Assessing global interrater agreement is difficult as most published indices are affected by the presence of mixtures of agreements and disagreements. A previously proposed method was shown to be specifically sensitive to global agreement, excluding mixtures, but also negatively biased. Here, we propose two alternatives in an attempt to find what makes such methods so specific. The first method, , is found to be unbiased while at the same time rejecting mixtures, is detecting agreement with good power and is little affected by unequal category prevalence as soon as there are more than two categories.

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5965630PMC
http://dx.doi.org/10.1177/0013164416654740DOI Listing

Publication Analysis

Top Keywords

global interrater
8
interrater agreement
8
unbiased estimate
4
estimate global
4
agreement
4
agreement assessing
4
assessing global
4
agreement difficult
4
difficult published
4
published indices
4

Similar Publications

Background: The LGBTQI+ population makes up at least 7.6% of the US population. LGBTQI+ populations are at increased risk of experiencing LGBTQI+-related discrimination and cis-heteronormativity in healthcare leading to poorer health outcomes throughout the cancer care continuum.

View Article and Find Full Text PDF

Importance: Although clinical practice guidelines exist for the treatment of infantile hemangiomas (IHs), recommendations are heterogeneous, and wide practice variations in IH management have been reported.

Objective: To analyze the degree of agreement in treatment choices for IH among pediatric dermatologists in North America and Europe and assess whether there are differences across IH risk categories.

Design, Setting, And Participants: This cross-sectional interrater and intrarater agreement study was conducted through a survey based on the Spanish Academy of Dermatology and Venereology IH prospective cohort.

View Article and Find Full Text PDF

Background: We aimed to characterize the histologic gut phenotype of pediatric primary sclerosing cholangitis (PSC)-associated inflammatory bowel disease (IBD) against non-PSC colitis, and to assess Nancy Index (NI) performance in pediatric PSC-IBD.

Methods: Single-center retrospective cohort study including children diagnosed with PSC-IBD or non-PSC colitis (ulcerative colitis [UC] or IBD-unclassified) from 2000 to 2018, with diagnostic intestinal biopsies. Biopsies were re-reviewed by two independent pathologists who assessed microscopic disease distribution, NI scores, and specific histological features in the right and left colons, overall and stratified by endoscopic severity (moderate-severe vs.

View Article and Find Full Text PDF

Ophthalmologists and ophthalmology residents (ORs) are increasingly turning to the internet for medical information, underscoring the significant role that YouTube videos, particularly three-dimensional (3D) ones, play in lifelong learning. This study aimed to compare the content and quality of 3D YouTube videos with two-dimensional (2D) videos as supplementary educational tools for vitreoretinal surgery. Data collected included video length (minutes), time elapsed since upload (days), number of views, likes, dislikes, vitreoretinal surgery type, and visualization system.

View Article and Find Full Text PDF

Introduction: Latest epidemiological metrics put a global prevalence of 20.6 million people suffering from spinal cord injury (SCI), leading to a burden of functional disability, deterioration in quality of life and reduced life expectancy. A thorough statement of diagnostic methods and treatment protocols for swallowing disorders after SCI stands as a major priority to streamline patient care and cost-sharing.

View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!