Reliability of trachoma clinical grading--assessing grading of marginal cases.

PLoS Negl Trop Dis

F.I. Proctor Foundation, San Francisco, California, United States of America; Department of Ophthalmology, University of California, San Francisco, San Francisco, California, United States of America; Department of Epidemiology & Biostatistics, University of California, San Francisco, San Francisco, California, United States of America.

Published: May 2014

Background: Clinical examination of trachoma is used to justify intervention in trachoma-endemic regions. Currently, field graders are certified by determining their concordance with experienced graders using the kappa statistic. Unfortunately, trachoma grading can be highly variable and there are cases where even expert graders disagree (borderline/marginal cases). Prior work has shown that inclusion of borderline cases tends to reduce apparent agreement, as measured by kappa. Here, we confirm those results and assess performance of trainees on these borderline cases by calculating their reliability error, a measure derived from the decomposition of the Brier score.

Methods And Findings: We trained 18 field graders using 200 conjunctival photographs from a community-randomized trial in Niger and assessed inter-grader agreement using kappa as well as reliability error. Three experienced graders scored each case for the presence or absence of trachomatous inflammation-follicular (TF) and trachomatous inflammation-intense (TI). A consensus grade for each case was defined as the one given by a majority of experienced graders. We classified cases into a unanimous subset if all 3 experienced graders gave the same grade. For both TF and TI grades, the mean kappa for trainees was higher on the unanimous subset; inclusion of borderline cases reduced apparent agreement by 15.7% for TF and 12.4% for TI. When we assessed the breakdown of the reliability error, we found that our trainees tended to over-call TF grades and under-call TI grades, especially in borderline cases.

Conclusions: The kappa statistic is widely used for certifying trachoma field graders. Exclusion of borderline cases, which even experienced graders disagree on, increases apparent agreement with the kappa statistic. Graders may agree less when exposed to the full spectrum of disease. Reliability error allows for the assessment of these borderline cases and can be used to refine an individual trainee's grading.

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4006735PMC
http://dx.doi.org/10.1371/journal.pntd.0002840DOI Listing

Publication Analysis

Top Keywords

experienced graders
20
borderline cases
20
reliability error
16
field graders
12
kappa statistic
12
apparent agreement
12
graders
10
cases
9
graders disagree
8
inclusion borderline
8

Similar Publications

Clinical Relevance: Clinical evaluation of the optic nerve using 3-D stereo disc photographs is considered the gold standard for estimating vertical cup-to-disc ratios. Ultra-widefield retinal imaging has gained increasing popularity to document and screen the health of the retina and optic nerve.

Background: Glaucoma is often first identified or suspected based on initial optic nerve assessment.

View Article and Find Full Text PDF

Exposure to family violence and school violence among Palestinian high schoolers: The moderating effect of gender and school climate.

Child Abuse Negl

December 2024

Paul Baerwald School of Social Work and Social Welfare, The Hebrew University of Jerusalem, Mt. Scopus, Jerusalem 91905, Israel. Electronic address:

Background: There is a serious dearth of research on the phenomenon of school violence in the Arab world. Moreover, studies have examined the relationship between students' exposure to family violence (EFV) and school violence.

Objective: This study measured the correlation between Palestinian high schoolers' EFV (i.

View Article and Find Full Text PDF

Particulates Matter: The Influence of Cumulative Local Air Pollution Exposure on Sixth-Grade Academic Achievement in California.

Environ Health Insights

November 2024

Industrial Strategies Division, California Air Resources Board, Sacramento, CA, USA.

We examine the influence of exposure to fine particulate matter (PM 2.5) in ambient air over the previous 6 years on the average standardized test score performance in math, English language arts (ELA), and overall for sixth graders at a sample of California public school districts from 2015 through 2018. Public health research suggests that children exposed to localized air pollution may suffer from cognitive impairment during testing or chronic conditions such as asthma that could influence their academic performance.

View Article and Find Full Text PDF
Article Synopsis
  • * A total of 597 AMD patients were analyzed, with 332 having SDDs and 265 without, using optical coherence tomography (SD-OCT) and reviewing electronic medical records.
  • * Results showed that AMD patients with SDDs are significantly more likely to be diagnosed with CVD and have procedures such as percutaneous transluminal coronary angioplasty (PTCA), suggesting a possible shared mechanism between SDDs and severe cardiovascular conditions.
View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!