Interrater reliability of criteria used in assessing blunt head injury patients for intracranial injuries.

Objective: To determine the interrater reliability of potential predictor variables that may be used to construct a clinical decision rule for emergency computed tomography of the head in blunt head injury victims.

Methods: As a substudy of the NEXUS II study of blunt head trauma, physicians from 21 different emergency departments performed paired evaluations of patients undergoing computed tomography of the head after blunt head injury. Each physician independently determined, for each subject, the presence or absence of each of 19 separate clinical characteristics. The physicians were either residents or attending physicians in the participating emergency departments. Paired responses on a sample of 3,951 patients were compared for raw level of agreement and for interrater concordance using the kappa statistic. If the lower margin of the 95% confidence interval for raw agreement was at least 85% and was equal to or greater than 0.50 for kappa, this was predetermined to represent substantial interrater agreement.

Results: There was substantial interobserver agreement by both measures for 17 of the 19 candidate variables in patients with blunt head trauma. Interobserver agreement was substantial for all candidate variables except presence of seizure (kappa = 0.57 [95% CI = 0.47 to 0.67]; raw agreement = 96.5%) and abnormal cerebellar function (kappa = 0.54 [95% CI = 0.41 to 0.67]; raw agreement = 96.5%).

Conclusions: The clinicians in our study had a substantial level of agreement regarding most clinical criteria assessed in this large sample of patients with blunt head injury.