Assessing appropriateness of pediatric emergency department visits: is it even possible?

Objectives: Numerous studies reported on the frequency of, and factors associated with inappropriate or unnecessary emergency department (ED) visits using clinician judgment as the gold standard of appropriateness. This study evaluated the reliability of clinician judgment for assessing appropriateness of pediatric ED visit.

Methods: We conducted a retrospective cohort study comparing 3 clinicians' determination of ED visit appropriateness with and without guidance from a three-question structured algorithm. We used a cohort of scheduled ED return visits deemed appropriate by the index treating clinician between May 1, 2012, and April 30, 2013. We measured the level of agreement among three clinician investigators with and without use of the structured algorithm.

Results: A total of 207 scheduled ED return visits were reviewed by the primary clinician reviewer who agreed with the index treating clinician for 79/207 visits (38.2%). Among a random subset of 90 return visits reviewed by all three clinicians, agreement was 67% with a Fleiss' Kappa of 0.30 (0.17-0.44). Using a three-question algorithm based on objective criteria, agreement with the index treating provider increased to 115/207 (55.6%).

Conclusions: Although an important contributor to pediatric ED overcrowding, unnecessary or inappropriate visits are difficult to identify. We demonstrated poor reliability of clinician judgment to determine appropriateness of ED return visits, likely due to variability in clinical decision-making and risk-tolerance, social and systems factors impacting access and use of health care. We recommend that future studies evaluating the appropriateness of ED use standardized, objective criteria rather than clinician judgment alone.