An Event-based Approach to Measurement: Facilitating Observational Measurement in Highly Variable Clinical Settings.

Background: Translational research in medical education requires the ability to rigorously measure learner performance in actual clinical settings; however, current measurement systems cannot accommodate the variability inherent in many patient care  environments. This is especially problematic in emergency medicine, where patients represent a wide spectrum of severity for a single clinical presentation. Our objective is to describe and implement EBAM, an event-based approach to measurement that can be applied to actual emergency medicine clinical events.

Methods: We used a four-step event-based approach to create an emergency department trauma resuscitation patient care measure. We applied the measure to a database of 360 actual trauma resuscitations recorded in a Level I trauma center using trained raters. A subset ( = 50) of videos was independently rated in duplicate to determine inter-rater reliability. Descriptive analyses were performed to describe characteristics of resuscitation events and Cohen's kappa was used to calculate reliability.

Results: The methodology created a metric containing both universal items that are applied to all trauma resuscitation events and conditional items that only apply in certain situations. For clinical trauma events, injury severity scores ranged from 1 to 75 with a mean (±SD) of 21 (±15) and included both blunt (254/360; 74%) and penetrating (86/360; 25%) traumatic injuries, demonstrating the diverse nature of the clinical encounters. The mean (±SD) Cohen's kappa for patient care items was 0.7 (±0.3).

Conclusion: We present an event-based approach to performance assessment that may address a major gap in translational education research. Our work centered on assessment of patient care behaviors during trauma resuscitation. More work is needed to evaluate this approach across a diverse array of clinical events.