Investigating faculty assessment of anesthesia trainees and the failing-to-fail phenomenon: a randomized controlled trial.

Purpose: A growing body of evidence has shown that supervisors may "fail to fail" trainees even when they have judged their performance unsatisfactory. This has significant implications for the implementation of a nationwide competency-based education model of residency training. The objective of this study was to determine the incidence of "failing to fail" clearly underperforming residents.

Methods: Study participants were recruited via an email invitation sent to all departments of anesthesia at each of the hospitals affiliated with the University of Toronto. They were randomized into a high-stakes (assessment would affect the resident's progress) or low-stakes (assessment would not affect the resident's progress) group and asked to assess the performance (fail or pass grade) of a struggling resident. Participants assessed a video depicting an actor managing a scripted simulation scenario. It contained several critical clinical mistakes constituting a clear fail performance. The purpose of the study was only disclosed following the assessment.

Results: Of the 288 email invitations sent (144 in each group), 158 (54%) participants completed the study, with 93 in the high-stakes group and 65 in the low-stakes group. Twenty-eight participants (17.7%) failed to issue a failing grade, including 23.1% (15/65) in the high-stakes group and 13.9% (13/93) in the low-stakes group (P = 0.14).

Conclusions: Though often discussed, this is the first study to quantitatively show that the "failing-to-fail" phenomenon likely occurs during residency training performance evaluations. Passing underperforming learners can potentially affect patient safety and result in severe personal consequences to the learner. The results indicate the need for better performance assessment training for faculty members.