The Quality of Operative Performance Narrative Feedback: A Retrospective Data Comparison Between End of Rotation Evaluations and Workplace-based Assessments.

Objective: To describe the quality of operative performance feedback using evaluation tools commonly used by general surgery residency training programs.

Summary Of Background Data: The majority of surgical training programs administer an evaluation through which faculty members may rate and comment on trainee operative performance at the end of the rotation (EOR). Many programs have also implemented the system for improving and measuring procedural learning (SIMPL), a workplace-based assessment tool with which faculty can rate and comment on a trainee's operative performance immediately after a case. It is unknown how the quality of narrative operative performance feedback delivered with these tools compares.

Methods: The authors collected EOR evaluations and SIMPL narrative comments on trainees' operative performance from 3 university-based surgery training programs during the 2016-2017 academic year. Two surgeon raters categorized comments relating to operative skills as being specific or general and as encouraging and/or corrective. Comments were then classified as effective, mediocre, ineffective, or irrelevant. The frequencies with which comments were rated as effective were compared using Chi-square analysis.

Results: The authors analyzed a total of 600 comments. 10.7% of EOR and 58.3% of SIMPL operative performance evaluation comments were deemed effective (P < 0.0001).

Conclusions: Evaluators give significantly higher quality operative performance feedback when using workplace-based assessment tools rather than EOR evaluations.