Evaluation of the validity of medical students' self-assessments of proficiency in clinical simulations.

Background: The accuracy of self-assessments has not been well supported in the literature. This study was undertaken to examine the validity of medical students' ratings of their proficiency during encounters with simulated patients and simulation devices.

Methods: Confidential self-assessments for 10 skills were collected from 195 students during a formal clinical skills assessment related to 3 cases at the end of a surgery clerkship. The cases required students to gather data from simulated patients and perform procedures such as rectal examinations, nasogastric tube insertions, and suturing on bench simulation models. The patients were trained to assess student performance.

Results: There were significant differences between student self-assessments and simulated patient scores for general clinical skills as opposed to procedural skills. Students' mean self-assessments in the data gathering and interpersonal skills were 2-6 % points higher than ratings of their proficiency by simulated patients. However, self-assessments on procedures were 5-8 points lower than patient ratings. The median correlation between self-assessments and patient ratings for general clinical skills such as data gathering and interpersonal skills was 0.08 (not significant), whereas the median correlation between student and patient ratings in procedures was 0.22 (P < 0.01).

Conclusions: Third-year medical students' self-assessments for specific procedures are more valid than self-assessments of general clinical skills. Students are less confident in their procedural skills compared with general clinical skills. Although self-assessments should not be used as the sole measure of performance in clinical simulations, self-assessments for specific procedures can provide supplemental information on proficiency.