A first look at the Accreditation Council for Graduate Medical Education anesthesiology milestones: implementation of self-evaluation in a large residency program.

Study Objective: The objective was to determine if there is a correlation between resident postgraduate year (PGY) of training and self-evaluation of performance using the Accreditation Council for Graduate Medical Education milestones.

Design: Survey.

Setting: Residency program at a large academic center.

Patients: Residents and Faculty Clinical Competency Committee (CCC).

Interventions: None.

Measurements: Resident and CCC milestone scores.

Main Results: Correlation coefficients for average score for each milestone vs PGY level ranged from 0.80 for receiving and giving feedback to 0.95 for anesthetic choice and conduct. All milestones showed a relatively linear relationship with PGY of training, and none were found to be consistently reached very late or very early in training. When examining variation across the scores for the individual residents, the distributions for PGY-2 and -3 appeared to be wider than those for PGY-1 and -4. The intraclass correlation coefficients ranged from 0.718 to 0.928.

Conclusions: There was a remarkable degree of consistency in the relationship between level of training and resident self-assessment score for every milestone, as well as strong agreement between the resident and CCC faculty scores. Examination of the variance in the scores, when interpreted in light of our particular training program's characteristics, suggests that the milestones accurately reflect the progression in skill across the residency. In addition, given the concordance between the self-evaluation scores and the CCC faculty scores, self-evaluation may be a reasonable starting point as programs begin the daunting task of determining scores for each of the 25 milestones as part of the biannual evaluation process.