Examining rater and occasion influences in observational assessments obtained from within the clinical environment.

Background: When ratings of student performance within the clerkship consist of a variable number of ratings per clinical teacher (rater), an important measurement question arises regarding how to combine such ratings to accurately summarize performance. As previous G studies have not estimated the independent influence of occasion and rater facets in observational ratings within the clinic, this study was designed to provide estimates of these two sources of error.

Method: During 2 years of an emergency medicine clerkship at a large midwestern university, 592 students were evaluated an average of 15.