RESIDENTS AS INTERVIEWERS: A COMPARISON OF RESIDENT AND FACULTY EVALUATIONS OF EMERGENCY MEDICINE RESIDENCY APPLICANTS.

Background: Interviews are an integral component of the residency selection process. Many programs use current residents as interviewers in addition to faculty. Although the reliability of interview scores between faculty members has been examined, little is known about the reliability between resident and faculty interviewers.

Objective: This study evaluates the reliability of residents as interviewers compared with faculty.

Methods: A retrospective analysis of interview scores from the 2020-2021 application cycle was performed at an emergency medicine (EM) residency program. Each applicant participated in five separate one-on-one interviews led by four faculty members and one senior resident. Interviewers assigned applicants a score from 0 to 10. Consistency between interviewers was measured using the intraclass correlation coefficient (ICC). Generalizability theory was used to measure variance components including applicant, interviewer, and rater type (resident vs. faculty) and their impact on scoring.

Results: There were 250 applicants interviewed by 16 faculty members and 7 senior residents during the application cycle. The mean (SD) interview score given by resident interviewers was 7.10 (1.53) and the mean (SD) score given by faculty was 7.07 (1.69). There was no statistically significant difference between the pooled scores (p = 0.97). Reliability between interviewers was good to excellent (ICC = 0.90; 95% CI 0.88-0.92). The generalizability study showed most score variance was attributable to applicant characteristics and only 0.6% was attributable to interviewer or rater type (resident vs. faculty).

Conclusions: There was strong concordance between faculty and resident interview scores indicating reliability of EM resident scoring compared to faculty.