Examiner training: A study of examiners making sense of norm-referenced feedback.

Examiner training has an inconsistent impact on subsequent performance. To understand this variation, we explored how examiners think about changing the way they assess. We provided comparative data to 17 experienced examiners about their assessments, captured their sense-making processes using a modified think-aloud protocol, and identified patterns by inductive thematic analysis. We observed five sense-making processes: (1) (2) (3) (4) , and (5) . Three observed meta-themes describe the of examiners' thinking: - where examiners expressed curiosity over how their judgments compared with others', but they also expressed guardedness about the relevance of the comparisons; - where examiners' interpretation and attribution exhibited cognitive anchoring, personalization, and affective bias; - where examiners expressed openness to change, but also loyalty to their judgment-framing values and aphorisms. Our examiners engaged in complex processes as they considered changing their assessments. The 'stabilising' mechanisms some used resembled learners assimilating educational feedback. If these are typical examiner responses, they may well explain the variable impact of examiner training, and have significant implications for the pursuit of meaningful and defensible judgment-based assessment.