An investigation into the use of a structured clinical operative test for the assessment of a clinical skill.

Aim: This study was designed to investigate the level of agreement between a group of assessors observing students undertaking a structured clinical operative test.

Method: 3 assessors agreed a series of criteria to assess the performance of undergraduate students in the recording of a dental impression. Guidelines for assessing whether the students adequately performed in relation to each criteria were also agreed. Following preliminary validation between the assessors, 2 assessors independently scored the performance of each student by reference to the agreed criteria, and the levels of agreement between assessors were compared. The 3 assessors worked in pairs with each other on three groups of students who were in the early stages of their clinical course.

Results: A total of 39 clinical dental students were assessed in the recording of a dental impression. The 3 pairs of assessors had satisfactory levels of agreement in the study with similar judgements being made on 90% or more of the 12 criteria assessed. Some differences existed between the pairs of assessors. Certain criteria were more easily judged than others and this was reflected in the level of agreement seen. For over 90% of the criteria, positive assessments were made by each of the 3 assessors, and although there were minor variations between the pairs of assessors, this may have reflected the ability of the groups of students studied.

Conclusion: The study showed that different assessors were generally able to make agreed judgements on performance criteria in a structured clinical operative test. In setting up performance assessment it is necessary to have close collaboration between assessors to make clearly defined criteria so that judgements are not too subjective. Furthermore, for the assessment of more complex clinical skills, great care is needed in assembling criteria that can be used reproducibly, and sufficient preparation time for the assessors is critical.