Regulatory end-point assessment of the consultation competence of family practice trainees in Kuwait.

Background: No single approach to the regulatory assessment of global consultation competence has been shown to possess the required levels of validity, reliability and feasibility.

Objective: To evaluate the approach adopted in Kuwait to the regulatory end-point assessment of the global consultation competence of family practice trainees with particular reference to validity, reliability and feasibility.

Methods: Family practice trainees in Kuwait were individually and directly observed for 3 hours in consultation with a minimum of 10 patients by a pair of examiners. Performance was judged against the explicit criteria of consultation competence as contained in the Leicester Assessment Package (LAP).

Results: The marks independently allocated by the pairs of examiners to 126 trainees between 1994 and 2001 were within five percentage points on 91% of occasions. A reliability coefficient of 0.82 was achieved when two examiners independently marked candidates consulting with 10 real patients; this rose to 0.95 at the critical 50% pass-fail margin. The main sources of variance contributing to the reliability of marks allocated were candidate performance (42%) and the interaction of candidate performance across cases, i.e., case specificity (30%). The clinical challenges presented by the patients were judged by both examiners to be sufficient to enable performance to be assessed across the seven LAP consultation categories as follows: behaviour and relationship with patients (100% of consultations), interviewing/history taking (100%), record keeping (99%), patient management (99%), problem solving (98%), physical examination (95%), and anticipatory care (86%). Each assessment involved a pair of examiners and lasted approximately 3.5 hours.

Conclusion: The Kuwait clinical examination achieves high content validity and authenticity as it uses direct observation of performance, validated and explicit criteria against which performance is judged, and real patient challenges. It can discriminate between different levels of consultation performance and satisfies the recognized reliability threshold for regulatory examinations (0.82 vs 0.80). Accordingly, we recommend the use of such an approach in the regulatory end-point assessment of the global consultation competence of trainees in family practice. Such an approach is more valid, and is likely to be more feasible, than simulated surgeries or the short-case OSCE format.