Objective evaluation of ERCP procedures: a simple grading scale for evaluating technical difficulty.

Background: and objective: Endoscopic retrograde cholangiopancreatography (ERCP) is a technically demanding endoscopic procedure that varies from a simple diagnostic to a highly complex therapeutic procedure. Simple outcome measures such as success and complication rates do not reflect the competence of the operator or endoscopy unit, as case mix is not taken into account. A grading scale to assess the technical difficulty of ERCP can improve the objectivity of outcome data.

Methods: A I to IV technical difficulty grading scale was constructed and applied prospectively to all ERCPs over a 12 month period at a single centre. The procedures were performed by two senior trainees and two experienced consultants (trainers). The grading scale was validated for construct validity and inter-rater reliability at the end of the study using the chi(2) test and kappa statistics.

Results: There were 305 ERCPs in 259 patients over the 12 months study period (males: 112, females: 147, age range 17-97, mean 70.3 years). There was overall success in 244 (80%) procedures with complications in 13 (4%): bleeding in five (1.6%), cholangitis in one (0.3%), pancreatitis in five (1.6%), and perforation in two (0.7%). Success rate was highest for grade I, 49/55 (89%), compared with grade IV procedures, 8/11 (73%). There was a significant linear trend towards a lower success rate from grade I to IV (p=0.021) for trainees, but not for trainers. Complications were low in grade I, II, and III procedures, 12/295(4%), compared with grade IV procedures, 1/11(9%). The inter-rater reliability for the grading scale was good with a substantial agreement between the raters (kappa=0.68, p<0.001).

Conclusion: Success and complications of ERCP by trainees are influenced by the technical difficulty of the procedure. Outcome data incorporating a grading scale can give accurate information when auditing the qualitative outcomes. This can provide a platform for structured objective evaluation.