Performance of an Algorithm Grading Surgery-Related Adverse Events According to the Clavien-Dindo Classification.

Objective: To assess performance of an algorithm for automated grading of surgery-related adverse events (AEs) according to Clavien-Dindo (C-D) classification.

Summary Background Data: Surgery-related AEs are common, lead to increased morbidity for patients, and raise healthcare costs. Resource-intensive manual chart review is still standard and to our knowledge algorithms using electronic health record (EHR) data to grade AEs according to C-D classification have not been explored.

Method: The algorithm was developed in a research database containing all EHR data of Karolinska University Hospital Stockholm and returns a C-D grade for each AE within 30 days. This raw score was used to grade postoperative recovery of 1,379 elective colorectal procedures according to C-D classification and Comprehensive Complication Index® (CCI). Agreement with manual annotation of colorectal surgeon (gold standard) and research nurse (current practice) was assessed in a random sample of 399 procedures.

Results: For the C-D classification, kappa was 0.77 (95%CI 0.71-0.84) for algorithm vs surgeon and 0.74 (95%CI 0.67-0.82) for algorithm vs nurse. The kappa value increased to 0.89 (95%CI 0.84-0.95) after correction of misclassified annotations of surgeon. The intraclass correlation for CCI between algorithm and surgeon was 0.89 (95%CI 0.87-0.91) after correction and 0.76 (95%CI 0.71-0.80) for algorithm vs nurse.

Conclusion: The performance of the algorithm motivates in our opinion implementation to real-time data under continuous scientific evaluation of the impact on AEs in different types of surgery. In the future, local EHR data could be used to enhance risk prediction with machine learning techniques.