External validation of a machine learning based algorithm to differentiate hepatic mucinous cystic neoplasms from benign hepatic cysts.

Purpose: To externally validate an algorithm for non-invasive differentiation of hepatic mucinous cystic neoplasms (MCN) from benign hepatic cysts (BHC), which differ in management.

Methods: Patients with cystic liver lesions pathologically confirmed as MCN or BHC between January 2005 and March 2022 from multiple institutions were retrospectively included. Five readers (2 radiologists, 3 non-radiologist physicians) independently reviewed contrast-enhanced CT or MRI examinations before tissue sampling and applied the 3-feature classification algorithm described by Hardie et al. to differentiate between MCN and BHC, which had a reported accuracy of 93.5%. The classification was then compared to the pathology results. Interreader agreement between readers across different levels of experience was evaluated with Fleiss' Kappa.

Results: The final cohort included 159 patients, median age of 62 years (IQR [52.0, 70.0]), 66.7% female (106). Of all patients, 89.3% (142) had BHC, and the remaining 10.7% (17) had MCN on pathology. Agreement for class designation between the radiologists was almost perfect (Fleiss' Kappa 0.840, p < 0.001). The algorithm had an accuracy of 98.1% (95% CI [94.6%, 99.6%]), a positive predictive value of 100.0% (95% CI [76.8%, 100.0%]), a negative predictive value of 97.9% (95% CI [94.1%, 99.6%]), and an area under the receiver operator characteristic curve (AUC) of 0.911 (95% CI [0.818, 1.000]).

Conclusion: The evaluated algorithm showed similarly high diagnostic accuracy in our external, multi-institutional validation cohort. This 3-feature algorithm is easily and rapidly applied and its features are reproducible among radiologists, showing promise as a clinical decision support tool.