From pixels to patient care: deep learning-enabled pathomics signature offers precise outcome predictions for immunotherapy in esophageal squamous cell cancer.

Background: Immunotherapy has significantly improved survival of esophageal squamous cell cancer (ESCC) patients, however the clinical benefit was limited to only a small portion of patients. This study aimed to perform a deep learning signature based on H&E-stained pathological specimens to accurately predict the clinical benefit of PD-1 inhibitors in ESCC patients.

Methods: ESCC patients receiving PD-1 inhibitors from Shandong Cancer Hospital were included. WSI images of H&E-stained histological specimens of included patients were collected, and randomly divided into training (70%) and validation (30%) sets. The labels of images were defined by the progression-free survival (PFS) with the interval of 4 months. The pretrained ViT model was used for patch-level model training, and all patches were projected into probabilities after linear classifier. Then the most predictive patches were passed to RNN for final patient-level prediction to construct ESCC-pathomics signature (ESCC-PS). Accuracy rate and survival analysis were performed to evaluate the performance of ViT-RNN survival model in validation cohort.

Results: 163 ESCC patients receiving PD-1 inhibitors were included for model training. There were 486,188 patches of 1024*1024 pixels from 324 WSI images of H&E-stained histological specimens after image pre-processing. There were 120 patients with 227 images in training cohort and 43 patients with 97 images in validation cohort, with balanced baseline characteristics between two groups. The ESCC-PS achieved an accuracy of 84.5% in the validation cohort, and could distinguish patients into three risk groups with the median PFS of 2.6, 4.5 and 12.9 months (P < 0.001). The multivariate cox analysis revealed ESCC-PS could act as an independent predictor of survival from PD-1 inhibitors (P < 0.001). A combined signature incorporating ESCC-PS and expression of PD-L1 shows significantly improved accuracy in outcome prediction of PD-1 inhibitors compared to ESCC-PS and PD-L1 anlone, with the area under curve value of 0.904, 0.924, 0.610 for 6-month PFS and C-index of 0.814, 0.806, 0.601, respectively.

Conclusions: The outcome supervised pathomics signature based on deep learning has the potential to enable superior prognostic stratification of ESCC patients receiving PD-1 inhibitors, which convert the images pixels to an effective and labour-saving tool to optimize clinical management of ESCC patients.