Large language model-augmented learning for auto-delineation of treatment targets in head-and-neck cancer radiotherapy.

Background And Purpose: Radiation therapy (RT) is highly effective, but its success depends on accurate, manual target delineation, which is time-consuming, labor-intensive, and prone to variability. Despite AI advancements in auto-contouring normal tissues, accurate RT target volume delineation remains challenging. This study presents Radformer, a novel visual language model that integrates text-rich clinical data with medical imaging for accurate automated RT target volume delineation.

Materials And Methods: We developed Radformer, an innovative network that utilizes a hierarchical vision transformer as its backbone and integrates large language models (LLMs) to extract and embed clinical data in text-rich form. The model features a novel visual language attention module (VLAM) to combine visual and linguistic features, enabling language-aware visual encoding (LAVE). The Radformer was evaluated on a dataset of 2985 patients with head-and-neck cancer who underwent RT. Quantitative evaluations were performed utilizing metrics such as the Dice similarity coefficient (DSC), intersection over union (IOU), and 95th percentile Hausdorff distance (HD95).

Results: The Radformer demonstrated superior performance in segmenting RT target volumes compared to state-of-the-art models. On the head-and-neck cancer dataset, Radformer achieved a mean DSC of 0.76 ± 0.09 versus 0.66 ± 0.09, a mean IOU of 0.69 ± 0.08 versus 0.59 ± 0.07, and a mean HD95 of 7.82 ± 6.87 mm versus 14.28 ± 6.85 mm for gross tumor volume delineation, compared to the baseline 3D-UNETR.

Conclusions: The Radformer model offers a clinically optimal means of RT target auto-delineation by integrating both imaging and clinical data through a visual language model. This approach improves the accuracy of RT target volume delineation, facilitating broader AI-assisted automation in RT treatment planning.