Comparative evaluation of image-based vs. text-based vs. multimodal AI approaches for automatic breast density assessment in mammograms.

Comput Methods Programs Biomed

MRI unit, Radiology department, HT Médica, Carmelo Torres n 2, 23007, Jaén, Spain.

Published: October 2024

Background And Objectives: In the last decade, there has been a growing interest in applying artificial intelligence (AI) systems to breast cancer assessment, including breast density evaluation. However, few models have been developed to integrate textual mammographic reports and mammographic images. Our aims are (1) to generate a natural language processing (NLP)-based AI system, (2) to evaluate an external image-based software, and (3) to develop a multimodal system, using the late fusion approach, by integrating image and text inferences for the automatic classification of breast density according to the American College of Radiology (ACR) guidelines in mammograms and radiological reports.

Methods: We first compared different NLP models, three based on n-gram term frequency - inverse document frequency and two transformer-based architectures, using 1533 unstructured mammogram reports as a training set and 303 reports as a test set. Subsequently, we evaluated an external image-based software using 303 mammogram images. Finally, we assessed our multimodal system taking into account both text and mammogram images.

Results: Our best NLP model achieved 88 % accuracy, while the external software and the multimodal system achieved 75 % and 80 % accuracy, respectively, in classifying ACR breast densities.

Conclusion: Although our multimodal system outperforms the image-based tool, it currently does not improve the results offered by the NLP model for ACR breast density classification. Nevertheless, the promising results observed here open the possibility to more comprehensive studies regarding the utilization of multimodal tools in the assessment of breast density.

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.cmpb.2024.108334DOI Listing

Publication Analysis

Top Keywords

breast density
20
multimodal system
16
external image-based
8
image-based software
8
nlp model
8
achieved accuracy
8
acr breast
8
breast
7
multimodal
6
density
5

Similar Publications

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!