CACER: Clinical concept Annotations for Cancer Events and Relations.

J Am Med Inform Assoc

Department of Information Sciences and Technology, George Mason University, Fairfax, VA 22030, United States.

Published: November 2024

AI Article Synopsis

  • The study aims to analyze the relationship between cancer drugs and their associated symptoms by extracting structured information from oncology clinical notes using a new corpus, CACER, which includes detailed annotations of over 48,000 medical problems and drug events.
  • Transformer-based models such as BERT, Llama3, Flan-T5, and GPT-4 were evaluated for their ability to extract events and relationships from clinical narratives, with BERT and Llama3 performing the best overall.
  • The research concludes that while large language models like GPT-4 are capable, they did not outperform smaller models like BERT, emphasizing the effectiveness of well-annotated training data.

Article Abstract

Objective: Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes.

Materials And Methods: We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48 000 medical problems and drug events and 10 000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL).

Results: In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks.

Discussion: The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models.

Conclusions: We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC11491616PMC
http://dx.doi.org/10.1093/jamia/ocae231DOI Listing

Publication Analysis

Top Keywords

medical problems
12
clinical concept
8
concept annotations
8
annotations cancer
8
cancer events
8
events relations
8
clinical narratives
8
narratives oncology
8
cacer novel
8
novel corpus
8

Similar Publications

Association of smartphone overuse and neck pain: a systematic review and meta-analysis.

Postgrad Med J

January 2025

Department of Orthopedics, Hualien Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, Hualien, Taiwan.

Background: Smartphone overuse is associated with both psychological and physical health problems, including depression and musculoskeletal disorders. However, the association between smartphone overuse and neck pain remains unclear. We performed a meta-analysis to examine the relation between smartphone overuse and neck pain, and to identify high-risk usage patterns.

View Article and Find Full Text PDF

Introduction: Anemia is a severe public health problem in India, affecting more than 50% of individuals across most age groups. The Anemia Mukt Bharat (AMB) program, with a target of a three-percentage point reduction in anemia prevalence per year, developed a monitoring mechanism based on a set of 18 indicators and six key performance indicators (KPIs) derived from routine reporting in the Health Management Information System (HMIS). The study's objective was to assess the status of anemia control measures in the district of Faridabad, Haryana, India, using AMB HMIS indicators from April 2018 to March 2019.

View Article and Find Full Text PDF

Aim Preventing leaving-without-being-seen (LWBS) in children is crucial due to their inability to seek medical care independently. Because there are no studies of LWBS in Japan, the extent of this problem in Japan and its impacts on healthcare are uncertain. The present study seeks to fill this gap by investigating LWBS after triage and identifying the associated factors.

View Article and Find Full Text PDF

It is critical to recognize pulmonary embolism as soon as possible in patients who have gastrointestinal problems pre- and post-surgery. Even in the absence of conventional risk factors, the Factor V Leiden mutation emphasizes the importance of a thorough thrombophilia assessment. To effectively manage and prevent thrombotic episodes, prompt anticoagulant medication and genetic screening for family members are essential.

View Article and Find Full Text PDF

Background: Persistently high rates of inhaler errors and poor adherence among Chronic Obstructive Pulmonary Disease (COPD) patients contribute to ineffective symptomatic control, high care burdens, and increased healthcare resource utilization.

Objective: This study aimed to report (i) nurses-identified common problems and errors of inhaler use in COPD patients, (ii) nurses' attitudes, practices, training needs and required support in inhaler education.

Methods: An online questionnaire survey was conducted with nurses working in Hong Kong from May to June 2023 using an exponential, non-discriminative snowball sampling strategy.

View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!