AI Article Synopsis
- The study investigates the effectiveness of the Generative Pre-trained Transformer 4 with vision in diagnosing vitreoretinal diseases in real-world settings at Bascom Palmer Eye Clinic, analyzing retrospective patient data from 2010 to 2023.
- Results showed low diagnostic accuracy, with 13.7% for open-ended questions and 31.3% for multiple-choice questions, while ICD-10 coding accuracy was similarly low, particularly struggling with complex cases.
- The findings suggest that while the model has potential in clinical care, particularly for standardized questions, it has significant limitations in handling more complex medical inquiries.
Article Abstract
Purpose: This study evaluates a large language model, Generative Pre-trained Transformer 4 with vision, for diagnosing vitreoretinal diseases in real-world ophthalmology settings.
Methods: A retrospective cross-sectional study at Bascom Palmer Eye Clinic, analyzing patient data from January 2010 to March 2023, assesses Generative Pre-trained Transformer 4 with vision's performance on retinal image analysis and International Classification of Diseases 10th revision coding across 2 patient groups: simpler cases (Group A) and complex cases (Group B) requiring more in-depth analysis. Diagnostic accuracy was assessed through open-ended questions and multiple-choice questions independently verified by three retina specialists.
Results: In 256 eyes from 143 patients, Generative Pre-trained Transformer 4-V demonstrated a 13.7% accuracy for open-ended questions and 31.3% for multiple-choice questions, with International Classification of Diseases 10th revision code accuracies at 5.5% and 31.3%, respectively. Accurately diagnosed posterior vitreous detachment, nonexudative age-related macular degeneration, and retinal detachment. International Classification of Diseases 10th revision coding was most accurate for nonexudative age-related macular degeneration, central retinal vein occlusion, and macular hole in OEQs, and for posterior vitreous detachment, nonexudative age-related macular degeneration, and retinal detachment in multiple-choice questions. No significant difference in diagnostic or coding accuracy was found in Groups A and B.
Conclusion: Generative Pre-trained Transformer 4 with vision has potential in clinical care and record keeping, particularly with standardized questions. Its effectiveness in open-ended scenarios is limited, indicating a significant limitation in providing complex medical advice.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1097/IAE.0000000000004204 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Identifying Protein-Nucleotide Binding Residues via Grouped Multi-task Learning and Pre-trained Protein Language Models.
J Chem Inf Model
January 2025
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China.
The accurate identification of protein-nucleotide binding residues is crucial for protein function annotation and drug discovery. Numerous computational methods have been proposed to predict these binding residues, achieving remarkable performance. However, due to the limited availability and high variability of nucleotides, predicting binding residues for diverse nucleotides remains a significant challenge.
View Article and Find Full Text PDFChatGPT Assisting Diagnosis of Neuro-Ophthalmology Diseases Based on Case Reports.
J Neuroophthalmol
October 2024
Department of Ophthalmology (YM, MD, PAL, JWF, TJH, SY), University of Tennessee Health Science Center, Memphis, Tennessee; Department of Ophthalmology (MYK), University of Colorado School of Medicine, Aurora, Colorado; and Department of Genetics, Genomics, and Informatics (SY), University of Tennessee Health Science Center, Memphis, Tennessee.
Background: To evaluate the accuracy of Chat Generative Pre-Trained Transformer (ChatGPT), a large language model (LLM), to assist in diagnosing neuro-ophthalmic diseases based on case reports.
Methods: We selected 22 different case reports of neuro-ophthalmic diseases from a publicly available online database. These cases included a wide range of chronic and acute diseases commonly seen by neuro-ophthalmic subspecialists.
Widespread use of ChatGPT and other Artificial Intelligence tools among medical students in Uganda: A cross-sectional study.
PLoS One
January 2025
Department of Pediatrics and Child Health, Makerere University, College of Health Sciences, Kampala, Uganda.
Background: Chat Generative Pre-trained Transformer (ChatGPT) is a 175-billion-parameter natural language processing model that uses deep learning algorithms trained on vast amounts of data to generate human-like texts such as essays. Consequently, it has introduced new challenges and threats to medical education. We assessed the use of ChatGPT and other AI tools among medical students in Uganda.
View Article and Find Full Text PDFA dataset and benchmark for hospital course summarization with adapted large language models.
J Am Med Inform Assoc
December 2024
Department of Radiology, Stanford University, Stanford, CA 94304, United States.
Objective: Brief hospital course (BHC) summaries are clinical documents that summarize a patient's hospital stay. While large language models (LLMs) depict remarkable capabilities in automating real-world tasks, their capabilities for healthcare applications such as synthesizing BHCs from clinical notes have not been shown. We introduce a novel preprocessed dataset, the MIMIC-IV-BHC, encapsulating clinical note and BHC pairs to adapt LLMs for BHC synthesis.
View Article and Find Full Text PDFDetection of Cognitive Impairment through Spontaneous Speech Acoustic Markers Using Transformer‐based Machine Learning Approach.
Alzheimers Dement
December 2024
University of California, Irvine, Irvine, CA, USA
Background: The increasing prevalence of cognitive impairment and dementia threatens global health, necessitating the development of accessible tools for detection of cognitive impairment. This study explores using a transformer‐based approach to detect cognitive impairment using acoustic markers of spontaneous speech.
Method: Recordings of unstructured interviews from baseline visits were obtained from participants of The 90+ Study, a longitudinal study of individuals older than 90 years.
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!