Introduction: ChatGPT, developed by OpenAI, represents the cutting-edge in its field with its latest model, GPT-4. Extensive research is currently being conducted in various domains, including cardiovascular diseases, using ChatGPT. Nevertheless, there is a lack of studies addressing the proficiency of GPT-4 in diagnosing conditions based on Electrocardiography (ECG) data. The goal of this study is to evaluate the diagnostic accuracy of GPT-4 when provided with ECG data, and to compare its performance with that of emergency medicine specialists and cardiologists.
Methods: This study has received approval from the Clinical Research Ethics Committee of Hitit University Medical Faculty on August 21, 2023 (decision no: 2023-91). Drawing on cases from the "150 ECG Cases" book, a total of 40 ECG cases were crafted into multiple-choice questions (comprising 20 everyday and 20 more challenging ECG questions). The participant pool included 12 emergency medicine specialists and 12 cardiology specialists. GPT-4 was administered the questions in a total of 12 separate sessions. The responses from the cardiology physicians, emergency medicine physicians, and GPT-4 were evaluated separately for each of the three groups.
Results: In the everyday ECG questions, GPT-4 demonstrated superior performance compared to both the emergency medicine specialists and the cardiology specialists (p < 0.001, p = 0.001). In the more challenging ECG questions, while Chat-GPT outperformed the emergency medicine specialists (p < 0.001), no significant statistical difference was found between Chat-GPT and the cardiology specialists (p = 0.190). Upon examining the accuracy of the total ECG questions, Chat-GPT was found to be more successful compared to both the Emergency Medicine Specialists and the cardiologists (p < 0.001, p = 0.001).
Conclusion: Our study has shown that GPT-4 is more successful than emergency medicine specialists in evaluating both everyday and more challenging ECG questions. It performed better compared to cardiologists on everyday questions, but its performance aligned closely with that of the cardiologists as the difficulty of the questions increased.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1016/j.ajem.2024.03.017 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Who is coming in? Evaluation of physician performance within multi-physician emergency departments.
Am J Emerg Med
January 2025
Department of Emergency Medicine, Yale University School of Medicine, New Haven, CT, USA; Center for Outcomes Research and Evaluation, Yale University, New Haven, CT, USA.
Background: This study aimed to examine how physician performance metrics are affected by the speed of other attendings (co-attendings) concurrently staffing the ED.
Methods: A retrospective study was conducted using patient data from two EDs between January-2018 and February-2020. Machine learning was used to predict patient length of stay (LOS) conditional on being assigned a physician of average speed, using patient- and departmental-level variables.
National early warning score 2 plus non-invasive capnography and perfusion index to estimate poor outcomes in emergency departments.
Am J Emerg Med
January 2025
Faculty of Medicine, Universidad de Valladolid, Valladolid, Spain; Emergency Department, Hospital Clínico Universitario, Gerencia Regional de Salud de Castilla y León, Valladolid, Spain.
Background: The study of the inclusion of new variables in already existing early warning scores is a growing field. The aim of this work was to determine how capnometry measurements, in the form of end-tidal CO2 (ETCO2) and the perfusion index (PI), could improve the National Early Warning Score (NEWS2).
Methods: A secondary, prospective, multicenter, cohort study was undertaken in adult patients with unselected acute diseases who needed continuous monitoring in the emergency department (ED), involving two tertiary hospitals in Spain from October 1, 2022, to June 30, 2023.
Comparing Virtual Reality-Based and Traditional Physical Objective Structured Clinical Examination (OSCE) Stations for Clinical Competency Assessments: Randomized Controlled Trial.
J Med Internet Res
January 2025
Institute of Medical Teaching and Medical Education Research, University Hospital Würzburg, Würzburg, Germany.
Background: Objective structured clinical examinations (OSCEs) are a widely recognized and accepted method to assess clinical competencies but are often resource-intensive.
Objective: This study aimed to evaluate the feasibility and effectiveness of a virtual reality (VR)-based station (VRS) compared with a traditional physical station (PHS) in an already established curricular OSCE.
Methods: Fifth-year medical students participated in an OSCE consisting of 10 stations.
Evaluating ChatGPT's Efficacy in Pediatric Pneumonia Detection From Chest X-Rays: Comparative Analysis of Specialized AI Models.
JMIR AI
January 2025
Department of Radiology, Children's National Hospital, Washington, DC, United States.
Predicting the risk of intensive care unit admission in patients with COVID-19 presenting in the emergency room: Development and evaluation of the CROSS score.
Clin Infect Dis
January 2025
Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Department of Infectious Diseases, Respiratory Medicine and Critical Care, Berlin, Germany.
Background: Existing risk evaluation tools underperform in predicting intensive care unit (ICU) admission for patients with the Coronavirus Disease 2019 (COVID-19). This study aimed to develop and evaluate an accurate and calculator-free clinical tool for predicting ICU admission at emergency room (ER) presentation.
Methods: Data from patients with COVID-19 in a nationwide German cohort (March 2020-January 2023) were analyzed.
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!