Background: It is thought that ChatGPT, an advanced language model developed by OpenAI, may in the future serve as an AI-assisted decision support tool in medicine.
Objective: To evaluate the accuracy of ChatGPT's recommendations on medical questions related to common cardiac symptoms or conditions.
Methods: We tested ChatGPT's ability to address medical questions in two ways. First, we assessed its accuracy in correctly answering cardiovascular trivia questions ( = 50), based on quizzes for medical professionals. Second, we entered 20 clinical case vignettes on the ChatGPT platform and evaluated its accuracy compared to expert opinion and clinical course. Lastly, we compared the latest research version (v3.5; 27 September 2023) with a prior version (v3.5; 30 January 2023) to evaluate improvement over time.
Results: We found that ChatGPT latest version correctly answered 92% of the trivia questions, with slight variation in accuracy in the domains coronary artery disease (100%), pulmonary and venous thrombotic embolism (100%), atrial fibrillation (90%), heart failure (90%) and cardiovascular risk management (80%). In the 20 case vignettes, ChatGPT's response matched in 17 (85%) of the cases with the actual advice given. Straightforward patient-to-physician questions were all answered correctly (10/10). In more complex cases, where physicians (general practitioners) asked other physicians (cardiologists) for assistance or decision support, ChatGPT was correct in 70% of cases, and otherwise provided incomplete, inconclusive, or inappropriate recommendations when compared with expert consultation. ChatGPT showed significant improvement over time; as the January version correctly answered 74% (vs 92%) of trivia questions ( = 0.031), and correctly answered a mere 50% of complex cases.
Conclusions: Our study suggests that ChatGPT has potential as an AI-assisted decision support tool in medicine, particularly for straightforward, low-complex medical questions, but further research is needed to fully evaluate its potential.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1080/00015385.2024.2303528 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Augmented prediction of vertebral collapse after osteoporotic vertebral compression fractures through parameter-efficient fine-tuning of biomedical foundation models.
Sci Rep
December 2024
Department of Medical Device Development, Seoul National University College of Medicine, Seoul, Republic of Korea.
Vertebral collapse (VC) following osteoporotic vertebral compression fracture (OVCF) often requires aggressive treatment, necessitating an accurate prediction for early intervention. This study aimed to develop a predictive model leveraging deep neural networks to predict VC progression after OVCF using magnetic resonance imaging (MRI) and clinical data. Among 245 enrolled patients with acute OVCF, data from 200 patients were used for the development dataset, and data from 45 patients were used for the test dataset.
View Article and Find Full Text PDFAttention-guided convolutional network for bias-mitigated and interpretable oral lesion classification.
Sci Rep
December 2024
Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Canada.
Accurate diagnosis of oral lesions, early indicators of oral cancer, is a complex clinical challenge. Recent advances in deep learning have demonstrated potential in supporting clinical decisions. This paper introduces a deep learning model for classifying oral lesions, focusing on accuracy, interpretability, and reducing dataset bias.
View Article and Find Full Text PDFThe impact of "nurse-led education" vs. "nurse-led education plus peer support" on patients' decision for peritoneal dialysis treatment in chronic kidney disease patients.
Ther Apher Dial
December 2024
Department of Nephrology, Ankara Bilkent City Hospital, Ankara, Turkey.
Introduction: End-stage kidney disease patients face a critical decision regarding kidney replacement therapy options, which include kidney transplantation, hemodialysis, or peritoneal dialysis (PD). This study aims to evaluate the impact of nurse-led education (NE) alone vs. NE combined with peer support on the patients' decision over PD treatment in chronic kidney disease patients.
View Article and Find Full Text PDFImproving the Evaluation of Aesthetic Outcomes in DIEP Flap Breast Reconstruction: Validation of the Aesthetic Grading Tool.
J Surg Oncol
December 2024
Department of Surgery, Plastic and Reconstructive Surgery Service, Memorial Sloan Kettering Cancer Center, New York City, New York, USA.
Introduction: This study aimed to develop and validate an aesthetic grading tool (AGT) for bilateral DIEP flap breast reconstruction and investigate the correlation of BREAST-Q scores with perceived aesthetic outcomes.
Methods: The AGT utilized a Likert scale to rate aesthetic outcomes based on photographs of post-reconstruction breasts. The validation involved iterative testing with healthcare providers and patients.
Calibrated Ecosystem Models Cannot Predict the Consequences of Conservation Management Decisions.
Ecol Lett
January 2025
School of Mathematical Sciences, Queensland University of Technology, Brisbane, Queensland, Australia.
Ecosystem models are often used to predict the consequences of management interventions in applied ecology and conservation. These models are often high-dimensional and nonlinear, yet limited data are available to calibrate or validate them. Consequently, their utility as decision-support tools is unclear.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!