Background: ChatGPT, an artificial intelligence (AI) text generator trained to predict correct words, can provide answers to questions but has shown mixed results in answering medical questions.
Purpose: To assess the reliability and accuracy of ChatGPT in providing answers to a complex clinical question.
Methods: A Population, Intervention, Comparison, Outcome, and Time (PICOT) formatted question was queried, along with a request for references. Full-text articles were reviewed to verify the accuracy of the evidence summary provided by the chatbot.
Results: ChatGPT was unable to provide a certifiable response to a PICOT question. The references cited as evidence included incorrect journal information, and many study details summarized by ChatGPT proved to be patently false, including providing fabricated data.
Conclusions: ChatGPT provides answers that appear legitimate but may be factually incorrect. The system is not transparent in how it gathers data to answer questions and sometimes fabricates information that looks plausible, making it an unreliable tool for clinical questions.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1097/NNE.0000000000001436 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Preparing the AI-assisted animal scientist: faculty and student perspectives on enhancing animal science education with artificial intelligence.
Anim Front
December 2024
Department of Animal Science, Iowa State University, Ames, Iowa 50011, USA.
Recent advancements in large language models (LLMs) like ChatGPT and LLaMA have shown significant potential in medical applications, but their effectiveness is limited by a lack of specialized medical knowledge due to general-domain training. In this study, we developed Me-LLaMA, a new family of open-source medical LLMs that uniquely integrate extensive domain-specific knowledge with robust instruction-following capabilities. Me-LLaMA comprises foundation models (Me-LLaMA 13B and 70B) and their chat-enhanced versions, developed through comprehensive continual pretraining and instruction tuning of LLaMA2 models using both biomedical literature and clinical notes.
View Article and Find Full Text PDFExpert Consensus on Developing Information and Communication Technology-Based Patient Education Guidelines for Rheumatic Diseases in the Korea.
J Korean Med Sci
January 2025
Department of Rheumatology, Hanyang University Hospital for Rheumatic Diseases, Seoul, Korea.
Background: This study aimed to identify key priorities for the development of guidelines for information and communication technology (ICT)-based patient education tailored to the needs of patients with rheumatic diseases (RDs) in the Republic of Korea, based on expert consensus.
Methods: A two-round modified Delphi study was conducted with 20 rheumatology, patient education, and digital health literacy experts. A total of 35 items covering 7 domains and 18 subdomains were evaluated.
Gene expression and immunohistochemistry analysis of ADAMTS-1 and its substrates in odontogenic keratocyst.
Diagn Pathol
January 2025
Cell Culture Laboratory, School of Dentistry, Federal University of Para, Rua Augusto Correa, 01 Guama, Belem, PA, 66075110, Brazil.
Background: Considering the significant participation of the microenvironment in the local aggressiveness of odontogenic keratocysts, this study aims to evaluate the expression of ADAMTS-1 and its substrates, versican, aggrecan and brevican in this locally invasive odontogenic cyst.
Methods: Immunohistochemistry and polymerase chain reaction (PCR) were conducted on 30 cases of odontogenic keratocysts (OKCs) and 20 dental follicles (DFs).
Results: The immunohistochemical expression of these proteins was predominantly cytoplasmic and granular across all samples.
Evaluation of Generative Artificial Intelligence Models in Predicting Pediatric Emergency Severity Index Levels.
Pediatr Emerg Care
January 2025
University of California Davis School of Medicine, Sacramento, CA.
Objective: Evaluate the accuracy and reliability of various generative artificial intelligence (AI) models (ChatGPT-3.5, ChatGPT-4.0, T5, Llama-2, Mistral-Large, and Claude-3 Opus) in predicting Emergency Severity Index (ESI) levels for pediatric emergency department patients and assess the impact of medically oriented fine-tuning.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!