Accuracy of Chatbots in Citing Journal Articles.

JAMA Netw Open

Learning Health Community, Palo Alto, California.

Published: August 2023

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10410472PMC
http://dx.doi.org/10.1001/jamanetworkopen.2023.27647DOI Listing

Publication Analysis

Top Keywords

accuracy chatbots
4
chatbots citing
4
citing journal
4
journal articles
4
accuracy
1
citing
1
journal
1
articles
1

Similar Publications

Objective: Artificial intelligence (AI) chatbots, including chat generative pretrained transformer (ChatGPT) and Google Gemini, have significantly increased access to medical information. However, in pediatric orthopaedics, no study has evaluated the accuracy of AI chatbots compared with evidence-based recommendations, including the American Academy of Orthopaedic Surgeons clinical practice guidelines (AAOS CPGs). The aims of this study were to compare responses by ChatGPT-4.

View Article and Find Full Text PDF

Evaluation of different artificial intelligence applications in responding to regenerative endodontic procedures.

BMC Oral Health

January 2025

Department of Endodontics, Faculty of Dentistry, Marmara University, Başıbüyük, Başıbüyük Yolu Marmara Üniversitesi Başıbüyük Sağlık Yerleşkesi 9/3, Başıbüyük - Maltepe, PO Box: 34854, İstanbul, Turkey.

Introduction: The integration of artificial intelligence (AI) technologies in healthcare is revolutionizing the workflows of healthcare professionals, enabling faster and more accurate patient treatment. This study aims to evaluate the accuracy of responses provided by different AI chatbots to questions that dentists might ask regarding regenerative endodontic treatment (RET), a procedure that shows promising biological healing potential.

Methods: A total of 23 questions related to RET procedures were developed based on the American Association of Endodontists (AAE) 2022 guidelines.

View Article and Find Full Text PDF

The potential of large language models (LLMs) in medical applications is significant, and Retrieval-augmented generation (RAG) can address the weaknesses of these models in terms of data transparency and scientific accuracy by incorporating current scientific knowledge into responses. In this study, RAG and GPT-4 by OpenAI were applied to develop GuideGPT, a context aware chatbot integrated with a knowledge database from 449 scientific publications designed to provide answers on the prevention, diagnosis, and treatment of medication-related osteonecrosis of the jaw (MRONJ). A comparison was made with a generic LLM ("PureGPT") across 30 MRONJ-related questions.

View Article and Find Full Text PDF

Objective: Coronary artery disease (CAD) is the leading cause of morbidity and mortality globally. The growing interest in natural language processing chatbots (NLPCs) has driven their inevitable widespread adoption in healthcare. The purpose of this study was to evaluate the accuracy and reproducibility of responses provided by NLPCs, such as ChatGPT, Gemini, and Bing, to frequently asked questions about CAD.

View Article and Find Full Text PDF

Large language models (LLMs) are fundamentally transforming human-facing applications in the health and well-being domains: boosting patient engagement, accelerating clinical decision-making, and facilitating medical education. Although state-of-the-art LLMs have shown superior performance in several conversational applications, evaluations within nutrition and diet applications are still insufficient. In this paper, we propose to employ the Registered Dietitian (RD) exam to conduct a standard and comprehensive evaluation of state-of-the-art LLMs, GPT-4o, Claude 3.

View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!