Introduction: Artificial intelligence and machine learning are popular interconnected technologies. AI chatbots like ChatGPT and Gemini show considerable promise in medical inquiries. This scoping review aims to assess the accuracy and response length (in characters) of ChatGPT and Gemini in medical applications.
Methods: The eligible databases were searched to find studies published in English from January 1 to October 20, 2023. The inclusion criteria consisted of studies that focused on using AI in medicine and assessed outcomes based on the accuracy and character count (length) of ChatGPT and Gemini. Data collected from the studies included the first author's name, the country where the study was conducted, the type of study design, publication year, sample size, medical speciality, and the accuracy and response length.
Results: The initial search identified 64 papers, with 11 meeting the inclusion criteria, involving 1,177 samples. ChatGPT showed higher accuracy in radiology (87.43% vs. Gemini's 71%) and shorter responses (907 vs. 1,428 characters). Similar trends were noted in other specialties. However, Gemini outperformed ChatGPT in emergency scenarios (87% vs. 77%) and in renal diets with low potassium and high phosphorus (79% vs. 60% and 100% vs. 77%). Statistical analysis confirms that ChatGPT has greater accuracy and shorter responses than Gemini in medical studies, with a -value of <.001 for both metrics.
Conclusion: This Scoping review suggests that ChatGPT may demonstrate higher accuracy and provide shorter responses than Gemini in medical studies.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC11830737 | PMC |
| http://dx.doi.org/10.3389/fdgth.2025.1482712 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Accuracy of Large Language Models for Literature Screening in Thoracic Surgery: Diagnostic Study.
J Med Internet Res
March 2025
Department of Thoracic Surgery, West China Hospital of Sichuan University, Chengdu, China.
Background: Systematic reviews and meta-analyses rely on labor-intensive literature screening. While machine learning offers potential automation, its accuracy remains suboptimal. This raises the question of whether emerging large language models (LLMs) can provide a more accurate and efficient approach.
View Article and Find Full Text PDFComparative Analysis of Artificial Intelligence Platforms in Generating Post-Operative Instructions for Rhinologic Surgery.
Indian J Otolaryngol Head Neck Surg
January 2025
Department of Otolaryngology- Head and Neck Surgery, Rutgers New Jersey Medical School, Newark, New Jersey USA.
Recently, artificial intelligence (AI) platforms such as ChatGPT and Google Gemini have progressed at a rapid pace. To allow for optimal medical outcomes and patient safety, it is crucial that patients have clearly written post-operative instructions. Patients are increasingly turning to AI platforms for medical information.
View Article and Find Full Text PDFAdvancing AI-driven thematic analysis in qualitative research: a comparative study of nine generative models on Cutaneous Leishmaniasis data.
BMC Med Inform Decis Mak
March 2025
Mohammed VI International School of Public Health, Mohammed VI University of Sciences and Health, Casablanca, Morocco.
Background: As part of qualitative research, the thematic analysis is time-consuming and technical. The rise of generative artificial intelligence (A.I.
View Article and Find Full Text PDFLarge Language Models in peri-implant disease: How well do they perform?
J Prosthet Dent
March 2025
Associate Professor, Department of Preventive Dentistry, Periodontology and Implant Biology, School of Dentistry, Aristotle University of Thessaloniki, Greece; Associate Professor, School of Dentistry, European University Cyprus, Nicosia, Cyprus; and Adjunct Associate Professor, Hamdan bin Mohammed College of Dental Medicine, Mohammed bin Rashid University of Medicine and Health Sciences (MBRU), Dubai, United Arab Emirates.
Statement Of Problem: Artificial intelligence (AI) has gained significant recent attention and several AI applications, such as the Large Language Models (LLMs) are promising for use in clinical medicine and dentistry. Nevertheless, assessing the performance of LLMs is essential to identify potential inaccuracies or even prevent harmful outcomes.
Purpose: The purpose of this study was to evaluate and compare the evidence-based potential of answers provided by 4 LLMs to clinical questions in the field of implant dentistry.
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!