Background: In November 2022, OpenAI released a chatbot named ChatGPT, a product capable of processing natural language to create human-like conversational dialogue. It has generated a lot of interest, including from the scientific community and the medical science community. Recent publications have shown that ChatGPT can correctly answer questions from medical exams such as the United States Medical Licensing Examination and other specialty exams. To date, there have been no studies in which ChatGPT has been tested on specialty questions in the field of nephrology anywhere in the world.
Methods: Using the ChatGPT-3.5 and -4.0 algorithms in this comparative cross-sectional study, we analysed 1560 single-answer questions from the national specialty exam in nephrology from 2017 to 2023 that were available in the Polish Medical Examination Center's question database along with answer keys.
Results: Of the 1556 questions posed to ChatGPT-4.0, correct answers were obtained with an accuracy of 69.84%, compared with ChatGPT-3.5 (45.70%, = .0001) and with the top results of medical doctors (85.73%, = .0001). Of the 13 tests, ChatGPT-4.0 exceeded the required ≥60% pass rate in 11 tests passed, and scored higher than the average of the human exam results.
Conclusion: ChatGPT-3.5 was not spectacularly successful in nephrology exams. The ChatGPT-4.0 algorithm was able to pass most of the analysed nephrology specialty exams. New generations of ChatGPT achieve similar results to humans. The best results of humans are better than those of ChatGPT-4.0.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC11295106 | PMC |
| http://dx.doi.org/10.1093/ckj/sfae193 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Current context and the experience of Brazilian Medical Societies with Specialist Title Exams.
Rev Col Bras Cir
January 2025
- Faculdade de Medicina do ABC, Santo André, SP, Brasil. Presidente da Associação Médica Brasileira (AMB).
The editorial discusses the need for the Specialist Title to be valued by Medical Societies, as a stage after the conclusion of medical residency and a mandatory prerequisite for taking the tests. Finally, it shows the experience of seven Medical Societies with their specialist title exams.
View Article and Find Full Text PDFA Comparative Study of ChatGPT and BingAI in Answering the National Eligibility Entrance Test for Postgraduates (NEET-PG)-Style Practice Questions: A Cross-Sectional Analysis.
Cureus
December 2024
Internal Medicine, Ross University School of Medicine, Saint Michael, BRB.
Purpose: The integration of artificial intelligence (AI) into medical education has witnessed significant progress, particularly in the domain of language models. This study focuses on assessing the performance of two notable language models, ChatGPT and BingAI Precise, in answering the National Eligibility Entrance Test for Postgraduates (NEET-PG)-style practice questions, simulating medical exam formats.
Methods: A cross-sectional study conducted in June 2023 involved assessing ChatGPT and BingAI Precise using three sets of NEET-PG practice exams, comprising 200 questions each.
Cost-Benefit Analysis of in vivo Reflectance Confocal Microscopy for Melanoma Diagnosis in a Real-World Clinical Setting.
Risk Manag Healthc Policy
January 2025
Department of Clinical Internal, Anesthesiological and Cardiovascular Sciences, Dermatology Clinic, Sapienza University of Rome, Rome, Italy.
Background: In a recent prospective, multicenter, two-arm randomized controlled trial (RCT), we demonstrated that adjunctive reflectance confocal microscopy (RCM) in routine clinical practice provides clinical benefits, including safe melanoma detection and a 43.3% reduction in the number needed to excise (NNE).
Methods: A cost-benefit analysis was conducted based on NNEs for standard care (5.
Evaluating ASC-4 Transfer Rates in Cataract Surgery: Insights into Timing and Causes of Hospital Transfers.
J Cataract Refract Surg
January 2025
Wolfe Eye Clinic, West Des Moines, IA, US.
Purpose: To describe the causes, timing, and contributing factors of direct hospital transfer cases from an ophthalmology-specific ambulatory surgery center and to identify potential strategies for decreasing future transfers.
Setting: A large ophthalmology surgery center in Des Moines, Iowa.
Design: Retrospective review.
Genetic context modulates aging and degeneration in the murine retina.
Mol Neurodegener
January 2025
The Jackson Laboratory, Bar Harbor, ME, 04609, USA.
Background: Age is the principal risk factor for neurodegeneration in both the retina and brain. The retina and brain share many biological properties; thus, insights into retinal aging and degeneration may shed light onto similar processes in the brain. Genetic makeup strongly influences susceptibility to age-related retinal disease.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!