Purpose: The use of AI-powered technology, particularly OpenAI's ChatGPT, holds significant potential to reshape healthcare and medical education. Despite existing studies on the performance of ChatGPT in medical licensing examinations across different nations, a comprehensive, multinational analysis using rigorous methodology is currently lacking. Our study sought to address this gap by evaluating the performance of ChatGPT on six different national medical licensing exams and investigating the relationship between test question length and ChatGPT's accuracy.
Methods: We manually inputted a total of 1,800 test questions (300 each from US, Italian, French, Spanish, UK, and Indian medical licensing examination) into ChatGPT, and recorded the accuracy of its responses.
Results: We found significant variance in ChatGPT's test accuracy across different countries, with the highest accuracy seen in the Italian examination (73% correct answers) and the lowest in the French examination (22% correct answers). Interestingly, question length correlated with ChatGPT's performance in the Italian and French state examinations only. In addition, the study revealed that questions requiring multiple correct answers, as seen in the French examination, posed a greater challenge to ChatGPT.
Conclusion: Our findings underscore the need for future research to further delineate ChatGPT's strengths and limitations in medical test-taking across additional countries and to develop guidelines to prevent AI-assisted cheating in medical examinations.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC11082010 | PMC |
| http://dx.doi.org/10.1007/s10439-023-03338-3 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
CaMKIIγ advances chronic intermittent hypoxia-induced cardiomyocyte apoptosis via HIF-1 signaling pathway.
Sleep Breath
January 2025
Nantong Key Laboratory of Translational Medicine in Cardiothoracic Diseases, and Research Institution of Translational Medicine in Cardiothoracic Diseases, Affiliated Hospital of Nantong University, Nantong, Jiangsu, 226001, China.
Background: Our previous study have demonstrated chronic intermittent hypoxia (CIH) induced cardiomyocyte apoptosis and cardiac dysfunction. However, the molecular mechanisms are complicated and varied. In this study, we first investigated the CaMKIIγ expression and signaling pathway in the pathogenesis of cardiomyocyte apoptosis after CIH.
View Article and Find Full Text PDFRapid and Sustained Resolution of Peristomal Pyoderma Gangrenosum With Aerosol Steroid Treatment.
J Wound Ostomy Continence Nurs
January 2025
Kyriaki Stefania Mitsaki, MBBCh, BSc (Hons), MSc, MRCP, Department of Dermatology, Northwick Park Hospital, London North West University Hospital NHS Trust, London, UK.
Background: Peristomal pyoderma gangrenosum (PPG) is a non-infectious neutrophilic dermatosis most commonly seen in the context of ostomies in inflammatory bowel disease. The lack of established treatment guidelines and high-quality evidence in the form of randomized controlled trials present a major challenge in PPG management, owing to the rarity of the condition. Treatment can be further complicated by difficulties in maintaining the stoma pouch seal with conventional topical corticosteroids.
View Article and Find Full Text PDFPsychometric accuracy of the Dutch Child and Adolescent Trauma Screener.
Eur J Psychotraumatol
December 2025
Department of Child and Adolescent Psychiatry, Amsterdam University Medical Centre, Amsterdam, The Netherlands.
The aim of this study is to investigate the psychometrics of the Dutch version of the Child and Adolescent Trauma Screener (CATS-2). By this, an international recognized instrument to screen symptoms of post-traumatic stress (PTSS) in children and adolescents according to the Diagnostic and Statistical Manual for Mental Disorders, 5th edition (DSM-5) becomes available for Dutch youth. Based on the validated CATS-2 we established the Dutch version, named the KJTS.
View Article and Find Full Text PDFEvaluating the Safety and Usability of an Over-the-Counter Medical Device for Adults With Mild to Moderate Hearing Loss: Formative and Summative Usability Testing.
JMIR Hum Factors
January 2025
Center for Research and Innovation in Systems Safety, Department of Anesthesiology, Vanderbilt University Medical Center, 2525 West End Avenue, Suite 800, Nashville, TN, 37203, United States, 16153431528.
Background: Only 15% of the nearly 30 million Americans with hearing loss use hearing aids, partly due to high cost, stigma, and limited access to professional hearing care. Hearing impairment in adults can lead to social isolation and depression and is associated with an increased risk of falls. Given the persistent barriers to hearing aid use, the Food and Drug Administration issued a final rule to allow over-the-counter hearing aids to be sold directly to adult consumers with perceived mild to moderate hearing loss at pharmacies, stores, and online retailers without seeing a physician or licensed hearing health care professional.
View Article and Find Full Text PDFNext generation bioelectronic medicine: making the case for non-invasive closed-loop autonomic neuromodulation.
Bioelectron Med
January 2025
SecondWave Systems Incorporated, Head Quarters, Minneapolis-Saint Paul, MN, 55104, USA.
The field of bioelectronic medicine has advanced rapidly from rudimentary electrical therapies to cutting-edge closed-loop systems that integrate real-time physiological monitoring with adaptive neuromodulation. Early innovations, such as cardiac pacemakers and deep brain stimulation, paved the way for these sophisticated technologies. This review traces the historical and technological progression of bioelectronic medicine, culminating in the emerging potential of closed-loop devices for multiple disorders of the brain and body.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!