We aimed to assess ChatGPT's performance on the Clinical Informatics Board Examination and to discuss the implications of large language models (LLMs) for board certification and maintenance. We tested ChatGPT using 260 multiple-choice questions from Mankowitz's Clinical Informatics Board Review book, omitting 6 image-dependent questions. ChatGPT answered 190 (74%) of 254 eligible questions correctly. While performance varied across the Clinical Informatics Core Content Areas, differences were not statistically significant. ChatGPT's performance raises concerns about the potential misuse in medical certification and the validity of knowledge assessment exams. Since ChatGPT is able to answer multiple-choice questions accurately, permitting candidates to use artificial intelligence (AI) systems for exams will compromise the credibility and validity of at-home assessments and undermine public trust. The advent of AI and LLMs threatens to upend existing processes of board certification and maintenance and necessitates new approaches to the evaluation of proficiency in medical education.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10436139 | PMC |
| http://dx.doi.org/10.1093/jamia/ocad104 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Implementing Findable, Accessible, Interoperable, Reusable (FAIR) Principles in Child and Adolescent Mental Health Research: Mixed Methods Approach.
JMIR Ment Health
December 2024
Karakter Child and Adolescent Psychiatry University Centre, Nijmegen, Netherlands.
Background: The FAIR (Findable, Accessible, Interoperable, Reusable) data principles are a guideline to improve the reusability of data. However, properly implementing these principles is challenging due to a wide range of barriers.
Objectives: To further the field of FAIR data, this study aimed to systematically identify barriers regarding implementing the FAIR principles in the area of child and adolescent mental health research, define the most challenging barriers, and provide recommendations for these barriers.
The role of sonic hedgehog signaling in the oropharyngeal epithelium during jaw development.
Congenit Anom (Kyoto)
December 2024
Department of Molecular Craniofacial Embryology and Oral Histology, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University (TMDU), Tokyo, Japan.
Sonic hedgehog (Shh) is expressed in the oropharyngeal epithelium, including the frontonasal ectodermal zone (FEZ), which is defined as the boundary between Shh and Fgf8 expression domains in the frontonasal epithelium. To investigate the role of SHH signaling from the oropharyngeal epithelium, we generated mice in which Shh expression is specifically deleted in the oropharyngeal epithelium (Isl1-Cre; Shh). In the mutant mouse, Shh expression was excised in the oropharyngeal epithelium as well as FEZ and ventral forebrain, consistent with the expression pattern of Isl1.
View Article and Find Full Text PDFClinical Impact of TP53 Mutations in Patients with Head and Neck Cancer Who Were Treated with Targeted Therapies or Immunotherapy.
Cancer Res Treat
December 2024
Divison of Medical Oncology, Department of Internal Medicine, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Korea.
Purpose: TP53 mutations are common in head and neck squamous cell carcinoma (HNSCC). We evaluated their clinical impact in patients treated with targeted agents or immunotherapy in the KCSG HN15-16 TRIUMPH trial.
Materials And Methods: We analyzed clinical characteristics and outcomes of patients with TP53 mutations in the TRIUMPH trial, a multicenter, biomarker-driven umbrella trial in Korea.
Identifying Symptoms of Delirium from Clinical Narratives Using Natural Language Processing.
Proc (IEEE Int Conf Healthc Inform)
June 2024
Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, FL, USA.
Delirium is an acute decline or fluctuation in attention, awareness, or other cognitive function that can lead to serious adverse outcomes. Despite the severe outcomes, delirium is frequently unrecognized and uncoded in patients' electronic health records (EHRs) due to its transient and diverse nature. Natural language processing (NLP), a key technology that extracts medical concepts from clinical narratives, has shown great potential in studies of delirium outcomes and symptoms.
View Article and Find Full Text PDFIdentification and characterization of a novel chromosome-encoded aminoglycoside -nucleotidyltransferase gene, in sp. TYF-12 isolated from the marine fish intestine.
Front Microbiol
December 2024
Department of Laboratory Sciences, The People's Hospital of Yuhuan, Yuhuan, China.
Background: The mechanisms underlying the resistance of the genus to aminoglycosides are complex, which poses a challenge for the efficient treatment of infectious diseases caused by these pathogens. To help clinicians treat infections more effectively, a more comprehensive understanding of antibiotic resistance mechanisms is urgently needed.
Methods: Plates were streaked to isolate bacteria from the intestinal contents of fish.
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!