Introduction: Publicly available AI language models such as ChatGPT have demonstrated utility in text generation and even problem-solving when provided with clear instructions. Amidst this transformative shift, the aim of this study is to assess ChatGPT's performance on the orthopaedic surgery in-training examination (OITE).
Methods: All 213 OITE 2021 web-based questions were retrieved from the AAOS-ResStudy website (https://www.aaos.org/education/examinations/ResStudy). Two independent reviewers copied and pasted the questions and response options into ChatGPT Plus (version 4.0) and recorded the generated answers. All media-containing questions were flagged and carefully examined. Twelve OITE media-containing questions that relied purely on images (clinical pictures, radiographs, MRIs, CT scans) and could not be rationalized from the clinical presentation were excluded. Cohen's Kappa coefficient was used to examine the agreement of ChatGPT-generated responses between reviewers. Descriptive statistics were used to summarize the performance (% correct) of ChatGPT Plus. The 2021 norm table was used to compare ChatGPT Plus' performance on the OITE to national orthopaedic surgery residents in that same year.
Results: A total of 201 questions were evaluated by ChatGPT Plus. Excellent agreement was observed between raters for the 201 ChatGPT-generated responses, with a Cohen's Kappa coefficient of 0.947. 45.8% (92/201) were media-containing questions. ChatGPT had an average overall score of 61.2% (123/201). Its score was 64.2% (70/109) on non-media questions. When compared to the performance of all national orthopaedic surgery residents in 2021, ChatGPT Plus performed at the level of an average PGY3.
Discussion: ChatGPT Plus is able to pass the OITE with an overall score of 61.2%, ranking at the level of a third-year orthopaedic surgery resident. It provided logical reasoning and justifications that may help residents improve their understanding of OITE cases and general orthopaedic principles. Further studies are still needed to examine their efficacy and impact on long-term learning and OITE/ABOS performance.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC11025881 | PMC |
| http://dx.doi.org/10.2106/JBJS.OA.23.00103 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
The secretion of TGF-β3 by adipose-derived stem cells inhibits melanin synthesis and its impact on the cAMP/PKA signaling pathway.
Arch Dermatol Res
January 2025
Burn and Wound Repair Center, The Third Hospital of Hebei Medical University, No. 139, Ziqiang Road, Shijiazhuang, Hebei Province, 050035, China.
This study aimed to investigate the role of transforming growth factor-beta 3 (TGF-β3) secreted by adipose-derived stem cells (ADSCs) in suppressing melanin synthesis during the wound healing process, particularly in burn injuries, and to explore the underlying mechanisms involving the cAMP/PKA signaling pathway. ADSCs were isolated from C57BL/6 mice and characterized using flow cytometry and differentiation assays. A burn injury model was established in mice, followed by UVB irradiation to induce hyperpigmentation.
View Article and Find Full Text PDFComparison of the natural course of clinical and radiologic features in 13 patients with TRPV4-related skeletal dysplasias.
Pediatr Radiol
January 2025
Department of Pediatric Genetics, Istanbul University-Cerrahpaşa, Cerrahpasa Medical Faculty, 34098, Cerrahpasa, Istanbul, Turkey.
Background: Heterozygous TRPV4 mutations cause a group of skeletal dysplasias characterized by short stature, short trunk, and skeletal deformities.
Objective: The aim of this study is to compare the natural history of clinical and radiologic features of patients with different TRPV4-related skeletal dysplasias.
Materials And Methods: Thirteen patients with a mutation in TRPV4 were included in the study, and 11 were followed for a median of 6.
Sex-Based Differences in the Arthroscopic Treatment of Femoroacetabular Impingement Syndrome: 10-Year Outcomes With a Nested Propensity-Matched Comparison.
Am J Sports Med
January 2025
American Hip Institute Research Foundation, Des Plaines, Illinois, USA.
Background: Sex has been associated with different pathologic characteristics in painful hips undergoing hip arthroscopic surgery.
Purpose: To compare minimum 10-year patient-reported outcomes (PROs) and survivorship in patients who underwent primary hip arthroscopic surgery for femoroacetabular impingement syndrome and labral tears according to sex.
Study Design: Cohort study; Level of evidence, 3.
Coronal Plane Alignment of the Medial Column in Isolated Hallux Valgus, Isolated Flatfoot, and Combined Hallux Valgus-Flatfoot.
Foot Ankle Int
January 2025
Department of Orthopedics and Rehabilitation, Carver College of Medicine, University of Iowa, Iowa City, IA, USA.
Background: To provide improved treatment for hallux valgus (HV), we sought to understand more about the pathophysiologic connection between flatfoot deformity and HV by comparing coronal plane alignment of the medial column of the foot for patients with isolated HV, isolated flatfoot, and combined HV-flatfoot vs controls.
Methods: This study retrospectively assessed a consecutive series of 33 patients with combined symptomatic and radiographic HV and flatfoot, 33 isolated symptomatic HV, 33 isolated symptomatic flatfoot, and 33 controls. The medial column alignment was assessed in the coronal plane using 3-dimensional weightbearing computed tomography (WBCT); rotation was measured for the navicular, medial cuneiform, and first metatarsal (M1).
Symptomatic Accessory Navicular Treated With Endoscopic Accessory Navicular and Partial Navicular Resection.
Foot Ankle Int
January 2025
Center for Foot and Ankle Surgery, Department of Orthopedic Surgery, Yashio Central General Hospital, Saitama, Japan.
Background: This study aims to report the results of the patients with symptomatic accessory navicular (AN) who underwent endoscopic AN and partial navicular resection.
Methods: The medical records of patients with type 2 symptomatic AN who underwent the aforementioned surgery at our hospital from November 2019 to May 2022 with a follow-up of >2 years were reviewed. Data on clinical, radiographic, and patient-reported outcomes were obtained.
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!