Background: NLPs such as ChatGPT are novel sources of online healthcare information that are readily accessible and integrated into internet search tools. The accuracy of NLP-generated responses to health information questions is unknown.
Methods: We queried four NLPs (ChatGPT 3.5 and 4, Bard, and Claude 2.0) for responses to simulated patient questions about inguinal hernias and their management. Responses were graded on a Likert scale (1 poor to 5 excellent) for relevance, completeness, and accuracy. Responses were compiled and scored collectively for readability using the Flesch-Kincaid score and for educational quality using the DISCERN instrument, a validated tool for evaluating patient information materials. Responses were also compared to two gold-standard educational materials provided by SAGES and the ACS. Evaluations were performed by six hernia surgeons.
Results: The average NLP response scores for relevance, completeness, and accuracy were 4.76 (95% CI 4.70-4.80), 4.11 (95% CI 4.02-4.20), and 4.14 (95% CI 4.03-4.24), respectively. ChatGPT4 received higher accuracy scores (mean 4.43 [95% CI 4.37-4.50]) than Bard (mean 4.06 [95% CI 3.88-4.26]) and Claude 2.0 (mean 3.85 [95% CI 3.63-4.08]). The ACS document received the best scores for reading ease (55.2) and grade level (9.2); however, none of the documents achieved the readibility thresholds recommended by the American Medical Association. The ACS document also received the highest DISCERN score of 63.5 (57.0-70.1), and this was significantly higher compared to ChatGPT 4 (50.8 [95% CI 46.2-55.4]) and Claude 2.0 (48 [95% CI 41.6-54.4]).
Conclusions: The evaluated NLPs provided relevant responses of reasonable accuracy to questions about inguinal hernia. Compiled NLP responses received relatively low readability and DISCERN scores, although results may improve as NLPs evolve or with adjustments in question wording. As surgical patients expand their use of NLPs for healthcare information, surgeons should be aware of the benefits and limitations of NLPs as patient education tools.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1007/s00464-024-11221-y | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
A Novel Technique for Inguinal Hernia Repair in Patients With Nonmalignant Ascites.
Cureus
December 2024
Surgery, Morsani College of Medicine, University of South Florida, Tampa, USA.
Introduction: We report a novel approach to open inguinal hernia repair in patients with known ascites in which the cord, hernia sac, and attached testicle on the affected side are repositioned into the retroperitoneum through the inguinal ring. By avoiding invasion of the peritoneum and limiting dissection of the sac off the spermatic cord, the risk of ascites leak and testicular ischemia is theoretically decreased.
Methodology: This is a retrospective case series report.
Strangulated Amyand's hernia with testicular necrosis in an adult: A case report.
Int J Surg Case Rep
January 2025
Department of Medicine, College of Medicine and Health Sciences, University of Gondar, Gondar, Ethiopia.
Introduction: Amyand's hernia is a rare condition defined by the presence of the vermiform appendix within an inguinal hernia sac. The occurrence of Amyand's hernia with testicular necrosis is particularly uncommon, further complicating its clinical presentation and management.
Case Presentation: A 50-year-old male presented with a two-year history of progressive right scrotal swelling, acutely worsened over four days with pain and fever.
Comparison of trans-umbilical single-port laparoscopic complete extraperitoneal closure and laparoscopic intracorporeal closure for pediatric inguinal hernia: a randomized controlled study.
Front Pediatr
December 2024
Department of Pediatric Surgery, Children Hospital of Guizhou Province, Affiliated Hospital of Zunyi Medical University, Zunyi, Guizhou, China.
Background: The purpose of this study was to compare the outcomes of Trans-umbilical single-port laparoscopic complete extraperitoneal closure (LCEC) and laparoscopic intracorporeal closure (LIC) for inguinal hernia by analysis of follow-up data over 5 years.
Methods: In this prospective randomized controlled trial, 524 children with inguinal hernia were randomly assigned to undergo LCEC or LIC between August 2016 and December 2017. The primary outcome measures were the success and recurrence rates.
Surgeons transitioning from laparoscopic to robotic-assisted inguinal hernia repair: a prospective analysis of efficiency.
Hernia
January 2025
Department of Surgery, Baptist Health South Florida, Miami, FL, USA.
Article Synopsis
- The study assesses surgeon efficiency when transitioning from laparoscopic to robotic-assisted inguinal hernia repair across multiple centers and surgeons.
- Experienced surgeons with no prior robotic experience participated, measuring efficiency by procedure durations and outcomes post-surgery.
- The results show that surgeons improved their efficiency over time, especially in unilateral repairs during the mid-phase of the study, although variations existed between individuals and factors like practice patterns and external circumstances impacted the results.
Chronic postoperative inguinal pain (CPIP) after pediatric inguinal hernia repair-a retrospective analysis.
Hernia
January 2025
Department of General, Visceral, Transplant, Vascular and Pediatric Surgery, University Hospital of Wuerzburg, Wuerzburg, Germany.
Background: Surgical treatment of inguinal hernias in children is one of the most common operative procedures worldwide. During surgery for inguinal hernias in adults, chronic pain develops in approximately 10% of all cases. In children, there has been limited research to determine whether they may also develop this chronic postsurgical inguinal pain (CPIP).
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!