JMIR Ment Health
Brightside Health, San Francisco, CA, United States.
Published: August 2024
Background: Due to recent advances in artificial intelligence, large language models (LLMs) have emerged as a powerful tool for a variety of language-related tasks, including sentiment analysis, and summarization of provider-patient interactions. However, there is limited research on these models in the area of crisis prediction.
Objective: This study aimed to evaluate the performance of LLMs, specifically OpenAI's generative pretrained transformer 4 (GPT-4), in predicting current and future mental health crisis episodes using patient-provided information at intake among users of a national telemental health platform.
Methods: Deidentified patient-provided data were pulled from specific intake questions of the Brightside telehealth platform, including the chief complaint, for 140 patients who indicated suicidal ideation (SI), and another 120 patients who later indicated SI with a plan during the course of treatment. Similar data were pulled for 200 randomly selected patients, treated during the same time period, who never endorsed SI. In total, 6 senior Brightside clinicians (3 psychologists and 3 psychiatrists) were shown patients' self-reported chief complaint and self-reported suicide attempt history but were blinded to the future course of treatment and other reported symptoms, including SI. They were asked a simple yes or no question regarding their prediction of endorsement of SI with plan, along with their confidence level about the prediction. GPT-4 was provided with similar information and asked to answer the same questions, enabling us to directly compare the performance of artificial intelligence and clinicians.
Results: Overall, the clinicians' average precision (0.7) was higher than that of GPT-4 (0.6) in identifying the SI with plan at intake (n=140) versus no SI (n=200) when using the chief complaint alone, while sensitivity was higher for the GPT-4 (0.62) than the clinicians' average (0.53). The addition of suicide attempt history increased the clinicians' average sensitivity (0.59) and precision (0.77) while increasing the GPT-4 sensitivity (0.59) but decreasing the GPT-4 precision (0.54). Performance decreased comparatively when predicting future SI with plan (n=120) versus no SI (n=200) with a chief complaint only for the clinicians (average sensitivity=0.4; average precision=0.59) and the GPT-4 (sensitivity=0.46; precision=0.48). The addition of suicide attempt history increased performance comparatively for the clinicians (average sensitivity=0.46; average precision=0.69) and the GPT-4 (sensitivity=0.74; precision=0.48).
Conclusions: GPT-4, with a simple prompt design, produced results on some metrics that approached those of a trained clinician. Additional work must be done before such a model can be piloted in a clinical setting. The model should undergo safety checks for bias, given evidence that LLMs can perpetuate the biases of the underlying data on which they are trained. We believe that LLMs hold promise for augmenting the identification of higher-risk patients at intake and potentially delivering more timely care to patients.
Download full-text PDF |
Source |
---|---|
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC11329850 | PMC |
http://dx.doi.org/10.2196/58129 | DOI Listing |
J Orthop Res
January 2025
Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, New York, New York, USA.
Previous studies suggest a relationship between femoroacetabular impingement (FAI) and femoral neck stress fractures (FNSF), due to pathologic biomechanics in the setting of femoral head abutment (cam morphology) and/or acetabular overcoverage (pincer morphology). The purpose of this study is to evaluate the association between cam or pincer morphology and FNSF, compared to a control group of patients without hip pain. A retrospective review of the electronic medical record at a single institution was queried for patients with FNSF over a 10-year time period from January 2011-2021.
View Article and Find Full Text PDFJ Emerg Med
January 2025
Emergency Medicine Department, Salmaniya Medical Complex, Kingdom of Bahrain.
Background: Emergency departments (EDs) around the world are facing a crippling crisis of overcrowding, a complex problem caused by a variety of factors. One contributing factor is the overutilization of EDs by patients with frequent visits.
Objective: This study aims at measuring the prevalence of this phenomenon and better understanding the characteristics of high utilizers.
Urology
January 2025
Department of Urology, Glickman Urological and Kidney Institute, Cleveland Clinic Foundation, Cleveland, Ohio, USA.
Objective: To measure patient knowledge about Benign prostatic hyperplasia (BPH) and identify factors associated with knowledge deficiencies among those newly presenting to our urology clinic.
Methods: Adult men presenting as new patients to our institution's urology clinic regardless of chief complaint were invited to complete a 26-item multiple choice questionnaire to assess basic knowledge about BPH, related symptomatology, and treatment options prior to their initial consultation. Responses were correlated to demographic variables using ANOVA and multivariable linear modeling.
Radiol Phys Technol
January 2025
Department of Radiological Sciences, Ibaraki Prefectural University of Health Sciences, 4669-2, Ami, Ibaraki, 300-0394, Japan.
The purpose of this study is to clarify the influence of acquiring medical record information and laboratory data on the sensitivity of detecting imaging findings among Japanese radiological technologists (RTs). RTs were presented with patient's information in three distinct sequences for detecting imaging findings. True positives (TP) were identified and categorized into three groups: Group 1 (image + chief complaint), Group 2 (image + chief complaint + medical record), and Group 3 (image + chief complaint + medical record + laboratory data).
View Article and Find Full Text PDFCureus
December 2024
Otolaryngology-Head and Neck Surgery, Kagawa University, Takamatsu, JPN.
Primary nodular fasciitis of the nasal cavity is quite rare, and only a few cases have been reported. The patient was a 40-year-old man whose chief complaint was a nasal tumor. We suspected fibrosarcoma and operated.
View Article and Find Full Text PDFEnter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!
© LitMetric 2025. All rights reserved.