Publications by authors named "M Spotnitz"
Article Synopsis
- Large language models (LLMs) show potential in summarizing medical evidence, but using proprietary models can lead to issues like lack of transparency and reliance on specific vendors.
- This study focused on enhancing the performance of open-source LLMs by fine-tuning three models—PRIMERA, LongT5, and Llama-2—using a dataset of 8,161 systematic reviews and summaries.
- Fine-tuning resulted in significant performance improvements, with LongT5 performing similarly to GPT-3.5 in certain settings, indicating that smaller models can outperform larger models in specific tasks, like summarizing medical evidence.
Article Synopsis
- Large language models (LLMs) show potential in summarizing medical evidence but are often limited by issues such as lack of transparency when using proprietary models.
- This study examines the effects of fine-tuning open-source LLMs like PRIMERA, LongT5, and Llama-2 to enhance their performance, using a dataset of systematic reviews and summaries.
- Results indicate that fine-tuning improves the performance of open-source models, with LongT5 performing nearly as well as GPT-3.5, and smaller fine-tuned models sometimes outperforming larger models in evaluations.
Purpose: The specific aims of this paper are to (1) develop and operationalize an electronic health record (EHR) data quality framework, (2) apply the dimensions of the framework to the phenotype and treatment pathways of ductal carcinoma in situ (DCIS) using Research Program data, and (3) propose and apply a checklist to evaluate the application of the framework.
Methods: We developed a framework of five data quality dimensions (DQD; completeness, concordance, conformance, plausibility, and temporality). Participants signed a consent and Health Insurance Portability and Accountability Act authorization to share EHR data and responded to demographic questions in the Basics questionnaire.
Introduction: Electronic Health Records (EHR) are a useful data source for research, but their usability is hindered by measurement errors. This study investigated an automatic error detection algorithm for adult height and weight measurements in EHR for the All of Us Research Program (All of Us).
Methods: We developed reference charts for adult heights and weights that were stratified on participant sex.
Article Synopsis
- C2Q 3.0 is a new system that uses GPT-4 technology to automate the process of identifying eligible patients for clinical trials by turning trial eligibility texts into database queries.* -
- The system's performance was tested through concept extraction from clinical trials, where it scored 0.891 for accuracy, and it found multiple errors in the SQL queries generated, with logic errors being the most frequent.* -
- Overall, while C2Q 3.0 showed strong coherence in reasoning, there’s still room for improvement in readability, highlighting the need for further research to enhance the reliability of AI in clinical settings.*