Systematic review identifies the design and methodological conduct of studies on machine learning-based prediction models.

J Clin Epidemiol

Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands; Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.

Published: February 2023

Background And Objectives: We sought to summarize the study design, modelling strategies, and performance measures reported in studies on clinical prediction models developed using machine learning techniques.

Methods: We search PubMed for articles published between 01/01/2018 and 31/12/2019, describing the development or the development with external validation of a multivariable prediction model using any supervised machine learning technique. No restrictions were made based on study design, data source, or predicted patient-related health outcomes.

Results: We included 152 studies, 58 (38.2% [95% CI 30.8-46.1]) were diagnostic and 94 (61.8% [95% CI 53.9-69.2]) prognostic studies. Most studies reported only the development of prediction models (n = 133, 87.5% [95% CI 81.3-91.8]), focused on binary outcomes (n = 131, 86.2% [95% CI 79.8-90.8), and did not report a sample size calculation (n = 125, 82.2% [95% CI 75.4-87.5]). The most common algorithms used were support vector machine (n = 86/522, 16.5% [95% CI 13.5-19.9]) and random forest (n = 73/522, 14% [95% CI 11.3-17.2]). Values for area under the Receiver Operating Characteristic curve ranged from 0.45 to 1.00. Calibration metrics were often missed (n = 494/522, 94.6% [95% CI 92.4-96.3]).

Conclusion: Our review revealed that focus is required on handling of missing values, methods for internal validation, and reporting of calibration to improve the methodological conduct of studies on machine learning-based prediction models.

Systematic Review Registration: PROSPERO, CRD42019161764.

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.jclinepi.2022.11.015DOI Listing

Publication Analysis

Top Keywords

prediction models
12
methodological conduct
8
conduct studies
8
studies machine
8
machine learning-based
8
learning-based prediction
8
study design
8
machine learning
8
[95%
8
studies
6

Similar Publications

Purpose: This study aimed to initially test whether machine learning approaches could categorically predict two simple biological features, mouse age and mouse species, using the retinal segmentation metrics.

Methods: The retinal layer thickness data obtained from C57BL/6 and DBA/2J mice were processed for machine learning after segmenting mouse retinal SD-OCT scans. Twenty-two models were trained to predict the mouse groups.

View Article and Find Full Text PDF

Rats and mice rapidly update timed behaviors.

Anim Cogn

January 2025

Neuroscience Department, Oberlin College, 173 Lorain St, Oberlin, OH, USA.

Keeping track of time intervals is a crucial aspect of behavior and cognition. Many theoretical models of how the brain times behavior make predictions for steady-state performance of well-learned intervals, but the rate of learning intervals in these models varies greatly, ranging from one-shot learning to learning over thousands of trials. Here, we explored how quickly rats and mice adapt to changes in interval durations using a serial fixed-interval task.

View Article and Find Full Text PDF

Immune checkpoint inhibitors have improved the treatment of metastatic renal cell carcinoma (RCC), with the combination of nivolumab (NIVO) and ipilimumab (IPI) showing promising results. However, not all patients benefit from these therapies, emphasizing the need for reliable, easily assessable biomarkers. This multicenter study involved 116 advanced RCC patients treated with NIVO + IPI across nine oncology centers in Poland.

View Article and Find Full Text PDF

Currently, the World Health Organization (WHO) grade of meningiomas is determined based on the biopsy results. Therefore, accurate non-invasive preoperative grading could significantly improve treatment planning and patient outcomes. Considering recent advances in machine learning (ML) and deep learning (DL), this meta-analysis aimed to evaluate the performance of these models in predicting the WHO meningioma grade using imaging data.

View Article and Find Full Text PDF

Background: T cells are involved in every stage of tumor development and significantly influence the tumor microenvironment (TME). Our objective was to assess T-cell marker gene expression profiles, develop a predictive risk model for human papilloma virus (HPV)-negative oral squamous cell carcinoma (OSCC) utilizing these genes, and examine the correlation between the risk score and the immunotherapy response.

Methods: We acquired scRNA-seq data for HPV-negative OSCC from the GEO datasets.

View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!