Learning curve analyses in spine surgery: a systematic simulation-based critique of methodologies.

Background Context: Various statistical approaches exist to delineate learning curves in spine surgery. Techniques range from dividing cases into intervals for metric comparison, to employing regression and cumulative summation (CUSUM) analyses. However, their inherent inconsistencies and methodological flaws limit their comparability and reliability.

Purpose: To critically evaluate the methodologies used in existing literature for studying learning curves in spine surgery and to provide recommendations for future research.

Study Design: Systematic literature review.

Methods: A comprehensive literature search was conducted using PubMed, Embase, and Scopus databases, covering articles from January 2010 to September 2023. For inclusion, articles had to evaluate the change in a metric of performance during human spine surgery across time/a case series. Results had to be reported in sufficient detail to allow for evaluation of individual performance rather than group/institutional performance. Articles were excluded if they included cadaveric/nonhuman subjects, aggregated performance data or no way to infer change across a number of cases. Risk of bias was assessed using the Risk of Bias in Nonrandomized Studies of Interventions (ROBINS-I) tool. Surgical data were simulated using Python 3 and then examined via multiple commonly used analytic approaches including division into consecutive intervals, regression and CUSUM techniques. Results were qualitatively assessed to determine the effectiveness and limitations of each approach in depicting a learning curve.

Results: About 113 studies met inclusion criteria. The majority of the studies were retrospective and evaluated a single-surgeon's experience. Methods varied considerably, with 66 studies using a single proficiency metric and 47 using more than 1. Operating time was the most commonly used metric. Interval division was the simplest and most commonly used method yet inherent limitations prevent collective synthesis. Regression may accurately describe the learning curve but in practice is hampered by sample size and model choice. CUSUM analyses are of widely varying quality with some being fundamentally flawed and widely misinterpreted however, others provide a reliable view of the learning process.

Conclusion: There is considerable variation in the quality of existing studies on learning curves in spine surgery. CUSUM analyses, when correctly applied, offer the most reliable estimates. To improve the validity and comparability of future studies, adherence to methodological guidelines is crucial. Multiple or composite performance metrics are necessary for a holistic understanding of the learning process.