Beyond hazard ratios: appropriate statistical methods for quantifying the clinical effectiveness of immune-oncology therapies - the example of the Netherlands.

Background: The Dutch Committee for the Evaluation of Oncological Drugs evaluates the effectiveness of new oncological treatments. The committee compares survival endpoints to the so-called PASKWIL-2023 criteria for palliative treatments, which define if treatment effects are considered clinically relevant. A positive recommendation depends on whether the median overall survival (OS) is below or above 12 months in the comparator arm. If the former applies, an OS benefit of at least 12 weeks, and a hazard ratio (HR) smaller than 0.7 are required. If the latter applies, an OS or progression free survival (PFS) benefit of at least 16 weeks, and an HR smaller than 0.7 are required. Nonetheless, the median survival time may not be reached and the proportional hazards (PH) assumption, quantified by the HR, is likely violated for immuno-oncology (IO) therapies, deeming these criteria inappropriate.

Methods: We conducted a systematic literature review to identify statistical methods used to represent the clinical effectiveness of IO therapies based on trial data. We searched MEDLINE and EMBASE databases from inception to August 31, 2022, limited to English papers. Methodological studies, randomized controlled trials, and discussion papers recognising key issues of survival data analysis of IO therapies were eligible for inclusion.

Results: A total of 1,035 unique references were identified. After full paper screening, 17 publications were included in the review. Additionally, 43 papers were identified through 'snowballing'. We conclude that the current PASKWIL-2023 criteria are methodologically incorrect under non-PH. In that case, single summary statistics fail to capture the treatment effect and any measure should be interpreted in combination with the Kaplan-Meier curves. We recommend 'parameter-free' measures, such as the difference in restricted mean survival time, avoiding assumptions on the underlying survival.

Conclusions: The HR is commonly used to assess treatment effectiveness, without investigating the validity of the PH assumption. This happens with the application of the PASKWIL-2023 criteria for palliative oncology treatments, which can only be valid under a PH setting. Under non-PH, alternative treatment effect measures are suggested. We propose a step-by-step approach supporting the choice of the most appropriate methods to quantify treatment effectiveness that can be used to redefine the PASKWIL-2023 criteria, or similar criteria in other clinical areas.