Classifying responders and non-responders; does it help when there is evidence of differentially responding patient groups?

Introduction: Continuous trial outcomes are often dichotomized into 'response' and 'non-response' categories prior to statistical analysis. This facilitates the interpretation of results, but generally reduces statistical power. Exceptions may occur when response in the study population is heterogeneous, and outcomes are bimodally distributed. We explore whether bimodality is present in antidepressant trial data and whether dichotomizing then indeed results in more powerful statistical tests.

Methods: The distributions of relative changes from baseline (rCFB) on the Hamilton depression rating scale (HAM-D) were estimated using pooled data from nine antidepressant trials. T-tests on rCFB scores and chi-square tests on dichotomized outcomes were compared to assess the consequences of dichotomization, using both the commonly applied cutoff (i.e. rCFB > 50%) and an estimated cutoff that provided optimal separation of the mixture of two normal distributions that best fitted the pooled placebo outcomes. The power of both tests was also evaluated for simulated scenario's that varied the degree of bimodality and the treatment effect and sample size.

Results: Placebo and treatment groups showed evidence of bimodality. The estimated cutoff closely matched the commonly applied cutoff. Nevertheless, t-tests generally yielded smaller p-values than chi-square tests. Simulations showed that dichotomization only provides superior power when bimodality was considerably more marked than observed in the empirical data.

Conclusion: Antidepressant trial outcomes showed bimodality, suggesting differential response among patient groups. This heterogeneity in outcome distributions should be reported more often, since a comparison of means does not adequately summarize the differences between treatment groups. However, simply dichotomizing outcomes is not an appropriate alternative as it reduces statistical power.