Poor allocation concealment methods are associated with heterogeneity in age and statistical significance of the primary outcome: Review of recent trials published in four general medical journals.

Objective: To assess the association of the quality of allocation concealment with heterogeneity in age, the P value of the primary outcome and statistical significance of the primary outcome.

Study Design And Setting: We extracted data from articles published in four major medical journals in 2017 and 2018 that reported the results of randomized controlled trials. The outcome measures were the quality of allocation concealment used in the trial, the P value of the primary outcome, whether the P value of the primary outcome was statistically significant and the level of heterogeneity in age between the treatment groups (measured using the I statistic). The association between the quality of allocation concealment and the P value of the primary outcome was assessed using a kernel density plot, while the association between the quality of allocation concealment and whether the P value was statistically significant was assessed using logistic regression.

Results: Trials that used inadequate concealment methods were more likely to report statistically significant findings than trials that used good or adequate methods (OR 1.90; 95% CI: 0.91 to 3.95; P = .09). The values of I for trials that used good, adequate, inadequate and unclear concealment methods were 0%, 1.0%, 32.6%, and 93.8%, respectively.

Conclusion: There is evidence of an association between poor allocation concealment methods and statistical significance of the primary outcome. Trials that use inadequate allocation concealment methods are more likely to have statistically significant P values compared with trials using good or adequate allocation concealment methods.