Why the p Value Alone Is Not Enough: The Need for Confidence Intervals in Plastic Surgery Research.

Background: The p value is one of the most used descriptors in statistical analysis; however, when reported in isolation, it does not convey the effect size of a treatment. The reporting of confidence intervals is an essential adjunct to determine the clinical value of treatment, as it permits an assessment of the effect size. The authors assessed the reporting of confidence intervals in clinical trials within the plastic surgery literature.

Methods: The seven highest impact plastic surgery journals were screened using MEDLINE for clinical trials in the years 2006, 2009, 2012, and 2015. Studies were randomized based on a predetermined sample size, and various characteristics (e.g., Jadad quality score, reporting of statistical significance, journal impact factor, and participation of an individual with formal research training) were documented.

Results: Two independent reviewers analyzed 135 articles. There was substantial interrater agreement (kappa = 0.78). Although 86.7 percent of studies reported a p value, only 25.2 percent reported confidence intervals. Of all journals assessed, Plastic and Reconstructive Surgery most frequently reported confidence intervals. The quality of the studies had a median Jadad score of 2 of 5. Bivariate analysis revealed that higher Jadad score and involvement of an individual with formal research training were associated with reporting of confidence intervals. Multivariate analysis revealed similar findings, whereas journal impact factor, year of publication, and statistical significance were not correlated with confidence interval reporting.

Conclusions: Confidence intervals are underreported in the plastic surgery literature. To improve reporting quality of clinical trials, results should always include the confidence intervals to avoid misinterpretation of the effect size of a statistically significant result.