https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=29776399&retmode=xml&tool=Litmetric&email=readroberts32@gmail.com&api_key=61f08fa0b96a73de8c900d749fcb997acc09 297763992019060320210109
1471-22881812018May18BMC medical research methodologyBMC Med Res MethodolSimulation-based power calculations for planning a two-stage individual participant data meta-analysis.41414110.1186/s12874-018-0492-zResearchers and funders should consider the statistical power of planned Individual Participant Data (IPD) meta-analysis projects, as they are often time-consuming and costly. We propose simulation-based power calculations utilising a two-stage framework, and illustrate the approach for a planned IPD meta-analysis of randomised trials with continuous outcomes where the aim is to identify treatment-covariate interactions.The simulation approach has four steps: (i) specify an underlying (data generating) statistical model for trials in the IPD meta-analysis; (ii) use readily available information (e.g. from publications) and prior knowledge (e.g. number of studies promising IPD) to specify model parameter values (e.g. control group mean, intervention effect, treatment-covariate interaction); (iii) simulate an IPD meta-analysis dataset of a particular size from the model, and apply a two-stage IPD meta-analysis to obtain the summary estimate of interest (e.g. interaction effect) and its associated p-value; (iv) repeat the previous step (e.g. thousands of times), then estimate the power to detect a genuine effect by the proportion of summary estimates with a significant p-value.In a planned IPD meta-analysis of lifestyle interventions to reduce weight gain in pregnancy, 14 trials (1183 patients) promised their IPD to examine a treatment-BMI interaction (i.e. whether baseline BMI modifies intervention effect on weight gain). Using our simulation-based approach, a two-stage IPD meta-analysis has < 60% power to detect a reduction of 1 kg weight gain for a 10-unit increase in BMI. Additional IPD from ten other published trials (containing 1761 patients) would improve power to over 80%, but only if a fixed-effect meta-analysis was appropriate. Pre-specified adjustment for prognostic factors would increase power further. Incorrect dichotomisation of BMI would reduce power by over 20%, similar to immediately throwing away IPD from ten trials.Simulation-based power calculations could inform the planning and funding of IPD projects, and should be used routinely.EnsorJoieJ0000-0001-7481-0282Centre for Prognosis Research, Research Institute for Primary Care and Health Sciences, Keele University, Keele, Staffordshire, ST5 5BG, UK. j.ensor@keele.ac.uk.BurkeDanielle LDLCentre for Prognosis Research, Research Institute for Primary Care and Health Sciences, Keele University, Keele, Staffordshire, ST5 5BG, UK.SnellKym I EKIECentre for Prognosis Research, Research Institute for Primary Care and Health Sciences, Keele University, Keele, Staffordshire, ST5 5BG, UK.HemmingKarlaKInstitute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK.RileyRichard DRDCentre for Prognosis Research, Research Institute for Primary Care and Health Sciences, Keele University, Keele, Staffordshire, ST5 5BG, UK.engSRF-2017-10-002DH_Department of HealthUnited KingdomJournal ArticleMeta-AnalysisResearch Support, Non-U.S. Gov't20180518
EnglandBMC Med Res Methodol1009685451471-2288IMAlgorithmsBody Mass IndexComputer SimulationFemaleGestational Weight GainphysiologyHumansModels, StatisticalOverweightphysiopathologyprevention & controlPregnancyPregnancy Complicationsphysiopathologyprevention & controlRandomized Controlled Trials as TopicETHICS APPROVAL AND CONSENT TO PARTICIPATE: Not Applicable. COMPETING INTERESTS: The authors declare that they have no competing interests. PUBLISHER’S NOTE: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
201710242018415201852060201852060201964602018518epublish29776399PMC596020510.1186/s12874-018-0492-z10.1186/s12874-018-0492-zRiley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ. 2010;340:c221. doi: 10.1136/bmj.c221.10.1136/bmj.c22120139215Simmonds M, Stewart G, Stewart L. A decade of individual participant data meta-analyses: a review of current practice. Contemp Clin Trials. 2015;45:76–83. doi: 10.1016/j.cct.2015.06.012.10.1016/j.cct.2015.06.01226091948Huang Y, Mao C, Yuan J, et al. Distribution and epidemiological characteristics of published individual patient data meta-analyses. PLoS One. 2014;9:e100151. doi: 10.1371/journal.pone.0100151.10.1371/journal.pone.0100151PMC406379124945406Krumholz HM. Why data sharing should be the expected norm. BMJ. 2015;350:h599. doi: 10.1136/bmj.h599.10.1136/bmj.h59925655137Hingorani AD, Windt DA, Riley RD, et al. Prognosis research strategy (PROGRESS) 4: stratified medicine research. BMJ. 2013;346:e5793. doi: 10.1136/bmj.e5793.10.1136/bmj.e5793PMC356568623386361Brookes ST, Whitely E, Egger M, Smith GD, Mulheran PA, Peters TJ. Subgroup analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for the interaction test. J Clin Epidemiol. 2004;57:229–236. doi: 10.1016/j.jclinepi.2003.08.009.10.1016/j.jclinepi.2003.08.00915066682Hróbjartsson A. Why did it take 19 months to retrieve clinical trial data from a non-profit organisation? BMJ. 2013;347.24298001Pastorino U. Systematic review of multiple studies of prognosis: the feasibility of obtaining individual patient data. In: Altman DG, Trivella M, Pezzella F, Harris AL, Auget J-L, Balakrishnan N, Mesbah M, Molenberghs G, editors. Advances in statistical methods for the health sciences. Boston: Birkhäuser; 2006. pp. 3–18.Simmonds MC, Higgins JP. Covariate heterogeneity in meta-analysis: criteria for deciding between meta-regression and individual patient data. Stat Med. 2007;26:2982–2999. doi: 10.1002/sim.2768.10.1002/sim.276817195960Kovalchik SA. Aggregate-data estimation of an individual patient data linear random effects meta-analysis with a patient covariate-treatment interaction term. Biostatistics. 2013;14:273–283. doi: 10.1093/biostatistics/kxs035.10.1093/biostatistics/kxs035PMC359092423001065Kovalchik SA, Cumberland WG. Using aggregate data to estimate the standard error of a treatment-covariate interaction in an individual patient data meta-analysis. Biom J. 2012;54:370–384. doi: 10.1002/bimj.201100167.10.1002/bimj.20110016722685003Kontopantelis E, Springate DA, Parisi R, Reeves D. Simulation-Based Power Calculations for Mixed Effects Modeling: ipdpower in Stata. J Stat Softw. 2016;1(12).DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–188. doi: 10.1016/0197-2456(86)90046-2.10.1016/0197-2456(86)90046-23802833Hartung J, Knapp G. A refined method for the meta-analysis of controlled clinical trials with binary outcome. Stat Med. 2001;20:3875–3889. doi: 10.1002/sim.1009.10.1002/sim.100911782040IntHout J, Ioannidis JP, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Med Res Methodol. 2014;14:25. doi: 10.1186/1471-2288-14-25.10.1186/1471-2288-14-25PMC401572124548571Fisher DJ, Copas AJ, Tierney JF, Parmar MK. A critical review of methods for the assessment of patient-level interactions in individual participant data meta-analysis of randomized trials, and guidance for practitioners. J Clin Epidemiol. 2011;64:949–967. doi: 10.1016/j.jclinepi.2010.11.016.10.1016/j.jclinepi.2010.11.01621411280Hua H, Burke DL, Crowther MJ, Ensor J, Tudur Smith C, Riley RD. One-stage individual participant data meta-analysis models: estimation of treatment-covariate interactions must avoid ecological bias by separating out within-trial and across-trial information. Stat Med. 2017;36:772–789.PMC529954327910122Burke DL, Ensor J, Riley RD. Meta-analysis using individual participant data: one-stage and two-stage approaches, and why they may differ. Stat Med. 2016;PMC529799827747915Riley RD, Kauser I, Bland M, et al. Meta-analysis of randomised trials with a continuous outcome according to baseline imbalance and availability of individual participant data. Stat Med. 2013;32:2747–2766. doi: 10.1002/sim.5726.10.1002/sim.572623303608Whitehead A, Whitehead J. A general parametric approach to the meta-analysis of randomized clinical trials. Stat Med. 1991;10:1665–1677. doi: 10.1002/sim.4780101105.10.1002/sim.47801011051792461Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions: John Wiley & Sons. 2011.Veroniki AA, Jackson D, Viechtbauer W, et al. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods. 2016;7:55–79. doi: 10.1002/jrsm.1164.10.1002/jrsm.1164PMC495003026332144Kontopantelis E, Reeves D. Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: a simulation study. Stat Methods Med Res. 2012;21:409–426. doi: 10.1177/0962280210392008.10.1177/096228021039200821148194DerSimonian R, Kacker R. Random-effects model for meta-analysis of clinical trials: an update. Contemp Clin Trials. 2007;28:105–114. doi: 10.1016/j.cct.2006.04.004.10.1016/j.cct.2006.04.00416807131Knapp G, Hartung J. Improved tests for a random effects meta-regression with a single covariate. Stat Med. 2003;22:2693–2710. doi: 10.1002/sim.1482.10.1002/sim.148212939780Hartung J. An alternative method for meta-analysis. Biom J. 1999;41:901–916. doi: 10.1002/(SICI)1521-4036(199912)41:8<901::AID-BIMJ901>3.0.CO;2-W.10.1002/(SICI)1521-4036(199912)41:8<901::AID-BIMJ901>3.0.CO;2-WHartung J, Knapp G. On tests of the overall treatment effect in meta-analysis with normally distributed responses. Stat Med. 2001;20:1771–1782. doi: 10.1002/sim.791.10.1002/sim.79111406840Sidik K, Jonkman JN. A simple confidence interval for meta-analysis. Stat Med. 2002;21:3153–3159. doi: 10.1002/sim.1262.10.1002/sim.126212375296Sidik KJ, J. N. On constructing confidence intervals for a standardized mean difference in meta-analysis. Comm Statist-Simulation Comput. 2003;32:1191–1203. doi: 10.1081/SAC-120023885.10.1081/SAC-120023885Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika. 1934;26:404–413. doi: 10.1093/biomet/26.4.404.10.1093/biomet/26.4.404Thangaratinam S, Rogozinska E, Jolly K, et al. Effects of interventions in pregnancy on maternal weight and obstetric outcomes: meta-analysis of randomised evidence. BMJ. 2012;344:e2088. doi: 10.1136/bmj.e2088.10.1136/bmj.e2088PMC335519122596383Kahan BC, Rushton H, Morris TP, Daniel RM. A comparison of methods to adjust for continuous covariates in the analysis of randomised trials. BMC Med Res Methodol. 2016;16:42. doi: 10.1186/s12874-016-0141-3.10.1186/s12874-016-0141-3PMC482722327068456Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006;25:127–141. doi: 10.1002/sim.2331.10.1002/sim.233116217841Altman DG, Royston P. Statistics notes: the cost of dichotomising continuous variables. BMJ. 2006;332:1080. doi: 10.1136/bmj.332.7549.1080.10.1136/bmj.332.7549.1080PMC145857316675816Vickers AJ, Altman DG. Statistics notes: Analysing controlled trials with baseline and follow up measurements. BMJ. 2001;323:1123–1124. doi: 10.1136/bmj.323.7321.1123.10.1136/bmj.323.7321.1123PMC112160511701584Ahmed I, Sutton AJ, Riley RD. Assessment of publication bias, selection bias and unavailable data in meta-analyses using individual participant data: a database survey. BMJ. 2012;344:d7762. doi: 10.1136/bmj.d7762.10.1136/bmj.d776222214758Crowther MJ, Lambert PC. Simulating biologically plausible complex survival data. Stat Med. 2013;32:4118–4134. doi: 10.1002/sim.5823.10.1002/sim.582323613458Arnold BF, Hogan DR, Colford JM, Jr, Hubbard AE. Simulation methods to estimate design power: an overview for applied research. BMC Med Res Methodol. 2011;11:94. doi: 10.1186/1471-2288-11-94.10.1186/1471-2288-11-94PMC314695221689447Landau S, Stahl D. Sample size and power calculations for medical studies by simulation when closed form expressions are not available. Stat Methods Med Res. 2013;22:324–345. doi: 10.1177/0962280212439578.10.1177/096228021243957822491174Feiveson AH. Power by simulation. Stata J. 2002;2(2):107–24.Browne WJ, Golalizadeh LM, Parker RMA. A guide to sample size calculation for random effects models via simulation and the MLPowSim software package. University of Bristol. 2009.Simmonds MC, Higgins JPT, Stewart LA, Tierney JF, Clarke MJ, Thompson SG. Meta-analysis of individual patient data from randomized trials: a review of methods used in practice. Clinical Trials. 2005;2:209–217. doi: 10.1191/1740774505cn087oa.10.1191/1740774505cn087oa16279144Fisher DJ. Two-stage individual participant data meta-analysis and generalized forest plots. Stata J. 2015;15:369–396.Partlett C, Riley RD. Random effects meta-analysis: coverage performance of 95% confidence and prediction intervals following REML estimation. Stat Med. 2017;36:301–317.PMC515776827714841Abo-Zaid G, Sauerbrei W, Riley RD. Individual participant data meta-analysis of prognostic factor studies: state of the art? BMC Med Res Methodol. 2012;12:56. doi: 10.1186/1471-2288-12-56.10.1186/1471-2288-12-56PMC341357722530717