Bias in animal studies of estrogen effects on cardiovascular disease: A systematic review and meta-analysis.

Background: Randomized controlled trials on menopausal hormone therapy in humans have not confirmed the benefit of estrogens on cardiovascular disease found in animal studies. Flawed methodology or publication bias in animal studies may explain the dicrepancy.

Objectives: The aim of this study was to investigate whether publication of the randomized controlled trials Heart and Estrogen/Progestin Replacement Study and Women's Health Initiative influenced study authors' assessment of research findings (confirmation bias) as well as to investigate publication bias and small-study effects in animal studies of estrogen effects on atherosclerosis.

Methods: The data source for this study was PubMed from inception to 2018. We selected animal studies with cardiovascular outcomes comparing 17-β-estradiol, its natural metabolites, or conjugated equine estrogen with control. Qualitative data were extracted on authors' conclusions about estrogen effects on cardiovascular disease, as well as quantitative data for atherosclerosis outcomes. Fixed- and random-effects meta-analyses were used. Publication bias/small-study effects were assessed using funnel plots and Egger's regression. Trim-and-fill plots and extrapolation from Egger's regression were used to adjust for publication bias. The main outcomes and measures were the primary study authors' interpretation of their own results, and estrogen effects on cardiovascular disease in general before and after publication of the Women's health Initiative study (2003). The effects of estrogens on atherosclerosis were measured as standardized mean difference between intervention and control.

Results: Of 1925 studies retrieved, 360 were eligible for analyses. Study-specific statements concluded that estrogens were protective against cardiovascular disease in 75% of studies before 2003 and 78% after, but the percentage of general statements about estrogens being cardioprotective changed from 70% to 40%. Meta-analyses showed less atherosclerosis in estrogen-treated animals. Extremely skewed funnel plots and  < .01 in Egger's regression suggested publication bias and/or exaggerated effects in small studies, which was more pronounced after 2002. There was substantial heterogeneity of effects (  = 68%-86%) overall and in all subgroups except cynomolgus monkeys (  = 9%), the only animal subgroup without clear bias. Adjusting for publication bias, overall estimates of estrogen effects on atherosclerosis were close to null effect.

Conclusions: We found substantial evidence of publication bias but not of confirmation bias. Publication bias and flawed small studies may partly explain why findings differ between animal studies and clinical trials on the effect of estrogens on cardiovascular disease.