Assessment of temporal instability in the applied ecology and conservation evidence base.

Outcomes of meta-analyses are increasingly used to inform evidence-based decision making in various research fields. However, a number of recent studies have reported rapid temporal changes in magnitude and significance of the reported effects which could make policy-relevant recommendations from meta-analyses to quickly go out of date. We assessed the extent and patterns of temporal trends in magnitude and statistical significance of the cumulative effects in meta-analyses in applied ecology and conservation published between 2004 and 2018.

Estimation of heterogeneity variance based on a generalized Q statistic in meta-analysis of log-odds-ratio.

For estimation of heterogeneity variance in meta-analysis of log-odds-ratio, we derive new mean- and median-unbiased point estimators and new interval estimators based on a generalized statistic, , in which the weights depend on only the studies' effective sample sizes. We compare them with familiar estimators based on the inverse-variance-weights version of , In an extensive simulation, we studied the bias (including median bias) of the point estimators and the coverage (including left and right coverage error) of the confidence intervals. Most estimators add to each cell of the table when one cell contains a zero count; we include a version that always adds .

On the Q statistic with constant weights in meta-analysis of binary outcomes.

Background: Cochran's Q statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value (under an incorrect null distribution) is part of several popular estimators of the between-study variance, [Formula: see text]. Those applications generally do not account for use of the studies' estimated variances in the inverse-variance weights that define Q (more explicitly, [Formula: see text]).

A double-Cox model for non-proportional hazards survival analysis with frailty.

The Cox regression, a semi-parametric method of survival analysis, is extremely popular in biomedical applications. The proportional hazards assumption is a key requirement in the Cox model. To accommodate non-proportional hazards, we propose to parameterize the shape parameter of the baseline hazard function using the additional, separate Cox-regression term which depends on the vector of the covariates.

Long term survival after a first transient ischaemic attack in England: A retrospective matched cohort study.

Objective: Transient ischaemic attacks (TIA) serve as warning signs for future stroke, and the impact of TIA on long term survival is uncertain. We assessed the long-term hazards of all-cause mortality following a first episode of a transient ischaemic attack (TIA).

Design: Retrospective matched cohort study.

On the Q statistic with constant weights for standardized mean difference.

Cochran's Q statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value is also used in several popular estimators of the between-study variance, . Those applications generally have not considered the implications of its use of estimated variances in the inverse-variance weights.

The effect of hormone replacement therapy on the survival of UK women: a retrospective cohort study 1984-2017.

Objective: To estimate the effect of estrogen-only and combined hormone replacement therapy (HRT) on the hazards of overall and age-specific all-cause mortality in healthy women aged 46-65 at first prescription.

Design: Matched cohort study.

Setting: Electronic primary care records from The Health Improvement Network (THIN) database, UK (1984-2017).

Cumulative meta-analysis: What works.

To present time-varying evidence, cumulative meta-analysis (CMA) updates results of previous meta-analyses to incorporate new study results. We investigate the properties of CMA, suggest possible improvements and provide the first in-depth simulation study of the use of CMA and CUSUM methods for detection of temporal trends in random-effects meta-analysis. We use the standardized mean difference (SMD) as an effect measure of interest.

Exploring consequences of simulation design for apparent performance of methods of meta-analysis.

Contemporary statistical publications rely on simulation to evaluate performance of new methods and compare them with established methods. In the context of random-effects meta-analysis of log-odds-ratios, we investigate how choices in generating data affect such conclusions. The choices we study include the overall log-odds-ratio, the distribution of probabilities in the control arm, and the distribution of study-level sample sizes.

Do statins reduce mortality in older people? Findings from a longitudinal study using primary care records.

Objective: Assess whether statins reduce mortality in the general population aged 60 years and above.

Design: Retrospective cohort study.

Setting: Primary care practices contributing to The Health Improvement Network database, England and Wales, 1990-2017.

A Q statistic with constant weights for assessing heterogeneity in meta-analysis.

The conventional Q statistic, using estimated inverse-variance (IV) weights, underlies a variety of problems in random-effects meta-analysis. In previous work on standardized mean difference and log-odds-ratio, we found superior performance with an estimator of the overall effect whose weights use only group-level sample sizes. The Q statistic with those weights has the form proposed by DerSimonian and Kacker.

Dynamic meta-analysis: a method of using global evidence for local decision making.

Background: Meta-analysis is often used to make generalisations across all available evidence at the global scale. But how can these global generalisations be used for evidence-based decision making at the local scale, if the global evidence is not perceived to be relevant to local decisions? We show how an interactive method of meta-analysis-dynamic meta-analysis-can be used to assess the local relevance of global evidence.

Results: We developed Metadataset ( www.

Estimation in meta-analyses of response ratios.

Background: For outcomes that studies report as the means in the treatment and control groups, some medical applications and nearly half of meta-analyses in ecology express the effect as the ratio of means (RoM), also called the response ratio (RR), analyzed in the logarithmic scale as the log-response-ratio, LRR.

Methods: In random-effects meta-analysis of LRR, with normal and lognormal data, we studied the performance of estimators of the between-study variance, τ, (measured by bias and coverage) in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect in the log scale, λ. We obtained additional empirical evidence from two examples.

Dynamic early identification of hip replacement implants with high revision rates. Study based on the NJR data from UK during 2004-2012.

Background: Hip replacement and hip resurfacing are common surgical procedures with an estimated risk of revision of 4% over 10 year period. Approximately 58% of hip replacements will last 25 years. Some implants have higher revision rates and early identification of poorly performing hip replacement implant brands and cup/head brand combinations is vital.

Methods for estimating between-study variance and overall effect in meta-analysis of odds ratios.

In random-effects meta-analysis the between-study variance ( τ ) has a key role in assessing heterogeneity of study-level estimates and combining them to estimate an overall effect. For odds ratios the most common methods suffer from bias in estimating τ and the overall effect and produce confidence intervals with below-nominal coverage. An improved approximation to the moments of Cochran's Q statistic, suggested by Kulinskaya and Dollinger (KD), yields new point and interval estimators of τ and of the overall log-odds-ratio.

Risk-adjusted cUSUM control charts for shared frailty survival models with application to hip replacement outcomes: a study using the NJR dataset.

Background: Continuous monitoring of surgical outcomes after joint replacement is needed to detect which brands' components have a higher than expected failure rate and are therefore no longer recommended to be used in surgical practice. We developed a monitoring method based on cumulative sum (CUSUM) chart specifically for this application.

Methods: Our method entails the use of the competing risks model with the Weibull and the Gompertz hazard functions adjusted for observed covariates to approximate the baseline time-to-revision and time-to-death distributions, respectively.

Estimation in meta-analyses of mean difference and standardized mean difference.

Methods for random-effects meta-analysis require an estimate of the between-study variance, τ . The performance of estimators of τ (measured by bias and coverage) affects their usefulness in assessing heterogeneity of study-level effects and also the performance of related estimators of the overall effect. However, as we show, the performance of the methods varies widely among effect measures.

Temporal Instability of Evidence Base: A Threat to Policy Making?

A shift towards evidence-based conservation and environmental management over the last two decades has resulted in an increased use of systematic reviews and meta-analyses as tools to combine existing scientific evidence. However, to guide policy making decisions in conservation and management, the conclusions of meta-analyses need to remain stable for at least some years. Alarmingly, numerous recent studies indicate that the magnitude, statistical significance, and even the sign of the effects reported in the literature might change over relatively short time periods.

Pitfalls of using the risk ratio in meta-analysis.

For meta-analysis of studies that report outcomes as binomial proportions, the most popular measure of effect is the odds ratio (OR), usually analyzed as log(OR). Many meta-analyses use the risk ratio (RR) and its logarithm because of its simpler interpretation. Although log(OR) and log(RR) are both unbounded, use of log(RR) must ensure that estimates are compatible with study-level event rates in the interval (0, 1).

Optimal SBP targets in routine clinical care.

Objective: Compare outcomes of intensive treatment of SBP to less than 120 mmHg versus standard treatment to less than 140 mmHg in the US clinical Systolic Blood Pressure Intervention Trial (SPRINT) with similar hypertensive patients managed in routine primary care in the United Kingdom.

Methods: Hypertensive patients aged 50-90 without diabetes or chronic kidney disease (CKD) were selected in SPRINT and The Health Improvement Network (THIN) database. Patients were enrolled in 2010-2013 and followed-up to 2015 (SPRINT N = 4112; THIN N = 8631).

Meta-analysis of binary outcomes via generalized linear mixed models: a simulation study.

Background: Systematic reviews and meta-analyses of binary outcomes are widespread in all areas of application. The odds ratio, in particular, is by far the most popular effect measure. However, the standard meta-analysis of odds ratios using a random-effects model has a number of potential problems.

Beta-binomial model for meta-analysis of odds ratios.

In meta-analysis of odds ratios (ORs), heterogeneity between the studies is usually modelled via the additive random effects model (REM). An alternative, multiplicative REM for ORs uses overdispersion. The multiplicative factor in this overdispersion model (ODM) can be interpreted as an intra-class correlation (ICC) parameter.

Survival prospects after acute myocardial infarction in the UK: a matched cohort study 1987-2011.

Objectives: Estimate survival after acute myocardial infarction (AMI) in the general population aged 60 and over and the effect of recommended treatments.

Design: Cohort study in the UK with routinely collected data between January 1987 and March 2011.

Setting: 310 general practices that contributed to The Health Improvement Network (THIN) database.