Correspondence measures for assessing replication success.

Given recent evidence challenging the replicability of results in the social and behavioral sciences, critical questions have been raised about appropriate measures for determining replication success in comparing effect estimates across studies. At issue is the fact that conclusions about replication success often depend on the measure used for evaluating correspondence in results. Despite the importance of choosing an appropriate measure, there is still no widespread agreement about which measures should be used.

The validity of causal claims with repeated measures designs: A within-study comparison evaluation of differences-in-differences and the comparative interrupted time series.

Modern policies are commonly evaluated not with randomized experiments but with repeated measures designs like difference-in-differences (DID) and the comparative interrupted time series (CITS). The key benefit of these designs is that they control for unobserved confounders that are fixed over time. However, DID and CITS designs only result in unbiased impact estimates when the model assumptions are consistent with the data at hand.

Design-Based Approaches to Causal Replication Studies.

Recent interest in promoting replication efforts assumes that there is well-established methodological guidance for designing and implementing these studies. However, no such consensus exists in the methodology literature. This article addresses these challenges by describing design-based approaches for planning systematic replication studies.

What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?

Given the widespread use of nonexperimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the same target population. The goal is to assess whether the stringent assumptions required for NE methods are likely to be met in practice.

Designs of Empirical Evaluations of Nonexperimental Methods in Field Settings.

Over the last three decades, a research design has emerged to evaluate the performance of nonexperimental (NE) designs and design features in field settings. It is called the within-study comparison (WSC) approach or the design replication study. In the traditional WSC design, treatment effects from a randomized experiment are compared to those produced by an NE approach that shares the same target population.

Assessing Correspondence Between Experimental and Nonexperimental Estimates in Within-Study Comparisons.

In within-study comparison (WSC) designs, treatment effects from a nonexperimental design, such as an observational study or a regression-discontinuity design, are compared to results obtained from a well-designed randomized control trial with the same target population. The goal of the WSC is to assess whether nonexperimental and experimental designs yield the same results in field settings. A common analytic challenge with WSCs, however, is the choice of appropriate criteria for determining whether nonexperimental and experimental results replicate.

Psychometric properties of the Chalder Fatigue Scale revisited: an exploratory structural equation modeling approach.

Objective: Previous validation studies of the Chalder Fatigue Scale (CFS) suffer methodological shortcomings. The present study aimed to re-evaluate its psychometric properties using exploratory structural equation modeling (ESEM).

Methods: A Chinese sample of 1259 community-dwelling residents completed the 11-item Chinese CFS and a variety of health measures (anxiety, depression, exhaustion, sleep disturbance, and quality of life).

Add-on effect of chinese herbal medicine on mortality in myocardial infarction: systematic review and meta-analysis of randomized controlled trials.

In China, Chinese herbal medicine (CHM) is widely used as an adjunct to biomedicine (BM) in treating myocardial infarction (MI). This meta-analysis of RCTs evaluated the efficacy of combined CHM-BM in the treatment of MI, compared to BM alone. Sixty-five RCTs (12,022 patients) of moderate quality were identified.

The effect of qigong exercise on immunity and infections: a systematic review of controlled trials.

The objective of this review was to summarize and critically evaluate the clinical evidence of the effect of qigong exercise on immunity and its efficacy in the prevention or treatment of infectious diseases. Thirteen databases were searched from their respective inceptions through January 2011, and all controlled clinical trials of qigong exercise on immunity and infections were included. Quality and validity of the included studies were evaluated using standard scales.

Tai chi exercise for patients with heart disease: a systematic review of controlled clinical trials.

Context: To summarize and evaluate the available evidence from controlled clinical trials of tai chi (TC) exercise for patients with heart disease.

Search Methods: Fourteen databases were searched up to November 2010 with the terms tai chi, taichi, tai ji, taiji, taijichuan, cardiac, heart, coronary, myocardial, and atrial fibrillation in the title, abstract, or key words. No language restrictions were imposed.

Future development of global regulations of Chinese herbal products.

Ethnopharmacological Relevance: GP-TCM is the first EU-funded Coordination Action consortium dedicated to traditional Chinese medicine (TCM) research. One of the key deliverables of the Work Package 7 in GP-TCM was to investigate information of the existing requirements for registration of TCM products listed by global regulatory bodies. The paper aims to collate data and draw comparison of these regulations.

A systematic review of the effectiveness of qigong exercise in supportive cancer care.

Purpose: Qigong as a complementary and alternative modality of traditional Chinese medicine is often used by cancer patients to manage their symptoms. The aim of this systematic review is to critically evaluate the effectiveness of qigong exercise in cancer care.

Methods: Thirteen databases were searched from their inceptions through November 2010.

A randomized experiment comparing random and cutoff-based assignment.

In this article, we review past studies comparing randomized experiments to regression discontinuity designs, mostly finding similar results, but with significant exceptions. The latter might be due to potential confounds of study characteristics with assignment method or with failure to estimate the same parameter over methods. In this study, we correct the problems by randomly assigning 588 participants to be in a randomized experiment or a regression discontinuity design in which they are otherwise treated identically, comparing results estimating both the same and different parameters.

Using Chinese version of MYMOP in Chinese medicine evaluation: validity, responsiveness and minimally important change.

Background: Measure Yourself Medical Outcome Profile (MYMOP) is a patient generated outcome instrument applicable in the evaluation of both allopathic and complementary medicine treatment. This study aims to adapt MYMOP into Chinese, and to assess its validity, responsiveness and minimally important change values in a sample of patients using Chinese medicine (CM) services.

Methods: A Chinese version of MYMOP (CMYMOP) is developed by forward-backward-forward translation strategy, expert panel assessment and pilot testing amongst patients.

Psychological resilience and dysfunction among hospitalized survivors of the SARS epidemic in Hong Kong: a latent class approach.

Objective: To examine trajectories of psychological functioning using latent class analysis on a sample of hospitalized survivors of the 2003 severe acute respiratory syndrome (SARS) epidemic in Hong Kong.

Design: A longitudinal study of 997 survivors, recruited from among 1,331 individuals hospitalized for SARS, were interviewed at 6, 12, and 18 months after hospitalization.

Main Outcome Measures: Psychological and physical functioning at each time point was measured using the 12-item Medical Outcome Study Short-Form Health Survey (SF-12).

Development and validation of an all-cause mortality risk score in type 2 diabetes.

Background: Diabetes reduces life expectancy by 10 to 12 years, but whether death can be predicted in type 2 diabetes mellitus remains uncertain.

Methods: A prospective cohort of 7583 type 2 diabetic patients enrolled since 1995 were censored on July 30, 2005, or after 6 years of follow-up, whichever came first. A restricted cubic spline model was used to check data linearity and to develop linear-transforming formulas.

Impacts of chronic kidney disease and albuminuria on associations between coronary heart disease and its traditional risk factors in type 2 diabetic patients - the Hong Kong diabetes registry.

Background: Glycated haemoglobin (HbA1c), blood pressure and body mass index (BMI) are risk factors for albuminuria, the latter in turn can lead to hyperlipidaemia. We used novel statistical analyses to examine how albuminuria and chronic kidney disease (CKD) may influence the effects of other risk factors on coronary heart disease (CHD).

Methods: A prospective cohort of 7067 Chinese type 2 diabetic patients without history of CHD enrolled since 1995 were censored on July 30th, 2005.

Prognostication in severe acute respiratory syndrome: a retrospective time-course analysis of 1312 laboratory-confirmed patients in Hong Kong.

Background And Objective: The temporal importance of prognostic indicators for severe acute respiratory syndrome (SARS) has not been studied. This study identified the various clinical prognostic factors for SARS and described the temporal evolution of these factors in the course of the SARS illness in Hong Kong in 2003.

Methods: A retrospective analysis of the entire Hong Kong cohort of 1312 laboratory-confirmed SARS patients aged 15-74 years was undertaken.

Non-invasive versus invasive mechanical ventilation for respiratory failure in severe acute respiratory syndrome.

Background: Severe acute respiratory syndrome is frequently complicated by respiratory failure requiring ventilatory support. We aimed to compare the efficacy of non-invasive ventilation against invasive mechanical ventilation treating respiratory failure in this disease.

Methods: Retrospective analysis was conducted on all respiratory failure patients identified from the Hong Kong Hospital Authority Severe Acute Respiratory Syndrome Database.