Correspondence measures for assessing replication success.

Given recent evidence challenging the replicability of results in the social and behavioral sciences, critical questions have been raised about appropriate measures for determining replication success in comparing effect estimates across studies. At issue is the fact that conclusions about replication success often depend on the measure used for evaluating correspondence in results. Despite the importance of choosing an appropriate measure, there is still no widespread agreement about which measures should be used.

Cluster-Based Input Weight Initialization for Echo State Networks.

Echo state networks (ESNs) are a special type of recurrent neural networks (RNNs), in which the input and recurrent connections are traditionally generated randomly, and only the output weights are trained. Despite the recent success of ESNs in various tasks of audio, image, and radar recognition, we postulate that a purely random initialization is not the ideal way of initializing ESNs. The aim of this work is to propose an unsupervised initialization of the input connections using the K -means algorithm on the training data.

The Role of Sample Size to Attain Statistically Comparable Groups - A Required Data Preprocessing Step to Estimate Causal Effects With Observational Data.

Propensity score methods provide data preprocessing tools to remove selection bias and attain statistically comparable groups - the first requirement when attempting to estimate causal effects with observational data. Although guidelines exist on how to remove selection bias when groups in comparison are large, not much is known on how to proceed when one of the groups in comparison, for example, a treated group, is particularly small, or when the study also includes lots of observed covariates (relative to the treated group's sample size). This article investigates whether propensity score methods can help us to remove selection bias in studies with small treated groups and large amount of observed covariates.

Reservoir computing with biocompatible organic electrochemical networks for brain-inspired biosignal classification.

Early detection of malign patterns in patients' biological signals can save millions of lives. Despite the steady improvement of artificial intelligence-based techniques, the practical clinical application of these methods is mostly constrained to an offline evaluation of the patients' data. Previous studies have identified organic electrochemical devices as ideal candidates for biosignal monitoring.

Evaluation of a Non-Personalized Optopalatographic Device for Prospective Use in Functional Post-Stroke Dysphagia Therapy.

Objective: Stroke survivors commonly suffer from dysphagia, originating from oro-facial impairments which affect swallowing function. Functional therapy often employs tongue exercises that require the patient to perform short motion sequences. Evaluating the patient's performance on those exercises is difficult, because there is no reliable form of visual feedback.

Design-Based Approaches to Causal Replication Studies.

Recent interest in promoting replication efforts assumes that there is well-established methodological guidance for designing and implementing these studies. However, no such consensus exists in the methodology literature. This article addresses these challenges by describing design-based approaches for planning systematic replication studies.

Causal graphical views of fixed effects and random effects models.

Despite the long-standing discussion on fixed effects (FE) and random effects (RE) models, how and under what conditions both methods can eliminate unmeasured confounding bias has not yet been widely understood in practice. Using a simple pretest-posttest design in a linear setting, this paper translates the conventional algebraic formalization of FE and RE models into causal graphs and provides intuitively accessible graphical explanations about their data-generating and bias-removing processes. The proposed causal graphs highlight that FE and RE models consider different data-generating models.

When does measurement error in covariates impact causal effect estimates? Analytic derivations of different scenarios and an empirical illustration.

The average causal treatment effect (ATE) can be estimated from observational data based on covariate adjustment. Even if all confounding covariates are observed, they might not necessarily be reliably measured and may fail to obtain an unbiased ATE estimate. Instead of fallible covariates, the respective latent covariates can be used for covariate adjustment.

What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?

Given the widespread use of nonexperimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the same target population. The goal is to assess whether the stringent assumptions required for NE methods are likely to be met in practice.

Designs of Empirical Evaluations of Nonexperimental Methods in Field Settings.

Over the last three decades, a research design has emerged to evaluate the performance of nonexperimental (NE) designs and design features in field settings. It is called the within-study comparison (WSC) approach or the design replication study. In the traditional WSC design, treatment effects from a randomized experiment are compared to those produced by an NE approach that shares the same target population.

Assessing Correspondence Between Experimental and Nonexperimental Estimates in Within-Study Comparisons.

In within-study comparison (WSC) designs, treatment effects from a nonexperimental design, such as an observational study or a regression-discontinuity design, are compared to results obtained from a well-designed randomized control trial with the same target population. The goal of the WSC is to assess whether nonexperimental and experimental designs yield the same results in field settings. A common analytic challenge with WSCs, however, is the choice of appropriate criteria for determining whether nonexperimental and experimental results replicate.

Identification and Sensitivity Analysis for Average Causal Mediation Effects with Time-Varying Treatments and Mediators: Investigating the Underlying Mechanisms of Kindergarten Retention Policy.

Considering that causal mechanisms unfold over time, it is important to investigate the mechanisms over time, taking into account the time-varying features of treatments and mediators. However, identification of the average causal mediation effect in the presence of time-varying treatments and mediators is often complicated by time-varying confounding. This article aims to provide a novel approach to uncovering causal mechanisms in time-varying treatments and mediators in the presence of time-varying confounding.

Graphical Models for Quasi-experimental Designs.

Randomized controlled trials (RCTs) and quasi-experimental designs like regression discontinuity (RD) designs, instrumental variable (IV) designs, and matching and propensity score (PS) designs are frequently used for inferring causal effects. It is well known that the features of these designs facilitate the identification of a causal estimand and, thus, warrant a causal interpretation of the estimated effect. In this article, we discuss and compare the identifying assumptions of quasi-experiments using causal graphs.

Pretest Measures of the Study Outcome and the Elimination of Selection Bias: Evidence from Three Within Study Comparisons.

This paper examines how pretest measures of a study outcome reduce selection bias in observational studies in education. The theoretical rationale for privileging pretests in bias control is that they are often highly correlated with the outcome, and in many contexts, they are also highly correlated with the selection process. To examine the pretest's role in bias reduction, we use the data from two within study comparisons and an especially strong quasi-experiment, each with an educational intervention that seeks to improve achievement.

The Mechanics of Omitted Variable Bias: Bias Amplification and Cancellation of Offsetting Biases.

Causal inference with observational data frequently requires researchers to estimate treatment effects conditional on a set of observed covariates, hoping that they remove or at least reduce the confounding bias. Using a simple linear (regression) setting with two confounders - one observed (), the other unobserved () - we demonstrate that conditioning on the observed confounder does not necessarily imply that the confounding bias decreases, even if is highly correlated with . That is, adjusting for may increase instead of reduce the omitted variable bias (OVB).

Identifying Causal Estimands for Time-Varying Treatments Measured with Time-Varying (Age or Grade-Based) Instruments.

This commentary discusses causal estimands of same-age and same-grade comparisons for assessing grade-retention effects on student ability and performance. Using potential outcomes notation, we show that same-age and same-grade comparisons refer to different retention-promotion contrasts and therefore assess different causal questions. We also comment on deleting versus censoring records of students who dropped out of the study or do not belong to the treatment regimes under investigation.

Practice of causal inference with the propensity of being zero or one: assessing the effect of arbitrary cutoffs of propensity scores.

Causal inference methodologies have been developed for the past decade to estimate the unconfounded effect of an exposure under several key assumptions. These assumptions include, but are not limited to, the stable unit treatment value assumption, the strong ignorability of treatment assignment assumption, and the assumption that propensity scores be bounded away from zero and one (the positivity assumption). Of these assumptions, the first two have received much attention in the literature.

Quasi-Experimental Designs for Causal Inference.

When randomized experiments are infeasible, quasi-experimental designs can be exploited to evaluate causal treatment effects. The strongest quasi-experimental designs for causal inference are regression discontinuity designs, instrumental variable designs, matching and propensity score designs, and comparative interrupted time series designs. This article introduces for each design the basic rationale, discusses the assumptions required for identifying a causal effect, outlines methods for estimating the effect, and highlights potential validity threats and strategies for dealing with them.

Template-based Intensity-Modulated Radiation Therapy: A Cost-effective Intensity-Modulated Radiation Therapy Planning Procedure for Prostate Cancer.

Introduction: Intensity-modulated radiation therapy (IMRT) has been widely accepted for the treatment of prostate cancer. In comparison with traditional three-dimensional conformal radiation therapy (3D-CRT), it improves local control while minimizing side effects. However, IMRT comes at a significantly higher cost.

An introduction to modeling longitudinal data with generalized additive models: applications to single-case designs.

Single-case designs (SCDs) are short time series that assess intervention effects by measuring units repeatedly over time in both the presence and absence of treatment. This article introduces a statistical technique for analyzing SCD data that has not been much used in psychological and educational research: generalized additive models (GAMs). In parametric regression, the researcher must choose a functional form to impose on the data, for example, that trend over time is linear.

Knowledge and attitude of ICU nurses, students and patients towards the Austrian organ donation law.

Background: A survey on the knowledge and attitudes towards the Austrian organ donation legislation (an opt-out solution) of selected groups of the Austrian population taking into account factors such as age, gender, level of education, affiliation to healthcare professions and health related studies was conducted.

Methods: An online survey among 3 target groups (ICU nurses, health science students and non health science students) was performed and results were compared to the answers from transplantation patients to a paper questionnaire. A total of 8415 persons were asked to participate in the survey and 2025 (24%) persons correctly completed the questionnaire.

Template-based breast IMRT planning for increased workload efficiency.

Background: To be less resource intensive, we developed a template-based breast IMRT technique (TB-IMRT). This study aims to compare resources and dose distribution between TB-IMRT and conventional breast radiation (CBR).

Methods: Twenty patients with early stage breast cancer were planned using CBR and TB-IMRT.