Gender, Racial, and Socioeconomic Disparities on Social and Behavioral Skills for K-8 Students With and Without Interventions: An Integrative Data Analysis of Eight Cluster Randomized Trials.

Despite decades of concern about disparities in educational outcomes for low SES students and students of color, there has been limited rigorous study of programmatic approaches for reducing these disparities in elementary or middle schools. We conducted integrative data analysis (IDA) of the combined data from eight Institute of Education Sciences funded cluster randomized trials to address the research gaps on social and behavioral outcome disparities. The final analytic sample includes 90,880 students in varying grade levels from kindergarten to grade 8 in 387 schools in 4 states (Maryland, Missouri, Virginia, and Texas).

Identifying and Estimating Causal Moderation for Treated and Targeted Subgroups.

Extant literature on moderation effects narrowly focuses on the average moderated treatment effect across the entire sample (AMTE). Missing is the average moderated treatment effect on the treated (AMTT) and other targeted subgroups (AMTS). Much like the average treatment effect on the treated (ATT) for main effects, the AMTS changes the target of inferences from the entire sample to targeted subgroups.

Using Propensity Score Analysis of Survey Data to Estimate Population Average Treatment Effects: A Case Study Comparing Different Methods.

Background: Many studies in psychological and educational research aim to estimate population average treatment effects (PATE) using data from large complex survey samples, and many of these studies use propensity score methods. Recent advances have investigated how to incorporate survey weights with propensity score methods. However, to this point, that work had not been well summarized, and it was not clear how much difference the different PATE estimation methods would make empirically.

Power and Sample Size Determination for Multilevel Mediation in Three-Level Cluster-Randomized Trials.

Mediation analyses supply a principal lens to probe the pathways through which a treatment acts upon an outcome because they can dismantle and test the core components of treatments and test how these components function as a coordinated system or theory of action. Experimental evaluation of mediation effects in addition to total effects has become increasingly common but literature has developed only limited guidance on how to plan mediation studies with multi-tiered hierarchical or clustered structures. In this study, we provide methods for computing the power to detect mediation effects in three-level cluster-randomized designs that examine individual- (level one), intermediate- (level two) or cluster-level (level three) mediators.

Advances in Statistical Methods for Causal Inference in Prevention Science: Introduction to the Special Section.

The board of the Society for Prevention Research noted recently that extant methods for the analysis of causality mechanisms in prevention may still be too rudimentary for detailed and sophisticated analysis of causality hypotheses. This Special Section aims to fill some of the current voids, in particular in the domain of statistical methods of the analysis of causal inference. In the first article, Bray et al.

Sample Size Planning for Cluster-Randomized Interventions Probing Multilevel Mediation.

Multilevel mediation analyses play an essential role in helping researchers develop, probe, and refine theories of action underlying interventions and document how interventions impact outcomes. However, little is known about how to plan studies with sufficient power to detect such multilevel mediation effects. In this study, we describe how to prospectively estimate power and identify sufficient sample sizes for experiments intended to detect multilevel mediation effects.

The Incredible Years Teacher Classroom Management Program: Outcomes from a Group Randomized Trial.

This group randomized controlled trial (RCT) evaluated the efficacy of the Incredible Years Teacher Classroom Management Program (IY TCM) on student social behavioral and academic outcomes among a large diverse sample of students within an urban context. Participants included 105 teachers and 1817 students in kindergarten to third grade. Three-level hierarchical linear models (HLM) were conducted to examine the overall treatment effects on teacher-reported student behavior and academic outcomes.

It's all about balance: propensity score matching in the context of complex survey data.

Many research studies aim to draw causal inferences using data from large, nationally representative survey samples, and many of these studies use propensity score matching to make those causal inferences as rigorous as possible given the non-experimental nature of the data. However, very few applied studies are careful about incorporating the survey design with the propensity score analysis, which may mean that the results do not generate population inferences. This may be because few methodological studies examine how to best combine these methods.

Can Propensity Score Analysis Approximate Randomized Experiments Using Pretest and Demographic Information in Pre-K Intervention Research?

Background: It is unclear whether propensity score analysis (PSA) based on pretest and demographic covariates will meet the ignorability assumption for replicating the results of randomized experiments.

Purpose: This study applies within-study comparisons to assess whether pre-Kindergarten (pre-K) treatment effects on achievement outcomes estimated using PSA based on a pretest and demographic covariates can approximate those found in a randomized experiment.

Methods: Data-Four studies with samples of pre-K children each provided data on two math achievement outcome measures with baseline pretests and child demographic variables that included race, gender, age, language spoken at home, and mother's highest education.

Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators.

Mediation analyses have provided a critical platform to assess the validity of theories of action across a wide range of disciplines. Despite widespread interest and development in these analyses, literature guiding the design of mediation studies has been largely unavailable. Like studies focused on the detection of a total or main effect, an important design consideration is the statistical power to detect indirect effects if they exist.

Meaningful Effect Sizes, Intraclass Correlations, and Proportions of Variance Explained by Covariates for Planning Two- and Three-Level Cluster Randomized Trials of Social and Behavioral Outcomes.

Background: There is a need for greater guidance regarding design parameters and empirical benchmarks for social and behavioral outcomes to inform assumptions in the design and interpretation of cluster randomized trials (CRTs).

Objectives: We calculated the empirical reference values on critical research design parameters associated with statistical power for children's social and behavioral outcomes, including effect sizes, intraclass correlations (ICCs), and proportions of variance explained by a covariate at different levels (R ).

Subjects: Children from kindergarten to Grade 5 in the samples from four large CRTs evaluating the effectiveness of two classroom- and two school-level preventive interventions.

Does aggregate school-wide achievement mediate fifth grade outcomes for former early childhood education participants?

This study used a multilevel mediation model to test the theory that former early childhood education (ECE) attendees' 5th grade achievement is mediated by the aggregate school-wide achievement of their elementary school. Aggregate school-wide achievement was defined as the percentage of 5th graders in a school who were at/above academic proficiency in reading or math. Research questions were: (a) Do ECE program participants have better achievement at 5th grade compared with their matched peers who did not participate in an ECE program?; and (b) Is the association between ECE attendance and 5th grade academic performance mediated by school-wide achievement? Results indicated that children who attended prekindergarten (pre-K) and child care outperformed their matched peers who had not attended ECE programs; conversely, those children who did not attend ECE actually outperformed their Head Start counterparts.

Longitudinal associations between executive functioning and academic skills across content areas.

This study assessed 562 four-year-old children at the beginning and end of their prekindergarten (pre-k) year and followed them to the end of kindergarten. At each time point children were assessed on 6 measures of executive function (EF) and 5 subtests of the Woodcock-Johnson III academic achievement battery. Exploratory factor analyses yielded EF and achievement factor scores.