Expanding Education Researchers' Access to Classroom Observation Data With a Remote and Cost-Effective Video Data Collection Protocol.

The onset of the COVID-19 pandemic and associated long-term shifts to virtual instruction among most US schools presented notable challenges among education researchers. Ongoing projects conducted in school settings experienced sudden losses of access to teacher and student participants, in many cases leading to severe interruptions to data collection efforts. Perhaps most notably, upon returns to in-person instruction in the 2021/22 academic year most schools instigated strict policies limiting the number of non-school personnel who could enter school buildings, including researchers conducting in-person data collections.

Power and Sample Size Determination for Multilevel Mediation in Three-Level Cluster-Randomized Trials.

Mediation analyses supply a principal lens to probe the pathways through which a treatment acts upon an outcome because they can dismantle and test the core components of treatments and test how these components function as a coordinated system or theory of action. Experimental evaluation of mediation effects in addition to total effects has become increasingly common but literature has developed only limited guidance on how to plan mediation studies with multi-tiered hierarchical or clustered structures. In this study, we provide methods for computing the power to detect mediation effects in three-level cluster-randomized designs that examine individual- (level one), intermediate- (level two) or cluster-level (level three) mediators.

Statistical power in partially nested designs probing multilevel mediation.

Analysis of the intermediate behaviors and mechanisms through which innovative therapies come to shape outcomes is a critical objective in many areas of psychotherapy research because it supports the iterative exploration, development and refinement of theories and therapies. Despite widespread interest in the intermediate behaviors and mechanisms that convey treatment effects, there is limited guidance on how to effectively and efficiently design studies to detect such mediated effects in the types of partially nested designs that commonly arise in psychotherapy research. In this study, we develop statistical power formulas to identify requisite sample sizes and guide the planning of studies probing mediation under two- and three-level partially nested designs.

A Reliability Analysis of Entrustment-Derived Workplace-Based Assessments.

Purpose: To examine the reliability and attributable facets of variance within an entrustment-derived workplace-based assessment system.

Method: Faculty at the University of Cincinnati Medical Center internal medicine residency program (a 3-year program) assessed residents using discrete workplace-based skills called observable practice activities (OPAs) rated on an entrustment scale. Ratings from July 2012 to December 2016 were analyzed using applications of generalizability theory (G-theory) and decision study framework.

A robust alternative estimator for small to moderate sample SEM: Bias-corrected factor score path analysis.

Structural equation modeling with full information maximum likelihood estimation is the predominant method to empirically assess complex theories involving multiple latent variables in addiction research. Although full information estimators have many desirable properties including consistency, a major limitation in structural equation models is that they often sustain significant bias when implemented in small to moderate size studies (e.g.

Sample Size Planning for Cluster-Randomized Interventions Probing Multilevel Mediation.

Multilevel mediation analyses play an essential role in helping researchers develop, probe, and refine theories of action underlying interventions and document how interventions impact outcomes. However, little is known about how to plan studies with sufficient power to detect such multilevel mediation effects. In this study, we describe how to prospectively estimate power and identify sufficient sample sizes for experiments intended to detect multilevel mediation effects.

Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators.

Mediation analyses have provided a critical platform to assess the validity of theories of action across a wide range of disciplines. Despite widespread interest and development in these analyses, literature guiding the design of mediation studies has been largely unavailable. Like studies focused on the detection of a total or main effect, an important design consideration is the statistical power to detect indirect effects if they exist.

Planning community-based assessments of HIV educational intervention programs in sub-Saharan Africa.

A key consideration in planning studies of community-based HIV education programs is identifying a sample size large enough to ensure a reasonable probability of detecting program effects if they exist. Sufficient sample sizes for community- or group-based designs are proportional to the correlation or similarity of individuals within communities. As a result, efficient and effective design requires reasonable a priori estimates of the correlational structure among individuals within communities.

Intraclass Correlation Coefficients for Designing Cluster-Randomized Trials in Sub-Saharan Africa Education.

Objective:: Over the past two decades, the lack of reliable empirical evidence concerning the effectiveness of educational interventions has motivated a new wave of research in education in sub-Saharan Africa (and across most of the world) that focuses on impact evaluation through rigorous research designs such as experiments. Often these experiments draw on the random assignment of entire clusters, such as schools, to accommodate the multilevel structure of schooling and the theory of action underlying many school-based interventions. Planning effective and efficient school randomized studies, however, requires plausible values of the intraclass correlation coefficient (ICC) and the variance explained by covariates during the design stage.

Approximate measurement invariance in cross-classified rater-mediated assessments.

An important assumption underlying meaningful comparisons of scores in rater-mediated assessments is that measurement is commensurate across raters. When raters differentially apply the standards established by an instrument, scores from different raters are on fundamentally different scales and no longer preserve a common meaning and basis for comparison. In this study, we developed a method to accommodate measurement noninvariance across raters when measurements are cross-classified within two distinct hierarchical units.

Strategies for Improving Power in School-Randomized Studies of Professional Development.

Objectives: Group-randomized designs are well suited for studies of professional development because they can accommodate programs that are delivered to intact groups (e.g., schools), the collaborative nature of professional development, and extant teacher/school assignments.

Covariate Selection in Propensity Scores Using Outcome Proxies.

This study examined the practical problem of covariate selection in propensity scores (PSs) given a predetermined set of covariates. Because the bias reduction capacity of a confounding covariate is proportional to the concurrent relationships it has with the outcome and treatment, particular focus is set on how we might approximate covariate-outcome relationships while retaining the PS as a design tool (i.e.