The Case for Preregistering Quasi-Experimental Program and Policy Evaluations.

Eval Rev

Graduate School of Education, Stanford University, Stanford, CA, USA.

Published: March 2025

The recognition that researcher discretion coupled with unconscious biases and motivated reasoning sometimes leads to false findings ("p-hacking") led to the broad embrace of study preregistration and other open-science practices in experimental research. Paradoxically, the preregistration of quasi-experimental studies remains uncommon although such studies involve far more discretionary decisions and are the most prevalent approach to making causal claims in the social sciences. I discuss several forms of recent empirical evidence indicating that questionable research practices contribute to the comparative unreliability of quasi-experimental research and advocate for adopting the preregistration of such studies. The implementation of this recommendation would benefit from further consideration of key design details (e.g., how to balance data cleaning with credible preregistration) and a shift in research norms to allow for appropriately nuanced sensemaking across prespecified, confirmatory results and other exploratory findings.

Download full-text PDF

Source
http://dx.doi.org/10.1177/0193841X251326738DOI Listing

Publication Analysis

Top Keywords

case preregistering
4
preregistering quasi-experimental
4
quasi-experimental program
4
program policy
4
policy evaluations
4
evaluations recognition
4
recognition researcher
4
researcher discretion
4
discretion coupled
4
coupled unconscious
4

Similar Publications

The recognition that researcher discretion coupled with unconscious biases and motivated reasoning sometimes leads to false findings ("p-hacking") led to the broad embrace of study preregistration and other open-science practices in experimental research. Paradoxically, the preregistration of quasi-experimental studies remains uncommon although such studies involve far more discretionary decisions and are the most prevalent approach to making causal claims in the social sciences. I discuss several forms of recent empirical evidence indicating that questionable research practices contribute to the comparative unreliability of quasi-experimental research and advocate for adopting the preregistration of such studies.

View Article and Find Full Text PDF

Open science practices are designed to enhance the utility, integrity, and credibility of scientific research. This article highlights how preregistration in open science practice can be leveraged to enhance the rigor and transparency of single-case experimental designs within an applied behavior analysis framework. We provide an overview of the benefits of preregistration including increased transparency, reduced risk of researcher bias, and improved replicability, and we review the specific contexts under which these practices most benefit the proposed framework.

View Article and Find Full Text PDF

Artificial Intelligence (AI) systems are precious support for decision-making, with many applications also in the medical domain. The interaction between MDs and AI enjoys a renewed interest following the increased possibilities of deep learning devices. However, we still have limited evidence-based knowledge of the context, design, and psychological mechanisms that craft an optimal human-AI collaboration.

View Article and Find Full Text PDF

In neuroimaging studies, small sample sizes and the resultant reduced statistical power to detect effects that are not large, combined with inadequate analytic choices, concur to produce inflated or false-positive findings. To mitigate these issues, researchers often restrict analyses to specific brain areas, using the region of interest (ROI) approach. Crucially, ROI analysis assumes the a priori justified definition of the target region.

View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!