Experiment aversion does not appear to generalize.

Proc Natl Acad Sci U S A

Department of Management, School of Business and Social Sciences, Aarhus University, Aarhus V 8210, Denmark.

Published: April 2023

Over the past decade, governments and organizations around the world have established behavioral insights teams advocating for randomized experiments. However, recent findings by M. N. Meyer , , 10723-10728 (2019) and P. R. Heck, C. F. Chabris, D. J. Watts, M. N. Meyer, , 18948-18950 (2020) suggest that people often rate randomized experiments as less appropriate than the policies they contain even when approving the implementation of either policy untested and when none of the individual policies is clearly superior. The authors warn that this could cause policymakers to avoid running large-scale field experiments or being transparent about running them and might contribute to an adverse heterogeneity bias in terms of who is participating in experiments. In one direct and six conceptual preregistered replications (total = 5,200) of the previously published larger-effect studies, using the same main dependent variable but with variations in scenario wordings, recruitment platforms, and countries, and the addition of further measures to assess people's views, we test the generalizability and robustness of these findings. Together, we find that the original results do not appear to generalize. That is, our triangulation reveals insufficient evidence to conclude that people exhibit a common pattern of behavior that would be consistent with relative experiment aversion, thereby supporting recent findings by R. Mislavsky, B. Dietvorst, U. Simonsohn, , 1092-1104 (2020). Thus, policymakers may not need to be concerned about employing evidence-based practices more so than about universally implementing policies.

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10120089PMC
http://dx.doi.org/10.1073/pnas.2217551120DOI Listing

Publication Analysis

Top Keywords

experiment aversion
8
appear generalize
8
randomized experiments
8
aversion appear
4
generalize decade
4
decade governments
4
governments organizations
4
organizations established
4
established behavioral
4
behavioral insights
4

Similar Publications

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!