Quantifying the robustness of causal inferences: Sensitivity analysis for pragmatic social science.

Social scientists seeking to inform policy or public action must carefully consider how to identify effects and express inferences because actions based on invalid inferences may not yield the intended results. Recognizing the complexities and uncertainties of social science, we seek to inform inevitable debates about causal inferences by quantifying the conditions necessary to change an inference. Specifically, we review existing sensitivity analyses within the omitted variables and potential outcomes frameworks.

Hypothetical case replacement can be used to quantify the robustness of trial results.

Objectives: We apply a general case replacement framework for quantifying the robustness of causal inferences to characterize the uncertainty of findings from clinical trials.

Study Design And Setting: We express the robustness of inferences as the amount of data that must be replaced to change the conclusion and relate this to the fragility of trial results used for dichotomous outcomes. We illustrate our approach in the context of an RCT of hydroxychloroquine on pneumonia in COVID-19 patients and a cumulative meta-analysis of the effect of antihypertensive treatments on stroke.

The Push and Pull of School Performance: Evidence from Student Mobility in New Orleans.

We investigate student mobility in a choice-based system that has gone to scale, New Orleans, to gain insight into an underlying improvement mechanism of choice-based reform and its potential equity-related consequences. In contrast to typical analyses of mobility, this study distinguishes incumbent school characteristics that can cause students to search for a new school ("push" factors) from those features that can draw families to a new school ("pull" factors). We find evidence consistent with school performance playing both push and pull roles.