Background: Policy evaluation studies that assess how state-level policies affect health-related outcomes are foundational to health and social policy research. The relative ability of newer analytic methods to address confounding, a key source of bias in observational studies, has not been closely examined.

Methods: We conducted a simulation study to examine how differing magnitudes of confounding affected the performance of 4 methods used for policy evaluations: (1) the two-way fixed effects difference-in-differences model; (2) a 1-period lagged autoregressive model; (3) augmented synthetic control method; and (4) the doubly robust difference-in-differences approach with multiple time periods from Callaway-Sant'Anna. We simulated our data to have staggered policy adoption and multiple confounding scenarios (i.e., varying the magnitude and nature of confounding relationships).

Results: Bias increased for each method: (1) as confounding magnitude increases; (2) when confounding is generated with respect to prior outcome trends (rather than levels), and (3) when confounding associations are nonlinear (rather than linear). The autoregressive model and augmented synthetic control method had notably lower root mean squared error than the two-way fixed effects and Callaway-Sant'Anna approaches for all scenarios; the exception is nonlinear confounding by prior trends, where Callaway-Sant'Anna excels. Coverage rates were unreasonably high for the augmented synthetic control method (e.g., 100%), reflecting large model-based standard errors and wide confidence intervals in practice.

Conclusions: In our simulation study, no single method consistently outperformed the others, but a researcher's toolkit should include all methodologic options. Our simulations and associated R package can help researchers choose the most appropriate approach for their data.

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10538408PMC
http://dx.doi.org/10.1097/EDE.0000000000001659DOI Listing

Publication Analysis

Top Keywords

augmented synthetic
12
synthetic control
12
control method
12
confounding
9
state-level policies
8
simulation study
8
two-way fixed
8
fixed effects
8
autoregressive model
8
model augmented
8

Similar Publications

Winter-spring droughts exacerbated PM-O compound pollution? Evidence from China.

Sci Total Environ

December 2024

College of Environmental Science and Engineering, China West Normal University, Nanchong 637002, China. Electronic address:

With the impact of global climate change, drought events are becoming more frequent, making it critically important to quantitatively evaluate the effects of these events on air pollution. This study uses the augmented synthetic control method and the mediation effect model to quantitatively evaluate the impact effect of the winter-spring drought of 2023 on PM-O compound pollution and its driving factors with Chinese prefecture-level city data. This study indicates that: firstly, compared to non-drought periods, both the monthly averaged and diurnal variations pattern of PM and O significantly increased during drought periods.

View Article and Find Full Text PDF

Development of a microbiome for phenolic metabolism based on a domestication approach from lab to industrial application.

Commun Biol

December 2024

Tianjin Key Laboratory of Industrial Biological Systems and Process Engineering, Tianjin Institute of Industrial Biotechnology, Chinese Academy of Sciences, Tianjin, 300308, China.

Despite a lot of efforts devoted to construct efficient microbiomes, there are still major obstacles to moving from the lab to industrial applications due to the inapplicability of existing technologies or limited understanding of microbiome variation regularity. Here we show a domestication strategy to cultivate an effciient and resilient functional microbiome for addressing phenolic wastewater challenges, which involves directional domestication in shaker, laboratory water test in small-scale, gas test in pilot scale, water test in pilot scale, and engineering application in industrial scale. The domestication process includes the transition from water to gas, which provided complex transient environment for screening of a more adaptable and robust microbiome, thereby mitigating the performance disparities encountered when transitioning from laboratory experimentation to industrial engineering applications.

View Article and Find Full Text PDF

Purpose: This study explores the use of deep generative models to create synthetic ultrasound images for the detection of hemarthrosis in hemophilia patients. Addressing the challenge of sparse datasets in rare disease diagnostics, the study aims to enhance AI model robustness and accuracy through the integration of domain knowledge into the synthetic image generation process.

Methods: The study employed two ultrasound datasets: a base dataset (Db) of knee recess distension images from non-hemophiliac patients and a target dataset (Dt) of hemarthrosis images from hemophiliac patients.

View Article and Find Full Text PDF

A novel methodology for dataset augmentation in the semantic segmentation of coil-coated surface degradation is presented in this study. Deep convolutional generative adversarial networks (DCGAN) are employed to generate synthetic input-target pairs, which closely resemble real-world data, with the goal of expanding an existing dataset. These augmented datasets are used to train two state-of-the-art models, U-net, and DeepLabV3, for the precise detection of degradation areas around scribes.

View Article and Find Full Text PDF

Background: Cancer classification has consistently been a challenging problem, with the main difficulties being high-dimensional data and the collection of patient samples. Concretely, obtaining patient samples is a costly and resource-intensive process, and imbalances often exist between samples. Moreover, expression data is characterized by high dimensionality, small samples and high noise, which could easily lead to struggles such as dimensionality catastrophe and overfitting.

View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!