The reputation of medical education research: quasi-experimentation and unresolved threats to validity.

Background: Medical education research is often criticized for its methodological flaws. This raises questions about the prospect of evidence-based medical education practice. Critics call for more rigorous research with randomization, greater control, and tight execution. But randomization and control are rarely achieved in applied field settings, the site of most medical education research. Consequently, research in medical education has relied on quasi-experimentation, which by definition is compromised methodologically.

Summary: So is medical education research doomed to a reputation for flawed research that undermines the promise of evidence-based practice? We think not. We argue in this article that the problem is due to incomplete and uncritical use of quasi-experimentation. This practice ignores a critical step in the quasi-experimental process, a step that follows the design and execution of the research, that is, ruling out threats to validity posed by methodological flaws.

Conclusion: Validity threats must be evaluated critically when drawing research conclusions, to establish a credible evidence base for medical education practice.