Rapid Blueprinting: An Efficient Method for Designing Content of Assessments.

Problem: Many assessments in medical education involve measuring proficiency in a content area. Thus, proper content development (blueprinting) of tests in this field is of primary importance. Prior efforts to conduct content review as part of assessment development have been time- and resource-intensive, relying on practice analysis and then on linking methods. This monograph explores a "rapid, cost-effective" approach to blueprinting that allows efficient assessment development with rigor. Our investigation seeks to explore an efficient and effective alternate method for creating a content design (blueprint) for medical credentialing and evaluation examinations by focusing directly on assessment requirements.

Approach: We employed a two-phase process to propose a method. Phase 1 involved a 1-day direct meeting of content experts/practitioners. Phase 2 involved a corroboration survey sent to a wider group of content experts/practitioners. The rapid blueprinting method was applied to developing eleven blueprints (five for medical specialty certification; five for health professions certification; and one for in-training assessment).

Outcomes: The methods we used resulted in effective, well-balanced, operational examinations that successfully implemented the resulting blueprints in item writing assignments and test development. Assessments resulting from the use of the rapid blueprinting method also generated psychometrically sound inferences from the scores. For example, the assessments resulting from this methodology of test construction had KR-20 reliability coefficients ranging from .87 to .92.

Next Steps: This approach leveraged the effectiveness and feasibility of the rapid blueprinting method and demonstrated successful examination designs (blueprints) that are cost- and time-effective. The rapid blueprinting method may be explored for further implementation in local assessment settings beyond medical credentialing examinations.