Assessing outcomes through congruence of course objectives and reflective work.

Introduction: Course outcomes have been assessed by examining the congruence between statements of commitment to change (CTCs) and course objectives. Other forms of postcourse reflective exercises (for example, impact and unmet-needs statements) have not been examined for congruence with course objectives or their utility in assessing course outcomes. This study assessed the congruence of course objectives and statements of commitment to change, effects on practice, unmet-needs, and the utility of supplementing CTCs with other forms of reflective work in course evaluations.

Methods: A 3-module course on Alzheimer's disease and other dementias provided end-of-course CTC statements, follow-up data, and statements of effects on practice and unmet needs. Statements were aligned to module objectives and analyzed descriptively.

Results: Of the 932 physicians who registered for 1 of the 3 modules, 404 provided CTCs, 302 provided impact statements, and 265 provided unmet-needs statements. Sixty percent of the CTCs could be assigned to an objective for their module, and between 14% and 25% of CTCs were assigned to objectives for another module. Three-quarters of CTCs were fully or partially implemented. Physicians did not have an opportunity to implement the new content in 70% of nonimplemented CTCs. Fewer impact and unmet-needs statements were congruent with course objectives than CTCs.

Conclusions: Commitment-to-change statements had more congruence with objectives than did impact or unmet-needs statements. These latter statements, particularly those that could not be assigned to an objective, may reinforce and supplement the information provided by CTC analyses.