Simulation-based ongoing professional practice evaluation in psychiatry: a novel tool for performance assessment.

Background: Ongoing professional practice evaluation (OPPE) activities consist of a quantitative, competency-based evaluation of clinical performance. Hospitals must design assessments that measure clinical competencies, are scalable, and minimize impact on the clinician's daily routines. A psychiatry department at a large academic medical center designed and implemented an interactive Web-based psychiatric simulation focusing on violence risk assessment as a tool for a departmentwide OPPE.

Methods: Of 412 invited clinicians in a large psychiatry department, 410 completed an online simulation in April-May 2012. Participants received scheduled e-mail reminders with instructions describing how to access the simulation. Using the Computer Simulation Assessment Tool, participants viewed an introductory video and were then asked to conduct a risk assessment, acting as a clinician in the encounter by selecting actions from a series of drop-down menus. Each action was paired with a corresponding video segment of a clinical encounter with a standardized patient. Participants were scored on the basis of their actions within the simulation (Measure 1) and by their responses to the open-ended questions in which they were asked to integrate the information from the simulation in a summative manner (Measure 2).

Results: Of the 410 clinicians, 381 (92.9%) passed Measure 1,359 (87.6%) passed Measure 2, and 5 (1.2%) failed both measures. Seventy-five (18.3%) participants were referred for focused professional practice evaluation (FPPE) after failing either Measure 1, Measure 2, or both.

Conclusions: Overall, Web-based simulation and e-mail engagement tools were a scalable and efficient way to assess a large number of clinicians in OPPE and to identify those who required FPPE.