Determining Component Weights in a Communications Assessment Using Judgmental Policy Capturing.

Objectives: Tools are needed for determining appropriate weights for complex performance assessment components in medical education. The feasibility of using judgmental policy capturing (JPC), a procedure to statistically describe the information processing strategies of experts, for this purpose was investigated.

Methods: Iterative JPC was used to determine appropriate weighting for the six core communication skill scores from a communications objective structured clinical examination (OSCE) for medical students using a panel of four communication skill experts.

Results: The mean regression weights from the panel indicated they placed less importance on information management (8.5%), moderate and nearly equal importance on rapport building (15.8%), agenda setting (15.4%), and addressing feelings (14.1%), and greater importance on active listening (20.1%) and reaching common ground with the patient (25.5%).

Discussion: JPC is an effective procedure for determining appropriate weights for complex clinical assessment components. The derived weights may be very different for those assessment components.