The Impact of item flaws, testing at low cognitive level, and low distractor functioning on multiple-choice question quality.

Background: This study investigated the impact of addressing item writing flaws, testing at low cognitive level and non-functioning distractors (< 5 % selection frequency) in multiple-choice assessment in preclinical medical education.

Method: Multiple-choice questions with too high or too low difficulty (difficulty index < 0.4 or > 0.

Measuring inter-rater reliability of the sequenced performance inventory and reflective assessment of learning (SPIRAL).

Purpose: To measure the inter-rater reliability of the Sequenced Performance Inventory and Reflective Assessment of Learning (SPIRAL), a twenty-three item scoring rubric designed to assess first and second-year students' competencies such as "acquisition of knowledge," "peer teaching and communication skills," and "professional behavior" in a problem-based learning (PBL) curriculum at the University of North Dakota School of Medicine and Health Sciences.

Method: In 2001, the authors constructed a 69-item multiple-choice questionnaire consisting of descriptions of "prototypical" (representing real students) PBL student performances. For each of the 23 SPIRAL items, the authors chose competency descriptions of an "emerging," a "developing," and an "advanced" student (69 prototypical students).