OSCE Standard Setting: Three Borderline Group Methods.

High-stakes assessments must discriminate between examinees who are sufficiently competent to practice in the health professions and examinees who are not. In these settings, criterion-referenced standard-setting methods are strongly preferred over norm referenced methods. While there are many criterion-referenced options, few are feasible or cost effective for objective structured clinical examinations (OSCEs).

Augmenting physician examiner scoring in objective structured clinical examinations: including the standardized patient perspective.

In Canada, high stakes objective structured clinical examinations (OSCEs) administered by the Medical Council of Canada have relied exclusively on physician examiners (PEs) for scoring. Prior research has looked at using SPs to replace PEs. This paper reports on two studies that implement and evaluate a standardized patient (SP) scoring tool to augment PE scoring.

Core principles of assessment in competency-based medical education.

The meaningful assessment of competence is critical for the implementation of effective competency-based medical education (CBME). Timely ongoing assessments are needed along with comprehensive periodic reviews to ensure that trainees continue to progress. New approaches are needed to optimize the use of multiple assessors and assessments; to synthesize the data collected from multiple assessors and multiple types of assessments; to develop faculty competence in assessment; and to ensure that relationships between the givers and receivers of feedback are appropriate.

Development and testing of an objective structured clinical exam (OSCE) to assess socio-cultural dimensions of patient safety competency.

Background: Patient safety (PS) receives limited attention in health professional curricula. We developed and pilot tested four Objective Structured Clinical Examination (OSCE) stations intended to reflect socio-cultural dimensions in the Canadian Patient Safety Institute's Safety Competency Framework.

Setting And Participants: 18 third year undergraduate medical and nursing students at a Canadian University.

A method for identifying extreme OSCE examiners.

Background: Performance assessments rely on human judgment, and are vulnerable to rater effects (e.g. leniency or harshness).

Quality assurance best practices for simulation-based examinations.

Whether used for formative or summative evaluation, health professions schools, residency programs, continuing medical education programs, and specialty boards using simulation-based assessment must consider quality assurance methods to ensure reliable results. This article addresses the content, training, and administrative protocols that are necessary for findings based on valid, reliable, and fair assessments.

Influence of physicians' management and communication ability on patients' persistence with antihypertensive medication.

Background: Less than 75% of people prescribed antihypertensive medication are still using treatment after 6 months. Physicians determine treatment, educate patients, manage side effects, and influence patient knowledge and motivation. Although physician communication ability likely influences persistence, little is known about the importance of medical management skills, even though these abilities can be enhanced through educational and practice interventions.

Doctor scores on national qualifying examinations predict quality of care in future practice.

Objectives: This study aimed to determine if national licensing examinations that measure medical knowledge (QE1) and clinical skills (QE2) predict the quality of care delivered by doctors in future practice.

Methods: Cohorts of doctors who took the Medical Council of Canada Qualifying Examinations Part I (QE1) and Part II (QE2) between 1993 and 1996 and subsequently entered practice in Ontario, Canada (n = 2420) were followed for their first 7-10 years in practice. The 208 of these doctors who were randomly selected for peer assessment of quality of care were studied.

Does the gender of the standardised patient influence candidate performance in an objective structured clinical examination?

Context: The objective structured clinical examination (OSCE) requires the use of standardised patients (SPs). Recruitment of SPs can be challenging and factors assumed to be neutral may vary between SPs. On stations that are considered gender-neutral, either male or female SPs may be used.

The use of standardized patient assessments for certification and licensure decisions.

Although standardized patients have been employed for formative assessment for over 40 years, their use in high-stakes medical licensure examinations has been a relatively recent phenomenon. As part of the medical licensure process in the United States and Canada, the clinical skills of medical students, medical school graduates, and residents are evaluated in a simulated clinical environment. All of the evaluations attempt to provide the public with some assurance that the person who achieves a passing score has the knowledge and/or requisite skills to provide safe and effective medical services.

Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities.

Context: Poor patient-physician communication increases the risk of patient complaints and malpractice claims. To address this problem, licensure assessment has been reformed in Canada and the United States, including a national standardized assessment of patient-physician communication and clinical history taking and examination skills.

Objective: To assess whether patient-physician communication examination scores in the clinical skills examination predicted future complaints in medical practice.

Is it feasible to include a technical skill station on a national licensing examination?

The purpose of the current study was to assess the feasibility and validity of including a technical skill station on a national licensing examination. At the 2003 Medical Council of Canada Qualifying Examination, 745 test takers participated in a pilot station assessing the ability to perform a technical procedure. Checklists and rating scales were used for scoring.

A comparison of physician examiners and trained assessors in a high-stakes OSCE setting.

Background: The Medical Council of Canada (MCC) administers an objective structured clinical examination for licensure. Traditionally, physician examiners (PE) have evaluated these examinees. Recruitment of physicians is becoming more difficult.

A sequenced OSCE for licensure: administrative issues, results and myths.

In response to stakeholder demands for a more cost-effective clinical examination, the Medical Council of Canada adopted a sequenced format for the OSCE component of its licensure examination. The sequenced OSCE was administered in 1997 at 14 sites and assessed 1,796 test takers. The 10-station screening test had an alpha = 0.

Using the Judgments of Physician Examiners in Setting the Standards for a National Multi-center High Stakes OSCE.

In 1994 and 1995, the Medical Council of Canada used an innovative approach to set the pass mark on its large scale, multi-center national OSCE which is designed to assess basic clinical and communication skills in physicians in Canada after 15 months of post-graduate medical training. The goal of this article is to describe the new approach and to present the experience with the method during its first two years of operation. The approach utilizes the global judgments of the physician examiners at each station to identify the candidates with borderline performances.