Development, Implementation, and Meta-Evaluation of a National Approach to Programmatic Assessment in Canadian Family Medicine Residency Training.

The growing international adoption of competency-based medical education has created a desire for descriptions of innovative assessment approaches that generate appropriate and sufficient information to allow for informed, defensible decisions about learner progress. In this article, the authors provide an overview of the development and implementation of the approach to programmatic assessment in postgraduate family medicine training programs in Canada, called Continuous Reflective Assessment for Training (CRAFT). CRAFT is a principles-guided, high-level approach to workplace-based assessment that was intentionally designed to be adaptable to local contexts, including size of program, resources available, and structural enablers and barriers.

Association of a Competency-Based Assessment System With Identification of and Support for Medical Residents in Difficulty.

Importance: Competency-based medical education is now established in health professions training. However, critics stress that there is a lack of published outcomes for competency-based medical education or competency-based assessment tools.

Objective: To determine whether competency-based assessment is associated with better identification of and support for residents in difficulty.

Family physician practice visits arising from the Alberta Physician Achievement Review.

Background: Licensed physicians in Alberta are required to participate in the Physician Achievement Review (PAR) program every 5 years, comprising multi-source feedback questionnaires with confidential feedback, and practice visits for a minority of physicians. We wished to identify and classify issues requiring change or improvement from the family practice visits, and the responses to advice.

Methods: Retrospective analysis of narrative practice visit reports data using a mixed methods design to study records of visits to 51 family physicians and general practitioners who participated in PAR during the period 2010 to 2011, and whose ratings in one or more major assessment domains were significantly lower than their peer group.

Defining competency-based evaluation objectives in family medicine: professionalism.

Objective: To develop and describe observable evaluation objectives for assessing competence in professionalism, which are grounded in the experience of practising physicians.

Design: Modified nominal group technique.

Setting: The College of Family Physicians of Canada in Mississauga, Ont.

Defining competency-based evaluation objectives in family medicine: procedure skills.

Objective: To develop evaluation objectives for assessing competence in procedure skills using a key-features approach. This was part of a multiyear project to develop competency-based evaluation objectives for Certification in Family Medicine.

Design: Nominal group technique.

Defining competency-based evaluation objectives in family medicine: communication skills.

Objective: To provide a pragmatic approach to the evaluation of communication skills using observable behaviours, as part of a multiyear project to develop competency-based evaluation objectives for Certification in family medicine.

Design: A nominal group technique was used to develop themes and subthemes and to identify positive and negative observable behaviours that demonstrate competence in communication in family medicine.

Setting: The College of Family Physicians of Canada in Mississauga, Ont.

Involving users in the refinement of the competency-based achievement system: an innovative approach to competency-based assessment.

Background: Competency-based assessment innovations are being implemented to address concerns about the effectiveness of traditional approaches to medical training and the assessment of competence.

Aim: Integrating intended users' perspectives during the piloting and refinement process of an innovation is necessary to ensure the innovation meets users' needs. Failure to do so results in no opportunity for users to influence the innovation, nor for developers to assess why an innovation works or does not work in different contexts.

Defining competency-based evaluation objectives in family medicine: key-feature approach.

Objective: To develop key features for priority topics previously identified by the College of Family Physicians of Canada that, together with skill dimensions and phases of the clinical encounter, broadly describe competence in family medicine.

Design: Modified nominal group methodology, which was used to develop key features for each priority topic through an iterative process.

Setting: The College of Family Physicians of Canada.

Competency-based achievement system: using formative feedback to teach and assess family medicine residents' skills.

Problem Addressed: Family medicine residency programs require innovative means to assess residents' competence in "soft" skills (eg, patient-centred care, communication, and professionalism) and to identify residents who are having difficulty early enough in their residency to provide remedial training.

Objective Of Program: To develop a method to assess residents' competence in various skills and to identify residents who are having difficulty.

Program Description: The Competency-Based Achievement System (CBAS) was designed to measure competence using 3 main principles: formative feedback, guided self-assessment, and regular face-to-face meetings.

Improving the Quality and Capacity of Canada's Health Services: Primary Care Physician Perspectives.

Objective: This study set out to identify the perspectives of family physicians (FP/GPs) on the quality and capacity of the services they provide and of the system in which they work, to assess their responsiveness to potential changes and to determine their suggestions for future directions to enhance primary care services.

Methods: Thematic results from prior focus groups with FP/GPs provided direction for a questionnaire sent to practitioners in the urban study area. Seventy-four questions, most using a five-point Likert scale, were grouped into 10 sections: physician issues (based on themes from the focus groups), access to specialist services, workload, scope of practice, primary care physician networks, interdisciplinary collaborative practice, complexities and challenges of family practice, future directions, comments and demographics.

Defining core procedure skills for Canadian family medicine training.

Objective: To create a list of core and enhanced procedures suitable for family medicine training.

Design: Mailed or e-mailed survey using a Delphi technique.

Setting: Randomly selected family physician practices across Canada.

The influence of residents training level on their evaluation of clinical teaching faculty.

Background: Evaluations by learners are the most common sources of information on teaching. There is some debate about the role of these assessments, but the overall evaluation of faculty by learners was found to be valid and reliable.

Purpose: The purpose of this study was to examine the relationship between the level of training of family medicine residents and their evaluation of emergency medicine clinical teachers over time.

Resident evaluation of clinical teachers based on teachers' certification.

Objective: To examine the influence of emergency medicine (EM) certification of clinical teaching faculty on evaluations provided by residents.

Methods: A prospective cohort analysis was conducted of assessments between July 1994 and July 2000 on residents' evaluations of EM faculty at the University of Alberta, Edmonton, Canada. Resident- and faculty-related variables were entered anonymously using the validated evaluation tool (ER Scale).