Fairness: the hidden challenge for competency-based postgraduate medical education programs.

Competency-based medical education systems allow institutions to individualize teaching practices to meet the needs of diverse learners. Yet, the focus on continuous improvement and individualization of curricula does not exempt programs from treating learners in a fair manner. When learners fail to meet key competencies and are placed on probation or dismissed from training programs, issues of fairness may form the basis of their legal claims.

Medical student resilience strategies: A content analysis of medical students' portfolios.

Introduction: Stress and burnout among medical students is a well-recognized concern. A student's ability to employ resilience strategies to self-regulate behaviour is critical to the student's future career as a physician.

Methods: We retrospectively reviewed a sampling of year 1, 2 and 5 portfolio essays focused on the Personal Development competency and performance milestones, written by 49 students from three different classes in a 5-year programme devoted to training physician investigators.

The Learning Environment Counts: Longitudinal Qualitative Analysis of Study Strategies Adopted by First-Year Medical Students in a Competency-Based Educational Program.

Purpose: The move toward competency-based education will require medical schools and postgraduate training programs to restructure learning environments to motivate trainees to take personal ownership for learning. This qualitative study explores how medical students select and implement study strategies while enrolled in a unique, nontraditional program that emphasizes reflection on performance and competence rather than relying on high-stakes examinations or grades to motivate students to learn and excel.

Method: Fourteen first-year medical students volunteered to participate in three, 45-minute interviews (42 overall) scheduled three months apart during 2013-2014.

Factors influencing students' receptivity to formative feedback emerging from different assessment cultures.

Introduction: Feedback after assessment is essential to support the development of optimal performance, but often fails to reach its potential. Although different assessment cultures have been proposed, the impact of these cultures on students' receptivity to feedback is unclear. This study aimed to explore factors which aid or hinder receptivity to feedback.

Time to Loosen the Apron Strings: Cohort-based Evaluation of a Learner-driven Remediation Model at One Medical School.

Background: Remediation in the era of competency-based assessment demands a model that empowers students to improve performance.

Aim: To examine a remediation model where students, rather than faculty, develop remedial plans to improve performance.

Setting/participants: Private medical school, 177 medical students.

Targeted Feedback in the Milestones Era: Utilization of the Ask-Tell-Ask Feedback Model to Promote Reflection and Self-Assessment.

Background: The Accreditation Council for Graduate Medical Education's Milestones Project focuses trainee education on the formation of valued behaviors and skills believed to be necessary for trainees to become independent practitioners. The development and refinement of behaviors and skills outlined within the milestones will require learners to monitor, reflect, and assess their own performance over time. External feedback provides an opportunity for learners to recalibrate their self-assessments, thereby enabling them to develop better self-monitoring and self-assessment skills.

Association of research self-efficacy with medical student career interests, specialization, and scholarship: a case study.

This study used variables proposed in social cognitive career theory (SCCT) to focus the evaluation of a research curriculum at the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University (CCLCM). Eight cohorts of CCLCM medical students completed a web-based version of the six-scale Clinical Research Appraisal Inventory-Short Form (CRAI-SF) items at matriculation (n = 128) or graduation (n = 111) during 2009-2013. Parametric statistics were used to compare CRAI-SF scales to domains proposed in SCCT: trainees' characteristics (gender, training level, advanced degree), career interests, career intentions (medical specialty), and performance (peer-reviewed publications and required thesis topic).

A systematic approach toward building a fully operational clinical competency committee.

Background: The Accreditation Council for Graduate Medical Education has offered minimal guidelines for the creation and implementation of clinical competency committees (CCCs). As surgical residency programs may differ greatly in terms of size and structure, requirements that are too specific throughout the process could place some programs at a great disadvantage.

Objective: The purpose of this article is to address some of the common considerations all surgery residency programs will face.

Supporting students in self-regulation: use of formative feedback and portfolios in a problem-based learning setting.

Background: The widely recognized need for students to self-regulate their behavior and learning extends to the multiple dimensions of professionalism.

Aim: This study examines the extent to which students self-regulate professionalism behaviors related to work habits and interpersonal skills in a PBL setting.

Methods: Formative feedback on works habits and interpersonal skills provided by peers and tutors to a Year 1 cohort (n = 32) over the course of a year-long PBL experience (5 blocks) was examined for comments on targeted areas for improvement (TAFIs) and observed improvements.

Beyond assessment of learning toward assessment for learning: educating tomorrow's physicians.

Beyond its importance in informing high-stakes decisions, the assessment process can also be designed to foster learning. To be effective, this requires developing a program in which curricular experiences, assessment practices and support activities are aligned to provide an educational culture that encourages self-regulated learning. We describe a program (based at Cleveland Clinic Lerner College of Medicine) in which explicit performance standards align these components and provide a roadmap for students to manage their learning.

Student perspectives on assessment: experience in a competency-based portfolio system.

Despite considerable evidence recognizing the importance of learners' perceptions of the assessment process, there is little literature depicting the participants' experience. We aim to capture these perceptions in order to gain insights into the strengths and weaknesses of a competency-based assessment system. Cleveland Clinic Lerner College of Medicine has implemented a learner-centered portfolio assessment system built around competency standards and continuous formative feedback.

Evidence within a portfolio-based assessment program: what do medical students select to document their performance?

Background: Decisions about performance in programs of assessment that provide an array of assessment evidence require judgments about the quality of different pieces of assessment data to determine which combination of data points best represent a trainee's overall performance.

Aim: In this article, we examine the nature of evidence selected by first-year medical students to include in a portfolio used to make promotion decisions.

Methods: We reviewed portfolios to examine the number, type, and source of assessments selected by students (n = 32) to document their performance in seven competencies.

Does students' gender, citizenship, or verbal ability affect fairness of portfolio-based promotion decisions? Results from one medical school.

Purpose: Measurement experts use four criteria to examine the fairness of tests: (1) equitable treatment for examinees, (2) equal outcomes for subgroups, (3) absence of bias, and (4) equal opportunity to learn. These criteria apply to portfolios just as they do to other assessments. This report examines the fairness of portfolio-based promotion decisions for medical students at the Cleveland Clinic Lerner College of Medicine.

Evaluation of essay questions used to assess medical students' application and integration of basic and clinical science knowledge.

Background: Educators need approaches to assess medical students' abilities to apply and integrate concepts essential to medical practice.

Description: We used a multimethod approach to examine the quality of essay questions intended to elicit medical students' ability to apply and integrate their understanding of medical concepts.

Evaluation: Three educators assigned essay questions (n = 120) to one of four levels of cognition.

Methods to assess students' acquisition, application and integration of basic science knowledge in an innovative competency-based curriculum.

Background: The Cleveland Clinic Lerner College of Medicine was designed to encourage medical students to pursue careers as physician investigators. Our faculty decided that assessment should enhance learning and adopted only formative assessments to document student performance in relation to nine broad-based competencies. No grades are used to judge student performance throughout the 5-year program.

The portfolio approach to competency-based assessment at the Cleveland Clinic Lerner College of Medicine.

Despite the rapid expansion of interest in competency-based assessment, few descriptions of assessment systems specifically designed for a competency-based curriculum have been reported. The purpose of this article is to describe the design of a portfolio approach to a comprehensive, competency-based assessment system that is fully integrated with the curriculum to foster an educational environment focused on learning. The educational design goal of the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University was to create an integrated educational program-curriculum and instructional methods, student assessment processes, and learning environment-to prepare medical students for success in careers as physician investigators.

Peer assessment of professional competence.

Background: Current assessment formats for medical students reliably test core knowledge and basic skills. Methods for assessing other important domains of competence, such as interpersonal skills, humanism and teamwork skills, are less well developed. This study describes the development, implementation and results of peer assessment as a measure of professional competence of medical students to be used for formative purposes.

Psychometric properties of an instrument designed to measure the educational quality of graduate training programs.

To assess the quality of residency education programs at an academic medical center for purposes of enhancing individual graduate medical education programs, we asked residents and fellows (N = 419) to evaluate their training programs using a Web-based questionnaire (response rate = 70%). Kruskal-Wallis tests, factor analysis, correlations, generalizability/decision studies, and mean plots were used to examine trainee responses and to assess the questionnaire's measurement properties. Exploratory factor analysis indicated that the instrument had a three-factor structure that correlated highly with overall program rating.

Comprehensive assessment of professional competence: the Rochester experiment.

Background: A required 2-week comprehensive assessment (CA) for 2nd-year medical students that integrates basic science, clinical skills, information management, and professionalism was implemented.

Description: The CA links standardized patients (SPs) with computer-based exercises, a teamwork exercise, and peer assessments; and culminates in student-generated learning plans.

Evaluation: Scores assigned by SPs showed acceptable interrater reliability.

A user's guide to enhancing geriatrics in an undergraduate medical school curriculum: the ten-step model to winning the "geriatric game".

In 1999, the University of Rochester School of Medicine and Dentistry committed to major restructuring of its undergraduate medical school curriculum. A distinguishing feature of this reform was the planned emphasis on and integration of several core topics or themes throughout the 4 years of the curriculum. One of these curricular themes was aging.