Recommendations to address and research systemic bias in assessment: perspectives from directors of research in medical education.

Introduction: Addressing systemic bias in medical school assessment is an urgent task for medical education. This paper outlines recommendations on topic areas for further research on systemic bias, developed from a workshop discussion at the 2023 annual meeting of the Society of Directors of Research in Medical Education.

Materials And Methods: During the workshop, directors engaged in small-group discussions on guidelines to address bias in assessment practices following a proposed categorization of 'Do's,' 'Don'ts,' and 'Don't knows' and listed their insights using anonymous sticky notes, which were shared and discussed with the larger group of participants.

Using Natural Language Processing to Visualize Narrative Feedback in a Medical Student Performance Dashboard.

Problem: Clinical competency committees rely on narrative feedback for important insight into learner performance, but reviewing comments can be time-consuming. Techniques such as natural language processing (NLP) could create efficiencies in narrative feedback review. In this study, the authors explored whether using NLP to create a visual dashboard of narrative feedback to preclerkship medical students would improve the competency review efficiency.

Students as Teachers: Development and Implementation of a Workplace-Based Assessment.

Students as Teachers programs are prevalent, though assessments within these programs are lacking. A workplace-based assessment for clinical teaching was developed to foster formative feedback and support learner growth. Feedback narratives were analyzed to identify student teaching behaviors and demonstrated the themes medical knowledge, professionalism, communication, and teaching skills, which were subcategorized as clinical relevance, learner stage appropriateness, use of evidence-based teaching strategies, learning environment, feedback-related, and time-appropriate.

Precision Education: The Future of Lifelong Learning in Medicine.

The goal of medical education is to produce a physician workforce capable of delivering high-quality equitable care to diverse patient populations and communities. To achieve this aim amidst explosive growth in medical knowledge and increasingly complex medical care, a system of personalized and continuous learning, assessment, and feedback for trainees and practicing physicians is urgently needed. In this perspective, the authors build on prior work to advance a conceptual framework for such a system: precision education (PE).

Utilizing Natural Language Processing of Narrative Feedback to Develop a Predictive Model of Pre-Clerkship Performance: Lessons Learned.

Background: Natural language processing is a promising technique that can be used to create efficiencies in the review of narrative feedback to learners. The Feinberg School of Medicine has implemented formal review of pre-clerkship narrative feedback since 2014 through its portfolio assessment system but this process requires considerable time and effort. This article describes how natural language processing was used to build a predictive model of pre-clerkship student performance that can be utilized to assist competency committee reviews.

Entrustable Professional Activity-Based Summative Performance Assessment in the Surgery Clerkship.

Objective: The objectives of this study were to 1) assess the performance Entrustable Professional Activities (EPAs) when integrated into the summative assessment of third-year medical students on the surgery clerkship and 2) to compare EPAs to traditional clinical performance assessment tools.

Design: EPA assessments were collected prospectively from a minimum of 4 evaluators at the completion of each surgical clerkship rotation from November 2019 to June 2019. Overall EPA-based clinical performance scores were calculated as the sum of the mean EPA score from each evaluator.

A Qualitative Analysis of Narrative Preclerkship Assessment Data to Evaluate Teamwork Skills.

: Students entering the health professions require competency in teamwork. : Although many teamwork curricula and assessments exist, studies have not demonstrated robust longitudinal assessment of preclerkship students' teamwork skills and attitudes. Assessment portfolios may serve to fill this gap, but it is unknown how narrative comments within portfolios describe student teamwork behaviors.

Description and Early Outcomes of a Comprehensive Curriculum Redesign at the Northwestern University Feinberg School of Medicine.

In 2012, the Northwestern University Feinberg School of Medicine launched a redesigned curriculum addressing the four primary recommendations in the 2010 Carnegie Foundation for the Advancement of Teaching report on reforming medical education. This new curriculum provides a more standardized evaluation of students' competency achievement through a robust portfolio review process coupled with standard evaluations of medical knowledge and clinical skills. It individualizes learning processes through curriculum flexibility, enabling students to take electives earlier and complete clerkships in their preferred order.

What Is the Relationship Between a Preclerkship Portfolio Review and Later Performance in Clerkships?

Purpose: Medical educators struggle to find effective ways to assess essential competencies such as communication, professionalism, and teamwork. Portfolio-based assessment provides one method of addressing this problem by allowing faculty reviewers to judge performance, as based on a longitudinal record of student behavior. At the Feinberg School of Medicine, the portfolio system measures behavioral competence using multiple assessments collected over time.

Feasibility and Outcomes of Implementing a Portfolio Assessment System Alongside a Traditional Grading System.

Purpose: Portfolios are a powerful tool to collect and evaluate evidence of medical students' competence across time. However, comprehensive portfolio assessment systems that are implemented alongside traditional graded curricula at medical schools in the United States have not been described in the literature. This study describes the development and implementation of a longitudinal competency-based electronic portfolio system alongside a graded curriculum at a relatively large U.

Development of a metacognitive effort construct of empathy during clinical training: a longitudinal study of the factor structure of the Jefferson Scale of Empathy.

Empathy is crucial for effective clinical care but appears to decline during undergraduate medical training. Understanding the nature of this decline is necessary for addressing it. The Jefferson Scale of Empathy (JSE) is used to measure medical students' clinical empathy attitudes.