Computer Adaptive vs. Non-adaptive Medical Progress Testing: Feasibility, Test Performance, and Student Experiences.

Background: Computerized adaptive testing tailors test items to students' abilities by adapting difficulty level. This more efficient, and reliable assessment form may provide advantages over a conventional medical progress test (PT). Prior to our study, a direct comparison of students' performance on a computer adaptive progress test (CA-PT) and a conventional PT, which is crucial for nationwide implementation of the CA-PT, was missing.

Studies on Reliability and Measurement Error of Measurements in Medicine - From Design to Statistics Explained for Medical Researchers.

Reliability and measurement error are measurement properties that quantify the influence of specific sources of variation, such as raters, type of machine, or time, on the score of the individual measurement. Several designs can be chosen to assess reliability and measurement error of a measurement. Differences in design are due to specific choices about which sources of variation are varied over the repeated measurements in stable patients, which potential sources of variation are kept stable (ie, restricted), and about whether or not the entire measurement instrument (or measurement protocol) was repeated or only part of it.

Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents' Performance in Pediatric Emergency Medicine.

Purpose: Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs).

Method: EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children's Hospital Medical Center, a quaternary children's hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance.

Utility of a multimodal computer-based assessment format for assessment with a higher degree of reliability and validity.

Multiple choice questions (MCQs) suffer from cueing, item quality and factual knowledge testing. This study presents a novel multimodal test containing alternative item types in a computer-based assessment (CBA) format, designated as Proxy-CBA. The Proxy-CBA was compared to a standard MCQ-CBA, regarding validity, reliability, standard error of measurement, and cognitive load, using a quasi-experimental crossover design.

Factors Influencing Professional Identity Construction in Fourth-Year Pharmacy Students.

To explore the key factors that influence professional identity construction in fourth-year pharmacy students enrolled in a Doctor of Pharmacy program. A single-site instrumental case study of current fourth-year pharmacy students from the Leslie Dan Faculty of Pharmacy, University of Toronto, was used. Thirteen students participated in semistructured interviews.

The limited use of instructional design guidelines in healthcare simulation scenarios: an expert appraisal.

Background: Systematic reviews on simulation training effectiveness have pointed to the need to adhere to evidence-based instructional design (ID) guidelines. ID guidelines derive from sound cognitive theories and aim to optimize complex learning (integration of knowledge, skills, and attitudes) and learning transfer (application of acquired knowledge and skills in the workplace). The purpose of this study was to explore adherence to ID guidelines in simulation training programs for dealing with postpartum hemorrhage (PPH), a high-risk situation and the leading cause of maternal mortality worldwide.

Supervisory dyads' communication and alignment regarding the use of workplace-based observations: a qualitative study in general practice residency.

Background: In medical residency, performance observations are considered an important strategy to monitor competence development, provide feedback and warrant patient safety. The aim of this study was to gain insight into whether and how supervisor-resident dyads build a working repertoire regarding the use of observations, and how they discuss and align goals and approaches to observation in particular.

Methods: We used a qualitative, social constructivist approach to explore if and how supervisory dyads work towards alignment of goals and preferred approaches to performance observations.

Ottawa 2020 consensus statement for programmatic assessment - 1. Agreement on the principles.

Introduction: In the Ottawa 2018 Consensus framework for good assessment, a set of criteria was presented for systems of assessment. Currently, programmatic assessment is being established in an increasing number of programmes. In this Ottawa 2020 consensus statement for programmatic assessment insights from practice and research are used to define the principles of programmatic assessment.

When Feedback Backfires: Influences of Negative Discrepancies Between Physicians' Self and Assessors' Scores on Their Subsequent Multisource Feedback Ratings.

Introduction: With multisource feedback (MSF) physicians might overrate their own performance compared with scores received from assessors. However, there is limited insight into how perceived divergent feedback affects physicians' subsequent performance scores.

Methods: During 2012 to 2018, 103 physicians were evaluated twice by 684 peers, 242 residents, 999 coworkers, and themselves in three MSF performance domains.

Composite Reliability of Workplace Based Assessment of International Medical Graduates.

This article was migrated. The article was marked as recommended. Introduction All developed countries depend on International Medical Graduates (IMGs) to complement their workforce.

Improving the Competency of Nurses.

To decrease infusion pump administration errors, time-consuming training is often initiated. The aims of this study were twofold: to develop minimum competency requirements for programming and operation of infusion pumps and to develop and validate a test for nurses based on those requirements. The test was completed by 226 nurses between May and December 2017.

A history of assessment in medical education.

The way quality of assessment has been perceived and assured has changed considerably in the recent 5 decades. Originally, assessment was mainly seen as a measurement problem with the aim to tell people apart, the competent from the not competent. Logically, reproducibility or reliability and construct validity were seen as necessary and sufficient for assessment quality and the role of human judgement was minimised.

A Historical Discourse Analysis of Pharmacist Identity in Pharmacy Education.

To determine the discourses on professional identity in pharmacy education over the last century in North America and which one(s) currently dominate. A Foucauldian critical discourse analysis using archival resources from the () and commissioned education reports was used to expose the identity discourses in pharmacy education. This study identified five prominent identity discourses in the pharmacy education literature: apothecary, dispenser, merchandiser, expert advisor, and health care provider.

Driving lesson or driving test? : A metaphor to help faculty separate feedback from assessment.

Although there is consensus in the medical education world that feedback is an important and effective tool to support experiential workplace-based learning, learners tend to avoid the feedback associated with direct observation because they perceive it as a high-stakes evaluation with significant consequences for their future. The perceived dominance of the summative assessment paradigm throughout medical education reduces learners' willingness to seek feedback, and encourages supervisors to mix up feedback with provision of 'objective' grades or pass/fail marks. This eye-opener article argues that the provision and reception of effective feedback by clinical supervisors and their learners is dependent on both parties' awareness of the important distinction between feedback used in coaching towards growth and development (assessment for learning) and reaching a high-stakes judgement on the learner's competence and fitness for practice (assessment of learning).

The Incidence and Nature of Adverse Medical Device Events in Dutch Hospitals: A Retrospective Patient Record Review Study.

Objectives: Despite widespread use of medical devices and their increasing complexity, their contribution to unintended injury caused by healthcare (adverse events, AEs) remains relatively understudied. The aim of this study was to gain insight in the incidence and types of AEs involving medical devices (AMDEs).

Methods: Data from two patient record studies for the identification of AEs were used.

Students' perceptions towards self-directed learning in Ethiopian medical schools with new innovative curriculum: a mixed-method study.

Background: Self-directed learning (SDL) is an appropriate and preferred learning process to prepare students for lifelong learning in their professions and make them stay up-to-date. The purpose of this study was to explore preclinical students following a hybrid curriculum in Ethiopia experiences to SDL and the support of several learning activities from the curriculum on their SDL. A mixed-method research design was employed.

Assessment in the context of problem-based learning.

Arguably, constructive alignment has been the major challenge for assessment in the context of problem-based learning (PBL). PBL focuses on promoting abilities such as clinical reasoning, team skills and metacognition. PBL also aims to foster self-directed learning and deep learning as opposed to rote learning.

Meaningful feedback through a sociocultural lens.

This AMEE guide provides a framework and practical strategies for teachers, learners and institutions to promote meaningful feedback conversations that emphasise performance improvement and professional growth. Recommended strategies are based on recent feedback research and literature, which emphasise the sociocultural nature of these complex interactions. We use key concepts from three theories as the underpinnings of the recommended strategies: sociocultural, politeness and self-determination theories.

Relationships as the Backbone of Feedback: Exploring Preceptor and Resident Perceptions of Their Behaviors During Feedback Conversations.

Purpose: Newer definitions of feedback emphasize learner engagement throughout the conversation, yet teacher and learner perceptions of each other's behaviors during feedback exchanges have been less well studied. This study explored perceptions of residents and faculty regarding effective behaviors and strategies during feedback conversations and factors that affected provision and acceptance of constructive feedback.

Method: Six outpatient internal medicine preceptors and 12 residents at Brigham and Women's Hospital participated (2 dyads per preceptor) between September 2017 and May 2018.

Exploring Validity Evidence Associated With Questionnaire-Based Tools for Assessing the Professional Performance of Physicians: A Systematic Review.

Purpose: To collect and examine-using an argument-based validity approach-validity evidence of questionnaire-based tools used to assess physicians' clinical, teaching, and research performance.

Method: In October 2016, the authors conducted a systematic search of the literature seeking articles about questionnaire-based tools for assessing physicians' professional performance published from inception to October 2016. They included studies reporting on the validity evidence of tools used to assess physicians' clinical, teaching, and research performance.