Medical Student Experiences With Electronic Health Records Nationally: A Longitudinal Analysis Including School-Level Effects.

Purpose: Increasingly, medical school graduates have been expected to be competent in the knowledge and skills associated with effective electronic health record (EHR) use. Yet little is known about how student experiences with EHRs have changed over time or how these trends vary by medical school. This study examined shifts in U.

Surgical Trainee Performance and Alignment With Surgical Program Director Expectations.

Objective: To examine the alignment between graduating surgical trainee operative performance and a prior survey of surgical program director expectations.

Background: Surgical trainee operative training is expected to prepare residents to independently perform clinically important surgical procedures.

Methods: We conducted a cross-sectional observational study of US general surgery residents' rated operative performance for Core general surgery procedures.

A Proposed Blueprint for Operative Performance Training, Assessment, and Certification.

Objective: The aim of this study was to propose an evidence-based blueprint for training, assessment, and certification of operative performance for surgical trainees.

Summary Background Data: Operative skill is a critical aspect of surgical performance. High-quality assessment of operative skill therefore has profound implications for training, accreditation, certification, and the public trust of the profession.

Medical student experiences with accessing and entering patient information in electronic health records during the obstetrics-gynecology clerkship.

Background: Medical school graduates should be able to enter information from patient encounters and to write orders and prescriptions in the electronic health record. Studies have shown that, although students often can access electronic health records, some students may receive inadequate preparation for these skills. Greater understanding of student exposure to electronic health records during their obstetrics and gynecology clerkships can help to determine the extent to which students receive the educational experiences that may best prepare them for their future training and practice.

Number of Operative Performance Ratings Needed to Reliably Assess the Difficulty of Surgical Procedures.

Objective: The profession of surgery is entering a new era of "big data," where analyses of longitudinal trainee assessment data will be used to inform ongoing efforts to improve surgical education. Given the high-stakes implications of these types of analyses, researchers must define the conditions under which estimates derived from these large datasets remain valid. With this study, we determine the number of assessments of residents' performances needed to reliably assess the difficulty of "Core" surgical procedures.

Electronic Health Record Use in Internal Medicine Clerkships and Sub-internships for Medical Students Graduating from 2012 to 2016.

Background: As electronic health records (EHRs) became broadly available in medical practice, effective use of EHRs by medical students emerged as an essential aspect of medical education. While new federal clinical documentation guidelines have the potential to encourage greater medical student EHR use and enhance student learning experiences with respect to EHRs, little is known nationally about how students have engaged with EHRs in the past.

Objective: This study examines medical student accounts of EHR use during their internal medicine (IM) clerkships and sub-internships during a 5-year time period prior to the new clinical documentation guidelines.

Medical Student Use of Electronic and Paper Health Records During Inpatient Clinical Clerkships: Results of a National Longitudinal Study.

Purpose: An important goal of medical education is to teach students to use an electronic health record (EHR) safely and effectively. The purpose of this study is to examine medical student accounts of EHR use during their core inpatient clinical clerkships using a national sample. Paper health records (PHRs) are similarly examined.

Longitudinal assessments in continuing specialty certification and lifelong learning.

Longitudinal assessment (LA) involves the regular, spaced delivery of a limited number of questions on practice relevant content on a computer or mobile internet platform. Depending on the platform, participants may indicate relevance of the content to their practice and confidence in their answer prior to receiving immediate feedback (including critiques) on each question. Individual dashboards may be included to assist participants in tracking progress and identifying areas of strength and weaknesss across a content blueprint.

How Many Observations are Needed to Assess a Surgical Trainee's State of Operative Competency?

Objective: To establish the number of operative performance observations needed for reproducible assessments of operative competency.

Background: Surgical training is transitioning from a time-based to a competency-based approach, but the number of assessments needed to reliably establish operative competency remains unknown.

Methods: Using a smart phone based operative evaluation application (SIMPL), residents from 13 general surgery training programs were evaluated performing common surgical procedures.

Exploring the Relationships Between USMLE Performance and Disciplinary Action in Practice: A Validity Study of Score Inferences From a Licensure Examination.

Purpose: Physicians must pass the United States Medical Licensing Examination (USMLE) to obtain an unrestricted license to practice allopathic medicine in the United States. Little is known, however, about how well USMLE performance relates to physician behavior in practice, particularly conduct inconsistent with safe, effective patient care. The authors examined the extent to which USMLE scores relate to the odds of receiving a disciplinary action from a U.

Trends in national licensing examinations in medicine.

Context: As a contribution to this special issue commemorating the journal's 50th volume, this paper seeks to explore directions for national licensing examinations (NLEs) in medicine. Increases in the numbers of new medical schools and the mobility of doctors across national borders mean that NLEs are becoming even more important to ensuring physician competence.

Objectives: The purpose of this paper is to explore the use of NLEs in the future in the context of global changes in medical education and health care delivery.

US Medical Student Performance on the NBME Subject Examination in Internal Medicine: Do Clerkship Sequence and Clerkship Length Matter?

Background: Prior to graduation, US medical students are required to complete clinical clerkship rotations, most commonly in the specialty areas of family medicine, internal medicine, obstetrics and gynecology (ob/gyn), pediatrics, psychiatry, and surgery. Within a school, the sequence in which students complete these clerkships varies. In addition, the length of these rotations varies, both within a school for different clerkships and between schools for the same clerkship.

Team cohesiveness, team size and team performance in team-based learning teams.

Objectives: The purpose of this study was to explore the relationships among variables associated with teams in team-based learning (TBL) settings and team outcomes.

Methods: We administered the National Board of Medical Examiners (NBME) Psychiatry Subject Test first to individuals and then to teams of Year three students at four medical schools that used TBL in their psychiatry core clerkships. Team cohesion was analysed using the Team Performance Scale (TPS).

International variation in performance by clinical discipline and task on the United States medical licensing examination step 2 clinical knowledge component.

Purpose: To investigate country-to-country variation in performance across clinical science disciplines and tasks for examinees taking the Step 2 Clinical Knowledge (CK) component of the United States Medical Licensing Examination.

Method: In 2012 the authors analyzed demographic characteristics, total scores, and percent-correct clinical science discipline and task scores for more than 88,500 examinees taking Step 2 CK for the first time during the 2008-2010 academic years. For each examinee and score, differences between the score and the mean performance of examinees at U.

The relationship between the american board of anesthesiology part 1 certification examination and the United States medical licensing examination.

Background: The graduate medical education community uses results from the United States Medical Licensing Examination (USMLE) to inform decisions about individuals' readiness for postgraduate training.

Objective: We sought to determine the relationship between performance on the USMLE and the American Board of Anesthesiology (ABA) Part 1 Certification Examination using a national sample of examinees, and we considered the relationship in the context of undergraduate medical education location and examination content.

Methods: Approximately 7800 individuals met inclusion criteria.

Changes in anatomy instruction and USMLE performance: empirical evidence on the absence of a relationship.

Anatomy instruction has evolved over the past two decades as many medical schools have undergone various types of curricular reform. To provide empirical evidence about whether or not curricular changes impact the acquisition and retention of anatomy knowledge, this study investigated the effect of variation in gross anatomy course hours, curricular approach (stand-alone versus integrated), and laboratory experience (dissection versus dissection and prosection) on USMLE Steps 1 and 2 Clinical Knowledge (CK) scores. Gross anatomy course directors at 54 United States schools provided information about their gross anatomy courses via an online survey (response rate of 42%).

A multilevel analysis of examinee gender, standardized patient gender, and United States medical licensing examination step 2 clinical skills communication and interpersonal skills scores.

Background: Women typically demonstrate stronger communication skills on performance-based assessments using human raters in medical education settings. This study examines the effects of examinee and rater gender on communication and interpersonal skills (CIS) scores from the performance-based component of the United States Medical Licensing Examination, the Step 2 Clinical Skills (CS) examination.

Method: Data included demographic and performance information for examinees that took Step 2 CS for the first time in 2009.

The relationship between USMLE Step 2 CS patient note ratings and time spent on the note: do examinees who spend more time write better notes?

Background: During the United States Medical Licensing Examination Step 2 Clinical Skills examination, examinees rotate through 12 standardized patient (SP) encounters. Examinees have 25 minutes per encounter to interact with SPs and complete postencounter patient notes (PNs), and they may end the SP interaction early to spend extra time on the PN. The current work assesses the time examinees are spending on PNs and whether this is related to performance on the PN.

Practical considerations in equating progress tests.

Progress testing offers a unique contribution to medical education assessment in that it provides a tool for monitoring examinees' growth over time and their progress toward graduation objectives. However, one of the most important psychometric requirements of progress testing has typically been neglected in past applications: scores across the time and test forms are not commonly placed on the same scale. Equating is a method used to achieve this property.

Progress testing in clinical science education: results of a pilot project between the National Board of Medical Examiners and a US Medical School.

Background: Though progress tests have been used for several decades in various medical education settings, a few studies have offered analytic frameworks that could be used by practitioners to model growth of knowledge as a function of curricular and other variables of interest.

Aim: To explore the use of one form of progress testing in clinical education by modeling growth of knowledge in various disciplines as well as by assessing the impact of recent training (core rotation order) on performance using hierarchical linear modeling (HLM) and analysis of variance (ANOVA) frameworks.

Methods: This study included performances across four test administrations occurring between July 2006 and July 2007 for 130 students from a US medical school who graduated in 2008.

Using multivariate generalizability theory to assess the effect of content stratification on the reliability of a performance assessment.

In recent years, demand for performance assessments has continued to grow. However, performance assessments are notorious for lower reliability, and in particular, low reliability resulting from task specificity. Since reliability analyses typically treat the performance tasks as randomly sampled from an infinite universe of tasks, these estimates of reliability may not be accurate.

Use of multimedia on the step 1 and step 2 clinical knowledge components of USMLE: a controlled trial of the impact on item characteristics.

Background: During 2007, multimedia-based presentations of selected clinical findings were introduced into the United States Medical Licensing Examination. This study investigated the impact of presenting cardiac auscultation findings in multimedia versus text format on item characteristics.

Method: Content-matched versions of 43 Step 1 and 51 Step 2 Clinical Knowledge (CK) multiple-choice questions describing common pediatric and adult clinical presentations were administered in unscored sections of Step 1 and Step 2 CK.

Assessing potentially dangerous medical actions with the computer-based case simulation portion of the USMLE step 3 examination.

Background: The 2000 Institute of Medicine report on patient safety brought renewed attention to the issue of preventable medical errors, and subsequently specialty boards and the National Board of Medical Examiners were encouraged to play a role in setting expectations around safety education. This paper examines potentially dangerous actions taken by examinees during the portion of the United States Medical Licensing Examination Step 3 that is particularly well suited to evaluating lapses in physician decision making, the Computer-based Case Simulation (CCS).

Method: Descriptive statistics and a general linear modeling approach were used to analyze dangerous actions ordered by 25,283 examinees that completed CCS for the first time between November 2006 and January 2008.

Relationship between performance on part I of the American Board of Orthopaedic Surgery Certifying Examination and Scores on USMLE Steps 1 and 2.

Background: This study investigated the strength of the relationship between performance on Part I of the American Board of Orthopaedic Surgery (ABOS) Certifying Examination and scores on United States Medical Licensing Examination (USMLE) Steps 1 and 2.

Method: USMLE Step 1 and Step 2 scores on first attempt were matched with ABOS Part I results for U.S.