Proximity to Practice: The Role of Technology in the Next Era of Assessment.

The integration of technology into health professions assessment has created multiple possibilities. In this paper, we focus on the challenges and opportunities of integrating technologies that are used during clinical activities or that are completed by raters after a clinical encounter. In focusing on technologies that are more proximal to practice, we identify tradeoffs with different data collection approaches.

Examining ChatGPT Performance on USMLE Sample Items and Implications for Assessment.

Purpose: In late 2022 and early 2023, reports that ChatGPT could pass the United States Medical Licensing Examination (USMLE) generated considerable excitement, and media response suggested ChatGPT has credible medical knowledge. This report analyzes the extent to which an artificial intelligence (AI) agent's performance on these sample items can generalize to performance on an actual USMLE examination and an illustration is given using ChatGPT.

Method: As with earlier investigations, analyses were based on publicly available USMLE sample items.

Assessing the validity of test scores using response process data from an eye-tracking study: a new approach.

Understanding the response process used by test takers when responding to multiple-choice questions (MCQs) is particularly important in evaluating the validity of score interpretations. Previous authors have recommended eye-tracking technology as a useful approach for collecting data on the processes test taker's use to respond to test questions. This study proposes a new method for evaluating alternative score interpretations by using eye-tracking data and machine learning.

A Natural-Language-Processing-Based Procedure for Generating Distractors for Multiple-Choice Questions.

One of the most challenging aspects of writing multiple-choice test questions is identifying plausible incorrect response options-i.e., distractors.

Detecting High-Functioning Autism in Adults Using Eye Tracking and Machine Learning.

The purpose of this study is to test whether visual processing differences between adults with and without high-functioning autism captured through eye tracking can be used to detect autism. We record the eye movements of adult participants with and without autism while they look for information within web pages. We then use the recorded eye-tracking data to train machine learning classifiers to detect the condition.