A framework for rigorous evaluation of human performance in human and machine learning comparison studies.

Rigorous comparisons of human and machine learning algorithm performance on the same task help to support accurate claims about algorithm success rates and advances understanding of their performance relative to that of human performers. In turn, these comparisons are critical for supporting advances in artificial intelligence. However, the machine learning community has lacked a standardized, consensus framework for performing the evaluations of human performance necessary for comparison.

Association of Disease Severity and Socioeconomic Status in Black and White Americans With Multiple Sclerosis.

Objective: To compare clinical and imaging features of multiple sclerosis (MS) severity between Black Americans (BAs) and White Americans (WAs) and to evaluate the role of socioeconomic status.

Methods: We compared BA and WA participants in the Multiple Sclerosis Partners Advancing Technology Health Solutions (MS PATHS) cohort with respect to MS characteristics, including self-reported disability, objective neurologic function assessments, and quantitative brain MRI measurements, after covariate adjustment (including education level, employment, or insurance as socioeconomic indicators). In a subgroup, we evaluated within-race, neighborhood-level indicators of socioeconomic status (SES) using 9-digit zip codes.