Electronic health record problem lists: accurate enough for risk adjustment?

Objectives: To determine whether comorbidity information derived from electronic health record (EHR) problem lists is accurate.

Study Design: Retrospective cohort study of 1596 men diagnosed with prostate cancer between 1998 and 2004 at 2 Southern California Veterans Affairs Medical Centers with long-term follow-up.

Methods: We compared EHR problem list-based comorbidity assessment with manual review of EHR free-text notes in terms of sensitivity and specificity for identification of major comorbidities and Charlson Comorbidity Index (CCI) scores. We then compared EHR-based CCI scores with free-text-based CCI scores in prediction of long-term mortality.

Results: EHR problem list-based comorbidity assessment had poor sensitivity for detecting major comorbidities: myocardial infarction (8%), cerebrovascular disease (32%), diabetes (46%), chronic obstructive pulmonary disease (42%), peripheral vascular disease (31%), liver disease (1%), and congestive heart failure (23%). Specificity was above 94% for all comorbidities. Free-text-based CCI scores were predictive of long-term other-cause mortality, whereas EHR problem list-based scores were not.

Conclusions: Inaccuracies in EHR problem list-based comorbidity data can lead to incorrect determinations of case mix. Such data should be validated prior to application to risk adjustment.