Inconsistent report cards: assessing the comparability of various measures of the quality of ambulatory care.

Background: Report cards based on various performance measures have become increasingly common for rating hospitals and health care plans. However, little has been done to create report cards at the ambulatory clinic level, nor has there been much comparison of the potential components of report cards.

Objectives: To create a report card for ambulatory clinics based on different data collection methods and to assess the correlations of clinic scores across various domains of quality.

Research Design: Cross-sectional chart review (n = 3614), patient (n = 2180), and physician surveys (n = 169).

Subjects: Sample of outpatients ages 20 to 75 and their primary care providers in 11 ambulatory clinic sites in the Boston-area from May 1996 to June 1997.

Measures: Performance on various quality indicators for each site.

Results: Report card scores for five quality domains (performance on HEDIS-like measures, clinic function, patient satisfaction, diabetes guideline compliance, asthma guideline compliance) were created for each site. None of the five domain scores were significantly correlated with any of the other domains. In addition, there was substantial intraclinic variation in domain scores when compared with the corresponding mean domain score across all clinics. Additional clinic domain scores were created by limiting measures to those found on chart review or survey alone. The chart review and survey domain scores for each clinic were also not significantly correlated.

Conclusions: Report cards that emphasize only one domain of quality or use limited data collection methods may provide incomplete or inconsistent information to health care consumers about the overall quality of an outpatient clinic.