Exploring the determinants of NHS performance ratings: lessons for performance assessment systems.

Objectives: A basic tenet of effective performance management is that decision makers should be held responsible only for aspects of performance over which they have control. We examine the degree to which variations in the performance of health care organizations are explained by a range of factors that are subject to differing degrees of managerial control.

Methods: We use multiple regression methods and data on 304 National Health Service (NHS) Primary Care Trusts (PCTs) in England for 2002/03 to analyse the relationship between their performance, as measured by 'star ratings' and nine key performance targets, and a large number of explanatory variables. We classify the explanatory variables into five groups along a spectrum from 'no managerial control' to 'total managerial control'. We also analyse the relationship between PCTs that are service purchasers and their main providers to assess the degree to which their performance is linked.

Results: For all of the key performance targets, most of the variation in performance is unexplained, although variables in groups 1, 2 and 3 (less managerial control) explain the largest proportion of variation. We find some evidence that socioeconomic and geographic factors have an impact on performance. We also find a positive relationship between the performances of purchaser and provider organizations.

Conclusions: The star ratings did not allow for the different environmental circumstances within which PCTs operate and which affect their performance. Policy-makers should exercise caution in using such performance indicators to regulate health care organizations.