Out-of-hours primary care: development of indicators for prescribing and referring.

Background: Dutch general practitioners have reorganized their out-of-hours primary health care to general practice cooperatives. Good insight into the quality of delivered medical care is important to make the accountability of health practitioners and managers transparent to society and to identify and minimize medical errors.

Objective: Development of a set of quality indicators for internal quality improvement in out-of-hours primary clinical care.

Method: A systematic approach combining the opinion of three different general practitioner expert panels, and an empirical test in daily practice. The indicators were based on clinical, evidence-based, national guidelines. We tested the validity, feasibility, reliability and opportunity for quality improvement.

Results: Of the 80 available national clinical guidelines, 29 were approved and selected by the first general practitioner expert panel. Out of these 29 guidelines, 73 indicators concerning prescribing and referring were selected by the second panel. In an empirical test on 36 254 patient contacts, 7344 patient contacts (22.7%) were relevant for the assessment of these 73 indicators. Six indicators were excluded because they scored more than 15% missing values. In total, 38 indicators were excluded because the opportunity for quality improvement was limited (performance score > or =90%). In the final meeting, the third general practitioner expert panel excluded five indicators, leading to a final set of 24 indicators.

Conclusion: This study shows the importance of subjecting indicators to an empirical test in practice. The national clinical guidelines are only partially applicable in the assessment of out-of-hours primary care. They need to be expanded with topics that are related to general practitioner care in an out-of-hours setting and acute medical problems.