The immunization delivery effectiveness assessment score: a better immunization measure?

Background: Childhood immunization measures, such as the Health Employer Data Information Set (HEDIS) or the National Immunization Survey, assess the percentage of children up-to-date for a specified series of vaccinations. In particular, the HEDIS assesses immunization delivery to children enrolled in managed care organizations (MCO). Such measures do not assess the timeliness of immunization delivery with reference to recommended age standards. To achieve maximal protection against vaccine-preventable diseases, children should receive all immunizations within recommended age intervals-fully "on-time."

Objective: The Immunization Delivery Effectiveness Assessment (IDEA) is a novel immunization measure that assesses, on a continuous scale, the timeliness of administration of each vaccination with reference to recommended age intervals. Specifically we ask: 1) Do absolute immunization rates differ between HEDIS and IDEA? 2) Does relative MCO performance differ when assessed by the 2 performance measures? 3) How well do MCOs perform relative to the standard of fully on-time immunization? The health services implications of using the timeliness standard to assess childhood immunization delivery is discussed.

Methods: A vaccine-dose IDEA score was developed for each of the 14 vaccination events in the 4:3:1:3:3 combination series (4 diphtheria-pertussis-tetanus:3 polio:1 measles-mumps-rubella:3 Haemophilus influenzae type B:3 hepatitis B). Assessing the actual age of administration with reference to the recommended age of administration generates the vaccine-dose IDEA score. A child's composite IDEA score is obtained by averaging the 14 vaccine-dose IDEA scores. These composite IDEA scores, when averaged among children sampled within the MCO, constitute the MCO's immunization score.

Setting: Retrospective analysis of childhood immunization datasets from a convenience sample of 6 MCOs in 5 states.

Results: HEDIS rates ranged from 57% to 75%. IDEA scores ranged from 80% to 90%. Relative rankings of MCO immunization performance were different using HEDIS rates and IDEA scores, respectively. At most, 16% of children in any of these MCOs received all of their immunizations fully on-time. From 47% to 77% of children experienced at least 3 delayed immunizations.

Conclusions: An immunization measure based on timeliness of administration yields both absolute and relative differences in MCO childhood immunization performance when compared with HEDIS rates. By assessing delivery of each component vaccination, the IDEA score permits more detailed analysis of immunization patterns within an MCO and focuses improvement efforts.