Measurement and improvement as a model to strengthen immunization information systems and overcome data gaps.

Background: IIS are important tools in the public health system and exist to improve and protect the nation's health from vaccine-preventable diseases. A network of 62 independent state, territorial, and jurisdictional immunization information systems (IIS) are operated within the United States. These systems are relied upon to implement an increasingly complex vaccination schedule, consolidate and create comprehensive immunization records, as well as monitor vaccine safety, efficacy, and support vaccine delivery. Despite their importance and necessity, the number of varying systems, coupled with jurisdictional policy and resource limitations, presents challenges with standardization, interoperability, data exchange, and the capture of complete immunization records. In partnership with the Centers for Disease Control (CDC) and IIS partners, the American Immunization Registry Association (AIRA) instituted its Measurement and Improvement (M&I) Initiative in 2015 as an innovative effort to evaluate and increase alignment of IIS with national functional standards. Lessons and strategies can be adapted for broader implementation as global systems develop methods to better achieve 2030 Sustainable Development Goal (SDG) targets, particularly related to global population health and infrastructure.

Methods: AIRA works closely with its partners to propose, vet, and refine processes and measures that can be compared across IIS, resulting in a uniform, standardized approach for measurement. The M&I Initiative is conducted as a third-party, independent evaluation through AIRA connecting with IIS pre-production systems to test the IIS response to test messages and measures across multiple content areas prioritized by the IIS community. The process includes three stages: 1) Testing and Discovery, 2) Assessment, and 3) Validation. Content areas currently evaluated include clinical decision support, interoperability transport, HL7 submission/acknowledgement, HL7 message query/response, and data quality. Testing is performed using the AIRA-developed Aggregate Analysis Reporting Tool (AART), an electronic testing tool and user interface specifically designed to compile and visualize results from the measures and tests.

Results: The M&I Initiative is voluntary with 86 percent (50/58) of the IIS programs targeted for measurement participating as of Spring 2020. To date, AIRA has actively measured standards alignment and published data on Validation in the first three content areas of Transport, Submission/Acknowledgement, and Query/Response. Thirty-one individual IIS have been validated in at least two of these three content areas. The number of IIS meeting one primary Transport measure has increased from 19 to 39 in three years, an increase of 105 percent. The number of IIS who were able to process the submission of a correctly formatted full immunization record for a patient jumped from 17 to 34, a 200 percent increase from baseline. Similarly, the number of IIS sending standards-conformant HL7 Acknowledgment messages has increased fourteen-fold since measurement began in 2017. The number of IIS who were able to process and respond to a query requesting a patient's evaluated immunization record and forecast increased from nine to 42, a 367 percent increase from baseline. Within the first two quarters of assessment, the percentage of IIS meeting the CDS measures aimed at supporting IIS alignment with ACIP recommendations increased 15 percent from baseline.

Conclusion: The M&I initiative has helped to reduce variability across IIS and strengthen immunization data in IIS that is more complete, accurate, and can be utilized with confidence. The successes and experience offer an innovative model that could be adapted to standardize measures of success and data-sharing capabilities across global borders, particularly of value in achieving SDGs aimed at ensuring healthy lives and promoting well-being for all ages through strengthened immunization systems.