Evaluation criteria for district health management information systems: lessons from the Ministry of Health, Kenya.

Introduction: There has been no comprehensive evaluation of the district health management information systems (DHMISs) since the establishment of these systems by the Ministry of Health (MoH), in Kenya. This is partly due to lack of defined criteria for evaluating the systems. The objective of this study is to design evaluation criteria for assessing the viability, sustainability and ultimate contribution of DHMIS in the management of the district health system (DHS) in Kenya.

Methods: This descriptive cross-sectional study was undertaken in three DHSs in Kenya. Empirical evidence was collected through interviews, complemented by a comprehensive review of relevant literature, reports and operational manuals of various health information systems in Kenya.

Results: A set of evaluation criteria for DHMISs in Kenya was designed for each of the three phases of implementation: phase one-pre-implementation evaluation criteria (categorized as policy and objectives, technical feasibility, financial viability, political viability and administrative operability) to be applied at the design stage; phase two-concurrent (operational) implementation evaluation criteria to be applied during implementation of the new system; phase three-post-implementation evaluation criteria (classified as internal-quality of information; external-resources and managerial support; ultimate-systems impact) to be applied after operating the implemented system for at least 3 years.

Conclusions: In designing a DHMIS model there is need to have built-in these three sets of evaluation criteria which should be used in a phased manner. Pre-implementation evaluation criteria should be used to evaluate the system's viability before more resources are committed to its implementation; concurrent (operational) implementation evaluation criteria should be used to ascertain the status of the on-going implementation with the view to either fine-tune or abandon it altogether before more resources are used on it; and post-implementation evaluation criteria should be used to assess its overall effectiveness (if it has achieved its hypothesized benefits) towards the management of DHS.