EVALUATION Mid-Term Evaluation of the Integrated Disease Surveillance and Response Project
Total Page:16
File Type:pdf, Size:1020Kb
EVALUATION Mid-Term Evaluation of the Integrated Disease Surveillance and Response Project December 14, 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Bob Pond, Hammam El Sakka, Joseph Wamala and Luswa Lukwago, Management Systems International. MID-TERM EVALUATION OF THE INTEGRATED DISEASE SURVEILLANCE AND RESPONSE PROJECT DECEMBER 14, 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Bob Pond, Hammam El Sakka, Joseph Wamala and Luswa Lukwago, Management Systems International. MID-TERM EVALUATION OF THE INTEGRATED DISEASE SURVEILLANCE AND RESPONSE PROJECT Contracted No. DFD-1-00-05-00251-00, Task Order No. 2 Services under Program and Project Offices for Results Tracking (SUPPORT) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS List of Tables ............................................................................................................................. ii List of Figures ........................................................................................................................... ii Acknowledgements .................................................................................................................. iii Executive Summary................................................................................................................... 1 Introduction and Background ................................................................................................... 6 Methodology and Data Limitations ......................................................................................... 11 Findings .................................................................................................................................... 15 1. Effectiveness of the overall project—Part I ..................................................................... 15 I. Are the project’s core activities being implemented as planned? .............................................. 15 2. Timely reporting and response ....................................................................................... 25 II. Are IDSR data accurately recorded and reported on a timely basis to higher levels within the system? ............................................................................................................................... 25 III. When outbreaks and epidemic alerts are reported, are they responded to promptly and effectively? ................................................................................................................... 33 3. Integration ...................................................................................................................... 36 IV. How have/are other disease surveillance programs being integrated into IDSR? .............. 36 4. Capacity building ............................................................................................................ 38 V. What key strategies is WHO using to build the necessary institutional capacity within the Ministry of Health to manage and run the disease surveillance and response program once the project ends? ....................................................................................... 38 5. Effectiveness of the overall project—Part II ................................................................... 40 VI. Are IDSR data used for decision-making? If so, how? ............................................................ 40 VII. What are the MoH, other donor, and stakeholder impressions of the program? .............. 42 VIII. What, if any, are major overall shortcomings that should be addressed for effective project implementation? .................................................................................................... 43 Conclusions ............................................................................................................................. 45 IX. What are the critical lessons learned from the program experience to date? ....................... 45 Recommendations ................................................................................................................... 47 Annexes ................................................................................................................................... 52 Annex 1: Scope of Work ...................................................................................................... 52 Annex 2: Funding for IDSR ................................................................................................ 67 Annex 3: Assessment Tools ................................................................................................ 69 Annex 4: Photos .................................................................................................................. 83 Annex 5: The IDSR M&E Plan from the proposed five-year Plan of Action ..................... 88 Annex 6: WHO’s documentation of IDSR basic training .................................................. 93 Annex 7: Trends in the completeness of weekly reporting from three states ..................... 95 Annex 8: Examples of data demanding action ................................................................... 96 Annex 9: What must be done before the national public health laboratory will be functional? ................................................................................................................... 98 Annex 10: Use of surveillance data for decision-making .................................................... 99 Annex 11: Proposed scope of work for a short-term consultant to strengthen implementation of IDSR in South Sudan .................................................................. 101 Annex 12: A balanced set of indicators for monitoring and evaluation of IDSR in South Sudan ............................................................................................................... 103 Annex 13: Persons interviewed and sites visited ................................................................ 105 Annex 14: Documents reviewed ......................................................................................... 108 MID-TERM EVALUATION OF THE INTEGRATED DISEASE SURVEILLANCE AND RESPONSE PROJECT DECEMBER 14, 2011 i LIST OF TABLES Table 1: Topics and questions specified by USAID’s scope of work for the IDSR mid-term evaluation .......................................................................................................................... 11 Table 2: State and county surveillance officers and health facility staff receiving IDSR in-service training as a percentage of those who were interviewed by the evaluation team ............................. 18 Table 3: Distribution of weekly IDSR reporting booklets, by state and as a percentage of the number of functional health facilities. Source: WHO national log for distribution of IDSR materials .......................................................................................................................... 20 Table 4: Availability of basic IDSR materials by sites visited by the evaluation team ........................................... 20 Table 5: Availability of IDSR weekly reporting booklets at sites visited by the evaluation team, by type of health facility .......................................................................................................................... 21 Table 6: Percentage of health workers at health facilities visited by the evaluation team who reported that they had received IDSR in-service training, by state .................................................... 23 Table 7: Percentage of health workers at health facilities visited by the evaluation team who reported that they had received IDSR in-service training, by type of health facility ....................... 23 Table 8: Number and percentage of health workers by IDSR knowledge score ................................................... 24 Table 9: Percentage of health workers scoring 8 or higher, by type of health facility. .......................................... 24 Table 10: Percentage of health workers scoring 8 or higher, by cadre of worker .................................................. 24 Table 11: Percentage of health facilities reporting at least 50% of weeks during 2011, by state ......................... 28 Table 12: Completeness of IDSR weekly reports for August observed at health facilities, by county visited by the evaluation team .................................................................................................................. 29 Table 13: Uses of equipment distributed by the IDSR program, as reported by State Surveillance Officers .......................................................................................................................... 32 Table 14: Uses of equipment distributed by the IDSR program, as reported by County Surveillance Officers .......................................................................................................................... 32 Table 15: Types of analysis observed at state, county and facility level ................................................................... 40 LIST OF FIGURES Figure 1: Population by county of South Sudan, from the census of 2008 ........ Error! Bookmark not defined.