Evaluation of the First Five Years of GAVI Immunization Services Support Funding
Total Page:16
File Type:pdf, Size:1020Kb
Cambridge, MA Lexington, MA Evaluation of the Hadley, MA First Five Years of Bethesda, MD Chicago, IL GAVI Immunization Services Support Funding Prepared for The GAVI Alliance Grace Chee Abt Associates Inc. Natasha Hsi Abt Associates Inc. Kenneth Carlson Abt Associates Inc. Slavea Chankova Abt Associates Inc. Patricia Taylor John Snow, Inc. September 2007 Evaluation of the First Five Years’ of GAVI Immunization Services Support Funding is a publication of Abt Associates, commissioned by the GAVI Alliance. The opinions expressed herein are those of the authors and do not necessarily reflect the views of Abt Associates, the GAVI Alliance or any of its alliance partners. September 2007 Recommended Citation Grace Chee, Natasha Hsi, Kenneth Carlson, Slavea Chankova, Patricia Taylor. September 2007. Evaluation of the First Five Years’ of GAVI Immunization Services Support Funding. Bethesda, MD: Abt Associates Inc. CONTENTS Acronyms....................................................................................................................................... v Acknowledgements...................................................................................................................... vii Executive Summary ...................................................................................................................... ix 1. Background to the Study........................................................................................................ 1 2. Approach and Methods .......................................................................................................... 5 2.1 Quantitative Analysis ......................................................................................................5 2.1.1 Quantitative Data Collection .................................................................................. 5 2.1.2 Data Sources........................................................................................................... 5 2.1.3 Data Cleaning Process ............................................................................................ 7 2.1.4 Data Limitations ..................................................................................................... 7 2.1.5 Modeling Methods.................................................................................................. 8 2.1.6 Indicator of Immunization Performance............................................................... 10 2.2 In-depth Country Studies...............................................................................................11 2.2.1 Country Selection ................................................................................................. 11 2.2.2 Data Collection..................................................................................................... 12 2.2.3 Data Sources......................................................................................................... 13 2.2.4 Data Limitations ................................................................................................... 14 3. Findings................................................................................................................................ 17 3.1 Experience of the ISS Scheme ......................................................................................17 3.1.1 Expenditures of GAVI ISS Funds ........................................................................ 17 3.1.2 Application Processing and Funding Transfers.................................................... 20 3.1.3 Programming and Management of ISS Funds...................................................... 20 3.1.4 Experience with Countries Under Stress .............................................................. 23 3.1.5 Integration of ISS Scheme with Overall Health System Planning and Budgeting24 3.2 Impact of ISS Funds on Routine Immunization Financing ...........................................25 3.2.1 Use of ISS Funds to Fill Gaps .............................................................................. 25 3.2.2 Significance of ISS Funds Relative to Total Financing for Routine Immunization .......................................................................................... 27 3.2.3 Changes in Government and Other Donor Funding ............................................. 27 3.2.4 ISS Displacement of Other Sources of Funding................................................... 28 3.3 Impact of ISS Funds on Performance of Immunization Programs................................31 3.3.1 Impact on DTP3 Coverage ................................................................................... 31 3.3.2 DTP3 vs. Measles Coverage Rates....................................................................... 34 3.3.3 Impact on Geographic Equity of Coverage .......................................................... 34 3.3.4 Impact on Consistency of Coverage..................................................................... 36 3.4 Factors Correlated with Differences in Performance ....................................................37 3.4.1 External Factors.................................................................................................... 37 3.4.2 Immunization Program Factors ............................................................................ 38 3.4.3 Management and Planning of ISS funds .............................................................. 41 3.4.4 Use of ISS Funds .................................................................................................. 42 3.5 Impact of ISS Design and Structure on Immunization Performance ............................43 3.5.1 Flexibility in Use of ISS Funds ............................................................................ 43 3.5.2 Reward-based Funding......................................................................................... 44 4. Discussion and Recommendations....................................................................................... 49 Abt Associates Inc. Contents i 4.1 Discussion of Findings ..................................................................................................49 4.1.1 Experience of the ISS Scheme.............................................................................. 49 4.1.2 Impact of ISS Funds on Overall Immunization Financing................................... 50 4.1.3 Impact of ISS Funds on Performance of the Immunization Program................... 50 4.1.4 Factors Correlated with Differences in Performance ........................................... 53 4.1.5 Impact of ISS Design and Structure on Immunization Performance.................... 54 4.2 Recommendations .........................................................................................................55 4.2.1 Policy and Design Recommendations .................................................................. 55 4.2.2 Management Recommendations........................................................................... 58 4.2.3 Monitoring and Evaluation Recommendations .................................................... 59 Annex A: Executive Summary from Evaluation of GAVI Immunization Services Support Funding (Chee et al, August 2004).................................................................... 61 Annex B: Terms of Reference in GAVI Issued RFP ................................................................... 71 Annex C: WHO/UNICEF Estimates and Official Country Estimates of DTP3 Coverage Rates............................................................................................................... 75 Annex D: Key Informants in In-depth Study Countries .............................................................. 77 Annex E: Functions of ICCs and Technical Teams in In-depth Study Countries........................ 83 Annex F: Countries Receiving Rewards, LICUS and non-LICUS Countries ............................. 85 Annex G: Integration of ISS within NIP and Health System Planning........................................ 87 Annex H: Difference between Observed and Model Estimates of DTP3 Coverage Rate, Residuals by Country...................................................................................................... 89 Annex I: Regression Results with Logistic Transformation of WHO/UNICEF Estimates of DTP3 Coverage as Dependent Variable..................................................... 93 Annex J: Regression Results with Various Alternative Model Specifications ............................ 95 Annex K: Country Reward Status, sorted by DTP3 Coverage Rate.......................................... 101 Annex L: References.................................................................................................................. 103 List of Tables Table 1: Summary of Key Indicators and Data Source ..................................................................6 Table 2: Comparative Indicators for In-depth Study Countries....................................................11 Table 3: Illustrative List of In-country Informants.......................................................................13 Table 4: Expenditure Rate of ISS Funds (US$)............................................................................20 Table 5: Summary of Management Procedures for ISS Funds.....................................................21 Table 6: Indicators of ISS Experience for LICUS and non-LICUS Countries.............................24 Table 7: Total Non-vaccine