Health Results Innovation Trust Fund Mid Term Review Final Report
Total Page:16
File Type:pdf, Size:1020Kb
Health Results Innovation Trust Fund Mid Term Review Final Report Prepared for // IOD PARC is the trading name of International Jo Keatinge, Meena Gandhi, Organisation Development Ltd// DFID, Ingvar Olsen, Norad, Monique Vledder, World Bank Omega Court 362 Cemetery Road Date // 4 April 2018 Sheffield S11 8FT By// Emma Henrion, Annalize United Kingdom Struwig, Enrique Wedgwood Young, Madeleine Guay, Tel: +44 (0) 114 267 3620 Christiane Duering www.iodparc.com i Contents Acknowledgements iii Acronyms iv Executive Summary 6 Evaluation results 7 The Health Results Innovation Trust Fund 13 The Global Financing Facility 18 Purpose of the evaluation and methodology 20 Objectives 20 Methodology 20 Limitations to the evaluation 21 Findings 23 Relevance 23 Effectiveness 27 Efficiency 37 Impact 45 Equity 55 Sustainability 58 Review of progress on recommendations from the 2012 HRITF Evaluation 62 Conclusions 67 Recommendations 71 Learning for future RBF programmes 72 Annex 1: Terms of Reference I Annex 2: Evaluation framework and questions I Annex 3: Summary of HRITF country grants VI Annex 4 HRITF Results Framework XIII Annex 5: Global stakeholders interviewed XVI i Annex 6: HRITF Country Pilot Grants Projections, Actuals and Disbursements between 2010 and 2019 for case study countries XVII Annex 7: HRITF Disbursements Summary by Activity Category US$ Million XX Annex 8: Summary of completed impact evaluations XXI Annex 9: Deep Dive case studies summaries XXIII Nigeria Case Study XXIII Zimbabwe Case Study XXVI Cameroon case study XXVII Annex 10: References and documents reviewed XXXV ii Acknowledgements Many thanks are due to many people for their help and contributions to this report. The HRITF (now GFF) Secretariat at the World Bank provided excellent briefings, and patiently sourced documents and data for the evaluation team and introductions to country teams, and also provided comments on draft findings and reports. Dinesh Nair, Petra Vergeer, Augustina Nikolova and Oleg Kucheryavenko were assiduous in their support and provision of information. Norad and DFID both provided perspicacious comments and helpful background information. The three deep dive country teams, Dr Ayodeji Oluwole Odutolu and Dr Ayodeji Gafar Ajiboye (Nigeria), Dr Paul Jacob Robyn and Dr Jean Claude Taptue (Cameroon), and Ronald Mutasa and Chenjerai Sisimayi (Zimbabwe) were generous in their time in organising visits and providing documentation and commentary. Technical Team Leads for the virtual case studies, Tawab Hashemi (Afghanistan), Maud Juquois (Benin and Senegal), Haidara Ousmane Diadie (Burkina Faso), Alain- Desire Karibwami (Burundi), Hadia Samaha (DRC), Anne Margreth Bakilana (Ethiopia), Andrew Sunil Rajkumar (Haiti), Rianna Mohammed (Liberia), Ha Thi Hong Nguyen (Tajikistan), Collins Chansa (Zambia), were equally helpful. In all the country case studies, virtual and deep dive, we also acknowledge the time and contributions from many different country stakeholders, including government officials, regional and local staff, health facility staff and service users, civil society representatives, NGOs and development partners. Members of the PBF Community of Practice also generously volunteered to contribute their opinions and experiences and added a valuable perspective to the evaluation. Thanks are due to Bruno Meessen and Maxime Rouve at the Tropical Institute of Antwerp for facilitating our use of the collectivity.org platform. In addition, many global stakeholders representing development partners, donors, academia, and global health consultants provided insightful inputs and wider perspectives which helped to deepen and contextualise our understanding of the HRITF and Results Based Financing. The evaluation was undertaken by IOD PARC (www.iodparc.com ) in 2017. The team leader is Emma Henrion. The team members are Roger Drew (until May 2017), Annalize Struwig, Enrique Wedgwood Young, Madeleine Guay, and Christiane Duering. Quality assurance was provided by Nick York and Franke Toornstra. For further information, contact [email protected]. iii Acronyms AfDB African Development Bank ANC Ante-Natal Care CAR Central African Republic CCA Caisse Autonome d’Amortissement CHW Community Health Worker CO Country Office CoP Community of Practice CPGs Country Pilot Grants CTN Cellule Technique Nationale DAC Development Assistance Committee DEC Development Economics Group DFID Department for International Development DHIS District Health Information System DLI Disbursement-linked indicators DPO Development Policy Operation DRC Democratic Republic of Congo GAVI Global Vaccine Alliance GFF Global Financing Facility GoC Government of Cameroon GoR Government of Rwanda HCC Health Centre Committees HF Health Facility HMIS Health Management Information System HNPGP Health Nutrition and Population Global Practice HRITF Health Result Innovation Trust Fund IAWG Inter-Agency Working Group IBRD International Bank for Reconstruction and Development IDA International Development Association IE Impact Evaluation IEG Independent Evaluation Group of the World Bank INGO International Non-Governmental Organisation IPF Investment Project Finance K&L Knowledge and Learning LGA Local Government Area (Nigeria) LMICs Low- and Middle-Income Countries MoH Ministry of Health MoPH Ministry of Public Health NGO Non-governmental Organisation iv NSHIP Nigeria State Health Investment Project OECD DAC Organization for Economic Cooperation and Development Development Assistance Committee PAD Project Appraisal Document PBF Performance Based Finance P4R Programme for Results PIU Project Implementation Unit PNC Post Natal Care RBF Result-Based Financing RFHP Regional Funds for Health Promotion RMNCH Reproductive Maternal, Neonatal and Child Health RMNCAH-N Reproductive Maternal, Neonatal, Child and Adolescent Health, and Nutrition TTL Task Team Leader UNDP United Nations Development Programme WB World Bank WHO World Health Organisation v Executive Summary The Health Results Innovation Trust Fund - overview The Health Results Innovation Trust Fund (HRITF) is a World Bank-managed multi-donor trust fund, established in 2007 with support from the Government of Norway, joined by the Department for International Development (DFID), UK, from 2009. It supports low and middle-income countries to design, implement, monitor and evaluate results-based financing (RBF) interventions in the health sector to improve maternal and child health. The Fund supports RBF projects in 28 countries in Africa, South East Asia, Eastern Europe, Central Asia, Latin America and the Caribbean. The HRITF will complete in 2022 and funds are fully committed. As of August 2017, the total value of donor pledges to the HRITF was US$478.2 million. A primary objective of the HRITF programme is to build and share an evidence base from different RBF approaches. To build the evidence base, the Trust Fund has supported a variety of RBF approaches and has been developing a learning portfolio of rigorous impact evaluations and other studies. There is a variety of mechanisms such as a website, global workshops, publications, tools and guidance, to share evidence. When the HRITF was established, its main intended impact was on outputs contributing to maternal and child health outcomes through supporting design, implementation and monitoring and evaluation of RBF mechanisms as well as developing and disseminating the evidence base for implementing successful RBF mechanisms. Over time the HRITF has developed an appreciation of RBF’s contribution to health system strengthening, which is now seen by the Trust Fund as one of the main areas for potential impact. The Global Financing Facility In 2015 the Global Financing Facility in Support of Every Woman Every Child (GFF) was established as a mechanism to support delivery of the Sustainable Development Goals (SDGs) for Reproductive Maternal Neonatal, Child and Adolescent Health, and Nutrition (RMNCAH-N). The GFF has a more strategic focus on financial and health system reform, results, and sustainability and does not prescribe a specific model such as RBF. It is based on a partnership with bilateral and multinational donors, foundations, and the private sector. The GFF had US$762 million equivalent contributions as of October 2017. The GFF provides a successor fund to the HRITF. The HRITF is now working within the framework of the wider GFF. The World Bank Secretariat for the HRITF took on a dual role in 2015 and is now also the Secretariat for the GFF Trust Fund Committee. Evaluation objectives • Objective 1: To assess the performance of the HRITF against the given goals and outputs of the programme (as described in the results framework) identifying strengths, weaknesses and lessons learnt. • Objective 2: To determine what progress has been made in addressing the recommendations from the previous 2012 evaluation. • Objective 3: To make recommendations to inform on-going and future programming specifically aimed at (a) improving the performance of the current HRITF programme from a donor, implementer and country level perspective and (b) supporting the design and implementation of future RBF approaches being considered. Method The review was based on a document and data review; three “deep dive” visits to Cameroon, Nigeria and Zimbabwe (see Annex 9); and 12 virtual case studies (Afghanistan; Benin; Burkina Faso; Burundi; Democratic Republic of Congo; Ethiopia; Gambia; Haiti; Liberia; Senegal; Tajikistan; 6 Zambia). The country visits and case studies included interviews with selected country