Decentralized Evaluation
Total Page:16
File Type:pdf, Size:1020Kb
based decision making decision based - d evaluation for evidence d evaluation Decentralize Decentralized Evaluation Evaluation of the Intervention for the Treatment of Moderate Acute Malnutrition in Ngozi, Kirundo, Cankuzo and Rutana 2016–2019 Prepared EvaluationFinal Report, 22 Report May 2020 WFP Burundi Evaluation Manager: Gabrielle Tremblay i | P a g e Prepared by Eric Kouam, Team Leader Aziz Goza, Quantitative Research Expert ACKNOWLEDGEMENTS The evaluation team would like to thank Gabrielle Tremblay for facilitating the evaluation process, particularly the inception and data collection mission to Burundi. The team would also like to thank Patricia Papinutti, Michael Ohiarlaithe, Séverine Giroud, Gaston Nkeshimana, Jean Baptiste Niyongabo, Barihuta Leonidas, the entire nutrition team and other departments of the World Food Programme (WFP) country office in Bujumbura and the provinces of Cankuzo, Kirundo, Ngozi, Rutana and Gitega for their precious time, the documents, the data and the information made available to facilitate the development of this report. The evaluation team would also like to thank the government authorities, United Nations (UN) agencies, non-governmental organizations and donors, as well as the health officials and workers, Mentor Mothers, pregnant and breastfeeding women, and parents of children under five who agreed to meet with us. Our gratitude also goes to the evaluation reference group and the evaluation committee for the relevant comments that helped improve the quality of this report, which we hope will be useful in guiding the next planning cycles of the MAM treatment program in Burundi. DISCLAIMER The views expressed in this report are those of the evaluation team and do not necessarily reflect those of the WFP. Responsibility for the views expressed in this report rests solely with the authors. Publication of this document does not imply that WFP endorses the views expressed. The designations employed and the presentation of material in the maps do not imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory, or maritime area, or concerning the delimitation of its frontiers or boundaries. Evaluation Report: Moderate Acute Malnutrition Treatment Program, WFP Burundi – May 2020 | P a g e ii Table of contents Executive Summary ............................................................................................................................... 6 1. Introduction ................................................................................................................................... 1 1.1 Evaluation Overview ............................................................................................................... 1 1.2 Background ............................................................................................................................. 2 1.3 Evaluation Methodology and Limitations .............................................................................. 4 2. Results of the Evaluation ................................................................................................................ 9 2.1 Evaluation Criterion 1 – Relevance ......................................................................................... 9 2.2 Evaluation Criterion 2 – Coherence ...................................................................................... 12 2.3 Evaluation Criterion 3 – Effectiveness ................................................................................. 13 2.4 Evaluation Criterion 4 – Efficiency ....................................................................................... 22 2.5 Evaluation criterion 5 – Impact ............................................................................................ 28 2.6 Evaluation Criterion 6 – Sustainability ................................................................................ 32 3. Conclusions and Recommendations ............................................................................................ 36 3.1 General Review/Conclusions ................................................................................................ 36 3.2 Lessons Learned and Good Practices .................................................................................... 37 3.3 Recommendations ................................................................................................................ 38 APPENDICES ...................................................................................................................................... 44 Appendix 1. Terms of Reference .................................................................................................... 45 Appendix 2. Map ............................................................................................................................ 71 Appendix 3. Evaluation Matrix ...................................................................................................... 72 Appendix 4. Theory of Change Designed to Guide the Evaluation of MAM Treatment in the Provinces of Cankuzo, Kirundo, Ngozi, and Rutana in Burundi ......................................................... 83 Appendix 5. Documents Gathered ................................................................................................. 84 Appendix 6. Key Stakeholder Interviews ....................................................................................... 89 Appendix 7. Health Centers Selected for Data Collection .............................................................. 92 Appendix 8. Data Collection Tools ................................................................................................. 93 Appendix 9. Additional Results on Effectiveness .......................................................................... 113 Appendix 10. Additional Results on Efficiency .............................................................................. 124 Appendix 11. Bibliography ............................................................................................................. 126 LIST OF ACRONYMS .......................................................................................................................... 129 Tables Table 1. Annual grant from the Food for Peace Fund (FFP), 2016–2019 .......................................... 2 Table 2. Limitations of the evaluation with the measures taken to mitigate them ............................ 8 Table 3. Mapping of health policies/plans in Burundi and their links with nutrition ....................... 9 Evaluation Report: Moderate Acute Malnutrition Treatment Program, WFP Burundi – May 2020 | P a g e iii Table 4. Average quantities of PlumpySup (kg) distributed per child under five years of age, 2016– 2019 ...................................................................................................................................................... 16 Table 5. Average quantity of CSB++ distributed, by PLW, 2016–2019 ........................................... 17 Table 6. Difference between the actual versus planned disbursement rate during the implementation period of the 2016–2019 program (%) ..................................................................... 24 Table 7. Expenditure by category for the MAM treatment program, 2016–2019 (USD)................. 25 Table 8. Cost per beneficiary (children < 5 years or PLW) admitted/treated and per beneficiary cured (USD) ......................................................................................................................................... 26 Table 9. Overview of the level of integration of MAM treatment in each pillar of the health system .............................................................................................................................................................. 34 Table 10. Health centers selected for secondary quantitative data collection in the provinces of Ngozi and Rutana ................................................................................................................................ 92 Table 11. Health centers selected for qualitative primary data collection (purposive sampling) ...... 92 Table 12. Performance indicator collection sheet ............................................................................... 93 Table 13. Admission data collection sheet: ITP, OTP, SFP ................................................................ 94 Table 14. Data collection sheet: Mid-upper arm circumference (MUAC) at admission in OTP and SFP ....................................................................................................................................................... 95 Table 15. Data collection from MUAC at discharge (non-compliance rates) in OTP and SFP .......... 96 Table 16. Data collection sheet: OTP and SFP time to discharge ....................................................... 97 Table 17. Data collection sheet: OTP and SFP reference type ............................................................ 97 Table 18. Data collection sheet: ITP, OTP, SFP defaulter over time .................................................. 98 Table 19. Data collection sheet: Time to default before OTP and SFP ............................................... 98 Table 20. Chi2 test to compare data from databases and registration books .................................... 119 Table 21. Total actual expenditure versus planned expenditure in