PHOTO CREDIT: MAMY TIANA RAKOTOARIMANANA EVALUATION REPORT MADAGASCAR RURAL ACCESS TO NEW OPPORTUNITIES FOR HEALTH AND PROSPERITY (RANO-HP) EX-POST EVALUATION WASH Ex-Post Evaluation Series – Water Communications and Knowledge Management (CKM) Project June 2017 This document was produced for review by USAID. It was prepared by ECODIT LLC and Social Impact Inc. Under the Water CKM Project, IDIQ NO. AID-OAA-I-14-00069; TASK ORDER NO. AID-OAA-TO-15-00046. AUTHORITY Prepared for USAID under the Water and Development Indefinite Delivery Indefinite Quantity Contract No. AID-OAA-I-14-00069, Task Order No. AID-OAA-TO-15-00046, awarded September 17, 2015, entitled “Water Communications and Knowledge Management (CKM) Project.” This final report is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents of this report are the sole responsibility of ECODIT LLC and do not necessarily reflect the views of USAID or the United States Government. This report was prepared by the Water CKM project team, comprised of ECODIT LLC and Social Impact Inc. I | E3/WATER CKM PROJECT – RANO-HP EVALUATION USAID.GOV ACKNOWLEDGMENTS We would like to thank the implementing partners Catholic Relief Services (CRS), CARE, Caritas, and Voahary Salama for their availability for questions throughout the planning and execution of this evaluation. We would also like to thank Jonathan Annis and Avo Ratoarijaona of CARE and Chris Seremet of CRS, in particular, for answering our countless requests for documentation and for their consistent high-quality feedback. We are grateful to Joshua Poole of CRS for keeping his door open to us throughout the evaluation process. We are appreciative of the Malagasy Raitra Consulting Company for providing us with the endline data set from the 2013 evaluation they conducted of RANO-HP. We are also grateful to Agence Capsule for conducting the 2016 household survey and open defecation free verification and to our qualitative research team for their high-quality work and for all their efforts in reaching remote locations to interview beneficiaries of RANO-HP. The local qualitative team included Jean Eugene Injerona, Mamy Tiana Rakotoarimanana, Jocelyn Rakotonirina, and Neila Andrianaina. We would also like to thank Amédé Abdérémane Rafidimanatsoa for teaching Capsule enumerators the RANO-HP open defecation free verification process. Finally, we would like to thank everyone who took the time to speak with us about RANO-HP. USAID.GOV E3/WATER CKM PROJECT – RANO-HP EVALUATION | II TABLE OF CONTENTS Authority ......................................................................................................................................... i Acknowledgments ......................................................................................................................... ii Acronyms ....................................................................................................................................... v Executive Summary ..................................................................................................................... vi Background and Methods ........................................................................................................................................... vi Findings .......................................................................................................................................................................... vii Conclusions ................................................................................................................................................................. viii Recommendations ........................................................................................................................................................ x Introduction ................................................................................................................................... 1 Brief Overview of activity and Budget ..................................................................................................................... 1 Evaluation Questions .................................................................................................................... 6 Methodology .................................................................................................................................. 7 Overview ......................................................................................................................................................................... 7 Quantitative Methods .................................................................................................................................................. 7 Qualitative Methods ..................................................................................................................................................... 8 Limitations..................................................................................................................................................................... 10 Findings ........................................................................................................................................ 12 Evaluation Question 1: To What Extent Are the Levels of Sanitation Facility Functionality and Hygiene Usage/Behaviors That Were Measured at the Close of the RANO-HP Activity Still Observed Three Years Later? .................................................................................................................................................................. 12 Evaluation Question 2: Which Factors Influenced the Ability to Sustain Sanitation and Hygiene Facilities and Behaviors? Why? ................................................................................................................................. 18 Conclusions .................................................................................................................................. 23 Evaluation Question 1: To What Extent Are the Levels of Sanitation Facility Functionality and Hygiene Usage/Behaviors That Were Measured at the Close of the RANO-HP Activity Still Observed Three Years Later? .................................................................................................................................................................. 23 Evaluation Question 2: Which Factors Influenced the Ability to Sustain Sanitation and Hygiene Facilities and Behaviors? Why? ................................................................................................................................. 24 Recommendations ...................................................................................................................... 27 III | E3/WATER CKM PROJECT – RANO-HP EVALUATION USAID.GOV FIGURES Figure 1. Three-year sustainability of key RANO-HP sanitation and hygiene outcomes (HH survey) ..... vii Figure 2. RANO-HP Implementers, Activity Locations, and Locations of Evaluation Data Collection …...2 Figure 3. Map of RANO-HP intervention areas ......................................................................................................... 3 Figure 4. Three-year sustainability of latrine use outcomes (HH survey) ......................................................... 12 Figure 5. Three-year latrine use (private or shared) sustainability, by region (HH survey) .......................... 13 Figure 6. A monoblock with attendant in Ilaka Est, Atsinanana Region. ............................................................ 14 Figure 7. Three-year sustainability of handwashing outcomes (HH survey) ..................................................... 15 Figure 8. Three-year sustainability of handwashing station, by region (HH survey) ....................................... 16 Figure 9. Three-year sustainability of additional hygiene and water outcomes (HH survey) ........................ 16 Figure 10. A latrine with a washable slab and tippy tap in Anosy Region .......................................................... 19 USAID.GOV E3/WATER CKM PROJECT – RANO-HP EVALUATION | IV ACRONYMS BCC Behavior Change Communication CHW Community Health Worker CKM Communications and Knowledge Management CLTS Community-Led Total Sanitation CRS Catholic Relief Services CWSBP Commune Water and Sanitation Business Plan E3 USAID Bureau for Economic Growth, Education and Environment E3/W USAID/E3 Water Office FAA Fonds d’Appui Pour l’Assainissement GSF Global Sanitation Fund FGD Focus Group Discussion GOM Government of Madagascar HH Household IDIQ Indefinite Delivery/Indefinite Quantity KII Key Informant Interview MFI Microfinance Institution NGO Nongovernmental Organization ODF Open Defecation Free PPP Public-Private Partnership RANO-HP Rural Access to New Opportunities for Health and Prosperity SIT Sustainability Index Tool SO Strategic Objective UNICEF United Nations Children's Fund USAID US Agency for International Development VSLA Village Savings and Loan Association WADI Water and Development Indefinite Delivery Indefinite Quantity Contract WASH Water, Sanitation, and Hygiene WHO World Health Organization WSSCC Water Supply and Sanitation Collaborative Council V | E3/WATER CKM PROJECT – RANO-HP EVALUATION USAID.GOV EXECUTIVE SUMMARY BACKGROUND AND METHODS In the first of a series of independent ex-post evaluations of USAID–funded water, sanitation, and hygiene (WASH) activities, this ex-post performance evaluation