EVALUATION Midterm Evaluation of the Sanitation Service Delivery (SSD) Project Final Report August 2017 This document was prepared for review by the United States Agency for International Development (USAID). It was prepared by the Analytical Support Services and Evaluations for Sustainable Systems (ASSESS) activity, a partnership of the United States Department of Agriculture/Foreign Agriculture Service (USDA/FAS), the University of Rhode Island (URI), Delaware State University (DSU), and Kwame Nkrumah University of Science and Technology (KNUST) August - 2017 Disclaimer: The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Cooperative Agreement #: AID -624-14-P-000004 USAID Cognizant Technical Office: West Africa Environment Team Cover Page Photos Credit: ASSESS, April 2017 1 TABLE OF CONTENTS Table of Contents .................................................................................................................................. 2 List of Figures ........................................................................................................................................ 4 List of Tables ......................................................................................................................................... 5 Acronyms ............................................................................................................................................... 6 Acknowledgment ................................................................................................................................... 8 Executive Summary .............................................................................................................................. 9 1. Introduction and Background ....................................................................................................... 14 1.1. Background ................................................................................................................................ 14 1.2. SSD Project Objectives and Results .......................................................................................... 14 2. Evaluation Scope, Questions, and Methods .................................................................................. 16 2.1. Evaluation Scope ....................................................................................................................... 16 2.2. Evaluation Questions ................................................................................................................. 16 2.3. Evaluation Approach and Methods ............................................................................................ 18 2.3.1. Methods for Data Collection and Analysis ......................................................................... 18 2.3.2. Evaluation Limitations ........................................................................................................ 20 3. Findings and recommendations ..................................................................................................... 22 3.1. Business Models Developed ...................................................................................................... 22 3.1.1. Definition of BM ................................................................................................................. 22 3.1.2. Business Models Developed in Ghana ................................................................................ 22 3.1.3. Business Models Developed in Côte d’Ivoire ..................................................................... 24 3.1.4. Business Model Developed in Benin .................................................................................. 25 3.1.5. Summary of Business Model Status in April 2017 ............................................................. 26 3.2. Relevance and scalability of BM ............................................................................................... 27 3.2.1 How to assess the success of a BM? ............................................................................. 27 3.2.2. Key contextual factors impacting the success of business models ..................................... 30 3.3. Effectiveness and Relevance of the Approach ........................................................................... 58 3.3.1. Identification and preparation process ................................................................................ 58 3.3.2. Decision-making process .................................................................................................... 61 3.3.3. Model development and testing process ............................................................................. 63 3.3.4. Relevance and scalability of the lean startup method ......................................................... 64 3.4. Effectiveness of the Project’s Knowledge, Information and Learning Strategy ........................ 65 3.4.1. Limitations of the methods in Knowledge and Information Sharing .................................. 65 3.4.2. Effectiveness of SSD’s knowledge sharing plan and impact on sanitation market ............ 66 4. Conclusion ....................................................................................................................................... 70 Annexes ................................................................................................................................................ 71 2 Annex I: Synthesis Table of Recommendations ............................................................................... 71 Annex II: List of Documents Reviewed ........................................................................................... 78 Annex III: Evaluation Statement of Work ........................................................................................ 84 Annex IV: Team Composition .......................................................................................................... 92 Annex V: Evaluation Matrix ............................................................................................................. 93 Annex VI: Evaluation Field Schedule ............................................................................................ 109 Annex VII: List of Interviewees and Participants to the FGDs ...................................................... 113 Annex VIII: Data Collection Instruments ....................................................................................... 120 Annex VIII-A: Key Informant and FGD Guide .......................................................................... 120 Annex VIII-C: Interview Guide – Consumers ............................................................................ 121 Annex VIII-D: Interview Guide – Entrepreneurs ....................................................................... 126 Annex VIII-E: Interview Guide – Local Government ................................................................ 128 Annex VIII-F: Interview Guide – Microfinance Institutions ...................................................... 131 Annex VIII-G: Interview Guide – Implementing Partners ......................................................... 134 Annex VIII-H: Project System Review Checklist ...................................................................... 137 Annex IX: Vacuum Truck Operator (VTO) Business Models Analysis Matrix ............................. 138 Annex X: Example of methodology for first 2 steps of the process ............................................... 145 Annex XI: Disclosure of Any Conflicts of Interest ........................................................................ 146 3 LIST OF FIGURES 4 LIST OF TABLES Table 1. SSD Project’s Objectives and Results .................................................................................... 15 Table 2. ASSESS Understanding of the Evaluation Questions ............................................................ 17 Table 3. Samples of the evaluation ....................................................................................................... 19 Table 4. Status of BM developed - March 2017 ................................................................................... 26 Table 5. Comparison between the targets of SSD and other projects ................................................... 28 Table 6. Key contextual factors and key business model orientations.................................................. 40 Table 7. Summary of loans with SSD support ...................................................................................... 56 Table 8. Potential business models identified ....................................................................................... 59 Table 9. Stage of development of the scalable models ......................................................................... 60 5 ACRONYMS ABMS : Association Béninoise pour le Marketing Social (PSI Bénin) AfWA : African Water Association ASSESS : Analytical Support Services and Evaluations for Sustainable Systems BM : Business Model CLTS Community-Led Total Sanitation DNSP : National Direction of Public Health (Benin) DST : Direction des Services Techniques ECOWAS : Economic Community of West African States EHO : Environmental Health Officers (Ghana) EHSD : Environmental Health and Sanitation Department FGD : Focus Group Discussion FSM : Fecal Sludge Management