Enhancing Integration of Displacement Affected Communities in Somalia Endline Evaluation Report 1 July 2020 Legal Notice and Disclaimer All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means without prior approval in writing from Consilient and Concern Worldwide. This report is not a legally binding document. It is a collaborative informational and assessment document and does not necessarily reflect the views of any of the contributing partners in all of its contents. Any errors are the sole responsibility of the authors. Cover Photo source: Distribution of hygiene kits, Afgoye. Photo taken by Yusuf Mohamed Mahi-Field Officer, SHACDO, 22nd May 2019. Prepared by: Brenton Peterson, Director of Research Manar Zaki, Research Manager Zuzana Machova, Research Officer Copyright © 2020 Consilient and Concern Worldwide ENHANCING INTEGRATION OF AFFECTEDDISPLACEMENTSOMALIA COMMUNITIESININTEGRATION ENHANCING CONTENTS LIST OF ACRONYMS ................................................................................. 6 EXECUTIVE SUMMARY .............................................................................. 7 Process evaluation .......................................................................................... 7 OECD DAC Criteria .......................................................................................... 7 IASC/REDSS Framework analysis .......................................................................... 9 1. INTRODUCTION AND BACKGROUND .......................................................... 11 1.1. Program and geographic context ................................................................. 11 1.2. Overview of program activities ................................................................... 13 2. RESEARCH PURPOSE, SCOPE AND LIMITATIONS ............................................ 15 3. METHODOLOGY ................................................................................. 17 3.1. Analytical frameworks .............................................................................. 17 3.1.1. Process Evaluation 17 3.1.2. OECD DAC Criteria 18 3.1.3. Log frames 19 3.2. Data Collection ...................................................................................... 20 3.2.1. Achieved sample 20 3.2.2. Timeline of Data Collection 21 3.3. Desk review .......................................................................................... 21 3.4. Research ethics ...................................................................................... 22 4. PROCESS EVALUATION ......................................................................... 23 4.1. Program approach and design ..................................................................... 23 4.2. Consortia governance and coordination ......................................................... 24 4.3. Government engagement .......................................................................... 26 4.4. Community Engagement ........................................................................... 29 4.5. External coordination and cooperation .......................................................... 31 4.6. Learning and adaptation ........................................................................... 32 5. OECD DAC ANALYSIS ........................................................................... 35 5.1. Relevance ............................................................................................ 35 5.1.1. To what extent are the objectives of the programme still valid? 35 5.1.2. Are the activities and outputs of the programme consistent with the overall goal, the attainment of its objectives, and their intended impacts and effects ? 37 5.2. Effectiveness ......................................................................................... 39 5.2.1. To what extent were the objectives achieved? 39 EIDACS ENDLINE EVALUATION Page 3 of 121 5.2.2. What were the major factors influencing the achievement or non-achievement of the objectives? 46 5.3. Impact ................................................................................................ 49 5.3.1. What has happened as a result of the programme? 49 5.3.2. What real difference has the activity made to the beneficiaries? 53 5.3.3. How many people have been affected? 54 5.4. Efficiency ............................................................................................. 55 5.4.1 Cost-effectiveness 55 5.4.2 Were objectives achieved on time? 59 5.4.3 Was the programme implemented in the most efficient way compared to alternatives? 60 5.5. Sustainability ........................................................................................ 61 5.5.1. Identification of critical gaps after the end of the contract and recommendations for the Exit Strategy and transition to other DS programmes 61 5.5.2. What are the major factors that can influence the achievement or non-achievement of the sustainability of the programme? 63 6. LOGFRAME ANALYSIS ........................................................................... 64 6.1. Program logframe analysis ......................................................................... 64 6.1.1. Impact indicators 64 6.1.2. Outcome indicators 67 6.1.3. Output indicators 69 6.2. IASC DS Framework Analysis ....................................................................... 70 7. CONCLUSION .................................................................................... 73 7.1 Process evaluation ................................................................................... 73 7.2 OEDC DAC Criteria ................................................................................... 74 7.3 Log frame analysis ................................................................................... 75 8. KEY RECOMMENDATIONS ...................................................................... 77 8.1. Recommendations on key measures of DS that could be included as indicators in future programming .............................................................................................. 77 8.2. Strategic and programmatic recommendations to inform future programming, donors, and other DS consortia and programmes ................................................................... 78 8.3. Identification of critical gaps after the end of the contract and recommendations for the Exit Strategy and transition to other DS programmes ............................................... 79 9. ANNEXES ......................................................................................... 80 Annex 1: COvid-19 prodecured for research .......................................................... 80 Annex 2: Tools and Question Matrix ................................................................... 83 Annex 3: Updated Log Frame Matrix ................................................................... 83 Annex 4: ReDSS Framework Indicators ............................................................... 104 Annex 5: Distributions of indicators .................................................................. 110 Annex 6: Regression analysis results of impact indicators ........................................ 115 Annex 7: Regression Analysis Results of Outcome Indicators ..................................... 116 Annex 8: Regression analysis results of output indicators ......................................... 117 Annex 9: Complementary regression analysis results of IASC indicators ........................ 118 Annex 10: Mapping of challenges to effective achievement of project objectives ........... 119 Annex 11: Number of beneficiaries ................................................................... 121 EIDACS ENDLINE EVALUATION Page 5 of 121 LIST OF ACRONYMS CAP – Community Action Plans CLTS – Community Led Total Sanitation CMU – Consortium Management Unit CSC – Consortium Steering Committee DAC – Displacement Affected Communities DRC - Danish Refugee Council DS - Durable Solutions EIDACS - The Enhancing Integration of Displacement Affected Communities EU – European Union FFS – Farmer Field School FGD – Focus Groups Discussions FGS – Federal Government of Somalia GREDO - Gargaar Relief and Development Organisation HC – Host Community HLP – Housing, Land, Property IDP – Internally Displaced People IGAD – Intergovernmental Authority on Development KII – Key Informants Interviews MCH – Mother and Child Heath MOPIED – Ministry of Planning, Investment and Economic Development NRC - Norwegian Refugee Council OECD DAC – Organisation for Economic Co-operation and Development, Development Assistance Committee ReDSS - Regional Secretariat for Durable Solutions SHACDO – Shabelle Community Development Organization SME – Small and Medium Enterprise SOP – Standard Operating Procedures SPL – Somali Peace League SWS – South West State TWG – Technical Working Group EXECUTIVE SUMMARY The present report is the final output of external evaluation contracted to Consilient and carried out form 27th of April to 1st of July 2020. The purpose of this evaluation is to assess the effectiveness of Enhancing Integration of Displacement Affected Communities in Somalia (EIDACS), compile programme consortium best practices, develop a list of recommendations on key measures of Durable Solutions and future programming, and identify gaps for future programming. The assessment undertakes three distinct analytical frameworks, namely process evaluation, OECD DAC criteria, and