End Term Evaluation of Reducing poverty and gender inequality in Sarepul province of Afghanistan by improving household income and financial resilience for rural women through enterprise and job creation Final Report Hand in Hand Afghanistan Submitted by Assess, Transform & Reach Consulting (ATR) June 2018 End Term Evaluation – Hand in Hand Afghanistan TABLE OF CONTENTS TABLE OF CONTENTS _____________________________________________________________ 2 ACRONYMS ____________________________________________________________________ 5 EXECUTIVE SUMMARY ___________________________________________________________ 6 PART 1 - INTRODUCTION _________________________________________________________ 11 A. Purpose of the evaluation _________________________________________________________ 11 B. Organisation context _____________________________________________________________ 11 C. Logic and assumption of the evaluation ______________________________________________ 11 D. Overview of UK Aid Direct funded activities __________________________________________ 11 PART 2 – EVALUATION METHODOLOGY _____________________________________________ 17 A. Evaluation Plan _________________________________________________________________ 17 1. Quantitative data collection _______________________________________________________________ 17 2. Qualitative data collection ________________________________________________________________ 18 E. Strengths and Weaknesses of the Designed Methodology _______________________________ 18 F. Summary of problems and issues encountered ________________________________________ 18 PART 3 - FINDINGS ______________________________________________________________ 20 A. Overall Results __________________________________________________________________ 20 B. Assessment of accuracy of reported results ___________________________________________ 20 C. Relevance ______________________________________________________________________ 20 1. To what extent did the grantee support achievement towards the SDGs? __________________________ 20 2. To what extent did the project target and reach the poor and marginalised? ________________________ 22 3. How well did the project respond to the needs of target beneficiaries, including how these needs evolved over time? __________________________________________________________________________________ 23 4. How were the beneficiaries involved (what mechanisms were used), at which stages and what worked well/can be improved? What other actors were involved and when? __________________________________ 24 D. Effectiveness ___________________________________________________________________ 26 1. To what extent are the results that are reported a fair and accurate record of achievement? Under this activity we expect the evaluation team to validate and triangulate the results that we have reported in our logframe. ___________________________________________________________________________________ 26 2. What has happened because of DFID funding that wouldn’t have otherwise happened? ______________ 28 3. To what extent has the project used learning to improve delivery? ________________________________ 28 4. What are the key drivers and barriers affecting the delivery of results for the project? ________________ 29 E. Efficiency ______________________________________________________________________ 31 1. To what extent did the project understand cost drivers and manage these in relation to performance requirements? ______________________________________________________________________________ 31 2. To what extent did the programme deliver results on time and on budget against agreed plans? _______ 32 3. To what extent has the overall programme implemented Value for Money principles in its approach? Specifically related to: ________________________________________________________________________ 33 4. Toolkits: Has the programme delivered VFM through the procurement and through the use of the toolkits? How effective were the toolkits? ________________________________________________________________ 33 5. Staff costs: Has the programme had a sufficient staff cohort to deliver its objectives? ________________ 34 2 End Term Evaluation – Hand in Hand Afghanistan F. Sustainability ___________________________________________________________________ 35 1. To what extent is there evidence that the benefits delivered by the project will be sustained after the project ends? _______________________________________________________________________________ 35 2. Has the social structure of 1) the Self Help Groups 2) Associations & 3) the enterprises created, promoted female and male cohesion and collaboration within these groups? ____________________________________ 36 3. What are the key factors that ensure the functionality and sustainability of 1) the Self Help Groups and 2) Associations e.g. leadership, location, support etc. & 3) the enterprises created?_________________________ 37 4. To what extent has the project leveraged additional resources from other sources? What effect did this have on the scale, delivery and sustainability of the activities? ________________________________________ 39 G. Impact _________________________________________________________________________ 39 1. To what extent and how has the project affected people in ways that were not originally intended? ____ 39 2. To what extent and how has the project build the capacity of civil society? _________________________ 40 3. How many people received support from the project that otherwise would have not received support? _ 40 4. To what extent has our Theory of Change been demonstrated as set out above and in our logframe? ___ 40 5. To what extent did the project mainstream gender equality in the design and delivery of activities (and or other relevant excluded groups)? _______________________________________________________________ 42 6. Has the programme had any impact on household and community gender dynamics, intended or unintended? Specifically looking at changes to: ____________________________________________________ 44 a. Men's attitudes towards women ___________________________________________________________ 44 b. Decision-making, regarding income spent and other key decisions ________________________________ 44 c. Female participant mobility _______________________________________________________________ 45 d. Female participant self-image and confidence ________________________________________________ 46 e. Any negative unintended impacts on women or girls ___________________________________________ 46 PART 4 – CONCLUSIONS AND RECOMMENDATIONS ___________________________________ 48 A. Conclusions ____________________________________________________________________ 48 1. Summary of achievements against evaluation questions ________________________________________ 48 2. Summary of achievements against rationale for UK Aid Direct funding _____________________________ 49 3. Overall impact and value for money of UK Aid Direct funded activities _____________________________ 49 B. Lessons learnt___________________________________________________________________ 49 1. Training _______________________________________________________________________________ 49 2. Gender ________________________________________________________________________________ 50 3. Partnerships ____________________________________________________________________________ 51 C. Recommendations _______________________________________________________________ 51 ANNEXES _____________________________________________________________________ 53 A. Summary of findings according to OECD-DAC criteria ___________________________________ 53 B. Project coverage area ____________________________________________________________ 57 C. Methodology ___________________________________________________________________ 58 1. Primary and secondary data review _________________________________________________________ 58 2. Quantitative data collection _______________________________________________________________ 59 3. Qualitative data collection ________________________________________________________________ 59 4. Approach to gender ______________________________________________________________________ 59 5. Evaluation questions and data collection plan _________________________________________________ 61 6. Logic and assumptions of the evaluation _____________________________________________________ 65 7. Evaluation team _________________________________________________________________________ 65 8. Training of field staff _____________________________________________________________________ 65 9. Data collection __________________________________________________________________________ 66 3 End Term Evaluation – Hand in Hand Afghanistan 10. Field staff work plan ___________________________________________________________________ 66 11. Analysis of quantitative and qualitative data ________________________________________________ 66 12. Quality Assurance _____________________________________________________________________ 66 13. Confidentiality ________________________________________________________________________ 67 14. Ethical considerations __________________________________________________________________ 67 E. Research Tools __________________________________________________________________ 69 1. Tool 1: Survey __________________________________________________________________________ 69 2. Tool 2: FGD with Beneficiaries _____________________________________________________________