OXFAM External Evaluation Report MANAGING RISKS IN SMALLHOLDER AGRICULTURE THE IMPACTS OF R4 ON LIVELIHOODS IN TIGRAY, ETHIOPIA FROM 2012 TO 2016 Malgosia Madajewicz The Center for Climate Systems Research Columbia University Asmelash Haile Tsegay Freelance Consultant Addis Ababa, Ethiopia Ruiwen Lee Columbia University September 26, 2017 As part of our commitment to accountability and learning, Oxfam will share conclusions and recommendations from evaluations. Internally we will share with relevant stakeholders, ensuring that they have an opportunity to participate in discussion of those results in meaningful ways. We will also publish the evaluation reports on our website in accessible language. As a rights-based organization, accountability, particularly to the communities we seek to serve, is of the highest importance to us. For Oxfam, accountability requires Oxfam to regularly and honestly assess the quality of its work, share and learn from its findings with primary stakeholders, and apply that learning in future work. This is an independent impact evaluation of Oxfam America’s R4, Rural resilience program. The program has been operating in in Ethiopia since 2009; it was expanded to Senegal in 2013 and more recently, in 2015, to Zambia and Malawi. This evaluation covers the work undertaken between the years 2015 and 2016.It also includes regression analysis, for some variables, for the periods 2009 – 2012 – 2015 – 2016. The major evaluation activities took place between March 2016 and September 2017. The evaluation was carried out by Malgosia Madajewicz, from Columbia University, through a competitive process and reflects the findings as reported by them, and it was validated with stakeholders. The evaluation was managed by Julio Espinoza, Monitoring, Evaluation and Learning Advisor from Oxfam America, and commissioned by Sophie Romana, Coordinator of the Community Finance department, Oxfam America. For additional information regarding the evaluation Terms Of Reference, please refer to the report appendices. ii OXFAM External Evaluation Report CONTENTS List of Tables ........................................................................................... iv List of Figures .......................................................................................... iv Acronyms ................................................................................................ vi Executive Summary ................................................................................ 01 The R4 program .................................................................................... 01 Program theory in the context of Tigray .................................................. 01 The purpose of the evaluation ................................................................ 02 Evaluation methodology ......................................................................... 02 Summary of the results ........................................................................... 03 Recommendations.................................................................................. 04 Introduction ............................................................................................ 07 The R4 program Program theory in the context of Tigray The purpose of the evaluation Overview of the results Evaluation Design .................................................................................. 10 Key evaluation questions ........................................................................ 10 Evaluation team ...................................................................................... 11 Evaluation methodology ......................................................................... 11 Limitations .............................................................................................. 16 Context ................................................................................................... 18 The R4 Program ..................................................................................... 20 Program Theory ..................................................................................... 22 Evaluation Findings ................................................................................ 25 Comparison of R4 and control villages Protection for food-insecure households: food security and coping strategies Changes in livelihood security Contribution to impacts and main implementation issues: weather index insurance and disaster risk reduction Progress toward R4 objectives in each district Enabling environment iii Women’s empowerment Conclusions and Recommendations ...................................................... 66 References ............................................................................................. 71 Tables .................................................................................................... 73 Appendix 1: Terms of reference.............................................................. 80 Appendix 2: Theory of change ................................................................ 83 Appendix 3: Performance of R4 components .......................................... 85 Appendix 4: Baseline questionnaire ........................................................ 92 Appendix 5: Focus group discussion and interview guides ................... 131 Appendix 6: Photographs of disaster risk reduction activities ................ 143 LIST OF TABLES Table 1. Sample size by treatment status and gender ............................ 15 Table 2. Comparison of R4 and control villages ...................................... 73 Table 3. Effect of R4 on food security ..................................................... 74 Table 4. Effect of R4 on diversification of income ................................... 75 Table 5. Effect of R4 on accumulation of livestock .................................. 76 Table 6. Effect of R4 on accumulation of savings ................................... 78 Table 7. Effect of R4 on borrowing behavior ........................................... 79 Table A3.1. Savings at RUSACCOs and VESAs .................................... 85 Table A3.2. Amount of loan provided in five districts since 2014............. 86 Table A3.3. Loan repayment plan versus achievement .......................... 86 Table A3.4. Enrollment & payment of cash for the premium in 2016/17...87 Table A3.5. Enrollment of farmers by district. ......................................... 87 iv OXFAM External Evaluation Report LIST OF FIGURES Figure 1 Map of Tigray ........................................................................... 11 Figure 2 Percentage of households who report food shortages .............. 28 Figure 3 Percentage of female-headed households who report food shortages ......................................................................................................... 28 Figure 4 Most commonly reported primary coping strategies .................. 33 Figure 5 Percent of members of female-headed households in Raya Azebo who report non-agricultural income sources ......... 36 Figure 6 Number of big livestock owned by female-headed households . 38 Figure 7 Main sources of loans by gender and by year ........................... 43 Figure 8 Insurance enrollment over time ................................................. 52 LIST OF ABBREVIATIONS AND ACRONYMS DA Development Agent DRR Disaster Risk Reduction DECSI Dedebit Credit and Savings Institution FGD Focus Group Discussion HARITA Horn of Africa Risk Transfer for Adaptation project OA Oxfam America v PSNP Ethiopia’s Productive Safety Net Program REST Relief Society of Tigray RUSACCO Rural Saving and Credit cooperative RWH Roof Rainwater Harvesting Technology WFP World Food Programme ii EXECUTIVE SUMMARY The changing climate is straining agricultural livelihoods around the world, especially for smallholder farmers. Adverse weather events, such as drought, affect crop production directly by damaging crops. They also have indirect effects. First, farmers who lack savings and access to credit sell productive assets during a drought to pay for food. Second, as uncertainty about the crops increases, farmers reduce investments in crop production to maintain cash to cover expenses in case of poor harvests. Both of these behaviors reduce farmers’ ability to generate income in good seasons. A drought can launch a descent into poverty, which is difficult to reverse. The R4 program This evaluation investigates how an innovative, integrated risk management program, the R4 Rural Resilience Initiative (R4), is impacting food and livelihood security among smallholder farmers in the northern state of Tigray in Ethiopia. Oxfam America (OA) and the World Food Program (WFP) launched R4 in 2011, building on the accomplishments of the Horn of Africa Risk Transfer for Adaptation (HARITA) initiative, which OA, the Relief Society of Tigray (REST) and Swiss-Re implemented in Tigray in 2009. R4 has expanded to reach over 28,000 smallholder farmers in Ethiopia and Senegal. The program integrates four risk management strategies: improved management of natural resources and diversification of livelihoods (risk reduction), weather index insurance (risk transfer), microcredit (prudent risk taking), and savings (risk reserves). The critical innovation that HARITA pioneered and R4 is continuing is a partnership between weather index insurance and national safety nets, the Productive Safety Net Program (PSNP) in the case of Ethiopia, which allows farmers to pay for