Acknowledgements

This evaluation would not have been possible without the support of a number of individuals and institutions. The University of Notre Dame Pulte Institute research team would like to express their gratitude to the following: USAID Ethiopia for funding the Livelihoods for Resilience-Oromia Activity and midterm evaluation. The staff of Catholic Relief Services who are responsible for implementing the Livelihoods for Resilience-Oromia Activity, principally Ian de la Rosa (Chief of Party) and Maria Alemu (Monitoring Evaluation Accountability and Learning Manager) for funding this evaluation, as well as for their guidance and support during the study preparation, implementation and reporting. The field staff from the Ethiopian Catholic Church Social & Development Commission of Meki for coordinating field interventions and sharing information related to the Activity and its participants. The key informants from the Women, Children and Youth Affairs office, Kebele/Woreda Food Security Task Force Office, Rural Savings and Cooperatives, Kelata Savings and Credit Cooperative Union, Metemamen Microfinance Institution, Adami Tulu Agricultural Research Centre, proprietors of various agro-dealers, and the staff of World Vision for sharing their insights and experiences. Yisak Tafere for recruiting and mobilizing high quality field researchers and completing the data collection so effectively and on time. The many field researchers who collected data from the various target communities and stakeholders. The various communities in Ethiopia for their cooperation. The community members who carved out time to participate in group discussions and interviews deserve a special note of thanks.

1

List of abbreviations and Acronyms Abbreviation Description Activity Feed the Future Ethiopia Livelihoods for Resilience Oromia Activity ATJK Adami Tulu Jido Kombolcha AWP Annual Workplan CA Community Animators CRS Catholic Relief Services CSA Climate-Smart Agriculture DA Development Agent DFSA Development Food Security Activity ETB Ethiopian Birr FA SILC Field Agents FFP Food for Peace FGD Focus Group Discussions FSP Financial Service Providers FSTF Food Security Task Force FY Fiscal Year GoE Government of Ethiopia GRAD Graduation with Resilience to Sustainable Development HH HH IFd Innovation Fund IPTT Indicator Performance Tracking Table ITT Indicator Tracking Table KFC Kebele Feedback Committee KFSTF Kebele Food Security Task Force KII Key Informant Interview LG Livelihood Group MCS Ethiopian Catholic Church Social Development Coordination Office of Meki MEAL Monitoring, Evaluation, Accountability and Learning MGs Marketing Groups MTEWs Marketing and Technical Extension Workers MFI Microfinance Institution MTE Mid Term Evaluation MoU Memorandum of Understanding NDIGD Notre Dame Initiative for Global Development PSNP Productive Safety Net Program Q Quarter RF Results Framework RuSACCOs Rural Savings and Credit Cooperatives

2

SACCOs Saving and Credit Cooperatives SILC Savings and Internal Lending Community SOW Scope of Work TFH/TIH The Faithful House/The Islamic House ToR Terms of reference ToT Training of Trainers TVET Technical and Vocational Education and Training USG United States Government WCYA Women, Children and Youth Affairs WFSTF Woreda Food Security Task Force WV World Vision Y Year YLG Youth Livelihood Group

3

Table of Contents Acknowledgements 1 Executive Summary 5 Midterm Evaluation 15 Background 15 Activity Intervention and Strategy 15 Evaluation Methodology 18 Objectives 18 Evaluation Questions 18 Evaluation Implementation 19 Analysis 21 Limitations of the MTE 21 Main Findings 22 LG Formation 22 Livelihood Pathways 24 Market Systems and Value Chain Development 25 Financial Services 28 Innovation Fund 30 Nutrition 32 Women’s Empowerment 32 Youth Empowerment 34 Collaboration 34 Resilience Capacities of LGs 36 Sustainability and Graduation 38 Financial Management 41 Recommendations 43 Annexes 49

4

Executive Summary

In August 2019, CRS-Ethiopia commissioned the Notre Dame Initiative for Global Development (NDIGD)1 to conduct a Midterm Evaluation (MTE) of its Feed the Future Ethiopia “Livelihoods for Resilience Oromia” Activity (referred to hereafter as “Activity”). This MTE is an important component of the Activity’s Monitoring, Evaluation, Accountability and Learning (MEAL) plan, enabling Catholic Relief Services (CRS) to assess progress from the perspective of its participants and reorient the Activity’s strategies and interventions as necessary to improve their relevance. The evaluation employed qualitative methods—Focus Group Discussions (FGDs), Key Informant Interviews (KIIs), and site and intervention observations—to collect information from participants and key stakeholders. The Pulte Institute and CRS designed this evaluation to be participant informed and prioritized, instructing the research team to collect primary data from program participants and stakeholders. The evaluation reviewed routine monitoring data, partner and donor reports, and other pertinent Activity documents. Researchers from NDIGD visited field sites from August 24-31, 2019, to meet with Activity participants, implementing staff, and other pertinent stakeholders, amassing a total of 24 FGDs and 32 KIIs across six of the nine Activity Woredas. To ensure balance in perspectives and learning, researchers conducted six FGDs for each of the following demographic segments: adult male, adult female, youth male, and youth female. Out of nine Woredas where the Activity implements its program, six Woredas were selected for the primary data collection. The selection criteria for selecting six Woredas include political (administrative) zones considering two Woredas from each zone, remoteness/ accessibility to services (rural and semi-urban), diversity in livelihoods, number of program participants and overlap with other programs as well as donor interest for future interventions. Based on these indicators and more supporting information from CRS, the researchers selected two Woredas from each zone as follows: • West Arsi: Negele Arsi2 and Shala • Arsi: Dodota and Sire • East Shewa: Adami Tulu Jido Kombolcha (ATJK) and Boset

The research team has organized the findings of this MTE under the following thematic areas.

LG Formation Based on the primary data collected and review of Activity documents, the Activity interventions appear to be meeting planned benchmarks regarding formation of Livelihood Groups (LGs). As of Year 3 (Y3) Quarter 3 (Q3), the Activity has reached 29,000 participants3, covering 24,500

1 NDIGD changed its name to Pulte Institute in November 2019 because of an endowment from the Pulte Family Charitable Foundation. 2 At the time of the agreement, the Woreda was called Arsi Negele, but this report reflects the updated and current name of the Woreda. 3 All figures are as of Year 3, Q3.

5 households (HHs) across nine Woredas. These participants are spread across 1,385 LGs. That said, FGDs ascertained significant attrition within LGs, particularly the Youth Livelihood Groups (YLGs). This attrition appears to be tied in part to participant expectations that the Activity would provide more handouts, as have other projects serving this population in the past. Additionally, consistent participation in the groups has struggled amidst members migrating to city areas in search of employment opportunities. Some members complained that they are participating in “training after training”, and that they are having a hard time even recalling the names, content, and applications of all the trainings. Others complained that delays or cancellation of trainings without advanced notice have caused them to lose valuable workdays. This seemed to be an issue more at the start of the Activity, and some participants acknowledged that communication and scheduling of trainings has improved since. Livelihood Pathways Activity interventions include assisting Productive Safety Net Programme (PSNP) HHs to determine livelihood pathways, strengthening capacity around these pathways, facilitating business plan development, bolstering existing support systems, and brokering linkages with value chain actors or financing options. The Activity offers three distinct pathways to support efforts of PSNP HHs to move out of poverty: on-farm, off-farm, and employment. By Y3 Q3, 18,472 members adopted the on-farm pathway and are receiving related capacity strengthening trainings. This represents 64 percent of the total 29,000 participants reached by the Activity. These members are involved in various farming actions, such as vegetable gardening, growing maize with improved technologies, or fattening cattle for market; however, the availability of inputs, including financing for market-based actions, and market fluctuations continue to challenge HHs in this pathway. The off-farm pathway aims to increase the income of vulnerable HHs and diversify their income sources. By Y3 Q3, 2,344 members (8 percent of total Activity participants) participated in off- farm livelihood interventions. Some members indicated that they have already taken up various off-farm value chain actions (e.g. food catering shops, retail shops, animal-drawn carts), made possible through the support they obtained from the Activity. For example, some members have started to prepare business plans and explored financing options from FSPs. The training that the Activity provided proved instrumental in starting their businesses and, more importantly, managing them successfully. Members report being involved in running petty trades or opening shops locally after participating in LGs. Some on-farm Marketing Groups (MGs) were also established, and group leaders were trained on business management skills. Conversely, the employment pathway is an area that would benefit from further investment of time, and human resources, coupled with thoughtful planning, monitoring, and adaptation. This pathway aims to increase the income from gainful employment and focuses on increasing access to different employment opportunities. The knowledge and skills that participants will obtain from the training should increase their employability, which could positively affect their long-term earning

6 potential, however this will need to be monitored more consistently. By Y3 Q3, 196 HHs espoused this pathway as the principal component of their livelihood portfolio. The Activity plans to support 1,470 HHs with this livelihood pathway. At the moment, this pathway is underachieving, and LG members, especially youth, expressed a disconnect between what the Activity had promised and what it is doing. Their biggest concern is that there is a significant waiting period before they are able to receive the vocational skills trainings. In some groups, youths expressed that they have been waiting for more than one year to receive the training. As a result, it was difficult to secure an accurate understanding of the effect of these trainings. They feel that this training is crucial to increasing their employability. Without it, many need to rely on their grade 10 credentials, which are not sufficient to secure reliable wage-based jobs.

Market Systems and Value Chain Development The Activity commissioned various studies during Years 1 and 2 at the community and Woreda levels to understand the prospects of on-farm and off-farm value chain activities in Activity areas. These studies made several recommendations for the uptake and implementation of value chain interventions. One recommended area for value chain development was livestock. Recommendations focused on providing support to private feed dealers, establishing linkages between veterinary services, facilitating the sale of products on the market, improving access to financing for investments into the value chain, and increasing women/youth participation. In line with the livestock recommendation, among others, the Activity provides trainings and forms MGs to catalyze input and output markets. During the FGDs, members mentioned how the training they received helped them learn how to start a business. Some members are starting to participate in livestock or crop-based value chains, but these still have not been widely adopted due to insufficient access to financing, inadequate support systems and lackluster market incentives. The Activity is also supporting agro-dealers as a way to stimulate improved access to farming inputs such as pesticides, herbicides, and fertilizer. This materialized in the form of grants to help agro-dealers upgrade their infrastructure as needed, ideally with the hope that these agro-dealers could be prompted to improve their supply to rural areas. In reality, some agro-dealers expressed reluctance in expanding their businesses to Activity areas. FGDs revealed that farmers might not even be aware of the existence of agro-dealers or the effects of their offerings like seeds and fertilizers because the farmers have been working mostly with farmers’ unions to date. Problems for the agro-dealers include the cost associated with holding inventory and the fact that they usually need to extend products to farmers on credit, which they may or may not make back. Usually, this is done sparingly to those farmers known by the agro-dealers, but it poses a significant risk to these small businesses regardless. As an alternative to agro-dealers, group members suggested working with farmers’ unions, as farmers’ unions are trusted and more stable. Study participants noted that they mostly work with

7 these unions to procure inputs such as seeds and fertilizer. The Activity is currently working with three Woreda-level farmers’ unions to source agriculture inputs given that the Kebele-level unions appear to be weak. Financial Services Members acknowledged that, despite the onset of the Activity, LG members still need to worry about how and where to secure loans when they need emergency funds or business financing. In Activity Woredas, lenders, be it local predatory moneylenders or MFIs, are not interested in extending credit to the poor. To some extent, the establishment of Savings and Internal Lending Communities (SILC) in each LG has empowered members by the fact that they do not need to beg for small loans, particularly from predatory moneylenders. That said, in many cases HHs perceive that the financing that SILC affords is not enough to initiate or scale up a small business, although this has not stopped some participants from accessing credit through SILC and making notable gains. This in turn necessitates that they seek out options for larger loan sizes. The Activity has responded to this issue of members needing larger loans by trying to link them with external FSPs such as private MFIs or government-supported RuSACCOs; however, this has presented significant challenges. High default rates from previous lending experiences are giving these FSPs pause before entering a similar cycle. Additionally, MFIs voiced problems with liquidity in that the demand by participants for financing outweighs the number and capacity of FSPs who are available to lend. To address the default problem, the Activity has initiated a fund that will provide loan guarantees to FSPs. The purpose of these guarantees is to create incentives and reduce the perceived risk associated with lending to PSNP HHs. By Y3 Q3, six FSP (3 MFIs and 3 SACCO Unions with their 22 RuSACCOs) have linked to Activity members and provided guarantees for up to 25% of the total financing request. Many members, however, have not received credit from such arrangements as the MFIs still do not have enough cash in hand, and delays in forming business plans have further inhibited members from securing loans on time. For those who have, loan sizes are still significantly smaller than anticipated. Innovation Fund The Activity promotes innovation to support livelihood pathways (Strategic Objective 4/SO4). Under this objective, the Activity is working with universities, researchers, the private sector and other entities to develop technologies that are best suited for the local context. By Y3 Q3, there were 14 innovative initiatives led by 11 grantees. The Activity (via CRS cost share) provides funds to institutions to develop approaches and technologies that are based on the value-chain assessments. Of the innovation funds observed, some participants are selected for demonstration sites where the members have a chance to witness these innovative technologies or approaches and how they might help them increase their agricultural or livestock production. This intervention is designed to also reap benefits for participating farmers through enhanced productivity. Farmers who have an opportunity to view field activities can adopt these practices on their own lands, which helps scale the overall effect of the Activity. This practice has proved promising for

8 programs in the past, however it was not possible to see an effect to these demonstration sites given that they were established recently. Nutrition For the Activity’s objective to promote optimal health practices (SO1), the Activity intervened in aspects of nutrition that will mostly benefit lactating women and children. Activity interventions focused on promoting nutrition-rich crops, training on homestead gardening, and cooking demonstrations incorporating locally available food items. Nearly half of the total LG members participated in nutrition-sensitive agriculture trainings. During the FGDs, members shared how they have engaged in gardening and producing nutritious vegetables like cabbage and lettuce over the past years. To further the uptake of vegetable farming, the Activity delivered dietary diversity messages during SILC meetings, functional literacy trainings and cooking demonstration sessions. The messaging helped change attitudes and practices of participants to grow nutritious vegetables. As results, Although participants are growing nutritious vegetables in their field, we suggest that the Activity team continue to monitor the actual consumption habits of these households to assess how well they are utilizing these nutritious foods (as opposed to selling them in the marketplace, for example). Women’s Empowerment The Activity aims to address gender inequality by integrating women more purposefully in its interventions, and its monitoring system effectively tracks female participants separately. Female members are now able to engage in meetings with male members, discuss their own issues, save money, participate in income generating activities, and work on various on-farm and off-farm income pathways. As per the responses from male and female members, in most areas women do not have problems participating in Activity interventions, and both husbands and wives are supposed to own equally the money contributed in savings and SILC share-outs. That said, the evidence collected indicates that this is not happening in all cases. Women in SILCs experience problems showing up on time to meetings because of their HH duties that men do not share. Some women pointed out that greater say is given to men around how money is controlled, which limits the degree to which they benefit from the Activity. In all the group trainings, the participation of women was equal to that of men. Nevertheless, in some groups the women expressed that they were not treated equally or afforded equal voice in their daily group level activities. Additionally, while both males and females have been working on various value chains, the research team found that males are more likely to lead those value chain interventions. Some women also noted the long distances they must travel to participate in meetings, which inhibits the full and active participation. Youth Empowerment The Activity has registered 3,609 members across 146 YLGs, and it estimates that approximately 1.3% of members within the mixed LGs could be classified as Youth. That said, the scattered youth population and migration in some areas has presented a challenge in maintaining functioning LGs. In some Woredas, youth are leaving their groups. The sentiment is that the Activity is not offering

9 much value to youth who do not have money to save and do not qualify for credit from a FSP. Furthermore, delays in implementing vocational trainings in Technical and Vocational Education Training Centers (TVETs) have disheartened youth from continuing. For some, the Activity has created an added burden. In one FGD, a few youth claim to be losing focus on their education, while attempting to engage in market-based interventions and being expected to support their families with domestic needs. Collaboration Inclusion of different stakeholders during the planning, implementation and supervision process improves effective collaboration among the various stakeholders. The Activity was successful in involving government counterparts and line agencies in Activity implementation, as confirmed by interviews with key informants. These include the local offices of Women, Children and Youth Affairs (WCYA), the Woreda/Kebele Food Security Task Force Office (FSTF), and various Development Agents (DAs) who provide extension services within Kebeles. While most of the government officials were happy with Activity’s collaborative efforts, some informants pointed out opportunities for improvement, as they felt the Activity did not provide sufficient details on what interventions they would be implementing, when, and where. Communication with government stakeholders has also been inconsistent, which can affect their timely support of the Activity, although this is more pertinent with new government officials who have changed since the start of the Activity. In Woredas shared with one or more additional U.S. government-funded activities, Livelihoods for Resilience-Oromia has collaborated well with other Activities. This has helped to develop synergies and to strengthen the Activity’s impact. For example, CRS Development Food Security Activity (DFSA) aims to strengthen PSNP capacity and provides food or cash assistance to PSNP HHs. Livelihoods for Resilience-Oromia does not provide this kind of support to members; however, it is working alongside the DFSAs with CRS and WV to determine how best to complement each other. Sustainability and Graduation For the sustainability of development activities, four factors were identified as key: • a sustained source of resources • sustained motivations • sustained technical and managerial capacity • linkages to government organizations and/or other entities

Examining Livelihoods for Resilience Oromia’s interventions in light of these critical factors, many have the potential to endure after the Activity ends. That said, more work needs to be done to ensure this. As mentioned above, currently the Activity struggles to maintain consistent membership within the LGs, linking to FSPs, encouraging HHs to adopt a variety of livelihood options, creating employment pathway options for youth, and fully involving women across the board. The Activity can work with LGs to establish linkages with local and external organizations

10

(government offices, MFIs, etc.), provide greater opportunities for building technical and managerial capacities of participants, encourage participants to save more for sustained sources of financing, and constantly motivate and mentor participants to maintain their groups and trial different livelihood options in a low-risk environment. All group participants and key informants believe that SILC interventions will be sustained. The participants do not know when the project ends, but they are optimistic that they will maintain these and work towards diversifying income sources with the knowledge and skills they receive from the Activity. KII respondents from government offices promised to provide continued support to group members even after the Activity ends. Apart from sustainability, the aim of the Activity is to support the graduation of 18,375 HHs out of 24,500 (75 percent of total HHs) to become self-reliant (as per the PSNP guidelines that define graduation to happen when “HHs achieve food self-sufficiency in the absence of external support''). Through the focus group and KIIs, researchers learned that no discussions are taking place within the groups around graduation. Furthermore, neither participants nor Activity team members are clear on the criteria for graduation. To be eligible to graduate, members need to demonstrate meeting certain criteria to the Government of Ethiopia (GoE), although the criteria themselves are ever changing. Given the time remaining for the Activity, the discussion on graduation should be started within the LGs and the target adjusted to reflect a more realistic number of HHs that can achieve graduation readiness rather than actually graduating. The Activity staff and Kebele officials are well placed to lead this discussion in the groups.

Recommendations The following recommendations have been assembled by the Pulte Institute’s research team to improve the delivery of the Activity in terms of its expected outcomes. These critical priority recommendations incorporate input from Activity participants, in addition to practices that the Pulte team has observed as having had beneficial impacts on similar programs in other parts of East Africa, such as Ethiopia and South Sudan. These should be taken as recommendations that are likely to offer the greatest return on investment (human or financial) based on participant responses. Critical priority recommendations 1. Promote greater diversity in livelihood pathways adopted by individual HHs as a way to increase resilience. The Activity is based on PSNP guidelines for selecting livelihood pathways for safety net HHs. Approximately 88% of HHs have selected on-farm activities as their pathway for livelihoods support; however, increasingly crop-based livelihoods have been challenged by market fluctuations and climate change. Going forward, the Activity should promote greater diversification of revenue sources, encouraging a balance of livestock or other economic activities for HHs to complement their crop-based primary livelihood.

11

2. Better communicate and disseminate trainings to participants and government partners in order to fit individual needs and promote sustainability. Trainings help transfer knowledge and skills, and members are cognizant of this. Nevertheless, instead of providing blanket trainings to all members, learning outcomes could improve if trainings are delivered based on need and on a timely basis. Additional trainings should only be rendered after evaluating the effectiveness of prior trainings and other educational activities. In addition, representatives from the government should be accompanying CRS partner staff on monitoring and technical visits. The Activity could also look at ways to second (i.e., temporarily assign) government staff to the Activity as a way of further embedding the Activity objectives, trainings and resources into government strategies while strengthening the capacity of government staff. 3. Identify and effect mechanisms to reduce risk to private sector collaborators (i.e., agro- dealers) to incentivize more consistent and sustained support. Activity grants to agro- dealers were employed to build up or upgrade existing infrastructure. These could incentivize agro-dealers to expand their businesses to rural areas ultimately if dealers can find a way to direct their upgrades to shape the market better. The Activity could support agro-dealers to better market their services to farmers while also re-sizing their product offerings by making small packages of products, to meet the needs of farmers who may not have the cash flow to afford larger volumes. For example, agro-dealers could provide products in smaller volumes for quarter-acre plot sizes, which may be easier for smallholder farmers to afford, while increasing margins slightly for the dealers. To fully understand the potential for agro-dealers and incentivize them better, the Activity could carry out a market study that outlines the number of participants in each area, what their specific growing issues may be, their willingness to spend on product offerings, and entry or exit barriers affecting them. 4. Better manage expectations of participants around availability and conditions of financing options. Linking members to external financial services is a sustainable solution to meet members’ long-term financial needs, but there is widespread confusion and disagreement among members about this effort. Activity staff should provide more information around group saving requirements, the amount members can expect to access from FSPs, the expected interest rate, and repayment periods. Conversely, FSPs are concerned that borrowers will not pay back loans on time. FSPs could offer incentives in the form of interest waivers or partial rebates for those who pay ahead of the maturity date to encourage borrowers to pay early. Further, the provision of a loan guarantee fund also mitigates FSPs’ concern on borrowers’ default to some extent; however, these should be applied strategically and in ways that can improve the credit histories of individuals or groups at large. In the Woredas where the LGs started working with FSPs, members are experiencing problems receiving loans on time. In some Woredas, this is partially linked to them not having a business plan. In response, the Activity should offer staff support to assist

12

members with the preparation of qualified business plans as quickly as possible and in tandem with future trainings with the help of DAs and the Woreda agriculture office. The Activity could also help HHs improve their future credit profiles by integrating a credit log for SILC members that records loans and payback rates. Over time, this could serve as a record of borrowing history and assist FSPs to value properly the risk associated with granting loans to individuals. 5. Push boundaries further to promote greater gender equity in uptake of Activity interventions. Livelihoods for Resilience-Oromia has included a number of interventions to address gender inequality issues within the area. That said, there is an opportunity for the Activity to explore this further and better understand what is inhibiting women’s full and equal participation, such as subdued voice during meetings, housework and long travel distances. While not all of these factors may be within the Activity’s control, the team should increase efforts to affect those factors that it can influence. 6. Improve communication to youth of opportunities from the Activity and conditions. Youths reported problems that fall into the following three camps: a) raising enough money to build regular savings, b) securing loans from FSPs, and c) participating in vocational trainings. Youths depend on parents for saving money, and often cannot pull together money for group savings without parental support. This emerged in particular for female youths. Given this, it would be worth testing in some areas if youth engagement would improve if they were combined into mixed groups rather than forming separate YLGs. This emerged as a suggestion from the evaluation of CARE’s similar Activity, however this idea has not been tested as far as we could tell. Developing an evidence base around this would be important in order to affect GoE PSNP policy and would need to consider the effect on both male and female youths separately. Further, youths feel that they are waiting a long time to receive promised trainings. The Activity should communicate more effectively with youth around training plans to help them manage their timelines and expectations, and then monitor closely the progress made on vocational trainings for youths. 7. Collaboration could benefit from additional investments in time by Activity staff. The existing collaborations between the Activity and other organizations and government were found to be effective overall. Government counterparts are mostly happy with the collaboration; however, a few believe that the Activity did not involve them adequately, but this should be balanced with the high turnover witnessed among government officials during the implementation period. Going forward it will be important to review present levels of collaboration and ensure that the Activity is reaching out to a broader cross- section of stakeholders.

13

8. Increase understanding among Activity staff of graduation requirements and strategy. The ultimate aim of the Activity is to support the GoE in graduating 18,375 HHs out of 24,500 HHs who are LG members from PSNP status. Given the time remaining for the Activity, and in the absence of a clear graduation strategy, this goal will be difficult to meet and needs to be prioritized. This will be particularly challenging due to the drought and other climatic factors affecting some areas, which make it more difficult for HHs to pass the graduation threshold securely. Activity management should discuss this with all members of the Activity team and key stakeholders to ensure that all have a clear understanding of what graduation means and define a clear strategy for reaching this target. It is important to note that while graduation is the overarching goal of the project, the actual determination on number of HHs to graduate along with which HHs will specifically graduate is the sole determination of the GOE. Therefore, while the project should retain a focus on graduation, it should expand its focus to include HH resilience, which CRS and MCS are better able to influence.

14

Midterm Evaluation

Background The Feed the Future Ethiopia Livelihoods for Resilience – Oromia (Activity) is a five-year USAID-funded activity, which started in 2017. The Activity intends to support livelihood options of chronically food insecure HHs in the Oromia region. The goal of the Activity is to improve the sustainable economic well-being of PSNP HHs by building on the push-pull Feed the Future strategy, the PSNP livelihoods framework, CRS’s pathway to prosperity model, and the success achieved under Feed the Future Ethiopia’s previous Graduation with Resilience to Achieve Sustainable Development (GRAD) Activity. The strategic objectives, intermediate results and interventions are designed to create collectively the push-pull elements necessary to generate livelihoods and employment opportunities and increase integration with wider markets that will support graduation of HHs from PSNP. Hence, the activity operates on both sides of the push-pull equation, equipping clients for basic market entry, including financial literacy, access to savings, and business planning and skills, while also focusing on higher-end access to markets, strengthening of the formal financial sector and business support services, and encouraging private sector innovations that could eventually catalyze wider market development. Activity Intervention and Strategy The Activity works closely to assist 24,500 HHs enrolled in the PSNP, with the aim of successfully graduating 18,375 of them from PSNP support over the Activity’s duration. Four high-level outcomes contribute to this aim: • Increased income and diversification through on-farm opportunities, including crop and livestock marketing systems • Increased income and diversification of off-farm livelihood options • Increased income from gainful employment • Increased innovation, scaling and sustainability of livelihood pathways.

1. Increase and diversify income through on-farm opportunities Establish livelihood groups: The Activity organizes clients into livelihood groups (LGs), which serve as an entry point into the livelihood interventions the Activity is implementing. Through the groups, clients receive training in financial education, nutrition and climate-smart agriculture (CSA), and access services that will serve as the foundation to their “pathway to prosperity.” By building on the LGs, income-generating subgroups are formed with a range in focus. They provide opportunities for members to gain an understanding of markets, leverage prices through aggregation and collection points, improve networking and relationships, manage transport challenges, and share labor and market information. The Activity is designed to focus particularly on facilitating inclusion of women and youth into livelihood activities.

15

Enhance agricultural inputs and outputs: LGs participate in trainings on CSA, learning how to protect and improve their natural assets in the face of recurrent drought and general climate instability. The promotion of these techniques epitomizes the Activity’s efforts to bolster resilience, improve nutritional outcomes, and help prevent families from becoming even more vulnerable. Promote agricultural value chains: The Activity promotes selected crop and livestock commodities as recommended by value chain assessments sponsored by the Activity. CRS provides technical information on inputs and their appropriate use. Engage FSPs: As clients become conversant with savings and financial literacy skills, the Activity facilitates linkages with FSPs, including MFIs, especially looking for ways for women and youth to engage.

2. Increase and diversify income through off-farm activities Through the Activity’s interventions, HHs will also increase their income and diversify economic activities through off-farm opportunities. Women and youth are equipped with knowledge, skills, and capital to start small enterprises to carry out value addition, processing and trading of various commodities with a focus on nutrient-rich value chains. Building on career planning exercises, youth identify livelihoods that align with their career aspirations and abilities, with selected livelihoods filling a clear gap in the local economy and identified through labor and market assessments.

3. Increase income from gainful employment Through the Activity’s interventions, HHs increase their income sources from various employment opportunities found locally and in surrounding areas. The Activity offers life skills trainings to youth and links them with prospective employers. Additionally, the Activity is designed to organize trainings for traditional leaders on youth engagement and decision-making roles, who later develop plans to sensitize other community members about the value of youth as productive members of society. Further, the Activity provides trainings to YLG members in positive youth development in order to involve them in income generating activities.

4. Increase innovation, collaboration, scaling and sustainability of livelihood pathways Livelihoods for Resilience-Oromia leverages the vast USAID resources in the area, facilitated by other activities such as the CRS DFSA. These interventions include the formation of SILC or other savings-based groups in addition to seed and goat distributions. It has also tried to complement these other activities by identifying knowledge gaps. This is most evident through the crop and livestock exploratory assessments commissioned by Livelihoods for Resilience-Oromia. Through the Innovation Fund, CRS provides small grants to catalyze market development and encourages off-farm enterprises in nutrition and climate-smart interventions. These grants seek to remove barriers to entry for product and business model development that would not be possible

16 without external investment. Grants aim to solve some of the more intractable livelihood development problems in Ethiopia and generate learning that can further inform the Activity. The fund is key to the testing of new Activity interventions, and the Activity closely monitors the implementation and results of initiatives funded through this mechanism. Interventions like the Innovation Fund contribute to the larger learning agenda for Livelihoods for Resilience-Oromia and other similarly focused livelihoods activities.

17

Evaluation Methodology

Midway through Activity implementation, CRS-Ethiopia commissioned the NDIGD at the University of Notre Dame to conduct its MTE. The MTE is an important part of the activity’s MEAL plan, enabling CRS to assess progress of the Activity, reorient Activity strategies and interventions as necessary, and determine the relevance of its strategies and interventions in addressing the needs of targeted groups (See Annex 1 for the MTE Term of Reference - ToR). As a complement to the various assessment and monitoring that the Activity is shepherding, this evaluation offers an opportunity for the Activity to gauge its implementation to date through the lens of its participants. The following were the specific objectives of the MTE. Objectives ■ To evaluate the quality and effectiveness of service delivery, the strengths and weaknesses of Activity implementation and management, and the technical quality of Activity inputs and outputs in terms of adherence to terms agreed to by USAID and acceptability and perceived value to target communities, identifying factors that enhance or detract from the quality, acceptability, and usefulness of implementation and outputs; ■ To present evidence of changes (intended and unintended) associated with activity interventions, assess how well the observed changes reflect the Results Framework (RF); ■ To recommend adjustments to the RF, activity design, resource allocation, Activity implementation and management, and M&E Plan that could improve the likelihood of achieving desired results by the Activity’s end, based on the evidence collected and conclusions drawn for the evaluation; and, ■ To determine the extent to which outcomes, systems, and services are designed and being implemented to continue after the Activity ends and assess progress made on implementing sustainability strategies. The evaluation employed qualitative methods—FGDs, KIIs, and site and intervention observations—to collect information from participants and key stakeholders (see Annexes 3 to 5 for data collection tools). The evaluation also reviewed routine monitoring data, partner and donor’s reports, and other Activity documents (see Annex 7 for the list of documents reviewed). Researchers from the University of Notre Dame visited Activity sites from August 24-31, 2019, in order to collect data from Activity participants, implementing staff, and other key stakeholders. During this field research component, the team observed group activities and participants’ behaviors. In most of the groups, discussions went well; the participants were engaged in the discussion and expressed their thoughts, feelings and concerns without hesitation. Evaluation Questions In line with the aforementioned objectives, the MTE aimed to answer the following research questions through a well-designed evaluation method:

18

1. How well have the Activity’s interventions met planned schedules, participant numbers, and outputs? What factors promoted or inhibited adherence to schedules? How were problems and challenges managed? 2. What are the strengths of and challenges to the overall Activity design, implementation, management, communication, and collaboration (with other USAID grants and government interventions) so far? What factors appear to promote or challenge the Activity operations or effective collaboration and cooperation among the various stakeholders? 3. What are the strengths of and challenges to the efficiency of intervention implementation? What factors in the implementation and context are associated with greater or lesser efficiency in producing higher quality outputs? What are the strengths and weaknesses of measures taken to ensure gender equity regarding access to, participation in, and benefit from the activity? 4. What changes—intended/unintended—do community members and other stakeholders associate with the interventions? What factors appear to promote and deter these changes? How do the changes correspond to those hypothesized by the Activity’s RF? 5. How could the activity be modified to improve its acceptability to targeted communities or the efficiency and effectiveness of its implementation? How should the Activity’s RF be refined? 6. Which Activity outcomes (services, goods or structures) are likely to be sustained after the Activity end? Who will help to sustain Activity outcomes? Evaluation Implementation Out of nine Woredas where the Activity implements its program, six Woredas were selected for the primary data collection. The selection criteria for selecting six Woredas include: political (administrative zones) considering two Woredas from each zone, remoteness/ accessibility to services (rural and semi-urban), diversity in livelihoods, number of program participants and overlap with other programs as well as donor interest for future interventions. Based on these indicators and more supporting information from CRS, the researchers selected two Woredas from each zone as follows: Table 1. Woreda Selection Criteria for Primary Data Collection

Woreda Zone Comments

Negele Arsi West Arsi Overlap with other USAID projects

Shala West Arsi Diverse livelihood zones and remote

Dodota Arsi Recommended by CRS Project Team

Sire Arsi Recommended by CRS Project Team

ATJK East Shewa First of two target Woredas in this zone

Boset East Shewa Second of two target Woredas in this zone

19

A training of the field researchers was conducted from August 22 to 23, 2019 in Addis Ababa (see Annex 8 for training schedule). The training to field researchers provided a background to the Activity and evaluation objectives and covered interview skills, data documentation, quality control, and data protection techniques. During the training, the field researchers also practiced by role-playing interviews in local languages (i.e., Afaan and Oromo). The field research team comprised eight research assistants (five males and three females) and one coordinator. The team field tested the tools on August 24, 2019, to ensure the language used in the interview was understandable, questions were relevant, and timing was adequate for conducting the interviews. The field testing was done in Dodota Alem Kebele, Dodota Woreda, which is one of the Activity sites. No major changes were required in the tools following the field testing. Afterwards, the research team started data collection in the selected communities. A total of 24 FGDs were completed across the six selected Woredas. Table 2 shows the number of FGDs completed per Woreda by adult male, adult female, youth male, and youth female categories. For the monitoring of data collection, two NDIGD staff were also present in the field. During NDIGD staff’s field visit, they also spoke with the Activity implementers, government officials, village leaders, and Activity participants. Table 2. Number of FGDs completed by Woreda S/N Name of Woreda Name of Kebele FGDs Completed by category Total selected Adult male Adult Youth Youth female female male 1 Adami Tulu Desta Abijata 1 1 1 1 4 2 Negele Arsi Dakha Horekelu 1 1 4 Dakha Harangana 1 1 3 Shala Wendo Koisha 1 1 4 Fende Ejersa 1 1 4 Boset4 1 1 2 5 Dodota5 Dire Qiltu 1 1 4 Amigo Dabaso 2 6 Sire6 Borera Chirao 1 1 3 1 6 Total 6 6 6 6 24

4 The research team was not able to find enough Youth LD in Boset, so it only conducted two FGDs there. 5 In Dodota, two Kebeles were selected, the first being Dire Qiltu. The researchers selected this Kebele for two reasons. The kebele had 12 LG and no separate YLG. Therefore, two adult FGDs were conducted in this kebele. The other kebele selected was Amigna Dabaso. This kebele was selected due to the availability of two adult LG and two YLG. Two FGD with YLG-boys were conducted, as the girls LG were not available, because only 9 girls included in the YLG. Among them, three married, two migrated and only four are active. Among the four again, only two girls were reachable.

6 To compensate for the lack of a youth female FG in Dodota, the researchers prioritized Kebeles in Sire that would ensure the availability of the girls in the LG. The Woreda has 13 kebeles under PSNP, containing a total of 126 LGs, of which 16 are YLG. Borera Chirao kebele was ultimately selected, allowing for three FGDs with youth female groups.

20

In addition to this, the research team also conducted KIIs with the following categories of people (see Annex 6 for the list of key informants):

Table 3: Number of KIIs Completed S/No Categories of KIIs No of KII completed 1 Women, Children and Youth Affairs official 6 2 FSTF officials 4 3 Agro-dealers 6 4 Micro-Finance Institutions 5 5 Innovation fund recipients 2 6 Collaborator within World Vision 1 7 CRS staff (from head office) 3 8 MCS staff 5 Total 32 Analysis Analysis of the transcripts focused on content and context, descriptions, language, and narratives that reveal respondents’ respective viewpoints on the process and impacts of Activity interventions. The qualitative analysis team read all typed transcripts and developed and categorized a hierarchy of concepts and themes that were featured in the transcripts, supporting documents, and literature. Systematically developed codes were manually assigned to all concepts and categories for further analysis in ATLAS.Ti software. As this evaluation was participant informed and prioritized, the researchers included narratives and snippets of interviews to illustrate findings from the study. Limitations of the MTE The research team acknowledge following limitations of this evaluation: 1. The evaluation focuses on six out of nine Activity Woredas. As these Woredas were selected purposefully, so the findings might not be generalizable to all the Activity Woredas. Similarly, there were 143 Kebeles in which Activity has program, but the researchers focused on only 12 Kebeles due to time and budget limitations. 2. Because of continuous rain, some Kebeles were not accessible. 3. The program implementer supported to identify the Activity LGs for the primary data collection, which might have introduced biases on group selection process. 4. In some places, it was hard to find enough girl youths at times. The MCS teams at the Kebele level described the difficulty of having youth organized under an LG, particularly girls who migrate to different places, including the Middle East or work in factories with low wages.

21

Main Findings

The findings of this MTE reflect the perspective of participants and key stakeholders involved with the implementation of the Livelihoods for Resilience – Oromia Activity. Findings have been presented by thematic area rather than in response to the specific Research Questions, as per the request of the CRS Ethiopia team. CRS and the Pulte Institute expect that this format will be more useful for CRS and USAID to incorporate the findings and subsequent recommendations. Activity thematic areas look at livelihood group formation with PSNP HHs; livelihood pathways; facilitation of market linkages and value chains; linkage to financial services; provisions to catalyze innovation; delivery on nutrition; opportunities to empower marginalized groups such as women and youth; collaboration with other United States Government (USG) funded actors; a review of sustainability and graduation; and an assessment of the project through the lens of resilience. These findings draw from participant perspectives as a way for the Activity implementers to gauge how they are delivering on participant expectations.

LG Formation Based on the primary data collected and review of Activity documents, it appears that the Activity interventions are proceeding as planned, although with some notable areas for improvement. The overall Activity participant number has reached 29,000, covering 24,500 HHs in nine Woredas. These participants are organized into 1,385 livelihood groups (LGs). The Activity aims to form 1,470 LGs over the life of the Activity. This, however, was based on the assumption of an average 20 members per LG. This group average will likely increase to approximately 25, meaning that the Activity may not need to achieve the original number of LGs to meet its participant target. The Activity has formed YLGs in Activity Woredas, with a total of 3,609 members across 146 YLGs. Within the mixed livelihood groups, about 1.3 percent members are youth7. The strength of the Activity design hinges on the ability to provide individuals and HHs with the skills and opportunity to become economically self-sufficient and secure. The establishment and development of LGs is at the center of the Activity’s success. The Activity focuses on improving the livelihoods of poor PSNP HHs, by encouraging them to participate in various on-farm and off- farm activities. The participants comprise males, females, and youths. FGD participants mentioned that the Activity members comprise safety net participants who self-selected into the LG, rather than being selected by the Activity implementer, Ethiopian Catholic Church Social & Development Commission of Meki (commonly referred to as MCS). Members developed group by-laws building upon a sample ToR provided by MCS. This helps constituent members to realize ownership of their group. The Activity is building new institutions in communities that are supposed to remain viable for extended periods and support the livelihood of poor families.

7 Percentage secured from a MCS source.

22

The Activity’s 2019 Year 3 (Y3) third-quarter (Q3) report shows that most of the planned interventions are on track, with some exceptions. For example, the target for organizing climate- smart agriculture training could not be met because of the onset of the planting season. The expected participants had to work in their fields, so they could not attend the training. There appears to be variation in the functionality of LGs. In some FGDs, members were not fully aware about the existence of a LG and some had even forgot its name, while others have experienced attrition of members. The attrition was especially noted within the YLGs. In other places, a written regulation was present, which guides the LG with specific rules outlined. Examples of these rules include meeting every two weeks at 2:00pm. Members who arrive late or miss a meeting will be fined 5 and 10 Ethiopian Birr (ETB) respectively. Members who happen to fight with each other during the meeting are fined 10 ETB. In some places, remoteness and terrain pose issues for service provision. The FGD participants of Wanda Kosha Kebele in Shala mentioned that they have to travel to Woreda headquarters, which is 67 Km away, to receive services from government programs or from an FSP. This entails additional cost to members to access the services they need. Some members in Dodota Woreda mentioned that the scattered population and youth migration in some places is a challenge to maintaining a functioning LG. The area is not densely populated, and many LG members have migrated to the nearby town and others skip the regular LG meetings, which poses challenges to continuing the group in the future. In some places, Activity implementation early on was affected by unrest, but it appears that the situation has improved, and no major problems have been reported recently. For example, in Sire Woreda, the members said, “[We] established the LG two and a half years ago, but it was not active for around a year due to the instability related to the political unrest in the country. So, they reactivated the group this year again, and it is six months since they [have] resume[d] the SILC activities.” Several social benefits have come about from the LG groups. The LGs formed because Activity implementation improved members’ intimacy with one another and created an environment for them to know each other better. Building networks locally helps to develop absorptive capacity of HHs who might need to exchange assistance with one another in case of shocks. While most of these benefits were intended and planned, there were also unintended benefits. Some members shared that because of SILC in LGs, there is less violence in the community. Moreover, they do not feel the need to borrow money from local predatory lenders, which used to generate conflict between borrowers and lenders when the borrowers failed to pay back money on time. There are also spillover effects in that community members who are not participants of PSNP are envious of the LG, particularly of SILC interventions. For example, in Negele Arsi, villagers sometimes request the LG members to admit them into their groups. Many of the people who have learned saving habits from the Activity have started their own savings. This should help FSPs to expand their services in the future as well.

23

Livelihood Pathways The Activity has three distinct pathways to support the safety net HHs to move out of poverty: on- farm, off-farm and employment activities. During the focus group discussions, group members spoke about the positive changes in their lives after joining these pathways. Increasing income and diversifying income sources from crop and livestock marketing is the goal of the on-farm pathway. By Y3 Q3, 18,472 members, which represents 64 percent of total participants, took up the on-farm pathway and associated capacity strengthening trainings, which is higher than the established Q3 targets. There are some indications that members are already involved in various on-farm activities, like vegetable gardening or growing maize with improved technologies or fattening animals for the market as a result of the trainings; but the availability of input such as financing for activities and market fluctuations have posed sizeable challenges to fully realizing this pathway. The off-farm pathway aims to increase income of vulnerable HHs by helping them adopt off-farm income generating activities. By Y3 Q3, 2,344 members (8 percent) had taken up off-farm livelihood pathway actions. There were some indications that the members are involved in various off-farm value chain activities, which is possible from the support they received from the Activity to prepare a business plan and secure financing from external FSPs. So far, 1,772 members have received technical trainings on selected off-farm value chain activities. These members also received training and support on the use of financial and business development services. The training that the Activity provided to the members was instrumental in starting businesses and running them successfully. Members report running petty trades or opening shops locally after participating in their LGs. Some on-farm MGs were also established to support on-farm activities of members. The main challenge to realize this pathway fully is the inadequate access to financing.

Conversely, the employment pathway aims to increase the income from gainful employment and focuses on increasing access to various employment opportunities. By Y3 Q3, 196 members took up this pathway as their principal livelihood. This pathway is underachieving, and LG members, especially the youth, are not fully satisfied with what the Activity is providing. Their biggest concern is that they have to wait a long time for the trainings, which might otherwise open up employment opportunities for them. Vocational training is crucial to increase their employability, as otherwise many would have to compete for positions with just a grade 10 education. Some youth members mentioned that they have waited for the last couple of years for vocational trainings promised by the Activity while they could have attended the TVET on their own or with the help of relatives. During the field visits, researchers found that the youth who wanted to participate in these trainings did not know the reason for the delay. The Activity is cognizant of youth issues and the importance of bringing life skills trainings to them. While there were some delays in implementing youth trainings in all Woredas, there are Woredas where the Activity has already started technical training for the youth.

24

While the Activity placed special emphasis for youth participation in the Activity, for some it created an additional burden. In one youth FGD in Shala, most of the participants (except one) were students who are engaged in petty trading of potato and onion on market days. They attend school during one shift, are engaged in trading after school, and pay their saving contribution from this trading. This burden, however, made them not focus on their education as much. They strive to trade but are also expected to support their family through domestic activities. While the major strength of the Activity is the provision of training on technical and entrepreneurial skills, the participants also point out major challenges in implementing those interventions. A major weakness of the Activity according to participants had to deal with its unreliability in implementing interventions as per the initial timeline communicated to participants. According to LG members, the Activity staff promise to organize trainings but then change the dates with little notice. A lack of clarity among members on what the Activity will provide and how they can prepare was also cited as an issue. Another challenge includes LG members having higher expectations from the Activity than what it was designed for. In some communities, LG members benefited from previous activities, like GRAD, that extended in-kind support to its participants through a crisis modifier. This has been a source of discouragement for some members. In ATJK, researchers observed that goat sheds were constructed. Members said that the Activity had promised to provide goats as well, however none of the participants have received these to date. In some Woredas, youth do not feel particularly excited, and are leaving their groups. They mentioned that the Activity does not have much to offer youth who do not have money to save and consequently do not qualify to receive credit from a FSP.

Market Systems and Value Chain Development The Activity commissioned different studies at the community level to understand the prospects of various value chain interventions to support the livelihood of men, women and youth in the communities. These studies made several recommendations around value chain options. The researchers examined whether those recommendations were implemented and the outcomes. One recommended value chain area involved livestock. Livestock are largely owned by smallholder farmers and are an integral complement to the agricultural production system. The government is also stressing their commercialization. The recommendations sought to improve upon the value chain by ● providing support to private feed dealers, ● establishing linkages with veterinary services, ● facilitating the market, ● improving financial access for investing in the value chain, and ● increasing women/youth participation in the value chain.

25

In line with these recommendations, the Activity is working with agro-dealers and innovation grantees to find a sustainable solution to feed problems for livestock and poultry. The Activity is also working with government offices responsible for providing veterinary services and training for business skills. During the FGDs and KIIs, respondents mentioned how the training the members received was helpful for them to find out how to start a business. Some members mentioned receiving seeds of various crops (beans, teff, carrot and onion) and egg laying chicken breeds; however, these were in CRS DFSA overlap areas. Some members have started livestock or crop based value chains, but they have not been widely adopted. Reasons for this include lack of access to financial resources, poor support systems, and latent markets. Some LG members’ success stories indicate that, with the training sessions and financial linkages established by the Activity, they have been able to expand their own businesses and invest in new ideas. This aligns with the goals outlined in the RF to increase vulnerable HHs’ income and diversification through off-farm livelihood options and on-farm opportunities. The increased use of financial and business development as well as increased sustainability of livelihood pathways has been a positive change noticed within communities. Some families that used to rent out their land to others have now started farming on their own when they have access to financial resources. This is a positive change. To implement on-farm activities effectively farmers must have access to a proper support network for inputs; however, a sufficient supply of agro-dealers are not available within close proximity. In response, the Activity invested in agro-dealers to upgrade their infrastructure and trained them in business management. The existing agro-dealers are limited to city centers and are selective in their product offerings. Some supply only cattle feed, while others supply herbicides and pesticides or fertilizers. This means that farmers cannot procure all the necessary inputs from a single source. Further, for many members the quantity of inputs is not affordable. For example, a farmer may need pesticides for a quarter of a hectare of his/her land but the quantity/package that exist in the market does not fit with what the farmer needs. Most of the time the available quantity/package is larger than required, and buying a large package is costly for farmers. The other problem is that agro-dealers do not want to work in remote areas where most of the rural Kebeles are located. To mitigate this issue, the Activity is working with select agro-dealers to expand their services in rural areas where the LG members are. Agro-dealers mentioned that they found the business training offered through the Activity helpful. Moreover, the Activity is supporting the agro-dealers to expand their businesses by connecting with the feed suppliers. However, agro-dealers have found that the feed provided is expensive and out of reach of poor members. A dealer in Negele Arsi mentioned, “We attempted to make an agreement with this producer but the price of their product was too expensive. The farmers in our local area cannot afford the price.” Given the fact that the customers of these newly supported agro-dealers are PSNP HHs, it might be challenging for them to afford expensive feed for their livestock and still derive a profit. The Activity should explore affordable input options before connecting them with agro-dealers. There is an initiative to engage agro-dealers to support

26 agricultural production, but it has just started, and none of the members mentioned benefitting from this yet. The Activity has strengthened or supported 18 agro-dealers as of Y3 Q3. It is worth noting that one agro-dealer in ATJK, who has already received support from the Activity to expand his business was hesitant to extend to Activity areas and is expecting more support from the Activity before doing so. It is not clear from the conversation whether this agro-dealer will eventually offer services to rural Activity participants in the future. The money he received has already been spent to construct a warehouse, and as of now, he does not plan to open branches in the Activity area without additional resources. In Negele Arsi, the agro-dealer was expecting support to invest in additional stock, although this is not possible under this Feed the Future award. Examining changes in relation to the recommendations made in the livestock value chain study, researchers documented success stories from the members. Very few members, however, were involved in value chain interventions in the sampled LGs. Some members shared how they benefitted from credit received from a savings and credit cooperative in fattening cattle. In a women’s FGD in Boset, the members stated that they received 7,000 ETB from external financial institutions for cattle fattening and profited from selling the cattle. Similar success stories were shared within the women FGD in Dodota where members used a loan from group savings and invested in a potato value chain. There were challenges too. The value chain activities are not well established in all Activity areas, as a supportive infrastructure – electricity for poultry, quality roads for transportation, and market to sell products – is not available in all Activity areas. In addition to these factors, the drought and other adverse climatic factors limited the production of various crops. Value-chain related actuals are lower than their targets stated for Q3, which can be attributed mostly to not having sufficient staff at the Kebele level. This forced the community animators (CAs) to stretch themselves well beyond their comfort zone and affected the quality of the trainings at times. This problem of understaffing has been resolved through the Activity’s recruitment of 143 Marketing and Technical Extension Workers (MTEWS) at the local level. The market is a determining factor for the success of any value chain. Some Activity Woredas have good market opportunities, but they are underutilized. In the Negele Arsi FGDs, participants stated that because of the presence of lodges and hotels in the area, the demand of goats for meat is high. Currently, the demand is being met by bringing goats from other areas because the local population does not have the awareness and the finance to tackle this need. Further, the hotels and lodges ask for cabbage, carrots and a large number of eggs, but the local people are unable to supply these products in the quantities required. They note the lack of experience, business acumen, and start-up financing are the main hindrances. The members stated that continuous follow up after the training by the Activity staff would be effective to maintain motivation. The follow up helps ensure they are on the right track or determine what extra support may be needed. Activity staff, however, have not followed up with all groups. This has been a weakness in the past. In one FGD in Dodota, the members mentioned

27 that after the organization and formation of the livelihood group, the Activity office has not followed up and monitored the progress of the group. This is expected to improve with the recent hiring of 143 MTEWs. There are challenges for youths to engage fully in value chain activities. The main constraint for youth is getting access to finance. The researchers elaborate how the access to finance has limited the ability of youths in the following sections.

Financial Services Financial services are mechanisms to assist LG members with funding for livelihood opportunities. Members admitted that, despite the onset of the Activity, securing a loan for emergencies or for investing in a business remains a concern. Lenders, be it local predatory lenders or MFIs, are not inclined to extend credit to the poor. That said, with the establishment of a SILC in each LG, members feel somewhat empowered to avoid begging for money in all conditions. Most community members admitted to having a poor saving culture before, but savings activities have helped people see that something big could come from small things that appear insignificant at first. Both the male and female participants are accessing internal loans. In 2019 Q3, more females received an internal loan than male participants did. For example, out of 1,867 borrowers during the quarter, 1,017 were female who borrowed 436,000 ETB of the total 802,000 ETB provided to the members. From external sources, however, the percentage of female Activity participants who borrowed is lower than the number of male participants who borrowed for the same period. Out of 707 members who borrowed in Q3, 327 were female, who borrowed 1,875,535 ETB of the total 4,143,160 ETB.

Members asserted that the availability of financial resources through SILC increased in a relative way after joining the group, though this has not helped them engage in business activities due to the small amounts saved. One group cited a scenario of a savings corpus worth 10,000 ETB where five of its members request a loan, which affords a loan of only 2,000 ETB for each request. They perceived that this amount would not be enough to start a business. Generally, participants do possess other necessary resources (experiences of doing businesses and skills), but are not finding adequate financing to initiate a business. In the Sire women FGD, a member expressed that, “in order to improve the Activity’s acceptability and enhance its benefits to targeted HHs, the amount of loan should be enlarged; and additional trainings on how to strengthen the LGs and mobilize additional money should be delivered for all members.” Members are linked to external FSPs, but this approach is fraught with challenges. Because of the high default of previous loans among the members, FSPs are reluctant to provide loans to current borrowers when there is a risk of default. Another issue with procuring the services from a FSP is adequate liquidity, since there are not enough FSPs available to extend credit for all participants who request it. As a result, the Activity is far from meeting its target for borrowing from FSPs

28

(1,037 as of Q3 versus 6,615 Y3 target). To address the default problem, the Activity is providing a fund to offer loan guarantees to FSPs. The purpose of this guarantee fund is to mitigate the risk for FSPs and incentivize them to stimulate lending. Nevertheless, the FSPs are still hesitant to offer credit to all eligible members as they say the guarantee fund does not cover their cost in the case of higher default rates among borrowers. Interestingly, 54 percent of people accessing SILC loans during Y3 Q3 are women, while 54 percent of those accessing loans from FSP during Y3 Q3 are men. In some FSP branches, the branch managers were not aware of the details of the agreement between the FSP head offices and the Activity guarantee fund. KIIs with local branch managers revealed this communication gap between the branch office and the head office regarding this agreement. The managers had a hard time explaining how the guarantee fund works and what types of support the Activity provided for expanding the capacity of FSPs. Keeping the agreement, a secret between FSP management and the Activity may be intentional to incentivize borrowers to pay back the loan in a timely manner, but branch offices serve as the primary interface to borrowers. Without detailed knowledge about the contract, they are less willing to issue a loan, collect prepayment, and deal with defaulters. Going forward it will be important to provide clear information on the contract to all related staff, at least at the branch manager level, in order to avoid confusion. FSP officials want to improve the system on how the Activity works with SILC and FSPs. In one KII in Dodota, the FSP respondent pointed out the lack of follow up from the Activity staff and recommended that Activity employees follow up and support SILCs since some lack various requirements (i.e. treasury recording), assist the community in business plan preparation, and report progress to the FSP when needed. Not all members are comfortable with the idea of securing credit from a FSP, as some members believe the interest rates that FSPs charge are too high. Moreover, for some, the upfront saving requirement may be a limitation, as some members cannot find the money that has to be saved before qualifying for a loan. This is especially problematic for youth who depend on their parents for saving money. Further to this, the lack of farmland or other assets was cited as a constraint for some members to receive credit from a FSP, as some of them were already forced to sell their land during an emergency and now do not have any assets to put against the guarantee. Youth in particular face similar challenges when attempting to access loans. Many are unable to save money through the group or offer collateral. Since youth depend on their parents for their savings, this limits their ability to accumulate adequate amounts to quality for FSP loans. In addition, few possess land at their age to put forward as collateral. Youth participants in all the FGDs expressed these concerns. Youth who were working as daily laborers in the community developed better saving practices, even before the Activity started, and a small number of youth members were able to secure loans with the assistance of the Activity. In these cases, the Activity

29 facilitated links to financial services for them with minimal interest rates, worth 10 times the amount they saved in their groups. Some members follow the Sharia principle and do not want loans. The members in ATJK Woreda cited their religious law as a barrier to receiving loans with interest. They stated that they have agreed with the Activity implementers in this regard. The implementers have agreed to work with the participants in way that is sensitive to their religious values. In Negele Arsi, members who borrow money from the group pay a 10 percent return per month, but do not view this as interest. Instead, participants have labeled the additional payment as a thank you. Some members opposed the idea of sharing the default risk across the group to secure a loan. One member shared her experience working with the Oromia Credit and Saving Share Company (OCSSCO). Although she paid all her debts to the company, she was forced to pay for those members who defaulted. As a result, she now wants to access credit independently. A separate but notable concern was raised in Negele Arsi by the manager of a MFI who was stressing about a promise that the Activity made around the provision of a motorbike for their RuSACCOs to help them make frequent visits to the target group. He also referenced money and office equipment that had been committed by the Activity to the FSP, none of which have been received. It would be worth it for the Activity to look into this by either fulfilling this request or managing expectations to ensure the manager continues engaging with the Activity.

Innovation Fund Innovation funds are provided as part of CRS’ cost share to innovators to develop technologies that benefit LG members. By Y3 Q3, there were 14 innovative initiatives led by 11 grantees8. One grant was provided to Adami Tulu Agricultural Research Centre develop moisture conservation technology for maize cultivation in moisture deficit areas. The technology conserves moisture for crops and helps improve harvests when rainfall is poor. This type of simple technology will help farmers immensely. Other notable innovations being supported are as follows: ü Arsi University, College of Agriculture and Environmental Science – fattening approaches for small ruminants. ü Bora Dembel Farmers’ Cooperative Union – animal feed processing unit at ATJK. ü NASIBA Animal Nutrition – a packaging machine and connection to agro-dealers to complement NASIBA’s patent around MIN Chick, a mineral supplement for poultry egg production, shell quality and egg productivity.

8 The grantees include: EIAR-Melkasa Research Center, Adami Tulu Agricultural Research Centre, Bora Denbel Multi Purpose Cooperative Union, Arsi Universty College of Agriculture and Environmental Science, Kaleta Saving and Credit Cooperative Union, Medina Hamza and Medina Poultry production, Mirko and Mekdes Agricultural Input suppliers Enterprise, METEMAMEN Micro Financing Institute Share Co., NASIBA Animal Nutrition (NAN), Selam Awassa Business Group PVT.LTD.Co (SABG), Shayashone Trading PLC.

30

ü Selam Awassa Business Group PVT. LTD. Co – firm that has an agreement with Melkasa Research Center to multiply and distribute the mechanized bean thresher developed by the research center and delivery of sample threshers to selected MGs and training schedules. ü Shayashone Trading PLC – sale of Purdue Improved Crop Storage (PICS) bags, inclusive of training manuals, brochures and other promotion materials in preparations for distributors’ trainings in Q4. ü Medina Hamza and Medina Poultry Production Enterprise – construction of a hatchery and feeding production house. ü Mirko and Mekdes Agricultural Input suppliers Enterprise – delivery of a feed machine. ü Kaleta Saving and Credit Cooperative Union – agreement between the Union and a software company to improve the Union’s accounting system. Furthermore, the Union produced coin boxes to be distributed to RuSACCOs' LG members. ü Metemamen Micro-Financing Institute Share Co –agreement between the micro- finance institute and M-Birr to support the introduction of mobile and agent banking services. Some participants are selected for demonstration sites where LG members have a chance to observe these innovative technologies to increase their agricultural or livestock production. The farmers who have an opportunity to observe the field activities can adopt the practice in their own lands, which will help scale up the Activity. One grantee shared his story on how he is involved in promoting a new technology in LGs: “As I told you we have two Woredas, the training was given for a day for a Kebele. As I told you within a Kebele there are two farmers groups. Similarly, for goat fattening the same training was delivered. We spent most of our time delivering training for farmers.” Training from researchers might be different from the training provided by a DA. Such trainings might be more problem focused and useful for the farmers. Since the grant fund came via Activity cost share, the grantee targeted for their field activities farmers who are under the Activity. For example, 596 farmers are included for the intervention in Negele Arsi and ATJK Woredas for the goat fattening and moisture conservation grants. During the field work, the research teams also observed the demonstration plots and goat sheds. Factors such as a farmer’s willingness to be part of the Activity helps to accomplish the planned interventions on time. Researchers found this lacking in some places. For the goat fattening technology, the innovative fund grantee promoted fodder plantation to feed the goats, but participating farmers were requesting payment for the land where the goat feed is being grown. Further, the understanding in moisture conservation technology was that the labor contribution would come from the participating farmers and the research center was not expected to contribute labor. Nevertheless, there were farmers who were not willing to contribute to the labor. Such types of behavior from the participating farmers makes implementation less efficient.

31

Nutrition For promoting optimal health behavior, the Activity intervened in an area of nutrition that will mostly benefit lactating women and children. Activity interventions focused on promoting nutrition rich crops, homestead gardening, and cooking demonstrations with locally available food items. Nearly 12,000 members have received nutrition sensitive agriculture training, and over 17,800 members have received the homestead gardening training. These are two areas where the Activity is over-performing versus its Q3 targets. During the FGDs, members shared how they are engaged in gardening and produced nutritious vegetables. In Sire, members mentioned that the LG members have begun to grow vegetables like cabbage and lettuce over the last three years. Seedling/seed have been provided in CRS DFSA overlap areas. Dietary diversity messages were delivered using different platforms including SILC meetings, functional literacy trainings, and cooking demonstration sessions. Beyond the LGs, vegetable farming not expanded to other community members, but Woreda-level nutrition experts are being hired facilitate the support needed to do the training. In KIIs, the agro-dealers who are working with poultry stated that egg-laying hens are being distributed to participants, which also supports the nutrition of children in the family in addition to providing income from chicken sales. In Dodota, an agro-dealer bought 500 additional improved breeds of chicken with support from the Activity. According to him, farmers will benefit from his service since he is planning to provide quality chickens. To inform trainings on dietary diversity and budgeting for nutritious foods, the Activity collected data for a Cost of Diet assessment to understand HH costs of food based on current dietary habits. The Activity plans to present the findings in Q4 to key stakeholders. Building on these findings, the Activity will prepare a curriculum to train participants on 1) HH budgeting and resource allocation for nutritious foods and 2) options for planning cost-effective nutritious meals. The Activity also collaborates with other programs to implement nutrition interventions. For example, in the CRS DFSA area, CRS DFSA distributes seeds of different vegetables. The seeds are used to grow nutritious vegetables to improve the health of Activity members. During the field work, the research teams also observed the vegetable gardens where the participants were growing seasonal vegetables.

Women’s Empowerment Women’s and men’s roles, ambitions, access to and control over resources, and influence over decision-making are some of the crucial factors that need to be considered in any development Activity. These dynamics change over time. Sometimes the Activity interventions influence these changes. In each role, women and men may act together or separately, depending on their own concerns and interests. Interventions to address gender inequality must be based on a nuanced picture of these complex gender relations and the drivers of change to avoid creating conflict in social groups.

32

The prevalence of illiteracy among women within Activity areas compounds the challenge of adopting new knowledge and skills. Information disseminated through public gatherings may also be out of reach for many women because of restrictions on their participation in mixed gatherings, particularly in more conservative communities. In response, the Activity is delivering functional literacy trainings to women within the target area. These structural barriers to the improvement of rural women’s livelihoods are not easy to address. The Activity is cognizant of these issues and aims to tackle gender inequality by involving women in Activity interventions. The Activity also tracks women participants separately in its monitoring system to make sure the target is met for women members. Women members appreciate the Activity’s design that encourages their participation in functional literacy, leadership and other group level interventions. Regarding the special benefits women receive from the Activity, participants in Boset FGD said, “women benefited from the Activity because they equally participate in meetings with fellow men, which means a lot and contributes to gender equality as well as equity.” The key informants of all WCYA offices appreciate the Activity for its approach to empower women by providing opportunities to women in rural communities. They appreciate its integrated package that includes organizing women in a group, providing them with literacy and livelihood related trainings, promoting SILC, and linking them to FSPs. Women members are now able to engage in meetings with each other, discuss their own issues, save money and engage in income generating activities. One of the participants in ATJK said, “We are able to talk with you, but this was impossible in the past”. Members cited a case where a woman who is a member of the LG won a court case. In the participants’ view, this woman won the case against her husband because she had access to financial resources from the group savings, coupled with a sense of empowerment engendered through the Activity. As per the opinion given by the women members, in most areas they do not encounter problems with participating in Activity interventions. The numbers of men and women participating in Faithful House trainings is another area where the Activity is over-performing versus the established Q3 targets. Faithful House trainings proved instrumental for promoting better understanding of different roles of husbands and wives within a HH. Both husbands and wives mutually own the money contributed as savings to a SILC. Thus, if the wife wants to participate in a SILC, they will discuss its usefulness and make decisions together. Sometimes both a husband and wife can participate in one group, and if the wife misses the meeting the husband will cover the expected contribution. That said, women in the SILC face a problem of showing up on time to meetings due to the burden placed on them to complete all HH chores before attending. In some communities, women said that they need to ask permission from their husbands if married or from their fathers if unmarried to participate in the meeting and other interventions. In the Sire youth FGD, one member raised the example of a girl from his neighborhood with whom he started their LG. Ultimately, she left

33 the group because her father opposed the idea of going to Sire for a training as he could not trust others enough to send his daughter to spend nights and days there. Some members in Dodota also pointed out the preferential treatment provided to men at the expense of women in the groups if all the group leaders are men. In some interventions, women’s participation is lower compared to that of men. In trainings around on-farm value chain interventions, only about 40 percent of participants were women. Fewer women participated in climate smart agriculture and vocational trainings compared to men in Q3.

Youth Empowerment As referenced earlier in this report, the Activity reports supporting 3,609 members across 146 YLGs, which are divided into male groups and female groups. To add to this, an estimated 1.3 percent of the mixed livelihood groups qualify as youths. While this may have been the case earlier on during LG formation, the scattered population and aggressive youth migration in some areas is proving a challenge to maintaining functioning YLG and youth participation. In some Woredas, youth are leaving their groups, sensing that the Activity does not offer much to those who do not have money to save and do not qualify for credit from a FSP. This is particularly true of the female YLG, which are further inhibited by the responsibilities of marriage, HH duties, conservative families that restrict their participation, and the increased challenge for them to save with no access to resources. Delays to implementing vocational trainings in technical and vocational teaching school (TEVTS) have exacerbated youth attrition. Some youth members mentioned losing two years waiting for vocational trainings promised by the Activity, while they could have attended the TVET on their own or with the help of relatives. While the Activity placed special emphasis for youth participation, for some it has created an additional burden. Some youth claimed to be losing focus on their education and strive to trade while also supporting their family with domestic activities. On the positive side, the Activity is designed to address expressed needs of youth through the provision of vocational training. The knowledge and skills they expect to obtain from the Activity should increase their employability, which could positively affect their retention in LGs and their long-term earning potential. For this reason, it will be important for the Activity to move forward with these trainings and communicate revised schedules to youth in a timely basis.

Collaboration In Woredas where another U.S. government-funded activity is being implemented, Livelihoods for Resilience – Oromia works in collaboration with the other activity while implementing its own interventions. This helps to develop synergy to strengthen the cumulative impact. For example, CRS and WV DFSAs strengthens PSNP capacity and provides food aid or cash assistance to PSNP HHs. Livelihoods for Resilience – Oromia does not provide this type of support to LG members.

34

Inclusion of different stakeholders during the planning, implementation and supervision process improves effective collaboration among the various stakeholders. The Activity was successful in involving government counterparts in Activity implementation and worked with the local office of Women, Children and Youth Affairs, Woreda/Kebele FSTF offices, and various DAs who provide extension services in Kebeles. Some government officials even attend the trainings. Continued communication and discussion with stakeholders and providing them with the activity plan in a timely manner is critical to increasing their support in the overall Activity. Key informants confirmed that the Activity is working with government line agencies in implementing the Activity interventions. A key informant of the FSTF in Negele Arsi Woreda says, “[The Activity] contacts our office to work with us. [The Activity] works with our food security staff. We two deliver awareness creation together. We recruit participants of the Activity together with Catholic Relief Service (CRS).” However, not all the stakeholders are happy with how the Activity is working in its Woredas. A FSTF official in Sire and WCYA official in Dodota claimed that they do not know much about the Activity, nor do they know the staff working in the Kebele. Other key informants indicated poor communication between the Activity and stakeholders, particularly regarding potential monitoring roles. This appears to be the case mostly in situations where the government offices have experienced high turnover. One key informant in Boset stated that: “The organization (i.e. CRS) by itself didn’t explicitly tell what the concerned stakeholders are supposed to carry out (i.e. their duties and responsibilities, procedures of monitoring and evaluation, financial allocation for the same, the way things get documented and the like). We have repeatedly talked to the organization so that it adjusts this issue, but it remains the same.” A key informant in Sire made a similar observation and mentioned a loose collaboration between stakeholders and the Activity. According to the informant, the Activity staff do not let them know the details of the intervention ahead of time, which creates confusion. This might be a problem of a particular staff or stakeholders, but the Activity should correct these types of issues.

At the beginning, there was confusion in an overlapping Woreda on how to implement Activity interventions if two entities are doing similar interventions. This was mostly related to DFSA interventions being implemented by WV in Siraro Woreda. There were a number of conversations between CRS and WV to sort out overlapping issues. After such conversations a memorandum of understanding (MOU) was prepared that explained who will be doing what in which areas. For example, since WV’s DFSA was already establishing LGs in some areas, CRS focused on LG establishment in other areas. WV trained LGs on saving and landing processes. Once the group had savings, members could start preparing business plans and start seeking financial resources. In terms of its scope, WV recruited participants from the LGs and trained them on financial literacy, gender equality, nutrition and sanitation, climate change and adaptation, livelihood pathways, and on small-scale irrigation. The coordination with CRS’ Livelihoods for Resilience –

35

Oromia is going well, although the interventions under this Activity have started late in this Woreda, particularly the linking of participants to credit facilities. The Activity also collaborates extensively with the private sector. Currently, the Activity is working with 18 agro-dealers, 6 FSPs, and 14 innovation fund grantees who are supporting its interventions in varied ways. The Activity supported these stakeholders by providing funding to agro-dealers to expand their business, to FSPs by providing training and other logistical support, and to innovation fund grantees by providing grants to conduct the research and development actions. The support the Activity provided to these stakeholders were crucial to strengthen their capacity generally, which benefits Activity participants.

Resilience Capacities of LGs This report applies the USAID Resilience Framework to examine community level resilience in Activity areas, noting the contribution of Activity interventions across the range of capacities that influence overall resilience. Measuring the changes in these capacities is beyond the scope of this midterm evaluation, observing instead general progress. If the Activity aims to conduct a final evaluation of the Activity, a well-developed survey can capture resilience outcomes, which could be compared with baseline data to measure impact. Absorptive capacity: Absorptive capacity speaks to HHs’ assets, the availability of informal safety nets, access to cash and savings, credit, and networking at the local level. Absorptive capacity is the basis of how rural HHs buffer short-term disturbances as well as the beginning phase of larger shocks. HHs that are equipped with various skills and access to different resources in case of emergencies are more likely to tackle and bounce back from a disturbance. The following Activity’s interventions directly strengthened the absorptive capacity of HHs. • Increased the access of safety nets population to cash by promoting savings behaviors; • Increased access to loans by utilizing group savings or by linking to external FSPs; • Creation of a forum locally to strengthen the social network. LG groups are a proven mechanism to expand the local network of women, adult men and youth, who meet and plan regularly and help each other.

HHs that are involved in savings and credit as well as expanding their social networks are more likely to be resilient and retain their assets in the event of a short-term emergency or shock. Access to social funds and other funding mechanisms available through the group reduces the need to sell their fixed assets. The Activity provides various capacity building interventions (training on leadership, business development, etc.) to help members maintain these interventions after the Activity is over. Adaptive capacity: Adaptive capacity encompasses providing motivation and aspirations to LG members, supporting agricultural practices, facilitating input markets, providing vocational skills, and business skills, and diversifying income sources. Adaptive capacity supports systems at the community level to adjust by ensuring that systems function if a shock exceeds the community’s

36 absorptive capacity. It is the ability of the community to adjust itself to sustain and function as usual when there is a shock. These adjustment practices are incremental and draw from learning through failures and successes that add to adaptability. For example, when rural HHs experience food scarcity, HHs adopt different strategies to increase production from the land they owned. HHs might practice new moisture conservation practices in the event of frequent droughts and learning from the first year could be important for the continuation of soil conservation practices for a better harvest in coming years. The Activity interventions under this capacity are:

• Enhanced mobilization of PSNP HHs to join LGs by motivating them to engage in livelihood activities that will improve their livelihoods; • Provision of trainings to improve business skills of members who are now running successful businesses. The Activity offers these trainings based on the chosen livelihood pathways. For example, the Activity delivered training on pepper and haricot bean production for on-farm livelihood pathway members. • Provision of trainings on goat fattening and poultry production for HHs involved in the livestock value chain; • Expansion of agro-dealer services in rural areas to improve the input market, and provision of trainings focusing on vocational skills that would help involve HHs in wage employment; • Engagement with vocational training centers to create a training curriculum for Activity youth. In light to this capacity, there is also a need to redefine the concept of resilience, which can only be achieved by diversifying income sources. At its start, it appears that the Activity assumed that resilience could be achieved by producing a single crop, such as onion. Diversifying income sources is critical for HHs to adapt in case of a shock. It is especially necessary for PSNP HHs to have more than one income source to be resilient in light of increased climate related shocks. Hence, the Activity should focus on diversifying income sources rather than just strengthening only one income source. There is also a gender element of adaptive capacity given the widespread inequality in social, economic and educational disparity between men and women in the Activity area. Women headed HHs might suffer more from shocks if illiteracy among women is very high. Illiteracy can cause barriers in understanding and recall of training content. The Activity should develop different strategies (i.e. more practical trainings) than classroom-based theoretical trainings for women, to address this type of a barrier. Transformative capacity: Transformative capacity encompasses factors like the availability of extension services, market infrastructure, and research institutions in solving local problems. Transformative capacity creates a favorable environment for rural HHs to maintain their livelihoods and may require institutional reforms, behavioral changes and technological innovations. Factors like socioeconomic policies, land-use policies, resource management trends,

37 institutions and technology may limit the performance of transformative resilience. The Activity is cognizant of this and is implementing the following interventions accordingly:

• Provision of trainings to DAs to improve their skills and linking them to LGs for solving various problems related to crop production, animal fattening and natural resource management; • Improved market infrastructure with live market information; • Increased engagement with research institutions to develop different scalable solutions to local development problems, by which members will benefit most; • Increased linkages to financial institutions and strengthening of their capacities so that they can expand their service areas. Transformative capacity is critical for the graduation of HHs. If supporting infrastructure, like market access or availability of financial resources is available, then LG members will be less likely to return to the safety net category. The trainings (capacity strengthening/knowledge transfer) provided to Woreda cooperatives experts, SACCO unions and RuSACCO staff and MFI experts will arm them with the knowledge and skills needed to address local problems. These absorptive, adaptive and transformative capacities that the Activity is building will be instrumental for sustaining the Activity’s outcomes after it ends.

Sustainability and Graduation From a development perspective, programs should be designed and implemented in such a manner that they produce products and services, and outcomes beyond the life of the Activity. It is also imperative that programs are not only sustained, but also benefit additional people by expanding to new areas. Development outcomes are both enhanced and more reliable when there is a high return on investment on Activity interventions. For the sustainability of a development Activity, four factors are found to be critical9:

• a sustained source of resources • sustained motivations • sustained technical and managerial capacity • linkages to government organizations and/or other entities Examining the Activity’s intervention in light of these critical factors, the Activity’s interventions may endure after the Activity ends, however more works need to be done to ensure this. The Activity LGs require a variety of resources in order to sustain themselves in the future. The principal resource need is financing, which the group is generating through the savings that the members collect regularly. Group savings and savings in social funds are critical to serve

9 Sustaining Development: A Synthesis of Results from a Four-Country Study of Sustainability and Exit Strategies among Development Food Assistance Programs. USAID/FANTA III (2016)

38 members’ needs. In addition to financial resources, the Activity is also building human resources through trainings that are needed to lead the group successfully. Participants and stakeholders appear to be highly motivated to sustain the Activity’s interventions. In all FGDs, members affirmed that they want to keep their group intact and keep SILC going in perpetuity. One participant in Boset said, “When a father has a son, he will enable him to stand by himself.” The Activity has assisted them to stand by themselves in providing sustainable knowledge, and they feel confident in their ability to keep the Activity going. They do not know when the Activity ends, but they are optimistic that they will keep savings activities running and work towards diversifying income sources with the knowledge and skills received. The Activity is working towards building technical and managerial capacity locally. In addition to providing trainings on various technical and managerial skills to the group members, the Activity has been working in close collaboration with the concerned stakeholders at the Woreda and Kebele levels. The Activity is providing trainings to local government partners to help them build their capacity and support the groups after the Activity ends. The Activity works with agricultural staff (livestock and crop) and Child, Youth and Women offices as well as with MFIs to build their capacity. The Activity has been involving these stakeholders in Activity planning and implementation. Thus, after the termination of the Activity, it should be easy to handover the Activity interventions and tools to local stakeholders for future application. Working with the stakeholders, the Activity is placing greater emphasis on linking LGs to government organizations and other entities. The Activity’s approach to sustainability is rooted in the belief that the private sector (e.g., agro-dealers, FSPs, etc.) will play a key role after the Activity ends. Various supports (i.e. resources and capacity building) the Activity provides to these private sector actors to strengthen their services is also crucial. Most key informants believe that SILC will continue after the Activity ends. However, a key informant from WCYA in Negele Arsi doubts that activities will be sustained and suggests involving technical committees in the Woreda to support this objective. Similarly, a FSTF informant mentioned that the livelihood groups are informal entities and may not function as they do today when the Activity ends. During the KIIs, the private sector actors expressed their commitment to support LG members. The agro-dealers expect to continue serving members, but the challenge is whether they will be profitable when the Activity is no longer available to support them. The FSPs also expect to continue their support after the Activity is phased out; however, they will need to ensure that default cases do not increase, which could otherwise cut into their profitability. The Activity intervention that is more likely to generate sustainable impact is the youth training on vocational education. The knowledge and skills they will obtain from the Activity should increase their employability, which could positively affect their long-term earning potential. The

39 challenge will be to see how to keep youth involved in the Activity, avail them access to credit, and stimulate them to contribute to value chains. Based on a few project resources, the ultimate goal of the Activity is to support the GoE graduation of 18,375 HHs out of 24,500 (75 percent of total HHs) to become self-reliant, although this is not reflected in the annual work plan. Through the focus group and key informant interviews, researchers learned that no discussions are taking place within the groups around graduation. To be eligible to graduate, the members have to meet certain criteria. The PSNP manual explicitly mentions that the key criteria for graduation is “HHs achieve food self-sufficiency in the absence of external support.'' The manual also suggests that the graduation should be evidence based and draw upon case-by-case assessments of a HH’s circumstances. This process should be participatory in nature. All these requirements require heavy preparation on the process. In general, the research team did not find members cognizant of graduation and its requirements, which begs that CRS and MCS relook at this target and gauge how to revise it. In Sire, FSTF planned to graduate 130 HHs in the coming year, but preparations are not adequate to make this happen. The PSNP manual states the key criteria for graduation is that “HHs achieve food self- sufficiency in the absence of external support,” and graduation should be evidence-based, drawing on case-by-case participatory assessments of a HH’s circumstances. Ultimately, the GoE determines graduation whereby FSTFs conduct a wealth ranking exercise to assess which PSNP clients should be put forward for graduation. The Activity should carry out this wealth ranking exercise for all of its PSNP clients to help it decide which members may be eligible for graduation. All these requirements require heavy preparation. It is urgent that implementing staff and participants understand the prerequisites of graduation and that the Activity team include graduation related activities in the plan. Furthermore, since only the GoE can officially move people out of the PSNP, the Activity should modify its language to concentrate more on raising HHs to “graduation readiness” rather than actually graduating them.

Modification in Activity Based on the findings, the evaluation team does not see need for major changes to the Activity’s Results Framework (RF). Nevertheless, some refinements could be conceptualized for the livelihood pathways and concerning targets. The Activity has three distinct pathways to support the safety net HHs to move out of poverty: on-farm, off-farm and employment activities. At the beginning, the Activity assumed that the on-farm activities may include 80 percent HHs, but in actuality it great to 88 percent. Likewise, the Activity assumed that the off-farm activities would include 15 percent HHs; however, it eventually became 11 percent. Similarly, the Activity assumed that 5 percent of them would engage in employment-related activities, but only 1 percent are represented in this category. All these changes should be reflected in the redesigned plan.

40

Table 4: Pathways and HHs Involvement At the Activity level, modifications will be needed Livelihood Expected Actual to improve the Activity Pathway Involvement Involvement effectiveness and Activity On-Far m 80% 88% outcomes. Some members in ATJK mentioned that they are Off-Far m 15% 11% overwhelmed by the volume Employment 5% 1% of trainings provided to them. Instead of repeatedly rendering trainings, the effectiveness of each training should be assessed afterwards, and resources should be invested in follow up as needed. In some groups, participants could not even recall the name or content of the trainings in which they participated. The Activity is at risk of not meeting its target for borrowing from FSPs. The Activity currently states a target of 13,000 members to borrow from an FSP by end of Activity, but to date only 1,111 members have borrowed from external sources. The Activity plan should revise the target to reflect what is possible. Another change that is needed in the Activity plan is to improve discussions around graduation processes within the groups. According to its factsheet, Livelihoods for Resilience-Oromia aims to graduate about 75 percent of its participating HHs in LGs (18,375 out of 24,500) by end of program; however, this is not reflected in its results framework or annual work plan. In general, the research team did not find members cognizant of graduation and its requirements. It is urgent to include graduation related activities in the plan and start communicating with the stakeholders and members about the prerequisites of graduation and the general process to achieve it. Also, since only the GoE can officially move people out of PSNP status, the Activity should concentrate more on raising HHs to “graduation readiness” rather than actually graduating them. There were not many tangible achievements under strategic objective three (SO3), which states that vulnerable HHs will increase their income from gainful employment. This is due in large to delays in implementing skills-based trainings and the poor availability of jobs locally. The target for life skills training to the youth and their linkages to potential employers may not be met. Amid these challenges, the Activity should consider setting more realistic targets for these interventions.

Financial Management By most accounts, the Activity appears to be spending its resources on schedule and in compliance with the terms of the award. As of the end of Y3 Q3, the Activity estimated spending amount was $4,775,964 of a $6,860,660 obligation (70 percent expended). Of the $1,132,775 Activity cost share (10 percent of USG obligation) for the life of the award, the estimated Activity expenditure as of June 30, 2019 is $420,318. This accounts for 37 percent of the total committed amount for

41 the cost share. This may be a little low given that the Activity is more than 50 percent complete, however future investments in Innovation grants which are being funded from cost share will likely make up for this.

42

Recommendations

The research team has assembled the following recommendations to improve implementation and delivery of the Activity on its expected outcomes. These recommendations have been categorized as critical or other recommendations. Critical recommendations link closely to input from Activity participants in addition to practices that the Pulte team has observed as furthering the effects for similar programs in East Africa. These should be taken as recommendations that bear the greatest return on investment (human or financial) based on the findings from participants. The other recommendations tie more closely to good practices observed by the Pulte team, and may not originate from suggestions by participants. That said, they are presented as ways to address issues or concerns raised directly by participants. Critical priority recommendations 1. Promote greater diversity in livelihood pathways adopted by individual HHs to bolster further HH level resilience. The Activity is based on PSNP guidelines for selecting livelihood pathways for safety net HHs. Approximately 88 percent of HHs have selected on-farm activities as their pathway for livelihoods support; however, increasingly agriculture has been challenged by market fluctuations and climate change. The net profit farmers receive from their growing practices has been shrinking. Crop-based value chains are more susceptible to price fluctuations than those of livestock are. Going forward, the Activity should promote greater diversification of revenue sources, encouraging a balance of livestock or other economic activities for HHs to complement their crop-based primary livelihood. Having diversified income streams can increase the resilience of HHs to shocks (market or climate related or otherwise) while still supporting the necessary food production element within Ethiopian society. Meeting the goals of the employment pathway will be challenging. Vocational trainings for youth will create new opportunities, and many youths during the FGDs expressed their willingness to participate in these trainings. Given the time remaining for this Activity, more work needs to be done to realize this objective, especially coordination with vocational training schools and prospective employers. It is also important to review how the youth who already graduated from such trainings are doing and if there is any need to revise the curriculum to fit better the realities of the labor market.

2. Communicate and disseminate trainings in order to fit better with group or individual HH needs and constraints. Trainings help transfer knowledge and skills, which members recognize. Nevertheless, some members complained that they participate in “training after training”, and that they are having a hard time recalling the names of all the trainings, content, and applications. Others complained that delays or cancellation of training without advanced notice have caused them to lose valuable workdays.

43

Instead of providing blanket trainings to all members, learning outcomes could improve if trainings are delivered based on need. Additional trainings should only be rendered after evaluating the effectiveness of prior trainings and other educational activities. In addition, representatives from the government should be accompanying CRS partner staff on monitoring and technical visits. There was an indication from several KIIs that local government officials are participating in trainings. These officials should receive the full range of training materials if they are not doing so as a way of scaling up the Activity after activities reach their end date. The Activity could also look at ways to second government staff to the Activity as a way of further embedding the Activity objectives, trainings and resources into government strategies while strengthening the capacity of government staff. 3. Improve consistency of communication across the board of Activity interventions and timelines. A number of respondents expressed concerns around what they were told they could expect from the Activity. Specifically, participants in Dotota, ATJK, and Negele Arsi understood they would receive goats and seeds or other inputs. Those in Dotota, Shala, and Sire were promised trainings that have not taken place or experienced delays. Across all the Activity areas interviewed, HHs assumed that they would benefit of benefitting from large loans with no interest and connection to FSPs that the Activity has not been able to deliver on. These expectations should be carefully managed with participants, as some respondents reported turning down other employment opportunities in order to capitalize on promises from the Activity that never materialized or materialized late with little updates by the Activity team. Better communication by the Activity will help participants look at the Activity more realistically and make informed decisions concerning in which trainings and events they participate. 4. Support agro-dealers more comprehensively around entry barriers while strengthening alternative supply options for farmers. Incentivizing agro-dealers to expand their businesses to rural areas could be effective only if dealers can find a way to shape the market to make such an expansion sustainable. Activity grants to businesses support costs associated with upgrading infrastructure, but these will not be enough to sustain businesses in rural areas if they cannot find a way to extend product or service offerings in a way that makes sense for them and for farmers. FGDs revealed that farmers may not even be aware of the existence of agro-dealers or the effects of their offerings like herbicides if they have been working only with farmers’ unions to date. Problems include the cost associated with holding inventory and the fact that they usually need to extend products to farmers on credit, which they may or may not make back. Usually, this is done sparingly to those farmers known by the agro-dealers, but it does pose a significant risk to these small businesses regardless. The Activity could support agro-dealers to better market their services and consider re-sizing their product offerings in smaller volume sizes as one way of meeting farmer needs while reducing their risk. For example, dealers could provide products in smaller volumes for quarter-acre plot sizes, which may be easier for

44

smallholder farmers to afford given cash flow constraints, while increasing margins slightly for the dealers. This could result in smaller inventory needs and increased profits, which may help agro-dealers feel more comfortable with the assumed risk of expansion into rural areas. The Activity could also support agro-dealers with a market assessment, so they understand better the market potential, inclusive of number of potential customers, their service needs, willingness to pay, and entry or exit barriers. Also, group members suggested working with farmers’ unions instead of private agro- dealers, as farmers’ unions are trusted and more stable. Study participants noted that they mostly work with these unions to procure inputs such as seeds and fertilizer. This may entail increasing their understanding of how specific product offerings (i.e. agricultural inputs) could support farming efforts. 5. Better assess client readiness, business skills, and loan recovery before accessing loans; and manage expectations of participants around availability and conditions of financing options. Linking members to external financial services is a sustainable solution to meet members’ long-term financial needs, but there is widespread confusion and disagreement among members about this effort. Activity staff should provide more information around group saving requirements, the amount members can expect to access from FSPs, the expected interest rate, and repayment periods. Conversely, FSPs are concerned that borrowers will not pay back loans on time. FSPs could offer incentives in the form of interest waiver for those who pay ahead of the maturity date to encourage borrowers to pay early. Further, the provision of a guarantee fund also mitigates FSPs’ concern on borrowers’ default to some extent, however these should to be applied strategically and in ways that can improve the credit histories of individuals or communities at large. In the Woredas where the LGs started working with FSPs, members are experiencing problems receiving loans on time. This is linked in part to some not having a business plan, which should be prepared with the help of DAs in collaboration with the Woreda agriculture office. FSPs will not provide a loan without a business plan, which meant that some LG members could not qualify for loans, despite participating in trainings on loan management alongside saving and credit activities provided by the Activity. In response, the Activity should offer staff to assist members with the preparation of qualified business plans as quickly as possible and in tandem with future trainings. Members expressed concern around the loan amounts provided by FSPs. In some Woredas, members were able to access 3,000 ETB only, with the exception of a few who received 6,000 ETB. This in turn is linked to poor savings being reported by members. In response, the Activity team should continue working with SILC and HHs to incentivize savings and confirm this link between savings and FSP loan amounts to help manage member expectations.

45

The Activity could also help HHs improve their credit profiles by integrating a credit log for SILC members that records loans and payback rates. Over time, this could serve as a record of borrowing history and assist FSP to value properly the risk associated with granting loans to individuals. Accurate credit histories could have a powerful effect in overcoming the credit stigma associated with past borrowing and could qualify current participants who can demonstrate responsible borrowing alongside their business plans for larger loan amounts from either FSP or other formal lending options.

6. Push boundaries further to promote greater gender equity in uptake of Activity interventions. The Activity has included many actions to address gender inequality issues within the Activity area. In all the group trainings, the participation of women was equal to that of men; however, in some groups the women expressed that they were not treated equally. While both males and females have been working on various value chains, it was found that males are more likely to lead those value chain interventions. There is an opportunity for the Activity to explore this further and better understand what is inhibiting women’s full and equal participation. While not all of these factors may be within the Activity’s control, the Activity team should increase efforts to affect those factors that are. 7. Improve communication to youth of opportunities from the Activity and conditions. Youths reported problems that fall into the following three camps: a) securing money for building regular savings, b) securing loans from FSPs, and c) participating in vocational trainings. Youth depend on parents for saving money, and often cannot pull together money for group savings without parental support. This emerged in particular for female youths. Given this, it would be worth testing in some areas if youth engagement would improve if they were combined into mixed groups rather than forming separate YLGs. Developing an evidence base around this would be important in order to affect GoE PSNP policy; however, a test would need to consider the effect on both male and female youths separately. It should also review which youth age segments fare better from a LG or YLG structure. Further, youths feel that they are waiting for a long time to receive promised trainings. Some youths were under the impression they would get the training opportunities if they agreed to quit their jobs in the cities and returned to their villages to join the LG groups. The Activity should communicate more effectively with youths around training plans to help them manage their timelines and expectations, and then monitor closely the progress made on vocational trainings for youth. 8. Increase understanding among Activity staff and LG members of graduation requirements and strategy to achieve graduation readiness. The ultimate aim of the Activity is to support the GoE in graduating 18,375 HHs out of 24,500 HHs who are LG members from PSNP status. This constitutes 75 percent of total Activity HHs. Despite this, researchers learned that groups are not discussing graduation, and that neither participants nor Activity team members are clear on the criteria for graduation. Given the time remaining for the Activity

46

and in the absence of a clear graduation strategy, this target will be difficult to meet and needs to be prioritized. This will be particularly challenging due to the drought and other climatic factors in some areas, which will make it more difficult for some HHs to pass the graduation threshold. Activity management should discuss this with all members of the Activity team and key stakeholders to ensure that all have a clear understanding of what graduation means and define a clear strategy for reaching this target for “graduation readiness” rather than actual graduation, given that only the GoE can officially graduate PSNP HHs. This will be critical in helping group members to plan.

Other recommendations: 1. Dedicate more time of Activity staff to follow up and mentoring of HHs as they initiate value chain interventions. FGD participants as well as key informants voiced a desire for more follow-up visits from the Activity field staff. This might pose an extra burden for the Activity but making frequent visits to the LG meeting and other activities would be helpful to maintain motivation levels. This should not be as much of an issue now that the Activity has invested in a new team of 143 MTEWs. 2. Incentivize innovations that can assist HHs and communities in navigating inefficient market dynamics, such as crop price fluctuations. Innovations have the greatest chance of success in being sustained and scaled up if they can assist LGs prosper amidst unfavorable market dynamics. One important area that was not included is post-harvest storage. Many respondents complained about the effect of price fluctuations in the market for their crops. Effective post-harvest storage options could enable them to wait and sell their crops during off-peak periods when prices may not be as depressed. Participants also expressed issues concerning financial support for farmer participation in innovation fund activities. The Activity and innovation fund grantees should clarify all requirements and expectations clearly before commencing with research and development activities in farmers’ fields. 3. Collaborate more closely and frequently with government (especially Kebele level structures), given high turnover of government officials. The existing collaborations between the Activity and other organizations and government was found to be effective overall. Government counterparts are mostly happy with the collaboration; however, a few believe that the Activity did not involve them adequately. Going forward it will be important to review present levels of collaboration and ensure that the Activity is reaching out to stakeholders on a recurring basis, particularly given the turnover of officials. 4. Coordinate better with CRS DFSA livelihood transfers and Diner Fairs to maximize investments and influence provision of inputs relevant to Livelihoods Resilience – Oromia Activity value chains. There are more opportunities for the various USG-funded activities to leverage each other’s strengths within overlap areas. Interventions such as livelihood

47

transfers and DiNER Fairs offer an opportunity to link HHs with resources that could support the uptake of value chain, particularly in light of the dearth of financing options. 5. Leverage seasonality of income when engaging LG savings groups. As the Activity works with LGs to expand the savings culture, it would help groups to consider months when their revenue stream may be greater than other months and that would allow them to save more. This could include the post-harvest season for farmers, adjusting for repayment of any loans they may have needed to draw on during the planting season, or after selling livestock. Assisting groups to enhance their savings levels by exploiting market seasonality will enable them to increase their corpuses and then extend bigger loans or qualify for more financing from FSPs.

48

Annexes

Annex 1: Midterm Evaluation ToR

Feed the Future Ethiopia Livelihoods for Resilience – Oromia Catholic Relief Services Ethiopia Country Activity July 03, 2018 V1.0

Document Change History VERSION NUMBER DATE DESCRIPTION V1.0 July 03, 2018 Initial draft submitted to USAID V1.1 October 29, 2018 Revised ToR submitted to USAID V1.2 November 6, 2018 Second revision of the ToR submitted to USAID

I. INTRODUCTION

I.A. Midterm Evaluation Overview Feed the Future Ethiopia Livelihoods for Resilience - Oromia is a five-year USAID funded activity started on February 01, 2017. The activity intends to support livelihood options of chronically food insecure HHs in Oromia region. With the aim of assessing progress of the activity and to suggest future directions to better achieve activity’s objectives, CRS has planned to conduct an internal MTE. The evaluation will mainly employ qualitative methods such as FGDs, KIIs and site and activity observations to collect information from participants and key stakeholders. The evaluation will also review routine monitoring data, partner and donor reports and other Activity documents. The annual outcome indicator data from year2 Annual Participant Based Survey (ABBS) shall be available for use at MTE. The MTE will be led by an external consultant. The Livelihoods for Resilience – Oromia activity Chief of party and Monitoring, Evaluation, Accountability and Learning (MEAL) manager, the CRS Ethiopia country Activity MEAL unit and regional MEAL advisors will assist in the process. The implementing partner will also involve through information provision and facilitation of deliverables of this mid-term evaluation.

49

I.B. Background: CRS and Implementing Partner (MCS) CRS/Ethiopia has been engaged in various development and humanitarian services for over fifty years now with policies and strategies ranging from emergency and relief to development. This has been a long history of achievement, challenges and lessons learned in impacting on lives of so many Ethiopians who have lived in different conditions and levels of poverty. Implementing partner of this activity is Ethiopian Catholic Church Social & Development Commission of Meki (commonly referred to as MCS). MCS has been working in the Activity area since 1995 focusing on food security, education, water and sanitation, cooperative promotion, emergency relief and recovery, peace building, migrant support, and rural and urban women’s support. I.C. Background: Activity Goal and Objectives The goal of Livelihoods for Resilience-Oromia activity is to improve the sustainable economic well-being of Productive Safety Net Activity (PSNP) HHs by building on the Push and Pull strategy of USAID’s Feed the Future, the PSNP Livelihoods framework, and the success achieved under the past Feed the Future Ethiopia GRAD Activity.

The activities conducted under Livelihoods for Resilience – Oromia will be consistent with the Results Framework (RF) below;

GOAL: PSNP HHs have improved their sustainable economic well-being Strategic Objective 1: Vulnerable HHs have increased income and diversification through on-farm opportunities including crop and livestock market systems IR 1.1: Women and youth have increased access to and control of HH resources IR 1.2: Smallholder men, women and youth have increased use of improved agricultural techniques and technologies IR 1.3: Smallholder men, women and youth have increased use of financial services IR 1.4: Smallholder men, women and youth have increased engagement with agricultural commodities in markets Strategic Objective 2: Vulnerable HHs have increased income and diversification through off-farm livelihood options IR 2.1: Men, women and youth engage in off-farm livelihoods activities IR 2.2: Men, women and youth have increased use of financial and business development Services IR 2.3: Men, women and youth have increased diversification of off-farm IGAs and nonfarm Enterprises Strategic Objective 3: Vulnerable HHs have increased their income through gainful Employment IR 3.1: Jobseekers adopt increased use of employment knowledge, skills and information for job-seeking IR 3.2: Job seekers are linked with prospective employers Strategic Objective 4: Livelihoods for Resilience - Oromia and other USAID Ethiopia and GoE interventions have increased innovation, scaling and sustainability of livelihood pathways

50

IR 4.1: Livelihoods for Resilience - Oromia, USAID Ethiopia and GoE have improved layering, sequencing and cost effectiveness of interventions IR 4.2: Livelihoods for Resilience - Oromia continuously identifies and addresses knowledge gaps, successes and failures important to the Activity IR 4.3: Successful organizations effectively utilize innovation grant for sustained innovation, scaling and sustainability of livelihood pathways.

The Feed the Future Livelihoods for Resilience - Oromia activity promotes a demand-driven agricultural extension approach that links private sector actors such as agro dealers with DAs and real-time market information. The strategic objectives, and the intermediate results and activities will collectively create the Push and Pull elements necessary to generate livelihoods and employment opportunities and increased integration with wider markets that will support graduation of HHs from PSNP. The activity will operate on both sides of the push-pull equation, equipping clients for basic market entry, including financial literacy, access to savings, and business planning and skills, while also focusing on higher-end access to markets, strengthening of the formal financial sector and business support services and encouraging private sector innovations that will catalyze wider market development. Livelihood Groups (LGs) are primary entry point to deliver the activity’s interventions. All members of LGs are PSNP HHs who self-select and established a group of 20-30 members. Men and women PSNP clients are member of LGs. LG members receive various skill trainings to enable them to be more resilient to shocks arising from poor nutrition and health, climate change and market fluctuations. Catholic Relief Services (CRS), in consortium with the Ethiopian Catholic Church Social & Development Commission of Meki (commonly referred to as MCS) implements the activity in nine Woredas across three zones in Oromia region: namely, Zeway Dugda, Negele Arsi, Heben Arsi, Shala, ATJK, Dodota, Sire, Siraro, and Boset. The activity will assist 24,500 PSNP HHs with a goal to successfully graduate 18,375 PSNP HHs.

Feed the Future Ethiopia Livelihoods for Resilience – Oromia and Food for Peace (FFP) Title II Development Food Assistance Activity (DFSA) activities overlap in 4 Woredas of MCS implementation area: Namely, Shala, Negele Arsi, Heben Arsi and Zeway Dugda. In addition, Livelihoods for Resilience - Oromia overlaps with WV’s DFSA in Siraro Woreda. In the overlap Woredas, Livelihoods for Resilience – Oromia and both CRS and WV DFSAs implement livelihood component of the PSNP in collaboration.

II. PURPOSE OF THE EVALUATION

II.A. Purpose of the Midterm Evaluation The midterm evaluation is an important part of the activity’s MEAL plan, enabling CRS to assess progress of the activity and reorient Activity strategies and activities as necessary.

51

II.B. Key Audiences and Uses

CRS, implementing partner (MCS) and USAID will play active role in the process of this MTE and will be the primary users of data and information generated. The MTE results will be used to measure progress of the activity under implementation and suggest where and how implementation can be improved in the remaining Activity time-period. III. Evaluation QUESTIONS / OBJECTIVES

III.A. Objectives The MTE will review quality of implementation of the activity in producing planned outputs and outcomes, to assess the intended and unintended effects of these outputs, and to examine the progress to formulate recommendations to be implemented in the remaining life of the activity. Specific objectives; 1. To assess relevance of strategies and interventions in addressing the needs of targeted groups 2. To evaluate the quality and effectiveness of service delivery, the strengths and weaknesses of activity implementation and management, and the technical quality of activity inputs and outputs in terms of adherence to terms agreed to by USAID and of their acceptability and perceived value to target communities, identifying factors that appear to enhance to detract from the quality, acceptability and usefulness of implementation and outputs. 3. To present evidence of changes (intended and unintended, positive and negative) associated with activity interventions, assess how well the observed changes reflect the RF 4. To recommend adjustments to RF, activity design, resource allocation, activity implementation and management, and M&E Plan that could improve the likelihood of achieving desired results by the activity’s end, based on the evidence collected and conclusions drawn for the evaluation 5. Determine the extent to which outcomes, systems, and services are designed and being implemented to continue after the Activity ends and assess progress made on implementing sustainability strategies.

III.B. Key questions To address the above objectives, the MTE will focus on the following key questions: 1. How well have the activity’s interventions met planned schedules, participant numbers, and Outputs? What factors promoted or inhibited adherence to schedules? How were problems and challenges managed? 2. What are the strengths of and challenges to the overall activity design, implementation, management, communication, and collaboration (with other USAID grants and government

52

interventions) so far? What factors appear to promote or challenge the activity operations or effective collaboration and cooperation among the various stakeholders? 3. What are the strengths of and challenges to the efficiency of interventions’ implementation? What factors in the implementation and context are associated with greater or lesser efficiency in producing outputs of higher quality? Which interventions and implementation processes are more or less acceptable to members of the target communities and why? What are strengths and weaknesses of measures taken to ensure gender equity regarding access to, participation in, and benefit from activity interventions 4. What changes—expected and unexpected, positive and negative—do community members and other stakeholders associate with the activity’s interventions? What factors appear to promote and deter the changes? How do the changes correspond to those hypothesized by the activity’s RF? 5. How could the activity be modified to improve its acceptability to targeted communities or the efficiency and effectiveness of its implementation? How should the activity’s RF be refined or modified? 6. Which Activity outcomes (services, goods or structures) are likely to be sustained after the Activity end? Who will help to sustain Activity outcomes? IV. METHODOLOGY

IV.A. Approach Qualitative data collection approach and tools will be employed to collect primary data in response to objectives of this evaluation. There will also be review of Activity documents (quarter reports, monitoring data and annual participant-based survey results). The evaluation will exert maximum effort to ensure the data and information collected meets high quality standards. IV.B. Sources of Data and Data Collection Methods The midterm evaluation will employ qualitative data collection methods. Data will be collected both from primary and secondary sources.

Primary data will entail the following;

• Conduct focus group interviews with men, women and youth Activity participants (both HH and institutional) to determine Activity benefits and participants, service delivery quality, perception of the communities of services delivered, unanticipated consequences and possible areas of modification and design of the activity to ensure efficient and effective Activity delivery • Conduct key informant interviews with Activity staff and key stakeholders, including USAID, GoE local government officials, other INGOs, selected men, women and youth participants, the PSNP food security task forces at Kebele and Woreda level, market actors including private business sector partners

53

• Direct observations of learning sessions, LG activities and implementation offices through site visits to assess technical practices and quality of outputs, verify recorded outputs and assess the likelihood of achieving outcomes considering the outputs delivered and other contextual factors. Secondary data will be collected from; • Review of Activity records to establish outputs delivery and assess quality of MEAL system. These include Activity performance reports, annual survey report and reports of assessments and barrier analyses • Review of Activity and organizational documents to assess the quality and effectiveness and institutional strengthening activities. These include design documents (proposal, RF and logical framework), CLA Plan and CRS and WV DFSAs and ACTIVITY collaboration papers. The contractor may make request other documents they deem important for the evaluation. • At the discretion of the MTE team, they may decide to conduct analysis of secondary data from annual survey and the routine monitoring system The activity will produce and provide the above-mentioned documents and other key information as needed. The consultant will purposively select sample of implementation sites for primary data collection. The method will help the Activity team to actively participate and make suggestions if the MTE consultant’s proposals do not represent the overall Activity. The consultant will visit at least six implementation Woredas and number of Kebeles or community groups for site visits will be suggested later by the consultant. IV.C. Data Analysis Procedures Planning for data analysis helps to ensure that data collection instruments yield the information needed. Therefore, for this evaluation the collected qualitative data will be analyzed using qualitative data analysis matrix that summarizes the main findings of qualitative data using selected, pre-determined themes as per the objectives of this midterm evaluation. Gender and youth will remain as a major disaggregation requirement when analyzing results in this evaluation. V. MIDTERM EVALUATION TEAM

The MTE team will be led by a consultant who will be working closely by a mix technical leads in the areas of livelihood and markets, youth, gender and rural financial services. The MTE Team Lead will lead the process of ensuring qualified local interviewers are available to support in the field data collection process. The team lead will be responsible to provide a detailed proposal suggesting the required number of team members at various levels in consultation with CRS Livelihoods for Resilience-Oromia Chief of Party and MEAL manager. The consultant should also provide interviewers who are proficient in Afan Oromo.

54

The technical team lead and technical leads will form the Core team of the MTE. The Core team have substantial responsibility in the design and execution of the evaluation. They will participate in the full evaluation process. CRS/ET responsibilities - Provision of secondary data and other information including Activity implementation sites to help the MTE consultant to purposively sample activity site for primary data collection - Review important documents of this MTE (Inception report, data collection tools and reports at various stages) - Follow up on the midterm evaluation schedule Consultant responsibilities - Provide MTE plan/inception report including detailed implementation plan that specifies details for methodology and sampling of respondents for FGDs, KIIs and Site/activity observations, critical tasks, anticipated outputs, date-bound timelines, resource needs, and responsible person(s). - Develop data collection instruments and interviewer instructions/manual - Provide details of composition of a standard MTE team with assigned responsibilities of each team member (see Annex I for detail Annex I MTE team composition, qualifications and role) - Present key findings, conclusions, lessons learned and recommendation to CRS and stakeholders - Provide draft MTE for comment by CRS - Provide final MTE report that is approved by CRS: A final MTE report should be in English reviewed and approved by CRS Livelihoods for Resilience-Oromia CoP. The MTE report must clearly separate in different sections the evidence (i.e., raw data) collected by the evaluation team, the conclusions and recommendations that are based on the presented evidence. It should also state that sources of all evidence must be identified; conclusions must be based only on evidence presented in the report; and recommendations must directly correspond to the conclusions. In line with the requirements of the USAID’s evaluation policy, the report must describe the strengths and limitations of the evaluation methods and how and to what degree these factors influenced the process and findings of the evaluation. Once approved by CRS, the consultant must submit the report, supporting documents, and related data to CRS in time per agreement.

VI.REPORTING AND DISSEMINATION PLAN

VI.A. Midterm Evaluation Report The following is a proposed outline of the midterm evaluation report: • Cover page – contain Activity title, report title, Date of the Report and Logos of Feed The Future, USAID, CRS Ethiopia and the MCS

55

• List of abbreviations and Acronyms • Executive summary • Table of contents • List of tables and figures (if applicable) 1. Back ground 1.1 Overview of Activity strategies 1.2 Activity history and operating context 2. MTE purpose and objectives 2.1 MTE methodology, including sampling 3. MTE findings and discussion 3.1 Brief description of interventions 3.2 Service delivery strategies and approaches: quality, successes and challenges 3.3 Implementation progress and achievement of results 3.4 Meeting targets 3.5 Other achievements 3.6 Lessons learned and promising practices 3.7 Activity quality and cross-cutting areas 3.7.1 partnership 3.7.2 targeting 3.7.3 sustainability/exit strategies 3.8 Implementation processes 3.8.1 Monitoring and Evaluation 3.8.2 Collaboration learning and Adapting 3.8.3 Participant Accountability 3.8.4 General management Financial management Human resource management 4. Recommendations 4.1 Critical priority recommendations 4.2 Other recommendations 5. Annexes 5.1 MTE Scope of Work (SoW) 5.2 MTE plan and schedule 5.3 MTE methods 5.4 MTE tools (questionnaires, checklists, and discussion guides, etc...) 5.5 List of sites visited 5.6 List of key informants and communities visited 5.7 List of documents reviewed and other data sources 5.8 Signed disclosure of conflict of interest by evaluation team members

56

VI.B. Dissemination Plan

The initial report of this midterm evaluation will be prepared by the consultant and reviewed by technical leads at CRS and implementing partner level. After finalization and approval of the MTE report, key findings will be shared with CRS and IP Activity team, government sector offices participated in the evaluation, USAID and other relevant stakeholders that has interest on the PSNP Activity. For dissemination of results to these audiences a one-day workshop will be organized both at CRS and partner level. The report will also be uploaded on GATEWAY (CRS share point). In addition, CRS will make the final MTE report publicly available through the Development Experience Clearinghouse within three months of the evaluation’s conclusion. VII. SCHEDULE AND LOGISTICS

Final report of the MTE is expected to arrive at least by end of August 30, 2019. Activity Chief of party and MEAL manager at CRS level will maintain progress of the midterm evaluation schedule. VIII. DELIVERABLES AND TIMELINE

The following calendar is illustrative for completion of the work;

Deliverables When/Deadline Pre - Planning 1. Develop initial ToR June 2018 2. Submit ToR to USAID June 2018 3. Finalize initial ToR based on USAID review October 2018 4. Advertise TOR and hire consultant December 2018 - February 2019 5. Organize all documents and make them available to the MTE March 2019 team Planning 1. Review existing reports, documents and data March 2019 2. Develop and finalize MTE Plan March2019 3. Select sample communities/villages. March 2019 4. Develop tools (FGD guides, KII semi-structured April 2019 questionnaires, and site/activity observation checklists) Implementation 1. Train qualitative interviewers and field preparation April-May 2019 2. Conduct the field work (i.e., interviews, focus group discussion June 2019 and observations)

57

3. Present the preliminary observations to validate findings and July2019 interpretations (Validation workshop). 4. Revisit sites if there are major disagreements on the findings July 2019 5. Present detailed results, including preliminary July 2019 recommendations 6. Make a presentation of key findings and implications July 2019 Reporting 1. Prepare a draft report based on SoW guidance and share with July 2019 CRS 2. Address comments and incorporate inputs from stakeholders July 2019 3. Provide final MTE report July 2019 4. Approve final MTE report by CRS CR July 2019 5. Upload final and approved report on GATEWAY July-August 2019 6. Organize dissemination workshops at CRS and MCS level July – August 2019

IX. BUDGET

Feed the future Ethiopia Livelihoods for Resilience-Oromia, will cover all costs associated with this MTE.

X. ETHICAL CONSIDERATIONS

All data collection processes will be handled in line with ethical standards. The evaluation will take into account the informant’s demand for respect, integrity and dignity. Data collectors will be trained to be sensitive to the informants’ responses on themes and topics introduced during FGD sessions or KII.

All data collection procedures will be opened by introducing the data collectors to the informants by name and whom they are representing. Evaluation participants/respondents will also receive an explanation of the purpose of the evaluation and only participants who have formally consented and agreed to participation will be enrolled and interviewed.

The implementing partner (MCS) will prepare and provide a letter of cooperation to the data collection team before starting the field work.

XI. SUBMISSION OF PROPOSALS

All proposals should be submitted in line with this ToR in soft copies to CRS via ______(email address). Proposal should be submitted until ______(date) and any proposal received after the deadline will not be considered.

58

All proposals should be submitted per the following outline and based on information provided in this ToR; a) Background b) Workplan c) Detail MTE methodology, including sampling d) Training e) Field team with responsibilities f) Quality control mechanisms during data collection g) Data processing plan h) Data analysis and report writing i) Contractor division of labor based on the following template Name Sampling Tool Training Data Data Data Report plan development collection cleaning analysis writing

j) Contractor expertise and experience k) Progress updates – how update will be communicated to CRS l) Detailed budget for o Daily rate of key professionals o Travel costs o Field expenses (payment of field staff, travel cost of field staff and other costs) o Data cleaning, analysis and report writing o Other costs (if applicable)

59

Annex 2: MTE Team Composition, Qualifications, and Roles

The MTE team will be led by a consultant who will be working closely by a mix technical leads. The MTE Team Lead will lead the process of ensuring qualified local interviewers are available to support in the field data collection process. The team lead will be responsible to provide a detailed proposal suggesting the required number of team members at various levels in consultation with CRS Livelihoods for Resilience-Oromia Chief of Party and MEAL manager. The consultant should also provide interviewers who are proficient in Afan Oromo. The technical team lead and technical leads will form the Core team of the MTE. The Core team have substantial responsibility in the design and execution the evaluation. They will participate in the full evaluation process. Below is list of team members by position and responsibility that is required for this MTE; 1. Team leader The team leader must have the following qualifications • Formal education from a recognized university in a field relevant to evaluation (e.g., Activity evaluation, statistics, economics, agricultural economics, applied research or sociology) at a post-graduate or an evaluation professional continuing-education level. • 10 plus years extensive experience in evaluation using mixed methods of investigation (qualitative and quantitative) in developing countries. Experience of evaluating livelihood and economic well-being activities is highly desirable • Demonstrated strengths in organizing and leading qualitative evaluation teams, and communicating clearly and concisely • Excellent interpersonal, presentation, and communication skills, and a demonstrated ability to deliver a high-quality product 2. Technical Leads For this MTE, technical leads are required from the areas of livelihood and markets, youth, gender and rural financial services. The technical leads of these areas must have the following qualifications;

• A Masters or PhD in their respective sectors • Extensive (8+ years) experience using mixed methods (qualitative and quantitative) to evaluate development activities in developing countries • Multiple years of experience in applying his/her field of expertise to Activity design and oversight or previous experience evaluating activities in his/her field in developing countries (Ethiopian context preferred) • The following are the minimum specific qualifications of the technical sector leads • At least one member of the team must have substantial demonstrated experience in gender and youth integration

60

• The MTE team should comprise technical expertise from all activity sectors and activity management • No member of the MTE team should have had any prior input to the activity’s design or implementation 3. Field Interviewers • S/he must have, at minimum, a graduate degree with 5 years’ experience in a relevant field • S/he must have practical experience and conceptual clarity on qualitative data collection and analysis, team management and planning, ability to resolve field problems, persuasiveness, and communication skills • S/he should have good problem identification and writing skills (English and Afaan Oromoo) • S/he should be a good team player with strong leadership ability to uphold team spirit and ability to work under pressure/hardship, respect for teammates and Activity participants • A good listener and possess strong qualitative information interviewing skills • S/he should be familiar with regional/local context and culture

61

Annex 3: FGD Guides and Questions

Purpose: The purpose of this evaluation is to measure progress against stated goals and objectives, including the effectiveness, efficiency and timeliness of Activity interventions in achieving targets.

Procedures

1. The FGD is conducted by a facilitator with an assistance of a scribe (person who takes note and records the conversation). 2. The participants in the focus group will be the male members of a LG for a Male FGD, female members of a LG for a Women FGD, and youth (15-29 yrs) for a Youth FGD. 3. The LG for FGD will be selected randomly for each category. 4. There will be 8 to 10 participants in a FGD, who are the Activity participants. 5. A FGD will take for maximum of 2 hours. 6. The FGD will be conducted in local language. 7. The note taker will prepare a detailed report written in English 8. An attendance list has to be also prepared before starting the meeting.

Expected outputs from each Woreda/site: 1. A minimum of 4 FGDs (both adults and youth of both sexes) detailed report written in English 2. A minimum of 5 key informant interviews in audio and summary report

Discussion Guide:

Introduction • Thank you for coming – we are grateful for your time. • We are holding this discussion to assess the progress of Activity implementation and results toward the intended outcomes of the Activity. It will assess service delivery and outputs as outlined in the detailed implementation plan and Activity agreement. • Participation in this activity is voluntary. • The discussion will take about 2 hours. • Encourage everyone to participate in the discussion. • Consent: My name is [NAME] and I am working with University of Notre Dame research team and supporting them to collect information about the Activity. We are gathering information to review the quality of the implementation of the Activity in producing planned outputs and outcomes.

If you agree to participate in this interview, we will talk about your group, your HH, and your involvement in this Activity.

We will do everything in our power to protect the confidentiality of your responses. Your answers will be grouped with the answers of other people and your name will not be used. This research Activity may include tape- recording discussions or taking photographs during the discussion for use in our reports. If you do not wish to be photographed, please tell us and we will ensure that this does not happen.

62

There are no known risks associated with participating in this activity. You are free to ask questions at any time. You may withdraw from participation without penalty, but we would appreciate it if you can stay until the end.

** Set ground rules such as turning off the cell phones; provide opportunity to speak to other. As a first step, the facilitator and note taker will introduce themselves. Then they will ask the participants if they can tell their name, what they do for living, time lived in the community, and their experience with Activity.

Discussion questions:

1. What are the livelihoods sources for people in this village?

2. Are there any changes in the livelihoods (types) of people in the last three years in your villages? 2.1 Please tell me the types of occupations people have. 2.2 What about the creation of new employment, carrying out new activities, more income, etc.

3. What is the name of your group/s?

4. When did you form your livelihood group? What are the activities you are carrying out under the Activity? 4.1 Explain how long the group has been in existence. 4.2 Who makes up the leadership committee and how is this decided? Their roles? Term of services? 4.3 Tell us more about the process of forming the group. 4.4 Who are included in the groups (all HHs, only PSNP eligible families)? 4.5 Types of activities the group does? How the members are involved? What they do after joining the group? 4.6 How frequently do you meet? 4.7 How much do individuals contribute towards savings at each meeting?

5 What are the strengths of and challenges to the Activity’s implementation? 5.1 How do you feel about how the Activity is implemented/service is delivered? 5.2 What are the strengths and challenges of carrying out the activities? 5.3 What aspects of the Activity do you like most? Why? Please give an example. 5.4 What types of challenges (within the HHs, within the community/group, and external) do the members face with regards to successful implementation of the Activity activities? Please give an example.

6 How useful are the Activity activities in addressing your needs? 6.1 How quickly and efficiently are Activity activities organized? 6.2 Did you receive a schedule of Activity activities? Are activities organized according to the schedule? 6.3 Were those Activity activities able to address your needs?

63

7 What barriers or benefits, if any, do women face in participating in the Activity? 7.1 Any requirement for a husband’s approval, for example, in membership, meetings, savings? 7.2 Any time constraints or distance to travel in order to attend group meetings? 7.3 Any financial burdens if there is a payment structure? 7.4 Are there any special benefits women receive from the Activity?

8. Has the time you spent in involvement been a burden to you or affected your other activities? Explain how?

9. What barriers, if any, do young people face in participating in the Activity? 9.1 In collateral for loans, savings… 9.2 Any difference for male and female youth? If yes, how? 9.3 What about obstacles for really young youths (15-17) in accessing financial resources because of their age?

10. Have you been informed of the graduation criteria and the responsible body for participants and stakeholders including ACTIVITY? If not, why not?

11. How has the Activity contributed to the graduation process for PSNP participants? 11.1 Are you hoping to graduate from the PSNP? When and what do you need to graduate?

12. How did the Activity affect your access to finance/credit, if at all? 12.1 Tell us how the LG runs its financial resources? Does it ask for interest rate, loan, saving, etc. 12.2 Any contribution members make to social fund? 12.3 Do all members accept the type of financial /credit arrangement? Ask for the amount of interest rate and religious issues? 12.4 How would you describe your financial resources if you wanted to start a new business?

12.5 Has your access to required financial resources changed after joining the group? How? 12.6 Do you have the necessary resources (skills, options for securing loans from different sources, SILC and/or your own internal savings) to start a business? 12.7 Are there any changes in availability of financial resources after joining the Activity?

13. What support systems are available to assist with setting up a business? 13.1 Please tell us more about the availability of the input market, training, buyers for the produce, transport, your ability to negotiate with vendors, etc.? 13.2 Are you working with agro-dealers? If yes, please explain? 13.3 What support do they provide you with? 13.4 What are the shortcomings of the agro-dealer services? Any impact of seasonal price fluctuations? 13.5 Do you have extension services available locally? If yes, who provides it? Is there a cost for securing these services? 13.6 Do you have an opportunity to receive training on running a business?

64

14. What are your major products? 14.1 Where and to whom do you sell? 14.2 Do you have any market problems? 14.3 What support do you need in this regard?

15. What is the overall impact of the Activity for participants? 15.1 What types of changes can you see in the community after the intervention of the Activity? 15.2 Did the Activity enable you to provide more opportunities in the community? If yes, how? What type of opportunities? If no, why not?

16. Have there been any changes in the last two and half years that you did not expect? 16.1 What are the changes you observe emerging in your community? Please explain by giving some examples. 16.2 Are any of these changes related to the Activity interventions? How?

17. What factors appear to promote these changes? How do the changes correspond to benefits perceived or expected before joining the groups? 17.1 Before joining the group, what was your expectation from the Activity? 17.2 Do you think your expectations are met? If no, Why? If yes, how? 17.2.1 Do you think your expectations were realistic when you get to learn about the scope of the Activity? 17.3 What are the changes you find in your HHs and community because of this Activity? 17.4 If you find changes in your life, what factor(s) brought those changes? Are they related to the Activity intervention? If no, why not?

18. What factors appear to deter these changes? How do the changes correspond to those you expected before joining the groups? 18.1 What are the challenges to maintain any changes?

19. How could the Activity be modified to improve its acceptability and enhance its benefits to targeted HHs? 19.1 Are there any activities that you as participants do not like or want to include? 19.2 Do you want to modify any approach the Activity is currently employing for better results? 19.3 If you could make any changes, what would they be and why?

20. Imagine that the Activity ended tomorrow. Which Activity outcomes (services, goods or structures) are more likely to be sustained? 20.1 Which Activity outcomes do you expect will be continued? And, why? 20.2 Who will help to sustain Activity outcomes? 20.3 What are factors that promote continued flow of outputs after closing the Activity, beyond 2022 (after 2 and half years)? 20.4 Would you consider continuing within your group after the closure of the Activity? If yes, how?

65

21. Now we have finished our questions. Do you have anything else you would like to raise related to the Activity?

Closing: Thank you for your participation!

66

Annex 4: KII Guides and Questions

Respondents: The Activity implementers (CRS and MCS staff), social protection Activity (PSNP) head, the agriculture office head, the Women, Child and Youth Affairs office head.

Total number: 28-30 (22-24 from six Woredas and six from CRS and MCS) Time: approximately 45 minutes per interview

Based on the key research questions, the following question will be asked during the face-to-face interview.

Consent: My name is [NAME] and I am working with University of Notre Dame survey team and supporting them to collect information about the Activity. We are gathering information to review the quality of the implementation of the Activity in producing planned outputs and outcomes.

If you agree to participate in this interview, we will talk about Activity: Activity activities, implementation, challenges, outcomes, and suggestions for improvement.

We will do everything in our power to protect the confidentiality of your responses. Your answers will be grouped with the answers of other people and your name will not be used. This research Activity may include tape-recording discussions or taking photographs during the discussion for use in our reports. If you do not wish to be photographed, please tell us and we will ensure that this does not happen.

There are no known risks associated with participating in this activity. You are free to ask questions at any time. You may withdraw from participation without penalty, but we would appreciate it if you can stay until the end.

Do you agree for the interview? Proceed when they agree.

1. Implementers: Activity staff (CRS and MCS staff)

1. What are the main livelihood sources for people in this Woreda? 2. When did the Activity start implementing in this Woreda? 3. What activities is the Activity implementing? 4. How well have the activities’ interventions met planned schedules, participant numbers, and outputs? 4.1 What factors promoted or inhibited adherence to the original schedule? 4.2 How were problems and challenges managed? 5. What have been the strengths of and challenges to the overall activity design, implementation, management, and communication so far? 6. What factors in the implementation and context are associated with greater or less efficiency in producing higher quality outputs? 7. With which other organizations do you collaborate? 8. Have you mapped out the stakeholders particularly the agro dealers?

67

8.1 Are you working with agro-dealers? If yes, please explain? 8.2 What support do they provide to the LGs? 8.3 What are the shortcoming of the agro-dealer services? 8.4 How can these shortcomings be improved upon? 9. Who are the other stakeholders you collaborate with? (Finance, WCYA, PSNP) 10. Do you work with financial institutions in helping LG members benefit from credits? Explain how. 11. How would you describe the level and quality of coordination between different stakeholders to implement the Activity? 12. What factors appear to promote or challenge the implementation of the Activity or effective collaboration and cooperation among various stakeholders? 13. What challenges has the Activity faced in ensuring gender/age equity regarding access to, participation in, and benefit from the activity? 13.1 What are the barriers specific to women’s participation, if any? 13.2 What are the barriers specific to youth’s participation, if any? 14. How does the Activity ensure gender/age equity regarding access to, participation in, and benefit from the activity? 14.1 What specific measures were taken to mitigate barriers to women’s participation, if any? 14.2 What specific measures were taken to mitigate barriers to youth participation, if any? 14.3 What are the lessons learned in this regard? 15. Do community members and other stakeholders observe any change in the community because of the Activity intervention? If yes, in what ways? Please give an example. 16. Are LG members benefiting from the Activity? 16.1 Are these changes associated with a specific activity? 16.2 What is the impact of this intervention among the participants? 17. Which of its objectives has the Activity achieved so far? 17.1 What impact has the Activity had on different subsets of the target population, including the poor, women, young people and other vulnerable groups? 17.2 Are there different effects on men and women? 17.3 What about specific benefits for young people? 17.4 What factors appear to promote these achievements? 17.5 How do the changes/ achievements correspond to those hypothesized by the activity’s RF (Results Framework)? 18. What factors appear to deter these changes? 19. In your view, how could the activity be modified to improve its acceptability to targeted communities or the efficiency and effectiveness of its implementation? 19.1 How could it improve its impact on women? 19.2 What about young people? 20. How should the activity’s RF be refined? 21. Which Activity outcomes (services, goods or structures) are likely to be sustained after the Activity ends? 22. Who will help to sustain Activity outcomes after Activity ends? 23. How should hand over of the Activity take place in order to institutionalize it within the government or another organization?

68

24. What capacity building activities are you doing in order to make sure that LG can sustain the Activity? 25. Is it likely the same impacts could be achieved if the Activity were implemented in a different setting or on a larger scale? If yes, how? If no, why not? 25.1 Would the effect be different for men, women and young people? 26. What lessons have been learned so far best practices identified? 27. How were the lessons learned and best practices utilized and scaled up within the Activity, and how were they disseminated externally to other organizations? 28. How and to what extent were lessons and best practices from other organization utilized?

2. FSTF (Kebele/Woreda) 1. What are the main livelihood resources of people in this Woreda/Kebele? 2. What services does your office provide to PSNP HHs? 3. When did the Activity implementation start in this Woreda/Kebele? 4. What is your relation to the Activity? 4.1 How did you get involved in this Activity? 4.2 Are you/your office involved in the selection process of LG members? If yes, how? 4.3 Do you offer specific support so that women and young people can benefit from the Activity? How? 5. To what degree do you participate in the implementation of the Activity in your area? 6. How would you describe the level and quality of coordination between your office and Activity? 7. What are the specific activities you do that are associated with Activity activities? 8. What are the specific activities Activity does in relation to food security of the PNSP participants? 9. Have any HHs graduated from PSNP IV? How many? 9.1 If yes, what are the criteria for graduation? 9.2 If no, why not? 9.3 Do you have any plan for graduation in the next 2 years? 10. Have you communicated the graduation criteria and the responsible body to participants and stakeholders including Activity? If not, why not? 11. How has the Activity contributed to the graduation process of PSNP participants? 12. Who are the other stakeholders or Activity implementers that work with PSNP HHs in your area? 13. Compared to other stakeholders, how do you view the support from Activity in addressing food security of PSNP HHs? 13.1 Please compare with other overlapping Activities in your community, if they exist. 14. What are the challenges your office faces while coordinating support and services to women and young people in your community? 15. From your work experience with Activity so far, what are the strengths of and challenges to the overall implementation, management, communication, and collaboration so far? 16. How could the activity be modified to improve its acceptability to targeted groups such as women and youth? 29. If the Activity was to end tomorrow, what would your office do to continue supporting or coordinating the ongoing services to poor, women and young people? 30. Do you think graduation of PSNP HHs would be possible without the continued support of the Activity? If not, why?

69

31. How would you suggest improving the Activity?

3. Representative of Women, Children and Youth Affairs 1. What are the main livelihood resources of people in this Woreda? 2. When did the Activity implementation start in this Woreda? 3. What is your relation to the Activity? 3.1 How did you get involved in this Activity? 3.2 Were you involved in planning, design, serving committee members, meetings, supervision…? 3.3 Were you involved in the selection process of LG members? If yes, how? 3.4 Is there specific support you provide to help women and young people benefit from the Activity? How? 4. To what degree do you participate in the implementation of the Activity in your area? 5. What other organizations you work with in order to increase the benefit of the Activity to LG members? 6. How would you describe the level and quality of coordination between different stakeholders to implement the Activity? 7. Do you work with PSNP Activity implementers in targeting women and young people? 8. What challenges does your office face while coordinating support and services to women and young people in your community? 9. From your involvement with Activity so far, what are the strengths of and challenges to the overall implementation, management, communication, and collaboration? 10. What challenges does the Activity face in ensuring gender/age equity regarding access to, participation in, and benefits from the activity? 10.1 What are the barriers specific to women’s participation, if any? 10.2 What are the barriers specific to youth participation, if any? 11. What are the strengths of the Activity in ensuring gender/age equity regarding access to, participation in, and benefit from the activity? 11.1 What measures have been taken to mitigate barriers to women’s participation, if any? 11.2 What measures have been taken to mitigate barriers to youth participation, if any? 12. In your view, how could the activity be modified to improve its acceptability by targeted groups such as women and youth? 13. If the Activity was to end tomorrow, what would your office do to continue supporting or coordinating the ongoing services to women and young people? 14. How would you suggest improving the Activity?

4. Agro-dealers 1. Please tell us about your business, how it started, what it does? 2. What did you come to learn about the LRO Activity? 3. What is your relationship to the Activity? 4. How did you get involved in collaborating with the Activity? 5. What support did you receive from the LRO Activity? 5.1 Did you receive any capacity strengthening or financial support? 5.2 How useful was it? 5.3 How should it be promoted?

70

6. What is your relationship with LG members? 6.1 Who established your relationships with the LG members? 6.2 Do Activity provide you with any references to identify they are LG members? If yes, please explain. 6.3 What are the specific services you provide to the LG members? (Ask for inputs, veterinary drugs, feeds, etc.) 6.4 Do you provide services for the group or individual or both? Explain. 6.5 Who are your customers? Men, women, youth, or all? 6.6 Which ones do you think is more efficient, in group or individually? 6.7 What types of services or goods are in demand by the LG members? 6.8 If members are not using them as they want, what are the reasons? Do price fluctuations play a role? 6.9 Which of these are available or can you supply? 7. What is the role of the Activity in coordinating such support? 8. Have LG benefited from your service? If yes, how? 9. Have you benefited from the service you provide? How? 10. What are the challenges you face while providing the services? 10.1 Related to credit services? 10.2 Seasonality of inputs supply? 10.3 Price fluctuations? 10.4 Coordination problems? 10.5 Maintaining inventory? 10.6 Logistics? 11. Are others individuals or groups (e.g. Farmers’ cooperatives, private suppliers) providing similar services in your community? If yes, how does it affect your business? a. How do you differentiate yourself from these competitors? 12. What lessons can you share with us? 13. How is your engagement with LG members differ from non-LG customers? 14. Are you planning to continue working with LG members? 15. What would be the challenge in ensuring continuity? 16. How should Activity assist in order to continue improving your service to LG members? 17. Will you continue engaging with LG members beyond the life of the Activity? Why?

5. Micro- Finance Institution- manager - (Metemamen, Kelata SACCO Union, Vision Fund, Meklit, Awash, Duro Shala Union, …) 1. What is the name of your organization? When did you open a branch in this Woreda? 2. What does your institution do? (Ask for saving, credits, insurance…?) 3. Please explain the process of providing loans? 3.1 Do you ask for collateral or a guarantor? 3.2 What is the rate of interest and payment terms you provide? 4. How is your institution linked with the Activity? Please tell us how it started, any agreement? 4.1 What supports do you get from the Activity? (i.e. office supplies, training, logistics, loan guarantee fund, etc.) 4.2 What has to happen in order to improve your benefit from the Activity?

71

5. Do you provide special services to LG members? 6. How do you assess your engagement with LG members so far? 7. How do LG group members access the services you are providing? What are the requirements for the loans provided to the LG members? 8. Is there any limit for borrowing from your institution? What are the interest rates? 9. What is the repayment schedule for a loan you provide? 9.1 Do you monitor your loan provided to the LG members? 9.2 If yes, how? 10. What challenges do you face in providing services to LG members? 11. Do women face specific challenges around loan provision? If yes, what? 12. Do you provide loans to young people who are LG members? How? 12.1 How do you deal with any credit requests from LG members as young as 15-17? 12.2 Do you have special service arrangement for LG members, particularly to women and young people? 12.3 Has religion affected their ability to pay interest? How do you deal with this? 13. How is your relationship with LG members different from other customers (e.g. Traders, rich farmers, civil servants…)? 14. What is one success story you could share with us around your provision of credit to LG members? 15. What has to change from LG and Activity to improve your support to LG? 16. Will you continue providing credit to LG members after the Activity ends? If no, why not? If yes, how? 17. Do you have any suggestions on how to improve your relationship with Activity – and LG members?

Innovation Fund Grantee: Adami Tulu Agricultural Research Centre (two grants)

Innovations: 1. Support the demonstration and scaling up of Goat Fattening with available feeding technologies

1. What are the main issues with goat farming in the area and how is this grant trying to solve the problem? 2. What are the main activities under this grant? How did you use the grant? 3. When did you start working on this grant and what is the current status in terms of setting up the study sites and securing results? 4. Who are the intended users of the outcomes of this grant and how will they use the results/outcome to improve their livelihood? How might the outcomes affect men, women, or youth differently? 5. How will the Activity participants benefit from this innovation? Possible pathways? How did you involve the Activity participants in the demonstration activity? How are their knowledge and experiences being used during the demonstration period? 6. How will the results be shared with other stakeholders? 7. How do you scale up the results to solve goat farming issues locally, regionally and possibly nationally? 8. What are the strategies to sustain the results of this grant after the Activity ends?

72

9. What was the process to apply for the grant? Do you have any suggestions on how to improve the process? 10. Why do you think your grant was successful versus others?

2. Support participatory evaluation and demonstration of selected Moisture conservation technologies on Maize yield in Moisture deficit area

1. What are the main issues with moisture conservation in the areas, and how is this grant trying to solve this problem? 2. What are the main activities under this grant? How did you use the grant? 3. When did you start working on this grant and what is the current status in terms of setting up the study sites and securing results? 4. How are the farmers’ knowledge and experiences being used while developing new technology? 5. Who are the intended users of the outcomes of this grant and how will they use the results/outcome to improve their livelihoods? 6. How might the outcomes affect men, women, or youth differently? 7. How will the Activity participants benefit from this innovation? Possible pathways? How did you involve the Activity participants in the demonstration activity? 8. How will the results be shared with other stakeholders? 9. How do you scale up the results to solve soil conservation and maize farming problems locally and regionally? 10. What are the strategies to sustain the results of this grant after the Activity ends? 11. What was the process to apply for the grant? Do you have any suggestions on how to improve the process? 12. Why do you think your grant was successful versus others?

2. METEMAMEN Micro Financing Institute Share Co. Innovation: Introduce Mobile and agent banking services using M-Birr which allows clients to deposit, withdraw and transfer money using their mobile phone and to pay for bills

1. What are the main issues with the use of new technologies in banking services in the areas, and how is this grant trying to solve the problem? 2. What are the main activities under this grant? How did you use the grant? 3. When did you start working on this grant and what is the current status in terms of outcomes and results? 4. Who are the intended users of the outcomes of this grant and how will they use the results/outcomes of this grant? 5. How might the outcomes affect men, women, or youth differently? 6. How will the Activity participants benefit from this innovation? Possible pathways? How did you involve the Activity participants in the testing phase? 7. How the results will be shared with other stakeholders? 8. How do you scale up the results to increase access to people without bank accounts? 9. Will there be costs associated with access and use of this product? 10. What are the strategies to sustain the results of this grant after the Activity ends?

73

11. Did you also collaborate with other like-minded organizations during the development phase? 12. What was the process to apply for the grant? Do you have any suggestions on how to improve the process? 13. Why do you think your grant was successful versus others?

Questions for World Vision Collaboration

1. What are the activities you are doing in Oromia region? Which Woreda are you working in? Who is your major donor? 2. Are there similar Woredas where you and CRS work together? If yes, what are the activities you are doing in those Woredas? 4. How do you collaborate with CRS during the program implementation? Any examples? 5. On a scale of 1 to 5, how would you rate the quality of your collaboration (1 being very poor, and 5 being very strong)? 6. How do you solve any issues that may arise when you collaborate with CRS? How do you share responsibilities? Can you share some examples? 7. How do you complement each other and share ownership for joint outcomes? 8. How do you share lessons learned during program implementation with CRS and other stakeholders? 9. What efforts do you make to be intentional about your sharing learnings and ensuring complementarity? 10. What are your strategies for scaling up the program that you implement in collaboration?

74

Annex 5: Direct Observation

Woreda: ______Kebele ______Field Researcher ______Observation date______time______

Check list Comments (observation note)

Activities carried out (i.e. SILC)

Presence of gardening in HHs Livelihood activities (farming activities and off-farm activities) Presence of agro-dealer locally/in the Woreda Training activities Presence of MFIs/ RuSACCO s

Market places that LG could use

Other observations

75

Annex 6: List of Key Informants

No Name Woreda Role 1 Maria Alemu CRS, Addis Ababa Monitoring Evaluation Accountability and Learning (MEAL) Manager 2 Tesyaye Alene CRS, Addis Ababa Youth and Gender Technical Advisor 3 Tamrat W/giorgis CRS, Addis Ababa Financial Services for the Poor Technical Advisor 4 Safayo Ashuta ATJK ACTIVITY- Activity Manager 5 Mohammed Abe ATJK Manager of Mekilit MFI of Adami Tulu branch

6 Habtamu Asfawu ATJK Agro-dealer in ATJK Woreda

7 Zelalem A. Adami Tulu Agricultural Researcher, Innovative Grant Fund on Moisture Research Centre Conservation Technology on maize yield (IGFMCT) 8 Gemedu Teffo ATJK Woreda WCYA office

9 Abas Jundi Boset DA, Nura-Hase Kebele 10 Ahmed Hussein Boset Legal officer in the Woreda’s WCYA 11 Chala Degefu Boset MCS, Community facilitator 12 Mulat Alemayehu Boset Agro-dealer 13 Tilahun Bekele Dodota Process owner at the department of Women’s Mobilization and Support 14 Solomon Hailu Dodota FSTF 15 Kedir Geleto Dodota MCS Area Coordinator 16 Belachew Urgi Dodota Agro –dealer 17 Aman Lage Dodota Kelata SACCO Union

18 Aman Negele Arsi Woreda FSTF head and deputy of agriculture office 19 Tesfaye Hunde Negele Arsi Negele Arsi area coordinator for Activity and DFSA 20 Gadise Sime Negele Arsi Manager of Metemamen MFI of Negale Arsi branch 21 Zenebech Mekite Negele Arsi Agro-dealer in Negele Arsi Woreda 22 Milkeso Bekele Negele Arsi Woreda WCYA office 23 Bekalu Negele Arsi Mobile Banking innovation, Metemamen MFI 24 Amid Gamadi Shala Loan officer of Metemamen MFI of Shala branch 25 Mirko Shibiru Shala Agro-dealer in Shala Woreda 26 Sinkinsh Sida Shala Woreda WCYA office 27 Abaynesh Shiferaw Sire Officer at the department of Women’s Mobilization and Support 28 Getahun Berhanu Sire MCS Woreda Coordinator 29 Berhanu Asebe Sire FSTF 30 Diriba Kebede Siraro Collaboration, World Vision Staff for Collaboration 31 Eyobed Dejene Sire MFI, RuSACCOs’ Coordinator

76

Annex 7: List of documents reviewed and other data sources

1. Activity Proposal submitted to CRS by MCS. 2. Annex B - ACTIVITY Progress Against Annual Workplan_Q3_20190724 3. Catholic Relief Services, Feed the Future Ethiopia Livelihoods for Resilience - Oromia and Development Food Security Activity. 2018. Crop-Value/Market Systems Assessment Report. DEC: https://pdf.usaid.gov/pdf_docs/PA00SX4G.pdf 4. Catholic Relief Services, Development Food Security Activity and Feed the Future Ethiopia Livelihoods for Resilience - Oromia. 2017. Gender and Youth Assessment. Accessible on DEC: https://pdf.usaid.gov/pdf_docs/PA00STN9.pdf 5. Catholic Relief Services, Feed the Future Ethiopia Livelihoods for Resilience – Oromia. 2017. Multistakeholder Assessment of Youth Development. Accessible on DEC: https://pdf.usaid.gov/pdf_docs/PA00TCRQ.pdf 6. Catholic Relief Services, Feed the Future Ethiopia Livelihoods for Resilience - Oromia. 2018. Livestock Value-Chain Assessment. Accessible on DEC: https://pdf.usaid.gov/pdf_docs/PA00TQ9T.pdf 7. Catholic Relief Services, Feed the Future Ethiopia Livelihoods for Resilience - Oromia and Development Food Security Activity. 2018. Off-Farm Value Chain and Market Systems Analysis. Accessible on DEC: https://pdf.usaid.gov/pdf_docs/PA00T6FB.pdf 8. Catholic Relief Services, The Faithful House: Building Strong Families to Affirm Life and Avoid Risk Core Module Manual for Trainers (English and Afaan Oromo) 9. Feed the Future Ethiopia Livelihoods for Resilience_FY19 Q3 Narrative_20190821 10. Productive Safety Net Activity Phase 4: Activity Implementation Manual, Ministry of Agriculture

77

Annex 8: Training Activity Eliana Hotel, Addis Ababa

Time Activities Presenter 22-08-2019 9:00-9:15 Registration All 9:15-9:30 Welcoming and Introduction All 9:30 -10:15 A brief note on CRS Activity Interventions Maria (CRS) 10:15-10:45 Coffee/tea break All 10:45-11:30 Introduction to the CRS Midterm Qualitative Evaluation Lila/(UND) 11:30-12:30 Qualitative Process Evaluation Yisak 12:30-13:30 Lunch All 13:30-15:00 FOCUS GROUP DISCUSSION (FGD) GUIDES - review All 15:00-15:30 Coffee/tea break 15:30-17:00 FGD Role play on FGDs (4 FGDs), feedback & revision All 23-08-2019

9:00-10:00 KIIs Yisak/Lila/Paul 10:00-10:30 Coffee/tea break All 10:30-12:00 Role play on KIIs (4) & feedbacks All 12:00-12:30 Research Ethics Yisak 12:30-13:30 Lunch All 13:30-14:00 Reporting Format and Data Management Yisak

14:00-15:00 Fieldwork Daily Activities and Deliverables Yisak/Chanie

15:00-15:30 Coffee/tea break All 15:30-16:30 Administrative matters and fieldwork organization Yisak/Chanie

24-08-2019 Piloting at Activity site and refining the questions

78

Annex 9: Detailed Planning

Team 1: Abharam, Mesfin, Aster, Tefera Team 2: Toli, Seble, Solomon, Mestawot Day 1 (August 25) Team 1: Negele Arsi FGD Team 2: Boset FGD Day 2 (August 26) Team 1: Negele Arsi KII Team 2: Boset KII Day 3 (August 27) Team 1: Shala FGD Team 2: Dodota FGD Day 4 (August 28) Team 1: Shala KII Team 2: Dodota KII Day 5 (August 29) Team 1: ATJK FGD Team 2: Sire FGD Day 6 (August 30) Team 1: ATJK KII Team 2: Sire KII

79