EAAPP Eval Format FINAL 2

END OF PHASE 1 EVALUATION OF THE EAST AFRICAN AGRICULTURAL PRODUCTIVITY PROGRAMME - EAAPP Final Report

Kate Wellard, Tim Chancellor, Geresom Okecho, Sheba Ndagire and Stephen Mugarura

NATURAL RESOURCES INSTITUTE (NRI) UNIVERSITY OF GREENWICH

AFRICA INNOVATIONS INSTITUTE (AfrII)

March 2015

EAAPP I Evaluation Final Report NRI AfrII

Contents Tables...... 5 Figures...... 6 Acknowledgements...... 8 Acronyms...... 9 EXECUTIVE SUMMARY...... 10 1. INTRODUCTION...... 19 1.1. Background...... 19 1.2. Evaluation Objectives...... 19 1.3. Scope of Work...... 20 1.4. Evaluation approach...... 21 1.5. Methodology...... 24 2. PROJECT IMPACT...... 28 2.1 Overall project purpose – performance against indicators...... 28 Indicator 1: Rate of change in regional specialization and collaboration in agricultural research...... 28 Indicator 2: Rate of increase in information and knowledge transfer across national boundaries...... 33 Indicator 3: Rate of change in adoption of new technologies...... 37 Indicator 4: Rate of increase in land area with seeds of improved cultivars...... 38 Indicator 5: Increase in productivity at farm level over control technology for all disseminated new technologies 39 Indicator 6: Level of stakeholder satisfaction with the technologies and innovations by number of products users 39 2.2 Conclusions on performance against programme purpose...... 40 3. PROGRAMME EFFECTIVENESS – BENEFICIARY IMPACT...... 42 3.1 Introduction...... 42 3.2 Production of key commodities...... 42 3.3 Household incomes...... 46 3.4 Food security and nutrition...... 47 3.5 Overall status...... 48 3.6 Profile of beneficiaries – Equity and inclusion issues...... 50 Gender...... 50 Youth...... 52 Chronic illness and Disability...... 53 3.7 Scaling-out...... 54 3.8 Overall assessment of EAAPP impact...... 55 4. EAAPP PROGRAMMING...... 56 4.1 Programme design...... 56 Linkages in regional research and development...... 57 Investment priorities...... 59 4.2 Relevance of research...... 60 4.3 Quality of research...... 61 4.4 Regionality...... 63 4.5 Programme management and governance...... 66 4.5.1 Governance...... 66 4.5.2 Monitoring and Evaluation...... 68 Communications...... 69 4.6 Role of ASARECA...... 70 TRAINING...... 73 4.7 Enabling and constraining factors impacting on the programme and their handling by EAAPP...... 74 4.7.1 Policy and institutional barriers...... 74 4.7.2 Delivery of programme outputs, outcomes, objectives against budgets...... 74 4.7.3 Partnerships with universities...... 75 4.7.4 Partnership with implementers...... 75 4.7.5 International partnerships...... 76 4.7.6 Sustainability...... 77 4.8 Gender and equity...... 77 4.8.1 Gender...... 77 4.8.2 Youth...... 78 4.9 Economic analysis...... 80

2 EAAPP I Evaluation Final Report NRI AfrII

5. ENVIRONMENTAL SUSTAINABILITY...... 82 5.1 Introduction...... 82 5.2 Compliance with Environmental and Social Safeguard requirements...... 82 5.3 Environmental impacts...... 85 5.4 Agrochemical use and protection...... 88 5.5 Challenges in implementation of ESMP...... 90 5.6 Recommendations – Environmental assessment...... 91 6. EAAPP OVERALL ASSESSMENT AND RECOMMENDATIONS...... 92 6.1 EAAPP Overall assessment...... 92 6.2 Key Recommendations...... 93 Appendix 1 Evaluation Framework...... 96 Appendix 2 Selected Publications under EAAPP...... 100 Appendix 3 Key Persons Interviewed...... 101

Tables

Table 1 Household survey respondents by country and commodity...... 27 Table 2 Quality assurance process...... 28 Table 3: Number of regional research sub-projects and degree of partner country participation...... 30 Table 4: Thematic coverage of regional research sub-projects...... 31 Table 5: Numbers of scientists available to conduct research in regional centres of excellence...... 32 Table 6: Number of postgraduate students in EAAPP countries supported through the project...... 32 Table 7 Adoption of new varieties, breeds and management practices (% Households using in EAAPP project areas) 38 Table 8 Use of improved varieties and breeds by beneficiary and non-beneficiary households (%)...... 39 Table 9 Land Under improved cultivars (Ha)...... 39 Table 10 Change in productivity of EAAPP technology over control technology at farm level (%)...... 40 Table 11 Level of stakeholder satisfaction with technologies and innovations by users (%)...... 41 Table 12 Stakeholders reporting increased output as a result of improved technologies, 2014 (%)...... 41 Table 13: Average total production of key commodities by surveyed EAAPP beneficiary and non-beneficiary households in 2009 and 2014 (Kg)...... 43 Table 14 Average yield per hectare of key commodities of surveyed EAAPP beneficiary and non-beneficiary households (Kg/Ha)...... 44 Table 15 Average milk yield of surveyed EAAPP beneficiary and non-beneficiary households (Kg/animal/year)44 Table 16 Average net value of crop and livestock production of key commodities by beneficiaries and non- beneficiaries in 2009 and 2014 (US$ constant prices)...... 47 Table 17 Food sufficiency of surveyed EAAPP beneficiary and non-beneficiary households, 2009 and 2014 (% of households)...... 48 Table 18 Surveyed households consuming from all food groups in the past 24 hours...... 49 Table 19 Economic status of beneficiary and non-beneficiary households in 2009 and 2014 (self-reporting) (%)...... 50 Table 20 Average yield for key commodities by gender of household head – project beneficiaries (Kg/ha)...... 52 Table 21 Economic status reported by beneficiary households in 2009 and 2014 by gender of household head52 Table 22 Age groupings of household heads: beneficiaries and non-beneficiaries surveyed...... 53 Table 23 Disability status of EAAPP beneficiaries and non-beneficiaries surveyed...... 54 Table 24 Membership of an organised group by EAAPP beneficiaries and non-beneficiaries...... 55 Table 25: Number of non-project members learning and using EAAPP technologies disseminated by EAAPP beneficiaries (beneficiary reporting)...... 56 Table 26 Allocation of funding by component and country...... 61 Table 27 Returns to agricultural R&D investment with and without spillovers...... 65 Table 28: Typology of support from regional institutions...... 72 Table 29: Farmers trained under EAAPP projects (short-term training)...... 79 Table 30 Valuation of EAAPP net benefits...... 82 Table 31 ESMF activities carried out by EAAPP projects (%)...... 85 Table 32: Environmental and social impacts observed by EAAPP project implementers...... 87 Table 33 Case study of Projects Implemented in Tanzania...... 88 Table 34 Use of agrochemicals by farmers surveyed...... 90

Table 35 Type of protective clothing used by farmers surveyed...... 91

Figures

Figure 1 Increased regional specialisation in agricultural research...... 30 Figure 2 Animal genetic improvement: Development and sharing of common breeding approaches...... 35 Figure 3 Increasing rainfed wheat productivity...... 36 Figures 4a-d Yields of key commodities reported by beneficiary and non-beneficiary households in 2014 (Kg/ha; Kg/animal)...... 44 Figure 5 Increased competitiveness of rice value chains through partnerships and improved business skills....57 Figure 6 Partnerships with extension services are key to sustaining farm level innovation and need to be supported 76 Figure 7: Project implementers submitting ESMF reports over the previous 12 months...... 85

Acknowledgements

The Natural Resources Institute and Africa Innovations Institute gratefully acknowledge the support received during this assignment by EAAPP teams in Kenya, Tanzania, Uganda and Ethiopia: particularly National Coordinators, RCoE Coordinators, Research Coordinators and M&E specialists. All EAAPP staff and projects have provided timely and enthusiastic assistance to the team. We would particularly like to thank the EAAPP Coordinator, Vincent Akulumuka for his untiring support during the evaluation process. Finally, we would like to thank all project staff, stakeholders and farmers who generously gave their time to work with us in the field.

Acronyms

AFAAS African Forum for Agricultural Advisory Services AfrII Africa Innovations Institute AIS Agricultural Innovation System ART Assisted Reproductive Technology AR4D Agricultural Research for Development ASARECA Association for Strengthening Agricultural Research in Eastern and Central Africa CAADP Comprehensive Africa Agriculture Development Programme CFP Country Focal Point CGIAR Consultative Group for International Agricultural Research COMESA Common Market for Eastern and Southern Africa CORAF Conseil Ouest et Centre Africain Pour la Recherche et le Developpement Agricoles) EAAPP East African Agricultural Productivity Programme ECA Eastern and Central Africa EASCOM East African Seed Committee ESMF Environmental and Social Management Framework ESS Environmental and Social Safeguard FAAP Framework for African Agricultural Productivity FARA Forum for Agricultural Research in Africa IPR Intellectual Property Rights KRA Key Results Area M&E Monitoring and Evaluation MDTF Multi-Donor Trust Fund MTR Mid-term Review NARI National Agricultural Research Institute NARS National Agricultural Research System (government, private sector, NGOs) NPT National Performance Trial NRI Natural Resources Institute PAD Project Appraisal Document PDO Programme Development Objectives PI Principal Investigator PIP Project Implementation Plan PMP Performance Management Plan PMU Project Management Unit PPE Personal Protective Equipment PVS Participatory Variety Selection RCoE Regional Centre of Excellence S3A Science Agenda for Agriculture in Africa TIMPS Technologies, Innovations and Management Practices ToR Terms of Reference TOSCI Tanzania Official Seed Certification Institute WAAPP West African Agriculture Productivity Programme WECARD West and Central African Council for Agricultural Research and Development WB World Bank

EXECUTIVE SUMMARY

The East African Agricultural Productivity Programme (EAAPP) was conceived as a Regional Agricultural Research for Development initiative. The EAAPP Programme Development Objective is to: enhance regional specialisation in agricultural research; enhance collaboration in agriculture training, and technology dissemination; and facilitate increased transfer of agricultural technology, information and knowledge across national boundaries. EAAPP is a ten-year programme with two phases. Phase I, approved in 2009, focused on capacity building with the establishment of the Regional Centres of Excellence (RCoEs) through construction/improvement of infrastructure and human resource development; technology generation and dissemination; and improving seeds and breeds availability. EAAPP focuses on four commodities: cassava where Uganda is leading, dairy led by Kenya, rice by Tanzania and wheat by Ethiopia.

ASARECA commissioned the Natural Resources Institute (NRI) of the University of Greenwich in partnership with Africa Innovations Institute (AfrII) to conduct an end-of-project evaluation, including Economic Analysis and Impact Assessment focusing on outcome-level evaluation of the implementation of EAAPP phase I.

The overall evaluation objectives were to: critically undertake an economic analysis and assess achievements of the implemented projects; critically assess the performance of EAAPP in meeting its Development Objective; and generate key lessons to inform the design of the next potential phase of EAAPP.

The methodology combined participatory and conventional data collection methods from both primary and secondary sources for the four priority commodities across the four project countries. It included quantitative data collection and analysis (including a household survey with 1239 beneficiary and non- beneficiary households) and qualitative methods to provide detailed analysis and assessment of the quality of implementation, outputs and results.

Key Findings – Programme Impact

Indicator 1: Rate of change in regional specialization and collaboration in agricultural research Average level of regional specialization and collaboration across the four countries was 63%, an increase of 53 percentage points above the baseline and exceeding targets (EAAPP M&E data). Twenty-nine regional research sub-projects have been initiated with high levels of country participation. There appears to be some imbalance in the portfolio of sub-projects on cassava and wheat. Several sub-projects address value chain and marketing issues but there is a relatively weak emphasis on mechanization and post-harvest issues across all commodities. These impacts are especially important in reducing drudgery and increasing income-earning opportunities for women.

Short- and long-term training of research scientists has increased capacity of RCoE in each country, most notably in Ethiopia. Some of the increase is due to redeployment of existing scientists, while in Uganda, some is hiring of consultants rather than permanent staff. Seventy-five Masters students and 36 PhD candidates have been fully funded under the project. Similar numbers of men and women registered for Masters programmes in Tanzania and Uganda but only 2 in Ethiopia. There were only 7 female PhD students out of 36 across the four countries. A further 50 students have received partial funding from the project. Further capacity development is needed in livestock breeding (for animals and fodder), value addition (‘functional products’) and genomics.

Progress in acquisition of infrastructure has been behind schedule in all countries, mainly due to slow government procurement systems which are outside the control of the project. Delays in completion of the labs and commissioning of the lab equipment have affected the type and quality of research conducted.

However, the significant improvements in capacity to conduct research, support dissemination and host regional research exchanges should deliver results in a potential Phase 2.

Indicator 2: Rate of increase in information and knowledge transfer across national boundaries. The regional centres of excellence have developed 138 new technologies. Many are new varieties of cassava, rice, wheat and forage crops. Twenty-three new technologies have been disseminated across national boundaries including: two Tanzanian rice varieties released in Kenya and Uganda, and undergoing National Performance Trials (NPT) in Ethiopia; four clones of Napier grass from Kenya recommended for dissemination in Uganda; botanical seed of cassava with enhanced carotene sent to Ethiopia, Tanzania and Kenya; assisted reproductive technologies from Kenya sent to the other countries. This is a considerable achievement given the restrictions on transfer of crop varieties and cattle semen between countries.

The project has generated a large amount of dissemination materials in the form of leaflets, booklets, posters and manuals. All countries have prepared communication strategies. The project is yet to exploit the full potential of digital and other communication approaches.

Indicator 3: Rate of change in adoption of new technologies EAAPP M&E surveys show an increase in adoption of new varieties, breeds, and other selected management practices by farmers from 35 percent to 53 percent (2010-2014) in project areas. Large increases are seen in Ethiopia (all commodities) and Kenya, with smaller increases in Tanzania. Evaluation findings confirm that the proportion of beneficiaries using improved varieties of cassava, wheat and rice has increased in all countries between 2009 and 2014. In 2014 65 percent of targeted households were using improved cassava, 87 percent improved rice varieties and 97 percent improved wheat and adoption of improved breeds of dairy cows has also increased.

Indicator 4: Rate of increase in land area with seeds of improved cultivars EAAPP M&E estimates of land planted with improved cultivars are 2,755 ha in 2010, increasing to 12,807 ha in 2014. This is attributed to the substantial increase in production of planting material and farmer awareness in EAAPP project areas. Large increases are reported in Ethiopia, Kenya (cassava, rice, wheat) and Uganda (cassava). Beneficiary groups and key informants interviewed during the evaluation confirmed sizeable increases in areas planted for each of the commodities in the four countries.

Indicator 5: Increase in productivity at farm level over control technology for all disseminated new technologies Almost all technologies performed positively over control technologies, ranging from zero to 8% between 2010 and 2013 (EAAPP M&E). Large improvements in productivity of new technologies over controls were recorded in wheat (Kenya and Uganda) and cassava (Kenya). Rice and dairy technologies showed lower increases in over the project period. Findings from the evaluation survey on output per unit of land indicate higher increases in land productivity for all commodities except cassava in Uganda.

Indicator 6: Level of stakeholder satisfaction with the technologies and innovations EAAPP programme measures show farmer satisfaction with technologies has increased from 23 percent to 69 percent of households in targeted project areas. The greatest increases were in cassava and wheat. This may be partly the result of disease resistance traits of the new technologies. The evaluation survey found that a quarter to half of beneficiaries had experienced improvements in their production or incomes as a result of adopting the technologies.

Overall, programme targets have been met and, in some cases, exceeded. Thus the overall assessment is quite positive. Regional research projects are on target, though with some variation between commodities. Training targets are also on track.

All EAAPP partner countries had relatively high increases in their agricultural research and development spending between 2000 and 2011. All saw increased numbers of agricultural researchers during the same period, but the magnitude was much greater in Ethiopia. Phase 2 of the project needs to build in mechanisms for mentoring returning students and integrating them into RCoE activities.

Beneficiary impact Evaluation household survey findings indicate that EAAPP beneficiaries are already seeing a positive impact from their involvement in the programme in terms of: production, yields, incomes, food security and economic status. In a number of cases these improvements were found to be significantly greater than non-project households’.

Average yields for beneficiaries in 2014 were 15 tonnes per hectare for cassava, 7 for wheat and 9 for rice. These exceed regional productivity figures for rice and wheat. Despite these improvements, there is large variability in farmer yields across all the four commodities.

Net incomes from key EAAPP commodities of surveyed beneficiary and non-beneficiary households have increased over the project period. This has been most significant in dairy in Kenya, rice in Tanzania and cassava in Uganda. Non-beneficiaries have also increased their net incomes, partly through scaling out effects to neighbouring farmers and communities. Net incomes of beneficiaries from key commodities were significantly higher than those of non-beneficiaries in diary and rice. Low incomes were received by wheat farmers in Tanzania who complained of low prices. Returns for dairy farmers in Uganda were very low at $300pa, below the $1 per day income threshold.

Beneficiaries with household food surplus rose significantly in Ethiopia, Kenya and Tanzania, and slightly in Uganda. Beneficiaries in Ethiopia and Kenya were significantly more likely to be food surplus households than non-beneficiaries in EAAPP project areas in 2014. Nutrition security, indicated by the number of food groups consumed, was significantly higher for project farmers compared to non-beneficiaries for wheat in Ethiopia and dairy in Uganda, implying a positive impact on diversity of foods consumed in the household. However, beneficiary rice households in Kenya were less likely to have a diversified diet than non-project farmers. This may be because these are households who have so far not seen an increase in incomes through the project.

Poverty self-assessment showed significant improvements by 2014, with large numbers of households being lifted out of the poorest group. This could be the result of transmission of benefits to non-project members, as well as to non-EAAPP interventions in the area. Beneficiary households in Uganda and Ethiopia appear to have improved their economic status relative to non-project members in their community over the life of the project. Beneficiaries are also scaling-out technologies to non-beneficiaries at quite a high rate.

Gender: The Impact Evaluation looked at several aspects of gender, including inclusion and roles of women and female-headed households in EAAPP projects. Project staff have been trained in Gender mainstreaming and target women farmers. An exception was dairy farmers in Kenya where some groups were found to be all male. Before the start of EAAPP activities in 2009, yields of female-headed households were lower than those of their male counterparts for wheat, dairy and rice. By 2014, female- headed households were yielding more than male-headed households for wheat in Ethiopia, rice in Tanzania and cassava in Uganda. This implies that the technologies are having a positive gendered impact. Only in Kenya were female-headed dairy households lagging behind. The EAAPP Kenya programme is prioritising gender issues in an attempt to address this.

Programme design EAAPP is intended to enlarge the technology space of the four commodities through the establishment of RCoEs and lowering barriers of movement of technologies across borders. The theoretical basis for the programme is sound but the institutional context provides considerable challenges. Regional research has effectively been doubled in Eastern Africa as a result of the Programme. Regional spillovers have already been realised both inside and outside EAAPP countries.

The agricultural innovation system envisaged under EAAPP is a chain – from international and local research, extension and advisory services to production, distribution of inputs and on-farm application. Rice in Southern Tanzania is an excellent example of commercialisation of a RCoE variety. Putting an explicit Agricultural Innovation Systems model at the centre of the programme would support this approach and ensure it is relevant and farmer-led. Agricultural research and delivery need to be linked more closely, and other actors including universities and farmer organisations should be centrally involved in research and extension activities.

Options for organizing and funding regional research and extension include: collaboration between scientists and extensionists in different countries facilitated through regionally-managed projects; and Regional research and extension undertaken through projects funded by national governments or external donors.

The largest share of investment under EAAPP I goes to support Research, training and dissemination (Component 2) followed by Improved availability of seeds and breeding material (Component 3), except for Uganda. This would be expected to continue under Phase II. A significant share was also allocated to Support to research infrastructure (Component 1.1). Outputs were not fully complete and research facilities commissioned at the time of the Evaluation visits. Support to Harmonisation and regulations has been a critical investment in a public good which is expected to contribute to improved regulation and policy harmonisation.

Regionality - Policy ASARECA has facilitated harmonized NPT protocols for cassava, rice, wheat and pasture seeds under the project. It has coordinated a draft EAAPP intellectual property (IP) rights policy. On livestock, progress has been made in: drafting animal breeding policy and breeding rules in Kenya, now being used as a guide for other EAAPP countries; developing regional guidelines on procedures for movement and trade in heifers and germplasm; drafting a National Animal Breeding Policy in Ethiopia; development of a protocol for import and export of compounded dairy feeds and forages in Tanzania; developing regional guidelines for import and export documentation and procedures for dairy. Whilst good progress has been made several actions need to be fast-tracked: accreditation of the new laboratory in Naivasha, establishment of a Dairy Board in Ethiopia, and development of a livestock policy and finalising drafting legislation on animal feed in Uganda.

Institutional arrangements EAAPP activities at country level are managed through different institutional arrangements. Given very different national institutional contexts and capacities, it is not possible to come up with a blueprint for EAAPP country organisation. Rather, new countries joining the project need to critically assess: existing institutions and their decision-making processes, cost, sustainability. EAAPP units need to be staffed by dedicated full-time staff to improve efficiency and timeliness of activities, and to free up PCU staff to engage in relevant scaling out and partnerships.

Project M&E The harmonised Results Framework developed by ASARECA/EAAPP is a very useful tool which focuses on process and output indicators, and tracks progress at national and regional levels. Overall the M&E systems are very well designed and managed, providing timely results for project management. M&E officers have good capacity and are well motivated. The EAAPP M&E process is however fairly resource intensive, with a large number of quantitative indicators requiring frequent follow up.

Role of ASARECA The role of ASARECA is to work with the RCoEs in coordinating regional activities: providing assistance for networking, capacity building and technical backstopping, monitoring and evaluation, regional coordination and supervision, and policy analysis. However, ASARECA’s ability to contribute inputs is dependent on its having adequate financial and technical resources. Only Uganda has provided the full contribution.

General observations by EAAPP members were that ASARECA staff have done a good job, and EAAPP staff showed particular appreciation for ASARECA’s inputs in: facilitating regional meetings, identifying topics for regional research projects, developing guidelines on the description of the technologies, training in ESS and value chains, guiding the working group on policy harmonization and providing the M&E framework and the PMERL software. Overall, countries agree that ASARECA needs to have a strengthened role in EAAPP to deliver full regionality.

Project implementation

Delay in disbursement of funds has been a binding constraint to the efficiency of project implementation. Slow disbursement of funds tend to be a common feature of government-funded projects but the consequences are more severe in regional projects when activities need to be coordinated among countries.

Partnerships with Universities: Research carried out by postgraduate students in each of the countries was generally well integrated into sub-project activities and students have made an important contribution to the project. Challenges include matching research and academic objectives, delays in procuring consumables and lengthy response times from supervisors. A key lesson is on closer links between research institutes and universities in the research component.

Partnerships with implementers: EAAPP projects partner with a range of stakeholders including government at central and local levels. Partnerships tend to be on a one-off project activity basis. To improve buy-in and sustainability, EAAPP should look at providing a small funding allocation to local government departments of agriculture and livestock with a track record in project implementation.

International Partnerships: Partnerships with the CGIAR and advanced research institutes are beneficial but RCoEs will need to increasingly assimilate some of the functions traditionally played by these organizations. The CGIAR has a new capacity development strategy it has a reduced commitment to capacity development than in the past and RCoEs will need to help fill this space. The primary focus of the CGIAR will remain on upstream rather than adaptive research, but RCoEs will need to enhance their capacities in advanced research methods in order to engage effectively with the CGIAR and advanced research institutes in emerging areas of research. Under the CGIAR Consortium Research Programmes collaboration with national partners will need to be highly strategic and the RCoEs are well placed in this regard. The involvement of SROs and NARS in CGIAR priority setting has generally been limited and RCoEs will have a role to play in this in the future.

Sustainability Continued financial support for regional centres of excellence will depend on delivery of demonstrable benefits over and above those that each participating country might expect to achieve independently. Sustainability is being addressed through: support for dissemination and for the seed sector, and monitoring and evaluation of project benefits. The project needs a proactive public relations effort to build awareness among key stakeholders.

Gender and Youth Gender issues were included in the selection of commodities for EAAPP, but were not given serious consideration in project implementation until midway through the project, when the gender strategy was developed and gender Focal Persons appointed. Project targets can play a role in boosting female participation and could be set higher under EAAPP II. Achieving targets requires focused support on gender sensitization and programming for project partners and participants.

EAAPP is working with established youth groups and sometimes supports new groups to take advantage of emerging business opportunities. In Kenya the programme has drawn up a youth project/strategy focusing on creating employment and generating income for unemployed youths as well as influencing school children to appreciate agriculture as an important business.

Economic analysis Project benefits have been estimated using valuation of net increased agricultural production of beneficiaries over non-beneficiaries over the project life (Difference In Difference). This has been estimated at $307 for wheat, $175 for cassava and $391 for rice. Average value of dairy beneficiary production compared to non-beneficiaries is reported to have fallen over the period in real terms, due to low prices received by group members. However, dairy beneficiaries reported supplementing their incomes through periodic sales of calves, so relative incomes may not have changed significantly. Comparing the valuation of benefits against estimated programme costs indicates that the benefit-cost ratio would be expected to be positive midway through 2015. Even with quite large changes in assumptions, returns to the investment under EAAPP I look positive by end of 2015.

Environmental and Social Safeguards All EAAPP countries have developed Environmental and Social Monitoring Frameworks. Implementers have been involved in: preparation of Environmental and Social Management Plans (ESMP), environmental screening, Integrated Pest Management planning and Environmental Impact assessments, as well as Capacity Building and Training of trainers. Awareness of the ESMP is high amongst EAAPP project implementers and stakeholders. Potential negative environmental and social impacts included: pesticide use and waste generation, as well as a small number of cases of potential noise generation, clearing vegetation, resettlement and increased use of resources. Potential positive impacts were decreased use of water or energy resources and use of local knowledge. Training has been provided to beneficiaries in handling of pesticides, but few farmers were found to be using protective equipment due to high cost. Other challenges in implementation of the ESMP include: limited resources, limited staff for implementation; late incorporation of ESS in the project.

Key recommendations

1. Thematic areas: There is need for more emphasis on mechanization and post-harvest issues, given the potential beneficial impacts from mechanization, transport, storage, processing and packaging,

especially for women, in reducing drudgery and increasing income opportunities. 2. Long-term training: Increased investment in post-harvest and value chain/marketing, livestock breeding (animals and fodder) and genomics is needed.

3. Gender and training: Numbers of female PhD candidates are very low and need to be addressed. The project should review selection processes and learn lessons to guide future recruitment.

4. EAAPP Phase 2 needs to build in mechanisms for mentoring returning students and integrating them in RCoE activities. Governments should recognize the need to provide adequate incentives to retain new human resource capacity. 5. Regional dissemination: Methods used to disseminate new technologies need to be evaluated for inclusion and impact. The programme needs to identify and support ways to facilitate the dissemination and adoption of technologies through appropriate and sustainable channels.

6. The programme needs to exploit the full potential of communication approaches and support activities to document and share learning on where a particular communication approach has proved valuable.

7. RCoE websites need to be improved to give greater visibility to successful approaches used by the project and technologies developed and promoted.

8. Agricultural research and delivery need to be linked more closely, and other actors, including universities and farmer organisations, involved more centrally in research and extension activities. Multi-stakeholder and demand-led approaches are key to attaining EAAPP Project Development Objectives. An explicit Agricultural Innovation Systems model needs to be made central to the project.

9. Farm-level variability needs to be investigated during EAAPP II to understand causes and develop technology recommendations for a range of agroecological and socioeconomic conditions. Farmer- centred research methods (e.g. Participatory Technology Development) which involve farmers in all stages of the research process would be most appropriate.

10. Impact of EAAPP on food and nutrition security needs to be tracked by the project. A nutrition strategy needs to be developed and capacity built to implement it, to ensure the project maximises benefits and avoids negative impacts on vulnerable groups, including under-5 children.

11. Stronger capacity in research methods is needed to improve journal publication rates and research design. It is recommended that writeshops are held in which professional facilitators assist selected researchers to prepare manuscripts for journal publication.

12. Policy harmonisation: Closer harmonisation is needed to improve regional research effectiveness. ASARECA can play a critical role in strengthening the regional dimension of EAAPP, but this requires strengthening of the EAAPP Coordination unit.

13. Institutional arrangements: New countries joining the project need to critically assess: existing institutions and their decision-making processes, cost, sustainability in designing institutional arrangements. EAAPP coordination units should be staffed by dedicated full-time staff to improve efficiency and timeliness of activities, and increase engagement in scaling out and partnerships.

14. EAAPP M&E systems: as the programme moves to a new phase more attention is needed on lesson learning, evaluation and sharing best practices.

15. Role of ASARECA: ASARECA needs to have a strengthened role in EAAPP to achieve full regionality. Options for funding ASARECA inputs should be explored including: direct commissioning for specific inputs; funds for regional projects released to ASARECA; increased contributions from the current levels. Countries must also deliver on existing financial pledges.

16. Gender: Gender targets can play a role in boosting female participation and should be set higher under EAAPP II. Achieving targets requires support on gender sensitization for project partners and participants. Experiences on gender mainstreaming from pilots should be evaluated and scaled-out across EAAPP.

17. Youth: Successes and lessons from initiatives such as Kenya’s Youth Strategy should be shared amongst EAAPP countries and partners and mainstreamed through an EAAPP Youth Strategy.

18. Partnerships with universities - closer links between research institutes and universities would provide benefits in terms of arrangements for studentships. Universities also have specialist knowledge and facilities that can be utilised in project research and complement research institutions. The role of universities needs to be reviewed in any future phase.

19. Partnerships with local government – EAAPP should provide a small funding allocation to local government departments of agriculture and livestock that have a good track record in project implementation to support wider involvement of local development partners in project activities.

20. Environmental and Social Safeguards recommendations:  Conduct regular training on biosafety, and occupational safety and health at all levels to increase use of protective equipment and knowledge of safe varieties and safe practices of handling.  Provide access to finance for project participants to purchase protective equipment.  Support weather monitoring and keeping a log of chemicals used and trends in usage. Projects involving field trials and irrigation should monitor nutrient content in run-off and nearest water bodies.  Establish research on waste disposal including laboratory chemical wastes. Liaise with National Environment Authorities on hazardous waste management.

21. EAAPP should review lessons from the West African Agricultural Productivity Programme (WAAPP), including the role of Conseil Ouest et Centre Africain Pour la Recherche et le Developpement Agricoles/West and Central African Council for Agricultural Research and Development (CORAF/WECARD) in management of regional projects, and provision for an explicit regional harmonization component.

1 INTRODUCTION

2 Background

The Eastern Africa Agricultural Productivity Programme (EAAPP) was conceived as a Regional Agricultural Research for Development (AR4D) initiative to: (i) support AR4D activities to address regionally identified agricultural challenges for the selected commodities, (ii) generate improved agricultural technologies that spill over country boundaries, (iii) complement the Common Market for Eastern and Southern Africa (COMESA) initiatives, (iv) provide a platform for policy harmonization, and (v) complement the regional strategic approach to commercialize agriculture.

EAAPP aims to raise farm incomes, reduce poverty and improve food security by: (i) strengthening Regional Centres of Excellence (RCoE) in agricultural research in Ethiopia, Kenya, Tanzania, and Uganda and (ii) to support these centres in the production of selected varieties of improved seeds and breeds and to improve access of poor farmers to more productive crop varieties and improved dairy cattle. The project is contributing to CAADP Pillar IV on improving agricultural research and development.

EAAPP is a ten-year programme of two phases. Phase I was approved in 2009 and its implementation started in 2010. EAAPP Phase I focused on capacity building and concentrated on the establishment of the Regional Centres of Excellence through: construction/improvement of infrastructure and human resource development; technology generation and dissemination; and in improving seeds and breeds availability. EAAPP focuses on four main commodities: Cassava where Uganda is leading, Kenya leads on Dairy, Tanzania on Rice and Ethiopia on Wheat.

At the regional level, EAAPP also provides a vehicle for implementing the agricultural transformation agenda of ASARECA, which was created to enhance regional collective action in agricultural research for development, extension, training, and education. ASARECA has been mandated by COMESA and the Forum for Agricultural Research in Africa (FARA) to take the lead in coordinating implementation of CAADP Pillar IV and the application of the principles of the Framework for African Agricultural Productivity (FAAP) in Eastern and Central Africa.

ASARECA commissioned a team of multi-disciplinary consultants from the Natural Resources Institute (NRI) University of Greenwich in partnership with Africa Innovations Institute (AfrII) to conduct a detailed end of project evaluation, including Economic Analysis and Impact Assessment focusing on outcome-level evaluation of the implementation of EAAPP phase I. This report details the findings of the evaluation.

3 Evaluation Objectives

The main objectives of the evaluation were to

1. Critically undertake an economic analysis and assess achievements of the implemented projects. This includes determination of the number of direct (and indirect) beneficiaries as well as quantification of various benefits accrued to targeted stakeholders along the value chain (farmers, traders, processors, etc).

2. Critically assess the performance of EAAPP in meeting its Development Objective as stipulated in the Results Framework. This includes assessing i) Rate of change in regional specialization and

collaboration in agricultural research, ii) Rate of increase in information and knowledge transfer across national boundaries iii) Rate of change in adoption of new technologies (disaggregated by type ) iv) Rate of increase in land area with seeds of improved cultivars (%) v) Increase in productivity at farm level over control technology for all disseminated new technologies and vi) Level of stakeholder satisfaction with the technologies and innovations (%) by number of product users (by gender, age, and location).

3. Generate key lessons learned that may be helpful in the design of the next potential phase of EAAPP.

4 Scope of Work

To achieve the above-mentioned objectives, this evaluation was tasked to assess the relevance, efficiency, effectiveness, achievements and sustainability of EAAPP projects as well as determine the extent of performance towards the fulfilling the Development Objectives. To achieve this, the following tasks were to be undertaken (taking into account gender disaggregation):

Design and carry out a comprehensive survey on randomly selected stakeholders (farm households, researchers, advisory services, seed companies, implementing institutions, etc.). Carry out detailed economic analyses of EAAPP interventions in terms of return on investments (e.g. conducting cost-effectiveness & benefit-cost analyses). Assess and quantify the impacts of the EAAPP interventions (e.g. determine the number of direct and indirect beneficiaries, as well as the type and magnitude of the accrued benefits, etc). Determine the rate of change in regional specialization and collaboration in agricultural research. Determine the rate of increase in information and knowledge transfer across national boundaries. Assess the rate of change in adoption of new technologies. Assess the rate of increase in land area with seeds of improved cultivars. Determine the increase in productivity at farm level over control technology for all disseminated new technologies (%). Determine the percent change in yield of Cassava, Wheat and Rice and also Dairy (in terms of production and productivity). Determine the level of stakeholder satisfaction with the technologies and innovations (%) by number of products users (by gender, age, and location). Assess strengths and weaknesses of the implementation process, including targeting and reaching specific beneficiary groups (i.e. small scale farmers, women farmers, etc.). Generate robust evaluation evidence of the links between research outputs and their impact on the lives of poor people. Assess internal and external factors that influenced project implementation, including technical, managerial, organizational, institutional, socio-economic and political factors. Evaluate levels of conformity of project interventions with sound environmental management practices in general and the national legislative requirements of member countries, besides compliance with the ASARECA approved Environmental and Social Management Framework (ESMF). The audit shall: o Describe the major impacts including, but not limited to potentially affected ecological and socio-economic areas, identifying any environmental and social concerns o Review the efficacy and adequacy of ESMF and project documents in ensuring ESS compliance, including adequacy of existing internal controls to identify and mitigate adverse impacts

o Assess the level of awareness and sensitization about ASARECA’s ESMF and other relevant environmental standards, regulations, law and policy among project implementers o Examine monitoring programs, parameters, and procedures in place for control and corrective actions at organizational, program management and sub-project levels o Obtain views on social issues from project implementers and other potentially affected persons. Recommend corrective measures that will assist EAAPP to promote a safe and healthy environment during the implementation of future projects. Assess the effectiveness of selected partnerships formed or strengthened through the selected projects. Determine the effectiveness of the overall project implementation in the different communities in which they were being implemented and among other value chain actors. Monitor the adoption rate, effectiveness and sustainable use of the TIMPS. Assess the status of spillovers emanating from these projects, by determining which project outputs had a multiplier effect. Document lessons learned over the period studied, narrate case studies on success stories and provide recommendations for improving the implementation and/or design of the approach to ensure achievement of the objectives and benefit sustainability.

5 Evaluation approach

The evaluation addressed a wide set of evaluation questions and issues, including: project design, implementation, efficiency and sustainability and impact at household, community and regional levels. An evaluation framework was drawn up by the evaluation team to organise and systematically address these questions (Appendix 1).

Relevance: This covered relevance of EAAPP to regional activities and to national priorities, evidenced by ASARECA, CAADP and national Agricultural development and investment plans and strategic plans of National Agricultural Research Systems (NARS) and through discussions with national and local government policy makers and researchers. Relevance of EAAPP technologies to farmer priorities was determined through discussions with community groups and individual interviews with male and female farmers from diverse social and economic groups.

Development Impact: EAAPP’s Development Objective is to enhance regional specialisation in agricultural research, enhance collaboration in agricultural training and technology dissemination, and facilitate increased transfer of agricultural technology, information and knowledge across national boundaries. The evaluation assessed the extent of progress towards regional specialisation in agricultural research, rate of change in information and knowledge transfer across national boundaries, rate of change in adoption of new technologies, rate of change in land area with seeds of improved cultivars, change in productivity at farm level for all disseminated new technologies, and level of stakeholder satisfaction with the technologies and innovations by gender, age and location.

Beneficiary Impact: This is a central component of the evaluation and assessed the number of direct and indirect beneficiaries and quantification of intended and unintended benefits (positive and negative) accrued to targeted stakeholders along the value chain. Impact evaluation involved assessing changes in the well-being of individuals, households and communities that can be attributed to a particular project, programme or policy. Key questions considered in the impact evaluation included:

 What is the (net) impact attributable to the project? What would have happened without the project? Measuring attribution involved an identifiable counterfactual (control group) and a single primary cause and effect. Impact on targeted beneficiaries of EAAPP projects was assessed through a household survey with a control group (non-beneficiaries) and before and after project intervention livelihood indicators.

 Did the project make a difference? Assessing contribution involved examining whether and how the intervention has contributed to impact. The evaluation used the project theory of change to identify project and non-project influencing factors and their likely contribution to change.

 What is the potential wider impact of the project? Examining potential for scaling-up and scaling- out included examination of the conditions under which project outcomes have occurred and whether these can be applied elsewhere. The evaluation examined project context and identified structures, processes and institutions (including those created by the programme) and their potential for scaling out project impact.

Project efficiency: The efficiency of the project and its strategies in delivering outputs, outcomes and objectives against budgets were assessed. Key measures used included:

 Cost-benefit analysis – estimated value of inputs and outputs in monetary terms. Estimates were made based on cost of projects and changes in value of agricultural output over the project period.

Gender and equity: The effectiveness and extent to which gender and equity issues have been incorporated into project design and implementation was examined. Information was collected from projects on participation of women in projects and from Survey beneficiaries on changes in the status of participating women and vulnerable groups (including those with chronic illness). Key indicators include:

 Number of women and vulnerable groups and percentage of total beneficiaries

 Changes in knowledge, outputs, income and decision-making of women and vulnerable groups

 Existence of Programme and project gender and vulnerable group inclusion strategies.

Environmental sustainability: The ASARECA approved Environmental and Social Management Framework (ESMF) and the extent of awareness and compliance with Environmental and Social Safeguards (ESS) was assessed from a review of ESS documentation and interviews with project managers. The evaluation also examined major impacts to potentially affected ecological and socioeconomic areas of selected projects. Practices of project beneficiaries were examined through the household survey to assess the adequacy and implementation of controls and practices to mitigate adverse impacts (e.g. around use of pesticides).

Sustainability and scaling-out: Lessons on project sustainability were assessed through document review and key informant interviews including how EAAPP best practice technologies and processes are being integrated with national innovation systems - including official National Agricultural Research Systems (NARS), the private sector and farmer organisations. Evidence was also collected on how EAAPP has contributed to strengthening national research and policy-making processes in different member countries.

Programme design and management: Strengths and weaknesses of project design and implementation, including in targeting and reaching beneficiary groups, were assessed. Likely effects of organisational structure, staffing, procedures and management approach on project implementation and results were analysed. Impact of enabling and constraining factors on programmes and how these have been handled by EAAPP were assessed.

Lessons: The evaluation looked at how EAAPP technologies, innovations and processes contributed to project outcomes, or not (intended and unintended impacts), drawing out lessons for development, targeting and implementation of future programmes.

6 Methodology The methodology used involved a mix of participatory and conventional data collection methods from both primary and secondary sources for the four priority commodities (cassava, dairy, rice and wheat) across the four project countries. The approach combined quantitative data collection and analysis of statistically representative data and qualitative methods that permitted in-depth analysis and assessment of the quality of implementation, outputs and results.

1. Desk Review: Review of relevant project documents included: Project Appraisal Document (PAD), Project Implementation Plan (PIP), Performance Monitoring Plans (PMPs), Progress and Performance Reports, End-of-project reports and Case studies of project innovations, lessons and impacts etc. This was used to guide site selection, stakeholder interviews, site/field visits and farmer interviews, and as a benchmark for outcomes (verified during the evaluation). Secondary data and literature were also researched to inform Goal and Purpose indicators, including FAOSTAT data.

2. Key informant interviews: A checklist based on questions in the Evaluation Framework (Appendix 1) guided the interviews that were carried out during field visits to projects in the four EAAPP countries (Uganda, Tanzania, Kenya, and Ethiopia). Interviews specifically targeted stakeholders from public agricultural research and extension institutions, agricultural R&D specialists, policy makers, donors, EAAPP staff, NGOs, private sector, farmer organisations, Universities and other stakeholders involved in implementation of the project. Similarly, officials from government ministries were interviewed to assess any changes in policy harmonization with respect to cross-border transfer of germplasm, knowledge and information, capacity development of scientists, partnership and sustainability issues.

3. Focus group discussions: This method was utilized to collect qualitative primary data by soliciting information from groups of people whose views related to EAAPP beneficiary assessment. The FGD had the advantage that the participating members were able to state how they benefitted from the project and had the opportunity to suggest ways of improving service delivery. The discussions were guided by evaluation framework content relevant to the issues of the survey. Male and female only group participants, as well as mixed groups were interviewed, depending on the sociocultural context. The results of the FGD were analyzed and used to give meaning to the quantitative indicators and to provide data on contextual factors that are relevant for beneficiary assessment.

4. Beneficiary impact assessment: Structured questionnaires were administered to 1,239 households in the four project countries, of which 671 were direct EAAPP project beneficiaries and 568 non- beneficiaries, as a control (Table 1). Non-beneficiaries were selected with similar socio-economic

characteristics to beneficiaries. They were further stratified into Control 1 (C1) – 281 (spillover, neighbouring beneficiaries) and Control 2 (C2) – 287 (a minimum of 10 Km away from beneficiary villages). Beneficiary households were randomly selected from databases provided by project managers in each country. Around 81 percent of beneficiary households interviewed were male headed and 19 percent female headed; compared to 79 and 21 percent of non-beneficiary households respectively.

The investigation included: demographic profiling of beneficiary and non-beneficiary households, farmer exper iences and evaluation of technologies developed and disseminated under the project, membership of commun ity groups, changes in key crops and livestock production and utilization, changes in income and assets, househ old nutrition, decision-making within the household (gendered) and environmental knowledge and practices. T he emphasis was on changes occurring at household level since project implementation commenced in 2010.

Data analysis

Data analysis was based on triangulation of qualitative and quantitative data from relevant sources, content analysis, computation of frequencies and analysis of difference from the standardized instrument using SPSS and interpretation of qualitative analysis.

Methodological and Data Collection Challenges

A number of methodological and practical challenges were faced during the impact assessment. Although baseline data was available for some sites this did not cover all areas in the evaluation. This necessitated use of recall survey methods and sampling of control households. Recall methods produce less accurate data, particularly over the 5 year period for which data was sought. Farmer estimates of production and farm area are also likely to have inaccuracies: however, there is no indication of systematic bias. Paired samples of equivalent numbers of control (non-beneficiary) households were interviewed in neighbouring villages to project sites.

Table 1 Household survey respondents by country and commodity

Country Commodity Beneficiary Non- Non- Non- Total Beneficiary Beneficiary beneficiary (C1) (C2) Total ETHIOPIA Cassava 51 15 26 41 92 Wheat 117 38 70 108 225 Total 168 53 96 149 317 KENYA Rice 42 17 16 33 75 Dairy 125 59 34 93 218 Total 167 76 50 126 293 TANZANIA Wheat 41 18 13 31 72 Rice 114 58 51 109 223 Total 155 76 64 140 295 UGANDA Cassava 144 57 62 119 263 Dairy 37 19 15 34 71 Total 181 76 77 153 334 EAAPP Total 671 281 287 568 1239 Source: Evaluation survey

Quality Assurance

A Quality assurance framework was dawn up for the assignment with appropriate actions taken to ensure quality of process and findings (Table 2). This included:

Recruitment, Training and Fieldwork supervision: Teams of enumerators and supervisors competent in carrying out field work specified in the terms of reference and particularly in conducting individual interviews were recruited and trained detailing specific requirements of the survey including the questionnaire. Each team was headed by a supervisor who quality assured work in the field. The ability to interpret the tools in local dialect was an important criterion for selection of field supervisors and enumerators.

Table 2 Quality assurance process

Issue Action to ensure quality Theoretical underpinnings Evaluation framework with clear logic Robust survey tools Relevant questions Evaluation rigour Pre-test of survey instruments and training data collectors Careful data collection and recording Systematic, thorough analysis Researcher conduct Ethical behaviour – gain consent Participation of target group Respect for different perspectives Credibility of results Triangulation Validation by informants and peer review Link between data and conclusions Utilisation of findings Relevance to policy Link to research Clear reporting and active dissemination Source: Evaluation team

7 PROJECT IMPACT

7.1 Overall project purpose – performance against indicators

The EAAPP Project Development Objective (PDO), as stated in the PIP, is to: Enhance regional specialisation in agricultural research; Enhance collaboration in agriculture training, and technology dissemination; and Facilitate increased transfer of agricultural technology, information and knowledge across national boundaries.

In the EAAPP Key Results Area (KRA) six Verifiable Indicators are specified. These outcome indicators were assessed by the Impact Evaluation team using primary and secondary sources:

Indicator 1: Rate of change in regional specialization and collaboration in agricultural research The rate of change in regional specialization and collaboration in agricultural research is a composite indicator which has several components. This indicator relates to the joint planning and implementation of agricultural research, training and dissemination activities in at least two EAAPP project countries.

Progress is measured by identifying numbers of regional sub-projects (30% weighting) and training and dissemination activities (30%), advances in policy harmonization (10%), monitoring evaluation systems (15%) and capacity development (15%). The average baseline value for this indicator across the four countries was 10% in 2010, ranging from zero in both Kenya and Tanzania to 35% in Uganda. The baseline figures do not fully take into account projects funded through ASARECA’s previous Networks, Programmes and Projects and its current programme structure which cover the four commodities within EAAPP. Similarly, with regard to training and dissemination, initiatives such as the East Africa Dairy Development Project were operational at the start of EAAPP.

Based on data received from the four countries the ASARECA M&E unit estimates the average level of regional specialization and collaboration to be 63%, indicating an increase of 53 percentage points above the baseline and exceeding the mean target increase. The greatest contribution to the increase is considered to come from joint planning and implementation of agricultural research, training and dissemination among the partners. By September 2014 a total of 29 regional research sub-projects had been initiated, 21 of which involve participation by organizations in each of the four countries (Table 3). The research topics for some of the 29 sub-projects are not directly relevant to all of the countries. For example, crop and animal diseases such as Napier grass stunt and east coast fever are not present in Ethiopia. This accounts for why Ethiopia is not involved in sub-projects addressing these constraints.

Regional research projects

The high degree of country participation in regional sub-projects is noteworthy. This demonstrates that countries believe the research issues to be relevant to their needs and are willing to commit resources to participate in them. There are differences among sub-projects in the level of participation of different countries. This is partly due to a planned process in which individual countries are involved in components of sub-projects in which they have a particular interest or have the expertise to make a significant contribution. In some cases, however, the involvement of some countries was lower than planned and this has adversely affected the progress of activities. In Ethiopia, there were difficulties for staff to obtain authorization to travel overseas for meetings. This situation appears to have improved as the project has progressed. Also in Ethiopia, there has been a high turnover of staff involved in sub-projects and this has had an impact on the rate of progress in implementation. A common constraint across all countries has been delays in procurement and in the disbursement of funds to sub-projects. These issues are discussed further below.

Table 3: Number of regional research sub-projects and degree of partner country participation

Commodity No. regional projects No. countries participating Planned (actual) 2 3 4 Cassava 8 (8) - - 8 Dairy 5 (5) - 5 - Rice 15 (10) - 2 8 Wheat 6 (10) 1 2 7 Total 34 (33) 1 9 23 Source: Country implementation progress reports for 2014

The number of sub-projects under different thematic areas is shown in Table 4. Some sub-projects address more than one thematic area. Therefore, sub-projects have been assigned to the thematic area which is most central to their activities. This analysis reveals that there is no obvious omission in the thematic areas covered and that there is a fairly even distribution of sub-projects across these themes. It is arguable that there is an imbalance in the portfolio of sub-projects on cassava and wheat. Pests and diseases are a major

22 EAAPP I Evaluation Final Report NRI AfrII constraint to production of both crops, but three out of eight sub-projects on this theme represents quite a high proportion of the overall research effort. Whilst, there are several sub-projects addressing value chain and marketing issues there is a relatively weak emphasis on mechanization and post-harvest issues across all commodities. This will need to be reviewed if a new phase of the project is approved, given the high potential for beneficial impacts from advances in mechanization, transport, storage, processing and packaging. These impacts are especially important for women; for example in reducing drudgery and increasing income-earning opportunities.

Table 4: Thematic coverage of regional research sub-projects

Thematic area Commodity Total Cassava Dairy Rice Wheat Crop or animal breed improvement 2 1 1 1 5 Crop management - - 3 2 5 Pests and diseases 3 2 2 3 10 Mechanization/post-harvest - - 1 2 3 Value chains/ marketing 2 1 1 1 5 Regulatory/policy - 1 1 - 2 Promotion/dissemination 1 - 1 1 4 Total 8 5 10 10 3329 Source: Evaluation team, derived from EAAPP reports and interviews

Figure 1 Increased regional specialisation in agricultural research

Research capacity - scientists

The project has undertaken short- and long-term training of scientists in order to enhance human resource capacity in the regional centres of excellence. Each country conducted a capacity needs assessment and the assessments formed the basis for identifying both staffing needs and training requirements to address capacity gaps. Long-term training has taken the form of support for Masters and PhD students in priority disciplines (Table 6). The disciplinary areas chosen for the studentships are judged to be relevant. However, in view of the greater capacity gaps in post-harvest and value chain/marketing a proportionally higher level of investment in these areas would have been justified.

The number of research scientists available to contribute to activities in the regional centres of excellence is an important measure of the capacity of national systems to conduct regional research in the priority

23 EAAPP I Evaluation Final Report NRI AfrII commodities. This number has increased substantially in each country since 2010, most notably in Ethiopia (Table 5). There has been an increase in the number of scientists in each qualification category. Perhaps of greatest significance for the increased capacity to conduct high quality research is the increase in the number of PhD holders. It should be noted that the minimum entry level for an agricultural scientist is generally considered to be a Bachelors degree but the ‘Other’ category includes technicians and other staff with a lower qualification. Therefore, the increase in the number of research scientists is encouraging but it should be placed in context. The figures do not necessarily imply that there has been an overall increase in human resource capacity to conduct agricultural research in the countries. Much of the increase is likely to be due to redeployment of existing scientists, rather than hiring of new staff. In Uganda, some of the increase is attributable to the hiring of new staff as consultants rather than permanent staff and so it is not guaranteed that they will remain within the research system after the project ends.

Table 5: Numbers of scientists available to conduct research in regional centres of excellence

Qualification/country Kenya Ethiopia Tanzania Uganda BSc 28 (12) 35 (-) 39 (15) 17 (7) MSc 42 (31) 64 (12) 77 (32) 31 (17) PhD 26 (13) 23 (4) 31 (13) 12 (8) Other 84 (14) 52 (8) 73 (33) 27 (13) Total 180 (70) 174 (24) 220 (93) 87 (45) Note: Figures in brackets denote the baseline numbers in 2010 Source: ASARECA M&E Summary 2014

The available data do not show the disciplinary expertise of the research scientists and so it is not possible to assess whether there is enhanced capacity in the disciplines where this is most needed. Nor does it necessarily follow that the scientists have the type of skills that will enable them to participate effectively in regional research; for example, the capacity to do multi-disciplinary research and to engage effectively with other actors. However, this can be addressed by training.

Table 6: Number of postgraduate students in EAAPP countries supported through the project

Country Masters Masters PhD Male Female Total completions Male Female Total Tanzania 7 7 14 6 6 0 6 Uganda 6 5 11 2 8 0 8 Kenya 24 12 36 7 9 5 14 Ethiopia 12 2 14 5 6 2 8 Total 49 26 75 20 29 7 36 Source: Country implementation progress reports for 2014 * = data not available

Seventy-five Masters students and 36 PhD candidates have been fully funded by the project (Table 6). The highest number are from Kenya. Looking at the gender of postgraduate students, similar numbers of men and women registered for Masters programmes in Tanzania and Uganda but only 2 women in Ethiopia. There were only 7 female PhD students amongst a total of 36 across the four countries. Identifying female candidates for PhD studentships in agricultural research in East Africa is a well-documented challenge. Nonetheless, the limited number of female students is a setback and the project should review the selection process and learn lessons that will guide future recruitment processes. In Ethiopia, a total of 44 further PhD students have received partial funding from the project. Similarly, in Kenya, a further 7 PhD students are partially funded by the project.

Further capacity development is needed in livestock breeding (for animals and fodder) and value addition (‘functional products’). There is also limited capacity in genomics.

Acquisition of relevant infrastructure according to plan Improving research infrastructure and equipment was considered a key component of building capacity to support regional specialization. The project has supported new acquisitions and rehabilitation of a broad range of infrastructure and equipment including: laboratories, resource centre, resident blocks, vehicles, farm machinery, irrigation facilities, offices, Information and communication technology, laboratory equipment, water systems, greenhouses.

In Kenya, the focus is on construction of a resource centre and accommodation at the RCoE in Naivasha. In Tanzania, infrastructure has been split between: rehabilitation of the Agricultural Seed Agency (ASA) Seed farm, which was seen as a priority for the government, and renovation of labs. In addition, all countries have bought vehicles and ICT equipment.

Progress in acquisition of infrastructure is measured against plans. In all cases progress has been behind schedule, mainly due to slow government procurement systems which are outside the control of the project. By 2014 EAAPP PMP was reporting an acquisition rate of 83.6 percent compared to 99 percent target. A sizeable proportion of the equipment had not been commissioned by the time of the evaluation, though this was expected to have happened by the beginning of 2015. Civil works are particularly slow: ranging from 45% (Uganda) to 70 percent in Kenya. Completion of infrastructure should be a priority during the no-cost extension period.

The delay in the completion of the labs and commissioning of the lab equipment has affected the type and quality of the research conducted during Phase 1. Similar delays, such as that to the ASA seed farm development programme mean that increased seed production is not fully on stream. However, commissioning of all buildings and equipment is a key objective of the No-Cost Extension phase. The significant improvements in capacity to conduct research, support dissemination and host regional research exchanges should deliver results in a potential Phase 2.

Indicator 2: Rate of increase in information and knowledge transfer across national boundaries

An early activity in the project was to identify promising commodity-based agricultural technologies that have potential for scaling out, both within the country in which they originated and in other EAAPP partner countries. ‘Technologies’ are defined as: technologies, innovations and management practices (TIMPs). Guidelines were provided by ASARECA on the information needed to document technologies in an inventory of the target commodities in each of the countries. These guidelines on the documentation of the agricultural technologies include eleven points, many of which relate to information that indicates the suitability of the technologies for a given biophysical and socio-economic environment. These were intended to characterize the technologies in a way that would help potential users to assess their suitability. Careful thought has been given to the guidelines and they provide a useful basis for evaluating the scope for adoption of a technology.

The degree to which the guidelines were followed and the information shared among partners seems to have varied among countries. In Tanzania, for which detailed documentation on technologies for rice, cassava and wheat is available on the regional rice centre of excellence website, the guidance was followed closely. The amount of publicly-available information on technologies for the target commodities in the other partner countries is considerably lower. For example, there is a list of wheat technologies on the

25 EAAPP I Evaluation Final Report NRI AfrII website of the regional wheat centre of excellence. No details are provided other than the name of varieties and type of agricultural machinery or crop management practice.

According to the data compiled by ASARECA there were a total of 344 technologies identified for the four commodities available in the four countries, with the majority (211) identified in Kenya. By September 2014, a total of 138 ‘new’ technologies had been developed by the regional centres of excellence, distributed fairly evenly across the commodities. Details of these technologies are not readily accessible but from country progress reports and discussions with project staff it appears that many of these are new varieties of cassava, rice, wheat and forage crops. For example, in Ethiopia eight wheat varieties have been developed and in Uganda seven new cassava varieties are listed in the most recent progress report.

Twenty-three new technologies have so far been disseminated across national borders (EAAPP PMP 2014). These include: two Tanzanian rice varieties released in both Kenya and Uganda, and undergoing National Performance Trials (NPT) in Ethiopia; four clones of Napier grass from Kenya recommended for dissemination in Uganda; botanical seed of cassava with enhanced carotene sent to Ethiopia, Tanzania and Kenya; assisted reproductive technologies from Kenya sent to the other countries (Box 1). This is a considerable achievement given the restrictions on transfer of crop varieties and cattle semen between countries.

Box 1: New technologies disseminated across national borders under EAAPP I  Two Tanzanian rice varieties released in Kenya: TXD306 and Komboka  Two rice varieties released in Uganda  Four rice varieties under NPT in Ethiopia in 2014 - expected to be released in 2015  Four clones of Napier grass from Kenya (16702, 16805, Kakamega 1 and Kakamega 2) have shown high tolerance to Napier stunt disease and good yields in Uganda and have been recommended for dissemination to farmers.  Botanical seed of cassava from Uganda with enhanced beta carotene sent to Ethiopia, Kenya and Tanzania.  High yielding cassava varieties from Uganda (mainly NASE 14) are being grown in Ethiopia and are contributing to a significant increase in production, stimulated by the demand for cassava flour as a cheaper substitute for wheat flour in products such as enjera.  Two farms in Kenya have sold 154 young bulls to buyers in Tanzania.  Tube silage making technology transferred from Kenya to Uganda.  Assisted reproductive technologies from Kenya to other countries  Bull semen from Kenya to other countries

Source: EAAPP country progress reports

A sub-project on animal genetic improvement led by the Kenya Agricultural and Livestock Research Organisation (KALRO) in Kenya provides a good example of how regional benefits can be captured in spite of such restrictions. The researchers originally planned to exchange cattle breeds among countries but they soon realized that there were different preferences and requirements in their respective countries. Consequently, the emphasis was placed on developing and sharing common approaches to breeding and capacity building. The aim of the project was to increase the dairy production potential of indigenous Zebu breeds through targeted selection and cross-breeding.

Breeding schemes involving the use of both artificial insemination and bulls were established in each of the four countries. Sites were chosen near the Kenya, Uganda and Tanzania borders to facilitate interaction among the partners. Assisted reproductive technology (ART), involving hormone treatment protocols, was

26 EAAPP I Evaluation Final Report NRI AfrII used to accelerate genetic gain in selected locations in Kenya. This led to a substantial improvement in the number of successful calvings and as a result over 470 young breeding animals were reared for distribution. ART was also used to support artificial insemination schemes and camps in Kenya and Uganda each delivered more than 800 inseminations. Thus the ART approach has good potential to accelerate gene flow and, in time, genetic improvement for dairy production among Zebu cattle populations.

Figure 2 Animal genetic improvement: Development and sharing of common breeding approaches

Another example of the value of a regional approach is seen in the sub-project on "Increasing rainfed wheat productivity in Eastern Africa through variety development (Bread and Durum Wheat)". A substantial number of the technologies identified in the baseline inventory were improved wheat varieties. These varieties were widely grown but their productivity was declining due to their susceptibility to rust diseases. This is a clear case in which regional collaboration is essential to address a problem that affects all the countries and for which a solution is urgently needed. A wide range of screening and breeding activities have been carried out in the project. These activities include screening germplasm in the international nursery, making crosses and evaluating segregating populations, conducting yield trials, and carrying out maintenance breeding and seed production. A key consideration in this research is to identify sources of resistance which are effective in given locations and to introgress this resistance into locally adapted improved varieties which are preferred by farmers. Thus the requirement is to develop a range of varieties which are suitable for particular well-characterized locations, rather than attempt to produce a small number of varieties for wide adaptation.

Figure 3 Increasing rainfed wheat productivity

Regional Dissemination: The programme has used various methods to facilitate the dissemination of new technologies between countries. These include exchange visits for researchers, farmers, and agricultural extension officers; attendance at agricultural shows by a wide group of stakeholders, including traders; regional training activities; and electronic platforms such as web sites. For example, a rice processor from Tanzania was able to forge links with regional markets at the Kenya agricultural show and is now exporting to Kenya. Exchange visits are valued by those who have been involved. Researchers who were interviewed

27 EAAPP I Evaluation Final Report NRI AfrII stated that such visits were essential to develop a clear understanding of the research issues being addressed by sub-project teams. Such interaction is clearly an important part of the research process. However, a challenge for the project is to identify and support ways that will facilitate the dissemination and adoption of technologies through institutionalised processes.

The project has generated a large amount of dissemination materials in the form of leaflets, booklets, posters and manuals. The extent to which this has been done is commendable and, based on the samples viewed, the information provided is useful and is well presented. All countries have prepared communication strategies: the strategy developed for Tanzania is available on the website of the regional rice centre of excellence. The Tanzania strategy identifies a variety of communications channels and media that are appropriate for different target audiences. These include digital tools that have the potential to enhance interaction (such as Web 2.0 tools and videoconferencing) or expand the reach of the target audience (FM radio, television, video). There is not much evidence that the project has exploited the full potential of such communication approaches; or that it has supported activities that document and share learning on the situations in which a particular communication approach has proved to be valuable (See Section 4).

Indicator 3: Rate of change in adoption of new technologies

Adoption refers to acceptance and practice of agricultural technology, i.e. application and continued use. It is the point at which a technology (breeds, new varieties, and management practices) is chosen or selected for use by individuals or organizations. Therefore, rate of adoption is measured as a percent of the baseline value, and is a function of government policies, technological change, environmental concerns, demographic factors, market forces, delivery mechanism, and availability of information and education, etc. The technologies for tracking include: new varieties of pasture and fodders, new or improved breeds, management practices, models, approaches and systems.

This indicator seeks to assess the number of existing and new varieties, breeds, and other selected management practices preferred by farmers. EAAPP M&E surveys show an increase in adoption from 35 percent on average to 53.2 percent (2010-2014) in EAAPP project areas. The breakdown by country and commodity is shown in Table 7. Particularly large increases are seen in Ethiopia (all commodities) and Kenya. Smaller increases are seen in Tanzania.

Table 7 Adoption of new varieties, breeds and management practices (% Households using in EAAPP project areas)

Improved practices Kenya Ethiopia Tanzania Uganda 2010 2014 2010 2014 2010 2014 2010 2014 New Varieties 32 82 2 30 28 32 67 77 Cassava Mgt. practices 43 83 2 30 59 69 67 77 New Varieties 8.5 56 4 10 7 8 36 20

Dairy Breeds 21.3 51 1 12 18 21 56 Mgt. practices 45 84 2 12 39 47 36 67 New Varieties 20 97 4 15 35 44 54 5 Rice Mgt. practices 40 86 2 15 77 68 70 25

New Varieties 78 90 36 65 87 97 12 0 Wheat Mgt. practices 30 88 36 65 64 79 3 4 Source: EAAPP PMP 2014

The Evaluation team assessed use of new varieties and breeds by EAAPP beneficiary and non-beneficiary households surveyed (Table 8). Findings show that the proportion of beneficiaries using improved varieties of cassava, wheat and rice has increased in all countries between 2009 and 2014. This increase was statistically significant in all countries except for cassava farmers in Ethiopia. In 2014 65 percent of project households were using improved cassava, 87 percent improved rice varieties and 97 percent improved wheat. Adoption of improved breeds of dairy cows by beneficiary households has also increased.

There is a close match in findings from the Independent evaluation and EAAPP PMP on use of improved varieties by project beneficiaries for: rice in Kenya (97 percent in both sources) and wheat in Tanzania (97 percent). Uganda figures for cassava from the PMP are higher than the evaluation team’s (77 to 58 percent). For Ethiopia the evaluation team findings are considerably higher compared to the PMP figures. These differences may be due to differences in sampling, or increased uptake since the last reported EAAPP survey.

Table 8 Use of improved varieties and breeds by beneficiary and non-beneficiary households (%) Country Crop 2009 2014 p-value p-value 2009-2014 Ben. Non Ben. Non 09 14 Ben. Non ETHIOPIA Cassava 68.4 80.0 95.5*** 66.7 0.447 0.001 0.104 0.189 Wheat 63.7 60.2 97.5* 82.4 0.613 0.000 0.000** 0.000** KENYA Rice - - 97.1 96.3 - 0.868 ** ** Dairy cows 90.5 89.2 96.4 92.9 0.788 0.261 0.033* 0.159 TANZANIA Wheat 55.6 25.0 96.7** 43.3 0.049 0.000 0.004** 0.163 Rice 28.4 35.7 84.8** 64.7 0.151 0.001 0.000** 0.000** UGANDA Cassava 43.0* 30.0 58.0** 37.3 0.039 0.001 0.000** 0.012* Dairy cows 9.5 7.7 12.2 10.5 0.855 0.851 0.329 0.433 ALL Cassava 45.6 37.0 64.8** 43.1 0.143 0.000 0.000** 0.127 Wheat 61.8 51.9 96.7** 73.7 0.257 0.000 0.000** 0.000** Rice 28.7 36.0 86.6** 70.7 0.277 0.001 0.000** 0.000** Dairy cows 73.9 75.6 73.9 77.7 0.972 0.487 0.019* 0.159 Source: Evaluation survey

Indicator 4: Rate of increase in land area with seeds of improved cultivars

EAAPP M&E estimates of land planted with improved cultivars are 2,755 ha in 2010, increasing to 12,807 ha in 2014 (breakdown in Table 9). This increase of 10,000 ha (368%) is attributed to the substantial increase in production of planting material and farmer awareness in EAAPP project areas. Large increases are reported in Ethiopia, Kenya (cassava, rice, wheat) and Uganda (cassava).

Table 9 Land Under improved cultivars (Ha) Improved practices Kenya Ethiopia Tanzania Uganda 2010 2014 2010 2014 2010 2014 2010 2014 Cassava 75.9 560.4 30 201 23.9 40.7 493 1941.1

Dairy 482.6 462.4 28.5 121 35 47 17 Rice 120 868.3 150 616 141.7 187 151 32.8 Wheat 108 2886.8 688 4906 144.5 157.2 66.3 Source: EAAPP PMP 2014

It was not within the scope of the evaluation to independently assess increase in total area under improved varieties. However, interviews with beneficiary groups and key informants for each of the commodities in the four countries confirmed sizeable increases in areas planted. For example, improved wheat and rice varieties were introduced to farmers in new areas in Kenya by the project and uptake has taken off rapidly over the past 4 years.

Indicator 5: Increase in productivity at farm level over control technology for all disseminated new technologies

Changes in productivity (ratio of outputs to inputs) over control technology recorded by EAAPP PMP are shown in Table 10. Almost all technologies performed positively over control technologies, ranging from zero to 8% between 2010 and 2013. Large improvements in productivity of new technologies over controls were recorded in wheat (Kenya and Uganda) and cassava (Kenya). Rice and dairy technologies showed lower increases in over the project period (rice in Uganda showed no improvement over control technology).

Table 10 Change in productivity of EAAPP technology over control technology at farm level (%)

Commodity Kenya Ethiopia Tanzania Uganda 2010 2013 2010 2013 2010 2013 2010 2013 Cassava 5.3 13.8 2.5 5 5.21 7.1 5 5.7 Dairy 3.8 4.8 3 3.2 2.8 3.2 1.3 1.48 Rice 3.2 3.6 1.5 3.3 1.27 1.8 1.24 1.24 Wheat 2.85 9.2 1.25 3 6 7.4 2.2 22.6 Source: EAAPP PMP

Computing changes in total productivity at smallholder farm level requires detailed data collection over a regular period and was not within the scope of the evaluation. However, findings by the evaluation household survey on output per unit of land indicate far higher increases in land productivity (yield per hectare table) for all commodities (except cassava in Uganda). Whilst this is only a partial measure, it suggests that EAAPP PMP figures represent the lower limit of productivity change achieved under the programme.

Indicator 6: Level of stakeholder satisfaction with the technologies and innovations by number of p roducts users

This indicator measures stakeholder appreciation of the attributes of the new technology or innovation and the extent to which they are accepted by end-users (Table 11). EAAPP programme measures show farmer satisfaction with technologies has increased from an average of 22.8 percent to 68.8 percent of households in targeted project areas. The greatest increases in satisfaction were seen in cassava and wheat. This may be partly the result of disease resistance traits of the new technologies.

Table 11 Level of stakeholder satisfaction with technologies and innovations by users (%)

Kenya Ethiopia Tanzania Uganda Total 2010 2014 2010 2014 2010 2014 2010 2014 2010 2014 Satisfaction 66 84 56 75 20 74 50 78 48 77.8 by users % Source: EAAPP PMP reports

The evaluation survey assessed farmers’ satisfaction with the EAAPP technologies as shown by effect of the improved technologies and innovations on production and incomes. Percentage of farmers reporting a positive effect of EAAPP technologies to date is shown in Table 12. This indicates that around a quarter to half of beneficiaries had experienced improvements in their production or incomes as a result of adopting the technologies. Note that the figures are lower than EAAPP PMP figures since respondents did not receive training in all EAAPP technologies; and because this a tougher measure - it looks at satisfaction as evidenced by impact on production.

Table 12 Stakeholders reporting increased output as a result of improved technologies, 2014 (%)

Kenya Ethiopia Tanzania Uganda All Cassava - 37 - 53 46 Dairy 45 - - 78 53 Rice 74 - 66 - 68 Wheat - 35 12 - 29 Source: Evaluation survey

7.2 Conclusions on performance against programme purpose

Overall, project targets have been met and, in some cases, exceeded. Thus the overall assessment is quite positive. Regional research projects are on target, though with some variation between commodities, as are training targets.

All EAAPP partner countries had relatively high increases in their agricultural research and development spending between 2000 and 2011. They also each experienced an increase in the number of agricultural researchers during the same period, but the magnitude of this increase was much greater in Ethiopia 1. In each of the countries, Bachelors degree holders accounted for a large proportion of the increase. Due to freezes in public sector recruitment in the 1990s the age structure of staff became skewed towards senior scientists. Many of these senior scientists are now approaching retirement age of have already retired. This has led to a problem of mentoring of younger researchers; as, for example, has been documented in Tanzania2. It will be important in a Phase 2 of the project to build in mechanisms for mentoring returning students and integrating them effectively in RCoE activities. One option would be to have a specific provision requiring the involvement of a returning student in any new research proposal.

The training of staff at Masters and PhD level through EAAPP will help to redress this imbalance and should bring wider benefits within the respective NARS, beyond the target commodities in the program. However,

1 Beintema, N. and Stads, G. (2014) Taking Stock of National Agricultural R&D Capacity in Africa South of the Sahara. A STI Synthesis Report. International Food Policy Research Institute, Washington.

2 Beintema, N., Lwezaura, D. and Rahija, M. (2013) Agricultural R&D Indicators Fact Sheet: Tanzania. International Foo d Policy Research Institute and Department of Research and Development.

31 EAAPP I Evaluation Final Report NRI AfrII it is also important that governments recognize the need to provide adequate incentives to ensure that this new human resource capacity is retained. The departure of staff to better paid positions in the private sector and academia is to some extent inevitable, but reward systems need to be adjusted to ensure that the gap in remuneration does not become too large. In the case of Ethiopia, the government stipend provided to PhD students is very low and does not incentivize staff already in paid employment to register. EAAPP partner countries should give serious consideration to this issue if the recent human capacity gains are to be sustained and this could be included in the policy agenda for a possible second phase of the project.

It is important to have accurate information on the adoption rate of improved crop varieties in order to be able to measure the impact of investment in crop breeding and associated extension activities. An analysis of historical data revealed that adoption levels of improved varieties vary substantially, and that by the late 1990’s estimated adoption levels of wheat, rice and cassava were 66%, 45% and 22%, respectively across sub-Saharan Africa3. A subsequent dataset for 2010 did not include figures for wheat and rice but showed that adoption levels of cassava had increased significantly to 39%. More recent surveys using genetic characterization of varieties grown by farmers indicates that adoption rates based on household surveys are generally underestimates (L. Oruko, personal communication) with farmer to farmer diffusion of good- fit technologies, and so the actual levels of adoption are likely to be higher.

3 Alene, A., Yigezu, Y., Ndjeunga, J., Labarta, R., Andrade, R., Diagne, A., Muthoni, R., Simtowe, F., and Walker, T. (201 1) Measuring the effectiveness of agricultural R&D in sub-Saharan Africa from the perspectives of varietal output and adoption: Initial Results from the Diffusion of Improved Varieties in Africa Project. Paper presented at the ASTI/IFPRI- FARA Conference on Agricultural R&D: Investing in Africa’s Future, Accra, Ghana, 5-7 December 2011. International Fo od Policy Research Institute, Washington. Note: the surveys on which the data are based were taken from a selection of countries; five for wheat, ten for rice an d eleven for cassava.

8 PROGRAMME EFFECTIVENESS – BENEFICIARY IMPACT

8.1 Introduction

An assessment of the impact of EAAPP I at household level was based on findings from the Evaluation survey of 1240 beneficiary and non-beneficiary households. Findings cover: programme impact on output and productivity of key commodities, Incomes, overall farmer evaluation of technologies, food security, equity/Inclusion and scaling-out to non-beneficiaries.

8.2 Production of key commodities

The report presents output data on key commodities: rice, cassava, wheat and dairy. Average production (kg) of key commodities by beneficiary and non-beneficiary households sampled for 2009 and 2014 is given in Table 13 (farmer estimates). Production for both beneficiaries and non-beneficiaries has increased over the project life for all commodities except wheat, where non-beneficiary production fell. Beneficiaries have seen significant increases in their production of all EAAPP commodities in all four countries, except milk in Uganda. For non-beneficiaries, significant increases have been seen in Kenya and Ethiopia (cassava).

Table 13: Average total production of key commodities by surveyed EAAPP beneficiary and non-benefici ary households in 2009 and 2014 (Kg)

2009 2014 2009-2014 Non- Non- Benef- Non- Beneficiary Beneficiary Beneficiary Beneficiary iciary Benef. % Country Commodity n Mean n Mean n Mean n Mean % change change Ethiopia Cassava 24 598 15 503 49 3718* 32 2902 522%* 477%* Wheat 107 2379 89 2724 115 3115 104 1091 31% -60% Kenya Rice 7 50 0 0 18 210 19 126 320%** ** Milk 63 6286* 40 4355 107 8112 74 5560 29%* 28%* Tanzania Wheat 21 502* 17 359 29 1111 27 763 121%* 113% Rice 99 2195 87 1810 109 3921* 102 2615 79%** 44% Uganda Cassava 144 3560 104 2815 142 6436 107 4812 81%* 71% Milk 21 1276 10 416 33 1775 17 808 39% 94% ALL Cassava 168 4158* 119 3318 191 10154 139 7714 144%** 132%* Wheat 128 2881 106 3083 144 4226 131 1854 47%** -40% Rice 101 2245 87 1810 127 4131* 121 2741 84%** 51%** Milk 84 7562* 50 4771 140 9887 91 6368 31%* 33%* Source: EAAPP Evaluation household survey * Denotes significant change p<0.05; **Highly significant change p<.01

Average yields of EAAPP commodities obtained by surveyed project beneficiaries and non-beneficiaries indicate productivity improvements from the new technologies. Average yields for beneficiaries in 2014 were 14.6 tonnes per hectare for cassava, 7.2 tonnes per hectare for wheat and 8.8 tonnes per hectare in 2014. These exceed regional productivity figures for all rice and wheat (FAOSTAT 2015). Project beneficiaries experienced significant increases in yields of all commodities except cassava in Ethiopia and wheat in Tanzania. Increases for non-beneficiaries were smaller and not statistically significant, except for

33 EAAPP I Evaluation Final Report NRI AfrII cassava. Milk yields increased significantly from 7560 kg/animal to 9887 kg/animal for beneficiaries, whilst for non-beneficiaries, increases were smaller though still significant.

Table 14 Average yield per hectare of key commodities of surveyed EAAPP beneficiary and non-beneficia ry households (Kg/Ha) Country Crop 2009 2014 2009-2014 Beneficiary Non- Beneficiary Non- Beneficia Non- Beneficiary Beneficiary ry Benef. n Mean n Mean n Mean n Mean change change Ethiopia Cassava 24 2452 15 2726 115 7984* 104 3708 226% 36% Wheat 107 2668* 89 3710 49 4256 32 3968 60%** 7% Kenya Rice 0 0 0 0 18 1289 19 970 Tanzania Wheat 21 1146* 17 933 29 2900 27 1649 153% 77% Rice 99 3268* 87 2386 109 7524* 102 3229 130%* 35% Uganda Cassava 144 4723 104 5641 142 6598 107 7716 40%* 37% ALL Cassava 168 7175 119 8367 257 14582 211 11424 103%** 37%* Wheat 128 3814* 106 4643 78 7156 59 5617 88%** 21% Rice 99 3268* 87 2386 127 8813** 121 4199 170%* 76% Source: EAAPP Evaluation household survey * Denotes significant change p<0.05; **Highly significant change p<.01

Table 15 Average milk yield of surveyed EAAPP beneficiary and non-beneficiary households (Kg/animal/y ear) 2009 2014 2009-2014 Country Beneficiary Non-Beneficiary Beneficiary Non-Beneficiary Beneficiary Non-Benef. n Mean n Mean n Mean n Mean Change Change Kenya 63 6286* 40 4355 107 8112** 74 5560 86%* 28%* Uganda 21 1276 10 416 33 1775 17 808 327% 94% All 84 7561* 50 4771 140 9887 91 6368 107%* 33%* Source: EAAPP Evaluation household survey * Denotes significant change p<0.05 **Highly significant change

Despite these improvements there is large variability in yields of both beneficiaries and non-beneficiary households across all the four commodities (Figures 4a-d). This highlights that average yields are an insufficient measure of performance at farm level. A number of factors may be behind this variation: physical factors including soils and rainfall; input use (type, quantity, quality, timing) and management factors. It is recommended that variability at farm level is investigated during EAAPP II to understand causes and adapt technology recommendations. Farmer-centred research methods (e.g. Participatory Technology Development) which involve farmers in all stages of the research process would be most appropriate for this.

Figures 4a-d Yields of key commodities reported by beneficiary and non-beneficiary households in 2014 (Kg/ha; Kg/animal)

Wheat Yield Variability 2014

0 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 126

Beneficiary Non Beneficiary

Rice Yield Variability 2014

0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97

Cassava Yield Variability 2014

0 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 126 131 136 141

Milk Yield Variability 2014

0 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 126 131 136

8.3 Household incomes Incomes of project beneficiaries derived from EAAPP commodities were estimated by the evaluation team through valuation of the four key agricultural and livestock commodities. Net incomes are presented in Table 16: Incomes less input costs, using constant 2014 prices.

Net real incomes for key EAAPP commodities of surveyed beneficiary and non-beneficiary households have increased considerably over the project period. Non-beneficiaries have also increased their net incomes, partly through scaling-out effects to neighbouring farmers and communities. However, net incomes of beneficiaries from key commodities were significantly higher than those of non-beneficiaries in diary, wheat and rice. Project dairy farmers in Kenya received an estimated average annual net income of over $3000 in 2014; for rice farmers in Tanzania the average was over $1300, $943 for cassava farmers in Tanzania and $1136 for wheat farmers in Ethiopia. Low incomes were received by wheat farmers in Tanzania who complained of low prices. Returns for dairy farmers in Uganda were very low at $300pa, below the $1 per day minimum income level. Farmers would not be able to support their families on these enterprises alone.

Table 16 Average net value of crop and livestock production of key commodities by beneficiaries and no n-beneficiaries in 2009 and 2014 (US$ constant prices)

2009 2014 % change 2009-2014 Country Commodity Non- Non- Non- Beneficiary Beneficiary Beneficiary beneficiary beneficiary beneficiary Wheat 1,106 939 1136 794 52%* 25%* Ethiopia Cassava 99 60 399 145 497%** 257%

Rice 108 60 247% 1425% 42 5 Kenya

Dairy 3026 2365 94% 124% 2,125 1,438

Wheat 265 206 189% 127% 135 133 Tanzania

Rice 1084 726 82% 50% 877 712

Cassava 566 464 68% 46% 643 608 Uganda

Dairy 291 458 18% 206% 471 286

Wheat 905 785 995 53% 26% 1422

All Cassava 531 485 734 384% 180% 955

Rice 89% 60% 920 718 1,741 1,148

Dairy 2,596 1,724 4,091 80% 137% 4,672 Total All 928 711 1804 1262 91% 50% Source: Evaluation survey

8.4 Food security and nutrition

Food and nutrition security is integral to household wellbeing, although there are currently no indicators on household food or nutrition security in the EAAPP PMP. The Impact Evaluation used Food available to the household from any source relative to requirements in 2009 and 2014 as an indicator of household

Food Security. Beneficiaries with a food surplus increased significantly in Ethiopia, Kenya and Tanzania, with a non-significant increase in Uganda. The proportion of non-beneficiaries who experienced a surplus also increased significantly in Ethiopia and Tanzania but fell in Uganda. Beneficiaries in Ethiopia and Kenya were significantly more likely to be food surplus households than non-beneficiaries in EAAPP project areas in 2014.

Table 17 Food sufficiency of surveyed EAAPP beneficiary and non-beneficiary households, 2009 and 2014 (% of households)

Change 2009 2014 p-value Non- Non- Non- Ben Non Count Food Security Benefici Benefici Beneficia Benefic Benefici Beneficiar ry Status ary ary ry iary ary y Ethiop Surplus 16.7% 10.1% 37.5%** 17.4% 125%** 73%** 0.000 0.000 ia Met requirement 45.2% 50.3% 50.6% 63.1% 12% 25% Deficit 29.8% 32.2% 7.1% 18.1% -76% -44% Kenya Surplus 61.7% 61.1% 67.1%* 60.3% 9%* -1% 0.047 0.549 Met requirement 22.8% 23.0% 20.4% 25.4% -11% 10% Deficit 8.4% 6.3% 6.0% 7.9% -29% 25% Tanza Surplus 31.0% 26.4% 42.6% 34.3% 38%** 30%* 0.000 0.013 nia Met requirement 41.9% 44.3% 38.7% 43.6% -8% -2% Deficit 15.5% 14.3% 8.4% 11.4% -46% -20% Ugand Surplus 24.2%* 21.7% 28.6% 17.8% 18% -18% 0.115 0.670 a Met requirement 41.2% 36.8% 41.8% 47.4% 1% 29% Deficit 23.1% 33.6% 20.3% 29.6% -12% -12% Source: EAAPP Evaluation household survey * Denotes significant change p<0.05; **Highly significant change p<.01not add to 100 due to non-responses

Nutrition Security was assessed by the number of food groups consumed by household members in the past 24 hours (Table 18). Food consumption from all food groups was significantly higher for project farmers compared to non-beneficiaries surveyed for wheat in Ethiopia and dairy in Uganda, implying a positive impact on diversity of foods consumed in the household. However beneficiary rice households in Kenya were less likely to have a diversified diet than non-project farmers. This may be because these are households who have so far not seen an increase in incomes through the project. However, it is important that the project tracks the nutritional impact of its interventions: in this case the switching to rice may be reducing crop diversity.

Table 18 Surveyed households consuming from all food groups in the past 24 hours

Country Commodity Beneficiary Non-Beneficiary P-value Cassava 39.2% 41.5% 0.498 Ethiopia Wheat 56.4% 43.5% *0.036 Rice 61.9% 78.8% *0.092 Kenya Dairy 71.2% 64.5% 0.183 Wheat 48.8% 58.1% 0.294 Tanzania Rice 71.1% 65.1% 0.211 Cassava 78.6% 78.8% 0.546 Uganda Dairy 97.3% 55.9% **0.000

Cassava 68.4% 69.2% 0.904 Wheat 54.4% 46.8% 0.202 All Rice 68.6% 68.3% 1.000 Dairy 77.2% 62.2% **0.006 All 67.3% 61.9% *0.056 Source: EAAPP Evaluation household survey * Denotes significant change p<0.05; **Highly significant change p<.01

8.5 Overall status

Poverty self-assessment provides an indicator of household economic status. Households were also asked to assess their overall economic status relative to other members of the community in 2009 and 2014 on a six step scale. In 2009 there was no significant difference in economic status of beneficiaries and non- beneficiaries, except in Ethiopia, where beneficiaries reported being more likely to be poor or very poor than other community members. By 2014, all groups reported having significantly improved their status, with large numbers of households being lifted out of the very poor group. This may be the result of transmission of benefits to non-project members as well as to other interventions in the area. In 2014 beneficiaries indicated they were being significantly better off than non-beneficiaries in Uganda and Ethiopia. Thus beneficiary households appear to have improved their economic status relative to non- project members of their community over the life of the project. This indicates that technologies disseminated under EAAPP are beginning to have an impact at household level.

Table 19 Economic status of beneficiary and non-beneficiary households in 2009 and 2014 (self-reporting) (%)

2009 2014

Beneficiaries Non- Beneficiaries Non- Country Crop beneficiaries beneficiaries Ethiopia Very poor 10.8* 9.1 1.2** 1.3 Poor 47.6 33.6 3.0 8.1 Moderately poor 30.1 46.2 24.6 37.6 Moderately better off 10.8 10.5 52.1 41.6 Better off 0.6 0.7 19.2 11.4 Kenya Very poor 10.3 12.4 2.4 1.6 Poor 27.9 28.9 3.0 6.5 Moderately poor 37.0 38.0 16.5 26.6 Moderately better off 23.0 17.4 42.7 41.1 Better off 1.8 3.3 35.4 24.2 Tanzania Very poor 15.8 14.0 0.6** 2.9 Poor 54.6 52.7 18.7 24.1 Moderately poor 23.0 28.7 47.1 47.4 Moderately better off 4.6 3.9 20.0 16.8 Better off 2.0 0.8 13.5 8.8 Uganda Very poor 17.7 14.6 0.0* 3.4 Poor 40.0 41.7 10.9 13.1 Moderately poor 29.7 34.0 32.8 36.6 Moderately better off 9.1 7.6 40.8 40.7 Better off 3.4 2.1 15.5 6.2 ALL Very poor 13.7 12.5 1.1** 2.3

Poor 42.2 39.3 8.8 13.0 Moderately poor 30.1 36.9 30.0 37.3 Moderately better off 12.0 9.7 39.2 35.1 Better off 2.0 1.7 20.9 12.3 Source: EAAPP Evaluation household survey  Denotes significant change p<0.05; **Highly significant change p<.01

8.6 Profile of beneficiaries – Equity and inclusion issues

The evaluation examined equity and inclusion issues in ASARECA OP-1 projects, including: gender participation and decision-making; youth participation and participation by households with disability/chronic illness.

Gender

The Impact Evaluation looked at several aspects of gender, including inclusion of women and female- headed households in EAAPP projects and project impact on female-headed compared to male-headed households on yields, income and poverty status.

The survey team interviewed an average of 40-50 percent women farmers in the sampled projects. Project staff explained that they purposively recruit women farmers and in some cases have women only groups. An exception was dairy farmers in Kenya where some groups were found to be all male.

Average yields per hectare for key commodities reported by project beneficiaries by gender of household head are shown in Table 20. Before the start of EAAPP activities in 2009, yields of female-headed households were lower than those of their male counterparts for wheat, dairy and rice. Reasons may be lower access to resources – land, labour, inputs and capital; knowledge; and social and institutional constraints. Only female-headed cassava growing households in Uganda were getting higher yields than male-headed households. Yields of all commodities have increased significantly over the project period, particularly for female-headed households. Thus by 2014, female-headed households were yielding more than male-headed households for wheat in Ethiopia, rice in Tanzania and cassava in Uganda (though the differences were not statistically significant given small sample size). This is a significant achievement for the households concerned and for the project, implying that the technologies are having a positive gendered impact. Only in Kenya were female-headed dairy households lagging behind their male counterparts. The EAAPP Kenya programme is now prioritising gender issues in an attempt to address this.

Table 20 Average yield for key commodities by gender of household head – project beneficiaries (Kg/ha)

2009 2014 p-values Male-headed vs Commo Male-headed Female-headed Male-headed Female-headed female-headed Country dity n Mean n Mean n Mean n Mean 2009 2014 Ethiopia Wheat 94 2693 5 1606 100 4232 6 4660 .561 .737 Kenya Dairy 56 6555 7 4133 95 8352 12 6212 .059 .938 Tanzania Rice 30 3496 32 3053 47 6765 40 8526 .739 .318 Uganda Cassava 69 4281 26 6632 69 6746 25 7352 .974 .108

Source: EAAPP Evaluation household survey * Denotes significant change p<0.05; **Highly significant change p<.01

Changes in economic status and poverty levels reported by beneficiary households by gender of household head are shown in Table 21. This uses a self-assessment scoring system of 6 steps, with 1 being very poor and 6 being well-off, relative to other community members. In 2009, male-headed beneficiary households in Ethiopia, Tanzania and Uganda ranked themselves at an average of 2.5 (poor-moderately poor) whilst women were significantly lower at 2.15 (poor). Interestingly, female headed households in Kenya ranked themselves at 3 (moderately poor) compared to 2.74 for male-headed households (difference not significant). By 2014, households in all countries were reporting higher scores: 3.77 for male-headed households (moderately better off) compared to 3.55 for female-headed households. However, the gender gap had narrowed with no significant difference in economic status between male- and female-headed beneficiary households in all countries. Findings indicate that the programme is having a positive impact on both male and female-headed households, with female-headed households generally seeing a greater improvement.

Table 21 Economic status reported by beneficiary households in 2009 and 2014 by gender of household head

2009 2014 p-values Male-headed Female-headed Male-headed Female-headed 2009 2014 Country n Mean n Mean n Mean n Mean Ethiopia 151 2.46* 15 2.13 152 3.88 15 3.8 .043* .797 Kenya 145 2.74 20 3.05 144 4.08 20 4.1 .211 .915 Tanzania 89 2.28 63 2.14 91 3.16 64 3.42 .303 .093 Uganda 125 2.5* 50 2.16 125 3.72 49 3.14 .036 .431 All 510 2.52** 148 2.27 512 3.77* 148 3.55 .003 .015 Source: EAAPP Evaluation household survey * Denotes significant change p<0.05; **Highly significant change p<.01

Youth Inclusion of Youth in project activities is seen as important in EAAPP and youth activities are being promoted (for example seed producer groups in Kenya and Tanzania) but no targets or indicators on youth beneficiaries were available. Definitions of youth varied across countries in the region. In the Evaluation youth was defined as 16-30 years (young farmers), middle aged as 31-45 years (experienced farmers), and older household heads as 46+ years. Although household heads are distributed across all age groups, beneficiaries surveyed as a whole were significantly less likely to be in the younger age range and more likely to be in the older age range than non-beneficiaries.

Table 22 Age groupings of household heads: beneficiaries and non-beneficiaries surveyed

Age Group Country Commodity P-value (years) Beneficiary Non-Beneficiary Ethiopia Cassava 16-30 43.1% 39.0% 0.253 31-45 49.0% 41.5%

46+ 7.8% 19.5% Wheat 16-30 29.3% 32.1% 0.613

Age Group Country Commodity P-value (years) Beneficiary Non-Beneficiary 31-45 36.2% 39.6% 46+ 34.5% 28.3% Kenya Rice 16-30 7.1% 12.1% 0.588 31-45 33.3% 24.2%

46+ 59.5% 63.6% Dairy 16-30 3.2% 7.8% 0.321 31-45 29.6% 28.9%

46+ 67.2% 63.3% Tanzania Wheat 16-30 9.8% 22.6% 0.114 31-45 31.7% 41.9%

46+ 58.5% 35.5% Rice 16-30 18.1% 22.1% 0.532 31-45 42.9% 35.6%

46+ 39.0% 42.3% Uganda Cassava 16-30 24.4% 33.0% 0.286 31-45 39.3% 37.5%

46+ 36.3% 29.5% Dairy 16-30 16.7% 25.8% 0.452 31-45 38.9% 25.8%

46+ 44.4% 48.4% 31-45 31.7% 28.1% 46+ 62.1% 59.5% 16-30 19.2% 24.8% All All *0.062 31-45 37.3% 35.2%

46+ 43.5% 40.0% Source: EAAPP Evaluation household survey

Chronic illness and Disability

Information on disability and chronic illness of household members was collected during the Evaluation survey (Table 23). Overall around 17 percent of beneficiaries and 15 percent of non-beneficiaries reported a person with a disability or chronic illness in the household, suggesting that EAAPP does not exclude farm households on the basis of disability. In Uganda, dairy farmer project members were more likely to have a disability or chronic illness than non-project members.

In Kenya the programme is targeting an HIV/AIDS group with improved dairy goats. Group members interviewed have seen very positive benefits in terms knowledge on goat husbandry, nutritional and health benefits from goats’ milk, and improved self-esteem. However, members observed they had not seen any income benefits since the quantity of milk produced did not give a surplus for sale. It is therefore recommended that the programme assesses the viability of projects targeted towards vulnerable groups in terms of economic benefits as well as other non-cash benefits such as nutrition. With the experience gained from dairy goats, Kenya group members may now be in a position to take on improved dairy animals.

Table 23 Disability status of EAAPP beneficiaries and non-beneficiaries surveyed

Country Commodity Beneficiary Non-Beneficiary P-value Ethiopia Cassava 11.8% 7.3% 0.726 Wheat 17.1% 9.3% 0.116 Kenya Rice 14.3% 12.1% 1.000 Dairy 12.8% 9.7% 0.525 Tanzania Wheat 4.9% 16.1% 0.132 Rice 14.0% 13.8% 1.000 Uganda Cassava 26.2% 29.7% 0.581 Dairy 29.7% 11.8% 0.084 Cassava 22.4% 23.9% 0.800 All Wheat 13.9% 10.8% 0.483 Rice 14.1% 13.4% 0.868 Dairy 16.7% 10.2% 0.126 All All 17.1% 15.0% 0.315 Source: EAAPP Evaluation household survey * Denotes significant change p<0.05; **Highly significant change p<.01

Group membership

Membership of a group can assist household and community development through the opportunities it provides such as access to training, inputs, credit, joint marketing. Beneficiary households were significantly more likely to be members of a group (EAAPP or other) than non-group members: 68 percent compared to 51 percent (Table 24). The difference was also significant in Tanzania and Uganda where group membership ranged from 60 to 84 percent.

Table 24 Membership of an organised group by EAAPP beneficiaries and non-beneficiaries

Country Crop Beneficiary Non-Beneficiary Ethiopia Cassava 54.9% 43.9% Wheat 73.5% 63.0% Kenya Rice 54.8% 54.5% Dairy 76.0% 64.5% Tanzania Wheat 82.9%* 51.6% Rice 68.4%** 42.2% Uganda Cassava 62.8%** 34.7% Dairy 59.5% 64.7% Cassava 60.7%** 37.1% Wheat 75.9%** 60.4% All Rice 64.7%** 45.1%

Dairy 72.2% 64.6% All 68.0%** 51.0% Source: EAAPP Evaluation household survey * Denotes significant change p<0.05; **Highly significant change p<.01

8.7 Scaling-out

EAAPP aims to make an impact beyond its direct beneficiaries (project beneficiaries) to non-project members. These may be through extension and other development partners or directly through farmer-to- farmer dissemination. Evidence of farmer-to-farmer scaling-out was estimated by the evaluation team from project beneficiaries interviewed. This includes demonstrating technologies to other farmers and sharing planting materials etc. There are several challenges with these estimates - there may be double- counting of non-project beneficiaries taking up the technologies after interaction with several different stakeholders: alternatively there could be unknown scale-out to other farmers.

Farmer reporting indicates that they have shared EAAPP technologies with an average of 11.8 other farmers: the highest being dairy farmers in Kenya, cassava in Uganda and wheat in Ethiopia (Table 25). Of those non-project members who have learnt about the new technology or practice, farmers estimate that 5.7 have used the technology. This implies a multiplier of almost 6, and represents a significant factor in scaling-out new practices within communities.

Table 25: Number of non-project members learning and using EAAPP technologies disseminated by EAAP P beneficiaries (beneficiary reporting)

Learnt technology Used technology Country Crop n Mean n Mean ETHIOPIA Cassava 51 8.3 51 5.7 Wheat 117 13.5 117 6.3 KENYA Rice 42 6.7 42 5.0 Dairy 125 16.7 125 7.2 TANZANIA Wheat 41 5.9 41 5.0 Rice 114 5.1 114 3.9 UGANDA Cassava 145 17.1 145 6.4 Dairy 37 7.1 37 2.8 Cassava 196 14.9 196 6.2 All Wheat 158 11.5 158 6.0 Rice 156 5.5 156 4.2

Dairy 162 14.5 162 6.2

All 672 11.8 672 5.7 Source: EAAPP Evaluation household survey * Denotes significant change p<0.05; **Highly significant change p<.01

8.8 Overall assessment of EAAPP impact

Findings from the evaluation household survey indicate that a majority of EAAPP beneficiaries are already seeing a positive impact from their involvement in the programme, in terms of production, yields, incomes, food security and economic status. Improvements were found to be significantly greater than those of non- project households in a number of cases. Beneficiaries are also scaling out technologies to non- beneficiaries at quite a high rate. EAAPP wheat, cassava and rice technologies were found to be delivering significant benefits to female-headed households: only women dairy farmers in Kenya remain worse off.

There is still considerable variability in yields amongst beneficiary households, suggesting that many

44 EAAPP I Evaluation Final Report NRI AfrII farmers are not getting optimal performance from EAAPP technologies. This points to the need to develop a range of technologies to meet farmers’ diverse agroecological and socioeconomic conditions. The improved fodder programme in Kenya is a good example of this approach, promoting a range of varieties and intercrops for farmers to adapt on their own farms. Use of participatory technology development approaches – going beyond participatory variety selection - including participatory needs assessment, technology design, testing, monitoring and evaluation is recommended.

9 EAAPP PROGRAMMING

9.1 Programme design

The EAAPP project design is based on the following rationale (EAAPP Appraisal):

 Growth in agricultural productivity is necessary to trigger economic development in Africa;

 Agricultural technology is fundamental to growth in productivity;

 The rate of agricultural development depends on national institutions, levels of investment and the policy context;

 Size matters in technology systems: Technology investments in one country can create substantial spillovers in a region with shared agroecological zones and markets – if technologies can flow across boundaries this can increase technology space;

 Regional research accounted for only 1 percent of overall agricultural research expenditure in 2009 (US$25 million pa);

 Agricultural research in Africa is highly fragmented – researchers are spread across many small programs. There is a lack of critical mass to address complex problems.

 Improved regional technology systems can improve short-term food security and long-term growth. Farmers need access to improved cultivars and livestock.

EAAPP is intended to enlarge the technology space of the four commodities through the establishment of RCoEs and lowering barriers of movement of technologies across borders. The theoretical basis for the programme is sound – based on evidence from IFPRI and others – though the institutional context provides considerable challenges to operationalization of the model. Regional research has effectively been doubled in Eastern Africa as a result of the Programme. Some regional spillovers have already been realised both inside and outside EAAPP countries (see Regionality).

The agricultural innovation system envisaged under EAAPP is a chain – from international and local research, extension and advisory services to production, distribution of inputs and on-farm application. This innovation system – if well-designed – will generate, disseminate, spill-over, borrow-in, and facilitate adoption of technology (PAD 2009: 1). This model which has been adopted by EAAPP starts with research and ends with farmers and variously includes other stakeholders in the innovation system – extension, input providers, buyers, processors. Rice in Southern Tanzania, is an excellent example of

45 EAAPP I Evaluation Final Report NRI AfrII commercialisation of a RCoE variety, through mobilisation of key stakeholders from research, extension, farmer groups and the private sector, with increased competitiveness along the value chain.

Figure 5 Increased competitiveness of rice value chains through partnerships and improved business skill s

Some EAAPP research projects e.g. Fodder legume seed production in Kenya (which offers a wide range of fodder technologies for different farmer requirements), and a number of others using Participatory Breeding approaches, take an explicit farmer-centred approach. The project has also provided an incentive for researchers to link with value chain partners in response to farmer needs (e.g. wheat breeders in Tanzania).

Putting an explicit Agricultural Innovation Systems model at the centre of the programme would drive this approach forward and ensure that the research is relevant and farmer-led, and that the innovation is broad-based and self-sustaining. This is very important as the project needs to ensure that platforms are in place to sustain development and uptake after phase-out of EAAPP. Proven models in the region (such as Innovation Platforms for Technology Adoption, successfully used by ASARECA4) should be appraised for use in EAAPP projects. Platforms facilitated by long-term local institutions (e.g. Farmer Organisations) often have more chance of success compared to those led by researchers.

Linkages in regional research and development

The institutional environment within which agricultural research and extension is undertaken is continually changing. There is now an increasing emphasis on integrated agricultural research for development (IAR4D) and innovation systems thinking. The Science Agenda for Agriculture in Africa (S3A) highlights the importance that policy makers and research managers should attach to the emergence of a more integrated system of research and higher education at the regional level (FARA, 2014 5). This implies that agricultural research and delivery need to be linked more closely and that universities should be involved more centrally in research and extension activities. Facilitating such a process is complex and requires developing capacities in a wider range of skills than under previous, more compartmentalised, systems.

4 Kimenye, L and McEwan, M (2014) Scaling up dissemination and adoption of agricultural technologies using innovati on platforms – Lessons from Eastern and Central Africa, ASARECA

5 FARA (2014) Science agenda for agriculture in Africa (S3A): “Connecting Science” to transform agriculture in Africa. F orum for Agricultural Research in Africa (FARA), Accra.

Sharing experiences from different countries can play an important part in developing such systems at the national level.

There are various options for organizing and funding regional research and extension. Collaboration between scientists and extensionists in different countries may be facilitated through programmes or projects managed by a regional entity such as ASARECA. Such programmes and projects may be commissioned on a competitive or non-competitive basis. Competitive schemes are more transparent and may help to raise research and extension quality. However, they may disadvantage countries in which research capacity is limited, unless specific provision is made to encourage their participation. Regional research and extension may also be undertaken through programmes or projects funded by national governments or external donors without the involvement of a regional entity.

Regional collaboration between researchers and extensionists may be organized through participation in networks in which exchange of information, materials and technologies can be facilitated as well as joint activities undertaken. Another model involves the establishment of regional research centres. These may be sited in a specific location or be virtual centres in which different organizations collaborate in a decentralized system. An example of the former in the ECA region is Biosciences East and Central Africa (BECA) which is located in the International Livestock Research Institute in Nairobi. BECA manages advanced laboratories which are made available to national scientists who do not have access to similar facilities in their own countries.

The S3A recommends support for regional centres of excellence to share knowledge and facilities. This is considered to be an essential step in enhancing scientific capacity in all countries in Africa so that science- based solutions can be utilised effectively for development impact. EAAPP has placed regional research centres of excellence at the core of its regionality approach. The potential benefits of regional centres include:

 The provision of shared facilities and technical support that may be accessed by national scientists from several countries.

 The functioning of such centres as hubs which facilitate collaboration with other partners, including CGIAR Centres and advanced research institutes (ARIs) in a way that minimises dispersion of effort.

Such collaboration may be project-based, but can also involve longer-term partnerships in which African scientists will play a prominent role.

The EAAPP model for regional research centres involves upgrading existing centres in partner countries through infrastructure development and human capacity development. These centres are commodity based and represent priority areas of research in the countries in which they are located. This approach should provide a strong incentive for the host country to maintain and further develop the centres of excellence, especially as the funding for the initial infrastructure and training is provided in the form of a loan rather than a grant from the World Bank. The EAAPP approach also involves support for regional research and development projects in which participants have access to facilities in the regional centres.

The role of regional research in the innovation chain is between international and local research. The RCoEs have linkages with the CGIAR in terms of sourcing breeding material, facilities and in some cases joint research (e.g. the Rice RCoE with IRRI) though these linkages pre-date EAAPP. There appears to be

47 EAAPP I Evaluation Final Report NRI AfrII more interest in collaboration as the RCoE are strengthened and this should be explored to maximise use of research resources available in the region.

Investment priorities

In terms of investments in the system, the largest share under EAAPP I goes to support Research, training and dissemination (Component 2) followed by Improved availability of seeds and breeding material (Component 3), except Uganda (Table 26). This would be expected to continue under Phase II. A significant share was also allocated to Support to research infrastructure (Component 1.1). Outputs were not fully complete and research facilities commissioned at the time of the Evaluation visits, but it is expected that these will make a significant contribution to research capacity by the end of the Programme period. Support to Harmonisation and regulations has been a critical investment in a public good which is expected to contribute to improved regulation (e.g. TOSCI and KEPHIS) and policy harmonisation.

Table 26 Allocation of funding by component and country

Component Projects Ethiopia Kenya Tanzania Uganda Total US$M Component 1 Strengthening RCoEs 3.9 6.5 7.7 6.7 24.8 1.1 Support to research 3.3 5.9 5.9 4.3 19.4 infrastructure 1.2 Capacity building 0.6 0.6 1.8 2.4 5.4 Component 2 Support to Research, Training 11.1 13.5 14 12.4 51.0 and Dissemination 2.1 Technology generation 6.4 9.8 7.0 8.7 31.9 2.2 Training and dissemination 4.7 3.7 7.0 3.7 19.1 Component 3 Improved Availability of Seeds 7.5 7.5 5.8 8.6 29.4 & Livestock Breeding Materials 3.1 Improving availability of seeds 1.7 4.0 3.8 4.8 14.3 and breeding materials 3.2 Support to Business 8.8 1.0 0.8 0.6 11.2 Development 3.3 Support to Harmonisation of 0.7 2.5 1.2 3.2 7.6 policies and regulations Component 4 Project Management and 3.8 2.5 2.5 2.3 11.1 Coordination 4.1 Project management and M&E 3.1 1.7 1.7 1.5 8.0 National 4.2 Project management and M&E 0.7 0.8 0.8 0.8 3.1 Regional TOTAL PROJECT COSTS 30 30 30 30 120 Source: EAAPP reports

9.2 Relevance of research

Identification of commodities

The rationale for the selection of the commodities for inclusion in the programme was provided by a study conducted for ASARECA by the International Food Policy Research Institute (IFPRI) (Omama et al., 2006).

The approach was to first characterize agricultural development domains using a geographical information system and to analyse the potential of different agricultural and non-agricultural subsectors with a multi- market model. The potential returns to regional cooperation in agricultural development were then explored using a model that quantified the effects of productivity-enhancing investments in agricultural research and development.

The analysis showed that the greatest reductions in poverty would accrue from growth in the agricultural subsector for which demand in the region is strongest; in particular, staples, livestock products, oilseeds, and fruits and vegetables. The results of the modelling indicated that investment in the dairy subsector had the greatest potential for contributing to growth in the ECA region and that cassava also ranked highly. Projections showed that staple crops and livestock products resulted in the greatest gains in gross domestic product. There was a close match between the priorities for Kenya, Tanzania, and Uganda and the ECA region as a whole. Cereals and milk were found to be the main priorities for Ethiopia.

Cassava is listed as a priority commodity in the Uganda Agricultural Sector Development Strategy and Investment Plan (DSIP). As the second ranking subsistence crop in terms of production it is important for its poverty reducing potential and it has scope for commercialization. The import substitution potential from cassava is estimated at USD 30,658,000 per year. Dairy and rice are priority commodities in the national agricultural strategies for Kenya and Tanzania, respectively.

The project focal districts were consultatively identified through stakeholder forums and professional group meetings and interventions rolled out along the value chains of the four commodities i.e. Dairy, Wheat, Rice and Cassava. This was an important part of the process of developing local ownership.

9.3 Quality of research

We considered that, at this stage of the project implementation, the most appropriate way to measure the quality of research being undertaken is to assess the systems that have been put in place to commission and support new research and to review research findings; and to review the research approaches used by sub-project teams. The specific approaches and methodologies used to design and implement regional research projects are not described in detail in the reports that we had access to. However, we were able to meet with members of selected project teams and to gain insights into how they developed their ideas and carried out research activities. It is too early to use standard ‘third-party’ measures of research quality such as the number and quality of journal publications, although we will comment on this later in this section with a view to suggesting how these might be enhanced.

Each of the partner countries established their own procedures for commissioning research projects. Only Kenya and Tanzania used a competitive process by issuing an open call for proposals. Competition is generally considered to enhance the quality of research proposals as well as providing transparency in the award of projects. However, in situations where the number of potential applicants is low it is not always the most efficient way of commissioning new research. In the case of Kenya, which has a relatively large and well-resourced agricultural research system, this limitation does not apply to the same degree. Moreover, the production of a grants manual by the project management unit in Kenya provided useful guidance to applicants on procedures and on what is expected from the research process, including highlighting cross-cutting objectives such as gender, environmental and social issues.

The development of proposals by research teams should be regarded as an opportunity to strengthen the capacity of researchers. This also applies to unsuccessful applicants as feedback can be given on aspects of proposals that are weak and recommendations made for improvement. It is not clear how systematically

49 EAAPP I Evaluation Final Report NRI AfrII this was done but this should be an integral part of any future research commissioned under a new phase of the project. The research centre of excellence reviews and conferences were an important means of engaging researchers in a peer review of their research. The opportunity to present research findings in a peer group and the requirement to submit manuscripts for inclusion in formal reports from these events help to ensure that the quality of the work is of a high standard. The project placed considerable emphasis on these events, which were also an integral part of the planning process, and the written outputs should be widely circulated among ASARECA’s network of scientists in the region.

The formulation of relevant research questions and the use of methodologies that are appropriate to the research objectives of sub-projects is an important dimension research quality. In view of the fact that much of the work is of an applied nature and aims to have developmental impact, the use of participatory approaches is essential for many activities. In the sub-projects which it was possible to review in more detail, mainly through direct discussion with team members, there were examples of good practice in using participatory approaches. In some sub-projects, farmers were involved in the evaluation of candidate crop varieties for release. For example, in the sub-project on ‘Genetic Improvement of Cassava (Manihot esculenta Crantz) in Eastern Africa’ farmers and Ministry of Agriculture staff were involved in multi-location evaluation of 26 cassava genotypes at 11 sites over a three-year period. There are few documented cases where farmers were actively involved at an early stage in the development of a technology, for example through participatory varietal selection (PVS) in plant breeding activities. PVS was used in some of the wheat breeding in Ethiopia and a gender study was conducted on this work. Unfortunately, the release of a report on the findings has been delayed due to the transfer of the researcher who carried out the study.

An interesting model for engaging farmers in the research and innovation process has been developed in Ethiopia. This involves the establishment of farmer innovation groups. These groups are focused on particular commodities and aim to generate improved economic returns for their members based on the development of simple, low-cost and environmentally friendly innovations. The establishment of this initiative pre-dates EAAPP, but the project is working with farmer innovation groups as a way of supporting the promotion and adaptation of new technologies.

One of the constraints faced by national agricultural systems in most countries in sub-Saharan Africa is the lack of advanced laboratory facilities. This restricts the type of research which can be undertaken and means that researchers are heavily reliant on collaboration with CGIAR centres and advanced laboratories in other continents to carry out many analytical procedures. The construction of new laboratories and the rehabilitation of existing laboratories in the regional centres of excellence was expected to expand the range of research activities that can be undertaken. So far, the benefits from this aspect of infrastructure development in the project have been limited. This is mainly due to the delays in procurement which have meant that improved facilities are not yet available for sub-project teams. Similarly, delays in procurement of equipment have affected dissemination activities such as access by dairy farmers to bull semen due to limited availability of liquid nitrogen.

Once the new laboratory facilities are in use the regional centres of excellence will have the capacity to do more advanced research in areas such as molecular biology, nutrition and value addition. The availability of these new should also help to leverage additional resources as other donor agencies identify them as suitable locations to support programmes they wish to fund. There is already some evidence that this may be happening; for example, for planned molecular research at the regional cassava centre of excellence. Meanwhile, effective use has been made of existing partnerships with advanced laboratories in Europe and the United States of America; for example, with molecular characterization work being conducted at Cornell University and the University of Davis, California.

One of the factors constraining high quality research at the regional centres of excellence, and within the wider national agricultural research systems, is the limited capacity in research methods. Some capacity in statistics is available but this expertise is often not adequately oriented to research design and analysis in agricultural systems; particularly, addressing the requirements of inter-disciplinary work and combining appropriate quantitative and qualitative methods. The need for greater capacity in research methods is seen in the rather conventional approaches being used in some of the crop management research, such as fertilizer trials. These approaches do not take sufficient account of extrapolation domains and tend to result in blanket recommendations that are not appropriate for many locations. It is recommended that options for research methods support are explored, including short-course, online courses and tailored support.

Stronger capacity in research methods would also result in an improvement in the number and quality of journal papers. In the medium- to long-term this will be achieved through better designed research which can be published in higher quality journals. In the short-term research methods support may be used to assist researchers to write up their current research for publication. There are a large number of short papers which are included in Conference Proceedings, some of which are likely to be suitable for journal publication if appropriate research methods support is provided. It is recommended that ‘writeshops’ are held in which professional facilitators assist selected researchers to prepare manuscripts for journal publication.

A list of journal papers from project research is shown in Appendix. This is not a comprehensive list and it is known that other papers have been submitted or are in preparation. The papers published to date tend to be rather descriptive and many appear in online journals for which the peer review process is not very rigorous. This is not to discourage researchers from publishing in online journals but they should aim high in order to make the most effective contribution to new agricultural knowledge.

9.4 Regionality

In theory, regional integration and collective action in agricultural R&D among neighbouring countries can lead to economies of scale and spillover effects that permit research systems to jointly achieve critical masses and cost savings needed to address problems beyond the capacities of individual systems. IFPRI/ASARECA estimated returns to agricultural research based on a 1 percent increase in investment in research and development by three large innovating countries (Kenya, Uganda, Tanzania). This shows that in addition to gains of over $10 million from milk, cassava and rice in the innovating countries, there are spillovers to other ECA countries. For milk spillovers are almost $3 million, for cassava $2.6 million, and rice $1.4 million. The results suggest that milk and cassava would be especially big winners of region R&D investments. Investments in rice are also likely to give high gains to the region (159%). Returns to wheat were not calculated, indicating limited regional spillovers (Table 27).

Table 27 Returns to agricultural R&D investment with and without spillovers

Commodity Regional gains Incremental spillover Gains to region from without spillovers (US$000 pa) spillovers % (US$000 pa) Milk 4,456 2,984 67 Cassava 5,200 2,581 50 Rice 854 1,355 159 Wheat Na na Na Source: IFPRI/ASARECA

We have noted above that closer harmonization of policies is needed to make a regional research approach more effective. ASARECA, through its Policy Analysis and Advocacy Programme, coordinates a regional working group on policy which includes representatives from relevant Ministries in member countries. EAAPP has provided a channel for harmonizing policies on seed, intellectual property rights and dairy and progress in each of these areas is summarized below.

Seed In 2002, agreements were signed on harmonization of national seed policies in ECA. The technical agreements on seed policy harmonization were in five key areas: variety evaluation and release; seed certification; plant variety protection; phytosanitary regulations and; seed import and export procedures. However, the agreements were not implemented, partly due to differences among countries in the provisions of National Performance Trials (NPT). Working through the Eastern Africa Seed Committee (EASCOM), ASARECA has facilitated the development of harmonized NPT protocols for cassava, rice, wheat and pasture seeds6.

Further discussion is needed by members of the expert group to reach agreement among the countries on three outstanding issues. These are the number of seasons over which the trials need to be conducted; the number of test sites required; and whether on-farm trials should be optional or mandatory in evaluating the performance of the test lines. It is envisaged that once the expert group has reached consensus on these issues the revised protocols will be submitted to the national EAAPP steering committees. They will then be reviewed by the relevant agencies in the partner countries and before being adopted as standard protocols by the National Variety Release Committees.

Intellectual property rights policy ASARCEA has coordinated the development of a draft EAAPP intellectual property rights (IPR) policy 7. It has done this through leadership of an expert group which includes representatives from each of the EAAPP countries. The IPR policy addresses the generation, ownership, protection and use of intellectual property. It is designed to provide incentives to enhance research and product development that will lead to economic growth which will benefit citizens in EAAPP countries and in the EAC region. The policy aims to facilitate improved regional access and utilization of new agricultural technologies; streamline the transfer of materials and knowledge among countries; reward research and innovation effort through the recognition and protection of intellectual property; and make suitable provision for benefit sharing and resolving disputes.

Under the IPR policy, it is proposed that an Intellectual Property Management sub-committee of the Regional Steering Committee will be established. ASARECA is responsible for initiating the establishment of this IPM sub-committee and will act as the secretariat. The IPM sub-committee will oversee the setting up of an EAAPP IP Management Office in each country. These offices will be responsible for ensuring that ‘all eligible and appropriate intellectual property rights arising from EAAPP project work are protected expeditiously’. An interesting and commendable feature of the policy is that the IP Management Offices may initiate and establish Community Intellectual Property Management Committees in which local communities are represented.

6 Akao, G, Waithaka M, Kyotalimye M. (2013) National Performance Trials: harmonized protocols for cassava, rice, wh eat and pasture seeds. ASARECA (Association for Strengthening Agricultural Research in Eastern and Central Africa), E ntebbe.

7 EAAPP/ASARECA (2013) Eastern Africa Agricultural Productivity Program (EAAPP) Intellectual Property Policy: final dr aft November 2013.

The draft IPR policy is an important step forward in creating a common framework for the management of intellectual property in the ECA region. It was discussed by the project Steering Committee in February 2014 and was subsequently shared with Permanent Secretaries in EAAPP partner countries. The Kenya Project Steering Committee has approved the draft IPR policy and decisions from the other countries are pending. A plan has been developed to implement the policy once it has been approved in all the countries, but this will take time to deliver. For example, national audits reports of IP generated in participating institutions will be prepared and stakeholder meetings held to validate the findings, raise awareness of the issues and build consensus on the roles and responsibilities of different institutions. Therefore, other EAAPP partner countries should be urged to ratify the policy as soon as possible so that it can be put into operation before a possible Phase 2 of the project starts.

Dairy Little attention has previously been given to harmonizing policies and regulations governing the production and movement of livestock and livestock products in the ECA region. Standards for feedstuffs and dairy products vary among countries and there is weak enforcement of these standards. Through EAAPP, with the support of ASARECA through its Policy Analysis and Advocacy Programme (PAAP), progress has been made in several areas.

Achievements include:

• Kenya has led the development of a draft animal breeding policy and breeding rules and this is being used as a guide for similar policies in other EAAPP countries. • Regional guidelines on procedures for movement and trade in heifers and germplasm were developed by the Kenya dairy policy team. This is designed to address the constraints caused by current restrictions on the import and export of bull semen and live animals.

• The Ethiopia EAAPP dairy policy team and other national partners have finalised the drafting of a National Animal Breeding Policy.

• Tanzania has led the development of a protocol for import and export of compounded dairy animal feeds and forages.

• Uganda is leading the development of regional guidelines for import and export documentation and procedures for dairy.

Whilst good progress has been made in a relatively short time there are several actions which need to be fast-tracked. There is currently no reference laboratory for dairy analysis in the ECA region and accreditation of a new laboratory in Naivasha needs to be done as soon as possible. Ethiopia does not have a Dairy Board or a comparable institution and EAAPP plans to provide guidance on how it can be established, under the auspices of the Eastern Africa Dairy Regulatory Authorities Council (EADRAC). Uganda does not have a livestock policy and also needs to advance its draft legislation on animal feed. Work has been done in all the EAAPP countries to review and revise existing standards in the dairy sector but compliance remains weak, particularly in the area of feeds.

Project staff interviewed spoke positively about the benefits of participating in regional projects and highlighted the benefits of exchange visits. However, the contribution of partners in other countries does not come through strongly in activity reports; see, for example, the 2013-14 report from the RCoE in which there is no mention of contributions to projects from partners outside Uganda. This illustrates the importance of effective communication in the management and monitoring of projects. Several participants in EAAPP projects acknowledged that communication needs to be improved.

If a second phase of EAAPP is approved, one way of strengthening the regional dimension of the project would be to enhance the role of ASARECA. Some country managers of EAAPP have suggested that ASARECA might hold some funds for regional sub-projects. Others have drawn attention to the limited capacity currently available at ASARECA. If ASARECA is to play a more prominent role in the coordination of regional activities in any further phase of EAAPP the coordination unit would need to be strengthened.

9.5 Programme management and governance

9.5.1 Governance As a regional program, EAAPP implementation is based on partnerships and collaboration among participating countries. Countries work together in different ways: by undertaking joint technology generation, dissemination and training activities; coordinating their respective national activities with each other in order to achieve a shared regional objective; and sharing knowledge and technological outputs from their national programs throughout the region.

Implementation of EAAPP takes place at the national level with regional activities coordinated through annual work programming exercises. EAAPP implementation relies on existing regional platforms, networks and partnerships to share information and create opportunities for collaboration.

EAAPP activities at country level are managed through different institutional arrangements:

In Kenya EAAPP is implemented by the Ministry of Agriculture (MoA), Ministry of Livestock Development (MoLD) and Kenya Agriculture and Livestock Research Organisation (KALRO), formerly Kenya Agricultural Research Institute (KARI). The MoA takes the lead in the overall coordination of the project. An EAAPP coordination unit was established by MoA, housed in an independent office with full-time dedicated staff. However, the Country coordinator maintains close contact with the PS Agriculture. Project oversight is provided by the existing Joint Agriculture Sector Coordination Steering Committee.

In Tanzania, EAAPP is implemented under the Agriculture Sector Development Programme (ASDP). The Department of Policy and Planning within the Ministry of Agriculture, Food and Cooperatives (MoAFC) is responsible for the overall coordination of the project and M&E with the Deputy Director of Research as National Coordinator. MoAFC and Ministry of Livestock Development and Fisheries implement specific project components. The arrangement has the advantage of strong continuity and steamlining within existing government programmes. A disadvantage is a heavy workload and competing priorities by Planning and Research staff coordinating the EAAPP project. Oversight is provided by the existing Basket Fund Steering Committee. The Rice RCoE was originally constituted as a networking arrangement between several different institutions including zonal research institutions, universities, Kizimbani Agricultural Experiment station, coordinated by the RCoE coordinator at Katrin. However, the mid-term review concluded that this arrangement diluted the concept of a RCoE and recommended that research activities be focused on Katrin research station, with other sub-centres testing the technologies. This arrangement is creating a stronger clustering of scientists which should attract more research activities. However, the location and road access to Ifakara remains an issue.

In Ethiopia, EAAPP is implemented by the Ministry of Agriculture and Rural Development (MoARD) and the semi-autonomous Ethiopian Institute of Agricultural Research (EIAR), with a coordination unit within MoARD based in the Rural Capacity Building Project coordination unit. Implementation of training and dissemination, most activities under the third component and M&E are coordinated by MoARD. EAIR is responsible for implementation of component I, technology generation, seed multiplication activities, and internal M&E. The wheat RCoE is coordinated from Kulumsa research centre by the RCoE coordinator.

In Uganda the EAAPP project coordinator reports to the Director General of the National Agricultural Research Organization (NARO). His coordination role is constrained by the fact that he has limited ability to exert influence in the Ministry of Agriculture Animal Industries and Fisheries (MAAIF). Project Focal persons in MAAIF report to line managers in the Ministry rather than to the project coordinator. However, following the mid-term review the Permanent Secretary specified that all reports from MAAIF project staff should be sent directly to the project coordinator.

Stakeholders report various strengths and weaknesses of each of the programme management structures. Given very different national institutional contexts and capacities, it is not possible to come up with a blueprint EAAPP country organisation. Rather, new countries joining the project need to make a critical assessment of factors including: existing institutions and their decision-making processes, cost, sustainability. It is however, recommended that EAAPP units are staffed by dedicated full-time staff to improve efficiency and timeliness of activities, and to free up PCU staff to engage in other relevant scaling out and partnerships.

9.5.2 Monitoring and Evaluation

Monitoring and evaluation of the planned activities under EAAPP are coordinated by ASARECA. The RCoE as well as the National Coordinators play a major role in coordinating the project implantation, while the M&E Officers ensure effective monitoring, data collection and reporting. ASARECA has developed the M&E framework for the project and provided training for partners to facilitate evidence-based and adaptive management: the Performance Monitoring Plan (PMP).The PMP is based on the ASARECA M&E framework, which is in line with the Comprehensive Africa Agricultural Development Program (CAADP) framework that is adopted by many of the Africa Union (AU) member states. ASARECA also has an ongoing role in ensuring that the results targets for the project are monitored and achieved, in collaboration with the team members from all the four Centres of Excellence, has designed. Each RCoE has also developed its own PMPs.

The PMP lays out the monitoring and evaluation system that ASARECA and the RCoEs will implement to determine the project’s success. This PMP establishes linkages between (i) the approaches, indicators, milestones and targets described in the Project Appraisal Document; (ii) the activities described in the same document work plan; and (iii) Country Project Implementation Plans (PIPs). The PMP presents and defines project-specific objectives, terminology, beneficiary populations, indicators, measurements, and targets. It also develops the monitoring and evaluation system to be used for data collection, analysis and reporting.

The harmonised Results Framework is a very useful tool which focuses on process and output indicators, and tracks progress at national and regional levels. The consolidated Results Framework focuses on the indicators listed in the PAD, but also gives room for identification of custom indicators. These custom indicators allow project implementers to track performance within their RCoE, besides addressing all the standard (mandatory) indicators.

To ensure effective performance monitoring bi-Annual Joint Review Meetings (Portfolio Reviews) are held with RCoE Coordinators and M&E Officers. These meetings provide a platform for collective assessment of progress against milestones (the Annual Workplan and Budget). These reviews are targeted to lead to management decisions about program and project implementation and feedback.

Overall the M&E systems are very well designed and managed, providing timely results for project management. M&E officers have good capacity and are well motivated. The EAAPP M&E process is

55 EAAPP I Evaluation Final Report NRI AfrII however fairly resource intensive, with a large number of quantitative indicators - defined in the PAD and reported to the World Bank - requiring frequent follow up. Some case studies of success stories are being documented but there is less attention to lesson learning, evaluation and sharing of best practices e.g. the different models being tried out on value addition, scaling out. There is a need to ensure learning is maximised across the project.

Data collection limitations: self-reporting has a tendency to over-report positive results and under-report negative results. However the system of external mid-term reviews and end of project evaluations provides a counter-balance to check information collected. There are also challenges of reporting against indicators such as progress in infrastructure development. The current reporting system involves M&E and RCoE national and research coordinating teams meeting to review progress and assign percentage achievements e.g. progress with contracts. This is a useful process in terms of monitoring for project and financial management but is not a fully objective measure.

Communications

A wide range of communication approaches are available within the programme. An impressive range of written materials have been produced, in the form of leaflets, booklets and manuals. However, there is not much evidence to show that the project has exploited the full potential of communication approaches; or that it has supported activities that document and share learning on the situations where a particular communication approach has proved to be valuable. It is important to know what has been learned about the effectiveness of different types of communication approaches to disseminate new technologies. For example, extension staff need to know when specific tools such as FM radio or SMS messages add value to more conventional methods of information dissemination. Are digital technologies being exploited to their full potential? For example, the use of participatory video to demonstrate the value of new technologies would be a useful way to document the practices and promote their uptake by other farmers.

There are sub-projects which are comparing the use of different extension approaches. Sub-project 3 in the Regional Cassava Centre of Excellence on ‘Enhancing uptake of technologies along cassava value chain’ has gathered useful data on different pathways used for training and dissemination activities, including various forms of media. It will be very useful for lessons from this sub-project to be documented and shared with other CoEs. Each of the countries has a website for its regional centre of excellence. In the case of Tanzania the website for the rice regional centre of excellence is embedded in the ‘e-rails’ platform managed by the Forum for Agricultural Research in Africa. The websites of the other countries are stand- alone. ASARECA has a page for the project which can be accessed through a tab on the home page of its website, but there is little information on the project or the technologies it is promoting. This is surprising, given ASARECA’s purpose: enhanced utilization of agricultural research and development innovations in eastern and central Africa. The website of the rice regional centre of excellence provides the most detailed information but all of the websites need to be improved to give greater visibility to the approach that the project is using and the technologies that are being developed and promoted. Specific recommendations for improving the websites are as follows:

 The ASARECA website (http://www.asareca.org/content/eaapp) should be updated and include downloadable key documents and information on Best Practices. Best Practice information should include details of the initial inventories of GIMPS; as, for example, are available on the Rice Regional Centre of Excellence website. Broken links should be fixed (e.g. to the RDCoE) and links added to the websites of the RWCoE and RCCoE.

 Best Practices and Project Outputs should be added to all the RCoE websites. Outputs should include Conference Proceedings, references or links to journal publications and information about new developments in policy harmonization.

 Each RCoE website should have a link to other RCoE websites.

 Online tools should be provided to facilitate communication between partners in regional sub- projects. This may require training in the use of a free online facility such as Dgroups or Basecamp. ASARECA is best placed to facilitate the provision of this support.

9.6 Role of ASARECA

EAAPP contributes to the Common Africa Agriculture Development Programme (CAADP) agenda. ASARECA is mandated by COMESA to lead in coordinating the implementation of CAADP Pillar IV (Improving agricultural research, technology dissemination and adoption) and the application of the principles of the Framework for African Agricultural Productivity (FAAP) in East Africa.

This is how ASARECA’s role is stated in the Uganda PAD (pp 51-52).

“The RCoE will work with ASARECA in coordinating regional activities associated with their RCoE program of activities. ASARECA will play an important role in facilitating regional implementation by providing assistance for networking, capacity building and technical backstopping, monitoring and evaluation, regional coordination and supervision, and policy analysis. The RCoE will establish an Implementation Agreement with ASARECA detailing the nature of technical support and coordination expected from ASARECA, and the level of financial support to be provided by the host country. The table summarizes the specific activities ASARECA will undertake in project implementation, together with experiences from countries for each type of role.

Other regional institutions such as African Forum for Agricultural Advisory Services (AFAAS) will also work with ASARECA in supporting implementation of the project’s regional technology training and dissemination activities. ASARECA will contract AFAAS to provide specific services related to the provision of technical assistance, facilitation of knowledge sharing and exchange, and regional coordination activities related to extension/advisory services activities.”

This is how ASARECA sees its role in EAAPP (extracted from EAAPP page, ASARECA’s website):

“ ASARECA has offered to use its expertise in coordinating regional research for development, extension, training and education to facilitate spillovers of technologies and innovations that will be generated through EAAPP.”

Specific roles played by ASARECA in EAAPP are summarised in Table 29 (Column 2), together with feedback from the ASARECA coordinator and from EAAPP stakeholders:

Table 28: Typology of support from regional institutions Typology of Support from Type of support provided by Stakeholder feedback on Regional Institutions - Planned ASARECA support received from ASARECA (Source: ASARECA EAAPP (Source: Stakeholder interviews (Source: Uganda PAD) coordinator and website) and reports) 1. Convening Facilitate regional strategic To give a regional perspective; ASARECA has done a good job in meetings involving the four target identify regional priorities; modalities facilitating regional meetings, countries to develop regional of operation, including the role of the especially in identifying the operational frameworks for RCoEs. topics for regional research establishment of CoEs. Such ASARECA is facilitating strategic projects (All countries).

57 EAAPP I Evaluation Final Report NRI AfrII meetings will be jointly convened meetings for the four countries to Tanzania (Progress report for by the target countries (CoEs) and develop operational frameworks for January to July 2014 (pp 40-41): ASARECA. RCoEs; defining the responsibilities of ASARECA’s contribution is Facilitate development of regional each RCoE to the sub-regional recognized in providing technical strategies and priorities for the stakeholders; defining the mode of expertise and guidance on four commodities. Each CoE will operation of the RCoEs individually enhancing the regional convene regional meetings for and in relation to others; defining dimension in sub-projects. This commodity based strategy outputs and the manner in which has helped to build the capacity development and priority setting they will be shared amongst of staff; also to build in and ASARECA will only play a participating countries and other performance management, supportive role and ensure that countries in the sub-region, and environmental and social regional interests are taken into facilitating development of regional management and gender account. strategies for the four commodities. mainstreaming plans. ASARECA has convened two planning workshops for these participating countries to agree on roles and responsibilities of each RCoE and the mode of operation of the RCoEs. 2. Networking Networking and information sharing Facilitate information sharing Facilitating communication between Communication of knowledge platforms to enable sharing of countries; development of and information among benefits and spillover of communication strategies; ICT tools researchers in the RCoEs should technologies and innovations e.g. social media for communication. be strengthened e.g. through developed by CoEs to other electronic platforms. ASARECA countries. Facilitate collaboration with national, regional and international centers. 3. Technical backstopping Assist the CoEs in developing ASARECA has over the years Guidelines on the description of principles and generic guidelines developed in-house expertise for the technologies provided by and standards for call of regional out-scaling agricultural technologies, ASARECA were positively proposals, review and approval. innovations and best practices; and received so would like to put this expertise at the disposal of the RCoEs. But inventories of technologies not yet fully described by the countries Backstop outscaling of Training in ESS, value chains – technologies, innovations and best viewed positively practices across sub-region. Uganda: Limited technical support from ASARECA for crop commodities; positive on support for dairy. 4. Monitoring and Evaluation Provide capacity building for Regional M&E activities will primarily ASARECA provision of the M&E monitoring and evaluation, focus on tracking the extent to which framework and the PMERL participate in impact assessment, EAAPP is making spillovers happen software – very positive participate and facilitate program across the sub-region. reviews. ASARECA has developed a results framework – “Proud of what was done”. 5. Policy harmonization and advocacy Facilitate rationalization and ASARECA pledges to facilitate the Guidance for the working group harmonization of policies, rationalization and harmonization of on policy harmonization - procedures and regulations aimed policies, procedures and regulations positive at common standards in the aimed at creating common standards Kenya: Effective regional different countries. in the participating countries. This working group on policy includes will be attained by establishing the Ministries; coordinated by

status of policies and procedures ASARECA through the PAAAP. affecting the four commodities; Harmonization of policies is analysing and developing policy important and progress has options and advocating for and been made; shortening of supporting implementation of the procedures needed for options. germplasm dissemination if appropriate testing already done in the country where a variety or breed was developed. Countries have the right to audit these procedures. Registration systems for livestock differ among countries; e.g. it is difficult to bring bull semen from Ethiopia to Kenya where there is a strong demand (esp. Jersey). Tanzania: greater efforts are needed in harmonizing policies across countries (e.g. IPR) and packaging of information and technologies at regional level. Stronger lobbying and advocacy for recommended policy options is required. 6. Capacity Building Participate in capacity building ASARECA will organize and facilitate ESS and Value chain training – organized by the CoEs. regional training workshops for RCoE good. managers to equip them with the Ethiopia: Training in Arusha on tools and skills to transform the value chain analysis was good. RCoEs into more effective research Support on gender was weak. institutions Example: value chains, about which little is known. Short courses on topics such as leadership. Training ASARECA will collaborate with institutions and organizations that have expertise for training agricultural extension workers in undertaking the training. Original role was for ASARECA to help identify universities to host postgraduate students – Comment: May not be feasible 7. Governance Participate in the regional governance of EAAP project through representation of ASARECA Board on the regional governance unit. 8. Administration and Management and coordination management of EAAP Reporting and accountability of ASARECA has pledged to ensure the Vincent coordinates ASARECA ASARECA managed resources and highest level of professionalism in inputs and supports regionality - activities. managing EAAPP affairs and facilitating people to work resources. EAAPP is like a new baby together. Provides Secretariat in the house and ASARECA is paying for the Regional Steering deserving attention to this newborn. Committee (Permanent We will do all it takes in collaboration Secretaries etc).

with the RCoEs to see the baby grow and complete its lifespan. 9. Agricultural research for development Participate in the review and approval of regional research components. Source: EAAPP PAD; ASARECA and EAAPP stakeholders

Overall, based on the information on its website, ASARECA’s interpretation of its role in EAAPP, is broadly in line with what is indicated in the Uganda PAD. However, ASARECA’s ability to contribute inputs is dependent on its having adequate financial and technical resources. Only Uganda has contributed the full amount (US$200,000) that each country was intended to allocate to ASARECA. This has meant that ASARECA inputs on policy through PAAP, for example, have had to be resourced through the ‘flexible’ window of ASARECA’s CGS.

General observations were that ASARECA staff have done a good job, and EAAPP staff showed particular appreciation for ASARECA’s inputs in:  Facilitating regional meetings  Identifying topics for regional research projects  The guidelines on the description of the technologies which were provided by ASARECA  Training in ESS, value chains.  Guiding the working group on policy harmonization  Providing the M&E framework and the PMERL software

Overall countries agree that ASARECA needs to have a strengthened role in EAAPP if full regionality is to be achieved. Options for funding ASARECA inputs include: direct commissioning for specific inputs; funds for regional projects released to ASARECA; increased contributions from the current levels (and countries to meet existing pledged contributions).

9.7 Enabling and constraining factors impacting on the programme and their handling by EAAPP

9.7.1 Policy and institutional barriers ASARECA has been active in seeking supporting policy harmonization in this area for almost a decade. However, the procedures for the import and export of plant materials remain highly restrictive in several countries in the region, notably Ethiopia and Kenya. Until more streamlined procedures are introduced the present constraints on regional seed trade will remain. ASARECA has continued to facilitate policy change in the exchange of plant materials between countries during the project. Draft legislation has been prepared for review in national parliaments and it is hoped that this process will lead to the desired results within the next one or two years.

9.7.2 Delivery of programme outputs, outcomes, objectives against budgets One of the key constraints to the efficiency of project implementation identified by EAAPP staff is the delay in the disbursement of funds. It is critical for the smooth functioning of the project that funds are available at the time when they are needed and delays in the release of funds compromise the effective implementation of planned activities. The effects are especially serious for sub-projects as the absence of funds may result in delayed planting of field trials or the postponement of other activities such as training events. Slow disbursement of funds tend to be a common feature of government-funded projects but the

60 EAAPP I Evaluation Final Report NRI AfrII consequences are more severe in regional projects when activities need to be coordinated among countries.

Programme management unit (see: Governance). A coordinating unit is needed with a large number of implementing agencies. A key dimension is building team work. The project has “really strengthened bonds between the various agencies”. There is a cost but this should be related to performance. The experience with previous World Bank projects is that if a coordinating office is housed within one agency it is seen as a project of that agency. There is a cost in paying for office space but this is more than repaid through the greater ownership and efficiency that comes with an independent unit.

Ethiopia may be spreading resources too thinly by involving large numbers of research centres over a wide area. This can be good for technology dissemination but is potentially high cost and may be reducing the quality of research.

The Tanzania programme had a shift in direction at Mid-Term Review, when the decision was made to re- focus resources on the RCoE with other centres relocating staff to the Centre to create a critical mass of research human and physical resources to catalyse research.

9.7.3 Partnerships with universities

Postgraduate studentships. In general, the research carried out by postgraduate students in each of the countries was well integrated into sub-project activities and students have made an important contribution to the project. However during a discussion held with four postgraduate students in Uganda it became apparent that the particular needs of the sub-projects were not always well suited to the academic requirements of postgraduate study. Several students reported that there were difficulties in aligning their proposed research objectives with the academic requirements imposed by their supervisors at Makerere University. There were also long delays in the procurement of consumables which affected the progress of their research. Lengthy response times from supervisors in providing feedback on written work submitted to them by the students led to further delays.

A key lesson from these experiences is that there should be closer links between the research institutes and universities in the arrangements for the studentships and the planning of the research component. This disjuncture probably reflects the generally limited involvement of universities as partners in EAAPP activities. This needs to be reviewed in any further phase of the project as it is important to link different agricultural knowledge centres to enhance efficiency. Universities have specialist knowledge in certain areas in which agricultural research institutes have limited capacity; for example, in value addition and markets. Universities also have advanced laboratory facilities that can be utilised in project research. There are some examples where partnerships with universities have been successfully carried out in EAAPP. In cassava research a useful contribution has been made by Kenyatta University by involving postgraduate students in the development of protocols for the regeneration of cassava plantlets. In wheat research in Ethiopia, Haramaya University has participated in socio-economic studies and project staff have access to the newly-established Biotechnology Research Institute at Bahir Dar University. This type of collaboration can be highly productive and should be encouraged in future research.

9.7.4 Partnership with implementers

EAAPP projects partner with a range of stakeholders including government at central and local levels. With the process of decentralisation put in place across the region, decision-making and responsibility for extension and dissemination activities are now at local government level. At this level partnerships with

61 EAAPP I Evaluation Final Report NRI AfrII the programme tend to be on one-off project activity basis, which has limitations in terms of sustainability. In theory, local government can allocate funds for maintaining activities after EAAPP funding has finished, but in practice, agriculture is generally low in the list of priorities at local level. To improve buy-in by local governments and increase sustainability, EAAPP could look at providing a small funding allocation to local government departments of agriculture and livestock with a good track record in project implementation.

Figure 6 Partnerships with extension services are key to sustaining farm level innovation and need to be supported

9.7.5 International partnerships

International collaboration is primarily with CGIAR institutes: for example, the Rice RCoE collaborates with Africa Rice, IRRI and IITA in capacity building, sharing and evaluating research materials from advanced yield to on-farm trials before variety release. The RCoE also collaborates with the Chinese Demonstration Centre at Dakawa to evaluate hybrid lines. JICA has been supporting the systems of rice intensification in 20 irrigation schemes (see also JICA support to rice research at Namulonge).

There is also collaboration with advanced research institutes, particularly in molecular breeding techniques and disease characterization; for example, in connection with the international wheat rust nurseries and identification of strains of cassava virus diseases and candidate resistance genes against these pathogens.

Partnerships with the CGIAR and advanced research institutes are beneficial but RCoEs will need to increasingly assimilate some of the functions traditionally played by these organizations. The CGIAR has a new capacity development strategy it has a reduced commitment to capacity development than in the past and this is unlikely to change. The RCoEs will need to help fill this space. The primary focus of the CGIAR will remain on upstream rather than adaptive research, but RCoEs will need to enhance their capacities in advanced research methods in order to engage effectively with the CGIAR and advanced research institutes in emerging areas of research. Under the CGIAR Consortium Research Programmes collaboration with national partners will need to be highly strategic and the RCoEs are well placed in this regard. The involvement of SROs and NARS in CGIAR priority setting has generally been limited and RCoEs will have a role to play in this in the future.

9.7.6 Sustainability Continued financial support for regional centres of excellence will depend on delivery of documented benefits in excess of those that each participating country might expect to achieve independently. The inclusion of support for dissemination and for the seed sector addresses sustainability, as does emphasis

62 EAAPP I Evaluation Final Report NRI AfrII on monitoring and evaluation. The project will need a proactive public relations effort to build awareness among key stakeholders.

The long term sustainability of regional investments is also supported by the commodity focus of each regional center of excellence. The four selected commodities are of high national and regional priority. The commodity of highest national and regional priority, however; i.e., maize, is conspicuously absent from the list. A decision to concentrate maize research in one of the countries at an early stage of regional cooperation would very likely have been rescinded at the first set-back in results or shortfall in production since the strategic importance of the commodity is so high. The participating countries appropriately chose to open a new phase of cooperation under the EAAPP with products other than maize.

9.8 Gender and equity

9.8.1 Gender ASARECA has developed a Gender mainstreaming strategy and has provided support to EAAPP through training of key staff. Each partner country has an EAAPP Gender Focal Point and has prepared a gender action plan. Gender Focal Persons have been supported with training (e.g. Gender Development Management course in Swaziland). A Gender Mainstreaming Training Workshop was held with 42 key project implementers (17 men and 25 women). Gender sensitization was given to the PCU members at one of its quarterly meetings and a Gender Mainstreaming Training Manual has been prepared. EAAPP projects have been supported to ensure that at the design and subsequent stages proposals consider gender responsiveness of research, as key to relevance and quality of research.

Gender studies have been planned by each country. In Ethiopia, a study was conducted in Ethiopia on ‘Gender perspectives and dynamism in wheat production in Ethiopia: the case of wheat belt areas of Arsi zone’.

The main findings were:

 Female headed households have difficulty in leasing land for wheat production as their limited social contact within the community places them at a disadvantage.  Decisions on the sale of grain are usually made by men, although women in a male-headed household are sometimes authorised by the man to sell grain if money is needed in an emergency.  There are clear gender divisions of labour in both crop and livestock enterprises.  During the peak season of wheat production women and men contributed 34% of and 64%, respectively, of labour for wheat farming activities. However, women spent about 17 hours per day at work in total, compared with 12 hours for men.

 Women farmers are more technically efficient than men farmers. The mean technical efficiency indices for women and men were 0.808 and 0.700, respectively. Scale efficiency indices were 0.780 and 0.816 for women and men, respectively.

Gender reporting by projects shows the distribution of female and male farmers and other stakeholders in training and other project activities (Table 28). Overall, the ratio of males to females trained under EAAPP projects is 2:1, with the lowest proportion of women trained in Ethiopia, attributable to prevailing social and cultural factors. In Kenya, where training was initially male-dominated, particularly in dairy, the project has taken steps to increase female participation, including establishment of women-only groups. This appears to have had a positive impact, with Kenya training close to equal numbers of men and women in 2013.

Table 29: Farmers trained under EAAPP projects (short-term training)

Kenya Ethiopia Tanzania Uganda Total Percent

Male 26,312 27761 9454 11263 74,790 68.0%

Female 19,115 6472 4545 5019 35151 32.0%

Total 45,427 34233 13999 16282 109941 100.0%

Source: PMP data

According to the Project Appraisal Document (World Bank 2009), gender issues were included in the selection of commodities of focus, and were to be explicitly considered in design of training and dissemination. However, in practice, gender was not given serious consideration in project implementation until midway through the project, when the gender strategy was developed and gender Focal Persons appointed.

Gender issues were one of the 11 criteria listed in the guidelines for documenting good practices. In most of the good practice descriptions in the Tanzania good practice inventory (the only one available online), though, there is limited information provided and in many cases the word ‘None’ is included, indicating that the authors consider there are no gender issues.

EAAPP project targets on training and participation currently more or less reflect existing gender imbalances. However targets can play a role in boosting female participation and could be set higher under EAAPP II. Achieving targets would require focused support on gender sensitization and programming for project partners and participants.

EAAPP Kenya was selected to participate in a JICA-funded gender mainstreaming programme, aiming to change mindsets and increase women’s control over resources including labour, land and incomes. This pilot should be closely monitored and evaluated and successes scaled out across EAAPP.

9.8.2 Youth Definitions of youth vary across the region from 14-15 up to 25-35 years. The age group constitutes around 50 percent of the region’s population and represents a considerable potential resource to the agricultural sector. The small number of young people venturing in commercial farming as an occupation is seen as a concern by agricultural policy makers as negatively impacting on the sector growth. The majority of farmers are aged over 50 years and their children are reluctant to join the farming occupation due to low returns to investment and drudgery associated with farming operations.

The government of Kenya has made it mandatory for projects and programs to focus on youth and integrate them into the planning and implementation of activities and give them opportunities for income generation and employment (Kenya Integrated Youth Action Plan). The youth are seen as a key resource that can be tapped for youth economic empowerment and food security. Young people are energetic and ready to venture in initiatives that promise to generate quick income. Thus, there is need to address issues affecting youth by developing strategies that provide the youth with meaningful opportunities in agribusiness. It is important to integrate the youth in identifying opportunities along commodity value chains and to enable them to learn from successful cases of youths already practicing agriculture through exposure tours and peer review fora. Youth also require initial support in setting up businesses since they lack capital and may not have security required for borrowing from financial institutions. Youth require

64 EAAPP I Evaluation Final Report NRI AfrII mentoring from experienced business practitioners who would also provide linkages to markets and other service providers including financiers.

EAAPP is working with established youth groups and in some instances supports new groups to take advantage of emerging business opportunities. An example was seen in Kenya of a group of young people who had succeeded in developing a wheat seed farm and were expanding into agro-inputs, transport and butchery businesses.

In Kenya the programme has drawn up a youth project/strategy focusing on creating employment and generating income for unemployed youths as well as influencing school children to appreciate agriculture as an important business. The overall objective is to increase youth employment and income generation by engaging young people in agribusiness opportunities along the dairy, cassava, wheat and rice value chains. Specific objectives are

 To disseminate promising technologies to school and out-of-school youth in EAAPP Cluster Sub Counties

 To promote establishment, strengthening and growth of agribusiness initiatives among youth

 To promote school agricultural programs within EAAPP sub counties.

It is recommended that: Elements and lessons from national initiatives to engage young people in commodity value chains should be distilled and shared among EAAPP countries and mainstreamed into an EAAPP Youth Strategy.

9.9 Economic analysis

Estimated project benefits

Project benefits in terms of change in incomes of EAAPP beneficiaries over the project period have been estimated using valuation of net increased agricultural production of beneficiaries over non-beneficiaries (Difference In Difference). Estimates of the change in value of key crops and livestock commodities by beneficiaries and non-beneficiaries between 2009 and 2014 are derived from Section 3.3. Table 30 shows average increases in net value of agricultural production in 2014 compared to 2009 of $307 for wheat beneficiaries over non-beneficiaries, $175 for cassava beneficiaries and $391 for rice beneficiaries (2014 constant prices). The average net value of dairy production by beneficiaries has increased more slowly than non-beneficiaries’. This appears to be the result of low market prices being paid to many EAAPP group members. However, dairy beneficiaries reported supplementing their incomes through sales of calves, so the difference in net incomes may not have fallen significantly and is treated as zero.

The numbers of EAAPP direct beneficiaries are estimated by the project at 108,633 for wheat; 245,251 for cassava; and 265,125 for rice (EAAPP/ASARECA M&E 2015). Of these, the proportions reporting increased output/income as a result of taking up the technology are 29 percent for wheat producers, 46 percent for cassava and 68 percent for rice. This gives an estimated 421,141 direct beneficiaries reporting improved production.

Total net value of the increased production of beneficiaries compared to non-beneficiaries in 2014 is estimated at $9.68 million for wheat producers, $19.93 million for cassava growers and $70.65 million for

65 EAAPP I Evaluation Final Report NRI AfrII rice producers. This represents a net increase in agricultural production value of US$100,262,104 (change in beneficiaries compared to non-beneficiaries) across EAAPP countries. Benefits shown are for one year only (2014 compared to 2009).

Table 30 Valuation of EAAPP net benefits

Change in net value Direct of agricultural Total net value of Direct beneficiaries production 2009- beneficiaries Commodity beneficiaries reporting 2014 by EAAPP increased (total) increased beneficiaries vs. non- production (US$) production beneficiaries (US$) Wheat 307 108,633 31,538 9,682,150

Cassava 175 245,251 113,880 19,929,036

Rice 391 265,125 180,693 70,650,918

Dairy - 180,882 95,030 -

All 325 799,891 421,141 100,262,104 Source: Own calculations from EAAPP M&E data and Evaluation survey

Benefit-cost estimates

EAAPP total project costs are estimated at $120 million (PAD). Comparing the valuation of benefits, $100.3 million, against estimated programme costs indicates that in around 1.2 years project benefits would be expected to have equalled costs. Thus the benefit-cost ratio (not discounted) would be expected to be positive midway through 2015.

The analysis assumes a constant level of benefits to beneficiaries. Whilst projects surveyed were generally those expected to show impact (after around 3 years of implementation), in many cases benefits had only just begun accruing to beneficiaries during the reporting period (2014) and were expected to increase. Changing the assumptions would change the project pay-back period and/or benefit-cost ratio. Even with quite large changes in assumptions (e.g. reducing overall benefits by 40 percent), returns to the investment under EAAPP I look positive by end of 2015.

It should be noted that an economic analysis of individual projects was beyond the scope of this programme evaluation. However, it would be useful to evaluate a cross-section of EAAPP projects to assess the relative economic benefits of different types of projects by different implementers (e.g. government and small-scale seed producers) and the combination of projects which would deliver Project Development Objectives most efficiently. Cost effectiveness of different approaches to achieving regional impact could also be assessed e.g. different communications approaches, exchange visits etc. This would assist in identifying ‘best bets’ for support under the proposed phase 2 of EAAPP.

10 ENVIRONMENTAL SUSTAINABILITY

10.1 Introduction

This section presents an evaluation of levels of conformity of project interventions with sound environmental management practices in general and the national legislative requirements of member countries, and compliance with the ASARECA Environmental and Social Management Framework (ESMF).

Following the terms of reference for the impact evaluation, we set out to assess compliance with Environmental and Social Management Framework (ESMF) by

 Reviewing the efficacy and adequacy of ESMF and project documents in ensuring ESS compliance, including adequacy of existing internal controls to identify and mitigate adverse impacts

 Assessing the level of awareness and sensitization about ASARECA’s ESMF and other relevant environmental standards, regulations, law and policy among project implementers

 Examination of monitoring programs, parameters, and procedures in place for control and corrective actions at organizational, program management and sub-project levels

 obtaining views on social issues from project implementers and other potentially affected persons

 describe major impacts on the environment including, but not limited to potentially affected ecological and socio-economic areas

Project proposals informed the evaluation on how environmental issues might have been incorporated in program design and a combination of survey responses, observations, key informant interviews and review of progress reports were used to evaluate the level of incorporation of environmental issues in implementation. In order to assess awareness and level of implementation of the EAAPP Environmental and Social Management Framework (ESMF), a self-reporting survey was conducted with project implementers as the target group. The survey questionnaires were administered to key implementers at Research Centres of Excellence and PIs. Fifteen responses were received. Progress reports and project appraisal documents also provided information of the level of awareness and sensitization.

10.2 Compliance with Environmental and Social Safeguard requirements

Awareness and Implementation of the ESMF

Review of project appraisal documents and consultation with implementers at Research centres of excellence revealed that all implementers at the Centres of Excellence level are aware of the ASARECA Environmental and Social Management Framework (ESMF). Based on the ASARECA Framework, an Environmental Management Framework was developed for each EAAPP country. Its implementation is monitored by Environmental focal persons at the regional centres of excellence.

Monitoring records and appraisal documents indicate that trainings for project implementers on the identification of likely impacts, preparation and implementation of Environmental and Social Management Plans (ESMPs) have been organised.

For example, Progress reports from Kenya indicate that implementation of the Environmental and Social Management Framework has mainly been through capacity building and development of recommended ESS instruments (screening and Management Plans). Fifty Seven (57) Principal Investigators received

67 EAAPP I Evaluation Final Report NRI AfrII training as Trainers of Trainers within the first half of 20148. The participants were trained on ESS instruments and guided to carry out screening of their respective projects/activities, develop Environment and Social Management Plans (ESMP)/ Integrated Pest Management Plans (IPMP) and the monitoring plans. The training also covered safe use of chemicals, disposal of the empty containers and wrappings. The Pest Control and products Board (PCPB) approved list of pesticides was circulated to guide the field users. Training also included a review of an Environment and Social Safeguard Action Plan that was developed to guide in mainstreaming of ESS issues in project implementation.

Additionally, at farmer/ household level, attempts have been made to disseminate key components of the ESMF including promotion of Integrated Pest Management, on safe use of chemicals and proper disposal of empty containers and wrappings. In Kenya; for example the Wheat Appraisal report of July 2014 reports that all farmers use agrochemicals on their farms and that majority have received training regarding the safe use of chemicals. However there was low participation by farmers in EAAPP capacity building activities: attending farmer training (27.3%), field days (24%) and demonstrations (19.9%).

It also reports on the proportion of farmers exposed to technologies on pests and disease control (13.2%), weed control (13.2%), use of improved wheat varieties (12.8%), land preparation and planting method (12.6%), use of soil fertility management technologies (12%) and safe use of chemicals (11.8%).

A Regional training workshop was also organised by ASARECA in collaboration with the resource persons from the World Bank in September 2013 involving relevant stakeholders from the four EAAPP countries.

In Ethiopia, a total of 156 people (119 men and 37 women) were trained in 2014 following the regional training of trainers of September 2013. Training concentrated on World Bank guidelines for ESS, the need to practice IPM as the first option to minimize pests; effects of pesticides and herbicides; the need for personal protective equipment (PPE); chemical handling & management. Training of implementers is reported to have highly benefited farmers in availing them with safer pesticides, and access to knowledge on appropriate handling and PPE. Similarly, training on food safety with emphasis on Hydrogen cyanide in cassava was conducted in all regions. In August 2014, a sub-project screening workshop was conducted in Ethiopia including all EAAPP countries.

Youth Groups have also been supported to establish/strengthen economic initiatives including feed business, disease control and provision of services. The groups have been trained on safe use of acaricides and proper disposal of the containers.

A survey conducted with project implementers during this impact evaluation revealed that all respondents were aware of the Environmental and Social Management Framework. Respondents however unanimously indicated that as much as they were now aware of the EAAPP/ASARECA ESMF, it was presented to them 3 years into project implementation.

EAAPP project implementers’ were involved in a wide range of ESMF activities. Around fifty percent of those interviewed stated that their projects were involved in: preparation of Environmental Management plans, environmental screening (Table 31), Integrated Pest Management planning and Environmental Impact assessments, as well as Capacity Building for ESS and Training of trainers.

Table 31 ESMF activities carried out by EAAPP projects (%)

8 EAST AFRICAN AGRICULTURAL PRODUCTIVITY PROJECT KENYA. REGIONAL DAIRY CENTER OF EXCELLENCE (RDCoE) TECHNICAL PROGRESS REPORT (JANUARY- JULY, 2014). NAIROBI : s.n., 2014

ESMF Activity Projects engaged in ESMF activities (%) Environmental Screening 53.3% Environmental Impact Assessment 46.7% Preparation of Environmental Management Plan 60.0% Preparation of a Resettlement Action Plan 6.7% Integrated Pest Management Planning 53.3% Capacity building for Environmental and Social Safeguarding 53.3% Budgeting for Environmental monitoring 33.3% Training of Trainers for ESS 53.3% None of the above 6.7% Source: Evaluation survey

Environmental reporting is an important component of ESMF activities. Over the past 12 months, around 20 percent of project staff interviewed had submitted quarterly ESMF reports, one third had submitted six- monthly reports, and 8 percent annual reports only. Around 40 percent had not submitted a report in the past 12 months (Figure 2).

Figure 7: Project implementers submitting ESMF reports over the previous 12 months

10.3 Environmental impacts

Impacts on the social and ecological environment were predicted in ESMF documents for EAAPP projects. The Environmental and Social Management Frameworks anticipated low/ negligible prospect for large- scale, significant or irreversible environmental impacts associated with EAAPP, however, potential risks or negative impacts that could arise were identified. These included:

i. Increased vulnerability to pests due to poor pesticide management or introduction of new cultivars

ii. Localized agro-chemical pollution and reduction of water quality from agro-chemical use or poor handling of pesticides and disposal of empty chemical containers; and

iii. Land or water degradation due to the rehabilitation of small scale irrigation systems or the construction or rehabilitation of buildings

iv. Unintended movement or transmission of genetically modified organisms (GMOs) within or between countries as a result of field trials or other research activities

v. Unintended alteration of livelihoods or destruction of physical cultural property as a result to land cultivation during research or seed multiplication activities

Based on project reports, none of these negative impacts were experienced during project implementation. From the evaluation survey, project implementers reported potential environmental and social impacts (negative and positive) of their projects (Table 32). The main potential negative impacts were pesticide use (different agrochemicals, including herbicides) (85 percent) and solid or liquid waste generation. There were also a small number of cases of potential noise generation, clearing vegetation, resettlement and increased use of resources. Potential positive impacts were decreased use of water or energy resources and use of local knowledge. ESMP documents show impact indicators including poor handling of waste, delays in restoration following ground disturbance and absence of guidelines on management of waste from laboratories (including wasted genetic material) have been identified as causes for concern.

Table 32: Environmental and social impacts observed by EAAPP project implementers

Environmental and Social Impacts Negative Percent of cases Positive impacts identified impacts Noise generation during construction x 7.7% or operations Solid or Liquid waste generation x 23.1% Pesticide use (any agrochemical x 84.6% including herbicides) Genetically Modified Organisms - 0% Clearing a significant amount of x 7.7% vegetation/trees Resettlement or compensation of x 15.4% residents Use of local knowledge x 23.1% Increased use of resources e.g. energy x 7.7% or water Decreased use of resources e.g. energy x 15.4%

or water

Table 33 summarises findings from the Environmental Monitoring Unit regarding environmental impacts experienced in Tanzania across a range of projects implemented under the EAAPP9. Findings indicate that there is urgent need to build capacity of all project implementers and to develop guidelines on handling the various streams of wastes associated with project activities, occupational safety and health especially of laboratory workers.

Table 33 Case study of Projects Implemented in Tanzania PROJECT ESMP IMPACT Establishing and Promoting Fertilizer Recommendations for Upland Rice Eco-system in the Eastern Zone yes Awareness creation Use of Lablab (Dolicus lablab) to Improve Soil Fertility and Value Addition in the Rice/ Legume Cropping System for Better Farmer Awareness creation on fertilizer Livelihoods yes application Enhancing Adoption of Appropriate Policies and Safety Awareness creation on safety of Standards of Feeds and Milk for Improved Livelihoods yes products on the market Improving soil fertility management in rice ecosystems yes Awareness creation Extent of salt affected land and characterization for salt tolerance of rice varieties grown in irrigation schemes of Trained farmers on Toxicity of run off Tanzania and Kenya yes associated with use of UREA Enhancing farmers’ household food security and income through development of high yielding drought tolerant rice with related Increased biosafety & controlled diversified products and by-products yes disposal of waste Promotion of NERICA varieties to increase production and Reduced social impacts flood retention productivity of upland rice yes structure installed Improving Livelihood through Enhanced Wheat Production in Tanzania, Kenya and Ethiopia yes ESMP formed following monitoring Increasing Competitiveness of Rice Value Chain in ECA Regions: Rice Value Chain Analysis and Development yes Farmers sensitization Testing and Promotion of Mechanization Technology for Improving Rice Productivity yes Provision of protective gear Enhancing Rice Productivity through Integrated Disease Management: Determination of Virulence Diversity, Resistant Germplasm and Introduction of Management Options for Brown Spot Disease Pathogen yes Promoted Awareness on lab safety Development of Integrated Technological Packages for Potential soil and water contamination Increasing Rice Production in Low land Rain fed Rice yes from pesticides Cassava Improvement through Germplasm Deployment and Management of cassava brown streak Sustainable Management of Cassava Brown Streak yes disease Increasing Rice productivity though enhanced water Management yes n/a Cassava improvement in Tanzania through germplasm yes Health hazards identified; increased deployment and sustainable management of Cassava Brown awareness for workers and

9 EAST AFRICA AGRICULTURAL PRODUCTIVITY PROJECT REGIONAL RICE CENTRE OF EXCELLENCE. REPORT OF THE MONITORING/SUPERVISION ON THE INTEGRATION OF ENVIRONMENT AND SOCIAL SAFEGUARDS (ESS) ON EASTERN AFRICA AGRICULTURAL PRODUCTIVITY PROJECT (EAAPP). s.l. : Republic of Tanzania Ministry of Agriculture Food Security and Cooperatives, 2014.

PROJECT ESMP IMPACT

Streak Disease management Reduced fire risk, reversal of degradation; increased adaptation to Production of Quality Pasture Seeds yes climate change - growing own pasture Rehabilitation of irrigation field trials at Ukiruguru Mwanza yes Impacts on flood retention Evaluation of Integrated Disease and Pest Management Options Reduction of risks associated with for cassava in Tz yes pesticide use Improving Indigenous cattle for dairy production through targeted selection and crossbreeding in ECA Countries yes Farmer training on husbandry Poor sanitation & waste management, low level of community stakeholder engagement, creation of vector breeding sites, encroachment on CIVIL WORKS None community area

A review of monitoring reports on civil works undertaken within Tanzania reveals that Environmental and Social safeguard requirements were not considered prior to commencement of all civil works. This resulted in impacts on local community for example conflict during some rehabilitation works. Other impacts from construction activities observed included delays in restoration of earth burrow pits and low consideration for re-vegetation in some areas.

10.4 Agrochemical use and protection

A survey was undertaken by the evaluation team at farmer level to establish whether there might be a significant difference in the use of agrochemicals between EAAPP beneficiaries and other households (see sample population in section 2). Results indicate that in Ethiopia there is a significant difference between beneficiaries and non-beneficiaries using chemicals on their farms. No significant difference was implicated in Kenya or Uganda, whilst in Tanzania results indicated that beneficiaries were using more chemicals than non-beneficiaries (Table 34).

Table 34 Use of agrochemicals by farmers surveyed

Country Crop Beneficiary Non-Beneficiary P-Value ETHIOPIA Cassava 62.7% 48.8% 0.179 Wheat 94.0% 86.1% 0.046 KENYA Rice 83.3% 69.7% 0.161 Dairy 72.8% 80.6% 0.179 TANZANIA Wheat 82.9% 54.8% 0.009 Rice 81.6% 59.6% 0.000 UGANDA Cassava 40.0% 29.7% 0.081 Dairy 51.4% 58.8% 0.527 All Cassava 45.9% 34.6% 0.031 Wheat 91.1% 79.1% 0.003 Rice 82.1% 62.0% 0.000

Dairy 67.9% 74.8% 0.200 All All 70.2% 61.4% 0.001

Use of Personal Protective Equipment (PPE)

The farmer survey found that use of protective equipment (PPE) whilst handling agrochemicals and doing other agricultural work was very limited in Uganda with up to 93.4% of beneficiaries reporting to use no PPE and a slightly higher 96.1% of the non-beneficiaries not using any PPE (Table 35). Beneficiaries in Kenya (80.2%) and Tanzania (84.5%) also reported non- usage of protective equipment (PPE). Compared with the other countries, beneficiaries in Ethiopia used more PPE with 60% reporting use. High cost was the main reason cited by beneficiaries for not using protective clothing.

Table 35 Type of protective clothing used by farmers surveyed

Country Type of protection Beneficiary Non-Beneficiary ETHIOPIA None 30.4% 35.6% Gloves 3.6% 1.3% Nose masks 6.0% 2.7% Overalls 0.6% Others 59.5% 60.4% KENYA None 80.2% 86.5% Gloves 12.6% 2.4% Nose masks 0.8% Eye goggles 0.6% Overalls 3.0% 4.8% Others 3.6% 5.6% TANZANIA None 84.5% 87.1% Gloves 7.7% 7.1% Nose masks 5.2% 0.7% Overalls 0.7% Others 2.6% 4.3% UGANDA None 93.4% 96.1% Gloves 3.3% 2.0% Nose masks 0.5% Eye goggles 1.1% 0.7% Overalls 0.5% Others 1.1% 1.3%

10.5 Challenges in implementation of ESMP

Awareness of the ESMF is high amongst EAAPP project implementers and stakeholders. However, there have been a number of challenges in implementation of the ESMP by projects. These include:

• Limited resources: Vehicles & Limited funding, slow procurement of printing service for the safety manuals

• Low level of human resource allocation for implementation

• Incorporation of ESS was done later in the project. The training was done in the 3rd (2013) year of implementation

• High cost of protective clothing, so that even though project beneficiaries have been trained in use of PPE, the majority are not using it.

10.6 Recommendations – Environmental assessment

Recommendations based on the impact evaluation include:

 Regular capacity building for laboratory staff particularly on topics related to handling of chemicals in order to avoid any health risks.

 Weather monitoring should be undertaken alongside keeping a log of chemicals used and the trends in usage. It is particularly important that projects involving field trials and irrigation have in place a mechanism to monitor nutrient content in run-off and nearest water bodies.

 Regular training on biosafety, and occupational safety and health is recommended at all levels of implementation including at farm level to increase usage of protective equipment and knowledge of safe varieties and safe practices of handling.

 Provision of access to finance for project participants (groups) to purchase protective equipment, e.g. a revolving fund.

 Increase funding for implementation of mitigation measures

 Establish research on waste disposal including laboratory chemical wastes. Liaise with National Environment Authorities on hazardous waste management

 Monitoring erosion from field trial sites

 Where project activities involve setting up or renovation of physical infrastructure, construction supervision of contractors by qualified personnel must be enhanced in order to minimise the resultant impacts observed from construction works.

11 EAAPP OVERALL ASSESSMENT AND RECOMMENDATIONS 11.1 EAAPP Overall assessment The evaluation team considered the question: what is now required to enhance the regional dimension in a possible Phase 2 of EAAPP? The following approach is proposed to guide project preparation:

1. Review lessons from the West African Agricultural Productivity Programme (WAAPP).

a. Conseil Ouest et Centre Africain Pour la Recherche et le Developpement Agricoles/West and Central African Council for Agricultural Research and Development (CORAF/WECARD) plays a role in the management of regional projects in WAAPP. It is recommended that the project looks at this and draws out lessons: How does this operate and would such a model improve the coordination of regional activities under EAAPP?

b. Under WAAPP there is a component which addresses regional harmonization much more explicitly than in EAAPP. Might a similar arrangement be appropriate for a Phase 2 of EAAPP?

2. Greater attention needs to be given to regionality of extension structures and delivery mechanisms in upscaling EAAPP technologies.

3. Centralized or decentralized Centre of Excellence model? In Tanzania, which originally had a decentralized approach, facilities are now being concentrated at Katrin. The expectation is that this will leverage internal and external resources. A similar approach should be considered for Ethiopia where activities are quite widely dispersed.

What should be the aim by the end of a possible Phase 2?

 Economies of scale may be achieved through the operation of regional centres of excellence but they still have to be financed from somewhere. A sustainable funding model needs to be developed which may involve a combination of public and private funding. Long-term support from central government budgets will have to be at the core of any sustainable funding approach.

 The centres of excellence should be capable of hosting collaborations with international research centres, along similar lines to a recently announced link between the John Innes Centre in the UK, the Shanghai Institute of Plant Physiology and Ecology and the Institute of Genetics and Developmental Biology in Beijing, which will investigate challenges facing agriculture and human health.

11.2 Key Recommendations 1. Thematic areas: There is a fairly even distribution of research efforts across themes. However there is need for more emphasis on mechanization and post-harvest issues given the potential beneficial impacts from mechanization, transport, storage, processing and packaging. These impacts are especially important for women - in reducing drudgery and increasing income-earning opportunities. 2. Long-term training: The disciplinary areas chosen for the studentships are judged to be relevant. However, in view of the greater capacity gaps in post-harvest and value chain/marketing a proportionally higher level of investment in these areas is required. Further capacity development is also needed in livestock breeding (animals and fodder). There is also limited capacity in genomics.

3. Gender and training: In long-term training there are similar numbers of men and women in Masters programmes, but numbers of female PhD candidates are very low and need to be addressed. The project should review selection processes and learn lessons to guide future recruitment.

4. Phase 2 needs to build in mechanisms for mentoring returning students and integrating them in RCoE activities. Governments must recognize the need to provide adequate incentives to retain new human resource capacity.

5. Regional dissemination: various methods have been used to disseminate new technologies between countries. These need to be evaluated for inclusion and impact. The programme needs to identify and support ways to facilitate the dissemination and adoption of technologies through institutionalised processes.

6. The programme has generated a large amount of relevant dissemination materials: leaflets, booklets, posters and manuals and is exploring other media. It needs to exploit the full potential of such communication approaches and support activities to document and share learning on where a particular communication approach has proved to be valuable.

7. RCoE websites need to be improved to give greater visibility to the approaches the project is using and the technologies being developed and promoted.

8. Agricultural research and delivery need to be linked more closely and other actors, including universities and farmer organisations, should be involved more centrally in research and extension activities. Multi-stakeholder and demand-led approaches are key to attainment of Project Development Objectives. Putting an explicit Agricultural Innovation Systems model at the centre of the project would ensure research is relevant to all stakeholders along the value chain.

9. There is large variability in farmer yields across all four commodities. It is recommended that farm-level variability is investigated during EAAPP II to understand the causes and develop technology recommendations for a range of agroecological and socioeconomic conditions. Farmer-centred research methods (e.g. Participatory Technology Development) which involve farmers in all stages of the research process would be most appropriate for this.

10. Impact of EAAPP on food and nutrition security needs to be tracked by the project. A nutrition strategy needs to be developed and capacity built to implement it, to ensure that the project maximises benefits and avoids negative impacts on vulnerable groups, including under-5 children.

11. Research methods support: Stronger capacity in research methods is needed to improve journal publication rates and better designed research. It is recommended that ‘writeshops’ are held in which professional facilitators assist selected researchers to prepare manuscripts for journal publication.

12. Policy harmonisation: Closer harmonisation if needed to improve regional research effectiveness. ASARECA’s Policy Analysis and Advocacy Programme coordinates a regional working group on policy. ASARECA can play a critical role in strengthening the regional dimension of EAAPP, but this requires strengthening of the EAAPP Coordination unit.

13. Institutional arrangements: New countries joining the project need to critically assess: existing institutions and their decision-making processes, cost, sustainability in designing institutional arrangements. A coordinating unit is needed with a large number of implementing agencies. It is

recommended that EAAPP units are staffed by dedicated full-time staff to improve efficiency and timeliness of activities, and to free up PCU staff to engage in scaling out and partnerships.

14. EAAPP M&E systems are well designed and managed, providing timely results for project management. However, it is resource intensive and focused on the PMP which has a large number of quantitative indicators which require frequent follow up. As the programme moves to a new phase more attention is needed on lesson learning, evaluation and sharing best practices.

15. Role of ASARECA: ASARECA needs to have a strengthened role in EAAPP to achieve full regionality. Options for funding ASARECA inputs should be explored including: direct commissioning for specific inputs; funds for regional projects released to ASARECA; increased contributions from the current levels. Countries must also deliver on existing financial pledges.

16. Gender: EAAPP programme targets reflect existing gender imbalances. Targets can play a role in boosting female participation and should be set higher under EAAPP II. Achieving targets requires support on gender sensitization for project partners and participants. Experiences on gender mainstreaming from pilots should be evaluated and scaled-out across EAAPP.

17. Youth: EAAPP has identified as key stakeholders in agricultural commercialisation. Successes and lessons from initiatives such as Kenya’s Youth Strategy should be shared amongst EAAPP countries and partners and mainstreamed through an EAAPP Youth Strategy.

18. Partnerships with universities - closer links between the research institutes and universities would provide benefits in terms of arrangements for the studentships. Universities also have specialist knowledge and facilities that can be utilised in project research and complement research institutions (e.g. in value addition and markets). The role of universities needs to be reviewed in any future phase.

19. Partnerships with local government – Extension and other services are provided at local government level on a one-off activity basis. To improve buy-in by local governments and increase sustainability, EAAPP should look at providing a small funding allocation to local government departments of agriculture and livestock with a good track record in project implementation.

20. Environmental and Social Safeguards recommendations:  Weather monitoring and keeping a log of chemicals used and the trends in usage. Projects involving field trials and irrigation should have in place a mechanism to monitor nutrient content in run-off and nearest water bodies.  Regular training on biosafety, and occupational safety and health at all levels of implementation including at farm level to increase usage of protective equipment and knowledge of safe varieties and safe practices of handling.  Provision of access to finance for project participants (e.g. groups) to purchase protective equipment, e.g. a revolving fund.  Establish research on waste disposal including laboratory chemical wastes. Liaise with National Environment Authorities on hazardous waste management 21. In preparation for a possible Phase 2, EAAPP should review lessons from the West African Agricultural Productivity Programme (WAAPP), including the role of Conseil Ouest et Centre Africain Pour la Recherche et le Developpement Agricoles/West and Central African Council for Agricultural Research and Development (CORAF/WECARD) in management of regional projects, and an explicit regional harmonization component.

Appendix 1 Evaluation Framework Issue Questions Indicators Data sources and methods Relevance  How relevant is EAAPP design/strategy to its aims of  Robustness of ToC/intervention logic Documentation review - ToC, EAAPP PAD, enhancing sustainable productivity, value added and  Alignment to needs and priorities of regional (e.g. CAADP, FAAP, ECA), PMP, CAADP, NDP, ADP, AIPs, Strategic competitiveness of the sub-regional agricultural system? national, NARI and other value chain actors plans of NARIs, needs assessments  How relevant is the program to regional, national and  Role and performance of EAAPP RCoEs vis-à-vis other regional initiatives, Interviews, focus group discussions with farmer priorities? NARS, CGIAR EAAPP staff, agricultural R&D specialists,  What is the strategic positioning of EAAPP as a  Balance of EAAPP investments: infrastructure, research and training, seeds policy makers, donors, other stakeholders knowledge hub in agricultural R&D in ECA (in the context and breeds, management & coordination of other national/regional programs)? Program  How far have program outputs, outcomes and objectives 1 Rate of change in regional specialization and collaboration in agricultural National and regional reports – impact – been achieved: research – no/value specialised and joint research projects leveraged; no./ government statistics, ECA reports, FAO, extent of a) Enhanced regional specialisation in agricultural capacity of researchers COMESA, UN COM-TRADE, IFPRI performance research 2 Rate of increase in information and knowledge transfer across national Including: cassava, rice, wheat, dairy – total towards b) Enhanced collaboration in agriculture training and boundaries – no. technologies, information transferred; no./type of policy/ production, yield, value of total production fulfilling technology dissemination, regulatory changes initiated – 2009-2014 overall c) Facilitate increased transfer of agricultural technology, 3 Rate of change in adoption of new technologies (disaggregated by type ) HH survey Development information and knowledge across national boundaries  New varieties, breeds & management practices, Objective  New handlings and processing methods,  Improved dairy genetic materials 4 Rate of increase in land area with seeds of improved cultivars (%) 5 Increase in productivity at farm level over control technology for all disseminated new technologies (%) 6 Level of stakeholder satisfaction with the technologies and innovations (%) by number of products users (by gender, age, and location ) Increase in research scientists working in regional research projects on the basis of % of total research staff of the RCoE; Increase in the number of new technologies developed by the RCoE; Increase in existing and new technologies disseminated in more than one Program Country; Increase of cultivars for selected commodities registered in more than one Program Country based on the number per selected commodity; Percentage of regional research and training and dissemination activities implemented according to the Annual Regional Research and Budget Plans; Harmonized M&E system for RCoEs in cooperation with ASARECA developed, adopted, and implemented. Effectiveness -  How far have program outcomes been achieved A. Number of direct and indirect beneficiaries in the value chain (disaggregated Document review, key informant Impact on  What is the impact of EAAPP interventions on by gender) (accessing technologies; adopting technologies) interviews, focus group discussions, program beneficiaries along the value chain (direct and indirect B. Type and magnitude of benefits to beneficiaries: changes in output, yields household survey: beneficiaries beneficiaries)? C. Changes in knowledge A. Demographics D. Change in adoption B. Land under key crops, production, value, E. Change in land area with improved cultivars amount sold, prices 2008-2012 F. Changes in yields – production and productivity C. purchased inputs; yields G. Stakeholder satisfaction with technologies D. enterprises; income H. Changes in net benefits from EAAPP technology E. Food groups, 24 hours

I. Change in income F. Main staples before, now; months J. Food and nutrition intake availability K. Changes in food security G. Changes in livelihoods and assets L. Change in technology uptake by non-beneficiaries H. Net benefits from technologies Programme  How efficient has the program and its strategies been in Unit cost of delivering outputs; benefit: cost ratios for outcomes Economic analysis, household surveys, Key Efficiency delivering Program outputs, outcomes, objectives against informant interviews, cost benefit analysis, budgets? cost effectiveness assessment  What have been the rates of return EAAPP of investments? Gender and  How effectively have gender and equity issues been No. and changes in gender sensitive programming Household surveys, Key informant equity issues incorporated into program design and implementation? No. and % women, vulnerable groups in program; activity levels; interviews, gender and poverty analysis  How successful has the programme been in targeting and Changes in women’s knowledge, yield, income, livelihoods reaching specific beneficiary groups – small-scale farmers, women Environmenta  How far have environmental issues been incorporated in Levels of conformity of project interventions with sound environmental ESS audit, review of ESMF and project l sustainability program design and implementation? management practices, national, ASARECA ESFM (selected projects) documents  Are projects compliant with environmental and social Major impacts to potentially affected ecological and socioeconomic areas; ESS checklist for KII, RCs, PIs safeguards requirements and enforcing policy and Adequacy of ESMF and project docs to ensure ESS compliance, existence of legislative requirements? adequate controls to identify/mitigate adverse impacts Awareness and sensitisation on ASARECA ESMF etc. among project implementers Views on social issues from project implementers Monitoring programs, parameters, procedures in place for control/corrective actions at organisation, PM, project levels Sustainability  How effective are selected partnerships formed or Level of integration between national research systems, including national Document review, KII, SWOT, policy and strengthened through the selected projects? agricultural research institutions, private sector processes analysis and uptake pathways Partnerships (national/regional and project levels) Extent of EAAPP contribution to strengthening regional and national research and  What are the options/potential for scaling-up and policy making processes replicability? Evidence of change in levels of collaboration and likelihood of continuing collaboration (national and project levels) Evidence of exit strategy Contribution of ASARECA in facilitating partnerships, learning, policy engagement etc. Programme  What have been the strengths and weaknesses of the Change in capacity (training and staff numbers) (through EAAPP) and performance KII, SWOT and institutional analysis, Burn performance implementation process, including targeting and of staffing, procedures, including financial procedures, management rate and reaching beneficiary groups (small-scale farmers, approach implementati women). Responsiveness of program/project to internal and external factors on  What internal and external factors have influenced program/project implementation, including technical, managerial, organisational, institutional, socioeconomic and political factors?

80 FINAL REPORT – EAAPP I Evaluation NRI AfrII

Appendix 2 Selected Publications under EAAPP

Agza, B., Melesse, K., Funga, A. and Melesse, K. (2013) Assessment of knowledge gap and factors affecting consumption of dairy products in Ada’a and Lume districts of East Showa Zone, Ethiopia. African Journal of Food Science and Technology 4(9): 201-210. A

Atuhaire, A.M., Mugerwa, S., Okello, S., Lapenga, K., Kabi, F. and Lukwago, G. (2014) Prioritization of Agro- industrial By-products for Improved Productivity on Smallholder Dairy Farms in the Lake Victoria Crescent, Uganda. Frontiers in Science 4(1): 1-7. A

Melesse, K., Agza, B. and Melesse, A. (2014) Milk marketing and post-harvest loss problem in Ada’a and Lume districts of east Shoa Zone, Central Ethiopia. Sky Journal of Food Science 3(4), 27-33. A

Mwambene, P.L., Chawala, A., Ilatsia, E., Das, S.M., Tungu, B. and Loina, R. (2014) Selecting indigenous cattle populations for improving dairy production in the Southern Highlands and Eastern Tanzania. Livestock Research for Rural Development 26 (3) A http://www.lrrd.org/lrrd26/3/mwam26046.html

Tadesse, T., Degu, G., Shonga, E., Mekonen, S., Addis, T. and Yakob, B. (2013) Current status, Potentials and challenges of Cassava production, processing, marketing and utilization: Evidence from Southern Ethiopia. Greener Journal of Agricultural Sciences 3 (4): 262-270. A

Yami, M., Solomon, T., Begna, B., Fufa, F., Alemu, T. and Alemu, D. (2013) Source of technical inefficiency of smallholder wheat farmers in selected waterlogged areas of Ethiopia: A translog production function approach. African Journal of Agricultural Research 8(29): 3930-3940.

Appendix 3 Key Persons Interviewed Country Person Position Organisation Kenya Jane Muriuki National Coordinator EAAPP Coordination Unit Jedidah Maina Research EAAPP Coordination Unit Boniface Mutua M&E Specialist EAAPP Coordination Unit Andrew Karanja Co-TTL Kenya World Bank Tobias Onyango RDCoE Coordinator RDCoE T. Lanyasunya Director, KALRO Dairy Institute, Naivasha Joseph Kamau Seed Dairy Institute, Naivasha Douglas Indetie M&E Officer Dairy Institute, Naivasha Evans Ilatsia Livestock breeder Dairy Institute, Naivasha J. M. Muia Animal feed specialist Dairy Institute, Naivasha Stella Makokha Socio-economist Dairy Institute, Naivasha Mwaicimo Director, Animal Health KALRO Muguga John Mugambi PI, sub-project KALRO Muguga John Ndirangu PI, sub-project KALRO Muguga Julius Muturi PI, sub-project KALRO Muguga T. Riungu Director, Muguga South F. Muyekho PI, sub-project I. Kariuki PI, sub-project N. Kanegeni PI, sub-project J. G. Mureithi Acting Dep. Dir., Livestock KALRO Research, F. P Wandera EAAPP Desk Officer KALRO R. Oduor PI, sub-project, Kenyatta University Maina Gachara Veterinary Surgeon University of Nairiobi Ruth Wanyera National Wheat Research EAAPP/KALRO Coordinator Lucy Karani Research Officer KALRO Seed Unit Njoro John Kimani PI EAAPP project KALRO Mweya Agatha Wamuyu Thuo Executive Committee County Government of Member, Agriculture, Nyandarua Livestock and Fisheries Eluid Mutura Kinya + 9 Chairman and committee HILLTEN Dairy Self Help Group group members members – Kirimangai Location John Muigai Njihia Chief Kirimangai Location William Gitonga Asst. Chief Kirimangai Location Margaret Wanjeri Asst. Chief Kirimangai Location Kiarie Milkah Thiongo Manager Tulaga Farmers Cooperatice Chair and members Wheat seed producers Chebaibai group Chair and members Self-Help Group NGOPELWA Tanzania Hussein Mansoor National Coordinator and Ministry of Agriculture, Food Assistant Director Crop and Cooperatives 82 FINAL REPORT – EAAPP I Evaluation NRI AfrII

Country Person Position Organisation Research, (MAFC)/EAAPP Coordination Unit Ruth Kamala Deputy National Coordinator MAFC/EAAPP Nkori Kibanda RCoE Coordinator and ARI Katrin/EAAPP Director Deogratias Lwezaura M&E Specialist EAAPP MAFC/E Coordination Unit AAPP Zainab Semgalawe Co-TTL World Bank Fidelis Myaka Director of Research and Ministry of Agriculture, Food Development and Cooperatives January Mafuru Zonal Director Northern Zone Agricultural Research Institute Jerome Jonathan Agronomist & Assistant ARI Katrin Mghase Director, Sophia Kashenge- Rice breeder, ARI Katrin Killenga Shadrach Kihombo Training manager, ARI Katrin Fidelis Myaka Director of Research & MAFC Development, Ombaeuli Lemweli Assistant Director, Crop MAFC Monitoring and Early Warning Shakwaanande Natai Head, Environmental MAFC Management, Joyce Mvuna Assistant Director, Extension, MAFC Mshaghuley Ishika Principal Agricultural MAFC Research Officer John Banzi Principal Economist MAFC Angelo Mwilawa Research Scientist Tanzania Livestock Research Institute Benjamin Kiwovele Agricultural Zonal Research Uyole ARI Coordinator Peter Mlegulah Production Manager Raphael Group Limited Victor Iuvinga Deputy DAICO Njombe Town Council Hamis Mtwaenzi Acting Director TOSCI Wallace Karia District Executive Director Mvomero District Council Daina Muywanga District Agriculture, Irrigation Mvomero District Council and Cooperatives Officer Kenji Shiraishi Extension/Monitoring JICA TANRICE-2

Ethiopia Yitaye Alemayehu National Coordinator EAAPP Coordination Unit/ Ministry of Agriculture Alemayehu Assefa Research Coordinator Ethiopian Institute of Agricultural Research Terefe Fitta M&E Specialist EAAPP Coordination

Country Person Position Organisation Unit/Ministry of Agriculture Teshome Beyene Gender Specialist EAAPP Coordination Unit Assaye Legesse EAAPP TTL World Bank Kassahun Melesse Dairy focal person Deberezeit Mekonnen Mukuria Research Extension Deberezeit Specialist Tadesse Desalegn Director Kulumsa and Kulumsa Agricultural Research Technical Coordinator Centre Wheat RCoE Dawit Habte Soil & Water management Kulumsa Daniel Kassa Plant pathologist Kulumsa Tesfaye Solomon Socio-economist Kulumsa Kassahun Melesse Dairy focal person Deberezeit Bayuh Belay Centre Director and Fogera Rice Research and National Rice Research Training Centre Coordinator Worku Biureta Agricultural mechanisation Lawayew Ayal Ahmara Region EAAPP Assistant Coordinator Tadese Kedese Forage extension Agricultural Office Andasa Mamo Tessema Vice Head Agricultural Office Andasa Solomon Orion Head of Livestock Dept Bahir Dar Zuria Zelalem Tadesse Agronomist Fogera Rice Research Centre Tadesse Lakew Rice Breeder Fogera Rice Research Centre Yizegau Bayines + 12 New Condition/ Andasa group members Environment Milk Production & Marketing Association Kassahun Agumasae + Gombat Farmer Research Gombat Kebele 6 group members Group Getenet Adebabay + 5 Quahar Kebele Farmers’ group members Research group Welelaw Lakew + 19 Bura Research Extension Bura & Shina Kebele group memebrs Farmer Group Welelaw Lakew + 19 Bura Research Extension Bura & Shina Kebele group members Farmer Group Uganda George Lukwago National Coordinator EAAPP Coordination Unit Sheila Butungi EAAPP focal person, NAGRC Mityana Jolly Kabirizi Director, NaLLIRI Wakiso Chris Omongo Coordinator CRCoE Namulonge Arthur Wasukira Programme leader, Cereals Namulonge Jimmy Lamo EAAPP Focal person, Rice Namulonge Cedric Mutyaba Director, NARL Namulonge Joseph Oryokot Co-TTL World Bank Arthur Wasukira Programme leader, Cereals Namulonge Michael Waithaka Programme leader, PAAP ASARECA Vincent Akulumuka EAAPP ASARECA 84 FINAL REPORT – EAAPP I Evaluation NRI AfrII

Country Person Position Organisation George A. Maitek Director NgeZARDI Charles Aben PI – T&D NAADS Christopher Bukenya Manager Technical Services NAADS Chris Owiny Lamo District Production Officer Molly Egwang + 20 Chairperson Agency for Food Security members Network Hellen Mary Akiror Secretary AFAMCOS David Etilu Secretary Production AFAMCOS Joseph Egabu Treasurer AFAMCOS Charles Kiirya + 5 Chairperson Agateraine Farmers Group members Hellen Ameja + 9 Group members KAUSON Dairy members Gaston Ogwang + 10 Association members Dokolo Farmers Association - members rice Tom Olum + 6 Group members Obanga Beri - Rice Farmers members group