Evaluation of FAO's Activities on Capacity Development in Africa

PC 104/5 August 2010 E

Hundred and Fourth Session

Rome, 25 – 29 October 2010

EVALUATION OF FAO’S ACTIVITIES ON CAPACITY DEVELOPMENT IN AFRICA

1. This cover page provides the background and key issues for the attention and guidance of the Committee on the attached Evaluation of FAO’s Activities on Capacity Development in Africa and the Management Response. Background 2. The Paris Declaration and Accra Agenda for Action recognised that limited capacity is one of the major constraints to development in Africa and highlighted the need for development assistance to be better coordinated and led by Africans. Capacity Development (CD) is part of the core mandate of FAO, as explicitly recognised in the Strategic Framework 2010-2019. 3. At its October 2008 session, the FAO Programme Committee selected FAO‟s Capacity Development Activities in Africa as one of the priority areas for evaluation. The Evaluation was conducted by an independent external team between June and December 2009, using a mix of tools to draw evidence, including six country case studies. The Evaluation was expected to serve as a vehicle for accountability and learning, as well as to assist in validating and refining the FAO Corporate Strategy on Capacity Development. Key Issues in Evaluation Report  Overall, FAO needs to raise the profile and understanding of CD in the Organization. CD needs to be the focus of most, if not all, of FAO activities including assembly and provision of information, support to international instruments, norms and standards, policy advice, advocacy, the formation and dissemination of knowledge products, technology development and transfer, and networking.  FAO‟s CD performance in Africa has been mixed. Most interventions are relevant, many have been effective, but few have been sustainable. The Evaluation noted a number of successes, principally where FAO had engaged continuously over time and across all three dimensions (individual, organization and enabling environment), most obviously in plant protection, statistics and increasingly in transboundary animal diseases.  While FAO CD work only occasionally targeted organizations, much of the most effective work the Evaluation team members saw in the countries visited, involved just these cases of enhancing capacities of local, national and regional organizations, mainly in the public sector.  There is very little emphasis given to sustainability and too much given to immediate results and outputs. This is evident in the project timeframes and modalities; the lack of understanding by FAO staff of the importance of process in CD; the lack of focus on institutionalising CD activities and building political will to sustain them; and also to the limited motivation and opportunity for follow-up and for monitoring and evaluation by FAO staff responsible for the CD work.

This document is printed in limited numbers to minimize the environmental impact of FAO's processes and contribute to climate neutrality. Delegates and observers are kindly requested to bring their copies to meetings and to avoid asking for additional copies. Most FAO meeting documents are available on the Internet at www.fao.org K8635

2 PC 104/5

 FAO produces much valuable knowledge relevant to CD, but its uptake in Africa is limited, mainly because the region is constrained by poor communications infrastructure. Additional investment, and partnering, is required to ensure wider access to FAO‟s normative products.  The regional and subregional offices need their capacities strengthened, more resources and better links to expertise in headquarters in order to lead capacity development in Africa. Where additional resources are not available, it may be necessary for FAO to reduce the number of projects and programmes in selected areas. FAO should also work more in partnerships to share some of the costs and to increase its outreach. It should continue to move away from direct implementation to more facilitation and mentoring of local implementers. Management Response 4. FAO management appreciated the evaluation process and methodology and fully accepts six of the nine recommendations and partially accepts three. Many of the topics included in the recommendations have been reviewed by the Interdepartmental Working Group (IDWG) on CD led by the Office of Knowledge Exchange, Research and Extension (OEK), and many of them are included in the Corporate Strategy on CD and its associated Implementation Plan. 5. The recommendations will contribute to the ongoing organizational change process and will involve actions to: i. strengthen decentralized offices in Africa; ii. change CD programming and uptake of a series of new CD tools and practices; iii. change CD partnerships; iv. change post descriptions and include CD in performance evaluation of FAO staff and consultants; v. train staff on CD; and vi. improve distribution and uptake of products in Africa. 6. It is recognized that allocation or redirection of resources will be required to ensure implementation of all the recommendations, and that a phased approach will need to be adopted over the coming years. The potential resource implications will be quantified for possible consideration in the preparation of the Programme of Work and Budget 2012-13. 7. To ensure the change process needed for improving FAO‟s work on CD in Africa, and more widely, its compatibility with the ongoing Organizational change and available resources, Management will give priority to activities which can be addressed quickly, while allowing more time for the recommendations requiring further analysis or sequenced implementation. Management will integrate evaluation follow-up actions into the ongoing IPA change process where relevant, to ensure synergy with the organizational reform.The action table will be further refined as funding and priorities are defined over the coming years. Guidance sought 8. The Programme Committee may wish to provide its views and guidance on the key issues in the Evaluation report and the proposed follow-up actions by Management.

Queries on the substantive content of this document may be addressed to:

Mr. Robert Moore, Director, Office of Evaluation (Evaluation Report) Tel. (06) 5705-3903 Mr. Anton Mangstl, Director, Office of Knowledge Exchange, Research and Extension (Management Response) Tel. (06) 5705-3371

Evaluation of FAO’s Activities on Capacity Development in Africa

Final Report

March 2010

Table of Contents

Pages

I. Introduction 1

II. Capacity Development: What is it? 1

III. The Changing Environment 2

A. INTERNATIONAL COMMITMENT TO MORE COORDINATED AND DEMAND-DRIVEN DEVELOPMENT 2

B. AGRICULTURE AND CD ARE HIGH ON THE INTERNATIONAL AGENDA 3

C. ADVANCEMENTS IN COMMUNICATIONS 3

D. CHANGES AT COUNTRY LEVEL 3

IV. CD in the Context of FAO‟s New Strategic Framework and Reform 3

V. Evaluation Scope, Methodology, Focus and Limitations 5

A. SCOPE 5

B. METHODOLOGY 5

C. FOCUS AND LIMITATIONS 7

VI. Overview of FAO‟s CD Work in Africa 7

VII. Findings 11

A. RELEVANCE OF FAO‟S WORK 12 Priorities as defined in National Medium-Term Priority Frameworks 13 Country priorities for CD according to interlocutors 13 Coordination with other CD initiatives 14 Relevance of FAO’s work from the perspective of the three CD dimensions 14

B. DESIGN, APPROACH AND MODALITIES 17

Overall findings 18 Taking into account the context and identification of needs at the individual, organisation and institutional levels 18 Participation of beneficiaries/stakeholders in the design 19 Modalities of CD interventions 20

Strategic selection of participants, facilitators and implementing agents 20 Ensuring the institutionalisation of interventions 22 Project length, follow-up and exit strategy 22

C. EFFECTIVENESS 23 Overall findings 23 The policy/enabling environment 24 Organisations 25 Individuals 27

D. IMPACT 28 Overall findings 28

E. SUSTAINABILITY AND FOLLOW-UP OF CD ACTIVITIES 30 Overall findings 30 Longer time-frame, follow-up and continued engagement 30 Institutionalisation of results and processes 32 Developing capacity at the decentralized/local level 32 Encouraging networking 32 Facilitating government commitment 33

F. GENDER 33 Overall findings 33

G. PARTNERSHIPS 34 Overall findings 35 African inter-governmental organisations 35 UN agencies 36 National and regional research organisations and universities 37 Civil Society and the private sector 38

VIII. FAO‟s Normative CD Products 38

A. APPROACH 39

B. RELEVANCE TO THE NEEDS 39

C. DESIGN AND PRODUCTION QUALITY 40

D. GENDER 40

E. ACCESS AND FORMATS 41

F. CAPACITY BUILDING PORTAL 42

G. USE OF NORMATIVE CD PRODUCTS 42

IX. FAO‟s Ability to Deliver CD 43

A. OVERALL FINDINGS 43

B. UNDERSTANDING CD AND FAO‟S ROLE IN IT 43

C. INSTITUTIONAL AND ADMINISTRATIVE ARRANGEMENTS TO DELIVER CD 44

Regional and Sub-regional Offices 44 Country offices 46

D. PROJECT MODALITIES 46

X. Conclusions 47 General overview 47 Specific conclusions leading to recommendations 49

XI. Recommendations 53

A. MAINSTREAMING CD 53

B. SUSTAINABILITY AND EFFECTIVENESS OF FAO CD INITIATIVES 54

C. DEVELOPING CAPACITY FOR POLICY ANALYSIS, FORMULATION AND IMPLEMENTATION 55

D. DOCUMENTING AND PROMOTING SUCCESSFUL INNOVATIONS 55

E. ACCESS TO KNOWLEDGE AS A KEY COMPONENT OF CAPACITY 56

F. INCREASED INVESTMENT IN FAO CAPACITY IN AFRICA 56

List of Annexes

Annex 1: Terms of Reference Annex 2: Evaluation Methodology Annex 3: The Team Annex 4: The Expert Review Panel Annex 5: List of People Met Annex 6: List of Key Documents Consulted Annex 7: Overview of FAO‟s CD Activities in Africa Annex 8: Assessment of FAO‟s Normative Products Related to CD in Africa Annex 9: Meta-evaluation of Country, Programme and Project Evaluations Annex 10: Country Reports a. Kenya b. Malawi c. Tanzania d. Uganda e. Ghana f. Burkina Faso Annex 11: Beneficiary Assessment Reports a. Kenya b. Malawi c. Tanzania d. Uganda e. Ghana f. Burkina Faso

Composition of the Evaluation Team

Core Evaluation Team Prof. Kay Muir-Leresche, Team Leader Prof. Malcolm Blackie, Regional Expert South and East Africa Mr. André Damiba, Regional Expert West Africa Mr. Mark Stiles, Assessment of FAO‟s Normative Products Ms. Cristina Lopriore (resource person), Meta-synthesis of Evaluations

Country Consultants, Beneficiary Assessments Dr. Florence Birungi Kyazze - Uganda Ms. Aggripina Mosha - United Republic of Tanzania Dr. Esther Njuguna - Kenya Dr. Alexander Phiri - Malawi Dr. Daniel Sarpong - Ghana Mr. Bakiené Son - Burkina Faso

Evaluation Office Ms. Anna Guerraggio, Evaluation Officer Ms. Rachel Sauvinet-Bedouin, Senior Evaluation Manager

Acknowledgements

This evaluation has been a long and demanding process that would not have been possible without the support and information provided by many FAO staff at headquarters and in the field offices. The evaluation team members also benefited extensively from the time, perspectives and data which national government officials, development partners and people in communities shared with them.

The team would like to express its warmest gratitude to Ms. Nadine Monnichon, who provided highly valuable administrative assistance throughout the evaluation, and to Ms. Carlotta De Vivanco in the FAO Evaluation Service for her key contribution to the background research work.

Acronyms

ACP African, Caribbean and Pacific Group of States AGP FAO Plant Production and Protection Division AGPT Assemblée générale de projet de territoire AGRA Alliance for a Green Revolution in Africa ANAFE African Network for Agriculture, Agro-forestry and Natural Resources Education ANSA Food Security and Nutrition Association of Mozambique ARENET Agriculture Research Extension Network ASARECA Association for Strengthening Agricultural Research in Eastern and Central Africa CCA Common Country Assessment CAADP Comprehensive Africa Agriculture Development Programme CA-SARD Conservation Agriculture for Sustainable Agriculture and Rural Development CBO Community-based Organisation CCRF Code of Conduct for Responsible Fisheries CD Capacity Development CILSS Comité permanent inter-états de lutte contre la sécheresse dans le Sahel COMIFAC Central African Forest Commission CORAF Conseil Ouest et Centre africain pour la recherche et le développement agricoles COREP Comité régional des pêches du Golfe de Guinée CSO Civil Society Organisation DAC Development Assistance Committee EC European Commission ECDPM European Centre for Development Policy Management ECOWAS Economic Community of West African States EMPRES Emergency Prevention System for Transboundary Animal and Plant Pests and Diseases ESWD FAO Gender, Equity and Rural Employment Division FAO Food and Agriculture Organization of the United Nations FAOR FAO Representative FARA Forum for Agricultural Research in Africa FFS Farmers‟ Field School FNPP FAO/Netherlands Partnership Programme FPMIS Field Programme Management Information System FSA Fish Stocks Agreement FSIA Food Security Information for Action FSIS Food Security Information System GAINS Ghana Agricultural Information Network System GDP Gross Domestic Product GIPD Gestion intégrée de la production et des déprédateurs GIS Geographic Information System GMO Genetically Modified Organism GTZ Deutsche Gesellschaft für Technische Zusammenarbeit

HDI Human Development Index HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome HQ Headquarters HR Human Resources ICRAF World Agro-forestry Centre ICT Information Communication Technology IDP Internally Displaced People IDWG Inter-departmental Working Group IEE Independent External Evaluation ILO International Labour Organisation ILUA Integrated Land Use Assessment IPC Integrated Food Security Phase Classification IPPC International Plant Protection Convention IT Information Technology ITPGR International Treaty on Plant Genetic Resources JFFLS Junior Farmer Field and Life School JP Joint Programme KAINET Kenya Agricultural Information Network KCEF Outreach and Capacity Building Branch [FAO] KEPHIS Kenya Plant Health Inspection Services KIPRA Kenya Institute for Policy, Research and Analysis LinKS Gender Biodiversity and Local Knowledge Systems for Food Security LVFO Lake Victoria Fisheries Organisation MAAIF Ministry of Agriculture, Animal Industries and Fisheries MAC Market Access Company MDG Millennium Development Goal MoA Ministry of Agriculture NARO National Agricultural Research Organisation NBI Nile Basin Initiative NEPAD New Partnership for Africa‟s Development NFMA National Forest Monitoring and Assessment NGO Non-governmental Organisation NMTPF National Medium-term Priority Framework OECD Organisation for Economic Co-operation and Development OFZ Onchocerciasis-freed Zone Prods-PAIA Integrated Production Systems-Priority Area for Inter-disciplinary Action RAF Regional Office for Africa ROPPA Réseau des organisations paysannes et de producteurs agricoles de l’Afrique de l’Ouest RUFORUM Regional University Forum for Capacity Building in Agriculture SADC Southern African Development Community SEAGA Socio-economic and Gender Analysis Programme SFC Sub-regional Office for Central Africa SFE Sub-regional Office for Eastern Africa SFS Sub-regional Office for Southern Africa SFW Sub-regional Office for West Africa

SIFSIA Sudan Institutional Capacity Programme: Food Security Information for Action SRO Sub-regional Office SSA Sub-Saharan Africa SSC South-South Cooperation TA Technical Assistance TAD Transboundary Animal Disease TCE Emergency and Rehabilitation Division TCP Technical Cooperation Programme TO Technical Officer ToR Terms of Reference UEMOA Union économique et monétaire ouest-africaine UN United Nations UNAIDS The Joint United Nations Programme on HIV/AIDS UNCLOS United Nations Convention on the Law of the Sea UNDAF United Nations Development Assistance Framework UN DaO UN Delivery as One UNDP United Nations Development Programme UNFPA United Nations Population Fund UNIDO United Nations Industrial Development Organisation UNIFEM United Nations Development Fund for Women WFP World Food Programme

Executive Summary

Introduction i. The Paris Declaration and Accra Agenda for Action recognised that limited capacity is one of the major constraints to development in Africa and highlighted the need for development assistance to be better coordinated and led by Africans. This requires a change in approach to development and emphasises the importance of capacity development in particular. ii. At its October 2008 session, the FAO Programme Committee selected “FAO‟s work on Capacity Development in Africa” as one of the priority areas for evaluation. The evaluation was conducted from June to December 2009, using a mix of tools to draw evidence. It carried out an in-depth analysis of FAO‟s capacity development work in six case-study countries: Burkina Faso, Ghana, Kenya, Malawi, Tanzania and Uganda. It also completed an inventory of CD activities at country level, covering the 48 countries of Sub-Sahara Africa. It finally drew evidence from a review of selected FAO normative products, available records on projects and programmes, and a meta-synthesis of past evaluations, including country evaluations of FAO‟s work in Congo DRC, Sudan, Sierra Leone and Mozambique. iii. This evaluation serves as a vehicle for accountability and learning by providing an evidence-based analysis of the current status of FAO‟s work in the area of CD in Africa. The evaluation is forward-looking and formative and provides guidance and recommendations to improve the work of the Organization on capacity development.

Overview of FAO CD activities in Africa iv. FAO CD activities permeate nearly all the work FAO does. FAO is recognized as an important source of knowledge, and about half of the field projects (including within the Emergency Programme) FAO carried out in the period 2000-2008 included a significant component of CD. v. FAO‟s work has been more heavily focused on individuals (75%), and primarily on transferring technical skills. Some projects have rather specifically targeted organisational capacity (23%) or the enabling environment (27%). FAO is increasingly being called on to assist countries in setting the agenda for agriculture, fisheries and forestry and in liaising with donors. The Organization has significantly assisted governments in developing food policies, and aligning plant protection, food safety, transboundary animal diseases, fisheries and forestry regulations and policies with international norms and conventions. Only in very few cases, where FAO had a consistent and continuous engagement over time, did FAO intervention address all three CD dimensions.

Core Findings and Conclusions vi. Capacity development (CD) is the process whereby individuals, groups, private and public sector organisations all enhance their systems, resources, skills and knowledge to better address development.

vii. Capacity Development (CD) is part of the core mandate of FAO, as explicitly recognised in the Strategic Framework 2010-2019. In line with the Organization‟s mandate, FAO CD focus is on food security, rural poverty reduction and sustainable natural resource management. FAO has been active in CD across Departments and Divisions. Yet, interpretation of CD and recognition of its role varied throughout the Organization, with many equating CD with the one- off training of individuals. CD is a process and requires improving the functioning of the individuals and organisations. To be effective, CD also needs to address the enabling environment (policies, norms, values, legislation) to ensure there are incentives for improving capacity to address these issues and to be adaptable to changing circumstances. viii. FAO‟s CD performance in Africa has been mixed. Most interventions are relevant, many have been effective, but few have been sustainable. The Evaluation noted a number of successes, principally where FAO had engaged continuously over time and across all three dimensions, most obviously in plant protection, statistics and increasingly in transboundary animal diseases. This continuous engagement over a long period, across dimensions, allows for the building of a critical mass of skills, institutional memory and the policies, norms, values and structures to support the work in those areas. FAO has also achieved widely recognised success in integrating CD into pilot projects testing new technologies using effective CD approaches such as Farmer Field Schools. There have also been some good examples of policy assistance which has effectively and sustainably strengthened policy analysis and implementation capacity in Burkina Faso, Mozambique and Zanzibar among others. ix. Several factors contributed to the effectiveness:  adequate participatory planning, needs and context assessment;  appropriate consideration of the enabling environment, including institutional linkages and challenges;  long-term planning and involvement with appropriate follow-up;  the use of national consultants with strong FAO back-up; and  engagement across time with successive projects. x. However, despite many effective and relevant interventions, the Evaluation found that FAO CD activities are, for the most part, unsustainable. There is very little emphasis given to sustainability and too much given to immediate results and outputs. This is evident in the project timeframes and modalities; the lack of understanding by FAO staff of the importance of process to CD; lack of focus on institutionalising CD activities and building the political will to sustain them; and also to the limited motivation and opportunity for follow-up and for monitoring and evaluation by FAO staff.

Capacity Development Dimensions xi. While FAO CD work seldom targeted organisations, much of the most effective work the evaluation team members saw in the countries visited involved the enhanced capacities of local, national and regional organisations, mainly in the public sector. xii. With regard to CD targeting individuals, the country field visits and beneficiary assessments found strong appreciation of the FFS approach which is being taken up by donors, development agencies and governments but the evaluation found that the principles are sometimes poorly understood. The evaluation felt that FAO should make a more concerted effort

iii to document key principles of its successful approaches and make these available to a wide range of audiences. xiii. Stakeholders in several countries called on FAO to shift emphasis from policy development to policy implementation; to enhance capacities at the national and district levels and translate policies into action. In many cases, this requires better inter-departmental linkages, between national and local government and with other stakeholders. Some FAO projects specifically addressed these linkages but far greater emphasis needs to be given to them and to policy analysis and implementation. FAO can work with partners to help strengthen capacity on the frontline, particularly important in the increasing number of countries emphasising decentralisation. Approach to CD and soft skills’ development xiv. The country case studies and the meta-synthesis of evaluations reflected the demand for more CD of business, financial and marketing skills. They also underscore the importance of soft skills such as confidence, negotiating skills, teamwork, creativity, adaptability, leadership and trust. These are often best developed through using a participative approach to CD, emphasising process in delivering specific skills training. xv. FAO has provided little support to strengthening farmers‟ lobbying efforts for services so essential to building farmers‟ voice. In contrast, FAO‟s efforts to facilitate the participation of Africans at regional and international fora, workshops and other exchanges to strengthen their knowledge and build their confidence, has been important to developing soft skills and sustaining capacity. However, FAO interventions often fail to devote sufficient attention and resources to building the political will necessary to consolidate and sustain CD. This emphasises the importance of strengthening FAO country offices and devoting significantly more resources and attention to engaging with policy-makers. xvi. FAO produces much valuable and relevant knowledge as an important contribution to CD but its uptake and use in Africa is limited. Africa is constrained by poor communications infrastructure which means that additional investment, or partnering, is required to ensure wider access to FAO‟s normative products. This may also involve the more selective production of materials in order to ensure that resources are available for effective distribution. There is no point in producing materials which do not reach their intended audience. FAO capacities to address CD and Partnerships xvii. The Evaluation found that the Regional and Sub-regional Offices need their own capacities strengthened, more resources and better connectivity to expertise in headquarters in order to lead capacity development in Africa. At country level, FAO lacks the capacity to take its expected role in the many committees established to determine priorities, co-ordinate donor activity and facilitate interactions between government and donors and with other stakeholders. The Evaluation considers that FAO will lose both relevance and opportunities unless it is able to significantly strengthen effective capacity in Africa. xviii. In the spirit of the Accra Agenda for Action and to help address capacity constraints in implementing projects and in dealing with the emerging needs of decentralised systems, FAO should strengthen endogenous capacity, and be encouraged to partner more effectively. Setting priorities for CD within a strategic framework such as the NMTPF should take into account the context of the country, FAO‟s own comparative advantage to address national needs and the potential for partnering with local, regional and international agencies, including universities and research institutes.

iv xix. FAO has limited resources and needs to take the cost implications of investing in CD into consideration. Where the necessary resources cannot be made available for CD, the Evaluation considers that it may be necessary for FAO to reduce the number of projects and programmes to ensure that it has the human and financial resources to make sustainable CD interventions in selected areas. FAO can also work more in partnerships to share some of the costs involved in implementation and to increase its outreach. Recommendations xx. This Evaluation found that the more successful modalities for CD intervention were where the emphasis was on the way FAO engaged rather than on the production of outputs. The Evaluation has distilled the factors that have led to sustainable and effective CD and to the changing climate for development. It has highlighted the challenges confronting FAO as a result of increased in-country collaboration and decision-making by donors, as well as the implications of increased decentralisation and commercialisation. FAO needs to take these into consideration when making decisions on resource allocation, establishing priorities and strategies and motivating staff to ensure that it can be at the forefront of strengthening sustainable capacity to deliver on the MDGs. xxi. The new CD strategy recognises that FAO needs to work more systematically across the three dimensions of CD helping facilitate nationally-led processes and the endogenous strengthening of existing capacities in the countries where it works. This involves a major shift in FAO‟s approach to CD giving more emphasis to process, partnering and to the enabling environment. The Evaluation provides some recommendations to strengthen this change which are summarized below and detailed in the main report: Recommendation 1: It is recommended that the Interdepartmental Working Group (IDWG) on Capacity Building take steps to ensure that FAO staff and partners have a common conceptual understanding of CD and FAO‟s role in it. Recommendation 2: It is recommended that senior management, under the guidance of the IDWG, incorporate CD into the mandates, work programmes and post descriptions of all relevant programming staff. Recommendation 3: It is recommended that senior management, with the guidance of the IDWG, review and, where necessary, revise FAO‟s systems to improve the effectiveness and sustainability of CD initiatives. Recommendation 4: It is recommended that the IDWG develop, and senior management, implement, guidelines for projects and programmes that emphasise effective CD practices, such as participatory approaches that build ownership, sustainability and partnerships. Recommendation 5: It is recommended that FAO senior management ensure that staff, in particular those in decentralized offices and FAO Representatives, place increased emphasis on partnerships in their CD activities in Africa. Recommendation 6: It is recommended that FAO staff, in particular FAO Representatives, place increased emphasis on facilitating the development of national capacity for policy analysis and implementation Recommendation 7: It is recommended that FAO staff give priority to, and improve, the documentation and dissemination of successful CD initiatives, methods and normative products. Recommendation 8: It is recommended that FAO senior management take immediate steps to improve the distribution and uptake of FAO‟s products for CD in Africa.

Recommendation 9: It is recommended that FAO invest more heavily in the capacity of its decentralized offices in Africa to engage significantly in developing capacities for the agriculture, forestry and fisheries sectors and to respond to emerging demands of African member countries.

Introduction 1. Capacity development (CD) is at the core of the mandate of the Food and Agricultural Organization of the United Nations (FAO). Yet, no comprehensive assessment of FAO‟s work in this area has ever been undertaken. At its October 2008 session, the FAO Programme Committee selected the “FAO‟s work on Capacity Development in Africa” as one of the priority areas for evaluation. 2. The purpose of the evaluation is to serve as a vehicle for accountability and learning by providing an evidence-based analysis of the current status of FAO‟s work in the area of CD in Africa. The evaluation is forward-looking and formative and provides guidance and recommendations to improve the work of the Organization on CD in Africa, and perhaps other regions, in the future. 3. Furthermore, the evaluation is expected to assist in validating and refining the FAO Corporate Strategy on Capacity Building and defining the parameters for its implementation. 4. The evaluation is addressed to FAO member countries through the Programme Committee, Senior Management and FAO staff working on CD as well as to clients of FAO‟s capacity CD products and services, in particular in Africa. 5. The evaluation was carried out from June to December 2009 by a team led by Prof. Kay Muir-Leresche.

Capacity Development: What is it? 6. Reduced to its essentials, CD empowers people and organisations to be able to adapt to change and to have the confidence and skills to hold themselves and outside agencies accountable, and to improve their enabling environment. CD improves ability to:  decide, act, commit;  respond to stimuli (adapt and renew);  deliver and engage (produce outputs); and  develop, use, maintain and improve relationships, both internally and externally. 7. The Organisation for Economic Co-operation and Development/Development Assistance Committee‟s (OECD/DAC) Network on Governance and the European Centre for Development Policy Management (ECDPM) are among a number of agencies involved in researching what constitutes good practices in CD. Stiles (2009) reviews the literature and he notes that “there is no single best method of capacity development; much depends on the development circumstances and context. In most situations, a combination of methods will be needed to achieve desired results. Those engaged in strengthening capacities must also pay close attention to process: how the methods and tools are applied is equally as important as the methods and tools selected. The development community has learned that organisational capacity cannot be transferred in a linear process through technical assistance and training; it must be developed over time with a variety of interventions and with experimentation to find the best combinations of methods and tools.” 8. For this evaluation, as outlined in the terms of reference (ToR), the following definition of CD is used:

Box 1: Definition of CD CD is understood as the process whereby individuals, groups, private and public sector organisations all enhance their systems, resources and knowledge as reflected in their improved abilities to perform functions and solve problems, in order to better address hunger, poverty reduction and sustainable natural resource management objectives.

9. This definition makes explicit that CD is directed at empowering beneficiaries to address food security, improve livelihoods and ensure that systems of resource use are sustainable. It acknowledges that the evaluation will take into account all of FAO‟s CD initiatives and not only those directed at government agencies, and it highlights the fact that the focus of CD is to improve African rural development effectiveness. 10. CD engages all FAO departments, involves relationships, cuts across sectors and disciplines, and is a process within a system and not a discrete activity or set of initiatives. For CD to be effective, the process of engagement is as, or more, important than the outputs produced. 11. As indicated above, developing effective capacity is a process and it involves all three dimensions: individuals, organisations and the environment. The evaluation considers FAO‟s activities in relation to all three dimensions. 12. The enabling environment relates to political commitment and vision; norms, values and institutions; policy, legal and economic frameworks; institutional relations; sector strategies and overall resources; the general development level (health, literacy, etc), and financial resources. This evaluation focuses on interventions that are directed towards improving incentives, policies, legislation, regulations and accountability for sustainable development, food security and poverty reduction. FAO assists countries both to enhance the enabling environment for agriculture and to become more effective in formulating and implementing policies. 13. The organisation dimension relates to political and administrative bodies (government agencies, political parties, international and regional bodies), technical bodies (inspectorates, laboratories, research, extension), economic and social bodies (enterprises, commerce chambers, consumer groups, producer associations, CBOs, universities). Interventions designed for the organisation dimension are usually associated with improving the organisational system rather than individual competencies. This can involve the setting-up of veterinary laboratories, information systems, development of restructuring plans, etc. 14. The individual dimension relates to improving the ability of individuals of all ages and providing them with knowledge and a skill set (both technical and soft) that improves their ability to contribute and that enhances the effectiveness of any group or organisation they are associated with. 15. Each dimension relates closely to the other in an embedded system where the whole is the sum of the parts. It is possible to take one of the parts and change it but for success and sustainability, the changes need to be linked to, supported and reinforced through all dimensions. The emphasis is on building the capacity of the system at country level.

The Changing Environment 16. On field visits in Africa, the evaluation team witnessed new approaches to development assistance, opportunities created by new technology and the changing nature of structures, policies and capacities in the countries visited. These trends, which are briefly outlined below, are important to the context of CD and underpin many of the evaluation‟s recommendations. B. International Commitment to More Coordinated and Demand- driven Development 17. The Paris Declaration and the Accra Agenda for Action called on development partners to provide a more coordinated approach to assistance in close cooperation with governments and with specific efforts to enhance endogenous capacity. The evaluation found that many of the donors in the countries they visited are committed to this approach. Increasingly, programme priorities and funding arrangements are discussed at multi-donor and government fora within the country which requires strong country presence for effective participation.

C. Agriculture and CD are high on the International Agenda 18. Over the past several decades, development assistance in Africa has focused largely on non-agricultural areas. This now appears to be changing, as more development practitioners recognise the importance of agriculture, forestry and fisheries in relation to economic growth and the achievement of the Millennium Development Goals (MDGs) and climate change mitigation. As a result, agriculture and natural resource management are now given more prominence in development assistance.

D. Advancements in Communications 19. Technological advancements in communications provide new opportunities for accelerated development. The evaluation team noted that policy-makers, senior technocrats, scientists, research institutes and universities in Africa have access to the Internet. However, in many places, poor connectivity and the high cost of access restrict Internet use, making off-line access important. Some farmers‟ organisations use e-mail to communicate with representatives who access the Internet from cafés or government offices when travelling in remote parts of the country. Cellular telephones have made access to market information and financial transactions viable for many, especially in remote communities. Coverage is, however, limited and not all people can afford regular access. There is a real danger that the digital divide could increase inequities both within and between countries.

E. Changes at Country Level 20. The evaluation team noted increased emphasis on decentralisation in all countries visited, especially Uganda, Tanzania and Ghana. Capacity constraints at decentralised levels are affecting the ability to implement policies and programmes and to institutionalise innovations. Innovative mechanisms are required to enhance capacity at the frontlines to ensure that policies and strategies once developed at the centre are effectively implemented in rural areas. 21. The evaluation noted a strong emphasis on the commercialisation of agriculture and renewable resource production systems. National development strategies called for more emphasis on market links, inputs, processing, storage and finance. However, increased commercialisation could take the spotlight away from sustainable food systems, nutrition, poverty alleviation and services for marginalised groups. In addition, many of the countries emphasised irrigation as a way to reduce the effects of climate change, intensify production and ease land pressure. 22. Although capacity constraints in Africa are severe, the evaluation found advanced skills available in some of the countries visited. Endogenous capacity can be further enhanced by employing local and regional organisations as contractors. CD interventions should be differentiated to fit needs; some countries may require CD support at high levels while others may not. CD in the Context of FAO’s New Strategic Framework and Reform 23. The Director-General‟s Reform Proposals (2005) highlighted “Capacity Building” as one of the main areas of priority for member countries. The Independent External Evaluation (IEE) Report (2007) noted “Capacity Building” as a major theme for reform and advised the Organization to develop a corporate strategy to “take full account of countries‟ capacities, needs and partnerships opportunities” (Recommendation 3.24). FAO began work on a capacity development strategy in 2008 under the guidance of the then Outreach and Capacity Building Branch (KCEF) with inputs from an Inter-Departmental Working Group (IDWG).1

1 A review of FAO‟s draft CD strategy appears in Section IX.

24. The Organization has frequently stated the importance of CD work across its technical areas. It has made CD a priority for extra-budgetary resources, and has emphasised the need for staff to take maximum advantage of partnerships and networking opportunities in areas where FAO has the expertise and the comparative advantage. Yet, the Organization recognizes that a shift in the way FAO provides CD is needed - from the Organization being an implementer of supply-driven CD to the Organization being a facilitator of demand-driven CD. 25. In order to adapt to the changing architecture of development cooperation, FAO recognises that it needs to adjust the way it operates and has taken steps to do so. In response to the reforms proposed in the IEE Report (2007), the FAO Immediate Plan of Action for the biennium 2010-11 focuses on four key themes:  Managing for results through the definition of a results framework that comprises FAO‟s vision, three Global Goals and eleven Strategic Objectives, together with the related means of actions;  Functioning as One, including assigning a stronger role to Regional Offices in decision- making and investing in the capacities of decentralized offices to operate with the same corporate tools and facilities as headquarters (HQ);  Introducing a performance management system and reviewing human resources (HR) policies; and  Reforming the administration, which includes establishing a joint procurement unit for the three Rome-based agencies. 26. CD is one of the eight core functions in the new FAO Strategic Framework (2010-19), under the heading “Technical support to promote technology transfer and build capacity.” This covers both technical and functional capacities2 in all three CD dimensions, in order to facilitate the alignment of internal processes for the “new CD approaches [to be] fully institutionalised within existing systems and procedures.” 27. The New Strategic Framework mentions CD with reference mainly to the following three key areas:  Policy assistance work (especially on food security and nutrition), which includes not only CD for policy formulation, implementation and monitoring, but also institutional strengthening/restructuring and identification of Members‟ priorities for the development of programmes;  Trade-related CD, to assist countries in the definition of strategies and policies to facilitate the response of the private sector to new market demands and opportunities. In addition, the Framework underlines the importance of building the smallholder farmers‟ capacities along the value chain, to lift their productivity and move out of subsistence into market-oriented farming; and  Access to, and analysis and management of, data and statistics, which includes strengthening countries‟ capacities to collect, compile, analyse and disseminate relevant and timely data, and facilitating access to FAO‟s information products and services. Evaluation Scope, Methodology, Focus and Limitations

F. Scope 28. As described in the ToR (Annex 1), the evaluation focuses on Africa and more particularly on Sub-Saharan Africa, where the needs are considered the greatest and where many countries lag behind with respect to the attainment of the MDGs. Capacity-related constraints arising from a combination of individual, organisational and institutional/societal factors have

2 Technical capacities are defined as the capacities „to carry out the tasks required to intensify production in a sustainable manner, manage resources and eventually improve food security.” Functional capacities are rather linked to “the areas of policy, knowledge, partnering and implementation/delivery”.

5 often been identified as major hindrances to Africa‟s development. This is particularly true in agriculture, which accounts for a large share of the gross domestic product (GDP) at regional and country levels, and where the “huge paucity of trained personnel”3 and the weak state of organisations and institutions are key factors contributing to the sector‟s poor performance. The evaluation includes CD initiatives undertaken since 2000, in order to assess the long-term processes involved in producing results. 29. Given the breadth of CD as described earlier and the diversity of FAO‟s CD activities, the evaluation team defined the scope of the evaluation to include:  FAO initiatives at the country or regional level established specifically to enhance capacity as well as those that contribute to CD as part of other project or programme objectives. For these initiatives, CD is an explicit and substantial component of the product or the service. Global projects and international initiatives are also taken into account to the extent that they cover African countries in a significant manner; and  only those normative products which were adapted or developed at the country/regional level through a pedagogical approach. General knowledge-sharing events, such as conferences and seminars, and materials, such as technical papers and communication and information products (for example, data bases), were not assessed.

G. Methodology 30. The evaluation methodology is detailed in Annex 2. 31. The evaluation began with an inception phase from March to June 2009 with the objective of refining the scope of the evaluation, clarifying the analytical frameworks, developing the methodological tools and providing details on organisational aspects of the evaluation. 32. During the inception phase, the team prepared and used as a reference tool an evaluation matrix, which included questions to be addressed and indicators. The evaluation drew on various methodologies to assess the results and the processes followed in FAO‟s CD activities, including: logic models; participatory and systems approaches; and benchmarking against good practices and lessons. 33. The evaluation used a mix of tools, including:  an inventory of CD activities at country level, covering the 48 countries of Sub-Sahara Africa. The evaluation team developed the inventory starting with a review of the countries‟ portfolios using FAO‟s Field Programme Management Information System (FPMIS). These reviews were complemented by requests to divisions, interviews with staff and the country case studies. The team used the inventory to prepare the overview which is included in Annex 7;  meta-synthesis of evaluations, the objective of which was to review, aggregate and synthesize the findings and recommendations on CD of 33 selected country project and programme evaluations carried out by the FAO Evaluation Service since 2003. The full report is included as Annex 9;  country case studies and complementary field visits: The country case studies and field visits constitute the backbone of the evaluation. The objective of the case studies was to provide a comprehensive assessment of the relevance and effectiveness of CD activities in six countries in Africa. The evaluation team selected the countries on the basis of well- defined criteria outlined in the Inception Report. The countries were: Burkina Faso and Ghana in West Africa; and Uganda, Kenya, Malawi and Tanzania in East and Southern Africa. The country case studies were carried out in two phases. In the first phase, local experts with support from the core team completed the field work. Their main task was to collect information on a selected number of CD activities in the field through

3 Independent External Evaluation of FAO, 2007, page 166.

participatory workshops, focus discussions, surveys and interviews with the beneficiaries of CD activities and other stakeholders. The second phase of the country case studies was part of the core team‟s field visits of about a week to each of the six selected countries. By interviewing a much wider group of stakeholders, the team supplemented the beneficiary analyses. The team consulted partnering agencies, existing and potential partners, as well as bilateral and multilateral agencies. Team members prepared comprehensive country reports (Annex 10) to serve as the basis for general findings. A list of all persons met is provided in Annex 5. The beneficiary assessments are provided in Annex 11;  additional field visits: In addition to the six countries visited and with a view to assessing FAO‟s institutional arrangements and capacity to provide technical support to countries, the core team visited FAO‟s Regional Office for Africa (RAF) in Ghana, Sub-regional Office for Eastern Africa (SFE) in Ethiopia, Sub-regional Office for West Africa (SFW) in Ghana, Sub-regional Office for Central Africa (SFC) in Gabon and Sub-regional Office for Southern Africa (SFS) in Zimbabwe; and  assessment of normative CD products: A purposive sample of thirty-one normative products including guidelines, manuals, compact discs (CDs) and e-learning tools, was assessed with regard to the quality and relevance of their content and, where appropriate, for their effectiveness in the field, including their accessibility to the targeted audience, their relevance to the objectives and the country contexts, their ability to be interactive and their potential to remain relevant over time or to adapt to change. The full report appears in Annex 8. 34. In total, the evaluators interviewed 410 people: 89 FAO staff at HQ, 40 at the Regional and Sub-regional Offices, 27 staff in African countries and 254 stakeholders/partners involved in FAO CD activities.4 35. Two initiatives facilitated the analysis of findings and report writing:  team discussions held after the field visits had ended in November 2009, during which findings from the different evaluation tools were triangulated and refined; and  presentation and discussion of preliminary results to HQ and regional staff. 36. The evaluators balanced the opinions expressed in interviews with evidence gathered in documents and used triangulation to validate their major findings. In making judgements, team members also drew from their personal experience, as well as from established good practices in capacity development effectiveness.

37. Finally, in February 2010, a Peer Review Panel5 provided feedback on the technical soundness of the conclusions and recommendations of the evaluation.

H. Focus and Limitations 38. The evaluation dealt with the following broad questions as outlined in the ToR:  Is FAO‟s approach towards CD in line with emerging good practices? Does it promote greater ownership of partner countries?  Do the CD initiatives respond to articulated or implicit demand and in what way do they contribute to poverty reduction?  What are the strengths and weaknesses of FAO‟s work on CD?  Has the Organization focused its work where it has a comparative advantage in CD support?

4 The complete list appears in Annex 5. 5 Members of the Panel are presented together with their report in Annex 4.

 Has it worked effectively with national institutions and in partnership with other agencies?  What should the role(s) of FAO on CD be in the future?  How does the FAO corporate strategy on capacity building reflect the above concerns? 39. With regard to the latter, the final document was not available to the team at the time of writing this report. However, the team reviewed draft versions of the strategy and, while not providing an in-depth analysis, the team referred to the strategy in the Section IX of the report, dealing with FAO‟s concept and approach to CD, as well as in the Sections X and XI on conclusions and recommendations. 40. There are many limitations to the evaluation; the most challenging relates to the representativeness of the information collected against an area of work that cuts across nearly all FAO‟s programmes. In this respect, the evaluation made general conclusions, drew from multiple lines of evidence, used many examples to support its findings and employed a wide variety of evaluation tools.

Overview of FAO’s CD Work in Africa 41. This section provides an overview of CD found in the documents pertaining to all FAO field projects (national, regional and international, excluding TeleFood) implemented in Sub- Saharan Africa (SSA) since 20046. The overview covers some 3,000 projects, excluding those with a budget of less than US$1 million between 2004 and 2009. It also excludes normative CD activities usually carried out by HQ but which are not reflected in projects in Africa. The overview assesses the extent to which CD activities deal with each of the three dimensions of CD (individual, organisation, and the enabling environment), their geographical coverage and how they relate to a number of indices. The projects deal with the technical and functional capacities that FAO has developed through software (for example, technical assistance, training, workshops, knowledge and information sharing), hardware (for example, provision of agricultural inputs, infrastructure and equipment, including IT facilities and laboratories) and, to a much lesser extent, financial assistance.7 The following is a summary of the full report which appears in Annex 7. 42. The analysis found that about half8 of FAO‟s in-country projects had a strong CD component:9  of these, three quarters focused on developing the capacities of individuals10 through technical training, workshops, study tours and demonstration sessions;  twenty-three percent included the enhancement of organisational capacities11 through restructuring, establishment of units and committees and the development/strengthening of information management systems;  twenty-seven percent dealt with the enabling environment by either closely working with national governments on defining policies and programmes or by creating or

6 The Overview draws its information from FPMIS with the focus on field projects. Therefore, it does not include CD work carried out at headquarters and/or not reflected in projects. 7 The evaluation considered hardware and financial assistance support as an integral part of CD activities only when provided in conjunction with software components. 8 In Mozambique, Zimbabwe, Sierra Leone, Gambia, Senegal, South Africa and Malawi, some 60% of projects had a strong CD focus. 9The evaluation identified CD content in relation to each project‟s immediate objectives, activities and target dimension(s). Members of the evaluation team assessed the degree of CD focus on a scale of 1 (minimum) to 5 (maximum). Those with a level greater than 3 were deemed to have a “strong CD component.” 10 Primarily the capacities of government officials or farmers, including Farmers‟ Field Schools (FFS). 11 The capacities of government departments and agencies and, to a lesser extent, farmers‟ associations and small businesses.

strengthening institutional linkages among stakeholders, including decentralized government units, non-government organisations (NGOs) and the private sector. This share would have been significantly higher had the evaluators included support provided through regional projects, such as those in which FAO plays a key role with respect to the dissemination/implementation of international norms and standards and where it acts as an honest broker in the governance of transboundary issues and the management of natural resources; and  at the national level, most CD interventions targeted only one dimension, with thirty percent targeting two dimensions and only four percent targeting all three dimensions. The analysis is focussed on individual, national projects separately whereas an analysis of projects over time is more likely to reflect activities across the dimensions.12

12 This is reflected later in the report where findings show that over time FAO intervened across all three dimensions for many of its most effective and sustainable interventions: Kephis, Nile Basin Initiative, forestry, fisheries and in statistics.

Figure 1

FAO CD Dimensions (national projects only)

80% 70% 60% 50% 40% 75% 30% 20% 23% 27% 10% 0% Individuals Organizations Enabling Environment CD Dimension

43. Amongst national projects, 47 percent included some CD targeting government departments and agencies13 and 42 percent targeted farmers and community-based organisations (CBOs), in particular through the establishment of FFS. Only 11 percent of FAO interventions included strengthening the private sector and these primarily provided training and facilitation of institutional linkages to small business enterprises and agri-business processing units. 44. FAO‟s CD activities cut across all of the Organization‟s technical areas. As Figure 2 indicates, CD efforts at the national level are concentrated in plant production and protection, land and water management, animal health and production, food security, information management systems and statistics. Policy assistance, including that pertaining to food security, forestry, fisheries and other technical areas, represents 12 percent of all FAO‟s CD work in countries. The CD component of emergency and rehabilitation interventions is also significant14. At 1 percent, “markets and trade” is the area where FAO has put the least effort into developing capacities. Relatively few country forestry projects have focused on CD15, this may be explained by the fact that much of FAO‟s forestry work is done through global support programmes and on a regional basis. Rural infrastructure and agro-industries are concentrated in just a few countries such as Ghana, South Africa and Zambia. Few projects dealt specifically with gender and HIV/Aids.

13 Including regional organisations, laboratories and national research centres. 14 This includes many FFSs for internally displaced people (IDPs) in seed distribution, resettlement and livelihoods programmes, as well as initiatives to improve the enabling environment, including FAO‟s leadership role in agricultural sector recovery and in co-ordinating government and development partners efforts. 15 There are some CD activities in the work relating to the national forest monitoring and assessments in Angola, Cameroon, Comoros, Congo, the Gambia, Mozambique and Tanzania and the integrated land use assessment in Kenya and Zambia.

Figure2

FAO CD Activities per technical area (national projects only)

14%

13%

12%

11%

10%

8% Policy Assistance Plant Production & Protection

7% Food Security

6% Natural Resources Management Animal Animal Production & Health

5% Emergency & Rehabilitation

3% Statistics & Information Systems

Nutrition and Consumer Protection Forestry

2% Fisheries & Aquaculture

Agro-Industries Rural Infrastructures &

1% Trade & Markets Gender/HIV 0% Technical areas

45. When the national data are aggregated, the findings across sub-regions are similar, except in Southern Africa where FAO‟s CD work is focussed more on the public sector (57%) than in West Africa (38%). The share of CD interventions on forest management in Central Africa is in line with the SSA regional average of 3 percent16 which is low given the extent of forest resources. Only 3 percent of the projects on statistics and information management were found in West Africa, well below the SSA average. This small share included activities in Niger, Burkina Faso, Gambia and Liberia, with no projects in a number of West African countries17. 46. As shown in Table 1 below, FAO CD activities are proportionately higher in East and West Africa regions where under-nourishment and the human development index are high.

16 This figure does not include two regional CD projects, one with the Central African Forest Commission (COMIFAC) and the other aimed at enhancing food security through training on non-forest wood products. 17 This does not include regional and inter-regional projects such as the EC/FAO Food Information for Action project and Early Warning Information System project (GTFS/INT/928/ITA) which covers Benin, Burkina Faso, Guinea-Bissau, Côte d‟Ivoire, Mali, Niger, Senegal and Togo.

Table 1: FAO CD Focus by Sub-region and Main Indicators of Needs

Under-nourishment Agriculture CD focus in national HDI (2009) Sub-region (% of population (% of GDP projects 2002-04) 2006) (% of total CD in Africa) Central 18% 148.6 36.2 21.9

East 30.7% 161.6 42.3 31.1

South 22.9% 146.8 31.9 21.6

West 28.4% 165.1 24.9 37.0

47. Table 1 masks FAO‟s under-investment in CD in some Central African countries, since most of its CD activities in that region are concentrated in the Democratic Republic of Congo. The Overview (Annex 7) shows relatively fewer CD projects for some countries with high levels of undernourishment in West and Central Africa including Chad, Togo, Central African Republic, Rwanda and Guinea-Bissau. 48. The Overview also highlights inequities in the distribution of CD activity across “conflict and post-conflict” countries. The Democratic Republic of Congo, Sudan, Somalia and Mozambique have a relatively high concentration of CD projects, whereas others in similar situations, such as Angola, Côte d‟Ivoire, Chad and Rwanda, have far less.

Findings 49. The findings presented below take into account the evaluation questions outlined in the methodology section using the following criteria: relevance, design, effectiveness, sustainability, impact, gender, partnerships and FAO‟s own capacity to deliver CD. The findings from the evaluation of normative products are presented separately thereafter. 50. The meta-evaluation (Annex 9) synthesized the strengths and weaknesses in CD in the past evaluations. A summary of findings is illustrated in Figure 3 which shows:  good/satisfactory scores in three areas: quality/relevance of types/formats of CD materials, facilitation of technical/policy initiatives and the provision of technical assistance and training;  mixed scores in the following areas: awareness/use/utility of normative products, partnerships for developing and enhancing the capacities of individuals and organisations to deliver CD and gender and other cross-cutting issues; and  unsatisfactory or poor scores in relation to the provision of effective follow-up, support to ownership of CD interventions and monitoring and evaluation frameworks.

Figure 3: FAO’s CD Performance (as evidenced in evaluation reports)

Relevance to needs

Building on FAO's comparative advantage

Relevance Coordination with other CD initiatives

Needs at indiv/org/EE levels considered

Responsive to national demand & needs

Participation in CD design

Promoting ownership

Objectives clearly defined in desired outcomes

Major goals reflected (FS, poverty, NRM)

Cost-effectiveness & sustainability considered Design of CDDesignintervention of Gender & other crosscutting issues

Monitoring & Evaluation frameworks Good Capacity for CD delivery via FAO techn. units Good-Satisf. Interdepartmental coordination at HQ Satisfactory Mixed Efficiency Coordination of project management HQ vs. Field Unsatisfactory Relevance/quality CD materials Unsatisf-Poor Poor Normative products: awareness, use & utility Not available Facilitating technical/policy initiatives

Technical assistance & training

Supporting CD (ownership) Effectiveness

Partnering on CD activities

Promoting better CD technologies

Impact of CD activities at ind/org/EE levels Impact Enhanced client capacities

Enhanced local leadership

Up-scaling of CD activities by trained people

Social capital created Sustainability Changes in policies & program implementation

Adaptation & dissemination of CD approaches

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

51. The country case studies showed similar results. Many interventions in all three dimensions of CD were well implemented with good short-term outcomes, but very few had sustained their momentum long enough to show lasting results in the form of strengthened enabling environments, more effective organisational behaviour and the application of individual skills over time. I. Relevance of FAO’s Work 52. The analysis of CD activities presented in the overview18 shows that FAO has intervened to varying degrees in all sectors relevant to its mandate. The country case-studies indicated that FAO is, for the most part, aligned with country and community needs. The meta-evaluation

18 See Annex 7.

13 confirmed that about 70 percent of FAO‟s projects reflected country priorities, despite a dearth of proper needs assessments. 53. The vast majority of interventions at field level, including activities carried out in technical areas and in relation to FAO‟s role as an “honest broker”, appear to have capitalized on the Organization‟s comparative advantages. There were, however, instances in which stakeholders blamed FAO for having taken a “recipe” approach, with a standard package of planned and implemented CD interventions and little attention given to local needs or contexts, thus reducing the potential for ownership and sustainability.19 Priorities as defined in National Medium-Term Priority Frameworks 54. The evaluation found that National Medium-Term Priority Frameworks (NMTPFs) were generally aligned with country national priorities as expressed in national development plans and United Nations Development Assistance Frameworks (UNDAFs). In most case-study countries, NMTPFs were in place or planned and appeared to have been driven by the national agricultural framework policies and formulated with the participation of the government. FAO staff, and FAO Representatives in particular, acknowledged the importance of NMTPFs for planning CD interventions in accordance with national priorities. 55. However, none of the documents reviewed incorporated an analysis of CD needs or set clear priorities to pursue in relation to CD. Instead, most established general areas of intervention, leaving the evaluators to conclude that FAO appears to respond more to short-term, ad hoc country requests than to long-term, mutually established CD goals and priorities. Interviews with government officials and the country evaluations confirmed that FAO has yet to use the NMTPFs as a strategic approach to its interventions. The evaluation suggests that this may not occur until FAO in country offices are strengthened so that they are able to be proactive in pointing out to governments unidentified needs and emerging issues. 56. Notwithstanding the importance of setting priorities for CD, the evaluation acknowledges that FAO needs to be flexible and ready to respond to urgent needs, such as those related to the control of diseases and pests. Despite funding constraints, the evaluation team found evidence that many of FAO‟s CD interventions were timely and relevant in supporting CD related to contingent needs, as in the Technical Cooperation Programme (TCP) project on banana wilt disease in Uganda. Country priorities for CD according to interlocutors 57. Evidence from the beneficiary assessments and the country visits confirmed that FAO‟s CD interventions have been relevant to perceived needs. For the most part, FAO has responded well to country needs as expressed in various national policy documents analysed in the country reports (Annex 10). Interlocutors identified many capacity gaps where they considered FAO well suited to provide more assistance. These included the prevention and control of Transboundary Animal Diseases (TADs), irrigation and water management, data management and agriculture statistics, and the commercialisation of agricultural products addressing the full value chain.

58. Stakeholders also identified climate change, HIV/Aids and gender as important areas where FAO should be more involved. The evaluation team suggests that FAO ought to assess its potential role and comparative advantage in developing capacities in these cross-cutting areas and, when implementing projects, consider partnering with organisations that specialize in them.20 59. The evaluation team‟s interactions at the sub-regional offices brought to light another issue worthy of FAO‟s attention - large land leases to foreigners. Government officials need to

19 Projects which aim to introduce Food Security Information Systems, in countries where the promotion of HQ-led analytical tools was perceived by country partners as too supply-driven, constitute a good example in this sense. 20 For more details, see Section VII.G on Partnerships.

14 ensure that compensatory mechanisms, for surrounding communities in particular, are included in the land deals and that sound environmental practices are respected. Some require CD assistance in formulating new policies. 60. Many interlocutors, including the private sector and CBOs, said that it was important for FAO to support the development of soft skills relevant to individual and organisational effectiveness. The country visits and the meta-synthesis noted many requests for training in such areas as ICT, participatory research, policy analysis and strategic planning, budgeting, marketing and management along with technical skills. Communication and writing skills were also frequently identified, along with soft skills related to leadership, creativity, negotiation, advocacy and motivation. In fact, many CD interventions fail because they have not taken into account soft skills. The process and modality of engagement can help to develop these skills, as will partnering to incorporate specialist input where required. Coordination with other CD initiatives 61. The meta-evaluation found that FAO coordinated its CD activities with other CD initiatives of national governments and/or development partners in only half the cases reviewed. Similar findings were reflected in the country case studies, confirming that FAO sometimes collaborates with governments and partners in other CD efforts, but inconsistently. The evaluation team found some notable successes, particularly in the area of food security information. For example, the final evaluation of the EC/FAO Food Security Information for Action (FSIA) Programme highlights links to food security information systems, and thereby to local capacities, resources and action plans, thus potentially enhancing the sustainability of the programme‟s outcomes. The complementary CD initiatives of Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ) conducted through FAO‟s food security policy support in Mozambique provide an example of successful CD coordination. Relevance of FAO’s work from the perspective of the three CD dimensions a) The enabling environment dimension 62. With the increased emphasis on country-led decisions on development needs, many interlocutors perceive FAO as particularly well placed to facilitate decision-making groups related to agriculture, forestry and fisheries. In some of the countries visited, the evaluation team noted that FAO is under-utilizing what is acknowledged as its comparative advantage as honest broker, facilitator and expert adviser to the national working groups, donors and UN clusters on development. One exception was Zimbabwe, where the team found FAO successfully playing this role in FAO‟s Emergency Programme, raising the profile of, and respect for, FAO amongst a wide range of stakeholders. 63. From evidence gathered, FAO has been instrumental in responding to country requests for policy assistance, particularly with respect to integrating food security issues into national agricultural and development policies. Examples include:  Burkina Faso, where FAO‟s CD activities under PCA Norway21 are aimed at improving capacities and providing technical assistance in the analysis of poverty and food security impacts and formulation of agricultural and rural development policies;  Kenya, where FAO worked with the government to develop the National Food and Nutrition Policy; and  Tanzania, where FAO has been active in supporting the National Irrigation Policy and Transitional Strategy and in assisting with CD for the National Strategy for Growth and Reduction of Poverty. 64. The relevance of FAO‟s assistance on policies and regulatory frameworks on plant protection and food safety, essentially the International Plant Protection Convention (IPPC) and

21 PCA Norway Strategy A1 Objective 1: Inclusion of Food Security Objectives, Policies, Programmes and Monitoring Mechanisms in PRSPS and Other Policy (FNOP/INT/109/NOR).

Codex, emerged quite prominently in the course of the interviews with government representatives. FAO support in these areas was much needed and appreciated, having contributed to increasing the countries‟ ability to export products, protect agriculture and adapt to the increasing sophistication of urban consumer demands. The evaluation found evidence of relevant work in many countries in FAO support to policies and regulatory frameworks for fisheries and forestry. In particular, FAO‟s contribution to the creation22 of national fora for policy dialogue within the framework of the National Forest Programme Facility and the National Forest Monitoring and Assessment (NFMA) and Integrated Land Use Assessment (ILUA) processes were particularly appreciated in that they involved numerous actors, including Civil Society Organisations (CSOs) working on forest management, some at sub-national levels. The evaluation found that while FAO contributed directly to policy development in the past, more recently its work has focussed on facilitation, coordination and the setting of norms, including through the FAO Emergency Programme. This work has also helped develop relevant soft skills and assisted countries to graduate from humanitarian aid to development assistance. 65. However, weak capacity on the part of governments to implement policies once they are developed severely hampers FAO from achieving sustainable results in the policy/enabling environment dimension. Stakeholders in several countries called on FAO to shift emphasis from policy development to policy implementation, claiming that the Organization has not done enough to enhance capacities at the national level to translate policies into action. However, this would require more inter-departmental linkages and assistance to governments to engage with other actors. FAO‟s track record here has been mixed, with positive examples from its work on statistics in Burkina Faso and its support to the Food Security Steering Groups in Kenya, and negative examples from its forest decentralization initiatives in Burkina Faso and its work on statistics in Malawi. 66. Responding to the demand for CD interventions on policy implementation requires development actors to strengthen weak capacity at local and provincial levels, as most countries decentralise government services. An evaluation of FAO‟s cooperation in the Democratic Republic of Congo reinforced this point, as follows: The ongoing decentralization process at the provincial level sets a challenge and, at the same time, represents an opportunity for FAO. There is a great need to assist governance and provincial public institutions. Support to capacity building provided by FAO, which concentrated on central services, would have gained efficiency by conforming to the decentralization process with stronger synergy. In that sense, the extreme weakness of provincial services of agriculture, forestry and fishery constitutes an area where FAO will have an important role to play.23 67. A number of interlocutors thought that FAO could do more to assist in developing capacity at local levels by leveraging frontline partnerships such as those it developed when implementing FFSs. The evaluation met with a number of producer organisations that expressed interest in bridging the capacity gap between central and local levels, thus enabling FAO to remain relevant as decentralisation proceeds. 68. FAO has been active in developing platforms for information management in some African countries. Stakeholders have appreciated these efforts, particularly the facilitation of exchanges between agencies and the improved flow of information resulting from them. The evaluation team was impressed with the Ghana Agricultural Information Network System (GAINS) and the Kenya Agricultural Information Network (KAINET) which stakeholders widely acclaimed for linking research, extension, farmers and implementing agencies and for facilitating

22 In Ghana, Burkina Faso and other countries. 23 Evaluation de la coopération de la FAO en République démocratique du Congo (2003-07), 2008, page 7.

16 effective information exchange and partnering.24 They also praised the Rural Knowledge Networks in East Africa for their innovation in linking farmers, small traders and service providers to market information. Interlocutors found such initiatives highly relevant and they would like to scale them up. b) The organisational dimension 69. FAO has assisted governments to strengthen their organisational capacities. It has been particularly active in all the case study countries with respect to government institutions supporting plant protection and transboundary animal diseases (through the set up and strengthening of laboratories as well), food safety, statistics and information management. 70. Much of FAO‟s work along this dimension has concentrated on data and information management across a wide range of technical areas, from food security to animal health, fisheries, agriculture statistics, and so on, all of which developing partners and donors recognise as important. Although FAO‟s work in data collection and analysis responds to crucial information needs, most interlocutors told the evaluators that FAO needed to be more involved in facilitating communication among ministries and departments involved and in establishing effective information-sharing mechanisms across sectors. 71. FAO has also embarked upon a few relevant and highly appreciated interventions on organizational restructuring. For example, in Kenya, FAO assisted the government in establishing departments for agribusiness and agricultural policy to better serve commercial interests and marginalised communities. 72. At the regional level, FAO has played a limited, but highly relevant, role in developing the capacity of technical organisations such as the Comité régional des pêches du Golfe de Guinée (COREP), Central African Forest Commission (COMIFAC), International Institute for Water and Environmental Engineering, Lake Victoria Fisheries Organisation, and the Nile River Basin Initiative, which are critically important to Africa‟s development. The much needed support to the establishment and development of the Emergency Prevention System for Transboundary Animal and Plant Pests and Diseases (EMPRES) in the Western Region is a good example of FAO‟s relevance in the organizational dimension of CD. 73. The evaluators encountered wide-spread appreciation for FAO efforts in Burkina Faso, Niger and Senegal to strengthen the capacity of producer organisations to engage in policy deliberations. Similar types of interventions were deemed extremely relevant in other countries the evaluation team visited, among them Ghana, Uganda, Kenya and Tanzania, where there are strong producer and distributor organisations. Some stakeholders also suggested that FAO should help FFS groups have more voice and be better networked and linked to markets, services and the policy and advocacy initiatives of established producer organisations. c) The individual dimension 74. The evaluation team found it more difficult to assess the relevance of FAO‟s work aimed at developing the capacities of individuals due to the large number of interlocutors and project beneficiaries and to the largely anecdotal nature of the available information. However, the team found sufficient information in the beneficiary assessments and the country visits to conclude that FAO‟s contribution to the enhancement of technical knowledge and competences of individuals was, for the most part, relevant and valuable. The evaluation considers that it is important for FAO to continue engaging in these field activities, both to provide practical experience to its normative work and, more importantly, to develop the capacity of countries to test the appropriateness of new approaches and technologies to meet their needs.

24 In Uganda, a similar experience with the setting up of ARENET was at the end less positive, not in terms of links created and appreciated by stakeholders, but due to flaws in project design relating to communications and the website.

75. However, as pointed out earlier, while the development of technical competencies appears to have relevance, FAO has paid little attention to developing the “soft skills” that are likely to significantly influence performance in the organisational and policy/enabling environment dimensions. To highlight one example: fish farmers in Ghana, while showing sincere appreciation for the training they received on tilapia aquaculture techniques, nevertheless expressed concern that they lacked the skills and organisational capabilities to conduct their work in this field in a business-like manner.

J. Design, Approach and Modalities 76. The Evaluation assessed the design, approach and modalities of CD activities against internationally recognised good practices as defined in the Evaluation Inception Report and outlined in Box 2.

Box 2: Good Practices in the Design of CD Interventions25  Understanding context: The initial context analysis and problem definition, identifying sources of change, delivery, learning and sharing of experiences, is critical to effective CD;  Needs assessment: CD activities should be based on comprehensive capacity needs assessment, including not only individual CD needs but also organisational and institutional analysis that considers both formal and informal aspects and incentive structures related to the context in which the CD efforts are focused.  Local partners: In the implementation phase, CD providers should give priority to local/regional suppliers of CD services, build endogenous capacity for CD, and ensure that such suppliers have sufficient time to learn and understand the context so that the approach and instruments used are appropriate.  Advocacy and convening role: It is important to contribute to CD by playing a neutral convening role, facilitating partnerships between governments, NGOs and donors and advocating for all stakeholders to fulfil CD commitments.  CD in high-level planning processes: Advocating for CD within CCA and UNDAF processes and identifying and exploiting organisational reform and strategy revision opportunities are considered good practices that ensure training and human capacity formation are fully integrated as specific high-level development objectives.  Flexible approaches should be applied: Modalities of CD interventions should be combined and adapted to the context in the design and implementation of CD interventions.  CD in fragile states: CD for government should be maintained as an important objective in fragile states, balanced with CD of non-state actors in ways that reinforce the development of public sector capacity in the longer term while avoiding a “centralizing effect.”  Monitoring and evaluation of CD: It is important to examine results as well as impact levels – measuring benefits for individuals, the Oganization and final beneficiaries.  Partnerships: Contractual arrangements are as conducive to CD as collaborative and collegiate partnerships where reciprocal learning occurs, decision-making authority is shared or handed over and there are higher levels of partner ownership. Software  Of the various training modalities, peer learning/training is highlighted as good practice; peers are well trusted (this is key for skills transfer) and have come from relevant institutional settings.  Knowledge and information-sharing are key to promoting broad-based CD, particularly when technology broadens and improves access to public goods. Project staff should play a facilitating role, avoiding substitution (filling in for gaps in capacity) and resisting the temptation to assume leadership of the CD initiative. Hardware  Investment in IT in particular is one of the most effective ways of boosting capacity. Ample examples exist to demonstrate that new technologies can be lower in cost than old technologies.

25 These practices are based on a literature review carried out for the evaluation.

Overall findings 77. FAO‟s CD efforts have been more successful when projects have been designed with a good understanding of the local physical and social conditions of the project, and a clear understanding of existing capacities and capacity gaps. 78. The evaluation confirmed the great importance of the process by which projects are implemented26. Promising cases were found where projects were designed with an emphasis on process as much as on output. Emphasis on process enables the development of soft skills to accompany the transfer of technical skills.27 79. The evaluation found that the design of FAO‟s CD initiatives was generally poor and lacking a clear assessment of needs. All design-related issues except one (responsiveness to stakeholders‟ demand) scored unsatisfactory or poor in 50-70 percent of the projects scrutinized in the meta-synthesis. Areas of particular weakness included: design of appropriate monitoring and evaluation frameworks; translation of objectives into clear desired outcomes; cost-effectiveness and sustainability considerations; and the extent to which CD initiatives took into account the needs of all three dimensions of CD. Evidence from the case study countries was mixed, with a number of the beneficiary assessments praising FAO for a participatory approach to design but failing to pay adequate attention to the development context. Taking into account the context and identification of needs at the individual, organisation and institutional levels 80. The meta-synthesis found that FAO failed to systematically undertake proper needs assessments before planning CD interventions. As was clearly pointed out in the Mid-term Evaluation of the Sudan Institutional Capacity Programme: Food Security Information for Action (SIFSIA)28 in 2009, “Future training activities should be preceded by a clearer assessment of individual as well as institutional and organisational capacity building needs, and a strategy developed for addressing them all”. 81. When the design of a CD intervention does not begin with a proper assessment of the needs, time, physical constraints and institutional framework, there is less likelihood of success, as the Beneficiary Assessment of the Acacia Operation project in Kenya demonstrates. There, the outputs were of poor quality and outcomes were marginal, largely because the design failed to adequately consider the above-mentioned factors and because of limited participation. Implementers needed to pay much more attention to the context. For example, the project failed to consider the weak capacity of the beneficiaries, the difficulties inherent in developing the capacities of migrant pastoralists, as well as the challenges of implementing and monitoring field activities involving nomadic peoples in remote areas. The Beneficiary Assessment also found that project implementers needed to deal more effectively with certification and marketing issues. 82. Poor needs identification across the three dimensions of CD was one of the weakest design factors that emerged from the meta-evaluation. A number of project evaluations noted that weak needs assessments contributed to missed opportunities for more strategic targeting of CD, ineffective project implementation and lack of sustainable results. One-off training, targeting

26 Process was specifically recognised as important by counterparts in the Obsolete Stockpiles programme in the Tanzania beneficiary assessment because it provided them with exposure to a range of skills and knowledge and helped raise local leadership awareness of the issues. 27 Process was specifically recognised as important by counterparts in the Obsolete Stockpiles programme in the Tanzania beneficiary assessment because it provided them with exposure to a range of skills and knowledge and helped raise local leadership awareness of the issues. 28 Mid-term Evaluation of the Sudan Institutional Capacity Programme: Food Security Information for Action (SIFSIA) South Programme, 2009, page 28.

19 government officials was found to be of limited impact, especially in a complex arena such as international trade, even when FAO included the application of knowledge to policy planning and implementation in the training package, as it did in various programmes in Kenya, Malawi and Tanzania. 83. The country case studies also demonstrated poor results when CD activities were implemented as a “standard package” with insufficient attention paid to the context and to the three dimensions of CD. The introduction of CountryStat in Malawi and of IPC tools in Kenya are examples of projects where FAO should have given more consideration to institutional linkages in the design and planning stage, particularly with respect to government organisations, notably the relationship between the Ministry of Agriculture (MoA) and the National Institute of Statistics, and between central and decentralized levels. Participation of beneficiaries/stakeholders in the design 84. OECD/DAC and the UN community recognise that the participation of beneficiaries in the design of projects can help build ownership, encourage better understanding of the context and ensure that resources are deployed to their best advantage. The complex Onchocerciasis-freed Zone (OFZ) project29 in Ghana and Burkina Faso (which required an interdisciplinary approach, the development of soft skills for cross-border planning and the involvement of a range of diverse agencies), the joint UN initiative in Northwestern Tanzania30 and the Nerica rice project31 in Uganda demonstrate why strong participation in the design and implementation of CD projects is important to their success. 85. Beneficiaries in the Nerica rice project were involved in setting the project goals and in implementation. As a result, they were able to clearly recall the objectives and CD activities when questioned by the evaluation team. The participants in these FFSs continue to recruit and train new members. The project also engaged extension workers and government officials as implementing partners and thus built capacity to scale out to new areas. Representatives from the Ministry of Agriculture, Animal Industries and Fisheries (MAAIF) expressed the need for more initiatives that link them with farmers and with new technology. 32 86. In the case of the project, Assistance préparatoire au recensement général de l'agriculture et de l'élevage,33 the full involvement of national counterparts in the design helped redirect the focus of the intervention towards CD areas and to modalities more appropriate to local needs. 87. Beneficiaries not being adequately consulted in the design and planning phase has led to disagreements and undermined the sustainability of results. For example, in the Projet d’intensification agricole par la maîtrise de l’eau dans les Régions du Centre-Sud et du Centre- Ouest34, the training was not carried out as planned because staff, who were not involved in the design and planning of the project, had different assignments. Said one trainer, “We learn just like that, that the selected trainees had to leave for an entire month.”35 Similarly, a project36 designed

29 Socio-economic Development Programme for the Transborder Onchocerciasis-freed Zone of Burkina Faso and Ghana (GCP/RAF/376/BEL) 30 Strengthening Human Security through Sustainable Human Development in Northwestern Tanzania (UNTS/URT/002/UNJ) 31 Dissemination of NERICA and Improved Rice Production Systems to Reduce Poverty and Food Deficit in Uganda (GCP/UGA/035/JPN) 32 Extract from the Beneficiary Assessment of the GCP/UGA/035/JPN project. 33 TCP/BKF/3102. 34 GCP/BKF/049/SPA. 35 Extract from the Beneficiary Assessment of the project.

20 to improve food security in cross-border districts of Burundi, Rwanda and Uganda in support of the modernization of agriculture under the New Partnership for Africa‟s Development (NEPAD) framework, featured inadequate consultation with local authorities and agriculture staff in its design. As a result, implementing staff had to negotiate at length with farmers on the use of an outside agency (instead of a local cooperative) to manage the store, and many participants complained that the project wasted resources on a potato warehouse that was much smaller than required. The beneficiary assessment of the Livestock/Wildlife integration project in Tanzania37 featured similar evidence. Offices provided under the project were unused and field activities had limited impact, in part as a result of inadequate participation during the project design process. 88. With respect to the design of policy support activities, it appears that beneficiaries have seldom been included in the design, even in cases where participatory implementation has subsequently been adopted. Too often, policy assistance has emphasized outputs rather than the processes required to ensure effective ownership and outcomes. Modalities of CD interventions 89. FAO used a range of modalities including workshops, peer training, short courses with local, regional or international trainers and a range of practical exposure ranging from international engagements to Farmers‟ Field Schools with either or both farmer facilitators and extension workers. Overall, hands-on and practical exposure were recognised as the most effective mode for developing skills, as in the initiative to transfer practical skills to orphans and disadvantaged groups in Malawi.38 90. With regard to CD targeting farmers, the country field visits and beneficiary assessments found strong appreciation of the FFS approach and farmer-to-farmer experience sharing as the best method to develop the capacities of farmers, provided they received sufficient backstopping and support from implementing agencies and extension agents. The evaluation team found that facilitation and participative approaches enhancing individuals‟ soft skills were more effective than sole formal training with technical assistance. This highlights the importance of process over output. The project “Strengthening the Organisational Capacity of Fish Farmer Groups in Ghana”39 is one among many that demonstrated the importance of practical training for the immediate application of knowledge and skills, as well as the frequent need to accompany such training with initiatives to strengthen management and organisational capacities. Without the latter, the beneficiaries, in the case of the above Fish Farmer Group project, were unable to scale up and move beyond subsistence farming. Strategic selection of participants, facilitators and implementing agents 91. CD interventions need to carefully identify beneficiaries (individuals and/or organisations), facilitators and implementing agents. In order to achieve CD objectives, programmes and projects must involve the right people at the right time, in the right combination. Selection strategies may vary depending on the CD objectives. 92. Many of the projects reviewed for the evaluation used inclusive criteria to select participants, while others paid insufficient attention to gender when selecting participants. For example, a training programme for the small-scale dairy sector in Ghana40 selected participants

36 Improvement of food security in cross-border districts of Burundi, Rwanda and Uganda, in support of the modernization of agriculture under NEPAD framework (GTFS/RAF/391/ITA). 37 Novel forms of livestock and wildlife integration adjacent to protected areas in Africa (GCP/URT/124/UBG). 38 Protecting and Improving Food and Nutrition Security of Orphans and HIV/AIDS Affected Children (GCP/RAF/388/GER). 39 TCP/GHA/2904. 40 Training Programme for the Small-scale Dairy Sector (TCP/GHA/0167).

21 who were too far from markets to benefit from the training. Implementers should have selected closer participants or provided additional support to the dairy farmers from regions far from markets. The Conservation Agriculture for Sustainable Agriculture for Rural Development initiative in Kenya underscores the importance of selecting relevant trainers and participants: “The beneficiaries considered that it would have been more sustainable if the facilitators trained were local farmers [trained extension agents had been transferred] and if local garage owners had also received training in tool fabrication services rather than only Ministry of Agriculture officials.”41 93. The beneficiary assessments and the meta-evaluation found farmers and local facilitators to be the most successful in carrying out CD in field projects in many instances. In some cases, the use of farmers as facilitators was considered more effective than the use of extension agents. In others, the integration of extension agents was found to be important to sustainability as in Uganda in Nerica Rice project42 where locally recruited extension agents were available. 94. Back-up support from either an extension service or implementing agency is often needed towards the end of a project in order to ensure that capacities once developed are institutionalised. There is a need to ground the projects in the agencies that will be in a position to provide post- project support. This was a shortcoming of the Prods-PAIA conservation agriculture initiative in Burkina Faso, despite its significant success in a range of technical innovations including introducing a technology brought from pastoralists in the North and the use of the same pastoralists to engage with farmers in the South. 95. The selection of facilitators and trainers for policy interventions did not emerge as an issue, although the evaluation team noted that using staff from the organisation/government agency as facilitators created space for individuals to apply their new skills. The use of senior staff also creates greater institutional will for the changes. As stated in the evaluation of the EC/FAO Food Security Information for Action Programme: “Better results were obtained in countries where the national institutions concerned with food security were strongly committed. This indicates that when higher-level department representatives engage in the programme, there is greater potential for enhanced performance and an increased level of benefits through training and capacity building activities.”43 This was particularly evident when national facilitator/trainers are included as counterparts to employed outsiders. 96. The evaluation found a clear demand for the use of locals. They also noted that in many countries, there is useful local capacity that needs the chance to gain experience and so foster endogenous capacity. FAO is doing this in many instances and has had some good experiences. As the country case studies in Appendix 10 show, in some cases there was not sufficient mentoring by FAO staff and implementers were not given access to FAO normative products. 97. In quite a few cases, particularly in Uganda and Kenya, stakeholders criticised FAO for having followed a standardised approach to the selection of implementers and facilitators, thus not always making the best choice or creating the best fit with local needs. Projects appear to not always have been designed to develop the capacity of the local CD implementers. There were a number of exceptions, including the CD support FAO provided to Nutrition projects in Malawi using Bunda College and to the Food Security and Nutrition Association (ANSA) of Mozambique as attested in the recent evaluation of the EC/FAO Joint Evaluation: Food Security Information for Action Programme.

41 Extract from the Beneficiary Assessment of the project “Supporting Conservation Agriculture for Sustainable Agriculture and Rural Development (CA for SARD)” (GCP/RAF/413/GER). 42 Agriculture and Rural Development through Innovative Rice-based Farming Systems for Food Security and Poverty Reduction in Uganda (GCP/UGA/035/JPN). 43 EC/FAO Joint Evaluation: Food Security Information for Action Programme, 2009, page 32.

98. The use of South-South Cooperation (SSC) as a mechanism for implementing CD was not much in evidence. The evaluation found that SSC was highly appreciated on paper and probably effective when trainers had the right technical skills, as well as cultural acclimatisation and ongoing mentoring. Yet, in the few instances where the evaluation team came across SSC, the results were disappointing. It appeared that language and cultural barriers often hampered effective engagement. For example, in Ghana, only one of the original team of Chinese trainers stayed on, was reasonably effective and remained to the end of contract. In Central Africa, the SSC also experienced poor results. Ensuring the institutionalisation of interventions 99. The support to national and local institutions that provide a legal framework or that have a normative function is of paramount importance to the effectiveness and sustainability of CD initiatives. But strengthening appropriate institutions is not easy. The evaluation found some examples of effective institutionalisation, such as the Conservation Agriculture for Sustainable Agriculture and Rural Development (CA-SARD) initiative in Tanzania, the OFZ project in Ghana and Burkina Faso and the EMPRES programme. These projects were designed to ensure that the agencies that would need to continue to provide services at the end of the project were involved in project design and closely consulted throughout the implementation process. The institutions themselves were also strengthened as part of the project. The importance of linking projects into systems that provide ongoing support has been clearly highlighted in the country reports (Annex 10). 100. The process of selecting the right organisations to strengthen can be problematic. The evaluation team found instances where FAO should have taken more care to assess institutional relationships when selecting regional and national organisations to strengthen. Such was the case in CountryStat in Malawi (and, to a lesser extent, a number of other countries), where the Ministry of Agriculture and other government institutions often had little incentive to cooperate. In the project on the promotion of organic Karité Butter44, the Ministry of Trade in Burkina Faso, which is in charge of promoting exports, was not involved in project planning or linked in to the project afterwards. This sullied relationships and jeopardized the sustainability of the initiative. The evaluation team noted that FAO has not always recognised the factors that help or hinder successful relationships among institutions, including those that affect relatively simple tasks, such as the sharing of information. Project length, follow-up and exit strategy 101. Effective CD interventions often take a great deal of time, particularly when dealing with organisational and institutional capacities and factors in the enabling environment that support or impede performance. 102. The evaluation team found many examples, in particular TCPs, where timelines were too short to achieve sustainable capacity. In many cases, there were some good results but there was too little time to achieve all the objectives and some components had to be abandoned. In others, there was not sufficient interaction to consolidate gains.45 For example, the forest decentralization project in Burkina Faso46, the Programme pour le développement de la pêche and the OFZ project ended before results were fully realized.

44 Increasing incomes of small farmers through exports of organic and fair trade tropical products (GCP/RAF/404/GER). 45 Appui à la préparation d‟un plan de réforme institutionnelle et juridique pour la décentralisation dans le secteur forestier (TCP/BKF/2904). 46 Appui à la préparation d‟un plan de réforme institutionnelle et juridique pour la décentralisation dans le secteur forestier (TCP/BKF/2904).

103. A widely expressed frustration is captured in the following statement voiced to the evaluation team by one project participant: “Mwatisiya m’malele mwana akungobadwa.” (“You have left us hanging immediately after the birth of the child.”)47. This suggests that where FAO cannot extend the length of its involvement, it must work with others who can continue “to nurse the child.” 104. The evaluation team found that project designs seldom included strategies for follow-up and exit, which further compromised the achievement of sustainable results. As noted in the evaluation of the EC/FAO Food Security Information for Action, FAO should “develop sound hand-over strategies from the beginning to ensure the sustainability of programme activities, including periodic and continuous assessment of progress made in various capacity development activities”48.

K. Effectiveness Overall findings 105. In assessing effectiveness, the evaluation attempted to identify what capacities have been strengthened and where. The evaluation found some successful projects with solid CD activities across FAO‟s broad mandate, and in all three CD dimensions. In particular, the evaluation team was positively impressed by the results achieved when developing the capacities of:  farmers, including youth and pastoral people within the framework of Farmers‟ Field Schools, with respect to production techniques and good practices;  government officials, on the understanding and application of international conventions; and  government and government-affiliated organisations involved with plant protection and food safety, statistics and data management. 106. Many factors contributed to the effectiveness of CD interventions, including relevance, timing and especially, institutional anchorage, which FAO accomplished through:  adequate planning and needs assessment;  appropriate consideration of the enabling environment, including institutional linkages and challenges;  long-term planning and involvement, to the extent possible, with appropriate follow-up (either through re-training or advice); and  use of national consultants with strong FAO back-up. 107. The following sections highlight examples of achievements and issues according to the dimension they most concentrated on. The effectiveness of CD activities within each dimension is explored below. This should not detract from one of the evaluation‟s main findings, which is that FAO CD efforts have been most effective when the Organization has intervened over time, through a series of projects, across all three CD dimensions. FAO has been effective in all three dimensions in different situations. It has focussed more on the individual but has been effective, in some instances, in both developing capacity for policy formulation and in improving the effectiveness of organisations. There were also too many instances where FAO did not take into account other CD dimensions and this affected the results of CD interventions. The policy/enabling environment 108. FAO has engaged widely with governments in developing policies across most areas of its mandate and its contribution to policy formulation and adoption has been widely appreciated. While many projects have produced useful policy documents, less evident to the evaluation team was the development of local capacity for policy analysis, formulation and implementation. In its

47 Beneficiary Assessment, Malawi. 48 EC/FAO Joint Evaluation: Food Security Information for Action Programme, 2009, page 7.

24 policy assistance work, there has been a tendency for FAO to focus on outputs rather than processes that ensure local ownership and sustainable capacity development. 109. The evaluation team determined that CD is often not considered an integral part of FAO policy assistance. For example, CD objectives are frequently absent from policy assistance plans and from the terms of reference of policy experts. This issue came to light in interviews with FAO staff in Malawi and Ethiopia. The evaluation team did, however, find examples in Zanzibar and Mozambique where close cooperation with counterparts resulted in the effective transfer of policy development skills to local officials.

Box 3: Influencing Policy Processes The FNPP-supported engagement to integrate food security concepts into national agricultural policies is an excellent example of what FAO can achieve in CD. In Mozambique and Zanzibar, the project carefully assessed the context, problems and opportunities. Policy assistance did not rely exclusively on short-term visits by experts but rather involved local counterparts, local consultants and ongoing support from FAO experts over time. The project emphasized developing the capabilities of those involved in the process and measured results in broader terms than policy outputs. The project recognized the importance of building political will, taking into account a variety of human factors, the need for ownership and the necessity of developing the soft skills of staff engaged in CD. The project established trust and developed stable relationships through a participative approach targeting key stakeholders, including those responsible for implementation in the districts. To the extent possible, the project built in follow-up assistance. Lessons have been documented in several publications and shared in print and electronically on FAO‟s EasyPol website.

110. FAO has, in a few cases, played an important role in developing enabling environments by fostering coordination among government ministries and departments in defining cross- disciplinary policies and programmes, various national stakeholders including NGOs working in a defined sector and governments involved in the management of transboundary resources. 111. The evaluation team found good examples in this respect, including the following:  in Kenya, where FAO established working groups representing a broad range of stakeholders to work on developing a Food Security Policy;  in Uganda, where FAO provided leadership for the National Food Security cluster;  In Ghana, where the Organization helped establish the National Forum for Forest Resources Management, as well as broad-based multi-stakeholder steering committees in the ten administrative regions of the country;  in Tanzania, where FAO‟s Joint Programme on wealth creation, employment and economic empowerment has strengthened dialogue on agribusiness between the Ministry of Agriculture and the Ministry of Industry and Trade; and  in regional initiatives, where FAO facilitated dialogue and the formulation of policies on the prevention and control of transboundary animal diseases (TADs) and on water management, as in the Nile basin, Lake Victoria Lower Kagera River and elsewhere. 112. Yet, the evaluation found many examples where FAO missed opportunities to play a coordinating and facilitating role, owing mainly to a lack of human resources in-country and, in some cases, weak project design and planning. 113. FAO has been particularly effective in developing the capacities of government officials to understand and apply international regulatory frameworks,49 such as the International Plant Protection Convention (IPPC), the International Treaty on Plant Genetic Resources (ITPGR), Codex Alimentarius, and international conventions pertaining to forestry and fisheries sectors. Most of the concerned interlocutors met in the country visits expressed great appreciation of

49 Most of FAO programmes relating to these international frameworks include capacity development activities, generally on two aspects: (i) increasing members‟ effective participation to regional and international debates and negotiations and (ii) strengthening countries‟ capacity to comply with and implement these regulatory frameworks.

FAO‟s work in this area. In particular, many claimed that FAO‟s support for their participation in international fora was useful in building their confidence to negotiate and increasing their attentiveness to international issues of relevance to their countries. They also indicated that international exposure and regional exchanges helped them legitimise international instruments and provided fertile grounds for advocacy and government support. 114. Box 4 provides a clear example of how activities designed to affect the enabling environment led to successful CD achievements.

Box 4: The CD Potential of Regulatory Frameworks - Example from the Code of Conduct for Responsible Fisheries “The information obtained in the fisheries focus area indicates a strong and growing awareness at national level of the value and usefulness of international soft law. On the one hand, the international nature of the soft-law development process has allowed countries collectively to achieve results which each would otherwise have struggled with individually (such as practical implementation of the fisheries elements of UNCLOS [UN Convention on the Law of the Sea] or commitments of the FSA [Fish Stocks Agreement]). Their experience in developing and using international soft law has helped national fisheries officials get accustomed to a single body of terminology and concepts. In essence, no matter what language or dialect they speak, they all speak “Fisheries.” Finally, soft law enables the development of more soft law. For example, the problems of artisanal fisheries are the same all over the world. Through the linkages developed in negotiating and using the Code of Conduct for Responsible Fisheries and other soft international instruments, many countries are able to establish partnerships to find means to address this common concern.” 50

Organisations 115. While most FAO CD work did not target organisations, much of the most effective work the evaluation team members saw in the countries visited involved the enhanced capacities of local, national and regional organisations, mainly in the public sector. This was done through organisational restructuring, the introduction of data management systems, the provision of equipment and tools (for example, in national laboratories51), training and other interventions. 116. Some of the most successful examples the evaluation team encountered included: a) Government organisations  Assistance on restructuring the Kenya Ministry of Agriculture and establishing the Agribusiness and Policy Units. This was done in cooperation with the Kenya Institute for Policy, Research and Analysis (KIPRA), thus contributing to enhancing local capacities as well as ensuring greater relevance and likely sustainability.  Introduction of new tools and techniques for statistical data gathering and analysis in Burkina Faso, Niger, Uganda and Tanzania, especially as part of the assistance to the organisation of the national census. In Uganda, this also successfully combined with the introduction of CountryStat.  Development of the Kenya Plant Health Inspection Services (KEPHIS) as a centre of excellence. KEPHIS has now become an active and self-sustaining organisation, servicing the private and public sectors and the region (see Box 5).  Support to the Nile Basin Initiative, which involves the governments of nine riparian countries and a wide range of international donors. Support included the development of information products that integrate technical water resources and water use data with other demographic, socio-economic and environmental data, which were assembled using geographic information system (GIS) technology.

50 Extract from: Evaluation of FAO Corporate Strategic Objective B1, page 24. 51 The evaluation is aware of many examples of successful CD interventions in the field of animal health, including the setting up of national laboratories for the prevention and control of TADs in Sebeta (Ethiopia). For more details, please refer to the II Real Time Evaluation of FAO‟s Work on HPAI.

 Long-term support and advice to the Lake Victoria Fisheries Organisation on drafting a Regional Plan of Action and on the harmonization of data collection tools; and  Long-standing assistance to the Comité permanent inter-états de lutte contre la sécheresse dans le Sahel (CILSS) in the field of food security and early warning information systems for which the organisation is now a reference in the region.

Box 5: Organisational CD: The Example of KEPHIS FAO has supported KEPHIS consistently over time and continues to maintain close links; KEPHIS has become a centre of excellence in the region. Through FAO‟s assistance in training trainers and in developing manuals, the Institute now has a critical mass of staff, maintains strong ties with ministries and other stakeholders, and works on plant safety nationally, regionally and internationally. With FAO assistance, KEPHIS is now able to monitor pesticide residues and set standards. Through an FAO „focal point‟, KEPHIS scientists have been able to source direct assistance and advice. FAO played an important role in assisting KEPHIS with the revision of legislation and mentoring the development of new legislation. FAO support has also been important in helping KEPHIS obtain funding for research and in providing national guidance to the debate on genetically modified organisms (GMOs). FAO provided training to four government institutions involved in GMO regulation and the monitoring of biosafety measures and regulations. It also helped equip a laboratory to test for the presence of GMOs in imported commodities. b) Non-governmental and private sector organisations  Enhancement of the production and organisational management capacities of the Tulaga dairy cooperative in Kenya52 as well as their capacities in governance and social cohesion, through the development and use of an open-source licensed Business Management Software for Producer Organisations. This resulted in much more efficient payment systems and higher milk yields and incomes as well as increased membership;  Strengthening the capacities of the Réseau des organisations paysannes et de producteurs agricoles de l’Afrique de l’Ouest (ROPPA) in Burkina Faso, providing training on a number of agriculture-related issues, supporting the set-up of new producer and service provider unions, and enhancing the links among member organisations and their partners;  Other smaller interventions in support of national/regional farmers‟ organisations and cooperatives as in Ghana, Kenya and Uganda, as well as the Eastern Africa Farmers‟ Federation. FAO successfully contributed to the development of management and advocacy skills, and facilitated attendance of farmers at international fora; and  Support to a few private agribusiness units, for instance, Weija Agriculture Development Ltd in Ghana and to farmers‟ groups, such as the Productrices de beurre de karité au Burkina Faso. FAO enhanced their marketing and trade capacities, including elements of organisational management. 117. In general, the evaluation found that FAO is not providing much support to national producer and distribution organisations. FAO could engage more proactively with these organisations to implement projects. This would create multiplier effects at the community level. The evaluation team visited a number of dynamic producer organisations53 with strong links throughout the countryside that advocate farmer-friendly policies and services. Some appear to be well-placed to assist in scaling up successful pilot projects, although most still require additional CD and backstopping during the implementation process to further develop their capacity. The FAO country offices did not appear to be well linked to organisations outside of government.

52 Computerisation of Agricultural Cooperatives and Emerging Producer Associations in Kenya (TCP/KEN/2907). For more details, see Annex 10 - Kenya Beneficiary Assessment and sourceforge.net/projects/coopworks. 53 For example, the Farmer and Fisherman‟s Award Winners Association in Ghana, Uganda National Farmers Federation, Kenya National Federation of Agricultural Producers and the Federation of Associations of Ghanaian Exporters.

They have limited capacity to engage even at government level which is FAO‟s primary entry point. Individuals 118. As highlighted in the Overview, a large majority of FAO CD projects aim at developing the capacities of individuals, mainly government staff and/or farmers and farmers‟ groups. The evaluation team found most technical training designed to transfer technologies and promote good agricultural practices to be relevant and effectively delivered. 119. The evaluation found the FFS approach to have been the most effective in transferring technologies and establishing groups. The FFS model has been widely recognised as a good practice and it has been applied in many countries. 120. Increasingly, the FFS approach is applied successfully in rehabilitation programmes, working with young farmers through the Junior Farmers‟ Field and Life School, as in Malawi, Uganda, Mozambique, Sudan School, and more recently with agro-pastoral communities in the semi-arid, northern regions of Kenya and Ethiopia. In Uganda, FFSs were also engaged in response to an outbreak of plant disease (see Box 6), providing a rapid and effective response to a contingent problem.

Box 6: CD in Response to Emergencies Banana wilt emerged in the 1990s as a devastating disease of the staple food of many Ugandans and one for which there was no immediate solution. A dedicated team of national specialists at the National Agricultural Research Organisation (NARO), in collaboration with FAO and international scientists, developed a practical and effective control programme. The challenge then was to train extension agents and farmers in this new technology. FAO and the NARO scientists used FFS to disseminate the information and to promote the adoption of the new control methods. Today, banana wilt has been reduced to a problem of manageable dimensions and the new control methods are widely understood, adopted and practised. This is a testament to successful CD and partnering.

121. Stakeholders widely appreciated FAO‟s initiatives on conservation agriculture using FFS in Tanzania, Kenya and Zimbabwe. A project in Burkina Faso, which included such innovative practices as live fencing, indigenous legumes and silage preparation, generated additional revenues. Twelve regional chambers of agriculture and members of the Confédération paysanne du Faso are now upscaling the project together with other national and international partners, including research institutes such as the World Agro-forestry Centre (ICRAF), CORAF, the Alliance for a Green Revolution in Africa (AGRA) and the Forum for Agricultural Research in Africa (FARA) among others. 122. The evaluators took note of several initiatives focussed on agro-processing and marketing which aimed to link farmers and producers to the markets by looking more comprehensively at the value chain. Among those highlighted in the country case studies are:  the sericulture and silk processing development project in Ghana54 which provided new skills and methods to small-scale farmers for the production of cocoons and to silk industries for the production of raw silk, silk yarns and silk fabrics. The project also established sales linkages to the local and foreign silk markets;  the organic and fair trade regional projects in Ghana and Burkina Faso,55 which trained small producers in harvesting and inspection for organic agriculture and fair trade, and facilitated partnerships between farmers‟ groups and importers; and  the Rural Knowledge Network pilot project for East Africa,56 which set up market access companies and developed networks and learning platforms with them.

54 TCP/GHA/2902. 55 Increasing incomes of small farmers through exports of organic and fair trade tropical products (GCP/RAF/404/GER).

123. Despite such encouraging outcomes, the evaluation found processing and distribution lacked the necessary focus and resources. Most of the national policy documents and interlocutors emphasised the importance of FAO taking into account all aspects along the full value chain. 57 There was strong demand for assistance throughout Africa according to the SFE technical expert. 124. The evaluators interviewed many people, including internally displaced people (IDP) preoccupied primarily with food security and nutrition, who expressed their appreciation for FAO‟s efforts to link farmers to markets. The evaluation team noted the cases of the sweet potato sales to southern Sudan and of the young animals‟ sales in Uganda as related success stories. 125. The beneficiary assessments show clearly that farmers consider the acquisition or enhancement of business, management, financial and related soft skills as having changed their lives for the better. The assessments underscore the importance to effectiveness of such attributes as confidence, negotiating skills, teamwork and trust. 126. Although many of FAO‟s CD projects in Africa have enhanced the capacities of individuals, the evaluation found most achievements limited in scale and often lacking the critical mass necessary for significant change. Most of FAO‟s CD initiatives targeted at individuals are not scaled up. When FAO has strengthened local institutions, facilitated their access to resources, and widely replicated that experience, as it has done in a number of IDP-targeted projects in Uganda, it has often been more effective and had a more sustainable impact. But this has not been the pattern with the majority of FAO‟s CD interventions. Even with FFSs, one of FAO‟s most effective CD field interventions, the evaluation team found evidence of some farmers‟ groups and networks that were disbanded once the project ended.

L. Impact Overall findings 127. It is generally acknowledged by the development community that it is difficult to assess programme impact, especially programmes that are focused on CD. Impact cannot be assessed until long after a programme has ended and attribution is extremely challenging because there are usually many factors that affect impact beyond the programme itself. Furthermore, recent research on measuring CD underscores the difficulties associated with ascribing impact to specific interventions and suggests alternatively that development agencies focus on processes and include measures of engagement and outcomes.58 What needs to be taken into consideration is the extent to which the capacities of individuals, organisations and the society have been strengthened to contribute more effectively to sustainable livelihoods and improved food security. In the developing world, these capacities include the ability to participate in debates, to advocate for resources and to partner with, rather than be the clients of, development agencies. 128. Many of the evaluation reports included in the meta-synthesis highlighted the difficulty in assessing impacts. Most projects lacked capacity needs assessments and few had adequate baseline data or benchmark indicators to assess performance. Where impact was assessed, it usually correlated positively with good practices, such as integrating CD into the planning phase, building in measures to ensure sustainability, and forging successful partnerships. 129. During their field visits, members of the evaluation team noted examples of positive impact of CD activities for individuals, among them FFS projects, conservation agriculture initiatives, rural information networks linking farmers to the markets, and a project focused on the computerisation of a dairy cooperative in Kenya.

56 GCP/RAF/401/IFA. 57 See Annex 10, Country Reports. 58 Morgan, P. (2006). The Concept of Capacity – Maastricht: European Centre for Development Policy Management.

130. The case studies confirm one of the findings of the Sustainable Fisheries Livelihoods Programme evaluation,59 that is, that the most visible impact of CD activities has been increased production and income for primary beneficiaries. For example, the establishment of FFSs in the Programme sous-régional de formation participative en gestion intégrée de la production et des déprédateurs (GIPD)60 in Burkina Faso resulted in increases in crop production ranging from 20 to 200 percent.61 Similarly, in Ghana, the introduction of techniques to protect crops and reduce damage from elephants contributed to raising agriculture production by 70 percent as well as generating new income from the commercialisation of chilli, which was used to keep the animals away.62

Box 7: Impact of FAO CD - Example from a Project in Uganda I have never known I could use the small spaces I have at home to raise vegetables. With the training I got from CESVI/FAO, my chicken droppings, goat pellets and cow dung will not go to waste anymore because they are food to vegetables that I will grow in the micro-gardens. I will hardly go to the market now to buy onions, tomatoes, cabbages and eggplants, because I have realised I can keep a small garden here and maintain it wet regardless of the rains since this can be watered.

Aldo, 60 years old, Ugandan farmer, beneficiary of the Livelihoods Support for Pastoralists and Agro-pastoralists in the Karamoja Region

131. The AGPT reports from West Africa highlight positive impact from CD support: Each project, as planned, has contributed to strengthening the Farmers’ Organisations and NGOs, facilitated the dialogue among them, set their structures up, refined their vision and enabled them to defend their own interests and effectively operate within the socio-economic context.63 132. Despite good performance generally, some FFSs have had less impact than expected because they lacked links to local support systems, networking and follow-up. The beneficiary assessments echoed this finding, pointing to the need for more emphasis on scaling up, stronger linkages and a better understanding of country contexts on the part of those designing and implementing FFS projects.64 133. In the area of organisational strengthening, there are several examples where FAO has contributed to sustainable, national centres of excellence, such as KEPHIS in Kenya and the veterinary laboratory in Sebeta, Ethiopia. The Lake Victoria Fisheries Organisation is an example of where FAO has contributed to forming a sustainable regional organisation. 134. FAO‟s track record with respect to policy development is also mixed. Although FAO assistance has resulted in countries adopting new agricultural policies or adapting old ones, the evaluation team found little evidence of these policies having been implemented. Notable exceptions include FAO‟s work with respect to Codex Alimentarius, the International Plant Protection Convention (IPPC) and some international agreements pertaining to forestry and fisheries. Where the team found evidence of impact, they also found recurrent training, long timelines, sound needs assessments and a great deal of follow-up.

59 “There is evidence to indicate that SFLP interventions have had an impact on target local communities (e.g. local organisations) and on many individuals in them. This is arguably the level where SFLP has had the most visible impact on the primary beneficiaries.” 60 GCP/RAF/009/NET. 61 Evidence from the Beneficiary Assessment (see Annex 10). 62 Ensuring Farmers‟ Livelihoods and Food Security around Kakum Conservation Area (TCP/GHA/2905). 63 Evaluation tripartite de l‟axe thématique « Approche participative et gestion de terroirs » du Programme de coopération FAO-Belgique pour la période 2004-2007, page 37. 64 Extract from the Malawi Country Report, which the evaluation team found to be mostly true elsewhere in the region.

M. Sustainability and Follow-up of CD Activities Overall findings 135. Sustainability begins with sound design and implementation processes that reflect many of the effective practices outlined earlier. The evaluation team found that FAO projects in Africa generally give sustainability short shrift. Examples of this include an absence of an after-project vision in the design of CD interventions, a bias toward short-term, output-oriented projects and a dearth of motivation for FAO staff, consultants and implementing partners to ensure sustainability. The meta-evaluation showed that sustainability scored low (see Figure 3) and remains among the most serious challenges for FAO‟s CD interventions. 136. As already mentioned in previous sections, projects seldom take a long-term approach, infrequently address all three CD dimensions and rarely take into account that CD is an iterative process that requires constant adjustment and follow through. The most successful and sustainable of the interventions were those that provided: inter-disciplinary support across all three CD dimensions over time; on-going links with, and access to, FAO expertise; and activities anchored in local or national institutions. Longer time-frame, follow-up and continued engagement 137. CD is a process and can rarely be achieved within the framework or timeframe of a single project. The case study countries and many evaluations, including those of FAO work in statistics, demonstrate the importance of a long-term approach to CD with ongoing engagement. The long run sustainability of capacity building projects is put at risk due to limited follow-up. Given the longer gestation periods associated with CD efforts, such as an agricultural census, the short time-frame of TCP projects limits sustainability. The auto-evaluation also found that project length affected sustainability and that project results remain an important issue in many countries.65

Box 8: Beneficiaries’ Perspectives on Sustainability

Projects like this one are like marriage. When you just get married, sometimes you do not know each other very well. But with time, you tend to understand each other better. Similarly, in this project, by the time we thought things were being understood, you say “we are going“. In this way, you leave us in suspense and lost! Farmer in Malawi66

There is a great possibility that half of our farmers will fall back to the old ways ... It takes three farming seasons and projects need an introduction time, then implementation, then weaning and then closure. Farmer in Tanzania67

138. The meta-evaluation showed that training generally lacked proper follow-up and mentoring support to ensure that new-found skills and knowledge were applied on the job and reflected in improved individual and organisational performance. The meta-evaluation showed that continued engagement and follow-up are usually essential to address institutional issues, including those related to mandate, incentives, norms, regulations and tenure, and to achieve

65 Auto-Evaluation of FAO Activities in Technical Support Services to Member Nations and the Field Programme (Programme Entity 222S1) and FAO/World Bank/USDA Initiative for Agricultural Statistics in Africa (Programme Entity 222A2), 2006, page 13. 66 Beneficiary Assessment Report – Malawi - Project Small-scale Industry Development Enhancing Food Security and Developing Sustainable Rural Livelihoods (GCPS/MLW/030/NOR)- Farmer, Chilembwe Section, Mpilisi EPA. 67 Beneficiary Assessment Report – Tanzania: A Joint UN Response for Northwestern (UNTS/URT/002/HSF) Tanzania - Farmer Field and Life School Beneficiary.

31 sustainable results. The mid-term evaluation of an institutional capacity development programme in Sudan supports these findings: … The training of individuals should be linked to particular outputs and expectations of performance within their respective ministries, attention should be given to follow-up support and mentoring, and there needs to be closer monitoring of the impact of investment in training … Large numbers of staff have benefited, but more follow-up is needed to address institutional and organisational issues and to provide continued on-the-job support so that trainees are able to apply their skills.”68 139. Most FAO CD activities are implemented through projects. The Organization has insufficient capacity to work beyond project frameworks. Where there was evidence of post- project follow-up, the evaluation found that this was frequently due to personal commitment on the part of FAO staff who often had to use resources from other projects. There were, however, some examples where FAO has engaged consistently, across the dimensions, through a series of projects over many years to enhance national and regional organisations, such as the Lake Victoria Fisheries Organisation, Tanzania Bureau of Statistics and KEPHIS. Where this has occurred, the evaluation found that it had often resulted in markedly improved organisational performance, increased confidence and, in some cases, the ability to provide services to other African countries. Institutionalisation of results and processes 140. The evaluation team found evidence of sustainability where projects had developed exit strategies, built ownership of stakeholders early on and fostered institutional commitment. Developing the skills of individuals or groups will not necessarily lead to sustainable results unless these skills are put to use in organisations that allow for continued change and adaptation, and provide incentives for their use. Many of the most successful CD interventions featured links to national or local institutions that provided ongoing support and the impetus for continued change. The following are noteworthy examples of partnering involving government institutions, CSOs and the private sector:  the Tulaga dairy cooperative project in Kenya69 involved a software and cell phone company to improve banking services. The companies continued to provide support after the project ended; and  the Beach Management Units on Lake Victoria have been closely linked to the Lake Victoria Fishery Organisation (LVFO), providing them with continued support, although the organisation needs to advocate for additional resources to provide backstopping. 141. The evaluation found that most of these partnerships were an effective first step, but that few have been sufficiently institutionalised. Generally, FAO needs to ensure that CD results and processes are anchored in institutions that have the capacity to back up and scale up. Developing capacity at the decentralized/local level 142. Increased emphasis on decentralisation of implementation highlights the urgency to fill critical capacity gaps at local levels. The evaluation team found a significant lack of local capacity and weak institutional linkages at local levels in the countries visited. Although many field projects helped form farmer groups, many of these groups remain vulnerable without programmes to strengthen local government authorities responsible for providing services to them. The evaluation team heard many calls for CD to strengthen local institutions, particularly with respect

68 Sudan Institutional Capacity Programme: Food Security Information for Action (SIFSIA) – OSRO/SUD/620/MUL/EC – Mid-Term Evaluation of the SIFSIA Programme – June 2009, page 14. 69 Computerisation of Agricultural Cooperatives and Emerging Producer Associations in Kenya (TCP/KEN/2907) but there were problems with little follow-up with the Ministry of Cooperatives, thus limiting scale-out potential.

32 to management systems and the development of soft skills. These demands were most vocal in relation to complex local initiatives such as public-private partnerships. 143. Developing capacity at the local level for information management and policy formation and implementation is essential to the sustainability of FAO interventions at national levels. If there is poor information from local levels and insufficient capacity to implement policies, then it is unlikely that the best efforts at the national level will be sustainable. The evaluation found little evidence of the „trickle-down‟ effect to district levels resulting from FAO‟s information and policy initiatives at higher levels. FAO‟s work on food security policy in Mozambique and Zanzibar were notable exceptions. Encouraging networking 144. The evaluators note that many of the most sustainable projects reviewed were those that were linked on a continuous basis to networks for ongoing professional development and retooling. Although some field projects have helped replicate farmers‟ groups, diversify crops and strengthen marketing, these gains were often limited unless the groups had on-going access to services and support. FAO has provided little support to strengthening farmers‟ lobbying efforts for services and has provided few incentives to ensure sustainability. 145. In contrast, FAO‟s efforts to facilitate the participation of Africans at regional and international fora, workshops and other exchanges to strengthen their knowledge and build their confidence, has been important to sustaining capacity. Many beneficiaries underscored the lasting benefits of opportunities provided by FAO to develop their confidence, enhance their communication and negotiation skills and exchange ideas. Such exposure helped some individuals influence the informal enabling environment. This was evident, for instance, in the Nile River Basin Initiative and in plant protection interventions in all the case study countries referred to earlier. Facilitating government commitment 146. The sustainability of projects often depends on the recipient government‟s commitment to follow up and provide sufficient resources. The evaluation team‟s country visits highlighted the need for greater capacity on the part of government officials to advocate for the resources required to scale up and follow through. 147. However, the evaluation team found that FAO interventions in Africa fail to devote sufficient attention and resources to building the political will necessary to bring about and sustain change. More active engagement of the FAO Representatives with governments, civil society and the private sector is needed to develop the required enabling environment, which includes motivation, incentives and will in order to expand sustainable capacities once developed.

N. Gender 148. Strong efforts are required to: effectively mainstream gender equality into agricultural projects; deal with the many gender-related underlying causes of poverty and food insecurity; and ensure that gender analysis is featured prominently in policy decision-making. At HQ, FAO has taken steps in recent years to give greater prominence to gender in its corporate strategies. For example, the Gender and Development Plans of Action for the periods 2002-2007 and 2008-2013 acknowledge the importance of promoting gender equality. The inclusion of a gender-specific goal70 among FAO‟s 11 strategic objectives has the potential to pave the way for gender mainstreaming. The Gender, Equity and Rural Employment Division (ESWD) recently developed a guide to gender mainstreaming within FAO‟s New Strategic Framework71 and Africa is a priority for gender-focussed interventions of ESWD, representing 37 of the top 50 countries of

70 Gender equity in access to resources, goods, services and decision-making in the rural areas. 71 Gender Equity in Agriculture and Rural Development, FAO, 2009.

33 focus for the division. Furthermore, the Division, through the FAO Regional Office for Africa, supported Ministries of Agriculture in selected African countries to elaborate sector-specific Gender and Agricultural Development strategic policy and planning documents in follow-up to the 1995 Beijing Conference on Women. Overall findings 149. FAO‟s CD work on gender in Africa is carried out mainly through: i) specific field projects that deal with gender issues; ii) support to the National Institutes of Statistics on collection and analysis of gender-disaggregated statistics; iii) integration of gender concerns into land and water management policies in Lusophone countries; and iv) the diffusion and capacity development training based on gender analysis tools produced by the Socio-economic and Gender Analysis Programme (SEAGA). In addition, the Gender Biodiversity and Local Knowledge Systems for Food Security (LinKS) project and the Dimitra initiative have worked along the individual and the policy/enabling environment dimensions to strengthen the capacities of government officials and rural populations on gender-related concerns. Capacity building activities were also organized in Africa (and Asia), under the Project on Gender Analysis in Farmers‟ Water Management in support of the Special Programme for Food Security (SPFS). 150. Most staff interviewed by members of the evaluation team appeared to recognise the need to consider the gender dimensions of their work. A few initiatives in the field, such as many of the FFS projects, have tried to ensure gender representation in projects. However, for the most part, CD projects in Africa pay lip-service attention to gender and do not fully understand the gender dimensions of interventions.72 Gender considerations did not score satisfactorily in many of the evaluations analysed as part of the meta-synthesis. The beneficiary assessments and the interviews the evaluation team carried out in the field confirmed a shallow understanding of gender equality and its important role in relation to Africa‟s successful development. An indicator of this weak understanding was that many people interviewed in the field believed that they had fulfilled FAO‟s expectations with respect to gender by ensuring that women participated in project activities in numbers equalling or approximating men. Those responsible for CD interventions, including those carried out through normative work, generally viewed gender as a stand-alone set of activities and not as a cross-cutting concern. Only eight of the more than thirty normative products reviewed contained references to the gender dimensions of the subject-matter (see Section VIII of this report). As noted in the evaluation of the Approche participative et gestion de terroirs73, the lack of attention to gender equality has serious consequences leading to methodological and operational gaps in each of the activities. 151. FAO partners and FAO staff recognize that FAO does not have the exclusive expertise on gender per se, but has a comparative advantage on mainstreaming gender into agriculture, among other things by establishing links among the Ministry of Agriculture and other key stakeholders working on gender. Gender is an area that requires FAO‟s urgent attention. Where it has insufficient expertise, there are opportunities for partnering as the Organization has successfully done in Uganda (see the next section on partnerships). FAO should do much more sensitising of its own staff on the meaning of gender equality, its importance and on practical ways of integrating it into CD initiatives. In the context of the One UN Initiative, FAO also has to communicate what its added value is in terms of mainstreaming gender in the agricultural sector and food security initiatives. The outcomes and recommendations from the forthcoming gender audit to be carried out with UNIFEM will probably contribute to the formulation of a corporate capacity development strategy for gender mainstreaming in FAO activities.

72 For example, women are called to meetings when they have other responsibilities, as in the example where training was planned for women in a fishing project at Lake Victoria. The training coincided with the time when women help send men out on the boats, and so few participated. 73 Evaluation de l‟Axe thématique “Approche participative et gestion des terroirs” du Programme de coopération FAO- Belgique pour la période 2004-07.

152. To institutionalise gender mainstreaming and enhance gender results across the Organization, FAO has appointed a gender focal point in each division at HQ and has, in parallel, carried out CD activities for FAO Representatives in Africa and Assistant Representatives in West Africa, and emergency coordinators and project managers in Somalia, Sudan and Kenya. CD workshops on gender mainstreaming in emergency and rehabilitation programmes were also organised with TCE in selected countries in Southern Africa. With only one dedicated staff member at the Regional Office for Africa (RAF)74, the Organization‟s field capacity is severely limited. Clearly greater resources, more partnering and more training on gender mainstreaming to staff in HQ and in the field are needed in order to achieve FAO‟s gender-related objective. The IEE raised this same issue.

O. Partnerships 153. FAO expertise is required to enhance capacity in a wide range of areas and across all CD dimensions. As clearly pointed out in FAO‟s draft “Corporate Strategy for CD”, it is important that FAO engages more in partnerships in order to adequately cover this broad spectrum of CD work and, at the same time, focus on its areas of comparative advantage. 154. For the purpose of this evaluation, partnership means “cooperation and collaboration between FAO and external parties in joint or coordinated action for a common purpose.”75 Contractual relationships for implementation are not considered to be partnerships, in that they do not include significant resource sharing. The evaluation focused on FAO‟s partnerships for CD at regional and country levels. The evaluation does not address general issues relating to corporate institutional partnerships. Overall findings 155. In FAO, where partnerships have been established, they are most often a result of individual rather than institutional relationships. Exceptions are the Delivery as One (DaO) countries where FAO is formally required to partner. 156. Where FAO is engaged in partnerships, the Organization may not be fully exploiting its benefits and it is sometimes perceived as not fulfilling its partnership role, according to reports received by the evaluation team in the field. The Organization has often been blamed for its slow decision-making processes, as well as for its complex and inappropriate financial and contractual arrangements. In a few cases, the evaluation team heard complaints that FAO did not always acknowledge the capacities of its partners and involve them in planning and decision-making. 157. FAO needs partnerships in order to fulfil its obligations to develop endogenous capacity under the Paris Declaration, to meet its requirements under the One-UN reforms and to leverage additional resources. 158. The report next provides an analysis of the types of partnerships common to FAO in its CD work in Africa. African inter-governmental organisations 159. FAO has worked quite extensively with African regional and sub-regional political and economic integration organisations, yet it has done so more by providing policy, technical and financial support than by partnering. A few examples are:  support to the NEPAD Secretariat (advisory services) and close collaboration on the development of the Comprehensive Africa Agricultural Development Programme (CAADP);

74 In Uganda and Malawi, FAO hired two consultants to work on gender-specific projects. While this has been much appreciated, it raises concerns about sustainability. 75 Thematic Evaluation of FAO Partnerships and Alliances, FAO PBEE, 2006, page 7.

 institutional strengthening of the Central African Forest Commission (COMIFAC) and assistance to the Comité régional des pêches du Golfe de Guinée (COREP) in developing its strategic plan and programme of action;  support to the Economic Community of West African States (ECOWAS) to set up a food security monitoring and evaluation mechanism; and  assistance to the South African Development Community (SADC) on policy development and the creation of a data management system for seeds. 160. In each of the above examples, FAO‟s support has been primarily directed at producing outputs rather than at engaging in processes aimed at enhancing organisational capacity and partnership. 161. The evaluation team observed more collaborative and durable relationships on technical matters with regional organisations such as the CILSS and AFRISTAT. FAO has also provided CD support in nutrition, pesticide use safety and early warning and food security information systems. Under an EC-sponsored programme, FAO is working collaboratively with the CILSS to develop regional food security frameworks and to undertake joint assessment missions. 162. FAO has also helped establish and strengthen regional institutional arrangements to deal with trans-border resource management issues. FAO‟s support to the NBI is a good example.76 This collaborative arrangement has produced analyses of outstanding quality and accessibility, which now inform regional policies and programmes. It also contributed to enhancing skills which are being used to develop an innovative and carefully formulated national water policy in Uganda. 163. Many of these regional organisations, including ones with overlapping mandates such as ECOWAS and the Union économique et monétaire ouest-africaine (UEMOA) require ongoing FAO technical, facilitative and financial support. Despite the difficulties inherent in working with regional organisations, opportunities remain for more authentic partnering that fully exploit these organisations‟ convening power and political mandates. UN agencies 164. In the countries visited, members of the evaluation team found few concrete partnerships between FAO and other UN organisations, except in Tanzania and, to a lesser extent, Uganda and Malawi. In Malawi, they found considerable scope for more partnerships under UN DaO, despite FAO‟s limited capacity for engagement there. The Evaluation found that there is only limited capacity in most countries for FAO to be well represented on the many coordinating committees. 165. FAO‟s cooperation efforts in Tanzania date back to 2005, when the project “Strengthening Human Security through Sustainable Human Development in North Western Tanzania” began. FAO, the United Nations Industrial Development Organisation (UNIDO), the United Nations Development Programme (UNDP) and the World Food Programme (WFP) have since worked together on household food security, on livelihood and poverty concerns of small- scale farmers in refugee villages and on FFSs programmes and training to improve agricultural methods and entrepreneurial skills. 166. In the context of UN DaO - Tanzania being one of the eight pilot countries77 - the collaboration entered an important new phase in 2007-2008. FAO is now involved, to varying degrees, in six Joint Programmes (JPs), all concerned with CD. The JP1 entitled “Wealth Creation, Employment and Economic Empowerment” is particularly noteworthy. Despite some

76 The Nile Basin Initiative Secretariat is hosted in Entebbe (Uganda) and serves nine riparian countries, namely Congo, DRC, Egypt, Ethiopia, Kenya, Rwanda, Sudan, Tanzania and Uganda. The Secretariat was established to provide the analytical capacity for the development, allocation, management and protection of Nile water resources. More detail is available in the Uganda Country Report in Annex 10. 77 Together with Rwanda, Mozambique and Cape Verde in Africa.

36 initial delays, the UN partners involved, primarily UNIDO and the International Labour Organization (ILO), agree that the programme presents great potential to increase synergies based on each agency‟s comparative advantage. They acknowledge that the agencies still work in parallel rather than together but maintain that significant improvement has come about from having established a common framework and divisions of labour. The experience helped FAO reduce some of its transaction costs, such as those related to mobilizing resources. It has also been advantageous in other ways, such as helping FAO to work at sub-national levels, complementing FAO‟s technical inputs with more holistic programming, linking horizontally along the value chain, and enabling each agency to specialise in its area of comparative advantage. However, such coordinated programming requires a high level of in-country expertise in order to participate fully and to partner effectively. Furthermore, it is complex, with diverse programming cycles, varying decision-making procedures at country-level and sometimes incompatible systems for monitoring and information sharing. 167. In Malawi, FAO participates in five UNDAF cluster groups and chairs the one on Sustainable Economic Growth and Food and Nutrition Security. The Organization plays an active and well-appreciated role on gender and HIV/AIDS issues in partnership with other UN agencies and, in 2009, has for the first time received resources from the One UN Fund to promote the use of fruit trees under a joint project with WFP. However, delays in transferring the funds seriously hamper the project‟s implementation and undermine FAO‟s capacity to participate as a reliable partner. Interviews with donors and partners confirmed that FAO has insufficient resources in the country to adequately take part in working groups and effectively partner in such joint programmes. 168. In Uganda, stakeholders appreciated the collaboration between FAO and other UN agencies involved in a project to counter gender-based violence. Among other things, the project has allowed FAO to identify the impact of GBV on food security and livelihoods and bring to the UN Joint project an important livelihood perspective for capacity development through the FFS and JFFLS approach. 169. Overall, partnerships with UN agencies are poor when not tied to formal mechanisms, as UN DaO and joint programmes are. Limited collaboration with the UN limits the reach of FAO‟s CD interventions and its work in technical areas where the Organization has insufficient resources and capacities. Despite agriculture having a prominent role in UNDAF and UN DaO programmes, FAO‟s weak capacity in many African countries is a serious constraint. It often prevents the Organization from fully participating in planning and decision-making and, at times, from taking a lead role when it is appropriate and expected. National and regional research organisations and universities 170. The evaluation team found that FAO has missed many opportunities to partner with national, regional and university organisations and networks on CD initiatives. This was evident in the meta-synthesis78 and from visits to institutions such as the Forum for Agricultural Research in Africa (FARA), Association for Strengthening Agricultural Research in Eastern and Central Africa (ASARECA), African Network for Agriculture, Agroforestry and Natural Resources Education (ANAFE) and Regional University Forum for Capacity Building in Agriculture (RUFORUM). People interviewed were keen to partner with FAO in areas of training, technical input, post-graduate studies and research. Many felt that partnerships would in turn provide FAO with effective platforms to disseminate its knowledge and apply its expertise.

78 The evaluation of FAO‟s Activities in Fisheries Exploitation and Utilization highlighted the potential to network with training and capacity building institutions. The evaluation proposed a “twinning process” (see note below) whereby a “senior” centre of excellence (institutions with internationally-recognised training capabilities) is paired with another institution in a long-term cooperative arrangement to jointly prepare selected training materials and to train trainers.

171. There are opportunities for FAO to “twin”79 with these organisations and others and to assist them in their efforts to link research institutes and universities more closely to rural people and their development issues. FAO could also help them adapt curricula, instructional methods and information systems to take advantage of modern technologies and advancements in learning80. 172. SFE has made an innovative effort to provide opportunities for students and lecturers on sabbatical and exchange to carry out research on topics of mutual interest. The collaboration between FAO and graduate students and lecturers provides an excellent opportunity to develop the capacities of African researchers, to enhance networking and to apply the results of agricultural research. The Alliance for a Green Revolution in Africa (AGRA) indicated an interest in working with FAO on some of the post-graduate programmes it sponsors. 173. It is important to note that the quality of human resources in research centres and universities varies considerably across Africa, with the best quality found in East, West and Southern Africa. This suggests that FAO could play an important role in strengthening agricultural research capacity in Central Africa, utilising the organisations from other regions wherever possible. Civil Society and the private sector 174. The evaluation team found FAO‟s record of partnerships with civil society and the private sector thin with many missed opportunities. The team believes FAO should consider partnering more with farmers‟ organisations and other CSOs, especially in West and East Africa, where capacities are growing but where support is still required. Involving these organisations in service delivery may require large initial investment, but such involvement has the potential for greater sustainability. Given the scarcity of FAO resources in-country, such partnerships could help FAO reach the decentralised and local levels more effectively. 175. Many of those interviewed in the countries the evaluation team visited said that CSOs have networks reaching into remote regions and that they provide opportunities for linking farmers to markets and to national policy making organs. In addition, they said that emerging regional federations of farmers‟ associations could assist in disseminating FAO materials and information on appropriate technologies, effective practices and international conventions to farmers. 176. As well, engaging in partnerships with international NGOs provides opportunities for FAO to reach out more widely and to engage in mutual learning and exchange. The Dimitra project is a good example of partnerships promoting exchanges among ten African organisations. 177. FAO appears to have had little experience partnering with the private sector, especially small business enterprises. The evaluators found a few exceptions in the country case studies. In Ghana, for example, FAO has partnered with an organic processing company. In East Africa, it developed a relationship with the Rural Knowledge Networks, which links farmers and rural traders. However, in this latter example, FAO has had limited success in identifying gaps in

79 Twinning is perceived and practiced by aid organizations in a variety of ways. The World Bank, for example, defines twinning as a “process that pairs an organisational entity in a developing country with a similar but more mature entity in another country.”[The World Bank Handbook on Technical Assistance, 1993] The Bank recognizes the method as one of its four primary technical assistance delivery modes, along with long-term advisors, short-term advisors, and training. It uses twinning to transfer relevant operational knowledge between two organizations similar in function and structure. The EC has employed numerous twinning projects since 1998 to facilitate the accession process for countries that wish to join the European Union (EU). EU experts may be seconded to participate in a twinning arrangement for over a year and may be expected to participate in short-term missions, training and technical supports. 80 Many universities are embarking on new, more inter-disciplinary, practical degrees, such as a regional Master in research methods organized by RUFORUM and Jomo Kenyatta Agricultural University of Technology. FAO could provide these organisations with knowledge materials, assist with teaching, arrange joint workshops or module courses, and engage students in field work.

38 relation to market access companies, even though other aspects of the project have been reasonably successful. 178. Others than these, however, the Evaluation has not had the chance to see any relevant instance of partnership between FAO and the private sector. In fact, the case-study countries show that FAO is currently more assisting, rather than partnering with, small traders and processors.

FAO’s Normative CD Products 179. One of FAO‟s four main areas of activity is “putting information within reach” in order to aid development. FAO plays a normative role by setting international standards, promoting these standards and establishing guidelines and effective practices in matters pertaining to its mandate. Its professional staff collect, analyze and disseminate a vast amount of normative data. Much of this work goes beyond the scope of this evaluation. However, the evaluation included an assessment of the normative CD products from the perspective of African users81. This section draws essentially from this assessment and complements this information with findings gathered during the field visits. The assessment, which appears in full in Annex 8, is based on a sample of 31 normative CD products, produced between 2000 and 2008. It focused largely on issues of relevance, quality and use.

P. Approach 180. The bulk of FAO‟s normative CD products stems from discrete projects that donors support with extra-budgetary assistance. Some of these are one-off initiatives with little follow-up after the product has been produced and distributed. Some products are, however, more programmatic in nature, in that they are part of a more comprehensive CD initiative that sometimes involves multiple-year funding and multiple interventions. For example, the Sustainable Agriculture and Rural Development Initiative, which links resources, expertise, knowledge and technologies to the needs of rural communities and disadvantaged groups, produced an excellent web-based series of concise, practical solutions to technical problems as part of a larger program of technical assistance involving government and non-governmental organisations and institutions. 181. Many of FAO‟s technical products feature experts talking to experts or experts talking to farmers. Although often justified, lessons from the development community suggest that reversing the power balance so that experts respond to questions put to them by farmers and/or farmers speak directly to other farmers, can help to bring about behavioural change. This, FAO has done through its exemplary FFS model in Africa and elsewhere, but lessons from these projects need to be collated into generic guidelines for establishing and institutionalizing FFS. The Nutrition and Consumer Protection Division appears to be following a similar pedagogical model in Les bonnes pratiques d’hygiène dans la préparation et la vente des aliments de rue en Afrique, which emphasizes two-way dialogue between trainers and street vendors as a means of encouraging the hygienic handling of food.

Q. Relevance to the Needs 182. The assessment found that most materials produced for CD are based on needs identified in the field and/or at international fora. Most, but not all, appear to be relevant to the target countries‟ development priorities. However, no formal mechanisms exist at headquarters to align FAO‟s normative products to the priorities of African countries, as formalized in NMTPFs and/or

81 The evaluation‟s inception report clarifies the normative CD products covered as follows: “only those which have been adapted or developed at country/regional level through a pedagogical approach will be considered (i.e. the simple existence of such material at country/regional level will not be assessed). Particular attention will be given to activities and products, including global ones, which target Sub-Saharan African countries, and/or are important in terms of their potential use and impact in the Region.” [page 2 ]

39 other strategic documents. Most divisions emphasize the need for a strong field presence in order to understand the needs and demands and to develop relevant products. There did not appear to be close partnering with national research organisations and universities to complement field presence and provide local knowledge. 183. The assessment found that information in several of the products reviewed is available from other sources, including, on occasion, other UN agencies and other documents produced by FAO. Duplication suggests the need for greater collaboration both internally and externally. The assessment also found one incidence where the impetus for a normative product was primarily driven by the staff‟s own personal academic goal rather than responding to the priority needs of a country. Although such cases appear to be few, they are cause for concern, given the many unmet capacity needs in the field. 184. Many of the Organization‟s normative products are geared to experts and to specialized institutions. The evaluation suggests that there are significant capacity gaps at the community level and within municipal governments in Africa that are not well served by FAO. In Mali, for example, local representatives reported that the terms used in some FAO instructional manuals were too advanced for frontline staff and that some topics were too difficult for trainers to handle. It would appear that more CD materials need to be adapted to local conditions and replicated in local African languages, particularly where literacy levels are low among farmer groups, as in many Sub-Saharan countries. Furthermore, few of FAO‟s CD materials deal with the “soft” side of development, with matters pertaining to organisational culture and the formal and informal systems of incentives and rewards that often shape the behaviour of individuals in organisations.

R. Design and Production Quality 185. Access is first about users being able to understand, internalize and apply the information they receive in ways that make sense within their own cultural and organisational contexts. Many of FAO‟s normative products are, from a pedagogical perspective, exceptionally well produced. All e-learning products reviewed, for example, featured self-assessments, individualized learning pathways, tutorials, checklists and links to related learning resources. Some CD-ROM products, such as the Land and Water Digital Media Series, fail, however, to exploit the medium‟s interactive learning potential. Some print materials are gold standards. Fire Management, Voluntary Guidelines, Principles and Strategic Actions were among the best reviewed, and Negotiation and Mediation Techniques for Natural Resource Management serves as a model for how to structure and design a manual for effective CD. 186. Many materials examined for the evaluation were designed with input from user groups, which is good practice. Some benefited from rigorous field testing, a practice that FAO should encourage for all instructional materials. Relatively few materials assessed were developed with FAO interdepartmental collaboration, confirming again FAO‟s insufficient inter-disciplinary approach to conducting CD work. 187. Although most of FAO‟s normative CD products have been produced in multiple languages, resources are needed to translate more of them into local languages in order to reach a wider spectrum of Africans.

S. Gender 188. Insufficient attention to gender is a serious weakness in many of FAO‟s normative products aimed at CD in Africa. FAO provides the instructional designers of e-learning materials with guidelines on gender equality, but not its instructional writers. Gender analysis is not carried out consistently at the design stage and there does not appear to be the kind of rigorous check on the content and images for gender sensitivity that one would expect of an organisation committed to gender mainstreaming.

189. Yet, the FAO‟s ESWD has developed some specific gender analysis methods and tools, mainly within the framework of SEAGA and the Sustainable Livelihoods Programme, to integrate socio-economic and gender issues at different levels and within a variety of technical areas. Handbooks are available for programming and policy interventions at institutional and field levels (communities, households and individuals). Handbooks are also available for technical sub- sectors, such as statistics82, microfinance, irrigation, emergency and rehabilitation programmes, livestock, household resources management and so on. The LinKS project on gender, biodiversity and local knowledge systems for food security has supported the development of post-graduate training materials for universities and a manual for facilitators on participatory approaches to local knowledge and bio-diversity management for food security.83 190. The visits of the evaluation team to the case study countries revealed little evidence of a wide distribution and use of the above and other gender-specific CD tools. It seems that significantly more effort could be made to raise awareness and strengthen the distribution of these materials, as the discussion at the Ministry of Gender in Uganda suggested.84 This could help ensure that gender equality considerations are more consistently integrated into CD planning and implementation.

T. Access and Formats 191. Print is by far the most prevalent format for FAO‟s normative products. The Organization distributes hard copies, often through its Regional Offices and partner institutions. However, the team found these very poorly distributed in Africa and difficult for many potential users to access. Increasingly, FAO makes its products available online and on compact disk (CD-ROM). Most materials are free to download from the Internet, but some CD-ROMs, such as those for the Codex Alimentarius Programme and the Land and Water Digital Media Series, must be purchased at a cost that is prohibitively expensive for most Africans. 192. The distribution of, and access to, FAO‟s normative products is weak in Africa. This was a consistent finding of the beneficiary assessments and field visits. Many local libraries and FAO offices have inadequate supplies of FAO‟s most important materials. Most FAO resource libraries in the field have been closed, and locally engaged consultants and contractors have often had to produce new materials without reference to existing materials. Such was the case, for example, with soil and irrigation learning resources in Zimbabwe. 193. Although FAO‟s use of the Internet significantly extends the reach of its products, the Internet is not without its drawbacks, particularly for Africans, for whom Internet access and bandwidth are often limited. FAO staff have taken measures to compensate for Africa‟s Internet connectivity and bandwidth constraints85. They have, for example, made CD-ROMs of many normative products available for free or at low cost and they have prepared materials that take minimal bandwidth and that can be downloaded in sections and printed according to the needs of users. Yet, according to some FAO staff, even printing from CD-ROMs is problematic in parts of Africa, as reliable photocopiers are often in short supply. A staff member in the field noted that by moving away from print, FAO was effectively transferring the cost of knowledge access to those most in need and least able to afford it. 194. Access difficulties persist even when users have reliable high speed Internet connections and plenty of bandwidth. For example, there is so much information on FAO‟s website that it can

82 Cooperation between ESS, ESWD and RAF have involved a series of joint publication of manuals and guidelines on gender and statistics. 83 More recently, the Regional Offices have launched a toolkit for the collection and analysis of sex-disaggregated data at the request from the African Commission on Agricultural Statistics (AFCAS). 84 Staff that were unaware of much of FAO‟s work on gender. 85 Connectivity issues remain a significant barrier for the 93% of Africans who have no Internet access (www.internetworldstats.com, June 2009).

41 be challenging for users to find the information they are seeking. There is no site map accessible from FAO‟s home page and some of FAO‟s search engines, such as that of the Capacity Building Portal, fail to turn up many of the Organization‟s best-known normative products. 195. A general concern, which applies to FAO and many other organisations that put most of their written materials online, is that they fail to adapt them for Internet use, which forces many readers to download and print the materials for study offline. 196. Increasingly, FAO staff are turning to distance and e-learning for CD. Among the available self-learning courses are those in the IMark series, Food Security Information for Action series and Codex E-learning series. IMark, which has 135,000 registered users worldwide, appears to be particularly cost-effective. However, there appears to have been limited integration of FAO e-learning activities. They are expensive to develop and to run and need to be closely evaluated to assess their effectiveness as a CD intervention for Africa. It appeared that a general weakness of many of FAO‟s e-learning courses is that few incorporate tutors and/or local peer learning groups, which may, in part, explain why many who enrol fail to complete the courses. 197. FAO makes limited use of some “old” formats, such as radio, and it has yet to fully embrace new digital technologies such as video streaming, which, again, may only be applicable in certain environments. In the field visits, there was a call for mobile village video training sessions along the lines of the old-fashioned mobile film evenings.

U. Capacity Building Portal 198. FAO‟s Capacity Building Portal is a one-stop gateway to FAO‟s CD resources. It provides access to FAO‟s published learning materials, to a series of good practices and case studies in the Capacity Building Good Practices Case Studies Series86 and to FAO‟s learning services. It also features links to FAO‟s e-learning resources. Despite its potential as a convenient platform for all FAO CD products, the portal, which is still under development, needs considerably more work to fulfil its mandate. For example, the portal‟s search engine malfunctions and many of FAO‟s best CD resources are on separate sites, requiring users to click numerous times in order to reach the information they seek. In order for the portal to be more comprehensive and search efforts less time-consuming, the website should be linked to the document repository of FAO‟s David Lubin Library. 199. The portal could benefit from featuring links to some of the best international research and networks on CD, such as the research of the European Centre for Development Policy Management (www.ecdpm.org) and the network Capacity.org (www.capacity.org/). The portal might also be enhanced through partnerships with African CD organisations and networks, such as FARA, ASARECA, ANAFE and RUFORUM.

V. Use of Normative CD Products 200. The utility and impact of FAO‟s normative products for CD are difficult to determine because the Organization collects little data on use and does little evaluation of the same. Evaluation team visits to case-study countries noted that the distribution and uptake of FAO‟s products are weak. Few interlocutors knew how to access FAO materials and even fewer spent time surfing for information. In other cases, the use of FAO‟s materials may be limited for reasons related to the enabling environment, such as limited incentives, lack of certification associated with training courses and, in particular, lack of follow-up on the part of FAO. 201. High cost, poor connectivity and low bandwidth limit uptake. However, increasingly, many more stakeholders have access to e-mail. Most interlocutors indicated that they would welcome being part of dedicated list-serves that provide abstracts and other files for reading off-

86 The Series is available on the CB Portal. It is an inter-departmental effort to document FAO good practices on capacity development and analysis of the critical success factors.

42 line. Several research organisations, universities and government departments indicated a willingness to identify participants for target lists and to host or facilitate such services.87 Most indicated that the best use of materials occurred when the products were introduced at workshops with regular contact and updating thereafter. 202. Although the number of products examined was small, the evaluation gathered sufficient evidence to conclude that the quality of FAO‟s products varies considerably. Some set the gold standard; others leave room for improvement, most often in relation to incorporating gender concerns. The volume of materials produced over the past seven or eight years likely outstrips that of all other UN organisations. While this is commendable, the evaluation suggests the need for greater emphasis on distribution and consistency in quality. There is no point in producing materials which cannot be physically or cognitively accessed. 203. There are major gaps in the distribution and uptake of normative CD products in Africa. FAO relies too much on Internet distribution, when access and bandwidth are problematic in much of Africa. The Organization needs to take into account the reality of the African context. It needs to employ other means of distribution and invest more in face-to-face CD on the ground with a variety of partners who have more effective reach through their membership. Distribution of printed materials is costly, but a range of appropriate local partners can considerably reduce those costs.

FAO’s Ability to Deliver CD

W. Overall Findings 204. The evaluation found that FAO has insufficient capacity in Africa to engage meaningfully in the increasingly country-focussed decisions on CD. 205. The evaluation found that the constraints on capacity arise from a number of factors, including the following:  the increasing demand on country offices to coordinate development assistance and to support governments‟ lead role;88  the different understandings of CD across the Organization and of FAO‟s role in it;  inadequate institutional and administrative arrangements, including insufficient horizontal and vertical coordination within the Organization and difficulties in accessing FAO technical expertise and knowledge products;  inadequate human resources to meet demands and weak operational capacities, especially in Regional and Sub-regional Offices and country offices; and  inappropriate timeframes and regulations governing projects.

X. Understanding CD and FAO’s Role in It 206. CD is a core function of FAO but discussions with interlocutors indicated that CD is not yet well integrated into all its work; nor is it considered a core function by all. The Paris Declaration and Accra Agenda for Action call on development agencies to facilitate nationally-led processes and the strengthening of endogenous capacities. This involves a change in FAO‟s understanding of, and approach to, CD. The evaluation team found that few FAO staff understood CD as a multidimensional concept. The meta-synthesis indicated a tendency on the part of many to equate CD with skill enhancement, to the exclusion of the wider organisational context and enabling environment. The consequences of this narrow understanding are highlighted in a recent evaluation as follows:

87 The FANPANR Climate Change listserv was cited as an excellent example. 88 As outlined in the section on changing environment, there are many clusters and working groups that FAO is expected to participate in and a number where its leadership is expected.

[...] a number of training efforts were performed in a void as the trainees did not return to a work place where they could put their new skills to use. It has taken some time to recognise the importance of capacity building (in terms of training and hardware) at state level, but the necessary corrections have now been put in place.89 207. This lack of clarity over what constitutes CD was also evident from the inconsistent use of CD terminology in documents and reports. In some instances, FAO staff used CD interchangeably with training. 208. FAO‟s new strategy on CD recognises that FAO needs to work more systematically across the three dimensions of CD. FAO also needs to provide a clearer perspective for staff on how to enhance soft skills, strengthen organisational effectiveness and influence the enabling environment. It needs to clarify that technical staff are not required to become experts in these fields, but rather to understand how these skills can be developed by the mode of approach used, and, when it is appropriate, to partner to achieve the best effects. 209. The new strategy provides a way forward but needs to be clearly interpreted for non- specialists in CD and disseminated throughout the Organization. Interviews conducted for the evaluation proved that FAO should more systematically involve staff from HQ and the field in fine-tuning the strategy and disseminating it widely. Ongoing consultations by FAO‟s Outreach and Capacity Building branch (KCEF) are a step in the right direction but need to be taken further to ensure ownership and use of the strategy. 210. Many interlocutors requested that FAO clarify its approach to CD and that FAO officers, particularly those in Regional, Sub-regional and country offices, be given new tools and clear direction as to how CD should be incorporated into their programmes. 211. The evaluation team found that FAO staff have received little recognition for their work in CD. Furthermore, some expressed concern regarding how CD will work and can be accounted for in FAO‟s new results-based framework. Because CD is a process and produces few immediately measurable outputs, there is a risk that CD work will be marginalised if these aspects of CD are not taken into account. 212. Of the 40 post descriptions the evaluation team reviewed, only 19 included CD among the duties of technical officers. None, except the description for Investment Officers in the Sub- regional Offices, indicated CD experience as a selection criterion. Nor did CD experience appear to feature prominently in the ToRs of consultants.

Y. Institutional and Administrative Arrangements to Deliver CD 213. FAO country offices are at the forefront of FAO‟s delivery of CD. The Sub-regional Offices are the first port of call for providing support to country offices in delivering CD efficiently and effectively. 214. Strengthening decentralization is a major component of FAO reform.90 The Regional Office (RAF), working with Sub-regional Offices, is progressively taking on new responsibilities. The decentralized offices will be expected to play a greater role in strategic planning and priority- setting at corporate and strategic levels and to support the development of NMTPFs and the identification of sub-regional priorities.

89 Mid-Term Evaluation of the Sudan Institutional Capacity programme: Food Security Information for Action (SIFSIA) – OSRO/SUD/620/MUL – June 2009.

90 Sub-regional Offices for Eastern Africa (SFE), Central Africa (SFC) and West Africa (SFW) have only recently been established and their staffing is incomplete.

215. The evaluation team visited FAO‟s Regional and Sub-regional Offices for Africa and attempted to assess their ability to perform their enhanced roles against the above background. Regional and Sub-regional Offices 216. Over the period of the evaluation, the RAF and the SFW have suffered from either none or high turnover in leadership. It is only in 2009 that the heads of these offices have been appointed. With eight vacant professional posts, it may be difficult for the RAF and the SFW to respond to increasing demands. In these offices, the evaluation noted little focus on CD and problems with staff morale, motivation and working conditions. There appeared to be little cohesion among work teams and officers were individually responsible for achieving their work objectives. There did not seem to be a commitment to CD nor to an understanding of the important role technical officers (TOs) have in this regard. With a few notable exceptions, these officers viewed their role primarily as responding to requests from countries to provide specific technical advice. 217. The physical infrastructure in the Regional and Sub-regional Offices is not conducive to team work. It appeared that neither the Country Office in Ghana nor the SFW were benefiting from being located with the RAF. There was little contact between SFW and regional TOs, with some exceptions. Most regional TOs appeared to be more involved with initiatives outside Ghana and outside SFW. The evaluation team speculated as to whether SFW might be more appropriately located in Dakar, along with many of the other UN and Regional Offices. 218. Many TOs felt disconnected from their parent technical divisions. There were notable exceptions, including the Plant Production and Protection Division (AGP) which had actively engaged its field officers in developing their strategy on CD, soliciting their input and providing them with regular feedback on technical and administrative issues. In the Regional and most Sub- regional Offices, new staff are seldom given an orientation and links into headquarters are unclear and often depend on personal relationships. One TO indicated that he kept links with HQ active by designing projects to cover costs of HQ staff expertise. The poor linkages, and in particular the lack of clear focal points, was highlighted when some Emergency Programme officers indicated that they had so few entry points to FAO expertise and knowledge products that they often turned to other agencies for help. This also highlighted the problems of access to expertise and knowledge raised by non-FAO stakeholders. Regional and sub-regional officers felt isolated from new developments in their fields and from the retooling opportunities available to HQ staff. Some felt that they had insufficient exposure and training on emerging and cross-cutting issues. 219. The evaluation found that the standard mix of technical skills at the Sub-regional Offices was not well matched with available resources and needs. Some SRO managers raised this concern. For example, SFC requires more support on forestry and environment issues with less on livestock; SFS indicated the need for an emphasis on policy and food security. The evaluation team noted that the Immediate Plan of Action provides for a review of the initial round of skill mix in the context of the Programme of Work and Budget for 2010-2011. Officers considered it important to provide more access to skills related to interdisciplinary and holistic approaches, with more emphasis on the cross-cutting and emerging skills. There was an expressed need for staff training to work in multidisciplinary teams. Some considered it important for TOs to have more field experience and to be sensitized to the importance of assessing capacity needs when providing TA. 220. At both sub-regional and country levels, there needs to be much closer liaison between the emergency and regular programmes, especially in SFS. Good effort was being made in SFE for One-FAO with the regular programme and the emergency unit collaborating in the delivery of services. The evaluation found that SFE was set up to engender a multi-disciplinary team approach and that managers were taking active measures, like those at SFC, to motivate and train their staff.

221. The evaluation found little acknowledgement of the important role of knowledge dissemination as part of CD. Only SFE was seriously engaged in efforts to ensure its leadership in making FAO‟s normative products widely available and accessible. This was also true at the country level except in Zimbabwe.91 Most offices did not appear to have a strategy for communication and many had closed their library.92 Country offices 222. The evaluation found inadequate capacity to fulfil the demands required of FAO, even in countries with strong representation, as in Tanzania. This is a major constraint in the countries where UN agencies have to engage with government in a common effort. 223. FAO country offices have insufficient HR capacity to engage effectively in intensive policy dialogue. While countries draw on the capacity of the Regional and Sub-regional Offices, many staff in country offices noted that expertise was not always available. They stressed the need to have senior experts from FAO available at the country level to engage in the policy dialogue. 224. The evaluation noted that the situation for taking up that role was better in those countries where there is an active Emergency Programme that provides additional human and financial resources. The strong coordination role of TCE in Zimbabwe has gained it widespread recognition amongst the donor community and development partners. It is an excellent example of how FAO can provide platforms for effective engagement, enhancing the capacity in the enabling environment to deliver services under difficult circumstances. In Zimbabwe, FAO plays an active networking and leadership role in strengthening the capacity of civil society, the public service and the private sector through the Agricultural Working Group. In Uganda, TCE and the FAOR collaborate closely, and the Organization is active in, and well-respected for, its role in leading the Food Security Cluster and in working with other UN agencies and partners. However, as the office and development partners noted, FAO is constrained by a lack of capacity to fully represent the Organization in every important forum. 225. Even where FAO is active, partners perceive the FAOR as having limited decision- making power. In some of the countries where the FAOR was either too busy, or did not have the requisite profile93 to take the lead, the assistant FAOR was sometimes not the appropriate person to assume this role. 226. The evaluation team noted the lack of FAO capacity in all the countries visited. In some countries, FAO is only marginally represented in multidonor groups and is not assuming a leadership role to the extent that it should.94 Given the increasing tendency for decisions to be taken in-country, this is likely to have negative repercussions.

Z. Project Modalities 227. Most CD activities at the country level are carried out through projects. 228. As noted earlier, the timeframe of projects is usually too short to achieve CD objectives. The inadequate time available for projects, particularly if logistics create delays, was one of the issues most highlighted by implementing partners and beneficiaries (Annex 11). The ability to deliver CD is severely impacted when there is inadequate time for projects. The process of

91 In Zimbabwe, the monthly newsletter distributed widely by FAO was an excellent mechanism for bringing attention to new normative products. 92 There were few documents after 2003 available in any of the libraries visited with the exception of the SFE. Where the library had been relocated to a government institution or university, no effort had been made to ensure that there was a continuous flow of information and a capacity to manage the collection. 93 A good understanding of FAO and of Africa. 94 Malawi was particularly challenged in this respect and given its experimental engagement in One-UN, the strong presence of donors in the country, and the lack of capacity within the country, FAO‟s lack of presence is keenly felt.

46 engagement is particularly important to both the effectiveness and sustainability of interventions. Those preparing project proposals frequently overstate what can be reasonably achieved in the allotted time in order to meet the criteria for acceptance. Project implementation is often stalled owing to delays in financial and administrative procedures, further reducing the time available for implementation. CD cannot be rushed. It requires time-consuming participatory engagement. 229. Some of the administrative restrictions on TCPs pose problems for CD. Of particular concern were the TCP restrictions that limit follow-up. As indicated earlier, insufficient follow- up was often the main reason for weak or unsustainable results. Furthermore, the common ceiling for all TCP projects fails to account for disparities with respect to the size of the country, its population, infrastructure and institutional circumstances, all of which affects project design and implementation processes.95 230. Implementing partners operating on very short Letters of Agreement are at a particular disadvantage with respect to CD initiatives. It is impossible to retain staff in situations where, for example, there may be a 2-to-5-month delay in obtaining subsequent agreements. Partners in Uganda complained of this regulation and its deleterious effects. 231. The evaluators found that CD as a component requirement of TCP projects was too often restricted to the training of individuals, to the neglect of other equally or more important dimensions of capacity.

Conclusions General overview 232. The Evaluation has dealt with CD, a subject-matter that is high on the Agenda for Africa and is widely acknowledged as critically important for the achievement of the three Global Goals and the MDGs. The evaluation began with ambitious terms of reference – to assess FAO‟s performance in Africa with respect to the relevance, efficiency, effectiveness and sustainability of its CD initiatives, and to answer a number of challenging questions regarding the Organization‟s approach to CD. Although the evaluation cannot be definitive in all aspects of its mandate, it has gathered sufficient information to form conclusions about FAO‟s performance in Africa and to suggest ways to guide the Organization‟s future CD work. 233. The Evaluation found FAO‟s CD performance in Africa has been mixed. Most interventions are relevant, many have been effective, but few have been sustainable. In general, they have been of high quality and have exploited FAO‟s comparative advantages with respect to its technical competence and its role as an honest broker. While the evaluation has documented some reports of operational delays, FAO‟s CD initiatives in Africa have, as a whole, been reasonably efficient. 234. The Evaluation noted a number of successes, principally where FAO had engaged continuously over time and across all three dimensions, most obviously in plant protection, statistics and increasingly in TADs. This continuous engagement over a long period, across dimensions, allowed for the building of a critical mass of skills, institutional memory and the policies, norms, values and structures to support the work. FAO has also achieved widely recognised success in integrating CD into its pilot projects testing new technologies using effective CD approaches such as FFS. In a few countries, FAO also showed strong leadership in national agricultural policy and development agenda as a facilitator between government, donors, the private sector and civil society organisations. There have also been some good examples of policy assistance which has effectively and sustainably strengthened policy analysis and implementation capacity in Burkina Faso, Mozambique and Zanzibar among others.

95 It was pointed out that the ceiling is the same for Nigeria and for Cape Verde Islands.

235. However, despite many effective and relevant interventions, the Evaluation found that FAO CD activities are, for the most part, unsustainable. There is very little emphasis given to sustainability and too much given to immediate results and outputs. This is evident in the project timeframes and modalities; the lack of understanding by FAO staff of the importance of process to CD; lack of focus on institutionalising CD activities and building political will to sustain them; and also to the limited motivation and opportunity for follow-up and for monitoring and evaluation by FAO staff. 236. Although the Evaluation noted some stellar examples of FAO‟s CD initiatives at the organisational and policy/enabling environment levels, the bulk of its efforts have been directed at individuals, contrary to lessons from the development community which stress the need for attention at all three levels and, in particular, to the formal and informal systems, norms and values that shape individual and organisational behaviour. Consistent with this, the Evaluation concludes that much of FAO‟s most effective CD work in Africa has occurred when the Organization has intervened over a long period of time and across all three capacity dimensions. 237. FAO produces much valuable and relevant knowledge as an important contribution to CD but its uptake and use in Africa is limited. Africa is constrained by poor communications infrastructure, which means that additional investment, or partnering, is required to ensure wider access to FAO‟s normative products. This may also involve the more selective production of materials in order to ensure that resources are available for effective distribution. There is no point in producing materials which do not reach their intended audience. 238. The Evaluation found that the Regional and Sub-regional Offices need their own capacities strengthened, more resources and better connectivity to expertise in headquarters in order to lead capacity development in Africa. At country level, FAO lacks the capacity to take its expected role in the many committees established to determine priorities, coordinate donor activity and to facilitate interactions between government and donors and with other stakeholders. The Evaluation considers that FAO will lose both relevance and opportunities unless it is able to significantly strengthen effective capacity in Africa. 239. To help address capacity constraints in implementing projects and in dealing with the emerging needs of decentralised systems, FAO needs to strengthen endogenous capacity, and be encouraged to partner more effectively. Setting priorities for CD within a strategic framework such as the NMTPF should take into account the context of the country, FAO‟s own comparative advantage to address national needs and the potential for partnering with local, regional and international agencies. 240. Furthermore, FAO has limited resources and needs to take the cost implications into consideration when taking on board a more sustainable and effective approach to capacity development. If enhancing Africa‟s capacity to improve sustainable livelihoods is the core function of the Organization, then it is important that due recognition be given to CD in budgeting, staff work programmes and assessments. Where additional resources are not available, the Evaluation considers that it may be necessary for FAO to reduce the number of projects and programmes to ensure that it has the human and financial resources to make sustainable CD interventions in selected areas. FAO can also work more in partnerships to share some of the costs involved in implementation and to increase its outreach. It can continue to move away from direct implementation to more facilitation and mentoring of local implementers. FAO is under increasing pressure to address new areas. Many of these are cross-cutting and require more interdisciplinary and holistic approaches. For some, existing staff can be retooled (climate change, land leases, soft skills, etc.); for others, some additional investment and partnering will be necessary (addressing the full value chain, gender, etc.) and for still others, working closely with partners is the most relevant option (management and accounting skills, CD for the frontline, etc.). 241. This Evaluation has shown that the more successful modalities for CD intervention are where the emphasis is on the way FAO has engaged rather than on the production of outputs. The

Evaluation has distilled the factors that have led to sustainable and effective CD and to the changing climate for development. It has highlighted the challenges confronting FAO as a result of the Accra Agenda for Action, increased in-country collaboration and decision-making by donors, as well as implications of increased decentralisation and commercialisation. FAO needs to take these under advisement when making decisions on resource allocation, establishing priorities and strategies and motivating staff so that it can be at the forefront of strengthening capacity to deliver on the MDGs. 242. In order to achieve the critical mass needed to strengthen the capacity of African organisations and institutions to develop more effective agricultural and natural resource management systems that keep pace with rapidly changing technology, climate change and the vagaries of international markets, FAO, along with other development agencies, has to invest more and change its approach to CD and be accountable for it. Specific conclusions leading to recommendations Mainstreaming CD (Recommendations 1 and 2) 243. Capacity Development is a core function of the Organization and is recognised as a key constraint to effective development in Africa by the international community. The Strategic Framework 2010-2019 includes CD as a core function and some of the organisational results outlined in the Strategic Objectives include CD. The Evaluation considers that CD needs greater prominence in the FAO Strategic Framework. The title of Core Function (e) should be worded to reflect that CD is more than the transfer of technology. In addition, CD should be more clearly articulated within the organisational results of all the Strategic Objectives. The Evaluation notes the need for FAO to address the potential contradictions in a result-based performance system that relies heavily on outputs in a specified timeframe, while CD is a process reflecting engagement over time. The Evaluation considers that greater recognition needs to be given to the importance of including capacity strengthening into all FAO work in Africa. Given the importance of CD, it may be appropriate to review whether the FAO Programme Committee, the Regional Conferences and the FAO Governing Bodies are sufficiently involved in providing political oversight over FAO‟s CD agenda. 244. Furthermore, it is important that there is a common understanding of what capacity development means. The Evaluation found that there is no common vision and definition for capacity development within the Organization. The Organization‟s new CD strategy articulates the main issues. Yet, it will entail a great deal of work to raise awareness and implement the measures required to ensure that CD is integral to FAO‟s work throughout Africa. 245. Given the differing interpretations of what constitutes capacity development across the Organization, as well as differing perspectives on the role of FAO in developing capacity, it is important that the new Capacity Development Strategy be widely shared and debated and a common understanding accepted. The Evaluation found that many staff recognise the importance of capacity development as central to FAO‟s mandate, but that for most staff, this means developing individual capacities through training. A better understanding of what constitutes good practice for CD, of how to enhance soft skills, of organisational strengthening and of how to influence the enabling environment is required. 246. Overall, FAO needs to raise the profile and understanding of CD in the Organization. CD needs to be the focus of most, if not all, of FAO activities including assembly and provision of information, support to international instruments, norms and standards, policy advice, advocacy, the formation and dissemination of knowledge products, technology development and transfer, and networking. This does not mean that all activities should have a specific training component, but rather that all activities and products take into account engagement in ways that best strengthen the capacity of clients. At country level, CD needs to be explicitly incorporated into the NMTPF so that there is a more consistent, relevant and coordinated approach to capacity development within each country.

The Effectiveness and Sustainability of Initiatives (Recommendations 2, 3 and 4) 247. This evaluation shows that FAO needs to continue to emphasise participative CD modalities. Lessons from this evaluation highlighted that the following principles need to be more consistently incorporated into the FAO approach to CD: with regard to individual training, engaging the employer organisations early in project design to ensure strategies for using the new skills of the trainees; using a participative approach with beneficiaries so that they are actively engaged in design and implementation; continuously interacting with implementing agencies and ensuring adequate participation of stakeholders all along the process; using local implementers where possible (even though it may initially require additional resources); learning by doing; facilitating and mentoring rather than simply transferring skills; emphasising soft skills by providing opportunities to participate in and take responsibility for the projects; provide opportunities to exchange ideas and developing networks; and, finally and most importantly, following up on the interventions through on-going engagement, support and the provision of active contact points. 248. These were all shown to contribute to more effective and sustainable capacity development. FAO staff need to recognise the importance of the process rather than outputs in strengthening capacity and be prepared to invest a significant proportion of resources in the process. They need to be encouraged and motivated to incorporate these and other recognised good practices more systematically into their approaches. At the same time, they need to be encouraged to form partnerships with other agencies to address areas for which they have less relative advantage. 249. The Evaluation has also noted common shortcomings hampering effectiveness, namely: weak CD needs assessments; lack of baseline and monitoring data; limited access/reference to many of FAO‟s excellent normative products; weak integration of gender concerns; and insufficient attention to documenting and disseminating successful interventions and lessons. 250. The Evaluation has also found numerous challenges for FAO in achieving more sustainable CD initiatives. The most common among them relate to: the lack of motivation for staff to design projects using modalities that emphasise the process of learning and developing capacities, which is time-consuming even for FAO staff to plan; project timeframes that do not take into account the iterative nature of effective CD; cumbersome project administrative procedures; difficulties in overcoming the weak ownership of CD interventions at the local level and/or lack of political will at national level; and a lack of follow-up and follow-through to ensure that capacities once developed are utilized, institutionalised, scaled out and sustained. 251. Addressing the inadequacy of the current project modalities (especially under the TCP) to enable more long-term and sustainable CD work requires restructuring and a reform of the TCP and PPRC96 criteria. Addressing the other issues requires providing staff with guidance on how to engage in project activities in ways that encourage effective and sustainable CD and with the motivation/incentives to follow up their activities. Partnerships (Recommendation 5) 252. The Evaluation noted that the CD environment in Africa is dynamic and variegated, while FAO‟s approach to CD has often been uniform and overly prescriptive. As more skilled individuals and organisations emerge in Africa, FAO‟s role needs to shift from implementing CD directly on its own to supporting CD in partnership with others, as stipulated in the new CD strategy. The need for a strengthened partnership approach is underscored by the fact that there are many new capacity demands in Africa, such as those related to management, marketing and advocacy. There are also capacity demands created at sub-national levels, as a result of government decentralization and in areas that affect agriculture and food security but for which FAO has neither the mandate nor the capacity to take the lead.

96 FAO Programme and Project Review Committee.

253. Also changing is the way development organisations do their business. As more and more CD is planned and coordinated under multi-donor and One UN mechanisms, FAO will face some difficult choices owing to its limited capacity in many African countries. As the new CD strategy suggests, it will need to strengthen its own capacities, find niches based on its distinct comparative advantages and make a concerted effort to partner with others. 254. The Evaluation has provided evidence of the need for, and efficacy of, partnerships for CD. FAO needs to partner with regional organisations to leverage their platform and to assist them in developing their capacity. Partnering requires high up-front transaction costs in which staff may initially be reluctant to invest, since they are under pressure to produce results over the short term. However, these costs are an investment in CD for Africa and in more effective future partnerships for FAO. 255. To address severe capacity constraints at the local level and complementing FAO‟s work in a holistic approach and for policy implementation in countries that are decentralization and given that FAO has limited capacity for engaging on the frontline, strengthening capacity of local implementing partners is particularly important. 256. FAO is currently working with a number of private companies and NGOs to implement projects and is using local consultants. The Evaluation found that closer liaison and mentoring and a move towards greater partnering would help to further strengthen endogenous capacity. In some countries, there were a range of national farmers‟, commodity, traders‟ or exporters‟ associations which have wide reach and the potential and desire to partner with FAO. The Evaluation considered this to be an area which FAO should pursue more vigorously. 257. The Evaluation endorses the IEE recommendation of much closer collaboration by FAO with universities and national and regional research institutes. FAO should investigate how it could more effectively partner with these institutions. There are interesting opportunities to be explored such as partially seconding faculty to serve on FAO-led projects and committees to provide a more structured, reliable and institutionalised access to these human resources than the current system of ad hoc consultancies. These institutions, together with various CSOs (including national farmers associations) could also be useful partners in the dissemination of FAO normative products and FAO could provide lectures from visiting technical staff to universities. Developing Capacity for Policy Analysis, Formulation and Implementation (Recommendation 6) 258. FAO was found to have been effective in assisting countries in policy formulation but less successful in enhancing local capacities to analyse and formulate policies. It is important that policy assistance incorporate CD specifically as its main objective to ensure that countries become more effective at developing their own policies. There was also a strongly identified need for FAO to provide much more CD for policy implementation. This involves more interaction at the frontline where capacities are weak and calls for the identification/mapping of all the stakeholders (and their CD needs) involved in the implementation. 259. FAO is engaged in ensuring that policies address the MDGs and improve sustainable livelihoods. The Evaluation found that most FAO policy assistance was taking this into account with its focus on food security policies. At the same time, FAO should assist countries to anticipate change and analyze the implications of new trends. The Evaluation recognized that, in the changing landscape, FAO also needs to develop capacity for policy formulation across the value chain, on climate change mitigation, irrigation (with policies that are sustainable and relevant to smallholders), and help countries ensure that externalities and local communities are considered when determining land leases. Documenting successful innovation and better dissemination of knowledge products (Recommendations 7 and 8) 260. The widespread calls by outside agencies for documentation on FFS led the Evaluation to conclude that it is important for FAO to document successful innovations. If initiatives are to be

51 scaled-up by other agencies, it is particularly important that the principles be clearly articulated in a set of generic guidelines to avoid distortion of the concept. These guidelines should demonstrate the principles through examples that show the broad range of applicability and the importance of flexibility. CountrySTAT, Rural Knowledge Networks, Computerization for Cooperatives, Conservation Agriculture, the mapping and analytical techniques developed for the Nile Basin Initiative are other examples. The Evaluation also found that providing exposure to success stories is important to build political will, indirectly assist beneficiaries to access further funding, and increase opportunities for partnerships and scaling up. 261. The Evaluation noted serious shortcomings with regard to the access and use of FAO‟s normative products in Africa. A much more effective dissemination and communication strategy needs to be developed to ensure that significantly more focus is given to making the normative products available to a wide range of African stakeholders. FAO currently makes limited effort to track readership and use. Investing heavily in producing normative products is wasted unless it reaches the target audiences. FAO also needs to ensure that its knowledge products are used in FAO interventions, workshops and trainings by providing consultants with access. 262. While knowledge dissemination is costly, it is as important as the generation of normative products. To reduce some of the distribution costs FAO needs to partner with universities, government, private sector and NGOs, research and documentation centres which can make available both hard and soft copies of their normative products. It can also work with these agencies to use list-serves to provide their members with information on new materials and offer options to order copies by email that enable off-line downloads for those with limited internet access. Increased Investment in FAO Capacity in Africa (Recommendation 9) 263. The Evaluation found that there was a significant lack of capacity in most of the country, Sub-regional and Regional Offices. The Evaluation noted that the lack of human and financial resources and the limited authority of the decentralised officers impede their ability to effectively develop capacity in Africa. The Evaluation noted that there is an urgent need to ensure that FAO staff in Africa are strengthened in number, are much better linked to headquarters, that they are better motivated and better prepared, and that they receive regular training and retooling to ensure that they can be at the forefront of their fields in developing capacities in Africa. In addition to investment in training, motivating and reinforcing links with HQ, the Evaluation found that greater cognisance of sub-regional capacity constraints should be considered in the allocation of Technical Officers. There was a call to ensure that every office had one or more TOs with skills in cross-cutting and inter-disciplinary approaches to assist in the development of the multidisciplinary teams. 264. In the country offices, as outlined in the Findings, the capacity is mixed but in all cases the skills and resources were not adequate for the important role FAO has in developing capacity to address the intractable poverty and agricultural development issues. There are significant demands in some countries from both governments and development partners for FAO to play a greater role in facilitating coordination working groups. FAO has been successful in participating actively in some, primarily those where FAO Offices have the additional manpower, motivation and resources that come with Emergency work. FAO needs to recognise the situation and put in place a strategy that takes into account its financial and human resource constraints but which will assure FAO, at least in some countries, the visibility and funding opportunities that sustained engagement can bring. 265. Capacity development is a process and the country offices will be required to assist in the continued engagement of projects with FAO even after project completion. They will need to be active in providing links between stakeholders and FAO expertise and they will need to be involved in anticipating the changes required in CD for that country. This means that to be effective, country offices need to be significantly strengthened. Resources are limited and so it may be appropriate for human and financial resource to be more concentrated in fewer countries.

266. One suggestion is that FAO be selective in strengthening country offices where it is most needed with respect to food security and poverty indicators and where agriculture, forestry and fisheries play an important role in the economy. In addition, FAO needs to take into consideration the opportunities and responsibilities in countries working towards One UN. 267. Strategies to supplement human resources at country level need to ensure that they do not undermine local capacity. The SFE approach of using seconded nationals to primarily address issues within the host country, and spend only a limited time on regional issues, is a good strategy to strengthen capacity of the host country, SFE and the individuals concerned. It is also important that all supplementary volunteers have an understanding of the context they are working in. The evaluation found only limited, and relatively unsuccessful, south-south engagement. The Evaluation did see potential for improving human resources within countries and the region through much closer engagement of FAO with national and regional universities and research institutes, as outlined above under Partnerships. 268. There was strong feedback through national plans and interviews for FAO to provide more CD on marketing, value addition and trade. The countries are emphasizing commercialization. FAO will need to consider its comparative advantage and priorities and develop a strategy with partners that will address the emerging CD needs of African countries. 269. Finally, it is important for the African offices to be directly involved in setting the priorities for CD. It is suggested that the Regional Conferences play a role in examining and building consensus on CD priorities. These could then be fed into the country discussions and NMTPFs which would also be aligned with the UNDAF process in the country. The Regional Conferences could then be more involved in providing oversight to FAO‟s CD Agenda. 270. The recommendations which follow are forward-looking and aimed at assisting FAO towards the realization of its new CD strategy. They are also intended to complement the Agenda for Corporate Action 2009-2011 as proposed in FAO‟s new CD strategy and are in line with the general direction of the IPA-driven reform agenda. Recommendations

AA. Mainstreaming CD 271. Recommendation 1: It is recommended that the Inter-departmental Working Group (IDWG) on Capacity Building take steps to ensure that FAO staff and partners have a common conceptual understanding of CD and FAO‟s role in it. This will, inter alia, require the IDWG to:  clarify for staff throughout the Organization what is meant by CD and what approach FAO will take in developing, monitoring and evaluating it;  reassess the role of the Programme Committee, the Regional Conferences and the FAO‟s Governing Bodies in providing political oversight over the CD agenda;  disseminate the corporate strategy widely within the Organization;  provide short, jargon-free briefs outlining the key principles and examples of how these can be applied in the three CD dimensions, in particular, with respect to improving soft skills, strengthening organisations, enhancing the enabling environment and contributing to improving functional capacities;  provide training in CD for all staff, especially those on the frontline in decentralized offices; and  develop strategies in all sectors for CD based on needs assessments that include actionable guidelines. 272. Recommendation 2: It is recommended that senior management, under the guidance of the IDWG, incorporate CD into the mandates, work programmes and post descriptions of all relevant programming staff. 273. Suggested in the implementation of this recommendation are:

 the systematic inclusion of CD into the duties of all relevant technical officers throughout the Organization. CD should be included in their work programmes and performance appraisals;  minimum levels of CD field experience and training to provide CD services as a requirement for posts in Regional and Sub-regional Offices;  the provision of incentives to encourage staff to engage in, and learn from, CD through advocacy and the incorporation of CD into responsibilities, work programmes and performance appraisals;  the requirement for staff to dedicate time to follow up on CD initiatives after projects end in order to institutionalise and upscale their benefits wherever possible; and  the establishment and promotion of networks and CD contact points so that FAO staff in the field and other stakeholders can more easily access FAO‟s CD expertise and knowledge.

BB. Sustainability and Effectiveness of FAO CD Initiatives 274. Recommendation 3: It is recommended that senior management, with the guidance of the IDWG, review and, where necessary, revise FAO systems to improve the effectiveness and sustainability of CD initiatives. 275. Bearing in mind the need for flexibility as each CD situation is different, this recommendation would entail:  ensuring, wherever possible, that the right resources - people, funding and administrative supports - are in place to allow for CD engagement for long periods of time until needed changes are institutionalised and successful pilot initiatives are scaled out;  ensuring that FAO‟s work modalities and projects are sufficiently flexible to allow for iterative CD programming featuring experimentation, monitoring and learning; this could involve restructuring TCP limitations, modalities and criteria - for example, increasing the funding ceilings for CD work in larger countries; changing TCP rules so that TCPs can be used for follow-up, and ensuring that there is adequate follow-up to all TCPs;  ensuring the application of sound monitoring, evaluation and reporting practices in a way that enables them to form a part of the CD and that encourages an adaptive management approach and experimentation; and  reviewing the results-based framework, taking into account the long-term nature of CD and the need to provide for an iterative process.97 276. Recommendation 4: It is recommended that the IDWG develop, and senior management implement, guidelines for projects and programmes that emphasise effective CD practices, such as participatory approaches that build ownership, sustainability and partnerships. 277. The guidelines should, for example, cover the following critical elements:  conducting capacity needs assessments as a first step in project design;  understanding the organisational and institutional context, including the various factors in the enabling environment that may influence individual and organisational behaviour;  systematic consideration for gender aspects in all stages of project and programme design and implementation, not limiting it to the equal participation of women into project activities;  encouraging beneficiary and other stakeholders to participate in the design and implementation of projects, particularly those agencies responsible for post-project support;  encouraging interdisciplinary and internally coordinated approaches to the design and implementation of CD projects and programmes;

97 "Improving the Results of Learning for Capacity Building" has some useful proposals for more CD relevant assessment systems http//capacitydevelopment.ning.com/page/presentations-from-june-2009

 using implementation strategies that systematically make maximum use of local experts and institutions, e.g. farmers‟ associations, universities and research institutes;98 and  providing mentoring, monitoring, evaluation and exit strategies that take into account the iterative nature of successful CD. 278. Recommendation 5: It is recommended that FAO senior management ensure that staff, in particular those in decentralized offices and FAO Representatives, place increased emphasis on partnerships in their CD activities in Africa. 279. Implementing this recommendation could entail:  recognising and budgeting for the transaction costs of establishing these partnerships;  developing guidelines to provide staff with more information on opportunities and mechanisms for partnering; and  carrying out sub-regional and country assessments of the opportunities for partnering at all levels.

CC. Developing Capacity for Policy Analysis, Formulation and Implementation 280. Although FAO has generally been effective assisting African countries with policy formulation, it has been less successful strengthening local capacities for policy analysis and implementation. 281. Recommendation 6: It is recommended that FAO staff place increased emphasis on facilitating the development of national capacity for policy analysis and implementation. 282. Implementing this recommendation could entail:  ensuring that counterparts are closely involved in policy formulation and are provided with opportunities to improve analytical skills. CD should be in the terms of reference of any consultants hired to provide policy support in countries to ensure that they do not only focus on producing policy outputs;  ensuring that all policy assistance includes sufficient follow-up and includes advocacy for implementation and CD for developing operational plans to implement policies and strategies;  finding ways, including through partnerships with local actors, to support CD at sub- national level in countries that are decentralizing; and  facilitating mechanisms for country, regional and HQ staff to identify and anticipate emerging policy needs.

DD. Documenting and Promoting Successful Innovations 283. Recommendation 7: It is recommended that FAO staff give priority to, and improve, the documentation and dissemination of successful CD initiatives, methods and normative products. 284. Implementing this recommendation should include:  documenting, showcasing and disseminating successful case studies widely to development partners, government agencies, national producer and distributor organisations, universities and research organisations, NGOs and CBOs;  documenting and disseminating the core principles of FAO CD approaches; in particular those which are, or could be, taken up widely by other agencies. Generic guidelines should be made available in different formats relevant for a range of audiences;

98 FAO needs to build up its database of relevant institutions and individuals and engage with those that have some skills and potential to be effective partners.

 establishing an independent review of FFS which is becoming a hallmark of FAO CD product with a view to perhaps establishing a centre of excellence to provide support for FFS initiatives and ensure high standards; and  encouraging more inter-departmental, collaborative learning and active participation in regional and international CD networks.

EE. Access to Knowledge as a Key Component of Capacity 285. Recommendation 8: It is recommended that FAO senior management take immediate steps to improve the distribution and uptake of FAO‟s products for CD in Africa. 286. Possible steps include:  developing communication and distribution plans for key products tailored to varied African contexts;  overcoming or compensating for connectivity problems in many parts of Africa by providing alternative access for low bandwidth and by disseminating CD-ROM and printed materials;  ensuring that all divisions and the central distribution centre provide materials to partners and document centres in Africa;  systematically tracking the distribution and use of materials;  making FAO‟s normative products available to stakeholders, including outside organisations, consultants and service providers;  strengthening library services in decentralized offices, including the establishment of libraries in all the Regional and Sub-regional Offices and in country offices partnering with institutions, such as research institutes and universities that have dissemination capacities; and  promoting networks that will assist in the distribution and dissemination of FAO products.

FF. Increased Investment in FAO Capacity in Africa 287. Recommendation 9: It is recommended that FAO invest more heavily in the capacity of its decentralized offices in Africa to engage significantly in developing capacities for the agriculture, forestry and fisheries sectors and to respond to emerging demands of African member countries. 288. Implementing this recommendation could entail:  a capacity needs assessment in each Region at the Regional Conferences and then in the sub-regions, to assist in developing a locally relevant CD strategy and priorities for the NMTPF and to inform the allocation of the correct mix of skills to each office;  ensuring that the necessary human and financial resources accompany the decentralization of the TCP approval process to enable the African offices to incorporate CD concerns;  more systematic partnering with regional and local CD institutions, such as research institutions and universities to fill up gaps, complement expertise available in decentralized offices and further enhance partner capacity;  the use of the SFE host-country model with seconded officers: an approach that largely deploys the officers in the host country so that FAO does not deplete scarce skills in the host country, but still benefits from additional human resources;  careful selection if engaging in south-south exchanges and using volunteers to ensure that their profiles and skills reflect a thorough knowledge of the African context as well as some CD knowledge and experience; and  development of a plan to increase the capacity of FAO field officers. This should include orientation, training, retooling, greater exposure to HQ activities and to current trends in

56 officers‟ fields, and motivation and incentives to ensure that CD is given a higher profile in their work. A strong focal point on CD in HQ.

Annex 1 - Terms of Reference

1. Background

1. The international community has in recent years increasingly focused its attention on country ownership for development effectiveness. It is at the core of the Paris Declaration on Aid Effectiveness (March 2005) in which donors committed themselves to support the partners’ national development strategies and to use to the extent possible the countries’ own institutions and systems. 2. Capacity development has been assigned a central role to play in this context. The Paris Declaration refers to capacity issues all throughout, as both a means and an explicit end of national development and to ensure the success of poverty-reduction strategies. Donors and partner countries agreed that a concerted strategy lacking strong capacity development foundations is unsustainable and unlikely to lead to success, even if substantially increased funds are allocated. 3. In parallel, a new concept of capacity development was envisaged, in which the respective roles and responsibilities of donors and partners were re-defined. Serious concern emerged that previous capacity development approaches had too often been out of synch with the countries’ priorities and realities. In Paris, partner countries committed “to integrate specific capacity strengthening objectives in national development strategies and pursue their implementation through country led capacity development strategies where needed” (OECD, 2005). Donors in turn committed to “align their analytic and financial support with partners’ capacity development objectives and strategies [...]” (ibid). 4. The UN system itself has played an important role in the rise of capacity development from being a technical issue to the centre stage of the global political agenda. In 2004, the UN General Assembly explicitly acknowledged that capacity development is instrumental for the achievement of the Millennium Development Goals (Resolution 59/250). The Assembly called upon UN organizations to “provide support to the efforts of developing countries to establish and/or maintain effective national institutions” and to use to the extent possible national execution and local expertise and technologies, also within the framework of South- South Cooperation. 5. Upon request by the same General Assembly, the Chief Executives Board (CEB) for Coordination initiated reflections on how to enhance the effectiveness of the capacity development efforts of the UN development system, including through the improvement of monitoring and evaluation of results. In October 2006, the newly created UN Development Group (UNDG) Task Team on Capacity Development produced a position statement on “Enhancing the UN’s contribution to national capacity development”, which defined the role of UN Country Teams and suggested a few steps to integrate capacity development into UNDAFs and Country Programmes. 6. Further to this, in 2007, UNDG commissioned a comprehensive review of capacity development policies and practices within the UN system, to be incorporated into the Triennial Comprehensive Policy Review (TCPR, A/RES/62/208). 7. It is now universally acknowledged that capacity development is, at the same time, the “how’ and “why” of development. Both the 2007 UN TCPR and the most recent Accra Agenda for Action (OECD, 2008) reiterate the importance of country ownership over development, and the key support that donors and multilaterals could provide for it. Evidence shows that progress has been made, in terms of governments’ capacities to implement policies and manage resources through their own institutions and systems. Yet, the progress is slow, and the sustainability of external support to capacity development remains an issue.

1 1.1. FAO and Capacity Development

8. FAO does not use the term “capacity development” although the draft corporate strategy recognizes the conceptual differences in the terminology1. FAO thus defines capacity building as “a process whereby individuals, groups, institutions, organisations and societies enhance their abilities to identify and meet development challenges in a sustainable manner.”2 9. Almost all the work carried out by the Organization relates to capacity development. According to an Issues Paper prepared by the Knowledge Exchange and Capacity Building Division (KCE) for internal use,3 capacity building in FAO encompasses a variety of activities, which deal with what has been elsewhere defined as both “the hardware and software” aspects, i.e.: • Training and education: workshops, distance learning, training, enhancing skills, competences and behaviours. • Technical cooperation and assistance: guidelines, methodological approaches, procedures, methodological support • Promotion/advocacy activities: seminars, policy support, public awareness activities supporting standard setting and normative work • Provision of knowledge/information, support to knowledge-exchange: access to databases and information, establishment of and support to thematic networks • Provision of infrastructure/equipment: agricultural inputs, IT, vehicles.

10. In general terms, FAO work on CD thus includes a wide range of normative products (guidelines, procedures, and policy documents), training and workshops, data and information sharing, and inputs provision. Furthermore, FAO work on CD involves, with different modalities and to different degrees, almost all the departments and divisions (e.g. nutrition, forestry, fishery, statistics, etc.) in the Organization.

11. The first effort to assess FAO’s capacity development activities dates to 2001, when a thematic review of FAO’s external training activities (1994-1999) was undertaken at the request of the 86th session of the Programme Committee. In response to the review, FAO senior management requested the establishment of an interdepartmental Task Force on FAO’s Training Activities with a view to ensuring follow-up on the recommendations and identifying resource implications of the proposed improvements. In April 2002, the Task Force prepared a report with specific recommendations on i) a corporate approach and a structured planning process for external training; ii) staff competencies in internal training; and iii) training support capacity. However, although approved by the Senior Management Committee in January 2003, the recommendations in the report could not be implemented due to a combination of circumstances, including resource constraints (KCE Issues Paper, 2007).

12. In 2005, the FAO Director General, in his reform proposal “A vision for the 21st Century”, called upon new and stronger attention to capacity development for individuals, organizations. The thematic focus of capacity-building activities was on the provision of training opportunities, including: learning programmes on policy targeted at senior policy- makers and analysts; institution-building for cooperatives, farmers’ organizations, chambers

1 In FAO terminology, “capacity building” and “capacity development” are used interchangeably. The former will be used here only when explicitly refers to FAO’s documents. 2 FAO Internet Site – KCE Web Page on Capacity Building 3 KCE, Capacity Building at FAO Issues Paper, Consultation Draft, June 2007

2 of agriculture; training of technicians; information on fellowships. The document, and the focus areas for capacity building identified there, provided the basis for the set up of four Interdepartmental Working Groups (IDWGs)4.

13. More systematic efforts to rationalize and enhance the capacity development activities of FAO started in 2007, when the newly established Knowledge Exchange and Capacity Building Division (KCE) was asked to coordinate the follow up to the recommendations of the FAO Conference Resolution 2005/13 on the TCPR and the conclusion of the Interim Report to the Council (October 2006) about the need of a new approach to capacity development, in line with the UNDG concepts and policies.

14. The FAO Independent External Evaluation further recommended the development of a capacity building strategy, “following an assessment of the needs of capacities of countries at different stages of development and in different parts of the world” (IEE, 2007).

15. In 2008, FAO launched a “Capacity Building Portal”5, as an entry point to FAO Learning Resources or services, to help FAO Member States, partners and other development actors to access knowledge, information, and training tools in the Organization’s mandate areas.

16. More recently, as part of FAO’s Reform, a new corporate strategic framework is being developed. Capacity building is a core function applied to Organization Results contributing to nearly all the strategic objectives. More specifically, a corporate strategy to enhance FAO support to capacity building is one of the primary tools for the achievement of the Organizational Result X2: “Client capacities made sustainable and adapted to deal effectively with global challenges through FAO services”.

17. A newly established IDWG on Capacity Building6 is now tasked with the preparation of the FAO Corporate Strategy of Capacity Building and has become the overarching coordination mechanism for the related interdepartmental groups. A number of internal consultations took place and six country missions were fielded.

2. Purpose

18. Capacity development is at the core of FAO’s mandate. Yet, no thorough and comprehensive assessment of FAO’s work in this area has been accomplished. At its October 2008 session, the FAO Programme Committee selected the “FAO’s work on Capacity Development in Africa” as one of the priority areas for evaluation.

19. The Evaluation will serve as a vehicle for accountability and learning by providing a rigorous evidence-based analysis of the current status of FAO’s work in the area of capacity development in Africa. The Evaluation will be forward-looking and formative and will provide guidance and recommendations to improve the work of the organization on capacity development in Africa, and perhaps other regions, in the future.

4 IDWGs on: Training for Technicians, Capacity Building on Policies, Fellowships, and Institution Building. A fifth IDWG was subsequently established to harness some of the outputs of these four groups into a corporate web-based portal on capacity building. 5 http://www.fao.org/capacitybuilding 6 The four previous groups joined the newly created IDWG, still as separate Task Forces.

3 20. The Evaluation will be timely in providing input for validating and refining the FAO Corporate Strategy on Capacity Building and defining the parameters for its implementation. It will also contribute and inform the ongoing debate within the UN agencies on how to monitor and assess capacity development initiatives. 21. The Evaluation will be addressed specifically to FAO member countries through the Programme Committee, Senior Management, and FAO staff working on capacity development as well as clients of FAO’s CD products and services in particular in Africa.

EVALUABILITY ASSESSMENT

3. The Concept of Capacity Development

22. The concepts of capacity and capacity building are so all-encompassing that practitioners have found it difficult to make operational sense of them.7 There is, however, a consensus that the term capacity building is no longer appropriate and should be substituted with capacity development or enhancement.8 The “building” metaphor does not actually reflect the time dimension of capacity development, rather suggesting the lack of pre-existing capacities. 23. The shift in terminology reflects a conceptual change in both the policy and approach towards capacity. As mentioned above, until recently CD was viewed mainly as a technical process, involving the transfer of knowledge or organizational models. Similarly, not enough thought was given to the broader political and social context within which CD efforts take place, and the importance of country ownership on knowledge acquisition of CD initiatives was not adequately emphasized. 24. The new consensus reflected in the current literature and articulated strongly in the 2005 Paris Declaration sees CD as a necessarily endogenous process, strongly led from within a country, with donors and other partners playing a supporting role. The existence of three dimensions – individual, organizational, and policy/enabling environment - on which CD objectives may be defined and pursued is acknowledged. CD starts with individuals as they are “the basic unit of capacity, in whom skills and knowledge are vested.”9 Although important, the success of programmes aiming at strengthening human skills depends crucially on the organizations in which people work. In turn, power structures and institutions (what Douglass North defined as “the formal and informal rules of the game in a society”) shape and constrain the functioning of particular organizations. Organizations, and the policy environment thus influence individuals’ capacities by creating incentive structures that either give or deny them opportunities to make good use of their abilities and skills. 25. There are a multitude of definitions in the literature. The OECD-DAC (2007) provides a useful definition highlighting the three levels of capacity development and linking capacity development to results and performance: “Individuals, groups and organizations, institutions and countries develop, enhance and organize their systems, resources and knowledge; all reflected in their abilities, individually and collectively, to perform functions, solve problems and achieve objectives.” 26. Further adapting this definition for FAO specific mandate areas, the evaluation will use the following definition:

7 A survey carried out in the framework of an evaluation of the World Bank’s CB activities in 2005 found that there was no consensus on what CB is about among staff. 8 Still, FAO documents use the two terms interchangeably. 9 Lopes C. Theisohn T (2003)

4 Capacity development is understood as the process whereby individuals, groups, private and public sector organizations enhance their systems, resources and knowledge as reflected in their improved abilities to perform functions and solve problems in order to better address hunger, poverty reduction and sustainable natural resource management objectives.

4. Scope of the Evaluation

27. The evaluation will focus on Africa and more particularly on Sub-Saharan Africa, where the needs are considered the greatest and many countries lag behind with respect to the attainment of the Millennium Development Goals. Capacity-related constraints, arising from a combination of individual, organizational and institutional/societal factors10, have been time and again identified as a major hindrance to Africa’s development. This is particularly true in agriculture, which accounts for a large share of GDP at regional and country level, and where the “huge paucity of trained personnel”11 and the weak state of organizations and institutions are often cited as key factors affecting the poor performance of the sector 28. Given the breadth of the concept of capacity development, and the diversity of FAO activities and interventions falling within it, it is important to clearly define the scope of the evaluation, by reaching a common understanding on which products and activities are to be included. 29. In a first attempt to define its coverage, the evaluation will include FAO initiatives at country or regional level for which capacity development is an explicit and substantial component part of the product or the service. Within normative products12, only those which have been adapted or developed at country/regional level through a pedagogical approach will be considered (i.e. the simple existence of such material at country/regional level will not be considered as capacity development). Particular attention will be given to activities and products, including global ones, which target Sub-Saharan African countries, and/or are important in terms of their potential use and impact in the Region. The Evaluation Inception Phase will be essential in further refining the scope of the evaluation. 30. While upon initial review it appears that Member State Governments have been the major beneficiaries of capacity development efforts, the evaluation will consider all CD initiatives undertaken for government also including those specifically targeting communities, civil society and the private sector. 31. Taking into account the long term processes involved in capacity development activities, the Evaluation will cover an eight-year period (2000-2008).

5. Issues to be addressed

10 Norad 2008: The first level or unit of analysis is the individual, who is to acquire new or additional skills and knowledge. The second level is the organisation, producing new organisational structures, instruments and processes. The third is institutional or the societal level: changes to framework laws, formal and informal rules of behaviour etc.

11 IEE, 2007 12 The term “normative” has as defined in the “Note on “normative and operational” work in FAO” presented to the FAO Program Committee, Rome 8 – 12 May, 2006 as standard setting work related to international governance such as Conventions and Treaties, Public Information Goods such as outlooks and status reports, advocacy to combat hunger and malnutrition, and information systems functions: collection and dissemination of technical information and global monitoring and alerts, as well as knowledge management.

5 32. Within the defined boundaries, the Evaluation will seek to the extent possible to identify FAO’s comparative advantages13 in capacity development and to assess the extent to which FAO’s work on in this area is based on its technical and functional areas of comparative advantage, and particularly on the adoption of “a multidimensional and integrated approach”14 defined in partnership with, and in relation to the role played by, other organizations. 33. The evaluation will attempt to address the following broad questions: • Is FAO’s approach towards capacity development in line with emerging good practices? Does it promote greater ownership of partner countries? • Do the capacity development initiatives respond to articulated or implicit demand and in what way do they contribute to poverty reduction? • What are the strengths and weaknesses of FAO’s work on capacity development? • Has the Organization focused its work where it has a comparative advantage in capacity development support? • Has it worked effectively with national institutions and in partnership with other agencies? • What should be the role(s) of FAO on capacity development in the future? • How does the FAO corporate strategy on capacity building reflect the above concerns?

34. The Evaluation will more specifically focus on the following issues:

Relevance and Coherence of FAO’s Work

35. The relevance and coherence of FAO’s work in this area will be assessed against: i) the needs of the Region and the countries; ii) the National Medium-Term Priority framework at country level if relevant; and iii) other sources of support to capacity development and FAO’s comparative advantage. In particular, the Evaluation will look at the: • Extent to which FAO has identified country capacity needs (as expressed through consultative processes, analysis and review of country strategic documents) and reflected them in programme and project interventions and strategies (including in recently- developed NMTPFs, where applicable); • Extent to which FAO interventions at field level are based on an analysis of the organization’s comparative advantages. • Extent to which it is co-ordinated with other international and national capacity development initiatives

Design of CD activities

36. The design of CD activities will be assessed against identified good practices, looking in particular at the respective roles of FAO and national counterparts and how sustainability concerns are adequately considered in the planning phase. In particular, the Evaluation will look at the: • Extent to which the design of the CD initiative have taken in due account the needs at individual, organizational and policy dimension in order to identify CD priorities. This

13 The ability of the Evaluation Team to do so will depend very much on the availability of assessments of other organizations involved in CD in rural development and agriculture and the extent to which the Evaluation Team will be able to interact with other organizations at country level. 14 Strategic importance of capacity building in the context of FAO Reform and the IEE - Keynote by J.M. Sumpsi, ADG Technical Cooperation Department, January 2008

6 will include an examination of the way in which gender equity has been analyzed at all three levels; • Extent to which the CD needs have been identified in a participatory way, tailoring the programme and materials to the different target groups; • Extent to which CD activities have been designed to promote greater ownership in the region, countries and amongst the client groups; • Extent to which the objectives of the initiatives have been clearly defined in terms of desired outcomes; • Extent to which cost-effectiveness and sustainability issues have been adequately pondered in the design and planning of projects and activities; • Extent to which the CD design takes into account emphasis on the three major goals of food security, poverty reduction and sustainable natural resource management; • Extent to which the CD design takes account of gender and other cross-cutting issues; • Extent to which CD initiatives are relevant to stakeholders needs and in particular whether they have taken into account the interdisciplinary nature of the problems faced in rural development; • The quality/adequacy of the monitoring and evaluation framework designed for CD initiatives.

Efficiency of FAO’s Work

37. The extent and quality of the technical, operational and other support provided by FAO to respond in a timely manner to agreed priorities and requests will be assessed. In particular, the Evaluation will look at the: • Extent to which FAO has developed its own in-house capacity and ensure appropriate resources to deliver capacity development; • Extent to which the current institutional set up allows for adequate and efficient coordination among different departments units in Headquarters; • Timeliness and degree of coordination in the management of projects and activities between HQs and decentralized offices (RAF, Sub Regional Offices, and FAO Representations); Adequacy and continuity in the two-way communication flow.

Effectiveness of FAO’s Work

38. The extent to which outputs and intermediate outcome objectives have been or are likely to be achieved will be assessed against what is stated in programme and project documents. In particular, the Evaluation will look at: • The quantity, quality and appropriateness of types or format of capacity development materials (e.g. training sessions, E-learning courses, practical guidelines etc) in regard to the needs of the target group; • Awareness and use of FAO’s normative products and knowledge sharing tools; • Role and effectiveness of FAO as a facilitator of regional and national technical and policy initiatives; • FAO’s performance and effectiveness as a provider of training in the areas within its mandate; • FAO’s performance and effectiveness in partnership for developing and enhancing the capacities of local individuals and organizations to deliver capacity development; • FAO’s performance in developing and promoting new and better technologies for CD.

Impact of FAO’s Work

39. To the extent possible, the evaluation will attempt to identify and assess the impacts of CD activities at the levels of individual and organisations and on the enabling environment through the collection of evidence with respect to demonstrated improvements in capacities of government, civil society and private sector to deliver services, develop policies/strategies, and perform individual and state functions in rural development, agriculture and natural resource management.

Sustainability of FAO’s Work

40. The sustainability of the results achieved by FAO’s interventions will be assessed within the broader regional and national context and over a long term horizon. In particular, the Evaluation will look at the: • Extent to which FAO assistance has been delivered in ways that enhance the capacities of the client groups and organizations; • Extent to which FAO has supported and encouraged leadership by regional or national actors; • Extent to which trained actors have been/or are involved in scaling out their CD activities at regional and national levels to further disseminate the knowledge and skills acquired; • Social capital and networking created through the CD activities; • Extent to which the support provided and the lessons learned by working together have been integrated into policy and institutional changes, planning, and implementation; • Extent of adaptation and further dissemination of CD approaches and information/training materials at local, national and regional levels.

Partnerships

41. Given the plethora of actors working on capacity development at international, regional, and national level, the Evaluation will seek to assess whether partnership building was strategic and efficient for creating complementarities and synergy among different actors, and based on considerations about FAO comparative advantages. The evaluation will attempt to identify what institutional partnerships have been best for FAO to make a difference on capacity development. The Evaluation will look at the efficiency and effectiveness of the cooperation with: • UN agencies, within UN Country Teams or the Delivery as One Framework when relevant; • Multilateral, bilateral and regional and national actors, and whether the latter have been adequately identified, including non-state stakeholders such as CSOs/NGOs, farmers/community groups and the Private Sector

Cross-Cutting Issues

42. Major cross-cutting issues such as gender concerns, HIV/AIDs and its impacts on agriculture development, climate change will be addressed all throughout, and particularly in the assessment of the design and effectiveness of FAO’s work.

8 43. The appropriateness and effectiveness of FAO’s CD work in both stable and fragile states will be considered and compared.

44. Staff turnover, transfer and brain drain are major challenges in African countries. HIV/AIDS is also an important cause of attrition for national human resources in many African countries today. The evaluation will examine how these issues have been addressed in countries where this represents a particular challenge.

6. Approach and Methodology of the Evaluation15

Analytical Framework

45. CD interventions in FAO differ widely in terms of the level and people they target, and the nature of the activities themselves. The complexity of the topic and the importance of a three-level analysis (individuals, organizations and policy/enabling environment) acknowledged, there is yet little agreement about how to more specifically measure the concept of Capacity Development. In large part, this is due to the fact that CD is a multidisciplinary and non linear process, and as such it is difficult to measure. “The overall enhancements in capacities are not attributable to any single external factor but can only be explained by the successful combination, customization, and sequencing of various CD modalities interacting successfully with contextual elements of the surrounding environment”.16 46. However, measurement is possible even when qualitative in nature and involving a time frame. To be operational, indicators of CD activities should relate to the two fundamental questions of “capacity for what” and “of whom”. This understanding has naturally important implications for any assessment exercise, when the “what” and “whose” add to the “why” and “how”. Both quantitative and qualitative indicators will be combined to measure the perceptions and points of view of different target groups involved in the CD. 47. The use of a log frame and of performance indicators has been sometimes criticized, as only of partial use for the assessment of capacity development, in that it produces little insight into the human behavioural aspects to do with learning, attitudes and values or organizational change. As such, the Evaluation will complement a logical framework approach (adopted when CD interventions are well defined and time-bound) with participatory approaches and system thinking to seize the needs of individuals and organizations, as well as the institutional opportunities and constraints in a complex environment. 48. In addition, specific methods and tools will be developed to assess the quality, use and effectiveness of normative products and policy work with CD objectives. The evaluation will also relate to the relative budget allocations and provide comparative analysis of the more cost-effective approaches. While much of the analysis is, by its nature, subjective, triangulation, cross-country experience and explicit rationale for the findings will be used to offset some of the bias. Whenever possible, gender-sensitive indicators will be incorporated for most activities.

Methods and Tools

15 A number of evaluations on CD have been carried out in the past few years. The proposed methodology is very much inspired from these evaluations. In particular, the Evaluation of WFP’s Capacity Building Policy and Operations was found most comprehensive and useful, as it uses differentiated frameworks according to the nature of the activities evaluated. 16 FAO, Draft Corporate Strategy on Capacity Development, 2009

49. To address the breadth and complexity of the topic, the Evaluation will refine its scope and define the evaluation tools during an Inception Phase, where a number of data collection techniques will be used to prepare the ground for the Evaluation, including: • Inventory and mapping of regional and national CD activities in Africa (objectives, activities, and inputs by FAO); • Meta-analysis of programme, project and country evaluations, carried out during the evaluation period; • Surveys of staff at HQs and in decentralized offices on CD activities relevant to Africa (products and activities, success stories); • Preparation of a paper on good practices in Capacity Development to serve as a basis for benchmarking CD activities; • Preparation of an Evaluation Matrix and definition of relevant indicators; • Selection of a sample of countries for case studies (see below); • Standardized format for the country case studies; • Standardized formats for the desk analysis of a sample of projects and normative products; • Other evaluation tools as appropriate; • The Inception Phase will clarify to what extent and how impacts will be assessed;

50. The Evaluation will build on the previous analysis and proceed with the assessment by enlarging and deepening the data and information collected through: • Desk Reviews of a sample of projects and normative products directed at Sub- Saharan African countries; • Desk reviews (and possible field assessment) of a sample of regional and national projects; • Country case studies where a comprehensive assessment of all CD activities will be conducted by the Evaluation. The selection of countries will take the following elements into account: i) sub-regional balance; ii) diverse policy and institutional characteristics; iii) coverage of all types of FAO CD activities and intervention; and iv) avoidance of overlap with other recent or ongoing assessments (including the studies which are part of the preparation of the forthcoming Corporate Strategy). • Interviews with key informants and focus group discussions at Headquarters and in decentralized offices 51. The Evaluation will ensure that stakeholders with diverse views will be consulted to ensure the assessment, findings and recommendations are based on a comprehensive understanding of diverse perspectives on issues, performance, and outcomes. Consultative workshops are envisaged at various stages of the Evaluation Process.

7. Phases and Deliverables

Phase Period Main Output Responsibility Literature Review of studies and evaluations on capacity development, PBEE/OE February-March 09 Draft Approach Paper undertaken inside and outside FAO

Inception Phase Evaluation Team March – April 2009 Inception Paper Leader

10 Desk Reviews Evaluation Team May-September 09 Assessment Reports

Country Case Studies and Evaluation Team Field Missions

Interviews and analysis by July-November 09 Country Reports the Evaluation Team at HQs

Draft of the Evaluation Evaluation Team Report (December 09) Draft Evaluation Report Leader

Final draft Evaluation Evaluation Team Report Evaluation Report and Leader MR presented to the January 2010 Draft of the Management Programme Committee Response (May 2010) Relevant FAO Divisions

8. Organization of the Evaluation

8.1 Evaluation Team

52. The evaluation will be conducted by a core team of experts led by an external independent consultant, who will have an extensive knowledge and experience on capacity development. 53. The core team will include two evaluation experts from the Evaluation Office who will, among other things, coordinate the field work. A consultant will also be hired to specifically assess normative products. 54. Country case studies will be carried out by local teams and coordinated by a core team member.

8.2 Management

55. The Evaluation will be managed by a Senior Evaluation Officer, assisted by another Officer. These Officers will serve on the core team.

8.3 Quality Assurance Mechanisms

56. In order to ensure the technical soundness of the evaluation, the following mechanisms are proposed: • Internal Reference Group (RG) which will serve as a sounding board at different stages of the evaluation and will provide feedback on the Approach Paper, the Inception Report and the draft Evaluation Report. Given its coverage at HQs, it is suggested to use the IDWG on Capacity Building as the RG, and complement its membership with adequate representation from decentralized offices, through focal points in the Regional and Sub- Regional offices. • Peer Review: A group of experts on CD will review the draft report when it is near final stage and provide feedback on the technical soundness of the conclusions and recommendations of the evaluation;

11 Annex 2 – Evaluation Methodology

1. The Inception Phase

1. The evaluation began with an inception phase from March to June 2009 with the objective of refining the scope of the evaluation, clarifying the analytical frameworks, developing the methodological tools and providing details on organisational aspects of the evaluation. 2. Inception work started with a literature review of key documents produced by other UN agencies including evaluation reports, the World Bank, the OECD/DAC, and other research and think-thank institutions, as the European Centre for Policy Development. The aim was to get an overview of state-of-the-art knowledge on the topic, and issues to be taken into account by the evaluation. The literature review was also used for the development of a set of good practices and lessons learned, against which to benchmark FAO CD activities. 3. As per terms of reference, the Evaluation was to cover Sub-Saharan Africa and FAO’s CD activities over the 2000-2008 period. On that basis, the Team conducted interviews at HQ (including within the context of an IDWG on Capacity Building meeting) and in- depth research of what CD work concretely means for FAO, to get a first sense of the type and nature of the activities to be covered, as well as the level and people the Organization targets. 4. The team prepared an evaluation matrix1 to be used as a reference tool. As such, for each evaluation criteria identified in the Evaluation’s Terms of Reference, it listed questions to be addressed, defined indicators, and indicated the main sources and data collection methods the Team would use. 5. Recognizing the complexity of the CD concept and the difficulties of measuring achievements, The evaluation drew on various methodologies to assess the results and the processes followed in FAO’s CD activities, including: logic models; participatory and systems approaches; and benchmarking against good practices and lessons. The different tools (described in details in section 2 below) were conceived to complement each other. 6. The Evaluation was led by an independent external expert, and managed by the Evaluation Service. In addition to the Team leaders, the core Team was composed of two sub-regional experts (one for East-Southern Africa and one for West Africa), six national consultants (to carry out the country case studies), Capacity Development Specialist (FAO’s normative work), and a resource person (to conduct a meta-synthesis of existing evaluations). The six country case studies were carried out by six national experts. The Evaluation was managed by FAO Evaluation office and two evaluation officers were involved in all stages of the evaluation process.

1 Enclosed at the end of this Annex

1 2. The Evaluation tools

7. The Evaluation drew on various methodological tools to assess both the results and the processes followed in Capacity Development activities. These included:

8.1. Documents Review

The review included a number of FAO documents and reports, namely: • Strategic and planning documents, among others, Medium Term Plans and Programme Implementation Reports, the Immediate Plan of Action for FAO Renewal (2009-11), the Strategic Framework for FAO (2010-2019), the Gender and Development Plans of Action (2002-07 and 2008-13), etc. • The various rafts of the FAO Corporate Strategy for Capacity Building; • Project documents, with special consideration to projects with a CD focus and, those highlighted by the FAO staff as success stories; • Documents provided by FAO staff, notably in relation to policy support, institutions strengthening, various training activities by technical departments, e- learning, research and extension work

The review extended to cover the most relevant web repository of FAO information and on-line documents, as well as an analysis of the structure, content and setting of the CD materials in the Organization’s Capacity Building Portal.

8.2.The inventory of CD Activities for the Overview2

The inventory aimed at identifying and understanding FAO’s CD activities at country level, through a systematic review of projects’ objectives and activities in the period under assessment (2000-2008). While the list of CD activities was kept broad on purpose (in line with the CD definition), an assessment of the “CD content” of each project on a scale from 1 to 5 was made on the basis of a set of combined criteria.3 The review was conducted in two stages: (i) A first analysis of the projects’ portfolio in ten countries to inform the scope of the Evaluation - by clarifying the type, modalities, and targets (what, how and for whom) of CD interventions - and assist in the selection of countries for the case studies (see below) and; (ii) a final draft including an inventory of country-based CD activities that formed the basis for the Overview. The FAO Field Programme Management Information System (FPMIS) was the main source of information, complemented by information from FAO divisions. The desk review covered the forty-eight countries in the Sub-Saharan region4, including national regional, global and inter-regional projects. With respect to the two latter, only activities with a large African coverage were taken into account. A total of some 3,000 projects were reviewed.

2 See Annex 7 3 The projects’ rating was based on a combination of the following criteria: i) the project explicitly aims at developing capacity; ii) CD activities form a major part of the project’s components; iii) CD interventions are thorough and multifaceted; iv) the whole CD process appears as participatory, but does not limit itself to sharing experiences and knowledge; v) the contribution by FAO does not only include the provision of equipments and material inputs; vi) the scope of the project is wide and resources allocated are significant; vii) learning events are not isolated exercises, rather pave the way to the implementation of new concepts and technologies.

4 This does not include those implemented in countries (like Comoros, Djibouti, Mauritius, Sao Tome, Seychelles) where the total budget for the period 2004-2009 was less than USD 1 million (Source FPMIS).

8.3. Interviews at Headquarters

The Evaluation used the Inter-Departmental Working Group on Capacity Building as its Reference Group. IDWG members were consulted throughout the evaluation process, both on an individual basis and as a group. The IDWG – CB assisted the Team in collecting information from various divisions of FAO and provided advice and feedback at different milestones of the evaluation process. In addition, the Team organized meetings with FAO staff on key areas (including on normative work, see below) to supplement information collected in the field. This included interviews with a few officers from the Outreach and Capacity Building branch (KCEF), the Emergency Division (TCE), the Office for Coordination and Decentralization (OCD), and the Gender, Equity, and Rural Employment Division (ESWD). A Gender Focal Point meeting was also organized to discuss the adequacy and effectiveness of FAO CD activities related to gender mainstreaming. In total, the Team interviewed 89 FAO staff at Headquarters. The list of staff consulted is included in Annex 5.

8.4.Field visits

Given the Evaluation’s mandate to focus on CD activities at country level, visits to the FAO Offices in the field (the Regional Office for Africa, the Sub-Regional Offices, and a sample of FAO Country Representations) and country case studies constituted the backbone of the evaluation methodology. The countries for case studies were selected on the basis of the following criteria: • geographical balance among sub-regions; • amount of resources allocated and ongoing projects in the period under consideration; • adequate coverage of main technical areas of FAO CD field activities; • adequate coverage of different types of CD activities (e.g. training, workshops, policy advice, strengthening networking and coordination mechanisms, organizational restructuring, development, dissemination of new agricultural techniques); • adequate coverage of three CD dimensions (individual, organisations, enabling environment); • exclusion of countries and activities already covered by recent evaluations5; The Evaluation Team then visited Burkina Faso, Ghana (West Africa), Uganda, and Kenya (East Africa), Malawi and Tanzania (Southern Africa). 8.4.1. Country cases studies The country case studies were carried out in two phases: • In the first phase, under the supervision of the regional expert, in each country a national consultant carried out four to five beneficiary assessments of a selected number of projects6. The main task was to collect information through participatory workshops,

5 Namely: Mozambique, Sierra Leone, Democratic Republic of Congo, and Sudan. Information from these country evaluations, as well as from selected project and program evaluations, was inserted as evaluation evidence through a meta-analysis (see below). 6 The selection of the beneficiary assessment by the Core Team, in consultation with staff at HQ and in the field, was based on a number of criteria, including: i) Explicit CD objectives and/or CD activities being a major part of a project’s components; ii) Adequate coverage of FAO main technical areas of interventions; iii) Project span and activities’ implementation status; iv) Significant allocation of resources.

3 focus discussions, surveys and interviews with both clients of CD activities and other stakeholders. To the extent possible, participants from CD activities were asked to report the Most Significant Changes that had occurred as a result of the project. These individual anecdotes were in some cases complemented by a short simple questionnaire to be responded to individually or by working groups at the workshop, as appropriate and depending on literacy levels and logistics. To ensure consistency in evaluation across projects and countries, checklists and guides were provided for adaptation to the local circumstances. Evidence from the interviews and the field visits was then put together into Beneficiary Assessments reports7, according to a standard format developed by the Core Team. • The second phase of the country case studies was part of the complementary field visits by the Core Team and focused on CD activities carried out with national Governments, regional organisations (where located in the selected countries)8, NGOs, CBOs and other implementing agencies, as well as educational and research institutions. Together with meetings with FAO staff in the Representations, the Team conducted individual and group interviews with all the stakeholders, trying to obtain information relevant to the systemic issues of FAO Capacity Development effectiveness and to answer the core evaluation questions as in the Terms of Reference. The country visits also included interviews with bilateral and multilateral agencies, to better understand the context and assess existing and potential partnerships. In a few cases (Burkina Faso and Ghana), one-day field visits were organized to supplement the information collected during the beneficiary assessments. The Team interviewed a total of 27 staff in FAO Country Representations, and 254 stakeholders/partners involved in FAO CD activities.9

8.4.2. Visits to the Regional Office for Africa and the Sub Regional Offices

In addition to the country case studies, the Team visited the Regional Office for Africa (Ghana), and all the Sub Regional Offices, namely SFS (Zimbabwe), SFC (Gabon), SFW (Ghana), and SFE (Ethiopia). The team interviewed a total of 40 FAO staff, with a view to understanding sub-regional priorities for CD as well as FAO’s capacity to provide its CD services and products.

8.5. Meta –synthesis of Country, Programme and Project Evaluations

A resource person to the Evaluation was tasked with the preparation of a paper10 summarizing findings and conclusions of relevant evaluations carried out by FAO over the last six years. The objective of the synthesis was to provide the Evaluation with insights on Capacity Development in specific sectors or areas of work of the Organization. It also broadened the base for informing the Evaluation by providing additional assessment of projects and programmes which were not covered by the country case studies or the project reviews. The meta-synthesis focused on the extent to which i) CD concept and practice had been assessed

7 See Annex 10 8 The following (sub)regional African organizations whose capacities FAO has worked towards developing were visited in the field: African Union (AU); the West Africa Economic and Monetary Union (UEMOA); the Comité Inter-Etats de Lutte contre la Sécheresse au Sahel (CILSS); the Economic Community of Central African States (CEEAC); the Comité régional des pêches du Golfe de Guinée (COREP); the Central African Forest Commission (COMIFAC), the Lake Victoria Fisheries Organization (LVFO). In addition, the Team met with staff from a few regional capacity development organisations such as FARA, ANAFE and RUFORUM. 9 See Annex 5 10 See Annex 9

4 in past evaluations of FAO’s CD activities; ii) FAO’s activities for CD support had addressed CD across different dimensions, sectors, functional capacities, software modalities and iii) FAO’s performance in developing African capacity to address the needs of the poor and the environment and how this could be done better in the future. A sample11 of 30 evaluations was included for review, 24 of which were country, programmes and projects evaluations, and another 6 were corporate evaluations. Each evaluation report was reviewed in detail and relevant evidence entered into a grid which served to systematize information on the issues of interest to the meta-synthesis, notably: i) record the various components of CD support used in FAO’s activities; and ii) determine the quality of FAO’s CD activities based on the questions and indicators of the Evaluation matrix.

8.6.Assessment of FAO’s Normative Products related to CD in Africa

A sample12 of normative products, covering a range of CD tools, including guidelines, manuals, CDs, e-learning tools, was selected from the CD Portal and in consultation with staff from technical units in Headquarters. An independent consultant was tasked with assessing the quality and relevance of content of these materials and also, where appropriate, their effectiveness in the field (including their accessibility to the targeted audience, their relevance to the objectives and the country contexts; the ability to be interactive, their potential to remain relevant over time or to adapt to change).The consultant interviewed 27 FAO staff members in HQ about the design, development and distribution of normative materials, with particular reference to FAO’s work in Africa. In addition to the desk review of the selected 31 normative products, producers and users of these materials were interviewed on their relevance and effectiveness in the case study countries by the core team.

3. Debriefing and Report Writing

8. Once the field missions were completed and the other methodological components finalized, the Team shared and discussed its preliminary findings with the IDWG on Capacity Building and, through a video-conference, with staff of the Sub-Regional Offices (Central and Southern Africa)13. All the officers who had provided background information and/or were interviewed were also invited to attend the presentation at Headquarters. 9. A first draft report was prepared and widely circulated to all FAO staff interviewed and/or concerned at Headquarters and in the field offices. Comments were taken in due consideration and clarifications made, to the extent they were relevant and deemed in line with the independent findings by the Evaluation Team. 10. The second draft was then reviewed by an Expert Review Panel14, which provided feedback on the technical soundness of the conclusions and recommendations of the Evaluation. The Panel was composed of six experts on Capacity Development, who were selected on the basis of their knowledge and experience in this area.

11 Evaluations were selected based on the following criteria: i) Focus on activities for which there was a clear intent to develop capacities, ii) Focus on African countries; iii) Adequate coverage of programme, project, country and corporate evaluations; iv) Adequate coverage of specific sectors or areas of work of FAO; v) Assessment of projects and programmes not covered by the country case studies or the project reviews. 12 The consultant developed some criteria to guide the final sample selection from the preliminary list prepared by the core Evaluation Team. These included materials: i) developed and/or used by FAO and its partners between 2000 and 2008; ii) two or more samples from each department; iii) illustrative of the quality of the materials produced and distributed by the department; and iv) intended for use in developing capacity, broadly defined, particularly in relation to Africans. 13 Staff from RAF and the other Sub Regional Offices was invited, but could not attend. 14 Short bios of the Peer Review Panel members are included in Annex 4.

5 Issues / Questions Indicators Data Collection Methods and Sources Relevance Project documents Verification in countries with relevant stakeholders: field Extent to which FAO has identified country capacity Explicit reference to background elements visits and interviews with key staff from FAO, 1.1 needs and reflected them in programme and project and national strategies/policies on CD in the Governments, regional, national and sub-national interventions and strategies project documents institutions Interviews with FAO staff in HQs Extent to which FAO interventions at field level are Clear definition of FAO's comparative FAO Project documents 1.2 based on an analysis of the organization's advantages and explicit reference to them in Interviews with staff from FAO and other (partner and comparative advantages planning documents non-partner) key actors in CD Explicit reference to partners' policies in FAO Project documents planning documents Interviews with staff from FAO and other (partner and Extent to which FAO interventions are coordinated 1.3 Agreements and other evidence of (formal non-partner) key actors in CD with other international and national CD initiatives and informal) partnerships at regional and Field visits country level Thematic study on partnerships Design Explicit reference at the three levels of Extent to which the design of CD initiatives has interventions and implications being Project documents taken in due account the needs at individual, pondered in project documents and other 2.1 Interviews with internal stakeholders in HQs and field organizational and policy level in order to identify background studies for development of offices CD priorities normative products and implementation of other CD activities Reference to PRSPs and other strategic/planning national policy documents Evidence of meetings with key stakeholders Verification in countries with relevant stakeholders: Field Extent to which CD initiatives are responsive to in the planning phase visits and interviews with key staff from FAO, demand and relevant to stakeholders' needs and in Reflection of meetings' discussion in project Government, regional, national and sub-national 2.2 particular whether they have taken into account the documents and other background studies for institutions interdisciplinary nature of the problems faced in rural development of normative products and FAO Project documents development implementation of other CD activities PRSPs and other strategic/planning policy documents Evidence of complementarities between the various FAO CD activities within a country and/or the region

6 Issues / Questions Indicators Data Collection Methods and Sources Evidence of meetings with key stakeholders Verification in countries with relevant stakeholders: Field Extent to which the CD needs have been identified in in the planning phase visits and interviews with key staff from FAO, a participatory way, tailoring the programme and Reflection of meetings' discussion in project 2.3 Government, regional, national and sub-national materials to different target groups and working with documents and other background studies for institutions them in development and delivery development of normative products and FAO Project documents implementation of other CD activities Reference to collaborative partnerships with national and sub-national actors and Extent to which CD activities have been designed to assignation of respective roles and FAO Project documents 2.4 promote greater ownership in the region, countries responsibilities Interviews with FAO staff at HQs and in the field and amongst client groups Mention and consideration of trickle-down effects in planning documents Extent to which the objectives of the initiatives have Ouputs and outcomes defined in relation to FAO Project documents and in particular any (explicit or 2.5 been clearly defined in terms of desired outcomes objectives not) logic framework Extent to which the CD design takes into account Evidence of links between inputs, expected emphasis on the three major goals of food security, FAO Project documents and in particular any (explicit or 2.6 outputs and outcomes, and overall poverty reduction and sustainable natural resource not) logic framework organizational objectives management Mention and evidence of adequate Extent to which cost-effectiveness and sustainability consideration given to cost-effectiveness and FAO Project documents 2.7 issues have been adequately pondered in the design sustainability issues in project and other Interviews with FAO staff at HQs and in the field and planning of projects and activities background documents Mention and evidence of adequate consideration given to gender and other Extent to which the CD design takes into account cross-cutting issues in project documents and Project documents (incl. budget) 2.8 gender and other cross-cutting issues other background planning documents Interviews with FAO staff at HQs and in the field Budget resources specifically allocated to gender and other cross-cutting issues The monitoring and evaluation framework designed Evidence of sound M&E framework for Project documents 2.9 for CD initiatives is of good quality and adequate project and other CD initiatives Any M&E tool for CD initiatives other than project Efficiency Extent to which FAO has developed its own in-house Level of FAO Human and financial Budget and Implementation reports 3.1 capacity and ensure appropriate resources to deliver resources allocated to CD initiatives across Number of staff (and time) devoted to CD activities CD throughout its technical units Departments

7 Issues / Questions Indicators Data Collection Methods and Sources Extent to which the current institutional set up allows Evidence of role played by the IDWGs 3.2 for adequate and efficient coordination among (timing and frequency of meetings) Interviews with key staff at HQs different department units at Headquarters Evidence of other cooperation mechanisms Evidence of roles and responsibilities shared Timeliness and degree of coordination in the between HQs and decentralized offices in management of projects and activities between HQs Interviews with key staff at HQs and in field offices 3.3 project management and decentralized offices - Adequacy and continuity Implementation reports Timing and frequency of contacts between in the two-way communication flow HQs and DOs Effectiveness Assessment of normative CD products and Peer review The quantity, quality and appropriateness of types or of their quality Evidence of quality and appropriateness of 4.1 format of CD materials in regard to the needs of the Verification in countries with relevant stakeholders and format and target of CD materials target group beneficiaries Implementation reports Implementation reports Awareness and use of FAO's normative products and Evidence of use by and utility to key 4.2 Field interviews with key beneficiaries knowledge sharing tools beneficiaries Surveys and participatory workshops Number of regional and national meetings Implementation reports FAO was a facilitator for Verification in countries with relevant stakeholders and Role and effectiveness of FAO as a facilitator of 4.3 Qualitative evidence and perceptions of FAO beneficiaries: Interviews with policy makers, regional and national technical and policy initiatives effectiveness in playing this role (incl. any representatives of regional organizations and follow up to the initiatives) Governments Quality and adequacy of content of training provided by FAO Evidence of actual use of notions and Implementation reports FAO's performance and effectiveness as a provider knowledge shared through training and Verification in countries with relevant stakeholders and 4.4 of technical assistance and training in the areas technical assistance beneficiaries: Interviews with beneficiaries of training within its mandate Evidence and perceptions over FAO's activities comparative advantages as a training and Surveys and participatory workshops technical assistance provider in the areas within its mandate Evidence of use made of capacities Verification in countries with relevant stakeholders and FAO's performance and effectiveness as a provider 4.5 developed beneficiaries through interviews of CD interventions Evidence of ownership and follow-up to CD

8 Issues / Questions Indicators Data Collection Methods and Sources interventions at individual, organizational Review of relevant decision-making and policy and and institutional levels. strategy documents

Review of resources (investments) made to ensure sustainable results of the CD interventions FAO's performance and effectiveness in partnership Verification in countries with relevant stakeholders and Evidence of quality, role and value added of 4.6 for developing and enhancing the capacities of local beneficiaries: Interviews with partners and key FAO in partnering on CD activities individuals and organizations to deliver CD stakeholders Overview of main technologies adopted by FAO and other main players involved in CD activities, including evolution over time FAO's performance in developing and promoting Evidence of adoption and implementation of 4.7 Verification in countries with relevant stakeholders and new and better technologies for CD new technologies beneficiaries: Interviews with users of technologies for CD

Impact Qualitative evidence of changes resulting MSC Techniques applied to field visits Impact of CD activities at the levels of individuals 5.1 from CD activities at individual, Interviews with beneficiaries (farmers, Govt and other and organizations and on the enabling environment organizational and institutional levels regional/national institutions' staff) Sustainability Qualitative and quantitative evidence of Extent to which FAO assistance has been delivered MSC Techniques applied to field visits capacities of client groups and organizations 6.1 in ways that enhance the capacities of client groups Interviews with beneficiaries (farmers, Govt and other being enhanced (both actual and perceived and organizations regional/national institutions' staff) differences) Qualitative evidence of regional and national MSC Techniques applied to field visits Extent to which FAO has supported and encouraged actors leading development strategies and 6.2 Interviews with beneficiaries (farmers, Govt and other leadership by regional or national actors policies, following specific FAO regional/national institutions' staff) intervention(s) Quantitative evidence of trained people Extent to which trained actors have been or are leading the implementation of development involved in scaling out their CD activities at regional MSC Techniques applied to field visits strategies and policies 6.3 and national levels to further disseminate the Interviews with beneficiaries (farmers, Govt and other Qualitative evidence of knowledge and skills knowledge and skills acquired (are the skills being regional/national institutions' staff) acquired thanks to specific FAO used?) intervention(s) being of use

9 Issues / Questions Indicators Data Collection Methods and Sources Quantitative and qualitative evidence of MSC Techniques applied to field visits Social capital and networking created through the contacts established within and among 6.4 Interviews with beneficiaries (farmers, Govt and other CD activities countries following a specific FAO regional/national institutions' staff) intervention(s) Extent to which the support provided and the lessons learned by working together have been integrated Qualitative evidence of changes in policies MSC Techniques applied to field visits 6.5 into policy and institutional changes, planning, and and policies' implementation being Interviews with beneficiaries (farmers, Govt and other implementation (i.e. are there steps to ensure attributable to FAO specific intervention(s) regional/national institutions' staff) retention of institutional memory) Quantitative and qualitative evidence of FAO MSC Techniques applied to field visits Extent of adaptation and further dissemination of CD products and projects being considered as Interviews with beneficiaries (farmers, Govt and other 6.6 approaches and information/training materials at good benchmark and adopted in other regional/national institutions' staff) and other actors at local, national and regional levels contexts national levels involved in CD programmes Partnerships Qualitative evidence of cooperation mechanisms being (formally and informally) Interviews with UN Agencies and FAO staff in country established among UN agencies Efficiency and effectiveness of cooperation with UN Thematic Study on Partnership Cross-references to roles (linked to 7.1 agencies, within UN Country Teams or the Delivery UNDAF, PRSPs, DoU planning documents (plus comparative advantages) and activities of as One Framework when relevant eventually other UN agencies' country strategies) UN agencies within respective policy, FAO project documents projects and other CD background documents Qualitative evidence of cooperation Efficiency and effectiveness of cooperation with mechanisms being (formally and informally) Interviews with staff from other multilateral, bilateral, multilateral, bilateral, regional and national actors. established with multilateral, bilateral, regional and national bodies Extent to which the latter have been adequately regional and national actors Interviews with staff from NGOs, representatives from 7.2 identified, including non-state stakeholders such as Qualitative evidence of background analysis the private sector and community groups CSOs/NGOs, farmers/community groups and the and inclusion of private sector organizations Thematic Study on Partnership private sector and CBOs into FAO project and non-project FAO projects documents activities

10 Annex 3 - The Team

Professor Kay Muir Leresche, Team Leader

Professor Kay Muir Leresche, South African, is an Agriculture and Natural Resources economist, specialized in policy analysis. After more than twenty years as a Professor at the Department of Agricultural Economics and Extension of the University of Zimbabwe, Professor Muir Leresche has since 2002 been working as a self-employed consultant on rural services delivery, land reform, agro-biodiversity and sustainable use of resources. Professor Muir Leresche has worked closely with regional associations and UNDP on capacity development issues, and established the SEMCA (Sustainability, Education, and Management of Change in Africa) community of practice. In the past few years, Professor Muir Leresche has also collaborated with the FAO Plant Production and Protection Division and the Evaluation Service on the Evaluations of the FAO-Netherlands Partnership Program on Policy Development and the FAO-Norway Program Cooperation Agreement.

Professor Malcolm Blackie, Regional Expert Eastern and Southern Africa

Professor Malcolm Blackie, Zimbabwean, is an expert on agricultural development in Southern and Eastern Africa. After having served as lecturer on farm management in the Pacific and Head of the Department of Land Management at the University of Zimbabwe, Professor Blackie has then worked as senior scientist in the Agricultural Sciences Division of the Rockefeller Foundation in Southern Africa. In the past ten years, Professor Blackie has provided backstopping and advice to various development activities in the region, including to the Regional Forum for Development in African Agriculture (RUFORUM) and, among others, to the Governments of Tanzania and Malawi in setting up public/private partnerships for input supply to smallholders.

Mr. André Damiba, Regional Expert West Africa

Mr. André Damiba, Burkinabe, is an international consultant specialized on small-business enterprises and community development. In the last twenty years, Mr. Damiba has worked in more than twenty African countries supporting the formulation of development policies and programmes, and took part in several international assignments for various UN agencies, the European Commission, and the West African Development Bank. Among these, Mr. Damiba was the Regional Coordinator of the UNDP Program “Réseau Afrique 2000” on community integrated development, Principal Technical Advisor for the UNDP Capacity Building Program in Cameroon, and Team Member of the IFAD Country Program Evaluation in Niger.

Mr. Mark J. Stiles, Independent Consultant, Assessment of FAO’s Normative Products

Mr. Mark J. Stiles, Canadian, holds a Master of Adult Education at the St. Francis Xavier University (Nova Scotia). Mr. Stiles has more than thirty-years of experience in program evaluation, organizational capacity assessment, gender equality, training, and institutional strengthening. Mr. Stiles has recently been the Team Leader of a major review of UNESCO’s capacity building function, and evaluated the UN ESCAP approach to Capacity Development in relation to its research and analytical work, its normative role and its technical cooperation assistance to Member countries. Mr. Stiles has important competences on gender issues, and lately trained CIDA staff and partners in gender-sensitive Results Based Management.

1 Ms. Cristina Lopriore, Independent Consultant, Meta-synthesis of evaluation

Ms. Cristina Lopriore, Italian, MSc in Biomedical Sciences (University of Leiden) and MSc in Nutrition (London School of Hygiene & Tropical Medicine), is a specialist in public health nutrition and food security. She has over 10 years’ continuous professional experience in the area of nutrition assessment, analysis and planning, in both humanitarian and development contexts. Her field experience focuses especially on the design, implementation and management of maternal and child nutrition interventions. She has served full-time with research institutions and NGOs, and has collaborated extensively as an international consultant principally to UN agencies. In the last four years, Ms. Lopriore has collaborated with FAO as Training and Liaison Officer for the EC/FAO Food Security Information for Action Programme. In this context, Ms. Lopriore provided technical support to the design, implementation and evaluation of training and designed normative materials on the assessment and analysis of nutrition and food security, with a special focus on the development of associated indicators and tools.

Ms. Rachel Sauvinet-Bedouin, FAO Office of Evaluation, Evaluation Manager

Rachel Sauvinet-Bedouin, French, MSc in Economics (University of Paris IX Dauphine) and a post master degree in development economics (University of Paris X Nanterre), has more than 20 years of professional experience among which a large part related to evaluation. She is a senior evaluation officer in FAO Office of Evaluation, where she leads and manages major complex and strategic evaluations and report directly to Senior Management and Governing Bodies of the Organization. She also works on the development of evaluation tools and standards, including in relation to the accountability of humanitarian assistance, through her active participation to various peer networks. Over these years, she has been working for multilateral agencies, specializing in the fields of rural development, food security and vulnerability information, analysis and policy and disaster preparedness. Prior to her joining FAO, she was project economist in the Agriculture Department of the Asian Development Bank.

Ms. Anna Guerraggio, FAO Office of Evaluation, Evaluation Officer

Ms Anna Guerraggio, Italian, holds an MSc in International Institutions Management (Bocconi University) and an MSc in Development Management (London School of Economics). Ms. Guerraggio started her career as Training and Education Officer at the Institute for International Political Studies (ISPI) and as Researcher at CARE International in London. Ms. Guerraggio joined FAO in 2006 as Research Assistant to the Independent External Evaluation (IEE). Since 2008, she works as Evaluation Officer in the Office of Evaluation with competences on economic analysis, capacity development, and gender.

2 Evaluation of FAO’s Activities on Capacity Development in Africa

Annex 4

The Expert Review Panel

Background

1. At its October 2008 session, the FAO Programme Committee selected the “FAO’s work on Capacity Development (CD) in Africa” as one of the priority areas for evaluation. As part of the Evaluation quality assurance process, a panel of six experts1 on Capacity Development reviewed the draft report at its final stage, identified gaps in the analysis and provided feedback on the technical soundness of the conclusions and recommendations of the evaluation. 2. The Expert Panel met in FAO Headquarters, from 1-3 February 2010. After a short presentation of the draft Evaluation Report by the Team Leader, the Panel had a first discussion with the Evaluation Team, followed by a few meetings with key FAO managers2 who gave their views on Capacity Development in FAO, also with relation to what is contained in the draft report. 3. The following paragraphs outline the main conclusions reached by the Panel on Capacity Development in FAO and on the relevance, appropriateness and feasibility of the Evaluation report’s conclusions and recommendations for the enhancement of capacities in Africa.

General Observations on Capacity Development in FAO

4. FAO has gone through a period of reforms, since the IEE. This is well intended, but it should be acknowledged, it puts considerable stress on resources. The Panel noted some reticence in commitment to CD. Perhaps, this occurs because it comes in the midst of a whole set of reforms. FAO needs to be part of the international CD community, and to think through new modalities of partnerships to re-position itself vis-à-vis other stakeholders and the countries themselves 5. The Panel is concerned that emphasis on CD in FAO appears to be driven from external sources and from senior managers, with mediocre support at middle management level where the actual changes would be effected. FAO should in any case grasp this opportunity, even though externally motivated, and position itself to be recognized for its CD work. To create the required ownership within the Organization however, FAO will need to be much more proactive. For the change to take root and be internalised, and to overcome the perceived resistance will take time and resources. 6. A basic assumption of the expert panel is that in theory FAO is facing three corporate options in terms of its CD work; (1) it can provide relatively modest CD support to improve technical functions in the form of training, mentoring, organizational restructuring; (2) FAO can provide a more complex range of CD services including overall capacity assessment, organizational reforms and assistance to complex sector reforms; (3) FAO could make CD its main core function issue and offer a more complex range of CD services for many types of organizations in many different sectors including contributions to national capacity development strategies and partnerships with other international development agencies. 7. The implication of the options in 5 above for FAO is that: Option1: means no current action with change only in the medium to long term,

1 Short bios of Panel Members are included at the end of the Annex 2 The Expert Panel met with the following FAO staff members: David Conte, Andrew Shepherd, Shivaji Pandey, Parviz Koohafkan, Richard China, David Phiri, Abdul Kobakiwal, Carel Kallens, Eve Crowley, Ilaria Sisto, Denis Herbel, Stephen Rudgard, Sally Berman, and Cecilia De Rosa.

1 Evaluation of FAO’s Activities on Capacity Development in Africa

• Option2: incremental change, and • Option 3: major structural changes in FAO. 8. The view of the expert panel is that FAO concentrate building its capabilities to achieve the second option above. 9. The expert panel thinks that FAO does not need to make CD their main core function as the implication would be a major structural change within FAO’s mandate 10. The Panel is concerned that emphasis on CD in FAO comes from external sources and senior managers, with less support at middle management level where the actual changes will be effected. FAO should in any case grasp this opportunity, even though externally motivated, and position itself to be recognized for its CD work. However, to create the required ownership within the organization, FAO will need to be much more proactive. For the change to root and to overcome resistance, it will take time and resources to build ownership. 11. The IDWG institutional system for coordination should be given a stronger mandate to push the agenda for FAO to be fully involved in CD activities. It is apparent, however, that currently, there is a general lack of advocacy for CD in the Organisation, unlike other cross-cutting issues such as gender and climate change. It has to be recognised that mainstreaming CD is more difficult and as is handled now is perceived to impose an extra burden on managers. To encourage its adoption therefore, a way would have to be found to lower transaction costs. In support of this, FAO needs to have strong CD Focal Points in every Department. To drive the change that gives CD more prominence will require real champions in the Organisation beyond mere coordination. Effectively mainstreaming CD into organizations takes an evolutionary path and requires significant effort and motivation to be successful. For CD mainstreaming, the Organization has to consider what the implications are in terms of additional human resources required, as well as to address the tension between promoting CD and using an RBM system. The Panel expresses surprised that there are no significant and specific funds allocation to CD, although it has been declared a core function. Without the required resources, CD is unlikely to be meaningfully incorporated into staff work plans. The findings and recommendations of this Evaluation should further strengthen the importance of FAO specifically addressing staff motivation (incentives) to guarantee incorporation of CD into projects and programmes. 12. The Panel appreciated the work so far done on the CD Strategy. Yet, it was clear that the Strategy would bear further strengthening, by more explicitly incorporating effective approaches to partnership, networking and advocacy.. The point made by the Evaluation that national planning strategies and mechanisms offer critical entry points for CD is a key factor. Fragmentation and lack of coherence of FAO support especially at the country level can be counter-productive, with too many projects not linked together. 13. The Panel noted that there appears to be a disconnect between CD and knowledge management. FAO has the capacity to generate knowledge, especially in the discharge of its normative mandate. It has not however been successful in transferring same, not even in partnership with other stakeholders. It has to be strongly stated that CD is ultimately building learning capabilities which goes beyond just knowledge management, and the Institution has to worry about the ultimate use of the knowledge it generates. The evaluation makes these points and the Panel would like to re-emphasize that it is unnecessary to reinvent the wheel as to the most effective process for success in this regard. In many instances it is possible to tap into the wide body of existing knowledge on CD. 14. As is often the case in other UN Specialized Agencies, the development of “soft skills” tends to be neglected, and to be not highly rated, with prominence given largely to technical expertise. The Panel observed that the skills needed for real CD therefore do not appear to be appreciated in the Organization. FAO can still be very technical while at the same time combining this with a different approach with sustainability in mind. The Evaluation makes these points but it needs to place more emphasis on them.

2 Evaluation of FAO’s Activities on Capacity Development in Africa

15. In order to provide effective motivation for CD and to encourage FAO staff to put emphasis on the process which helps to build soft skills, ownership and sustainability, FAO will finally need to address the best way of assessing the extent to which staff have given attention to CD, and to devise a reward system for those who comply..

General observations on the Report

16. The Panel would like to recognize first the enormity and difficulties of undertaking an evaluation of all FAO CD work in Africa. This is a broad cross-cutting issue which needs a long gestation period to register impact. Effective evaluation of CD is something therefore the global community is still struggling with, both in terms of content and methodology. 17. Bearing this in mind, the Panel acknowledges that the report is well balanced and fair, and meets appropriately the requirements as outlined in the Terms of Reference.. 18. The report is timely and relevant, and is supportive of the call for focus on CD highlighted in the FAO Independent External Evaluation (IEE), and the high priority as pointed out to the Panel in meetings with FAO staff. This is buttressed as well by the prominence CD is accorded on the international development agenda.. 19. In this vein, the Panel wishes to draw attention to some weaknesses in the report. In several areas, topics were not treated with the kind of depth which would allow for the drawing of definitive conclusions. This could be due to resource and time constraints, or indeed limitations imposed by the remit of the evaluation itself. The language and structure of the report would not also generate passion, being measured and basically non-controversial, and not provocative enough to attract the kind of attention the topic deserves. This could have been cured by frequent references to, or evidence from the country reports to substantiate findings in the report. Given the ongoing internal debate on CD in FAO, it would be also useful to clearly draw attention to the value addition the report lends to the process of policy setting, with clear proposals for innovation, and identifying more strongly particular areas of weakness which call for stronger action. 20. The report needs to present the Organisation with strategic choices, with different scenarios as to how to ensure that CD is put in the centre of programming. The report could have made proposals which would help FAO to think more out of the box, with guidance on the most effective way the CD Strategy should be implemented; in particular on ways of planning, priority setting and allocation of resources. It is important that CD is not paid only lip-service within the organization, with business continuing as usual. For example the following options might have been presented by the Evaluation, with justification: • no current action, with change contemplated only in the medium to long term, • gradual incremental change, and • major structural changes in FAO. 21. In the opinion of the Panel, major structural change may be desirable but would be inappropriate at this stage. The gradual incremental change option is what the Panel thinks the Evaluation should have strongly recommended, clearly articulated and expanded upon.

General observations on the Report 22. The Panel appreciates that the FAO committed resources to carry out evaluation on the topic. 23. The Evaluation methodology is appropriate, given where FAO is and the use to which the report will be made. The country and beneficiary assessments reports are a good way to support findings, and more reference to these should be included. 24. The use of national consultants for the evaluation is appreciated since it ensures incorporation of local knowledge and expertise.

3 Evaluation of FAO’s Activities on Capacity Development in Africa

25. The main report is clear, succinct, focused and to the point. 26. 27. The Report is sufficiently forthright in drawing attention to some of the weaknesses. 28. 29. The Report, unlike in the case of some other reports commissioned by FAO seems to have stimulated a reaction in house with staff taking the time to comment. 30. The Panel however observes that the selection of countries might not have adequately reflected fully the reality on the ground in spite of the appropriateness of the criteria used for selection. This is because the countries selected appear to be among the more successful in Africa. The evaluation’s conclusions would be more robust had some relatively weaker countries where the context and CD needs are different been included in the list of case studies. 31. It would have been informative for the report to have presented a comparative analysis which would have rated the performance of FAO in CD with respect to UN and other similar organisations. 32. The report also appears not to have been influenced enough by the current political and economic debate in Africa. This is a major weakness since the effectiveness of CD is dependent on the socio-political environment of a country. Some of the country reports attempt such analysis and present relevant evidence, but unfortunately this is not carried into the report. It is necessary for this to be corrected by incorporating such elements from the Beneficiary Assessments and Country Reports.

Specific Recommendations for amendments to the Report

33. Recommendation 2: Change incentives to clarify that it is not financial incentives being referred to. Rather state specifically that CD needs to be incorporated into responsibilities, work programmes and performance appraisals. In addition, the recommendation should include HQ staff as well. 34. Recommendation 3: The Panel feels that having an FAO contact point for project activities is not enough. It is institutional networking, not only people, and partnerships that count. This can also be reinforced in the recommendation relating to the distribution of normative work. 35. Recommendation 4: The Panel feels that the wording on this recommendation needs to be clarified. All staff and not only senior management should be the target for the guidelines. In addition, guidelines should also stress the importance of adequate assessment of national priorities in CD, as CD efforts should be demand driven. It is important for FAO to engage in capacity needs mapping in Africa to direct intervention points and strengthen support. 36. Recommendation 6: Capacities to implement policies are equally important, as planning and drafting. There is need for an operational plan to implement strategies. 37. Recommendation 10. The Panel would like to see an additional recommendation under Section F. The Panel makes a strong recommendation that FAO needs to invest more resources into agricultural distribution, and not , and not be so lopsided in favour of production. It should also provide more support to CD for markets and trade issues which have currently assumed great importance on Africa’s development agenda. Crop production and agribusiness should be tackled in tandem. Marketing is one of the pillars of CAADP which FAO is involved in strengthening, and this continues to require support in terms of capacity development. This point should be emphasized throughout the report.

Priorities for CD in Africa The following suggestions are also offered by the Panel for consideration by FAO:

4 Evaluation of FAO’s Activities on Capacity Development in Africa

38. Programmes should cover the full value chain, with emphasis on markets and trade as well. 39. It is necessary to strengthen the capacities of individuals, institutions and systems to empower them with the ability to manage any eventuality in the often volatile and fluid changing physical, social and economic environment which Africa often has to grapple with. . 40. People, including public servants tend to be mobile in Africa, with serious implications for institutional memory-a situation which demands that CD efforts should not target individuals exclusively. CD approaches must aim to strengthen as well organizations and institutions not only of government but also of the private sector and civil society. . 41. It is essential to ensure that all knowledge produced is accessible to African countries, is packaged in a user-friendly manner and is widely used. This is critical and is an important component of FAO’s mandate, particularly as the Organization generates so much cutting edge knowledge and information. It becomes increasingly important to engage with local partners to develop knowledge relevant to the actual needs and demands of beneficiaries as assessed through consultations, especially in light of the reduced FAO field presences..

Procedural Issues

42. The Panel would like to advise the FAO Office of Evaluation to revise the Expert Panel mechanism by involving the Panel early in the evaluation process, in order that members can have better appreciation of what the exercise entails. There could for example be an early meeting to enable Panel members to comment on the Inception Report.. The amount of time (2.5. days) allowed for the meeting of the Expert Panel on the whole is sufficient. The use of the time and the agenda however need to be reorganised to enable the Panel to have more time for discussions with the Evaluation Team, and also for members to formulate their independent comments and recommendations.

5 Evaluation of FAO’s Activities on Capacity Development in Africa

Expert Panel Members Bios

Ms. Mary Chinery Hesse, former Chief Advisor to the President of Ghana and Permanent Secretary of the Ministry of Finance and Economic Planning in Ghana, has a long and distinguished career as civil servant in her country and at the United Nations. Ms. Chinery-Hesse was the first African woman appointed at USG level in the United Nations, as Deputy DG of the International Labour Organization. Ms. Chinery-Hesse also served as Chairperson of the UN Consultative Committee on Programme and Operational Questions for several years, and of the Commonwealth Expert Group of Eminent Persons on Structural Adjustment and Women. Among various high level commitments, Ms. Chinery Hesse is currently a Member of the Board of the Global Humanitarian Forum in Geneva.

Mr. Peter Morgan is an independent consultant specialized on capacity and institutional development. Mr. Morgan started his career as Assistant Secretary for Agriculture in the Government of Kenya, and then worked for more than 20 years for the Canadian International Development Agency (CIDA) up to level of Senior Director of the Organization’s Area Coordination Group. Mr. Morgan has an extensive experience on capacity development issues, within ECDPM and as free lance consultant, involved in the review of several CD Programs for UN agencies and bilaterals. Mr. Morgan has also a long record of studies on organizational assessment and analysis, and published a number of articles and papers on the topic. Since 2006, Mr. Morgan is a Member of the Capacity Development Advisory Panel to AusAID.

Ms. Ndey-Isatou Njie has thirty years of experience in environment, and she currently heads the Capacity development team in the Regional Service Centre in Dakar in Senegal. Prior to this, Ms. Njie was the UNDP Capacity 2015 Coordinator for Africa and Capacity Development Adviser to UNDP country offices in integrating capacity development in their programming processes. Ms. Njie also served as the Regional Coordinator for UNDP Capacity 21, and provided support to countries in sub-Saharan Africa in environment and sustainable development planning and implementation processes. Ms. Njie worked as a Strategic Planning Adviser for Africa for the World Conservation Union and served as the Executive Director of the National Environment Agency in the Gambia.

Professor Mandivamba Rukuni is Founder and Director of the Wisdom Afrika Leadership Academy Harare. A graduate of the University of Zimbabwe, Prof. Rukuni’s career over the past 25 years has largely been as an academic, up to the level of Dean of Agriculture at the University of Zimbabwe. His most substantive academic work has been in the areas of food security, institutional development, smallholder irrigation development, agricultural and R&D policy, land tenure and community based natural resources management. Prof. Rukuni has also been a consultant for a wide array of development assistance organizations (including the World Bank, USAID, and CIDA) and, after 2000, Regional Director for Africa Programs with the Kellogg Foundation in South Africa. Prof. Rukuni has served on several public sector boards in Zimbabwe including the Grain Marketing Board, the Agricultural and Rural Development Authority, and as chair of the Agricultural Research Council.

Dr. Paul Thangata is a Research Fellow with the International Food Policy Research Institute (IFPRI) in the Knowledge, Capacity, and Innovation Division. He conducts research on organizational efficiency and effectiveness of agricultural R&D, and supports capacity building and knowledge management to strengthen human and institutional capacity of NARS including universities and farmer organizations/ cooperatives. Dr. Thangata has over 15 years experience directly related to agriculture R&D management and food security programs. He began his career in 1988 as a Research Assistant with the Department of Agricultural Research, Ministry of Agriculture, in Malawi. He later joined the International Crops Research Institute for the Semi-Arid Tropics and later the World Agroforestry Centre in Zimbabwe. In 2006, he joined the SADC Secretariat in Botswana and the Centre for Agricultural Research and Development for Southern Africa

Dr Aissetou Drame Yaye is the Executive Secretary of the African Network for Agriculture, Agroforestry and Natural Resources Education (ANAFE). From 1987 – 1994, Dr. Yaye worked for Forest Management/Development Projects in Senegal and Niger and as a Trainer in Entomology at the

6 Evaluation of FAO’s Activities on Capacity Development in Africa

Regional Training Centre of AGRHYMET in Niger. In 2000, she was appointed as a Lecturer on Forest Management and Entomology at the Faculty of Agriculture of the Abdou Moumouni University of Niamey (Niger) before taking a sabbatical leave to join the Forum for Agricultural Research in Africa (FARA) in Ghana where she served as Programme Officer of the Strengthening Capacity for Agricultural Research and Development in Africa programme. Among others, Dr. Yaye is a Member of the Advisory Board of Africa-US Higher Education Initiative, the Advisory Committee on Science and Technology of the Technical Centre for Rural and Agricultural Cooperation, and the African Women in Science and Engineering (AWSE).

7 Annex 5 – List of People Met

FAO HQ1

Name and Service Achouri, Moujahed (FOMC) Kenny, Mary (AGNS) Ahmed, Mariam (UNC) Khan, Daud (OCDO) Ahmed, Shukri (ESTG) Kidane, Weldegabher (TCAS) Allara, Manuela (AGPP) Kidane, Weldeghaber (TCAR) Alvarez, Isabel (NRRD); Kienzle, Josef (AGST) Bamba, Madhy (TCOS) Klompenhouwer, Yves (OCDC) Basharat, Ali (OCDD) Kobakiwal, Abdul (TCOS) Bellu, Lorenzo (TCAS) Kueneman, Eric (AGPC) Berman, Sally (KCEF) Laub, Regina (ESWD) Besbes, Badi (AGAP) Maetz, Materne (TCAS) Bessy, Catherine (AGNS) Mangstl, Anton (KCED) Bianchi, Gabriella (FIMF) Metzner, Rebecca (FIEP) Blum, Magdalena (NRRR) Moon, Richard (AFHT) Boutrif, Ezzeddine (AGND) Morgera, Elisa (LEGN); Brunet, Fran (TCOM) Mrema, Geoffrey (AGSD) Bunning, Sally (NRLA); Mueller, Alexander (NRD) Burchi, Stefano (LEGN); Munro-Faure, Paul (NRLA) Caprazli, Kafkas (ESSS) Nadeau, Andrew (KCEF) Charrondiere, Ruth (AGNA) Nadeau, Andrew (KCEF) China, Richard (TCAD) Nichterlein, Karin (NRRR) Cistulli, Vito (TCAS) O’Farrell, Claire (NRRR) Clarke, Renata (AGNS) Otte, Joachim (AGAL) Colbert, Patricia (ESWD) Phiri, David (TCAS) Conte, David (PBEP) Price, Thomas (ODGS) Crowley, Eve (ESWD) Pulvenis, Jean Francois (FIED) DiBiase, Dominique (TCAP) Raswant, Suzanne (TCER) Doyle, Baker (AGSF) Ratsimba, Mahazosoa (TCOM) Egal, Florence (AGNP) Riemenschneider, Charles (TCID) Frattini, Rosana (KCEF) Rocchigiani, MariaGrazia (KCEF) Fredrix, Marjon (AGPP) Rudgard, Stephen (KCEF) Gambino, Regina (AFSP) Santacoloma, Pilar (AGSF) Gasperini, Lavinia (NRRR) Scaglia, Jean-Alexandre (TCEO) Gommes, Rene (NRCB) Senahoun, Jean (ESTG) Grainger, Richard (FIES) Settle, William (AGPP) Greboval, Dominique (FIEP) Sisto, Ilaria (ESWD) Guei, Gouantoueu (AGPS) Smulders, Mark (ESAF) Heilandt, Tom (AGNC) Souvannavong, Oudara (FOMC) Herbel, Denis (ESWD) Steinfeld, Henning (AGAL) Hinrichs, Angela (TCER) Thomas, Laurent (TCED) Jacqueson, Patrick (TCER) Trine, Francoise (ESAF) Jones, Jeffrey (AGPP) Trossero, Miguel (FOIP)

1 This list includes also the Members of the IDWG on Capacity Buidling and those FAO staff consulted via email Josserand, Henri (ESTG) Van der Wulp, Harry (AGPP) Jutzi, Samuel (AGAD) Vidar, Margret (LEGN); Keita, Naman (ESSS) Weibe, Keith (ESAC) Kenmore, Peter (AGPP)

Burkina Faso

Name Position Institution Amadou Keita Enseignat chercheur 2ie Goupe ETSHER/EIER Koffi Séwa Da Silvera Ingénieur géologue 2ie Goupe ETSHER/EIER Henriette Ouédraogo Association Ragussi - Club des Productrices Présidente du Beurre de Karité Biologique Banque Mondiale Souhaieb Kahyati Chargé de Programmes Bureau de Coopération italienne Assétou Nikiéma Club des Productrices du Beurre de Karité Présidente Biologique Sebastien Ouedraogo Club des Productrices du Beurre de Karité Conseiller Biologique Dramane Coulibaly Coordinateur PRA/SA-LCD-POP Comité Inter-Etats de Lutte contre la DEV Sécheresse au Sahel Abdou Karim Keita Comité Inter-Etats de Lutte contre la Expert PRA/SA Sécheresse au Sahel Abdoulaye Saley Moussa Comité Inter-Etats de Lutte contre la Expert Securité Alimentaire Sécheresse au Sahel Mohahmadou Hardaoun Comité Inter-Etats de Lutte contre la Expert GRN/FRP Sécheresse au Sahel Bernard Dembele Comité Inter-Etats de Lutte contre la Expert Nutrition Sécheresse au Sahel Ibrahim Lumumba Idi Issa Comité Inter-Etats de Lutte contre la Expert PRA/SA Sécheresse au Sahel Coudy Ly Wane Comité Inter-Etats de Lutte contre la Expert Nutrition Sécheresse au Sahel Sibiri Jean Ouedraogo Comité Inter-Etats de Lutte contre la Expert GRN/ Fertilité Sécheresse au Sahel Nétoyo Laomaibao Comité Inter-Etats de Lutte contre la Chef DREAM Sécheresse au Sahel Dao Bassiaka Président Confédération paysanne du Faso Oumar Seydina Traoré Directeur DADI Kaboré Kardiatou DGRH Do Christophe Ouattara aquaculture DGRH Désiré Coulibaly Législation DRGH Kaboré Etienne Direction de l’Organisation des Producteurs et Directeur de l’Appui aux Institutions Rurales Abel Nonguierma chef de service organisations Direction de l'Organisation des Producteurs et paysannes de l'Appui aux institutions Rurales Adama Dolkoum Directeur Direction des Forêts Gaston Georges Ouédraogo Direction générale de l’Agence de Promotion Directeur général des Produits Forestiers non Ligneux Joachim Ouédraogo Direction générale de la Conservation de la Directeur général Nature Damos Poda Agence nationale des produits forestiers Moussa Kaboré Directeur DPSA, MAHRH Korogho Responsable de service Direction de la Protection des végétaux Raki Kiema Service des interventions protection des végétaux Direction de la Protection des végétaux Moumouni Ouédraogo Secrétaire Permanent Confédération paysanne du Faso Jean-Pierre Renson Coordonateur des Opérations Agricoles d'Urgence et de Rehabilitation FAO François Rasolo Représentant FAO Daouda Kontougoumde Chargé de programme FAO Gnissa Konaté Institut National de l’Environnement et de Directeur Recherches Agricoles Francois Lompo Directeur Adjoint Chargé des Institut National de l’Environnement et de Programmes Recherches Agricoles Seraphine Kaboré Sawadogo Institut National de l’Environnement et de Chargée de projets Recherches Agricoles Clémentine Dabiré Institut National de l’Environnement et de département production végétaux Recherches Agricoles Jean Baptiste Tanga Institut National de l’Environnement et de chargé de programme Recherches Agricoles Adama Compaoré Ministère de l’Agriculture, de l’Hydraulique Secrétaire Général et des Ressources Halieutiques Salimata Somé Traoré Gouverneur du CFC et Conseiller Ministère du Commerce, de la Promotion de des Affaires Economiques l’Entreprise et de l’Artisanat Babacar Cissé Coordonnateur Résident Système des Nations Unies et Représentant Résident PNUD PNUD Gabriel Aimé Diasso Programme de Gestion Intégrée de la Coordonnateur Production et des Déprédateurs Amadou Thombiano Programme de Gestion Intégrée de la chargé de suivi évaluation Production et des Déprédateurs Ousséini Ouédraogo Réseau des organisations Paysannes et des Producteurs Agricoles de l’Afrique de l’Ouest Magha Mohamadou Réseau des organisations Paysannes et des Producteurs Agricoles de l’Afrique de l’Ouest Saliou Ndiaye Réseau des organisations Paysannes et des chargé de suivi évaluation Producteurs Agricoles de l’Afrique de l’Ouest François Kaboré Union des Producteurs de Coton du Burkina, Président de l'Association des Producteurs de Président Coton Africains Directeur de l'Agriculture et de la Kolado Boccoum UEMOA Securité Alimentaire Amadou Hebie Chargé de Programme Securité Alimentaire Union Européenne Senior Agricultural Services Ibrahim Nebie Specialist World Bank

Ethiopia

Name Position Institution Former Director Food Science and Negussie Retta Addis Ababa University Nutrition Program AU Directorate of Rural Economy and Solomon Haile Mariam Advocacy Officer PATTEC Agriculture AU Directorate of Rural Economy and Levi Uche Madueke Assistant PATTEC Coordinator Agriculture Monitoring and Evaluation Expert AU Directorate of Rural Economy and Hassane Mahamat Hassane PATTEC Agriculture Solomon Assefa Director General Ethiopian Institute of Agricultural Research Sub Regional Coordinator for Mafa Chipeta Eastern Africa and FAO FAO Representative in Ethiopia Getachew Abebe Livestock Expert FAO Emergency and Rehabilitation Marjolaine Martin Greentree FAO Coordinator Eshete Dejen Fisheries Technical Officer FAO Satoyama Takanori Policy Officer FAO Agriculture and Food Security Florence Rolle FAO Officer Selamawit Dagne Librarian FAO Lamourdia Thiombiano Senior Soil Resources Officer FAO Emmanuelle Guernebleich Livestock Officer FAO Susan Minae Agribusiness Officer FAO P Anandajayasekeram Manager Capacity Strengthening ILRI Former FAO Representative Hashim Gamal A-Shami Yemen

Gabon

Name Position Institution Coordonnateur du Programme Joël Beassem Régional de Sécurité CEEAC Alimentaire Conseiller du Ministre de l’Economie Forestière et Emile Mamfoumbi Kombila COMIFAC Coordonnateur national de la COMIFAC Emmanuel Sabuni Kasereka Administrateur COREP Guy Anicet Rerambyath Secrétaire General a.i. COREP FAOR in Gabon & Sub-regional Benoit Horemans FAO Coordinator for Central Africa Odile Angoran Policy Officer FAO Sourakata Bangoura Land & Water Resources Officer FAO Plant Production & Protection Sankung Sagnia FAO Officer Jean Claude Nguinguiri Forestry officer FAO Kebkiba Bidjeh Consultants CTPD FAO Ibrahim Salami Director project EC funding FAO Aristide Ongone Obame Coordonnateur du PSSA Ministry of Agriculture

Ghana

Name Position Institution Bureau of Integrated Rural Ernestina Fredua Antoh Senior Research Fellow Development, Kumasi Senior Research Fellow, Bureau of Integrated Rural Dr. Paul Sarfo-Mensah Director of BIRD Development, Kumasi Bureau of Integrated Rural S.E. Edusah Senior Research Fellow Development, Kumasi Bureau of Integrated Rural Akwasi Owusu-Bi Senior Research Fellow Development, Kumasi National project coordination Kweku Amoako Atta DeGraft Johnson UNEP/GEF IAS project CABI Africa Yaw Osei-Owusu Country Director Conservation International Annor Nyarko CSIR Environmental Protection Sam Adu-kumi Deputy Director Agency FAOR and Sub Regional Musa Mbenga Coordinator West Africa FAO Assistant Director General and Regional Representative for Maria Helena Semedo Africa FAO Hannah Clarendon Crop Protection Officer FAO John Moehl Regional Aquaculture Officer FAO Brahim Kebe FAO Mouslim Maiga FAO Irene Frampong FARA Women Farmers Organization Lydia Sasu Network of Ghana Federation of Agriculture Marjorie Abdin Vice President Exporters Federation of Agriculture Millicent Madjitey Officer Manager Exporters Owusu Ansah Forestry Commission, Kumasi Joel Sam GAINS CSIR Francis Dzah Ghana Statistical Service

Kwaku Saiw and Representatives of Ashanti Fish Kumah Farms/Ashanti Fish Farmers Association Farmers Association Emelia Monney Ministry of Agriculture Twumasi Ankrah SPFS Coordinator Ministry of Agriculture National Farmers and Fishermen Awards Winners Philip Abayori President Association of Ghana Monitoring and Evaluation Angela Dansson Officer PPMED Vesper Suglo PPRSD Eunice Adams PPRSD UN Resident Coordinator and Daouda Toure UNDP Resident Representative UNDP K.K. Kamaluddeen Country Director UNDP Pa Lamin Beyai Economic Advisor UNDP Artemy Izmestiev UNDP Afreh Nuamah Professor University of Legon Vida Ofori WAD Foods K. Osman Gyasi Agricultural Economist World Bank Sericulture promotion and Paul Kwasi Ntaanu President Silk Network Development Association Chairman Sericulture Sericulture promotion and Kwame Owusu K. Association Development Association Lina Otoo Head of Policy Unit Ministry of Agriculture

Kenya

Name Position Institution Rufaro Madakadze Program Officer Education and AGRA Training Aissétou Dramé Yayé Executive Secretary ANAFE Arid Lands Resource Management Project, Office Mary M. Mwale Drought Management Officer of the President Roger Day Deputy Director Development CABI Africa Cotton Development Alex K. Mungai Promotion Officer Authority Cotton Development Powon Micah Pkopus Managing Director Authority Eastern Africa Farmers Mainza Mugoya Program Officer Federation Programme Officer Agriculture Japhet Kiara and Rural Development Embassy of Sweden National Consultant Crop Paul A. Omanga Production FAO National Consultant Agricultural George A. Odingo Production FAO Food Security and Early Michael Makokha Odera Warning Systems Co-ordinator FAO International Livestock Robert Allport Consultant FAO Joseph Matere GIS Information Specialist FAO Paul Mbithi Mutungi Pastoral Field School Specialist FAO Castro Camarada FAO Representative FAO Assistant FAO Representative Augusta Abate (Programme) FAO Kenya Agricultural Research Ephraim A. Mukisira Director Institute Kenya National Federation of Kanywithia Mutunga Chief Executive Officer Agricultural Producers Kenya National Federation of Judith W. Nthiga Programme Manager Co-operatives Ltd General Manager Inspection Kenya Plant Health Washington Otieno Operations Inspectorate Service Anthony Gikandi Muriithi Agricultural Economist Ministry of Agriculture Priscilla Muiruri Head of Market Intelligence Ministry of Agriculture Beethoven Mwangi IPC Analyst - KFFSG Member Ministry of Agriculture Deputy Commissioner for Ministry of Co-operative Phanice Kajumbe Cooperative Development Development and Marketing Ministry of Irrigation and Alex Mwangolo Deputy Director Agriculture Water Management Ministry of Livestock and Peter M. Nzungi Chief Fisheries Officer Fisheries Development Ministry of Livestock and Beatrice Nyandat Principal Fisheries Officer Fisheries Development National Humanitarian Affairs Thomas Nyambane Officer UN-OCHA Patrick Lavand'homme Deputy Head of Office UN-OCHA

Malawi

Name Position Institution Frank Kufakwandi Resident Representative AfDB Moses Kwapata Principal Bunda College William Mfitelodze Dean Faculty of Agriculture Bunda College Victor Mhoni National Coordinator CISANET DAES, Ministry of Pearson Soko Senior Agribusiness Officer Agriculture DAES, Ministry of Andrex Makungwa Agribusiness Officer Agriculture DAES, Ministry of Lucy Nyirenda Agribusiness Officer Agriculture DAES, Ministry of Grace Phiri Agribusiness Officer Agriculture DAES, Ministry of Grace Malindi Director Agriculture Friday Njaya Chief Fisheries Officer Department of Fisheries Steve Donda Deputy Director of Fisheries Department of Fisheries Dennis Kayambazinthu Director Department of Forestry Deputy Director Irrigation Winston Sataya Services Department of Irrigation Principal Secretary for Nutrition, Department of Nutrition, HIV Mary Shawa HIV & AIDS & AIDS Head of Section Rural Enrica Pellacani Development and Food Security European Union Food and Nutrition Security Ruth Butao Ayoade Policy Advisor FAO Pinit Korsieporn Representative FAO Rose Bell Programme Officer FAO Wells Kumwenda Project Manager FAO Elizabeth Sibale Project Manager FAO Assistant FAO Representative Alick Nkhoma Responsible for Programmes FAO Blessings Mwale Project Manager FAO Katrien de Pauw Head of Cooperation FICA Mamda Reinford Programme Officer JICA Sonoyama Hideki Project Formulation Advisor JICA Assistant Resident Daisuke Saito Representative JICA Secretary for Agriculture and Ministry of Agriculture and Andrew Daudi Food Security Food Security Ministry of Agriculture and Isaac Chirwa Principal Statistician Food Security Lindiwe Chide Planning Officer Ministry of Education Beatrice Makwenda Policy Coordinator NASFAM Dyborn Chibonga CEO NASFAM Gloria Chimalizeni Farm Service Officer NASFAM Festus Kaupa Executive Director Natural Resources College Office of the UN Resident Nguyen Thi Ngoc Van Head Coordinator Marita Sorheim Rensvik Second Secretary Royal Norwegian Embassy Anne Callanan Deputy Country Director WFP Norah Mwamadi Senior Programme Assistant WFP Dr. Limbambala HIV Country Officer WHO Hardwick Tchale Agricultural Economist World Bank

Tanzania

Name Position Institution Emmanuel Mwachullah Director of Policy, Planning and Advocacy Agricultural Council of Tanzania Delegation of the European Anne-Claire Leon First Secretary Natural Resources Commission Consultant (Auto-Evaluation FAOR Robert Otsyina Tanzania) Development Associates Ltd Darren Welch Head of Office DfID Tom Sillayo Training Manager Faida Market Link Co. Ltd. James Yonazi Assistant FAO Representative FAO Gerald Runyoro Programme Assistant FAO Aisya Frenken Programme Officer FAO Chief Technical Advisor Improving Labour Michael Lerner Law Compliance ILO Philippe Vanhuynegem Chief Technical Advisor ILO Coop Africa Ministry of Agriculture Food Security Rose Anne Mohamed Principal Agricultural Officer and Cooperatives Principal Economist Development Assistance Ministry of Agriculture Food Security Margaret Ndaba and International Cooperation and Cooperatives Ministry of Agriculture Food Security Richard N. Magoma Officer, Plant Health Service and Cooperatives Ministry of Agriculture Food Security William Stephen AD Plant Health Service and Cooperatives Ministry of Agriculture Food Security William Riwa Agricultural Inputs Service and Cooperatives Ministry of Livestock Development Catherine Joseph Director of Policy and Planning Division and Fisheries Ministry of Livestock Development Hosea Gonza Mbilinyi AD Fisheries and Fisheries Ministry of Livestock Development Yassn E. Mndeme Principal Fisheries Officer and Fisheries Ministry of Livestock Development C.G. Mahika Director Aquaculture Development and Fisheries Ministry of Livestock Development Ritta Maly AD Aquaculture and Fisheries Ministry of Livestock Development Baralla S.M. Mngulwi AD Fisheries Marketing and Infrastructure and Fisheries Ministry of Livestock Development Margaret Rose Bakuname AD Livestock Extension and Fisheries Ministry of Livestock Development Jo Mollel AD Veterinary Public Health and Fisheries Coordinator National Forest and Beekeeping Ministry of Natural Resources and Deusdedit Bwoyo Programme Tourism Ministry of Natural Resources and Juma S. Mgoo Acting AD Forest Development Tourism Ministry of Natural Resources and Nurudin Chamuya NPG NAFORMA Tourism Raphael Daluti AD Irrigation Ministry of Water and Irrigation Susan Masagasi Field Operations Manager Tanzania Agricultural Partnership Charles M Ekelege Director Tanzania Bureau of Standards Tanzania Chamber of Commerce, Emmanuel Nnko Marketing Director Industry and Agriculture Tanzania Chamber of Commerce, Mwenze Kabinda IT Manager Industry and Agriculture Tanzania Chamber of Commerce, Adam Zuku Senior Chamber Development Officer Industry and Agriculture Godwin Ndossi Managing Director Tanzania Food and Nutrition Centre AlberIc Kacou Resident Coordinator UN RC Office Gianluca Rampolla del Tindaro Senior Adviser UN RC Office Alain Noudehou Country Director UNDP Juliet Kabege Programme Officer UNIDO Andrea Antonelli Programme Officer UNIDO Ronald Sibanda Country Director WFP Mohamed Belhocine Country Representative WHO Joyce Urasa CountryStat Focal Point National Bureau of Statistics Said Aboud Director Agriculture Statistics National Bureau of Statistics Thomas M. Mnunguli Head, Engineering Standards Department Tanzania Bureau of Standards Joseph B Masikitiko Finance and Administration Manager Tanzania Bureau of Standards

Uganda

Name Position Institution Andre Krummacher Country Director ACTED Regional Coordinator Eastern Africa Plant Abebe Demissie Genetic Resources Network ASARECA Jennifer Johnson Martinesi Country Director ASB Uganda Franklyn Omaki Programme Officer CESVI Salvatore Creti Country Director CESVI Anthony Kerwegi Executive Director Concerned Parents Association Bernard Crabbé Head of Agriculture and Rural Development Delegation of European Commission Fausto Prieto Perez Technical Assistant ECHO Deputy Emergency and Rehabilitation Kennedy Igbokwe Coordinator FAO Charles Owach Assistant FAO Representative Programme FAO Percy Misika FAO Representative FAO James Okoth Emergency Programme Officer FAO Charles Musinguzi Former National Consultant HPAI FAO Monitoring and Evaluation Officer Bernard Mmwesigwa Emergency Unit FAO Rachel Nandelenga Information and Communication Officer FAO National Programme Manager Emergency James Robert Okotzi Unit FAO Alexis Bonte Emergency Coordinator FAO Institute for International Cooperation Fabio Gigantino Programme Officer and Development International Institute of Rural Julius Ben Adome Programme Officer Recostruction Oliva Mkumbo Senior Scientist Lake Victoria Fisheries Organization Brian Eric Marshall Resource Monitoring Specialist Lake Victoria Fisheries Organization Elisabeth Birabwas Information and Communication Officer Lake Victoria Fisheries Organization Caroline Kirema-Mnkasa Senior Fisheries Management Officer Lake Victoria Fisheries Organization Komayombi Bulegeya Commissioner Crop Protection MAAIF Paul Laboke Senior Agricultural Officer MAAIF Godfrey Byarwanga Draughts Officer MAAIF MAAIF Department of Animal Androa Denis Senior Veterinary Officer Production and Marketing MAAIF Department of Animal Dick Kirumira Principal Pasture Agronomist Production and Marketing MAAIF Department of Crop Federica Nshemereirwe Program Officer Production and Marketing MAAIF Department of Crop Alex Lwakuba Assistant Commissioner Crop Production Production and Marketing MAAIF Department of Fisheries Bakunda Aventino Senior Fisheries Officer Resources MAAIF Department of Livestock Nicholas Kauta Commissioner Health and Entomology MAAIF Phytosanitary Quarantine Ephrance Tumuboine Senior Agricultural Inspector Inspection Service Annunciata Hakuza Economist MAAIF Planning Department Andre Sarich Programme Officer MAAIF Rice Industry Secretariat Maggie Mabweijano Assistant Commissioner Gender and Women Ministry of Gender, Labour and Social Kyomukama Affairs Development John Kashaka Muhanguzi Permanent Secretary Ministry of Local Government MWE Directorate of Environment Rachel Musoke Commissioner Forest Sector Support Affairs MWE Directorate of Water Richard Cong Commissioner Water Production Development MWE Directorate of Water Sottie Bomukama Director Development MWE Water Resources Monitoring and Nebert Wobusobozi Commissioner Assessment Department MWE Water Resources Monitoring and Kyosingira Wilson Fred Assistant Commissioner Water Resources Assessment Department MWE Water Resources Monitoring and Caroline Nakalyango GIS Expert Assessment Department National Agricultural Advisory Grace Opio Aluoch Ocheng Product Development Officer Services Secretariat National Agricultural Advisory Alyce Nakagwa Information and Communication Officer Services Secretariat National Agricultural Research Ambrose Agona Director Organization National Agricultural Research Jerome Kubiriba National Banana Research Programme Organization National Agricultural Research Japheth Magyembe Coordinator Competitive Grant Scheme Organization National Agricultural Research Joyce Adupa Head Agriculture Information System Organization Coodinator Agricultural Research National Agricultural Research Magyembe Mwesigwa Competitive Fund Scheme Organization Secretariat Assistant Commissioner for Coordination and Timothy Lubanga Monitoring Office of the Prime Minister Avu Elhy Biliku Principal Planner Office of the Prime Minister David Wamata Principal System Analyst Office of the Prime Minister Saul Kylyune System Analyst Office of the Prime Minister OPM Department of Disaster Pamela Komujuni Disaster Management Officer Management, Relief and Refugees Plan for Modernisation of Agriculture Tom Mugisa Programme Officer Technical Service Secretariat Moses Osiru Programme Manager RUFOFUM Patrick Okello Principal Statistician Agricultural Statistics Uganda Bureau of Statistics Seth Mayinza Director Uganda Census of Agriculture Uganda Bureau of Statistics Henry Ngabirano Managing Director Uganda Coffee Development Authority Mackay Aomu Deputy Executuve Director Uganda National Bureau of Standards Augustine Mwendya Director Uganda National Farmers Federation Silla Ristimaki Program Officer UN RCO Jane Kalweo Institutional Development Advisor UNAIDS Nea-Mari Sylvia Heinonen Program Officer Monitoring and Evaluation UNFPA Harriet Akullu Program Specialist Gender Justice UNIFEM Hugh Greathead Grants Director World Vision

Zimbabwe

Name Position Institution Faculty of Agriculture, Charlie Mtisi Dean University of Zimbabwe Faculty of Agriculture, Dr. Nyamangara Deputy Dean University of Zimbabwe Faculty of Agriculture, Dr. Whuta Dept of Soils and Irrigation University of Zimbabwe Sub-Regional Coordinator for Gaoju Han Southern Africa and FAOR FAO Zimbabwe Cary R. Joseph Senior Policy Officer FAO Martin Ager Water Resources Manager FAO Georges Codjia Nutrition Officer FAO Rene Czudek Forestry Officer FAO Aubrey Harris Senior Policy Officer FAO Plant Protection and Production Joyce Mulila-Mitti FAO Officer Animal Health and Production Kwaku Agyeman FAO (consultant) Jacopo Damelio Emergency Unit FAO HIV/Aids and Livelihoods Loveness Makonese FAO National Officer Brighton Nhau Monitoring and Evaluation FAO Karsto Kwazira Agricultural Officer FAO Tendai Mugara Monitoring and Evaluation FAO Agricultural Adviser Emergency Michael Jenrich FAO Unit Kazito Mazvimavi ICRISAT

Raymond Kujeke Consultant IFAD

Mandivamba Rukuni Former Executive Director Kellogg Africa Sustainable Agricultural Trust Bryan Saunders SAT The African Capacity Adeboye Adeyemo Building Foundation Caesar Chidawanyika World Bank Mavis Chidzonga ZimTrade

Annex 6 – List of key documents consulted 1

• S. Ballit, Communication for Isolated and Marginalized Groups”. Paper prepared for the Ninth United Nations Roundtable on Communication for Development, Rome, 2004 • H. Baser, H. and P. Morgan, Capacity, Change and Performance. ECDPM, 2008 • M. Blagescu and J. Young, Capacity Development for Policy Advocacy: Current thinking and approaches among agencies supporting Civil Society Organisations, ODI, 2006 • Danida, Between Naivety and Cynicism: A Pragmatic Approach to Donor Support for Public Sector Capacity Development, 2004 • C.Eicher, Building African Scientific Capacity in Food and Agriculture, 2009 • S. Ferris and P. Robbins, Development Market Information Services in Eastern Africa. The FOODNET experience. International Institute of Tropical Agriculture, 2004 • High Level Forum on Aid Effectiveness, Paris Declaration on Aid Effectiveness, 2005 • High Level Forum on Aid Effectiveness, Capacity Building, 2008 • International Development Research Centre, Good Practice for Capacity Development, 2007 • T. Land et al., Capacity development: between planned interventions and emergent processes. Implications for development cooperation, ECDPM, 2009 • A. Milèn, What do we know about capacity building?, World Health Organization, 2001 • Y. Mizrahi, Capacity Enhancement Indicators, World Bank Institute, 2004 • Norad, Synthesis Study on Best Practices and Innovative Approaches to Capacity Development in Low-income African Countries, 2008 • OECD, Living up to the Capacity Development Challenge: Lessons and Good Practice, 2005 • OECD, The Challenge of Capacity Development - Working towards good practice, 2006 • OECD, Concepts and Dilemmas of State Building in Fragile Situations and Ensuring Fragile States are Not Left Behind, 2008 • S. Otoo et al., The Need for a Conceptual and Results-oriented Framework in Capacity Development: Discussion of a New Approach, World Bank Institute, 2009 • T. Theisohn, Considerations on capacity building at the policy level in the UN, 2008 • United Nations Development Group, Enhancing the UN’s contribution to National Capacity Development, 2006 • T. Tooth, The Use of Multi-media in Distance Education, Commonwealth of Learning, 2000 • United Nations Development Group, Capacity Assessment Methodology – User Guide, 2008 • United Nations Development Program, Capacity Development Practice Note, 2008 • United Nations General Assembly, Triennial comprehensive policy review of operational activities for development of the United Nations system, 2008 • United Nations Population Fund, Thematic Evaluation of UNFPA Support to National Capacity Development, 2001 • D. Watson, Monitoring and Evaluation – Aspects of Capacity and Capacity Development, ECDPM, 2005 • World Bank, Capacity Building in Africa – An OED Evaluation of World Bank Support, 2005

1 The list does not include any of the FAO documents, publications, and reports the Team consulted. • World Bank, Institutionalization of Monitoring and Evaluation Systems to Improve Public Sector Management, 2006 • World Bank, Using Training to Build Capacity for Development – An Evaluation of the World Bank’s Project-based and WBI Training, 2008 • World Food Program, Evaluation of WFP’s Capacity Building Policy and Operations, 2007

Evaluation of FAO’s Capacity Development Activities in Africa

Annex 7 - An Overview of FAO’s Capacity Development Activities in Africa

Table of contents

1. Objective...... 3 2. Methodology...... 3 3. What is Capacity Development in FAO?...... 3 4. FAO CD Activities by technical area...... 12 4.1. Animal Production and Health...... 12 4.2. Nutrition and Consumer Protection (incl. food safety)...... 13 4.3. Plant Production and Protection...... 13 4.4. Rural Infrastructures and Agro Industries...... 14 4.5. Statistics and Information Systems...... 15 4.6. Trade and Markets...... 16 4.7. Fisheries and Aquaculture...... 17 4.8. Forestry...... 18 4.9. Natural Resources Management and Environment...... 19 4.10. Food Security (The Special Programme and other holistic interventions)20 4.11. Policy Support...... 20 4.12. Emergency Operations and Rehabilitation...... 21 4.13. Gender and HIV/AIDS...... 22 4.14. Other (essentially CD work to strengthen farmer groups and associations)...... 22

1 Evaluation of FAO’s Capacity Development Activities in Africa

List of acronyms

CAHW Community Animal Health Worker CBO Community Based Organization CD Capacity Development CECAF Fishery Committee for the Eastern Central Atlantic COMIFAC Central African Forest Commission COREP Comité Régional des Pêches du Golfe de Guinée DaO Delivery as One ECOWAS Economic Community of Western African States FFS Farmer Field School FIVIMS Food Insecurity and Vulnerability Information Management System FMPP FAO Multidonor Partnership Programme FPMIS FAO Field Programme Management Information System GDP Gross Domestic Product GHG GreenHouse Gas GIS Geographic Information System GMO Genetically Modified Organism HACCP Hazard Analysis and Critical Control Points HDI Human Development Index HQ Headquarters IDP Internally Displaced People IPC Integrated food security Phase Classification IPPC International Plant Protection Convention KCEF FAO Outreach and Capacity Building Branch MAC Market Access Company MoA Ministry of Agriculture NGO Non Governmental Organization PCA Program Cooperation Agreement SSA Sub Saharan Africa SPFS Special Programme for Food Security TAD Transboundary Animal Disease TCP Technical Cooperation Programme UN United Nations UNAIDS Joint UN Programme on HIV/AIDS UNDP UN Development Programme UNFPA UN Population Fund UNIDO UN Industrial Development Organization UNIFEM UN Development Fund for Women WTO World Trade Organization

2 Evaluation of FAO’s Capacity Development Activities in Africa

1. Objective

• The Overview aims at identifying capacity development activities in Africa as comprehensively as possible. In doing so, the Overview also provides quantitative evidence on the ways CD is approached and implemented through field projects in Africa. The Overview does not consider CD activity exclusively supported from Headquarters (HQ).

2. Methodology

• The Overview covers all the projects (including the regional and interregional/global ones, but excluding Telefood) implemented in Sub Saharan Africa, from 2000 onwards1. • All the relevant projects’ ProDocs (from FPMIS) have been reviewed, in order to identify CD content in there, in terms of immediate objectives, activities, and target dimension(s). On the basis of this, the degree of CD-focus was assessed on a scale from 1 (min) to 5 (max). • Results by country were then put together and analysed, with a view to understand: 1. how much FAO is committed to develop capacities through projects, using as an indicator the share of CD-focussed projects (i.e. those ranked 3-5 in the above scale) 2. “what CD is about”, in terms of approach adopted, activities implemented, and technical areas of intervention. 3. whom FAO CD interventions target, breaking this down by CD dimension (individual, organization, enabling environment) and categories (Government and public sector, farmers and CBOs, small business enterprises and private sector).

• For reasons of clarity, the description of CD activities is by technical area, mostly in line with the HQ technical division responsible for the supervision. The overview also attempts to do an analysis of the commonalities across dimensions and sectors and highlight sub-regional differences. • Issues of relevance are considered to the extent possible, by making reference to agro- ecological zones and carrying out a simple correlation analysis between amount of national CD-focussed projects (measured in terms of number of projects, not of budget allocated) on one side and few socio-economic indicators, such as the Human Development Index, the share of undernourished population, and the agricultural GDP on the other2.

3. What is Capacity Development in FAO?

• As stated in the latest draft Capacity Development Strategy, FAO supports the development of both technical and functional capacities in order to achieve the three global goals in the Organization’s Strategic Framework 2000-2015. Capacity understood this way thus does not include only the technical skills required to intensify production and sustainably manage natural resources to improve food safety and security for all, but also the ability to formulate and implement policies and programs, as well as to partner with other key

1 The Overview covers some 3,000 projects, not including those implemented in countries (like Comoros, Djibouti, Mauritius, Sao Tome, Seychelles) where the total budget for the period 2004-2009 was less than USD 1 million (Source FPMIS). 2 Sources: UNDP HDR 2009; FAO State of Food Insecurity in the World 2006, FAOSTAT

3 Evaluation of FAO’s Capacity Development Activities in Africa stakeholders. The production, management and exchange of information and knowledge cut across. • Both types of capacities unfold across the three dimensions (individual, organization, and the policy/enabling environment) and are developed through different modalities: software (technical assistance, training, workshops, knowledge and information sharing), hardware (provision of agricultural inputs, infrastructures and equipments, including IT facilities) and, to a significantly lesser extent, financial assistance.3 • Capacity Development is at the core of FAO’s work in the field. CD-focussed projects are about half of the total (49.4%) nationally-implemented ones4: 1. Three quarters (75%) aim at developing the capacities of individuals (either Government/public sector officials or farmers, essentially) through technical training, workshops, study tours, demonstration sessions (incl. within FFS); 2. 23% (either the same projects or others) rather focus on the enhancement of organizations’ capacities (essentially Governments and, to a lesser extent, formalized farmers’ associations and small business enterprises) through restructuring, set up of units and committees, development/strengthening information management systems; 3. 27% address the policy/enabling environment dimension, by either closely working with national Governments and regional institutions on the definition of policies and programs and by creating/strengthening the institutional linkages among different stakeholders (incl. decentralized Government units, NGOs and the private sector). It is worth noting that the 27% share goes up to a much higher level when including the support provided through (essentially) regional projects, where FAO plays a key role with respect to definition/implementation of international norms and standards, and as honest broker in the governance of transboundary issues (such as animal diseases) and the management of natural resources (water).

Figure 1: FAO CD dimensions

FAO CD Dimensions (national projects only)

80% 70% 60% 50% 40% 75% 30% 20% 23% 27% 10% 0% Individuals Organization Policy/Enabling Environment CD Dimension

• Two thirds of the national projects with a strong CD component address one dimension only. Three quarters of these (49% of the total) focus on enhancing the capacities of

3 In line with the general understanding of CD by the Evaluation, hardware and financial assistance support have been considered as integral part of CD activities only when provided in conjunction with software components. 4 The analysis through FPMIS does not in fact always allow for certainty of implementation of regional and inter-regional projects in specific countries. For this reason, the figures (but not the description of activities) in the Overview will reflect national projects only.

4 Evaluation of FAO’s Capacity Development Activities in Africa

individuals, being them either farmers or government officials, mainly through training and workshops. About 30% cover two dimensions, mostly starting with training of individuals and then extending the intervention to either organizational strengthening or facilitating coordination and institutional links. The remaining 4% of national CD projects address all three dimensions.

Figure 2: Combination of CD Dimensions

CD Dimensions (FAO national projects only)

70%

13% 60%

4% Individuals, Organization and 50% Enabling Environment Organization and Enabling environment 40% Indidual and Enabling environment

Individual and Organization 30% 2% 49% Enabling environment

12% 20% Organization

Individual

10% 14%

4% 0% One dimension only Two dimensions Three dimensions together

• FAO CD activities primarily targets staff from Governments (also local ones, although in fewer cases) and Government-affiliated institutions (47%), the latter including regional organizations, laboratories, national research centres. The Organization works also extensively with farmers and CBOs (42%), whose capacities are enhanced mainly through the set-up of FFS and the organization of demonstration sessions on agricultural production techniques. The private sector remains residual, with some 11% of FAO interventions aiming at strengthening small business enterprises and agri-business processing units, basically through training and set up of institutional linkages.

5 Evaluation of FAO’s Capacity Development Activities in Africa

Figure 3: FAO CD Target

FAO CD Target (national projects only)

50% 45% 40% 35% 30% 25% 47% 42% 20% 15% 10% 11% 5% 0% Government Civil Society Private Sector (Farmers) (Small Business) CD Dimension

• FAO CD activities cut across all the technical areas the Organization works into, with quite significant differences among them though. • CD efforts have more evidently focused on few sectors, namely: plant production and protection (on average, some 14% of total CD interventions per country); land and water management (12%); animal health and production (11%), SPFS and other “holistic” food security programs (11%), statistics and information management systems (11% altogether). Policy assistance (including in specific technical areas, such as forestry and fisheries) also figures out quite high (12%), as CD in emergency and rehabilitation interventions do (11%).5 • “Markets and trade” emerges as the area where FAO has put the least efforts in developing capacities through projects (1% on average in SSA), with few CD activities at national level focusing on negotiation skills and information management implemented in equally few countries such as Mozambique, Lesotho, Rwanda and Tanzania.6 • Less attention, on average and in relative terms, has also been given to develop capacities on gender and HIV/AIDS issues per se (2%, when considering only national projects with a specific focus), forestry (4%, most of the work done at HQ or through global support programs such as FMPP or PCA Norway) and rural infrastructures and agro- industries (5%, with activities here as well concentrated in few countries such as Ghana, South Africa, Zambia).

5 As described in more details in the dedicated following section, CD work in emergency and rehabilitation mainly comprises the work on seeds distribution and multiplication (whenever significant training and/or setting up of a seed chain was there) and the support to the MoA Emergency Unit, including coordination of various stakeholders in the humanitarian field. 6 The implementation of the USD 8 million ‘All ACP Support Programme on Agricultural Commodities” has in fact just started, with few workshops organized at regional level.

6 Evaluation of FAO’s Capacity Development Activities in Africa

Figure 4: FAO CD activities per technical area

FAO CD Activities per technical area (national projects only)

14% Plant Production & 13% Protection Policy Assistance 12% Animal Production & Health 11% Natural Resources Management 10% Food Security Emergency & Rehabilitation 9%

Statistics & Information 8% Systems

7% Nutrition and Consumer Protection 6% Fisheries & Aquaculture 5%

Rural Infrastructures & 4% Agro-Industries Forestry 3%

2% Gender/HIV 1%

Trade & Markets 0% Technical areas

• No outstanding difference emerges from comparing aggregate figures at sub-regional level:7  The share of national CD-focussed projects ranges from 45% to 50% all throughout, with slightly higher figures in West and Southern Africa;  The dimensions’ variance is not significant with respect to the average of interventions targeting individuals, organization and enabling environment, as above;  Shares of target population (Government, civil society, private sector) differ less than 2% (in absolute terms) from the regional averages, with the only exceptions of Southern Africa (where the focus is much more on the public sector than civil society, 57% and 38% respectively, compared to 47% and 42% at regional level) and less attention given to develop the capacities of Governments and Governments- affiliated organizations in West Africa (38%)

• The same can be said when relating the amount of CD work carried out in the sub- regions with average indicators of needs, such as the HDI, share of undernourishment, and agriculture’s GDP share. As it is evidenced in the table below, there is a general linear relationship between FAO CD-focussed projects and these indicators of needs, although the correlation among these two sets of variables is far from being strong (0.35, 0.43 and 0.22 respectively): e.g. in East Africa, where some 31% of FAO CD efforts in Africa are concentrated, the average HDI is quite high (expressing strong need), as the level of undernourishment and the importance of agriculture to national economy are. Yet, the general low

7 Defined according to the FAO SROs geographical coverage, as in COIN.

7 Evaluation of FAO’s Capacity Development Activities in Africa share of CD work (through national projects) in Central Africa despite a significant level of undernourishment is worth noting, with most of activities implemented in Congo DRC.

Table 1: FAO CD focus by sub-region and main indicators of needs

CD focus in national Under- Agriculture Sub-region projects HDI (2009) nourishment GDP (2006) (Out of total CD in (2002-04) Africa) Central 18% 148.6 36.2 21.9 East 30.7% 161.6 42.3 31.1 South 22.9% 146.8 31.9 21.6 West 28.4% 165.1 24.9 37.0

• Still focusing on sub-regional aggregates and looking at discrepancies among CD- focussed activities in different technical areas, it is interesting to point out how:  FAO has significantly contributed strengthening the capacities of both Governments and farmers in the area of animal production and health in East Africa (some 21% of CD-focussed projects, compared to a 12% SSA regional average), thanks to a number of agro-pastoral FFS set up and the enhancement of organizational and policy capacities in the area of TADs’ prevention and control;  Still on the “positive” side, it is worth noting the higher share (5%) of project activities focusing on developing capacities in the area of gender and HIV/AIDS, especially with relation to nutrition issues.  The share of CD interventions on forest management in Central Africa is in line with SSA regional average (some 3%)8, but quite low when considering the extent of forest resources in the region and when compared to other “less relevant” focus areas as animal health and production (which gets about 11% of CD national activities in the country). Same lower share for natural resources management, something that is altogether less ‘worrisome’ though.  Fewer projects’ activities have concentrated on developing capacities through national projects in the areas of statistics and information management in West Africa, the share (3%) being well below the SSA average9. This includes some key interventions in Niger, Burkina, Gambia and Liberia, but, with no action in several other countries.

8 Yet, it is important to keep in mind that two regional projects contributed strengthening the capacities of COMIFAC and enhancing food security through training on non-forest wood products 9 As elsewhere, the share does not consider key regional and inter-regional proejects as the EC/FAO Food Information for Action

8 Evaluation of FAO’s Capacity Development Activities in Africa

Table 4: Comparison of CD activities among technical areas and sub-regions

Technical Areas

Sub Nutrition region and Plant Rural Emergenc Animal Statistics and Trade Natural Consumer Productio Infrastructur Fisheries and Food Policy y and Gender/ Health and Information and Forestry Resources Protection n and e and Agro- Aquaculture Security Assistance Rehabilita Production Systems Markets Management HIV (incl. Food Protection Industries tion Safety)

Central 11% 6% 20% 2% 14% 0% 11% 3% 4% 14% 10% 14% 0%

East 21% 7% 22% 3% 11% 1% 5% 3% 12% 3% 12% 21% 1%

South 9% 12% 6% 9% 11% 1% 6% 5% 17% 9% 13% 4% 5%

West 11% 7% 12% 4% 3% 0% 7% 6% 11% 16% 12% 10% 2%

9 Evaluation of FAO’s Capacity Development Activities in Africa

• Breaking these figures further down to country level, it is interesting to note that:

 CD efforts, as measured by number of national CD-focussed projects, have been more significant in few “conflict and post-conflict” countries like Congo DRC, Sudan, Somalia, and Mozambique. However, other “similar” countries – such as Angola, Sierra Leone, Ivory Coast, Chad, and Rwanda – have not received as much, with the share of CD activities in each country being less than 1% of total national projects in Africa.  When rather measuring CD efforts through the share of CD-focussed projects out of total national projects per country, figures tend not to diverge that much among countries (with few exceptions) and be similar in “development” and “emergency” contexts. More than 60% of national projects in Mozambique, Zimbabwe, Sierra Leone but also in Gambia, Senegal, South Africa, and Malawi focus on CD.  Linear correlation among CD activities and indicators of needs at country level is there. Yet, a simple correlation analysis between the share of CD activities in countries (out of total national projects implemented in Africa) and these indicators shows a certain “neglect” for some countries in West and Central Africa (e.g. Chad, Togo, Central African Republic, Rwanda, Guinea-Bissau) where HDI is high, as the level of undernourishment and the agricultural GDP are.

Table 2: Juxtaposition of HDI indicators and share of CD activities by country

Share of CD activities (out of total projects in HDI Africa) Niger 182 1.2% Sierra Leone 180 1.6% Central African Republic 179 0.9% Mali 178 0.9% Burkina Faso 177 1.3% Congo, Democratic Republic of 176 4.5% Chad 175 0.9% Burundi 174 1.7% Guinea-Bissau 173 0.8% Mozambique 172 2.5% Ethiopia, Federal Democratic Republic of 171 1.3% Guinea 170 0.7% Liberia 169 1.4% Gambia, Republic of 168 1.2% Rwanda 167 0.8% Senegal, Republic of 166 1.3% Eritrea 165 1.2% Zambia 164 0.8% Côte d'Ivoire, République de la 163 1.1% Benin, Republic of 161 0.4% Malawi 160 0.9% Togo 159 0.5% Nigeria 158 0.3%

10 Evaluation of FAO’s Capacity Development Activities in Africa

Uganda 157 1.5% Lesotho 156 0.8% Cameroon, Republic of 153 1.0% Ghana 152 0.7% Tanzania, United Republic Of 151 1.2% Sudan 150 5.0% Kenya, Republic of 147 1.2% Madagascar 145 0.8% Angola, Republic of 143 0.8% Swaziland 142 0.7% Congo, Republic of 136 0.8% South Africa, Republic of 129 0.5% Namibia, Republic of 128 0.4% Botswana, Republic of 125 0.1% Cape Verde 121 0.4% Equatorial Guinea 118 0.2% Gabon 103 0.3% Somalia na 2.3% Zimbabwe na 1.4%

Figure 5: Correlation between Agriculture GDP indicator and CD-focussed activities by country (share of total projects in Africa)

Correlation Agriculture GDP and CD-focussed activities (out of total projects in Africa) 6.0%

5.0%

4.0%

3.0%

2.0%

1.0%

0.0% 0 20 40 60 80

11 Evaluation of FAO’s Capacity Development Activities in Africa

Figure 6: Correlation between indicators of undernourishment and CD-focussed activities by country (share of total projects in Africa)

Correlation Undernourishment and CD-focussed activities (out of total projects in Africa)

5.0% 4.5% 4.0% 3.5% 3.0% 2.5% 2.0% 1.5% 1.0% 0.5% 0.0% 0 10 20 30 40 50 60 70 80

4. FAO CD Activities by technical area

4.1 Animal Production and Health

• On average, animal production and animal health get an almost equal share (5-6% each) of CD project activities in country. Figures do not differ much among sub-regions, with the exception of East Africa (mainly Ethiopia, Kenya, and Uganda) where these two areas get up to 9% and 12% respectively of CD-focussed national projects (as pointed out above). Yet, it is important recalling that most of the recent support in the area of animal health in Africa has been additionally channelled through regional projects (and thus not included in these figures) to prevent and control Avian Influenza and other TADs.

• On the animal production side, CD activities have mostly focused on the enhancement of community extension services and the set up of agro-pastoral FFS (also as one of the diversification components of the Special Programme for Food Security). Training is provided also specifically on identification and livestock marking, meat inspection and dairy production, up to supporting the preparation of investment and business plans to commercialize dairy products. Some interesting examples:  In DRC, farmers and livestock keepers have been trained in improved production and marketing techniques, including on basic management skills to set small community enterprises up.  The Regional Support Programme for the coordination and capacity strengthening for disaster and drought preparedness in the Horn of Africa have also worked on the improvement of mapping and visualization tools (incl. through GIS techniques) for program planning, monitoring and evaluation and provided training in this respect.  In Kenya, CD intervention focused on organizational strengthening, by supporting the re-definition of roles/responsibilities and priorities of the Kenya Dairy Board.

• On the animal health side, interventions have run across the three dimensions by:

12 Evaluation of FAO’s Capacity Development Activities in Africa

 providing training in the prevention and control of various animal diseases, including on epidemiology and on laboratory techniques; organizing on-farm demonstration sessions;  setting up and equipping early warning and rapid response systems (incl. definition of protocols and provision of lab equipments); installing TADInfo database; establishing and strengthening association of Community Animal Health Workers;  developing national preparedness and control emergency plans; supporting the draft of contingency plans; reviewing vaccination procedures; conducting table top simulation exercises; establishing a functioning commercial system linking the CAHWs and the drug stores.

• A number of guidelines and manuals (on biosecurity measures, diagnosis and control of animal diseases, as well as Livestock Emergency Guidelines) have been produced and generally distributed as extension materials.

4.2. Nutrition and Consumer Protection (incl. food safety)

• On average, CD activities related to nutrition and consumer protection get some 8% of all CD-focussed projects at national level. Yet, quite a huge discrepancy emerge between those countries (Somalia, South Africa, Lesotho, Rwanda, and Benin) where the focus on these issues is strong and (many) others with no project at all. No striking divergence rather figures out among the sub-regions.

• On the nutrition side, CD interventions mainly target individuals (trainers, farmers and students). Manuals on nutrition and the importance of dietary diversification have been developed and applied to horticulture and home gardening techniques in school gardens as well.

• More on the consumer protection and food safety/quality side, trainers, food inspectors, and producers have been trained in the application of good hygiene practices in food processing, food microbiology, analysis for mycotoxins in food, as well as repairing and maintenance of lab instruments and equipments. In addition, a regional project aimed at strengthening the Production and Quality Control of Gums and Resins in Africa through the set up of a quality control system, transferring technologies and providing training on monitoring quality production. Finally, a lot of CD work on food safety and quality has been carried out within the PCA Framework agreement: food control systems have been set up and training to national officials and food inspectors organized.

• National training and awareness workshops have also been organized to improve the understanding of the work of the three standard setting bodies (Codex Alimentarius, IPPC, and OIE) and their linkage with the SPS Agreement. National Committees on Codex have been set up, and the system for the control of food safety generally strengthened by enhancing the collaboration among different stakeholders and discussing/revising any inconsistency with international sanitary and phytosanitary norms and measures.

4.3 Plant Production and Protection

• Indeed, plant production and protection is, on average, the area where FAO CD efforts are concentrated into (with some 14% of CD in-country activities). This has been mostly true

13 Evaluation of FAO’s Capacity Development Activities in Africa in East and Central Africa, where more than 20% of CD activities focus on crops production and plant protection.

• The set up of FFS has been the primary mechanism to develop capacities (of farmers through extension officers), for what concerns the production of specific plants and crops (cassava, manioc, banana, rice among others), IPPM or more specifically only on surveillance/protection from plant diseases (as the banana bacterial wilt or the cassava mosaic Disease). Trainers from extension services have been instructed on farming techniques (including some very innovative ones, like the one for NERICA rice), seeds quality control, plant breeding, crops production and post-harvest storage. Farmers’ groups have been constituted and farmers trained. Training has in a few cases been extended to farm management (admin and record keeping).

• At a strictly policy level, consultations with Ministries and representatives of NGOs and sub-regional organizations have been organized within the context of the African Seed and Biotechnology Programme for African Union Members. FAO has provided support to National Seed Policies, including in a few cases through the set up of round- tables/institutional frameworks and coordination mechanisms where all the actors involved in seeds production and distribution have been invited to participate.

• On the plant protection side, interventions have ranged across the three dimensions, but mainly focused on:  Training of national Government officials and plant health inspectors on phytosanitary measures, procedures and systems (incl. on fruit fly surveillance, management and eradication through a regional project)  The exchange of official phytosanitary information under the revised IPPC (regional project). Regional and in-country workshops are conducted, as well as training materials developed to allow for the exchange to occur and phytosanitary information systems to be established;  An African Centre for Phytosanitary Excellence was set up in Kenya, with a unit for applied pest risk analysis and a training center in there.  Revision of phytosanitary legislation (to be coherent and in line with the IPPC) and regulations, including national quarantine border strategies for pest exclusion, procedures for pests’ listing and surveillance data management

 Training (of national teams) to address the fight against desert locusts (mainly through regional projects), which have included – among others - sessions on pheromone biology and the effects of Green Muscle; desert locusts control techniques, preparation of protocols and field testing, field demonstrations of application; development of Elocust data, introduction of RAMSES and the use of satellite images;

• A few workshops on biosafety and the use of GMOs in agriculture have been organized, together with hands-on training courses and study tours for officials from regulatory agencies and inspectors. Knowledge resources on the application of biotechnologies in agriculture have been there distributed.

4.4. Rural Infrastructures and Agro-Industries

14 Evaluation of FAO’s Capacity Development Activities in Africa

• FAO has not massively intervened to develop capacities in the area of rural infrastructures and agro-industries. On average, some 5% of CD-focussed projects have concentrated on this sector, mostly in few countries (Ghana above all, followed by a couple of projects each in Ethiopia, Eritrea, Niger, Cameroon, and South Africa). Southern Africa as whole figures out more prominently (9%), yet this being mainly due to fewer projects implemented there in absolute terms.

• CD activities have mainly targeted individuals (farmers and artisans, with attention paid to youth and gender equality issues), who have been trained on agro-processing, post- harvest handling (e.g. on manufacturing and handling metallic silos for grain storage), and farm management (incl. business practices and marketing). Training has included study tours and demonstration sessions/hands-on training; development and distribution of manuals for procedures of storage management and laboratory quality control; as well as sessions on business planning and monitoring. Within the framework of UN DaO JP1 in Tanzania, staff from LGAs has been trained on agribusiness, value chain concepts, methodology and selection per region. Activities have been jointly managed and backstopped with UNIDO.

• In addition, in quite a few cases, CD activities have also focused on the enabling environment dimension, by grouping producers together, setting mechanisms for Public Private Partnerships and facilitating the links among input suppliers, CBOs, market outlets, and financial institutions.

• For what finally concerns the organizational dimension, FAO has supported the enhancement of the national food processing and enterprise development training centre of Malkers, in Swaziland, where training on food processing, value-addition, marketing and micro-enterprise development.

4.5. Statistics and Information Systems

• Even excluding CountryStat (which is the main ongoing CD activity in terms of budget resources and spread, see below) from the count of projects and activities, statistics and information systems are still a very important area of CD intervention for FAO, with some a share of 9% of total CD efforts through national projects in SSA. There are no huge discrepancies among countries and sub-regions, with the only exception of West Africa (where the “share of CD relevance” is much lower than other sectors’, as pointed out above).

• On statistics, CD support unfolds mainly across the organization component, through the review of estimation methodologies and the development of data entry and processing program, especially in preparation for the National Agricultural Census. Operators (at central and decentralized levels) are trained, and study tours on specific topics (e.g. on agropastoral stats or crop monitoring systems) are organized. Training on statistics and socio-economic analysis is also carried out; with a view to better inform national strategies on rural development. • As pointed out above, recent CD efforts on statistics have derived from, and moved into, the implementation of the CountryStat project. CountryStat has been introduced in 17 African countries, national officials – mainly from the National Institute of Statistics but also (to a different extent and in different ways in various countries) the MoA - are trained, and institutional links (ideally) built up for handling interconnected statistical information and visual indicators on food and agriculture.

15 Evaluation of FAO’s Capacity Development Activities in Africa

• On food security and nutrition vulnerability information systems, CD efforts have concentrated as well on the organizational (and consequently also on the individual) component, through the introduction in country of FIVIMS and IPC classification systems. Here as well, training, workshops and study tours have been organized for national officials in data collection and methodologies for analysis, and training materials/tools developed. In quite a few countries, training has included drought monitoring, preparedness, mitigation and response, as well as (livestock) early warning information systems’ management in semi-arid areas. In a few cases, a dedicated food security monitoring unit within a Government Ministry has been set up, and coordination mechanisms among different organizations assessed, with ideal roles and responsibilities defined. • The EC/FAO Food Security Programme is similarly focused on strengthening analytical capacities in key national institutions through the provision of equipment, tools and training for the management and analysis of food security information and for the dissemination of the derived outputs. The Programme also aims to work on the policy/enabling environment CD dimension, by ensuring coordination among national and international stakeholders for a better use of synchronized data on food security for policies.

• On information systems for management, a few projects have provided for the development of low-cost marketable software prototype for agricultural cooperatives: staff and members have been trained in the operation and use, and training material on farm management surveys produced (e.g. TCP project in Kenya, which aimed at enhancing the capacities of a milk agricultural cooperative by computerizing its production management system).

• On land and water data and information systems, please see the following section on natural resources management.

• On information networks, KCEF has worked through few field projects on the development and customization of software and applications for the web repository of documents, in collaboration with universities (as in Kenya) and Government-affiliated institutions (GAINS in Ghana). Agriculture information networks are designed, in a few cases down to decentralized level where training workshops for NGOs and CBOs in pilot districts (as in Uganda) on media communication and information seeking skills are organized.

4.6. Trade and Markets

• CD work on trade and markets gets some 1% share of all CD activities through national projects. CD activities through punctual interventions in this area have mainly dealt with: the collection of agricultural markets’ data and strengthening marketing extension services (e.g. in Rwanda and Lesotho), set up of market linkages in seeds production (e.g. in Liberia), assistance to the organization and management of urban markets (Mozambique), and – within the framework of a regional project – training and policy advice for Government officials who are involved in WTO negotiations.

• At regional level, a project in West Africa has focused on the set up of an informal network for marketing of products from organic agriculture. Training in production, harvesting and inspection for organic agriculture and fair trade has been carried out, and partnership agreements signed between farmers groups and importers.

16 Evaluation of FAO’s Capacity Development Activities in Africa

• Another regional project, in East Africa this time, has focused on the set up of Market Access Companies and building commercial relationships between MACs and market chain players. Training on managing the online data platform has been provided, as well as workshops on how to use mobile technologies to access the data base have been run at national and district level.

• The international “All ACP Support Programme on Agricultural Commodities” may be included in this section, although its implementation has just started in most of the target countries and, mostly, its activities span across few technical areas (including plant protection). The idea is to support the creation of professional producer organisations and empower them to negotiate with other commodity stakeholders, through training on key managerial and marketing aspects, the development of producer-focused business development services, including instruments to ease the access to market. So far, a few workshops have been organized in West and East Africa to start linking producers’ organizations.

4.7. Fisheries and Aquaculture

• On average, CD interventions in the area of Fisheries and Aquaculture get a 7% share of all CD-focussed national project activities, with no major discrepancies among sub-regions (if we exclude a slight increase – up to 11% share – in Central Africa). Yet, it is worth recalling that quite a number of CD activities (especially along the policy/enabling environment dimension) are implemented in the framework of inter-regional and global projects.

• For what concerns the individual dimension, fish farmers are trained on new techniques and methodologies for hatchery and ponds’ management operations and - within the context of the SPFS – also on improved boat construction and maintenance for artisanal fisheries. • In aquaculture, farmers are also increasingly trained on aqua-business planning and record keeping. Guidelines are developed and resource management models prepared, ideally for the use of the farmers themselves. • In quite a few cases, efforts are there to form fisheries management groups (as in the case of national prawn fisheries or aquaculture associations), and training/workshops targeting cluster hub operators on joint management organized.

• On fisheries animal health, a regional training on diagnosis of and surveillance for Epizootic ulcerative syndrome has been carried out, and regional strategy on disease epizootics and capacities strengthening for aquatic animal health management finalized.

• FAO has also worked on the improvement of sanitary inspection and the quality of fishery products. Administrative officials and fishermen are trained in the inspection of fishery products and appropriate related technologies, including on traceability, eco-labelling, fair trade, laboratory practices, and quality improvement at beach level. FAO has also developed guidelines for establishing an aquatic animal health certification and quarantine inspection system. Workshops on the legal framework for responsible aquaculture development have been also organized, and government officials and fish industry representatives trained on the concept and implementation of HACCP quality assurance programs.

17 Evaluation of FAO’s Capacity Development Activities in Africa

• CD activities related to data management also significantly target national Governments and, to a lesser extent, staff from industrial fisheries. Strategies for the collection and analysis of data to monitor maritime artisanal fisheries are drafted, data management systems (incl. ARTFISH) introduced, and training on data collection (at decentralized level) and homogenization (at central level) carried out.

• At regional level, a few projects contribute to the enhancement of Governments’ capacities in handling and facing illicit and non-regulated fisheries through the definition of a strategy and a Plan of action, and the organization of a few training workshops. • In addition, in the CECAF region, FAO has contributed building up the capacities of staff from national fisheries research institutions to assess and monitor the status of pelagic and demersal resources, and to analyse fisheries management and exploitation options to ensure a sustainable use of such resources in both off-shore and coastal fisheries. A coordinated annual work-plan has been drafted, working groups created, and a CECAF strategy for linking NGOs and civil Society in the formulation and implementation of fishery management plans drafted. • Still, for what concerns organizational and institutional capacities development, FAO has provided support to the Comité Régional des Pêches du Golfe de Guinée (COREP) by setting up coordination mechanisms and training national officials on project planning. • On the other coast, FAO has facilitated the formation and initiation of a Regional Fisheries Arrangement for the management of sustainable non-tuna fisheries in the South- West Indian Ocean, where training and regional workshops on fisheries management and management of shark fisheries have been organized. In addition, support has been provided to the Lake Victoria Fisheries Organization to act as a hub and in the development of a medium- long term work plan.

4.8. Forestry

• CD work on Forestry issues through projects is not getting a high share of all CD- focussed projects, some 5% regionally without much discrepancy among sub-regions when looking at activities carried out only within national projects and not including TCP Facilities. Interventions are logically more focused in few countries, such as Congo DRC, Liberia, Mozambique, and Mali. Yet, as noted above, Central Africa as a whole does not probably get many resources for CD, at least not through national projects.10

• Most of the CD work on forestry unfolds around the policy/enabling environment dimension, by supporting the development of: i) national laws on forest resources management; ii) forestry sustainable development strategies; iii) institutional reforms and decentralization of the forestry sector; iv) development of transparent and competitive forest concession allocation procedures; v) national strategies for the promotion and valorisation of non-wood forest products. • This is done through the organization of workshops and training (incl. on policy and participatory techniques, land-use planning, community-based enterprise development using the Market Analysis and Development approach), the set up of platforms involving different stakeholders from civil society and private sectors as well, in quite a few cases down to decentralized level as well. In one case, the FAO Investment Centre has also provided support

10 These figures and partly also the description of the activities below do not include CD activities carried out within the framework of FNPP and FMPP programs.

18 Evaluation of FAO’s Capacity Development Activities in Africa to the compilation of the Farm Forestry Field School Management manual, used in ToT courses. • In a few cases (e.g in DRC and Gambia) CD activities have also targeted the staff of universities and agro-forestry research institutes, through training (also on the impact of HIV/AIDS on agriculture production and participatory curriculum development).

• In Central Africa, a regional project aimed at supporting COMIFAC, through policy support and the set up of national fora to validate the national components of the convergence plan.

4.9. Natural Resources Management and Environment

• Natural resources (land and water) management is a key area for FAO CD work. This being true almost everywhere in the region (with some 12% average share of total CD- focussed projects), CD efforts have been most significant in Southern Africa and few other countries, like Eritrea, Somalia, and Burkina Faso.

• At farmers (individual) level, FAO develops capacities mainly through the set up of FFS on land and water management, including on conservation agriculture, smallholder irrigation and water conservation techniques. Farmer field days and demonstration sessions are organized, participatory land and water use plans are sometimes developed and village councils/users’ networks set up. In a few cases, these interventions are scaled up to reach the policy level, where assistance is provided to the formulation of national policies on conservation agriculture and strategies for institutionalization of FFS. • Some training on technical, economical, and environmental aspects in soil and water conservation measures has been carried out within the framework of the ACACIA operation projects (technically backstopped by the Forestry Department). In addition, a few regional projects implemented in Southern Africa have specifically focused on training irrigation professionals both from the public and private sectors on design, construction and management of sustainable smallholder irrigation projects.

• On the management of transboundary (water) resources, FAO CD efforts have mainly concentrated on technical assistance and data management work, plus significantly on the policy/enabling environment dimension. In this respect, it is worth mentioning three projects:  The Nile Basin Initiative has focused on the development of information products for decisions on water policy and water resources management. Training on field data acquisition (through GIS tools as well) and processing has been carried out, and an Internet forum and user groups for hydro-meteorological network operation established. Training workshops on negotiation skills for the joint management of transboundary resources have been organized, and a Nile DST Task Force set up, also to work on the harmonization of national and indigenous water laws.  A framework and an implementation strategy for the management of the Lower Kagera River Basin has been developed, working at the policy/enabling environment dimension by providing legal and institutional review, and setting up a framework for stakeholders’ participation for information sharing and monitoring;  The Environmental Protection and Sustainable Management of the Okavango River Basin project has mainly developed capacities along the organization (OKACOM) and policy dimension, by working on the preparation and implementation of the Strategic Action Programme.

19 Evaluation of FAO’s Capacity Development Activities in Africa

• On land tenure, CD interventions mainly target national official by organizing ToT on a number of issues, including on: land use assessments and GIS software application for cadastrial mapping, cartographic and topographic techniques, agro-economic analysis for land concessions. Guidelines and best practices for dealing with access to land and natural resources, restitution of rights and land/property disputes have been produced. • Support has been provided to the digitalization of land administration information and the set up of an integrated information system for land registration. • Participatory territorial planning has been as well promoted, and women’s groups that advocate rights of women and other minority groups in relation to land created. • In Sudan and Angola11, FAO has also supported the development of a legal framework for land tenure and property issues, and worked through a few projects at least on the creation of an enabling policy, legal and institutional framework to coherently address policy, legal, and advocacy issues related to land tenure.

• On environment and climate change, training has been in a few cases (e.g. in Lesotho) provided on the risks and impact of climate change, as well as on viable production strategies and practices. This has included also field demonstration of adaptation practices at community level. • In addition the CARBOAFRICA regional project on “Qualification, understanding and prediction of carbon cycle and other GHG gases” works along the policy/enabling environment dimension, by enhancing institutional capacities to engage in the international debate on all aspects of climate change, through the harmonization and exchange of methodologies for flux measurements and ecological sampling and the delivery of Clean Development Mechanisms rules and practices to Government agencies.

4.10. Food Security (The Special Programme and other holistic interventions)

• The Special Programme for Food Security has been extensively implemented throughout Africa and, together with other “holistic” interventions on food security, undoubtedly constitutes one of the key areas of FAO CD interventions. It gets an 11% share of all CD-focussed national projects, going up to 15% in West and Central Africa12. • The SPFS mainly targets individuals (farmers and extension workers). It aims at developing capacities essentially through the organization of demonstration sessions and training on techniques (ranging from crop production and livestock rearing to agroforestry and natural resources management) and, in quite a few cases, soft and management/commercialization skills. The South-South Cooperation approach has been used extensively as a tool.

4.11. Policy Support

• According to what stated in ProDocs, FAO work on agriculture and food security policy includes a significant CD component. The share of CD-focussed policy support projects (out of total CD activities at national level) is consistently around 12% throughout the region, this figure almost doubling when including support through regional initiatives.

11 Indeed, Angola and Sudan prominently, and quite uniquely, emerge as the two focus countries for this type of work 12 SPFS diversification components (e.g. on water management or livestock) have not been included in the count, rather included in specific technical areas.

20 Evaluation of FAO’s Capacity Development Activities in Africa

• CD support in the policy area has included, among others13; i) assistance to the definition of national Agriculture Sector Development Strategies; ii) advice on the interpretation and implementation of international norms and standards (e.g. on food safety, animal and plant health, trade); iii) assistance in the preparation of Medium Term Programme and the formulation of bankable projects in support to the CAADP implementation; iv) support to Government in the formulation of projects (including adequate monitoring) within the framework of the Special Initiative on Soaring Food Prices; v)introduction and institutionalization of RuralInvest (by TCI).

• At regional level, on-the-job training on situation analysis, identification, formulation, appraisal, monitoring and evaluation has been provided to staff of the ECOWAS Secretariat, and plans and guidelines for implementation of Regional Programmes for Food Security developed. In addition, FAO has produced guidelines containing a set of criteria for prioritizing, selecting and operationalizing agriculture programmes and projects for use by NEPAD.

• CD work in the policy area has in quite a few cases been accompanied by the revision of the MoA’s organizational structure (as in Kenya), the set up of Task Forces and Working Groups, and some training on PCM, monitoring, communication negotiation skills. • In Sudan, a total of USD 51 million has been allocated (within the framework of two projects) specifically to “build human, organizational and physical capacities of key emerging public and private institutions in order to provide administrative, advisory and strategic guidance to the rural areas”. Together with the provision of equipments and physical means, training, workshops and study tours on priority areas (including on project cycle management, strategic planning, computer literacy, conflict prevention, land tenure, re-integration of returnees) have been run. A number of CD activities also target CBOs and non-state actors, such as setting up FFSs, strengthening community-based animal health networks, and assisting the emergence of private institutions and rural businesses organizations.

4.12. Emergency Operations and Rehabilitation

• The CD component of FAO emergency operations and rehabilitation projects is quite important. This kind of CD activities constitutes on average a tenth of all the work FAO does on CD, with naturally higher peaks in few countries in East and Central Africa (e.g. Sudan, DRC, Central African Republic, Uganda, Chad, Ethiopia, Somalia), plus Zimbabwe, Ivory Coast, Liberia, and Sierra Leone.

• CD interventions mainly target the capacities of IDPs and farmers, through the set up of FFS and the distribution of seeds. Yet, as stated in the introduction, these interventions have been considered as part of the evaluation scope only – and these were not isolated cases – when accompanied by interventions along the enabling environment dimension, e.g. providing for the set up of a seeds distribution chain.

• Quite a few national projects have rather focused on organizational strengthening (mainly through the set up of food security and vulnerability information systems, see above) and coordination issues, by supporting the Emergency Units within the MoA and strengthening synergies among different stakeholders working on humanitarian issues.

13 For policy support to specific technical areas, please see related sections.

21 Evaluation of FAO’s Capacity Development Activities in Africa

• As hinted above (in the policy section), FAO CD interventions in this area have also recently increased in the context of the Special Initiative on Soaring Food Prices. Together with punctual inputs distribution at national level, a few regional initiatives have focused on strengthening the capacities of national officials through the establishment of food situation monitoring and evaluation mechanisms/ data bases, as well as related training workshops on project monitoring and assessment methods.

4.13. Gender and HIV/AIDS

• CD interventions specifically focusing on gender and HIV/AIDS issues do not represent more than 2% of CD activities carried out at field level through national projects, this share going high up to 5% in Southern Africa, thanks to few projects in Mozambique, Tanzania, and Zimbabwe.

• CD activities through projects in this area have mainly focused on the individual dimension and included gender equality either as the aim and/or the content of the interventions. In this double respect, FAO has set up FFS to specifically address HIV/AIDS, Gender (and Gender Based Violence - jointly with UNIFEM, UNFPA, UNAIDS, and UNDP in Uganda) concerns. Here, and elsewhere (e.g. in Tanzania within the framework of the JP1) gender-sensitive rural information technologies have been used to promote awareness and develop capacities. • In addition, in quite a few countries (e.g. Mali and Cape Verde) trainers have been trained on the use of the SEAGA tool kit for inclusion of gender into socio-economic analysis. The training was in a few cases extended to NGOs staff, para-jurists and politicians.

• In Southern Africa, FAO has also provided support (through regional projects) to HIV/AIDS affected people by developing their capacities (through training essentially) and also by strengthening local districts’ capacity to coordinate the interventions of various organizations involved in food security and HIV/AIDS work and integrate the support to OVCs and PLWHA in district programmes. Resource materials on HIV/AIDS have also been developed within the framework of the PCA Norway Program (2008-09), and a reporting system to monitor HIV/AIDS on forestry-dependent rural communities developed.

4.14. Other (essentially CD work to strengthen farmers groups and associations)

• The CD work carried out by FAO to strengthen farmers groups and associations has evolved around the three dimensions, through:  Training (and production of training materials) on administrative, financial and business procedures for agricultural cooperatives development; organization of regional workshops on planning, development and monitoring of small rural development projects;  Support to the set up of professional farmers organization platforms, cooperatives, producers’ union, micro-enterprises and savings groups;  Draft guidelines for farmers fora development and support to draft a national strategy for sustainable farmers group development  Enhancing institutional links among farmer organizations and with national Governments (at central and district level)

22 Annex 8

An Assessment of FAO’s Normative Products Related to Capacity Development in Africa

A Component of an Evaluation of FAO’s Capacity Development Activities in Africa

Background

This assessment of normative products forms part of the evaluation of Food and Agriculture Organization’s (FAO) capacity development activities in Africa.

The terms of reference for the assessment of normative projects called for a purposive sample of guidelines, manuals, compact discs (CD-ROMs) and e-learning tools to be evaluated for quality, relevance and, where appropriate, effectiveness, including their: • accessibility to the target audience; • relevance to the objectives of the country contexts; • ability to be interactive; and • potential to remain relevant over time or to adapt to change.

The terms of reference stipulated that the assessment be a desk review.

Methodology

Methods and analytical tools The evaluator spent a week at FAO Headquarters in Rome in June 2009 where he met with the evaluation’s core team and interviewed 27 staff members from FAO’s divisions about the design, development and distribution of normative materials, with particular reference to FAO’s work in Africa. A list of people interviewed appears Appendix A.

The core evaluation team prepared a preliminary list of materials to be assessed, based in part on the recommendations of each division. The evaluator developed criteria to guide the final sample selection, as follows: • developed and/or used by FAO and its development partners between 2000 and 2008; • two or more samples from each department; • illustrative of the quality of the materials produced and distributed by the department; and • intended for use in developing capacity, broadly defined, particularly in relation to Africans.

Based on these criteria, the core team settled on 31 products (listed in Appendix B). The products included some of the major learning resources and recent capacity development initiatives of FAO, such as the Information Management Resource Kit (I-Mark), EasyPol, the Codex Alimentarius Programme, and the Food Security Information for Action materials.

Table 1 outlines the criteria the evaluator applied to assess the materials:

Table 1 Criteria for Assessing FAO’s Capacity Development Materials

Content Based on identified need / demand-driven Relevant to & consistent with the target country’s(ies’) development priorities & contexts Potential to remain relevant over time & to adapt to change Targeted at specific groups & organizations Cognitively accessible to target audience Appropriate level of information for target audiences Technically accurate Peer reviewed Current, up-to-date information, complete in scope Gender sensitive Gender equality principles well integrated Culturally appropriate (reflects target audience’s customs, languages, norms, beliefs) Rights-based Design, production Developed with the participation of intended beneficiaries Rigorously field tested in the development stage Produced in collaboration with other departments at FAO Produced in collaboration with other UN orgs Produced in collaboration with non-UN orgs Produced in collaboration with African governments Accessible to target audiences in relation to language, visuals, length & format Adequate guidelines explaining uses/applications User-friendly (contains helpful directional tools such as table of contents, indexes, maps, pointers & graphics Designed for interactive usage Information presented clearly & organized logically Strong production values Promotion, distribution Promoted to target audiences using a communication strategy & plans Accessible to target audiences in Africa at an appropriate cost Distributed via multiple channels (e.g., governments, libraries, universities, partners, Internet) Use Extent of use by target groups Used for capacity building at more than one level (individual, organizational, policy/institutional) Part of a larger program, directly or through partnerships. Embedded in larger program rather than one-off standalone Spin-off uses—unexpected uptake, multiple uses Feedback mechanism to FAO on relevance, effectiveness & use Evidence of a contribution to development results

Methodological constraints The evaluator encountered two major methodological constraints. First, the enormous volume of materials produced between 2000 and 2008 for capacity development purposes made it impossible to select a representative sample for the assessment, coupled with the limited available time and resources. Second, the evaluator was unable to visit the field to assess the extent to which the materials had been used or to see the many products that are produced in the field and that are unavailable at headquarters. Instead, he drew mainly from the data FAO staff could provide and from feedback from members of the evaluation team who conducted field visits in Africa. 3

Despite these constraints, the evaluator is confident that the findings and conclusions presented are valid and evidence-based.

Introduction

With about 1,600 professional staff and 2,000 support personnel, FAO’s mandate is to raise levels of nutrition, improve agriculture productivity, better the lives of rural populations and contribute to the growth of the world economy (http://www.fao.org/about/mission-gov/en/). One of the Organization’s four main areas of activity is “putting information within reach” in order to aid development (http://www.fao.org/about/about-fao/en/). In this role, FAO functions as a knowledge- based organization. Its agronomists, foresters, livestock and fisheries specialists, nutritionists, social scientists, economists, statisticians and other professionals collect, analyze and disseminate data via the Internet, workshops, seminars and multi-media channels to a wide range of audiences. The Organization publishes hundreds of newsletters, reports and books, distributes several magazines and thousands of CD- ROMs, and hosts dozens of electronic fora.

As a UN agency and an authoritative source of information, FAO plays a normative role by setting international standards, promoting these standards and establishing guidelines and effective practices in matters pertaining to its wide mandate. The Organization has set ambitious goals for itself in relation to capacity development, which is now one of FAO’s eight core functions.1

This assessment deals with the intersection between FAO’s normative products and capacity development. What is the nature of that relationship? Before presenting the findings of the assessment, it is important to deal with this question because the answer sheds light on the assessment’s limitations.

It is generally understood that capacity development is a complex process which takes place at several levels—individual, organizational and institutional—over time, and that it is often influenced by a host of factors, both formal and informal, pertaining to the development environment and organizational context (Baser and Morgan, 2008). Developing capacity can involve many different external interventions beyond training and technical assistance, and it is known that capacities can develop without external assistance (Land, Hauck and Baser, 2009). With this understanding, normative products supplement the process of capacity development, serving as tools to aid the development of capacity. The high quality of normative products, whether e-learning courses, guides or manuals, does not guarantee that capacity will be developed and sustained. It is how such products are made available, used and combined with other processes and contextual factors that is key to developing capacity. That is to say, normative products

1 For the purposes of this assessment, capacity development is defined broadly as the “process whereby people, organizations and society as a whole unleash, strengthen, create, adapt and maintain capacity over time” (Organization of Economic Cooperation and Development/Development Assistance Committee [OECD/DAC], 2006). 4

are important, but on their own most are likely to play a limited role in relation to developing the capacities of FAO’s partners.

Owing to the limitations of this assessment, the review is focused largely on issues of relevance, quality and utility rather than on the products’ impact.

Findings

Approach Many of the products reviewed stem from discrete projects supported by donors through extra-budgetary assistance. Some of these are one-off initiatives, which reflect donors’ short-term project orientation. In such cases, a need is identified, funding is secured, and a normative product is produced and distributed, but there is often little follow up. Sometimes FAO involves local institutions and encourages them to use normative products in their education and training programs, but only occasionally are there sufficient time and resources to strengthen the capacity of these institutions to follow through. The Codex Alimentarius Programme is one of several exceptions: it comprises learning materials, a trust fund and training programs to assist new member countries to participate in establishing food standards.

There is a trend on the part of some donors toward a more programmatic approach involving multiple-year funding, multiple interventions, a range of partners and longer timeframes for capacity development. A programmatic approach usually provides opportunities for follow up on the ground with beneficiaries, a range of interventions and monitoring and evaluation mechanisms which, if carried out well, can assess results, stimulate learning and lead to program improvements. FAO’s Multidonor Partnership Program is an important step in this direction. It brings donors into a programmatic approach with a wide variety of interventions, and it is tied in with institutional reforms of participating governments. This trend should be encouraged.

There are numerous examples where FAO has followed this approach. The Sustainable Agriculture and Rural Development Initiative, which links resources, expertise, knowledge and technologies to the needs of rural communities and disadvantaged groups, is one example. It has produced an excellent web-based series of concise, practical solutions to technical problems, such as the guide to Biocontrol-based Integrated Pest Management for the Diamondback Moth, as part of a larger program of technical assistance involving government and non-governmental organizations and institutions.

CountrySTAT, which is a framework for international cooperation on agricultural statistics, has the potential to become a programmatic approach if it receives sufficient, long-term resources. To date, FAO’s Statistical Division has produced training materials, technical guides and new software applications, and has developed an ongoing consultative process with national working groups, coupled with training sessions, technical back up and pilot initiatives.

Although these programmatic initiatives are commendable, most are confined to technical matters and rarely grapple with the formal and informal systems of institutions and their enabling environments, all of which have a major bearing on developmental outcomes. Relatively few normative products deal with the “soft” side of development, such as matters pertaining to organizational culture and the systems of incentives and rewards that shape individual behaviour.

FAO staff told the evaluator that donor funding restrictions often prevent them from taking an integrated approach to capacity development where, for example, programs could deal with agriculture, fisheries and forestry resources together. One said: “We need money that is not linked to specific countries for upstream investment in capacity development materials.” But FAO staff cannot always have it both ways, and donors should not shoulder the entire blame. FAO and the donors must work together to ensure that funding for capacity development is spent on well-designed programs that adhere to best practices and that reach the intended beneficiaries.

One further observation with regard to FAO’s approach is that many of its normative products of a technical nature feature experts talking to experts and sometimes experts talking to farmers. Although this is often justified, lessons from the development community suggest that reversing the power balance, so that experts respond to the questions put to them by farmers and/or farmers speak directly to other farmers, helps to foster behavioural change and the uptake of effective practices. This FAO has done this through its exemplary Farmer Field Schools model in Africa and elsewhere. Some divisions of FAO appear to be following a similar pedagogical model. For example, the recent capacity development approach of FAO’s Nutrition and Consumer Protection Division is to equip partners with tools to make their own decisions rather than telling them only what the experts recommend. The Division’s training tool, Les bonnes pratiquies d’hygiène dans la préparation et la vente des aliments de rue en Afrique, for example, emphasizes long-term, two-way dialogue between trainers and street vendors as a means of encouraging good hygienic practices, and the training package serves as a tool to support that change process. The approach recognizes that experts simply telling street vendors what to do often results in little positive change.

Needs What drives the development of FAO’s normative products? How are needs identified? Most materials produced for capacity development are based on needs identified in the field and/or at international fora. Of the materials reviewed for this assessment, all appear to have been based on an identified need, sometimes at the grassroots level, but more frequently by experts, country representatives and communities of interest. Most, but not all, appear to be relevant to and consistent with the target countries’ development priorities. Some highly technical products, the Information Management Resource Kit (I- Mark) for example, are useful, but may not be among the highest priorities of some under-developed countries, such as those in sub-Saharan Africa. At FAO headquarters, there is no formal mechanism to align normative products with the development frameworks of target countries. An assessment of the extent to which this happens in the field through ‘One UN’ initiatives was beyond the scope of this evaluation. 6

FAO’s forestry division says that project experimentation and successful field initiatives determine the norms and provide the basis for the development of normative products. Most other divisions also emphasize the need for a strong field presence in setting priorities and developing relevant products. Indeed, many of the most relevant products are produced in the field where capacity gaps are identified. Although this is no guarantee of effectiveness, such products are often best adapted to local and regional contexts.

Some products, such as those produced under the Codex Alimentarius Programme, are determined at special international meetings involving government departments, non- governmental organizations and experts concerned with food standards. Sometimes needs are identified by representatives of member states or regional commissions. However, in some instances, FAO staff claim that it is too costly to verify these needs. Often UN global agendas, such as those related to the Millennium Development Goals, influence what normative products get developed. This is inevitable for any UN organization and often works best when several UN organizations collaborate. Collaboration has the additional advantage of reducing the chance of duplication. The evaluator notes that some of the information in several of the products reviewed for this assessment is available from other sources, including on occasion other UN agencies and other FAO divisions. For example, some of the content in the I-Mark series is available from UNESCO and some of what is packaged in the Irrigation Guidelines as part of the Land and Water Digital Media Series can be found in other FAO resources aimed at developing capacity. Much of the information on participatory development tools and techniques, which appears in several of FAO’s normative products, are available elsewhere. This is not necessarily wasteful if the information has been adapted for different contexts.

The evaluator questions the need and priority of some products. Sometimes, either the information is available elsewhere without the need to adapt it, or the impetus for developing the product is driven by the producers’ themselves, as was the case with a new initiative that was brought to the evaluator’s attention. However, these cases appear to be few and are usually confined to highly technical areas where there may be a small but dedicated community of interest.

Are there important needs that have not been met? Whose capacity development needs are most important? Although the scope of the assessment did not allow for an in-depth exploration of these questions, they are, nonetheless, important. The evaluation suggests that there are unmet capacity gaps at the municipal level in Africa, owing to the devolution of government programs and services dealing with such important matters as food safety. Many of FAO’s normative materials are geared toward higher levels of government and specialized institutions. In Mali, for example, local representatives reported that the terms used in some FAO instructional manuals were too advanced for frontline staff and that some topics were too difficult for trainers to handle.

Relatively few products are available to guide cash-strapped local governments, which often lack expertise and relevant institutional supports. Is it the role of FAO to target local governments, and does it have a comparative advantage when it comes to meeting 7

needs at this level? These are questions beyond the scope of this assessment, yet important for FAO to answer in relation to its overall capacity development strategy.

Some capacity development materials appear to be geared toward mid-income countries and may not be suitable for much of Africa where capacity is often weakest. This may be true of parts of the I-Mark series and of such products as System of Integrated Agricultural Censuses, and Preparation of Contingency Plans for Foot-and-Mouth Disease. According to feedback from the field, there is demand for more materials to be adapted to local conditions and produced in local African languages, particularly where literacy levels are low among farmer groups, as in many sub-Saharan African countries.

The utility and uptake of some of the best of FAO’s materials is limited by lack of effective access. This is discussed next.

Access and formats One of the most consistent and pervasive findings of the beneficiary assessments and field visits conducted for this evaluation was that distribution of, and access to, FAO’s normative products are weak in Africa. Many local libraries and FAO offices have inadequate supplies of FAO’s most important materials for capacity development. FAO resource libraries in some African countries, such as Ghana, have been closed, according to the evaluation team. The field evaluators also heard repeatedly that locally engaged consultants and contractors were not given relevant FAO materials and were often engaged to produce new ones without reference to materials already produced. Such was the case with soil and irrigation learning resources in Zimbabwe. The reasons for this are varied. In Africa, significant gaps exist in FAO’s distribution chain and there are problems related to the formats of some materials, as discussed next.

Print is by far the most prevalent format for FAO’s normative products. The Organization prints hundreds of manuals, guides and learning materials each year. It distributes hard copies often through its regional offices and partner institutions, and increasingly it makes these materials available online and on compact disk (CD-ROM). Most are free to download from the Internet, but some CD-ROMs, such as those for Codex Alimenatius Programme and the Land and Water Digital Media Series (at US $40-$45), are prohibitively expensive for most Africans.

Although FAO’s use of the Internet significantly extends the reach of its products, it is not without drawbacks, particularly for Africans, for whom Internet access and bandwidth are often limited. Although Internet use in Africa is growing much faster than the world average, Internet penetration lags far behind, as shown in Figure 1. Africa has about 15 percent of the world’s population, but represents only about 4 percent of the world’s Internet users (www.internetworldstats.com, June 2009).

Figure 1 Internet Penetration, Africa 2009

Rest of World

World Average

Afr ica

0% 5% 10% 15% 20% 25% 30%

Source: Internet World Stats, June 2009

Not surprisingly, the top ten countries of Africa in relation to Internet use, as shown in Figure 2, are among the most prosperous.

Figure 2 Top Ten Countries of Internet Use, Africa 2009

Zimbabwe

Uganda

Tunisia

Kenya

Algeria

Sudan

South Africa

Morocco

Nigeria

Egypt

0 2 4 6 8 10 12 14 Millions of Users

Source: Internet World Stats, June 2009

Where there is Internet access in Africa, there are often problems associated with bandwidth. Without sufficient bandwidth, web pages can be frustratingly slow to load. Bandwidth in Africa is generally expensive, such that most universities cannot afford more than 1.544 Mbps, equivalent to the use in an average Western household with a high speed landline connection. With too many users sharing a small amount of bandwidth, African Internet service providers often experience congestion, resulting in disrupted access for users (www.inasp.org).

FAO staff have taken measures to overcome Africa’s Internet connectivity and bandwidth constraints. They have, for example, made CD-ROM copies of many normative products available for free or at low cost and they have begun to prepare 9

materials that take minimal bandwidth and that can be downloaded in sections and printed according to the needs of users. Despite such measures, connectivity issues remain a significant barrier for many Africans. According to some FAO staff, even printing from CD-ROMs is problematic in parts of Africa and reliable photocopying equipment is often in short supply. Some FAO staff interviewed contend that face-to-face learning through workshops and seminars is the preferred way of learning for most Africans. One staff member in Zimbabwe noted that by moving away from print, FAO was effectively transferring the cost of knowledge access to those most in need and least able to afford it. Many Africans interviewed by the evaluators in the field made a plea for continued print access in Africa.

Access difficulties persist even when users have reliable high speed Internet connections and plenty of bandwidth. For example, there is so much information on FAO’s website that it can be challenging for users to find a particular resource unless they know its precise name. Sometimes knowing the name of the document is not of much help. The evaluator, who has considerable experience with web-based information searches, was unable to find some of the normative materials that FAO staff recommended on the FAO website. The evaluator found it time-consuming to navigate among FAO’s various portals in search of data. Sometimes it took more than four clicks to find a document. Although there are site maps on the web pages of various divisions and special sites, such as FAO’s Best Practices Portal, there is no site map accessible from FAO’s home page. As noted later, some of FAO’s search engines, such as that of the Capacity Building Portal, fail to turn up many of FAO’s best-known normative products. Some key resources, such as the Land and Water Digital Media Series, which FAO invites people to purchase at significant cost, are not readily available from most public sources, such as university and municipal libraries, in the South and the North.2

A general concern, which applies to FAO and other organizations that automatically put most of their written materials online, is that they fail to adapt them for Internet use and learning. This forces many readers to download and print the materials for study offline. EASYPol has experienced this constraint. Print-based normative products for capacity development ought to be revised for online consumption since few readers are willing to wade through pages of dense text on their computer screens in order to get the information they are seeking. If the recent experience of EASYPol is an indicator, relatively few people are willing to download information.3

Increasingly, FAO staff are turning to distance and e-learning for capacity development. Among the available self-learning courses to date are those in the I-Mark series, Food Security Information for Action series and Codex E-learning series. Some argue that distance learning is cost-efficient considering that it can run up to $60,000 to hold one

2 For example, the evaluator was unable to find the Land and Water Digital Media Series in either of two major universities or the main public library in Ottawa, Canada’s capital. 3 At the time of this assessment, Commodity Chain Analysis, Constructing the Commodity Chain: Functional Analysis and Flow Chart, was EasyPol’s most downloaded document—at 336 downloads. The EasyPol document reviewed for this assessment, Influencing Policy Processes: Lessons from Experience, had been downloaded only twice, according to data on the EasyPol website 10

training event for 25-30 people when $250,000 will buy a 15-20 hour course which has the potential to reach thousands of participants. I-Mark, for example, has registered 135,000 users worldwide. This is quite impressive in itself, but what portion of these learners are Africans? Each mode of instruction has its strengths and weaknesses and likely a combination of classroom, field-based and distance learning is ideal.

A general weakness of many of FAO’s e-learning courses is that they do not always incorporate tutors and/or local peer learning groups. Both are considered effective practices (Tooth, 2000). This may, in part, explain why many who enroll fail to complete the courses even when they are designed in the effective self-paced asynchronous mode of most of FAO’s e-learning courses. For example, a 2009 assessment of Codex E- learning found that about three-quarters of those enrolled completed only one of the 13 lessons (Cwiek, 2009).

FAO makes limited use of some “old” but cost-effective formats, such as radio. It has yet to fully embrace new technologies such as video streaming for capacity development. It could, for example, make all of its digital videos, such as Potted Shrimps, available through streaming without significant cost. Some FAO staff feel that the Organization should do more to take advantage of new technologies in order to disseminate and/or promote its normative products and foster learning. Others would like to see FAO using a blend of old and new; for example, using radio to reach farmers and fishers and combining that with text messaging to enhance collaborative learning. However, caution is warranted as some experiments with new technologies, such as mobile phones in Africa, have produced poor results, particularly where literacy rates are low, local languages dominate and power supplies are intermittent (Ferris & Robbins, 2004).

Access means more that simply physical access. Access is also about users being able to understand, internalize and apply the information they receive in a way that makes sense within their cultural and organizational contexts. The report turns to this issue next as it assesses how FAO has developed and designed its normative products.

Design and production quality Most of FAO’s normative products are exceptionally well produced. All e-learning products reviewed were designed well, most with self-assessments, individualized learning pathways, tutorials, checklists and links to related learning resources. FAO staff report that e-learning course design and development is rigorous, often taking between 18 and 36 months per course. Some CD-ROM products could be better designed to enhance learning. Irrigation Guidelines, which forms part of the Land and Water Digital Media Series, for example, is mainly a compendium of technical papers and does little to exploit the medium’s interactive learning potential.

Most print materials feature good design and some serve as gold standards. For example, the new Land Degradation in Drylands, which will be ready in 2010, is likely to become a global standard. The Gender in Agriculture Source Book, which was produced in 2009 by FAO, The World Bank and the International Fund for Agricultural Development (IFAD), is likely the most comprehensive guide available anywhere on the subject. Fire 11

Management, Voluntary Guidelines, Principles and Strategic Actions is among the best reviewed; it was well designed and was prepared with a great deal of thought and consultation. Negotiation and Mediation Techniques for Natural Resource Management serves as a model for how to structure and design a manual for capacity development. The guidelines for stock breeding, now under development, provide an exemplary model of product development and design. Developed through consultation workshops with experts and regional consultations with government and NGOs, they feature the use of story-telling to get at key issues. A Guide to Bycatch Reduction in Tropical Shrimp-Trawl Fisheries makes excellent use of visuals, including colour photos and diagrams. Modernizing Irrigation Management—the MASSCOTE Approach uses useful case studies and excellent charts and photographs to help readers understand complex technical matters. Good Hygiene Practices in the Preparation and Sale of Street Food in Africa has to be one of FAO’s best-illustrated training resources. Many of the print materials for use in developing capacity have a number of useful design features, such chapter summaries, checklists, visual clues to aid information retrieval, indexes, effective use of duo-colour wash, and appropriate illustrations.

Many of the materials examined for the evaluation were designed with input from user- groups, which is always good practice. Some, such as Good Hygiene Practices in the Preparation and Sale of Street Food in Africa, benefited from rigorous field testing, a practice that FAO should encourage for all instructional materials.

Most materials are targeted at specific users, but some do not identify a target group and the target audience may not always be obvious. Products such as the Fire Management, Voluntary Guidelines, Principles and Strategic Actions that were developed in collaboration with a variety of external organizations were usually well designed. The evaluator notes, however, that relatively few of the materials assessed were developed with FAO interdepartmental collaboration. Although interdepartmental collaboration is not always necessary, the evaluator was surprised to see how few materials had received input from FAO’s development support communication experts. Fewer in number now than in the 1990s, these experts can help field test materials and assist designers in coming up with effective communication and behavioural change strategies to enhance the products’ developmental results.

Instructional design could be improved in a few of the materials examined, such as the Training Course for Instructors in Community-based Fire Management and Aqua- business Training Curriculum for East Africa, where there appears to be too much emphasis on lecturing and insufficient opportunity for interactive and experiential learning.

In a few cases among the products reviewed, FAO may be setting the bar too high in setting normative standards for developing countries. For example, Preparation of Foot- and-Mouth Disease Contingency Plans could leave readers from poor developing countries exasperated because they are likely to lack the resources to implement many of the guide’s recommended procedures.

FAO staff claim that they are challenged in reaching senior managers and decision makers with capacity development materials because these target groups are extremely busy. Materials targeted at senior officials, such as the Policy Learning Program, may need to be adapted to provide short, focused learning modules.

For some e-learning materials, FAO has produced PowerPoint presentations and instructional manuals for trainers to guide them in delivering face-to-face training. The I- Mark series is one example. This is good practice but needs follow up and the resources to make it happen.

Most of FAO’s key normative products have been produced in multiple languages. The Country STAT series, for example, was in English, French and Portuguese, the I-Mark materials in English, French and Spanish and the Fire Management series in five languages. FAO must continue to encourage this practice. However, more resources are needed to translate some of these materials into local languages in order to reach a wider spectrum of Africans at the grassroots level.

Gender Insufficient attention to gender is a serious weakness in many of FAO’s normative products aimed at capacity development in Africa. Of the more than 30 products reviewed, only eight contained references to the gender dimensions of the subject matter, and three of those were products that focused almost entirely on gender, such as the Gender in Agriculture Source Book. In some cases—for example, the products dealing with nutrition and people living with HIV/AIDS, the Guide for the Prevention and Control of Avian Flu in Small-scale Poultry, the Aqua-business Training Curriculum for East Africa, and Conducting Agricultural Censuses and Surveys—the absence of any discussion of gender is an egregious oversight. Modernizing Irrigation Management—the MASSCOTE Approach contains only one reference to gender and women are not mentioned at all. Influencing Policy Processes: Lessons from Experience has three references to gender but no in-depth analysis of the gender dimensions of policy influence.

FAO provides its instructional designers of e-learning materials with guidelines on gender equality, but not its instructional writers. Gender analysis is not carried out consistently at the design stage and there does not appear to be a rigorous check on the content and images for gender sensitivity. The fourth course in the I-Mark series deals with gender issues, such as equal access and inclusiveness. However, the images in the series could be more gender-sensitive; a majority of images shows males in dominant decision-making positions. Some materials—even the authoritative manual, Conducting Agricultural Censuses and Surveys—are weak on stressing the importance of collecting sex-disaggregated data.

A search for ‘gender’ using the EASYPol search engine turned up only two references. FAO staff in the gender division say that project proposals come to them for vetting but not normative materials intended for capacity development. There is, therefore, no 13

systematic assessment of the type one would expect of an organization committed to gender mainstreaming.

Use One FAO staff member, who had been working at the Organization for many years, summed up the use of normative products this way: “FAO is an amazing repository of knowledge, but we don’t do enough with it.” It is difficult to determine the extent to which this view is true or false because FAO collects little data on the use of its normative products and does little monitoring and evaluation of the same. The materials used in face-to-face training programs are not normally assessed to determine their contribution to capacity development, or for their continued use after the training. Some evaluations of e-learning have taken place, such as an evaluation of I-Mark and the recent internal assessment of Codex E-learning Course (Cwiek, 2009), but there are none that focus specifically on outcomes in Africa. The evaluator’s requests for a breakdown of countries from which individuals downloaded normative products produced little reliable data. Although it is feasible for FAO to obtain such information—for example, Easypol has a tool for monitoring downloads—the Organization does not do so systematically. Feedback for this evaluation from the field in Africa confirms that the distribution and uptake of FAO’s products are weak.

Several staff told the evaluator that they had few resources for monitoring. Understandably, there is not much left for monitoring when a donor invests $50,000 or so in a product for capacity development. Even when the investment is larger, monitoring and evaluation often get short shrift. One staff member explained: “Most donors have short timeframes that don’t allow us to do the important follow-up work.”

There are, however, inexpensive technologies available to FAO to facilitate feedback, some of which FAO already employs. For example the Modernizing Irrigation Systems- Technical Briefs, which is available on FAO’s website, allows users to rate the usefulness of each file. Regrettably, FAO rarely asks for, let alone collects, analyzes and reports on, such data. For example, very few of the normative products assessed for this evaluation invited users to provide feedback and made it easy for them to do so. Where electronic feedback to FAO is not feasible, the Organization could do more to encourage its field partners to gather data on the products in use.

Several staff interviewed said that FAO produces many guidelines, but “they are not taken up by governments.” Although there is little empirical data available to support this view, informal feedback from the field in Africa suggests that in some cases relevant agencies and people in governments do not always have physical and/or cognitive access to the materials. In some cases where they do, they may not have the resources to implement FAO’s guidelines. As well, they may not use the resources for a host of other reasons related to the enabling environment, such as limited incentives and follow up on the part of FAO and its partners. As noted earlier, FAO has done little to respond to this important dimension of capacity development.

Certification Many FAO staff, particularly those responsible in e-learning, told the evaluator that lack of certification was an issue for many potential learners from developing countries. Few, if any, FAO courses provide diplomas or credits towards degrees at recognized institutions. According to some staff, learners and their sponsoring organizations often asked about certification when applying to participate in FAO courses. Certification was often participants’ most frequent recommendation in response to questionnaires at the end of courses. This issue requires in-depth study in order to determine the extent to which the lack of certification is a major deterrent for potential learners in African countries. In the interim, FAO could do more to partner with academic institutions in the delivery of its courses, where feasible, as a step toward learner accreditation. The assessment returns to the issue of accessibility of capacity development materials with reference to FAO’s Capacity Building Portal.

Capacity Building Portal FAO’s Capacity Building Portal is “designed to help FAO's partners, member states and other international, national and local development actors to access knowledge, information, training tools, good practices, and services in Capacity Building in FAO's mandate areas” (http://www.fao.org/capacitybuilding/index.jsp?lang=en). As the term “portal” suggests, it is a one-stop gateway to FAO’s capacity development resources. It provides access to: FAO’s published learning materials; a series of good practices and case studies; FAO’s learning services; and links to its e-learning series, such as Codex Alimentarius, Food Security Information for Action, I-Mark and EASYPol, FAO’s online resources for policy making.

The portal, which is still under development, is good in theory, but problematic in practice. Its first major challenge is that many of FAO capacity development resources are on separate websites, requiring users to click several times in order to reach the information they are seeking. As a general rule of thumb in website design, three clicks is the maximum for accessing information; any more and some information seekers are likely to grow impatient and abandon their search. By example, after clicking the EASYPol link on the portal, one is led to the EASYPol website where one must first choose from among language options before getting to the EASYPol homepage. From there, it may take several more clicks to locate the information one is seeking.

There are other problems with the portal; among them the search engine. The Capacity Building Portal search engine was unable to find most of the products reviewed for this evaluation, including Modernizing Irrigation Management—the MASSCOTE Approach, Influencing Policy Processes and Gender in Agriculture Source Book. The evaluator had much better success finding FAO’s capacity development materials using the search engine on FAO’s home page. Yet, even this search engine turned up few normative materials related to some of FAO’s most successful capacity initiatives, such as Farmers’ Field Schools.

The portal could benefit from featuring links to some of the best international research and networks on capacity development, such as the research of the European Centre for 15

Development Policy Management on capacity development (www.ecdpm.org) and the network Capacity.org (www.capacity.org/). The portal could be enhanced through partnerships with African regional and sub-regional capacity development organizations and networks that would allow FAO to link into their information sharing platforms. Such partnerships have the potential to raise the visibility of FAO and provide broader access to its normative products.

Lessons

The following lessons derive from interviews with FAO staff and the literature reviewed for this assessment. Readers should note that not all lessons apply in every case because capacity development is contextual and each situation is different. That said, the following lessons may serve as a guide for FAO.

 Producing capacity development products as part of a comprehensive program of capacity development can help bring about more sustainable outcomes than producing them within discrete, one-off projects where there is little opportunity for follow up. Several FAO staff interviewed for this assessment pointed to this lesson, cautioning against one-off initiatives that have no follow up on the ground. Research on capacity development, such as that referred to in the introduction section of this report, also points to this lesson. Baser and Morgan (2009), for example, argue that capacity development requires changing complex systems often at the individual, organizational and institutional level and that single interventions, such training, are, on their own, usually insufficient.

 Long-term, multi-year capacity development programs can provide opportunities to test, adapt and refine normative products over time. Several staff interviewed for this assessment spoke of the long-term nature of capacity development and how products for capacity development need to be adjusted to specific development contexts and updated from time to time. The research commissioned by the European Centre for Development Policy Management, referred to in the introduction to this report (Baser and Morgan, 2008; and Land, Hauck and Baser, 2009), underscores the need for a long-term approach to capacity development.

 When collaborating with partners on the development of normative products, it is important to give the partners space to come up with new ideas, rather than simply handing them the drafts for comment. This lesson came from several staff in one of FAO’s divisions and was based on many years of experience on their part.

 Results can be stronger and more sustainable when user groups, including national governments and not just the experts, participate in the development and testing of the normative products. Government ownership is critically important. Individuals in two divisions of FAO pointed to the need for local involvement for follow up and sustainable results—a lesson that is echoed by the OECD/DAC— particularly where local organizations, such as governments and civil society 16

organizations, have the mandate and responsibility to provide services and/or to regulate.

 Sending countries documents outlining international norms and standards is unlikely to result in positive change. One of FAO’s longest serving staff members emphasized this point. He told the evaluator that FAO needs to undertake projects that illustrate the value of adhering to international norms and standards in order to generate the political will for change. In short, let field work inform the normative work and not vice versa.

 Positive developmental change can be enhanced when local or regional organizations and institutions are given the opportunity to adapt normative materials for ongoing use in the field. Several FAO staff members noted the need to adapt and contextualize normative products. Some products, such as Preparation of Foot-and-Mouth Disease Contingency Plans, stated this in their opening chapters. Involving regional organizations and institutions in adapting normative materials for local use makes sense. However, sometimes these organizations and institutions lack the capacity and resources for this task. This would suggest an important role for FAO as part of its capacity development strategy.

 Materials in themselves may not necessarily lead to development results. It is how the materials are used and combined with other interventions that make a difference with respect to capacity. This lesson comes from the practice of development communication, in which FAO is a recognized leader. Siliva Balit, a former staff member of FAO, emphasized this lesson in her paper, “Communication for Isolated and Marginalized Groups,” at the Ninth United Nations Roundtable on Communication for Development in Rome in September 2004.

Conclusions and Recommendations

This assessment has examined a handful of FAO’s normative products intended for capacity development, particularly in Africa. The most significant conclusion from the assessment is that there are major gaps in the distribution and uptake of the products in Africa. Although further investigation is needed to determine the reasons, this assessment concludes that FAO relies too heavily on Internet distribution when access and bandwidth are problematic in much of Africa. The Organization needs to employ other means of distribution and invest more in face-to-face capacity development and follow up on the ground with a variety of partners.

Although the number of products examined was small, the evaluator gathered sufficient evidence to conclude that the quality of FAO’s products varies considerably. Some set gold standards; others leave room for improvement, most often in relation to incorporating gender concerns. The volume of materials produced over the past seven or eight years likely outstrips that of all other UN organizations. While this is commendable, 17

this assessment suggests the need for better distribution and greater consistency in quality, and it puts forward a number of suggestions and lessons related to design and process to achieve that consistency. The assessment also emphasizes that there are no short cuts or technological fixes to capacity development. It points to the merits of long- term, programmatic approaches where FAO’s normative products are combined with other resources and interventions towards the challenging job of developing individual, organizational and institutional capacity.

With these conclusions and cautionary notes, the evaluator makes four recommendations:

Recommendation #1: It is recommended that FAO’s senior management take immediate steps to improve the distribution and uptake of FAO’s capacity development products for Africans. The assessment has found serious shortcomings with regard to the access and use of FAO’s normative products in Africa. Over the short term, the Organization needs to make greater efforts to ensure that its knowledge resources reach intended African audiences and that they are designed and utilized in ways that enhance the acquisition of knowledge and skills of various target groups. Over the long term, further assessment is needed to guide policy and planning in relation to the challenges of information access and use in Africa.

Recommendation #2: It is recommended that FAO’s senior management strengthen the monitoring and evaluation of its normative products and capacity development processes. The Organization needs to more closely monitor the distribution of its products so that it has evidence that they reach their intended audiences and are used effectively. It needs to know more about what kinds of products and capacity development interventions are most cost-effective and under what circumstances. Effective monitoring and evaluation are key to learning, adaptation and sound, evidence-based decision making.

Recommendation #3: It is recommended that FAO senior management develop measures to encourage consistent quality and efficacy in the development and use of normative products for capacity development. The measures need not be elaborate nor should they be burdensome. Simple check lists could be useful. Some of the indicators used to assess normative products for this evaluation (see Table 1) could be helpful in preparing checklists. FAO could do more to encourage greater interdepartmental collaboration, more consistent integration of gender equality and the participation of intended beneficiaries when producing its normative products.

Recommendation #4: It is recommended that FAO and its partners pay closer attention to developing materials aimed at strengthening the capacities of organizations and institutions, particularly in relation to the formal and informal systems and aspects of the enabling environment that affect performance. The assessment has shown that most of FAO’s normative products are directed at enhancing the technical knowledge and capabilities of individuals. But individuals 18 function in complex systems within organizations and institutions, and their performance is affected by a wide range of enabling factors. Although FAO may not always be the best equipped to deal with organizational and institutional systems, it must seek out partners who are, and together better understand how these systems affect capacity and its development. As a first step, FAO would do well to participate in some of the networks, such as Capacity.org (www. capacity.org/) and Govnet (www.oecd.org/), that provide fora for learning and exchange in relation to capacity development. 19

References

Ballit, S. (2004). “Communication for Isolated and Marginalized Groups”. Paper prepared for the Ninth United Nations Roundtable on Communication for Development, Rome, 6-9 September 2004.

Baser, H. and Morgan, P. (2008). Capacity, Change and Performance. Maastricht: European Centre for Development Policy Management.

Cwiek, M. (2009). “Codex E-learning Course, Report for a period of 8 months from May to December 2008.” Unpublished internal assessment, AGNS, January 13, 2009. Rome: Food and Agriculture Organization of the United Nations.

Ferris, S. and Robbins, P. (2004). Development Market Information Services in Eastern Africa. The FOODNET experience. Ibadan, Nigeria: International Institute of Tropical Agriculture.

Land, T., Hauck, V., and Baser, H. (2009). “Capacity development: between planned interventions and emergent processes. Implications for development cooperation.” Policy Management Brief No. 22, March 2009. Maastricht: European Centre for Development Policy Management.

Organization of Economic Cooperation and Development/Development Assistance Committee (2006). The Challenge of Capacity Development. Working Toward Good Practice. Paris: OECD/DAC.

Tooth, T. (2000). The Use of Multi-media in Distance Education. Vancouver: Commonwealth of Learning.

List of Staff Interviewed at FAO Headquarters

H. Steinfeld, AGAL A. Besbes, AGAP P. Kenmore, AGPP R. Charrondière, AGNA C. Bessy, AGNS M. Kenny, AGNS

R. Lamb, ASW

F. Dalla Valle, ESWD C. Djeddah, ESWD P. Colbert, ESWD N. Keita, ESS K. Caprazli, ESS

T. Farmer, FIEP D. Gréboval, FIEP, R. Metzner, FIEP N. Hishamunda, FIEP J. Toppe, FIEP P. Manning, FIE R. Lee, FIIU

J. Heino, FOD M. Achouri, FOMC

R. Frattini, KCEF C. Petracchi, KCEF A. Nadeau, KCEF

S. Bunning, NRLA R. del Castello, NRR

M Maetz, TCAS

Appendix B: Selected Products

Type Format Department

ool

Product T

aper

- line

Manual/Guide Learning Resource Framework Issue Analytical Other Print CD - ROM On Other AGP ES FR FO KC NR TC

Market-oriented Farm Management for Trainers of Extension     Workers, Africa, Training Materials for Agricultural Management, Marketing and Finance, 2007 Biocontrol-based Integrated Pest Management for the      Diamondback Moth (Plutella xylostella), Eastern and Southern Africa, 2007 A System of Integrated Agricultural Censuses & Surveys:      Vol. 1, World Programme for the Census of Agriculture 2010, 2006 Conducting Agricultural Censuses and Surveys, FAO, 1995     Training Course for Instructors in Community-based Fire      Management, FAO and Global Fire Monitoring Centre Wildlife Fire Training Centre, South Africa, 2004 Fire Management, Voluntary Guidelines, Principles and      Strategic Actions, Fire Management Working Paper FM 17E, 2006 Effective Record Keeping in Ghanaian Fish Farming      Associations, 2004 Aqua-business Training Curriculum for East Africa, n.d.    Farm Ponds for Water, Fish and Livelihoods, FAO     Diversification Booklet 13, 2009 Code of Safety for Fishermen and Fishing Vessels: Part A,     Safety and Health Practice, Safety and Health Practice, 2005 Potted Shrimps (DVD), 2006    A Guide to Bycatch Reduction in Topical Shrimp-Trawl     Fisheries, Revised edition, 2007 Modernizing Irrigation Management—the MASSCOTE     Approach, 2007 22

Negotiations and Mediation Techniques in Natural Resource     Management: Case studies and lessons learned, 2007 Preparation of Foot-and-Mouth Disease Contingency Plans,     2002 Guide for the Prevention and Control of Avian Flu in Small-    Scale Poultry, 2006 Guidelines to Designated National Authorities on the    Operation of the Rotterdam Convention, revised 2006 Influencing Policy Processes: Lessons from Experience,     2008 EASYPol (FAO website for developing policy-related    capacity) Socio-economic and Gender Analysis Programme, Field      Level Handbook, 2001 Gender in Agriculture Sourcebook, FAO, The World Bank       and IFAD, 2009 Participatory Communication Strategy Design: A Handbook ,     2004, 2nd Edition Participatory Rural Communication Appraisal: A Handbook,     2004, 2nd Edition Good Hygiene Practices in the Preparation and Sale of    Street Food in Africa: Tools for Training, 2007 Irrigation Guidelines on CD-ROM. Land and Water Digital    Media Series, 2000 Nutrition Care and Support for PLHIV. Country Experiences       of Capacity Building from Lesotho, Malawi, Mozambique, Swaziland, Zambia and Zimbabwe, 2008 Living Well with HIV/AIDS: A manual on nutritional care and       support for people living with HIV/AIDS, 2002 Getting Started! Running a Junior Farmer Field and Life      School: Empowering orphans and vulnerable children living in a world with HIV and AIDS, 2007 Food Security Information for Action: e-learning     Enhancing Participation in Codex Activities, Training      Package including CD, 2005 Information Management Resource Kit: Series of 4, 2006    

Annex 9

Evaluation of FAO’s Capacity Development Activities in Africa:

A meta-evaluation of country, programme and project evaluations

Cristina Lopriore

1 Acknowledgements

I would like to thank the members of the Core Evaluation Team who helped get the discussions started, particularly Rachel Bedouin for her coordination and support as well as Kay Leresche and Anna Guerraggio for their constructive comments. I also extend my gratitude to Nadine Monnichon for her dedicated commitment to seeing this consultancy through.

2 Table of Contents

EXECUTIVE SUMMARY ...... 5 1. INTRODUCTION ...... 13 2. META-EVALUATION OBJECTIVES, PURPOSE AND FOCUS ...... 13 3. METHODOLOGY ...... 14 3.1 SELECTION OF EVALUATIONS ...... 14 3.2 ANALYSIS AND TOOLS OF THE META-EVALUATION ...... 14 3.3 LIMITATIONS ...... 17 4. OVERVIEW OF FAO’S CAPACITY DEVELOPMENT ACTIVITIES ...... 18 4.1 OVERVIEW OF EVALUATIONS...... 18 4.2 CHARACTERISTICS OF CD SUPPORT IN FAO’S ACTIVITIES...... 20 5. QUANTITATIVE ASSESSMENT OF FAO’S PERFORMANCE IN CD SUPPORT ...... 23 5.1 RELEVANCE OF FAO’S CD ACTIVITIES ...... 23 5.2 DESIGN AND MODALITIES OF FAO’S CD ACTIVITIES ...... 23 5.3 FAO’S CAPACITY TO DELIVER CD SERVICES EFFICIENTLY ...... 26 5.4 EFFECTIVENESS OF FAO’S CD ACTIVITIES ...... 26 5.5 IMPACT OF FAO’S CD ACTIVITIES ...... 27 5.6 SUSTAINABILITY OF CD ACHIEVEMENTS ...... 27 5.7 PARTNERSHIPS IN FAO’S CD ACTIVITIES ...... 28 6. QUALITATIVE ANALYSIS OF FAO’S PERFORMANCE IN CD SUPPORT ...... 29 6.1 UNDERSTANDING OF CD CONCEPT AND PRACTICE IN EVALUATIONS AND PROJECTS ...... 29 6.2 RELEVANCE, DESIGN AND MODALITIES OF CD SUPPORT IN FAO’S ACTIVITIES ...... 30 6.3 IMPLEMENTATION, MANAGEMENT AND EFFICIENCY ...... 38 6.4 EFFECTIVENESS AND IMPACT ...... 40 6.5 SUSTAINABILITY OF PROJECT RESULTS ...... 43 6.6 PARTNERSHIPS...... 44 7. CONCLUSIONS AND RECOMMENDATIONS ...... 47 8. REFERENCES ...... 58

ANNEX 1: LIST OF EVALUATION REPORTS ...... 59

TABLE 1: DESCRIPTORS OF CD SUPPORT ...... 15 TABLE 2: SUMMARY OF EVALUATION CRITERIA USED TO ASSESS PERFORMANCE IN CD SUPPORT ...... 16

FIGURE 1: EVALUATIONS BY AFRICAN REGION ...... 18 FIGURE 2: EVALUATION TYPES ...... 18 FIGURE 3: EVALUATIONS BY THEME ...... 20 FIGURE 4 : CHARACTERISTICS OF CD SUPPORT IN FAO'S ACTIVITIES ...... 22 FIGURE 5: SOFTWARE SUPPORT AND SECTOR TARGETED IN FAO’S CD ACTIVITIES ...... 22 FIGURE 6: FAO’S PERFORMANCE SCORING IN CD SUPPORT AS EVIDENCED IN EVALUATION REPORTS ... ERROR! BOOKMARK NOT DEFINED.

BOX 1: FACTORS DETERMINING CD PERFORMANCE OF PROJECTS —EXAMPLE FROM FISHERIES ...... 31 BOX 2: REGULATORY FRAMEWORKS ARE AN EFFECTIVE SUPPORT TO CD —EXAMPLE FROM THE CODE OF CONDUCT FOR RESPONSIBLE FISHERIES ...... 32 BOX 3: FARMERS ORGANISATIONS IN BURKINA ASSESSED THEIR CD NEEDS ...... 35 BOX 4: TWO LESSONS FROM A NUTRITION PROJECT IN ZAMBIA ON PARTICIPATORY APPROACHES IN CD ...... 35

Acronyms

AGA Animal Production and Health Division (FAO) AGN Nutrition and Consumer Protection Division (FAO) ALRMP Arid Lands Resource Management Project (GoK) CCRF Code of Conduct for Responsible Fisheries CLCPRO Commission de lutte contre le Criquet pèlerin dans la Région occidentale CEP Champs Ecoles Paysans (Farmer Field Schools) CTA Chief Technical Advisor CD Capacity Development CNLA Centre national de lutte antiacridienne DRC Democratic Republic of Congo EE Enabling Environment EMPRES EMPRES Emergency Prevention System for Transboundary Animal and Plant Pests and Diseases / Système de prévention et de réponse rapide contre les ravageurs et maladies transfrontières des animaux et des plantes (FAO) EMPRES/RO Programme EMPRES en Région occidentale ESS FAO’s Statistics Service (ESS) FAO Food and Agriculture Organisation of the United Nations FAOR FAO Representative (or Representation) FFS Farmer Field Schools FNPP FAO Netherlands Partnership Programme FS Food security FSIA EC/FAO Food Security Information for Action Programme FSIS Food security information system(s) GIEWS Global Information Early Warning System (FAO) GOSS Government of Southern Sudan HQ Headquarters IEE Independent External Evaluation of FAO JFFLS Junior Farmers’ Field and Life Skills Schools (FAO) LSP Livelihoods Support Programme MDG Millennium Development Goals M&E Monitoring and Evaluation NGOs Non-governmental Organizations NMTPF National Medium Term Priority Framework (FAO) NPFS National Programme(s) of Food Security NRM Natural Resource Management OCD Office for Coordination & Decentralisation (FAO) OP/OPA Organisation Paysanne/Organisation Professionnelle Agricole(Farmers Organizations) PAN National Food Security Programme (Mozambican acronym for SPFS) PRSP Poverty Reduction Strategy Paper SIFSIA Sudan Institutional Capacity Programme: Food Security Information for Action SFLP Sustainable Fisheries Livelihoods Programme SPFS Special Programme(s) of Food Security SFLP Sustainable Livelihoods programme TA Technical Assistance TCE Emergency Operations & Rehabilitation Division (FAO) TCP Technical Cooperation Programme UN United Nations WFP World Food Program

4 Executive Summary

INTRODUCTION

Capacity development is at the core of FAO’s mandate. Yet, no thorough and comprehensive assessment of FAO’s work in this area has been accomplished. This meta-analysis of 30 country, programme and project evaluations assesses the characteristics and performance of FAO’s support to capacity development and contributes to provide the evidence-base and analytical under-pinning for the Evaluation of FAO’s Capacity Development Activities in Africa. This report contains the main findings and recommendations from the meta-analysis.

OBJECTIVES, PURPOSE AND FOCUS OF THE META-ANALYSIS

The objective of this meta-analysis is to review, aggregate and synthesize evaluation study findings on capacity development from a series of country, project and programme evaluations carried out by FAO over the past decade. The purpose of this review is to fulfill the need to widen the scope of the Evaluation benefits and use of results in institutional learning. The primary users of the meta-evaluation are the Core Evaluation Team members. The paper draws selectively on a sample of available evaluation reports. It should not be seen as an exhaustive review. The meta-analysis focuses on the extent to which (a) CD concept and practice have been assessed in past evaluations of FAO’s CD activities; (b) FAO’s activities for CD support have addressed CD across different dimensions, sectors, functional capacities, software modalities and (c) FAO’s performance in developing African capacity to address the needs of the poor and the environment and how this can be done better in the future.

FINDINGS OF THE QUANTITATIVE ANALYSIS (Chapters 4 and 5)

Characteristics of CD activities in projects: Out of the 30 evaluations, only 3 provided a comprehensive analysis of CD support whereas in half of the evaluations this analysis was weak or sketchy. As most of FAO’s work relates to CD, this finding calls for urgent revision of current evaluation methodologies to adopt a more consistent approach for assessing FAO’s CD support. FAO’s support to CD was characterized as follows: − CD support was prominent in nearly all FAO interventions, with CD either formulated as a specific objective of projects or largely reflected in project outcomes and activities. − Modality of CD support was predominantly software. − CD dimension was mostly targeting individuals and groups (80%), enabling environment (50%) and organizations (25%). − Functional capacities were mainly knowledge capacity (90%), followed by implementation and outreach/partnering capacity and lastly policy/normative development capacity. − The sector typically targeted was the Government (99%) vs. civil society, consisting mostly of farmers groups (70%), while very few included the private sector (30%). − Software modality of CD support was predominantly a package consisting of technical assistance, training, knowledge sharing/materials, institutional support (99% of

5 evaluations provided evidence of these activities) followed by partnerships and support to policy/legal frameworks (approx. 80%). − FAO’s role was mainly represented by analytical/normative functions and related knowledge management (80%).

Performance scoring of CD activities in projects:

Relevance: The majority of FAO’s activities is found to be both relevant to country CD needs and coherent with FAO’s comparative advantage, however in only half the cases were CD activities well coordinated with other CD initiatives.

Design of CD initiatives was generally poor. All design-related issues except one (responsiveness to stakeholders demand) scored unsatisfactory or poor in 50-70% of evaluations, and particularly in: design of appropriate monitoring and evaluation frameworks, translation of objectives into clear desired outcomes, cost-effectiveness and sustainability considerations and extent to which CD initiatives took into account needs at individual, organizational and policy levels.

FAO’s efficiency in delivery of CD services scored generally low. Although the capacity to deliver through its technical units was deemed mixed or satisfactory in over half the evaluations, evaluative evidence on the level and quality of coordination mechanisms between headquarters and field highlighted a strong need for improvement. Interdepartmental coordination across technical units in headquarters was rarely assessed.

Effectiveness of FAO’s performance in supporting CD showed mixed results: - Good-satisfactory scores in three issues: quality/relevance of types/formats of CD materials, facilitating technical/policy initiatives, provision of technical assistance and training; - Unsatisfactory or poor scores for provision of effective follow-up and support to ownership of CD interventions; - Mixed scores in two issues: awareness/use/utility of normative products and in partnership for developing and enhancing the capacities of individuals and organizations to deliver CD.

Impact of CD activities was consistently reported as being very difficult to assess. Evaluations assessed results often only at the outcome level because of lack of adequate impact indicators. Moreover, the scoring of evaluative evidence was not easy because results varied by CD dimensions (individuals/groups, organization, enabling environment) and, even within a single dimension, evidence was mixed. A preliminary cross-tabulation of performance scoring results suggests the following association between evaluation criteria: the greatest impact was achieved in those cases where the design of CD interventions was well pondered during the planning phase, where implementation was undertaken in ways that enhanced sustainability of results achieved and where successful partnerships were forged.

Evaluative evidence on the sustainability of CD interventions was not always thoroughly analyzed; overall performance on this evaluation criteria was mixed: - Good-satisfactory in terms of creation of networks and social capital through CD activities.

6 - Unsatisfactory or poor in at least two issues: the promotion of leadership by regional or national actors and in the integration of support provided into policy and institutional changes, programme planning and implementation. - Mix of both satisfactory and unsatisfactory in at least two issues: the delivery of assistance in ways that enhance capacities of client groups/organizations, and the adaptation and further dissemination of CD approaches and information/training materials. - No or poor evidence on the up-scaling of CD activities at regional and national levels to further disseminate the knowledge and skills acquired.

Scoring results on partnerships are mixed, although the paucity of in-depth analysis of this issue may not reflect the reality. Evaluations provided indeed little to no evidence of the efficiency and effectiveness of FAO’s cooperation with different development actors. Yet, recommendations by evaluators recurrently stressed the need to increase collaboration and work more in partnerships.

FINDINGS OF THE QUALITATIVE ANALYSIS (Chapters 4 and 5)

Underlying factors of successes and shortcomings in FAO’s CD performance:

• Understanding of CD concept and practice: evaluative evidence clearly shows there is generally a narrow, incomplete or unequal understanding of CD as a multidimensional concept, often minimizing it to training of individuals and sometimes to physical and financial resources.

Factors of relevance and design: • Importance of design ─ Despite the high relevance of CD activities, achievements varied considerably and evaluations attributed this variation to three crucial factors: (a) design of CD interventions, (b) partnership building with national institutions and (c) follow up of CD activities. The importance of CD design was well illustrated in two examples from the Fisheries evaluation (box 1: Promotion of small pelagic & Food safety and quality) • Importance of institutional support ─ The early set up of national institutional bodies (with related legal framework and norms) ensured better use and sustainability of programme support and guaranteed coherence and continuity of functions/institutional memory (less turnover of trained staff). A successful case is illustrated by the regional desert locust EMPRES programme: “La seule formation des cadres ne suffit généralement pas: elle doit s’accompagner d’un renforcement institutionnel et d’un travail sur les politiques de développement ”. • Training methods to enhance skills of individuals ─ Various types of software support were used in projects. Best results were achieved through practical and participative approaches which worked better than the classic support consisting of formal training with technical assistance. Practical and participative approaches allow immediate application of new learning and skills and provide direct and practical job aids. • Identification of needs at the individual, organization and institutional levels ─ The poor identification of needs at the three levels of CD was one of the weakest design factors that emerged from the meta-evaluation. Capacity needs assessments were generally not undertaken prior to CD interventions. This resulted in poor identification of CD needs at different levels, inadequate tailoring and reduced effectiveness of implementation and sustainability of CD activities.

7 • Responding to country demand/needs ─ Not always were CD interventions adequately responding to country demand and needs. In fact, some cases were found were interventions offered a “standard package of CD activities”, and evaluators questioned the appropriateness of the activities to the problems to be addressed. This was found especially in the remarkably uniformity of some TCP interventions and also in one food security information systems project where the promotion of headquarter-led analytical tools was perceived as supply driven by country partners. Negative consequences included: diminishing ownership of CD activities and real challenges during field implementation. As some evaluators concluded: “although the basic thrust of these initiatives was valid, greater care should be given to identify and tailor the intervention to the specific characteristics of each country, with particular attention to the choice of the implementing institution and of the target consumer groups”. • Addressing inter-disciplinary concerns and needs ─ The traditional type of CD support, for example in the livestock sector, has mainly consisted of technology transfer at the individual level. Evaluations increasingly emphasized that technical assistance projects needed to be more development-based interventions by paying greater attention to addressing underlying development needs as interdisciplinary concerns require changes at institutional and policy levels. • Sustainability: seeking synergies to ensure follow up ─ In this and other thematic evaluations undertaken in recent years, project design and follow-up have consistently scored lower among the various project components. One way to maximize sustainability of CD results is to effectively tie into other country CD initiatives during design stages. • Design of Logical frameworks, Monitoring & Evaluation ─ In the meta-evaluation, these frameworks stood out as the weakest design issues, both in terms of lack of clarity/coherence in the logical flow between objectives, results and activities, but also in the inadequate choice of indicators to monitor and report on. One of the criticisms of evaluators was that these frameworks often lacked specificity (e.g. to accommodate a flexible process approach) and focused more on measuring progress in activities while omitting to measure achievements at the outcome or impact levels. Another shortcoming was the insufficient guidance in the operationalizing programme in terms of describing activities, linkages, and outputs. • Gender considerations ─ Gender considerations did not score satisfactorily in many of the evaluations. Evidence suggests that CD interventions viewed gender as a stand alone set of activities and not as a cross-cutting concern.

Implementation, management and efficiency: Evidence from country evaluations especially indicated that FAO Representations faced many challenges, particularly in terms of the volume of operations to manage with often overstretched human resources, the financial and administrative tools available to FAOR being inadequate or insufficient, while coordination with headquarters witnessed fragmented linkages. • Limitations of TCP format ─ several evaluations questioned the suitability of TCP format for addressing complex or ambitious CD needs, for instance some TCP were regarded as having ‘too ambitious goals and objectives with respect to short duration’ • Insufficient country presence ─ programmes characterized by a limited country presence and relying on management decisions at headquarter level were unable to respond in a timely fashion to implementation challenges and opportunities.

Effectiveness and impact: • Difficulty of assessing impact of CD support ─ There is a general lack of comprehensive impact assessments of CD interventions. The real impact of CD interventions will show with time. For this reason, achievements are mostly reported in terms of outputs • Achieving impact requires long term efforts ─ Supporting CD requires a long term process and a comprehensive approach across the different levels (individual, organization, institution/policy). The concluding remarks of the Mozambique evaluation applies in general across all CD interventions: “Capacity building is a feature of nearly all FAO interventions but has often been a weak area in terms of impact. Some of the long-term interventions […] have been relatively successful because they have been able to take a broad approach to capacity building: they have provided training to staff over a long period and they have in addition to this been able to address an important dimension of capacity building by establishing appropriate organisational structures.” • Examples of successful results in FAO’s CD activities were found at various levels: organizational, institutional and policy levels. In general evaluations tended to find more positive CD results in those CD activities that had intervened at the organizational and policy/institutional levels. • FAO Normative products awareness/use/utility ─ Evidence from evaluations indicates there is inconsistent awareness of FAO’s global services products. • FAO’s role as convener, mediator and “honest broker” is widely appreciated ─ FAO was shown across evaluations to play a significant role at national level, leading and/or supporting inter-agency initiatives (e.g. clusters in short-term humanitarian response) and sectoral (longer term developmental approach) coordination relating to agriculture and food security. However, it was felt that this distinctive role could be put to use more effectively for closing the gap between the two tracks of emergency and development.

Sustainability: Scoring low across most evaluations, the sustainability and follow up of results achieved remain a serious challenge in most CD support interventions. • Follow up to training interventions ─ Most evaluations considered that implementation of training lacked proper follow-up and mentoring support to ensure that new-found skills and learning effectively translated into application and better job performance. • Unclear exit strategy for sustainability and follow up ─ Evaluations found that the design of many CD projects lacked some vision of “after-project”. Sustainability depended on the extent to which intervention envisaged an exit strategy ensuring a continuity and also on the ability of projects to formulate tools or mechanisms that allowed client groups to pursue and further engage in other partnerships or seek other funds. • Commitment by recipient Governments ─ Even when assistance provided was valuable and comprehensive, the ultimate change and sustainability of project contributions will depend on the recipient Government commitment to ensure sufficient budget for follow up and maintenance of activities.

Partnerships:

9 The nature of cooperation with a variety of development actors in CD interventions was overall insufficiently analyzed by evaluations. Nonetheless, evaluators unanimously stressed the need for FAO to increase efforts in establishing and working through partnerships with a range of actors especially given FAO’s generally weaker role as implementing agency compared to its other roles (convener/facilitator, analytical/normative). • Traditional versus new/innovative partnerships ─ Evaluations reviewed show that FAO interacts with and builds partnerships with a wide range of partners. Traditional partners are often government ministries, especially Ministries of Agriculture. In general, more attention should be paid in CD interventions to seeking partnerships also with the private sector. Evaluators agreed more efforts were needed to explore innovative partnerships, such as those in the private sector, that can bring valuable and sustainable solutions to problems. • Administrative challenges ─ Among one of the frequently cited constraint for FAO’s work in partnership was related to the limitations in the efficiency of administrative procedures.

MAIN CONCLUSIONS & RECOMMENDATIONS FOR IMPROVING CD PERFORMANCE:

Understanding of CD and support to CD by development practitioners: FAO programmes are primarily about supporting CD needs in developing countries, yet CD has served as a catch-all term for very different issues related to capacities. Those responsible for CD intervention programmes, and their evaluation, must be specific in portraying CD problems and proposed solutions.

Design and modalities of CD support in FAO’s activities: • Provide CD support as an integrated package — CD interventions need to address the CD challenges as a system, targeting needs across different dimensions and functional capacities. In key areas of FAO’s comparative advantage there must be more resources at field level to better respond to needs and demand for technical assistance. • Assess capacity needs before designing CD activities — In the face of continuing high demand for CD and assistance, several evaluations have stressed the importance of basing CD support activities on a careful and realistic assessment of FAO’s comparative advantage and not take on “the lead” where resources are limited. • Providing organizational support, and not just transfer of technical skills — Meta- evaluation findings indicated that FAO intervenes comparatively less to enhance organizational capacities. Evaluative evidence was very clear as to the need to encompass all the skills needed to perform different functions and not just focus on technical skills. As shown above, a careful analysis of capacity needs/gaps and constraints across different dimensions is necessary to ensure CD support is properly integrated. • Continue facilitating policy change, but increase country support for policy implementation —The perceived comparative advantage of FAO in acting as “a neutral, honest broker” to facilitate policy change was repeatedly underlined in evaluations; however, nearly all studies recommended to increase support to countries for implementing these policies/strategies. Efficiency:

10 • Prioritizing CD support is essential given FAO’s limited funds — In the face of continuing high demand for CD assistance and FAO’s overall weak capacity to deliver capacities, several evaluations emphasized the need to prioritize CD support based on a careful and realistic assessment of FAO’s comparative advantage, a clear identification of the “right mix of services”, and by avoiding taking on “the lead” where resources are limited. Evaluative evidence suggested that FAO focus more on acting as facilitator, working more through partnerships rather than direct provider of technical assistance. • Improving internal coordination — Evaluations did not consistently provide recommendations for addressing the overall weak coordination both at interdepartmental level as well as between field programmes. The issue was however clearly recognized as requiring a “cultural” change. Practical solutions put forward by evaluators ranged from developing a conceptual framework to the provision of incentives and raising awareness. • Adapt TCP modalities to development problems —Evaluations recognized the limitations of the TCP funding modality for addressing long term CD needs. The issue was flagged with regard to the need for technology transfer projects to be designed with their development context in mind if they were to have the desired impact. Recommendations included: paying greater attention to design and preparation in TCPs, carrying out more formal cost-benefit analysis and specific design of follow-up activities for these projects, strengthening TCP flexibility, simplifying categorization of TCP projects, and tying TCP priorities closely to the FAO Strategic Framework and country priority frameworks. Effectiveness and Impact: • Greater attention to developing Monitoring & Evaluation frameworks — Available literature on CD concludes that measuring the outcomes, let alone impact of CD interventions is quite challenging because of the complexity of CD and the difficulty of predicting outcomes at the outset. Nonetheless, most evaluations were often only able to assess achievements up to output level of the results chain and recommended improving monitoring and evaluation systems by identifying indicators that enable the programme to measure programme outcomes. • Coherence and synergy building between FAO initiatives for greater effectiveness — Evidence particularly from Africa suggests that the package of FAO field assistance has the highest profile and is most effective when unified behind a major thrust, e.g. around Farmers’ Field Schools (FFS) in Kenya, or around the Special Programme for Food Security (SPFS) in Burkina Faso. Good integration of emergency and non-emergency activities in support of improved crop production was evident in D.R. Congo and in Ethiopia. Evaluations recommended to increase synergies across FAO CD activities to maximize effectiveness, this was especially the case of small project funds that alone cannot address CD requirements (e.g. TeleFood funds) Sustainability and partnerships: • Planning of follow up and develop hand-over strategies from the start— Inadequate follow up was often cited as one of the weakest criterion of CD projects. This was an area especially of concern with TCPs which contribute to larger CD initiatives. Evaluations strongly recommended to design follow-up activities to ensure the post-project sustainability of project achievements. In this regard, evaluations underscored the necessity for developing linkages much more carefully at the design stage so as to build synergies and institutional collaboration with other ongoing initiatives at country level. • Reaching out to untraditional partners— FAO should expand its cooperation efforts beyond its traditional partners, to include other sectoral ministries and also to the private

11 sector. The choice of the institutional counterpart is especially important in food security projects and can influence positively or negatively on the course of programmes. For examp,le, in the Sudan SIFSIA-South programme has found its institutional counterpart in the Southern Sudan Commission for the Census, Statistics and Evaluation – an appropriate location for a project dealing with a cross-cutting topic such as food security. • Work increasingly through partnerships and alliances especially to increase training performance and dissemination— Considering the significant demand from countries, evaluations consistently recommended to work more through partnership with local service providers, by building alliances and making better use of available capacities to amplify and implement the messages and programmes. This recommendation came up very clearly in the Fisheries evaluation that put forward a number of concrete suggestions (e.g. identifying institutions with internationally-recognized training capabilities as ‘centres of excellence’; fostering twinning processes with training networks of training institutions, including South- South cooperation) • Establish and support partnership and cooperation with regional bodies— This need clearly emerged in the realm of international instruments where the development of the linkage between global instruments and national level implementation is a key role for regional action. As highlighted in the evaluation of international instruments, regional bodies have the ability to address any imbalance in focus of these instruments/frameworks (e.g. issues of importance of richer versus poorer countries), and to provide a linkage between international instruments and national implementation. Also, regional bodies may be specially qualified and situated to provide support services, interpretation and implementation tools and other and input of value to international instruments.

12 1. Introduction The international community has in recent years increasingly focused its attention on country ownership for development effectiveness, and on capacity development as a key factor to enhance the sustainability of results. In 2005, the FAO Director General Reform proposal “A vision for the 21st Century” called upon new and stronger attention to capacity development for individuals and institutions.

Capacity development is at the core of FAO’s mandate. Yet, no thorough and comprehensive assessment of FAO’s work in this area has been accomplished. The FAO Evaluation Service has initiated in 2009 an Evaluation of FAO’s Capacity Development Activities in Africa to serve as a vehicle for accountability and learning by providing a rigorous evidence-based analysis of the current status of FAO’s work in the area of capacity development. The Evaluation covers FAO projects, activities, and normative products aimed at developing the capacities of individuals, organizations, and institutions in Africa. The Evaluation is currently carrying out country case studies and undertaking field missions, selected on the basis of geographical, institutional background and thematic balance.

Five desk reviews were included as part of the Evaluation methodology. This meta-analysis, one of the five, is one contribution towards providing the evidence-base and analytical under- pinning for the comprehensive assessment of FAO’s work in capacity development. As most of FAO’s work relates to capacity development, this meta-analysis examines evidence from past evaluations on capacity development in specific sectors or areas of work of the Organization. It also broadens and deepens the data and information collected by the Evaluation through additional assessment of projects and programmes which are not covered by the country case studies or the project reviews.

2. Meta-evaluation objectives, purpose and focus The objective of this meta-analysis is to review, aggregate and synthesize evaluation study findings on capacity development from a series of country, project and programme evaluations carried out by FAO over the past decade. The intent of this exercise is to discover if there are any trends, patterns, or additional information that emerge from previous evaluations around the cross-cutting theme of capacity development (CD). The purpose of this review is to fulfil the need to widen the scope of the Evaluation benefits and use of results in institutional learning. In other words, to offer system-wide opportunity to learn from past experience and to extract good practices with a view to improving performance in CD support. The primary users of the meta-evaluation are the Core Evaluation Team members. The paper draws selectively on a sample of available evaluation reports. It should not be seen as an exhaustive review.

The meta-analysis focuses on the following questions: 1. Extent to which past evaluations of FAO’s activities assessed CD concept and practice; 2. Extent to which FAO’s CD activities have addressed, targeted and/or used different: • CD dimensions and modalities, • software modalities for CD support (including knowledge acquisition and use, policies and institutions affecting the enabling environment), • sectors and functional capacities; 3. Extent of FAO’s performance in developing African capacity to address the needs of the poor and the environment and how this can be done better in the future.

13 On the basis of information provided by evaluations, this report covers: • an overview of FAO’s practices in supporting CD of countries across different technical sectors; • a first identification of strengths (or drivers) and weaknesses in CD support of FAO’s activities; • a synthesis of key conclusions and recommendations for improving these CD practices.

3. Methodology

3.1 Selection of evaluations This meta-evaluation is based on a textual analysis of evidence presented in a purposive sample of evaluation reports publicly available in the Evaluation Service’s database of project evaluations. Evaluations were selected based on the following criteria: - Those that focused on activities for which there is a clear intent to develop capacities, - Completed since 2003, - Focusing on African countries; - Covering a range of programme, project, country and corporate evaluations, - Covering a representative range of specific sectors or areas of work of FAO; - Including additional assessment of projects and programmes not covered by the country case studies or the project reviews. Using these criteria, a sample of 30 evaluations was included for review, 24 of which were country, programmes and projects evaluations, and another 6 are corporate evaluations. The full list of evaluations is shown in Annex 1. An overview of the type, period, geographic and thematic coverage of evaluations is provided in chapter 4.

3.2 Analysis and tools of the meta-evaluation The methodology was developed based on the Terms of Reference and Inception Report of the Evaluation. A grid was specially designed as the main tool of the meta-evaluation to fulfil the dual need to get an overview of FAO’s support to CD and assess how well this support has performed. In other words, the grid had two functions: (a) record the various components of CD support used in FAO’s activities; and (b) determine the quality of FAO’s CD activities based on the questions and indicators of the Evaluation matrix, the core reference document of the Evaluation (Annex 2).

Each evaluation report was reviewed in detail and relevant evidence entered into the common grid in Excel which served to systematize information on the issues of interest to the meta- analysis (Annex 3). In order to quantify findings, each issue was scored using a scoring system explained hereafter. The transversal reading of the grid formed the basis for the analysis.

The following quantitative and qualitative analyses were carried out: a) Description of FAO’s approaches to CD support The first step determined the extent to which the evaluation report provided a comprehensive analysis of CD as a multidimensional concept and estimated the relative importance of CD in the project. Next, FAO’s CD activities were assessed against six descriptors of CD support (table 1). Each item was scored on a 3-point scale (high, medium, low) except for two items (sector and software support) for which a ‘Yes/No’ score was given. Findings are described in Chapter 4.

14 Table 1: Descriptors of CD support Type of CD CD Functional Sector software FAO’s role Modalities Dimensions capacities support • Software • Individuals & • Government • Policy & • Technical • Analytical • Hardware groups • Civil society normative assistance or normative • Financial • Organization • Private development • Training functions & • Enabling sector • Knowledge • Twinning & related environment • Outreach & partnerships knowledge partnering • Distance/e- management • Implementation learning & advocacy • Knowledge & • Convener Information for multi- sharing stakeholder materials engagement • Pay/Incentives & broker for support partnerships • Policy & Legal • Operational frameworks role (techn. • Institutional assistance, support training, provision of experts, etc)

15 b) Assessment of FAO’s performance in CD support Evaluative evidence regarding the quality of FAO’s CD support was seized and assessed against seven evaluation criteria, covering a total of 31 items. Table 2 lists these items by their short titles (refer to Annex 2 for a full description). Each item was scored on a scale from A=good to D=poor. Scoring was not always clear-cut and, in some cases, no scores were assigned either because information was missing or unclear. Chapter 5 presents the findings of the quantitative analysis. Table 2: Summary of evaluation criteria used to assess performance in CD support Criteria Issues assessed • country capacity needs identified • project builds on FAO and integrated in project comparative advantage Relevance strategies • coordination with other CD initiatives • needs at indiv./org./EE levels • major goals reflected (food considered security, poverty, sustainable • responsiveness to local NRM) demand/needs • cost-effectiveness & sustainability Design • participation in CD design considered • promotion of ownership • gender • objectives clearly defined in • Monitoring and evaluation outcomes frameworks • in-house capacity for CD • coordination (timeliness, degree) Efficiency delivery via FAO technical units in project management between • inter-depart. coordination at HQ HQ & decentralized offices • CD materials quality/relevance • provision of CD support • FAO normative products encouraging ownership awareness/use/utility • FAO partnering on CD activities Effectiveness • FAO facilitating technical/policy • FAO improving CD technologies initiatives • technical assistance and training delivery

Impact • impact of CD activities at individuals/groups, organization and enabling environment levels • enhance client capacities • changes in policies and • enhanced local leadership programme implementation Sustainability • up scaling of CD activities by • adaptation and dissemination of trained people CD approaches • social capital created • effectiveness of cooperation • effectiveness of cooperation with Partnerships within UN all actors

Based on this analysis, we identified some of the key strengths and weaknesses in CD support and recommendations that were highlighted by evaluators as being critical to enhance FAO’s performance in CD support. Chapters 6 and 7 present the main findings of this qualitative

16 analysis with the support of relevant quotes extracted from the various reports with the intent to illustrate, show specific examples, case studies and good practices.

3.3 Limitations Constraints in terms of time and resources available for this meta-analysis dictated limiting the absolute number of evaluations reviewed, subject to the need to represent the breadth of FAO’s areas of work and also include a reasonably wide range of evaluation types. In-depth screening of evaluative evidence using the grid proved more complex than originally planned for. The difficulty lied in the fact that evaluations did not necessarily follow a methodology consistent with the one of this Evaluation and similarly did not necessarily address issues of CD. The grid for assessing FAO’s performance in CD support was originally designed for project evaluations, but it did not apply well to the review of corporate evaluations.

Finally, limitations inherent to the evaluation material available also affected the level of interpretation needed for the qualitative assessment. A quality assessment of this type will always be subject to the individual interpretations of the reviewers. To counter this, the meta- analysis relied as much as possible on the indicators of the Evaluation matrix, as a means to quantify our scoring, as well as cross-checking. However, these indicators were not always thoroughly reported. For future meta-evaluations on CD, it is recommended to adapt these indicators further based on the learning process of this Evaluation, in accordance with FAO’s Corporate Strategy for CD, as well as the range of CD modalities being evaluated.

17 4. Overview of FAO’s Capacity Development Activities

This section gives an overview of the evaluation coverage and key characteristics and modalities for CD support in FAO’s activities as evidenced by past evaluations. It provides a basic understanding of the extent to which FAO’s activities have approached CD support through various entry points.

4.1 Overview of evaluations

The general characteristics of the evaluations sample are summarized in the figures below. The reports ranged in length and complexity, from 15 pages to 121 pages. The length of assignments typically varied around 2 to 3 weeks in the field – which should have allowed sufficient opportunity for gathering data and getting a reasonable understanding of the CD context.

Evaluations were distributed across the entire African continent, covering all subregions Figure 1: Evaluations by African region (Figure 1). Over 40% of 23% evaluations covered multiple countries or subregions. Ten 41% West reports were at the global level, East & Central and where available, evaluation Southern results from visits to African Multiple regions 23% countries were examined. 13%

Sixty percent of the evaluations were evaluations of field projects, a third were programme and/or corporate strategy evaluations, Figure 2: Evaluation types and only three were country level evaluations (Figure 2). The report 30% Project evaluation types ranged from ‘typical’ mid- term or final evaluations, to joint Country evaluation reviews of sectoral support. Out Programme & strategy of the 18 project evaluations evaluation 10% studied, 14 were final evaluations 60% and only 4 were mid-term reviews.

Despite this small variation1, it was nonetheless possible to compare evaluations with each other. The reporting format used by the majority of evaluations followed the results-based analytical framework i.e. assessing the causality chain of results (inputs/activities, outputs, outcomes and impact). Project achievements were commonly assessed against the core evaluation criteria of relevance, efficiency, effectiveness, sustainability and impact. Some

1 Generally the timing of the evaluation will alter its content and focus. Mid-term reviews are carried out during the on-going implementation and usually have a focus on studying specific problems, and making suggestions to improve the implementation for the remainder of the intervention. Final evaluations have a focus on gathering information for decision-making regarding future options (eg. replication in another phase, extension or closure), and to evaluate the relevance, impact and sustainability of the intervention.

18 reports also structured findings along the different phases of the project cycle (design, implementation, management, impact).

19 The thematic areas studied covered a wide spectrum of sectors, with all sectors in rural development clearly more represented (Figure 3). The second largest thematic area was statistics, databases and food-security related information systems and networks. Over half of the evaluated projects and programmes were in development context, a third mixed development and emergency, while only two were in emergency/rehabilitation. The time period evaluated varied from mid 1990’s up to 2009, although half the evaluations examined the recent 2003-2009 period, a third in 2000-2005, and the rest across the range mid-90’s to 2004.

Figure 3: Evaluations by theme All sectors in rural development

Statistics, databases & FS 3% 3% 7% 40% information systems 3% Crops 7% Fisheries

Land & water

7% Livestock 7% Nutrition & food security 23% Gender & people's participation

Research, extension & communication

4.2 Characteristics of CD support in FAO’s activities Out of the 30 evaluations, only 3 provided a comprehensive analysis of CD support whereas in half of the evaluations this analysis was weak or sketchy. As most of FAO’s work relates to CD, this finding calls for urgent revision of current evaluation methodologies to adopt a more consistent approach for assessing FAO’s CD support.

The key characteristics and modalities of FAO’s support to CD are summarized in Figures 4 and 5: − CD support was a prominent feature of nearly all FAO interventions. The CD content of evaluated projects/programmes was found to be high in 23 evaluations, with CD either formulated as a specific objective of projects or largely reflected in project outcomes and activities. − The modality of CD support: was predominantly software (28 evaluations scoring high) followed by financial (11 scoring high and 10 medium) and hardware (14 scoring medium). − The CD Dimension targeted for support was by and large individuals and groups (23 evaluations scoring high), then enabling environment and organisations (19 and 11 medium scores, respectively). − The functional capacities receiving CD support were predominantly knowledge capacity (28 evaluation scoring high), followed by implementation and outreach/partnering capacity (respectively, 17 and 16 high scores) and policy/normative development capacity (11 high scores).

20 − The sector targeted in CD interventions was typically the Government (29 evaluations) vs. civil society, consisting mostly of farmers groups (21), while few included the private sector (11). − Software type of CD support: the predominant software package consisted of technical assistance, training, knowledge sharing/materials, institutional support (29-30 evaluations provided evidence of these activities), closely followed by twinning/partnerships networking (26) then support to policy/legal frameworks (22). Little or no specific evidence of distance learning or other supporting incentives. − FAO’s role featured mainly as analytical/normative functions and related knowledge management (24 scored high) closely followed by its other two roles, namely operational functions (21 high scores) and convener and broker for partnerships (17 high scores).

Figure 4 : Characteristics of CD support in FAO's activities Analysis of CD support in evaluations CD content of project & programmes

Software

Hardware support

Modality of CD Financial

Individuals & groups

Organizations High Medium

CD dimensionCD Enabling environment Low Policy & normative n/a development

Knowledge

Outreach & partnering supported

Functional capacity Implementation

Analytical & normative, KM & advocacy Convener & broker for partnerships FAO's role Operational role

0% 20% 40% 60% 80% 100%

Figure 5: Software support and sector targeted in FAO’s CD activities

Government

Civil Society

Target Sector Private Sector

Technical assistance

Training Yes Distance/ E-Learning No not available Knowledge & Information sharing materials

Twinning & partnerships

Pay/incentives support Type of Software support

Policy & legal frameworks

Institutional support

0% 20% 40% 60% 80% 100% 22 5. Quantitative Assessment of FAO’s Performance in CD Support

This chapter presents the quantitative results of the assessment of FAO’s performance in CD support across seven evaluation criteria: relevance, design, efficiency, effectiveness, impact, partnerships, sustainability (see full description of individual items under each criteria in Annex 2). The performance scoring results are summarized in figure 3 and are commented in the sections hereafter.

5.1 Relevance of FAO’s CD Activities Key finding: The majority of FAO’s activities is found to be both relevant to country CD needs and coherent with FAO’s comparative advantage, however in only half the cases were CD activities well coordinated with other CD initiatives.

- Nearly 70% of the evaluations considered that interventions had well or adequately identified country capacity needs and reflected them in programme and project interventions and strategies, although evaluation reports hardly ever mentioned that capacity needs assessments had been carried (at individual, organization and enabling environment levels) prior to interventions. - The vast majority of interventions at field level were well or adequately built on FAO's comparative advantages, although none of the evaluations referred to an explicit analysis of this comparative advantage in planning documents. - One the other hand, the coordination with other international and national CD initiatives scored Good and/or Satisfactory in less than half of the cases.

5.2 Design and modalities of FAO’s CD activities Key finding: The design of CD initiatives was generally poor. All design-related issues except one (responsiveness to stakeholders demand) scored unsatisfactory or poor in 50-70% of evaluations, and particularly in: design of appropriate monitoring and evaluation frameworks, translation of objectives into clear desired outcomes, cost-effectiveness and sustainability considerations and extent to which CD initiatives took into account needs at individual, organizational and policy levels.

- Evidence from two thirds of the evaluations suggests that CD initiatives had insufficiently or poorly taken into account the needs at individual, organizational and policy levels at design stage. Only 4 evaluations found there was a good analysis of these needs and relevant implications pondered in project documents. - The extent to which CD initiatives were responsive to demand and relevant to stakeholders' needs scored “good and/or satisfactory” in half of evaluations, with at least a third being better than satisfactory. - Nearly 60% of the evaluations reported evidence was “middle, unsatisfactory or poor” in terms of participation levels in the design of CD interventions (generally in the form of inception meetings with key stakeholders in the planning phase) as well as evidence of programme design and materials being tailored to different target groups and working with them in development and delivery. Participation levels were found to be better than satisfactory in 8 evaluations. - Over 60% of the evaluations found that the design of CD activities was “middle, unsatisfactory or poor” in terms of ownership promotion through local collaborative

23 partnerships. The promotion of ownership in design of CD activities was better than satisfactory in 8 evaluations. - Objectives of CD initiatives were overall poorly defined in terms of desired outcomes with 14 evaluations scoring “middle or unsatisfactory” and 6 scoring “poor”. Only 5 evaluations deemed this issue to be better than satisfactory - The emphasis placed by CD activities on the three major goals (food security, poverty reduction and sustainable natural resource management) showed a mixed picture, with half the evaluations deeming this issue to be unsatisfactory-poor and half scoring middle, satisfactory or good. - Cost-effectiveness and sustainability issues were often inadequately or poorly pondered in the design of CD initiatives (17 evaluations scored this issue as mixed or less than satisfactory) - The monitoring and evaluation framework design of CD initiatives was found to be the weakest item of CD design, with as much as 13 evaluations scoring poor.

Figure 6: FAO’s performance scoring in CD support as evidenced in evaluation reports

24 Capacity needs identified

Building on FAO's comparative advantage

Relevance Coordination with other CD initiatives

Needs at indiv/org/EE levels considered

Responsive to national demand & needs

Participation in CD design

Promoting ownership

Objectives clearly defined in desired outcomes

Major goals reflected (FS, poverty, NRM)

Cost-effectiveness & sustainability considered Design CD of intervention Gender & other crosscutting issues

Monitoring & Evaluation frameworks

Capacity for CD delivery via FAO techn. units Good Interdepartmental coordination at HQ Good-Satisf.

Efficiency Coordination of project management HQ vs. Field Satisfactory Mixed Relevance/quality CD materials Unsatisfactory Normative products: awareness, use & utility Unsatisf-Poor Facilitating technical/policy initiatives Poor Not available Technical assistance & training

Supporting CD (ownership) Effectiveness Partnering on CD activities

Promoting better CD technologies

Impact of CD activities at ind/org/EE levels Impact Enhanced client capacities

Enhanced local leadership

Up-scaling of CD activities by trained people

Social capital created

Sustainability Changes in policies & program implementation

Adaptation & dissemination of CD approaches

Cooperation with UN agencies

Cooperation with all actors Partnerships 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

25 5.3 FAO’s capacity to deliver CD services efficiently Key finding: FAO’s efficiency in delivery of CD services scored generally low. Although the capacity to deliver through its technical units was deemed mixed or satisfactory in over half the evaluations, evaluative evidence on the level and quality of coordination mechanisms between headquarters and field highlighted a strong need for improvement. Interdepartmental coordination across technical units in headquarters was rarely assessed.

- The level of in-house capacity and allocation of appropriate resources to deliver CD through FAO’s technical units showed mixed findings: a third of evaluations scored this issue as “satisfactory or good”, a third scored “mixed” and the rest scored “less than satisfactory or poor”. - Inter-departmental coordination was not often referred to in evaluation reports (13 made no mention of it) and among the few that did, evidence indicated that working across departments was a major challenge and sometimes obstacle to efficient work. - Timeliness and degree of coordination in the management of projects and activities between headquarters and decentralized offices was generally low: 5 evaluations scored this issues as satisfactory or good, 6 scored mixed and as much as 14 scored “less than satisfactory or poor”.

5.4 Effectiveness of FAO’s CD Activities Key finding: The findings on FAO’s work performance and effectiveness are mixed: - Good-satisfactory scores in three issues: quality/relevance of types/formats of CD materials, facilitating technical/policy initiatives, provision of technical assistance and training; - Unsatisfactory or poor scores for provision of effective follow-up and support to ownership of CD interventions; - Mixed scores in two issues: awareness/use/utility of normative products and in partnership for developing and enhancing the capacities of individuals and organizations to deliver CD.

- The quantity, quality and appropriateness of types or format of CD materials with regard to the target group needs was considered to be good/adequate in most evaluations (20 scored satisfactory or good, 3 less than satisfactory). - The findings on the awareness and use of FAO's normative products and knowledge sharing tools were mixed: 9 evaluations scored this issue as satisfactory or good versus 7 scoring mixed, and 8 scoring less than satisfactory or poor. - FAO acting as facilitator of regional and national technical and policy initiatives was deemed generally satisfactory in most evaluations (20 evaluations scored this issue as satisfactory or good) - Provision of technical assistance and training was also deemed generally satisfactory in most evaluations (20 scored this issue as satisfactory or good) although often lacked follow-up to ensure use of capacities developed as indicated in the next issue. - Provision of CD support i.e. use made of capacities developed to ensure ownership and follow-up to CD interventions, was generally found unsatisfactory (21 scored “middle”, 4 scored “high”)

26 - The effectiveness of FAO in partnering to enhance local capacities to deliver CD showed mixed though generally positive findings: 15 scored good-satisfactory, 8 mixed of both satisfactory and unsatisfactory, and 4 unsatisfactory or poor. - Development and promotion of new and better technologies for CD was not consistently reported.

5.5 Impact of FAO’s CD Activities Key finding: The impact of CD activities was consistently reported as being very difficult to assess. Evaluations assessed results often only at the outcome level because of lack of adequate impact indicators. Moreover, the scoring of evaluative evidence was not easy because results varied by CD dimensions (individuals/groups, organization, enabling environment) and, even within a single dimension, evidence was mixed. Evidence of impact was found to better than satisfactory (5 evaluations), 9 satisfactory, 8 mixed and 4 less than satisfactory or poor. A preliminary comparison between findings of impact with those of other evaluation criteria suggests the following association: the greatest impact was achieved in those cases where the design of CD interventions was well pondered during the planning phase and where programme was implemented in ways that enhanced sustainability of results achieved and where successful partnerships were forged.

5.6 Sustainability of CD achievements Key finding: Evaluations provided little analysis of the sustainability of CD interventions, but the little evidence available on this issue showed overall poor performance: - Good-satisfactory only in the creation of networks and social capital through CD activities. - Unsatisfactory or poor in at least two issues: the promotion of leadership by regional or national actors and in the integration of support provided into policy and institutional changes, programme planning and implementation. - Mix of both satisfactory and unsatisfactory in at least two issues: the delivery of assistance in ways that enhance capacities of client groups/organizations, and the adaptation and further dissemination of CD approaches and information/training materials. - No or poor evidence on the up-scaling of CD activities at regional and national levels to further disseminate the knowledge and skills acquired.

- Both the delivery of CD assistance in ways that enhance capacities and the promotion of national leadership showed on average similar mixed scores: good-satisfactory (7 and 6, respectively), satisfactory or mixed (11 and 12, respectively) and less than satisfactory or poor (7 and 4). - The involvement of trained actors in disseminating and scaling out their CD activities at other levels was scarcely reported on (12 evaluations made no mention of this issue) and where available this was deemed satisfactory or good in only 5 cases. - The creation of social capital and networking through the CD activities was largely appreciated by most evaluations (good or better than satisfactory scores in 13 evaluations) - There was weak evidence of the integration of support provided into policy and institutional changes, planning, and implementation: good or better than satisfactory scores in 4 evaluations, whereas 9 scored satisfactory and 9 scored less than satisfactory or poor..

27 - The adaptation of CD approaches/info materials in other contexts scored good or better than satisfactory in 6 evaluations, 8 scored satisfactory and 9 scored less than satisfactory or poor.

5.7 Partnerships in FAO’s CD Activities

Key finding: Scoring results on partnerships are mixed, although the paucity of in-depth analysis of this issue may not reflect the reality. Evaluations provided indeed little to no evidence of the efficiency and effectiveness of FAO’s cooperation with different development actors. Yet, recommendations by evaluators recurrently stressed the need to increase collaboration and work more in partnerships.

28 6. Qualitative Analysis of FAO’s Performance in CD Support

This chapter presents the qualitative findings of the meta-analysis and highlights some of the strengths and weaknesses of FAO’s performance in CD support. It is not intended to be exhaustive but points out to significant evidence. Relevant quotes extracted from evaluation reports are shown in italics with a view to illustrate findings and provide examples of good practices.

6.1 Understanding of CD concept and practice in evaluations and projects

One striking aspect of this meta-evaluation, and thus indirectly of CD support in projects, is the generally narrow and unequal understanding of CD as a multidimensional concept. Several evaluations assessed CD results merely focusing on training of individuals, yet most of them acknowledged CD support involved other dimensions as well. The general emphasis on skill enhancement of individuals consequently excluded from the analysis the wider organizational context and enabling environment in which individuals and groups operate. This uneven understanding of CD and its implications for CD support among the different project stakeholders was highlighted as one of the causes of poor sustainability of projects achievements. Some relevant quotes include: “Il est apparu au cours de la mission que nombre des acteurs des projets, partenaires directs ou indirects, permanents ou occasionnels, ne saisissait pas l’envergure que recouvre le renforcement des capacités, le minimisant à la formation et parfois aux moyens matériels et financiers. Aussi, on rappellera que ce concept porte l’accent sur l’aptitude des individus, de leur organisations et de leur groupe social à, notamment, définir un but précis (vision), analyser leur milieu et s’y positionner (place, enjeux, devoirs, droits, responsabilités), formuler des stratégies, établir des plans d’action et les mettre en œuvre, acquérir et mobiliser des ressources, établir des rapports d’entraide et de partenariat, évaluer les résultats et faire les réajustements nécessaires. Le centre européen de gestion des politiques de développement définit la capacité comme étant « une combinaison de propriétés, de ressources, d’aptitudes et de relations qui permettent à un système humain de fonctionner, de survivre et de s’auto-renouveler ». Dans cette esprit, on parlera d’un renforcement de capacités porteur de résultats si les individus et/ou les organisations sont « à même d’agir, de s’organiser et d’influer sur leur environnement, s’ils établissent des rapports avec d’autres acteurs et qu’ils revêtent aux yeux de ces acteurs une certaine légitimité, s’ils ont l’aptitude à maîtriser le changement et à adopter des idées nouvelles, etc. ». (Evaluation tripartite «Approche participative et gestion de terroirs», p.37) “In view of the low levels of expertise and qualifications available within Government of Southern Sudan (GOSS) institutions, SIFSIA-S has correctly prioritised capacity-building activities. In this, it has achieved some results and reached large numbers of people, but a number of training efforts were performed in a void as the trainees did not return to a work place where they could put their new skills to use. It has taken some time to recognize the importance of capacity building (in terms of training and hardware) at state level, but the necessary corrections have now been put in place. (Evaluation SIFSIA- South Programme, §4.3.4)

This lack of clarity over what constitutes CD was also evident from the inconsistent use of CD terminology in documents and reports. The quote below shows how the meaning of CD

29 varied in the formulation of CD objectives. In some instances, CD was even used interchangeably with “training”. « Un premier regard croisé entre l’objectif global de l’axe thématique et les objectifs de développement des projets montre que deux projets (Burkina et Sénégal) évoquent une action orientée vers les capacités « appui au capacités productives » et « renforcement de la responsabilisation des organisations paysannes ». Dans le projet Dimitra, le renforcement des capacités est indirectement évoqué « accroissement du statut des femmes rurales ». […]A ce stade, il est donc possible d’affirmer que le renforcement des capacités est une orientation claire des projets, dès lors où il s’agit de consolidation, de structuration, d’appui aux organisations qu’elles soient à vocation agricole (Burkina, Sénégal, Niger) ou à vocation de mobilisation sociale (Dimitra) ». (Evaluation tripartite «Approche participative et gestion de terroirs», p.26-27)

6.2 Relevance, design and modalities of CD support in FAO’s activities

Relevance of CD support ─ All CD dimensions are important: Despite the narrow focus on a single CD dimension (technical training of individuals), evaluations found that CD activities were relevant to country capacity needs but they also recognized that, in practice, CD support required concerted action at different levels, particularly in terms of organizational and institutional know-how. “The level of expertise available within GOSS institutions has often found to be lacking: [The Sudan Institutional Capacity Programme: Food Security Information for Action] has therefore correctly emphasized capacity-building. The initial focus on GOSS-level staff is still evident but has been corrected with greater attention now being paid to state- level officers. The disconnect described above between training individuals and a work place that does not enable them to apply their skills may become less of an issue in the future as SPCRP has become operational and SIFSIA-S equipment is being delivered. Nevertheless, future training activities should be preceded by a clearer assessment of individual as well as institutional and organizational capacity-building needs, and a strategy for addressing them all.” (Mid-Term evaluation of the SIFSIA-South Programme, §4.3.1)

The relevance of CD activities was particularly strong where CD support targeted all three main dimensions of CD, giving a great importance to both enabling environment and organizational levels. « […] il est possible de conclure à une cohérence interne, plus implicite qu’explicite, de l’axe thématique ainsi qu’à une bonne pertinence des projets du programme, chacun dans son propre contexte. Les modalités d’intervention suivantes semblent particulièrement pertinentes: • l’ancrage dans des politiques de développement rural et agricole et de réduction de la pauvreté, • l’appui aux organisations civiles et rurales, dont la structuration paysanne/agricole, • la capacitation organisationnelle, technique et institutionnelle, avec des éléments pour alimenter une communication interactive entre projets (début de capitalisation, quelques voyages d’études entre projets, production d’informations, échanges d’informations et d’expériences, etc.)

30 • l’instauration de partenariats effectifs et solides, s’accordant aux orientations de développement, dont la décentralisation et la reconnaissance des courants sociaux (mouvement paysan et société civile), • l’association des organisations paysannes à la supervision des projets, • la recherche active d’un impact au niveau des politiques nationales. » (Evaluation axe thématique «Approche participative et gestion de terroirs», page 29)

Poor design of CD interventions: institution and of the target consumer A recurrent finding across evaluations was that despite the high relevance of projects Promotion of small pelagic: “Two of the TCPs had positive to country capacity needs, project results: both met national priorities and the Governments were committed to a positive outcome and follow-up, the achievements varied between countries and implementing national institutions were appropriate and high- were not always positive. Why? What quality expertise was provided through FAO. […] “On the makes some projects successful and others other hand, the other two had less positive results, mainly because of overly complex and poor design, weak not in effectively meeting CD needs? The implementing national institutions and insufficient follow-up.” Fisheries’ evaluation identifies three crucial factors which are also found in Food safety and quality: “TCPs on this topic were successful as they met a real need and contributed to other evaluations: (a) design of CD opening crucial export markets for benefiting countries. Staff interventions, (b) partnership-building with were trained, national capacities built and the responsible national institutions, and (c) follow-up to institutions in many cases are now capable of keeping up with subsequent adjustments and changes in relevant regulations. CD activities. These elements are In some African countries, the TCP helped to create important highlighted in two examples from the linkages between the private sector, the fisheries fisheries sector (see box 1). The evaluators administrations and National Standards Bureaux that did not exist before.” concluded that: “although the basic thrust of these initiatives was valid, greater care Extract from the Fisheries evaluation (paragraph 68) should be given to identify and tailor the groups”. intervention to the specific characteristics Box 1: Factors determining CD of each country, with particular attention performance of projects —Example to the choice of the implementing from Fisheries Software support ─ Addressing the enabling environment: At least half of FAO’s CD activities addressed the enabling environment dimension, and in particular norms, institutions, policy and legal frameworks. This entry point to CD was generally highly appreciated in evaluations. A clear example of how activities designed to affect the enabling environment led to successful CD achievements is shown in the evaluation of FAO Corporate Strategic Objective B-1 on international instruments, in particular with the experience of the Code of Conduct for Responsible Fisheries (CCRF) – see box 2. Evidence of the positive effects of developing this soft law instrument was demonstrated across various levels and functional capacities of CD.

Another advantage of addressing the enabling environment is linked to “The information obtained in the fisheries focus area indicates a strong and growing awareness at national level of the value and usefulness the issue of cost-effectiveness. One of international soft law. On the one hand, the international nature of evaluation put forward the following the soft-law development process has allowed countries cost-effectiveness principles that collectively to achieve results which each would otherwise have struggled with individually (such as practical implementation of the stand out when evaluating technical fisheries elements of UNCLOS or commitments of the FSA). Their assistance projects: experience in developing and using international soft law has • Development cooperation works helped national fisheries officials get accustomed to a single body of terminology and concepts. In essence, no matter what best in conducive policy language or dialect they speak, they all speak “Fisheries.” Finally, soft environment law enables the development of more soft law. For example, the problems of artisanal fisheries are the same all over the world. Through the linkages developed in negotiating and using the Code of Conduct for Responsible Fisheries and other soft international31 instruments, many countries are able to establish partnerships to find means to address this common concern.”.

Extract from: Evaluation of FAO Corporate Strategic Objective B1 (p.24) • Development should lead to knowledge and finance that will help to improvements in institutions and strengthen institutions and policies so that policies; services can be effectively delivered. • Effective development Box 2: Regulatory frameworks are an effective cooperation complements private support to CD —Example from the Code of Conduct investment; and for Responsible Fisheries • the true value of development projects is to provide a package of

“These are the principles against which the project should ultimately be measured. Accepting these parameters, the project has gone in the right direction: its contribution to development is to help improve service delivery by strengthening sectoral and local institutions, and its participatory approach to service delivery should ensure private sector/beneficiary involvement.” (Evaluation of a Belgium-funded nutrition and household food security project in Ethiopia, 2005, page 16)

Software support ─ Focusing on institutional and organizational aspects is a good CD practice: The early set up of national institutional bodies ensured better use and sustainability of programme support and guaranteed coherence and continuity of functions/institutional memory, with less turnover of trained staff. «Le Programme EMPRES dans la Région occidentale a mis l’accent sur la création d’unités nationales de lutte antiacridienne fortes et bénéficiant dans leurs statut, personnel, infrastructures, équipements et budgets d’une autonomie clairement définie. L’évaluation a permis de confirmer empiriquement la justesse de cette approche [institutionnelle], dans la mesure où les appuis du Programme ont été mieux utilisés et maintenus là où existait une unité autonome (Mauritanie, Mali) que là où cette unité manquait encore (Niger, Tchad) au moment où les appuis étaient apportés. Dans ce dernier cas, on assiste à une dispersion dans l’ensemble […] des formations et des moyens matériels mis à disposition par les différents projets[…] L'accent mis sur les aspects institutionnels de la lutte antiacridienne reflète l'état actuel des bonnes pratiques en matière de renforcement des capacités. La seule formation des cadres ne suffit généralement pas: elle doit s’accompagner d’un renforcement institutionnel et d’un travail sur les politiques de développement » (Evaluation Programme EMPRES en Région occidentale, p.19)

Software support ─ Enhancing skills of individuals: Various types of software support are used in projects. To enhance individuals soft skills, practical and participative approaches worked better than the classic support consisting of formal training with technical assistance. Practical and participative approaches allowed immediate application of new learning and skills and provide direct and practical job aids. « Dans trois des quatre projets, la composante renforcement des capacités était d’importance. Chacun d’entre eux a agi avec des approches différentes sachant que tous ont opté pour une démarche participative et un ancrage dans les pratiques et les activités. Les méthodes les plus appréciées, plutôt que des méthodes « classiques » sont certainement celles basées sur les échanges (CEP2), les rencontres (voyages d’études),

2 Champs Ecoles Paysans (Farmer Field Schools): « Là où les producteurs et les productrices se retrouvent pour apprendre ensemble à partir d’expérimentations en conditions réelles. C’est un bon lieu d’acquisition et de validation de connaissances techniques, d’auto-apprentissage et de valorisation des savoirs locaux. Il faut noter que des améliorations pourraient être faites pour une plus grande appropriation, par exemple, par le biais de l’auto-évaluation. »

32 l’acquisition de connaissances fondamentales (alphabétisation), les discussions (méthodes et outils participatifs). (Evaluation axe thématique «Approche participative et gestion de terroirs», p.37)

Identification of needs at the individual, organization and institutional levels: The poor identification of needs at the three levels of CD was one of the weakest design factors that emerged from the meta-evaluation. Evaluators criticized for example the lack of clarity of project documents with regard to the definition of project strategies or the establishment of counterpart conditions (see also logframe design issue below). Programme formulators needed to pay greater attention in the linkages existing across dimensions and sectors of CD so as to seize any recent changes in policy issues, or to identify differentiated CD needs of central versus decentralized services, all of which likely to affect project performance. These shortcomings resulted in missed opportunities for more strategic targeting of CD support, hampered effectiveness of project implementation and poor institutional sustainability or consolidation of CD activities. « Le projet a néanmoins eu certaines faiblesses. La plus importante tient à la conception même du projet qui a créé certaines ambiguïtés en mélangeant une stratégie d’appui au développement local centrée sur les instances locales de planification des infrastructures socio-communautaires et une stratégie de professionnalisation visant à transformer les producteurs en agents économiques pour être davantage compétitifs sur les marchés. Par conséquent, les activités n’ont pas suffisamment été orientées sur la connexion entre les OPA et les marchés et l’équipe du projet n’a pas suivi la recommandation de la FAO d’entreprendre une étude sur les filières. […] Aujourd’hui, dans le contexte d’ouverture des marchés, les OPA structurées au sein de filières spécialisées ont un rôle central à jouer pour faire face à une concurrence davantage exacerbée [...] Le document de conception du projet n’a pas pris en compte cette dimension dans la formulation de ses objectifs ce qui ne permet pas de définir de façon suffisamment explicite des activités centrées sur les fonctions économiques de ces structures. Le document a pensé intégrer les OP/OPA au niveau des instances de planification locales du développement et des futures instances de la décentralisation, alors qu’il s’agit davantage d’un dialogue politique fondé sur des négociations aux niveaux local, national et sous-régional pour soutenir la croissance des productions agricoles. Dans des économies libéralisées et ouvertes, la séparation des compétences entre le domaine du public et celui du privé est un élément fondamental pour clarifier les fonctions des différentes structures et aboutir à la construction d’une synergie public/privé gagnante pour les acteurs des filières agricoles. » (Evaluation du projet « Appui aux organisations paysannes de la province de la Kompienga dans le cadre de la sécurité alimentaire et de la décentralisation », p.5,13) “The link between communities and institutions was to be strengthened through capacity building at all levels and mainstreaming participatory approaches. Institutions included public agencies, NGOs and Community Based Organizations (CBOs). All these were often depicted in the ProDoc as a uniform group of stakeholders, in spite of their very different nature, mandate and capacities.” “[…]Reasons for these shortcomings [no evidence of consolidation and sustainability for other project components] are several, mostly stemming from a misconceived institutional set-up and unrealistic assessment of necessary competences and resources, including finance.” (Luapula Valley Project evaluation, §170) “Capacity building support meets real needs in the present situation of institutional weakness. Despite some positive results in the support to restructuring of the Min. of

33 Agriculture, Fisheries and Livestock (MAPE) and to the strengthening of statistical services, support to capacity building provided by FAO, which concentrated on central services, would have gained efficiency by conforming to the decentralization process with stronger synergy. In that sense, the extreme weakness of provincial services of agriculture, forestry and fishery constitutes an area where FAO will have an important role to play.” […] “The ongoing decentralization process at provincial level sets a challenge and, at the same time, represents an opportunity for FAO. There is a great need to assist governance and provincial public institutions. [...] Therefore, FAO must reconsider its positioning at national, provincial and local levels.” (Evaluation of FAO cooperation in DRC, §xx-xxx) Responsiveness to demand: Where CD needs were not adequately assessed, this had repercussions on the extent to which CD initiatives responded to demand and were relevant to stakeholders’ priorities. This risk was especially high when interventions offered a “standard package of CD activities”. A typical case are FSIS projects: the promotion of headquarter-led analytical tools was in one programme perceived as supply driven by country partners, hereby diminishing ownership of CD activities and resulting in real challenges during field implementation. Another example are Technical Cooperation Programmes. “No proper capacity needs assessment or FSIS constraint analysis was conducted in many of the countries. The work plans were mostly supply-driven by HQ. While this is fully in line with the recommendations from the Phase I final evaluation report that Phase II should limit the number of tools selected and promoted to all countries to optimize the efficient use of resources, this approach did not always respond to the priorities set by local FSIS partners, thereby limiting the programme’s ability to ensure relevance to local or national contexts. The approach met with criticism at the country level, where key stakeholders often complained they did not like the programme’s supply-driven approach.”. […] In many cases, the country counterparts responsible for integrating new tools and methods had limited knowledge of food security data and lacked experience with information technology. In some cases, tools were delivered without enough prerequisite training and technical support.” (EC/FAO FSIA evaluation, § 218) “Most TCP projects were remarkably uniform in their inputs. These invariably included some international and national consultants (the former more under Partnership Programmes in recent years); in-country training and a study tour abroad; and some equipment. Missions felt that there should be greater discrimination in the selection of project inputs, with each one selected on its individual merits and appropriateness to the problem being addressed.” (Evaluation Technical Cooperation Programme, §111)

Responsiveness is also a matter of timeliness in engaging in key national policy-making processes, as shown by the strategic use of TCP funds in Sierra Leone: “The use of TCP in the Agricultural Sector Review was particularly strategic, as without it, FAO would have been unable to play the leading role that it did in its implementation. Using TCP for “buy-in” to key national policy-making processes is extremely effective and the TCP Unit should be prepared to respond to such requests, if technically sound, in a rapid fashion”. (Sierra Leone Evaluation)

Fostering ownership and participatory approaches in formulating and implementing CD projects: Fostering ownership and participation in design of CD interventions are two closely related issues. Several examples of CD interventions demonstrated some successful approaches,

34 including: setting up of local participatory committees for project selection and project support (Burkina Faso, Senegal), signing Letters of Agreement with institutions in project districts (Ethiopia), and carrying out participatory assessments of capacity needs at baseline – see box 3. “A sense of ownership Box 3: Farmers organisations in Burkina assessed their became evident when the project was essentially « Après l’élaboration d’un guide de diagnostic des organisations paysannes, par government-run in the les experts du projet et la formation de l’équipe d’animation à son utilisation, le diagnostic initial participatif a été réalisé par les six animateurs du projet et dix interim phase between the animateurs temporaires recrutés à cet effet, sous la supervision des experts du first and current CTA. Projet. Sur un total de 212 Organisations Paysannes recensées dans la Government staff is province, 185 ont été touchées par cette activité.[…] Ce diagnostic initial a constitué une base essentielle de l’approche participative car il a permis involved in project d’avoir une meilleure connaissance des OP/OPA de la province operations at all levels, (caractéristiques générales, capacités organisationnelles, fonctionnelles et and ownership is financières) en vue de profiler son intervention. Il a permis de montrer que le monde rural de la Kompienga se mobilise rapidement et fortement face à un reinforced by providing nouvel enjeu mais que très peu d’organisations paysannes sont professionnelles Letters of Agreement to avec des unions embryonnaires sans lien fonctionnel avec les structures woredas. […]There has faîtières. A la suite de ce diagnostic initial, un diagnostic approfondi a été entrepris auprès des 72 OP/OPA sélectionnées par le projet pour en faire des been active involvement by partenaires privilégiés. Ces deux diagnostics ont été un aspect important de government staff in, and la responsabilisation des OP/OPA qui ont été confrontées à l’auto support for, the Community questionnement et à la planification de leurs activités. » Extract from: Evaluation «Appui aux organisations paysannes de la province de la Action Plan process.” Kompienga (p.18-19) (Evaluation of BSF- nutrition project Ethiopia, CD needs §4.3)

“Le projet a prévu de mettre en place un dispositif institutionnel temporaire au niveau provincial comprenant deux entités auxquelles le gouvernement a accepté d’intégrer des représentants d’Organisation Paysanne/ Organisation Professionnelle Agricole (OP/OPA): le Groupe d’Appui au Projet et l’Unité de Gestion du Projet. […] La tutelle reste la responsabilité du Ministère de l’Agriculture (maître d’ouvrage principal) mais la grande originalité est d’avoir délégué la maîtrise d’ouvrage au Groupe d’Appui au Projet à compétence régionale (maître d’ouvrage délégué) avec une forte représentation des OP/OPA[…]. Le but est de mieux prendre en compte les besoins des producteurs et de les responsabiliser davantage dans la prise de décision au niveau stratégique car le GAP assure les fonctions d’orientation, de supervision et de contrôle.” (Evaluation «Appui aux organisations paysannes de la province de la Kompienga, p.15)

An after action review of the project “Improving Household Food Security and Nutrition in the Luapula Valley, Zambia” identified some useful lessons for the design and implementation of CD interventions that use community empowerment as their project strategies – see box 4.

Box 4: Two lessons from a nutrition project in Zambia on participatory approaches in CD

35 Lesson: Building capacity for project implementation. The experience of many grass-roots community support organisations using participatory and integrated approaches is recent and limited. Often, existing support services are poorly equipped (in terms of human capacities and logistics) to identify and appropriately respond to the needs of community members. As a consequence, the need for capacity-building, especially with regard to participatory project planning and implementation, is tremendous and requires not only skilled staff, but also many more staff than are available. This is a problem especially in view of government budgetary cuts and down-sizing of essential services. A major lesson learned from the first phase of implementation is that project design should account for this need and that capacity-building should be closely integrated into field activities and well timed so as to ensure that support services staff have acquired the necessary knowledge and skills.

Lesson: Institutionalising the community action planning approach Linked to the issue of technical capacity of community support staff to implement and sustain participatory approaches in their daily work, is the observation that many community support staff are often unmotivated and hampered in their work by a lack of operational resources and opportunities. While governments and local institutions may institutionalise participatory approaches, rendering these approaches operational requires qualified staff and resources. Many local institutions lack the required resource base. Well-qualified individuals frequently seek better opportunities outside or on the margins of the public sector and are not easily available unless competitive incentives are provided. Others within the public sector often lack the resources and means for continued implementation of what was learned during the project and await opportunities provided by other projects. However, while projects are temporary features, the need and demand for services created through participatory projects remain and need to be addressed if the momentum is to be maintained. Several lessons can be learned in this respect. - The first lesson is that, when designing participatory and other projects, it is essential to objectively evaluate the capacity of the public service resource base to provide the required community support services during and following the termination of the project. Often, project resources are the only source available for public service operations. This may seriously hamper project operations, but also compromise the sustainability and effective institutionalisation of project experiences. - The second lesson is that projects should not only build capacity in community support services, but also increasingly focus on building capacity within the communities in order to strengthen the communities' resource base in terms of knowledge, social organisation, financial capital etc.. This should be part and parcel of project design, as opposed to a situation where this responsibility is deferred to the community support services after project closure. This will allow the communities to internalise project experiences within their own local institutions and structures. - The third lesson is a warning and relates to the need for project designers to refrain from shifting excessive responsibility to the communities. There is a limit to how much communities can do themselves. The public sector has, and always will have, an important responsibility for the provision of some basic services, not least to create an environment that enables communities to take charge. Source: Participation website of FAO, available at: www.fao.org/Participation/zambia-lesson.html

Addressing inter-disciplinary concerns and needs: The traditional type of CD support, for example in the livestock sector, has mainly consisted of technology transfer at the individual level. Evaluations increasingly emphasized that technical assistance projects needed to be more development-based interventions and pay greater attention to basic development needs as interdisciplinary concerns requiring changes at institutional and policy levels. “Some of the projects reviewed were conceived many years ago, when they were more relevant and made valuable contributions through direct technology transfers. Now, however, the surrounding environment has grown more complex as societies have opened, markets have been liberalized and communications have been revolutionized. There is a diminishing return on small technology transfer projects, whose impact is generally only local and short-term. In this evolving environment, any new technical intervention always needs to be set in a development context, which introduces consideration of broader social, economic, political and environmental issues in the design process.” (Livestock evaluation, §136-137)

Sustainability issues, a matter of seeking synergies and paying attention to follow-up requirements:

36 In this and other thematic evaluations undertaken in recent years, project design and follow- up have consistently scored lower among the various project components. One way to maximize sustainability of CD results is to effectively tie into country programme efforts during design stages. “The two most critical issues identified by field evaluation missions were project design and how to achieve sustainable effects and impact. Some design deficiencies were: overestimation of the project’s potential impact (especially for TCP projects), a lack of attention to synergies and follow-up requirements, and a pro-forma designation of poor farmers and especially women as the project’s target group when realistically these groups were too poor to raise the required capital necessary for participation in the project.. […] The lowest rated items project design and sustainable effects and impact] were often attributable to the fact that follow-up in some cases was not sufficiently considered before the project was undertaken; rather there were hopes that the eventual success of the activity would influence donors or host Governments to take appropriate follow-up action.” (Evaluation of FAO Activities in Crop Production, §136 and §98) “The most successful examples of the utilization of the skills, knowledge and tools introduced by the programme were in countries where it was linked during planning stages to existing food security information systems, and thereby to local capacities, resources and action plans, which maximized the sustainability of programme impact.” (Evaluation of EC/FAO FSIA Programme, §5)

Design of Logical frameworks, Monitoring & Evaluation: In the meta-evaluation, these frameworks stood out as the weakest design issues, both in terms of lack of clarity/coherence in the logical flow between objectives, results and activities, but also in the inadequate choice of indicators to monitor and report on. One of the criticisms of evaluators was that these frameworks often lacked specificity (e.g. to accommodate a flexible process approach) and focused more on measuring progress in activities while omitting to measure achievements at the outcome or impact levels. Another shortcoming was the insufficient guidance in the operationalizing the Programme in terms of describing activities, linkages, and outputs. “Despite the coherence between FSIA strategies and activities, the final evaluation team agrees with the Mid Term Review’s conclusion that, given the complexity of the programme, the description of objectives, outputs, activities and inputs included in the logical framework were too simplified to be useful as a global monitoring tool.[…] Although the programme objectively verified country level outputs with great success, the lack of specificity in the logical framework resulted in a failure to effectively or systematically identify, describe or monitor outcomes. […] “The monitoring and evaluation system was focused on measuring activities and outputs and producing progress reports, yet it did not attempt to capture measurable outcomes.” (Evaluation EC/FAO FSIA programme, §39, 217 and 87) “The SFLP design was very ambitious, seeking to effect national-level policy change from community-based initiatives, and then inform regional and global policy. The SFLP project design document outlined a flexible process approach, but lacked detail and focus, perhaps as a result of the design team’s intention to use an approach that could be adapted to facilitate participation and enhance ownership of activities. However, use of a process approach does not negate the need for clarity and focus within a design. […] Precise logframes are not possible in every intervention e.g. The final evaluation team is convinced that, when dealing with policy and institutional change, it is impossible to specify in detail at the outset, and even as the programme proceeds, the precise way in

37 which it will be implemented. A process approach is, therefore, not only the most appropriate, but appears to be unavoidable. However, to be effective, the flexibility offered by the process approach must be matched by dynamic rigour in its implementation. When decisions are made, it is important to document why they are made and what it is intended to achieve, and to ensure that the programme continues to move towards achieving its goal and purpose. A process approach does not preclude careful and regional monitoring against a management framework”. (Evaluation of the Sustainable Fisheries Livelihoods Programme, §252 and 259) “The [evaluation] team made reference to the M&E framework for the purposes of the evaluation, but found the indicators more a distraction than a useful guide. This is in part because most indicators have yet to be properly quantified, but even then many cannot realistically be measured and most do not provide a suitable measure of impact. For example: the “quality and extent of harmonisation of food security strategies produced” is a very difficult indicator to understand, let alone measure; “SIFSIA Steering Committee meeting on a regular basis” may be achieved but still does not promote cross- sectoral partnership (output 1); and “number of staff who acquire relevant skills and competencies through training” does not reflect how many are then using those skills in support of developing food security policy and interventions (output 2). (Mid-Term evaluation of the SIFSIA-North Programme, §5.5).

Gender considerations: Gender considerations did not score satisfactorily in many of the evaluations. It appeared that CD interventions viewed gender as a stand alone set of activities and not as a cross-cutting concern. One evaluation quoted below provided a clear analysis of the consequences of this failure across various aspects of CD performance. Greater attention to gender issues was consistently recommended by evaluations. « Le principe d’égalité de genre, retenu dans les orientations de travail tant du bailleur de fonds que de la FAO, n’était pas inscrit dans les principes du programme et peu abordé dans les documents de projets. Les questions de genre ont été peu prises en compte et, le plus souvent, avec une démarche méthodologique qui les abordait de façon séparée de l’action alors qu’elles auraient été d’autant mieux traitées dans une approche transversale. » (Evaluation axe thématique «Approche participative et gestion de terroirs», p.4) « La mission a ainsi pu constater que les actions ont moins bénéficié aux agricultrices, aux groupements féminins et aux femmes rurales. Cela s’est traduit, dans chaque projet, par toute une série de ruptures : (i) méthodologiques : traitement du genre dissociée de l’action, absence d’une situation de référence des disparités à réduire (BKF, NIG), (ii) opérationnelles : mauvais ciblage des apports, actions « passe partout » peu argumentées au plan économique et social (SEN, BKF), (iii) organisationnelles : niveau d’accès aux facteurs de production peu mesuré, comme le crédit (SEN, BKF, NIG), (iv) stratégique : absence de vision (BKF, NIG), (v) politiques : faible représentation des groupements féminins (NIG, BKF). (Evaluation axe thématique «Approche participative et gestion de terroirs», p.45)

6.3 Implementation, management and efficiency

The management and efficiency of programme implementation generally did not score high, as shown in the previous chapter. Evidence from country evaluations especially indicated that FAO Representations faced many challenges, particularly in terms of the volume of

38 operations to manage with often overstretched human resources, the financial and administrative tools available to FAOR being inadequate or insufficient, while coordination with headquarters witnessed fragmented linkages.

Efficiency – limitations of TCP format: The TCP was originally developed at a time when FAO had a strong and increasing Field Programme, which the TCP often served to compliment, through timely interventions that could not have been otherwise implemented. However, with the decline of donor-funded field projects in recent years, the role played by TCP has changed. In many areas, including crop production, TCP has become the most viable (or only) alternative to test normative concepts in a field setting. Yet, several evaluations questioned the suitability of TCP format for addressing complex or ambitious CD needs, for instance some TCP were regarded as having ‘too ambitious goals/objectives with respect to short duration’. “Work in crop production is often not amenable to the two-year limit on TCP projects. While some specific topics related to crop production may be amenable to TCP (e.g. specific types of training), other activities relating to promotion of specific crops may not because of the longer amount of time for results to appear. Consequently, in the absence of donor funding that could support more long-term initiatives, some TCP projects tended to over-estimate their potential impact without a realistic appraisal of the requirements in terms of time and resources. In addition, in some cases, delays in project approval or implementation (usually recruitment of staff) have led to projects starting later than originally foreseen. In some cases, this has led to projects effectively being implemented for one season shorter than it was designed, or a need (due to the statutory limit on TCP projects) to design and approve a follow-on project. (Evaluation of FAO Activities in Crop Production, §107-108) “This [greater attention to design and preparation in TCPs] is closely related to the observed evolution of TCPs away from the narrow transfer of a technology, a specific methodology or advice on a narrowly defined subject into increasingly complex issues of poverty alleviation and socio-economic development. As noted in greater detail below, this is a necessary and inevitable evolution (and a positive one), but it puts a strain on the existing TCP mechanism.” (Livestock evaluation, §133-134)

Too many activities, less strategic steering: Large field programmes with many activities put so much intensity and pressure on active implementation that they kept attention away from documenting strategic directions. In the case of the Sustainable Fisheries Livelihoods Programme (SFLP), according to evaluators, this activity focus may have impeded progress toward further developing and promoting the Programme’s regional aspects.

The risks of top-down management: Programmes with a limited country presence and relying on management decisions from headquarters were unable to respond in a timely fashion to implementation challenges and opportunities arising at field level. “There was a strong tendency to rely on top-down management rather than promote decentralization and delegate budgets and decision-making processes to the country level. [...] the administrative process of approval did not support a timely response to country requests, effectiveness of the programme at the field level was impaired. Delays in approval impaired the timely implementation of demand-driven activities, and in some

39 cases there were limited windows of opportunity for responding to donor contributions.” (Evaluation EC/FAO Programme, § 217)

6.4 Effectiveness and impact

Despite the dearth of comprehensive CD impact analyses provided by evaluation reports, several evaluations did show promising results especially for those CD activities intervening at the policy and institutional levels. It is clear however that in order to achieve impact interventions supporting CD require a long term process and a comprehensive approach across the different levels (individual, organization, institution/policy). The following quote from the Mozambique evaluation illustrates this point: “Capacity building is a feature of nearly all FAO interventions but has often been a weak area in terms of impact. Some of the long-term interventions like the marketing intervention within the framework of MIC and early warning have been relatively successful because they have been able to take a broad approach to capacity building: they have provided training to staff over a long period and they have in addition to this been able to address an important dimension of capacity building by establishing appropriate organisational structures. The impact of FAO capacity building is constrained by a number of factors like the lack of incentives to employees, the toll on human capacity taken by the HIV/AIDS crisis and the general low level of education. While these constraints go beyond FAO’s control, they should be taken into account in FAO’s capacity building strategies. FAO (and other organizations) may need to consider that long-term support to some vital areas of government is required. (Mozambique country program evaluation, §xxxv)

The difficulty of assessing impact of CD support: Evaluations had a hard time assessing the impact of CD activities. Often the real impact of CD interventions will show with time. For this reason, achievements are mostly reported in terms of outputs (see above M&E frameworks). It is difficult for projects to find the right balance between demonstrating short term achievements attributable to project intervention and fostering long term CD processes, including building ownership among stakeholders, within the limited project timeframe. “The design of Regular Programme Technical Projects poses considerable difficulty in terms of their contribution to key FAO strategic objectives. Impact assessment is constrained by the difficulties in determining cause-and-effect linkages between the outputs produced by the programme (e.g. trained individuals, publications, networks, workshop results) and developmental changes at country level.” (Fisheries Evaluation, §118) “Les projets ont su inciter le changement et mobiliser les acteurs pour améliorer leurs capacités et leurs compétences, ainsi que pour mieux définir des voies et des moyens pour répondre à leurs propres projets. Toutefois, il faut être modeste et se rappeler que le renforcement des capacités prend toute sa portée dans le temps et que les résultats ne sont pas totalement prévisibles, d’autant que les actions s’insèrent dans un environnement changeant et soumis à des systèmes vivants qui influent grandement sur les résultats et leur pérennisation..” (Evaluation axe thématique « Approche participative et gestion de terroirs», p.38) « Aussi, les résultats sont faibles en matière de professionnalisation. Les Unions qui ont été crées par le projet ne sont pas suffisamment fonctionnelles car elles ont été mises en

40 place davantage comme un objectif à atteindre que comme une réelle émanation de la base. Même si certains groupes à la base commencent à se professionnaliser, cela tient davantage au boom économique de la zone dû à l’arrivée d’opérateurs économiques qu’au personnel technique du projet qui n’a pas réussi à définir une stratégie visant à faire passer les organisations du stade d’OP à celui d’OPA. » (Evaluation «Appui aux organisations paysannes de la province de la Kompienga, p.5)

Examples of successful results of FAO’ s CD support at organizational, institutional and policy levels:

• Establishing and supporting national institutional structures: “Les pays membres de la CLCPRO, et plus particulièrement ceux de la ligne de front, ont fait d’énormes progrès dans l’organisation et la conduite de la lutte préventive depuis les débuts du Programme, surtout là où existe un centre antiacridien autonome. La création des CNLA autonomes restera sans doute l’un des plus grands acquis du programme. Une vraie coopération régionale est maintenant en place et se concrétise en de nombreux échanges d’expériences et de moyens matériels et humains. La gestion participative et transparente des ateliers est à souligner. La planification et la prise de décision conjointe avec “Africa Emergency Locust Project” (Banque mondiale) sont exemplaires.” (Evaluation Programme EMPRES en Région occidentale, p.38)

• Enhancing organizational capacity: Cependant, si l’on regarde la façon dont les Organisations Paysannes sont entrées dans le processus de déblocage des fonds, le Promotion paysanne dans le cadre de la gestion des terroirs pour la sécurité alimentaire/ PPSA (Promotion paysanne pour la sécurité alimentaire), sans volonté particulière, a donné l’occasion à quelques organisations, à des degrés divers, d’apprendre à analyser leur situation pour décider d’une action commune, d’acquérir un minimum de savoir faire pour le montage d’un dossier, de suivre une procédure d’allocation de ressources, d’accroître, à un moment donné, un capital de ressources matérielles et, dans certains cas, à le renouveler et à rendre des comptes. (Evaluation tripartite de l’axe thématique « Approche participative et gestion de terroirs », p.39)

• FAO’s assistance to policy and strategy development was generally highly appreciated by evaluations, as shown in Mozambique. However, it was felt that countries needed greater support with the implementation of policies and strategies: “FAO support to policy assistance is a prominent feature of the programme in Mozambique. This has generally led to positive results: solid policy/strategy documents appreciated and owned by stakeholders. Stakeholders appreciated FAO’s role as a broker, its adherence to high technical standards and its long-term standing as a dependable unbiased partner to government. Based on this FAO has provided policy assistance to sensitive or politicised areas where other actors cannot enter. An example of this was FAO’s involvement in the formulation of a law and regulatory frameworks regarding land for which there were conflicting views among stakeholders. FAO succeeded in promoting regulatory frameworks that protect community rights while at the same time promoting commercial investments in farming. The policy assistance seems to have been most effective where FAO was able to provide continuous support

41 throughout the policy formulation process through senior experts.” (Mozambique evaluation, §xxvii) “The achievement until now is impressive given the size of the Programme and the short time that has elapsed since the beginning of the second phase. This progress is particularly significant in the way it has raised awareness of the issues and in the case of food security, in relation to the work documenting and improving the existing policy, legal and regulatory frameworks and introducing new concepts, methodologies and best practices. However, one shortcoming of the work related to these frameworks is that in some countries, there is a widespread attitude that it assumes that if an idea or proposal is written and approved then it is already applied. As a result, future work should progress into designing implementing strategies and helping the countries in the practical implementation of the principles and actions that have already been delineated. This is particularly true in smaller and more vulnerable countries where FNPP could have a larger impact and where immediate application is urgently needed (e.g. Bhutan).” (Evaluation of FAO-Netherlands Partnership Programme, §.71)

The issue of ownership is linked to impact : The extent to which the training and institutional strengthening activities achieved lasting impact was directly associated interventions efforts to build effective ownership among participating institutions. A clear example of this comes from « The need for ownership was for the most part neglected in the programme. For long- term success, ownership must be linked directly to participating institutions and organizations. Lack of ownership was largely responsible for the relatively low level of achieved or expected impact from the training and capacity-building activities.» (Evaluation EC/FAO Programme, §112)

FAO Normative products awareness/use/utility: Evidence from evaluations indicates there is inconsistent awareness of FAO’s global services products. Government and other stakeholders refer to the information they get from FAO mostly through publications distributed by FAOR or by projects, through the Internet or through conferences and workshops. Although some projects make the efforts to translate project-related materials into local languages (e.g. West Africa youth programme, Axe thématique « Approche participative et gestion de terroirs »), this is only a minority. Language is recurrently cited as a limitation to facilitate spontaneous dissemination of normative products. There is an opportunity for FAO to better promote its global products by familiarizing partners with the range of global services offered “ […] the fact that not all Mozambican key staff are able to read English or French limits somewhat the use of the information. The fact that only a few have good Internet access is a further constraint. (Mozambique country programme evaluation, §228)

FAO is successful in its role as convener, mediator and “honest broker”: FAO was shown across evaluations to play a significant role at national level, leading and/or supporting inter-agency initiatives (e.g. clusters in short-term humanitarian response) and sectoral (longer term developmental approach) coordination relating to agriculture and food security. However, it was felt that this distinctive role could be put to use more effectively for closing the gap between the two tracks of emergency and development. “Ineffective (at times non-existent) linkages between short-term (humanitarian) coordination and longer term (developmental) coordination mechanisms are notorious

42 across the [Horn of Africa] region. This needs to be strengthened and FAO could play a key role in bridging this gap within the food security sector because it generally has a foot in both camps. No better example or context to do this than Kenya where FAO has been engaged with both the Agriculture & Livestock Working Group (of the KFSM) and the Agriculture Sector Working Group (ASCU) which are not linked through government.” (Evaluation of FAO’s Emergency & Rehabilitation Assistance in the Greater Horn of Africa, §330)

6.5 Sustainability of project results

Scoring low across most evaluations, the sustainability and follow up of results achieved remain a serious challenge in most CD support interventions. In the case of limited TCP funds to support, however, country statistical capacities has forced FAO’s Statistics Service (ESS) to help countries find other complementary funds in order to ensure sustainability of interventions, and this with reasonable success. “[…] The weakest criterion was that of sustainability and follow-up. This is an area that is especially a concern with TCP projects which are part of a larger initiative or seek to introduce changes to favour development of a particular group or activity (such as the SPFS diversification component or the small-scale dairy development projects), that they need to develop the foundation for a longer-term engagement. It is linked to the evolution towards more complex TCPs noted in the previous paragraph.” (Corporate Evaluation on Livestock, §133-134) “Long run sustainability of capacity building projects [is put] at risk [due to little follow- up]. Given the longer gestation periods associated with CD efforts, such as an agricultural census, this is one instance where the project duration limit of TCP could have come into play. The auto-evaluation also found that sustainability of project results remains an important issue in many countries. […] FAO develops a TCP project for capacity building (say, undertaking an agricultural census) with the country. But staff from ESS will not implement the TCP until the country has found a donor to support the census undertaking. In this way, the small TCP is used as seed money to attract donor support. As well, the donors like to see that FAO has committed to provide guidance and support for the census. (Evaluation FAO work in Statistics, §86-87)

Follow up to training interventions: Most evaluations considered that implementation of training lacked proper follow-up and mentoring support to ensure that new-found skills and learning effectively translated into application and better job performance. Linked to the issue of follow up, the meta-analysis found very few instances of CD interventions using a cascade or training of trainers approach for their training activities. This prevented trained actors from getting involved in scaling out their CD activities at other levels to further disseminate the knowledge and skills acquired. “There is a need to sharpen the approach to capacity-building in both sub-programmes, particularly around training to achieve maximum impact. For example, the training of individuals should be linked to particular outputs and expectations of performance within their respective ministries, attention should be given to follow-up support and mentoring, and there needs to be closer monitoring of the impact of investment in training. Although there has been pressure to deliver training as a tangible output, this must be better contextualised. […] As far as training is concerned, large numbers of staff have benefited, but more follow-up is needed to address institutional and organisational

43 issues and to provide continued on-the-job support so that trainees are able to apply their skills.”(Evaluation of SIFSIA programme, §4.2) “For capacity-building interventions to achieve substantial impact they must build and sustain a critical mass of expertise within the institutions and the relevant institutional networks. In subsequent programme phases, a training of trainers approach will contribute to this.” (Evaluation EC/FAO FSIA Program, §114)

Short project duration and unclear exit strategy for sustainability and follow up: Evaluations found that the design of many CD projects lacked some vision of “after-project”. Sustainability depended on the extent to which intervention envisaged an exit strategy ensuring a continuity and also on the ability of projects to formulate tools or mechanisms that allowed client groups to pursue and further engage in other partnerships or seek other funds. « La durée des projets pilotes ayant été courte et leur fin brutale, il y a malheureusement peu à escompter en matière de durabilité. Là se pose dans toute leur acuité les limites des engagements ponctuels sans anticipation de l’après projet et des conséquences du retrait. Néanmoins, le projet Sénégal a concrètement rodé un mécanisme de financement satisfaisant et produit un guide de procédures qui ont des potentialités de reprise par d’autres projets […]. Le projet de Burkina Faso a de son côté validé des outils de poursuite des actions basés sur le parrainage et l’auto-évaluation » (Evaluation axe thématique «Approche participative et gestion de terroirs», p.44)

Commitment by recipient Governments: Even when assistance provided was valuable and comprehensive, the ultimate change and sustainability of project contributions will depend on the recipient Government commitment to ensure sufficient budget for follow up and maintenance of activities. As the case of Mozambique shows, the Government did not commit sufficient financial and human resources support despite several years of projects investing in Early Warning Systems work.. “FAO has for a long period provided support to food security information and policy. This support focused on the development of improved methodologies and training for early warning, support to food security and nutrition strategies, policy formulation, vulnerability assessment methodology and development of food security and nutrition plans. Valuable contributions have been made, but Government financial support in these areas has been weak and there has been a high turnover of staff.” (Evaluation of FAO Cooperation in Mozambique, § xxiv)

6.6 Partnerships

The nature of cooperation with a variety of development actors in CD interventions was overall insufficiently analyzed by evaluations. Nonetheless, evaluators unanimously stressed the need for FAO to increase efforts in establishing and working through partnerships with a range of actors especially given FAO’s generally weaker role as implementing agency compared to its other roles (convener/facilitator, analytical/normative).

Traditional versus new/innovative partnerships: Evaluations reviewed show that FAO interacts with and builds partnerships with a wide range of partners: government institutions, donors, UN organisations, NGOs and private enterprises. Country-level activities are commonly implemented in close collaboration with national

44 counterpart institutions. Traditional partners are often government ministries, especially Ministries of Agriculture. In general, more attention should be paid in CD interventions to seeking partnerships also with the private sector. Evaluators agreed more efforts were needed to explore innovative partnerships, such as those in the private sector, that can bring valuable and sustainable solutions to problems. Although seldom mentioned, the cooperation with the private sector in CD interventions received positive appreciation in evaluations in the few cases available. “While contributions from partnerships have been quite positive, most of the partnerships FAO selected were with traditional partners (Ministries of Agriculture, Meteorology Services and National Statistics Offices). There has been little outreach to form new partnerships and build capacity more broadly. The exception to this is with the Integrated Phase Classification tool, where working relationships are being established with WFP, UNICEF and multiple NGOs, and with the FSIP (Food Security Information Products CD activity), which involved 50 different institutions in the five countries where the training workshops were conducted.” (Evaluation EC/FAO FSIA Programme, §83) “Considering the growing importance of the private sector in the food and agriculture chain in Mozambique, partnerships with private enterprise should be explored more fully as a way to address problems. A private trader capable of distributing new genetic fish stocks would promote sustainability of fish farming, and dialogue with seed companies might contribute to solving the problems of the seed quality at the Input Trade Fairs.” (Mozambique Evaluation, §68) « Le projet a […] réussi à construire une véritable synergie au niveau de la zone en développant des partenariats avec des structures publiques et privées. Il a ainsi permis de valoriser les services techniques déconcentrés et de mettre en relation les Organisations Paysannes avec le réseau des Caisses Populaires. » (Evaluation «Appui aux organisations paysannes de la province de la Kompienga)

Benefits of partnership building: The benefits from partnerships were seen both in terms of trust and ownership building, while the cumulative effects brought about by synergies among existing initiatives allowed FAO’s activities to reach greater leverage at outcome and impact levels. “FAO has, a special relationship with government. FAO primarily regards itself as a partner to government without any specific agenda, and Government in turn regards FAO as an honest broker to whom sensitive tasks can be entrusted […] (Evaluation of FAO Cooperation in Mozambique, §213) “Country-level activities are commonly implemented in close collaboration with national counterpart institutions. This contributed to national level capacity and commitment to the objectives and results of programme interventions/activities.” […] “At the country level, programme benefits were higher in countries with pre-existing institutional or programmatic frameworks where an attempt was made to build synergy and partnership with these initiatives.” (Evaluation EC/FAO FSIA Programme, §81)

Challenges in partnership building: Among one of the frequently cited constraint for FAO’s work in partnership was related to the limitations in the efficiency of administrative procedures” “Implementing partners (government departments, UN organisations and NGOs) generally find that dealing with FAO is complicated by its procedures (e.g. seed trade fairs that should follow the agricultural calendar but are delayed or have to be

45 cancelled). Government departments consistently pointed to their difficulties in finalising an activity based on Letter of Agreement (LoA).” (Evaluation of FAO Cooperation in Mozambique, §237)

46 7. Conclusions and Recommendations

This chapter draws on both the review of evaluations’ main recommendations related to CD and the meta-analysis findings presented in the previous chapter. While not intended to be exhaustive, it offers some relevant conclusions and recommendations for improving FAO’s performance in CD support.

7.1 Understanding of CD and support to CD by development practitioners FAO programmes are primarily about supporting CD needs in developing countries, yet CD has served as a catch-all term for very different issues related to capacities. Those responsible for CD intervention programmes, and their evaluation, must be specific in portraying CD problems and proposed solutions. Support to CD must be understood and approached concurrently across different dimensions in order to respond to different needs at the levels of individuals, organizations and the policy/institutional environment. For instance, one evaluation provided an innovative and comprehensive CD analysis of a regional thematic programme. The evaluation regrouped the multiple entry points or intervention modalities of the thematic axis into three categories which underpin a CD process (training of individuals, support to organizational structures, development of institutional framework).

7.2 Design and modalities of CD support in FAO’s activities

• Provide CD support as an integrated package: Evaluations overall agreed on the need to address the CD challenges as a system and to recognize needs across different dimensions and functional capacities. In key areas of FAO’s comparative advantage there must be more resources at field level to better respond to needs and demand for technical assistance. “The evaluation shares the view of the IEE that “capacity building must be delivered as an integrated whole bringing together technical cooperation, access to knowledge, experience and decision-making, with FAO both as a facilitator and provider.” […] The creation of regional teams in the Statistics Service (ESS) and an increased role of Regional and Country Offices in the FAO’s Statistical System will be key elements for the successful undertaking of such a review.” (Corporate evaluation of FAO’s role and work in statistics, §102) • Assess capacity needs before designing CD activities: Carrying out a careful diagnosis of capacity needs/gaps, including a constraints analysis, has been highlighted and recommended as an essential first step to design and tailor CD activities. This will allow interventions to address demand-driven needs at country level, especially when activities aim to support organizations. “Le diagnostic initial [des organisations paysannes] a constitué une base essentielle de l’approche participative car il a permis d’avoir une meilleure connaissance des OP/OPA de la province (caractéristiques générales, capacités organisationnelles, fonctionnelles et financières) en vue de profiler son intervention.” (Evaluation axe thématique «Approche participative et gestion de terroirs») “Conduct a capacity/needs assessment and FSIS constraint analysis before initiating any activities in country to make sure that the activities initiated by the programme address specific country needs, opportunities, contexts and possibilities for sustainability.” (Evaluation EC/FAO programme, p.8)

47 • Providing organizational support, and not just transfer of technical skills: Meta-evaluation findings indicated that FAO intervenes comparatively less to enhance organizational capacities. Evaluative evidence was very clear as to the need to encompass all the skills needed to perform different functions and not just focus on technical skills. As shown above, a careful analysis of capacity needs/gaps and constraints across different dimensions is necessary to ensure CD support is properly integrated. For example, to strengthen the capacities of farmer’s organizations, one evaluation recommended that activities be tailored at different needs in terms of both technical and organizational skills of farmers, while ensuring proper links and partnerships with other stakeholders are in place for their effective functioning: “Etant donné le nombre et la diversité des OP (Organizations Paysannes) et de leurs besoins, la mission préconise de garder présent à l’esprit lors de la préparation du futur programme, les éléments suivants: − poursuivre les actions visant le renforcement des capacités techniques et organisationnelles des OP dans leur diversité pour assumer leurs multiples fonctions, tout en se rappelant qu’il existe un large éventail d’OP à des degrés de maturité très variable, allant des OP aptes à négocier à haut niveau jusqu’à des OP fragiles qui ont peu ou pas reçu d’appuis et dont les activités restent limitées, voire précaires ; ceci plaide en faveur d’appuis diversifiés, adaptés aux conditions des différentes catégories d’organisations et aux différents niveaux. − travailler avec les groupements de base, bénéficiaires ultimes, en s’orientant vers le renforcement des capacités et des appuis appropriés et validés, mais aussi accompagner les niveaux intermédiaires indispensables au bon fonctionnement des services aux agriculteurs et aux agricultrices ; − promouvoir le partenariat pour que l’intervention soit négociée dès sa conception avec les OP elles-mêmes et les autres parties prenantes, et travailler en relation avec les collectivités locales et les chambres d’agriculture qui se mettent progressivement en place. − chercher à améliorer l’environnement économique, social et institutionnel des OP ; en effet l’appui institutionnel contribue certes à mieux fixer un projet, à négocier différemment, à mettre en œuvre efficacement des programmes, etc., mais il ne peut lever tous les obstacles; par exemple, les besoins en financement sont considérables au niveau des exploitations agricoles, ce qui suppose une meilleure articulation entre les intervenants agissant directement sur la production agricole et les institutions de micro financement.” (Evaluation axe thématique «Approche participative et gestion de terroirs», p.5, 57)

• Continue facilitating policy change, but increase country support for policy implementation: Overall, CD support aimed at improving the enabling environment (e.g. policy development, managing information systems, etc.) within various technical sectors (land, marketing, food security, etc) was generally praised by evaluations, confirming that FAO has a comparative advantage lies in its role as “a neutral, honest broker”. The perceived comparative advantage of FAO in supporting/facilitating policy change was repeatedly underlined in evaluations (e.g. Evaluation of Crop production, FNPP evaluation). However, nearly all studies recommended to increase support to countries for implementing these policies/strategies. In this respect, it was suggested to use pilot field projects strategically in order to influence policy processes (Livestock evaluation).

48 “With the rapidly expanding impact of globalization and related socio-economic changes, there will be an increased demand for independent and reliable advice on a variety of policy matters. A considerable amount of confusion still exists on biotechnology and related subjects. FAO should help its member countries in the development of their national strategies and related policies and procedures,…” (Crop production evaluation, §123) “FNPP should continue work to improve national and regional (e.g. Central America) policy, legal and regulatory frameworks and as far as possible, work towards the conceptual integration of the three thematic areas. In addition the Programme should do more work on designing strategies for implementing the policies, in particular in the more vulnerable countries with responsive Governments.” (rec#10, FNPP Evaluation) “FAO also engaged in some small-scale livestock distribution schemes. These have been of limited impact for others than the immediate beneficiaries and it is difficult to see a comparative advantage for FAO in undertaking these interventions. It is more appropriate for FAO to engage in promoting livestock development at the policy and planning level by assisting government in developing plans for increased commercial and family level livestock production.” (Evaluation of FAO-Mozambique cooperation, §xix) Les efforts de la Coordination pour harmoniser les activités environnementales à travers un CCE global doivent se poursuivre et s’orienter vers la mise en place des capacités nécessaires et le respect d’indicateurs et de normes prioritaires.” (recomm.10, Evaluation EMPRES programme) “Technology-based field projects still have their place, and a very important one. Fieldwork will always have a key role to play as a basis for AGA to achieve changes in policies, institutions and attitudes. But to the extent possible, field projects should ideally be used as part of the policy process, serving as demonstrations, or for piloting new initiatives suggested by the policy dialogue”. (Livestock evaluation, §139)

• How can CD projects adopt a gender approach? Review of evaluative evidence strongly suggests that there are considerable variations in CD interventions over how gender is understood and how the implications for mainstreaming. This was also reflected in the evaluations themselves where different criteria were used to assess gender mainstreaming (in some cases defined in terms of women’s participation, while in others the focus being on the relative rights of the two sexes). Whilst effective implementation of gender mainstreaming remains a challenge for most CD interventions, a number of entry points for gender mainstreaming were also identified, such as the following ones put forward by one evaluation: « Aussi, les projets, en plus d’être volontaristes et motivés pour aborder les questions de genre, doivent se doter, en se basant sur une véritable analyse socioéconomique selon le genre du contexte d’intervention (situation de référence), d’instruments « de genre » minimaux pour créer un environnement porteur. Il s’agit pour les citer : (i) d’une vision stratégique claire pour la réduction des disparités, dotée d’un plan d’action intégré à toutes les étapes et à tous les niveaux de ’intervention ; (ii) d’une formation des acteurs concernés, en particulier des cadres et des agents des projets, dans une optique de réinvestissement professionnel et de son suivi; (iii) des lignes directrices de mise en oeuvre, dont des ressources humaines équilibrées en terme de ratio hommes femmes ; (iv) des analyses régulières de situation selon le genre et des modes de production de données et d’informations sexospécifiques ; (v) des outils de budgétisation et de veille de

49 l’utilisation des ressources financières dans une optique de parité; (vi) des mécanismes de suivi et des indicateurs d’évaluation qualitatifs et quantitatifs, notamment concernant la participation de chaque catégorie d’acteurs et des bénéfices retirés; (vii) des modes d’information et de communication pour partager les ressources et les expériences (réseau, rencontres, sites web, etc.). […] Néanmoins, les trois projets nationaux auraient certainement eu plus d’impact sur la réduction de la pauvreté et la sécurité alimentaire si les questions de genre avaient été considérées à part entière, dès lors où les femmes et leurs structures sont des actrices incontournables à toutes les étapes de la production agricole (incluant la gestion et la transformation des denrées alimentaires), et souvent les plus nombreuses en tant qu’actives agricoles (entre autres à cause de l’exode rural masculin). (Evaluation de l’axe thématique «Approche participative et gestion de terroirs», p.42, 46)

• Targeting vulnerable groups in rural CD interventions: Projects targeting organizational capacities must take care not to preferentially focus on “higher” and better recognized organizations and to exclude smaller groupings of poorer farmers, especially vulnerable groups and women farmers. This should be done by carrying out a baseline participatory diagnostic to identify vulnerable groups. « Ce sont surtout les capacités organisationnelles d’OP déjà arrivées à un certain niveau d’autonomie qui ont été touchées ; les groupes les plus isolés, moins ouverts à l’information et moins habitués à interagir avec des projets de développement de ce type, dont les groupements féminins, ont pu être pénalisés. (Evaluation de l’axe thématique «Approche participative et gestion de terroirs», p.37)

7.3 Efficiency

• Prioritizing CD support is essential given FAO’s limited funds: Evaluations generally find CD initiatives are invariably small, often of too short duration and thus have only limited impact. As one evaluation put it: “Capacity building, while generally present in FAO activities, has been insufficient to create the necessary critical mass. More capacity building seems always to be required.” A number of evaluations questioned whether CD initiatives can actually address the profound capacity requirements that are universally cited as an obstacle to sustainable development interventions. “As FAO is not a funding agency, its possibilities for carrying out extensive capacity building will always be constrained. Accordingly, it may be able to make a greater impact through assessments of capacity building needs in particular areas and make this information available to Government and Development Partners. Such assessments should take into account capacities outside of government. In Sierra Leone, this would usually mean NGOs and to a much more limited extent, the private sector. At the same time, FAOR should create awareness among Government, donors, NGOs and private sector about FAO training materials and facilitate their dissemination. Co-funding arrangements and public-private partnerships for capacity building activities should be sought”. (Sierra Leone evaluation)

In the face of continuing high demand for CD assistance and FAO’s overall weak capacity to deliver capacities, several evaluations emphasized the need to prioritize CD support based on a careful and realistic assessment of FAO’s comparative advantage, a clear identification of the “right mix of services”, and by avoiding taking on “the lead” where resources are limited.

50 Evaluative evidence suggested that FAO focus more on acting as facilitator, working more through partnerships rather than direct provider of technical assistance. “The selection of the right mix of services to be provided by all partners is critical to maximise the impact of FAO services. It is critical that FAO define technical assistance support strategies through a clear identification of the mix of services that should be provided, in parallel and over time, in order to maximise the impact of its assistance and ensure its sustainability.” (Mozambique evaluation, §xxxviii) “Care needs to be taken that the expectations of response to requests for technological assistance do not exceed the breadth of staff skills and depth of resources available, giving more active consideration, for example, to adopting a facilitator role rather than providing technical assistance directly and working through partnerships at the technology-transfer interface at regional and national levels. […] §98 FAO has helped to develop national capacity in many countries, but there has been less effort at taking the opportunity to tap into this increased capability of national staff, to continue to evolve FAO’s role away from direct technical assistance, passing this responsibility to strong NARES (national agricultural research and extension services) and consultants from developing countries, for example. (§5., 76 and 98, Livestock evaluation)

The following recommendations of the evaluation on strategic objective B-1 (International instruments) are illustrative in this regard: “FAO shows a tendency to place a high value on “the lead” without a careful assessment of its capacity to assume this role and, sometimes, at the expense of the less visible role of technical expertise provider, in which it has frequently been very effective. Its main comparative advantages appear to be as provider of impartial expert/technical advice, as analyst/interpreter of particular technical or sectoral elements of the issue, and as enabler of integrated national level implementation. Plans for building capacity for implementation should be prepared, based on systematic needs assessment, taking into account alternative sources of supply. FAO should focus its resources for capacity building on meeting needs for the type of relatively short-term, technical forms of assistance that it can supply well. (Recommendation.4.8) In the development of international instruments, FAO should seek to build on its apparent comparative advantages as provider of impartial expert/technical advice, as analyst/interpreter of particular technical or sectoral elements of the issue, and as enabler of integrated national level implementation. It should only seek the “lead” in development of an international instrument after realistic assessment of the potential short- and long-term implications, advantages and disadvantages and taking into account its position relative to other stakeholders concerning the issue(s) to be addressed by the proposed international instrument. (Recommendation.5.1)

• Improving internal coordination: Evaluations did not consistently provide recommendations for addressing weak coordination both at interdepartmental level and generally between field programmes. The issue was however clearly recognized as needing a “cultural” change. Practical solutions put forward by evaluators ranged from developing a conceptual framework to the provision of incentives and raising awareness. “One of the major elements to promote interdepartmental work would be a strong conceptual framework that could serve as an amalgamating force between the work of the Departments. Even if there were such a framework, it is likely that for formal

51 interdepartmental work to be undertaken, it would require explicit incentives or authority” (Evaluation of FAO-Netherlands Partnership Programme, § 183) “The different lines of reporting (FAORs to OCD and ECs to TCE) can lead to difficulties and tensions at country level with respect to lines of responsibility and information sharing; this was quite evident. However, a better understanding of “emergencies” through training and awareness has been to a degree promoted across FAO as an organization and this has helped the “culture” of the Organization adapt to its increasing responsibilities in support of emergencies.” (Evaluation of FAO's Emergency & Rehabilitation Assistance in the Greater Horn of Africa, §280) “The Programme Management Unit (PMU) is an interesting temporary arrangement to break down the “FAO silo” approach. If successful, the PMU should be terminated. The formation of the technical teams – similar to World Bank set-up – was thought to be a step in that direction. (FAO senior management)” (Evaluation EC/FAO Programme, p.5)

• Adapt TCP modalities to development problems: Evaluations recognized the limitations of the TCP funding modality for addressing long term CD needs. The issue was flagged with regard to the need for technology transfer projects to be designed with their development context in mind if they were to have the desired impact, and this was applicable to TCPs in particular. In general (with some exceptions), TCP design documents were found to be quite cursory, with little description of the context in which the intervention was to take place. As a rule, TCPs with poor scores on clarity of objectives or on design and formulation also had low scores on sustainability. The following recommendations extracted from the Livestock evaluation were generally supported by other evaluations: “[…] the weakness in formulation and design was more significant, with nearly one third of the projects rated 'unsatisfactory' on this count. This was reflected in the recommendations of the field missions, two of which specifically recommended a greater attention to design and preparation in TCPs, and another which recommended carrying out more formal cost-benefit analysis and specific design of follow-up activities for these projects. In line with the Independent TCP Review, the team recommends that this added attention to design should be included as part of start-up activities of the first mission included in the TCPs rather than as a separate design activity, for reasons of economy. […], the team supports the TCP Review's recommendations to strengthen TCP flexibility, simplify categorization of TCP projects, and tie TCP priorities closely to the FAO Strategic Framework and country priority frameworks. (§132-133, Livestock evaluation)

7.4 Effectiveness and impact

• Greater attention to developing Monitoring & Evaluation frameworks: Available literature on CD concludes that measuring the outcomes, let alone impact of CD interventions is quite challenging because of the complexity of CD and the difficulty of predicting outcomes at the outset. Nonetheless, most evaluations were only able to assess achievements up to output level of the results chain and recommended greater care in the design of M&E systems which allow measuring impact in addition to outputs and outcomes. “Improve the monitoring and evaluation system by identifying indicators that enable the programme to measure programme outcomes. For example, outcome indicators could include policy changes that result from decisions that are based on better

52 information provided by the FSIS; allocation of budget resources to geographical areas on the basis of vulnerability status; or line item government resources used to pay for the implementation of introduced tools and maintenance of trained staff. These outcome measures should be agreed upon by in-country stakeholders.” (Evaluation EC/FAO program, p.6) “The evaluation team would strongly recommend that the M&E framework is revisited and that indicators are developed which reflect progress in achieving both results and impact against the overall objective, the outcome and the outputs. These indicators should be relatively easy to obtain, should be both qualitative and quantitative, useful to incorporate into reports, and ones which reflect the satisfaction of SIFSIA-N to beneficiaries and other users of SIFSIA information. They should also reflect the MDGs relevant to food security and incorporate issues of gender and environment which the current matrix does not.” (Mid-Term evaluation of the SIFSIA-North Programme, §5.5). “TPs must be designed so that the causal relationships between interventions and objectives are more clearly manifest and the underlying assumptions for the achievement of objectives stated. The MTP (or supplementary supporting documentation) should contain details about how outputs are expected to be used, what results are expected from that use, which indicators are proposed to determine whether outputs are in fact being used and how those indicators will be measured.” (Fisheries evaluation, §119) “The need for careful outcome monitoring and evaluation of results of TCP-funded pilot projects. Budget restrictions do not permit close monitoring or evaluation of most TCP projects, but in the case of pilot activities, assessments of results and outcomes should be required as part of achieving their objectives, without which the 'pilot' cannot be replicated or lessons learned.” (Livestock Evaluation, §152)

• Increase support to knowledge management and communication in CD projects: FAO is knowledge management organization. Its knowledge must contribute to CD efforts as much as other CD inputs. Projects generate knowledge and knowledge is a resource that requires specific management. Several evaluations recommended that knowledge management should be envisaged as a transversal component in CD projects and to increase visibility of project achievements and lessons learned. … Ce travail [de gestion des connaissances] demandera une équipe qui aura pour fonction de rassembler ce « capital », de le valoriser et de le formater suivant les publics et de le diffuser par différents modes. L’équipe responsable de la gestion des connaissances devra sans doute être ancrée à l’échelon sous-régional. Elle devra collaborer avec les équipes nationales ayant charge d’identifier les expériences méritant d’être partagées et agir en étroite collaboration avec le projet Dimitra et en phase avec le Siège de la FAO, afin de prendre en compte tous les échelons pour une bonne gestion des connaissances. Cette composante devrait aussi favoriser et prendre en charge des voyages d’échange entre OP de la sous-région. » (p.62)

• Coherence and synergy building between FAO initiatives for greater effectiveness Evaluations recommended to increase synergies across FAO CD activities to maximize effectiveness, this was especially the case of small project funds that alone cannot address CD requirements (e.g. TeleFood funds). “TeleFood funds should support clearly identified components of wider FAO projects and programmes, where they can be better targeted and monitored and fully meet the criteria of addressing the needs of the poor. […] In the case of Sierra Leone, the evaluation

53 recommended that TeleFood projects be linked to Farmers’ Field Schools (FFS) and FFS District Networks, in order to have better access to capacity building and advice.” (Sierra Leone evaluation, p.42) “Evidence particularly from Africa suggests that the package of FAO field assistance has the highest profile and is most effective when unified behind a major thrust, e.g. around Farmers’ Field Schools (FFS) in Kenya, or around the Special Programme for Food Security (SPFS) in Burkina Faso. Good integration of emergency and non-emergency activities in support of improved crop production was evident in D.R. Congo and in Ethiopia.” (Crop evaluation, §103)

7.5 Sustainability and partnerships

• Planning of follow up from the start and develop sound hand-over strategies from the beginning to ensure the sustainability of programme activities: Inadequate follow up was often cited as one of the weakest criterion of CD projects. This was an area especially of concern with TCPs which contribute to larger CD initiatives. Evaluations strongly recommended to design follow-up activities to ensure the post-project sustainability of project achievements. In this regard, evaluations underscored the necessity for developing linkages much more carefully at the design stage so as to build synergies and institutional collaboration with other ongoing initiatives at country level. “The weakest criterion was that of sustainability and follow-up. This is an area that is especially a concern with TCP projects which are part of a larger initiative or seek to introduce changes to favour development of a particular group or activity (such as the SPFS diversification component or the small-scale dairy development projects), that they need to develop the foundation for a longer-term engagement. It is linked to the evolution towards more complex TCPs noted in the previous paragraph. This observation leads the evaluation team to recommend, in agreement with the TCP Review, the importance of follow-up activities being planned from the start, and supports the Review's recommendation for the holding of a mandatory end-of-project discussion regarding outcomes and follow-up between FAO, government and relevant stakeholders, to increase sustainability of TCP activities.” (Livestock evaluation, §134) “Develop sound hand-over strategies from the beginning to ensure the sustainability of programme activities. Handover strategies involve a commonly held view by the government and FAO of the timeline, jointly developed criteria for transition, clear objectives and targets, monitoring tools, and robust outcome indicators. Handover strategies also involve periodic and continuous assessment of progress made in various capacity development activities. Successful handover strategies will depend largely on integration and complementarities with existing food security information systems in country and the value-added aspect of programme activities”. (Evaluation EC-FAO FSIA Programme, Rec#11)

• Reaching out to untraditional partners: FAO should expand its cooperation efforts beyond its traditional partners, to include other sectoral ministries and also to the private sector. The choice of the institutional counterpart is especially important in food security projects and can influence positively or negatively on the course of programmes. When the concept of FS was still tied to food availability/crop production, the Ministry of Agriculture have traditionally been (and still continue to be) the

54 preferred institutional counterpart for FAO programmes. This set up runs the risk of other sectors being less involved and projects achieving less impact on FS. In case of FS information projects, this has shown to play to detriment of a functioning info system (e.g. Angola). “The perceived comparative advantage of FAO in supporting policy change was repeatedly underlined to the different regional missions by governments, donors and technical experts. In order to achieve genuine impact in this area, however, FAO (and AGA) must reach beyond its traditional counterparts at the Ministries of Agriculture where possible, and seek the attention of higher, more strategic levels of policy-making such as the Ministries of Planning or Finance.” (Livestock evaluation §138.) “SIFSIA-S has found its institutional counterpart in the Southern Sudan Commission for the Census, Statistics and Evaluation – an appropriate location for a project dealing with a cross-cutting topic such as food security. Through the Steering Committee and Programme Technical Committee, SIFSIA-S liaises effectively with most major actors relevant to food security – with the exception of the Ministry of Finance & Economic Planning.” (Evaluation Sudan SIFSIA South)

• Support business oriented approaches for local CD: On the issue of sustainability of rural development projects, two common constraints were the high levels of illiteracy, and only a short period of project operation to allow capacity of community groups to sustain project-initiated activities without external (e.g. Gov and FAO) support. Recommendations of evaluators included the building of effective partnerships and linkages with local micro-finance institutions and equip project beneficiaries with business skills and plans. “Project follow up need to link up with cooperatives, Farmer Training Centres and local Development Committees, and discuss future approaches. Generally, the project should concentrate on developing low investment, low maintenance packages that can be afforded by rural households, and the input packages should be accompanied by locally adapted business plans that are acceptable for loan applications to micro finance institutions, cooperatives and savings and credit groups. Also, a savings culture among farmers should be encouraged. (Evaluation nutrition and household food security project in Ethiopia, p.17] Rural dvpt projects aimed at improving FSN, there is a need for micro-finance institutions and a more business-oriented approach to develop agriculture (Luapula, para 122)

• Supporting decentralized information systems where capacities are low: start on small scale For CD projects supporting FS information systems in contexts where existing capacity at decentralized levels are very low, evaluations recommended to start activities in selected number of areas in order to demonstrate how to operate a functioning FS information system. “During phase 1 the main focus of SIFSIA-S has been on GOSS institutions; however, in order to build up a functioning food security information system (or at least a partial model that can be replicated), SIFSIA-S needs to develop stronger links with state and even county level actors which are the building blocks. While SIFSIA-S could not possibly cover all 10 states of Southern Sudan, the project will need to build capacity at this level in selected areas in order to demonstrate the elements of a functioning information system that can generate reliable field level data on crops, livestock, and

55 other sources of livelihoods, as well as health/nutrition related data”. (Evaluation Sudan SIFSIA §4.3.3)

• Work increasingly through partnerships (and less on providing direct technical assistance) by promoting alliances especially to increase training performance and dissemination: Considering the significant demand from countries, evaluations consistently recommended to work more through partnership with local service providers, by building alliances and networks and making better use of available capacities to amplify and implement the messages and programmes. This recommendation came up very clearly in the Fisheries evaluation that put forward a number of concrete suggestions, among others: identify institutions with internationally-recognized training capabilities as ‘centres of excellence’; foster twinning processes with training networks of training institutions (incl. South-South cooperation) and integrating the course modules, extension booklets, or other materials (translated where necessary) in the FAO Training (or extension) Series with the involved institutions taking credit for their preparation. “To effectively carry out the technical activities, a much stronger bridge needs to be built up between institutes with the required specialized knowledge and Governments, NGOs, farmers and private sector at the field level. This is an appropriate role for FAO, because of the rapidly evolving need for specialized advice on a much wider range of production problems. Such collaborative arrangements will allow FAO to be a more effective broker and facilitator in addressing the increasing demand of member countries...” (Crop evaluation, §121) “§5. There is a continuing demand from member countries for the delivery of the more traditional approach of technology-centred projects. Given the significant requests in this area and the limited resources in AGA, active consideration could be given to adopting a facilitator role rather than providing technical assistance directly, working through partnerships at the technology-transfer interface at regional and national levels. […] §98 FAO has helped to develop national capacity in many countries, but there has been less effort at taking the opportunity to tap into this increased capability of national staff, to continue to evolve FAO’s role away from direct technical assistance, passing this responsibility to strong NARES (national agricultural research and extension services) and consultants from developing countries, for example. (Livestock evaluation, §5 and 98) “A key strategic advantage of FAO technical sectors (e.g.FII) was found to be their ability to create a network of training institutions to cater for most international training requirements. [In order to strengthen networking performance with training and capacity building institutions, the following suggestions…” (Fisheries evaluation, §88-91)

• Establish and support partnership and cooperation with regional bodies: This need clearly emerged in the realm of international instruments where the development of the linkage between global instruments and national level implementation is a key role for regional action. As highlighted in the evaluation of international instruments, regional bodies have the ability to address any imbalance in focus of these instruments/frameworks (e.g. issues of importance of richer versus poorer countries), and to provide a linkage between international instruments and national implementation. Also, regional bodies may be specially qualified and situated to provide support services, interpretation and implementation tools and other and input of value to international instruments.

56 “Cooperation with regional bodies and instruments, whether created under FAO’s mandate or not, should be an essential element of FAO’s use of international instruments, particularly at the level of national implementation. Such cooperation should also encompass all stages of any new instrument development or implementation. Corresponding support should be given to ensure adequate capacity at regional level, as part of coordinated capacity building plans for each instrument. “ (Evaluation FAO strategic objective B-1, rec# 4.11)

57 8. References

FAO Evaluations. Documents and Reports. Available at http://www.fao.org/pbe/pbee/en/docrep/index.html

Evaluation of FAO Capacity Development Activities in Africa – Background documents: - Terms of reference - Inception Report

Theisohn T. (2007) Towards enhancing the effectiveness of the UN system in supporting capacity development. Background study to the TCPR 2007. Volume I: Main Report. Final Report 10 August 2007. http://www.fao.org/capacitybuilding/pdf/FINAL_CD_report.pdf

Balogun P. (2007) Meta-analysis of country-level evaluations — Background analysis for the Triennial Comprehensive Policy Review of operational activities for development of the United Nations System. http://unuony.hypermart.net/middayforum/February%2020/docs/META%20ANALYSIS%20 REPORT%20V3%2020070225-Bacj.pdf

FAO corporate strategy for capacity building – DRAFT 16 April 2009

58 Annex 1: List of evaluation reports

Title of evaluation report Year Evaluations of projects and programmes Rapport de l’Evaluation à mi-parcours du Programme EMPRES composante Criquet 2009 pèlerin en Région occidentale EMPRES/RO (Desert Locust component) Evaluation finale independante: Programmes Spéciaux de Sécurité Alimentaire au 2009 Burkina Faso, Mali, Niger, Tchad et Soudan Financement Libyen (GCP /BKF/042/LIB, MLI/024, NER/040, CHD/026 et SUD/051) - MALI - Mai 2009 Evaluation finale independante: Programmes Spéciaux de Sécurité Alimentaire au 2009 Burkina Faso, Mali, Niger, Tchad et Soudan Financement Libyen (GCP /BKF/042/LIB, MLI/024, NER/040, CHD/026 et SUD/051) - BURKINA FASO - Mai 2009 Independent Evaluation of FAO-Netherlands Partnership Programme (Phase II) 2007 2004-2007 - Final Report Evaluation of FAO’s Emergency & Rehabilitation Assistance in the Greater Horn 2007 of Africa 2004-2007. Final evaluation report 25 October 2007 Support to Food Security Information Systems in Ethiopia (SFSISE) Mid-Term 2007 evaluation Mozambique: National Programme for Food Security (PAN II) 2007 GTFS/MOZ/076/ITA – Final Report. Final evaluation of the FAO/DFID "Livelihood Support Programme " 2007 GCP/INT/803/UK Formal name: "Improving Support for Enhancing Livelihoods of The Rural Poor" Final Evaluation of the Sustainable Fisheries Livelihoods Programme (SFLP) 2007 GCP/INT/735/UK Final Report 2 March 2007 Luapula Food Security, Nutrition Action and Communication Project, Zambia. 2006 (GCP/ZAM/059/BEL) Tripartite Evaluation Mission, Final report Aug 2006 Evaluation of Improving Nutrition and Household Food Security In Northern Shoa 2005 and Southern Zone of Tigray, Ethiopia (GCP/ETH/060/BEL) Final Report. Evaluation tripartite de l’axe thématique « Approche participative et gestion de 2007 terroirs » (APGT) du programme de coopération FAO-Belgique pour la période 2004-2007 Joint Evaluation of the Project: Technical Support to the Ministry of Agriculture 2005 (Eritrea), Food/2002/003-041 (GCP/ERI/008/EC) External Evaluation Mission - Support to the Food Security Department (Phase II) 2006 GSA/MINADER Angola - Project GCPS/ANG/027/EC - Final Report, April 2006 Projet d’appui à la fixation des jeunes dans leurs terroirs (PAFJT) BKF/98/006 (juin 2006 2000 - juin 2005) Rapport d’évaluation finale Burkina Faso: Projet « Appui aux organisations paysannes de la province de la 2006 Kompienga dans le cadre de la sécurité alimentaire et de la décentralisation » (GCP/BKF/041/BEL FAO/BEL/BKF) Projet d'appui à la définition des politiques de développement agricole en 2007 République Démocratique du Congo - GCP/DRC/031/BEL - Evaluation en cours de réalisation -Rapport Version finale Information products for Nile basin water resources management (project 2009 GCP/INT/945/ITA) Terminal evaluation report Rapport d’évaluation de la deuxième phase du projet DIMITRA (2002-2004) 2005 Sudan Institutional Capacity Programme: Food Security Information for Action 2009

59 (SIFSIA) - OSRO/SUD/620/MUL mid-term evaluation of the SIFSIA programme June 2009 EC/FAO Joint Evaluation: Food Security Information for Action Programme - 2009 GCP/GLO/162/EC -Final independent evaluation, April 2009 Corporate evaluations Independent Evaluation of FAO Corporate Strategic Objective B-1: “International 2009 instruments concerning food, agriculture, fisheries and forestry, and the production, safe use and fair exchange of agricultural, fishery and forestry goods” (Final report, Jan. 2009) Evaluation of FAO Activities in Crop Production - Sept 2003 2003 Evaluation of FAO Activities in Fisheries Exploitation and Utilization Programme 2004 2.3.3 - May 2004 Evaluation of Livestock Production, Policy and Information (Programme 2.1.3) – 2005 May 2005 Independent Evaluation of FAO’s Role and Work in Statistics –Oct 2008 2008 TCP Policy and operational framework of the Technical Cooperation Programme: 2004 Responding to a changing environment - Oct 2004 Country evaluations Evaluation of FAO Cooperation in Mozambique 2001 - 2005 (June 2006) 2006 Evaluation de la Coopération de la FAO en RDC (2003 – 2007) Rapport Final - Juin 2008 2008 Sierra Leone: Evaluation of FAO Cooperation In Sierra Leone 2001– 2006, Final 2007 Report Volume I: Main Report, Rome April 2007

60 Annex 2: Grid template used to systematize evaluative evidence

Sheet 1: DESCRIPTIVE OVERVIEW QUESTIONS CD lens of evaluation 1.A. To what extent has the evaluation assessed CD concept and practice of FAO’s activities? CD CONTENT CD content of project 1.B. How important is the level of CD content in FAO's activities? (e.g. CD explicitly formulated as a prog objective and translated into expected outcomes) Software CD 2.A. To what extent FAO's activities ACTIVITY Hardware address the different modalities of CD? support Financial Support Individuals and groups 2.B. To what extent FAO's activities CD DIMENSION Organization address the different dimensions of CD? Enabling Environment Government SECTOR Civil Society 3. Which sector is targeted for CD TARGETED intervention? Private Sector policy/normative dvpt

CD knowledge 4. To what extent software activities FUNCTIONAL address the different functional capacities outreach and partnering CAPACITY of CD? implementation

Technical Assistance Training 5. To what extent have software activities Twinning Org/Networks addressed both technical capacity soft Distance/E-Learning SOFTWARE skills AND the enabling environment Knowledge/Info Sharing, TYPE support (policy and org/institutional aspects) the materials different organizational or functional Pay/incentives support capacities of the enabling environment? policy/legal FW Institutional support analytical/normative 6. What role has FAO primarily played in functions and related the CD support? knowledge management - Normative and analytical functions that FAO role and and advocacy are required to manage global public comparative convener for multi- goods. Sound standards, high quality advantage in stakeholder engagement & analysis, reliable information, globally support of CD broker for partnerships brokered agreements, global advocacy and operational roles (TC, access to relevant knowledge training, provision of - Convener & broker for partnerships to find collective solutions to challenges

61 experts, etc) related to global public goods. Instrumental role in promoting CD as an endogenous, stakeholder-driven process. - Operational: understood as technical cooperation, assistance, provision of training, experts, etc.

HEADINGS CORE QUESTIONS OF THE Sheet 2: CD evaluation META-EVALUATION 1.1 country capacity needs identified + integrated in prodoc 1.2 prog builds on FAO RELEVANCE comparative advantage 5. How well has FAO’s 1.3 coordination with other CD support to CD performed in initiatives contributing to the 2.1 needs at ind/org/policy levels achievement of prog taken into account goals/objectives? 2.2 responsiveness to demand/needs >> Issues/Questions are those 2.3 particip identif of CD needs formulated in Annex 2 of 2.4 promotion of ownership EVALUATION MATRIX DESIGN OF CD 2.5 objectives clearly defined in (Inception report) INTERVENTION outcomes 2.6 major goals (FS, pov, NRM) >> Rating: A = good. B = reflected satisfactory. C = unsatisfactory. 2.7 cost-effect/sust considered D = poor 2.8 gender 2.9 M&E framework 3.1 in-house capacity for CD delivery throughout FAO technical units 3.2 inter-dept coordination at HQ EFFICIENCY 3.3 coordination (timeliness, degree) in the management of projects and activities between HQs and decentralized offices 4.1 CD materials quality/relevance 4.2 FAO Normative products awareness/use/utility 4.3 FAO facilitating techn/policy initiatives EFFECTIVE- 4.4 FAO providing techn assistance NESS and training 4.5 FAO providing CD support (ownership + follow up of CD interv) 4.6 FAO partnering on CD activities

62 4.7 FAO improving CD technologies 5.1 impact of CD activ at ind-org- IMPACT policy levels 6.1 enhance client capacities 6.2 enhanced local leadership 6.3 upscaling of CD activ by SUSTAIN- trained people ABILITY 6.4 social capital created 6.5 changes in policies/implem 6.6 spill over effect PARTNER- 7.1 coop within UN SHIPS 7.2 coop with all actors