Evaluation of FAO's Cooperation in Somalia 2007 to 2012

Food and Agriculture Organization of the United Nations Office of Evaluation

Evaluation of FAO’s Cooperation in Somalia 2007 to 2012

Final Report

May 2013

Food and Agriculture Organization of the United Nations

Office of Evaluation (OED)

This report is available in electronic format at: http://www.fao.org/evaluation

The designations employed and the presentation of material in this information product do not imply the expression of any opinion whatsoever on the part of the Food and Agriculture Organization of the United Nations (FAO) concerning the legal or development status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The mention of specific companies or products of manufacturers, whether or not these have been patented, does not imply that these have been endorsed or recommended by FAO in preference to others of a similar nature that are not mentioned.

The views expressed in this information product are those of the author(s) and do not necessarily reflect the views or policies of FAO.

FAO encourages the use, reproduction and dissemination of material in this information product. Except where otherwise indicated, material may be copied, downloaded and printed for private study, research and teaching purposes, or for use in non-commercial products or services, provided that appropriate acknowledgement of FAO as the source and copyright holder is given and that FAO’s endorsement of users’ views, products or services is not implied in any way.

All requests for translation and adaptation rights, and for resale and other commercial use rights should be made via www.fao.org/contact-us/licence-request or addressed to [email protected].

For further information on this report, please contact:

Director, OED Viale delle Terme di Caracalla 1, 00153 Rome, Italy Email: [email protected]

Table of Contents

1. Introduction ...... 1 1.1 Background to the evaluation ...... 1 1.2 Scope, approach, methodology & constraints ...... 1 1.2.1 Scope ...... 1 1.2.2 Timing and team ...... 2 1.2.3 Approach and methods ...... 2 1.2.4 Constraints...... 4 1.3 Structure of the report ...... 5 2. Context ...... 7 2.1 Political context...... 7 2.2 Livelihoods and the economy ...... 8 2.3 Humanitarian crisis ...... 10 2.4 The context of international engagement ...... 11 3. An overview of the FAO Somalia programme ...... 13 4. Relevance of the FAO Somalia programme ...... 15 4.1 To the context and needs ...... 15 4.1.1 Issues ...... 15 4.1.2 In the context of failed formal governance ...... 15 4.1.3 Relevance of the livestock, agriculture, and fisheries programmes ...... 16 4.1.4 Relevance to the context of protracted humanitarian crisis ...... 18 4.1.5 Flexibility to respond to the changing context ...... 19 4.1.6 Areas that deserve greater attention: urbanisation, natural resource management and nutrition ...... 19 4.2 To government strategy and priorities ...... 20 4.3 To UN programmes and priorities ...... 22 4.4 To FAO’s corporate framework ...... 23 5. Programme design and effectiveness ...... 25 5.1 Programme design and implementation to achieve results ...... 25 5.1.1 Outline ...... 25 5.1.2 Consultation with stakeholders ...... 25 5.1.3 Issues re data and analysis that underpin programme design and implementation ...... 27 5.1.4 Evidence of project logic and the use of logical frameworks ...... 28 5.1.5 Use of normative products ...... 29 5.1.6 Achieving results...... 30 5.1.7 Gaps that once addressed could enhance effectiveness ...... 31 5.2 Effectiveness in responding to humanitarian crises, with a particular focus on the famine of 2011 ...... 32 5.3 Coordination...... 36 5.4 Twin-track approach ...... 41 5.5 Advocacy and influencing ...... 42 6. Effectiveness of FAO’s management and operational systems...... 44 6.1 Leadership ...... 44 6.2 Managing a multi-nodal programme from Nairobi...... 45 6.3 From project to programme approach ...... 45 6.4 Scaling up and scaling down ...... 46 6.5 Human resources ...... 47

iii 6.6 Remote management, monitoring and evaluation...... 48 6.7 Communications management ...... 53 6.8 Decentralisation from FAO HQ to Nairobi ...... 53 6.9 Re-location of FAO from Nairobi to Somalia ...... 54 6.10 Technical support and knowledge exchange ...... 55 7. Efficiency ...... 58 7.1 Efficiency of FAO procedures, administration and structures ...... 58 7.2 Cost effectiveness ...... 59 7.3 Resource mobilisation ...... 61 8. Connectedness ...... 62 8.1 Capacity-development ...... 62 8.2 Sustainability of FAO interventions ...... 64 8.3 Resilience paradigm and strategy ...... 65 9. Cross-cutting issues ...... 69 9.1 Gender equality and gender sensitivity ...... 69 9.2 Partnerships and modes of implementation ...... 71 9.2.1 Strategic partnerships...... 71 9.2.2 Changing mode of implementation ...... 72 9.2.3 Implementing partnerships ...... 72 9.2.4 Public-private partnerships ...... 74 10. Conclusions and recommendations ...... 76 10.1 Conclusions ...... 76 10.2 Recommendations ...... 79

Annex 1: TOR Annex 2: Evaluation matrix Annex 3: Evaluation programme/ itinerary Annex 4: Evaluation team members Annex 5: List of people met Annex 6: Country Projects implemented by FAO Somalia between 2007 and 2012 Annex 7: Evaluation team’s analysis of the FAO Somalia Resilience Strategy, and of the proposed shift in paradigm and implementation Annex 8: Assessment of Central Emergency Response Fund Projects Annex 9: Assessment of the World Bank funded project “Drought Management and Livelihood Protection” (OSRO/SOM/112-122/WBK) Annex 10: A study of FAO Somalia’s Cash for Work programme

Acknowledgements

The evaluation team would like to thank FAO staff in the Somalia country office, at headquarters, and in the sub-regional office in Addis Ababa for participating in the evaluation, for answering our many questions and for providing information and documentation. Credit should go to the FAO Somalia country office for requesting this evaluation and for their openness in discussing key issues. The inputs of many other stakeholders, including government officers, donor agency officials, NGO staff, and staff of other UN agencies were critical to the team’s work and are much appreciated. We would like to acknowledge the insightful inputs of the communities and individuals interviewed for the CFW study in Gedo, and the hard work and dedication of the research team who carried out the study, led by Savina Tessitore. Some interviewees travelled long distances to meet with us where we were unable to meet them in situ. This included members of Farmers’ Associations, staff of NGOs and members of the Livestock Professional Associations. The efforts they made and their inputs and views are highly appreciated.

Excellent logistical and administrative support was provided to the team by Joy Mulema in Nairobi and by Nadine Monnichon in Rome. Special thanks to Aurelie Larmoyer and Lori Bell for their supportive and efficient management of the evaluation.

Nigel Nicholson and Mark Bradbury played a much-valued peer review role throughout the evaluation process, providing constructive and insightful inputs and feedback. We would like to thank the many people and organisations who took time to provide feedback on an earlier draft. All of those comments have been considered in this final version.

Composition of the Evaluation Team

Evaluation Team

Margie Buchanan-Smith, Team Leader Tim Leyland, animal production and health expert Yacob Aklilu, trade and market, private sector development expert Mohamood Noor, crop production expert Sadia Ahmed, gender equality, community-based development, partnerships expert Shukria Dini, gender equality, community-based development, partnerships expert Ian Robinson, crop production expert

FAO Office of Evaluation

Aurelie Larmoyer, team member and evaluation manager Savina Tessitore, team member and Cash-for-Work study coordinator Genny Bonomi, team member and evaluation analyst

v Acronyms

AMISOM African Union Mission in Somalia AU-IBAR African Union’s Interafrican Bureau of Animal Resources CAP Consolidated Appeal Process CBRWG Cash-Based Response Working Group CCPP Contagious caprine pleuro pneumonia CFW Cash for Work CHF Common Humanitarian Fund CISS Coordination of International Support to Somalia CMU Crisis Management Unit DFID Department for International Development (UK government) DoA Delegation of Authority EC European Commission EEZ Exclusive Economic Zone FAO Food and Agriculture Organisation of the UN FAOR FAO Representative FFS Farmer Field Schools FM field monitors FMT form management tool FSC food security cluster FSNAU Food Security and Nutrition Analysis Unit HQ headquarters HR human resources IAP Interim Assistance Plan IASC Inter-Agency Standing Committee ICU Islamic Courts Union IDDRSI Strategic Plan for Drought Disaster Resilience and Sustainability Initiative (IGAD) IGAD Intergovernmental Authority on Development IP implementing partner IPC Integrated Phase Classification system LEGS Livestock Emergency Guidelines and Standards LOA Letter of Agreement LPA livestock professional association MACC Monitoring Activities Call Centre MCIA Meat Control and Inspection Act M&E monitoring and evaluation MOU memorandum of understanding NDP National Development Plan NGO non-governmental organisation OECD-DAC Organisation for Economic Co-operation and Development – Development Assistance Committee OED FAO Office of Evaluation OiC Officer in Charge OIC Organisation of Islamic Cooperation OIG FAO Office of the Inspector General PET pictorial evaluation tool PPP public-private partnerships PPR peste des petits ruminants

vi PSNP Productive safety-net programme (Ethiopia) REOA FAO’s Emergency Regional office for Africa RTE real-time evaluation SACB Somalia Aid Coordination Body SAHSP Somali Animal Health Services Project SEAGA Socio-Economic and Gender Analysis SERECU Somali Ecosystem Rinderpest Eradication Coordination Unit SHEFA Shebelle Fruit Association SRO FAO’s Sub-Regional Office (Addis Ababa) SWALIM Somali Water and Land Information Management TCE Emergency Operations & Rehabilitation Division (FAO, Rome) TFG Transitional Federal Government TNG Transitional National Government TOR Terms of Reference UCT unconditional cash transfers UN United Nations UNCT United Nations Country Team UNEG United Nations Evaluation Group UNSAS United Nations Somali Assistance Strategy

vii

Map of Somalia

Source: http://www.un.org/depts/Cartographic/map/profile/somalia.pdf

viii

Executive Summary

Introduction

ES1. This evaluation of FAO’s work in Somalia covers the period 2007 to 2012 when the programme grew rapidly from expenditure p.a. of $14.2 million to $51.2 million, becoming FAO’s largest country programme, more complex and higher profile than most others. The evaluation was carried out between September 2012 and January 2013 by a ten-person team; it serves both a learning function for FAO and an accountability function for donors and the respective government authorities. The evaluation team deployed a range of evaluation methods and also included an in-depth study into FAO’s Cash-for-Work (CFW) programme in parts of Gedo region using a range of qualitative participatory data collection methods at community level. The major constraint faced by the evaluation was lack of access to many parts of Somalia. Where possible, the team found ways around this, but it severely limited access to local communities and beneficiaries.

Context

ES2. Somalia is the most enduring case of state collapse in recent times, during which internal conflict has prevailed across most of southern and central Somalia, associated with a rise in jihadist insurgency, currently spearheaded by Al Shabaab. Some of the most violent conflict in Somalia’s history took place between 2006 and 2010, directly contributing to the famine in 2011. Regional and global political agendas have had considerable influence in Somalia in the past eight years; the international community has attempted to play a key role in state-building and ‘stabilisation’. There have been high levels of political instability during the period that this evaluation covers. The appointment of a new government in September/ October 2012 was met with greater optimism than has been felt for many years, although the situation remains volatile. Somaliland and Puntland have followed a different political trajectory, declaring independence in the case of the former in 1991 and a semi-autonomous region in the case of the latter in 1998. With more established government and administrative structures, both have enjoyed greater stability but have been in political limbo as their future status remains unresolved.

ES3. The majority of Somalia’s population depend upon pastoralism and agro-pastoralism for its livelihood, but the lack of effective governance, law and order have taken their toll, especially in southern and central Somalia. It is now one of the poorest and most food insecure countries in the world. High levels of displacement have fuelled urbanisation, but a deeper understanding of how livelihoods and settlement patterns have changed in Somalia is hampered by a lack of trend data and analysis.

ES4. Somalia has experienced humanitarian crises over two decades, mainly triggered by conflict and lack of effective governance but compounded by recurrent droughts, floods and in recent years, rising global food prices. The most acute periods of humanitarian crisis have attracted greatest international attention, including the 2011 famine, although food security and other indicators frequently hit emergency levels in the periods between.

Overview of FAO Somalia programme

ES5. FAO in Somalia has been working in a context of weak or absent formal governance in which international engagement has prioritised political stabilisation objectives, especially since 9/11. FAO has engaged in both development and emergency interventions, often simultaneously, although both have been implemented under emergency management arrangements. Insecurity and lack of humanitarian access have meant reliance on remote management in large parts of the country with its associated challenges.

ES6. During the period 2007 to 2012, FAO’s expenditure totalled $211.8 million with almost 90 country-specific projects. One-third of expenditure was on crop sector and one- fifth on livestock sector. The crop sector has long been the focus of FAO’s work in Somalia, although the livestock programme has expanded in recent years. FAO projects have also focused on the private sector and trade, on the production and dissemination of information as a public good, on fisheries and on coordination. Since the famine of 2011, CFW interventions have been massively scaled up, representing over 30% of FAO’s total expenditure in 2011 and 2012.

Relevance of the FAO Somalia programme

ES7. FAO’s programme in Somalia has been highly relevant to the context of failed formal governance, for example providing public information services on food security (FSNAU) and natural resources (SWALIM), that would normally be associated with central government, supporting the livestock and meat industry by attempting to get the trade ban lifted on the export of live animals from Somalia, supporting the formulation of legal, regulatory and institutional frameworks for the domestic meat industry, and supporting the private sector to provide key inputs for crop and livestock producers. In these ways, it has stepped into the breach of lack of central government. Playing these roles, however, also carries a certain responsibility and there have been gaps, for example, in terms of long-term trend analysis by FSNAU. Overall, FAO’s interventions in the livestock, agriculture1 and fisheries sectors have been relevant to the context and needs although the evaluation questions whether they could have been even more relevant if there had been better data and analysis indicating the outcome and impact of different interventions (see below).

ES8. Work in the fisheries sector was slow to start in view of the fishing potential off Somalia’s coastline although there are plans to expand this area of work in the coming years.

ES9. The evaluation identifies three significant gaps in FAO’s work in Somalia: (i) on peri-urban agriculture as the process of urbanisation has accelerated; (ii) investment in natural resource management, following through on SWALIM’s mapping and analysis; (iii) as custodian of much nutrition data, playing a leadership role in addressing chronic under-nutrition through agricultural interventions.

ES10. Where government strategies exist, FAO’s programme appears to have been closely aligned. For example, FAO’s Interim Assistance Plans for Somaliland and Puntland in 2012

1 The term agriculture has been used throughout the report as synonyms for crop production.

x outline its plans and priorities, following the strategies of the respective authorities. FAO’s work in the livestock and crop sectors has mostly been welcomed by the different government authorities with a few exceptions. However, the CFW programme has been developed independently of government, an important gap that needs to be filled. While SWALIM has strong links with the respective authorities of Somaliland and Puntland, FSNAU has been more oriented to serving the needs of the international community, which it does well, especially informing the strategy for the Consolidated Appeal Process (CAP) each year. However, FSNAU’s aggregated analysis is less well-suited to meeting the needs of the respective government authorities. FAO’s programme has been broadly in line with key UN strategy and planning documents, including the UNSAS for 2011 to 2015 which stresses the need to address underlying structural causes of poverty and food insecurity.

Programme design and effectiveness

ES11. FAO projects have a mixed record in terms of consultation with stakeholders, especially in the design phase. There are good practice examples such as the stakeholder consultation that fed into the regulatory support provided to the domestic meat industry. Performance has been weaker in terms of consultation at the community level, especially where access has been possible, with a few exceptions such as the fisheries sector in Puntland. FAO’s information projects have been slow in carrying out formal surveys of their users’ needs despite providing a public information service.

ES12. Lack of outcome and impact monitoring across the FAO Somalia programme have hampered both design and implementation, and have meant a lack of rigorous analysis underpinning FAO’s programming choices and approaches. In the livestock sector, 80% of FAO’s expenditure has been on animal health, but almost all monitoring has been at the output level and there is little evidence that these interventions are addressing underlying causes of vulnerability. Indeed, livestock interventions may have had a comparatively larger positive impact on wealthier livestock-owning households. More needs to be known about the causes of vulnerability and about other trends e.g. trends in livestock ownership. In terms of livestock and meat marketing, the evaluation questions the focus on value addition at the consumption end of the chain; instead, greater emphasis should have been given to the production end of the chain where greater value can be added in producing better quality animals. Although remote management has hampered FAO’s ability to do outcome and impact monitoring, there are participatory appraisal and impact assessment techniques that it could have used, as demonstrated by the CFW study carried out for this evaluation.

ES13. FAO Somalia’s use of normative products and guidance to promote effective programming is mixed. There are positive examples in terms of food security information systems and some glaring omissions, for example the use of LEGS in the livestock sector.

ES14. Some of FAO’s projects have achieved impressive results, for instance the seed project that created two privately-owned and managed commercial seed supply companies which now supply farmers and also international agencies for their emergency seed distribution programmes, and some of the fisheries interventions. Some projects in the livestock and meat markets have achieved good results; where there have been setbacks these have related to finding appropriate ways of managing livestock markets. Slaughterhouse construction projects have been some of the poorest-performing, hampered by serious design faults and failure to put in place functioning management structures.

xi ES15. In terms of the 2011 famine, the FSNAU played a critical role in providing early warning of deteriorating food security from 2010, eventually alerting the world to famine in Somalia. Unfortunately, the early warning did not elicit a timely response and there were many excess deaths. Despite the considerable obstacles to a timely response being launched, there could have been much more effective advocacy by FAO Somalia and at the corporate level to draw attention to the seriousness of the situation. In addition, despite hosting FSNAU, FAO Somalia was as late as all other international agencies in responding, i.e. only when famine had been declared at the end of July 2011. It then scaled up fast, especially the CFW programme. The CFW study in Gedo region indicates that income generated through CFW had a positive impact on the food security of entire communities, directly through targeted households and indirectly through redistribution within the community. Its effectiveness in enhancing agricultural productivity was more complex to assess because of the range of factors affecting productivity, but there is evidence that the wealthiest farm- owners tended to capture the greatest proportion of benefits. A broader range of interventions may have been needed. Both the livestock and crop-based emergency interventions were scaled up rapidly and appear to have been effective although there were serious delays with some emergency interventions: in particular an emergency fodder scheme and an emergency fisheries project.

ES16. The merging of the two coordination clusters in March 2012 (agriculture and livelihoods led by FAO and food assistance led by WFP) into a single Food Security Cluster has been welcomed in terms of greater coherence of food security interventions, although the very large size of the cluster is a challenge to manage. Good preparation and effective collaboration between FAO and WFP both contributed to the success of the merger. Performance of the cluster tends to be rated higher by its members at national level in Nairobi than at district level within Somalia. The FSC is most valued for its information-sharing role, although its role (still being developed) in terms of technical support and guidance, and capacity development is much in demand. The UN-centric nature of the cluster system is an ongoing challenge to be managed; NGOs tend to feel little sense of ownership of the coordination mechanism and have organised alternative fora outside the cluster system when pursuing new initiatives, for example the Somalia Cash Consortium.

ES17. FAO Somalia has generally performed well in pursuing a ‘twin-track approach’, carrying out development and emergency programming alongside each other, most evident in the agriculture sector where it has adapted objectives of the CFW programme from rehabilitation of infrastructure to relieving food security during the 2011 crisis. A more systemic approach in the livestock sector, addressing issues such as feed and water as well as animal health, would strengthen the twin-track approach.

ES18. In terms of advocacy, FAO Somalia has played an effective role influencing the development and formulation of key government policies and strategies, especially in meat marketing but also in fisheries and animal health. The growing scale and profile of its work have given it greater leverage within the international community where it has been influential in debates on resilience. The area that deserves attention is ensuring that stronger advocacy accompanies FSNAU’s technical early warning.

Effectiveness of FAO’s management and operational systems

ES19. Effective leadership has been key to the growth, innovation and effectiveness of FAO in Somalia, although the fast-paced expansion has sometimes been at the expense of

xii monitoring of, and learning from the programme on the ground. Managing succession is an urgent challenge for FAO at the corporate level, to maintain the effectiveness of this high- profile country programme and to maintain donor support during a period of transition and re-location to Mogadishu.

ES20. One example of FAO Somalia’s leadership and innovation is in replacing FAO’s normal projectised approach with a programme approach based on pooled funding, one programme proposal and one programme report. The efficiencies of this shift are widely acknowledged although there are challenges in the complexity of the financial management system still to be addressed. Another example of FAO Somalia taking initiative is its proactive efforts in communications management, rarely given much attention by FAO country offices. It has drawn up a communications strategy for improving outreach to stakeholders ranging from donors to potential beneficiaries.

ES21. FAO Somalia managed well the programme surge in 2011, setting up a Crisis Management Unit. As FAO institutionally does not have a strong track record in this respect, FAO Somalia can provide good practice experience and learning. The challenge in 2013 is to ‘scale down’ as the emergency subsides and funding levels decrease.

ES22. The negative findings of an FAO audit in 2010 on human resource practices by FAO Somalia triggered greatly improved HR management. Nevertheless, there are still shortcomings including: long delays in staff recruitment, both for international and field- based positions; the limitations of FAO’s corporate HR policy which gives FAO Somalia little flexibility to hire staff for key positions, except as short-term professional staff or consultants; lack of induction processes; and poor communication between Nairobi and field offices. FAO Somalia must urgently strengthen its HR management capacity as it prepares to re-locate into Somalia.

ES23. FAO Somalia’s structural weaknesses in M&E became apparent with the scale-up in 2011. New methods and technologies for M&E were rapidly introduced, mostly geared to producing quantitative output-focused data. High levels of financial risk associated with remote management and a rapidly scaled-up programme placed the emphasis on compliance (although there have still been cases of diversion of funds, usually investigated quite rapidly). However, monitoring and assessing outcomes and impact, key to learning about whether an intervention was effective and the right choice, have been overlooked and are very weak. There has been a lack of strategic vision behind the M&E system and unclear roles and divisions between the M&E unit and the sectors.

ES24. Over the past year, the process of decentralization of responsibilities and authorities from FAO HQ to the country office has proceeded smoothly, in many ways formalizing the existing autonomy of the FAO Somalia country office. It was well-supported by the relocation of three HQ-based staff. In principle, FAO Somalia should now call on the Sub- Regional Office (SRO) in Addis Ababa as the first port of call for technical backstopping. However, this office has neither the resources nor the expertise to fulfill this role effectively; indeed, the country office has stronger technical expertise than the SRO in some areas and in practice most technical support where required has come from HQ. As decentralization proceeds, ways of ensuring that technical support is available, probably through a broader virtual technical network, will need to be established. The evaluation also highlights serious gaps in knowledge exchange, even within the Horn of Africa, that must be urgently addressed. Technically strong, it is important that FAO Somalia is sufficiently connected into

xiii the rest of the organization to be able to contribute to the accumulation and dissemination of good practices and lessons learned.

ES25. For many years, FAO Somalia has been managing a geographically disparate programme from Nairobi. It has managed some aspects of this well, including the monitoring of outputs and delivery. Others have been more challenging. Key stakeholders in Somalia, including government officials, have resented the Nairobi-based centre of gravity and FAO’s slowness (compared with other UN agencies) in locating international staff inside Somalia, even to the more stable north. Some of FAO’s field staff have also felt disincentivised and cut off from decision-making in Nairobi. Recent decisions to move some of FAO’s operational functions to Mogadishu and to strengthen the offices in Hargeisa and Garowe provide an opportunity to redress this. Nevertheless it is a risky move, especially relocation to Mogadishu, and is likely to be a turbulent time for the programme. This must be taken into account in the pace of programming during this period of transition which must be well- managed to maintain staff morale.

Efficiency

ES26. Since the 2010 audit, FAO Somalia has standardised and tightened its procedures - an appropriate response to remote management and to the growth of the programme that has helped build trust with its donors. In some cases, this has slowed delivery and decision- making and there are instances where greater flexibility would ensure that time-sensitive actions are not delayed.

ES27. Measuring cost-effectiveness was hampered by the way that financial expenditure data are aggregated. Staff costs appear to have fallen as a proportion of overall expenditure but the implications of this are unclear. At the individual project level, performance varies between projects completed in time and on-budget (rehabilitation of the livestock market in Hargeisa), and those which failed to meet their overall objectives (construction of the abattoirs in Boroma and Bosasso).

ES28. During the 2007 to 2012 period, FAO Somalia has had a highly successful resource mobilisation strategy, dramatically increasing its financial resources and diversifying its donor base. Importantly, during this period, it has evolved from following donor priorities to engaging with donors as a more equal partner and with a more strategic approach.

Connectedness

ES29. FAO Somalia has engaged in a wide range of capacity-development initiatives. SWALIM’s approach to capacity development in Somaliland and Puntland has been one of the more effective at government level. It offers valuable experience and learning, focused at the institutional level rather than the individual level which has been FSNAU’s approach to date and is vulnerable to staff turnover. FAO Somalia’s investment in Farmer Field Schools at community level has been well-received as a form of extension. Most capacity development efforts have been connected to individual projects and project goals. As FAO Somalia shifts to a programme approach and as opportunities open up for supporting the new government in Mogadishu, the country office needs to develop an overall approach to capacity development, guided by a conceptual framework that captures the individual, the organisation and the enabling environment.

xiv ES30. In the vacuum of formal governance, FAO Somalia has successfully created a number of sustainable models and institutions, especially for the provision of services and inputs to the crop and livestock sectors, for example the two seed companies that now operate entirely independently, and through support to the Livestock Professional Associations which represent a sustainable PPP model for the provision of animal health. In some trade projects, however, lack of market analysis and a clear business case puts their long-term sustainability in jeopardy.

ES31. FAO Somalia’s tripartite resilience strategy with UNICEF and WFP represents an important initiative to build connectedness into the programme, challenging the bifurcated aid architecture and attempting to address underlying structural causes of poverty and vulnerability. The strategy is focused at the household and community level. The evaluation urges a more systemic approach that incorporates issues of governance. Drawing on how the concept has been applied in ecology, it should also address connections across systems (for example livelihood systems with agro-ecological zones and social systems), with a long-term perspective. Certain knowledge gaps need to be plugged to inform the strategy: (1) how Somali society has transformed and adapted over the last decade and (2) learning from earlier efforts with similar objectives, for example ‘linking relief and development’. There is a gap between FAO Somalia’s vision and the current reality of what it is doing. Community-based development is not the organisation’s natural comparative advantage. It must therefore carry out an analysis of its current resources and skill-sets to see how these must be strengthened to implement the strategy.

Cross-cutting issues

ES32. Conflict and social upheaval in the last two decades have triggered shifting gender roles in Somalia. Many women have become the main breadwinner in the household yet do not have equal rights to men, for example with respect to land entitlements. These realities do not appear to be recognised by FAO. Despite early investment during the period of review in engendering the FAO Somalia programme, in 2007/08, this has not been followed through. By 2012, there was little evidence that the programme has been gender sensitive. Very little data is sex disaggregated: FSNAU is the exception with the only dedicated gender resource in the FAO country office. Using the gender marker, the evaluation team scored FAO Somalia as Gender Code ‘1’ meaning that gender dimensions are meaningfully included in only one or two of the three essential components: needs assessment, activities and outcome. FAO Somalia does not meet FAO’s corporate commitment to gender equity and there has been no accountability for this failing.

ES33. In terms of partnership, FAO Somalia has developed a number of significant and positive strategic partnerships, at the overall programme level and at the sector level. In terms of implementing partnerships, FAO depends upon a wide range of NGOs, especially for programming in large parts of south-central Somalia. These arrangements are principally contractual for service provision, appropriate where scale-up has been rapid and new NGOs have been engaged at short notice. These contractual relationships are less appropriate in development programming where a longer-term and more equal partnership should be the aim, thus attracting stronger INGOs that could otherwise be put-off by FAO’s dominant contractual approach. A more nuanced approach to partnership is needed, with clearer terminology. FAO Somalia has a mixed record of public-private partnerships. There are positive examples from the agriculture sector (the seed companies), from the fisheries sector (landing sites in Puntland), and from LPAs in the livestock sector, all of which offer valuable

xv learning about how and why these worked. PPPs for management of slaughterhouses have been much less successful, also offering valuable learning from their failings. A key learning is that building a sustainable PPP takes time and the length of that process may be at odds with tight project deadlines.

ES34. Overall, the evaluation rated FAO Somalia’s portfolio of programming as relevant to the context. Many parts of the programme appear to have been effective; where projects failed to meet their objectives, design faults were often the reason. FAO Somalia was late in responding to the 2011 famine although it did eventually and effectively scale up its humanitarian response. The evaluation highlights a pressing need for the country office to invest in more rigorous analysis to understand the outcomes and impact of its work and to ensure that programme choices are fully informed to maximise their relevance, especially to address poverty and the needs of the most vulnerable.

RECOMMENDATIONS FOR FAO SOMALIA

Recommendation 1: Despite the operational challenges, FAO Somalia needs to strengthen the culture and practice of community consultation within the programme, from the design stage onwards, paying attention to dynamics within communities so that all voices are heard. Strengthening these skill-sets is especially important as FAO-Somalia embarks on a journey of community-based resilience programming. Action: FAO Somalia

Recommendation 2: FAO Somalia must invest in more rigorous analysis to underpin its programming and to test some of the assumptions on which programme choices and decisions have been made, for example about the best ways to support vulnerable households. This is an opportunity to link to research findings elsewhere in the region, for example on pastoralist systems of production, and to explore their relevance to the Somalia context. To support this more rigorous analysis, peer review mechanisms for different parts of the programme should be considered, involving experts from within FAO (HQ and SRO) as well as experts from outside. Action: FAO Somalia, FAO HQ and SRO

Recommendation 3: Linkages between the livestock and agriculture sectors in FAO- Somalia should be strengthened, especially around improved animal nutrition and fodder production. Closer sectoral working should be facilitated by FAO-Somalia’s shift to a programme approach so that synergies between different sectors can be more fully developed. Action: FAO Somalia

Recommendation 4: In the livestock sector: a. More holistic programming should be developed to complement FAO’s animal health initiatives with more substantial efforts to improve livestock productivity across a range of production systems and to rehabilitate degraded rangelands. Such livestock programmes should work to address vulnerability and poverty; b. Market-oriented interventions should add value at the production end of the chain, on the farm and feedlots, so that quality meat and milk products can be produced, thus boosting farmer income and creating employment opportunities along the value chain;

xvi c. FAO Somalia needs to ensure Livestock Emergency Guidelines and Standards (LEGS) are utilised by its own staff and partners when addressing future emergencies; d. FAO HQ should develop standard designs for slaughterhouses, indicating minimum facilities to be included, to use as blueprints for future slaughterhouse construction. Action: FAO Somalia

Recommendation 5: There needs to be greater investment in trend analysis, drawing on FSNAU and SWALIM data. FAO Somalia is uniquely placed to carry out such trend analysis to contribute to a deeper understanding of how livelihoods, the economy, and in turn poverty and vulnerability have been affected by years of conflict and displacement. This should inform FAO’s strategic programme choices and decisions, for example whether the current priority in the programme given to crops rather than livestock is appropriate, and will be a key resource for national and international actors in Somalia. It may also provide insights into the causes of chronic food insecurity in Somalia. Action: FAO Somalia

Recommendation 6: FAO Somalia must ramp up its advocacy skills and resources to match its early warning expertise in FSNAU, to advocate for a timely response before famine conditions develop. Action: FAO Somalia

Recommendation 7: New programming should be developed in three areas: a. to address issues of natural resource management. Using SWALIM data and analysis, a strategy for FAO Somalia’s engagement in this sector should be drafted; b. in peri-urban agriculture with two objectives, first to support the livelihoods of vulnerable households living in or near urban areas, and second, to support peri-urban agricultural and livestock development; c. as the custodian of substantial nutrition analysis, FAO Somalia should feed this directly into its agriculture and food security interventions to address long-term chronic under- nutrition in Somalia, and to play a leadership role in demonstrating how agriculture and food-based approaches can contribute to tackling malnutrition. Milk production and consumption should be central to this; d. although not ‘new’ programming, FAO Somalia’s recent commitment to the fisheries sector must be sustained. Action: FAO Somalia

Recommendation 8: The FSC is still establishing itself. In co-coordinating the FSC: a. FAO must take steps to ensure the large size of the cluster does not detract from its effectiveness. Revitalising the technical working groups, for example, is an opportunity for more focused and manageable technical debate and development of guidance material; b. communication and consultation with the field must be stepped up to reverse the tendency towards Nairobi-centric decision-making; c. ways of engaging with other key actors who are not part of the FSC must be found eg with the OIC, and with development coordination bodies. Action: FAO Somalia

Recommendation 9: The current compliance focus of FAO Somalia’s M&E needs to be re- balanced so that equal attention is given to outcomes and impact and to ensure that the M&E system feeds into learning as well as fulfilling accountability requirements. Re-thinking M&E

xvii as part of the resilience strategy offers an opportunity to develop an overall M&E framework, to review the role of the M&E unit and its relationship to the sectoral teams, for example the M&E Unit could maintain a focus on compliance monitoring while the sectoral teams lead in learning and impact and outcome evaluation. This is also an opportunity to review the role that well-designed participatory and qualitative exercises could play in the challenging operational context of Somalia in capturing outcomes and impact, learning from elsewhere where access is constrained, and to ensure that M&E becomes gender-sensitive Action: FAO Somalia

Recommendation 10: FAO Somalia must develop a differentiated and more nuanced approach to partnership, at a minimum differentiating between (i) contractual service provision, driven by an LOA, when the term ‘partner’ may not be appropriate, and (ii) genuine partnership where two organisations come together as equals and the relationship is guided by the principles of partnership often articulated in memorandum of understanding (MoU). FAO Somalia should consciously decide which approach is most appropriate in which circumstances. This should be informed by a review of how current procurement and partner contracting rules may actually stifle true partnership and innovation to see where adaptations can be made. This will be particularly important as FAO Somalia develops its community-based resilience programme and aims to work in close partnership with NGOs. Action: FAO Somalia

Recommendation 11: FAO-Somalia should develop a strategy for mainstreaming gender across the organisation, supporting staff to implement gender-sensitive programmes and to consider what gender equity means. This requires leadership and also a senior management position to drive such a strategy, and to develop an appropriate structure whereby addressing gender is everyone’s job rather than just an adviser’s job (for example, a gender focal team should be considered). This also needs to be properly resourced over a realistic timeframe to avoid one-off interventions. Action: FAO Somalia

Recommendation 12: FAO Somalia should formulate a framework for capacity- development as part of its revised strategy, to guide a more coherent and effective approach, especially in capacity development of government. This should pay attention to the three dimensions of capacity-development: individual, organisational and the level of the enabling environment2, drawing on experience of what has worked and what has not worked so far. This should be aligned with a broader UN-wide coherent approach to capacity-development. For capacity-development with government, realistic and long-term timeframes are needed. All capacity-development programmes should be preceded by a thorough capacity assessment. Action: FAO Somalia

Recommendation 13: FAO Somalia’s resilience strategy should be extended to 15 years (broken down into 3 year programming cycles for funding and review purposes), should be broadened to be more systemic, addressing governance issues, and must be carefully rolled out to ensure that all FAO staff are fully on board. There should be investment in research from the beginning of the programme to run alongside programming, and a review of FAO Somalia’s current skill sets to assess how these must change and/ or be strengthened for

2 See FAO’s learning modules on capacity development

xviii effective implementation of the resilience programme. See also Annex 7 for more detailed recommendations on this issue. Action: FAO Somalia

Recommendation 14: FAO Somalia must strengthen its HR management capacity to address the weaknesses that have been identified in this evaluation eg slow recruitment, lack of induction, poor communication with field offices, and to support the proposed office move to Mogadishu as well as decentralisation to Hargeisa and Garowe. Support from FAO Rome will be needed. Action: FAO Somalia

Recommendation 15: The relocation of much of FAO Somalia’s operations from Nairobi to Mogadishu, Hargeisa and Garowe is likely to be a period of turbulence when there may be many disruptions to programming, including loss of staff. This must be taken into account in the pace of programming which should be focused less on growth and more on the quality of FAO Somalia’s core activities as well as carrying out deeper analysis of the effectiveness of FAO Somalia’s work. Managing an effective period of transition may even require some programmes to be temporarily downscaled. Action: FAO Somalia

RECOMMENDATIONS FOR LEARNING ACROSS FAO

Recommendation 16: On learning: a. Where FAO Somalia has developed examples of good practice, for example in scaling up and establishing the CMU during the 2011 famine, and in some of its PPP work, these should be documented and shared with the wider organisation b. FAO’s sub-regional office should initiate and facilitate learning across the region, between FAO country offices working on similar issues sometimes in similar contexts, for example on public information systems and capacity development with government. Action: FAO Somalia, FAO HQ and SRO

RECOMMENDATIONS FOR FAO ROME

Recommendation 17: Four recommendations for FAO Rome emerge from this evaluation: a. planning for succession of the leadership of FAO Somalia should be a priority; b. FAO must speed up and modernise its recruitment practices and improve its human resource practices if it is to be competitive in the international arena and if it is to attract high calibre candidates; c. FAO should learn from the programme approach that has been pioneered by FAO Somalia and should promote this at country level over the project approach. It should also develop a sound financial management system that is appropriate to a programme approach yet also accommodates the more specific reporting requirement of individual donors; d. how technical support is going to be provided within FAO’s decentralised structure requires further thought and planning, including how to integrate high levels of technical expertise that may exist within a country office. Action: FAO HQ

xix 1. Introduction 1.1 Background to the evaluation 1. Since 2006 FAO has been carrying out country evaluations with the intention of assessing the totality of FAO’s assistance provided to the respective member state. This evaluation of FAO’s work in Somalia is the twelfth in a series of country evaluations. Specifically requested in early 2012 by the Assistant Director General Technical Cooperation Division and by the Officer in Charge (OiC) for Somalia, this was judged to be a key moment to pause and take stock of the last six years of FAO’s work in Somalia. In the wake of the 2011 famine response, the programme had grown rapidly, from expenditure of $14.2 million in 2007 to expenditure of $51.2 million in 2012, becoming FAO’s largest country programme in the world by 2012. It is a larger, more complex and higher profile programme than most other FAO country portfolios. The country office has undergone a number of significant internal organisational changes, including the decentralisation of management of the programme in 2012 as part of FAO’s corporate decentralisation strategy, and also in 2012 the shift from a project-based approach to a programme approach. At the time of the evaluation a new government was being established in Mogadishu with a greater sense of optimism than for many years. Looking back at the experience of FAO’s programme in Somalia since 2007 and drawing learning from that experience can therefore inform the future strategic direction of the programme at a key moment in its evolution.

2. This country evaluation has two principal purposes, both of which are given equal weight: (1) lesson learning, particularly for the FAO country office for its future strategic direction, but also for the broader organisation as it seeks to learn from the experience of FAO Somalia at a time of significant institutional change; (2) accountability, both to the donors that have funded and supported the FAO Somalia programme, and also to the respective government authorities and government in different parts of Somalia3.

Overall the evaluation is intended to be forward-looking and constructive. 1.2 Scope, approach, methodology & constraints

1.2.1 Scope 3. This evaluation of FAO’s country portfolio is a strategic and high-level evaluation, addressing questions such as the overall relevance and effectiveness of the programme over the last six years and how it has adapted to the dynamic and challenging political context and to development and humanitarian needs in Somalia. This is not an evaluation at project level although it was necessary for the evaluation team to familiarise themselves with individual projects that FAO has implemented (many of which have been subject to their own evaluation) in order to carry out its analysis, and evidence at the project level is often cited to back up a more generic or strategic finding. As stated in the Terms of Reference (TOR) for the evaluation: ‘The Country Evaluation will aim to assess FAO’s performance in Somalia over the past six years (i.e. since 2007), identifying success factors and challenges. Based on the main

3 Although the TOR mention accountability to communities and vulnerable groups, a high-level strategic evaluation is probably not the best process for achieving accountability to local people.

1 findings, the evaluation will provide recommendations for maximizing the strategic relevance of FAO’s engagement in the country over the next several years, and towards improving the efficient implementation and effectiveness of FAO support to the country in its future operations’ (See Annex 1 for the full version of the TOR).

4. Section 3 provides an overview of the FAO Somalia programme. During the period 2007 to 2012 FAO implemented almost 90 country-specific projects with a total expenditure of $211.8 million. It is also engaged in 10 regional projects4. Particular features of the FAO Somalia programme, pertinent to this evaluation and highlighted in the inception report, include, first, working in a context of weak or absent formal governance where the international community (including parts of the UN), has attempted to played a key role in state-building and ‘stabilisation’. This has had major implications for how FAO has engaged in Somalia, differently from its usual mode of operation in countries where there is a functioning central government. This is explored in section 4 on the relevance of the programme and is a theme running through the evaluation. Second, FAO has been engaged in both development and emergency interventions, often simultaneously although both have been implemented under emergency procedures. While the FAO Somalia programme is not unique in engaging in this type of programming, it throws up a number of issues that have long-challenged the international aid community, including how to manage rapid scale-up and surge capacity, and how to strengthen the linkages between development and emergency programming. This is explored in sections 5.2 and 5.4. Third, insecurity and lack of humanitarian access have forced FAO (and other international actors) to rely on remote management of its programmes in large parts of the country, posing considerable challenges for management and monitoring. At the same time, FAO has engaged in operational programming to a greater extent than would normally be the case. These issues run through the report, and are particularly explored in section 6.6.

1.2.2 Timing and team 5. The evaluation was carried out between September 2012 and January 2013. An inception mission to Nairobi took place in September 2012, further scoping the evaluation task, mapping stakeholders, developing the evaluation matrix (see Annex 2) and planning for the full evaluation mission which took place between late October and late November 2012. An in-depth study into FAO’s Cash for Work (CFW) programme in Dollow, Belet Hawa and Luuq districts of Gedo region was carried out during October and November 2012. See Annex 3 for the evaluation itinerary.

6. The evaluation was carried out by a ten person team. All were recruited internationally, of which three are of Somali origin. Three members of the team work for FAO’s Office of Evaluation in Rome (OED). The tenth team member joined in December to carry out a follow-up mission in January. Annex 4 provides background on the evaluation team members.

1.2.3 Approach and methods 7. Four evaluation criteria had been selected for this evaluation: (i) relevance;

4 27 of FAO’s global and regional projects indicate Somalia as a recipient country. However, further consultation with budget-holders indicated FAO Somalia was actively engaged in 10 regional and global projects.

2 (ii) effectiveness and impact (considered together in the TOR and in the evaluation matrix); (iii) efficiency; and (iv) connectedness.

8. The many questions in the TOR were rationalised during the inception phase into seventeen key questions, organised according to the four criteria above. See the evaluation matrix in Annex 2. These questions guided interview checklists that team members drew up as well as organisation of the analysis and findings. The structure of the evaluation report follows the criteria and questions in the matrix.

9. The evaluation team have made reference to international standards and guidance material where appropriate, for example the OECD ‘Principles for Good International Engagement in Fragile States and Situations’, Livestock Emergency Guidelines and Standards (LEGS) for livestock sector interventions, and to FAO normative standards, for example on food security information systems and cluster coordination. As the conceptual framework underpinning the evaluation, the team used the adapted livelihoods framework for complex humanitarian emergencies. See Figure 1. This framework usefully pays attention to ‘Policies, Institutions and Processes’ and how they impact on livelihoods as well as household assets, particularly appropriate to the breadth of FAO’s programme in Somalia which ranges from rebuilding livelihood assets to strengthening institutions and influencing the policy and legal framework.

Figure 1: Adapted livelihoods framework for complex humanitarian emergencies5

5 Source: Lautze, S., and A. Raven-Roberts (2006)

3 10. The data collection methods used by the evaluation team were the following:

1. Documentation review, including project documents, a review of strategy and planning documents, of FAO’s information products, and of normative guidance material; 2. Review of available monitoring data and of evaluations of past FAO work as well as more generic evaluations eg the Inter-Agency Standing Committee (IASC) real-time evaluation; 3. Literature review, especially to inform the contextual analysis and in terms of recognised best practice and recent research findings in different sectors; 4. Semi-structured interviews with key informants from all stakeholder groups; 5. Group interviews with particular stakeholders eg with focal points of the Food Security and Nutrition Analysis Unit (FSNAU) in Somaliland; 6. Where possible, individual and group interviews with beneficiaries and non- beneficiaries at community level. Men and women were usually interviewed separately; 7. In-depth CFW study – see below; 8. Facilitated workshops with members of the Food Security Cluster (FSC) in Nairobi, Hargeisa and Mogadishu; 9. A commissioned workshop, run by the Rift Valley Institute in Nairobi, with selected policy analysts and practioners on the topic of resilience and FAO’s joint resilience strategy; 10. An evaluation sheet circulated to FAO implementing partners in the agriculture sector, to selected FAO staff and some other key informants for feedback on cross- cutting issues.

See Annex 5 for a list of people met by the evaluation team.

11. The in-depth CFW study used a range of qualitative participatory data collection methods at community level, triangulated with phone calls to beneficiaries to verify implementation and outcomes. The evaluation team also made use of the findings of a survey carried out by the Global Food Security Cluster of members of the food security cluster in parts of Somalia, as part of a pilot to monitor performance of food security cluster coordination.

12. As far as possible, the team triangulated their data collection and findings between different sources.

13. The written outputs of the evaluation – the inception report and the draft evaluation report – have been subject to a peer review process within OED. Two resource people with extensive experience and knowledge of Somalia have also provided inputs into, and peer reviewed early written drafts6.

1.2.4 Constraints 14. The major constraint that the evaluation team faced was insecurity and therefore lack of access to many parts of Somalia, the same constraint that the FAO Somalia staff face in implementation. This meant that the team had very limited access to local communities and

6 Mark Bradbury of the Rift Valley Institute and Nigel Nicholson, independent consultant.

4 to beneficiaries, and no access at all to local communities in south-central Somalia with the exception of three districts in northern Gedo region where the CFW study was carried out and where some additional evaluation field work was possible. Lack of access to communities was a major drawback and meant that the team had to rely upon ‘intermediaries’, for example members of farmer and livestock professional associations who were able to travel to Mogadishu at the request of the evaluation team, and members of local and international non- governmental organisations (NGOs), for information about implementation at community- level.

15. Logistical challenges associated with travel, specifically changes to scheduled air travel, prevented some members of the team reaching parts of Somaliland. The field work of one team member was extended in order to overcome this.

16. The lack of data and statistics was a further constraint. To some extent this is the result of the lack of functioning government and protracted conflict, which has meant a lack of macro-economic data, restricted access to many parts of the country to collect data, and inconsistencies in data collection between international agencies7. Protracted conflict means that the economy and livelihoods have undergone significant changes over time, but there is little reliable information and analysis about these changes or about longer-term trends. There is also a lack of data on outcomes and impact in terms of FAO programmes as most monitoring to date has focused on inputs and outputs (see section 6.6). In a short evaluation such as this one, with limited access to the field and extremely limited access to local communities, it has been impossible for the evaluation team to do any systematic data collection in order to assess impact with one exception: the CFW study in Gedo region where four weeks of intensive field work were carried out and it has been possible to establish the impact of the CFW programme in this part of Somalia.

17. Although the evaluation looked at a six-year time period, staff turnover and difficulties of accessing key stakeholders from the earlier period meant the evaluation focused more on the last two to three years, although trend analysis over the six year period has been carried out wherever possible. 1.3 Structure of the report 18. The report begins by describing the context in Somalia in which the FAO programme has been implemented over the last six years – chapter 2. Chapter 3 provides an overview of the FAO Somalia programme and how it has evolved over the period 2007 to 2012. Chapter 4 addresses the evaluation criterion of relevance in terms of the big picture. How relevant was the FAO Somalia programme to the context and needs in Somalia, to the strategies and priorities of the respective government authorities in Somalia, to UN programmes and priorities, and to FAO’s own corporate framework? Chapter 5 dives into the detail of programming. The first sub-sections review aspects of programme design, including the extent to which FAO has consulted with stakeholders, and the data and analysis that has underpinned programme choices and implementation. These sections also address issues of appropriateness, but in terms of the choice of interventions rather than the overall strategic direction (addressed in Chapter 4). Other aspects of programme design and implementation are explored, including the extent to which projects achieved results, although as mentioned in section 1.2.4 above there are limited data and analysis to draw upon of the impact of FAO

7 It should be noted that government data collection and statistics were problematic even before the war, but lack of stable government has posed further challenges since.

5 Somalia’s projects. Chapter 5 also evaluates how effectively FAO responded to the humanitarian crisis and famine in 2011/12, the effectiveness of its coordination activities, and the effectiveness of its advocacy and influencing work. Chapter 6 looks at FAO’s internal systems and procedures: how effective are its management and operational systems? While this mainly focuses on the internal systems of the FAO Somalia programme, these are evaluated in the context of wider systemic change within FAO, particularly in relation to decentralisation and the provision of technical support. Chapter 7 addresses the evaluation criterion of efficiency. Chapter 8 evaluates the connectedness of the FAO Somalia programme in terms of its work on capacity development, the sustainability of its interventions and finishes with an analysis of the resilience paradigm and strategy that is intended to guide FAO Somalia’s programming in the future. Cross-cutting issues are picked up in Chapter 9: the extent to which the FAO Somalia programme is gender-sensitive and addresses issues of gender equality, and issues of partnership. Chapter 10 presents the conclusions of the evaluation and the evaluation team’s recommendations.

2. Context 2.1 Political context8 19. Somalia has been described as the ‘longest case of complete state collapse in modern times’ (Bradbury, 2010: 2). Despite numerous internationally-backed attempts to restore national government in the years since the collapse of the Siad Barre government in 1991, until 2012 none has been successful. A Transitional National Government (TNG) was established in 2000 out of a national peace conference convened by the Djibouti government and the Intergovernmental Authority on Development (IGAD), but received little international backing. This was followed by a Transitional Federal Government (TFG) that was put in place in 2004 and lasted until September 2012 with the explicit support of Ethiopia and the broader international community (although there was a change of president and parliament during this period). However, the TFG lacked the capacity and resources to achieve its mandate, was threatened by the rise of the Islamic Courts Union (ICU) in 2006 in Mogadishu and across south-central Somalia. Only with the defeat of the ICU in 2007 was it able to establish a presence in Mogadishu, but it was unable to extend its influence beyond the capital. Indeed, the period 2006 to 2010 was one of the most violent in the history of Somalia.

20. During this vacuum of formal governance the country was divided into clan-based areas of influence, much of it controlled by war lords and militias, including Mogadishu. There has also been a rise in jihadist insurgency, currently spearheaded by Al Shabaab, a militant Islamic movement that, at its peak, controlled much of south-central Somalia, and in February 2012 announced that it had joined forces with Al Qaida9.

21. Following a lengthy and contested political process backed by the UN and other governments in the region, a post-transition government was appointed in August/ September 2012 (at the time this evaluation commenced). Federal Parliament was appointed by traditional leaders in September 2012 and elected a new President, who subsequently appointed a government and ten ministers. These developments have been met with greater hope and optimism about Somalia’s political future than there has been for some time, with many of the diaspora returning to Mogadishu and the new Somali National Government enjoying high levels of public support and legitimacy. Yet the new government faces many challenges in establishing itself, not least in re-asserting the authority of formal government and managing the transition from high levels of chronic violent conflict to non-violent and ‘normal’ political contestation (Menkhaus, 2010 and 2012b). Reclaiming ownership over developmental processes and policies, including in the agriculture and livestock sectors and in natural resource management will be a further challenge. Meanwhile significant parts of southern and central Somalia, some of the most productive agricultural areas, remain under the control of Al Shabaab and have been since 2008 despite significant military gains in the last twelve months by the African Union Mission in Somalia (AMISOM) in support of the Somali army.

8 Based on Bradbury (2010), Menkhaus (2010), Menkhaus (2012a), Menkhaus (2012b) and a paper on the Somalia context written by Nigel Nicholson, commissioned by OED, FAO 9 EIU Somalia Country Report May 2012

7 22. In short, internal conflict has prevailed across most areas of southern and central Somalia for more than two decades, influenced in the past eight years by both regional and global political agendas. Most of the population has lived with chronic insecurity in which acts of armed criminality, communal violence, and politically driven war are often indistinguishable (Menkhaus, 2012b:4).

23. The north of Somalia has followed a completely different political trajectory. Since 1991 the region of Somaliland has been a self-declared independent entity although a continued source of frustration has been the lack of international recognition of its independence. Puntland also declared itself a semi-autonomous region in 1998. Both have established governance and administrative structures ensuring relative peace and security (although it is a continued source of debate whether the government is a result of, rather than cause of peace and security10) but the border areas between the two remain contested. 2.2 Livelihoods and the economy 24. The majority of Somalia’s population depends upon pastoralism and agro- pastoralism for its livelihood. Thornton et al (2002) estimated that three-quarters of the population were livestock dependent in 2000. Only a small proportion is dependent solely upon settled agriculture, principally along the Shabelle and Juba rivers and in areas with more consistent rainfall such as Bay and Bakool. With more than fifty five percent of Somalia’s land classified as arid and semi-arid rangelands, extensive livestock production will remain a crucial land use option for the country. Indeed, in such an environment the management of mobile herds of livestock, to take advantage of variable rainfall and vegetation growth (pastoralism), has a comparative advantage over other land use options (Headey et al. 2012). Estimates of the comparative GDP shares of different sectors are generally based on historical data and therefore vary. The World Bank estimates the livestock sector accounts for around 25% of GDP (World Bank, 2006) whilst the US Government’s Central Intelligence Agency estimates this figure to be 40%11. Similarly national export earnings from the livestock sector vary from 50 to 80%, with the 80% figure being consistently quoted by FAO.

25. However, protracted conflict and the lack of effective governance, law and order, especially in southern and central Somalia, have taken their toll on many households, negatively impacting their livelihoods. Somalia is now one of the poorest12 and most food insecure countries in the world. Over 43% of the population is estimated to live on less than $1 per day and the unemployment rate is above 60%13. One of the major ways in which conflict has impacted on livelihoods and on settlement patterns over the last couple of decades is through waves of displacement, particularly affecting ‘minority groups’ associated with agro-pastoral production. This, in turn, has fuelled urbanisation as many of the displaced, or at least some members of displaced households, remain in urban areas even if others return to areas of origin seasonally for farming14. In this sense many of the displaced

10 See Menkhaus (2012b) 11 CIA World Fact Book https://www.cia.gov/library/publications/the-world-factbook/geos/so.html as of February 2013 12 The income per capita was estimated in 2002 to be US$226 compared to US$515 in Sub-Saharan Africa (World Bank 2009), although it should be noted that the economy of Somalia has never been well-understood 13 Somalia Joint Strategy Paper 2008-2013 14 According to the OCHA Situation Analysis of Mogadishu in February 2012, the IDP population in Mogadishu was estimated to be 184,000 in October 2011

8 have become the long-term urban poor. Urbanisation has also been fuelled by members of rural households seeking alternative employment opportunities.

26. An analysis of how livelihoods and settlement patterns have changed in Somalia is beset by a lack of data, especially trend data that tracks how livelihoods, the economy and settlement patterns have changed during two decades of conflict. In the livestock sector, for example, little work has been carried out in Somalia to assess trends in ownership and access levels or the drivers of those trends. The Somali livestock sector remains largely reliant upon community knowledge and external research carried out in the neighbouring Somali areas of Ethiopia and Kenya to hypothesise how ownership and herd size might have changed. Research in neighbouring Ethiopia and Kenya would suggest a trend towards smaller herds and sedentarisation as population growth, loss of grazing lands and water access, poor education, high food prices, lack of basic services and chronic conflict continue to threaten pastoralist livelihoods in Somalia. These smaller herds can no longer sustain family members (Devereux 2006, Davies and Bennet 2007). Recent research also reveals the concentration of livestock ownership and wealth amongst fewer groups (Aklilu and Catley, 2011).

27. In terms of agriculture, the high potential crop production areas which were the traditional bread-basket of Somalia in better times, are the irrigated farming areas along the two river basins of Shabelle and Juba in the south, and areas with more consistent rainfall such as Bay and Bakool. Apart from Mogadishu, the riverine valleys of the Shebelle and Juba water catchments provide some of the most valuable real estate in the country, and consequently have been the sites of some of the most damaging armed conflicts, fuelled by land grabs as much as ideology. Conflict and the consequent lack of investment resulted in deteriorating irrigation and flood control infrastructure in the irrigated sector to the point that an estimated 85-90% of the irrigated land pre-conflict (in the 1980s) had become rain-fed by 2004 (World Bank, 2007). As farming communities have become displaced during the conflict years, there has also been a change of ownership of land, particularly in the high potential areas where displacement from land began even before the war years (Besterman and Cassanelli, 1996). Cereal production appears to have declined during the conflict years, although the data is contested (see section 5.1 below). Although the high potential irrigated/ riverine agriculture areas are the most densely populated outside of the major towns, there is a relatively small percentage of the population entirely dependent on crop production. Most are agro-pastoralists with dual incomes from crops and animals, and the majority of the population are dependent on rainfed rather than irrigated agriculture.

28. Not only has production been affected by years of conflict and lack of access to basic services, so has trade and marketing. In terms of crops, market infrastructure has deteriorated during the conflict years, and farmers are also caught in the plight of having to sell their produce immediately after harvest at low prices because they are cash-strapped, do not own storage facilities, nor do they have the means to transport their products to markets that offer better prices. This impacts at the micro/ household level. In terms of the livestock trade, this is highly significant at the macro level as livestock account for an estimated 80% of Somalia’s foreign exchange revenue (World Bank, 2006). The country’s reliance on the livestock trade (mostly sheep and goats) emanates primarily from its proximity to the Middle East. However, the livestock trade was badly affected by the ban on live animal exports imposed by Saudi Arabia in 2001 and not lifted until November 2009. This ban forced traders

9 to set up export abattoirs and switch to chilled meat exports. This has not been successful and by the end of 2009 meat exports had ceased altogether and the abattoirs closed15.

29. Fishing is an important potential resource to Somalia as the country has one of the longest coastlines in Africa, extending for 3,300 km with two separate marine environments—the Indian Ocean in the South and the Gulf in the North16. Fishing is not yet a major livelihood for many because its potential has never been developed despite considerable investments in the 1980s, and it is not a traditional food source in Somalia. Nevertheless, this sector has been badly affected by the long-running civil war which has meant that Somalia has not had formal, internationally recognized control over its 200 miles of Exclusive Economic Zone (EEZ), there has been over-fishing and depletion of marine resources by illegal trawlers and gill-netters, and destruction of fishing vessels and fish processing facilities as a direct result of violent conflict.

30. As traditional livelihoods in Somalia have been affected by long-running conflict, the role of remittances in shoring up both the household economy and the national economy has been highly significant and of growing importance. International remittances are widely believed to be the largest contributor to GNP17, exceeding international aid flows, although they have taken a hit from the recent global financial crisis.

31. Conflict and the social upheavals it has triggered have brought significant changes to gender roles and relations. A ‘Gender Profile for Somalia’ carried out in 2007 notes that ‘women’s gender roles have been stretched beyond traditional limits to meet the new domestic, social and economic needs of the family and local community’ (Gardner, 2007:2), and many women are now the main breadwinner for the household. Yet women do not have the same access or rights as men to productive resources and assets such as land, nor easy access to the political domain although there is evidence that many women are now taking the main role in domestic decision-making (ibid). 2.3 Humanitarian crisis 32. Over the past two decades Somalia (primarily south-central) has been described as: ‘one of the world’s worst and most enduring humanitarian crises’ (Hammond and Vaughan- Lee, 2012:2). Conflict and the lack of effective governance, law and order in southern and central Somalia have been, and continue to be the principal factors affecting the humanitarian situation in the country, compounded by recurrent droughts and floods, and by external factors such as rising global food prices. The most acute periods of humanitarian crisis have attracted greatest international attention and have made international news, for example the 1992 famine that claimed the lives of a quarter of a million people after the Somali state collapsed, described as setting a benchmark for humanitarian crises in the post-Cold War

15 The abattoirs were beset by problems including low hygiene standards (resulting in the rejection of carcasses by importers); low carcass weight of 7 kg on average; certification problems; a flight ban imposed on old propeller aircraft being used for meat cargo; and shortages of supplies during the drought. Between 2006 and 2009, the abattoirs were only able to export a total of 811,519 chilled shoat carcasses or an average of 162,300 carcasses a year. Fierce competition from live animal exporters finally forced the closure of all abattoirs at the beginning of 2009. 16 Various estimates put Somalia’s sustainable marine fish production close to 300,000 tons per year with very high potential for tuna (as reported in presentation by FAO Somalia, November 2012) 17 Remittances to Somalia were estimated to be around 60% of GNP in 2004, according to the World Bank (2006). The EIU estimated remittances at around US$1 billion per year (EIU Somalia Country Report May 2012)

10 world (Bradbury, 2010: 2), and the most recent famine in 2011. However, during the periods between, food security and other indicators are frequently at emergency levels, for example maternal mortality is at 1,400/100,000, under-five mortality is at 142/1,000 and under-five acute malnutrition is consistently above 19%18. ‘Somalia – especially southern Somalia – has had indicators of humanitarian wellbeing that have been substantially worse than would be tolerated in other countries’ (Maxwell and Fitzpatrick, 2012:6) This ‘normalisation of crisis’, first raised in the late 1990s (Bradbury, 1998), has persisted in Somalia since.

33. Bradbury (2010:8) describes the high levels of displacement in Somalia as ‘a mark of a severe protection crisis in Somalia’. By the end of 2012 the estimated IDP population was 1.1 million19 although the number is in constant flux as some households split at the start of the agricultural season with some members returning to their areas of origin while others stay to benefit from humanitarian assistance and business opportunities in the urban areas.

34. In mid-2011 famine was declared by the UN in southern and central Somalia. An estimated 4 million people were in need of emergency food and medical assistance20. Hundreds of thousands were displaced, some crossing the border to Kenya where the population of Dadaab camp rose to over 450,000. Although labelled by many as a ‘drought crisis’, it is widely acknowledged that the real triggers of the famine were lack of access to food, in turn related to conflict and the denial of humanitarian access to a number of humanitarian agencies21. This came on top of rising food prices since 2009 (related to the global hike in food prices) and falling remittances from the Somali diaspora as a result of the global recession22. Recent analysis of those most affected by the famine draw attention to ‘minority’ populations, in particular Somalis of Bantu origin living along the Shabelle and Juba rivers, and poor agro-pastoralists from the Reewin clan, historically minority and marginalized populations who were also the main victims of the 1991/92 famine (Majid and McDowell, 2012). 2.4 The context of international engagement 35. In the two decades since state collapse in Somalia, levels of international aid and engagement have fluctuated widely according to regional and global geo-political and strategic interests. During the 1990s international aid provided through the Consolidated Appeals Process (CAP) dropped to less than $50 million. By 2009 this had increased to over $500 million, mostly a result of renewed political interest in Somalia (Bradbury, 2010).

36. 9/11 marked a turning point in the nature of international engagement in Somalia as the humanitarian imperative to protect and provide relief to war-affected civilians was superseded by international (and especially US) security concerns including concerns over transnational terrorism (ibid). Menkhaus (2010) traces Somalia’s long history of tension between political stabilisation objectives and humanitarian objectives, remarking that:

18 Source: UNOCHA CAP 2011 19 UNHCR figures published in the CAP of 2013 to 2015 20 OCHA estimated that only 2.2 million out of the 4 million people in need were actually being reached (as quoted in Hammond and Vaughan-Lee, 2012) 21 See Hammond and Vaughan (2012), and Darcy et al (2012) 22 Estimated by one source to have reduced by as much as 25% in 2009 (UNOCHA CAP 2010)

11 ‘This ‘instrumentalist’ approach to humanitarian aid by political actors pursuing stabilisation goals in Somalia has been at its most unvarnished form in the period since 2006, and is responsible for the deep tensions within the donor, aid and diplomatic community in Nairobi’ (Menkhaus, 2010: 338).

37. This has had a major impact on how humanitarian agencies and UN agencies such as FAO are perceived, especially perceptions of their neutrality and impartiality. In 2008, Al Shabaab made it clear that aid officials would be considered legitimate targets in the war, accusing them of collaborating with foreign forces (Bradbury, 2010). This highly politicised context for international aid to Somalia, including humanitarian aid, is one of the main reasons why it has become one of the most challenging and dangerous operating environments for international aid agencies. There has been a significant contraction of humanitarian space in Somalia, especially since 2010 (Hammond and Vaughan, 2012). By November 2011 most UN organisations and INGOs were expelled from Al Shabaab controlled areas.. Although FAO has been able to continue working in Al Shabaab controlled areas, FSNAU, an FAO project, has not and some of FAO’s implementing partners have also been expelled or denied access23.

38. On a more practical level, the location of most of the international aid community in Nairobi for over a decade, principally because Mogadishu has been too insecure, has been a source of frustration and resentment across Somalia, especially in the more secure Somaliland and Puntland. The centre of gravity for decision-making has been Nairobi and many government officials and civil society leaders have felt unable to engage in, and influence that decision-making. In Hargeisa that frustration has been compounded by lack of international recognition of the independence of Somaliland which, in effect, means that most aid resources from donor governments are channelled through UN agencies rather than provided directly to government.

39. Insecurity in many parts of Somalia, and/ or access being denied to some international agencies has meant that remote management has become the modus operandi for many agencies, including FAO, in many parts of the country. This poses immense challenges ranging from conducting needs assessments, to project or programme implementation, to monitoring and preventing the diversion of funds. In terms of the latter, the risks are particularly high in view of the prevailing culture of corruption in Somalia and was a major issue as humanitarian programming scaled up rapidly, disbursing large quantities of aid in response to the famine in 201124.

23 The reasons for FSNAU being expelled by Al Shabaab while the rest of FAO has not been are not entirely clear although this may be due to the fact that FSNAU is in the business of data collection and analysis which may be regarded with some suspicion. FAO programmes, on the other hand, fall in line with Al Shabaab emphasis on strengthening the productive sector. 24 Somalia was ranked at the bottom of the Corruption Perceptions Index 2010 published by Transparency International. See http://www.transparency.org/cpi2010/results

3. An overview of the FAO Somalia programme

40. The growth of the FAO Somalia programme between 2007 and 2012 is striking, in terms of overall scale as well as sectoral and geographic coverage. See Figures 2 and 3. In 2007 expenditure was just over $10 million and the EU was the main donor. By 2012 expenditure is more than US$50 million and there are now a number of major donors (including the EU/ ECHO, DFID, USAID, the World Bank and AusAid). Since 2010, there has been a significant increase in resources allocated to south-central Somalia, much of it emergency-oriented. See Annex 6 for a list of projects implemented by FAO Somalia between 2007 and 2012.

Figure 2. FAO Somalia programme expenditure, by year and by sector (USD millions) 70

60 USD Millions

- 2007 2008 2009 2010 2011 2012 (up to October)

SWALIM FSNAU M&E Fishery Live Stock Agriculture CFW

Source: FAO Somalia

Figure 3. FAO Somalia expenditure by geographic area, 2007 to 2011 60

USD Millions 40

0 2007 2008 2009 2010 2011

Somaliland/Puntland South Central Somalia National Scope or Projects delivering in more regions

Source: Office of Evaluation elaboration of FPMIS data

41. While strengthening food security and livelihoods have long been the focus of FAO’s work in Somalia, at the beginning of this period most FAO projects were agricultural, focused on crop production and irrigation, mainly in south-central Somalia in the irrigated areas of Lower and Middle Shabelle and the Juba basin. During the period 2007 to 2012 FAO has been de facto gatekeeper for funds for agricultural sector interventions in Somalia. It has

13 implemented a total of 38 projects across the three administrative areas of Somalia although most have continued to be in south-central.

42. Since the beginning of the period under analysis, the livestock component of the programme has expanded, although it has always been a smaller component in terms of expenditure than agriculture. In this sector FAO Somalia has relied upon a relatively narrow range of activities, mainly focused on animal health interventions that have been deemed practical for the extremely difficult working environment in Somalia. Although it has developed a number of broad development-orientated strategies (eg FAO/EC/World Bank (2004), FAO (2009), FAO (2010) these have tended to be overtaken by emergency events and activities have slipped back into a relatively narrow range, mostly related to animal health. Between 2007 and 2012 37 projects have included a livestock component.

43. A number of FAO projects have focused on the private sector and promoting trade, especially in the livestock and agriculture sectors.

44. The fisheries sector has been very small although a new strategy for work in this sector promises expansion in the years to come. Most work has been in Puntland so far25. The thrust of FAO’s interventions have been infrastructure (including the installation of processing centres) and training, and to a limited extent the provision of inputs such as fishing equipment.

45. Figure 2 shows how significant CFW has become. Initially implemented on a small- scale in 2007, in 2011 it was rapidly scaled up in response to the famine and the fact that FAO was the only UN agency with access to Al Shabaab-controlled areas of south-central. In 2011 it represented 38% of total FAO programme expenditure, and an estimated 30% in 2012.

46. For a number of years FAO has supported the production and dissemination of information and analysis as a public good available to national and international actors working in Somalia. This includes food and nutrition security and livelihood-related information provided by the FSNAU (which began in 1994 and has been managed by FAO since 2000), and the Somalia Water and Land Information Management project (SWALIM), operational since 2002. For many years both projects operated independently of the FAO Somalia programme; since 2010 they are managed within the country programme.

47. Coordination has become an increasingly important aspect of FAO’s work in Somalia, initially through the Agriculture and Livelihoods Cluster that FAO chaired, and since March 2012 through co-chairing with WFP the newly merged Food Security Cluster (a result of the merger of the Agriculture and Livelihoods Cluster with the Food Assistance Cluster).

48. The main strategic document produced by FAO Somalia since 2007 is the FAO Somalia ‘5 year Strategy and Plan of Action’ produced in early 2011 for the period 2011 to 2015. In May 2012 the joint Resilience Strategy was finalised by FAO, WFP and UNICEF; as a result, FAO Somalia is now re-working its ‘5 year strategy’ to fit the new resilience paradigm. These two documents were key reference points for the evaluation team.

25 This is because of insecurity in the south and Puntland has a better marine resource base than Somaliland.

4. Relevance of the FAO Somalia programme 4.1 To the context and needs

4.1.1 Issues 49. Two overriding issues must be borne in mind when considering the relevance of the FAO Somalia programme to the context of Somalia, and to the needs of the most vulnerable: first, the relevance of the programme to a context of failed formal governance, particularly in south-central Somalia – what is an appropriate role for a UN agency like FAO to play? Second, how relevant has the programme been in a context of protracted crisis where there are persistently high levels of humanitarian need, periodically becoming acute?

4.1.2 In the context of failed formal governance 50. In the context of failed central government, there are a number of ways in which FAO’s programme has been highly relevant. The organisation has played an important and appropriate role providing services that would normally be associated with central government, first, in terms of the public information systems that FAO has been running, both FSNAU and SWALIM. For many years FSNAU has been a key source of data and analysis on food security needs, indeed often the only source of comparable data across the country and across time that is available as a public good (in contrast to VAM which is principally for the use of WFP). Providing seasonal assessments of emergency and humanitarian needs, FSNAU has provided critical information for the planning of humanitarian action and food security interventions. SWALIM has played a similar role in relation to natural resources. Particularly important has been the role of custodian that it has played for Somalia’s historical database on natural resources, dating back to the 1960s, a wealth of information that could so easily have been destroyed and lost during the conflict years. SWALIM has re-established monitoring systems in Somalia, for example on rainfall and meteorology, and river monitoring for early warning and flood management. It has carried out new mapping of land and water resources, and recently of land degradation. All of these services have continued to be highly relevant as government has lacked the capacity and infrastructure to do such mapping or run the monitoring systems itself. This, too, has provided critically important information for development planning (eg to guide water drilling, and for the new water strategy in Somaliland) as well as early warning.

51. Taking over functions of government as sole provider of critical information that has informed policy and programming carries a certain responsibility. While FSNAU is widely credited, especially by the international community, for the role that it has played in providing regular food security assessments, it has been much weaker in tracing and analysing long-term trends and hence the transformation and changes in livelihoods in Somalia during the conflict years. In a context in which there has been major disruption and displacement over the last two decades, these are serious knowledge gaps that FSNAU should be well-placed to fill given its wealth of longitudinal data since the early 1990s. This gap has been recognised for some time and is supposed to be addressed in the next phase of the FSNAU project beginning in 2013. The capacity and resources to do so must be protected despite the short-term imperative to provide snapshot seasonal assessments. In the meantime the World Food Programme (WFP) (working closely with FSNAU and SWALIM staff) attempted a longer-term analysis by combining FSNAU and SWALIM data to provide some

15 preliminary trend analysis for the UN community in November 201226. More solid research is urgently needed to capture and analyse longer-term trends; FSNAU and SWALIM are uniquely placed to carry it out. Another knowledge gap that FSNAU is well-placed to fill is providing analysis and understanding of the underlying causes of chronic food insecurity.

52. Other ways in which FAO has played an appropriate role, effectively stepping into the breach of lack of central government, include the role it played in attempting to get the trade ban lifted on the export of live animals from Somalia, inviting Saudi veterinary officials to visit the quarantine facilities in Berbera and Bosaso, and when this failed, trying to arrange for OIE (the World Organisation for Animal Health) to do an inspection on behalf of Saudi officials. These efforts ultimately failed due to the political nature of the ban, but it was entirely appropriate for FAO to play this role. In 2010 Saudi Arabia did eventually lift its ban on the livestock trade and Somalia was recognised as free from rinderpest disease. Whilst this cannot be attributed to FAO alone they were important milestones for Somalia and FAO was part of the process of attaining eradication of rinderpest.

53. All of these examples show how important it has been for FAO Somalia to pay attention to the ‘policies, institutions and processes’ component of the livelihoods conceptual model in Figure 1, in the absence of a functioning central government.

4.1.3 Relevance of the livestock, agriculture, and fisheries programmes 54. In the livestock sector more broadly, FAO’s programme has been well-adapted to the Somali context, to the Saudi ban on live animal exports and to the final eradication of rinderpest from the Somali Ecosystem. Strong policy narratives at that time were that the trade ban was damaging the livelihoods of livestock owners and the final eradication of rinderpest was a global public good. International efforts, including FAO’s, therefore focused on meat exports, disease surveillance and live animal certification systems. In 2004, FAO in collaboration with the European Commission (EC) Somalia Office and the World Bank, drew up a broad ranging livestock strategy which included pillars such as livestock production, animal health and disease control, livestock marketing and trade, and capacity building, as well as cross-cutting issues such as land tenure, improved feeding, environmental assessment and education (FAO/EC/World bank 2004). In practice, most of FAO’s activities from 2004 have centred on disease surveillance and building the capacity of Somali livestock professionals, and on livestock marketing and trade with little attention given to the cross- cutting natural resources and land tenure issues, as described below (Masiga et al 2006, Hashi et al 2007). As explored in section 5.1 below, although the focus on animal health was in line with the prevailing policy narrative and appears to have been a donor priority, it has not been backed by evidence that this should remain the dominant focus.

55. FAO’s interventions in support of both the export meat trade and the domestic meat trade appear to have been relevant to the context. In terms of the former, it has attempted to increase acceptance of Somali meat exports through improved sanitary and phyto-sanitary standards of the export slaughterhouses. In terms of the domestic meat trade, FAO’s efforts have focussed on formulating regulatory frameworks, building institutional capacities (laboratory and meat hygiene units), training meat inspectors, meat vendors and other actors in the value chain, and initiating value addition practices, for example through bone crafts

26 See WFP’s presentation on ‘Trend Analysis of Food and Nutrition Insecurity in Somalia 2007-2011’, presented in November 2012

16 and soap making. These interventions were carried out in a country lacking such frameworks, capacity and support and are therefore judged to be relevant although, as argued later in section 5.1.3, a more relevant approach would have been improving production and productivity for both the domestic and export markets.

56. In terms of the crops sector, although FAO has implemented projects in both the high potential irrigated areas and in areas of rain-fed farming, most of the funding was allocated to the irrigated areas of Lower and Middle Shabelle and the Juba basin in south- central Somalia and to irrigated small farms in Somaliland and Puntland. On the one hand it may be argued that this is relevant because of the dilapidated state of infrastructure in the irrigated sector and the higher costs of production which should therefore be subsidised because of the importance of this sector to national food production and for the export of cash crops. These interventions are principally aimed at increasing food supply, although the export of surpluses of some commodities (eg sesame) generates income to meet the rising cost of farm labour and therefore offers employment opportunities for rural youth. On the other hand, questions remain about the extent to which a focus on the irrigated farming sector meets the needs of the most vulnerable farming households, many of which are dependent on rain-fed agriculture directly, rather than increasing labour opportunities in the irrigated sector. Finding ways of supporting the agro-pastoralist livelihoods of vulnerable households, without compromising the momentum being generated in the irrigated sub-sector, should be a priority for the new resilience strategy, including finding more ways of integrating the two sub- sectors. Indeed, during 2012 FAO Somalia has begun to make efforts to re-balance the focus on the irrigated sector by paying greater attention to the rainfed sector.

57. FAO’s focus on seed production and marketing appears relevant to the needs of farming households and to increasing crop production, particularly promoting improved seed varieties through the private sector in the absence of a functioning Ministry of Agriculture, at least in south-central Somalia. It is entirely appropriate that investment in increased crop production is being matched by support to marketing of that produce.

58. In terms of the balance of expenditure between sectors, the agriculture/ crops sector accounts for one-third of FAO’s total expenditure between 2007 and 2012 and the livestock sector accounts for just under one-fifth. (See section 3 above). This is somewhat surprising in view of the significance of livestock to household livelihoods and to the overall economy in Somalia as described in section 2.2 above, and is indicative of a crop-sector bias in the programme. This may initially have been a consequence of FAO responding to donor calls for proposals in the agriculture sector. It is less clear why it has persisted and therefore the rationale.

59. FAO’s fisheries programme has been slow to get off the ground in view of the potential for harvesting fish in the waters off Somalia. This has not been a priority area for FAO Somalia and has therefore been a missed opportunity until recently. Over the last couple of years, however, there has been renewed commitment, appropriately targeted at addressing three critical issues in the current context: providing training in various aspects of marine resource management and production as well as skills development in post-harvest processing, handling and maintenance of equipment; provision of boats, fishing gear and nets; and installation of processing centres at landing sites.

17 4.1.4 Relevance to the context of protracted humanitarian crisis 60. In responding to emergencies, FAO’s interventions overall appear to have been appropriate. In the livestock sector, between 2007 and 2010 FAO responded to a range of emergencies related to flooding along the Shabelle and Juba rivers, the threat of new disease outbreaks particularly of rift valley fever and escalating conflict in south central Somalia, localised drought and increasing vulnerability related to high food prices. Whilst the majority of the funds continued to be spent on animal health, particularly support to service provision by Livestock Professional Associations, there were also limited restocking, fodder production and water access initiatives. Although these initiatives were generally relevant to needs, there is a gap in knowledge about the causes of vulnerability of poorer livestock owners and little analysis over the last few years to address this gap which could lead to more nuanced interventions.

61. In the agriculture sector FAO’s emergency interventions mostly had a two-fold purpose, providing inputs for the next season and providing work opportunities to the greatest number of people in the shortest possible time through CFW. Input delivery programmes, especially seed distribution, avoided the mistake of misplaced distributions of exotic seed made in neighbouring countries (eg South Sudan)27. Instead, FAO Somalia supported distribution of the widespread, locally multiplied maize variety (Somtux) and local sesame varieties, benefiting from its seed multiplication and production work on the development side28.

62. FAO’s decision in 2011 to adopt cash-for-work as an emergency response modality was very relevant to meeting basic household needs in a context where the famine-affected populations’ access to food relief was severely limited by the presence of Al Shabaab which instead favoured interventions related to long-term productivity. Switching from mechanised canal refurbishment in FAO’s development projects to hand-desilting of secondary and tertiary canals was entirely appropriate to the need to create work.

63. The extent to which the CFW programme reached the most food insecure and met their needs could not be measured by the evaluation as the areas where the famine was most severe remain the most inaccessible. Where it was possible to carry out fieldwork, in the part of Gedo region which was not Al Shabaab-controlled, there is evidence that the income generated through CFW had a strong impact on the food security of entire communities, both directly through targeted households’ increased expenditure on food, and indirectly through redistribution by beneficiaries of the programme’s benefits to other vulnerable households and through other knock-on effects on the wider economy. The relevance of CFW in enhancing agricultural productivity is more complex to assess because of the many factors that may influence agricultural productivity (including pests, diseases and floods), although one year after the intervention in Gedo, farm owners and sharecroppers on the farms benefiting from canal rehabilitation were mostly expecting a good harvest which they attributed largely to FAO’s CFW programme. But it should be noted that the CFW study team found that often the choice of infrastructure did not strictly follow needs in the riverine

27 See FAO(1998 to 2003): ‘Special Notes for South Sudan – CFSAMs’, FAO, Rome, and ODI Seeds and Biodiversity Programme (1996) ‘Seed Provision During and After Emergencies’, Relief and Rehabiltation Network Good Practice Review No 4, Overseas Development Institute 28 Both the seed companies and farmers prefer the ‘Humera’ sesame variety imported from Ethiopia, but this can now be multiplied locally, through the seed companies

18 farming communities of Gedo region, and that wealthier farm-owners tended to capture the greatest proportion of benefits for themselves and their farms.

64. More detail on the effectiveness of FAO’s response to humanitarian crises, especially in response to the 2011 famine, is provided in section 5.2 below.

4.1.5 Flexibility to respond to the changing context 65. The dynamic and conflict-ridden context of Somalia means that programmes can quickly become irrelevant or impossible to implement as the context changes and may need to be adapted. There are a number of examples of that flexibility within the FAO Somalia programme. On the development side, projects that were designed to support the meat export sector had to be re-focused when the export slaughterhouses closed in 2009; resources were switched to supporting the domestic meat industry instead. On the emergency side, the CFW programme was appropriately re-focused when the humanitarian crisis intensified in 2011, from prioritising asset rehabilitation to prioritising cash distribution in order to assist drought- affected populations in otherwise inaccessible areas. While this is a positive finding there are other ways in which the CFW programme could have paid greater attention to the political and conflict context, for example in being more realistic about what could be achieved and better analysing the reality on the ground, as well as consciously adopting a ‘do no harm’ approach, for example ensuring no negative impacts on the livelihoods of the poorest (see section 5.2 below for examples), ensuring no negative impact on natural resources, and ensuring beneficiaries were fit for work.

4.1.6 Areas that deserve greater attention: urbanisation, natural resource management and nutrition 66. As the process of urbanisation has accelerated in Somalia, fuelled by displacement, FSNAU started to carry out assessments of urban livelihoods in recent years. This was an entirely appropriate response to the changing context and FSNAU has played an important role in building a database and knowledge about urban livelihoods. On the programming side, however, FAO has been slower to respond to this fundamental shift in settlement pattern in Somalia. It has started to work with pastoralists and ex-pastoralists living in peri-urban areas and IDP camps in 2012, for example implementing a small-holder dairy development project in Somaliland, but there is scope to do much more, and this may be particularly important to FAO’s efforts to build resilience – see section 8.3 below.

67. Perhaps the most significant and surprising gap in FAO’s programme over the last six years is the lack of investment in natural resource management, especially as FAO has hosted SWALIM and has therefore made a major contribution in terms of mapping and analysis of natural resources. Furthermore, environmental issues were identified as a cross- cutting issue in the 2004 livestock strategy that FAO co-authored (FAO/ EC/ World Bank, 2004). But these have not been followed through with programme interventions to support natural resource management, nor to address charcoal production (until the end of 2012), a major cause of environmental degradation. FAO Somalia’s senior management is well aware of this gap, claiming there has been a lack of donor interest in this topic, and efforts are now being made to address the complex issues of land tenure, but somewhat belatedly.

68. A third gap relates to nutrition. Although FAO Somalia probably generates more information and analysis on nutrition (through FSNAU) than any other FAO programme, revealing a state of chronic malnutrition in Somalia, this information is not carried through

19 into programme interventions. Instead, the data and analysis are principally used by WFP and UNICEF for emergency and health interventions. There is an opportunity here for FAO Somalia to play a stronger leadership role, to address chronic malnutrition more directly through its programme interventions, and to play a leadership role in identifying how agriculture and food-based approaches can contribute to tackling hunger and malnutrition. This would be directly in line with FAO’s new Nutrition Strategy, tabled in Rome in September 2012.

69. Overall the evaluation judges the FAO Somalia programme to have been relevant to the context and to needs over the last six years, although it has been slow to address certain issues and there are also concerns about programming continuing in certain directions without sufficiently rigorous analysis that confirms the appropriateness of those directions. This is further elaborated in section 5.1 below. 4.2 To government strategy and priorities 70. FAO is working with three different government authorities in Somalia: in Somaliland, Puntland and Mogadishu. FAO has played a constructive role in the development of government policy and recognises the policy and strategy of the three Administrations. In evaluating how relevant FAO’s programme has been to government strategy and priorities, the team has reviewed alignment with the following government policy and planning documents: in particular Somaliland’s National Development Plan (NDP) for 2012 to 2016, the Puntland five-year development plan from 2007 to 2011, and joint government-UN planning documents (reviewed in section 4.3 below). The team also interviewed senior government officials for their feedback.

71. A strong indication of FAO Somalia’s alignment with government strategy is its investment in Interim Assistance Plans (IAPs) for Somaliland and Puntland in 2012, in which FAO outlines its priorities and plans for working in both regions, in Somaliland until 2014 and in Puntland until 2015. The IAPs specifically follow government strategies, and in the case of the Somaliland IAP, it is: ‘embedded in the NDP and UNSAS principles’ (FAO, 2012: 4). For the rest of Somalia, the most relevant strategy and planning documents are those that have been drawn up jointly by government and the international community. See section 4.3 below.

72. In the livestock sector, common components of the livestock strategy of all three administrative areas include: (a) support to meat processors and exporters; and (b) assistance to the Somali Meat Traders Association and public regulatory institutions, including livestock ministries and local governments. FAO’s programme has directly targeted both of these. Indeed, much of this work accorded the respective governments the recognition they have sought and portrayed them as responsible entities engaged in the development process. The Meat Control and Inspection Act (MCIA) that was discussed and ratified by the lower house of parliament and passed to the upper house of parliament in Somaliland and the ratification of the ACT in Puntland indicate the buy-in of these two governments. In 2009 the TFG produced a three point livestock strategy that included (i) curative and preventive animal health services; (ii) linking irrigation initiatives and fodder production to pastoral dairy development and milk marketing in high population centres and (iii) linking better quality animals to both domestic markets in high population centres and to export markets. The focus of FAO’s livestock programme has mostly been in line with point (i). Interviews with officials in all three administrative areas confirmed they are supportive of FAO’s approach, especially when it involves direct support to their respective ministries in terms of training,

20 capacity building and financing, although there was a concern in relation to the Technical Cooperation Programme (TCP) project agreed with the TFG Ministry of Livestock (MOLFR) in late 2011 that Ministry officials had not been sufficiently consulted about the project design and felt unable to influence how it is being implemented although overall they welcomed it. As FAO shifts its decision making to Mogadishu such consultation should become easier and more ‘a matter of course’.

73. FAO’s projects in the crop sector similarly appear to be in line with government priorities and strategies, which are very broad, although some government officials in both Hargeisa and Garowe raised concerns about FAO’s greater focus on high potential irrigated agriculture and lesser focus on rainfed agriculture. This may reflect a general concern in the north that so much of the investment in agriculture has focused on south-central, although according to FSNAU data this is where the majority of cereal farmers are located and this has reflected donor priorities.

74. FAO’s support to the fisheries sector has been very much welcomed by government officials in Puntland, especially the installation of fish processing centres, three of which are on the Puntland side29. FAO’s support to the Ministry of Fisheries in Puntland appears to have contributed to the ministry’s credibility, not least for now having a cadre of trained staff.

75. While FAO’s information projects, FSNAU and SWALIM, have importantly filled a data information gap on food security and natural resources respectively, they are rather differently received by the respective government authorities. SWALIM has developed stronger links with the respective governments in Somaliland and Puntland, has embedded ‘data centres’ within six ministries, and there is evidence of its information and analysis being used by those two government authorities, for example for developing a water strategy for Somaliland. FSNAU has been more oriented to serving the needs of the international community. Although it has engaged recently in capacity-building of ‘focal points’ within the Somaliland and Puntland authorities, it is less embedded in the ministries and has been criticised by government officials in both Hargeisa and Garowe for its aggregated analysis which does not meet their needs, for the inaccessibility of some of the technical analysis, and for the timing of its outputs. The proposed Phase VII of FSNAU, due to start in March 2013, has been designed to address some of these issues with a much stronger presence in Hargeisa, Garowe and Mogadishu. It is worth noting, however, the contribution that FAO/ FSNAU made to the Somali Nutrition Policy 2011-2013 (along with WHO, WFP and UNICEF), now accepted as a national framework across all government authorities. From the limited engagement that the evaluation team had with government in Mogadishu, there was little evidence of much engagement with FSNAU and SWALIM, nor of their respective products being accessed or used by civil servants.

76. So far FAO Somalia’s CFW programme has been developed entirely independently of government without any systematic analysis of the implications of failed formal governance on the programme. This is an important gap that needs to be filled, not least to explore how local and national government structures can be engaged in a future phase.

29 Those same government officers, however, were unhappy with the World Bank’s insistence that the fish processing centres should be located on the Indian Ocean coast (which had been tsunami-affected) rather than the northern coastal areas of Puntland, as proposed by FAO, where most fishing communities are located.

21 4.3 To UN programmes and priorities 77. FAO’s programme appears to have been broadly in line with key strategy and planning documents, including the five-year Somali Reconstruction and Development Programme (RDP), developed by the UN and World Bank in cooperation with the Somali authorities of Somaliland, Puntland and central-south Somalia for 2008-2012, and the United Nations Transition Plan (UNTP) for 2008 to 201030, based on the RDP. Although the premise of both of these documents was achieving political settlement and relative peace in all parts of Somalia, which was not achieved within their respective timeframes, especially in south- central Somalia, FAO’s programme has been broadly in line with two of the three pillars: • strengthening essential basic services and social protection; and • creating an enabling environment for private sector-led growth to expand employment and reduce poverty. The third pillar of ‘deepening peace, improving security and establishing good governance’ is the pillar where least progress was made by the international community, but this is mostly beyond the mandate of FAO.

78. FAO’s new resilience strategy (see section 8.3 below) appears to be very much in line with the successor to the UNTP, the UN Somali Assistance Strategy (UNSAS) of 2011- 2015 (due to be reviewed at the end of the TFG’s term of office)31. Designed with greater flexibility to adapt to the Somalia context, the UNSAS stresses the need for a longer-term commitment to Somalia32 to address underlying and structural causes of poverty and to create a foundation for peace and security through the strengthening of social service delivery and improved livelihoods – all objectives of the new resilience programme.

79. FSNAU has been more relevant and oriented to the UN and to the wider international community than to the needs of government authorities. It plays a key role in informing the international community about food security conditions in Somalia, especially as a form of early warning. Its seasonal and rapid assessments inform the strategy for the CAP each year, as well as planning and proposal writing by many individual agencies, whether UN, donor agencies or NGOs, although as mentioned above it has been less relevant in providing analysis for long-term food security planning because of the lack of trend analysis. Donor feedback on the value of the FSNAU analysis was especially positive: ‘this is the only reliable source of information’ ‘it is extraordinarily useful and well-used’ ‘a sense of robust information’ Awareness of SWALIM products, on the other hand, is surprisingly patchy within the international community and is not widespread despite its potential usefulness. The recent Technical Review of SWALIM’s performance in reaching its target audience was critical of SWALIM’s dissemination and communications work, especially in Nairobi (Wafula, 2012).

30 This was structured around five strategic outcomes: (i) key federal, Somaliland and Puntland institutions administer and manage core government functions more effectively; (ii) local governance contributes to peace and equitable priority service delivery in selected locations; (iii) improved security and protection under the law for all; (iv) children, youth and vulnerable groups have increased, more equitable access to quality education and health services; and (v) vulnerable and marginalised groups have improved sustainable food security and economic opportunities. 31 The three UNSAS outcomes link to the MDGs and mirror the RDP pillars: (i) social services; (ii) poverty reduction and livelihoods; and (iii) good governance and human security 32 In accordance with the ninth Principle for Good International Engagement in Fragile States and Situations

22 4.4 To FAO’s corporate framework 80. Although FAO’s new corporate Strategic Framework was developed only in 2012 and will be implemented staring from the next biennium 2014/15, the evaluation team has used this as a point of reference as well as the previous corporate Strategic Framework (2010- 19), partly because the FAO Somalia programme has been forward-looking and is in line with many of the paradigmatic shifts in the new Strategic Framework. Overall, the FAO Somalia programme is well-aligned to both FAO’s previous and new Strategic Framework in addressing the main challenges it identifies as well as in meeting its strategic objectives (See FAO Council, 2012). Parts of the FAO Somalia programme directly relate to the new Strategic Objective 1: ‘eradicating hunger, food insecurity and malnutrition’, for example FSNAU’s work and the CFW interventions, although this has been an elusive goal in the Somalia context of protracted conflict and humanitarian crises. The major part of FAO Somalia’s programme, its interventions in livestock, crops and fisheries contribute to the new Strategic Objective 2: ‘increasing production in agriculture, fisheries and forestry in an economic, social and environmentally sustainable manner’, although insecurity and lack of access as well as short-term emergency funding mean that its ability to focus on the sustainability of its work has been limited. FAO Somalia’s livestock and meat marketing programme addresses the priorities identified for Africa, for example in supporting market- oriented infrastructure, strengthening sanitary measures and food safety systems to improve trade, and developing and implementing commercial policies and regulatory frameworks. These are in line with the new Strategic Objective 4, ‘enabling more inclusive and efficient agricultural and food systems at local, national, regional and international levels’.

81. FAO Somalia’s information projects, FSNAU and SWALIM are in line with one of FAO’s core functions identified in the corporate strategic framework: ‘assemble, analyze, monitor and improve access to data and information, in areas related to FAO’s mandate, including global and regional trends and perspectives and associated responses by governments and other stakeholders (e.g. policies, legislation and actions)’ (FAO Council, 2012: 17), contributing to evidence-based decision-making.

82. FAO Somalia’s new resilience progamme is in line with the new Strategic Objective 5: ‘increase the resilience of rural livelihoods to threats and crises’, and could potentially play a leadership role for the organisation (and beyond) in piloting ways of achieving this objective. In terms of leadership more broadly, FAO Somalia offers an interesting and important model for the wider organisation of how it can ‘play a leading role when activities are linked to its mandate’ (FAO Council, 2012: 17).

83. There are also gaps or weaknesses where FAO Somalia has not yet contributed to the organisation’s strategic objectives, for example in addressing natural resource management and land degradation, part of strategic objective 2 in the new framework, more fully articulated in the previous strategic framework as Strategic Objective F: sustainable management of land, water and genetic resources and improved responses to global environmental challenges affecting food and agriculture. (See FAO, 2009). Although SWALIM has provided valuable information that could contribute to this objective, FAO Somalia has not directly engaged in programming in this area. In the livestock sector, FAO Somalia has also been weak in ‘basing policy and practice on timely and reliable information’ (FAO, 2009:20) – see section 5.1 below – although the proposed programme approach and the resilience strategy should improve this. Commitment to gender equity is most clearly stated in FAO’s previous corporate strategy as Strategic Objective K: ‘gender equity in

23 access to resources, goods, services and decision-making in the rural areas’. As described in section 9.1 below, FAO Somalia has done little to address this objective.

5. Programme design and effectiveness 5.1 Programme design and implementation to achieve results

5.1.1 Outline 84. This section explores the effectiveness of the design and implementation of FAO’s programme, particularly in terms of: (i) the extent to which they are based on consultation with key stakeholders; this is key to ensuring interventions are appropriate to the context at the micro- level; (ii) the assumptions and analysis on which individual projects have been based and therefore their logic to achieve desired results; (iii) their use of the logframe; (iv) the use of normative products; (v) achieving results.

The first sub-sections review aspects of programme design, although these also impacted on implementation and hence on overall effectiveness. This section ends by identifying gaps that need to be filled to enhance effectiveness.

5.1.2 Consultation with stakeholders 85. FAO projects have a mixed record in terms of consultation with key stakeholders. Some projects have an excellent record, for example the project designed to address the legal, regulatory and institutional framework for meat marketing and to promote the domestic meat market (project OSRO/SOM/608/EC), which consciously adopted a participatory approach to consensus-building amongst relevant stakeholders, including other UN bodies and NGOs. The success of this process is one of the major strengths of the project. Another positive example is the consultative and participatory process that FAO Somalia engaged in, in formulating its 2010-2015 strategic plan. The process was managed by external facilitators and participants were given space and time to articulate their views. This process was often referred to by government officials interviewed by the evaluation team, particularly in Somaliland and Puntland, who had appreciated being part of it and the openness with which FAO had conducted it.

86. Where consultation has been weakest is at the community level. There are mitigating circumstances in parts of the country where access is constrained; nevertheless, more could have been done, for example using participatory methods. This appears to have been a design flaw in the CFW programme. Although project documents generally mention the need to involve communities in the design phase, fieldwork in Gedo revealed that this had not been done systematically and there is no reason to believe that stakeholders were more involved elsewhere, especially in more remote or insecure areas. Apart from recommending that local communities be consulted, project documents do not indicate how this should be done in practice, nor how power relations within communities and between different groups should be addressed to ensure the most vulnerable and marginalised voices are heard. In the livestock sector projects appear to have been designed with little or no direct consultation with rural communities and livestock owners. In the crop sector, FAO’s NGO implementing partners score themselves quite highly in terms of their own consultation with local communities although FAO staff are more sceptical of the extent to which there has been real community consultation. Nevertheless, the relationship between NGOs and the communities

25 grew with successive projects, creating better consultation and understanding in both directions. An unfortunate example of the consequences of lack of community consultation can be found in the cereal storage project (OSRO/SOM/811/EC) which provided metal silos of 250kg, way below the capacity of the commercial farms for which they were intended (which store 1-2-3 tonnes underground)33.

87. In the fisheries sector there appears to have been a better record of community consultation in Puntland. Indeed, there have been a range of surveys and appraisals exploring issues such as the constraints facing fishermen associations and the training and inputs needed for fish handling and processing operations.

88. The design of some projects has been based on thorough analysis and survey work. For example, the project on improving the sustainable use of plant genetic resources in Somalia (OSRO/SOM/005/EC) built on the earlier phase (OSRO/SOM/611/EC) during which a market survey had been carried out showing that 60% of farmers were interested in buying commercial seeds at reasonable prices. This informed project OSRO/SOM/005/EC and its private sector focus, thus expanding access by private seed producers to improved germplasm and supporting a private sector owned and managed system of seed multiplication, processing and marketing for commercial crops. Similarly, in south-central Somalia, interventions to support the meat trade were based on a survey which explored the poor status of slaughter facilities in the region, resulting in the rehabilitation and construction of slabs, the production of meat inspection manuals and the training of meat inspectors and slaughterhouse workers. But this has not always been the case and the section below on ‘achieving results’ reveals how design flaws have impeded performance of some projects.

89. FAO’s information projects, FSNAU and SWALIM, have been slow in carrying out formal surveys of their users’ needs despite providing a public information service. FSNAU has made most progress, carrying out a user survey in 2010 that fed into and informed a Communications Strategy formulated for 2011 to 2013. The user survey recorded positive feedback from FSNAU’s users: over 90% of respondents described the FSNAU information as ‘very useful’ and the other 10% described it as ‘useful’. The survey also provided useful pointers to how FSNAU information could be made more accessible and relevant to users’ needs, particularly to non-technical users. SWALIM has not carried out a user survey but instead commissioned a Technical Review in 2012 of its performance in reaching its target group (although this appears to have been done without visiting Somalia) (Wafula, 2012)34. The Technical Review raised concerns that SWALIM had not carried out a clear stakeholder analysis that mapped the information needs of users, and that the project had not actively interacted with its users to get feedback on the relevance and usability of SWALIM’s products. In the next phase for SWALIM, phase 5 which will begin in 2013, development of an information/ communication management strategy is planned, as well as more active promotion and dissemination of SWALIM products. To achieve this will require a shift in mindset and new skillsets to the technical skillsets that have dominated SWALIM so far.

33 The initial decision to provide metal silos from underground stores was predicated on a) the high cost of an inappropriate style of underground store improvement developed by the NGO; b) underestimation of normal yields reported by farmers and FSNAU. 34 The EU Delegation had raised concerns about SWALIM’s communication and marketing problems

26 5.1.3 Issues re data and analysis that underpin programme design and implementation 90. Both project design and implementation have suffered from a lack of outcome and impact monitoring across the FAO Somalia programme. (See also section 5.2 below on monitoring and evaluation and the implications for accountability). While this may be partly a consequence of remote management, especially in south-central Somalia where there has been a focus on delivery and outputs, this justification does not hold across the programme.

91. In the livestock sector, approximately 80% of FAO’s expenditure has been allocated to animal health initiatives during the period 2007 to 2012, but animal health monitoring indicators primarily consist of numbers of animals treated or vaccinated by owner and location and little or no effort has been made to build the capacity of FAO’s partners to assess outcome or impact. Whilst this is challenging to do in the Somalia context, there are participatory appraisal and assessment techniques that could have been utilised in some areas. Until FAO builds community based monitoring into all programming it is not possible to say whether it has adopted the most effective approach to protecting, building and rebuilding the livelihood assets of pastoralists and agro-pastoralists in Somalia.

92. In the crop sector a fundamental flaw in project design, especially from 2011 onwards, is the ending of Letters of Agreement (LOA) between FAO and its implementing partners immediately after the work, or delivery of inputs, has been completed, and before the harvest period, therefore precluding responsibility to review outcomes. This may be a well- intentioned administrative procedure to close financial commitments, but it precludes any attempt to monitor outcome of the interventions at harvest-time and therefore to know whether the project worked and to learn from the intervention for the future. With its focus on high potential agricultural areas, FAO Somalia has invested substantially in canal refurbishment, but once again there is little evidence that it has seriously studied the impact of this work, including who benefits, apart from asking NGO partners to measure yields and area under irrigation.

93. This lack of rigorous analysis to underpin FAO’s programming choices and approaches means that work in a particular sector continues without sufficient knowledge and analysis of the extent to which it has been effective and therefore whether it is the most appropriate way to intervene. Based on the available evidence in Somalia and on research from elsewhere in the region, the evaluation team questions the appropriateness and therefore effectiveness of some of FAO’s programming strategies as follows:

1) On the livestock side, whilst FAO’s animal health work has probably slowed the increasing poverty and vulnerability of livestock owners in Somalia, it has not addressed the underlying causes of vulnerability. Indeed, for this reason there is a risk that it has had a comparatively larger positive impact for wealthier livestock owning households. There is no evidence that livestock owners with higher numbers of healthy animals support poorer more vulnerable livestock owners. Research of commercialised livestock systems in Somali areas in neighbouring countries suggest a growing wealth divide as richer members of the community capture markets and resources (Catley et al 2012). Information provided during interviews carried out by the evaluation team suggested that there is a significant problem of range degradation, with key resource areas being fenced off by wealthy households and returning diaspora through ‘investment licences’. Improved understanding of these trends within Somalia and of how different wealth groups within a community wish to utilise

27 their livestock (or lack of it) is crucial to developing effective short and long term livestock interventions that build the resilience of vulnerable communities to future shocks.

2) In terms of livestock and meat marketing, FAO Somalia has focused on value addition practices from the consumption and not the production end of the chain, but value addition begins before and not after the slaughter of animals. In other words, the animals must be of superior quality to produce good quality meat – value can be added at the production end of the chain. As indicated in one of the FSNAU reports, one of the problems with the export abattoirs was the low quality of animals. High rejection rates were also experienced in Bosasso port because of the failure to meet the 28 kg live-weight requirements of shoats by the Islamic Development Bank. The evaluation team therefore judges the recommendations made by consultants to make Somali meat internationally competitive through packaging and branding to have been neither relevant nor practical. Instead, and drawing on evidence of declining livestock numbers in Somalia (Githinji and Wanyama, 2010; FSNAU Deyr report, 2010/11; and the view of exporters), rehabilitation of rangelands, fodder and concentrate feed production and increasing milk and meat productivity per animal through value addition should have been given much greater emphasis.

3) In the crop sector, FAO Somalia’s programmes have relied heavily on FSNAU data on crop production that has in turn fed into the FSNAU’s livelihoods analysis. Yet there is growing evidence that FSNAU’s crop production data, based on oral exchanges with beneficiaries, have, for years, been seriously underestimated35. Key informants at local level have vested interests in lowering production estimates, to ensure communities continue to receive food aid and other relief assistance and to reduce tax liability. Recent exercises using more objective crop assessment methods (ie the pictorial evaluation tool, PET) have shown yields that may be five to ten times higher than FSNAU estimates36. FSNAU’s under-estimations may have affected assessments of the outcome of agricultural projects in terms of crop yields.

4) In the CFW programme the focus on process – monitoring delivery and outputs – has sidelined efforts to monitor and understand results and impact. To some extent this has been a consequence of the mode of remote management that FAO has been forced to adopt in south-central. However, the field study carried out for this evaluation showed how it is possible to evaluate results and impact using participatory methods, even where access is constrained.

5.1.4 Evidence of project logic and the use of logical frameworks 94. More rigorous analysis, as indicated above, could clearly strengthen the logic underpinning some of FAO Somalia’s work. In some instances it may be appropriate for the logic to evolve, for example in the CFW programme: the urgency of scaling up in response to

35 SWALIM data on cultivable area based on remote sensing also indicates FSNAU underestimation 36 In a workshop held in Mogadishu in December 2012, with maize growers and traders from south-central Somalia when the discrepancy in results was presented, it became clear that yields of irrigated maize may be three to five times as high as has been reported by FSNAU.

28 the 2011/12 famine when there was limited information from the ground is a case in point, but a logical model must then be built retrospectively, based on monitoring, especially outcome monitoring. The CFW programme has been slow to develop its logical model and one does still not exist.

95. The use of logframes is standard for most FAO projects. The evaluation team notes a tendency (not unique to FAO) to aspire to ambitious goals and objectives in the logframe that are unlikely to be attainable in the project time-frame and especially in the challenging and unpredictable context of Somalia. The meat marketing project (OSRO/SOM/608/EC), for example, aimed to develop and enforce a regulatory framework for meat safety and quality, strengthen public institutions to address market access and consumer safety of Somali domestic and export livestock products, and ensure that internationally recognised food safety and quality assurance practices were applied and monitored at all levels of the production/ export chain so that Somali meat and meat products were better position in the international market. This was to be achieved in 30 months, an almost impossible target in the context of Somalia with its weak systems of formal governance. Over-ambitious goals were noted in some other market-oriented projects, for example related to crop markets (OSRO/SOM/125/EC). With almost no outcome monitoring FAO have mostly relied on process indicators when preparing reports. The authors of these reports generally assume that an item or service, if delivered, has been effective and benefited the recipient and community. Reports rarely delved into demonstrating attribution for success or the difficulties and failings of a project. There may be an important learning here, not just for FAO, but also for donors. The latter, removed from the realities on the ground, may encourage and reward over- ambitious goals in their funding decisions. Ways of encouraging a more open and learning- oriented culture, in which learning from mistakes is incentivised, should be fostered

5.1.5 Use of normative products 96. FAO Somalia’s use of normative products and guidance is mixed. In terms of food security information, the experience of FSNAU has fed into, and in turn been informed by the FAO/ WFP Joint Strategy on Information Systems for Food & Nutrition Security 2012-2017 which aims to reorient investment toward long-term sustainable information systems for food & nutrition security (ISFNS) led by national and regional institutions. Design of the next phase of the FSNAU project takes this into account as well as encouraging stronger collaboration between WFP and FAO in running their respective food security information systems.

97. In contrast, FAO Somalia has made minimal use of LEGS, published in 2009, in its livestock work. The minimum standards common to all livestock interventions detailed in LEGS include community participation and monitoring and evaluation techniques designed to regularly assess the impact on livelihoods. LEGS advocates the use of a tool known as the Participatory Response Identification Matrix which could have been utilised in the Somali context to improve project design and effectiveness. Another example of where LEGS could have helped FAO improve design relates to the delayed 2011 drought-related livestock feed provision interventions. LEGS advocates realistic assessment of what is feasible in terms of the introduction of new fodder varieties and feed technologies, like the urea treatment of stova. Analysis of the type and quantity of livestock provided to IDPs and the poor could also have been more effectively determined through community consultation and lessons from previous FAO restocking initiatives.

29 98. In the other direction, the role of FSNAU in piloting and developing the Integrated Phase Classification system (IPC), now used internationally by many different agencies, was a significant contribution by FAO Somalia to normative work.

99. There is no evidence that FAO Somalia has taken on board FAO’s new corporate policy on gender. There was little awareness of the policy amongst staff and little or no reference made to it. Although more recent, there is similarly no evidence that FAO Somalia has taken on board FAO’s new corporate Nutrition Strategy.

5.1.6 Achieving results 100. Some of FAO’s projects have achieved impressive results, especially considering the challenging context in which they have been implemented. The seed project (OSRO/SOM/005/EC) is an example, and could be considered a flagship project for FAO Somalia for having achieved its goals and supported the creation of two privately owned and managed commercial seed supply companies in Somalia which are now major suppliers to farmers and also to international agencies for their emergency seed distribution programmes. Some of the fisheries interventions (eg GCP /SOM/043/SPA) have also had impressive results according to key informants interviewed, for example through the provision of relevant training which may have contributed to increased income of fishing households (although it was not possible for the evaluation team to triangulate this information through visits to the targeted fishing communities and attribution may be more complex to determine). Where outcome monitoring has been carried out for non-veterinary livestock interventions the outcomes have been shown to be positive. For example a post-distribution survey of a restocking initiative between 2009 and 2010 (OSRO/SOM/902/EC) showed that children in restocked households had improved access to milk. In addition, although not guided by LEGS, FAO’s non veterinary initiatives of restocking, fodder production, improved water access, policy development and institutional capacity building are all technical interventions listed as effective within the handbook for Livestock Emergency Guidelines and Standards (LEGS).

101. In terms of FAO’s projects in the livestock and meat markets in Somalia, some have worked well, such as the construction of the livestock market in Hargeisa town which is now securely enclosed and provided with infrastructure (OSRO/SOM/007/UK). This has been valued by stakeholders because it provides security for stock owners and traders and has generated a sense of ownership amongst livestock producers, middlemen and traders. But where this project has struggled is in terms of management of the market. Contrary to the plan, management has not yet been handed over to the private sector and is currently in the hands of the mayor who estimates he is making substantial losses due to fraud from service fee collectors. See Section 9.2.3 below on the learning from this project about the process of setting up public-private partnerships (PPPs). FAO’s rehabilitation of the meat market in Boroma also appears to have worked well (OSRO/SOM/007/UK). The meat vendors in the market (almost all women) have been trained in meat hygiene, appear to have a strong sense of ownership of their workplace and have maintained it as a clean, hygienic, spacious and well-organised environment.

102. FAO’s projects to build slaughterhouses have been less successful and offer important learning (OSRO/SOM/808/WBK). Two constructed slaughterhouses in Bosasso and Boroma were built with serious design faults, including construction of the entrance doors to the abattoirs which are too small for camels and cattle. At Boroma slaughterhouse the camels and cattle are now slaughtered outside in the dust, defying the original objectives

30 of the project, and the meat quarters are taken inside for further cutting. The water and drainage systems are inadequate and are now blocked and overflowing. The smell is so bad that the facility may soon have to be abandoned. The Bosasso slaughterhouse has not even been operational since it was completed. In both cases management issues have compromised performance and are linked to design flaws. Officials in both towns state that the actual investments made in the abattoirs was much less than what they expected from the inception meeting and that the abattoirs are not compliant with the original specifications. This is causing problems in transferring the management of the slaughterhouses to the private sector. Again, there are important learnings for PPPs. Some learning already seems to have been applied to a new project (OSRO/SOM/007/UK which will continue under OSRO/SOM/203/UK) which began in August 2012 and involves the construction of a slaughterhouse in Burao. In this case FAO Somalia has employed a resident engineer to supervise the construction work on site, a major improvement over previous arrangements.

103. The creation of the Somali Meat Development Association (SOMDA) and training of 40 individuals (mostly young women) in meat value addition to produce muqmad (a locally popular dried meat soaked with ghee) and in soap making and bone crafting has been well-received and is an innovative project using waste products for soap making (bone marrow) and bone crafts. However, it appears to have been designed and implemented without a thorough feasibility study or market analysis. Although the packaged muqmad is selling at high prices and there are plans to export this product to the Somali diaspora, the market opportunities for soap and bone crafts have not yet been proven. There is an important learning here, to ensure that a clear and robust business case can be made before investing in value addition production.

Evidence of results of the CFW programme are covered in section 5.4 below.

5.1.7 Gaps that once addressed could enhance effectiveness 104. A significant gap has been the lack of connection between FAO Somalia’s agriculture and livestock work. This particularly relates to fodder production which could play a key role in adding value at the production end of the livestock market chain through the use of crop stovers, oil cake and groundnut leaves for improved animal conditioning. Where forage production has been promoted in isolation in areas of food insecurity, there seems to have been a missed opportunity for dual purpose cropping, for example through over-sowing sorghum and using thinnings and stover for livestock37. There is now evidence of the agriculture and livestock teams working together more effectively; this collaboration should be encouraged and strengthened.

105. A second gap relates to the pressure on farmers to sell their produce soon after harvest because of immediate cash needs, as cited in FAO project proposals for projects OSRO/SOM/611/EC and OSRO/SOM/125/EC. This suggests a possible solution of providing short-term loans so that farmers can wait to sell their produce until prices are higher. The issue is beginning to be addressed by loans of agricultural inputs in-kind, paid back at harvest time. This constraint should be further considered in the new resilience programming.

37 Where this has been practised by agro-pastoralists in Haraghe in east Ethiopia it has produced more than 3 tonnes of stover per ha for animals and just under 2 tonnes of cereals per ha for human consumption. The benefits of this practice have been noted regularly in FAO/ WFP CFSAMs between 1999 and 2007.

106. Third, although milk is an essential commodity and recent research shows the importance of milk to the food security and well-being of children38 – indeed, this is not ‘new’ knowledge – this has not yet been given much attention in FAO Somalia’s programme. It is worth noting that the CFW study found that access to cash frequently allowed households to spare some milk for children’s consumption rather than sell it, but there is no indication that FAO Somalia is aware of this important impact of CFW on nutrition. As well as milk, supporting agricultural production to promote a diverse diet is also important. 5.2 Effectiveness in responding to humanitarian crises, with a particular focus on the famine of 2011 107. Hosting FSNAU, the main source of early warning and food security information on Somalia, means that FAO Somalia carries a particular responsibility for alerting the relevant actors to an impending crisis. The failure of early warning to elicit a timely response to deteriorating food insecurity in 2010/11, resulting in the declaration of famine on 20th July 2011, has been the subject of much research, analysis, and soul-searching. FAO Somalia and FSNAU senior managers have been active participants in these discussions39.

108. The widespread consensus, with which this evaluation concurs, is that ‘in the case of Somalia 2011 early warning information was adequate, timely, credible and pointed to clear, actionable interventions’ (Haan et al, 2012:77). The early warning was provided almost exclusively by FSNAU in collaboration with FEWSNET. It provided ample notice of deteriorating food security, providing a clear ‘famine’ warning in March 2011. The chronology of early warning is now well documented, beginning as early as August 2010 (see Hillbrunner and Moloney, 2012). The real-time evaluation (RTE) commissioned by the IASC comments that: ‘Despite the difficulties of obtaining information, Somalia has one of the most sophisticated early warning systems in Africa’ (Darcy et al, 2012: 28). But this system failed to trigger a timely response. The IASC RTE reports that the early warnings were ‘registered only to a limited extent’, and the financial request in the 2011 CAP was ‘actually lower than the previous year based less on anticipated needs that on what was felt to be a ‘realistic’ expectation of donor response at the time’ (ibid, 2012: 29), a fatal mistake that has been made in previous famine situations40. Global acute malnutrition (GAM) of 30 to 40% was registered in southern Somalia between late June and early July 2011, and there were tens of thousands of excess deaths, peaking in July 2011 (Hillbrunner and Moloney, 2012; 41 Checchi and Courtland Robinson, 2013)) .

109. Could anything more have been done by FAO to trigger a response, and indeed, did FAO Somalia heed the early warnings that FSNAU was providing with its own response? Hillbruner and Moloney (2012) have reflected on how the analysis could have been improved, for example through better understanding of the impact of WFP’s withdrawal of food aid and stronger analysis of poor households’ food insecurity in surplus-producing areas. The analysis should probably have extended to understanding the impact of the

38 See ‘Milk Matters’, Tufts/ FIC http://sites.tufts.edu/feinstein/program/milk-matters 39 See, for example, the online Special Issue of Global Food Security, dedicated to the Somalia famine of 2011- 12 http://www.sciencedirect.com/science?_ob=PublicationURL&_hubEid=1-s2.0- S2211912412X00026&_cid=280999&_pubType=J&_auth=y&_acct=C000228598&_version=1&_urlVersion= 0&_userid=10&md5=bf85cbe75e5db27b5f489ab15a2870dd 40 For example, the famine in Bahr El Ghazal in 1998 – see Buchanan-Smith et al (1999) ‘Evaluation of Danish Humanitarian Assistance to Sudan. 1992-1998. Final Report’, ODI, October 41 Importantly, FSNAU has commissioned a retrospective study on excess mortality during the 2011 famine.

32 withdrawal of other humanitarian agencies as well as WFP. A recent study co-commissioned by FSNAU indicates that increases in death and malnutrition among children under 5 started as early as October 2010 (Checchi and Courtland Robinson, 2013). But this additional analysis would not have made the difference. The warnings were clear. This shifts the spotlight to communication and advocacy. The shortcomings of the international humanitarian system in its systemic inability to respond to genuine early warning have been well-documented42. In the Somalia case this was exacerbated by the normalisation of crisis indicators over more than a decade, the remoteness of key decision-makers in Nairobi from the reality on the ground and the hostile political context affecting the aid policies of some donors, as well as competing early warning from other drought-affected countries in the Horn of Africa in 2011 (Lautze et al, 2012). All of this implies that enormous advocacy efforts would have been required with other UN actors and especially with donors to have mobilised an earlier response, probably involving the media. It is in meeting this challenge that FAO Somalia and FAO at the corporate level fell short. Advocacy and communication management have not been FAO’s strongest features over the years, although FAO Somalia has put a lot of effort into communications management as indicated in section 6.7 below. Ensuring that early warning expertise is matched and supported with appropriate advocacy skills is an important learning from this episode. When the famine was finally declared in July 2011, indicating that 2.8 million people were in a situation of acute humanitarian crisis, the response from donors was very rapid, but it was too late to save many lives.

110. FSNAU played an important role in the design of the international humanitarian response to the famine. Early discussions about the implications of WFP not being able to deliver food aid triggered FSNAU to conduct a market study to establish the feasibility of cash-based responses in February 2011. Jointly with FEWSNET, FSNAU advocated for cash-based responses based on its market analysis. (An evaluation of the unconditional cash and voucher response to the Somali famine is due to be published soon). During the famine, FSNAU continued to make important contributions that informed the cash-based response, producing a number of additional products to meet demand including bi-weekly market updates and data on the cost of a minimum basket of food per region. It also facilitated a round-table discussion between agencies and Somali cereal traders on the possible role of the monetisation of cereals as a market intervention in the response. These were all important and valuable contributions.

111. Despite hosting FSNAU, surprisingly FAO Somalia did not seem to be that much more responsive than other international agencies in taking action in response to the warnings of FSNAU. Although it was active in discussions about cash-based interventions, it did not scale up and launch its emergency interventions until famine in south-central Somalia had been declared. At this point it moved swiftly but this was already too late. FAO Somalia’s slow response cannot be entirely blamed on the slow response of donors. There is no evidence that it was lobbying hard to mobilise funds in order to launch an earlier response. When famine was eventually declared, a Crisis Management Unit (CMU) was established to oversee and manage FAO Somalia’s response, and an appeal document was released.

112. The most significant part of FAO’s eventual famine response was massively scaling- up its CFW programme. But the timing of this, when the population was already weakened by lack of food in parts of south-central Somalia, was not ideal and had consequences for the

42 See Buchanan-Smith and Davies, 1996; Bailey, 2012

33 types of intervention that were appropriate. There was a trade-off in the worst-affected areas between providing humanitarian assistance and building sustainable productive infrastructure that was not always taken into account. For example, building/ rehabilitating water catchments in the pastoral areas of Luuq district in southern Gedo was welcomed by beneficiaries because they were implemented before the rains so that the infrastructure could be of immediate use. But it appears that the number of workers employed was too few in view of the hard physical work and the fact that people had been weakened by lack of food43. This was the feedback from the local community, who greatly welcomed the CFW intervention but commented that water catchment rehabilitation required a bigger workforce. This kind of trade-off could have been more carefully assessed.

113. Overall, the Gedo study concluded that the CFW programme had been successful in improving access to food, its main objective, as households were able to buy more and better- quality food. Table 1 presents the combined impact, on households and in terms of creating infrastructure. In terms of infrastructure the impact was reduced by the fact that the intervention focused on canal rehabilitation. Sustainable changes in agricultural productivity require broader interventions, and CFW participants interviewed for the study frequently requested additional inputs such as fertiliser, support with land preparation, water pumps and animal health interventions.

Table 1: Positive impact of CFW, from Gedo study Impact and use of cash at household level Impact of infrastructure • Increased access to food basket (more and • Increased agriculture production with impact better quality food) and to household items on household income and food security • Increased health and animal health • Better access to water reducing the demands expenditure on women’s time to collect water • Asset protection and loan repayment • Savings on having to buy water • Productive small-scale investments, especially for women • Expenses related to social obligations, including social support • Improved access to loans • Multiplier effects on the broader economy

114. Although targeting criteria were set by FAO and implementing NGOs in terms of who should participate in the CFW programme, benefits were often redistributed at village level according to local social norms. On the one hand this meant that a higher number of households were reached than the number initially targeted, according to locally accepted criteria of justice and equity, but on the other hand it resulted in the exclusion of groups and individuals who were vulnerable but marginalised in the community, for example the Bantu community in Degreeb village, and enabled the capture of benefits by elites. This caused some social resentment within the community44. Overall, however, the CFW study found that this mode of implementation had been appreciated over food relief, illustrated by these two quotes: ‘cash is multi-purpose’, and ‘what will you do with the sugar you are given if you don’t have the tea or the milk?’ In the circumstances this was appropriate as markets were

43 This feedback is in spite of work load being reduced to one third compared with usual work standards ie beneficiaries had to move 1/3 of the soil normally required to be moved for the same wage. 44 For example, in one village the presence of CFW made it more difficult for non beneficiaries to access food on loan, because more cash was circulating, disincentivizing shop owners to sell on credit. In this village one of the non beneficiaries commented: ‘I am glad the CFW is over and that we are now all equal again’

34 functioning and it gave beneficiaries the choice of how to spend or use the cash with some using it productively for investment. From a gender perspective, it is acceptable, indeed appropriate for Somali women to handle small amounts of cash. Thus they remained in charge of the money they earned through CFW and could choose how to use it, for their own or for their households’ benefit. Overall, the field study in Gedo Region found that female beneficiaries had greatly benefited from participating in the CFW projects. Some female beneficiaries saved their earnings through involvement in merry-go-round groups or invested45 some of it. Although the overall impact of CFW projects on women may be short- lived, a number of female beneficiaries interviewed said they had felt empowered by gaining purchasing power46.

115. In terms of other interventions, FAO implemented an emergency animal health programme in response to the drought in 2011, providing livestock treatments and vaccination against PPR and contagious caprine pleuro pneumonia (CCPP) across the three regions of Somalia. The 2011/12 animal health response was a remarkable logistical achievement. It scaled up quickly, built on a solid base of FAO experience and contacts: over 2.5 million livestock were treated in 2011 alone, reaching almost 60,000 households, most in south-central Somalia. Rapid scale-up was possible by adopting a broad targeting approach and foregoing cost recovery from wealthier livestock owners. FAO was also innovative in the use of collapsible water tanks to assist the mustering of large numbers of stock for treatment and vaccination. Although the final reports on these emergency livestock interventions have not yet been produced, interviews with partners and a review of process indicators collected by the FAO monitoring and evaluation team suggest that FAO’s 2011/12 livestock response was well-delivered. A review of FAO’s monitoring figures of the average number of livestock vaccinated per household, assessed against FSNAU’s wealth ranking analysis suggests that FAO targeting has probably captured large numbers of poorer households47. The targeting approach used in 2011 was a much simplified version of the targeting approach it had used in a smaller but similar intervention in 2008. This simplified approach was appropriate to the urgency and scale of the response in 2011, and to the ready availability of funds. Monitoring the impact of this intervention could provide important guidance for the design of future emergency animal health interventions, especially as it is not clear how PPR is affecting poor pastoralists in Somalia even though the participatory techniques to find this out are readily available. PPR is considered to be endemic in Somalia and causes low level mortality in young stock. Whilst there is considerable evidence of the significant negative livelihoods impact for PPR epizootics (Meyers et al. 2009), there appears to be little evidence of the disease’s impact in endemic situations48.

45 With the cash earned from CfW activities, some female beneficiaries were either able to set up small stall, kiosks to engage in income generating activities or to enhance their existing informal businesses. 46 The recent evaluation of UCTs in Somalia shows that cash contributes favourably to the nutritional status of the household, in particular improving significantly the dietary diversity score. 47 The average number of livestock vaccinated against PPR was 50 shoats in rural areas of south central Somalia and 40 in peri-urban areas of Mogadishu. The average number of animal treatments carried out per household was just over 20. Whilst it is difficult to interpret average figures, FSNAU wealth ranking indicates a typical poor pastoralist household in central Somalia keeps about 60 sheep and goats and a pack camel while the very poor keep about 15 shoats (FSNAU 2011), hence the conclusion that large numbers of poorer households probably benefited from this intervention. 48 Research from neighbouring areas of Ethiopia suggests mortality due to PPR in endemic herds might be 3.3- 4.5% (Catley et al 2009a). This figure could be higher in Somalia due to the constant throughput of traded livestock. FAO Ethiopia has anecdotal information that mortality of young stock due to PPR could be as high as 10%.

116. On the agriculture side, the emergency interventions were essentially ‘off-the-shelf’ solutions, including seed distribution programmes that are appropriate for IDPs returning to their land after losing most of their assets. Such interventions are believed to have been completed within the cost/ time framework as the evaluation team could not identify any no- cost extensions that would mask delays in delivery. As described above, the fact that the seed distributed was mostly locally produced rather than exotic seed from other countries, is a credit to this part of FAO’s longer-term development work. Other inputs provided by FAO included fertiliser, tractor hours, irrigation hours and a limited number of irrigation pumps and integrated pest management.

117. There were serious delays with a couple of proposed emergency interventions. The first was an emergency fodder scheme. Despite this feeding initiative being far less ambitious in terms of numbers of households reached than the animal health intervention, FAO Somalia had failed to implement it by the end of 2012, let alone scale it up. The factors contributing to the failure which provide important learning can be summarized as follows: (i) proposals did not take reasonable account of the significant technical and organizational difficulties of introducing new crops and technologies into an insecure and drought affected environment; (ii) a proposal was made to grow commercial fodder in the dry season (off crop season) without doing a business case or risk analysis; (iii) FAO opted to tender for local partners rather than use INGOs with known experience and contacts in this area.

118. The other part of the emergency project that encountered severe delays was the emergency fisheries project (OSRO/SOM/201/MUL) which intended to support 2000 households in riverine communities in south-central with capacity-building and the provision of fishing and fish-processing sets. Delays in procurement of the fishing sets has postponed the launch of activities to February 2013, thus missing the critical emergency period49.

119. In conclusion FAO Somalia, through FSNAU, played a critical role warning of deteriorating food insecurity through 2010 and early 2011, and warning of the threat of famine from March 2011. FSNAU was at the heart of the decision to declare famine in July 2011. But FAO Somalia was as slow as all other international agencies in responding to its own warning. When famine was finally declared FAO scaled up rapidly and effectively, and most parts of its emergency programme were appropriate and effective, especially in supporting household assets (see Figure 1) although some components have been unacceptably late. 5.3 Coordination 120. The Somalia context poses particular challenges to the effective coordination of aid. The lack of a functioning central government means that there are no mechanisms for monitoring or holding NGOs to account, at least in south-central Somalia although they must register through the Ministry of Interior. The political context in south-central also means that

49 One of the cheapest suppliers was selected in the procurement process, but has since failed to deliver the materials

36 information-sharing can be sensitive, particularly for agencies working in Al Shabaab- controlled areas.

121. During the period covered by this evaluation, there have been two main coordination structures, the Coordination of International Support to Somalia (CISS, led by the UN) concerned with longer-term development and rehabilitation efforts, and the UN-led cluster coordination structure, focused principally on humanitarian programming. As so much international aid to Somalia has been humanitarian, cluster coordination has dominated so far.

122. Initially FAO led the agriculture and livelihoods cluster until this was merged with the food assistance cluster led by WFP in March 2012 to create the food security cluster (FSC). The merger had been encouraged by the UN Humanitarian Coordinator for Somalia and by the Global Food Security Cluster (FSC) in Rome. This has resulted in an extremely large cluster with a membership of more than 500, accounting for more than 50% of CAP funding. It therefore tends to dominate the overall cluster system. Meetings can also be very large, for example up to 100 may attend FSC meetings in Nairobi which can inhibit effective debate and also results in very long meetings. Nevertheless, the merger is generally regarded as a positive development that provides greater coherence to food security interventions and the FSC Secretariat is working to develop a differentiated structure of membership which may help to address the issue of unwieldy meetings.

123. The process of the merger and establishing the FSC were well-planned and implemented with support from the Global FSC. A process of consultation resulted in TOR being drawn up for the Somalia FSC. The FSC is modelled around a national coordination structure in Nairobi and a series of field-level regional FSCs, each with an elected vice- coordinator. This structure is still being rolled out and vice-coordinators were still being elected at the time of the field work for this evaluation. Vice-coordinators are voluntary positions, usually drawn from NGOs. The voluntarism of the role may need to be re-visited over the next year to review whether it is feasible for staff of local NGOs to perform this function effectively without any financial support, although they are provided with training.

124. There has been strong and effective collaboration between WFP and FAO in making the merger work, from the level of the cluster chairs (the heads of WFP and FAO Somalia respectively) who have provided strong leadership in the merger process, to the level of the full-time coordinators appointed by WFP and FAO respectively. Undoubtedly the fact that FAO was already a strong actor in Somalia facilitated both UN agencies coming together as equal partners.

125. The evaluation team carried out participatory visual exercises with FSC members in Nairobi, Hargeisa and Mogadishu in November 2012, whereby they scored the performance of the FSC against its TOR (a simple scale was used: 5 is excellent and 0 is very poor). See Figures 4 and 5 for the results from Nairobi and Hargeisa50. Generally the feedback on FSC performance was more positive in Nairobi, where most scores were above average, compared with Hargeisa where most scores were below average. In Mogadishu it was mixed. This pattern is also repeated in the findings of the performance monitoring survey recently carried

50 This particular exercise was not carried out in Mogadishu as FSC members were drawn from a number of different regional FSCs

37 out by the Global FSC where performance of the national cluster is generally rated more highly than FSC performance at district level in Banadir and Gedo respectively.

126. In the three locations visited by the evaluation team – Hargeisa, Nairobi and Mogadishu – the FSC was most valued for its role in facilitating information sharing between members although members also wanted more information from FAO fed back to FSC members so that it feels more like a two-way flow of information. In Nairobi the FSC was valued for strategic planning and for its role in building technical capacity51. The trainings that the FSC has provided in protection and in using the gender marker were highlighted as particularly important. But the procedures that FSC members must follow to secure funding was seen to be an obstacle to timely response. FSC members in Hargeisa rated the performance of the FSC poorly in terms of both emergency preparedness and timely response in Somaliland. Many of these concerns pre-date the FSC merger and relate to 2011 when there was famine in south-central Somalia and this became the focus of humanitarian efforts. There is a strong feeling amongst local NGOs in Somaliland that the drought and food crisis in the north were overlooked and that the cluster system failed to draw attention to this part of the country.

127. The FSC has a critical role to play in terms of technical support and guidance, and capacity development of NGOs. This role is still being developed, but is clearly much in demand from many FSC members who are seeking support in areas such as proposal writing, monitoring and reporting. FSC members in Hargeisa fed back that current training programmes of 3 to 4 hours are too short for some of the topics being covered. The technical working groups, for example on livestock and agriculture, do not appear to be functioning well within the FSC although the fisheries working group is meeting regularly. These technical working groups should be given urgent attention as this is where more nuanced technical debate and guidance may be possible.

Figure 4: Feedback from FSC members in Nairobi re FSC performance since March 2012

Very poor Excellent

Strategic Planning Average: 2.88

0 1 2 3 4 5

Information Sharing Average: 3.42

0 1 2 3 4 5

Emergency Preparedness Average: 2.40

51 However, the performance of the FSC at national level was regarded as ‘unsatisfactory’ in terms of ‘application and adherence to existing standards and guidelines’ in the performance monitoring survey recently administered by the Global FSC

38 0 1 2 3 4 5

Timely Response Average: 1.52

0 1 2 3 4 5

Addressing disruption of food security services in an emergency Average: 2.58

0 1 2 3 4 5

Improved technical capacity Average: 3.48

0 1 2 3 4 5

Figure 5: Feedback from FSC members in Hargeisa re FSC performance since March 2012

Very poor Excellent

Strategic Planning Average: 2.2

0 1 2 3 4 5

Information Sharing Average: 3.2

0 1 2 3 4 5

Emergency Preparedness Average: 1.80

0 1 2 3 4 5

Timely Response Average: 1.80

0 1 2 3 4 5

39 Addressing disruption of food security service in an emergency Average: 2.00

0 1 2 3 4 5

Improved technical capacity Average: 2.00

0 1 2 3 4 5

128. There are a number of challenges the FSC faces if it is to fulfil its potential:

(1) a general criticism of cluster coordination is the extent to which it is UN-centric52. This has shown up in Somalia in the way that NGOs have sometimes set up their own separate coordination mechanisms, feeling they have little sense of ownership of cluster coordination. For example, the Cash-Based Response Working Group was established in 200853. In 2011 a small group of 4 NGOs (ACF, DRC, Save the Children and Adeso) again self-organised outside the cluster system; during the 2011 famine they set up the Somalia Cash Consortium for humanitarian programming based on unconditional cash transfers. This second example indicates that the clusters were not sufficiently flexible to accommodate new initiatives, nor were they the chosen structure for NGO coordination around cash-based programming. (See also Hedlund et al, 2013, section 9 on coherence and connectedness). Finding ways in which the FSC can be more inclusive so that NGOs have a greater sense of ownership is a challenge for the next few years; (2) the huge membership of the FSC is a continuing challenge to manage. Until the regional FSCs are fully functioning, local NGOs in the field are likely to feel quite remote from the FSC. There is clearly resentment about the extent to which decision-making is currently happening in Nairobi; (3) the FSC currently has a weak relationship with CISS that needs to be strengthened, especially if opportunities open up for more developmental programming in the coming years; (4) there are some important actors in the food security field that are not currently part of the FSC, namely the Organisation of Islamic Cooperation (OIC) and the Turkish government and some of the organisations it is funding, both of which operate mainly at the Mogadishu level rather than in Nairobi. Finding ways of reaching out to these groups will be important for the FSC, especially if the centre of gravity of decision-making in the international aid community moves to Mogadishu;

52 This has been highlighted in global evaluations of the cluster system. See, for example, Stoddard et al, (2007) ‘Cluster Approach Evaluation. Final Report’ commissioned by OCHA; and Streets, et al (2010) ‘Cluster Approach Evaluation’. 53 The CBRWG originally emerged from CISS, but has since evolved into an independent forum on cash programming. See http://www.cashlearning.org/where-we-work/somalia-cash-based-response-working-group

40 (5) a generic challenge facing the cluster system is the fact that the Common Humanitarian Fund (CHF) is administered through the clusters. This inevitably introduces an element of competition between cluster members and can inhibit information sharing. This issue has been recognised at the global level, which is the level at which it will have to be addressed. In Somalia this is also seen to affect which agencies actively engage in the FSC – those most in need of funding – and this is clearly an incentive for FSC membership, seen by many as a retrograde step from the days of the Somalia Aid Coordination Body (SACB).

5.4 Twin-track approach 129. The ‘twin-track approach’ refers to the ability to carry out both development and emergency programming alongside each other, yet connected, a challenge that has long eluded international aid agencies. FAO Somalia’s commitment to a twin-track approach is clear from its strategy documents. The 2011 to 2015 strategy states: • ‘Protecting, promoting and rebuilding livelihoods and achieving food security for the Somalia people should be initiated in parallel with humanitarian responses’ (FAO, 2010:48); and • ‘The principle of building back better calls for a linkage at the early stages of humanitarian responses between short-term humanitarian actions and longer-term development interventions’ (ibid: 3).

130. Institutionally FAO Somalia has been well set up to achieve this when the whole programme was managed by TCE. In this sense it has been ahead of recent corporate efforts towards a more united approach. But how has it played out in practice?

131. On the agriculture side there have been strong links between the development and emergency work. The logic of the development programme, linking canal rehabilitation with improved seed development, improved storage facilities and pest control supports an overall plan to increase food availability. Parts of the emergency programme are connected to the outcomes of the development projects, for example seed distribution whereby locally produced seed can now be used. And the CFW programme shows how objectives can be temporarily shifted during a crisis when relieving food security is the overriding goal, thus switching the focus from the rehabilitation of infrastructure to cash-based employment to increase access to food and other essential services. Flexibility is required but does not fundamentally compromise the development process.

132. In the livestock sector FAO Somalia has tended to depart from its longer-term livestock development strategies when there is an emergency, contrary to the aims of the twin-track approach, although there is one programme (OSRO/SOM/201/MUL) – the emergency programme – that specifically aims to make the twin-track approach a reality through: (1) improved fodder production and distribution; (2) emergency treatment and livestock vaccination; (3) increasing numbers of livestock of vulnerable small-scale herders through redistribution/ restocking; and (4) CFW to rehabilitate and construct productive infrastructure eg water catchments.

133. Some of this programme may have worked to achieve this approach, for example emergency vaccination and treatment of animals may have helped secure live animal exports

41 in 2011/12, but there have also been missed opportunities, for example around disease surveillance. With the significant resources available for the 2011/12 PPR vaccination campaign and a relatively long 6-month planning phase FAO missed an opportunity to gain a better long-term understanding of the epidemiology of PPR in Somalia54 despite the FAO strategy prepared in advance of the campaign, stating that ‘without improved disease surveillance it will be impossible to establish the continuum from emergency project to development project’.

134. There is growing evidence from elsewhere of the limitations of a ‘single issue’ approach to addressing vulnerability of pastoralists and agro-pastoralists, and of the need instead for a multi-pronged approach to managing risk and reducing vulnerability (Devereux, 2006). For FAO Somalia this implies the need for a more systemic approach to realise the twin-track objectives of achieving both developmental and emergency goals, paying more attention to feed and water (highlighted by interviewees for this evaluation as the fundamental problem in Somalia’s dryland livestock systems during drought and recovery periods) as well as animal health. Impact assessments are essential to inform such an approach. This is where there is also an opportunity for greater collaboration between the agriculture and livestock sectors of FAO Somalia to establish the feasibility and profitability of fodder production rather than cereal production on irrigated farms.

135. FSNAU’s analysis is intended to inform both short-term emergency responses and longer-term strategic planning but has been much more oriented to the former. The lack of trend analysis which is critical to understanding how livelihoods and the economy have changed over time impedes longer-term planning, and thus does not really fulfil a twin-track approach.

136. The resilience strategy, discussed below, is an opportunity to fill some of these gaps and to develop more systemic thinking and programming to take the twin track approach to the next level. 5.5 Advocacy and influencing 137. There are two spheres in which FAO Somalia has conducted most of its advocacy and influencing: with the respective government authorities in Somalia and within the international aid community.

138. During the 2007 to 2012 period, most of FAO Somalia’s advocacy and influencing efforts with government have focussed on the meat market sector where it successfully advocated the respective governments of Somaliland and Puntland to formulate and ratify the Meat Inspection and Control Acts, and has subsequently supported the development of their meat markets as described above. FAO’s success with the Meat Acts has encouraged Veterinaires Sans Frontières Germany (VSFG) to embark on a similar process to formulate Milk Acts in Somaliland and Puntland.

54 No syndromic surveillance for PPR appears to have been carried out through the vaccination campaign. There is no indication from vaccination reports or LoAs that the campaign targeted recent and current outbreak areas, predicted high-risk areas (trade routes, markets, border crossing points etc). Sero-surveillance carried out was not linked to any study of declining herd immunity in vaccinated and unvaccinated groups of sheep and goats. The ‘before and after’ sero-surveillance carried out is of doubtful use due to delays in the arrival of sampling equipment and analysis of samples.

42 139. In the fisheries sector FAO has successfully advocated for, and supported the fishing regulation signed by the parliament and President of Puntland. This is an important first step although there are now challenges of implementation.

140. FAO provided technical and administrative support to the TFG Ministry of Livestock, Forestry and Range (MoLFR) from 2007 to 2008 to convene animal health policy workshops and in 2009 the TFG produced its own livestock sector strategy. FAO Somalia has also started to assist the respective Somali administrations to build policy and strategy based on up to date livestock information, exploring the feasibility of using aerial photographs for a livestock census although it appears that other approaches would be more accurate. As mentioned above (section 4.1.2), information provided by SWALIM has fed into the new water strategy developed by the authorities in Somaliland.

141. Ways of building on the work that FAO Somalia has supported so far are spelt out in the institutional capacity development programme of the revised strategy (currently in draft), whereby FAO Somalia aims to further support the respective government authorities to set up policy and regulatory units to cover agriculture, livestock, fisheries and natural resources. These will be important contributions that play to the comparative advantage of FAO’s technical strengths.

142. In terms of advocacy and influencing within the international community, FAO Somalia has been most active and influential in relation to the new resilience thinking that offers a paradigm shift that has gained wide popularity within the aid community. The OiC has led this work and is contributing to current debates on resilience programming related to Somalia, the Horn of Africa, and beyond. This is further discussed in section 8.3 below. On the other hand, FAO Somalia’s shortcomings in failing to advocate, or play a leadership role in scaling up the humanitarian response as food security indicators deteriorated rapidly in 2011, shows a weakness in advocacy, as described in section 5.2 above.

143. At the programme level there is an interesting example of FAO’s pioneering work being picked up by another agency. VSF Germany is using the training manual on soap and bone craft making, developed by FAO Somalia for SOMDA in Hargeisa, in its work in Puntland and is in the process of sub-contracting SOMDA to conduct the training in Puntland.

144. In conclusion, FAO Somalia has played an important role influencing the development and formulation of key government policies and strategies. (This is one example of FAO Somalia paying attention to the ‘policies, institutions and processes’ part of Figure 1 above). The growing scale and profile of FAO’s programme in Somalia has also given it visibility and leverage with the international community, most notably in current debates on resilience. The area that deserves greater attention in future, however, is ensuring that stronger advocacy accompanies the technical early warning that FSNAU is well-placed to provide, and that advocacy skills are strengthened within FAO Somalia.

6. Effectiveness of FAO’s management and operational systems 6.1 Leadership 145. Effective leadership is widely credited as having played a central role in the overall growth and effectiveness of the FAO Somalia programme in the last few years. This recognition comes from FAO Somalia staff, FAO headquarters and from key external stakeholders including donor representatives. The leadership qualities that interviewees value include:

1) strategic vision and a willingness to make bold strategic choices and to innovate; 2) ability to build a strong management team and to attract good staff; 3) high-quality management support which has helped to keep motivation levels high despite the challenging work environment; 4) supporting and managing the surge in response to the 2011/12 famine; 5) knowledge of FAO and of how to work with the system, yet ‘protecting’ the FAO Somalia team from certain interferences during the 2011/ 12 emergency phase; 6) relationship-building and communications with donors, and an openness and transparency about the FAO programme.

146. These strengths have also contributed to FAO’s leadership position within the international community, most evident in its current leadership on resilience thinking. Critically, it has raised FAO’s profile and its ability to engage with donors as an equal partner. In the words of one donor representative, FAO Somalia has become a ‘prominent player’ and is seen to be forward-thinking although this was seen to be ‘in spite of the challenges’ of normal FAO bureaucracy and procedures.

147. Although dynamic leadership has been key to growth, innovation and raising the profile of the FAO Somalia programme, the findings of this evaluation are a reminder that this fast-paced expansion has sometimes been at the expense of monitoring and assessing performance and progress, and therefore of learning from what is currently being done and fulfilling accountability requirements for increasingly large flows of aid resources.

148. At the moment, most of the leadership qualities are currently vested in one person, the current OiC. This carries a high level of risk in terms of succession. The FAO Somalia programme enjoys a high profile and is widely supported by a growing number of donors; some of them expressed their concerns about succession to the evaluation team. In the words of one: ‘if they mess up replacement, it would really rock donor confidence in FAO’. If the programme stalled, or at worst unravelled due to badly managed succession this could do serious reputational damage to the wider organisation. The evaluation team is aware of the commitment of the FAO’s current Director-General to improving recruitment of FAO Representatives (FAORs)55, but was not able to find evidence that FAO has done any succession planning although finding a high-calibre leader, especially in the context of relocating to Mogadishu, could take some time. As described in section 6.5 below, FAO does not have a strong record in this respect in terms of head-hunting to fill key positions.

55 With reference to the Director-General’s bulletin of 28th May 2012, No. 2012/25

44 6.2 Managing a multi-nodal programme from Nairobi 149. One of the dimensions of remote management necessitated by insecurity (especially in Mogadishu) has been running a geographically-disparate programme with many different nodes, managed from a third country, Kenya.

150. There are some aspects of this that FAO Somalia has managed well, for example it is widely credited with how it has grappled with the reporting dimension of remote management, at least in terms of delivery. See section 6.6 below. There are others that have posed more of a challenge, especially as the scale of the programme has rapidly expanded. One such challenge has been a tendency to centralise the locus of decision-making in Nairobi. This is resented by government officials in Somalia, some of whom have made unfavourable comparisons between FAO and other agencies that have moved their representatives to the field more quickly, especially to Somaliland and Puntland. In a meeting with one government ministry in Puntland, it was commented that: ‘in comparison with other agencies FAO’s presence is felt more outside of Somalia than within’. This may be unduly harsh but indicates the depth of frustration. The lack of awareness of FAO’s work in south- central by some government officials in Mogadishu implied FAO staff have had limited engagement at this level. Some UN and other international agencies have similarly expressed their disappointment with FAO’s Nairobi-based centre of gravity and slowness in locating international staff in Somalia. In December 2011 there were 46 international UN staff based in Somalia, but FAO had none. At the time of the evaluation there were just two internationally-recruited FAO staff in Hargeisa, both appointed in 2012, compared with WFP which had eight international staff and UNICEF, ten. The growing and increasingly active FAO office in Hargeisa has been welcomed by stakeholders in Somaliland, but still has limited decision-making powers.

151. The centralisation of decision-making in Nairobi has inevitably affected the level of involvement of FAO’s field-based staff, some of whom seem to be under-utilised in strategic decision-making considering their qualifications and experience. Field level staff interviewed by the evaluation team were often unaware of FAO Somalia’s vision for the future, nor had been consulted in developing the resilience paradigm (despite its strong field focus in implementation). Some did not know many of the FAO Somalia team based in Nairobi and had not had an organized induction process when they had joined FAO. All of this contributed to field staff feeling disincentivised and cut off from Nairobi and from the overall programme vision and decision-making. This also risks the Nairobi team being cut off from the day-to-day realities of Somalia.

152. Discussions with FAO’s CFW partners in Dollow highlighted how distance management meant that FAO Somalia’s relations with its implementing partners (IPs) were remote. This accentuated the gap between FAO and its partners in the field and limited the technical value that IPs could add. 6.3 From project to programme approach 153. With TCE’s encouragement, the current OiC has introduced a significant change in how the FAO Somalia programme is organised, from a project-based to a programme-based approach. While the latter is common-place for many international agencies, including WFP, it is not common in FAO. This is another example of innovation and leadership in the FAO Somalia programme.

45 154. When the FAO Somalia programme was smaller in scale there were some advantages to a projectised approach: it encouraged close contact between the FAO staff managing the project and the implementing partners and a sense of ownership by the respective FAO staff. As the programme grew, the first shift was to a sector-based structure (around April 2011) which enabled the development of a more coherent and strategic approach, but reporting was still done on a project basis. The first step towards a programme approach was the emergency programme in 2011 whereby funds were raised against a single programme/ appeal and were pooled, but agreements were signed with individual donors for individual projects. This became extremely demanding and inefficient, especially for reporting and for attribution. Learning from this experience, a programme approach has since been extended to the entire FAO Somalia programme based on the principles of pooled funding, one programme proposal and one programme report (although some modifications have had to be made to meet the requirements of specific donors). As well as gains in terms of strategic planning, the efficiencies of a programme approach are widely acknowledged by FAO Somalia and by FAO HQ in terms of the rationalization of staff time spent on functions such as monitoring and reporting. However, the financial tracking system established for this new approach, required to reconcile individual contributions with a pooled-fund management model, raises questions.

155. In the absence of an adequate corporate model, FAO Somalia pioneered its own ad- hoc financial management system, mirroring the ‘multi-donor trust-fund’ approach that is composed of ‘baby-projects’ against individual donor contributions. This system was supposed to allow reporting on overall programme achievements on the one hand while on the other hand keeping track of individual contributions. The challenges of donor earmarking and/ or individual reporting requirements preclude a true pooled-fund management approach. Despite the system’s complexity and its lack of system-based safeguards against overspending on individual contributions, which the evaluation sees as its main weakness, the Finance Department in HQ endorsed it. This is a reflection of their confidence in the current finance officer’s ability to manage the system and to set-up the necessary control mechanisms. Thus, the system established by FAO Somalia is regarded as adequate but is highly dependent on the current head of finance, a risky and untenable situation that needs urgent attention. An alternative is an ‘umbrella programme’ that combines both earmarked and truly- pooled contributions, but it would preclude reporting against the entire programme so is not seen to offer a better model.

156. A slightly different and earlier step towards a ‘one programme/one office’ approach was the restructuring that took place in 2010 to incorporate FSNAU and SWALIM, both of which had been independent projects, into the FAO Somalia programme. Despite early concerns, particularly on FSNAU’s side that this would compromise their independence and especially their perceived independence, these fears do not seem to have been realised. 6.4 Scaling up and scaling down 157. FAO institutionally does not have a strong track record in scaling up rapidly to respond to emergencies. Its reputation is as a more methodical and slower-moving development organisation. So when FAO Somalia had to scale up rapidly to respond to the 2011/12 famine, it was essentially on its own. Corporate support was minimal apart from support from TCE and some technical support from AGAH, and many corporate procedures inhibited rather than enhanced the operational surge, for example recruitment and procurement procedures. See section 7.1 below.

46 158. The evidence suggests that the FAO Somalia programme managed the surge well. The CMU, set up in mid-2011, seems to have functioned effectively, managing a heavy operational workload and large amounts of funding while setting up new operations in the field. The experience of CMU as an example of good practice should be documented for wider FAO learning. As FAO has few ‘off the shelf’ emergency management and administration procedures, full advantage should be taken of documenting FAO Somalia’s experience and learning.

159. The challenge in 2013 will be to ‘scale down’. As the acute emergency subsided and funding levels decrease, ways of scaling down to reflect this change in reality need to be found. An analysis of current human resources against the programme’s needs should be carried out and the team restructured accordingly. 6.5 Human resources 160. An audit of the FAO Somalia programme in 2010 highlighted sub-standard human resource (HR) management practices and a general lack of transparency. Ad hoc support was provided by FAO HQ to establish more sound working practices and greatly improved HR management was acknowledged by the recent financial management and administration audit published in July 2012.

161. Nevertheless the evaluation still encountered shortcomings related to human resource management. These include:

(1) Long delays in staff recruitment, both for international and field-based positions. One example is the amount of time it has taken to recruit the new Chief Technical Adviser (CTA) for FSNAU – well over nine months – at a critical time in the project’s evolution when it has had to ‘tread water’ rather than move forward, negatively impacting its credibility with the wider international community; another is the lack of a fisheries coordinator for years in FAO’s Sub-Regional Office (SRO) in Addis Ababa until October 2012, reportedly missed by FAO Somalia. FAO’s lengthy and bureaucratic recruitment procedures have been raised in numerous evaluations of FAO’s programmes around the world as a factor affecting both the effectiveness and efficiency of FAO’s work. Yet the problem persists with little evidence that FAO engages in head-hunting to fill key positions. This puts it at a major disadvantage to other international organisations that have more pro-active recruitment policies.

(2) FAO’s lengthy recruitment process for staff positions related to its corporate HR policy leaves FAO Somalia little flexibility to hire staff except as short-term professional staff or as consultants, even for key positions. In November 2012 only 16 of FAO’s international employees in the professional category held fixed term contracts (i.e. with benefits), while 25 were employed under consultancy contracts ranging from 3 to 11 months duration, including ten key positions, mainly heads of technical units. This issue has affected FSNAU staff who have been working for FAO for 12 to 15 years on one year contracts without job security or employment rights. This issue has been raised in the past, for example in an FSNAU evaluation in 2008 that made three pertinent recommendations: ‘FSAU needs to offer its staff including national staff with career perspective up to the senior management positions…

47 Maintaining a high level of staff competence should be a major aim of the FSAU’s human resources management… Other UN agencies (WFP, UNICEF) operating in Kenya and Somalia are providing better contract conditions to their staff. The FAO should re-examine the contracts of field staff on 11-month contracts’. (Bell et al, 2008:37 & 38)

Little action seems to have been taken in response, with the consequence that FAO is unable to provide attractive employment conditions or to compete with more secure contracts offered by many other UN and international agencies. This may become even more of an issue for FAO Somalia as it relocates to Mogadishu where the labour market is likely to be highly competitive as many international agencies return to the capital.

(3) Lack of induction processes for new staff: the need for well-planned induction is even greater as the FAO Somalia office has expanded, and lack of induction has caused some field-based staff to feel disconnected from the Nairobi office, as mentioned in section 6.2. This also causes inefficiencies as staff new to FAO have had to work out for themselves how FAO processes and procedures operate.

(4) Poor communication between Nairobi and field offices, which tends to marginalise the field staff, as mentioned in section 6.2, is essentially a local HR management issue.

162. In order to address these issues FAO Somalia needs either an HR professional within the office (which would be preferable), or further ad hoc support from FAO’s regional office or HQ. This input should also support FAO Somalia in scaling up and scaling down: scaling up when surge capacity needs to be built rapidly as in 2011, and scaling down, as in 2012/13 post-famine. This second part of the process requires an analysis of current staffing, capacity and training needs, especially as FAO Somalia changes gear and engages in community- based resilience programming.

163. These shortcomings in HR management and policy weaken FAO's ability to relocate successfully into Somalia (see below), especially if it is not able to offer attractive packages to staff from whom personal sacrifices are requested, for example living apart from their families. 6.6 Remote management, monitoring and evaluation 164. One of the major challenges of remote management is installing robust and reliable monitoring and evaluation (M&E) systems when field access is denied. Up to 2010 FAO Somalia did not have a systematic approach to M&E, nor dedicated staff. M&E was carried out on a project-by-project basis and the quality of the monitoring tended to vary according to the donors’ requirements and the allocation of resources to M&E in the respective project budget. Monitoring data mostly came from FAO’s implementing partners (IPs) which FAO would attempt to triangulate with information from key informants such as community elders and local authorities where possible, from FSNAU data, or from other NGOs. The monitoring was entirely output-oriented or activity- focused. Regular contact between FAO’s technical teams and IPs was the main means by which FAO kept some oversight on its field programme.

48 165. When the programme dramatically scaled up in 2011 with the emergency response, the structural weaknesses of this ad hoc approach to M&E became apparent. The large number of actors involved and the reshuffling of team responsibilities from 2011 meant that FAO Somalia could no longer maintain close contact with IPs and key informants, and allegations of diversion of funds in the CFW programme revealed systemic weaknesses and the failure of controls: there was no formal selection process for IPs and money vendors, for example, and reporting requirements were thin. Up to 2011 the CFW programme had been managed by the infrastructure unit and did not have a dedicated manager; when one was appointed in October 2011, there was no handover from the previous officer in charge. All these factors left the programme vulnerable to diversion of funds and collusion in the context of remote management, especially related to CFW, an area of programming in which FAO had limited experience and expertise.

166. In 2011 an M&E team was established within the CMU to provide oversight to the emergency programme. The M&E system was mainly built on quantitative and output focused data, collected through: 1) MACC (Monitoring Activities Call Centre), a phone-based beneficiary survey system aimed at verifying that activities had been implemented and outputs received; 2) Field monitors: 19 recruited in 2011 to cover south-central Somalia to do typical post- distribution monitoring / output counting; (10 more recruited in November 2012 for Somaliland and Puntland); 3) The use of remote sensing imagery (in three districts in Lower Shebelle) to confirm the progress of construction work through CFW against IP reports, drawing on SWALIM capacity to interpret the imagery; 4) FMT (form management tool) as an on-line instrument used to enter M&E data collected in the field and to reduce data processing time; 5) FAO is also planning to introduce a biometric information system for CFW interventions in 2013, to track beneficiaries and delivery.

167. The pros and cons of the monitoring tools used by FAO Somalia are presented in Table 2. Development of many of these tools demonstrates FAO’s efforts to gather information from the field and to account for its expenditure in Somalia, at least at an output level. Its initiatives in using new technologies are commendable, opening up new possibilities in a context where verification seemed almost impossible because of lack of access.

168. Despite these measures, FAO Somalia has not been able to protect its programme against diversion of funds. Given the challenging context in which it is operating, the pertinent issue is how quickly it picks up indicators of mismanagement and the speed with which it is able to investigate and respond to them. This issue has been addressed through successive audits of the FAO Somalia programme, the most recent being the financial and administrative audit carried out in July 2012. This evaluation notes and credits FAO’s efforts to address the high risks of operating in the Somalia context, for example an operational risk management framework was developed for South Central Somalia in November 2011, identifying the main risks faced, their significance, and measures that were in place to mitigate them. Risk logs were developed for the CFW programme and have informed how the MACC operates. When there have been indicators of the possible diversion of funds, the FAO Office of the Inspector General (OIG) has responded with an investigation and action has been taken. Thus, measures to manage high levels of financial risk associated with remote management appear to have been built into the daily work of FAO Somalia since 2011, in turn dependent on FAO’s monitoring capacity.

169. Nevertheless, the evaluation noted the following weaknesses in FAO Somalia’s capacity to monitor its programme: a. There was no framework and apparently no strategic vision behind the M&E system: the function of M&E (compliance or feedback into programmes) does not seem to have been clarified with the result that the control function has dominated and tools have been developed mainly to check on whether IP reporting was accurate. While this is clearly important, it has contributed to tense relations with IPs who feel they are not trusted; b. The roles and division of labour between M&E stakeholders have not been defined and none were assigned the function of collecting qualitative information to understand and learn from programme dynamics; c. The high cost of some of the technologies used, in particular the remote sensing, seems to have limited its use; the evaluation was not clear how effectively it may have supported improved monitoring on a large scale. The cost of the biometrics system is not clear to the evaluation team, but may be an issue; d. FSNAU data has not always been used to its full potential by FAO Somalia for monitoring purposes. Some sectors, including the crop sector, have made most use of FSNAU data. Others, including the livestock sector, do not appear to have maximised this opportunity56.

170. Above all, the focus on compliance means that FAO Somalia has not tackled outcomes and impact which are key to learning about whether an intervention was effective and the right choice, and key to fulfilling accountability requirements. Despite the large amount of resources invested in monitoring the CFW programme, for instance, they contributed little to understanding impact and outcomes. Meanwhile, donors are increasingly requesting data on outcomes and impact. As described in section 5.1 above, this has inhibited the kind of rigorous analysis that is needed to underpin FAO Somalia’s programme choices. Some parts of the FAO Somalia programme are shifting to more in-depth M&E that is better oriented to capturing outcomes and impact, for example the CFW programme has employed a senior consultant as deputy coordinator with dedicated M&E functions and a number of more in-depth studies are planned; the agriculture sector outsourced a crop performance assessment for the Gu season of 2012 and FMs are also using the PET tool for outcome monitoring of crops; FMs are to be supported to do more outcome monitoring and more qualitative data collection. But this shift in thinking is very recent and is yet to permeate the whole programme. In the context of Somalia insightful outcome and impact monitoring may be most easily achieved through one-off participatory and qualitative studies, as the CFW study for this evaluation demonstrated.

171. There are also indications that FAO’s compliance monitoring may have become too demanding. This is the feedback from some of FAO’s IPs, that the heavy load of monitoring requirements has become counter-productive and is distracting them from the intervention itself57. There is clearly a trade-off here, ensuring that monitoring of delivery is sufficiently

56 It is important that FSNAU is not drawn away from its core business, nor that its independence is compromised by being asked to play a role in monitoring FAO Somalia’s programme quality, or in attempting attribution of impact to FAO’s projects and programmes. Nevertheless, parts of the FAO Somalia programme could make greater use of FSNAU data and analysis in their monitoring 57 This feedback came through from the Livestock Professional Associations in a meeting with FAO in December 2012. They identified the heavy load of monitoring procedures – image taking, filling vaccination

50 rigorous in view of the risks of diversion, but does not impede the work. The pendulum may have swung too far in this respect and data and analysis for learning purposes appears to have been overlooked.

172. Gender has been badly neglected in FAO Somalia’s monitoring so far. Sex- disaggregated data on beneficiaries are not systematically collected and there have been no attempts to understand how interventions may have had a differential impact on the livelihoods of women and men, nor of minority groups. The majority of field monitors are men and have not been trained on gender issues.

173. M&E is now being re-thought in the light of the new resilience strategy. This is an opportunity to learn from what has worked and what has not so far. The priorities should be: (1) developing an overall framework for M&E and ensuring equal emphasis on establishing outcomes and learning as on compliance; (2) clarifying the roles and responsibilities for M&E between the M&E unit and the sectors. This is also an opportunity to redefine the role of the M&E Unit, which may be best-placed to continue focussing on compliance monitoring while the sectoral teams with greatest knowledge of the programmes and of cutting edge and relevant research in the region, commission and outsource periodic outcome-oriented evaluations and research-based studies to test the assumptions on which some of the programming is based; (3) learning from successful efforts to monitor and evaluate outcomes and impact in other challenging environments where security is an issue and access may be constrained58; (4) ensuring that a gender-sensitive approach to monitoring the resilience programme (and the rest of the FAO Somalia programme) is installed from the outset.

174. The estimated annual cost of the M&E unit is $1.7 million in 2012, including the M&E unit staff, field monitors and the call centre (but not the FMT or remote sensing). This accounts for approximately 3% of total programme costs in 2012. The proportion allocated to M&E is expected to rise to 4-5% of programme costs with the resilience programme, including the baseline survey, the resilience impact survey and remote sensing. These appear to be reasonable percentages if they result in improved outcome and impact data.

175. As FAO Somalia strengthens its own ability to do outcome and impact monitoring, it will then be in a stronger position to build the M&E capacity of the respective government authorities (which so far have also focused principally on output indicators, for example in monitoring the 2011/12 emergency livestock treatment and vaccination campaign).

forms, signatures, telephone numbers, collection of empty bottles during the field work etc – as an impediment to their work, overshadowing the real intervention of treatment/vaccination and have suggested that the monitoring procedures be simplified. Concerns about excessively demanding reporting were also raised during a meeting with some of the new IPs working with the Cash-Based Interventions Unit. 58 See, for example, work in Sudan and Ethiopia, as well as community-based systems developed by ActionAid in Somaliland in the 1990s. The Feinstein Center has pioneered an approach to ‘participatory impact assessment’, whereby project participants are included in the impact assessment process in a ‘learning partnership’ that also involves the donor and implementing partner (Catley et al., 2009b).

51 Table 2 Strengths and weaknesses of M&E tools currently used by FAO Somalia M&E Tool Strengths Weaknesses Monitoring • Innovative tool with strong potential • Small % of beneficiaries contacted Activities Call where access is restricted so far has limited the ability to Centre (MACC) • Can pick up problems related to generalise and draw conclusions. for phone-based implementation and raise red flags. • Questions arise about potential surveys • Numbers of telephone numbers is biases in beneficiary selection, if increasing, with potential to poorer households do not have improve representativeness. mobile phones and if there are 59 • FAO has started a beneficiary areas without network coverage . ‘hotline’, potentially increasing • Questionnaires have not been accountability to local people tested for potential bias where access is possible. • Questionnaires centred on process and compliance monitoring exclusively Field monitors • Where possible, first-hand field • Risks of insecurity compromising (FM) visits and reporting FM’s access • FMs provide a field presence and • Often dependent on IPs for access, therefore direct relationship with which may compromise communities and IPs. independence • Some capacity-building of FMs was • FMs may come under pressure done though still insufficient from their own communities not to • FMs have the potential to monitor report upon irregularities outcomes with more qualitative • Lack of capacity (eg. often FMs do approaches, but depends on not have an agricultural building their capacity and skills. background) or training in assessment methods. • Lack of numeracy skills raises a question mark on the validity of numerical information collected • Lack of supervision in the field • Missed opportunities to use FMs eg for targeting • Few female FMs Remote sensing • An objective source of information • Has strained FAO/IP relationships where access is restricted/ and sense of trust, and was impossible introduced after the work had been • Supported by SWALIM capacity completed • Depends on NGOs’ capacity to correctly map and measure the canals60. • Satellite imagery alone does not build understanding of what happens in the field. Access and direct contact with beneficiaries still necessary. • Costly instrument, but there may be alternatives to this cost in the near future. Commissioned • Independent team • Variability in quality, and extent to evaluations • Opportunity for more thorough and which evidence-based

59 FAO Somalia’s CFW team is currently undertaking research into the extent and disaggregation of mobile phone ownership which should help to inform the extent of bias this may introduce 60 In future IPs implementing CFW projects with FAO will be required to have a technical staff member amongst their staff to engage in the remote sensing

52 qualitative analysis • Difficulties in finding good evaluators • An aid environment that wants positive feedback and is less open to the risk of failure, even in high risk environments • Problems of access and therefore of triangulation of findings Form • Provides users with pre-designed • Depends on IPs and FMs being Management forms and questionnaires for data trained and having the hardware to Tool (FMT) collection allowing for implement the FMT (training needs standardization time) • Data collected in the field sent electronically to FAO, speeding up response time 6.7 Communications management 176. FAO has not usually invested in communications management at country level. Bucking this trend, FAO Somalia has made proactive efforts to establish a three year communications strategy and develop a variety of external communication products since 2010. It appears to have done so with no support from FAO HQ and in spite of some counterproductive decisions61. This indicates the priority that FAO Somalia has attached to establishing sound communication practices as a part of effective management. A communications specialist was hired, first to support dissemination of FAO Somalia’s public information systems, and then to address the programme’s wider communication needs. This has developed in two directions: i) outreach to donors and the media, FAO Somalia’s focus over the last 2 years (in phase 1) and ii) outreach to beneficiaries, more recently prioritized. The latter has happened too recently to be evaluated, but offers potential to strengthen accountability channels between FAO and its beneficiaries and to ensure they are informed of their rights and entitlements. In terms of the former, FAO Somalia has made important efforts to prepare simple communication products to improve accessibility and to attract media attention. But so far there has been no monitoring of communication outcomes, for example by tracking website hits, and therefore no accumulated evidence of the impact this has had on the media, donors and other audiences. This should be addressed as FAO Somalia continues to develop its communications work. 6.8 Decentralisation from FAO HQ to Nairobi 177. FAO-Somalia was selected as a pilot for decentralization of TCE. This was a rational choice as the country office already de facto managed emergency and development programmes jointly under the authority of the OiC. This, combined with the relative autonomy of the newly-arrived OiC who already enjoyed a high level of trust with TCE, meant that decentralization simply formalized the existing autonomy of the Somalia Country Office rather than drastically changing the centre of gravity of decision-making. Since 2011 FAO Somalia had a Delegation of Authority (DoA) for procurement of up to 250,000 USD, more recently extended to 500,000 USD. It is the only country in FAO with such a large DoA. This is only possible because it has an international procurement officer re-located to Nairobi from Rome, a privilege that few FAO country offices enjoy.

61 For example, FAO HQ made a very poor decision to close the webpage developed by FAO Somalia to provide information on its activities

53 178. Other factors which have contributed to the relatively smooth process of decentralisation from FAO HQ to FAO Somalia include: • a shared vision by OiC and TCE of the goal and process of decentralization; • available resources to invest in some key functions in FAO Somalia, such as procurement and technical expertise; • two HQ-based staff from TCE relocating to Nairobi between April and September 2012 as part of the decentralization process; • the findings of the negative audit in 2010 also gave FAO Somalia the opportunity to make changes that prepared it well for full decentralization.

179. Although decentralization formally shifted the OiC’s line management from Rome to the Regional Office for Africa (RAF) based in Accra, in practice this has not been a major change as the FAO Somalia office has maintained close links with HQ, partly due to its long- standing and privileged relationship with TCE and partly due to the OiC’s strong network of relationships with FAO Rome. Over time, of course, this network of relationships could weaken, in which case it must be replaced with alternative networks to ensure that neither technical support nor knowledge exchange with the wider organisation suffers.

180. As documented below, decentralisation of technical functions has not happened as smoothly as decentralisation of management. Although technical functions have moved to the SRO in Addis Ababa, there are not necessarily the people in place to fulfil those functions. Overall, there appears to be quite limited contact between the SRO and FAO Somalia. 6.9 Re-location of FAO from Nairobi to Somalia 181. As noted above, FAO has been slow to shift decision-making and to put international staff into Somalia, despite the relative peace and security in Hargeisa and Garowe. Recent political developments with the establishment of Somalia’s most credible national government for two decades means that it is now time for FAO to move the centre of gravity of its programme from Nairobi to Mogadishu. In many ways this is a decentralisation of FAO Somalia from Nairobi to Mogadishu, Hargeisa and Garowe.

182. At the time of the evaluation mission to Nairobi, the plan was to move FAO Somalia’s operational functions to Mogadishu while others, such as procurement, remain in Nairobi for the time being. By June/ July 2013 there should be around 15 international staff in Mogadishu, 6 international staff in Hargeisa and 5 international staff in Garowe. The estimated establishment costs are 5.6 million USD and 1.1 million USD for running costs.

183. Although this is a timely move to Mogadishu (and in the case of Somaliland and Puntland a tardy move), there are many risks associated with it. First and foremost is the security risk, particularly in Mogadishu, and the high costs associated with having a secure office and accommodation. Second is the risk to programme quality, and the ability to match staff capacity with an ambitious programme; it can be anticipated that some of the existing staff will not want to relocate and it may be hard to attract well-qualified international staff to Mogadishu. Competent local staff will also have to be recruited in what is expected to be a competitive job market. Third, the transition from a well-serviced Nairobi office to a new office in Mogadishu where there are bound to be technological and logistical disuptions brings the risk of a slow-down in programme delivery and performance. Fourth, the relocation to Mogadishu may coincide with scaling down the FAO Somalia programme so will need to be carefully managed to maintain staff morale. Fifth, as a more development-oriented

54 organisation, FAO is less experienced in managing this kind of transition than other more emergency-oriented UN agencies, and does not have the same level of corporate support.

184. These risks must be set against the benefits of re-locating decision-making and management to Somalia and the opportunity for closer relationships and dialogue with key Somali stakeholders, at least those based in Mogadishu, Hargeisa and Garowe, (although it will not build closer relationships with local communities unless FAO staff spend considerably more time in the field, in accessible areas, than is currently the case). It also means that some of the current weaknesses in administrative management, for example on HR described above, will need to be urgently addressed so that FAO Somalia is ready to relocate and to offer staff competitive packages (including the possibility of having their families in a nearby location62), and that potential risks and delays are built into current and future planning and programming. To achieve successful relocation in a relatively short time- period, FAO Somalia will require substantial support from FAO HQ and/ or from RAF. 6.10 Technical support and knowledge exchange 185. As the FAO Somalia team has grown in size and technical capacity, it has received relatively limited technical input and support from the rest of the organization. Since decentralization the first port of call for FAO Somalia for technical support should be the SRO in Addis Ababa. But this comprises a very small technical team of just seven technical experts supposed to service eight countries in east Africa (and has been beset by recruitment difficulties and delays: two of the seven positions were vacant in November 2012 although the role has sometimes been temporarily filled by a consultant) so has not had the necessary resources nor expertise. The main sectors that have received support from the SRO have been trade and livestock production. A source of frustration for FAO Somalia has been the need to request technical clearance from SRO in areas where the country office may have stronger technical expertise.

186. Surprisingly, FAO Somalia has had very limited interaction throughout the 2007 to 2012 period with FAO’s Emergency Regional office (REOA) in Nairobi, even during the recent emergency when exchanges were limited to the communication sector or to linking information systems (for example FSNAU sharing its data and experience with other countries in the Horn of Africa that did not have such developed systems). This may be related to the type of relationship that the REOA has with Country Offices, essentially dependent on requests from the latter to take action.

187. Despite decentralization, most technical support to FAO Somalia has come from FAO HQ, especially in the livestock sector (eg related to animal health) but less so for agriculture63. Until recently there were weak links with HQ for the fisheries sector. This seems to follow a familiar pattern in FAO whereby the level of exchange between sector teams and HQ staff is strongly related to the relationship individuals have with HQ, in turn a consequence of whether those staff have passed through Rome or not. Until TCE relocated

62 Policy Directive 2012/10 - Discontinuation of SOA: Transitional Measures dated 07 June 2012 has abolished the possibility of staff working in non-family duty stations having their families living in the Administrative Place of Assignment. This sort of administrative rule creates strong disincentives for staff to accept being posted in countries such as Somalia. 63 In terms of the agriculture sector, feedback from current FAO staff indicates that technical backstopping from HQ has been demand-driven with good and timely support from AGPM. FAO-Cairo provided support on irrigation design. However, the Sub-Regional Office in Addis has not always provided timely support when requested

55 staff to Nairobi, they were also able to liaise between FAO Somalia and technical expertise in HQ – a link which may weaken now.

188. SWALIM and FSNAU are the exceptions in that the lead technical officers for both projects are based in Rome and each project continues to be supported technically from FAO HQ. This has generally worked well although there are a number of cases of technical clearance being delayed, thus delaying the publication of reports. Both surprising and concerning, however, is the lack of experience sharing across the region. As both SWALIM and FSNAU pay increasing attention to capacity-building of government, there are valuable opportunities to share experience and to learn from the SIFSIA project (Sudan Institutional Capacity Programme: Food Security Information for Action) in Sudan and in South Sudan. The learning from South Sudan may be especially pertinent as FAO has been trying to build government capacity in data collection, analysis and food security policy-making (including natural resource mapping) since 2006, starting with low skill-sets and limited resources, and facing issues of high staff turnover as soon as they are trained, all familiar issues in the Somalia context.

189. This is indicative of obstacles to corporate knowledge exchange, in this case regional knowledge exchange. There are other positive examples of how FAO Somalia has contributed to FAO global knowledge. FSNAU, for example, pioneered the Integrated Phase Classification system (IPC) which is now used globally by FAO and other organizations. And the FAO Somalia CFW team is constructively feeding into guidelines FAO is developing on cash-based interventions. But one of the casualties of decentralization appears to be the current lack of mechanisms for FAO corporately to track its technical capacity and innovation at country office level. This means that opportunities for knowledge exchange may be missed. For example, there are ways in which SWALIM could serve the greater region, but these were not identified nor exploited during the 2011/12 Horn of Africa drought. The short-term contracts of many FAO Somalia staff also acts as a disincentive for them to get to know colleagues in technical divisions in Rome or in the regional offices, and therefore to contribute to organizational learning. The Functional Technical Networks recently established in FAO and to be managed from the SRO may begin to address this problem.

190. FAO’s Horn of Africa regional initiative on ‘Zero Hunger’ was intended to encourage greater collaboration across country offices, facilitated by the SRO, but progress appears limited so far.

191. Two learnings emerge from this analysis of how FAO Somalia has been supported technically: (1) as decentralization proceeds within FAO, it will not be possible to replicate the technical expertise that has been located in FAO headquarters in the regional and sub-regional offices. But the strong technical expertise that a country office like FAO Somalia has been able to develop could be taken into account in knowledge sharing mechanisms such as the Functional Technical Networks, already under discussion within FAO. Technical expertise could be mapped across the organization and some staff at country office level take on additional regional or corporate responsibilities. (2) technically strong, FAO Somalia has de facto been able to operate with a relatively high degree of autonomy. This raises two concerns, first to ensure that it is sufficiently connected into the rest of the organization from a technical point of

56 view and does not become isolated in what it is doing and that a two-way learning process is installed and working; second, the need for more rigorous analysis to underpin some of FAO Somalia’s programming highlights the importance of peer review, some of which should be provided from other parts of FAO, including the sub-regional office.

7. Efficiency 7.1 Efficiency of FAO procedures, administration and structures 192. Since the 2010 audit, FAO Somalia has moved to standardize its procedures (e.g. in procurement and recruitment) compared with the situation previously when each project had considerable autonomy to follow its own path. Overall this is a significant improvement. In line with the 2012 internal audit report, the evaluation considers this as indispensable in view of the increased risks associated with the Somalia context including remote management and especially since the programme has grown substantially. The evaluation recognizes that the trust established between FAO Somalia and its donors depends on the existence of solid and transparent mechanisms to ensure adequate control over the programme’s resources.

193. The tightening of procedures has, however, impinged on the timeliness of some actions related to programme delivery. Some procedures appear to have slowed down decision-making and/ or smooth implementation, especially concerning when actions are time-sensitive. For instance, for the delivery of training activities in the field FAO manages the procurement of small items through apparently quite burdensome procedures rather than authorizing implementing partners to purchase the materials themselves and present an invoice. Other examples include arrangements for short-term inputs that are needed within tight time-frames, for example to hire an editor or translator, where approval processes have been too slow to meet deadlines. There have also been cases of slow approval for Letters of Agreement when an NGO has been urgently needed to carry out a time-sensitive nutrition assessment.

194. A perceptions survey of FAO technical staff, carried out for this evaluation, revealed the low negative scores they give to bureaucratic delays related for instance to tendering processes for procurement, for both development and emergency projects. Poorly drafted technical specifications and lack of planning may be the reason for some lengthy procurement processes, but at the same time there is a sense of frustration amongst staff as they perceive a loss in efficiency related to a lack of flexibility in the application of procedures.

195. Two examples of how slow procedures have compromised programme effectiveness can be found in the Dur Dur project in Somaliland. First, farmers have received inputs such as seed and fertilizer too late for the growing season and they have had to keep them for the following growing season. The second example shows how FAO’s procedures could jeopardize its collaborative working arrangements with other agencies that have faster, more emergency-oriented procedures, in particular WFP and UNICEF as partners in the resilience strategy. The WFP food-for-work component planned as part of the Dur Dur project to pilot resilience programming was stalled because of FAO delays in signing the Memorandum of Understanding (MOU). Although the work was scheduled for June/ July 2012, the MOU had still not been signed by November 2012.

196. Some of these problems imply there may not be sufficient delegation of authority within the FAO Somalia programme, for instance on HR management where senior staff do not have the autonomy to hire fixed-term staff for key positions but instead have to seek approval through centralized HR processes. There is a similar issue in terms of delegation of

58 authority between HQ and Nairobi: sign-off on financial reports to donors has to be done in Rome which sometimes caused unhelpful delays64. Both FAO staff and donors are calling for more simplified and flexible procedures to match the pace and flexibility of programming that is necessary in the Somalia context.

197. FAO Somalia has shown initiative in trying to address some of these constraints. For example vouchers have been used to acquire livestock for restocking interventions when procurement rules would not allow procuring livestock with cash; and in 2011 livestock professional associations (LPAs) were paid an agreed fee for each treatment they carry out (as a service) rather than FAO Somalia having to procure the drugs they need65. This had the added benefit at the time of promoting the purchase of veterinary drugs from within Somalia rather than undermining local suppliers. The flexibility and innovation of the procurement and contracting teams within FAO Somalia must be acknowledged although there still seems to be a problem of FAO procedures being too slow and cumbersome for timely implementation of some activities, especially of emergency interventions. As discussed in section 9.2 below, a review of whether procurement procedures could be eased, whilst still ensuring adequate rigour in the context of remote management, should be carried out in the interests of partnership to implement community-based resilience programming. 7.2 Cost effectiveness 198. Measuring cost effectiveness in a context like Somalia is challenging, particularly in terms of making comparisons as so many projects and activities are context specific. And the costs of operating in an insecure environment are considerable, not only the costs of checks and balances associated with remote management but also the costs associated with maintaining a large well-staffed office in Nairobi, and the anticipated costs of having the heart of the programme based in Mogadishu where, in November 2012, it cost $700 per day to travel within the city in an armed convoy. An analysis of the cost effectiveness of the FAO Somalia programme is further hampered by the way in which the financial expenditure tracking system is set-up (see section 6.3), for instance aggregating data under the categories of ‘technical programme costs’ (TPC) and ‘common costs’ 66, hampers disaggregated analysis as some line items appear in both categories. Only by deconstructing the financial data and then reorganising it according to the main line items was the evaluation team able to analyse expenditure over the period 2007 to 201167.

199. Within these constraints the evaluation team has attempted to look at cost effectiveness in terms of: (1) the weight of various cost categories (such as staff, LoAs, inputs) related to overall expenses and to overall programme performance; (2) the length of LOAs, with particular reference to the agriculture sector; (3) how efficiently FAO has used its resources, especially its technical resources.

64 There are examples of delays in getting sign-off to the point that the reporting data are no longer valid and have to be amended 65 This mechanism is unlikely to be allowed in future. 66 TPC expenses can be directly attributed to the implementation of an activity, and comprise technical staff, training & capacity building costs, input procurement, while ‘common’ or fixed costs are committed no matter what activity is in the work programme eg procurement, administration and finance staff, oversight management, offices management, etc. 67 At the time of writing, the financial data for the latest trimester of 2012 was still being reconciled by the FAO Somalia team. For this reason the analysis of financial data has been restricted to the period 2007-2011

59 200. In terms of the first measure, analysis of the financial data from January 2007 to December 2011 shows that contracts (LoAs) were the main item of expenditure, accounting for at least 30% of total expenditure over this five year period. In 2011 LoAs accounted for as much as 50% of total expenditure, reflecting massive scale-up of the CFW programme (LoAs represented over 90% of the overall CFW budget in 2011). In the agriculture sector, the relative weight of LoAs has declined over time while expenditure on procurement has increased. This may reflect the fact that procurement has become more centralized and that IPs have less scope to purchase inputs directly. Procurement of inputs for the livestock sector also rose from 20% of expenditure in 2010 to 30% in 2011. Overall, procurement of inputs has been relatively stable since 2009, accounting for around 10% of annual budgets.

201. Staff costs are the second big expenditure item over the period, accounting for about 20% of the total, but these have consistently decreased in relative terms as shown in Figure 6 below. This may reflect FSNAU and SWALIM (for which staff costs are the main expenditure) becoming a smaller percentage of the overall programme as it has grown. Staff costs also represent the main cost (80%) of M&E. However, this is not necessarily an indication of greater efficiency as staff may be carrying unmanageable workloads, as some staff have indicated in the post-emergency phase.

Figure 6. The relative weight of staff costs in total expenditure

202. Travel costs have fallen slightly in relative terms, from about 10% of total expenditure up to 2009 to around 5% since 2010. General operating expenses represented around 5% of total expenditure. Both seem reasonable, especially considering the additional costs associated with remote management. Training expenditure is around 2%. Although this is in line with FAO standards, the evaluation findings indicate that FAO Somalia’s investment in staff training has not been enough.

203. In terms of LOAs, at least in the agriculture sector there has been a trend over the 2007 to 2012 period to increase the duration of LOAs. Thus, although overall funding has increased, the number of LOAs issued each year has fallen from a peak of 121 in 2009. This is regarded as a positive development if it means that LOAs have been extended to cover the period of harvest as well as the period when the activity has been carried out. As explained in section 5.1.3 above, this should facilitate measuring the outcome of projects not just outputs in order to inform future programming. Longer LOAs are also better-suited to working at community level. As pointed out in the evaluation of project OSRO/SOM/811/EC, remotely managed projects in south-central Somalia rely for their success on relationships being built up between implementors, local administrators and communities, and in turn these relationships are built on trust which requires time and continuity of personnel. Short-term LOAs that do not take either of these factors into account are therefore a false economy, and

60 this was borne out in interviews with some of FAO Somalia’s NGO IPs who expressed their frustration at the short duration of some LOA contracts with FAO Somalia.

204. In terms of individual projects cost effectiveness varies. Some, where actual expenditure was in line with the planned implementation timeframe and budget, appear to have been cost-effective, for example rehabilitation of the livestock market in Hargeisa. Others, which have not been completed on time, and/or which have not achieved their overall objective, for example the two abattoirs constructed in Boroma and Bosasso (see section 5.1.6 above), cannot be regarded as a cost-effective use of funds. 7.3 Resource mobilisation 205. Evolving from a handful of projects with a small turn-over of $1.6 million in 2005, FAO Somalia has progressively built its capacity and its credibility with donors, to reach a critical mass of projects with a total expenditure of $ 34 million in 2010, and expenditure of over $50 million in 2012. This rate of growth is remarkable.

206. Initially, FAO Somalia’s resource mobilization strategy depended mainly on responding to ‘calls for proposals’, especially from the EC. At this point FAO’s programme was donor driven although three-year EC contracts gave it some financial stability. Its ability to win proposals depended on its track record, the credibility of FAO technical expertise and its contextual knowledge. This was predominantly funding for agriculture. Major funds for livestock did not come through until 2011.

207. The 2011 crisis brought a surge in FAO’s funding and activities with an appeal for $70 million in late July 2011, against which FAO had received 100% coverage by September 2011. Rapidly scaling up, as described in section 6.4 above, FAO Somalia demonstrated a high absorption capacity. With WFP’s withdrawal from much of south-central Somalia, agencies like FAO stepped into the breach and donor funds for alternatives to food aid – for CFW and other livelihood activities in the case of FAO – were readily available. FAO’s credibility and profile with donors at this time was rapidly growing.

208. Becoming FAO’s largest country portfolio is an indication of how effectively FAO Somalia has mobilized resources over the six-year time period of this evaluation, especially in response to humanitarian crises. During this period FAO Somalia has clearly evolved from following donor priorities to engaging with donors as a more equal partner and with a more strategic approach. Its budget has remained high in 2012 but as the crisis of 2011/12 subsides it is likely to become harder to maintain those levels of funding as humanitarian aid is less readily available. This will be the challenge, not only for FAO but for the wider international community, with the introduction of a three-year CAP and no immediate humanitarian crisis, to maintain adequate funding levels, for example for effective and long-term resilience programming to be a reality.

8. Connectedness 8.1 Capacity-development 209. FAO Somalia has engaged in a wide range of capacity-development initiatives, from the private sector to government to farmer-level, although much less in the NGO sector apart from trainings provided to members of the Food Security Cluster, described in section 5.3 above.

210. In terms of capacity-building with government, SWALIM’s capacity-building programme has been one of FAO’s longest running, beginning in 2006 although really scaling up in Phase III from 2008. It is intended to continue through the next phase of the SWALIM project until 2017. So far it has focused on the governments of Somaliland and Puntland but has not yet been extended to the TFG in Mogadishu. The approach described in Box 1 appears to have been effective, and well-functioning data centres are now embedded in the ministries and their data and analysis are being used for policy formulation and strategic planning. Importantly, given the low starting capacity of the ministry, a long-term perspective is being adopted.

Box 1 Capacity-development by SWALIM in Somaliland and Puntland

SWALIM has established data centres in three ministries in each of the respective governments: Ministry of Agriculture, Ministry of Water and Ministry of Pastoral Development and the Environment. Two government staff are assigned to each of the data centres and are supported with training and equipment by SWALIM. The challenges of such a capacity-development initiative are considerable because of the very low skills base within the ministries and lack of resources. When staff are trained they become highly ‘employable’ and are likely to seek more remunerative work elsewhere, often with international agencies. SWALIM has responded to this by providing the data centre staff with a top-up to their salary; there seems to have been less turn-over in recent years as the data centres have stabilised. SWALIM plays a role in selecting the data centre staff from a list of candidates provided by the respective ministry. Training is mostly provided ‘on the job’ as more SWALIM staff have been posted in Hargeisa and Garowe respectively. The data centre staff are expected to train their ministry colleagues in using the data and resources.

211. There is much to learn from SWALIM’s approach, and FSNAU based their (more recent) capacity-development programme on SWALIM’s experience although they have come up with a rather different model. In collaboration with government they have appointed nine focal points in nine different ministries within the respective governments of Somaliland and Puntland. Their approach to capacity-development has thus been more focused on the individual than on the institution, unlike SWALIM. This is a high-risk strategy if the individual should leave. A rigorous course in food security has been designed by FSNAU, with a number of assignments that the focal points must do and pass. So far the approach has been quite academic. Although thorough, it is lacking on-the-job practical training and there appears to be limited contact between FSNAU focal points and SWALIM-supported data centres. The next phase of the FSNAU project envisages a more institutionally-oriented approach, modelled on the SIFSIA project in Sudan (from which much learning can be gained, as mentioned in section 6.9 above). This is an important development that pays much great attention to the institutional context than has happened so far. It also offers an opportunity for SWALIM and FSNAU to link their capacity-development efforts.

62 212. Apart from study tours and the above-mentioned SWALIM and FSNAU support, FAO Somalia has done relatively little other capacity-development work with Ministry of Agriculture staff in Puntland and Somaliland although it has helped to fund the construction of new office facilities in Hargeisa. At community level, it has invested in Farmer Field Schools (FFS) which have been well-received as a form of extension. But some of the village-based farmer training seems to be carried out in mass meetings of villagers which may raise awareness of issues and concerns but are unlikely to change behaviour.

213. On the livestock side, most of FAO’s capacity development has been in the animal health sub-sector through laboratory support or laboratory establishment (OSRO/SOM/120/ITA, 104/CHS, OSRO/SOM/905/SPA) and technical training, for example short courses on laboratory diagnostic skills and risk-based disease surveillance for government officers and for LPAs. In terms of meat marketing, FAO Somalia’s capacity- development work is very evident in Puntland and Somaliland in terms of support to the respective livestock ministries in their formulation of the Meat Acts and in developing the capacities of the hygiene units and laboratories. Training is also being provided to meat inspectors. In contrast, relatively little work has been done on national animal health policy and legislation as this has been covered by projects such as the Somali Animal Health Services Project (SAHSP) of which FAO is one implementing partner.

214. Capacity-development has been an important part of FAO’s relatively new programme in the fisheries sector using a training of trainers (TOT) approach. The Ministry of Fisheries in Puntland estimates that more than a thousand people have been trained as a result, including ministry staff, fishermen and members of fishing communities. At the local level, those trained in boat repair, engine repair and net mending say that production has increased as a result of more operational boats. The training support has been widely welcomed in this sector.

215. In the private sector, FAO Somalia’s capacity-development efforts have a mixed record related to its performance with PPPs. It has been most successful in establishing two fully-fledged seed companies – Daresalam and Sesima – and a fruit growers’ association by providing rigorous technical and managerial support in difficult circumstances. Some of the other PPPs have had a more chequered history, described in section 5.1.6 below. And some of the more recent private enterprises that FAO has helped form and train, for example SOMDA in Somaliland which is involved in soap and bone craft making, is still in an early stage of development.

216. So far FAO Somalia has engaged in limited capacity-development or training of its IPs in the CFW programme. As FAO Somalia builds a body of experience and knowledge about how to implement effective CFW schemes it should consider expanding this training and support to its IPs.

217. So far, FAO Somalia’s capacity-development efforts have been connected to individual projects and to achieving project goals. However, as the programme has grown and shifted from a project to a programme approach, and importantly, as new opportunities open up for supporting the new government in Mogadishu, it is timely for FAO Somalia to develop an overall approach to capacity-development, especially for government. The priority areas that have been identified are spelt out in the institutional capacity development component of the new strategy. This could usefully be complemented with an overall framework that would guide work in different sectors, paying attention to FAO’s learning

63 modules on capacity development. As many other UN agencies similarly engage in capacity- building with government, there is also a need for a UN-wide strategy, or at least a framework or set of principles to ensure consistency, coherence and coordination of approach68. 8.2 Sustainability of FAO interventions 218. To achieve sustainability in the recent and current context in Somalia is a huge challenge. Yet there are examples of how FAO has supported certain institutions to become independent and sustainable. One of FAO Somalia’s greatest success stories has been its support to the two seed companies, Daresalam and Sesima, which can now operate entirely independently and have recently purchased their own seed packaging machine. Al Mizan, the sesame oil trading company supported by FAO Somalia, has provided guaranteed market outlets to growers for both sesame seed and grain and has already established linkages with sesame oil importers in Europe and the Middle East on its own initiative. Daresalam supplies seed through its seven outlets to farmers and NGOs. The Shebelle Fruit Association (SHEFA) which has received technical support from FAO to test, import and distribute banana varieties, has demonstrated the association’s capacity by searching for export markets in Turkey on their own. The sustainability of this group of companies (project 611/ EC) is a major achievement.

219. In the livestock sector, as well as building the capacity of government staff as described above, FAO Somalia appears to have built a sustainable PPP model for the provision of animal health through support to the LPAs. Thus, private vets not only provide private services to farmers but can be contracted by government to carry out disease surveillance and services such as vaccination and emergency treatment campaigns on government’s behalf. In south central Somalia FAO contracts the LPA to carry out the service and funds the TFG MOLFR to monitor the work of the LPA. In 2011/ 12 FAO enabled the LPAs in south central to purchase their own drugs for animal treatments from local veterinary wholesale outlets. As well as supporting veterinary PPPs, FAO has recognised the important role that community-based animal health workers and other para- veterinary professionals play in enabling private veterinarians to work profitably in rural areas. FAO has provided limited support for the training of CAHWs (701/NOR, 704/NOR) and worked with their supervisors (803/ITA) to try to prevent drop outs. FAO has recently invested in the training of veterinary para-professionals by providing interim funding to the Sheikh Technical Veterinary School in Somaliland69. All these activities are supportive of a sustainable animal health delivery system in Somalia, and FAO Somalia appears committed to this model of facilitating PPPs in all three administrative areas. However, there is a policy risk that as government capacity strengthens, as it has in Somaliland, directly funded administrations may try to bypass the private sector.

220. Clear and well-thought through analysis underpinning FAO’s projects is critical to their viability and ultimately their sustainability. In the fisheries sector, until official exports begin, the setting up of fish processing centres (with cold chambers) in inland urban centres is critical to the success and sustainability of FAO’s interventions, ensuring there is a market outlet. The potential threat is the rising cost of fuel, which could force up the price of fish in inland areas and jeopardise the viability of the project. In some other market-oriented

68 This could be linked to the ‘New Deal for Engagement in Fragile States’, agreed at the Fourth High-Level Forum on Aid Effectiveness in Busan, which is an opportunity to define priorities with stakeholders in Somalia 69 IGAD will take over full time funding of the School from 2013

64 projects, for example the soap and bone-making project with SOMDA in Somaliland, the lack of market analysis means that the project is progressing without a clear business case that proves its sustainability, as mentioned in section 5.1 above.

221. There are other parts of FAO Somalia’s programme for which sustainability is still a distant goal, in particular the two information systems: FSNAU and SWALIM. As these have been replacing government services, for a couple of decades in the case of FSNAU and for one decade in the case of SWALIM, in the absence of a functioning central government sustainability has not been feasible. Nevertheless, this does not mean that eventual hand-over to government should not be the ultimate goal, and previous evaluations of both information systems have raised issues of sustainability and ownership. As the most recent evaluation of FSNAU commented: ‘Transfer (to government) means more sustainability but on the other hand less efficiency and effectiveness given the high turnover of qualified staff in the governmental services’ XX:5. Giving much greater emphasis to capacity development of government in the next few years, as planned and as described in section 7.1 above, is a very important next step. But a realistic timeframe for completing that capacity development and achieving sustainability must be set, in decades rather than in years. 8.3 Resilience paradigm and strategy 222. FAO, UNICEF and WFP in Somalia have taken the unusual step of coming together to formulate a joint resilience strategy, focused at the household and community level, in which resilience is defined as: ‘the ability to anticipate, resist, absorb and recover in a timely and efficient manner from external pressures and shocks in ways that preserve integrity and do not deepen vulnerability. This includes the ability to withstand threats and the ability to adapt if needed to new options in the face of crises’. (FAO Somalia et al, 2012a: 1) Originally presented at a workshop in Istanbul in May 2012, the three agencies have since begun the process of rolling out the strategy with some initial community consultations in Dollow district in Gedo region and in Burao in Somaliland, with some pilot programming (for example on the Dur Dur project), and by raising funds against the strategy. A common results framework was drafted in May 2012 which has since been developed further.

223. As well as reviewing the written strategy and other documents on resilience that FAO Somalia (in collaboration with WFP and UNICEF) has produced, the evaluation team interviewed a number of FAO staff and other stakeholders (eg IPs and non-IPs, government officers, donor agency staff) to gather their views on FAO Somalia’s new resilience strategy and programming. Hosted by the Rift Valley Institute, the evaluation team held a workshop on resilience in Nairobi in November 2012, exploring the origins of the concept and the aims and objectives of FAO Somalia’s resilience approach in the context of what has been attempted before in Somalia. Based on all of the above, the evaluation team’s analysis of the resilience strategy and the work done so far was written up and shared with the FAO Somalia team at the end of November 2012. The feedback on this part of the work was accelerated ahead of the rest of the evaluation because FAO Somalia’s thinking on, and implementation of the resilience strategy was developing fast so the evaluation needed to feed in immediately. Although the resilience strategy is recent and has been developed towards the end of the period that this evaluation is focused on, it is likely to have a major impact on FAO Somalia’s future strategic direction. The detailed feedback from the evaluation team, shared with FAO Somalia at the end of November 2012, is presented in Annex 7. This section summarises the team’s main findings and conclusions.

224. The rationale for developing the resilience strategy for Somalia relates to a number of factors70. First, there is a sense that two paradigms have dominated international interventions in the last couple of decades: humanitarian assistance and more recently state- building and stabilization, but that neither has really contributed to building the capacity of the Somali people to withstand shocks in a sustainable way. Indeed, international humanitarian aid has not had a strong record in preventing humanitarian crises in Somalia. The bifurcated structure of the aid system into development and humanitarian streams is not well-placed to address complex patterns of acute and chronic vulnerability. New ways of addressing both immediate symptoms as well as the underlying structural causes of vulnerability need to be found so that ‘at-risk’ populations can withstand future shocks sustainably. The joint strategy for enhancing resilience in Somalia is based on three building blocks: • strengthening productive sectors for vulnerable working populations; • basic services to protect human capital; and • predictable safety nets for a minimum of social protection.

225. As the new resilience paradigm sweeps through the international aid sector, it brings with it a window of opportunity to experiment with, and to develop more effective ways of programming, especially in contexts of complex and protracted crises, and an opportunity to challenge the bifurcated aid architecture. FAO Somalia sees this as an opportunity to shift the focus to issues of poverty and vulnerability, re-balancing a preoccupation with state-building in Somalia. Building on previous research carried out by FAO Somalia’s OIC on this topic, when working in FAO headquarters, FAO Somalia is now playing a leading role in promoting this new paradigm within the Somalia context, for example supporting IGAD’s ‘Strategic Plan for Drought Disaster Resilience and Sustainability Initiative’ (IDDRSI)71, and beyond, for example also inputting into the DFID global strategy on resilience. Alongside the other two UN agencies it can continue to capitalise on this leadership role as it pilots and documents its progress in resilience programming.

226. The resilience concept originates from ecology in the early 1970s (Holling, 1973). Revisiting those origins is an important reminder of some of the properties of resilience that are relevant to how it is now being applied in humanitarian and development programming. See Box 2 below. FAO Somalia has chosen to focus its resilience strategy at the household and community level, but a wider systems-approach would appear to be more appropriate to the concept. Indeed, this has triggered lively discussions, especially in Nairobi, about the extent to which governance issues should be included in the three-agency strategy, and therefore why other UN agencies such as UNDP are not key players in the strategy. If a systems approach to resilience is adopted, and it is recognised that government authorities are key stakeholders in interventions at the community level, especially local authorities, then the FAO Somalia approach to resilience does currently appear to be too limited, and needs to be broadened from the community and household level to incorporate and address governance issues. This implies close collaboration with other actors working on governance issues. Drawing on how the resilience concept has been used in ecology, if it is to make a substantial difference in terms of aid programming then it must be looked at in terms of connections across systems, for example linking livelihoods systems with agro-ecological zones/ marine

70 See FAO Somalia et al (2012b) 71 See IGAD (2013)

66 resources and with social systems, and with a long-term perspective72. FAO Somalia is proposing an eight-year timeframe to the resilience strategy, but this is still short. IGAD’s drought resilience strategy is fifteen years which seems a more reasonable timeframe.

Box 2 Key dimensions to the concept of resilience in ecology

• States and dynamics: rather than focus on equilibrium states, focus on the persistence of system elements and function, and distinguish between stability (the ability of a system to return to an equilibrium state) and resilience (the ability of a system to absorb disturbance and persist)

• Systems: resilience is a property of systems, and a function of the interactions and dynamics of different system components across multiple spatial and temporal scales ie actions and interventions on one scale have implications across other scales. Resilience thinking is embedded in the wider social-ecological system

• Adaptability and transformation: adaptability refers to the ability of a system to adapt to changing conditions in order to maintain the system in a desired state; transformation recognises that not all resilient states are ‘good’ and that periods of system change and reorganisation provide opportunities to leverage the system into a more desirable state

• Uncertainty: threats to system persistence can be multivariate and unpredictable, in other words uncertain. A resilient social-ecological system values future potential in balance with current realisation and utilisation of that potential. General and specific resilience may differ ie mechanisms that support resilience to one particular disturbance may undermine other aspects of resilience and reduce general resilience

Drawn from Worden (forthcoming)

227. It could be argued that parts of Somali society have been extremely resilient for decades, even centuries. For example the pastoralist system of production has persisted despite recurrent drought, conflict and the breakdown of formal governance. But there are also large gaps in knowledge and understanding of how Somalia society has transformed and adapted over the last decade, for example how livelihood systems and settlement patterns have changed with displacement and urbanisation. These are gaps that the resilience strategy needs to address, drawing upon the knowledge of Somali civil society as well as FSNAU’s historical database.

228. The motivation for resilience programming, addressing some of the shortcomings in how international aid has been provided and therefore the impact it has had, is not new. There have been earlier attempts in Somalia, particularly in the second half of the 1990s, to ‘link relief and development’, and to find ‘local solutions to local problems’, but with limited success73. As the wider aid community embraces the new resilience paradigm, it may be that circumstances are more conducive now to some of the fundamental shifts in programming that are required to make it work, than they were fifteen to twenty years ago. FAO Somalia’s resilience strategy would benefit from some review and analysis of these earlier efforts, to identify good practice and to learn from what did not work and why. There has been a

72 This point is reinforced by experience in Ethiopia where the large-scale and well-known safety net, the PSNP – Productive Safety Net Programme – is designed to shift millions of chronically food insecure people from recurrent emergency food aid to a cash-based and more predictable form of social protection, in other words to begin to address issues of resilience. Progress has been very slow. 73 As discussed in the resilience workshop in Nairobi, November 2012

67 tendency for FAO Somalia to present components of its resilience strategy as new and innovative. But some of what is being offered is recognised good development practice, for example demand-driven, bottom-up local development, even if it has not always been possible to achieve in the past.

229. Whether it is new or not, the direction that FAO Somalia is promoting, towards a more community-oriented model of development, is to be welcomed although this way of working has not been FAO’s natural comparative advantage. Therefore in order to realise the vision that it has set, FAO Somalia must carry out an analysis of its current resources and skill-sets and come up with a plan on how to build the skill set that will be necessary for community development work. The mode of implementation of FAO Somalia’s resilience strategy is not yet clear, the extent to which it plans to implement directly versus working with NGO partners and government authorities. Community-based development is more naturally an NGO area of comparative advantage. If NGOs are to play a central role in this strategy, then FAO must develop genuine partnerships with them, different to the contractual relationships it has with most of its partners. See 9.2.3 below. FAO Somalia must also find ways of speeding up its administrative procedures so that they are more in synch with the more rapid procedures of other UN agencies with which it will be working closely.

230. The three-agency resilience strategy of which FAO Somalia is part, has attracted a lot of attention and already has a high profile. What it is trying to achieve is important. But as ODI reminds us (Levine et al, 2012), there is no research evidence yet to demonstrate how resilience programming will deliver and be more effective than other types of programming. There is therefore a need to manage expectations, to point out the challenges and required time-scale to have an impact, as well as to monitor and document progress and learning.

9. Cross-cutting issues 9.1 Gender equality and gender sensitivity 231. As described in section 2.2, many Somali women have become the main breadwinner in the household yet do not have equal rights to men, for example to land. As described in an earlier evaluation: ‘Women have been disproportionately affected by the emergency due to pre-existing inequalities. They have been often excluded from assistance or involvement in the process of aid distribution’ (DARA, 2011: 44). To what extent has FAO Somalia taken account of structural gender inequality in Somali society and promoted ‘gender equity in access to resources, goods, services and decision-making in the rural areas’, in line with FAO’s corporate strategic commitment? Following the United Nations Evaluation Group (UNEG) guidelines, gender equality refers to: ‘...the equal rights, responsibilities and opportunities for women and men, girls and boys. It implies that the interests, needs and priorities of both women and men are taken into consideration, recognizing the diversity of different groups of women and men’ (UNEG, 2011: 13). To what extent has FAO Somalia’s programme been gender sensitive, in terms of setting out to understand the different experiences of women and men and power relations between them to inform programming and policy decisions? And to what extent has it adopted FAO’s gender policy.

232. In September 2007 FAO Somalia recruited nationally to appoint a gender/ HIV coordinator for the first time. The coordinator’s mandate was to mainstream gender in FAO Somalia’s country office and to provide support to emergency and development projects. The resources were too limited for such an ambitious objective. Instead, support appears to have been more ad hoc, reviewing projects to encourage greater gender sensitivity and having most input to FAO Somalia’s programme in Somaliland and Puntland where access for the coordinator was most feasible. However, from project visits by the evaluation team, for example to Dur Dur project in Somaliland, there was no evidence that the project had been engendered: the percentage of participating female-headed households was small and there was no evidence that female farmers had been consulted, although apparently efforts have been made to increase the number of women participating in the Farmer Field Schools.

233. Around the same time, towards the end of 2007, senior management in FAO Somalia agreed that staff should participate in training in Socio-Economic and Gender Analysis (SEAGA), along with staff from FAO Sudan and from REOA, and a budget was allocated. This is to be welcomed, but appears to have been a one-off and was not followed up. Thus, although a gender action plan was one of the outcomes of the training, few of the recommendations seem to have been taken up. Since then, over the last four years, the evaluation team was not aware of any gender training that has been offered to FAO Somalia staff.

234. Meanwhile, external requirements to demonstrate gender sensitivity became more rigorous. In 2007 it was clearly indicated that proposals to the CAP must address gender issues and gender marking was introduced. FAO Somalia appears to have been able to meet these requirements on paper, but this does not necessarily reflect the situation on the ground and the evaluation team saw little or no evidence of FAO’s programmes being engendered. In the livestock sector, for example, livestock ownership does appear to be sex disaggregated in Somaliland and south-central Somalia, but not in Puntland. FAO’s drought response livestock proposals state that at least 30 percent of the households targeted for treatment would be

69 female-headed, but a review of project reports reveals that data collected merely states the sex of the livestock owner rather than whether it is indeed a female- headed household, and decisions about targeting were generally made at the time of the animal treatment and vaccination by the veterinary teams in consultation with community elders and ‘other stakeholders’.

235. The ‘magic target’ of 30% of beneficiaries being women continuously reappears in FAO Somalia documentation although the rationale for this target is never given, and there is rarely guidance about how to target. This is the case in the CFW programme, which stipulates that 30% of beneficiaries should be women and youth, but with no indicative breakdown between these two groups nor rationale. Even though a corporate concept note on CFW in Somalia, written in February 2012, raises a number of gender issues to be considered when designing CFW programmes, including ways to reduce barriers to female participation and to create jobs that make use of women's skills and build their assets, there is no sign that these were concretely factored into any project's design or implementation until now. Fieldwork in Gedo Region revealed that women were never consulted and often excluded from participating, especially if they had no male relative to advocate on their behalf. Women interviewed expressed their dissatisfaction over the absence of consultation, although many did ultimately benefit from participating in the CFW projects as described in Section 5.2 above.

236. At least a decade ago, in an evaluation in 2003, FSAU was picked up for its gender weaknesses although it took some time for action to be taken. The first gender analyst was recruited in 2009. The current gender analyst was appointed in December 2011. This has been a key position, supported for six months in 2012 by a Gen-Cap adviser. There is evidence of FSNAU revising its methodologies for data collection, for example in order to collect data and analyse changing roles and responsibilities of men and women in livelihood activities, coping strategies, access to services, and to capture their differential needs. This is a major step forward. However, as the only dedicated gender resource in the large FAO Somalia office, the FSNAU gender analyst has quickly become overwhelmed with demands from elsewhere in the FAO Somalia office as well as from inputting into the clusters.

237. The evaluation team used the gender marker to assess the gender sensitivity of FAO Somalia’s programme in different sectors. In each sector the team scored FAO Somalia as Gender Code ‘1’: gender dimensions are meaningfully included in only one or two of the three essential components: needs assessment, activities and outcomes. The exception was the meat marketing project in Somaliland where 80 to 85% of the meat vendors are women and FSNAU did an assessment of the role of women. This project qualifies for Gender Code 2a: the project is designed to contribute significantly to gender equality and the different needs of women/girls and men/boys have been analysed and integrated well in all three essential components: the needs assessment, activities and outcomes. FAO Somalia’s principle means of addressing gender issues has been through targeting, applying a rule of thumb target that 30% of beneficiaries should be women, but with insufficient analysis, consultation nor consideration of what gender equity might mean in a particular project or programme.

238. In short, although a promising start was made in 2007 to address gender more seriously in the FAO Somalia programme, these efforts have soon petered out and gender

70 issues have not been given priority by senior management. Despite FAO’s corporate efforts to redress a poor corporate record on gender and development74, and the launch of a new FAO Gender Policy in 2012, there is no evidence that these efforts have yet reached FAO Somalia and there has been no accountability for this failing. There have been no efforts to bring in Somali gender expertise to support the programme. FAO Somalia’s own weak record on gender means that it has not been in a position to support government authorities in formulating agricultural and livestock policies that are gender sensitive and inclusive.

239. Ironically, on the few occasions when FAO Somalia has had dedicated gender resources, its contribution has been valued externally within the UN Country Team (UNCT) and IASC. Between 2008 and 2010 FAO co-chaired GenNet with OCHA Somalia. This was a Nairobi based Humanitarian Working Group comprising UN and NGO gender officers and focal points with a focus on humanitarian activities and was intended to engender cluster work, monitor the CAP and coordinate gender training for humanitarian staff. Between 2007 and 2009 FAO co-chaired the UN Gender Theme Group for Somalia with UNDP, and in 2012 FAO’s GenCap adviser supported the CAP process by training agencies in using the gender marker and supporting the cluster coordinators. Although the individual gender advisers may have made a substantial contribution outside FAO, progress within FAO Somalia has been minimal.

240. For gender equality to be taken on board seriously by FAO Somalia, senior management must take the lead with renewed efforts to embark on a process of gender mainstreaming that is properly resourced, and with staff at a sufficiently senior level75. 9.2 Partnerships and modes of implementation

9.2.1 Strategic partnerships 241. FAO Somalia has developed a number of significant strategic partnerships, some at the overall programme level and some at the sectoral level. These include:

(1) The close partnership with UNICEF and WFP in formulating and implementing the joint resilience strategy – an opportunity for three UN agencies to work much more closely at programme level than is usually the case, although this should also involve other UN agencies as appropriate, for example on governance issues.

(2) The close collaboration between FSNAU and FEWSNET, which was particularly important in warning of deteriorating food security and impending famine in 2011.

(3) A number of strategic partnerships in the livestock sector which provided a solid foundation for FAO animal health initiatives, including: a. being part of the Somali Ecosystem Rinderpest Eradication Project (SERECU) managed by the African Union’s Interafrican Bureau of Animal Resources (AU-IBAR), and the Terra Nuova-led consortium that made up the Somali Animal Health Services Project (SAHSP). Both projects were instrumental in demonstrating the eradication of rinderpest from Somalia and building

74 See the FAO Gender Audit and evaluation of ‘FAO’s role and work related to gender and development’ 75 As long as FAR has a vacant gender position, support may have to be forthcoming from elsewhere, eg from FAO HQ

71 capacity that FAO was successfully able to utilise for its complementary animal health interventions; b. FAO has a potential strategic partnership to develop through AU/IBAR’s new ‘Veterinary Governance’ project funded by the EC.

(4) FAO, particularly the investment centre in Rome, has been developing a strategic partnership with IGAD and its regional platform for drought resilience and sustainability. This platform will work closely with the Global Alliance for Action for Drought Resilience and Growth and is potentially important as FAO develops its resilience programme which will probably be more conflict-oriented.

242. FAO Somalia has developed strategic partnerships with the respective government authorities, especially in Somaliland and Puntland in the agriculture and livestock sectors. The partnerships are at an earlier stage with the ministries in Mogadishu. This means that FAO is well-placed to support future capacity development as well as policy and regulatory work.

243. FAO Somalia’s strategic partnerships are weakest in terms of its cash-based programming. Since the famine of 2011 there has tended to be a divide between agencies involved in unconditional cash transfers (UCTs) and those involved in CFW (most notably FAO). Efforts are now being made to bridge this divide with a planned pilot project to implement simultaneously unconditional and conditional cash transfers in Hiran, involving Save the Children, FAO Somalia and UNICEF. This kind of collaboration will be important for reciprocal learning, not least to identify in which circumstances which type of intervention is most appropriate, and to inform the resilience programming.

9.2.2 Changing mode of implementation 244. Up until 2009 FAO Somalia did much of its implementation through international NGOs. Under pressure from local administrations in Somaliland and Puntland it shifted to working more through government in both the livestock and agriculture sectors. This is positive in terms of developing stronger partnerships with government and beginning the long-term process of capacity-development. It is widely claimed that this is a more cost- effective approach as it avoids INGO administrative costs but it has not been possible to gain access to data that substantiates this claim. This shift in mode of implementation also comes with a cost: the loss of INGO-community relationships and local affiliations, loss of INGO technical expertise, and of their ability to reach remote areas. As the new Somali National Government establishes itself in Mogadishu, a similar shift in mode of implementation may take place in south-central Somalia as FAO builds its relationship with the new government. This is an opportunity to apply the learning from how the shift has been managed in the north, to manage tensions which may emerge between government and NGOs, to avoid NGO resentment and to ensure that NGO expertise is not lost.

245. In the livestock sector FAO Somalia has increasingly worked through LPAs, private sector organisations. This has been particularly important in ensuring FAO’s continued ability to support vulnerable populations in Al Shabaab controlled areas from 2009.

9.2.3 Implementing partnerships 246. FAO’s corporate ‘Policy and Strategy for Cooperation with Non-Governmental and Civil Society Organisations’ defines partnership as: ‘cooperation and collaboration between

72 FAO units and external parties in joint or coordinated action for a common purpose. It involves a relationship where all parties make a contribution to the output and the achievement of the objectives rather than a solely financial relationship’76. This policy appears to have been formulated with stable and peaceful contexts in mind in contrast to the Somalia context. Nevertheless, this definition has been used in evaluating the mode and nature of FAO Somalia’s implementing partnerships, in this section taken to refer to its partnerships with national and international NGOs.

247. Partnership with NGOs is critical to FAO’s work in Somalia, especially in large parts of south-central where FAO does not have secure access but where local organisations have been able to negotiate access. The range of NGOs with which FAO Somalia is partnering is extremely broad, in terms of capacity, expertise and experience, and there is not a common view amongst them about their experience of partnership with FAO although some common patterns and issues emerged in interviews with FAO’s IPs and in discussion with FAO staff.

248. First and foremost, most of FAO’s implementing partnerships are principally contractual, and therefore do not follow the definition above. This is immediately evident from the LoAs which use the term ‘service provider’ rather than ‘partner’, and which clearly spell out what is expected of the NGO, providing instructions and indicating how the NGO must comply with FAO requirements. This may be appropriate in certain circumstances, for example when FAO Somalia had to scale-up its emergency programme rapidly in 2011 and engaged new IPs at short notice. It is less appropriate in development programming where a longer-term and more equal relationship should be possible. This approach is resented by many NGOs who feel they have no input into needs assessments nor project design. Some compared FAO unfavourably with other UN agencies which allowed their IPs to draft their own proposals and then revised them together which could be a valuable learning process for the NGO. In the words of one of FAO’s IP staff members: ‘through its tenders, FAO-Somalia pretty much says to its partners, can you do this work for us rather than what is it that you can do for specific communities. The FAO’s tender is based on what FAO wants a particular community to have and not what the community actual wants. For instance, a particular community may want to have food but FAO wants to build four canals in a specific village’. This concern was echoed by members of farmers’ associations from Middle Shebelle, met in Mogadishu, who commented that they are informed by the NGOs about the intervention that FAO has chosen to implement eg canal rehabilitation, but are not consulted about their needs or priorities.

249. The predominance of the contractual relationship over a relationship of genuine partnership may act as a disincentive to stronger INGOs who choose not to partner with FAO because of their loss of independence. Another consequence of this one-way relationship is that FAO loses out on local NGO knowledge.

250. Many projects seem to have had a short time-frame, sometimes just three to six months, which is not conducive to FAO building a relationship with the IPs, nor the IPs building a relationship with local communities. Only where IPs have had repeat contracts

76 See the FAO Organization-wide Strategy on Partnerships document.

73 does it appear that consultation between FAO and the NGOs has improved over time, for example in the agriculture sector where the ‘gatekeeper’ of the funds, FAO, and the implementers, the NGOs, have become mutually dependent partners and a more constructive relationship has developed. The trend towards longer LoAs in the agriculture sector, mentioned in section 7.2 above, is seen as a positive development.

251. Communications and information-sharing, between FAO and its IPs and within the IPs, emerges as an issue. Some NGOs felt that FAO Somalia could improve the way it shares information with them, but it also emerged that information sharing is often a problem within NGOs, between senior NGO staff based in Nairobi and their staff implementing projects in the field, again a casualty of Nairobi-centric decision-making. FAO’s remoteness from the field and infrequent field visits, even to more secure areas, means that it loses touch with some of the realities of implementation on the ground. One example of this that emerged in Dollow district is the amount of time it takes IPs to negotiate the safe passage of seed and other agricultural inputs, which becomes a costly process, but does not seem to be fully appreciated by FAO and is not included in the project plan. This indicates a lack of flexibility to accommodate challenges faced in the field, not all of which may be predictable.

252. A fundamental issue is the mismatch between FAO’s reporting requirements, for timely written reports, and the ability of local NGOs to fulfil those reporting requirements. In the words of one of FAO Somalia’s staff members: ‘on the ground, they may be doing an excellent job. They are visible and interact with the targeted communities. But the problem is when FAO asks them to provide progress reports and information on how the work is going – getting such information at the agreed time becomes very difficult’. This raises an issue about how FAO Somalia can support and develop the capacity of its IPs, especially local NGOs. Most IPs interviewed commented on the limited training and capacity-building they had received from FAO, with the exception of some CFW training and training provided by the Food Security Cluster. The training needs they highlighted relate to basic but essential skills of financial management, administration and report writing.

253. There is clearly a compromise that FAO Somalia has to reach, between acting responsibly and professionally, for example in selecting its IPs and ensuring programme delivery, especially for service provision, and engaging in genuine partnership where the knowledge and experience of NGOs can be fully utilised. According to FAO Somalia staff in Nairobi, where possible they want to move away from highly contractual relationships with their IPs, to develop more genuine partnerships. This is positive, but will require considerable commitment to overcome the asymmetric power relationship that naturally exists between a large UN agency which controls the funding and a small NGO with limited resources. Overall, FAO Somalia must develop a more nuanced approach to partnership, consciously adapting the nature of the relationship to the situation, the NGO, and the desired outcome.

9.2.4 Public-private partnerships 254. FAO Somalia has a mixed record in implementing PPPs. On the one hand there are successful experiences to be built upon, but there are also failed attempts at public-private partnership which offer important learning to avoid the repeat of such failures.

255. There are positive examples from the agriculture sector – the seed companies that FAO supported that are now independent and sustainable – and from the fisheries sector where two landing (processing sites) in Puntland are effectively managed by private enterprises: the Bargal Fishing Company and Fish Net. Factors that have contributed to these

74 successes include: a clear vision from the outset, and clear and careful planning that explored feasibility and has resulted in a logical sequence of implementation that ended with handover to private companies. Establishing and supporting LPAs in the livestock sector is another example of FAO’s support in creating successful PPPs. In this case, continued support from FAO over time has been critical, for example in supporting the role of the LPAs in discussions with the governments of Somaliland and Puntland to ensure the private sector is not bypassed. This is a clear commitment in FAO’s 5-year strategy.

256. PPPs have been less successful in taking over management of the slaughterhouses that FAO has constructed in Somaliland and Puntland. As explained in section 5.1.6, serious design flaws have undermined the viability of the slaughterhouses in Bosasso and Boroma. The slaughter houses are not compliant with the original specifications, in the case of Boroma slaughter fees cannot be agreed upon and it is currently being operated by the municipality in a manner that defies its original purpose, is unlikely to be sustainable, and is not transparent. The Bosasso slaughter house is not even operational because of design faults. In Hargeisa, although the livestock market has been successfully rehabilitated, management of the market has not yet been handed over to the private sector. The learnings from these various experiences include:

(1) weakness at the design stage can compromise an entire project and its future sustainability in terms of management, an important learning for FAO which is not traditionally an operational agency; (2) clear sequencing and ongoing support from FAO is essential to the success of a PPP being launched and becoming sustainable; (3) building a sustainable PPP takes time, and the length of that process may be at odds with tight project deadlines. This has been the case with the Hargeisa livestock market. The NGO, Terre Solidali, was sub-contracted to facilitate hand over of management to a private company but the project was not completed during the period of Terre Solidari’s contract with FAO so the handover never took place and the market is currently managed by the municipality, apparently unsustainably. More realistic (and sometimes flexible) timeframes must be given for such complex institutional processes.

10. Conclusions and recommendations 10.1 Conclusions 257. The period that this evaluation covers – 2007 to 2012 – has been a time of great political instability in Somalia and a time when the country has been the focus of strong international political strategic interests. 2006 to 2010 have been some of the most violent years in Somalia’s history, directly contributing to the famine of 2011. The north – Somaliland and Puntland – have been more stable throughout this period and more developmental interventions have been possible, but they have also been in a state of ‘political limbo’ as the future status of these two territories remains unresolved. With the appointment of a new government in Mogadishu in September/ October 2012, there is a greater sense of optimism about the future than there has been for many years, but the situation remains fragile and volatile.

258. In this challenging and unpredictable context, FAO has succeeded in scaling up its programme in Somalia despite the challenges of remote management, has adapted well to the context of failed formal governance, and has made some major contributions. In a number of ways it has stepped into the vacuum of central government, providing an essential information service (FSNAU and SWALIM) to inform planning and decision-making by national and international stakeholders, it has supported the private sector to provide critical services for agriculture and for livestock (for example, private seed companies and the LPAs), it has rehabilitated dilapidated agricultural infrastructure (for example, irrigation infrastructure), it has supported the formulation of legal, regulatory and institutional frameworks for the meat industry, and it has supported Somalia to remain engaged in the international market (for example through the export of live animals)77. These are examples of how FAO Somalia has developed a programme that is relevant and appropriate to the context of Somalia, mostly in line with the priorities of the respective government authorities and drawing on FAO’s core strengths. But there are also gaps in FAO Somalia’s portfolio. First and most notably, there is a gap on natural resource management, a pressing need in the context of failed or failing governance. Despite the investment in SWALIM, FAO Somalia has not followed through (until very recently) on natural resource management. The second gap is urbanisation: as the settlement pattern in Somalia has changed rapidly with displacement, FAO Somalia must become more engaged in peri-urban agriculture, following the movement of people from rural to urban areas. A third gap is playing a leadership role in addressing chronic under-nutrition in Somalia through agricultural interventions. FAO has been slow to develop its fisheries programme, despite the fisheries potential in Somalia; this sector deserves much greater attention in the future. A question mark also hangs over the crop-bias of the programme and whether this should be redressed with a greater focus on livestock, reflecting the high livestock dependence of the population.

259. FAO, through FSNAU, played a vital role in providing early warning of deteriorating food insecurity in 2010/11 and made a major contribution to discussions and decisions about cash-based responses. Ultimately it was the key player alerting the world to famine in Somalia, at which point a large-scale international humanitarian response was launched. But this was too late to prevent many excess deaths. Early warning did not trigger

77 These are all examples of how FAO Somalia has, appropriately, make a significant contribution in terms of ‘policies, institutions and processes’ to support livelihoods – see Figure 1 in this report.

76 early response. There were many factors mitigating against a timely response, and this implies that warnings of crisis and famine should have been supported by massive advocacy, from FAO in Nairobi as well as from FAO in Rome to draw attention to the seriousness of the situation, especially from March 2011 when the first warning of famine was made. The fact that FAO itself did not launch an appeal for additional funds to scale-up its emergency response until famine was declared towards the end of July 2011 implies that it, too, was not sufficiently responsive to its own early warning of famine.

260. Despite these shortcomings, in many ways, growth, innovation and leadership have been the hallmarks of the FAO Somalia programme over the last six years. This has resulted in a dynamic, high-profile programme in which FAO is more operational than usual and enjoys a high level of donor support and confidence. It is a very different programme to most FAO programmes and offers an alternative model in which FAO has developed a leadership role, strategically and technically, and has pioneered approaches that are innovative to FAO, for example the programme approach which is an opportunity to be more strategic and should ultimately be more efficient. At the corporate level there is much for FAO to learn from the way that FAO Somalia has evolved, so that this is not a ‘one-off’.

261. The dynamism and leadership that have made all of this possible in FAO Somalia are a credit to the wider organisation. But they come with their challenges. On the one hand, the fast pace of growth was appropriate to the rapid scale-up that eventually took place in response to the 2011 famine, especially as livelihood activities replaced the usual large-scale emergency food aid response. FAO Somalia rose to the challenge and the scale-up and adaptation of the CFW programme eventually supported many households during a period of acute food insecurity78. But on the other hand, through this period of crisis and scaling up, there has been little or no time to take stock and the environment has not been conducive to pausing to question some of the basic assumptions which underpin large parts of the programme, further exacerbated by a lack of data on outcomes and impact. Now that the FAO Somalia programme is well-established, with a strong and diversified funding base, this is the time to carry out more rigorous analysis, particularly of interventions in the livestock and market sectors to explore whether these are indeed the most appropriate choice to fulfil FAO Somalia’s goals of strengthening food security, reducing poverty and addressing vulnerability, or whether there are other programme interventions that would achieve these goals more effectively.

262. Where FAO Somalia’s programmes have fallen short, design flaws mostly seem to be at fault, for example the construction of some slaughterhouses in the north, and in some cases setting up PPPs to manage projects. A general weakness across much of FAO Somalia’s programming has been lack of consultation, especially at the community level. Although there are parts of the country where this is difficult if not impossible, more could have been done elsewhere. This is an important message as FAO Somalia moves into community-oriented resilience programming: the incentives and skill-sets needed to carry out community consultation must be strengthened. There are instances of more effective community consultation within the programme, for example in the fisheries sector. The rest of the programme needs to learn from these good practice examples.

78 This is an example of FAO addressing the ‘assets’ component of the livelihoods model presented in Figure 1.

77 263. Apart from FSNAU, FAO Somalia’s performance in building a gender sensitive programme and in promoting gender equality is dismal. Early investment in 2007/08 in engendering the programme has not been followed through. Although proposals on paper may appear ‘engendered’, there is little evidence that this is followed through on the ground and much of the programme appears gender-blind.

264. As more of FAO’s programme has had to be managed remotely, especially during the emergency, it has taken its monitoring responsibilities towards its IPs seriously in terms of compliance and has implemented rigorous procedures. This is where most of its efforts and resources have been focused, on process and output monitoring. What is badly lacking is outcome and impact monitoring and therefore more insightful analysis of the consequences of FAO’s work on the ground. This is a major shortcoming, in meeting accountability requirements as FAO has been the recipient of ever-larger amounts of aid funding, and for FAO’s own learning. FAO Somalia is beginning to address this gap, albeit late in the day. In the context of Somalia this may need a very different approach to procedural and survey- based quantitative monitoring, drawing instead on more participatory and qualitative approaches that are more flexible and can be adapted to a volatile and insecure context.

265. As FAO Somalia has grown, it now has a large team and the functions of a large office to service, mostly Nairobi-based. FAO has been slower than most other UN agencies to deploy international staff into Somalia, especially into the more secure north. These two factors combined have contributed to a Nairobi-centric operation; there is a risk of much of the strategic thinking being developed in Nairobi (for example on the resilience strategy), to which some government officials and other stakeholders may be invited to contribute but over which they do not feel a sense of ownership. As the programme has grown, staff who are based in the field often feel out-of-touch with thinking and decision-making in Nairobi. The opportunity to change this is imminent with the planned move of many operational functions to Mogadishu and with more staff relocated to Hargeisa and Garowe. On the one hand this should root the programme more firmly in Somalia, especially if staff engage in more field travel and spend more time with local communities in areas that are accessible. On the other hand, no matter how well-managed the process is, this is likely to be a period of turbulence and transition as staff leave and new staff have to be found who are prepared to relocate to an environment that remains fragile and insecure (especially Mogadishu), from senior management downwards. This means a different pace of programming may be required, focused less on growth but instead on transition and on carrying out deeper analysis of the effectiveness of FAO Somalia’s work.

266. At the same time, FAO Somalia is re-orienting its way of working to follow the resilience paradigm. This offers a valuable opportunity to experiment with a different approach to programming that promises to be more appropriate in a situation of protracted crisis, and to work closely with two sister UN agencies. However, in an unpredictable context where many different approaches have been tried over the years, there is unlikely to be a ‘magic bullet’. A carefully calibrated process of ‘trial and error’ will be necessary, with a long time perspective and a systemic approach that addresses and incorporates governance.

267. FAO Somalia has modelled well what decentralisation from HQ to country office can look like and how it can work for the wider organisation. Despite this greater autonomy, however, there are still systemic weaknesses in the wider FAO that could threaten the future of the FAO Somalia programme. The most pressing issue is leadership and succession to ensure that all efforts are made to maintain and continue strong management and leadership

78 of the programme. The stakes are high and failure to manage succession could seriously damage the programme and FAO’s reputation more widely. Second, some of FAO’s procedures remain bureaucratic and cumbersome. They can, and have hampered FAO Somalia’s performance, in particular lengthy and delayed recruitment processes. Third, there is still a lack of clarity about how technical support will be provided post-decentralisation and how the wider organisation can make best use of high levels of technical expertise within a country office like FAO Somalia. 10.2 Recommendations

RECOMMENDATIONS FOR FAO SOMALIA

Recommendation 18: Despite the operational challenges, FAO Somalia needs to strengthen the culture and practice of community consultation within the programme, from the design stage onwards, paying attention to dynamics within communities so that all voices are heard. Strengthening these skill-sets is especially important as FAO-Somalia embarks on a journey of community-based resilience programming. Action: FAO Somalia

Recommendation 19: FAO Somalia must invest in more rigorous analysis to underpin its programming and to test some of the assumptions on which programme choices and decisions have been made, for example about the best ways to support vulnerable households. This is an opportunity to link to research findings elsewhere in the region, for example on pastoralist systems of production, and to explore their relevance to the Somalia context. To support this more rigorous analysis, peer review mechanisms for different parts of the programme should be considered, involving experts from within FAO (HQ and SRO) as well as experts from outside. Action: FAO Somalia, FAO HQ and SRO

Recommendation 20: Linkages between the livestock and agriculture sectors in FAO- Somalia should be strengthened, especially around improved animal nutrition and fodder production. Closer sectoral working should be facilitated by FAO-Somalia’s shift to a programme approach so that synergies between different sectors can be more fully developed. Action: FAO Somalia

Recommendation 21: In the livestock sector: a. More holistic programming should be developed to complement FAO’s animal health initiatives with more substantial efforts to improve livestock productivity across a range of production systems and to rehabilitate degraded rangelands. Such livestock programmes should work to address vulnerability and poverty; b. Market-oriented interventions should add value at the production end of the chain, on the farm and feedlots, so that quality meat and milk products can be produced, thus boosting farmer income and creating employment opportunities along the value chain; c. FAO Somalia needs to ensure Livestock Emergency Guidelines and Standards (LEGS) are utilised by its own staff and partners when addressing future emergencies; d. FAO HQ should develop standard designs for slaughterhouses, indicating minimum facilities to be included, to use as blueprints for future slaughterhouse construction. Action: FAO Somalia

79 Recommendation 22: There needs to be greater investment in trend analysis, drawing on FSNAU and SWALIM data. FAO Somalia is uniquely placed to carry out such trend analysis to contribute to a deeper understanding of how livelihoods, the economy, and in turn poverty and vulnerability have been affected by years of conflict and displacement. This should inform FAO’s strategic programme choices and decisions, for example whether the current priority in the programme given to crops rather than livestock is appropriate, and will be a key resource for national and international actors in Somalia. It may also provide insights into the causes of chronic food insecurity in Somalia. Action: FAO Somalia

Recommendation 23: FAO Somalia must ramp up its advocacy skills and resources to match its early warning expertise in FSNAU, to advocate for a timely response before famine conditions develop. Action: FAO Somalia

Recommendation 24: New programming should be developed in three areas: a. to address issues of natural resource management. Using SWALIM data and analysis, a strategy for FAO Somalia’s engagement in this sector should be drafted; b. in peri-urban agriculture with two objectives, first to support the livelihoods of vulnerable households living in or near urban areas, and second, to support peri-urban agricultural and livestock development; c. as the custodian of substantial nutrition analysis, FAO Somalia should feed this directly into its agriculture and food security interventions to address long-term chronic under- nutrition in Somalia, and to play a leadership role in demonstrating how agriculture and food-based approaches can contribute to tackling malnutrition. Milk production and consumption should be central to this; d. although not ‘new’ programming, FAO Somalia’s recent commitment to the fisheries sector must be sustained. Action: FAO Somalia

Recommendation 25: The FSC is still establishing itself. In co-coordinating the FSC: a. FAO must take steps to ensure the large size of the cluster does not detract from its effectiveness. Revitalising the technical working groups, for example, is an opportunity for more focused and manageable technical debate and development of guidance material; b. communication and consultation with the field must be stepped up to reverse the tendency towards Nairobi-centric decision-making; c. ways of engaging with other key actors who are not part of the FSC must be found eg with the OIC, and with development coordination bodies. Action: FAO Somalia

Recommendation 26: The current compliance focus of FAO Somalia’s M&E needs to be re- balanced so that equal attention is given to outcomes and impact and to ensure that the M&E system feeds into learning as well as fulfilling accountability requirements. Re-thinking M&E as part of the resilience strategy offers an opportunity to develop an overall M&E framework, to review the role of the M&E unit and its relationship to the sectoral teams, for example the M&E Unit could maintain a focus on compliance monitoring while the sectoral teams lead in learning and impact and outcome evaluation. This is also an opportunity to review the role that well-designed participatory and qualitative exercises could play in the challenging

80 operational context of Somalia in capturing outcomes and impact, learning from elsewhere where access is constrained, and to ensure that M&E becomes gender-sensitive Action: FAO Somalia

Recommendation 27: FAO Somalia must develop a differentiated and more nuanced approach to partnership, at a minimum differentiating between (i) contractual service provision, driven by an LOA, when the term ‘partner’ may not be appropriate, and (ii) genuine partnership where two organisations come together as equals and the relationship is guided by the principles of partnership often articulated in memorandum of understanding (MoU). FAO Somalia should consciously decide which approach is most appropriate in which circumstances. This should be informed by a review of how current procurement and partner contracting rules may actually stifle true partnership and innovation to see where adaptations can be made. This will be particularly important as FAO Somalia develops its community-based resilience programme and aims to work in close partnership with NGOs. Action: FAO Somalia

Recommendation 28: FAO-Somalia should develop a strategy for mainstreaming gender across the organisation, supporting staff to implement gender-sensitive programmes and to consider what gender equity means. This requires leadership and also a senior management position to drive such a strategy, and to develop an appropriate structure whereby addressing gender is everyone’s job rather than just an adviser’s job (for example, a gender focal team should be considered). This also needs to be properly resourced over a realistic timeframe to avoid one-off interventions. Action: FAO Somalia

Recommendation 29: FAO Somalia should formulate a framework for capacity- development as part of its revised strategy, to guide a more coherent and effective approach, especially in capacity development of government. This should pay attention to the three dimensions of capacity-development: individual, organisational and the level of the enabling environment79, drawing on experience of what has worked and what has not worked so far. This should be aligned with a broader UN-wide coherent approach to capacity-development. For capacity-development with government, realistic and long-term timeframes are needed. All capacity-development programmes should be preceded by a thorough capacity assessment. Action: FAO Somalia

Recommendation 30: FAO Somalia’s resilience strategy should be extended to 15 years (broken down into 3 year programming cycles for funding and review purposes), should be broadened to be more systemic, addressing governance issues, and must be carefully rolled out to ensure that all FAO staff are fully on board. There should be investment in research from the beginning of the programme to run alongside programming, and a review of FAO Somalia’s current skill sets to assess how these must change and/ or be strengthened for effective implementation of the resilience programme. See also Annex 7 for more detailed recommendations on this issue. Action: FAO Somalia

79 See FAO’s learning modules on capacity development

81 Recommendation 31: FAO Somalia must strengthen its HR management capacity to address the weaknesses that have been identified in this evaluation eg slow recruitment, lack of induction, poor communication with field offices, and to support the proposed office move to Mogadishu as well as decentralisation to Hargeisa and Garowe. Support from FAO Rome will be needed. Action: FAO Somalia

Recommendation 32: The relocation of much of FAO Somalia’s operations from Nairobi to Mogadishu, Hargeisa and Garowe is likely to be a period of turbulence when there may be many disruptions to programming, including loss of staff. This must be taken into account in the pace of programming which should be focused less on growth and more on the quality of FAO Somalia’s core activities as well as carrying out deeper analysis of the effectiveness of FAO Somalia’s work. Managing an effective period of transition may even require some programmes to be temporarily downscaled. Action: FAO Somalia

RECOMMENDATIONS FOR LEARNING ACROSS FAO

Recommendation 33: On learning: a. Where FAO Somalia has developed examples of good practice, for example in scaling up and establishing the CMU during the 2011 famine, and in some of its PPP work, these should be documented and shared with the wider organisation; b. FAO’s sub-regional office should initiate and facilitate learning across the region, between FAO country offices working on similar issues sometimes in similar contexts, for example on public information systems and capacity development with government. Action: FAO Somalia, FAO HQ and SRO

RECOMMENDATIONS FOR FAO ROME

Recommendation 34: Four recommendations for FAO Rome emerge from this evaluation: a. planning for succession of the leadership of FAO Somalia should be a priority; b. FAO must speed up and modernise its recruitment practices and improve its human resource practices if it is to be competitive in the international arena and if it is to attract high calibre candidates; c. FAO should learn from the programme approach that has been pioneered by FAO Somalia and should promote this at country level over the project approach. It should also develop a sound financial management system that is appropriate to a programme approach yet also accommodates the more specific reporting requirement of individual donors; d. how technical support is going to be provided within FAO’s decentralised structure requires further thought and planning, including how to integrate high levels of technical expertise that may exist within a country office. Action: FAO HQ

82 Bibliography

Aklilu, Y., and Catley, A. (2010). ‘Mind the Gap. Commercialization, Livelihoods and Wealth Disparity in Pastoralist Areas of Ethiopia’ UKAid and Tufts FIC, Medford, MA. December.

Bailey, R. (2012) ‘Famine Early Warning and Early Action: The Cost of Delay’, Chatham House, London, July

Bell, L., Vianney, L., Ondijo, D. (2008) ‘Joint Evaluation. Final Report. Support to the Food Security Analysis Unit (Phase V) – Somalia and Nutrition Information Project (Somalia)

Besterman, C., and Cassanelli, L.V. (1996) The Struggle for Land in Southern Somalia: the war behind the war, London: Haan

Bradbury, M. (1998) ‘Normalising the Crisis in Africa’, Disasters 22: 328-338

Bradbury, M. (2010) ‘State-building, Counterterrorism, and Licensing Humanitarianism in Somalia’, Briefing Paper, Feinstein International Center, Tufts University, September

Buchanan-Smith, M. and Davies, S. (1996) ‘Famine Early Warning and Response: The Missing Link?’, London: IT Publications

Catley, A., Abebe, D., Admassu, B., Bekele, G., Abera, B., Eshete, G., Rufael, T. and Haile, T. (2009a) ‘Impact of drought-related livestock vaccination in pastoralist areas of Ethiopia’. Disasters 33/4, 665-685 1

Catley, A., Burns, J., Abebe, D., & Suji, O. (2009b) ‘Participatory Impact Assessment, A Guide for Practitioners’. Medford: Tufts University

Catley, C, Lind, J. Scoones, I (2012) Pastoralism and Development in Africa, Dynamic Change at the Margins Eds. Andy Catley, Jeremy Lind, Ian Scoones. Chapter 7. ‘Moving Up or Moving Out? Commercialization, Growth and Destitution in Pastoralist Areas’ Andy Catley and Yacob Aklilu. Publisher Routledge Http://www.routledge.com/books/details/9780415540728/

Checchi, F. and Courtland Robinson, W. (2013) ‘Mortality among Populations of Southern and Central Somalia Affected by Severe Food Insecurity and Famine during 2010-2012’, a study commissioned by FAO/ FSNAU and FEWSNet from the London School of Hygiene and Tropical Medicine and the John Hopkins University Bloomberg School of Public Health, May

DARA (2011) ‘IASC Evaluation of the Humanitarian Response in South and Central Somalia 2005 – 2010’

Darcy, J. Bonard, P. and Dini, S. (2012) ‘IASC Real-Time Evaluation of the Humanitarian Response to the Horn of Africa Drought Crisis. Somalia 2011-2012’, May

Devereux, S. (2006) ‘Vulnerable Livelihoods in Somali Region, Ethiopia’. IDS Research Report No. 57. Sussex, UK: IDS

FAO (2009) ‘Strategic Framework 2010 to 2019’ Conference. Rome 18 to 23 2009

FAO (2012) ‘Somaliland. Poverty Alleviation and Increased Food Security, Contributing to the Way Forward to Economic Development, Peace and Security. Food Security Response and Sector Rehabilitation. Agriculture - Interim Assistance Plan 2012-2014’

FAO Council (2012) ‘Outline of the Reviewed Strategic Framework’. Hundred and Forty- Forth Session. Rome 11-15 June 2012

FAO Somalia, UNICEF Somalia, and WFP Somalia (2012a) ‘Somalia. A Resilience Strategy’, Istanbul II, May

FAO Somalia, UNICEF Somalia, and WFP Somalia (2012b) ‘Promoting resiliency for at risk populations: lessons learned from recent experience in Somalia’

FAO/EC/World Bank (2004) ‘Somalia. Towards a Livestock Sector Strategy. Final Report 2004-9’, Report No.: 04/001 IC–SOM

FAO Somalia (2010) ‘For A World Without Hunger. FAO Strategy and Plan of Action. 2011-2015’

FSNAU (2011) ‘Livelihood Baseline Analysis Addun pastoral. July 4, 2011’ . Accessed Dec 2012 http://www.fews.net/docs/Publications/so_Addun-Pastoral-Livelihood-Zone-Baseline- Report-June-2011.pdf

Gardner, J. (2007) ‘Gender Profile for Somalia – An Executive Summary’, EC Somalia Unit, Kenya and NORAD, January

Githinji and Wanyama (2010). Making markets work for the poor in Somaliland livestock value chain analysis report, 2010.

Haan, N., Devereux, S., and Maxwell, D. (2012) ‘Global Implications of Somalia 2011 for Famine Prevention, Mitigation and Response’ in Global Food Security, Vol 1, No.1: 74-79, November. Special Issue on the Somalia Famine of 2011-2012

Hammond, L and Vaughan-Lee, H. (2012) ‘Humanitarian Space in Somalia: a Scarce Commodity’ HPG Working Paper, April

Hashi, A.M., Castiello, M. and Musiime, J 2007 Background Analysis of Veterinary Sector Development in Somalia, FAO Somalia report 2007

Hedlund, K., Nicholson, N., Maxwell, D., and Majid, N. (2013, forthcoming) ‘Final Evaluation of the Unconditional Cash and Voucher Response to the Southern and Central Somalia Crisis 2011-12’, Humanitarian Outcomes

Hillbruner, C., and Moloney, G. (2012) ‘When Early Warning is Not Enough – Lessons Learned from the 2011 Somalia Famine’ in Global Food Security, Vol 1, No.1: 20-28, November. Special Issue on the Somalia Famine of 2011-2012

84 Holling, C. S. (1973) ‘Resilience and Stability of Ecological Systems’. Annual Review of Ecology and Systematics: 1–23.

IGAD (2013) ‘Somalia. Country Programming Paper. IGAD Strategic Plan for Drought Disaster Resilience and Sustainability Initiative (IDDRSI)’, Technical Working Document prepared with the support of the Technical Consortium (CGIAR & FAO), 7 January 2013. Draft

Landell Mills and VetEffect (2012) ‘Evaluation of the Somalia Food Security and Nutrition Analysis Unit (FSNAU)’ Revised Draft – Final Report, April

Lautze, S., and A. Raven-Roberts (2006) ‘Violence and complex humanitarian emergencies: Implications for livelihoods models’. Disasters 30 (4): 383–401.

Lautze, S., Bell, W., Alinovi, L., and Russo, L. ‘Early Warning, Late Response (Again): the 2011 Famine in Somalia’ in Global Food Security, Vol 1, No.1: 43-49, November. Special Issue on the Somalia Famine of 2011-2012

Levine, S., Pain, S., Bailey, S., and Fan, L. (2010) ‘The Relevance of “Resilience”’, HPG Policy Brief 49, September, ODI: London

Majid, N., and McDowell, S. (2012) ‘Hidden Dimensions of the Somalia Famine’ in Global Food Security, Vol 1, No.1: 36-42, November. Special Issue on the Somalia Famine of 2011- 2012

Masiga, W., Kajume, J. and Daborn, C. (2006) ‘An Assessment of Veterinary Service Delivery Systems in the Somali Ecosystem – Consultancy Report prepared for AU- IBAR/PACE-SERECU’, Nairobi, August 2006

Maxwell, D., and Fitzpatrick, M. (2012) ‘The 2011 Somalia Famine: Context, Causes and Complications’, in Global Food Security, Vol 1, No.1: 5-12, November. Special Issue on the Somalia Famine of 2011-2012

Menkhaus, K. (2010) ‘Stabilisation and humanitarian access in a collapsed state: the Somali case’, Disasters, Vol 34, ODI

Menkhaus, K. (2012a) ‘Somalia’s 20-Year Experiment in Hybrid Governance’ Network Al- Shahid, August 12 2012

Menkhaus, K. (2012b) ‘Conflict analysis: Somalia’, prepared for UN Political Office for Somalia, October

Meyers, C., De Haan, N., Kimani, T. and Amanfu W. (2009) ‘The Impact of Peste des Petits Ruminants (PPR) on Livelihoods in the Arid and Semi-Arid Lands of Kenya’. Report funded by FAO Kenya, coordinated by FAO ECTAD.

UNEG (2011) ‘Integrating Human Rights and Gender Equality in Evaluation – Towards UNEG Guidance’

85 Wafula, A. (2012) ‘Technical Review of SWALIM’s Performance in Reaching its Target Group and Responding to their Information Needs’, Wes and Abbott Consults Ltd, September

Worden, J. (forthcoming) ‘Resilience and Development. Ecological origins and opportunities in the Horn of Africa’, Rift Valley Institute

World Bank (2006) ‘Somalia – from Resilience towards Recovery and Development’

Annex 1 Terms of Reference

Table of Contents

1 Introduction ...... 4

2 Brief overview of the Context ...... 5

3 FAO’s Programme in Somalia ...... 8 3.1 Budget and Sectoral Focus of the Field programme ...... 8 3.2 FAO Structures in Support of Somalia ...... 13

4 Purpose of the evaluation ...... 14

5 Scope of the evaluation ...... 16 5.1 The evaluation framework ...... 16 5.2 Evaluation Questions ...... 18

6 Evaluation methodology and approach ...... 20 6.1 Preparatory Phase and Analysis of Existing Data ...... 20 6.2 Main Evaluation Mission...... 21 6.3 Dissemination/Follow-Up of Findings, Conclusions, Recommendations 22

7 Organization of the Evaluation...... 22 7.1 Roles and responsibilities ...... 22 7.2 Composition and profile of the evaluation team ...... 23 7.3 The Evaluation Report ...... 24 7.4 Tentative evaluation time schedule ...... 25

Annex 1: All country projects under implementation between Jan 2007-July 2012 Annex 2: Regional and Global Projects that Nominally Include Somalia Annex 3: Existing Independent Evaluation Reports of FAO Somalia Interventions Annex 4: Concept Note – Study of FAO Supported Cash for Work Interventions in Somalia Annex 5: Terms of Reference: Evaluation Consultative Groups Annex 6: Guidance: Inception Report Preparation and Country Evaluation Report Outline Annex 7: Acronyms Annex 8: Organogram FAO Country Office 2012

Introduction

1. The decision to undertake and independent evaluation of the FAO Programme in Somalia was based on a) a specific request from the ADG Technical Cooperation Division and the Officer in Charge (OIC) for Somalia1, and b) an analysis of the context in Somalia, the size of the overall FAO country programme expenditure over the past three biennium, and a preliminary review of existing monitoring and evaluation (M&E) data. Country evaluations are designed to assess the totality of the institution’s assistance provided to the FAO member state. The evaluation seeks to draw lessons and make recommendations that will be relevant and useful for FAO’s future engagement in the country. The evaluation will provide accountability to resource partners, Government and non-Government partners and communities. In addition to providing lessons specifically on FAO cooperation in Somalia, the evaluation will enrich FAO’s synthesis of findings and guidance for its country-level support – particularly important as the organization continues in its decentralization strategy.

2. The Terms of Reference (ToR) for the Evaluation have been developed based on a preliminary analysis of FAO’s portfolio of work in Somalia and scoping interviews with key FAO staff in HQ and the Officer in Charge (OIC) for Somalia. They are informed by an initial review of some key external documentation including the 2012 Inter-Agency Real Time Evaluation for Somalia, the WFP Country Evaluation Inception Report, existing OED evaluations of FAO projects in Somalia, and internal audit and risk assessment reports available. The TOR will be shared and commented on by a broad range of FAO staff and by the consultative groups2 in Rome and at country level – informing the subsequent inception report. FAO country evaluations generally cover five to six years of the organization’s involvement. Accordingly the evaluation will cover the 2007-2012 period.

3. The Terms of Reference represents an important guiding document for the independent evaluation team. The evaluation Team Leader, once recruited, will undertake additional preparatory review and analysis which will result in further refinements in the scope, key issues, tools and methods to be employed in the evaluation which will be presented in his/her inception report at the start of the country mission.

1 FAO Somalia has received requests for resource partners for an evaluation. 2 Consultative groups are comprised of key internal and external stakeholders to the evaluation (Government of Somalia representatives, donor representatives and senior FAO staff). The TORs for CGs are attached as annex 6. Brief overview of the Context3

4. Somalia is known to be amongst the poorest and most food insecure countries in the world. In recent years, assessments have estimated that approximately 25% of the population have not had access to sufficient food and have been regularly in need of emergency food assistance4. For the past decade Somalia has been among the world’s highest per capita recipients of humanitarian assistance.

5. The livelihoods for the majority of Somalia’s people depend on pastoralism and agro-pastoralism. Only a small proportion of the population is dependent solely upon settled agriculture, which is undertaken principally along the valleys of the Shabelle and Juba rivers and in areas with more consistent rainfall such as Bay and Bakool, traditionally the bread-basket of Somalia in better times. Somalia has one of the longest coastlines in Africa, so fishing is an important potential resource, but it is not currently a principal livelihood for many. The seasonal calendar in Somalia is important to all livelihood groups. There is considerable reliance upon the long Gu rains and the shorter, but important Deyr rains, at the end of the year for cereal and vegetable crops, but of equal importance for the regeneration of pastures and replenishment of rivers, dams and ground water supply. Figure 1 shows the associated seasonal events in Somalia. Figure 1: Somalia Seasonal calendar and critical events calendar5

6. Somalia has not had a functioning central system of government since the collapse of the Siad Barre government in 1991 following a prolonged and bitter civil war. In the north-west of the country, Somaliland has declared itself an independent entity (the Somaliland Republic) and has effectively established governance and administrative structures within its boundaries. Somaliland has not been accorded international recognition, principally because the African Union consider this might be detrimental to a longer term political solution for Somalia as a whole. Recently, however, there has been a growing recognition among the international community of the potential benefits of engaging with Somaliland’s institutions to address the multiple problems in the area, and steps have been taken accordingly.

3 Context analysis draws heavily from the WFP Country Evaluation TOR. 4 A large percentage of this population are internally displaced people with disrupted livelihoods and are reliant upon external assistance 5 source: FEWSNet 7. Puntland, in the north-east of the country, is a semi-autonomous region of the country which has not sought independence from Somalia, but has established its own political and administrative systems. These systems have been closely aligned to the government in the south. There is an ongoing border dispute with Somaliland which has been the cause of tensions between the two administrations over the past 12 years.

8. However, it is the south-central areas of Somalia where the political and military crisis has been most extreme. The current Transitional Federal Government (TFG) was initially established in 2004 following national reconciliation talks in Djibouti. The TFG has faced opposition from influential groups in Mogadishu, from the ICU and more recently the Sheebab. It has not been able to consistently maintain the seat of Government in the capital, Mogadishu.

9. There is a number of factors that make the humanitarian situation uniquely complex in Somalia: the lack of effective governance, security and access problems, law and order exacerbated by insurgencies and conflict are the principal drivers of the humanitarian crisis in south-central Somalia.

10. Furthermore, there is increasing evidence of droughts becoming more severe and frequent as a result of climate change. The drought in 2005-2006 following a series of failed rains was considered the worst in ten years; this was followed by a further period of “deepening” drought in 2009-10 (including in the north); and subsequently the lack of rain led to the declaration of famine during 2011 in parts of Bay, Bakool, Gedo, Juba, Lower and Middle Shabelle, in addition to IDP settlements in Mogadishu and the Afgoye corridor leaving a total of 4 million people in “crisis” representing over 50% of the Somalia population6. Often periods of drought in Somalia are punctuated by severe flooding along the Shabelle and Juba rivers, which was the case at the end of the 2006 Deyr rains running through into 2007.

11. There has also been a significant rise in food prices since 2008-09 due to global spikes in the cost of grain and fuel, which has increased the cost of all essential household commodities7. Added to this has been the impact of the global recession on remittances from the diaspora, a vital resource for poor households in Somalia8, which were estimated to be down by as much as 25% in 20099.

12. Then in recent years there has been a significant reduction of the humanitarian space in which aid organisations have been able to operate effectively and safely. Humanitarian aid has been viewed by some elements of the TFG as support to the “enemy” and obstacles have been set up by uncontrolled and predatory

6 The population in crisis include: 750,000 experiencing “famine/humanitarian catastrophe” in south- central areas; 1,800,000 who are in “humanitarian emergency”; 830,000 in “acute food & livelihood crisis” in urban and rural areas; and 910,000 IDPs (FSNAU Technical Series Report VI 42 October 2011) 7 The prices of imported rice and local cereals increased between 200-400% in the first six months of 2008 (FSNAU) 8 Total flow of remittances to Somalia has been estimated at USD 1.3-2 billion a year, of which the total remitted for relief and development purposes has been calculated to stand at USD 130-200 per year. Hammond et al (2011) Cash and Compassion: the role of the Somali diaspora in relief, development and pece building, New York, UNDP 9 UNOCHA CAP 2010 security forces to prevent or “tax” food aid movements. Similarly jihadist cells within shabaab threaten Somalis working for Western aid agencies or collaborating with the UN and Western NGOs. Shabaab has also been responsible for the expulsion of the two principal food aid agencies from south-central Somalia.

13. The complex political context in south-central Somalia now makes the country (along with Afghanistan) the most dangerous place in the world for humanitarian workers.10 Since March 2010 there are very few international aid workers based in south-central Somalia and all aid operations have been managed remotely from Kenya. Sixteen organizations11 were very recently banned by shabaab in November 2011. Both local and international aid agencies are now either not able to conduct operations at all or are operating at extremely limited capacity. Those that do operate have to adopt a very pragmatic approach.

14. A significant amount of humanitarian funding has gone into Somalia since 2007. Average ODA over the past five years is approximately 500 million/year – most of which has gone towards humanitarian assistance. The main OECD donors to Somalia include the USA, EU, Britain, and the Scandinavian countries.

15. The annual Consolidated Appeal Process (CAP) prioritises funding across the humanitarian sectors and is complemented by the Central Emergency Response Fund (CERF), the Humanitarian Response Fund (HRF) and the new Common Humanitarian Fund (CHF).

Table 1: Somalia Consolidated Appeal (CAP) 2007-2011 Principal Target Total required FAO request US$ FAO actual CAP factors population by CAP US$ and % (% on total funding and % (% required by CAP) on total required by FAO)

200712 Conflict 1,000,000 262,354,615 12,569,400 5% 6,608,533 53% and floods incl 400,000 IDPs 2008 Conflict 1,500,000 406,235,651 26,520,543 7% 7,243,279 27% chronic incl food 850,000 insecurity IDPs 2009 Conflict 3,200,000 918,844,550 20,869,580 2% 11,692,098 56% rising food costs chronic food

10 In the 16 months from July 2007, 30 aid workers were killed in Somalia including the assassination of the top national officer for UNDP in Somalia (June 2008) and several UN staff in synchronized suicide bombing by shabaab of targets in Puntland and Somaliland (October 2008) 11 While the list does not include FAO, it does include a specific FAO project that produces food and nutrition security information in the country, the FSNAU – as well as some of FAO’s implementing partners (NGOs). 12 The CAP 2007 was revised upwards from US$237,112,824 to $262,354,615 due to additional projects being added insecurity 2010 Conflict 3,640,000 689,008,615 17,867,500 3% 9,928,119 56% drought incl (including 910,000 northern (HE) areas) 1,180,000 global (AFLC) recession 1,550,000 (IDPs) 2011 Conflict 2,000,000 529,520,029 76,924,000 15% 47,071,773 61% chronic incl food 1,460,000 insecurity IDPs

2012 Conflict 4,050,000 1,521,930,554 180,834,782 12% 92,576,128 51% and Incl drought 910,000 recovery IDPs

FAO’s Programme in Somalia

16. FAO Somalia has a longer term FAO Somalia Strategy document finalized in early 2011, for the period 2011-2015. In addition, FAO also has rolling two year Plans of Action as well as a series of annual plans (sub-components of the annual appeal documents) that define its strategic objectives. As from 2011 FAO Somalia shifted its approach from project based to programme based. FAO is working to create a pooled fund to be able to receive un/lightly-earmarked resources from donors towards the overall programme13. Recently, following the drought crisis, FAO together with UNICEF and WFP, prepared a joint Resilience Strategy 2012-2020.

Budget and Sectoral Focus of the Field programme Field Pgm Delivery 2007 2008 2009 2010 2011 (Source: FPMIS) US$ 14,671 20,355 27,169 33,943 65,449

17. Over the period covered by the evaluation, FAO’s field programme has grown significantly, particularly since the 2011 drought crisis. The portfolio of interventions covered by the evaluation totals over US$ 200 million. FAO has designed and implemented roughly 100 country specific projects14 which have been complemented by a number of regional and global projects which have included Somalia. Main areas of intervention have included support for crop production (including irrigation), animal health and production, fisheries, cash for work15 and other employment related interventions, support for livestock/meat marketing and

13 The 2012 programme requested 172 million usd. As of June 2012, the programme is 13% funded. 14 See annex 1: list of projects considered within the scope of the evaluation. A complete portfolio analysis is also available. 15 Typical infrastructures rehabilitated through CFW include irrigation canals and irrigation systems, feeder roads and market facilities, slaughter slabs, water points and catchments for livestock and water for domestic use. trade, coordination of the agriculture and food security sectors, and provision of information and analysis on food and nutrition security, land and water16.

Figure 2. Number of projects by type 2006-2012.

18. With respect to the approaches that FAO has used in Somalia17, some of the main ways in which FAO has intervened include:

a) Increasing and protecting household and community assets

19. As part of the livelihood support and emergency interventions, FAO has provided inputs for crop production to vulnerable HHs and has delivered veterinarian treatments and vaccinations mainly through the Somali Veterinarian Professional Associations. In addition, FAO has implemented a large Cash for Work (CfW) programme with the immediate goal of improving poor HHs access to the minimum expenditure basket and at the same time increasing communities’ resilience through the rehabilitation of productive infrastructures including irrigation canals, water catchments, feeder roads and slaughter facilities. Through the private sector, FAO has also supported the rehabilitation of major irrigation schemes and other infrastructures such as market stores and facilities, veterinary laboratories and slaughterhouses. Some interventions in the fisheries sector (provision of fish landing infrastructure, fish processing equipment, distribution of boats and fishing gear, and training) have been conducted in particular in Puntland.

16 Specific thematic briefs have been prepared for each of these areas of intervention. 17 In corporate terms, these are commonly referred to as FAO’s “core functions”

b) Providing technical support and capacity development of individuals and organizations

20. In the livestock sector, FAO has provided training to public and private veterinarian staff and community animal health workers on disease investigation and treatment. FAO has participated in the SAHSP EU funded projects aiming at strengthening the capacity of the Somali communities, public and private sector institutions to better manage risks associated with trade limiting animal diseases and has supported the development of a livestock certification model for Somalia (SOLICEP). Over the period covered by the evaluation, other projects had among their objectives enhancing the Somalia surveillance system through support to data collection, analysis and reporting and control capacities (including diagnostic laboratory capacities).

21. Within the livelihood support projects, FAO has provided training to farmers covering use of improved farm inputs and good agricultural practices and several other aspects such as water and soil management, irrigation, and improved nutritional practices. In addition, FAO has provided training on post-harvest losses and better storage (including training on silos construction and provision of storage facilities), on pest and disease issues and on integrated pest management principles and standards. FAO has also implemented two projects on seeds improvement and on the development of a seed market through the promotion of commercial seeds companies (COSE).

22. As part of the rehabilitation interventions, FAO has contributed to the establishment of local organizations and has provided training on management and maintenance of the rehabilitated assets (such as the water user associations, Private- Public Partnership PPP companies/committees for the slaughter facilities, etc.).

23. In the meat market sector, FAO has supported the development of the Meat Inspection Control Act (MICA) and the Quality Assessment System in Somaliland and Puntland. FAO has provided training to public and private capacity for meat inspection and reporting systems and at community level on improved hygiene conditions for meat processing. FAO is implementing a project on the meat value chain to identify opportunities and constraints in the meat industry. In addition, FAO has provided support to the umbrella organization of meat exporters in Somalia (SOMEAT). Concerning the agriculture market, a new project has recently started with the aim of improving market linkages and access to local and international markets through activities such as mapping of farmers organizations, training of grading and labelling specialists, training of partners in marketing, etc.

c) Coordination

24. Coordination of multi-stakeholder efforts has become an increasingly important aspect of FAOs work in Somalia18. FAO has been the leading agency for the Agriculture Livelihood cluster and is now co-leading with the WFP the recently

18 The section on coordination is mainly based on information provided in the 2012 FAO Somalia Emergency Programme and in meetings with technical divisions. established Food Security Cluster. FAO also leads the Inter Cluster Coordination Group for cash based intervention in Somalia which was created in 2011.

25. FAO is participating in the Food Security and Economic Development Sector Committee (FSEDSC) of the Somalia Support Secretariat (SSS) including the cash based response, the agriculture and the livestock working groups and is contributing to the United Nations Somalia Assistance Strategy (UNSAS), for which FAO leads Pillar II “Poverty Reduction and Livelihoods” and to the United Nations Country Team (UNCT) on the piracy trust fund.

d) Information, Statistics and Knowledge

26. Over the period covered by the evaluation, FAO Somalia has continually supported the production and dissemination of food and nutrition security and livelihood related information and analysis (FSNAU project) through field based national staff data gathering in collaboration with a network of partners present in the country, analyses and disseminates the information. As part of a recently started project “Support to agricultural market in Somalia”, the FSNAU monitoring tools are being expanded to include a Market Information System.

27. The Somalia Water and Land Information Management (SWALIM) is the other main information system supported by FAO19. Under the current Phase, a transition strategy has started aiming at transferring the management of the programme to Somali entities. An emphasis is thus placed on building national capacities for allowing the transition.

28. Both FSNAU and SWALIM projects produce a large number of information and analysis products which can be accessed through their websites (http://fsnau.org/ and http://www.faoswalim.org/ respectively). Beneficiaries of the information systems supported by FAO are national line ministries and Somali institutions, civil society organizations, universities and the international community of donors, UN agencies and NGOs. The main aim and challenge of the information systems is to contribute to better informed decision making processes at national level and to better planning of development and emergency innervations, early warning and preparedness.

29. In a pilot project implemented in Somaliland, FAO has worked to develop and test a methodology to conduct an aerial/ground livestock survey and to collect and disseminate livestock data.

19 Over the period covered by the evaluation the following phases were implemented: Phase II (2004- 2007), Phase III (2008-2010) and the current Phase IV (2010-2013).

e) Partnerships

30. FAO has implemented its programme with and through a wide range of partners, some of the most common, are indicated by type of intervention in the table below. A new strategic partnership has been developed between FAO, WFP and UNICEF around a resilience framework for Somalia.

Top Civil Society Government Other (Community groups, Partners academia, private sector, (INGO/NNGO) consulting companies, etc) CFW ADA, COOPI, Aw-Faqi, Infrastructure User WOCCA (Women and Committees, Somalia Child Care association), Agronomist Association, SDRO, SAREDO, SAGRA, Local Authorities MURDO, COMORAD, AGROSPHERE, CEFA Crop Cooperazione MoA staff Farmer field school (FFS) Support Internazionale (COOPI), groups, water user associations Advancement for Small (WUAs), SATG, COSEs, Enterprise Programme Somalia Agronomists (ASEP), DRC, HARDO, Association (SAGRA), Local SOADO, ADA, authorities Agrosphere Livestock The 5 Professional MoL Somaliland, Local authorities Veterinarian MoLAH Puntland, Associations (in MoLFR in South Central particular SOWELPA Somalia and CERELPA), AU-IBAR COOPI, VSF-S, VSF-G MUGUGA laboratory in and Terra Nova. Kenya, Regional Livestock Officers, Veterinarian Diagnostic Laboratories in Somalia Market VSF-G, COOPI and MoL Somaliland, SOMEAT (Somalia Meat Interventions Terre Solidali. MoLAH Puntland, Exporters Association), (in particular Ministry of Health municipalities (as Meat) owners/administrators of local slaughter facilities), Veterinarian Professional Associations , slaughter slabs committees, Damug Modern Slaughterhouse Company DAMSCO and Puntland Slaughterhouse Company PSC both created with the support of FAO, Livestock Boards Fisheries Ministry of Fisheries Fishers’ associations Somaliland, Puntland and Central South Somalia FSNAU Main partners: • FEWSNET, WFP SWALIM Main partners: • See list on http://www.faoswalim.org/Useful%20Links

31. Major donors to FAO’s programme have been the EU/ECHO, USA, OCHA, the World Bank and UK. FAO has also benefited from pooled funding (CHS) and emergency appeals for Somalia.

32. With respect to the geographic distribution of FAO projects, the largest number of projects (45) have been implemented in south-central Somalia, followed by Somaliland (14) and Puntland (13). A quarter of the projects are national in nature or are implementing activities in all 3 regions. FAO’s programme in Somaliland and Puntland appears to have been fairly stable over the past 6 years – while in the south central region a significant increase in programme delivery is noted over the 2010/11 period.

Figure 3. Delivery of FAO Programme Budget by Year and Region (US$)

33. Specific community level target groups have included households affected by the 2004 Tsunami, households displaced or affected by the civil conflict, and households affected by the flooding in Somalia in 2008 and 2010. While vulnerabilities vary by livelihood group and the constantly shifting conflicts have affected most parts of Somalia at some point in time over the last decades, the most severely affected communities have largely been in south-central Somalia.

34. FAO has established a central monitoring unit in Nairobi and has 19 Somali staff based in the country who monitor implementing partner activities. In addition, FAO has recently established a monitoring approach based on random cell phone interviews with beneficiaries and has put in place a “hotline” which allows for direct beneficiary feedback. FAO is beginning to use remote sensing data to monitoring progress and completion of infrastructure rehabilitation work such as canals and water catchments.

FAO Structures in Support of Somalia 35. FAO has no formal representation in Somalia. A large office exists which includes 252 staff (including 35 internationals) who remotely manage20 the large emergency and recovery programme from a programme base in Nairobi. The most senior official is the Senior Emergency Coordinator – who is also the Officer in

20 90 FAO staff are based inside Somalia – the other 162 are in Nairobi. Charge (OiC). The current OiC has been in post since early 2010. A significant internal structural change that has occurred over the last 2 years is the integration of what used to be four separate FAO offices21 into one structure under the OiC (Annex 8. Country Structure).

36. Historically the OiC reported to and was operationally supported by the Emergency and Rehabilitation Department at FAO in Rome. Technical support has been provided both by the multi-disciplinary team in the Sub Regional Office in Addis Ababa, by the Regional Emergency Office for Africa and ECTAD (in Nairobi) and by technical divisions (principally AGP, NRL, AGA ESA, AGS, and AGN) at FAO in headquarters.

37. As of 1May 2012, as part of FAO’s corporate decentralization strategy, responsibility for the management of the projects was transferred to country level, to the OIC. Operationally, the Somalia office now reports to RAF (and the OiC to the ADG/RAF). During the initial transition period, TCE is continuing to provide support. Another important change in 2012 has been the development of a programme approach to resource mobilization at country level – which establishes a basis for creating a pooled fund, moving away from highly earmarked project funding towards a resourcing strategy that contemplates the programme as supported by an integrated system of human and physical resources.

Purpose of the evaluation

38. The Country Evaluation will aim to assess FAO’s performance in Somalia over the past six years, identifying success factors and challenges. Based on the main findings, the evaluation will provide recommendations for maximizing the strategic relevance of FAO’s engagement in the country over the next several years, and towards improving the efficient implementation and effectiveness of FAO support to the country in its future operations. Wherever possible, lessons learned and good practices will be identified and learning shared within the broader organization through the dissemination of the evaluation.

39. The main audiences of the evaluation to which the findings, lessons and recommendations will be offered are the FAO management in Somalia, the Government structures with which it works, and FAO’s resource partners in Nairobi. Given current efforts to strengthen decentralized management of FAO cooperation, other potentially important users of the evaluation report identified are the Regional Office for Africa (RAF), the Sub Regional Office in Addis Ababa and the Regional Emergency Office for Africa based in Nairobi. Other stakeholders to whom the report is intended include the FAO Programme Committee - and FAO HQ senior management and divisions providing operational and technical support to the country programme. Although not a target group for the evaluation report itself, the evaluation also focuses on providing accountability with respect to the communities and in particular vulnerable groups in Somalia that FAO has sought to assist. A broader stakeholder group will also take part in the evaluation through interviews and possibly as part of the country level consultative group.

21 Emergency Coordination Unit, FSNAU, SWALIM office

Scope of the evaluation

The evaluation framework 40. The country evaluation will cover the totality of FAO’s cooperation with Somalia over the period 2007 to 2012, including activities supported by the Regular Programme budget and voluntary resources, projects managed by HQ, SFE and the country office, the field programme and normative products, and all the major economic sectors. The evaluation will also assess the activities of country level senior management that were not necessarily carried out through projects but that involve the strategic management and direction of the programme and FAO’s external image and engagement (e.g. coordination, advocacy, resource mobilization and communication). Given the significant increase in overall programme resources since 2010 (see figure 3), more emphasis will be placed on evaluating FAO’s performance in the last 3 years.

41. In accordance with UN Evaluation Group Standards, the evaluation will assess the relevance, efficiency, effectiveness, impact and sustainability of all of FAO’s interventions in Somalia. It will also evaluate the extent to which they pursued gender equality and social inclusion22, using the new 2010 UN system evaluation guidelines developed for this purpose. Given the extensive emergency recovery programme that FAO has engaged in over the past six years, and the role that FAO has played in coordinating the response, additional evaluation criteria of connectedness23 and coordination will be added.

42. Evaluability Assessment: Due to the specificities of Somalia and fragmented governance situation, the Reconstruction and Development Programme (RDP), jointly developed in 2006 by the UN/WB with 3 Somali Administrations (TFG, Somaliland and Puntland), still remains the only overall development framework for Somalia. The development of a fully fledged UNDAF has been replaced by a UN Somali Assistance Strategy (UNSAS) with a 2011-2015 timeframe, which is based on the original RDP. UNSAS guides the short and longer term priorities of the UN in Somalia and has 3 main objectives related to governance, poverty reduction and livelihoods and revision of social services. Additionally on request of the UNSG, an Integrated Strategic Framework (ISF) was developed with specific focus on the peace building initiatives of the UN for Somalia. FAO’s performance in Somalia can therefore be assessed against the RDP (which comes to an end at the end of 2012) and UNSAS as they represent the overarching frameworks for Somalia. Evaluating organizational performance in a highly complex and constantly changing environment will require the mission to pay particular attention to FAO effectiveness in adapting to complexity, in recognition of the risks, opportunities and need for innovation.

d System. Internal Draft Document 2010. 23 Connectedness refers to the need to ensure that activities of a short-term nature are carried out in a context that takes longer-term problems into account (adapted from the concept of sustainability). Coordination refers to the systematic use of policy instruments to deliver humanitarian assistance in a cohesive and effective manner. (planning, data gathering, information management, mobilizing resources and ensuring accountability, negotiating a functional division of labour, negotiating and maintaining a serviceable framework with host political authorities). ALNAP Real Time Evaluation Guidelines 2011.

43. FAO’s global 2010-2019 strategic framework will also be useful in a general sense24, and emerging frameworks around strengthening resilience which are articulated in the FAO Somalia programme 2011-2015 which put a focus on twin track approaches to disaster risk management may be particularly relevant.

The main programmatic documents for FAO Somalia are: 1. The FAO 5 year Strategy and PoA 2. The Short term emergency response programme 3. The joint FAO/WFP/UNICEF resilience programme 4. Annual FAO components of consolidated annual appeals

44. With respect to the quality and quantity of data which will be available to the mission, there are a number of independent evaluations which have been undertaken which examine components of FAOs work in Somalia (annex 3) and it appears that some monitoring of outcomes using beneficiary assessments has been done in the most recent period. Although no formal programme baseline data exists, it may be possible to establish comparative data for some interventions from FSNAU25. The evaluation team will also have access to project progress/final reports and mission reports. Comprehensive programmatic audits have also been done which can be particularly useful for examining aspects of efficiency. Several risk assessment exercises have been carried out and the analysis is available. Finally, WFP has just completed a portfolio evaluation of their work in Somalia over the period 2006-2010. Given that both agencies are working towards a common goal of improved food and nutrition security in Somalia, much of the context analysis and benchmarking of FAO performance in situ will be possible through reviewing the major findings from the partner agency evaluation.

45. An important constraint to the collection of new/primary data for the evaluation is access. At the present time it is not possible for international consultants to travel in much of Somalia26 and in particular south-central Somalia. Elections were held in August 2012 and the establishment of a new Government may have an additional effect (positive or negative) on the security situation. For this reason, no random sampling methodologies will be feasible and the evaluation exercise will need to a) rely heavily on desk reviews and interviews with stakeholders outside of Somalia (largely in Nairobi), and b) explore alternative methodologies for collecting primary data e.g. remote sensing, telephone interviewing, etc. Local consultants may be recruited to collect specific date in south-central Somalia where access is the poorest. The best case scenario is that the evaluation mission will be able to undertake site visits in Somaliland, Puntland and specific locations in south central Somalia. The planned cash for work study will need to rely on local researchers and of necessity will be largely qualitative in nature.

24 particularly the results framework developed under Strategic Objective I: Improved preparedness for, and effective response to food and agricultural threats and emergencies. 25 The Food Security and Nutrition Analysis Unit is supported by FAO and has collected data on a range of food security and nutrition related indicators over more than a decade. 26 http://ochaonline.un.org/somalia/MapCentre/HumanitarianAccessMaps. Note even Somali consultants are limited in their movements and confined to regions of origin. Evaluation Questions

46. The focus of the evaluation is on the FAO Somalia programme as a whole, rather than on individual projects. Draft or tentative questions the evaluation will address are the following:

Relevance • Given the absence in the past of a clear and explicit strategy for the country, to what degree were FAO’s interventions in the various sectors and regions aligned with the government’s national strategy and sub national Government priorities? How has Somalia’s governance structure affected FAO approach and functions? How have FAO interventions been designed to use and strengthen governance systems where they exist? • What forces have shaped FAO’s programme at country level? Is FAO aligned with UN programmes and priorities in the country? To what extent has FAO played to its comparative advantage? Have coordination structures been appropriate for maximizing the synergies and minimizing duplication between FAO and other agency interventions? • In a highly complex and changing environment, has FAO’s analysis and planning mechanisms allowed it to respond flexibly to changing needs and priorities? Has FAO been able to strike the right balance between risks and benefits? • To what extent did FAO’s programme seek to address the needs of the most vulnerable- particularly women and more disadvantaged groups? How good was FAO’s analysis of the immediate and underlying causes of food insecurity and malnutrition? Is the information generated by the FSNAU (including the nutrition section) used to inform programming and how? Were interventions designed to strengthen resilience at household and community level, and address specific causes of malnutrition? How have risks, costs, and benefits been assessed and managed for the main areas of intervention given the security context? • What is the quality of FAO intervention designs – is the internal logic sound enough to allow for the achievement of the desired results? Are good practices (technical and humanitarian) built into the programme design? FAO Somalia has a new 5 year strategy that was not based on the corporate process/format (CPF) - what can be learned from this experience? • To what extent have key internal and external stakeholders participated in FAO’s new strategic plan 2011-2015 - and how well does the strategic document reflect Government priorities, diverse national needs and FAO’s comparative advantage?

Effectiveness and Impact • To what extent did FAO’s interventions in the different regions and sectors achieve their desired results (at household and organizational/institutional levels). Were the results indicators selected appropriate? What changed and for whom? Was coverage of the most vulnerable groups adequate? To what extent has FAO contributed to gender equity through these interventions? • How effective are FAO’s monitoring systems for informing programming and decision-making and ensuring accountability beyond the immediate outputs? How are evaluation recommendations (Somalia specific and corporate) and related new strategies (capacity development, gender, water, nutrition, food and nutrition information systems, etc) been applied in Somalia? How has data and analysis produced by FAO been used by internal and external stakeholders? • What internal or external factors inhibited or enabled the achievement of the immediate and long-term results? How have partnerships impacted on FAO’s ability to respond effectively in a volatile and highly insecure context? How has FAO managed the challenges compared to other UN agencies? • How effective was FAO in managing and coordinating the humanitarian response within the food security and agriculture sector with other agencies? To what extent has this lead to improved coverage, reduced gaps and strengthened technical quality humanitarian action in the agriculture sector? Have any innovations been successfully introduced? Given their role as a global Food Security Cluster lead, to what extent is FAO in Somalia meeting standards in terms of leadership, coordination and accountability? • To what degree did the OIC succeed in influencing state actors and/donor priorities with respect to the most relevant food security issues? Is FAO Somalia seen as leading on or following debates on rural livelihoods? • Have there been any unintended impacts (positive or negative) from FAO’s programme? • If successful approaches have been identified, to what extent and how can they best be scaled up? How “big” does the FAO programme need to be for FAO to be credible and influential?

Efficiency • To what extent were materials, funds, knowledge and staff delivered in a timely and cost-efficient way? How do in country sub-offices contribute to the efficiency and effectiveness of FAO’s work? • Have FAOs organizational and management structures and systems been conducive for the delivery of programme? Are they suitable for the new approaches that are being introduced in the Somalia programme? Given the size and resources of the Somalia programme, what role have HQ and decentralized offices played and to what extent have they added value? • What has FAO’s success been in raising funds for the programme? How efficient are the resource mobilization and programme management tools/mechanisms? Is there a way to have a more corporate approach that is still driven by country priorities, needs and opportunities?

Connectedness • To what extent will the various benefits of the interventions, in the form of livelihood improvements at the community level, continue after assistance ends? • Is there any evidence that institutional systems or services have been strengthened as a result of FAO interventions? • Has FAO sought, created, or participated in inter-sectoral initiatives to address food insecurity and malnutrition? Have synergies with other agencies’ programmes been created? If no, why, and if yes, what are the main lessons learnt that arose from this process? Has FAO, especially in its role as food security cluster lead, fostered synergies with the nutrition cluster and if so, how? • To what degree has resilience been built into the programme? Have FAO interventions build on indigenous coping strategies and mechanisms which have been locally developed (e.g. communication, governance, finance)? • Has FAO developed well defined exit strategies for their interventions? How effective has FAO been in linking farmers/fishers/pastoralists to input and output markets?

47. As mentioned above, the evaluation questions will be refined during the pre- mission inception phase by the Team Leader. The inception report represents a communication tool between the TL and FAO Office of evaluation and further orients team member’s work. The Inception report will include an evaluation matrix within which the revised evaluation questions will be indicated as well as the sources of data and methodology which will be used for answering the questions.

Evaluation methodology and approach

48. The evaluation’s findings, lessons and recommendations will be evidence- based and use, to the extent possible, a variety of different methods and tools to gather information in order to assess FAO performance against the standard OECD DAC evaluation criteria. To validate the qualitative and quantitative data gathered and to ensure that the assessment is systematic and objective, triangulation of information will be employed. The following methods and tools, amongst others, will be utilized:

Preparatory Phase and Analysis of Existing Data (July-Oct 2012) • Desk reviews of FAO’s Somalia project documentation and primary quantitative and qualitative M&E data from FAO projects to capture and quantify the main reported project results. • An extraction and synthesis of existing evaluations27 to identify common findings and areas of recommendation. • A desk review of a sample of key projects28 will be undertaken by the team to provide a sector specific overview of work done by FAO in each area. • An independent study of FAO’s cash for work interventions will be completed29. Other primary data gathering may also be commissioned as determined during the inception mission after a review of secondary monitoring data. • In preparatory mission by the Team Leader and OEDD will be undertaken which will result in an inception report which will further elaborate on key questions to be explored through the evaluation, summarize evidence gathered and analyzed in the preparatory phase, establish the specific mission plan (and approach and methodology to be used), propose a specific list of key informants to be met, and

27 The Office of Evaluation has managed a number of evaluations of FAO work in Somalia (Annex 4). Several resource partner evaluations have also been completed (FSNAU) and an Inter Agency Real Time Evaluation of the Drought Crisis Response was completed in 2012. 28 A simple of projects will be selected for in depth review based on their financial value, specific interventions included, special aspects (innovation, target group, etc) and delivery mode. 29 Concept Note: Annex 5 evaluation matrix for the evaluation, and allocate the specific roles, responsibilities and expectations with respect to individual team members. Key sources of reliable secondary data will be identified and arrangement for gathering any needed community level primary data will be made. • Preparation of a 2-3 page sector specific review by each team member demonstrating their understanding of the context, FAO work in the sector, and specific questions that they deem important to examine in their particular area.

Main Evaluation Mission (Nov 2012) • At the start of the mission, the team will be provided with an internal briefing on FAO as a global organization, Office of Evaluation norms, standards and procedures, any specific methods for the country evaluation, and on respective tasks of team members in the mission. • Semi-structured individual and group interviews (face-to-face and/telephone) with internal and external stakeholders at all levels in Somalia, and at HQ and Regional Office levels. • Workshops with OIC staff to identify issues, lessons learned and future directions. • Field visits to beneficiary communities and other project sites in Somaliland, Puntland and South-Central Somalia as security allows. • Assessments of the use and usefulness of FAO’s normative outputs in the different sectors (e.g. guidelines, capacity-strengthening material and other knowledge products).

49. The evaluation will adopt a consultative approach, seeking and sharing opinions with stakeholders at different points in time. Triangulation of information across stakeholders will be a key tool for gathering and validation of evidence. Among the main internal and external stakeholders to be consulted will be FAO staff at HQ, the Regional Office for Africa, the Sub Regional Office for East Africa (SFE), the OIC, government ministry officials in the different sectors and at different levels, the beneficiary communities, other UN agency staff at strategic and operational levels, multilateral and bilateral donors and coordinating agencies supporting Somalia, international and national NGOs, and private sector partners.

50. The actual evaluation mission involving the whole team will be carried out in November 2012 over a period of approximately one month. Note that past evaluations of FAO projects in Somalia have, without exception, remarked on the limits imposed by the volatile and insecure environment on implementation of programmes, constraints which inevitably impinge on the evaluation process too. The mission will need to work creatively to collect and analyze both primary and secondary data. Where physical access for interviewing and observation is not possible, other data gathering tools such as remote sensing, email/internet surveys, and telephone interviewing may be considered.

Dissemination and Follow-Up of Evaluation Findings, Conclusions and Recommendations (Dec 2012-Feb 2013)

51. Upon completion of the mission, the team will present its preliminary findings and recommendations to the OiC, Consultative Group (see below), senior programme/project staff, key government officials and interested partner representatives in a debriefing session. Comments and additional information from these stakeholders will be considered for the evaluation report. A similar debriefing will be held in Rome involving key senior management and technical staff who have supported the Somalia programme.

52. The final draft version of the report will be disseminated to and discussed by the CG and FAO management and their written comments and suggestions provided to the team leader before it is finalized. Specific annexes will be prepared to summarize results of TCP and CERF projects. A rating/scoring system is available and will be provided by OEDD for this purpose.

53. After the report has been finalized, it will be disseminated widely by email, website and through presentations at HQ and country level. The OIC and OSD will be jointly responsible for coordinating the Management Response to the report and its recommendations. Both the final report and FAO’s management response are public documents and should be fully disseminated both internally and externally30. This may include country level workshops as appropriate – to be determined by the CG during the presentation of the initial findings at the end of November. Follow up of the management response will be required during the one year period following the evaluation.

Organization of the Evaluation

Roles and responsibilities 54. The FAO Office of Evaluation (OED) is responsible for managing the Somalia evaluation and delivering the final evaluation report within a specific time schedule. The OED Evaluation Manager (EM) assigned is responsible for the formulation of the evaluation draft TOR and participates as a team member in the actual mission, contributing within her competency and mandate and to the writing of the report as defined by the Team Leader. The EM is responsible for reviewing the eventual complete draft report for the purposes of assuring quality and compliance with the TOR – and for promoting the dissemination of the report at different levels. As part of OEDs quality assurance measures, a second officer in the Office of

30 The Office of Evaluation posts all reports on the OED website. The FAO Representative is responsible for distributing the report to all key stakeholders at country level after its finalization. For further guidance, see the Guidance note for Conducting Country Evaluations: http://www.fao.org/evaluation/oed-about/guidelines/en/ Evaluation will serve as a Peer Reviewer of the inception report and final draft evaluation report.

55. As country evaluations, given their geographic specificity, are not presented individually to the PC, consultative groups (CG) are frequently established at HQ level comprising the specific concerned member state permanent representative and representatives from some of the main resource partners to the country programme. For the Somalia country evaluation, a CG will be established in Rome (annex 5) comprising these stakeholders plus several representatives from FAO senior management. This group will have the opportunity to comment on the TOR, the inception report and the draft Country Evaluation report. A similar advisory group may be established at country (Nairobi) level.

56. The Evaluation Team is responsible for the technical and substantive contents of the evaluation. An independent consultant with the required experience in evaluation and a relevant field for the evaluation will be hired as the Team Leader. S/he will be responsible for preparing an inception report, overseeing and guiding the evaluation team, developing any specific tools for the evaluation, discussing evaluation findings, conclusions and recommendations with the team and consolidating the inputs of the members to produce the final draft and the final evaluation report. The team members will prepare pre-mission desk review reports, participate in briefing meetings and in the preparation of evaluation tools, contributing according to their individual TORs with written inputs to the final draft and final report. The team will decide on the integration of comments and suggestions by FAO stakeholders and the Consultative Group, as appropriate.

57. The OIC, ADG RAF, and relevant key HQ units (OSD, TC, CSA and the knowledge departments) are responsible for contributing to the draft TOR and for assisting the evaluation team in its work. They are also required to participate in meetings with the team, to make available information and documentation as necessary, and to comment on the final draft report. The OIC (with support from OSD) also responsible for leading and coordinating the preparation of the FAO Management Response (MR) to the evaluation, in which it expresses its overall judgment of the evaluation process and report and accepts, partially accepts or rejects each recommendation. For accepted recommendations, responsibilities and timetable for implementation will also be indicated; for rejected recommendations, a justification should be provided. One year after the management response is issued, the FAO Representation will prepare the Follow-up report to the MR, to inform on progress in the implementation of the recommendations.

Composition and profile of the evaluation team 58. Team members should have experience in humanitarian programming in fragile states. An effort will be made to recruit Somalia nationals for some of the expert positions as well as to recruit both males and females within the team. The evaluation team reflected needed expertise and will tentatively consist of the following people:

• Team Leader - experience in leading strategic evaluations (covering analysis of core functions31 and strategic leadership, and advocacy). • OED staff (1-2) - evaluation. Expertise in cash transfer related interventions and institutional knowledge of FAO and its policies and strategies. Knowledge of FAO management systems and structures. • Agricultural expert, with wide experience in crop production and irrigation. • Livestock expert, with experience in both animal health and animal production. Ability to cover fisheries. • Private sector, markets and trade expert – with specific expertise in livestock related issues. Will also examine marketing issues related to crops and fisheries. • Community development specialist/social scientist – covering gender, humanitarian principles (eg. Do No Harm), targeting, and food security and nutrition.

59. Cash for Work – a separate community level assessment will be undertaken prior to the country mission to collect information from participating communities on changes evident (both intended and unintended) from FAO supported CFW interventions. A separate 3 person study/research team will be recruited for this purpose, gathering data in south-central Somalia during September and October.

The Evaluation Report

60. The Evaluation team will decide on the precise outline of the report, but the report will follow OED evaluation report standards and provide evidence-based findings related to the evaluation criteria outlined in Section 5 above and the revised questions developed. A guideline on the outline of the report is included as annex 6. Its lessons or conclusions will be founded upon its findings and oriented to provide guidance for FAO’s cooperation in Somalia and at the country level in general in the future. The recommendations, in order to be of maximum use to FAO, will be focused, clearly formulated, addressed to specific stakeholders with the ability to effect change and be actionable. The report will seek to be concise and include an executive summary. The Team Leader will deliver it to OED according to the final timetable determined. The CFW study conducted for the country study will be provided as a stand-alone annex to the report, with its findings incorporated into the main body of the document.

31 Policy advice, capacity development, partnership & interdisciplinary approaches, generating and disseminating information (including statistics and monitoring trends) and knowledge.

Tentative evaluation time schedule Preparation Phase Desk Review July-Sept 2012 Inception Mission Sept 2012 Establishment of Consultative Groups in Nairobi and Rome – Sept 2012 Evaluation Phase CFW Study Sept/Oct 2012 Main Evaluation Mission 28 Oct - November 30, 2012 (post mission debriefings in Nairobi 26th and Rome in early-mid December) Report-writing and Dissemination Phase Draft Report: January 2013 Final Report: February 2013 Dissemination of the Report and March 2013 Management Response

Annex 2 – Evaluation Matrix

Issues / Questions Criteria of judgement/ indicators Data collection methods and sources Relevance Analysis of FAO’s programme against evidence of need and Mapping of FAO’s programme against analysis of need – How relevant was FAO’s programme to the needs gaps in international assistance (especially in relation to o short and long-term (including review of FSNAU data) of the most vulnerable (particularly women and vulnerable groups) 1.1 Key informant interviews and group discussions at more disadvantaged groups)? Analysis of FAOs’s approach to targeting, in programme o community level and with international and national NGOs design and in practice and other actors

Analysis of FAO’s programme against key strategy o Review of documents indicating government priorities in documents and stated government priorities different parts of Somalia To what extent has FAO’s programme been Adaptation and innovation in the programme to respond to Key informant interviews with members of government - aligned with the strategy and priorities of o the lack of formal governance past and present, and with FAO staff 1.2 government authorities? How has the programme Assessment of the appropriateness of FAO’s modes of Key informant interviews with non-state actors eg private adapted to the lack of a functioning central o operating to the changing context, and to a context of weak sector, community-based institutions government? or failed formal governance o Review of relevant literature on working in a fragile or failed state Analysis of FAO’s programme against key UN strategy and programme documents To what extent has FAO’s programme been o Review of documents indicating UN priorities in Somalia 1.3 aligned with UN programmes and priorities in o Key informant interviews with members of UN agencies, Somalia? and with FAO staff

How well-designed are FAO’s interventions, and Analysis of FAO interventions against accepted international how well-implemented in terms of: norms and standards eg on accountability to beneficiaries, a) consultation with key stakeholders ‘Do No Harm’ etc b) sound internal logic to achieve the Evidence that an understanding of the political economy has desired results informed programme design and implementation and that Review of project and programme proposals c) flexibility to adapt to the changing context programming has been adapted to the changing context o 1.4 Key informant interviews with FAO staff and with key and changing needs o external stakeholders/ participants in FAO interventions d) sensitivity to conflict and to the political context

Issues / Questions Criteria of judgement/ indicators Data collection methods and sources Effectiveness and impact Comparison of actual results with intended results Review of project evaluations and other M&E data and How effective were FAO’s interventions in o Evidence of the impact of FAO’s programme – positive or documentation 2.1 achieving their desired results, and impact? negative, intended or unintended, since 2007 Interviews with actual and potential beneficiaries of FAO o Evidence of who gained and who lost from the interventions programmes Evidence of how effectively FAO’s management and monitoring systems have adapted to the context Review of FAO management structures and procedures Evidence of how FAO’s management structure and systems o over time – from documents and interviews with FAO staff How effective were FAO’s management and adapted to scaling up of the emergency programme, and Review of FAO risk management mechanisms over time, monitoring systems in a context of insecurity and subsequent scaling down o 2.2 with reference to recent audits, risk management remote access, and how have they adapted to the Evidence of effective risk management documents and interviews with FAO staff scaling up of the programme? Evidence of appropriate checks and balances built into Review of FAO’s M&E systems, from documents and monitoring and management systems o interviews with staff and other stakeholders Extent to which monitoring covers outcomes as well as outputs and feasibility of moving down the results chain Evidence of FAO leadership to the international response o Group and individual interviews with agencies Evidence that coordination by FAO has improved coverage, participating in clusters that FAO has coordinated for their the extent to which needs have been met, and the quality of feedback and perceptions (in Hargeisa, Mogadishu and How effective has FAO been in coordinating the the response as well as reducing duplication Nairobi) humanitarian response in the food security, The quality of the relationship between FAO’s coordination Interviews with FAO-appointed coordinators and with 2.3 o agriculture and livelihood sectors in Somalia? function and agencies participating in clusters that FAO has WFP co-coordinators coordinated o Administration and analysis of a questionnaire on cluster coordination o Review of documents relating to FAO’s coordination role eg cluster meeting minutes Evidence of how FAO responded to early warning, directly Review of publications on the 2011 Somalia famine and of and in influencing other actors o other evaluations eg RTE How effectively has FAO responded to acute Evidence of the timeliness of FAO’s emergency Review of FSNAU early warning data and how it was humanitarian crises, and especially to famine interventions o 2.4 used prevention and response in 2011? Evidence of the impact of FAO’s emergency interventions in Interviews with FAO staff, staff of IPs, other UN agencies addressing humanitarian needs o and government officers Evidence of most appropriate modes of response, especially Interviews at community level (if possible) in view of shift from food aid to cash in 2011 o How effective has FAO been in influencing state Assessment of FAO’s role in the formulation of government o Key informant interviews with key government personnel actors and international agencies in terms of policies and frameworks – past and present 2.4 addressing food security and livelihood issues and Assessment of FAO’s influence and leadership within the o Key informant interviews with key members of the in showing leadership? international community international community Issues / Questions Criteria of judgement/ indicators Data collection methods and sources o From secondary data, review of information systems provided by FAO and how they have been used Efficiency Perceptions of key stakeholders regarding the timeliness and cost-effectiveness of FAO’s Key informant interviews with FAO staff, other UN programming o To what extent has FAO conducted its organizations, government and donors Costs compared to similar FAO projects elsewhere 3.1 operations and delivered its programmes in Review of project budget documents and with other UN organizations in Somalia o Somalia in a timely and cost-effective way? Reference to documents outlining international Extent to which project designs and implementation o standards/principles included cost-effective measures according to recognized standards Review of FAO organisational procedures and o Key informant interviews with FAO staff and staff of To what extent have FAO’s organisational structures and comparison: other UN organizations procedures and structures facilitated the a) with those of other UN agencies o Review of documents that set out procedures and 3.2 cost-efficient and effective delivery of the b) pre and post decentralisation structures programme, and how has decentralisation o Review of documents that set out decentralisation impacted on this? process and objectives

How successful has FAO been in raising Amount of money raised by the OiC and by others at funds for the programme and how efficient FAO to support the country programme, including its Meetings with donors 3.3 are the resource mobilization and regional activities o FAO database: data review programme management mechanisms? Number of funding sources tapped o

Connectedness Evidence of a longer-term perspective being o Review of project and planning documents, and of To what extent will the benefits of FAO’s incorporated in project and programme planning and in monitoring data interventions continue after assistance exit strategies Interviews with key informants and project and 4.1 o ends? Evidence that benefits have/ will continue when programme beneficiaries assistance ends

To what extent have institutions within Review of appropriateness of capacity-building o Review of capacity-building assessment, planning Somalia (within government, the community programmes to the context and to existing capacity and strategy documents, and of project evaluations 4.2 and the private sector) been strengthened levels o Key informant interviews with participants of as a result of FAO’s interventions? Evidence of strengthened capacity in institutions that capacity-building efforts Issues / Questions Criteria of judgement/ indicators Data collection methods and sources FAO has supported, and assessment of contribution of FAO efforts

Intellectual and conceptual robustness of resilience o Review of documentation on resilience thinking and paradigm strategy What is the current and potential impact of Indication of how resilience paradigm will change the o Interviews with key informants 4.3 resilience thinking and programming on delivery and impact of FAO’s programme on the o Seminar/ workshop in Nairobi with small group of livelihoods? ground resource people/ policy analysts to explore resilience thinking in the historical context of aid programming in Somalia Cross-cutting Issues Review of project documents and project o Review of project documents implementation for evidence of gender sensitivity and o Interviews with participants and beneficiaries of of commitment to gender equality FAO projects 5.1 To what extent did FAO integrate gender Perceptions of stakeholders (internal and external) o Key informant interviews with FAO staff and with equality and mainstream gender in its regarding the extent of gender equality integration external stakeholders eg NGO staff, government projects, programmes, staffing and Review of data collection and analysis for evidence of officers management systems? gender disaggregation o Review of FAO data collection and analysis in Male-female staffing ratio by level of responsibility and different sectors evidence of commitment to gender equality amongst o Review of gender of those trained with FAO funding staff o Review of FAO staffing structure and of training provided to FAO staff

Review of the range of partnerships FAO has engaged in, and how they have evolved Review of what partnership means and how it is Key informant interviews with FAO’s partners – past 5.2 How strategic, effective and efficient have o perceived and present FAO’s partnerships been? Evidence of what FAO’s partnerships have achieved, Key informant interviews with FAO staff o how and why (eg compared with a single agency o Review of FAO’s partnership agreements working on its own)

Annex 3 Mission Itinerary Margie Sadia Mohamood Tim Yacob Shukria Aurelie Savina Date Day Buchanan Musse Noor1 Leyland Aklilu Dini Larmoyer Tessitore3 Smith Ahmed2 27-Oct Sat 28-Oct Sun 29-Oct Mon Nairobi Nairobi Nairobi Nairobi Nairobi 30-Oct Tues Nairobi 31-Oct Wed 01-Nov Thurs 02-Nov Fri Addis Ababa 03-Nov Sat Dollow Dollow Hargeisa Hargeisa 04-Nov Sun Nairobi Burao Burao 05-Nov Mon (Somaliland) (Somaliland) Dollow (South Central 06-Nov Tues Hargeisa Somalia) 07-Nov Wed (Somaliland) Hargeisa 08-Nov Thurs Hargeisa (Somaliland) Boroma 09-Nov Fri Garowe (Somaliland) 10-Nov Sat (Puntland) 11-Nov Sun Garowe Garowe Nairobi Bossaso Bossaso 12-Nov Mon Nairobi (Puntland) Nairobi (Puntland) Nairobi 13-Nov Tues Mogadishu Mogadishu 14-Nov Wed (South Central (South Central 15-Nov Thurs Somalia) Mogadishu Somalia) Mogadishu (South Central (South Central 16-Nov Fri Somalia) Somalia) 17-Nov Sat Genny 18-Nov Sun 19-Nov Mon Nairobi Nairobi Nairobi Nairobi Nairobi 20-Nov Tues Nairobi Nairobi Nairobi 21-Nov Wed 22-Nov Thurs 23-Nov Fri

Notes:

1. For the crop sector, Ian Robinson, an independent consultant, conducted follow-up interviews in Nairobi in late December and early January.

2. The evaluation mission initially included a visit to Borama (Somaliland). However, due to changes in the programme the visit was cancelled. In December, Sadia Mussa Ahmed visited Borama and she also conducted follow up interviews in Hargeisa.

3. The overall mission for the CFW study lasted from September 8 to November 24. Savina Tessitore spent the first part of the mission at FAO Somalia Office in Nairobi to conduct scoping interviews with internal and external stakeholders and further refine the study methodology and then one month in Dollow (18 October-15 November).

Annex 4 Team members

Name Background Responsibility Margie Buchanan-Smith Margie Buchanan-Smith has been working in the humanitarian Team leader, responsible also for FAO aid sector for over 25 years, as an evaluator, a policy information systems, coordination and researcher and adviser, a humanitarian programme manager resilience and as a trainer and coach. She has co-authored a recent guide on ‘Evaluating Humanitarian Action’ for ALNAP and facilitates annual training courses on evaluation. She has particular expertise on the Horn of Africa, especially Sudan, in food security and livelihoods. She is a Senior Research Associate with the Overseas Development Institute, London, and is a Visiting Fellow at the Feinstein International Center at Tufts University. M.Sc. Agricultural Economics

Tim Leyland Tim Leyland is a consultant, originally from Kenya, now Livestock sector based in New Zealand. He has over 25 years’ work experience in international development and research with a broad range of experience from community-based projects to training of trainers, privatisation of veterinary services, policy and legislative change at national and international levels and evaluations in Papua New Guinea, Afghanistan, Pakistan, Nepal, Jordan, Sri Lanka and Sub-Saharan Africa. Mr. Leyland has practical experience managing large multi- country development projects funded by USAID, DFID, the EC and NGOs. M.Sc. Tropical Animal Production and Health,

Mohamood Noor Since retiring from the World Bank in 2004, Mohamood Noor Agriculture sector: crops has been working as an agricultural consultant. He studied in the Universities of Arizona (BSc), University of Wyoming (MSc) and University of Massachusetts (PhD) in plant breeding (1963-1970). After completing his studies in US universities, he started his career as a post-doctoral fellow at the International Institute of Tropical Agriculture (IITA), Nigeria, (1970-71). He went back to Somalia and became the Head of the Central Agricultural Research Station (1971-73), the Dean of the Faculty of Agriculture and Vice-President of the Somali National University (1973-80) and Vice-Minister of Agriculture (1980-90). In June 1992 he joined the Special Program for African Agricultural Research (SPAAR), at the World Bank and retired from the World Bank as Senior Agriculturist. Since his retirement he has worked as an agricultural consultant with the World Bank and with other international organizations and NGOs until 2010. Ian Robinson Dr W. Ian Robinson has 42 years of wide ranging agricultural Supplementary evaluation of the agriculture development experience encompassing 11 years of project sector: crops implementation in the Middle East, Africa and South-East Asia and more than 30 years of short to medium term technical assistance with 300+ missions to programmes and projects in, mostly but nor exclusively, conflict and post-conflict zones for UN, NGOs, multi and bilateral agencies. Following 20 years with the Centre for Arid Zone Studies, Bangor, Wales (as founder member in 1985, then as Director between 1993 and 2005), he formed AA International Ltd based in Aberystwyth, to concentrate on food security assessment and programme evaluations. BSc Agricultural Science; PhD

Yacob Aklilu Yacob Aklilu is a senior livelihoods specialist with in-depth Trade and markets; private sector knowledge of humanitarian and development programming development and policies in Africa, including extensive reviews of food security policies and approaches in Ethiopia. An agricultural economist, he has more than 25 years experience of policy analysis and reform at national and regional levels. He has led the testing of alternative livelihood interventions in northeast Kenya and his livelihoods-based emergency intervention in Kenya received the ‘best project award’ from USAID in 2001. Yacob has specialist knowledge of livestock marketing at domestic, regional and international levels, and is instigator of the Pastoral Livestock Marketing Groups approach in Ethiopia. He has coordinated major humanitarian response efforts and development programmes in southern Africa for the United Nations, and participated in major analyses of livelihoods in Darfur and Ethiopia for USAID. Yacob is a Senior Researcher with the Feinstein International Center at Tufts University. Prior to joining FIC/Tufts he worked with UNICEF, the UN, FINNIDA and NGOs in Mozambique, Zambia and Iraq

Sadia Ahmed Sadia Musse Ahmed is the Country Representative of the Gender equality, community-based Pastoral and Environmental Network in the Horn of Africa development and targeting, partnerships, input (PENHA) for Somaliland/Somalia. For the past 20 years she into coordination, jointly with Shukria Dini held a number of different positions in the organization including Gender Program Coordinator. Before the collapse of the state of Somalia she was the Director for the Women’s Research Unit of the Somali Academy of Sciences and Arts (SOMAC). She is involved in a number of organisations in the Horn of Africa as a board member or chair. She is an international gender consultant for NOVIB and a member of the Gender Regional Network. She is the founder and Chair of the Gender Development and Policy Institute for Somali Speaking communities. BSc Social Anthropology; Gender and Development Course at IDS, Sussex University Shukria Dini Shukria Dini has over 10 years’ experience as a consultant in Gender equality, community-based the field of international cooperation and has participated in a development and targeting, partnerships, input number of assessments and evaluations for a range of into coordination, jointly with Sadia Ahmed international organizations including Humanitarian Outcomes Inc., OCHA, UNDP and UNPOS. She is the Founder and Director of the Somali Women’s Studies Centre. Shukria was part of the three-person team carrying out the IASC’s RTE of the humanitarian response to the crisis in Somalia in 2011, covering gender issue and the role of local actors. She was also a team member for the evaluation of emergency cash transfers and food voucher interventions in Somalia, carried out by nine INGOs and managed by UNICEF

BA in Political Science; MA in International Development Studies; Ph.D in Women’s Studies

Aurelie Larmoyer As an Evaluation Officer with FAO and previously with the Internal FAO management and administration World Food Programme, Aurelie Larmoyer has managed and issues participated in numerous project/programme and strategic evaluations including many covering humanitarian contexts, for example WFP’s school feeding interventions in emergencies and an impact evaluation of FAO’s CHF funded programme in Sudan as team leader. Previous work includes food security analysis with WFP and managing aid programmes in the field with international NGOs. M. Sc. Development and Humanitarian Aid

Savina Tessitore Savina Tessitore is an anthropologist by background. She has Cash for Work field study worked in various capacities for FAO-UN, private consultancy companies and research institutes, NGOs and local administrations. Her fields of expertise and research interests are mainly social protection and cash transfers, vulnerability reduction and its relationship with agency, power and rights, pastoralism, disaster risk management, rural development and food security. M.A. Development Studies

Genny Bonomi Genny Bonomi has been working with the FAO Office of M&E, documentation review and analysis Evaluation since 2010. As an Evaluation Analyst, Genny has supported the preparation of Country Evaluations for Zimbabwe and Sri Lanka, the evaluation of FAO’s response to the floods in Pakistan and the evaluation of the EC/FAO programme on linking information and decision-making to improve food security. Previously, she has worked for the Secretariat of the Development Assistance Committee at the OECD in the Peer Review and Evaluation Division. Genny holds a Ph.D in Economics.

ACRONYMNS

ADESO African Development Solutions

ADO

ASEP Advancement for Small Enterprise Program

AUSAID Australian Agency for International Development

BENALPA Benadir Livestock Professional Association

BVO

CEDA Community Empowerment and Development Action

CERELPA Central Region Livestock Professional Association

COGWO Coalition for Grassroots Women’s Organization

COOPI Cooperazione Internazionale

CNEA

CTA Chief Technical Adviser

CVL Central Veterinary Laboratory

DAI

DFCS Dollow Farmer Cooperative Society

DMA

DFID Department for International Development

DSPE Darusalaam Seed Production Enterprise

EC European Commission

ECTAD Emergency Centre for Transboundary Animal Diseases Operations

ECHO European Community Humanitarian Office

EU European Union

FAO Food and Agriculture Organization

FSNAU Food, Security and Nutrition Analysis Unit

GEELO Gender Education Empowerment & leadership Organization

HATI Horn of Africa Training Institute

ICRC International Committee of the Red Cross

IFAD International Fund for Agricultural Development

IFTIIN

JCC Jubbalandese Charity Centre

JICA Japan International Cooperation Agency

MALFR Ministry of Agriculture, Livestock, Forestry and Range

MOA Ministry of Agriculture

MOE Ministry of Environment

MOF Ministry of Fisheries

MOFMR Ministry of Fisheries and Marine Resources

MOL Ministry of Livestock

MOLAH Ministry of Livestock and Animal Husbandry

MURDO Mubarak for Relief and Development Organization

NAPAD Nomadic Assistance for Peace and Development

NERAD National Environment Research and Disaster-preparedness

PULPA Puntland Livestock Professional Associations

ROA Regional Office for Africa

RVI Rift Valley Institute

SAHSP Somali Animal Health Services Project

SAREDO Samawada Rehabilitation & Development Organization

SOADO Somali Organic Agriculture Development Organization

SATG Somali Agriculture Technical Group

SMPAH Standard Method Procedures in Animal Health

SOMDA Somali Meat Development Association

SOWELPA South West Livestock Professional Association

STVS Sheikh Technical Veterinary School

SWALIM Somali Water and Land Information Management

TFG Transitional Federal Government

TN Terre Nuova

TOT Trainer of Trainers

ULPA

UNHCR United Nations High Commissioner for Refugees

UNICEF United Nations Children’s Education Fund

UNPOS United Nations Political Office for Somalia

UON University of Nairobi

USAID United States Agency for International Development

WFP World Food Programme

WOCCA Woman and Child Care Organization

VAAW

VSF- G Veterinaires Sans Frontiers Germany

VSF- S Veterinaires Sans Frontiers Switzerland

FAO

Rodrigue Vinet, FAO Emergency Operation and Rehabilitation Division

Yves Klompenhouwer FAO Office of Support to Decentralisation

Luca Russo FAO Agricultural Development Economics Division

Sue Lautze FAO Emergency Operation and Rehabilitation Division

Chaya Nurosinghdass FAO Audit office

Graham Farmer FAO Food Security Cluster Coordination (ex-OiC FAO Somalia)

Charlotta Oqvist FAO Senior Finance officer

Fabio Quenel FAO Finance Officer

Velda Nylander FAO Finance Officer

Nadine Valat FAO Resource Officer

Eva Bolkart FAO Internal investigator

Jenny Ritter FAO Internal investigator

Emma Fitzpatrick FAO Global Cluster officer

Cristina Amaral FAO Chief - TCEO

Felix Njeumi FAO Animal Health Officer

Thomas Osborn FAO Senior Officer

David Brown FAO Fisheries and Aquaculture Officer

Helga Josupeit FAO Senior Fishery Officer

Toppe Jogeir FAO Fishery Industry Officer

John Latham FAO Senior Environmental Officer

Karen Frenken FAO Senior Water Resources Officer

Marjorie Smith FAO Audit Office

Rod Charters FAO Head of Regional Emergency office

Luca Alinovi FAO Officer in Charge

Rudi Van Aaken FAO Head of Programme Management Unit

Nicolas Tremblay FAO Programme Coordinator

Palmira Ciacciarelli FAO Planning and Coordination Unit

Consolata Ngemu FAO Planning and Coordination Unit

Francesco Giasi FAO Planning and Coordination Unit

Frank Nyakairu FAO Communications Officer

Alberto Moreno FAO International Security Officer

Zoe Druilhe FAO Resilience Focal Point

George Mvula FAO Programme Operations Officer

Francesco Baldo FAO FSC Coordinator

Sergio Innocente FAO Emergency Coordinator/ M & E

Virginia Vigliar FAO Monitoring Program Officer

Chana Opaskornkul FAO Monitoring Coordinator

Michael Oyat FAO M & E Officer (Hargeisa)

George Obhai FAO Monitoring & Evaluation Officer, Livestock Sector

Maureen Mungai FAO Project Monitoring Assistant (M & E)

Field Monitors (South Central) FAO Monitoring & Evaluation

Mohammed Jama FAO Field Manager- Hargeisa

Yusuf Moalim Amin FAO Liaison Officer- Mogadishu

Barkhad Shire FAO Field Support Manager- Garowe

Stoyan Nedyalkov FAO Procurement Officer

Agarwal, Rajiv FAO Operations Officer

Christian Abinassif FAO LOA Officer

Jose Lopez FAO Agriculture Coordinator

Laxman Reddy FAO Operations Officer, Agriculture Sector

Mohammed Salih FAO Marketing Officer- Agriculture Unit

Musse Shaie FAO Land Coordinator

Dario Cipolla FAO Head of Inputs & Knowledge

Julius Mwangi FAO Deputy Project Manager

Simeon Giuseppe FAO Cash FW

Laura Cortada-González FAO Research and Learning Officer

Julie Lawson McDowall FAO CFW Department Coordinator

Maximilla Omengo FAO Cash for Work Programme Officer

Paul Githumbi FAO Irrigation Engineer

Mohammed Barkhad FAO Agriculture Field Staff

Abdirashid Araye FAO Irrigation Engineer

Mohammed Warsane FAO Field Coordinator

Ahmed Nur FAO- Programme Officer

Mahdi Qayad FAO FAO Hargeisa

Sowda Mude FAO Local Researcher

Mohamed Jama FAO Team Leader

Mire Abdullahi FAO Team Member

Omar Mohamed FAO Team Member

Hoosh Banadir FAO Team Member

Cyprien Biaou FAO Livestock Coordinator

Daniel Wachira FAO National Operations Officer- Livestock Sector

Khalid Saeed FAO Emergency Livestock Response Officer

Sophycate Njue FAO Animal Health

George Matete FAO

Suleyman Khayre Mohamed FAO FAO Site Engineer

Abdillahi Rabille FAO Livestock Sector

Ahmed Aided FAO Livestock Sector

Ahmed Abdalle FAO Veterinary Officer

Fatuma Hassan FAO Veterinary Officer

Abshir Said FAO Veterinary Officer

Arne Andreaason FAO Fisheries Coordinator

Jorge Torrens FAO Fisheries Officer

Musse Gabobe Hassan FAO National Fisheries Consultant

Joy Mulema FAO Operations Assistant

Thomas Gabrielle FAO I.M Advisor

Tamara Nanitashvili FSNAU CTA a.i

Zoltan Tiba FSNAU Research Manager

Amos G Nyaoro FSNAU Operations Officer

Mohamed Moalim Hussein FSNAU Nutrition Focal Point for CSE

Abdulkadir M. Ahmed FSNAU Urban & IDPs Food Security Analyst

FSNAU Focal Points within Somaliland government

Robert Basil FSNAU Gender Analyst

Linda Panelli Gen-cap adviser

Nicholas Kerandi FSNAU Database

Tawakal Ahmed FSNAU Assistant- Livelihoods

Ahmed Mohammed FSNAU

Mahdi G. Qayaad FSNAU

Zoltan Balint SWALIM CTA

Hussein Gadain SWALIM Water Coordinator

Jeremiah G. Njeru SWALIM Information Management Co.

Evelyne Karanja SWALIM Information Officer

Mohamed Saeed SWALIM Water Resources Assistant

Mohamed Gees SWALIM Information Management Officer

FAO ECTAD

Dr. Bouna Diop FAO ECTAD Regional Director

Dr. Sam Okothe FAO ECTAD Epidemiologist

Dr. Joseph Litamoi FAO ECTAD Laboratory Adviser

Dr. Tabitha Kimani FAO ECTAD Socio Economist

Rob Alport FAO Kenya Assistant FAO Representative

Sadia Ahmed FAO/PENHA Evaluator

SFE- SUB REGIONAL OFFICE FOR EASTERN AFRICA (FAO)

Castro Camarada Sub- Coordinator Regional Office for Eastern Africa,

Fantahun Assefa SFE Field Programme Support and Monitoring Officer

Bernard Mtonga SFE Crop Production and Protection consultant

Leone Magliocchetti Lombe SFE Water management in Agriculture

Francois Tabsoba SFE CTA

Gigs Vant Klooster SFE Adviser

Mark Bowden UN RC/HC Somalia

Jo Nickolls UN Head of Humanitarian Coordinator’s Office

Kilian Kleinschmidt UN Deputy Humanitarian Coordinator, Mogadishu

Mohammed Barre UNDP Somalia Poverty Alleviation and Environment Protection

Abdul Qadir Rafiq UNDP Somalia Project Manager - Environment & Energy

Stephen Kinloch-Pichat UNDP Head of Sub-Office, Hargeisa

Emma Morley UNDP Project Manager, Somali Institutional Development Project

Bernard Mokam UNDP Head of Poverty Reduction and Environment Protection

Hassan Bilaal WFP VAM Officer

Mark Gordon WFP Cluster Coordinator

Simon Renk WFP Head of VAM

Regis Chapman WFP Head of Programme

Oliver Nkakudulu WFP Head of Area Office, Hargeisa

Allegra Baiochhi UN OCHA Deputy Head of Office

Kristine Verhoeven OCHA Senior Humanitarian Adviser, CAP focal point

Grainne Moloney UNICEF Kenya (ex-FSNAU) Chief, Nutrition Section

Nancy Balfour UNICEF Head of WASH

Peter Hailey UNICEF Senior Nutrition Manager

Mohamed Abdul Monen ROA/ UNEP

Ayaki Ito UNHCR Deputy Representative

Tariq Chaudhy UNPOS

James Wabacha AU IBAR Coordinator- SMPAH

Prof Ahmed El Sawalhy AU IBAR Director

Dr. Baba Soumare AU IBAR Chief- Animal Health

James Wabacha AU IBAR Advisor

ICRC

Mohamed Sheikh Ali ICRC Coordinator; Economic Security

Craig Strathern ICRC Delegate- Somalia

DONOR AGENCIES

Solomon Ngari AUSAID Senior Program Manager - Humanitarian

Shannon Ryan AUSAID Senior Program Manager (Food Security and Climate Change)

East and Horn of Africa – Regional Office

Emily Rainey AUSAID

Matti Karvanen Embassy of Finland Programme Officer, Security, Policy, Humanitarian Aid, Development Cooperation

Grith Anderson Danish Embassy Ambassador

Rebecca Terzeon DFID Somalia Deputy Head of DFID

Seb Fouquet DFID Somalia Humanitarian Adviser

Janil Shah DFID Somalia Deputy Policy Manager

Adhan Haji DFID Somalia PSD Adviser

Paul Machanga DFID Somalia Programme Officer

Chris Porter DFID Somalia Humanitarian Adviser

Linet Otibine DFID Somalia Humanitarian Programme Officer

Ricardo Losa Embassy of Spain Deputy Head of Mission

Morten Petersen ECHO

Isabel Faria de Almeida EU Head of Section Rural Development, Social Services & Infrastructure Somalia Unit

Baston R UNPOS

Yoichiro Toda JICA

Chiara Beatrice Chiwi Italian Cooperation

Sureka Khandagle USAID Regional Advisor- OFDA

Pete Mohan USAID Food for Peace

Sarah Ryman USAID

Paolo Toselli EU Food Security

Ernest Njoroge EC Livestock Expert

Calum McLean ECHO Advisor Food Security

Aldo Biondi ECHO Head of Office

Philippe Royan ECHO Representative for Somalia

Morten Petersen ECHO Somali Situation

Junail Shah DFID Somalia Policing Officer/ Resilience Focal Point

Caroline Rusten World Bank Senior Social Development Specialist

NGO’S

Luca Machina Antonio Agrosphere Head- Programme Officer

Abdulkadir Alim HATI Livelihoods

Mohamed Abdi Ibrahim DFCS Agriculture/ Food Security

Abdikarim BAshir DFCS Agriculture/ Food Security

Mohamed Rage DFCS Agriculture/ Food Security

Noor Hasssan DFCS Agriculture/ Food Security

Hawa Abdirahman DFCS Agriculture/ Food Security

Mohamed Jama Muse SATG Agriculturalist

Abdi Adow CEDA Executive Director

Ibrahim Omar Alle CEDA Field Coordinator

Abdikarim Mohid CEDA Field Supervisor

Degan Ali ADESO

Evelyn Njue COOPI Head

Alyssa Ricarda COOPI

Mahumud Hassan COOPI Project Officer

Abdikadir Mohid Abdullahi COOPI Team Leader- Dollow

Abdinasir Adan COOPI Agronomist

Abdullahi Mohamed Hassan COOPI Field Coordinator

Mohammed Sharrif Abdikadir COOPI Team Leader- Luq

Nur Musa Abdi NAPAD Finance and Administration Manager

Ahmed Abdirahman Farah NAPAD Livelihoods Programme Officer

Mohamed Ismail Ali NAPAD Assistant Livelihoods Programme Officer

Mohmmed Ahmed Arai NAPAD Executive Director

Abdi Ali Botan NAPAD Deputy Programme Manager

Ali Ahmed IFAD Regional Coordinator

Abdirao Omar IFAD Project Engineer

Mohamood Muse NERAD Executive Chairman

Ahmed Mohammed NERAD Deputy Chairman

Abdullahi Ali SESIMA Executive Chair

Ahmed Farah Al Mizan Executive Chair

Bube Abdulle DSPE Chairman

Mohamed Sichew DSPE Vice Chairman

Isack Al Mumin MURDO Chairman

Abdulkadir Ali Mohamed MURDO Programme Manager

Abdikarim Ibrahim Abdow HIJRA Programme Manager

Shure Madome Hassan HIJRA Field Officer

Raha Janegow Saacid Director

Sharife Addew COGWO Member

Mohamed Ali SAREDO Chairman

Mohamed Abdi SAREDO Field Focal Point

Abdi Khalif JCC Programme Officer

Bashir Ibrahim Adan JCC Field Officer

Ahmed BAshir SOADO Programme Manager

Abdifatah SOADO Agriculture Specialist

Ahmed SATG Agronomist

Adam Isse Ahmed SATG Agronomist

Adan Abdalla WOCCA Field Officer

Abdi Mohamud Ahmed Swisso- Kalmo Field Coordinator

Hassan M VAAW Beeyalay

Hassan M. Ali

Xasan Abikar Cali

Mohamed M. CERELPA Veterinary

Hussein AMin CERELPA Veterinary

Osman Bunde Abaid Kalundi Farmers Association Farmer

Qasim Dashow Abdullahi Kalundi Farmers Association Farmer

Hassan Nur Saleh Barey Farmers Association Farmer

Ali Mohammed Nuholo Barey Farmers Association Farmer

Cabdi S. WOCCA Farmer

Raqe Maxud Gafow IFTIIN Project Manager

Yasin Abdirahin D IFTIIN Field Staff

Hussein Ismail ADO Director

Adan Abdulahi BVO Director

Khadra Mohammed BVO Chairman

Nicolletta Buono VSF- G Programme Manager

Seif Maloo VSF- S Country Director; Kenya Somalia

Ahmed Takou BENALPA Secretary General

Ahmed Abdi Gedi SOWELPA Secretary General

Ali Roble SOWELPA Chairman

Nurta Sh. Mohamud BENALPA Chairperson

Hussein Hagi Adan SAHSP/ TN Central Coordinator

Ahmed Warsame Saeed Warsame Saeed Construction Company

Shuru Hussin Elmi SOMDA Member

Shuru Daalim Cali SOMDA Member

Abdirizak Warsane DAI/CNEA Animal Production Specialist

Sulub Ismail Ahmed VSF- G Project Officer- Dairy Project

Suweizava Y. Mohammed SOMDA Chairlady

Sinid Ibrahim SOMDA Vice Chairman

Sam Ogolla VSF-G Country Programme Manager, Somalia

Yasin Salah PULPA General Secretary

Abdisalam Warsane Veterinary Board Chairman

Abdirahman Aideed Oxfam Programme Director

Mohammed Adan Hassan Livestock Market Association Chairman

Ali Mohamed Ali ULPA General Secretary

Dario Zecchini Terre Solidali Regional Representative

Ahmed Farah Al Mizan Seed Co General Manager

Judith Gardner RVI Rapporteur

Jeff Worden RVI Consultant

Johara Bellali Save the Children Regional Programme Manager

Karimi Gitongo Save the Children Regional DRR ICCD Officer

Holly Welcome Radice Save the Children Food Security & Livelihoods Technical Manager

Abdiwahal Sh. ASEP Coordinator

Mark Bradbury RVI Regional Director

Khadar DMA Information/ Data Collection

Abdullahi I DMA Chairman

Ali Abdurahman DMA Head Of Resource/ Coordinator

Hana Abbas DMA Head of Early Warning

Abdulaziz Farah DMA Deputy Chairman/ Head of Programme

Mohamed Muse Ahmed Barfisco General Manager

Ali Muse Barfisco Member of Association

Kiim Aadan Iskaashata Beraha Coop Mobilizer

Ayaan Abdullahi Iskaashata Beraha Coop Secretary

Mohammed Adwe JDO

Zahara Mohamed GEELO Head

Mohamed Hassan ADA

Mohamed Ahmed ADA

Hassan Haj ADA

INDIVIDUALS/ BENEFICIARIES

Mohammed Farah Farmer (Beneficiary) Farm Owner

Mohamud Ahmed Farmer Farm Owner

Habibo Yusuf Bantaal Farmer Farm Owner

Fadumo Khalif Farmer (Beneficiary) Farm Owner

Abshiro Mohamed Farmer Farm owner

Mohamed Khuse Bulla Qalool Farmer Farm owner

Farah Mohamed Bulla Qalool Farmer Farm owner

Abdu Abdullahi Bulla Qalool Farmer Farm owner

Adan Mohamed Bulla Qalool Farmer Farm owner

Nuurta Abdi Bulla Qalool Farmer Farm owner

Mohamed Osman Farmer Farm owner

Dahir Abdullahi Farmer Farm owner

Abdi Osman Farmer Farm owner

Samo Awes Farmer Farm owner

Suban Qoriyow Farmer Farm owner

Anfou Abdi Farmer Farm owner

Abdullahi Adow Farmer Farm owner

Qoriyow Robow Farmer Farm owner

Mohamud Yusuf Farmer Farm owner

Hussein Affrin Bantaal Farmer Farmer Owner

GOVERNMENT

Abdirashid Hassan abdinoor Government Official District Commissioner- Dollow

Mohammed Muse Ahmed Local Government Hargeisa Livestock Market

Nageb Yusuf Mohammed Local Government Hargeisa Livestock Market

Somaliland

Dr Abdi Aw Dahir Ali Ministry Of Livestock Minister

Professor Farah Mohamoud Elmi Ministry of Agriculture Minister Geedole

Abdullahi Ismail Ministry of Agriculture Director General

Dr Ahmed Mohamoud Mohamed Ministry of Agriculture Director General of Livestock Ministry

Abdi Musa Mohammed Ministry of Agriculture Technical Advisor

Ibrahim Omar Kahin Ministry of Agriculture Director of Production

Saad A. Shire Ministry of National Minister Planning and Development

Dr Abdul Rashid Ahmed Guleid Ministry of National Director-General Planning and Development

Faysal Farah Ministry of Fisheries Deputy Production and Training

Al-aziz Salad Ministry of Fisheries Nugal Region Coordinator

Nasir Ibrahim Ministry of Water Manager, SWALIM Data Centre

Mohamed Hussein Ministry of Water Assistant, SWALIM Data Centre

Abdikami Yusuf Ministry of Agriculture Manager, SWALIM Data Centre

Idil Hussein Ministry of Agriculture Assistant, SWALIM Data Centre

Hussein M. Local Government Mayor- Hargeisa

Ibrahim Saeed Ismail Chamber of Commerce Advisor

Mahamed Shukri Jama Chamber of Commerce Chairman

Dr. Abdikarim Mahamoud Somaliland Veterinary Board

Abdirahman Mohamoud Maandeeq Slaughter House Manager

Mohamud Roble Hers Mandeeq Slaughterhouse Chairman of Board

Puntland

Hon. Mohamed Farah MOFMR Minister of Fisheries & Marine Resource

Mohamoud Abdulahi Salah MOFMR Vice Minister of Fisheries

Abdiwahid Hersi Joar MOFMR Director General

Abdulrahman M.W Ministry Of Fisheries Department of Direct Production

Abdikadir Sidi Ministry Of Fisheries Director of Planning

Nadifo Mohamud Yusuf Ministry of Fisheries Health

Eng M.F. Adam MOFMR Staff

Abdulkader M. Yousuf MOFMR Staff

Hon. Mohamoud Haji Salah MOA Minister of Agriculture & Irrigation

Jama Gesod Ministry of Agriculture

Alrhaman Dirie Arab Ministry of Agriculture

Ali Ahmed Omar Ministry of Agriculture R. Director

Hon. Saed Hassan Shire MOLAH Minister

Nur Axmed MOLAH Deputy Minister

Abdiwali Hersi Nur MOLAH Deputy Minister

Dr. M. A. Agaweyme MOLAH

Abdiaziz Botan MOLAH Admin/ Finance Director

M. Abdulkadir MOLAH Secretary

C/ulali Xirs MOLAH Staff

Dr. Ahmed H. MOLAH Animal Health

Dr. Farhan Ahmed Yousuf MOLAH Planning Director

Cumar Irbad MOLAH Consultant

Hon. Abdiqani Elmi Ministry Of Environment Minister of Environment

Mohamoud Ahmed Municipal Council of Burao Mayor- Burao

TFG

Hussein Mohamed Sheikh Transitional Federal Deputy Prime Minister Government

Mohamed Mohamud Ministry of Agriculture D.G

Abdirahman Nur Qeiliye Ministry of Agriculture Deputy D.G

Abdirahman Sharif Mukhtar Ministry of Agriculture Director of Animal Health Department

Heshi Osman Mohamed Ministry of Agriculture Director of Forestry and Range

Clrahim Adam Ministry of Agriculture Director of Planning and Training

Ibdi Welle MALFR Director: Production

Abdulkadir Hassan Warsame MALFR Director of Research & Extension Department

Hussein Muse Mahsha MALFR Director of Department

Yusuf Hussein Mohamoud MALFR Director- Administration & Finance

Mohammed Farah MALFR Director- Land, Water & Environmental Works

Abdi Ministry of Maritime Director General

Ali H. Ismail Ministry of Education Technical Adviser

Mohamed H. Ibrahim Ministry of Education Director General

Abdulla Jumall Disaster Management Chairman Agency

Aziz Fara Disaster Management Deputy Chair and Head of Agency Programmes

Khidir Disaster Management Information Officer Agency

Abdul Rahman Disaster Management Head of Response and Agency Coordination

Abbi Disaster Management Head of EW Agency

Abdikhayr Mohamed Ministry of Focal Point Agriculture/FSNAU

Ali Ahmed Omar Ministry of Agriculture Focal Point /FSNAU

Xaawo M Ministry of Fisheries Secretary

M. Nur Ministry of Fisheries Director

Abdi Adan Ministry of Fisheries Member

Abdikadir Barkhad Ministry of Fisheries Member

Noor Osman Ministry of Fisheries

Maxed Abshir Ministry of Fisheries Member

Maxudali Said Ministry of Fisheries Member

Bihel Said Ministry of Fisheries Member

Adan Fiin Ministry of Fisheries Member

Alkadir Mahomed Ministry of Fisheries Member

OTHERS

Abdulkadir Khalif STVS Tutor

Ibrahim Osman Suleiman STVS Tutor

Abdullahi Shikh STVS Dean of Studies

Nur A. STVS Acting Principle

Mubarak Hasan Jama CVL Head of CVL

Suad Hussein Nur CVL CVL Staff

Ayan Abdi Mohamed CVL CVL Staff

Prof. J.N. Ombui UON Chairman, Department of PHPT

Seth Kisia UON Lecturer- Veterinary, Anatomy & Physiology

Nigel Nicholson Freelance consultant working on FSNAU project formulation

Susanne Jaspars Policy researcher

Chris Print Independent Consultant

Mark Bradbury Rift Valley Institute Horn of Africa and East Africa Regional Director

Abdirahman Mahmood Hussein Consultant PPP

Abdullahi Muse TOT

Muse Warsame TOT

Abdirazaq Nadif TOT

Khalif Yusuf Ibrahim TOT

Hussein Dahir Hiddig Boat Company Manager

Annex 6 Country Projects implemented by FAO Somalia between 2007 and 2012 Total Actual Actual Project Symbol Project Title Budget EOD NTE (DWH) Somalia water and land information and GCP /SOM/045/EC 2004-10-01 2007-12-31 $4,781,250 management system (Phase II) Rehabilitation of livelihoods in the fisheries sector OSRO/SOM/508/CGC 2005-09-01 2007-02-28 $154,460 affected by the Tsunami Rehabilitation of livelihoods in the fisheries sector GCP /SOM/046/GER 2005-12-01 2007-02-28 $141,591 affected by the tsunami Support to the survival and recovery of children, OSRO/SOM/509/CEF women and vulnerable communities in fishing 2006-01-01 2007-04-30 $483,340 villages affected by the tsunami Community-based local economy development in OSRO/SOM/601/DEN 2006-02-01 2009-06-30 $1,242,989 Somaliland and Puntland ARDOPIS -Agricultural Rehabilitation and Diversification of High Potential Irrigation Schemes OSRO/SOM/510/EC 2006-03-01 2009-02-28 $2,839,314 in Southern Somalia in Lower Juba and Shabelle River Basin (Jamama, Agfoi and Balad Districts) Agricultural Rehabilitation and Diversification of High Potential Irrigation Schemes in Southern OSRO/SOM/511/EC 2006-03-15 2008-12-14 $3,567,399 Somalia (ARDOPIS) in Merka and Qoryooley districts (Somalia - Lower Shabelle) Provision of basic livelihood services to directly OSRO/SOM/603/SWE 2006-04-01 2007-07-31 $909,174 increase food access Nutrition Information Project for Somalia - Year OSRO/SOM/605/USA 2006-04-01 2007-06-30 $700,000 2006 Support to the Sustainable Management of the GCP /SOM/047/EC 2006-08-01 2009-12-31 $2,204,059 Shebelle and Juba Rivers in Southern Somalia Consolidation of Integrated Pest Management OSRO/SOM/607/EC 2006-08-01 2009-06-30 $1,414,853 Initiative in Somalia Support to the Food Security Analysis Unit - OSRO/SOM/604/EC Understanding Livelihoods in Somalia - FSAU 2006-08-31 2009-02-28 $5,486,777 Phase V Emergency livelihoods support to flood-affected OSRO/SOM/613/CHA 2006-11-27 2007-02-26 $508,200 riverine farmers

OSRO/SOM/612/EC Improvement of traditional grain storage system 2006-12-01 2007-03-31 $384,184

OSRO/SOM/608/EC Support to Pastoral Livelihood Development 2007-01-01 2009-06-30 $2,310,311

Nutrition Information Project, Food Security Analysis OSRO/SOM/702/USA 2007-01-01 2008-12-31 $2,085,000 Unit, Somalia (Year 2007) Emergency livestock disease surveillance and vaccination/treatment in support of pastoralist OSRO/SOM/701/NOR 2007-01-01 2007-12-31 $971,188 livelihoods in flood-affected areas of Southern Somalia Capacity Enhancement of the Principal Somalia UTF /SOM/041/SOM 2007-01-01 2007-12-31 $605,941 TFG Line Ministers Improvement and sustainable utilisation of plant OSRO/SOM/611/EC 2007-04-01 2010-03-31 $3,430,000 genetic resources programme Increased Capacity to Respond Effectively to OSRO/SOM/705/SWE Disaster through a Strengthening of the Somalia 2007-04-01 2008-12-31 $1,764,694 Information Base Total Actual Actual Project Symbol Project Title Budget EOD NTE (DWH) OSRO/SOM/704/NOR Rural livelihoods support in the Shabeelle Valley 2007-04-01 2008-07-31 $1,698,564

Livelihoods Support to Tsunami-Affected Fishermen OSRO/SOM/703/MUL 2007-05-01 2008-02-29 $237,884 Communities in Somalia Support to pastoral communities on livelihoods risk OSRO/SOM/706/ITA 2007-09-01 2008-12-31 $955,290 reduction

SOM/07/001/ /01/34 Support to the Somali meat export 2007-09-01 2008-11-30 $542,565

Protection, Reintegration and Resettlement of IDPs OSRO/SOM/712/UNJ 2007-11-01 2010-12-31 $700,002 in Bosaso Rehabilitation of Rural Infrastructure and Support OSRO/SOM/711/SWE for Populations in Food Security Crisis in Lower 2007-12-01 2008-12-31 $776,850 Shabeelle Somalia Water and Land and Information GCP /SOM/048/EC 2008-01-01 2010-01-31 $3,451,310 Management - PHASE III Livelihood support intervention in support of food OSRO/SOM/707/EC security for most vulnerable households in Southern 2008-01-01 2008-12-31 $1,035,503 Somalia Irrigation Consolidation and Development of better farming systems in the Middle and Lower OSRO/SOM/708/EC 2008-03-01 2010-09-30 $2,800,000 catchments of the Dur-Dur Watershed, Awdal Region, Somaliland Integrated intervention to strengthen the capacity to OSRO/SOM/802/SWE 2008-04-01 2009-09-30 $2,696,000 respond effectively to disaster in Somalia

OSRO/SOM/710/EC Somalia Animal Health Services - SAHSP Phase II 2008-04-01 2010-06-30 $254,090

Financial support to FSAU Livelihood Baseline OSRO/SOM/804/USA 2008-05-07 2009-02-28 $400,000 Activities Integrated Cash and Food for Work in Support of OSRO/SOM/807/CHA Populations in Food Security Crisis in Middle and 2008-07-07 2008-11-07 $2,000,001 Lower Shabeelle Regions of Southern Somalia Improve the level of preparedness in Somalia for OSRO/SOM/806/USA Rift Valley Fever (RVF) and other climate related 2008-07-15 2009-06-30 $900,000 diseases Support to Pastoral Communities on Livelihood Risk OSRO/SOM/803/ITA Reduction in the Bay, Hiraan, Middle and Lower 2008-09-01 2010-01-31 $899,921 Shebelle Regions OSRO/SOM/808/WBK Rapid Response Rehabilitation of Rural Livelihoods 2008-10-01 2010-09-30 $7,009,284 Support to vulnerable and food insecure populations OSRO/SOM/809/AUS in the DurDur watershed of the Awdal region 2008-10-15 2009-06-15 $134,771 through the distribution of seeds of cereal crops. OSRO/SOM/805/EC Somali Livestock Survey - Pilot Study 2008-11-01 2010-03-31 $220,000

OSRO/SOM/912/SWE SIDA Support to the livestock survey pilot study 2008-11-01 2010-03-31 $53,500 Improvement of storage facilities and post harvest, OSRO/SOM/811/EC handling and storage practices, in Bay Region of 2008-12-16 2011-07-15 $1,046,035 Southern Somalia Agricultural Rehabilitation and Diversification of OSRO/SOM/810/EC High Potential Irrigation Schemes in Southern 2009-02-01 2011-04-30 $1,977,943 Somalia Food Security & Nutrition Analysis Unit for Somalia GCP /SOM/041/MUL 2009-03-01 2013-02-28 $20,492,882 (FSNAU) Total Actual Actual Project Symbol Project Title Budget EOD NTE (DWH) Livelihoods support project for improved food OSRO/SOM/902/EC security for the most vulnerable households in 2009-04-01 2010-05-31 $4,185,527 Southern Somalia Support to Pastoral Communities on Livelihood Risk OSRO/SOM/905/SPA Reduction in the Gedo and Lower Juba Regions of 2009-04-15 2010-10-31 $1,305,774 Somalia Integrated Cash for Work and Productive Assets Rehabilitation/transfer in Support of Populations in OSRO/SOM/906/CHA 2009-04-23 2009-12-31 $2,500,000 Food Security Crisis in Mudug and Galgaduud Regions of Central Southern Somalia Somali Agri-Productive Livelihoods and Enterprises OSRO/SOM/904/QAC 2009-04-29 2010-12-28 $1,810,985 Project (SALEP)

GCP /SOM/042/EC Integrated support to rural livelihoods 2009-05-01 2011-07-31 $13,556,449 Time-Critical Emergency Livestock Vaccination and Treatment Project for the Protection of Productive OSRO/SOM/909/CHA Livestock Assets of Primary Importance to the 2009-09-01 2009-11-30 $640,181 Survival of Pastoralist Populations in Crisis in Gedo and Lower Juba Regions of Southern Somalia OSRO/SOM/910/CHA Pastoralists Emergency Response (PER) 2009-11-01 2010-01-31 $2,500,000 Integrated cash-for-work and productive assets rehabilitation in support of populations in food OSRO/SOM/911/CHA 2010-01-25 2010-04-24 $2,501,842 security crisis in Gedo Region of Central Southern Somalia. Somalia Water and Land Information Management GCP /SOM/049/EC 2010-02-01 2013-01-31 $5,049,999 System, Phase IV Emergency Livestock Disease Surveillance and OSRO/SOM/002/WBK 2010-03-03 2010-12-31 $900,000 Control and Promoting Competitive Meat Industry Improvement and Sustainable Use of Genetic Plant OSRO/SOM/005/EC 2010-04-01 2013-03-31 $2,394,000 Resources Programme (Phase II) Livelihoods protection and support for vulnerable OSRO/SOM/006/EC 2010-04-01 2011-01-31 $1,897,500 households in Gedo region, Somalia Improve livelihoods of the fishing, pastoral and agro pastoral communities, women groups and GCP /SOM/043/SPA 2010-06-01 2012-05-31 $2,437,229 association members in Iskushuban district, Bari region of Puntland Support to Pastoral Livelihood Development (Phase OSRO/SOM/004/EC II). Promoting an internationally competitive Somali 2010-08-01 2013-07-31 $2,400,000 meat industry Value chains and productive sectors in Somaliland OSRO/SOM/007/UK 2010-12-01 2012-07-31 $5,092,275 and Central Somalia (FAO component) Value Chains and productive sectors in Somaliland and Central Somalia (Component implemented by OSRO/SOM/010/UK 2010-12-01 2012-07-31 $1,973,400 Participating Organisations ILO,UNDP, and Save the children) Value chains and productive sectors in Somaliland OSRO/SOM/011/UK 2010-12-01 2012-07-31 $64,233 and Central Somalia (Agent Admin fee) Improving food security in agro-pastoral areas of OSRO/SOM/009/ITA 2010-12-15 2011-10-15 $1,634,877 Hiraan region in Somalia Support to the Implementation of the Tsunami OSRO/SOM/812/WBK 2010-12-20 2013-03-31 $1,579,646 Livelihood Recovery Project Livelihood support to IDPs and riverine communities OSRO/SOM/102/CHA 2011-03-01 2011-12-31 $2,489,090 in Humanitarian Emergency and Acute Food and Total Actual Actual Project Symbol Project Title Budget EOD NTE (DWH) Livelihood Crisis in southern Somalia

Capacity building project for effective OSRO/SOM/105/CHS implementation coordination and monitoring of 2011-03-30 2012-05-30 $357,000 cluster activities OSRO/SOM/101/EC Support to the Somali Animal Health Sub-sector 2011-04-01 2012-12-31 $287,758 Hydrogeological Assessment and Survey in OSRO/SOM/103/CHS SelectedAreas of Somaliland and Puntland - (CHF- 2011-04-07 2012-07-31 $499,946 DMA-0489-123) Livelihood support for agro pastoral communities in OSRO/SOM/106/CHS humanitarian emergency and acute food and 2011-04-20 2012-04-20 $2,115,233 livelihood crises in South Central Somalia Protecting pastoral community livelihood assets in South Central Somalia and enhancing the OSRO/SOM/107/CHS 2011-04-20 2012-04-20 $1,429,065 community capacity to cope with shock through an integrated approach Irrigation Consolidation and Development of better OSRO/SOM/108/EC farming systems in the Dur Dur Watershed, Awdal 2011-07-01 2014-06-30 $3,750,000 region, Somaliland - phase II Livelihood Support for Agro-pastoral Communities OSRO/SOM/111/CHA in Humanitarian Emergency and Acute Food and 2011-07-22 2012-01-20 $5,999,998 Livelihood crises in South Central Somalia Drought Management and Livelihood Protection OSRO/SOM/112/WBK 2011-08-01 2012-08-30 $4,666,667 project Drought Management and Livelihood Protection OSRO/SOM/122/WBK 2011-08-01 2012-08-30 $3,733,333 project Emergency Input Distribution Deyr 2011 for OSRO/SOM/117/SWI Livelihood Support to Rainfed and Riverine 2011-08-01 2011-12-31 $577,688 Households in Southern Somalia Livelihood support to pastoral and agro-pastoral OSRO/SOM/114/SPA 2011-08-03 2012-08-02 $2,816,901 households in Southern Somalia

OSRO/SOM/113/UK Livelihood support to famine affected households 2011-08-15 2011-12-31 $6,212,800

Livelihood Support to Famine Affected Households OSRO/SOM/109/AUL 2011-08-15 2012-08-15 $2,095,200 in South Central Somalia

OSRO/SOM/121/EC Emergency Crisis Response Project 2011-08-19 2012-11-18 $10,043,972 Livelihood Support to Agro-pastoral and Riverine OSRO/SOM/119/USA Households in Southern Somalia - (AID-OFDA-IO- 2011-09-01 2012-03-31 $4,000,000 11-00032) OSRO/SOM/118/SWI Emergency Crisis Response 2011-09-01 2012-05-31 $1,688,029

OSRO/SOM/115/BEL Emergency Drought Crisis Response in Somalia 2011-09-01 2012-08-31 $1,333,334 Support and diversification of sustainable livelihood OSRO/SOM/120/ITA opportunities in urban /peri-urban centres of 2011-11-01 2013-11-01 $1,333,332 Puntland Livelihood Support to Pastoral, Agro-pastoral and OSRO/SOM/124/USA 2011-11-08 2013-05-31 $19,288,162 Riverine Households in Southern Somalia

TCP/SOM/3302 TCP Facility 2011-11-15 2012-03-31 $49,637

Emergency support to animal health and production TCP/SOM/3301 2011-12-01 2012-11-30 $400,000 in drought stricken areas of Somalia Total Actual Actual Project Symbol Project Title Budget EOD NTE (DWH) Support to Agricultural Productivity and Grain OSRO/SOM/126/EC 2011-12-15 2013-12-14 $5,000,000 Storage in the Irrigated Areas of South Somalia

OSRO/SOM/125/EC Support to Agricultural Markets in Somalia 2011-12-15 2014-06-14 $2,750,000

Emergency Response Programme to the OSRO/SOM/201/MUL 2012-01-01 2013-06-30 $22,272,569 Humanitarian Crisis in Somalia (2012) Improvement of livelihood of vulnerable households OSRO/SOM/110/EC in urban and peri-urban areas of Central Somalia 2012-02-10 2014-12-31 $1,769,937 and Mogadishu (in collaboration with ILO) Integrated Assistance to Voluntary Returnees in OSRO/SOM/202/CHS 2012-05-17 2013-03-31 $3,204,659 Somalia

Regional Projects

Total Project Symbol Project Title Actual EOD Actual NTE Budget (DWH) Facilitating the formation and initiation of a Regional Fisheries Arrangement for the GCP /RAF/395/SWE 2005-03-01 2008-04-30 $764,709 management of sustainable non-tuna fisheries in the South-West Indian Ocean (SWIO) Rift Valley fever and climate related diseases OSRO/RAF/706/USA 2007-06-01 2008-03-31 $1,300,000 control in Eastern Africa Regional Support Programme for the coordination OSRO/RAF/801/EC and capacity strenghtening for disaster and drought 2008-01-01 2009-06-30 $4,027,892 preparedness in the Horn of Africa. Technical Assistance to the Somali Livestock MTF /INT/084/AU 2009-02-03 2011-07-31 $607,632 Certification Project (SOLICEP) Somali Ecosystem Rinderpest Eradication MTF /INT/074/AU 2009-04-29 2010-06-30 $153,217 Coordination Unit (SERECU) Project II Support to capacity building to promote formal TCP/RAF/3301 marketing and trade of livestock and livestock 2011-01-01 2012-09-30 $489,000 products from the Horn of Africa Support to Capacity building to promote formal TCP/RAF/3308 marketing and trade of fish and fish products from 2011-06-01 2013-05-31 $494,000 and within the Horn of Africa OSRO/RAF/113/FRA Emergency Drought Response 2011-10-28 2012-10-27 $2,743,484 OSRO/RAF/112/BEL Emergency Drought Response in the Horn of Africa 2011-12-01 2012-11-30 $1,291,990 OSRO/RAF/114/FRA Drought Recovery and Smallholder Adaptation 2012-05-01 2013-12-01 $1,300,000 Programme in Djibouti and Somalia

Annex 7 Evaluation team’s analysis of the FAO Somalia Resilience Strategy, and of the proposed shift in paradigm and implementation

Opportunities associated with the resilience strategy

o FAO is in a strong leadership position to develop resilience thinking and programming, intellectually, within the UN system and within the wider international community in Somalia and beyond, for example also inputting into the DFID strategy on resilience o As donors and other agencies currently express much interest in resilience, there is a window of opportunity right now to develop more effective ways of programming, and to challenge and change the current aid architecture eg pushing for longer-term time funding cycles o The changing political context and end of the transition period in Somalia means that there is an opportunity opening to work more closely with government and to engage in governance issues. This needs to be monitored carefully in order to identify opportunities o This is also an opportunity to move away from a purely state-building paradigm to refocus on issues of poverty and vulnerability o If the new government in Mogadishu signs up to the IGAD strategy/ platform on resilience, there will be an opportunity to engage with the IGAD strategy and to bring conflict more centrally into that strategy o The joint FAO/ WFP/ UNICEF resilience strategy is an opportunity for these three UN agencies to work much more closely together than has traditionally been the case, and to have a significant impact on the ground o At community level, people do not think in terms of agency mandates, so this is an opportunity for holistic programming according to community needs and priorities rather than driven by agency mandates and priorities. It is thus an opportunity for more integrated analysis at field level o This presents an important opportunity to learn about Somali resilience and traditional coping strategies at household and community level o FAO’s CFW experience is an opportunity to be capitalised upon in terms of building household and community resilience, by building assets and distributing cash at household level

Conceptually and strategically – issues and gaps identified by the team

o Drawing on the presentation at the resilience workshop re the history of the resilience concept from ecology (which looks at resilience over decades), and the experience of the PSNP in Ethiopia (which has contributed to resilience very slowly), the temporal dimension of FAO’s resilience strategy is too short and needs to be revisited. NB the timescale for IGAD drought resilience strategy is 15 years

o The ‘system’ that FAO is addressing in its resilience strategy – household and community level – is too narrowly defined. In order to build resilience governance issues must be addressed at all levels, from national to local. Local authorities are key stakeholders in interventions at the community level (and in the changing political context in Somalia it may be unacceptable NOT to be involving government authorities centrally in the resilience programme). This means working with agencies and projects that are working to strengthen governance.

o Importance of making connections across systems in the resilience strategy, ie linking livelihoods systems with agro-ecological zones/ marine resources and with social systems in terms of resilience thinking

o The resilience thinking and strategy has been developed and devised by the leadership of FAO Somalia and has therefore been driven from the top- down. It is not yet ‘owned’ or taken on board at the field level. o The strategy has some gaps in terms of analysis, specifically: o It is clear re the vision, but is lacking an analysis of FAO’s ability to deliver that vision and the distance that needs to be travelled for that to be possible, especially in terms of FAO’s resources and skill-sets on the ground o Much of what the resilience strategy is attempting to achieve has been tried before in Somalia, albeit not always successfully. The strategy is lacking acknowledgement of previous work done by other agencies, of good practice examples and of what can be learned from earlier efforts o The strategy also needs to be clearer that the most immediate factor affecting resilience in Somalia is violent conflict, and needs to acknowledge that one of the key factors building resilience has been diaspora remittances (note forthcoming FSNAU research analysis of this) o It should also be noted that what is being offered as innovation in the FAO strategy is not necessarily ‘new’. It is recognised good development practice, although FAO and others may have struggled to follow this practice in the past o There is an acknowledged (although not in the strategy) lack of research and understanding of how Somali society has transformed and adapted over the last decade, in other words of trends in how livelihood systems and settlement patterns have changed (eg with displacement and urbanisation), and of trends in vulnerability and resilience within different livelihood and population groups. This is a major gap that the resilience strategy needs to address urgently o The reality and implications of power dynamics, within and between communities and livelihood groups, between clans, and including gender inequalities, are currently missing in the strategy, yet are likely to be a critical determinant of vulnerability, and a critical factor to be taken into consideration in efforts to build resilience

Challenges in implementing the resilience strategy in practice

o There is a significant gap between the vision set out in FAO’s resilience strategy and the current reality of what FAO is doing and how it is working, and therefore its current skill-sets eg FAO does not have a comparative advantage or skill sets suited to community development work. o The lack of research and trend analysis of how local communities and households have developed their own resilience, right now and over time, is a major challenge to the programme, and means it must draw upon the knowledge of Somali civil society as much as possible, as well as FSNAU’s historical database o The FAO programme has an important role to play in helping to develop measures of resilience. The challenge is to ensure that these are appropriate to the context, accessible and easily understood o Insecurity and lack of access means that FAO may not be able to reach those who may be most vulnerable and most in need of being supported in terms of building their resilience as they are currently most affected by ongoing conflict o Volatility and unpredictability of the Somalia context are likely to be key challenges for some years to come o The roll-out of the strategy from the top-down and the fact that the three UN agencies are mostly not working together at field level is a challenge that will require a change in organisational culture in all 3 agencies, including FAO o The mode of implementation of FAO’s resilience strategy is still unclear: to what extent is FAO planning to implement directly versus working with NGO partners, government authorities etc? o FAO does not have a track record of real partnership with NGOs, yet such partnerships may be crucial to implementing the resilience programme and to ensure that this new paradigm is not imposed on FAO’s implementing partners o FAO’s administrative procedures are often slow and cumbersome, and there are already examples of how these are not in synch with the other UN agencies it is partnering, hampering joint programming on the ground

Recommendations 1) Extend the FAO resilience strategy to 15 years, broken down into 3 year programming cycles, for funding and ongoing review purposes, with clear milestones identified

2) FAO’s strategy on resilience should be broadened from the community and household level to incorporate and address governance issues. Implications: a. FAO’s capacity-development component of the strategy should be more explicitly related to its resilience programme b. FAO should work closely with agencies and programmes addressing governance issues in Somalia, and should pro-actively seek out partners on governance now

3) The FAO strategy should be explicitly broadened across sectors, to link livelihood systems with agro-ecological zones/ marine resources and to incorporate environmental analysis

4) Whilst recognising that any resilience programme needs to be large and robust if significant impact is to be achieved, as pointed out by ODI there is no research evidence yet to demonstrate how resilience programming will deliver and be more effective than other types of programming. There is therefore a need to manage expectations, to be clear that this first phase is experimental, to point out the challenges and the required time-scale to have an impact. In other words, it is important to be realistic and not to promise too much.

5) FAO must urgently define its proposed mode of implementation for the resilience programme (eg direct implementation, through NGOs, mixed modes etc)

6) A programme of roll-out is needed to the field level to ensure that all FAO staff are fully on board with the resilience strategy and understand the implications of working in this way with other agencies (esp with UNICEF and WFP in the first instance)

7) In order to deliver the vision set out in the resilience strategy, FAO must review its human resources and existing skill sets to assess how these must change. Implementation of the resilience programme is likely to require bringing in new skill sets

8) There is an urgent need to build a deeper understanding of how Somali society and livelihoods have transformed, and how they built their own resilience over time. Maximum use should be made of FSNAU’s historical database in carrying out trend analysis. Research is also needed to look in detail at what happened during the recent famine, how people coped and survived, and how they survived in other crises

9) There also needs to be investment in research from the beginning of the programme that is ongoing alongside the programme, and openness in terms of the current gaps in knowledge.

10) FAO’s experience of CFW should be coordinated with UNICEF’s experience of UCTs as a potential safety-net, and their respective impacts need to be further understood in terms of provoking sustainable change. Both need to be predictable and implemented over a sustained timeframe to contribute to resilience. This means that governments should be involved and alignment sought with their social and agricultural policies.

11) The resilience programme must address cross-cutting issues, in particular power dynamics (including clan dynamics) and gender equality

22nd November 2012 Annex 8: Brief – Central Emergency Response Fund (CERF)

Over the period covered by this evaluation, FAO has implemented 8 projects funded through the Central Emergency Response Fund. All of these projects have targeted regions in southern and central Somalia with the exception of one (OSRO/SOM/910/CHA) that has also covered some regions in Puntland and Somaliland. The duration of these projects ranged from 3 to 10 months with an average of 5 months.

Table 1: FAO/CERF Projects under implementation during the period 2006-2012

Beneficiaries Total Actual Actual Project Symbol Project Title by type of Budget EOD NTE activity (DWH)

OSRO/SOM/613/CHA Emergency livelihoods support to flood- 2006- 2007- Input $508,200 affected riverine farmers 11-27 02-26 Distribution 20,000 HHs OSRO/SOM/807/CHA Integrated Cash and Food for Work in Support 2008- 2008- CFW 35,299 $2,000,001 of Populations in Food Security Crisis in 07-07 11-07 HHs Middle and Lower Shabeelle Regions of Southern Somalia OSRO/SOM/906/CHA Integrated Cash for Work and Productive 2009- 2009- Restocking $2,500,000 Assets Rehabilitation/transfer in Support of 04-23 12-31 600 HHs, Populations in Food Security Crisis in Mudug Procurement and Galgaduud Regions of Central Southern of Fodder Somalia 2,006 HHs, Nutrition Support 11,250 HHs, CFW 6,812 HHs OSRO/SOM/909/CHA Time-Critical Emergency Livestock 2009- 2009- $640,181 Vaccination and Treatment Project for the 09-01 11-30 Vaccinations Protection of Productive Livestock Assets of and Primary Importance to the Survival of treatments Pastoralist Populations in Crisis in Gedo and 46,929 HHs Lower Juba Regions of Southern Somalia OSRO/SOM/910/CHA Pastoralists Emergency Response (PER) 2009- 2010- Vaccinations $2,500,000 11-01 01-31 and treatments 84,445 HHs OSRO/SOM/911/CHA Integrated cash-for-work and productive 2010- 2010- CFW 3,300 $2,501,842 assets rehabilitation in support of populations 01-25 04-24 HHs in food security crisis in Gedo Region of Vaccinations Central Southern Somalia. and treatments 40,000 OSRO/SOM/102/CHA Livelihood support to IDPs and riverine 2011- 2011- CFW 23,082 $2,489,090 communities in Humanitarian Emergency and 03-01 12-31 HHs of which Acute Food and Livelihood Crisis in southern 6,000 Somalia received also inputs OSRO/SOM/111/CHA Livelihood Support for Agro-pastoral 2011- 2012- CFW 13, 957 $5,999,998 Communities in Humanitarian Emergency and 07-22 01-20 HHs, Input Acute Food and Livelihood crises in South distribution Central Somalia 54,000 HHs

Through CERF funded projects, FAO delivered livestock vaccinations and treatments, provided re-stocking and fodder, agricultural inputs, carried out CFW interventions and nutrition support to vulnerable households. Over the period, the total CERF funds amounted to USD 19.1 million representing 16% of the total FAO emergency response.

What, if any, was the added value of having these projects funded by CERF rather than from other sources?

In the period covered by the evaluation, due to the security situation, FAO as other agencies experienced difficulties in accessing southern and central Somalia regions. In years when emergency funds were more limited, CERF has been a key donor for FAO. Excluding the 2011 drought response, when the attention of the international community has been high, 8 of the 19 emergency projects implemented in the region were funded through the CERF. On average, CERF funds represented 30% of the total emergency response in southern and central Somalia with peaks of 43% and 46% in 2009 and 2010 respectively.

FAO staff perception of the key comparative advantage of CERF relates to its predictability.

To increase efficiency and coherence, FAO has recently moved to a ‘programme approach’ (one programme, one budget, one report) in Somalia. FAO staff told the evaluation team that it is difficult to include CERF in the multilateral funded programme because of CERF specific reporting requirements. FAO’s 2012 emergency response programme in Somalia did not include CERF funding.

Did CERF funds lead to a fast response? Did CERF funds help respond to time critical needs?

Over the past two decades, Somalia has experienced a humanitarian crisis with wellbeing indicators - especially in southern Somalia - substantially worse than would be tolerated in other countries (see evaluation report section 2.3). Driving forces included the escalation of conflict, consecutive seasons of poor rainfall, increasing global food prices and the poor state of irrigation infrastructure due to a neglected agricultural sector.

Agricultural Input distribution and CFW

CERF funds were used in response to the sudden onset Deyr flooding in 2006 of the Juba and Shabelle rivers. The project provided seeds and tools to flood affected farms with the aim of supporting livelihoods and encouraging HHs to remain on their land. The ex-post assessment of the intervention carried out by an independent consultant that interviewed beneficiaries and implementing partners reported that for many farmers the support came at exactly the right time.

The first FAO’s Cash for Work (CFW) interventions on a large scale were funded by CERF who provided almost USD 2 million in response to the escalation of the conflict and food prices at exceptional record levels in 2008 (food prices had increased 700% in one year). CERF funds were used to support affected populations with cash for work initiatives aimed at increasing HH’s purchasing power and at the same time rehabilitating rural infrastructures.

In 2010, CERF funded CFW activities in the Gedo Region. In a meeting held by FAO in Mandera on the border with Kenya with leaders and elders it emerged that the intervention was timely, coming just a few months after the withdrawal of relief food distribution agencies from the region that had been followed by immediate soaring of food prices due to shortfall in supply. The minutes of the meeting report participants’ view that CFW improved HH’s income at the very opportune moment. The project also delivered inputs that provided a sustainable means for own household food production (source meeting in Mandera).

In 2011, two projects supporting CFW and agricultural input distribution were funded through CERF. Project OSRO/SOM/102/CHA final report concludes that input were procured in time for the Deyr 2011 season and CFW interventions started on time providing the targeted HHs with the means to meet the minimum food basket. However, as a result of poorly and unevenly distributed rains, there was substantial reduction in cereal production. The post distribution review indicates that farmers kept most of the seeds to plant for the Gu season while others bartered in exchange of food supply or reserved for self consumption.

For project OSRO/SOM/111/CHA, fragmentation of the government structures in the project areas resulted in local authorities delaying the implementation of CFW activities and input transportation. According to the final report, FAO’s local implementing partners were decisive to negotiate with local authorities and to obtain clearance for carrying out the activities. Clearance was not given for distribution of fertilizer. In addition, due to the heavy Deyr rains in middle Juba transportation of inputs was difficult resulting in further delays and CFW interventions in some areas had to be halted because the rains made it difficult for beneficiaries to work.

Project OSRO/SOM/111/CHA was part of the emergency response to the famine in 2011. However, with respect to FAO’s overall response to the famine, the country evaluation report noted that “despite hosting the Food Security and Nutrition Analysis Unit (FSNAU), surprisingly, FAO Somalia did not seem to be that much more responsive than other international agencies in taking action in the response to the [early] warnings [...] and did not launch its emergency interventions until famine in south-central Somalia had been declared (see section 5.2 of the evaluation report)”.

Livestock

During 2009 and 2010, CERF has been the main source of funds for vaccination and veterinarian treatment interventions. Over this period, 80% of pastoral HHs benefiting from emergency veterinarian interventions was reached through CERF funds (projects OSRO/SOM/906, 909,910, 911/CHA).

Project OSRO/SOM/906/CHA carried out vaccination to protect Somali livestock from possible outbreaks of the transboundary disease PPR originating in Kenya. Rapid response to disease outbreaks through vaccination is a proven effective method of controlling the disease and reducing disease impact. OSRO/SOM/906/CHA funding allowed innovative restocking and fodder production. Rapid restocking of displaced pastoralists with significant numbers of stock, feed and veterinary services can support them to recover their livelihoods.

Through project OSRO/SOM/909/CHA, in 2008/09, FAO continued to respond to reports of PPR related deaths involving larger numbers of sheep and goats in neighbouring Kenya. CERF funds were used to prevent PPR outbreaks benefiting pastoralist keeping sheeps and goats in the Kenya border region in south Somalia. These included PPR and CCPP vaccinations, treatments and vector controls. There is considerable evidence of the significant negative livelihoods impact for PP R epizootics (Meyers et al. 2009) and rapid response to the threat of outbreaks appears to be justified1.

The general treatments provided by OSRO/SOM/909,910,911/CHA were less time critical compared to vaccinations as the diseases treated were chronic problems for livestock production in Somalia.

Did CERF funds result in other funds being mobilized?

There is no evidence in final reports or from discussions with FAO Somalia Staff that CERF funds were catalytic in attracting other funds.

Others

Beneficiaries

In total, all CERF-funded projects reached 327,723 beneficiaries households of which 82,493 were through CFW interventions, 80,000 were through agricultural input distribution, 173,980 were through animal treatments and vaccinations and 11,250 through nutrition support interventions.

Gender

According to final project reports, 4 OCHA funded projects reached their target of 30% female beneficiaries. One project reported that only 18% of beneficiaries were female - the target not being reached due to the dictate of local administration. In 2 major livestock vaccinations and treatments projects, emphasis was reportedly put on targeting vulnerable female headed households during the selection process, however FAO and its partners neglected to monitor/report the actual proportion reached.

For the only project for which an independent assessment is available (OSRO/SOM/613/CHA) “the consultant could not see any indication of any women participation at all. The exclusion of women in participation was so serious in some villages that they were denied participation even in the interviews.” Whilst in both Hiran and Middle Shabeelle region, the consultant did meet with a few women farmers, who confirmed they had been beneficiaries of the distribution, it was clear that more attention needs to be placed on actively targeting women farmers in future distributions.

1 Meyers, C., De Haan, N., Kimani, T. and Amanfu W. 2009 The Impact of Peste des Petits Ruminants (PPR) on Livelihoods in the Arid and Semi-Arid Lands of Kenya. Report funded by FAO Kenya, coordinated by FAO ECTAD.

Annex 9: Project Assessment

Drought Management and Livelihood Protection Project (OSRO/SOM/112-122/WBK)

The Office of Evaluation (OED) had planned a specific-project evaluation of the “Drought Management and Livelihood Protection Project (OSRO/SOM/112-122/WBK)” funded by the World Bank. However, when OED was requested to undertake a comprehensive independent evaluation of FAO’s work in Somalia over the past three biennia, an agreement was reached with the World Bank to assess the project as part of the broader country evaluation with a specific project annex for three main reasons:

• The nature of the project (agricultural input distribution, animal vaccinations and treatments, cash for work) was very consistent with many other projects implemented by FAO over the evaluation period;

• To develop a more coherent and strategic approach, in April 2011, FAO Somalia moved from a project approach to a sectoral-based structure. As a result of the new approach adopted, funds raised against the 2011 drought response appeal were pooled together and used as a basket of resources. The World Bank fund was also implemented through this approach.

• In March/April 2012, a Mid-Term Review of the project had been conducted by an independent consultant. The purpose of the review was to assess the overall progress of project implementation against the work plan, to identify and document lessons learnt to date and corrective actions and to make recommendations to the project stakeholders for the remaining part of the project. At the time of the Mid-Term Review, FAO Somalia had already fully implemented the CFW component, a large share of the agricultural input distribution activities1 and 40% of the livestock activities.

The independent consultant that carried out the MTR in 2012 met with the project management, reviewed relevant documents and carried out consultations/interviews with project staff in FAO Somalia Office and with representatives of some implementing partners based in Nairobi. However, due to the security situation in Somalia, he could not visit the project sites nor meet with beneficiaries.

As part of the comprehensive country evaluation, an in depth study of FAO’s Cash For Work (CFW) activities was organized. To get more information on this specific project, sites where CFW activities had been funded by the World Bank were added in the sample. The fieldwork was conducted in 7 villages of the Gedo region in southern-central Somalia from October 18 to November 15, 2012. The study gathered primary data through a range of participatory data collection methods at community level2. In three of the 7 villages included in the study, activities had been funded by the World Bank. Due to security constraints, this was the only area were the team could have direct access to local communities and beneficiaries in south- central Somalia. The full CFW study report is published as a separate annex.

1 The mid-term review covered the Deyr season 2011 representing 80% of the total number of beneficiaries. 2 See the CFW study for a description of the methodology adopted. This annex draws on the findings emerging from the country evaluation and CFW study that apply to the types of interventions funded through the World Bank and on the Mid-Term Review main conclusions and recommendations. The first section of the annex provides an overall description of the projects’ intended objectives and outcomes including information on project outputs. Section 2 summarizes main findings of the country evaluation and the MTR organized around the five evaluation criteria: relevance, effectiveness, impact, efficiency and connectedness. Section 3 provides findings on the World Bank funded activities covered by the CFW case study, in particular the rehabilitation/construction of three water catchments.

1. Short description of activities implemented by the project

The “Drought Management and Livelihood Protection Project” funded by the World Bank started in August 2011 and was initially planned as a one year project. The total budget of the project amounts to USD 9 million funded by a grant of USD 5 million from the Global Facility for Disaster Reduction and Recovery (GFDRR) Multi-Donor Trust Fund and USD 4 million from the State and Peace Building Fund (SPF). A no-cost extension was agreed with the World Bank which resulted in and extension of the end date to November 2012.

The project development objective was to provide targeted emergency support to drought affected populations in Somalia including: i) cash for work to preserve the livelihoods and ii) the distribution of agricultural inputs, livestock feed or veterinary services in order to support the recovery of agriculture and livestock productive capacity.

The project had two main components:

• Component 1: Targeted Cash for Work aiming at improving food access through markets for the most vulnerable population groups affected by the drought and also contributing to rehabilitate infrastructure/assets that improve long term drought resilience. The expected outcome of the targeted Cash for Work component was to provide work for 15,000 individuals affected by the famine that would enable them each to support one household with sufficient food.

• Component 2: Recovery of Agriculture and Livestock Productive Capacity aiming at preserving household assets and fostering the rapid recovery of the livelihoods of farmers and pastoralists through better animal health and nutrition during the drought and by providing crop production inputs for the next crop season. The expected outcome of the Recovery of Agriculture and Livestock Productive Capacity component was provision of crop or livestock support to about 35,000 households, thus facilitating a rapid recovery immediately after the drought.

The table below reports for each component of the project outcome indicators as reported by FAO in the Results Framework and outputs achieved by the project up to October 2012.

Table 1: Activities implemented through this project Actual Project Outcome Indicators Outputs Achieved Beneficiaries Component I : Targeted Cash for Work 865,000 person days of cash-for- work completed at a rate of USD 70 per month per beneficiary, Infrastructure rehabilitated: 54 disaggregated by gender with a catchments; 25 canals (53km); 9 16,168 HHs target of 30 percent women feeder roads (298 km)3 (planned beneficiaries 15,000 HHs). Component II: Recovery of Agriculture and Livestock Productive Capacity

35,000 Heads of Household, of which female headed households are represented at a higher rate than their share in the area, are:

2011 Deyr season: 290 t maize distributed to 14,500 HHs in riverine areas, 461 t sorghum seeds to 30,753 HHs in rain fed 45,253 HHs areas. Distribution of fertilizer benefitting 13,220 HHs (87.5 UREA and 268.8 DAP)4 2012 Gu season: FAO procured 80 t maize seeds and 112 t sorghum seeds. Due to lack of clearance from local authorities the distribution of 49 t of • provided with drought sorghum was called off and seeds recovery crop input packages, returned to the supplier. The disaggregated by gender of 8,166 HHs maize was distributed to 4000 head of household. HHs in riverine areas and 62.5t of sorghum seeds were distributed to 4,166 HHs in rain fed areas. Distribution of fertilizer benefitting 11,050 HHs (UREA 150 and DAP 137.5) 2012 Deyr season: according to information provided in October to the evaluation team FAO Expected 3,820 HHs intended to distribute 76 t maize seeds and 38 t of cowpeas to 3,820 riverine HHs.

3 Source: Mid-term review (2012) 4 For the Deyr season 2011, the project procured 1000 t. of urea and 342 t. of DAP. Due to lack of authorization from local authorities distribution was put on hold. During the Gu season 2012 FAO decided to distribute the fertilizer kept in store from the previous season. As per information provided in October 2012, 325 t of UREA and 63.8 t. of DAP have been received by NGO but not distributed and 162.5 t. UREA and 265.4 t. DAP are still kept in store in Mogadishu. Estimated number 2012 Support to the vaccination of PPR vaccination campaign: 6,310,328 goats and sheep against 157,777 HHs PPR (2012) (6,311,090/40)

• provided with drought Support to the vaccination of Estimated number 2012 recovery livestock packages, 49,500 animals against CCPP CCPP vaccination disaggregated by gender of (2012) campaign: 2,470 HHs. head of household. Provision of selective animal treatments to 12,911 animals Estimated number animal (internal/external parasites, treatments: 430 HHs infectious diseases and blood parasites)

2. Findings from the country evaluation and Mid-Term Review

Relevance5

Regarding the agricultural sector (component II), as noted by the evaluation report, “FAO’s emergency interventions mostly had a two-fold purpose, providing inputs for the next season and providing work opportunities to the greatest number of people in the shortest possible time through CFW”. The evaluation found that FAO Somalia had appropriately supported the “distribution of the locally multiplied maize variety (Somtux) and local sesame varieties benefiting from its earlier seeds multiplication and production work on the development side”.

Funds provided by this project have been pooled with funds from other donors to fund vaccinations and treatments in Somaliland, Puntland and southern-central Somalia. The activities funded by the World Bank were not different from the standard vaccination and treatment package that FAO implemented and the evaluation did not find anything specific to report about this project. Regarding relevance of the overall emergency livestock interventions, including the animal health interventions the evaluation found that “although the initiatives implemented were generally relevant to needs, there is a gap in knowledge about the causes of vulnerability of poorer livestock owners and little analysis over the last few years to address this gap which could lead to more nuanced interventions”.

In relation to CFW (component I), it should be noted that this type of intervention was favoured in Al Shabaab-controlled areas over unconditional cash transfers. The CFW study found that communities were on the whole extremely positive about cash-based programming. What they appreciated in particular was the capacity of CFW to grant both short and long-term benefits and create durable assets. In the three villages covered by the study where CFW was used to construct/rehabilitate water catchments beneficiaries said that the intervention arrived at the right time since people were recovering from the drought and they were expecting the rain. Regarding the CFW activity directed at rehabilitation of canals, the evaluation noted that it was an appropriate way of enhancing agricultural productivity.

5 The findings presented in this section are drawn from the evaluation main report section “Relevance of the livestock, agriculture, and fisheries programmes (4.1.4)”, mainly paragraph 61, 63 and 65. However, the case study found that “often the choice of infrastructure did not strictly follow needs in the riverine farming communities of Gedo region”.

The evaluation noted that “the CFW programme was appropriately re-focused when the humanitarian crisis intensified in 2011, from prioritising asset rehabilitation to prioritising cash distribution in order to assist drought affected populations in otherwise inaccessible areas”. However, “the CFW programme could have been more realistic about what could be achieved and better analysing the reality on the ground, as well as consciously adopting a ‘do no harm’ approach, for example, ensuring no negative impacts on the livelihoods of the poorest, ensuring no negative impact on natural resources and ensuring beneficiaries were fit for work6”.

Effectiveness and Impact

As shown in table 1, the project has fully implemented and reached many more beneficiaries compared to what was initially planned. While data on outputs are available, the evaluation found a general lack of outcome data partially explained by the security situation that reduces FAO access to the field where activities were implemented7.

For the agricultural sector, the Mid-Term Review (April 2012) observed that although harvesting had already been completed, the effect of the improved agricultural inputs distributed on production were not known. The MTR recommended conducting post-harvest data analysis to assess the impact of the inputs on crop yields (Recommendation 1). The MTR also noted that some of the field monitors may not have had adequate training to conduct accurate post-harvest analysis. In addition, the methods the monitors used were not standardized. It therefore recommended training the monitors on standardized methods of postharvest analysis to build their capacity for the job (Recommendation 5).

The country evaluation observed that FAO has already made some progress on these two points. For the Gu season 2012, FAO has outsourced to an external organization a crop performance assessment and the results are already available. In addition, FAO field monitors have been trained using the Pictorial Evaluation Tool (PET) to carry out crop performance assessment. The result of the Gu 2012 assessment where made available to the evaluation team during the mission.

Another point made by the MTR related to the agricultural sector was the fact that poorly resourced farmers are unable to buy expensive commercial chemical fertilizers for use in growing crops for subsistence purposes. In addition, they may not have access to the commodity when required. It was recommended training the farmers in preparation and use of bio-fertilizers to maintain soil fertility (Recommendation 2). The country evaluation did not address this aspect.

With respect to livestock, the evaluation suggested that monitoring the impact of FAOs interventions “could provide important guidance for the design of future emergency animal health interventions”. This has been observed also for the PPR vaccination campaign. For instance, the livestock expert in the mission observed that “it is not clear how PPR is affecting poor pastoralists in Somalia even though the participatory techniques to find this out

6 The CFW study provides a more in-depth analysis of these aspects. 7 See evaluation report section 5.1.3 “Issues re data and analysis that underpin programme design and implementation”. are readily available. PPR is considered to be endemic in Somalia and causes low level mortality in young stock. Whilst there is considerable evidence of the significant negative livelihood impact for PPR epizootics (Meyers et al. 2009), there appears to be little evidence of the disease’s impact in endemic situations”. The evaluation concludes on this point that FAO missed an opportunity to gain a better long-term understanding of the epidemiology and impact of PPR in Somalia.

For the CFW programme, although it was not possible (for security reasons) to measure the extent to which it reached the most food insecure and met their needs - since the areas where the famine was most severe were inaccessible to the team, the case study showed evidence that the CFW had a strong impact on food security of the entire communities, both directly through targeted households’ increased expenditure on food, and indirectly through redistribution by beneficiaries of the programme’s benefits to other vulnerable households and through other knock-on effects on the wider economy. The study showed that HHs participating in the programme had improved their access to food, which was the main expected outcome. They were able to buy more and better-quality food and at the same time, thanks to the rehabilitated infrastructures, they had increased access to water; however, some conflict over natural resources emerged (see section 3 below on the results of the case study on water catchments in Gedo).

Efficiency

Since the World Bank project was implemented through a sector-based approach, the observations made in the evaluation report regarding the efficiency of FAO Somalia in the response to the 2011 drought apply also to this particular intervention (see section 7 of the main country evaluation report for an analysis of efficiency of FAO procedures, administration and structures).

The move to a sector-based approach brought a series of advantages. For instance, it allowed for a more rational allocation of staff resources. Positive changes also occurred in areas such as monitoring and evaluation that as part of the move was centralized and no longer project based. Having a centralized M&E unit allowed for the introduction of several innovative monitoring tools (see section 6.6 on the evolution of the M&E function).

In the livestock sector, the World Bank project funded the PPR8 (2012) vaccination campaign, the CCPP9 vaccination campaign (2012) and provided selective animal treatments (see table 1). The project Mid-Term Review concluded that there was a need to put more effort into the vaccination campaigns as implementation had experienced some delays. However, at the time of the country evaluation, the number of animals reached by the PPR vaccination campaign to which the WB project contributed totalled more than 6 million. The evaluation team livestock expert observed that overall the vaccination intervention “was a remarkable logistical achievement […].” Interviews with partners and a review of process indicators collected by the FAO monitoring and evaluation team suggest that FAO’s 2011/12 livestock response was well-delivered”10.

8 Peste Des Petits Ruminants 9 Contagious caprine pleuro pneumonia 10 Findings regarding the livestock sector reported are drawn from the country evaluation report section 5.2 “Effectiveness in responding to humanitarian crises, with a particular focus on the famine in 2011” paragraph 115. Connectedness

As in other animal health interventions, funds of the WB project have been used by FAO Somalia in south-central Somalia to contract Livestock Professional Associations (in particular SOWELPA, CERELPA and BENELPA) to provide services and to contract the TFG MOLFR to monitor the work of the LPAs. This was recognized by the evaluation as a sustainable PPP model for the provision of animal health. In the North, however, the World Bank project funded directly the government. The evaluation saw a policy risk that as government capacity strengthens as it has in Somaliland, directly funding the public service may result in an undermining of the private sector (see country evaluation main report section 8.2).

Two recommendations of the Mid-Term Review relate to sustainability of the intervention and in particular to training. The MTR noted that some of the NGOs involved in the distribution of the agricultural inputs had limited agricultural background and therefore lacked adequate capacity to train farmers on good agricultural practices. The MTR recommended that a needs assessment of NGOs be conducted and their capacity to deliver be upgraded through training. The MTR also noted that the training provided to the beneficiaries about the improved agricultural inputs was viewed as being inadequate in terms of both duration and coverage. In addition, poorly resourced farmers were unable to buy expensive quality seed for their subsistence farming. The MTR recommended upgrading the training to cover a broader spectrum of GAP, including seed selection, and considering introducing demonstration plots and Farmer Field Schools in the training in accessible areas (MTR recommendation 3, 4).

The CFW programme has been developed entirely independently of government, a gap that needs to be filled to explore how local and national government structures can be engaged in a future phase.

Gender

For component I (CFW), the project framework reports a target of 30% women beneficiaries and for component II (agriculture) the project document specifies that female headed households should be represented at a higher rate than their share in the area. Even so, the project interim report does not provide figures on actual beneficiaries disaggregated by sex.

The CFW study showed that there has been no attention in the design of the intervention on gender issues and women have not been involved in community consultations and in some cases they have been excluded from participating. The country evaluation is generally critical of FAO Somalia’s lack of gender sensitivity: “Although proposals on paper may appear ‘engendered’, there is little evidence that this is followed through on the ground”11. It also questions the 30% target for women beneficiaries: “The ‘magic target’ of 30% of beneficiaries being women continuously reappears in FAO Somalia documentation although the rationale for this target is never given, and there is rarely guidance about how to target. This is the case in the CFW programme, which stipulates that 30% of beneficiaries should be women and youth, but with no indicative breakdown between these two groups nor rationale”.

11 See country evaluation report section 9.1 “Gender equality and gender sensitivity”. 3. CFW Study Main Findings

As mentioned above, as part of the Somalia Country Evaluation, a qualitative study on CFW was carried out covering seven villages in the Gedo region where FAO implemented CFW interventions. The study included three villages where CFW activities have been funded through the World Bank project to rehabilitate/construct three water catchments. This section summarizes findings from these three villages (please see the full CFW study report for a description of the methodology and a more in-depth analysis of the CFW intervention in the Gedo region).

Information on the villages and the water catchments rehabilitated/constructed

Shatolow: The majority of the population in Shatolow practice farming (90%) while the remaining 10% are pure pastoralists. The water catchment (WC) does not retain water for more than two weeks after the rain because the soil is sandy and water captured is conserved into jerrycans. The WC is fenced and the water is used as drinking water both by the nomadic and the settled population. Animals are excluded from accessing it and are taken to the river that is 6/7 km away.

Fardajiro: The population is pure pastoralist. In general there is scarcity of water in the area. The CFW intervention was the third rehabilitation of a preexisting water catchment. The WC is not fenced and animals can access it. As a consequence, it attracts many herds from nearby villages. It can hold water for about 1 month after the rain but due to the immense migration from other areas the water at the time of the study was almost finished.

Jilaab: The population is pure pastoralist. There is scarcity of water in the area. The semi-permanent residents expressed much concern over water shortages forcing them to travel a distance of 21 km to the river looking for water. This was a new WC constructed by the intervention. After the construction it rained only once and at the time of the case study it was empty. The WC is not fenced. As in Fardajiro, people from different places converged into Jilaab attracted by the availability of water. As a consequence, the water and also the pasture were consumed in less than a month. Due to scarce rain the water catchment never really filled up and at the time of the case study it was empty.

Findings across the three villages were consistent. In all the three beneficiaries said that the intervention arrived at the right time since people were recovering from the drought and they were expecting the rain (pre-raining season). The CFW was perceived as an important intervention that really made a difference.

Cash Component

Findings from the three villages show that the cash received by beneficiaries increased their access to food, they were able to afford more meals per day and in some cases they also could buy more and preferred food. In addition, beneficiaries could spare animals since thanks to the intervention they were not forced to sell them in the market. This is an important result considering that at the time of the intervention negative terms of trade for animals were prevailing and animals were not in good body condition as a consequence of the drought. Many beneficiaries were also able to pay back their loans and avoid incurring further debt. However, the study found that the cash provided by the programme had a different impact on wealthy and poor families, with the poorer spending more money in food, basic needs and debt repayment and the wealthier spending more in clothes and being able to spare more animals.

Overall, during the CFW beneficiaries were relying less on coping strategies such as taking loans or collecting firewood. Several beneficiaries told the team that thanks to the cash they were able to keep the milk for home consumption instead of selling it, something that had a potential positive effect on nutrition. This was observed in wealthier families participating in the programme since they are the ones owning more animals.

The increased amount of cash circulating in the village had also a multiplier effect on petty trade. In particular, in Shatolow, beneficiaries said that interventions helped the whole village since tea shops (small kiosk owned by women) and restaurants were open and money circulated, while there was no significant increase in prices. Two beneficiaries who were skilled workers said that people had more money to pay them to build or complete their houses.

In addition, some beneficiaries bought medicines for their animals in a period were animals were weak because they were recovering from the crisis. Other beneficiaries spent the money for medical assistance and human medicines.

The study also showed some negative effects brought by the intervention. From interviews with non-beneficiaries, it emerged that there was resentment for having been excluded from the programme. Cash became a source of inequality in the village. Women in Shatolow for instance talk about the “pain” of seeing other women working, gaining wages and being able to spare some animals while they could not.

In addition, being part of the programme functioned in some cases as collateral with beneficiaries able to get additional credit compared to non-beneficiaries for which access to credit became difficult. All women non beneficiaries interviewed (4) in Shatolow complained that during the CFW intervention it was not possible for them to get food on credit in the village and they had to incur additional expenses to travel to other villages for getting it. Significant is the view expressed to the team by one non beneficiary woman: “thanks God the cash is not anymore there and people are the same again”.

Infrastructures

Several beneficiaries and non-beneficiaries, for the majority women, said that they really appreciated the intervention since before they used to walk long distance to fetch the water. In addition, the water catchments (in the cases where water was available) prevented them to buy expensive water from berkads. From the field work, it emerged that building something that was going to last was much valued by the population.

Despite the positive aspects related to this intervention, issues of conflict over natural resources emerged. In the two villages where the water catchments were not fenced, the population complained that newcomers attracted by the availability of water were coming with their animals depleting their pasture and consuming the water. Women were the most vocal since animals owned by women are in general more vulnerable and they cannot take their animals too far in search of water. This is a serious negative side effect of the intervention that should be taken urgently into account for planning new interventions in a context of conflict over scare natural resources.

In Jilaad and Fardajiro, there was agreement on the necessity of fencing the water catchments. More than one person mentioned the need to establish rules and regulations for accessing it that would be applied to both resident population and newcomers. Many people agreed that if a fee was introduced it would have been for both groups; one person proposed that resources collected could be used for maintenance. In Jilaab, where the water catchment was new, one person proposed the establishment of a water catchment management committee that was already present in Fardajiro. Fencing the water catchments would also enable clean water for domestic use. However, the new-comers were worried about the idea of fencing the WC. They did not profit from the CFW, but they were still benefiting from the infrastructure and they had some concerns regarding the introduction of rules that could limit their access.

Selection of infrastructures to be rehabilitated/constructed

Although the decision to rehabilitate or built the water catchments was logical and reasonable seeing the problems in accessing water, there are questions of efficiency. One of the water catchment has never really filled up and therefore it has not been useful even if the people wanted it.

In Shatolow, one of the members of the local administration complained that if the implementing partner had consulted with the community this project would not have been implemented. The villagers said that the priority need in the village was safe drinking water though they welcomed the water catchments. On the other hand, everyone said that if they had been consulted they would have requested a piped water system with water coming from the river. However, it is not clear if this could have been done through a CFW intervention.

In Fardajiro, a young man mentioned the drilling of a borehole as a preferred intervention, but the idea was rejected by the rest of the community since they were worried that this could attract many pastoralists from other areas resulting in a threat for natural resources and a potential source of conflict. The community was welcoming instead the water catchment and during the interviews they discussed the need to fence it.

Heavy workload

In all three villages, beneficiaries said that the workload was heavy. This is in spite of workload apparently being divided by 3 during the emergency compared with usual work standards i.e. beneficiaries had to move 1/3 of the soil normally required to be moved for the same wage. In Jilaab, one beneficiary mentioned that those who worked suffered as some of them fell sick and were taken to the hospital. This is a problem that needs to be addressed. However, although the work was hard, most of the beneficiaries considered the intervention as something positive because it came at the right moment providing cash when it was really needed and in addition because the villages are now benefiting from the infrastructure.

One of the reasons why the workload was heavy is that the number of beneficiaries initially planned for the intervention was not sufficient to rehabilitate/construct a functional water catchment. In some cases, beneficiaries had to work more hours than the ones for which they were paid to complete the rehabilitation/construction. In Jilaab, for instance, beneficiaries mentioned that people selected for participating in the intervention should have included those who were strong enough to meet the work requirements, but some were instead physically small and weak. For this reason, they preferred doing the work in groups and not to follow the work norm given by the implementing partner.

Targeting

From wealth ranking of the beneficiaries and non-beneficiaries according to criteria agreed for each village during the FGD (focus group discussion), it is clear that there have been serious problems with targeting. Among beneficiaries several were not in need and in some cases wealthy while among non-beneficiaries several poor were present especially among women. In all the three villages, women were very vocal about women’s exclusion. For example, although in Shatolow the elders and local administration said that the targeting was done in an open way with almost all community members consulted, the women beneficiaries selected were all not in need while the non-beneficiary women were all poor. The non- beneficiary women complained of not having been consulted.

In Fardajiro, the chief informed the community that 100 beneficiaries were needed and those who were nearby and came first were registered. Those who came after were told that it was too late and that the required number had already been reached. Also in this village women said that they had not been considered because of their gender and there were no men who could complain on their behalf. They also mentioned that often organizations implementing interventions do not send to the village women with whom they can talk. They protested that at least the village women association could have been consulted.

Also in Jilaab, women non beneficiaries were all poor. Again, they complained that women should be consulted and that targeting should be done according to poverty and vulnerable criteria.

Long term resilience building

In general, the study showed that the impact of cash was of a short term nature while the sustainable changes were the ones linked to the infrastructure rehabilitated.

From discussion with beneficiaries, emerged a potential for other initiatives for the construction of different types of infrastructures such as for example schools that might involve other agencies.

Sharing

In all villages, cash partially benefited also non-beneficiaries since in some cases they received money or other goods from beneficiaries participating in the programme through sharing mechanisms that are part of the Somali culture. However, this is not necessarily an equitable redistribution of resources, and may leave out some marginalized groups or individuals. Therefore local sharing mechanisms cannot compensate for mistargeting. In addition, in all the villages, some people were allocated to work for poor and vulnerable people in the community that could not work. In the future, the element of sharing present in the Somalia society should be taken into account during the design of new CFW initiatives.

FAO Somalia is building on the experience of the CFW study in Gedo and it is currently conducting further in-depth studies on the impact of CFW in other parts of southern-central Somalia.

Overall, the WB funded project activities appeared to be relevant to needs and to have yielded positive results. There is a lot to be learned about how interventions could be adapted to the specific needs of specific groups. This will require FAO Somalia to invest greater time and effort to ensure greater dialogue with communities and improved monitoring the outcomes of its work.

Food and Agriculture Organization of the United Nations Office of Evaluation

“Like a good trip to town without selling

your animals”: A study of FAO Somalia’s

Cash for Work programme

Impact Evaluation report

June 2013

Food and Agriculture Organization of the United Nations

Office of Evaluation (OED)

This report is available in electronic format at: http://www.fao.org/evaluation

The views expressed in this information product are those of the author(s) and do not necessarily reflect the views or policies of FAO.

All requests for translation and adaptation rights, and for resale and other commercial use rights should be made via www.fao.org/contact-us/licence-request or addressed to [email protected].

For further information on this report, please contact:

Director, OED Viale delle Terme di Caracalla 1, 00153 Rome, Italy Email: [email protected]

Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Gedo Region, Somalia, October/November 2012

iii Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Table of Contents

1 Introduction: background and rationale ...... 1 2 Methodology ...... 5 2.1 Geographic and village sampling ...... 7 2.2 Field procedures ...... 10 3 Constraints...... 11 4 Findings ...... 14 4.1 Programming ...... 14 4.1.1 Targeting ...... 14 4.1.2 Unconditional Cash Transfers and coordination with agencies ...... 18 4.2 Village profiles ...... 19 4.3 Community assessment of the programme ...... 23 5 Impacts ...... 24 5.1 Cash use and impact ...... 25 5.1.1 Food security ...... 25 5.1.2 Nutrition...... 26 5.1.3 Other basic household needs ...... 27 5.1.4 Loans, loan repayment and savings ...... 27 5.1.5 Productive investment of cash ...... 28 5.1.6 Multiplier effects on village economy ...... 28 5.1.7 Impact of Infrastructures ...... 29 5.1.8 Other preferred interventions ...... 35 5.1.9 Workload and work measurement ...... 36 5.2 Social impact ...... 38 5.2.1 Expenditure in social obligations ...... 38 5.2.2 Other social impacts ...... 39 5.3 Impact on gender ...... 41 5.4 Unintended negative impacts ...... 44 5.5 Sustainability of change...... 46 6 Lessons learned and ways forwards ...... Error! Bookmark not defined.

Tables

Table 1: Outcomes and indicators for assessed projects ...... 2 Table 2: Unplanned actions introduced during the research process ...... 12 Table 3: Female non-beneficiaries’ remarks on exclusion ...... 14 Table 4: Wealth ranking and inclusion in CFW by wealth...... 15 Table 5: Team’s observations on the UCTs ...... 18 Table 6: Fieldwork sites ...... 21 Table 7: Sources of household income, expenditure items and coping strategies (IEC) ..... 23 Table 8: Beneficiaries’ comments on CFW ...... 24 Table 9: Households’ productive investments by village* ...... 29 Table 10: Water catchment villages ...... 31 Table 11: Ranking of preferred interventions in Fardajiro (F) and Jillab (J) ...... 33 Table 12: Sharing ...... 38 Table 13: The cost of exclusion ...... 40 Table 14: Perception of social impacts ...... 41 Table 15: Women’s empowerment ...... 43

iv Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Table 16: Women’s voices from Fardajiro ...... 44 Table 17: Findings from the telephone survey ...... 45

Pictures

Picture 1: Mapping in Fardajiro ...... 7 Picture 2: Mapping in Gawido ...... 10 Picture 3: Wealth Ranking in Jillab ...... 17 Picture 4: Male IECs in Hammare ...... 27 Picture 5: Canal in Degreeb...... 30 Picture 6: Water Catchment in Shatolow ...... 35 Picture 7: Water Catchment in Fardajiro ...... 37 Picture 8: Female IECs in Fardajiro ...... 42 Picture 9: Water Catchment in Jillaab ...... 48

v Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Executive Summary

This report is the result of a study of FAO Somalia’s Cash for Work (CFW) interventions implemented after the 2011 crisis. It was commissioned as part of the evaluation of FAO’s cooperation with Somalia, and mainly based on fieldwork conducted in 7 villages of the Gedo region, South Central Somalia in October and November 2012. The main underlying assumption to be tested was that by disbursing cash to targeted poor and vulnerable individuals and rehabilitating communal productive assets, households and communities would experience short and long term positive gains. The study’s scope was to understand the positive and negative, intended and unintended impacts of cash inflow and infrastructures, as reported by community members. Fieldwork was conducted by a team of four local researchers and a team leader who possessed basic knowledge of household survey data collection techniques and a native understanding of social and economic dynamics. The methodology was based on iterative training and learning, immersion and accumulation of information, adaptability and flexibility of the approach, and triangulation. Four participatory tools were used to guide group and individual interviews with community members.

An analysis of the outcome of community wealth ranking exercises revealed exclusion and inclusion errors in targeting. Different livelihood zones presented different problems, with particular challenges tied to targeting of pastoralist communities. Generally, local criteria of selection of beneficiary households, sometimes tied to local concepts of justice, prevailed over the projects’ targeting rationale. Overall communities expressed a positive assessment of cash-based interventions for the freedom of choice they offer, and of CFW for the short and long term gains it guarantees.

Indicators of success were an increase in agricultural production due to the rehabilitated infrastructures and minimisation of the impact of coping strategies on livelihoods. More and more preferred food was consumed, sometimes beyond the duration of the programme, and negative coping strategies affecting household nutrition were curbed. To a degree the wages were invested productively, though the impact was not sustained or widespread. Multiplier effects on the village economy were common, though short-lived. Infrastructures were appreciated for their longer term benefits, though choice of asset and technical specifications were often questioned, and infrastructures were found to have variable and not always positive effects on different groups. Benefits deriving from CFW were widely shared, though it is unclear to what extent this sustained the most vulnerable through hard times. Women raised many complaints regarding their marginal role in the programme. Some unintended negative impacts were documented, including the adoption of new unsustainable food consumption patterns leading to household debts, resentment and inequality, exclusion or exploitation of Bantu individuals, depletion of pastures by incoming herders attracted by the rehabilitated water catchment,

Among the most significant lessons emerging from this research are the need to: better coordinate with other agencies and acknowledge informal community redistribution mechanisms and local concepts of justice in planning; undertake a more careful assessment to understand the implications of choosing to rehabilitate one type of infrastructure over another; ensure the inclusion of those who are fit for work but marginalised; implement programmes over a longer timeframe, and employ more workers; deliver CFW as part of a package of interventions, targeted according to categories and needs. Finally, impact assessments should be routinely carried out as part of a broader M&E and learning system.

vi Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Acknowledgements

The research on which this report is based, as well as the report itself, are the outcome of the efforts of many. The main author, Savina Tessitore, would like to thank them here. Lori Bell, officer at FAO’s office of evaluation (OED), first had the idea of undertaking a study of the cash for work component of the Somalia country programme as part of the country evaluation, and contributed to the initial development of the concept and methodology. Aurelie Larmoyer, who managed the evaluation, supported the process to its final outcome. The whole evaluation team offered ideas and encouragement and the country evaluation team leader, Margie Buchanan-Smith generously contributed concrete inputs and insights as well as in-depth comments to the final draft, as did Genny Bonomi, OED Evaluation Analyst. Genny was a supportive colleague throughout, and responsible for scoping FAO Somalia’s documentation on Cash for Work, producing a manageable synthesis.

Special thanks are due to all the OED staff in Rome, particularly Nadine Monnichon and Heather Young who provided backstopping during the fieldwork in Somalia and beyond. The FAO Somalia office in Nairobi gave full access to information, was open to discussion and invested much time and efforts to assist the research process and broaden my understanding of how this assistance modality evolved. Particular thanks go to Giuseppe Simeon, Julie MacDowall, Dan Opiyo, Maximilla Omengo and Paul Githumbi of the Cash for Work Unit for the great amount of time, efforts and patience they contributed to the project. Precious practical assistance was provided by Joy Nasyimu, Joy Mulema, Elijah, Ken Gonesi and Paul Thiga. José Lopez, Sergio Innocente, Virginia Vigilar, Thomas Gabrielli, Baru Adan, Palmira Ciacciarelli, Francesco Baldo and Rudi Van Aaken all offered contextual information that shed light on different aspects. As Officer in Charge, Luca Alinovi’s belief in transparent disclosure, and the value of research and learning had a clear influence on the whole country programme and made him an early and convinced supporter of this research.

Outside of FAO, the concept on which this research was based, as well as the analysis, benefited from encounters and inputs from Nigel Nicholson, Mike Brewin, Kerren Hedlund, Nisar Majid and Dan Maxwell.

In Dollow, the District Commissioner Abdirashid Hassan Abdinoor Mohamed welcomed me, as did Ahmed Arai and Abdi Ali Botan respectively Executive Director and Deputy Programme Manager of the implementing NGO, NAPAD. The NAPAD Livelihoods Programme officers for the region, Ahmed Abdirahman Farah and Mohamed Ismail Ali moved to Dollow throughout my stay. Their knowledgeable inputs were an invaluable contribution in the research process. Abdifatah Osman Hussein, FAO Field Liason Officer facilitated the research in many ways, not least by assisting me in the training, providing background information on the area, and taking me for walks around Dollow. By responding to the team’s inputs, communities and community members in the seven villages literally made this research possible. Its first aim was to channel their voices to reach policy-makers and improve effectiveness in enhancing people’s livelihoods and long-term prospects.

Lastly, the report that follows is based on the work of Mohamed Jama Aden, Mire Abdullahi, Hoosh Banadir Sheikh, Omar Mohammed and Sowdo Mude Billow, who carried out the fieldwork in Gedo, and helped me analyse the information they collected. Siham Rayale patiently provided essential verification of the data collected by interviewing many

vii Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme beneficiaries by phone. This research could not have been undertaken without their intelligent and good-humoured commitment, which went well beyond their contractual obligations. They are all to be considered co-authors of this report.

Any errors are of course to be considered the responsibility of the lead author alone.

viii Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

1 Introduction: background and rationale

1. Cash for Work (CFW) constitutes a large and rising proportion of FAO Somalia’s country portfolio. During the 2011 Horn of Africa crisis, FAO scaled up the CFW component of its projects in response to increased need of assistance1, once market assessments had confirmed that cash-based responses were a viable response option. The rationale for interventions with a work/asset building component was to set the ground for medium term agricultural recovery, while providing much needed immediate relief to food insecure households and stimulating local markets through the injection of cash2.

2. This report is the result of a study of FAO Somalia’s cash for work interventions implemented after the 2011 crisis, mainly based on fieldwork conducted in 7 villages of the Gedo region, South Central Somalia from October 18 to November 15, 2012. The study was commissioned as part of the evaluation of FAO’s cooperation with Somalia, to gain an understanding of the effects of CFW assistance on beneficiary households and communities. The overall Somalia country evaluation was undertaken over the second semester of 2012 and assessed the totality of FAO’s assistance to the country. It aimed at: 1) identifying success factors and challenges for learning purposes for the FAO Somalia programme and FAO at the corporate level; 2) providing accountability to donors and to the respective government authorities; and 3) maximizing the strategic relevance and effectiveness of FAO’s engagement in the country.

3. The choice to undertake an in-depth study of the cash for work component of FAO’s work was initially determined by: a. significant and increasing financial resources allocated by FAO Somalia to CFW interventions (including a large ongoing USAID funded project)3; b. the need to assess what is a relatively new but expanding area of FAO’s work; c. an explicit call for an independent evaluation of this component of FAO’s work raised in the OSRO/SOM/711/SWE final project report ; d. the need to answer questions regarding the appropriateness of cash-based interventions in a complex and volatile context such as Somalia, in view of better articulating risks, benefits and operational benchmarks. e. the opportunity it provided to collect information on the ground, considering that the Somalia Country Evaluation Team was otherwise severely limited by the difficult situation in the country, tight UN security rules, as well as the short timeframe envisaged for the main evaluation mission4

4. The main assumption underlying FAO Somalia’s CFW projects to be tested by the study was that by simultaneously disbursing cash to targeted poor and vulnerable individuals

1 Following the famine declaration the CFW budget for October 2011-March 2012, grew to 22 million dollars, from the 3.6 million dollars of the previous months of 2011. 2 For a more detailed analysis of cash-based assistance modalities in Somalia, and of FAO’s approach to CFW see the concept note prepared for this study, annex I. 3 The OSRO/SOM/124/USA project focuses exclusively on cash for work and targeted over 62,000 households during the famine response contributing USD 11,500,000 to the CAP. For the November 2012/March 2013 phase its estimated budget is over USD 26,000,000. 4 The Evaluation Team visited Somalia over four weeks, from 27 October to 23 November, while the time devoted to the study was significantly longer, from 8 September to 24 November, with four weeks (18 October- 15 November) dedicated exclusively to fieldwork.

1 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme and rehabilitating communal productive assets, households and communities would experience short and long term positive gains. If proven correct, this would align with FAO’s twin track approach to development, aimed at ensuring immediate food security while building long term sustainable livelihoods.

5. Other CFW objectives mentioned in FAO documents5 are:  empowering households  reaching the most vulnerable  injecting cash in the local economy  providing people with work opportunities  minimizing stripping of household assets  reducing and eliminating migration and preserving communities and families.

Table 1: Outcomes and indicators for assessed projects Projects Dates of Expected Indicators implementation results/outcomes related to CFW OSRO/SOM/122/WBK 28/10/2011 to Ensure immediate income 865,000 person days of cash- 20/1/2012 to the most vulnerable for-work programme completed households through CFW at a rate of USD 70/month per activities, rehabilitate beneficiary disaggregated by productive gender with a target of 30 infrastructures/assets percent women participating.

OSRO/SOM/124/USA 7/1 to 10/3 2012 Improved access to the 63,090 HH have access to the minimum food basket for minimum food basket for at at least three months least three months. through CFW; improved The selected productive resilience in the targeted infrastructures/assets are household that will be rehabilitated and functioning by benefiting from the the end of the project period, rehabilitated and directly contribute to infrastructure/s. restored production. Source: CFW brief based on final reports and project documents – FAO Somalia Country Evaluation

6. Overall, the CFW programme’s underlying logic and theory of change are not well framed in project documents. No evidence was found of a debate within FAO Somalia regarding the adoption of CFW as a modality of assistance , its articulate implications, or its relative advantages and disadvantages vis-à-vis other forms of cash-based interventions, in particular Unconditional Cash Transfers.

7. A logical framework was not developed as part of the projects’ design. Generally the projects do not seem to have been based on an analysis of causal chains. No documents were found providing clear links between assessment of needs, implementation modalities, and expected outputs/outcomes/impact. Furthermore, it is not clear how the attainment of some of the ambitious outcomes envisaged in project documents, such as enhancing household and

5 Documents referring to these objectives include implementation guidelines of cash for work projects in Somalia (annex to the project document of OSRO/SOM/124/USA), the Emergency Division’s February 2012 concept note “cash-for-work in Somalia: linking relief to development”, and a review of final reports and project documents of FAO Somalia projects with a cash for work component. The terminology here is verbatim.

2 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme community resilience, or protecting the most vulnerable from resorting to harmful coping strategies (e.g. asset stripping, forced migration and sexual exploitation)6 was to be measured.

8. These inherent weaknesses can be attributed to a number of factors: a. The ad hoc and non-systematic evolution of this modality of intervention within FAO and the country programme, mainly driven by circumstances, partially by opportunistic considerations. b. the rapid expansion during the 2011 crisis, reaching a much larger scale than was originally envisaged, and this, in an extremely short timeframe 7 c. the turnover of CFW management at the peak of the famine response, and the failure to hand over responsibility from one programme coordinator to the next. d. the absence of a solid M&E system, that would enable learning.

9. Post-implementation M&E has mainly been centred on risk management and process verification, understandably a priority, seeing the inherent difficulties in monitoring and enforcing Implementing Partners’ (IP) compliance with agreed terms in the complex Somali environment. Furthermore, the need for compliance monitoring was heightened by serious episodes of diversion uncovered during the 2011 emergency. An unfortunate side- effect of this focus was the failure to accumulate knowledge about outcomes and impacts. This gap in planning was addressed once the peak of the crisis abated, to the point that intense learning and monitoring are now incorporated into the upcoming phase.

10. The scope of the study were the intended and unintended, positive and negative self- reported impacts of the cash inflow and the rehabilitated productive infrastructures on communities and beneficiary and non beneficiary households. Besides a review of secondary data, the main source of information was fieldwork carried out in seven villages of 3 accessible districts of the Gedo region, in South Central Somalia. Information on implementation and opinions and preferences expressed by community members were collected using a set of well-known participatory tools tailored to the situation and the research questions.

11. The main aim of the study was learning: by uncovering social and economic dynamics triggered by the intervention at household and community level, the study attempted to develop indications and lessons to be incorporated into programming of the next phases, and into FAO operations, policy and strategy at corporate level. A second objective was to provide useful pointers for future and in-depth research in other areas of Somalia. Finally, though strictly qualitative data do not allow generalizations, a further aim was to provide donors, partners and other agencies working on Cash-Based Programming with direct information from inhabitants of Somali villages where projects are remotely managed, and whose point of view and voice are seldom reported.

12. Underlying questions guiding the study were:

 How were infrastructures and sites selected? Who used the infrastructures and who benefited? What were the short and long term, positive and negative impacts of rehabilitated infrastructures at household and community level?

6 Expected outcomes of OSRO/SOM/807/CHA and OSRO/SOM/124/USA respectively 7 The budget rose from USD 2.2 million in 2010, to USD 22.1 from October 2011 to March 2012, the months during which the famine was declared.

3 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

 How were beneficiaries selected? How was the selection process perceived by the community? How did the aim of targeting vulnerable households meet the capacity and needs of the most vulnerable, with little or no labour force?  How was the cash used? Did the cash injection improve immediate household food security? Did the cash injection have any multiplier effects? Did it have unintended positive or negative effects at household and community level? Did it have different effects according to recipients’ social and economic profile?  What were the programme’s negative or positive social impacts, if any?  Did the programme have any long-term impact and if so, what were its drivers?  Were any measures adopted to ensure that the programme followed “do no harm” principles? And if not, what were the consequences?  Was the programme gender-sensitive? Did it include 30 to 40% female beneficiaries as it set out to do, according to project documents? What were its impacts on women and gender relations, and to what extent were women involved in the different phases of programming? What were women’s perceptions of the programme and its impacts, both positive and negative?

13. The overall context of the interventions and of the study only allowed an analysis of beneficiaries’ perception of impacts. The broad focus of the study questions on “how did it work?” and “for whom did it work best?”8 rather than “did it work?” and “to what extent?”, provided strong grounds for the choice of a qualitative approach to data gathering. Qualitative and participatory methodologies, along with the rigor and skill-set they demand, including undertaking contextual analysis and close and repeated interrogation of sources, proved to be particularly appropriate under the circumstances.

14. The use of participatory tools allowed for more in-depth contextual analysis and for direct involvement of beneficiaries in understanding processes and analyzing the multiple outcomes of the CFW. In an extremely opaque context such as Somalia, the importance of capturing underlying dynamics which drive the capacity of interventions to have a positive impact cannot be overstated.

15. The study attempted, among other things, to understand the processes leading to the success or failure of the stated CFW objectives, and the underlying causes of either. It examined whether these objectives were realistic, if they were compatible with each other, and if not, what were the underlying reasons of such incompatibility. Efforts were made to capture how objectives and outcomes were viewed and prioritized by individuals and how they affected different groups of individuals within the selected communities.

16. Critically, rather than measuring the CFW programme’s achievements against ideal conditions and abstract “best practices”, the study attempted to account for what could concretely be done in the very constrained circumstances of Somalia. This approach was deemed particularly appropriate, considering the need to answer the humanitarian imperative during the famine crisis, when the CFW’s main aim shifted and the food security component of the programme gained prominence over productivity-enhancement.

8 Sterne, Stame and Mayne (2012)

4 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

2 Methodology

17. Fieldwork in selected villages was conducted by a team of four local researchers, and one team leader. The team members had undergone the FSNAU enumerator training and possessed basic knowledge of household survey data collection techniques in the Somali context. As local researchers, they possessed a native understanding of social and economic dynamics which, under continuous supervision and intense training, generated a steep learning curve and the honing of their skills and analytical capacity.

18. The methodology was tailored to the setting, and explicitly sought to circumvent some of the difficulties outlined in the preceding section, in order to reinforce the robustness of data and analysis. As identified by Sterne et al. (2012), difficulties in evaluating impact may arise from either the complexity of an intervention’s aim, or from the setting in which it is embedded, Both applied to the case under examination, which presents many of the attributes defining an intervention as complex9 with a number of methodological implications, foremost among which is “the design of an iterative or staged evaluation strategy that can respond to ‘emergence’” (Sterne 2012: 60).

19. Retrospectively the methodology may be said to have been guided by the following:

 Training and Learning: After the UNDSS SSAFE (Safe and Secure Approaches to Field Environment) training in Dolo Ado, Ethiopia, the team underwent four days training to prepare for the fieldwork, including familiarization with the tools 10. The principal objective was to acquaint the team with the CFW programme, the study aims and the data collection toolkit. The training was also geared to instill a sense of ownership of the entire research process. This was crucial, seeing that the international coordinator had no access to villages and could not directly take part in the fieldwork. Building capacity to act autonomously involved a more in-depth training than would usually be the case for similar assignments, and included background information on the programme and elements of theory of cash-based programming. With the same intent, the team members were strongly encouraged to participate actively, by asking questions and contributing ideas during the training. As part of the learning process, field simulations and testing of tools were included in the training sessions11.  Immersion, accumulation and reiteration Data were gathered at a small scale and over an extended timeframe, within the limits of the FAO Somalia evaluation schedule. Research was based on an iterative process and “snowballing”, progressively building on knowledge and information gathered. Findings were immediately fed back into the open-ended and evolving approach. Each village was visited over three days, with the second day dedicated to debriefing in Dollow with the coordinator. Regular and extensive debriefing provided an opportunity to

9 Relevant attributes identified include: Overlap with other interventions with similar aims; multiple and diverse activities and projects; likely impacts of the programme were long-term; working in areas of high risk or uncertainty; programmes working indirectly’ through ‘agents’ and often at different levels and stages (Sterne 2012: 58). 10 The training was led by the team coordinator with the assistance of the FAO Somalia liaison officer for field monitors. See annex VIII for an outline of the training module 11 The testing took place in Dollow at the end of the training module, with 3 women and 3 men beneficiaries from a neighbouring village where CFW had been implemented

5 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

examine unexpected issues, identifying problems, gaps and interesting questions specific to each village which had emerged during the first day of fieldwork, to be further investigated on the third day.  Flexibility and adaptation: Flexibility and adaptability were critical to the overall approach, seeing that the challenges encountered were seldom predictable. See Table 2 for a list of unplanned actions introduced following the emergence of unexpected events and findings. The necessity to adapt was driven by the complex context, but also by the underlying conviction that a pragmatic approach, aiming at what was feasible and tailoring the research to challenges as they emerged, would yield more robust results than following widely accepted “best practices”12.  The choice of qualitative methods allowed the necessary flexibility to adapt the methodology and reframe questions as circumstances changed and the research progressed. The possibility of comparing qualitative data across space and time was an additional advantage of the overall approach.  Field researchers were also encouraged to stray, if necessary, from predetermined question lists, to probe interlocutors to understand underlying social dynamics, as well as to be imaginative in the face of unexpected problems. Using their own best judgment at all times was a tenet of the research process. Based on the principle of flexibility, it relied on the team’s capacity, developed during training, to manage research tools and understand context, as well as on the trust relationship established within the team and with the team coordinator.  Triangulation: information collected directly from informants was supported through and triangulated with several other sources: 1) the team made photographic and audio recordings of their visits; 2) beneficiaries in Gedo and other regions (Bay, Lower and Middle Shebelle, Middle Juba and Hiiran) were contacted by phone and a survey was administered summarizing the main research questions; 3) both before and during the fieldwork, staff from NAPAD (Nomadic Assistance for Peace and Development), the implementing NGO, were available to answer questions arising from analysis of the project documents and from the fieldwork; 4) contact with researchers conducting parallel evaluations of cash-based interventions in Somalia resulted in a fruitful exchange of opinions and findings.

12 For example triangulation with information collected through primary quantitative data collection, e.g. larger- scale surveys, could ultimately prove an unsuitable approach under the circumstances

6 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Picture 1: Mapping in Fardajiro

2.1 Geographic and village sampling

20. The Dollow, Luuq and Belet Hawa districts in the Northern part of Gedo were selected as the most easily accessible in South Central Somalia, in consultation with FAO security officers and FSNAU experts.

21. Villages were selected in close coordination with FAO’s liaison officer for field monitors and with NAPAD, FAO’s Implementing Partner (IP). Within limits, sites were selected to isolate variables of interest and include the widest range of relevant characteristics within the study sample, e.g. livelihoods base, type of infrastructure, timing of the intervention, and geographical location. On the other hand, it proved impossible to compare projects implemented by different NGOs, since, as mentioned, one of the two IPs in the selected districts was under investigation at the time. These limitations reduced the sites which could be visited to two projects: OSRO/SOM/122/WBK and OSRO/SOM/124/USA.

22. It is well tested and documented that by involving different sectors of a population in the analysis, participatory methodologies may elicit a broader spectrum of opinions and deepen the understanding of development processes, potentially capturing the voices of the most vulnerable and marginalized. By recognizing different voices, the use of participatory

7 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme tools may uncover implicit trade-offs, and reveal who stands to gain and loose from a given intervention13.

23. Beyond data collection, these tools were used to trigger discussions among community members, bringing issues and concerns to the forefront as well as possible solutions. Consensus was a possible outcome of the conversations, but more often they allowed different points of view to emerge. For instance, in one of the villages, Degreeb, alternative storylines emerged regarding the inclusion of women and Bantus, while in Fardajiro respondents disagreed on the computation of workload and wages. In both cases the divergence was an indication of underlying design and implementation failures, and the first step towards their analysis and potential solution.

24. Data collection was based on four principles, discussed during team training and debriefing sessions: optimal ignorance, underscoring the need to stay focused and strike a balance between collecting as much information as possible in relevant areas of interest, while remaining ignorant in less significant ones; appropriate imprecision, accepting a certain degree of imprecision and avoiding wasting time for unimportant details; learning with and from the people, recognising that communities possess a wealth of knowledge and are the primary source of information, and the best placed to talk about their own needs and priorities; triangulation seeking different sources of information and understanding issues emerging from various perspectives, including different team members’14.

25. Four participatory tools were used during the fieldwork15:

1. Social and resource mapping: During the first focus groups community leaders and other community members were asked to draw a map of the village. The aim here was to collect information on community characteristics and the spatial distribution of relevant landmarks, with a focus on people’s perceptions of village space and resources, and their implications on relationship between groups within the community. This exercise proved particularly relevant in riverine agricultural villages, as it highlighted social cleavages, more marked than in pastoralist and mixed villages,

13 ”Participation” is not uncontroversial, and much has been written regarding its potential and actual capacity to enhance development interventions. In evaluating the effectiveness of a participatory approach, it is useful to make a distinction between instrumental, cognitive and transformative aims (Giles 2001). An early and well-known proponent of participation was Robert Chambers (1983), in the eighties, when participatory approaches were starting to gain traction. Almost two decades later a critical review of the participatory approach was edited by Cook and Kothari (2001) under the title Participation: the new tyranny? Hickey and Moyan (2004) later attempted to rescue the concept by broadening its scope in the collective volume Participation, from Tyranny to Transformation? Exploring New Approaches to Participation in Development. Joseph Stiglitz, in his 2002 paper “Participation and Development”, has been a more recent supporter of participation as the foundation of long-term, sustainable development A useful review of the use of participatory tools and approaches in M&E is the Governance and Social Development Resource Centre (GSDRC) online topic guide: http://www.gsdrc.org/go/topic-guides/measuring- results/participatory-tools-and-approaches 14 The source for this approach was the methodological note for the Impact Evaluation of FAO interventions carried out under the Common Humanitarian Fund (CHF) in Sudan, annex 5: Guiding principles for FDG and SSI (FAO 2012a). See also here, annex III CFW study - indications for fieldwork 15 Important sources for tools and guidance notes used during fieldwork were FAO (2012b) and Oxford Policy Management (2012). See also annexes to the present report:IV Participatory tools, V CFW PRA tools and VI, fieldwork guide.

8 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

and the benefits brought by the rehabilitated infrastructures, which were more uneven in agricultural villages than in pastoralist ones. The maps also provided an important visual reference during the debriefing sessions with the team coordinator. 2. Well-being and targeting analysis: The aims of this exercise were to a) solicit estimates of village wealth distribution, b) gain a picture of CFW beneficiaries disaggregated by self-reported wealth groups, and c), analyse poverty characteristics, their perception and how they related to inclusion/exclusion in the programme. A discussion was elicited around: 1) the proportion of village inhabitants belonging to different well-being categories as previously defined by respondents, 2) the proportion of CFW beneficiaries from each wealth category, and, 3) the proportion, among non-beneficiaries, of households with and without labour capacity. 100 counters were distributed among a group of respondents16, who, following a discussion, were asked to enter the counters in a matrix to reflect these proportions. 3. Income Expenditure and Coping (IEC) matrix: two groups of four male and four female beneficiaries in each village were asked to illustrate the changes in their income, expenditure and coping patterns, before, during and after the programme. The aims of this exercise were 1) to analyse the composition, characteristics, and income and expenditure patterns and coping strategies of individual households; 2) to verify whether such patterns and coping strategies had been modified by the CFW project, and if so in what way, and 3) whether changes caused by the CFW were sustained over time, analysing any emerging drivers of sustainable change. By registering the adoption of coping strategies across time, the matrix attempted to capture whether the programme had affected household vulnerability as well as poverty levels: since coping strategies are a response to risk, they were considered a more accurate predictor of vulnerability than income alone. 4. In each village between 3 and 8 male and female non beneficiaries were consulted through semi-structured interviews (SSI) guided by a question list. Their livelihoods and their level of vulnerability were noted. Questions centred on their perception of the programme (targeting, selection of infrastructures, reasons for exclusion, positive and negative impacts at household and community level)17.

26. Interventions involving water catchments presented specific problems for communities, analysed in greater detail in section 4.4.2. To gain a better understanding of people’s opinions regarding these issues, a simple ranking exercise was proposed in two of the three water catchment sites. After a discussion with a wide group of respondents, including community members and leaders, a list of preferred relief interventions was drawn up. Counters were equally distributed to participants, who were asked to place them on the listed items according to their personal preferences. This scoring exercise then triggered a discussion on potential winners and losers of each intervention.

16 Respondents were by and large the same as those participating in the previous exercise 17 See also annex VII non-beneficiaries question prompts

9 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Picture 2: Mapping in Gawido

2.2 Field procedures

27. Sequencing: The team first encountered Key Informants (KI) and community members, previously alerted of the visit by the IP18. After the social and resource mapping the team split. While two members visited the infrastructures, accompanied by one or more community members, the rest of the team facilitating the well-being and targeting analysis with the remaining group of villagers. Information and observations gathered in the course of the first two encounters helped the team to select participants for the following exercises. Authorities were sometimes asked to assist the team in the selection of respondents for the following exercises.

28. 2 to 3 team members were present during each exercise, one of them acting as the main facilitator, asking questions and discussing issues with informants, while another one took notes. Someone was in charge of the camera and of audio recording, taking pictures of the rehabilitated infrastructures and of salient moments of the encounters with communities.

29. Selection of respondents: though the details of selection procedures varied for each village, in general community members who had been present during the first Focus Group Discussion (FGD) were considered for further interviews, or asked to introduce the team to

18 see annex VI, fieldwork guide, for details on field procedures

10 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme other community members (snowball sampling). Male and female beneficiaries and non beneficiaries were mainly selected according to the following criteria:

Male respondents

30. Different selection criteria were considered simultaneously, based on observation and information gathered during the first session: 1. people who were silenced by other community members or gatekeepers, or ostracized in other ways; people who were quiet and might have had something to say in private; people who gave alternative versions of authorities’/gatekeepers’ narratives. under certain circumstances gatekeepers were also considered 2. people of different ages, as they have different experiences and opinions 3. people with specific professions and skills; in particular religious people and skilled and semi-skilled individuals, if present 4. people selected according to their appearance when this provided a clue to their status. 5. disabled people and people with special needs.

Female respondents

31. It was harder to collect information regarding women’s status and other characteristics during public gatherings, when women were generally silent, so the above criteria could not be applied. After the first two group exercises, women and men were separated, and female beneficiaries were requested to list their household (HH) assets (animals, farm ownership, occupation, HH size). The selection of four was then made to include the broadest possible range of self-declared wealth brackets and livelihoods. Female non-beneficiaries were selected among those identified as the poorest according to the same self-declared criteria.

3 Constraints

32. Insecurity and volatility in Somalia, a consequence of prolonged absence of a unified government and conflict, compounded by increasingly adverse climatic conditions, make data collection extremely challenging, further heightened when it comes to undertaking rigorous quantitative research.

33. Some of the challenges faced by this research process were19:

a. The extended period of statelessness in the country, and the resulting absence of centralized Statistics Offices, meant that little to no baseline information was available. Furthermore, benchmarking for ex-post comparison had not been a systematic requirement for NGOs implementing CFW; b. The short recall period for cash expenditure restricted the sites to those where interventions had been implemented over the last year; c. In the past Al Shabaab has often ostracized and attacked data collectors, and it is generally very suspicious of information gathering; in areas controlled by Al Shabaab and adjoining regions, enumerators are highly exposed to risks. d. The security situation had a double constraining effect: it imposed limitations on one side (on village and beneficiary sampling, team recruitment, scheduling and

19 See also annex II, “table of constraints to good practices in data collection.”

11 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

planning), and demanded determined measures to be followed on the other (e.g. SSAFE training, MORSS compliant guesthouse, travel requirements such as exclusive use of private vehicles, and, in the case of the team coordinator, accompanied by armed guards). e. To minimize security risks, the international consultant coordinating the research was unable to travel within the area, and could not directly supervise the team during fieldwork. f. A plethora of different agencies have been operating in the area over a long period: isolating a single intervention and attributing effects can prove to be complex; g. When FAO adopted CFW as the main modality to respond to the 2011 crisis, interventions shifted their focus from asset building to provision of cash to cover the minimum expenditure basket. The blurring of the line between long term livelihoods support and immediate food security objectives caused by this shift added a layer of complexity to the measurement of the programme’s outcomes and impact. h. Though its extent could not been fully ascertained, signals of diversion of some CFW funds emerged soon after the programme started scaling up in October 2011. This resulted in one of FAO’s main IPs in the Gedo region being investigated at the time of the study. Villages where this NGO had been implementing were excluded to avoid confusion between audit and evaluation functions, and to allow the investigation to proceed unhindered.

Table 2: Unplanned actions introduced during the research process  Interview with woman who was initially registered and later excluded (Hammare)  only 4 female non beneficiaries from each of the two settlements in the village were interviewed, and no female beneficiaries, as none were present (Degreeb)  3 male and 3 female newcomer households who had not been present during implementation were interviewed (Fardajiro)  Community ranking of preferred interventions (Fardajiro and Jillab)  Retargeting of research sites according to priorities uncovered during fieldwork  Rescheduling of visits according to new criteria emerging over the course of the reasearch  Team discussion when majority of team expressed concerns about security in pastoral areas20

34. Study findings don’t claim to be applicable to the whole of Somalia, and no final conclusions are drawn regarding the extent to which CFW as an assistance modality had specific positive or negative, intended or unintended impacts on a given number of individuals, households and communities. Also, due to the impossibility of ensuring rigorous quantitative data collection, a mix of quantitative and qualitative methods was not feasible.

35. Lastly, it should be added that no fieldwork could be carried out in the most impenetrable areas of South Central Somalia, the same which were hardest hit by the famine. Here access to food was most critical, and this is where research into the effectiveness of CFW as a famine relief measure would have been most appropriate, and indeed, poignant. This question, unfortunately not included in the present study, is still waiting to be addressed and answered by future research projects.

20 The team’s security was a priority, taking precedence over considerations linked to data collection, however, this also clearly had repercussions on sampling.

12 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

13 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

4 Findings

36. The first part of this section highlights findings on programming choices related to targeting (4.1.1), UCTs (4.1.2.1) and coordination with other agencies (4.1.2.2). The second and third part provide information on the characteristics of the selected villages (4.2) and on the overall assessment of the programme by the community (4.3). The fourth part (4.4) summarizes the main findings on impact, aggregating data from all field sites. As mentioned in the section on methodology, the intent is to sketch a broad picture of the changes occasioned by the CFW, provide lessons on risks and opportunities, indicate implications for future programming, and directions for future research.

4.1 Programming

4.1.1 Targeting

37. An analysis of the outcome of community wealth ranking exercises set against self- reported social characteristics of beneficiaries and non-beneficiaries clearly revealed a considerable extent of exclusion and inclusion errors in targeting. To varying degrees, inclusion of wealthier villagers in the program was found in all villages. One village, Jillab, was an extreme case of elite capture, with none of the 8 interviewed beneficiaries in the lower income bracket. A slightly different case was Degreeb, where all the workers were poor Bantu non-beneficiaries, forced to split 50% of their earnings with non-Bantus who were officially registered. At the same time, in all villages a number of cases emerged of poor female non-beneficiaries who had been excluded from participating. These women complained that they would have been fit for work, or that a male relative could have worked in their stead. Table 3 and 13 report women’s remarks regarding their exclusion.

Table 3: Female non-beneficiaries’ remarks on exclusion

Overall, women who were excluded from participating seem lack social capital. According to the women themselves, their exclusion was the consequence of their complete absence in decision-making fora, where selection criteria were established by community leaders. The following are some of the women’s remarks regarding their own exclusion: -“I was not related to powerful people in the village” -“Only elders know which criteria are used, we only heard that so and so is registered” -“Excluding women from the programme is men’s way to deny women’s rights” -“We were told it was men’s work, though later I saw some women working too” -“Women and the women’s association were not consulted or involved in targeting” -“I was shy and I was not able to complain with the men about having been excluded”

38. During the wealth ranking exercise, the population had been ranked in three income groups: “poor”, “ok” and “wealthy”. In terms of how villages represented participation in the CFW by income groups, three different trends emerged:

14 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

 Two villages, Unsi and Fardajiro, reported that CFW participants were proportionally divided by wealth, with around half the population in each income bracket having been registered as beneficiaries. These relevant inclusion and exclusion errors were consistent with beneficiaries’ and non-beneficiaries’ self- reported wealth status.  Three villages, Hammare, Degreeb and Shatolow, claimed to have excluded any non-poor households from the programme, though in Hammare and Shatolow this was manifestly in contrast with beneficiaries’ self reported wealth status. In other words, in these villages the rhetoric of targeting the poor was contradicted by the reality emerging through interviews with beneficiaries, some of whom turned out to be well-off, by village standards. Degreeb is a different case in that all those who worked were officially non-beneficiaries, forced to share their income with the owners of the land they were working on.  One village, Jillab, straightforwardly reported big exclusion and inclusion errors. The wealth ranking exercise revealed that 70% of the people considered “ok” had been included in the programme, against 32% of the poor. This was consistent with beneficiaries’ and non-beneficiaries’ self-reported status.

39. Inclusion and exclusion errors are primarily a reflection of elite capture in villages where the selection of beneficiaries was reportedly at the discretion of elders and other influential people. They also seem to embody an ethos of sharing regardless of social status, which probably indicates the political sensitivity of resource distribuition, including aid assistance, within the community. This was well captured by the words of a non-beneficiary woman in Unsi, in whose opinion “everyone should be included on an equal basis, because everything should be shared equally by everyone, and the poor should not be privileged”.

Table 4: Wealth ranking and inclusion in CFW by wealth

FGD Community wealth ranking Ben Non Ben* Poor OK Wealthy female male female male % % % % % % pop CFW pop CFW pop CFW Unsi 2 poor 1 Ok 5 poor ? 64 31 25 10 11 5 2 Ok 3 wealthy Hammare 4 poor 1 poor 1 poor ? 85 48 10 - 5 - 2 ok 1 1 wealthy wealthy Gaawido 4 poor 1 poor 3 poor 1 poor 62 52 26 22 2 - 2 Ok 1 wealthy Degreeb - 4 poor 4 poor* ? 91 45 5 - 4 - 3 OK 1 wealthy Shatolow 4 Ok 4 poor 4 poor 1 78 38 20 - 2 - wealthy Fardajiro 3 Ok 4 Ok 2 poor ? 68 22 22 10 10 4 1 Wealthy 1 ok Jillab 4 Ok 2 Ok 4 poor 1 87 28 10 7 3 - 2 wealthy wealthy * The wealth status of non-beneficiaries was not systematically collected **8 non beneficiary women were interviewed in Degreeb, 4 in each of the two hamlets

15 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

40. Settled farming communities, mixed agro-pastoralist villages, or prevailingly mobile pastoralist ones presented different targeting problems. Targeting mobile pastoralists poses a particular challenge as households abandon settlements during the dry season or in times of crisis. In the two pastoralist villages this was the situation in late 2011, at the time of implementation. Households were scattered when registration began and information reached them through word of mouth, so participants were selected on a “first come, first serve” basis, rather than through any other planned criteria.

41. Targeting the poorest within pastoralist communities presents a further problem, as the ordinary cycle of growth and collapse of herds compounded by other external factors means that even wealthier livestock owners are caught in what has been called “the paradox of wealth plus vulnerability” (Devereux 2006), facing moments of extreme vulnerability,. It can therefore be particularly difficult to assess the relative vulnerability of herders, and establish which households are most in need of assistance, especially during difficult times.

42. Ultimately, local criteria for the selection of beneficiary households seem to have prevailed over the project’s, not always clear or clearly executed, targeting rationale. Likewise, benefits were redistributed through informal mechanisms, following accepted social norms.

43. Some ways in which the programme’s targeting objectives were thus readjusted by communities were the following:

 As with any other resource, benefits deriving from the programme were inserted in the longer timeframes of ordinary social relations. Very often excluded households did not complain about this, as they believed that village leaders would include them in the next round of external assistance. In pastoral communities in particular, a widely accepted policy of “first come first serve” was adopted.  Assets are distributed following clan and sub-clan influence and/or numerical prevalence. Though clan considerations are paramount when it comes to resource distribution and use, the mechanisms behind them are mostly implicit, not transparent and difficult to assess. However, the research team remarked that completely disregarding clans when targeting may put the implementing NGO at risk, and trigger conflict within the community . Incidentally, this is a further reason to hire local staff to carry out fieldwork as they will be more knowledgeable of clan distribution and able to negotiate their way around it.  Quite often owners of viable farms impose their own farm’s sharecroppers working on their farms or other people from their own social networks within and outside the community as beneficiaries, sometimes bypassing community leaders’ own decisions. Some respondents denounced this as a further way for wealthier farmers to capture the programme’s benefits, and gave rise to many complaints.  A particular case was Degreeb, a village with a Bantu majority. Here, according to the Bantus themselves, all the workers were Bantus, though they were not formally registered, but required to share 50% of their wages with the “official” non Bantu beneficiaries, and sometimes a further 25% with the owners of the land they were working on. Remarkably, in this village all 65 farms owned by Bantus were excluded from benefiting from the infrastructures that were built, in favour of 9 non- Bantu landowners.  Other sharing mechanisms, listed in Table 12

16 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

44. A rooted sense of justice and the commonly felt obligation to reach out to the most needy emerged again and again during conversations with community members, often in relation to Islamic principles of charity and equity. A number of beneficiaries, for example, considered themselves to have been rightfully excluded in favour of more needy households. This discourse clashed with another trend, best captured by the saying “he who has breakfast also gets lunch”, meaning that wealthy people always receive assistance, since their status ensures that they will be able to repay any favour they are granted. The tension between these two principles has a strong influence over the allocation of resources and informal transfers. If, on the one hand, this may secure a degree of outreach to vulnerable households, especially in the presence of principled community leaders, it can also result in discrimination against certain groups. In particular minority clans, Bantus and women, who for different reasons generally possess little social capital, are usually excluded in favour of owners of viable farms and people with strong community ties.

Picture 3: Wealth Ranking in Jillab

17 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

4.1.2 Unconditional Cash Transfers (UCT) and coordination with agencies

UCT component of the CFW

45. When FAO scaled-up its CFW interventions in 2011 in response to the crisis, it was decided to distribute the equivalent of 2 weeks wages to participants before starting the work21, a provision which continued to be in place some time after the crisis was declared over22.

46. Ostensibly this measure was taken “to provide immediate access to food and raise people’s nutritional status before starting to work” according to a concept note on FAO Somalia and CFW (FAO 2012)23. The assumption that in the context of the Somali food crisis this amount of cash would grant access to nutritious food and raise nutritional status though, was untested. Furthermore, the study findings revealed some lack of clarity in planning the UCTs and in establishing their aims.

47. In the first place, it should be noted that no indication is provided in project documents of what the UCTs were destined for, nor are they ever mentioned in Letters of Agreement (LoA) with NGOs. According to NAPAD officers, they told participants to buy what was necessary to prepare them for 10 weeks work, be it food, water or tools. They claimed this was in line with indications received from FAO, but it was not possible to find confirmation or disprove this statement 24. In riverine areas many respondents claimed they were told to buy tools. A majority also said they didn’t comply, but used their own tools and bought food with the cash instead. In one pastoralist village the money was pooled to buy spades and what was left over redistributed to households. In this village old wheelbarrows and plastic sacks were used to dig the water catchments, as the UCTs were not enough to buy new ones.

Table 5: Team’s observations on the UCTs  Respondents had various names for the UCT component: "preface", "welcoming gesture", "introduction", "preparation", "unconditional cash"  Most people said they were told to buy tools with the $36, some people bought tools and also spent the cash for basic needs, others only spent towards fulfilling basic needs, without buying any tools.  Nobody used the cash exclusively to buy tools.  In all three pastoralist villages wheelbarrows were needed to move the earth, but the UCTs were not enough to buy them, so old wheelbarrows and plastic sacks were used instead.  Elderly people and those unable to work declared their preference for UCTs.

21 The workload was also reduced to one third (and not by one third, as erroneously stated in the concept note on CFW in Somalia (FAO 2012). For example canal work was reduced from 1.5 square metres/person/day to 0.5 square metres/person/day. 22 The 2 weeks UCT were last used for the cycle of CFW activities starting in December 2012 / January 2013, ending April / May 2013 23 The same aim was mentioned during a scoping meeting at FAO headquarters in Rome with senior Emergency officers . 24 The information gap on this issue is a clear example of how the high turnover in CFW management reflected on the programme, and indicates the absence of documentation regarding strategic choices.

18 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

There was no coordination between UCTs and CFW at agency level, though the two types of interventions could potentially be regulated at community level through elders' initiative

Coordination with other agencies

48. In the agricultural and agropastoralist villages visited, other agencies distributed UCTs in response to the famine25. Much overlap was found by this study between receipt of unconditional transfers and registration in the FAO CFW. This finding is consistent with the quantitative data collected in Gedo for the Cash and Vouchers Monitoring Group (CVMG) evaluation26 and, though the consequences and impact of this overlap were not investigated, it points towards a high degree of mistargeting and elite capture of both kinds of interventions, as well as a clear lack of coordination between them. More recently, however, in recognition of the need for better coordination, an attempt was made to couple FAO’s CFW with other agencies’ UCT interventions, by using the same IP.

Key learning 1: 27 At least nominally, discussions indicate widespread acceptance among respondents of targeting more vulnerable households with no labour capacity as recipients of UCTs. However, it is unclear whether the rationale for agencies’ choice of either form of assistance is coherently explained to communities. In one of the villages a certain degree of resentment was expressed regarding distribution of UCTs, and FAO should be careful, especially in non-crisis periods, about giving UCTs to households with working capacity, and should, in any case, be able to explain the rationale for doing so.

49. The two pastoral villages claimed they don’t usually receive any other forms of assistance. In the words of a man from Fardajiro: "we are always in this forest and hear the roaring of lorries carrying food relief to districts and riverine villages." Perhaps as a consequence of this exclusion, they were also the only communities who explicitly requested food aid.

4.2 Village profiles

50. Four of the seven villages visited were riverine agricultural communities, one is based on agro pastoralist and pastoralist livelihoods, and two rely exclusively on pastoralism. The main livelihood strategies in all these villages follow a geographical gradient, with greater reliance on livestock the further South from Dollow the village is located. In riverine communities the CFW projects rehabilitated or built canals, cleared farmland, and built roads.

25 Horn Relief was referred to, but there may others may have been present and not mentioned. 26 Analysis of the CVMG data indicates the following: “With regards to cash for work (CFW), the overlap was (…) significant particularly in Gedo where 10-25% of beneficiaries receiving cash and vouchers also participated in CFW as early as December 2011, Lower Juba (35%) particularly where Somali NGOs were simultaneously contracted by INGOs and/or by the FAO, and Mogadishu (30% of Concern beneficiaries and nearly 100% of DRC beneficiaries)”. 27 Key learning points referring to single issues and complementing the final section on overall lessons learned are to be found interspersed in the text.

19 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

In two of the pastoralist communities water catchments were rehabilitated and in the third a new water catchment was built.

20 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Table 6: Fieldwork sites28 Village Project, dates of Background information Infrastructure Dates of field implementation n of beneficiaries/n of HH main livelihoods visit

Luuq district Xammare OSRO/SOM/124/USA beneficiaries: 48/ 417 HH Permanent settlement, river 5 (300m) canals fully 27 and 29 Riverine 7/1 to 10/3 7/1 to 10/3 2012 fed farming along the Juba, few cattle and goats; rehabilitated, 1 (600m) October 2012 417 HH partially rehabilitated

Deegreeb OSRO/SOM/124/USA 7/1 to ben: 40/254 HH Permanent settlement, river fed 5 canals rehabilitated 3 and 4 November Riverine’ 10/3 2012 farming along the Juba, 4 hamlets 2 Somali and 2 Bantu 2012 communities. Bantus practice fishing Shatolow OSRO/SOM/122/WBK ben: 80/340 HHs (140 permanent) 90% 1 Water catchment 6 and 7 November Mainly agropastoralists, 28/10/2011 to 20/1/2012 agriculturalists (rainfed,ag) 10% pastoralists. 200 rehabilitated 2012 some pastoralists nomadic pastoralist HH seasonally resident in surrounding area Jillab OSRO/SOM/122/WBK ben: 35/178 HH Non permanent settlement, vacated 1 Water catchment (newly 10 and 12 Pastoralist 28/10/2011 to 20/1/2012 during the dry season, purely pastoralist built) November 2012

Belet Hawa district Uunsi OSRO/SOM/124/USA 7/1 to ben: 46/172 HH Permanent settlement, 3 2 Canals rehabilitated 23 and 24 Riverine 10/3 2012 hamlets divided along clans lines, Bantu day laborers bush clearing 1 road built October 2012 from outside the village; river fed farming along the Dawa with some livestock rearing. Water point on the riverbank is a source of water for livestock in the surrounding area Gaawiido OSRO/SOM/124/USA 7/1 to ben: 74/ 376 HH 3 Canals rehabilitated, 1 30 October and 1 Riverine 10/3 2012 Permanent settlement, river fed farming along the Juba, road built November 2012 Somali and Bantu communities Fardajirow OSRO/SOM/122/WB 28/10/2011 ben: 100/132 HH 1 Water catchment 8 and 11 Pastoralist to 20/1/2012 The village is rainfed, not permanent. Pastoralists move rehabilitated November 2012 during the drought season and come back during the rainy season. Some HHs remain all year round.

28 Social and resource village maps produced with the communities are included in annex XI.

21 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

51. Although land ownership in riverine villages is widespread, this fails to translate in equally widespread smallholder farming, as many of the poorer owners have what they call “bushy” farms, i.e. not cleared from weeds and bush, and lack pumps to draw water from the river and irrigate their fields, making their landholdings unviable even for subsistence farming. Frequently the solution for these poor villagers is to enter a burjwasi29 agreement with wealthier farmers. Burjwasi may take either one of two forms: labourers are paid half of what they harvest on the farms they work on, or alternatively, wealthier farm owners lease their water pumps to poorer ones in exchange for half their crops. The main social cleavage in riverine villages therefore runs between owners of viable farms and sharecroppers, regardless of land ownership. Members and leaders of farmer cooperatives tend to be owners of productive farms, rather than the totality of farm owners, or all villagers who practice farming.

Key learning 2: Cooperatives are usually an important counterpart for FAO’s IPs in deciding what kind of asset to build/rehabilitate, and on whose farms to do so. The membership of farmers’ cooperatives, therefore, has relevant consequences on programming Particularly if the aim is to target women and poor households, it is highly desirable that FAO Somalia and its IPs broaden the range of stakeholders they consult with, at the planning stage.

52. A further important social cleavage in villages follows clan lines, with majority clans often controlling productive land. The case of Bantu communities, excluded from ownership of productive land, and often in an exploitative relationship with Somali farm owners as labourers and sharecroppers, is an extreme case in point.

53. In villages prevailingly or exclusively relying on livestock, personal wealth is ranked according to animal ownership. The main indicator of status are animal heads, and particularly camels, which are owned exclusively by men. In the southernmost villages of Luuq and Belet Hawa districts, livelihood patterns are reflected in various degrees of residential mobility: farmers are permanent inhabitants of settlements, while herders move in search of pasture and water, splitting herds and families, if necessary. Camels are able to remain up to 30 days without water and often travel long distances in search of pasture, while goats can survive 5 days without water, and cows 3 days only. Characteristics of animal endurance therefore constrain the mobility of goat-owners, many of them female, and means they are more reliant on village natural resources. In sum, animal ownership and the mobility patterns it engenders have a direct relevance to timing and targeting of any form of assistance.

54. Table 7 provides summary of all the income and expenditure items mentioned by households in the seven villages visited, and of the coping strategies households adopt in times of distress.

29 burjwasi roughly translates as sharecropping

22 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Table 7: Sources of household income, expenditure items and coping strategies30 (IEC) Income Expenditure Coping strategies  sale of agricultural produce  Food and other basic HH  distress sale of livestock  sale of livestock needs  distress sale of fodder  farm labour  clothes and shoes  collection and sale of sisal  petty trade  medical expenses  firewood collection  sale of fodder  animal treatment  cutting food intake and  sale of milk  debt repayment meals  collection and sale of sisal  livestock  relief assistance  remittances  donkey cart and cart parts  loans  collection and sale of  farm implements  social support/zakat myrax (tree bark)  savings and informal  distress migration to IDP  sale of ropes and mats revolving loans camps  construction and repair of  social assistance/zakat  intensification of huts  school expenses agricultural activity  midwifery  madrasa fees  agricultural labour  transport  collection and sale of  fuel and batteries timber  water  eating wild foods  goods for trade (e.g. shop  collecting money at items, rope for mats) checkpoint

4.3 Community assessment of the programme

55. In broad terms communities were on the whole extremely positive about cash-based programming for the freedom of choice it offers. What respondents appreciated in particular about CFW is its capacity to grant both short and long-term benefits, and create durable assets. In this sense, households engaged in agricultural activities and with an active workforce unknowingly concurred with FAO’s rationale for CFW programming in Somalia based on the “twin track approach”.

56. Opinions on the way the programme was implemented were more varied, and in every community complaints were raised regarding choice of infrastructures, targeting of beneficiaries and community consultation. A word of caution was expressed by the study team regarding judgments arising from discussions with groups of village stakeholders, since these assessments inevitably also depended on other factors exogenous to the programme, and were often linked to to internal village politics31.

30 The relative importance of these items varies with the prevailing livelihood in the village, and level of household poverty/vulnerability 31 For example in Degreeeb the team noted the following: “Care should be taken before drawing a conclusion regarding the selection process of beneficiaries: Deegreeb is a village where we have jileec (those with soft hair, Somalis) and jareer (those with kinky hair, Bantus) and at the same time we have sub-clans of jileec. The team observed that clans as well as sub-clans are not on good terms with each other when it comes to selection of beneficiaries, hence one sub-clan may say selection was good and the other sub-clan may say it was poor selection. This means that the selection of Deegreeb beneficiaries was difficult to assess.” However, from what the team could observe, it did not seem that these and other underlying social dynamics were understood or taken into account at the planning stage.

23 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

57. The IP was never directly accused of willful mismanagement of implementation and payment modalities, including workload and timing of the intervention. On the other hand many complaints were raised about targeting procedures, and exclusion of important sectors of the population from consultation and information regarding the programme. As reviewed in section 4.1.1, this study confirms targeting to be a weak area of implementation.

Table 8: Beneficiaries’ comments on CFW

 “cash for work is ‘shaharsi fiican’: a good trip to the town without selling many of your animal."  “It is like you help yourself and you are paid for that.”  the cash received was “kordhis/turaanturo”, incremental, meaning it increased people’s power to purchase basic good and services.  "Whoever prefers food? We prefer cash because we can buy anything we need. With the cash for work I have the cash and something to rely on in the future as well. The food, once I have used it, it’s finished and nothing is left. If you have the sugar, but you can’t have the tea leaves and the milk, what good is the sugar to you? How can sugar be important to you if you don’t have anything to add to it? If you have some cash you are somehow rich because, at that time, you can buy everything”.  “Cash for work is multipurpose: people get money to buy whatever they want and also have durable assets.”  “It is much trouble, with little benefits”

5 Impacts

58. Perhaps the most straightforward aggregated indicators of the CFW achieving a degree of success in its aims are its capacity to boost agricultural production in villages where farming is practiced, and its ability to minimize or reverse the adoption of coping strategies erosive of households’ long-term livelihoods and well-being, including distress sale of livestock for herders.

59. In all the agricultural villages the general trend is a net increase in agricultural productivity32. In each of these villages this meant that at least some and often all interviewed beneficiaries produced more and up to twice their usual agricultural yield, and attributed this to the irrigation brought by the canals built through FAO’s intervention the previous year, which helped optimize water use. Though attribution of impact to a single intervention in the agricultural sector should be made with caution, there was general consensus among respondents that increased production was to be attributed to the infrastructures built or rehabilitated through the CFW. The roads built through CFW eased transportation of agricultural produce and water.

60. It is worth observing that in two of the four villages this positive outcome of the canals was set back for the first season after the intervention, by a crop disease (kudur in Somali) and floods respectively. In two out of four villages, therefore, the rehabilitated infrastructures were not sufficient in themselves to guarantee good agricultural output, an

32 In Unsi productivity increased for 2 seasons; in Hammare some respondents experienced two good agricultural seasons, while for others the first crop was destroyed by kudur disease; in Gaawido the first harvest was flooded, while all male respondents are now expecting a good harvest due to the canal; in Degreeb some improvement in harvested crops was reported.

24 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme indication of the vulnerability of these farming systems, and of the need for a more integrated approach.

61. Almost all interviewed beneficiaries remarked that during implementation they were able to avoid adopting negative coping strategies (see table 8 for coping strategies adopted during the crisis), temporarily protecting their assets from depletion. On the other hand, although the most extreme measures adopted at the height of the crisis were averted, with the programme’s end households almost invariably reverted to their habitual coping strategies, another indication of their vulnerability. Two women were rare cases where careful investments in petty trade meant that the impact of the cash was still felt a year after the intervention: while one of them opened a small shop that was still running, the other started using rope rather than natural fibres for her mat-making, translating into increased in value of her mats, which generated greater income.

5.1 Cash use and impact

5.1.1 Food security

62. The most widespread item bought with the cash by the great majority of beneficiary households was food, resulting both in more and more preferred food being consumed, and, in some cases, an increase in the number of daily meals. Many respondents appreciated the timeliness of the cash, allowing them to boost food consumption at a time when their households were highly food insecure.

63. High food prices were sometimes mentioned as a further contextual factor undermining households’ food security, and that the CFW income helped to offset. In one pastoralist village a respondent said: “the cash we received working on the CFW came at the right time as the cash helped people who were recovering from the 2011 drought”. In some cases, food consumption was unvaried, but the CFW earnings were used to buy food, mostly in order to save money, avoid loans or distress sale of animals33. Invariably, the few cases where the cash was not used to buy food were better off households, and instances of mistargeting.

64. In some cases the positive impact on food consumption lasted beyond the duration of the programme, either as a consequence of improved agricultural production due to the rehabilitated canal, or because food was still being bought with savings from CFW earnings. However, as with other effects of the cash, by and large the situation was reversed once the programme ended, and in a few cases food security even worsened compared with the period before the intervention. Except under specific circumstances, explained below and related to changes in the quality of food consumption, this has no relation to FAO’s projects. Rather, it signals the precarious agricultural basis of these rural households, which may very rapidly fall back to a state of destitution once assistance ends.

33 Some better-off recipients did not mention savings. In this case the claim that “the same amount and quality of food was bought with the money earned” might indicate a reluctance to admit that these households were not food insecure at all.

25 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

5.1.2 Nutrition

65. Another frequently documented impact was to curb negative coping strategies affecting household nutrition, including cutting meals, eating less, and eating less nutritious food34. For example, before the intervention a household in Jillab was eating maize mixed with water, and had reduced their meals to one35. Eating wild food was another coping strategy adopted during the crisis, often even when its negative impacts on health were acknowledged36.

66. Particularly in the three pastoralist villages hardest hit by the 2011 crisis, where the CFW represented a direct relief measure, the cash influx reversed the famine survival strategy of cutting meals, adopted during the crisis peak. Though some households bought the kind of food they preferred, there is no proof that this automatically resulted in better nutrient intake.

67. The cash went primarily towards buying starches (pasta in the first place, then rice, porridge, wheat flour)37 and sometimes meat. There are indications that switching to more preferred, but more expensive, foodstuff occasionally induced households to later take loans or cut quantity of food intake to keep up the new consumption habits acquired while the programme was running.

Key learning 3: The impact of greater cash inflow on food consumption habits can be significant, and needs to be better understood seeing its potential to have both positive and negative consequences.

68. On the other hand, wealthier households with enough livestock sell milk during times of distress, so another relevant and positive effect of the cash was to sometimes reverse this trend, destining the unsold milk for family consumption instead. The importance of availability and access to milk for children’s nutrition and well-being is well documented, including protection against wasting and stunting (Sadler et al 2009). Though generalizations should be made with caution, it seems that sparing milk as a consequence of the CFW income could have had an important nutritional impact, at least on some households and especially in the aftermath of the food crisis.

69. Overall data is too scarce to draw any firm conclusion about the way these trends affect nutrition, well-being and livelihoods, but what emerges clearly is that the nutritional impact of the cash for work at household level is an area requiring more in-depth scrutiny in the future, as also noted in Sadler et al 2009 (also reference Levine Bailey and Hedlund?).

34 These are well known short term strategies adopted by poor rural households in Africa to protect assets in times of crisis Devereux (1999) 35 Sadler and Catley (2009) on mixing grain and water as coping during seasonal shortage in Somali region 36 For example during the CFW, two male beneficiaries in Degreeb declared they stopped eating miraha, which are the fruits of a wild tree (garas),. In difficult times Bantus in Degreeb collect a herb called aran guryo, known to be very laxative, mixing it with another herb, rasow, to reduce its bitter/sour taste. 37 For pastoralist societies whose diets have been characterised as “protein-rich calorie-poor” (Sadler et al 2009: 16), the main challenge is adequate calorie intake rather than quality proteins, so in general the choice of increasing consumption of carbohydrates may be entirely rational in these contexts. On the other hand there is no proof that in the period under examination the interviewed households protein intake was in fact sufficient.

26 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

5.1.3 Other basic household needs

70. Expenses falling under bilkaqoyska, including food, medicines, clothes, shoes, paying debts, were not always distinguished by respondents, especially men who were less accurate in responding on expenditure towards household items. This is not surprising, since women hold a strong control over household budgets for daily expenses.

71. When these expenses were detailed, it emerged that the second most widespread use of the CFW wages was to purchase clothes and sandals for the whole family. Unsurprisingly, it seems that the wealthier the family, the greater the expenditure on clothes, otherwise usually bought during the Muslim festivity of Eid. Third in line were medical expenses, with three times more women spending on family healthcare than men. Other expenditure items included batteries, soap, agricultural tools, transport, school and madrasa fees, and animal health. Respondents of both sexes often stated that the effect of household expenditure on food and clothes was to “make the children happy”.

Picture 4: Male IECs in Hammare

5.1.4 Loans, loan repayment and savings

72. Credit has a crucial role in the village economy, to the extent that quite often households take and repay loans simultaneously. Families revert to credit for ordinary shopping, during a crisis and when facing extraordinary financial needs, and to buy goods for trade. It is not surprising, then, and consistent with findings of other concurrent Cash Transfer

27 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

(CT) evaluations in the region38, that almost all respondents, whatever their social ranking, used the CFW earnings to repay loans and/or to avoid buying on credit. In a few instances, the wages were insufficient to cover all household needs, in which cases loans were still taken while the programme was active, while a couple of beneficiaries claimed they spent the entire sum they earned for food, and were unable to repay any debts.

73. Registered beneficiaries also used their participation in the programme as collateral to buy on credit while the programme was running. The other side of the coin here was that non-beneficiaries, perceived to be less creditworthy, were excluded from accessing basic household items on credit. Some non-beneficiaries who reported having to travel to nearby towns to shop on credit, complained bitterly about this (see Figure 13).

Key learning 4: Since in CFW programming some non-beneficiary households may be the labour-poor and therefore most vulnerable ones, specific measures should be in place to ensure that the intervention does not inadvertently harm them. In this case, for example, the lack of credit could be offset by better coordination with agencies implementing UCTs, or by ensuring that labour- less households are “worked for” by relatives or other community members.

74. A few of the women were able to save some cash, and this resulted in a more protracted overall impact of the wages on basic household wellbeing. Significantly, though, one of the few households where this happened was a case of mistargeting, with both husband and wife registered, and therefore double wages. In general beneficiaries remarked that once debts had been repaid, very little was left to be saved. At least in one case the cash was set aside as a buffer against harder times ahead, with a temporal extension of the basic safety net function of cash-based assistance programmes.

5.1.5 Productive investment of cash

75. Wages were invested in small enterprises, particularly by women who are used to handling small amounts of cash and putting them to productive use, while men usually require larger sums for the same end. This is reflected in the wish expressed by one male beneficiary for the wages to be paid in single installments, as a larger sum could have more easily been invested in animals.

76. With the cash, women boosted or set up small restaurants or petty trade, bought rope for mat making, acquired livestock and, in one case, bought a tent for a son to teach the Koran in. Though some of these investments lasted well beyond the end of the programme, and were still relevant to household economy at the time of the study, it was generally acknowledged that the intervention was too short for the cash to have a sustainable and widespread impact.

5.1.6 Multiplier effects on village economy

77. Though multiplier effects on the wider village economy were short-lived, they were felt by many while the programme was ongoing. Both beneficiaries and non-beneficiaries in

38 See Longley, Dunn and Brewin (2012)

28 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme many of the sites remarked that the village economy was booming at the time due to the cash influx. This effect was mostly felt by women engaged in petty trade, and particularly by owners of small restaurants, as workers could afford to spend their earnings in their venues after work. Some respondents also reported a surge in sale of milk. Skilled workers also profited: for example in one village two builders were extremely busy during the programme, and attributed an increased demand to build or repair houses to the cash flow.

Table 9: Households’ productive investments by village* Livestock Family Petty School/ Ag Livestock Other healthcare Trade Madras tools medicines a fees m f m f m f m f m f m f m f Unsi 1 1 1 1 1 1 Hammare 4 1** 1 1 Gawido 1 2 1 2 Degreeb*** - 1 - - - - - Shatolow 2 4 Fardajiro 1 2 1 Jillab 2 1 3 Total 1 3 4 12 1 3 1 1 2 5 3 All villages 4 16 4 2 2 5 3 * Though a breakdown is presented of men’s (m) and women’s (f) answers, in most cases the productive investment is likely to benefit the whole household. ** in Hammare one of the interviewed male beneficiaries worked for his mother who was the official recipient. She was the one who used the income to set up a small shop. *** No female beneficiaries were interviewed in Degreeb

5.1.7 Impact of Infrastructures

78. All the assets built through the FAO interventions were appreciated for the longer term benefits they promised, and their potential to directly ameliorate people’s livelihoods. On the other hand, the type of asset chosen was questioned by some people in every village, as were the technical specifications adopted.

79. Besides the impact on agricultural production, another common consequence of asset rehabilitation was the savings it generated: well-functioning canals allowed people to spend less on pump fuel for field irrigation, while water catchments saved expenses for water bought from mobile trucks.

29 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Picture 5: Canal in Degreeb

Canals, Bush Clearing and Roads

80. When it comes to the choice of the kind of work to be carried out in rainfed agriculture villages, there is an inherent tension between rehabilitation of canals and clearing individual farms from the bush. As mentioned, though land ownership is widespread, the viability of farms depends on cleared land, and on access to a functioning pump drawing water from the river to the irrigation canals. Only once these conditions are met, do canals enhance agricultural yields, and thus benefit landowners. The choice to rehabilitate canals therefore inevitably benefited better off landowners whose farms were already viable, leaving poorer ones with unproductive and “bushy” farms to continue working as sharecroppers on others’ farms. On the other hand, to enhance non-viable farms’ productivity, bush clearing should be a preferred intervention, to be complemented with distribution of water pumps to truly benefit these poorer farmers.

81. Ultimately, enhanced agricultural production obtained through canal rehabilitation primarily favoured wealthier owners of functioning farms, though to a lesser degree it went to the advantage of the sharecroppers working on their farms as well.

Key learning 5: Failing to conduct a preliminary assessment to highlight who will benefit from the canals, risks increasing inequality in the long run, and excluding the poorest sectors of the population who do not own a viable farm. The trade-offs implied by favouring different asset rehabilitation should

30 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

therefore be understood beforehand and carefully assessed, possibly through an exercise involving all the sectors of a community.

82. During the first season after the canals were rehabilitated, in 3 out of 4 villages production improved unevenly, either because the work was completed after the planting season, or as a consequence of river flooding, excessive rain or plant disease. At the time of the fieldwork, farmers are expecting a good harvest in all 4 riverine villages, and attributed this to the rehabilitated canals.

83. Benefits of building roads leading to a river include wider river embankments and better access to the river for all animals, freeing people, and particularly women, from having to carry water and/or harvested produce, as this can now be accomplished by donkeys using the road.

Water Catchments

84. In Shatolow and Fardajiiro water catchments were rehabilitated by FAO39, while the one in Jillaab was newly built.. Water catchments were much appreciated as they saved people from having to walk to the river to fetch water, and buy it from mobile trucks. However, the water catchments raised a number of different issues in the three sites, as illustrated in the box below.

Table 10: Water catchment villages

Shatolow: The majority of the population in Shatolow practice farming (90%) while the remaining 10% are pastoralists. The water catchment does not retain water for more than two weeks after it rains, as the soil is sandy. The water catchment was fenced after FAO’s intervention. When it fills after the rains the water is stored in jerrycans and used as drinking water both by the nomadic and the settled population. Following rehabilitation and fencing of the water catchment, animals were excluded from accessing it, and they now drink from the river 6/7 km away. At the time of the field visit the water catchment was almost dry and the pastoralist population had left Shatolow, so the team could only interview the settled portion of the village’s inhabitants. Their claim that the present arrangements of a fenced water catchment exclusively destined for human consumption, was appreciated by the nomadic population could therefore not be verified by talking to the pastoralists themselves.

Fardajiro: Purely pastoralist population. In general there is scarcity of water in the area. The CFW intervention was the third rehabilitation of a pre-existing water catchment. The water catchment is not fenced and animals can access it, with the consequence that water is polluted by the animals and unsafe for human consumption. The unfenced water catchment attracts many herds from nearby villages. It can hold water for about one month, but due to significant migration from other areas at the time of the study the water was rapidly being depleted, as well as the pasture.

Jilaab: Purely pastoralist population. There is scarcity of water in the area. The semi-permanent residents expressed many concerns over water shortage, which forced them to cover a distance of 21 km to seek water from the river. The water catchment here was newly constructed through the FAO intervention, and unfenced. After the rehabilitation it only rained once and scarcely, so the water

39 In Fardajiro it had been built by CARE, and undergone a first rehabilitation through ASEP; in Shatolow FAO’s rehabilitation was the third, following ICRC and CARE, respectively through food and cash for work programmes.

31 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme catchment never really filled up. At the time of the team’s visit the population had temporarily vacated the village and some inhabitants had to be called back expressly to meet the team.

Conflicting water needs

85. The three non-riverine villages are in drought-prone areas, and the population is in dire need of water. The request of tapped water or other safe drinking water system was therefore prominent. However, there was no consensus over how to guarantee accessible public water, and this was a hotly debated topic during meetings with communities. In one of the villages, for example, a young man who requested a borehole was reprimanded and contrasted by the village elders.

86. Digging boreholes or cementing water catchments for water storage can bring uneven benefits, since stored water attracts large numbers of animals from the surrounding area. This can result in:  rapid depletion of natural resources (water and pasture)  risk of spread of animal disease  risk of conflict

87. As previously mentioned, men and women’s herds tend to have different water needs. Since camels can and do cover long distances in search of pastures, male camel herders are less concerned about the depletion of village resources by newcomers than goat- owning women are. In Fardajiro, for example, one of the pastoral villages visited, it was principally women who complained of the pasture’s depletion by newcomers. The women remarked on the paradox represented by semi-permanent pastoralist residents of Fardajiro moving away to preserve the pasture, while the livestock owners from surrounding villages moved their animals to browse around Fardajiro, preserving their own pasture, while spoiling it in Fardajiro.

88. According to respondents, access to a fenced water catchment could be regulated by requesting the same small fee from village inhabitants and newcomers, and investing the cash towards maintenance of the asset. The capacity to enforce such arrangements would ultimately need to be assessed through further investigation.

Key learning 6: The situation in Fardajiro was the starkest illustration of the multiple issues raised by water catchments and tied to use of and access to water in drought-prone pastoralist areas, where traditional water-management mechanisms have been eroded and are presently fragile, at best (Little 2003). Male camel herders seem to be merely opportunistic in seeking water, with little or no concern for long-term sustainability of surrounding pasture areas, since their animals are able to travel far in search of new ones.

32 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Table 11: Ranking of preferred interventions in Fardajiro (F) and Jillab (J) Preferred intervention HH HH Jillaab Yeey U Wanagsantahay yeey u xunthahay Fardajiro 11 HHs 7 HHs (4f present (3 f 8 (who is it good for?) (who is it bad for?) 3m) m of which 3 randomly non- chosen, 10 beneficiaries counters ) 10 counters each each

Water trucking to fill 17 It's good for cattle and goats (Gheeslaha: horned it is bad for camels because a lot of people will join berket with water animals) esp. kids and calves, and human beings that area and spoil the pasture. "People with camels prefer sites where there is no or scarce water; when people with shoats and cows have left the area and it rains this will be a good pasture for them. Camels can stay 31 days without water, so their owners have no interest in the presence of berkets or water catchments”

Water catchment 15 34 It’s good for almost everyone (F) it is not good for camel herders It is not good for through CFW* it is good for the village and the entire location natural resources because too many people will come and the people in the surrounding area. It's good and ruin the natural resources (F) It's for the horns (J) bad for the people with camels (J)

Food assistance 14 10 “It’s good for everyone here because we don’t It’s bad for agriculturalists (F) it's bad for those who cultivate” (F) it's good for the needy plant farms and for business people (J) people (J) Animal health 9 5 Good for everyone, but especially people with It doesn’t hurt anyone (F) It is bad for those big herds. (F) It is good for everybody who deal in animal medicines (J) and a priority (J) Restocking 7 5 it's good for everyone and especially for those It doesn’t hurt anyone (F) who have very few animals and are vulnerable, e.g. a widow with 5/7 shoats whose life depends on animals (F) Health post/medicine for 3 24 It is very important (F) It is good for everybody It doesn’t hurt anyone (F) It is bad for humans and it is a priority (J) people who have small dispensaries, and for crook

33 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

doctors (J) School 3 12 “Schools are needed for children. If we had Nobody said anything negative about schools in them we could be a good village and a Fardajiro. But if a child is good at herding animals he permanent settlement.” (F) will preferentially be kept herding, and the good for the whole village, for whoever has lazy/incapable child will be sent to school. Children children (J) specialise in goat, cow and camel herding. In every family there will be a certain proportion of children who will be kept herding, and others who will be sent to school (this is the principle for Madrasa attendance, and is applied to formal schooling too). In Jillab they said: “it's bad for the children who go after animals, because they are not good for school. As a parent you know if your child is good for livestock keeping or for school: everybody fits in a certain field" Good administration and 2 Good administration is necessary not just in good management of their village, but also in Belet Hawa and interventions (including Doolow. Aid agencies also need to be well FAO’s) administered

UCTs** 9 Good for everyone and especially for vulnerable “Bad for anyone who tries to rely on UCTs as their people (elderly and sick) way of making a living because the money will lead them astray” Rehabilitation of the 4 Good for animals and human beings Not bad for anybody***. shallow well Bush clearing**** 3 Road clearing***** 3 Good for the whole village and whoever is It’s not bad for anybody passing by Latrines 1 We don’t need them. They smell. They cannot be moved so they are useless to us. *In Jillab the specific request was to enlarge the water catchment through the CFW ** Respondents specified they meant “those from Horn Relief” *** However, the team observed that while men don’t care so much about the effects of increased water availability on village pastures because their animals are mobile, women are very unhappy with the idea of the shallow well because they depend on village pastures which can be negatively affected by the presence of the shallow well **** Though Jillab is a purely pastoralist village there are some farms which have been inactive since the war with Aidid ***** In Jillab there is a road from the village to the shallow well that the community would like rehabilitate

34 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Picture 6: Water Catchment in Shatolow

5.1.8 Other preferred interventions

89. Beyond the portfolio of three interventions implemented by FAO, several others were requested in all villages, either in parallel or in alternative to the current ones. Some were proposed as alternatives to the ones currently implemented, while some were complementary to them. Though agricultural support was also requested, often the demand was for other kinds of social support benefiting the whole village:

90. Agricultural interventions requested included: clearing the bush, especially from the weed “aligrob”; distribution of pumps; spare pump parts and fuel; tools for agriculture; fertilizers; crop disease and pest control; training on the use of mechanized tools; land preparation; flood control.

91. Other interventions: reopening of the village school and hiring of teachers; distribution of food and non-food items (pots, buckets, jerry cans); fishing nets, ropes and boats (Degreebe Bantus); water tapping system; lowering the embankment; restocking; animal drugs and vaccines; beekeeping. As mentioned in Fardajiro and Jillab an exercise to rank preferred interventions was conducted, the results of which are detailed in Table 8.

92. Based on their experience, NAPAD, the implementing partner, had some informed suggestions for other kinds of interventions: support to the establishment of beekeeping and fishing cooperatives; building of river embankments (through CFW); rather than simply

35 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme eradicating “aligrob”, which is a nitrogen producer, managing it through training in agroforestry techniques to protect fields from erosion and enrich soils.

Key learning 7: Overall the assessment revealed a number of issues tied to the choice of infrastructures and their impact. Each of the three typologies implemented by FAO implies potential winners and losers, Several trade-offs need to be weighted against each other through preliminary needs assessments or at least an analysis to understand who stands to gain and who loses and what is the more equitable option, considering private or public ownership and individual or collective use of the assets. Cost effectiveness should also be assessed in view of the fact that, for example, canal rehabilitation can be undertaken individually, while water catchment, bush clearing and roads require a critical mass of workers. A further question is whether the infrastructures could be rehabilitated in a more sustainable way, and how this could be done, once again considering the trade-off between providing technical inputs, the need to create labour and therefore avoid machinery, and the potential lack of environmental sustainability and erosion of work opportunities, and of social networks. The possibility of adopting a larger portfolio of options and of a more integrated approach with other forms of agricultural assistance should also be explored.

5.1.9 Workload and work measurement

93. Whether discussing work undertaken on canals or water catchments, complaints over the heavy workload were very frequent. Respondents mainly attributed this to mis- measurement of the ground, which failed to factor the unevenness of the terrain into the computation of the daily amount of soil to be dug. In one village, Jillab, only 35 beneficiaries were targeted to dig the new water catchment. Community members considered this figure to be far below optimal for the task, and one of the beneficiaries thus commented: “to construct a new water catchment with thirty five beneficiaries was not an easy task. This number was too low, especially seeing that some villagers were enfeebled.” In two cases one of the canals could not be fully rehabilitated, since only 300 metres of rehabilitation was planned, while the selected canals were 600 metres long.

94. It appears that in several instances work requirements were either not commensurate to what would have been necessary to complete functional infrastructures, or comparatively too heavy. This seems to have been a result of the infrastructure component having been planned mostly disregarding village variables in favour of a blanket approach, and, ultimately, of a failure to carry out a proper preliminary needs assessment in every village.

95. A further underlying weakness was the scarce technical support throughout, which certainly resulted in miscalculations of appropriate workload, but might also have affected overall quality of the infrastructures built. For example respondents said the newly built water catchment in Jillaab was too small, that it already needed rehabilitation, and that the pre- established number of workers was far too low for the workload.

Key learning 8: Lack of informed technical support negatively affected the programme’s outcomes and impacts in a number of ways, including short-lived

36 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

improvements to infrastructures. This should be remedied, considering that 40 a technical agency such as FAO could provide these inputs at little cost .

96. In some villages the community undertook further communal work after the CFW had ended. Where this meant enlarging the water catchment or widening the river embankment after the road was built, the work was explicitly driven by a common understanding that the whole community would gain by improving on FAO’s intervention. In other cases, as with land preparation following bush clearing, or digging of a garbage hole41, the effort was only indirectly related to the previous CFW, though people did mention that FAO’s intervention had spurred them to undertake further collective work, and in the case of the garbage hole, the tools bought with the FAO UCTs were used.

Key learning 9: The drive to independently undertake further communal work is an instance of a very positive and unexpected impact at village level. A better understanding of the conditions where it happens, could be useful in programming selection of targeted villages.

Picture 7: Water Catchment in Fardajiro

40 Phase IIA of the programme, implemented in 2013, seems to have at least partially addressed these failures through mandatory training of IPs. 41 In Gawido and Unsi respectively.

37 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

5.2 Social impact

5.2.1 Expenditure in social obligations

97. It has been remarked that cash transfers also affect the relational and symbolic dimensions of people’s lives, and as such may have an impact on well-being beyond the material sphere, with potentially pervasive and profound consequences on social relations (MacAuslan and Riemenschneider 2011). There were indications that various social processes were affected by FAO’s CFW, most directly through increased spending towards social obligations. This included saving for one son’s bride price, the payment of blood money, donations for the construction of a mosque, payment for ceremonial henna. Though no direct questions were asked regarding the value attributed to this kind of expenses, one can speculate that in a society where the “moral economy” is all-pervasive, the possibility of spending the cash on social networks and responding to social obligations will be highly valued by individuals and households.

Sharing and informal transfers

98. What emerged very clearly in all villages was that benefits deriving from the CFW were widely shared in a variety of ways. However, it is unclear how far these forms of redistribution at village level are effective in reaching out to the most needy and in sustaining them through bad times, and if they are capable of reducing their poverty and vulnerability in a significant and lasting manner. A further question is whether informal transfers are persisting or declining in significance and effectiveness through time, and if and how any such trend interacts with other transfers introduced by agencies through CFW and UCT programmes. These question may be particularly pertinent in the Somali political and economic space, where the mostly weak or non-existent presence of the state adds value to the safety net function of informal transfers within communities on one side, and to external agencies’ cash transfer programmes on the other. More research, including quantitative data collection and analysis, would be necessary to collect empirical evidence and reach a better understanding of this very important aspect of the programme’s impacts.

99. Whatever the dominant discourse, it appeared quite clear that the degree to which resources were redistributed was to a large degree dependent on power relations in each of the villages, and that where internal strife was rampant the rhetoric of sharing hardly translated into practice.

Table 12: Sharing

Redistributed CFW assets: -work -up to 50% of the wages earned through the CFW -food purchased with wages (e.g. sugar, tea) -food grown thanks to agricultural infrastructures rehabilitated/built (e.g tomatoes) -other minimum expenditure basket (MEB) items (e.g. phone top up cards, clothes)

Sharing vocabulary: NAFQEYBSI “soul sharing” “whatever we have we share” Request from those who don’t have to those who do, on an equal basis. Nafqeybsi can also be

38 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme undertaken on requesto of an elder. Much social stigma is attached to refusing a request of nafqeybsi. The request can come from a beneficiary who wants to share a workload he considers too heavy, or from a non-beneficiary asking a beneficiary for assistance.

QARAABODOON: “looking for relatives”, social assistance to relatives. People seeking assistance do not generally mention how much they request, but usually the amounts involved are relevant (e.g. 50 USD)

SHAXAAD: used when directly asking a friend, a relative or an acquaintance for money. For instance asking a friend for 2 dollars is shaxaad. The amount asked is usually small.

ISKUDULNOOLAAN: “blood sucking”, or exploitation, when someone more powerful forces someone less powerful to hand over part of their assets.

ZAKAT: traditional Islamic charity, one of the five pillars of Islam.

100. Sharing of such apparent magnitude and pervasiveness can have important implications on planning and outcomes, with three main areas involved:

1. By sharing benefits, communities alter the programme’s targeting mechanism and its underlying rationale. If on the one hand this may amplify its outreach, benefiting at least some vulnerable households with no labour capacity, on the other it also poses some questions. In particular, it would be important to verify who exactly is at the receiving end of informal redistribution, and whether there are any specific vulnerable groups and individuals who lack the social capital to participate in sharing networks. 2. When work is shared amongst households, it may happen at the request of the selected beneficiary, if the workload is perceived as too heavy, or the demand can come from a non-beneficiary to a friend or relative who was targeted, in order to partake in the programme. Alternatively, village authorities in charge of targeting sometimes decide to include more families than originally proposed by the IP to make sure that the rehabilitation is completed, to deflect potential tensions, or simply to distribute the incoming cash to a higher number of households. In this case a distortion in the way work distribution and workload were originally planned takes place. 3. Finally, redistribution of benefits alters the overall impact of the programme. A particular question to be considered is the cost-effectiveness of spreading benefits over a higher number of households. On the other hand considerations over effectiveness also need to be weighted against the potentially disruptive effects of engendering social resentment through unequal distribution.

5.2.2 Other social impacts

101. Other symbolic and relational impacts of cash transfers may include negative effects on households and communities. Mixed impact of CTs on social relations is well- documented, and, as pointed out by MacAuslan and Riemenschneider, negative social consequences can derive from the provision of cash, or be an effect of the very process of targeting (MacAuslan and Riemenschneider 2011: 64). On the other hand, the widespread informal sharing of benefits mentioned above can have the very significant symbolic effect of lubricating informal social networks to be activated as social safety nets in case of need. If this is the case, the amount that is shared is relatively unimportant, as it is the act of being

39 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme able to share in itself which assumes a social value and is practiced as a risk management strategy, rather than for its economic value42.

Table 13: The cost of exclusion

One non-beneficiary was unable to work at the time, and she did not have anyone to work for her. Though she recognized that the targeting was fair, as there were people who were more needy than she was, she was still in need of the cash brought by the programme. Her words are a precise illustrations of what failing to be on the programme meant for her: "I was not able to have food on credit any longer during the CFW because the shop owner only sold to people with ready cash and refused to sell on credit. So while before everyone was the same in not having cash, the inequality produced by the programme had a deeper effect as it also affected credit.” Another woman pointed out that lack of access to credit also meant incurring in extra travel expenses to find credit: "The people who had the (CFW) money could access food immediately without having to recur to credit, but I wasn’t able to access any and had to travel to Luq for credit. This proved not only more cumbersome, but also more expensive for me." Two women highlighted the personal feeling of confidence acquired by beneficiaries who were registered and received the cash. One of them said: "if a person has the money and you don’t, that makes a difference in the village. The person who has the money feels happy, proud, and comfortable because they can buy anything they want in the village, but we, we are not getting the money and while we are having lunch we have to think of supper and we have to look for it. " Semi Structured Interviews with female non-beneficiaries, Shatolow

"If you have the cash it is easier to obtain loans, because people trust that you will repay them. Now it is harder to obtain loans." IEC with female beneficiary, Gaawido

102. As the team remarked, “every humanitarian project creates a lot of resentment”, and the FAO CFW was no different in this respect: in the seven villages visited many of the excluded people expressed a degree of resentment and jealousy towards those who were registered as beneficiaries. Resentful groups included people previously dependent on the sisal which had been cleared by the intervention and whose livelihoods were partially disrupted; Bantus and others who felt marginalised by the village administration, the farmers’ cooperative and the implementing NGO; women who were not consulted; people who could no longer buy food on loan while the programme was ongoing, and those who were forced to sell their animals while other village inhabitants were able to spare theirs due to the wages they were earning. However, though this question was not explicitly addressed during the fieldwork, there were no indications that the perception of greater inequality introduced by the programme affected sharing networks or that it crowded out other informal mechanisms of assistance. Overall, it seemed that no negative material impact was registered due to people sharing or cooperating less during CFW, and that the increased inequality, just like the programme’s benefits, did not last beyond the duration of the programme (see also Table 14 below).

103. Conversely, some beneficiaries noted positive social impacts of the CFW. According to some respondents, a greater cash flow and the fact that in general everyone was better off, resulted in greater social harmony and defused conflicts and tensions over control and use of resources. Finally, some women remarked on the way control over cash gave recipients a

42 I am grateful to Laura Hammond for this important insight.

40 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme feeling of empowerment and the confidence to take loans, because they could rely on their earnings. The latter observation is a clear demonstration of the links between perceived relational and symbolic impacts, and material ones.

Table 14: Perception of social impacts

Non-beneficiaries expressed the profound feeling of discomfort when seeing the benefits brought to others by the programme as “feeling the pain”. Despite acknowledging that she was relatively better off than the selected beneficiaries, one non-beneficiary woman who was negatively affected by the lack of access to credit during the intervention conveyed her relief when it ended: “Now thank god the cash is not there, and all people are the same again”. Others noted the positive social impacts of the programme: “Thank god, the cash changed things for the whole community. Now I see people going to the market directly to buy food rather than having to go to the farms to collect sisal and then travel far to Belet Hawa to sell it. When cash is there people do not fight. The cash transformed relations within the community for the better, while people working as sharecroppers were previously fighting over land all the time”. In noting how the intervention increased inequality within the village, one of the non- beneficiaries also remarked on the feeling of confidence it gave beneficiaries: “I could see people were able to buy things with the money, while I didn’t even have animals to sell and help me to cope during the bad times. People who were working had the courage, the strength and the confidence to take loans, because they could rely on the CFW money, while I didn’t have that courage.”

5.3 Impact on gender

Gender and programming

104. Many complaints were expressed by female beneficiaries and non-beneficiaries for the way the programme was implemented. Women were very vocal in protesting their marginal role, and in reclaiming their rights as active participants throughout all phases of programming. They frequently lamented lack of information about programming modalities, and total exclusion from consultation at the planning stage. Women were also often barred from registering as beneficiaries, especially if they had no male relative who could work in their stead, and despite the nominal requirement in all project documents to reach a target of 30 to 40 per cent women beneficiaries.

41 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Picture 8: Female IECs in Fardajiro

Gender and cash

105. On the other hand, no female respondent made any reference to intra-household inequality or conflicts arising over control and use of the cash. Somali women are generally recognized as the principal managers of the family budget, and though some women beneficiaries did say they discussed expenditure of the earnings with their husbands, most of them made it quite clear that the final decision over use of the cash was theirs. For programming purposes, it is interesting to note that women who were formally registered but were not actually working themselves and were “worked for” by a male relative still claimed they had complete control over the wages.

106. However, this acknowledgement of women’s role should not automatically be taken to imply an absence of discussions between men and women over family expenditure, particularly in the case of multiple wives in polygamous households. On the contrary, polygamous male beneficiaries were extremely reluctant to disclose any arrangements they made for sharing the CFW wages among their wives, in itself an indication of the sensitivity of the subject, and of its potential to generate conflict. A better understanding of intra- household dynamics triggered by the intervention, and those between polygamous households in particular, could be reached through more in-depth dedicated research.

42 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

107. A subjective feeling of empowerment was mentioned by some women who were included in the programme, and conversely, a feeling of disempowerment for having been excluded.

Table 15: Women’s empowerment

One registered beneficiary from Gawido was only 17 years old. She lives with her husband, a child, and her two brothers. One of her brothers worked in her place, but she was the one who actually collected the money. She told the study team that from the moment she had an income she gained respect in the village, and that this lasted throughout the period of the intervention, but afterwards things went back to the way they were before. She runs a kiosk and used the CFW money to improve her business.

108. Some of the expenses incurred specifically by women included buying clothes for themselves, participating in merry-go-rounds, saving to hire a midwife for an upcoming delivery. As mentioned, when discussing household expenditure women, as opposed to men, were often able to give detailed accounts of cash expenditure for other family members’ needs, e.g. buying medicine for sick husbands, sending children to school or, in one case, helping a son to stop working at a checkpoint. Men on the other hand usually manage bigger sums of money and need such sums to make investments43. Perhaps not surprisingly, it was women rather than men themselves who reported male expenditure towards items such as cigarettes, khat and blood money, to which a certain degree of social stigma is attached.

109. Some of the findings on gender converge with those of a recent study on gender and UCTs in Somalia commissioned by the Cash Consortium44. Similar conclusions were reached regarding the status-enhancing role of cash, women’s capacity to manage and invest small sums against men’s bigger ones, and the problematic issue of splitting cash between polygamous HHs. The gender study also speculates that “personal spending of cash was more acceptable for men than for women”. What emerged from fieldwork in Gedo, on the other hand, was that women who were the registered recipients of the cash, used (and saved) it for their own priorities, which could or not be strictly personal. This suggests that rather than reaching such a strictly binary conclusion regarding “altruistic” and “selfish” spending by gender, it might be better to research into the differences between what women and men regard as “personal”. This could lead to a more nuanced interpretation, possibly indicating that personal spending for women might include spending for other family members too.

110. It emerges that, according to the commonly accepted discourse, informal community assistance systems require female-headed households to be taken care of by relatives or by the community at large. The fieldwork, though, revealed that this happened on an ad hoc basis. On the contrary, households excluded from all but minimal sharing were usually female-headed ones (also excluded from participation in the CFW), presumably due to low levels of social capital.

43 For example a male beneficiary complained that being paid in three instalments meant the sums were too small to be invested productively. 44 Wasilkowska, K. (2012)

43 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Gender and infrastructures

111. Other than a single example of a woman who owned a fully functional farm, canal rehabilitation directly benefited men as owners of functional farms, while female sharecroppers on these farms only benefited to a lesser extent. The land of the few farm- owning women was bushy and, had they been consulted, their priority would have been bush clearing over canal rehabilitation. As mentioned above, the different composition and needs of men’s and women’s herds meant that women were far more worried about the potentially disruptive effect of incoming herds attracted by rehabilitated water catchments on villages’ natural resources.

Key learning 10: Since land owned by women is invariably “bushy”, bush clearing interventions are to be preferred in order to sustain this small but important share of farm-owners. Such a measure would be a way of giving substance to the CFW’s stated objective of “empowering women”.

Table 16: Women’s voices from Fardajiro

Warsan wasn’t targeted because at the time she wasn’t in the village. “I had taken the shoats to look for a good pasture with my son. I would not have been able to work but I could have sent my son to work for me. I heard about the possibility from others, so I immediately went back to Fardajiro, but as soon as I arrived in town I found the NAPAD people surrounded by men. At the time I was not happy about the way the guys were conducting themselves, they should have given women a chance to be consulted. Men should consider that women are important in the village because they are the ones who are always responsible of taking care of the family. Men go away to town and can stay away for a month, while we are always here. They go with the camels, and as they are away they are not aware of what happens in the family.”

Halima wasn’t registered because at the time her husband was in Belet Hawa. She herself was unable to complain with the men that she wanted to be included in the programme, because she felt shy. “That’s the main problem women encounter: organizations coming here don’t send a woman who would be able to talk to the other women in the village.”

5.4 Unintended negative impacts

112. The programme’s outcomes were not always positive, and this is partially to be attributed to the programme’s weak design and lack of foresight, as well as to obstacles to obtaining preliminary information about villages.

113. Some actual and potential negative consequences of the programme have already been mentioned earlier:  Influx of cash in the village and registration in the programme employed as collateral to request loans meant that non-beneficiaries could not access credit while the programme was ongoing, and that they had to travel to nearby towns to buy food and basic household items on credit.  Food consumption patterns sometimes changed as household income increased. While the consequences on nutrition are still to be assessed, some households

44 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

incurred in debts for some time after the programme ended to sustain the new habits acquired.  Inclusion and exclusion from benefits can engender resentment and inequality within villages.  In the village with the biggest Bantu community, Bantus were not allowed to officially register on the programme, although they worked instead of the official beneficiaries, and were forced to give them half their wages. The fact that the registered beneficiaries were landlords on the fields where the Bantus were sharecropping indicates that, in this case at least, the programme’s presence did nothing to mitigate, and in fact perpetuated existing patterns of exploitation of this heavily discriminated portion of the population.  Pastures around the village were rapidly depleted by newcomer’s herds attracted by rehabilitated water catchment.

114. Other relevant unintended negative impacts were:  Water catchments reduced the need to move with animals in search of pasture and water during the dry season. Though subjectively much appreciated by interviewees, decreased mobility of nomadic communities can come at a cost to social relations, and negatively impact on natural resources.

Key learning 11: The implicit trade-offs should be considered when triggering processes affecting mobility patterns. In terms of design this should translate into a clearer distinction between the objective of reducing distress migration and displacement to IDP camps, and seasonal herd migration as an ordinary drought management strategy to optimize scarce resources.

115. In one pastoralist village some workers on the water catchment fell sick and had to be taken to the hospital. This was attributed to their bad body conditions, and to the workload, which was considered too heavy.

116. Land clearing interventions are the highest priority of bushy farm owners, but they may inadvertently translate in reduced income for the poorest households for whom sisal collection is a coping strategy in times of distress. Failure to sell sisal can also have secondary negative effects on households, for example one woman was forced to become a day labourer to make up for the income she had lost. Furthermore, in some cases after the intervention farm owners fenced areas which had not been cleared, and demanded that the sisal collectors, who had previously picked them for free, give them a percentage of what they earned from the sale.

117. Some village inhabitants observed that canal rehabilitation could be and usually was undertaken by individual farm owners, while roads, bush clearing and water catchment rehabilitation required a critical mass of workers and were therefore better suited to be the object of the CFW. A documented unintended negative consequence of canal rehabilitation was the loss of work for people who were usually paid to maintain the small tertiary canals.

Table 17: Findings from the telephone survey A telephone survey was conducted to triangulate information gathered through fieldwork and acquire information from other inaccessible famine-affected regions, tailoring question lists developed for the fieldwork to fit the purpose.. 80 respondents from the Bay, Middle Shabelle, Lower Shabelle, Middle Juba, and Hiraan regions were interviewed as well as 20 from Gedo Only 3 respondents were women.

45 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Telephone numbers had been provided by NGOs implementing in these districts, as required by Phase IIA LOAs. The greater anonymity afforded by phone interviews meant that information provided was not subject to public scrutiny, with positive and negative consequences: sensitive topics could be breached and discussed, and in fact emerged more frequently than in face to face encounters; however, there was no way of verifying the identity of respondents, nor their credibility. Accordingly, while findings corroborating fieldwork data can be considered validating of the latter, new issues that came up exclusively during phone interviews should be approached with caution as being indicative only, and deserving of further scrutiny through follow-up research.

Findings confirming fieldwork data: o Overall positive assessment of the CFW o Positive assessment of implementing NGOs (Gedo) o Social impacts: extensive sharing, some resentment engendered by exclusion o Emphasis on further infrastructure development, and broader portfolio of desired interventions (feeder and lorry roads) o A degree of productive investment of the cash, but longer term intervention would have enhanced it. o Most widely goods purchased with wages: more and better food, and HH items. o No long-lasting effect of the programme on most HHs

New findings and issues deserving further research o Women are more difficult to reach over the phone, probably due to different phone ownership between men and women o Mismanagement, including rregularity of payments and more workers than required for the job (one NGO, middle Juba) o Only one break allowed over the working day (in one village) o Some effect on preventing distress migration (in famine-affected districts) o Some effect on coping with the famine and its consequences (in famine-affected districts) o Request of training opportunities beyond agriculture e.g. mechanics o food/vouchers received in exchange of work45 (one village in famine-affected district) o some under 18 children worked on the CFW, helping family members who were registered workers46 (Gedo and famine-affected districts) o huge variation in wages within and among villages and some variation in days worked among villages (famine affected districts)47

5.5 Sustainability of change

118. Indications of any long term impacts are particularly relevant considering the CFW’s alignment with FAO’s strategic twin track approach, and its potential to induce lasting change through the infrastructure component.

119. A comparison of beneficiaries’ income-expenditure-coping patterns before, during and after the programme reveals that whatever changes brought by the cash were for the most

45 probing confirmed this was not the result of a confusion with other agencies’ assistance programmes 46 Children reportedly did the same tasks and worked the same hours as adults. It should be noticed, however, that no children were mentioned as being the officially registered beneficiaries of the CFW. Four HH reported children in the 10-13 age bracket working, while all the others were 14 and above. 47 Variation in income may have been due to some participants sharing the load of the programme and the wages with family members or friends not registered in the programme. Another reason for the variation in wages may be linked to underreporting of earnings to seem more eligible for future CFW programmes.

46 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme part not sustainable. Beneficiaries’ comments during Participatory Rural Appraisal (PRA) exercises and Focus Group Discussions were consistent with this finding.

120. In general the cash component’s impact was short lived, an exception being investment by women in human capital or in productive enterprises. For instance a woman who invested in a kiosk is a revealing example48: she presently uses the income from the kiosk to fulfill her family’s basic needs and cannot afford to reinvest it in the kiosk itself, and as things stand it seems her activity will not be sustainable for long. She explained that her revenue would have been higher if the programme had lasted longer and people still had more money to spend on the goods she sells.

121. Rehabilitated infrastructures on the other hand were for the most part still found to have an impact one year after the programme had ended, both directly (due to increased production in farms with canals, or by saving on water in the case of water catchments) and indirectly (through increased income from higher volume of sale of agricultural produce and fodder). However it was often mentioned that canals and water catchments deteriorated rapidly and were already in need of further work, so even this component can only be considered to have medium term impacts at best, and more work to consolidate infrastructures would be necessary to produce a more sustainable and significant change.

Key learning 12: Infrastructure management should be part of a package to enhance the sustainability of the CFW’s impact.

122. Ultimately two factors seem to have contributed to sustainable changes: women’s capacity to make small investments in petty trade with the cash they earned; and the infrastructures which were built or rehabilitated, though the latter may have disproportionally benefited better off owners of viable farms and pastoralists with bigger herds.

48 Female beneficiary, Hammare.

47 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Picture 9: Water Catchment in Jillaab

123. This section includes key lessons learned, from the study, to be to be rapidly and urgently incorporated into programming. Some detailed suggestions are also included in relation to areas and topics that would require further research and analysis.

Lesson learned 1: Coordination between agencies Trying to ensure food security through labour-based safety nets may be problematic, particularly during a crisis, and vulnerable households risk falling in an assistance void due to bad targeting and lack of labour capacity and social relations. Furthermore, weak coordination with other forms of assistance does not guarantee that they will be reached through other channels. Better coordination with agencies distributing UCTs is necessary, and, eventually, incorporation and acknowledgment of informal community redistribution mechanisms in planning (see also lesson 7).

Lesson learned 2: Timing of work Construction of water catchments immediately after the drought crisis presented specific problems. In the first place, the population had left and had to be expressly called back for registration, and this resulted in consistent inclusion and exclusion targeting errors. Secondly, though people were weak due to the food crisis, there was consensus among them that the timing of the CFW was right: it was a moment when people needed cash to avoid distress sale of their animals, and there was just enough time to build the water catchment before the rains filled it with water. This unaddressed paradox of seasonality, whereby the moment when people are farther from the village and most unfit for work is also when the benefits of

48 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

CFW are in greatest demand, needs to be addressed and solved, as a similar situation is likely to present itself in non-drought years as well. For example work could be provided in other moments of the year, ensuring that the water catchment is maintained and ready to collect water during the rainy seasons. Implementation at a time when households are present would ensure more systematic targeting.

Lesson learned 3: Targeting criteria From the design stage onwards local concepts of justice, encompassing clan and subclan relations, should be taken into account. Such concepts need to be better understood and analysed. When found to translate into support to the most vulnerable, they need to be incorporated into programming. On the other hand, if they are mechanisms that discriminate and exclude certain sectors of the population, ways to contrast them should be put in place. In particular the common use of “working for” households with no able bodied members should systematically be included in programming. This assigns a fixed quota of extra labour to groups of workers, eg 8 who undertake the workload of 10. This group of 8 give up the two extra wages to two households with no labour force.

Lesson learned 4: Choice of infrastructures In agricultural riverine communities, owners of viable farms tend to capture the greater share of the benefits brought by the CFW. In particular, they seem to be the main beneficiaries of the rehabilitated infrastructures. Though canals also benefit the sharecroppers on those farms, canal rehabilitation does not assist them to develop their own, often unproductive, farms. FAO should carefully assess the costs and benefits of targeting either group, in order to make an informed decision when directing its assistance. It is crucial to undertake a much more careful assessment to understand the implications of choosing to rehabilitate one type of infrastructure over another on communities, households and individuals.

More in general, the following are some of the measures to be adopted in order to optimize the impact of the infrastructure component of the CFW on agricultural productivity: a. a solid preliminary analysis needs to clarify who benefits from different kinds of interventions, where the greatest gains lie, what the trade-offs are between spreading benefits and intensifying production49, and if there are any trade-offs between food security and agricultural productivity-enhancing objectives. Preliminary needs assessments should analyse livelihoods, interactions between different livelihoods, use of natural resources and impact of the intervention on natural resources and livelihoods; b. every effort needs to be made to involve the whole community in programming, particularly at the needs assessment stage. To make sure that all clans are involved in the discussion, gatekeepers and majority clan leaders can be approached and clan presence mapped out with them. This will facilitate successive interaction with minority clans; c. a move should be made towards a more holistic approach, beyond the rehabilitation of a small portfolio of assets through the CFW, seeing that other factors influence agricultural production, particularly in a setting where most farmers are very vulnerable. For example, at time of the study many farmers expressed fear that

49 However there clearly are intrinsic limits to intensification of production, seeing the climate and the natural resource basis.

49 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

floods would spoil the expected good harvest. Building river embankments as part of the intervention could go some way towards preventing this.

Lesson learned 5: Targeting the most marginalized When targeting, specific measures need to be taken to avoid the exclusion of certain sectors, and particularly those who are fit for work but marginalized. These were identified as follows: Gender - Understanding livelihoods and vulnerabilities by gender; - interacting with members of women’s committees, where they exist, to make sure that women’s priorities are accounted for; - Requiring IPs to employ female staff to work specifically with women. Bantus - targeting specific geographical settlements and livelihoods (riverine fisheries) would enable to include them and assist this extremely marginalized and vulnerable group

Lesson learned 6: Sustainability and building resilience In view of local rural livelihoods’ extreme vulnerability, as also highlighted by the present study, the adoption of measures to transform them and build their resilience would be desirable.

Programmes should be implemented for a longer period, extending the impact of the CFW, and in some cases more workers need to be employed to ensure that the assets built are completed and consolidated. There is clearly a trade-off between reaching more beneficiaries throughout the country and spreading benefits thin, and concentrating inputs to achieve more durable change. In the context of the drought response when food security was the main objective, a case can be made for the former. Otherwise, it is recommended that concentrating resources in selected villages be considered as an option, for interventions to reach the productivity enhancing effect they set out to achieve.

Furthermore, households’ extreme exposure to shocks and stresses means that CFW needs to be part of a package of interventions, targeted according to household categories and needs. The request for a broader portfolio of projects, often beyond the reach of a single agencies’ mandate, also emerged during discussions with community members.

This lesson aligns with the joint FAO/WFP/UNICEF “resilience strategy” and programming, aiming to enhance individual, household and community resilience through coordination between agencies and integration of sectoral interventions50. As this study indicates, under the right circumstances CFW can play a part in enhancing household resilience, provided it is complemented by other livelihood supporting interventions.

Lesson learned 7: Targeting vulnerable households At present the project and other programming documents are unclear on the rules regarding individual and household registration and participation in the programme. Although the “Implementation guide of CFW projects”51 comprises “allowing someone else to do the work” among the ways to include labour-poor and time-poor households in the programme,

50 In particular CFW aligns with “strengthening productive sectors” and “establish predictable safety nets”, which have been identified as two of the three building blocks by to promote resilience in Somalia. 51 Annex 1 of OSRO/SOM/124/USA

50 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme how exactly this should be done is not specified. Research confirms that this is an appropriate and culturally acceptable way to i. involve vulnerable households, ii. ensure that women are targeted and control the income they are entitled to. It is therefore recommended that FAO embed this more firmly in programming, not least by specifying how it should be done concretely, and monitoring the risks it entails as well as the benefits it brings.

Lesson learned 8: Community consultations More rigorous requirements should be formally requested from NGOs to guarantee widespread community consultation. Although care needs to be taken to avoid overloading the compliance agenda and overburdening IPs52, a number of measures can be taken, including for example request of audio and visual proof of meetings with the community and documentation of needs assessment processes, using mixed gender teams to ensure women are reached and consultations held with village women’s groups. Measures to support the IPs would also be helpful to this end, for example through training53, and by ensuring that the consultation phase is properly funded.

Lesson learned 9: Carrying out impact assessments A shift from focusing solely on compliance monitoring is necessary, and in the future,e impact assessments need to be routinely carried out. This direction has already been taken, and a strong monitoring and learning system is being set up, which in the future will be an integral part of CFW activities. This is a welcome change and should be pursued consistently. The methodology to be adopted to conduct monitoring and learning should be carefully considered, taking lessons deriving from the present study into account. Quantitative research and analysis of impact could clearly strengthen findings, but its feasibility, reliability and cost-effectiveness should be assessed on a case by case basis.

Topics and issues for future research

Some further research areas of particular interest, in terms of feeding back into programming, are listed below. They have been ranked by priority. The first three are the most urgent, and should, if possible, immediately inform programming. i. A better understanding is needed of targeting within communities: how it is carried out, who it includes/excludes, and its underlying rationale. This would provide indications of measures to make the programme less exclusive and exposed to elite capture, and helps reframe targeting objectives and modalities accordingly. ii. If the CFW has the aim to improve livelihoods in the long run, a better understanding needs to be reached of the drivers of sustainable change. What are the contextual conditions whereby the programme can have an impact in the long run? This includes developing indicators of sustainability, and analyzing them throughout the project cycle. iii. Gathering empirical evidence of the outreach and magnitude of sharing and informal transfer mechanisms identified during the study: how many people they involve, how important they are in sustaining people through bad times and whether they are persisting or declining in significance over time.

52 This problem was raised by a number of NGOs interviewed during the FAO Somalia evaluation mission, see evaluation report REF. 53 Training was already in place for CFW IPs in Phase IIA, and has been intensified in Phase IIB.

51 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

The remaining 6 areas listed below, while still useful, are of secondary importance.

iv. Gaining a better understanding of the overall nutritional impact of the intervention, including trends of change in dietary habits by income bracket, and the longer-term impact of changes in dietary habits; v. It seems that occasionally the effects of the intervention were delayed by a whole season, as agricultural seasonality was not sufficiently considered in planning. It is also not completely clear whether the work interfered with households’ agricultural labour priorities. A better understanding needs to be reached of the way the CFW labour requirements interface with the timing of agricultural activities; vi. It seems that some wild plants have an important role in the livelihoods of the poorest54. In riverine villages it was found that interventions sometimes modified patterns of growth, established enclosure and common property regimes, and sale and processing modalities, bringing changes to accessibility of these plants, with important consequences on some vulnerable households. This is an indication that more research is needed on the effects of the infrastructures on natural resources and their management, on one side, and on the most vulnerable livelihoods, on the other; vii. Further research is needed regarding the economic and social consequences of work requirements on households. In particular if other family members are required to fill-in for the domestic and external work usually carried out by beneficiaries, and what the consequences are; viii. uncovering the dynamics of distribution of benefits between polygamous families. This is a problematic area of inquiry, since people are very reluctant to talk about the conflicts it generates, but it could be revealing of intra-household tensions that were not uncovered by the present study; ix. a deeper understanding of how the income gained through CFW is related to a subjective feeling of empowerment, mentioned by some beneficiaries, and particularly by women.

Two final reflections on lessons learnt on the methodology for impact assessments in the Somali context are:

1. Adopting a qualitative approach proved an effective way to work in the context of Somalia and to overcome the limits to experimental methods and statistical data collection in a constrained environment. 2. Working with participatory tools, which requires a high level of rigor, has the great advantage of allowing in-depth discussions with beneficiaries and non- beneficiaries, who may thus express their views and be listened to in a systematic and facilitated process. Ultimately, if used skillfully, such tools also allow the involvement of the communities themselves in the analytical process. Some of the issues uncovered by the study could probably only be revealed through such protracted and reiterative engagement with communities.

54 for example the bark and branches of acacias and other trees are split into very narrow threads, called mayrax, and put to various domestic uses; sisal is used for cloth and mat weaving; leaves of saarsaa tree are collected and eaten when no other food is available

52 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

References

Chambers, Robert (1983) Rural Development: Putting the Last First, Essex: Longman

Cook and Kothari (2001) Participation: The New Tyranny?, New York: Zed Books

Devereux, S. (1999) Making Less Last Longer: Informal Safety Nets in Malawi, IDS Discussion Paper 373, Brighton: Institute of Development Studies

Devereux, S. (2006) Vulnerable Livelihoods in the Somali Region, Ethiopia, IDS Research Report 57, Brighton: Institute of Development Studies.

FAO (2012a) ‘Concept Note for a Study of FAO Cash for Work Interventions in Somalia 2011/12’, mimeo, Rome: Food and Agriculture Organization

FAO (2012b) ‘Evaluation of FAO’s Interventions in Sudan funded by CHF - methodological note and Annexes’, Rome: Food and Agriculture Organisation

Hickey, S. and Moyan G. (eds) (2004) Participation - From Tyranny to Transformation: Exploring New Approaches to Participation in Development, New York: Zed Books

Little, P. (2003) Somalia: Economy without State, Oxford and Bloomington, Indiana: James Currey and Indiana University Press.

Longley, C., Dunn, S. and Brewin, M. (2012) ‘Cash and Voucher Monitoring Group: Final Monitoring Report of the Somalia Cash and Voucher Transfer Programme. Phase I: September 2011-March 2012’, Humanitarian Policy Group Report, London: Overseas Development Institute

MacAuslan, I. and Riemenschneider, N. (2011) ‘Richer but Resented: What do Cash Transfers do to Social Relations?’, IDS Bulletin 42.6: 60-66

Mohan, Giles (2001) ‘Participatory development’, in Desai, V. and Potter, R. (eds.),The Arnold companion to development studies, London: Hodder.

Oxford Policy Management, (2012), ‘Qualitative Research and Analyses of the Economic Impacts of Cash Transfer Programmes in Sub Saharan Africa: Research Guide’, mimeo, Oxford: Oxford Policy Management

Sadler, K. and Catley, A., (2009) Milk Matters: The Role and Value of Milk in the Diets of Somali Pastoralist Children in Liben and Shinile, Ethiopia, Boston: Feinstein International Centre

Sadler, K. et al (2009) Milk Matters: a literature review of pastoralist nutrition and programming responses. Addis Ababa: Feinstein International Centre , Tufts University and Save the Children

Stern, E., Stame, N. Mayne, J. (2012) Broadening the Range of Designs and Methods for Impact Evaluations. Report of a study commissioned by the Department for International Development, DFID Working Paper 38, London: Department for International Development

53 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Stiglitz, J. (2002) ‘Participation and Development: Perspectives from the Comprehensive Development Paradigm’, Review of Development Economics , 6.2: 163-182

Wasilkowska, K. (2012) ‘Gender and UCT Study: Summary of Initial Findings’ presentation at the Cash Based Response Working Group meeting, Nairobi, 5th December 2012.

54 Evaluation of FAO’s cooperation in Somalia, Annex 10, A study of FAO Somalia’s Cash for Work Programme

Annexes

I. Concept note II. Constraints to good practices in data collection III. Indications for fieldwork IV. Participatory tools - aims and procedures V. CFW PRA tools VI. Fieldwork guide VII. Q prompts for non beneficiaries VIII. Training schedule IX. Village maps X. Somali words and expressions XI. List of people met