The Pro-Poor Livestock Policy Initiative
Total Page：16
File Type：pdf, Size：1020Kb
EVALUATION OF THE PRO-POOR LIVESTOCK POLICY INITIATIVE
GCP/INT/804/UK
10 March - 15 April 2005
Peter Bazeley, (on behalf of DFID) Stephen Sandford (independent senior consultant) Daniel Shallon, (FAO Evaluation Service; Team Leader)
Rome, May 2005
Contents
ACRONYMS...... iv
EXECUTIVE SUMMARY...... v
I. INTRODUCTION AND CONTEXT...... 1
A. Scope and Methodology of the Evaluation...... 1
B. Origins of the PPLPI and Evolution of the Context and External Environment, 2000-05...... 2 Origins of the Project...... 2 Evolution of the Context Five Years Later...... 3
II. ASSESSMENT OF PROJECT OBJECTIVES AND DESIGN...... 5
A. Evolution of the Project Design and Log-frame...... 5 Early Phases of the Project...... 5 Revision of the Log-frame...... 5
B. Relevance and Justification...... 6
C. Assessment of Current Objectives and Focus of the PPLPI...... 7 Narrowing the Project Objective (the Log-frame "Purpose")...... 7 Strengthening Capacity in FAO...... 8
III. ASSESSMENT OF PROJECT IMPLEMENTATION...... 9
A. Implementation to Date: Assessment of Quality and Efficiency...... 9 Project Outputs and Activities...... 9 Output 1: Portfolio of livestock-related policy interventions for reducing poverty...... 10 Output 2: Increased awareness of the potential contribution of the livestock sector to poverty reduction...... 11 Output 3: Effective systems for livestock policy information, analysis and decision-support, and M&E...... 12 Output 4: Mechanisms established for effective stakeholder representation in the negotiation of policies and institutional changes...... 13
B. The Staffing and Structure of the PPLPI Project Team...... 14
IV. KEY ISSUES AND FUTURE DIRECTIONS...... 15
A. Project Governance and Management Structures...... 15 The Need for a Management Board...... 16
B. Beyond FAO: Role of the Focus Countries, Regional Hubs and Work at Global Level...... 17 Work in the Five "Focus Countries"...... 17 Regional Hubs and Work at Regional Level...... 19 Work at Global Level and with International Institutions...... 19 Creating a Strong PPLP Network - a neglected strategy...... 20
C. Cost-effectiveness and Use of Extra-budgetary Funding...... 20
D. Sustainability of Project Impacts...... 21
V. EVALUATION CONCLUSIONS...... 22
ii A. General Conclusions and Factors Which Will Affect Project Success...... 22 The log-frame...... 22 Impact on AGA...... 23 Integration with wider processes and complementarity...... 23 Strategic engagement through pilot country initiatives...... 23 Fostering innovation...... 23 Communicating...... 24
B. Recapitulation of the Evaluation's Main Recommendations for the Future of the PPLPI...... 24 Refocusing the Project Purpose/Objective on Strengthening FAO Capacity...... 24 Improving Project Governance and Providing an Enabling Environment...... 24 Working with Other Organisations: Global Mandates and Complementarity...... 25 Country and Regional Work...... 25 Implementation Issues...... 26
ANNEX I: LOG-FRAME OUTPUTS AND PERFORMANCE...... 27
ANNEX II: ELEMENTS FOR TERMS OF REFERENCE FOR THE MANAGEMENT BOARD...... 29
ANNEX III: ITINERARY, PEOPLE MET AND SUMMARY OF FINDINGS OF VISITS TO COUNTRIES AND INSTITUTIONS...... 31
ANNEX IV: TERMS OF REFERENCE OF THE EVALUATION...... 38
iii ACRONYMS
AG Agriculture Department (FAO) AGA Animal Production and Health Division (FAO) AGAH Animal Health Service (FAO) AGAL Livestock Information, Sector Analysis and Policy Branch (FAO) AGAP Animal Production Service (FAO) ALive 'African Livestock' platform - A Partnership for Livestock development, Poverty Alleviation and Sustainable Growth (World Bank) ASEAN Association of South-East Asian Nations AU/IBAR African Union's Inter-African Bureau for Animal Resources CONDESAN Consorcio para el Desarrollo Sostenible de la Ecorregión Andina DFID Department for International Development (UK) DTF Dairy Task Force (Uganda) EC European Commission ESAE Agricultural Sector in Economic Development Service (FAO) ESCB Basic Foodstuffs Service (FAO) ESNC Secretariat of the Codex Alimentarius (FAO) EU European Union GIS Geographic information system IBAR see "AU/IBAR" IFPRI International Food Policy Research Institute IGAD Inter-Governmental Authority on Development (an Eastern Africa REC) ILRI International Livestock Research Institute LEAD Livestock, Environment and Development Initiative M&E Monitoring and evaluation MDG Millennium Development Goals NEPAD New Partnership for Africa's Development NGO Non-governmental organisation OIE World Organisation for Animal Health OVI Objectively verifiable indicator PMA Programme for the Modernisation of Agriculture (Uganda) PPLPF Pro-Poor Livestock Policy Facility (interchangeable with PPLPI) PPLPI Pro-Poor Livestock Policy Initiative REC Regional Economic Communities RP Regular Programme (FAO) SADC Southern Africa Development Community SC Steering committee TCA Policy Assistance Division (FAO) TCI Investment Centre Division (FAO) UEMOA Union Economique et Monétaire Ouest-Africaine USAID United States Agency for International Development
NB: Paragraphs in 'italics' in the text of the report generally denote recommendations.
iv EXECUTIVE SUMMARY
1. A Evaluation Team, composed of two experts representing DFID and FAO and a third independent senior consultant, carried out the evaluation between 10 March and 14 April 2005 (with an interruption 20-28 March and a follow-up meeting 5 May). The Team had extensive consultations at FAO Headquarters, visited two of the Initiative's five pilot countries (Uganda and Peru) and spoke with officials at the African Union and its Inter-African Bureau for Animal Resources (AU/IBAR), the International Livestock Research Institute (ILRI), the World Organisation for Animal Health (OIE), the European Commission, the World Bank and USAID. In addition the Team met with government officials and NGO staff working in livestock and policy in Uganda, Kenya, Ethiopia and Peru. In all, the Team had detailed interviews with over 80 individuals in eight countries in four regions, as well as three workshops to discuss the conclusions. 2. Overall, the Mid-Term Evaluation Team was impressed with the quantity and the quality of the project activities during the first half of its implementation. This work has included significant studies and working papers which have formed the basis for dialogue with national, regional and international partners. The five pilot country programmes, though still new, are already producing interesting outputs which are drawing attention in the countries to pro-poor livestock issues. The project, as it now stands with the experiences and materials accumulated over the past two and a half years, has strong potential for achieving sustainable impact during its remaining life. Whether it does so will depend in large part on its ability to build strongly on its past to focus on a clearer strategy for the coming years.
General Conclusions and Factors Which Will Affect Project Success 3. Several conclusions emerge from the analysis in this report with regard to the present accomplishments and future strategic orientations of the project. The project will ultimately be judged on the extent to which it has successfully led to creation of enhanced capacity in FAO, in member nations and in other international organisations to formulate livestock sector and related policies and implementation plans that reduce poverty. The Evaluation Team highlights some major factors which emerged and which could affect the likelihood of achieving this outcome: 4. The log-frame: Despite a slow start, excellent progress has been made in generating new knowledge and partnerships, and in the establishment of solid, quality work in focus countries. Project activities and resources are well managed. However, the Team encountered an unexpected lack of common understanding - or agreement - about the project’s objectives and raison d’être. The (internally revised) project log-frame is not proving to be a sufficiently used management tool in this respect. It is in need of another review and rationalisation effort, preferably with the assistance of external expertise, as recommended below. 5. Impact on AGA: The Pro-Poor Livestock Policy Initiative has clearly had impact on the work and configuration of AGA, contributing to the Division’s efforts to achieve a better balance between policy and technical work, and to place greater emphasis than before on social equity objectives (especially poverty eradication). Good progress has been made towards mainstreaming project activities, particularly through the evolution of staff positions. However, integration of the project with the rest of AGA, and FAO more widely, was found to be patchy. The reasons for this are complex, coming both from the project itself and from the surrounding environment. There is certainly more to do, and this should become a priority for the remainder of the project. More proactive and constructive management of the environment into which the project is expected to deliver is a priority. 6. Integration with wider processes and complementarity: Since the project design phase, significant shifts and developments have occurred in the project’s external environment – in particular with the emergence of new African institutions, and the interests of other international organisations. Even in this changing context, the project remains highly relevant to contemporary development
v priorities and approaches. However, PPLPI cannot achieve its ambitious objectives alone: it will achieve them by working with and adding value to wider policy and institutional processes. There are encouraging examples of how PPLPI is successfully doing this at the level of national focus country work, but relatively less progress has been made in engaging with the international institutional architecture. This has led to FAO being marginalised in some of the international debate and activity being championed (e.g., by the World Bank and OIE). PPLPI and AGA more widely need to reflect on the likely scope and impact of these other programmes and seek to ensure complementarity as far as possible. As recommended, convening a high-level meeting to reflect on the changing institutional landscape and identify areas of mandate, comparative advantage and complementarity would be most useful at this stage. 7. Fostering innovation: PPLPI needs to have the space and the energy to innovate, to bring fresh perspectives and big ideas to the table, and to relate to a new set of clients in new ways. The commissioning and management of studies and field programmes is a part of this, but must not become the only end of the project. 8. Communicating: The communication of PPLPI output to its diverse constituency of clients cannot be passive, and it certainly cannot be assumed that academic papers and websites are reaching or influencing the right people and organisations. The development of a coherent communication strategy is very important. 9. Strategic engagement through pilot country initiatives: The Evaluation Team agrees, with some reservations, that the PPLPI should continue to invest in pilot projects in focus countries. There remains a need for the project and its staff to establish credibility ("proof of concept") with member nations and to be able to speak from field experience when affirming that policy assistance can make a tangible difference to poverty reduction. It follows that this field work must feed into wider policy processes at national, regional and international level.
Main Recommendations 10. At the cost of some repetitiveness, the full set of recommendations from the final section of the report is presented below.
Refocusing the project purpose/objective on strengthening FAO capacity 11. The most important overall recommendation of the Evaluation Team concerns the need to build on the strong body of experience and outputs accumulated over the past two and a half years and concentrate more deliberately on developing FAO/AGA's capacity to provide pro-poor livestock policy support to member nations and international organisations. 12. As a first action, AGA should sharpen the focus of the project's objective, as articulated in the Purpose statement of the log-frame. The Team recommends that AGA hold a professionally facilitated strategic planning and logframe revision workshop in this context (para. 36). The Evaluation Team is convinced that the achievement to a significant degree of the project's overall goal of poverty reduction through improved policy making in the livestock sector can only be reached in the long term by focusing more narrowly in the short term (life of the project) on strengthening FAO's capacity to provide effective support. In this context, the OVIs should all be reconsidered and amended appropriately, including recognition of the approved one-year extension to September 2008 (para. 46). 13. Suggestions as to how the project can contribute to strengthening FAO capacities include such initiatives as organising/sponsoring joint field visits both to PPLPI focus countries and to other AGA (or FAO) field activities; joint planning, proposal preparation, reviewing/auto-evaluating including bringing non-project staff into the project implementation process; reciprocal peer reviewing and discussion of each other's work; joint participation in international and regional meetings, workshops or training opportunities on related topics, and other means as determined during implementation. (para. 42)
vi 14. Ensuring the sustainability of acquired project knowledge and experience, as well as databases, information systems, communications and other strategies, will also involve gradually transferring responsibility for PPLPI work to more permanent staff, e.g., through redefining existing AGA posts (ongoing to some extent), co-funding posts with the RP, seeking to place project staff on vacancies that open up in AGA, etc. (para. 43)
Improving project governance and providing an enabling environment 15. The second most important recommendation concerns the environment in which the PPLPI (one of AGA's major investments) operates in AGA and in FAO. Management of this environment, which must be as receptive and supportive as possible, requires decision-making and configuration of resources beyond the project itself and beyond the Project Coordinator's mandate and ability to influence. The original project memorandum foresaw a "management task force" drawn from various parts of AGA and elsewhere in FAO but this was tried only briefly. (paras. 81-84) 16. This is a governance issue that the Division Director needs to tackle and he is the appropriate person to decide. However the Team strongly suggests that it would be of benefit to create a small "Management Board," consisting of the Division Director, the three Service/Branch Chiefs of AGA, the two non-chief Programme Entity Managers and the Project Coordinator. Board meetings might also include occasional participation of one or two people of at least Service Chief level from elsewhere in FAO outside AGA but in closely related activities (such as the Chief, Agricultural Sector in Economic Development Service, and the Director, Policy Assistance Division). 17. Such a Board would meet no more frequently than once every two months. Its job would be to support translation of project progress into 'impact', in particular impact on AGA. The concept is that these meetings should serve as 'mini-retreats' for reflection and discussion of strategic issues in the evolution of AGA and the project's role in that evolution. They should not resemble "management meetings" in the current sense of AGA's weekly management meetings. 18. The Management Board will need clear Terms of Reference (see suggested elements for this in Annex II) which define its role in creating and supporting the environment to promote the spread of knowledge, attitude, analysis and experience emerging from the project within AGA and FAO. 19. While budget-holder and day to day management responsibility would remain with the Project Coordinator, the Board could, if appropriate, provide approval of overall annual workplan and budget allocations. The Team also suggests that the Management Board might have direct control over a small part of the project budget (say 10% of remaining funds) to allocate to activities directly contributing to the exchange and transfer of capacity between the project and FAO's regular programme staff and activities like those suggested in the previous recommendation. 20. One specific issue which may benefit from the Board's attention: There is a fine line between achieving integration and mainstreaming of headquarters-based ('normative') project initiatives, and use of such a project to subsidize regular programme work. The Evaluation Team has some concerns that that line may be being crossed at times, and encourages AGA management (through the Management Board) to keep this under review. (para. 119)
Working with other organisations: global mandates and complementarity 21. The Team recommends continued activity by the project in the area of global decision making on livestock development and poverty, and in particular (para. 111): PPLPI should continue to commission and publish studies on international trade and other PPLPI issues relating to international public goods. The Animal Health Service (AGAH) should take the initiative in raising pro-poor issues in OIE in close consultation with the PPLPI. PPLPI should invest more time and resources in relationships with key international institutions (well identified in the project document) – both directly and through the mandated
vii interlocutors (for example the FAO Investment Centre [TCI] in the case of the World Bank and AGAH in the case of the OIE). PPLPI should examine the possibility of increasing its influence at the international level by appropriate briefing of civil society organisations and the mass media, that have an increasing influence both on developed countries' development aid and on trade policies. Substantial indirect influence may be achieved in this way. As part of the process of clarifying its objectives and mandate, PPLPI should convene reasonably high level meetings of the development agencies and sector institutions now working on livestock-sector policy and standard-setting, and on policies relating to livestock's contribution to poverty reduction (OIE, World Bank, IFPRI, ILRI...), to examine and better define institutional mandates, competencies and comparative advantages and, above all, to remove any duplication and competition for resources. (para. 37) A specific meeting (at least Director-level) should be convened by the PPLPI between the key international institutions involved in the World Bank's ALive initiative (WB, OIE, EU, FAO/TCI) in order to better define complementary roles in this particularly closely related activity.
Country and regional work 22. The Team supports continuation of the focus country activities, as learning activities. It is recommended that PPLPI formulate a more solid strategy for providing assistance to individual member nations consisting of direct participation of PPLPI in the policy formulation process, but predominantly focused on the vigorous dissemination of experiences and results through various means targeting particular information to different audiences, and in particular audiences within FAO/AGA. These could include publications and articles, but also visits, workshops and seminars, joint activities with others, etc. This is closely related to the recommendation relating to the need for a communications strategy. Of course, this recommendation also applies to exchange of experiences between focus countries. (para. 96) 23. The PPLPI needs to take a clear position on the future of the concept of regional hubs, which is at present confused. If this approach is to continue, then a specific strategy will need to be developed. (para. 101)
Implementation issues 24. Several recommendations refer to specific implementation issues. These include the following: 25. The livestock-dependent poor are very often women. The Team was surprised by how seldom women or gender more generally were present in discussions of PPLPI and in the documentation, both in Rome and in the focus country visits. As a general recommendation, the PPLPI should give more attention to gender issues. Greater gender balance in staff and the SC might help with this. (para. 46) 26. As Output 1 (developing a portfolio of policy interventions) is currently defined, it is not really possible to assess progress. The log-frame needs to be revised to incorporate both the need to build up a body of knowledge and the need to systematise the policy options, e.g. in terms of objectives and instruments - as is now being worked on by PPLPI. (para. 49) 27. Under Output 2 (increased awareness of the potential of PPLP), the Evaluation Team recommends that the emphasis on dissemination in the original project log-frame be restored. The PPLPI should focus on development of a Communications Strategy, as this will necessarily be a key element of future work in informing the policy process. It should also be clear that the stakeholders concerned are not just those of the focus countries. (para. 57) 28. With regard to Output 3 (effective information and decision support systems), the Team recommends that the Steering Committee consider whether the number of decision-making tools being simultaneously developed is too large for PPLPI adequately to supervise and/or to test and advise on. Early emphasis should be give to the question of M&E indicators where little progress
viii seems to have been made. Also, someone unconnected with developing the systems needs to do some simple "market research" to see what kind of systems are in practice "demanded" (and likely to be used); and what difficulty, if any, users have had in getting acquainted with the PPLPI tools. (para. 61) 29. The activities under Output 4 (effective stakeholder representation) are behind schedule. PPLPI needs to rethink its approach in ensuring stakeholder representation and to equip itself with adequate skills in institutional matters. Serious consideration should be given to recruiting an Institutional Development Specialist as was originally foreseen. Only by doing that is the PPLPI likely to achieve the quality and quantity of activities needed under this Output. (para. 66) 30. It is highly recommended that PPLPI pursue the establishment and development of a global network (and possibly regional networks) in support of pro-poor livestock policy, as foreseen in the project log-frame. (para. 111)
ix I. INTRODUCTION AND CONTEXT
1. Overall, the Mid-Term Evaluation Team was impressed with the quantity and the quality of the project activities during the first half of its implementation. This work has included significant studies and working papers which have formed the basis for dialogue with national, regional and international partners. The five pilot country programmes, though still new, are already producing interesting outputs which are drawing attention in the countries to pro-poor livestock issues. Through these various activities, the Pro-Poor Livestock Policy Initiative has been contributing to the shift in emphasis in AGA towards recognising the importance of policy support to strengthen technical work for food security and poverty eradication. 2. The Team feels strongly that it is now on this latter area, of strengthening FAO's capacity, that the project needs to concentrate during its remaining three and a half years of implementation. The project, as it now stands with the experiences and materials accumulated over the past two and a half years, has strong potential for achieving sustainable impact during its remaining life. Whether it does so will depend in large part on its ability to build strongly on its past to focus on a clearer strategy for the coming years.
A. Scope and Methodology of the Evaluation 3. This Mid-Term Evaluation examines progress towards the achievement of the higher-level objectives of the DFID-funded FAO project GCP/INT/804/UK: "The Pro-Poor Livestock Policy Initiative" (PPLPI). These objectives are ambitious and imply wide-ranging impact within FAO and externally. Given the long-term nature of the project in an evolving policy, institutional and development environment, the evaluation was, in particular, to reflect on the assumptions made about FAO's evolving role and comparative advantage, and about the external players, forces and influences that constitute the programme’s raison d’être. 4. The Evaluation Team assessed the continued relevance of the project as designed to development priorities and needs; the design of the project log-frame's Purpose and Outputs, including later modifications; Quality and adequacy of project design, including consideration of assumptions and risks as compared to actual evolution of the context; Efficiency and adequacy of project implementation; Project results, including outputs produced to date and progress towards achieving the log-frame Outputs, and looking particularly at (1) the status and quality of work on the interface between supra-national policy analysis and national policies and programmes, including the utility of the proposed ‘Hubs’ and sub-regional programmes and (2) progress on the objective of supporting FAO’s institutional development; The prospects for the new 'ways of working' introduced by the project to producing a sustained impact by the end of the project period. In particular, the Team considered: 1. the extent to which PPLPI’s staffing and work-stream is being, or will be, incorporated into FAO’s Regular Programme of Work and Budget; 2. the extent to which the implied shift in emphasis and direction conflicts with or supports other work-streams, priorities or mandates within FAO; and 3. progress in securing wider donor buy-in to the programme, in particular the Regional Hubs; The cost-effectiveness of the project, and in particular, whether the extra-budgetary support is being used as intended to promote and cover the costs of developing capacity, and not subsidising business as usual.
1 5. Based on this analysis the mission drew specific conclusions and makes proposals for further action by FAO to ensure sustainable impact, including the revision of the logical framework to clarify objectives. The mission also reflected on and reports the main lessons that can be learned from this project to date. 6. A team composed of Peter Bazeley (representing DFID), Stephen Sandford (independent senior consultant) and Daniel Shallon (FAO Evaluation Service, Team Leader) assembled in FAO headquarters in Rome on 10 March 2005 for initial briefings. Between then and 14 April 2005 (with an interruption 20-28 March), the Team had extensive consultations at FAO Headquarters, visited two of the Initiative's five pilot countries (Uganda and Peru) and spoke with officials at the African Union and its Inter-African Bureau for Animal Resources (AU/IBAR), the International Livestock Research Institute (ILRI), the World Organisation for Animal Health (OIE), the European Commission, the World Bank and USAID. In addition the Team met with government officials and NGO staff working in livestock and policy in Uganda, Kenya, Ethiopia and Peru. 7. Overall, the Team had detailed interviews with over 80 individuals in eight countries in four regions (see itinerary and people met in appendix). The Team also reviewed a large amount of documentation produced by the project and by its partners. During the last week of work, a workshop was held with project staff to discuss the Team's preliminary conclusions, and follow-up meetings were held with many of the earlier interviewees. A draft Aide-mémoire was produced and discussed at a wider meeting with staff from various parts of FAO. As this final discussion could not be attended by the donor (DFID) nor the Director, AGA, a follow-up day of meetings was arranged on 5 May 2005 to discuss the detailed comments provided by AGA on the Aide-mémoire. This report is the result of all this process.
B. Origins of the PPLPI and Evolution of the Context and External Environment, 2000-05
Origins of the Project 8. The PPLPI was conceived (initial discussions took place in 1999) at a time when AGA's priorities were evolving. Senior management in the Agriculture Department (AG) and the Animal Production and Health Division (AGA) were keen to reorient the organisation's Livestock Programme to achieve a better balance between policy and technical work, and to place greater emphasis than before on social equity objectives. 9. At the same time DFID was seeking with renewed vigour, and additional resources, more effective instruments and approaches to the reduction of poverty on a global scale. There was recognition of the need to address the general failure of small-scale technical projects to have significant impact on poverty reduction in the absence of wider policy and institutional reforms that create a better environment for sustainable social and economic development. DFID also recognised the uniquely important role that mandated international organisations with normative functions can or should play in informing polices and institutions from an expert but neutral perspective, especially in a new era of globalisation and increasing influence of supra-national policies, institutions and processes on poor people's livelihoods. 10. Recognising the complementarity between the re-orientation of thinking in (this part of) FAO and DFID's interests in improving the effectiveness of supra-national policies and institutions, DFID invited AGA to submit proposals for extra-budgetary support. Multiple proposals were generated from different parts of AGA and beyond. That process of generating and prioritising proposals caused considerable debate within the Division, much of which was positive but some of which led to polarisation and division. Not everyone agreed with the approach being pursued, and there were losers as well as winners when the funding was finally approved some two years later. Notwithstanding these tensions, AGA was resolute in pushing ahead with its vision of an enhanced policy and institutional capacity, and DFID supported this.
2 11. AGA's proposal was influenced by its work on the widely respected Livestock, Environment and Development (LEAD) initiative, which has mobilised donor funding to research and analyse policy and institutional issues surrounding the 'Livestock Revolution' and the management of the interface between livestock production and the global and local environment. But the PPLPI was to go further than this. It was to explore and inform1, within the context of FAO's normative mandate, the much wider policy and institutional forces that determine poor people's livestock-related livelihoods, recognising the changing nature of these forces and the multitude of actors and stakeholders. PPLPI was founded on the assumption that international normative organisations like FAO, along with the international private sector and civil society, are very influential in determining - directly or indirectly - the lot of the poor livestock keepers. Thus the PPLPI was about developing new approaches and ways of working in FAO/AGA to support its work with governments and international institutions in the long term. 12. Given this supra-national focus, DFID's interest was principally in establishing this capacity at FAO headquarters. But given FAO's emphasis on decentralising its work and increasing its impact at country level, AGA was also keen to build regional hubs and country programmes into the PPLPI. Although FAO's proposals for a larger-scale project were considered by DFID, they were not ultimately funded. FAO would instead seek funding for the non-headquarters work from other donors. 13. It is important to recognise how, at the time, PPLPI represented innovation and a departure from 'business as usual'. Few other organisations or donors were championing such a policy and institutional approach to development within the sector, and indeed, not all such agencies were entirely convinced that the refocusing of development efforts on poverty reduction at scale was much more than fashion or rhetoric. There was certainly not unanimity that FAO had a crucial role to play, or if it did whether it could rise to this new challenge. All the other key 'livestock donors' and agencies were consulted and involved in project identification and design but none, other than DFID, initially subscribed to the PPLPI.
Evolution of the Context Five Years Later 14. The project's internal and external environments have moved on since inception, and the project has both contributed to and been affected by these shifts. It is important that, mid-term, the project reflects on those changes and adapts.
Externally 15. Over the past five years, the international community has committed itself to achieving an impact on global poverty at an entirely different scale, with the Millennium Development Goals (MDGs) as the benchmark. While institutional inertia persists in the development community, and while we are certainly yet to see the level of progress needed, there can be no doubt that 'business as usual' will not be enough, and that major international development organisations such as FAO will need to respond robustly and radically, or risk being sidelined. PPLPI is well placed to be at the centre of such change and to help guide it, but it must reflect on, and relate to, how other institutions are also responding. 16. Shifts in four different sets of external institutions appear to be especially significant for PPLPI: i. A new institutional architecture is emerging in many developing regions, and in particular in Africa, in spite of obvious capacity constraints. The significance of these changes is that they are likely to cause a sea-change in the ways in which the international community supports Africa's development efforts. Much wider and stronger political forces are now behind the need to do things differently in Africa. The increasing importance of the Regional Economic Communities (RECs), for example, is a reflection of greater political will to promote regional integration and manage regionalism as a key element of African development. The development and growing acceptance by the donor community of the NEPAD framework ("New Partnership for Africa's Development") underlines the decision to manage African development policy from within Africa. The PPLPI must surely be a key element of FAO's response to these changes. 1 See Footnote Error: Reference source not found on the meaning of "to inform".
3 ii. Other international organisations are making their own adjustments to the new environment in the livestock sector, but the PPLPI isn't always part of the equation. Notable with respect to livestock and poverty is the World Bank's efforts to re-establish a more tangible presence in the sector, focussing on livestock sector policies with a poverty focus through its newly launched "African Livestock Platform" (ALive). Likewise, the World Organisation for Animal Health (OIE) is reaching out much further than it ever has before to influence development in the context of its trade and veterinary public health mandates. The International Livestock Research Institute (ILRI) has also greatly strengthened its own focus on livestock policy as a means to support poverty reduction. But while other organisations are now touching some of the same hotspots as FAO/PPLPI, there has been relatively little dialogue between the organisations, and the resurgence of institutional interest in the livestock sector is in need of an analysis of mandates and comparative advantage. iii. More broadly, the donor community is reflecting on the effectiveness of international governance and the role of international organisations in bringing about widespread, sustainable, social and economic development. As time is running out on the Millennium Development Goals, international organisations are under increasing pressure to demonstrate their impact, focus on comparative advantage and pitch their efforts at the points of greatest leverage. FAO is no exception, and PPLPI can play an important role in forming its future direction in this respect. iv. Finally, the extent to which the behaviour of the private sector and civil society shapes poor people's livelihoods, and the distinctly different way in which that behaviour is governed, is only just beginning to be addressed. How an inter-governmental organisation should address that fact is not a matter for this review, but it certainly calls into question the extent to which, for example, Ministries of Agriculture should to be PPLPI's most influential client.
Internally in FAO and AGA 17. There has undoubtedly been a remarkable shift in overall balance of work within AGA. PPLPI has both contributed to and been affected by that shift. A distinct shift in the balance between technical and 'policy/institutional' ('rules of the game') work is evident in the Medium Term Plan and the Programme of Work and Budget, in the Management Response to the recent external evaluation of elements of AGA's work, and in the reconfiguration over time of the Division's staff establishment. 18. There are varying views as to the extent to which this shift is attributable to the focus on policies, or to the resources, that PPLPI has brought to the table, both within FAO and externally. Some people credit PPLPI with having had considerable influence. 19. But the integration of PPLPI into the mainstream of AGA's (and the wider FAO's) other programmes has only been partial. The Evaluation Team was repeatedly confronted by the perception of interviewees that the PPLPI is not sufficiently integrated into the rest of AGA in particular, and into other parts of FAO in general. There are a number of reasons for this situation, many dating back to the project's long and difficult conception process whose deep divisions have not been completely put aside. The overall effect is to lessen the effectiveness of the PPLPI. The Director, AGA, is working to strengthen this integration and his efforts have already met with some, but not enough, success. 20. The debate about the balance that FAO as whole should assume between 'normative' and 'operational' work continues. Developing member countries tend to place more emphasis on the value of operational work than do many of the donor countries. While not denying the value of technology transfer, some donors are challenging how, in practice, small-scale field interventions can best be designed and implemented so that they do genuinely drive, support and inform the normative work of the organisation. The PPLPI has a role to play in clarifying the persistent dichotomy between the need for FAO's work to be grounded in field work while not compromising its unique position and comparative advantage as an intergovernmental organisation mandated with key normative functions.
4 II. ASSESSMENT OF PROJECT OBJECTIVES AND DESIGN
A. Evolution of the Project Design and Log-frame 21. Project design grew out of several workshops and consultations with other agencies and donors (the London workshop of September 2000 in particular) and with regional institutions and stakeholders in Asia, Africa and Latin America. In all, project preparation took over 18 months and was characterised by intense and protracted debate within AGA. The decision was made to proceed, but a considerable degree of design detail, including the refinement of the log-frame, was left to the inception phase.
Early Phases of the Project 22. The full project was signed in September 2001. It was preceded by a one-year separately funded preparatory phase (GCP/INT/778/UK - originally 6 months from Sept. 2000, then extended to 12 months). Even with the preparatory phase, the project got off to a slow start. Recruitment of suitable project staff proved difficult - both in terms of identifying suitable candidates and because of the complexities of FAO's internal approval process. The project manager came on board four months after the signing date (ten months after the recruitment process began), and the project team was not fully recruited until nearly another year later. 23. The prescribed 'Management Task Force' was reduced in practice to a technical and advisory role, and never properly established itself before being disbanded as an inappropriate tool. The Steering Committee met for the first time only in June 2003. 24. Crucially, although some other donors have had limited collaboration in some of the country-level work, only one other donor, the EC, has shown real interest in supporting the concept of regional hubs.2 AGA argued strongly that a degree of national and regional work was essential to the proper performance of the PPLPI, and so DFID approved the reallocation of some headquarters funding to the establishment of pilot activities in East and South Asia, East Africa, West Africa and the Andes. The original agreement was that this would be on the basis of FAO securing at least matching funds from elsewhere. 25. The tensions that surfaced during project design within AGA continued to affect the project after its inception. To an extent, project staffing reflected the fact that it was not going to secure the full cooperation of all parts of the Division, with a tendency therefore for project staff to duplicate rather than complement existing AGA skill sets (i.e., by adding stronger specialisation in policy and institutions). Reportedly because of FAO administrative reasons ("a staff member at D1 grade cannot answer to another D1"), the project never came under the line management of the Livestock Information, Sector Analysis and Policy Branch (AGAL) as originally intended in the project design document. In spite of AGAL having championed the project as its own initiative in the development phase and of the project being formally part of the Branch, the PPLPI ultimately became a parallel and largely autonomous unit.
Revision of the Log-frame 26. The log-frame was revisited in an internal workshop prior to the first Steering Committee meeting. The reasoning behind these log-frame changes was not documented, but both the Steering Committee and the first DFID review were content to accept the changes as refinements and clarifications rather than any more radical redirection of objectives. However, the Evaluation Team has not found the log-frame particularly helpful: it still leaves much open to interpretation. 27. The substantive changes were as follows:
2 FAO and the EC subsequently developed a proposal for EC funding for a Horn of Africa hub, now expected to come on stream later in 2005.
5 Original design document: Amended during inception to: Purpose Purpose International and national livestock policies that ensure equitable, Strengthened capacity in FAO, member safe and clean livestock farming formulated and promoted. nations and international organisations to formulate livestock sector and related policies and implementation plans that reduce poverty, whilst managing environmental and public health risks Outputs Outputs 1. Identification, analysis and targeting 1. Portfolio of livestock-related interventions for reducing poverty through Improved characterization and targeting of the different strata of policy and institutional change livestock-dependent poor achieved;
Enhanced understanding of the key-factors and institutions that affect poor people's livestock-related livelihoods obtained and key entry-points identified;
Information, knowledge and decision-support needs of key stakeholders and players in livestock policy making and institutional change effectively addressed and serviced. 2. Formulation and negotiation of policy options 2. Increased awareness and consideration of the potential contribution of livestock and Evaluated standards, codes and pro-poor livestock policy options the livestock sector to poverty reduction (international and national) for different types of livestock-based livelihoods;
'Tailor-made' pro-poor livestock policy options and institutional reform proposals at national level for incorporation into poverty reduction strategy papers, livestock development proposals and sector development strategies;
Effective platforms established for countries to negotiate international norms, standards and policies that better support poor people's livestock-dependent livelihoods 3. Dissemination and scaling-up 3. Effective systems for livestock policy information, analysis, decision-support and Comprehensive understanding of the factors and institutions that M&E affect poor people's livestock-related livelihoods achieved among key organizations and players (public, private and tertiary sector) in livestock policy making
Broad awareness among decision-makers and other interested parties of key-factors and issues that affect poor people's livestock-related livelihoods. 4. Mechanisms established for effective stakeholder representation in the negotiation of policies and institutional changes that better support poor people's livestock- dependent livelihoods
B. Relevance and Justification 28. The Review Team considers the project to have been at the outset and to remain highly relevant both to the needs and priorities of today's development agenda and to FAO/AG senior management's vision of the future. The international community's commitment to achieving poverty reduction at scale and its alignment behind the Millennium Development Goals only serve to increase the appropriateness of the policy and institutional approach that is being championed. The recent external
6 "Evaluation of FAO's Activities in Livestock Production, Policy & Information" also came to the same conclusion. 29. Similarly, if supra-national policies and institutions are assuming an increasingly important role in determining poor people's livestock-related livelihoods, then the relevance of and justification for such a project being housed in the main intergovernmental organisation mandated with informing such policies and institutions remain very strong. 30. However, there are several questions which will continue to be relevant as implementation proceeds: How influential can FAO aspire to be in this higher-level arena? Can it acquire a sufficient reputation to ensure it always has a place at the table? Will supra-national policies and institutions continue to grow in importance? And will information and intergovernmental influence be enough to make them more 'pro-poor'? Are other organisations filling the 'institutional vacuum' that existed when the PPLPI was conceived, and making FAO's role less unique or reducing its comparative advantage? 31. While the project's objectives are ambitious, the Review Team does not think that they are unrealistic. But they do imply a need for continuing innovation, a willingness to forge new and strategic partnerships where real leverage on policy processes can be exerted, and for coherence and complementarity between FAO's various interfaces with international institutions.
C. Assessment of Current Objectives and Focus of the PPLPI
Narrowing the Project Objective (the Log-frame "Purpose") 32. The project objectives have been subjected to various different interpretations. Different degrees of emphasis have been put at different times, and even at different places within the Project Memorandum, on: building FAO/AGA's capacity to provide support/assistance in pro-poor livestock policy, strengthening the capacity of FAO, other international organisations and member states to formulate pro-poor livestock policies, or actual formulation and promotion of livestock policies. 33. As noted above, the "Purpose" given in the revised version of the project's log-frame ("strengthening the capacity of FAO, international organisations and member nations to formulate livestock sector and related policies and implementation plans...") is hugely ambitious if, as suggested, it is to be achieved at meaningful scale by 2008. 34. But the introductory text in the Project Memorandum is clearer and more realistic: "… the implementation of a Pro-Poor Livestock Policy Facility within the Animal Production and Health Division of FAO that develops the capacity to inform [i.e., have an influence on3] national and international policy-making in support of poor people's livestock-related livelihoods." The focus of this statement is clearly on strengthening FAO's capacity to better assist (in future) its member nations and to have a positive influence on the international livestock policy environment.4
3 The use of the word "inform" in this context has apparently led to some misunderstanding, with several people taking it to mean only "to provide information (to)." In English, the verb "to inform" used in a context such as this has the meaning (from the Oxford English Dictionary): To give ‘form’, formative principle, or determinative character to; hence, to stamp, impress, imbue, or impregnate with some specific quality or attribute; thus also: to have an influence on (something); for example: "These guidelines will be used to inform any future decisions."
4 The Project Coordinator has described a desired end of project situation as (paraphrased): "recognition of FAO/AGA as a point of reference and an objective source of knowledge, expertise and information, through a 'public goods lens’ (with respect to livestock’s contribution to poor people’s livelihoods); ability to make explicit the trade-offs between equity, social, health and production objectives; provision of a resource that would naturally be included in relevant planning processes; and having a seat at relevant negotiating tables." The Team is in agreement with him that these are appropriate medium-term aims.
7 35. While some ranging and scoping has been valid and useful in its early stages, the project now needs to be clearer about this objective, and to focus down. This process of focusing-down should take account of the developing external institutional environment and concentrate on the areas where FAO has a mandate and comparative advantage. 36. AGA should sharpen the focus of the project's objective (as articulated in the Purpose statement); the Evaluation specifically recommends that the project focus more narrowly on strengthening FAO's capacity to provide effective support to member nations and international organisations in pro-poor policy-making in the livestock sector. The Team recommends that AGA hold a professionally facilitated strategic planning and logframe revision workshop in this context. 37. As part of the process of clarifying its objectives and mandate, PPLPI should convene a reasonably high (Director) level meeting of the development agencies and sector institutions now working on livestock-sector policy and standard-setting, and on policies relating to livestock's contribution to poverty reduction, (OIE, World Bank, IFPRI...) to examine and better define institutional mandates, competencies and comparative advantages and, above all, to remove any duplication and competition for resources.
Strengthening Capacity in FAO 38. As the focus of PPLPI narrows more specifically to strengthening the capacity of FAO (particularly AGA), the question arose of what this capacity building implies. ‘Capacity’ has been taken by this Evaluation to include all of the following: The ability to understand, among policy-makers and policy advisers, the impacts of policy interventions (or lack of them) on the poor people’s livelihoods. This impact is dependent in part on: o Information of the right quality, quantity and form, based on theory, evidence and synthesis of experience; o A critical mass of like-minded opinion – from the research community, in government fora, in civil society and among lobby groups; o Identification and support of 'champions': reinforcement and support to changes that are going in the right direction. The ability to participate in and influence the process of policy and institutional reform wherever it is happening (nationally and supra-nationally, within government and beyond the sphere of government). This would require from FAO, among others: o Engagement – often with new institutions and organisations, including new inter- governmental fora, the private sector, international NGOs o A recognised seat at discussion and negotiation tables o Convening power o Credibility o Respected and informed neutrality. An environment in which the interests of the poor can be recognised in policy making processes. This means ability to convene and inform policy-makers’ fora and international standard setting processes with better quality, quantity and legitimacy of representation. The ability to build coalitions and strategic partnerships for change, across multiple organisations and institutions, respecting mandates and comparative advantage. The ability to resolve conflicting and disparate policy options through evidence-based analysis. The ability (through information and analysis) to monitor and evaluate the impact of policy reforms, and to feed the results back into policy processes; The human resources (mandate, skills, attitude, practice and numbers) to inform policy and institutional processes credibly and appropriately. The right people, given the appropriate responsibilities in the right environment.
8 39. PPLPI is not a training unit that already has all the answers. Rather it is a learning project involved as much in doing the analysis as in disseminating the results. Possibly the greatest strength which the project has already is that of a 'mind-set', an awareness and attitude which accords high importance to policy processes in using livestock as an entry point for poverty eradication. 40. As a learning project, the PPLPI therefore has three targets for its capacity building: itself first of all, then FAO technical staff working in livestock (mainly in AGA), and thirdly other relevant FAO staff and units, such as the Policy Assistance Division (TCA) and the units working on economic analysis, food standards and trade issues in the Economic and Social Department (ESAE, ESNC and ESCB). 41. Working more closely together, with shared ownership of outputs and outcomes, will facilitate the transmission of the 'attitude' to FAO/AGA's work which comprises: (1) a greater focus on poverty eradication and (2) a greater focus on policy change as an important means in achieving FAO/AGA goals. The first should be (and is) easier: FAO in general already has a poverty orientation in its work (though the PPLPI can sharpen this and bring it more to centre-stage). But in the second case, policy change, there is a strong potential for FAO/AGA staff to better understand (1) what it means and how it can be promoted or influenced, and (2) how it relates to their own work and goals. PPLPI staff should be able to contribute to building awareness in these areas. Of course as the project's body of experience grows, it may be in a position to provide more targeted assistance to colleagues in specific areas of decision making. 42. The project can contribute to strengthening these capacities and attitudes through such things as organising/sponsoring joint field visits both to PPLPI focus countries and to other AGA (or FAO) field activities; joint planning, proposal preparation, reviewing/auto-evaluating including bringing non-project staff into the project implementation process; reciprocal peer reviewing and discussion of each other's work; joint participation in international and regional meetings, workshops or training opportunities on related topics, and other means as determined during implementation. 43. Ensuring the sustainability of acquired project knowledge and experience, as well as databases, information systems, communications and other strategies, will also involve gradually transferring responsibility for PPLPI work to more permanent staff, e.g., through redefining existing AGA posts (ongoing to some extent), co-funding posts with the RP, seeking to place project staff on vacancies that open up in AGA, etc.
III. ASSESSMENT OF PROJECT IMPLEMENTATION
A. Implementation to Date 5: Assessment of Quality and Efficiency
Project Outputs and Activities 44. Performance of the PPLPI in terms of each output is reviewed in summary terms here. More details of the OVIs set and the extent each has been realised is included in Annex I to this report. The Outputs of the PPLPI project, as revised in 2002/2003, were as follows: 1. Portfolio of livestock-related interventions for reducing poverty through policy and institutional change, subsequently defined by PPLPI as comprising the following distinct themes: a) The role of livestock in economic development and poverty alleviation. b) Policies, applied instruments and the political economy of policy reform. c) Promotion of access to markets for livestock and livestock products. d) Promotion of access to livestock services.
5 The source for much of the data in this section of our report is PPLPI Mid-Term Project Report: Summary of Project Activities and Outputs, of April 2005
9 2. Increased awareness and consideration of the potential contribution of livestock and the livestock sector to poverty reduction
3. Effective systems for livestock policy information, analysis, decision-support and M&E
4. Mechanisms established for effective stakeholder representation in the negotiation of policies and institutional changes that better support poor people's livestock-dependent livelihoods 45. The Activities proposed in the log-frame as necessary to achieve the outputs are too many and too wordy to be reproduced in full here. Moreover they were specified in very general terms which makes it difficult to assess to what extent they were carried out (e.g. an activity under Output 2 was "develop and implement targeted public relations strategies"). The number of activities proposed are 15 altogether, three each for Outputs 1, 3 and 4, and six for Output 2. 46. Two general observations: The present log-frame, which was most recently amended in July 2004 in discussions between AGA and DFID, still assumes a 6-year project ending in 2007, while a one-year cost- less extension until late 2008 has been agreed. The OVIs should all be reconsidered and amended appropriately. The livestock-dependent poor are very often women. The Team was surprised by how seldom women or gender more generally were mentioned in discussions of PPLPI, both in Rome and in the focus country visits. Only three of the 20 Working Papers give serious attention to gender issues (6, 10 and the unnumbered paper by Riviere-Cinnamond and Eregae). As a general recommendation, the PPLPI should give more attention to gender issues. Greater gender balance in staff and the SC might help with this.
Output 1: Portfolio of livestock-related policy interventions for reducing poverty
Factual Review 47. The PPLPI report progress (in PPLPI Mid-Term Project Report: Summary of Project Activities and Outputs, of April 2005) under this Output in terms of the number of studies carried out, the number of countries or regions covered by these studies, the number of key findings emerging from them, and the number of PPLPI publications arising from them. The PPLPI had produced 49 completed studies at evaluation time (several more were in progress or planned). Of these, 5 are global studies, 3 are regional and 41 are country level. The number of key findings drawn in these documents which have the potential to guide policy development and reform were assessed at 54. The project has produced 29 publications, excluding internal documents but including PPLPI working papers and conference papers.
Evaluation Team's Observations 48. Much excellent and necessary work has been done in terms of studies of background (the role of livestock), issues and some policy options. However the question of choosing a "portfolio" consisting of "generic pro-poor livestock sector-related policy options" (wording of the Output) is only just beginning to be tackled in a systematic way (matching policy instruments to policy objectives in various contexts - indicated in AGA's memorandum of 22/04/05). The word "portfolio" is itself unhelpful. It does not occur in the original project memorandum nor does the Steering Committee itself ever use it, but it crept into the revised 2002 logical framework. The original memorandum has a table listing different policy options, but these are very general, e.g.: - Enable access to financial services through partnerships - Promote low-cost technologies. - Legislative and norm changes that enable resource control and empowerment It is quite unclear how these could be built into a "portfolio".
10 49. As the "Output" is defined, it is not really possible to assess progress. The log-frame needs to be revised to incorporate both the need to build up a body of knowledge and the need to systematise the policy options, e.g. in terms of objectives and instruments as is now being worked on by PPLPI.
Output 2: Increased awareness of the potential contribution of the livestock sector to poverty reduction
Factual Review 50. The PPLPI reports progress under this Output in terms of the number of country, regional, and global workshops (or equivalent) in which PPLPI has participated, the number of workshops it has co- organised, and the number of pieces of work in which it has collaborated with other services in AGA or with other divisions in FAO. Participation by PPLPI in workshops organised by other organisations 246 Regional workshops co-organised by PPLPI 7 National workshops co-organised by PPLPI 12 Collaboration between PPLPI and other services within FAO-AGA 10 Collaboration between PPLPI and FAO Divisions other than AGA 12 51. In addition to these activities, there are a large number of useful publications arising from the project. Twenty PPLPI Working Papers have been published (the first seven in 2003) and a further two are in advanced draft form. A rough analysis of the status of the authors suggests that, of the total number of authors, about 64% were from developed countries, 25% from developing countries and 11% students under supervision. About 25-30% appear (from names) to be women. 52. Of the 20 Working Papers, 11 are matched by briefing papers summarising the results of the Working Papers in lay-persons' language. Twenty of the Working Papers are published in English, and only one each also in French or Spanish. In addition to these "official" publications, at least as many other papers have been presented at conferences or published elsewhere by PPLPI staff or by PPLPI consultants or close partners. Two collections of papers have been published as books in India. 53. The Working Papers, the Briefing Papers and some of the workshop publications are available on PPLPI's website. The records show that about 5-6000 document downloads from this website occur monthly, and that the two earliest Working Papers (which have been on the website for a significant amount of time) have been downloaded on average about 13,000 times each. However, although we do not have "origin of visitors" figures for the PPLPI site we know from the data for visitors to all AGA information products that the majority of visitors come from North America followed by Europe, and that access by developing countries, and by Africa in particular, is small (source: Auto- evaluation Report on livestock information activities in FAO).
Evaluation Team's Observations 54. The OVIs set for this output are very unambitious and the PPLPI seem to be interpreting them in an even less ambitious way (see Annex I). They will probably be achieved. 55. In the log-frame in the original project agreement one of the "Outputs" was called "Dissemination and Scaling Up" and related activities included "awareness campaigns to inform general public" and "development of an electronic Pro-Poor Livestock Policies Network." The words "dissemination", "general public" and "networks" have disappeared in the revised log-frame. 56. What has resulted has been an "awareness" programme which has deliberately concentrated on hub-regions and focus countries, and on workshops as a major tool (reaching the rather limited audience that has the time and the money to attend workshops). The websites are in principle available to all. In practice in many, especially African, countries, use of the internet except by those in academic circles, is rather limited. The Evaluation Team finds that PPLPI is significantly curtailing its impact by the restricted nature of its awareness raising efforts.
6 In half of these PPLPI made distinct presentations of some aspects of its work.
11 57. The Evaluation Team recommends that this emphasis on dissemination be restored. The PPLPI should focus on development of a Communications Strategy, as this will necessarily be a key element of future work in informing the policy process. It should also be clear that the stakeholders concerned are not just those of the focus countries.
Output 3: Effective systems for livestock policy information, analysis and decision-support, and M&E Table 1: Analysis of activities under Output 3 1. Baseline 1.1. Quantitative data Livestock data have been collected for the 5 hub regions. More comprehensive data information has been collected for the focus countries and has been compiled within GIS databases in Vietnam and and data Uganda ."Living Standards Measurement Survey" (LSMS) data have been obtained for 6 countries. A Global Livestock Information System, in order to manage and process sub-national data, has been developed 1.2. Qualitative data Especially for focus countries, and for the 4 themes of Output 1, data have been collated relating to legislative structure, institutions, contact details for networks, and various other documentation. 2. Analysis 2.1. Analysis of household types using survey data (LSMS). Detailed analysis has been done for Nepal and Vietnam and less detailed for Uganda. 2.2. GIS analysis Preliminary GIS analysis, related to trypanosomiasis control has been done in Uganda and more extensive analysis in Vietnam 2.3.Agricultural production systems mapping The project, in combination with others, is developing agricultural production systems classification systems. 2.4. Poverty mapping High resolution poverty maps have been developed for Uganda and Vietnam. 3. Decision 3.1. EXTRAPOLATE (Ex-Ante Tool for Ranking Policy Alternatives) is a decision support tool to assess the support tools impact of different policy measures. It is being "field tested" by PPLPI in Uganda, India, Senegal and Peru. 3.2. IPALP (Integrated Poverty Assessment for Livestock Promotion) is a tool designed to improve general understanding about the role of livestock in poverty eradication. The methodology has been developed using data from Vietnam and will be applied in other focus countries. 3.3. Dairy household model TIPICAL V4.0. This model is listed by PPLPI in their summary of project activities and outputs, but no details are given about its actual use by PPLPI 3.4. Policy menus A "knowledge system approach" currently at an early stage of development by the project 3.5. Livestock policy analysis tables This approach (methodology and analytic tool) is aimed to assist in identifying policy gaps and testing for policy coherence. It is at an early stage of development. 4. Information A brochure outlining the project's objectives, rationale and activities has been produced and widely dissemination disseminated. All project outputs are made available to the public via the PPLPI web-site. This website is integrated in the AGA website. Livestock data and sector briefs generated by the project are disseminated via GLIPHA (Global Livestock Production and Health Atlas) and via FAO's GIS portal, GeoNetwork. 5. Monitoring The project is drawing up a set of indicators by which to monitor and evaluate the impact of policy reform. and evaluation
Factual review 58. PPLPI report progress under the headings set out in the table above. They cite 11 project documents in relation to this work, but only two of these are published, and four are marked as still being "in preparation". None of these documents appears to deal with monitoring and evaluation.
Evaluation Team's Observations 59. Again the Evaluation Team is very impressed by the number and variety of the systems being developed and tested in respect of information, analysis and decision support. Clearly a large and well-organised effort has been made in this field and much very good work has been done. 60. We have some concern that for many developing countries the systems being elaborated may be too sophisticated for existing analytic and decision-making resources and the result will be either failure to use the systems or excessive dependence on outside technical assistance. Most pro-poor livestock policy formulation and implementation takes place in unsophisticated environments where the actors are doing many different jobs simultaneously. This makes it difficult for them to find time to be trained in complex systems. Some alternatives should be presented.
12 61. The Team recommends that the Steering Committee consider whether the number of decision- making tools being simultaneously developed is too large for PPLPI adequately to supervise and/or to test and advise on. Early emphasis should be give to the question of M&E indicators where little progress seems to have been made. Also, someone unconnected with developing the systems needs to do some simple "market research" to see what kind of systems are in practice "demanded" (and likely to be used); and what difficulty, if any, users have had in getting acquainted with the PPLPI tools.
Output 4: Mechanisms established for effective stakeholder representation in the negotiation of policies and institutional changes
Factual review 62. The PPLPI progress report (PPLPI Mid-Term Project Report: Summary of Project Activities and Outputs, of April 2005) displays a list which "provides an overview of policy issues that have emerged in the selected focus countries (or states within countries) and in which PPLPI is assuming the role of facilitating the policy dialogue between different stakeholders…..." The list reviews progress in each of the focus countries under five headings: Policy Goal, PPLPI concerns, Partners, Process and Partnership/funding agreements. The project reports that "Policy negotiations are taking place in all 5 focus countries in various platforms, which appear to have found acceptance and in most cases include government participation or have government endorsement"; and this corresponded to the Evaluation Team's observations in the countries visited. 63. Table 2 below analyses the degree of participation of the various stakeholders in the mechanisms for negotiating policies and institutional changes in the programmes in which PPLPI is taking part in the various focus countries.
Table 2: Analysis of participants in mechanisms for effective stakeholder representation
Number of partners cited who are: No. of meetings cited* with representatives international, traders or small producers NGOs who can speak of traders, and /or Country Sector/ sub- government, or private service (e.g. pastoralist for livestock- private service suppliers sector academic orgs. suppliers associations) dependent poor and/or producers Vietnam Pig 8 0 0 1 1 Dairy 5 0 0 1 0 India L/stock (Andhra service 3 0 0 0 4 Pradesh) delivery Uganda Dairy 5 1 0 2 2 Nagana & try- 5 0 0 0 0 panosomiasis Senegal L/stock Sector 3 1 0 1 1 Sheep market 4 1 0 0 1 Peru Camelids 4 0 1 2 2 * "Cited" means mentioned in PPLPI's progress report or orally to the Evaluation Team. "Partners" are those cited in the PPLPI progress report as "partners" in the policy formulation and negotiation process.
Evaluation Team's Observations 64. "Effective stakeholder representation" is partly about ensuring that all stakeholders are involved in policy formulation so that their combined and expert knowledge can lead to a more efficient policy process. But it is substantially concerned also with ensuring that one group of the stakeholders, the livestock-dependent poor, have an opportunity at least to identify their priorities even if, for the time being, their negotiating position is weak and they are more effectively represented on negotiating platforms by NGOS than by themselves. The fora/negotiating platforms established seemed to be more oriented towards the involvement of government and development organisations (including bilateral donors, multilaterals and foreign NGOs). Rather little success has been achieved in securing
13 the involvement of the indigenous private sector (commodity traders and service providers, who can often sabotage new policies) or the livestock-dependent poor (the ultimate beneficiaries). 65. The PPLPI's statement about facilitating the policy dialogue (para. 62 above) suggests that the project may be attempting a higher profile than it can or should present. 'Facilitating policy dialogues' implies having high credibility and visibility, enabling one to act as facilitator between actors and as supervisor of a process. This is not yet the case for the PPLPI, nor should it necessarily become so: rather the project should be seeking to confer to FAO/AGA the capacity to take on such a role, where and when appropriate. 66. The activities under this Output are behind schedule. PPLPI needs to rethink its approach in ensuring stakeholder representation and to equip itself with adequate skills in institutional matters. Serious consideration should be given to recruiting an Institutional Development Specialist as was originally foreseen. Only by doing that is the PPLPI likely to achieve the quality and quantity of activities needed under this Output. In the report of their second meeting, the Steering Committee also noted the lack of progress in this area and have urged a special effort.
B. The Staffing and Structure of the PPLPI Project Team 67. The project agreement between DFID and FAO was signed in September 2001 and was expected to run for 6 years (subsequently revised to seven years) from October 2001 for a total budget of £ 9 million. The budget, as set out in the project document, was as follows: Category of expenditure % of total cost Salaries (six professionals and one administrative staff) 43% Consultants and contracts (e.g. with research institutes) 30% Travel and training 12% Other costs 15% TOTAL 100%
Table 3: Staff positions in PPLPI Originally No. and person(s) in post Entry on Specialisation foreseen in early 2005 duty date Coordinator-senior livestock economist 1 1 (Joachim Otte) 01/02/2002 Veterinary public health specialist/Animal Health Officer 1 1 (Katinka de Balogh) 01/09/2002 Livestock Development Specialist/ 1 1 (vacant after resignation 17/06/2002 Animal Production and Health Officer in 2003) Livestock Information Officer 0 1 (Tim Robinson) 01/07/2002 Livestock systems analyst (GIS and modelling) 1 0 (replaced by above) Information Analyst 0 1 (Pius Chilonda). 12/12/2002 Animal health economist/Livestock economist 1 1 (Achilles Costales since 07/2002 15/11/2003) (N. Morgan) Institutional development services specialist 1 0 (Candidate selected in 2002 but declined) "/" indicates the distinction between the wording of the forecast and the present title of the post
68. The Project Coordinator took up his post in early 2002, about 4 months after the signature of the project. In addition to the project coordinator, there are 5 professional posts in the central (Rome) facility of PPLPI. The comparison of the actual present composition, in terms of qualification or specialisation, of these posts with what was originally foreseen in the project document forming part of the agreement is show in Table 3 above 69. While the table seems to show a substantial divergence between the specialisation forecast and those actually in post, in practice the main difference is the absence of an institutional development specialist, since the two "information" specialists have GIS and modelling skills. In early 2005 of five staff members in post, one is a woman and four are men. 70. As foreseen in the agreement a project steering committee (advisory) has been established with a membership, excluding FAO staff, of between 10 and 12 persons. The composition of the Steering Committee includes mainly officials of international or government organisations (8, of which 2 from
14 developing countries), along with two NGO representatives and one academic. Only two members of the Steering Committee are women. 71. In addition to staff located and working mainly in the central facility in Rome, the original project document foresaw five "regional programmes" However these were neither precisely defined, in the sense that a wide range of potential activities were listed without much guidance on focus; nor was finance for them provided from DFID but, instead, was "hoped for" from other sources. 72. In practice PPLPI has evolved a rather different programme outside FAO headquarters, involving "focus country" rather than regional activities (see also the observations under section IV.B below). These countries - Vietnam, India, Uganda, Senegal and Peru - have been the location of nearly half of the 41 country-level studies7 carried out under Output 1, although some non-focus countries have also been the subject of such studies. It is in the focus countries that PPLPI has made strong efforts to get involved directly in the process of policy formulation. 73. Some limited wider "regional" activities have also been undertaken by the PPLPI focal points in the focus countries (again, see under IV.B below). In the context of these decentralised activities, PPLPI has employed, on a part-time or short-term basis, 4 individuals or organisations.8
Evaluation Team's Comments 74. As already noted in previous sections, in the area of human resources the project needs (1) to recruit an Institutional Development Specialist, and (2) to pay more attention to gender issues.
IV. KEY ISSUES AND FUTURE DIRECTIONS
A. Project Governance and Management Structures 75. All the information reviewed by the Team indicated that the day-to-day management of PPLPI is satisfactory, although it was not possible to investigate this in detail. There was, however, some concern over higher levels of governance. Too little attention, in our view, is being paid to strategic issues (medium-term achievement of the project 'purpose'). The Team is of the opinion that there is the need to create the right environment within AGA for the project to flourish and have the desired impact on the Division and on FAO more generally. 76. The problem of the project's low level of interaction and integration within AGA and with the wider FAO is noted above in the sections on context and on project design. For the PPLPI to have a long-term impact on FAO's ability to assist its member countries by "fostering the policy dialogue in support of equitable, safe and clean livestock farming," it must be fully integrated in and interact continuously with the rest of AGA and relevant other sections of FAO. These counterparts, willingly or under some sort of incentive, must also be receptive to and promote this integration. 77. As already noted, the PPLPI, although formally part of the Livestock Information, Sector Analysis and Policy Branch (AGAL), does not in practice receive supervision from the Chief of AGAL, but instead the Coordinator works autonomously and reports directly to an already overburdened Division Director. The Steering Committee is structured to provide general and longer- term guidance on the direction in which the project is moving, which is seems to do satisfactorily. It
7 Altogether 49 studies are listed in a table in the PPLPI Mid-Term Project Report: Summary of Project Activities and Outputs, of March 2005. These include those done on a country, regional and global basis, introduced by the remark "PPLPI has conducted a number of field studies in selected countries of the regional hubs, and literature reviews." Several of the studies have been done by other organisations: about 30 of the 49 studies were done principally by PPLPI staff or consultants.
8 Dr. Vinod Ahuja in the South Asia Region, IBAR (Inter-African Bureau for Animal Resources) in East Africa, Dr. Cheikh Ly in West Africa, and CONDESAN (Consorcio para el Desarrollo Sostenible de la Ecorregion Andina) in the Andean Region, none yet in the Mekong Region.
15 meets too infrequently, however, and is, for the most part, too external to FAO to be able to monitor or support the project management environment. This task must be resolved internally within FAO. 78. The Team finds the current situation to be unsatisfactory and a priority for change. In line with the ongoing efforts to instil a 'corporate culture' of outcome/results-based management in FAO, the Team feels that there is a strong need for a higher level of management of the PPLPI, at the 'Purpose' level of the log-frame. What needs to be better managed is the external environment in which one of AGA's major investments operates, and that requires decision-making and configuration of resources beyond the project itself, and, obviously therefore, beyond the Project Coordinator's mandate and ability to influence. Even if the project was doing all the right things, we have concerns about the lack of impact those outputs would have, because the environment into which they are being delivered is not as receptive or supportive as it could be - and so that environment needs to be managed such that it becomes more receptive and supportive.
The Need for a Management Board 79. The original project memorandum foresaw a "management task force" drawn from various parts of AGA and elsewhere in FAO which should have had this sort of function. However, the concept of a task force was tried (albeit in a limited "technical and advisory" role - see the Project Governance Progress Report of 2002) but discontinued because it was apparently too large, membership was not at an adequately senior level, and its members felt it was not a good use of their time. The PPLPI also felt it was not serving a useful purpose. 80. The suggestion proposed by AGA during this evaluation, to use the weekly AGA Management Meetings to serve this purpose (and avoid the creation of "another layer of bureaucracy"), is more focused on oversight, supervision and monitoring functions. Management of project activities is the Project Coordinator's job, and whether his management of activities and resources results in the right quality, quantity and timing of outputs is a certainly a matter for a 'supervisory' function, but that is one for his line manager (supported by the weekly management meetings if needed), not for a management task force - or Management Board - of the sort we are suggesting. Such a Board would bring a breadth of view to the business of governance which cannot be looked for in one person. It should also lead to a greater sense of ownership of PPLPI by other parts of AGA. 81. This is a governance issue that the Division Director needs to tackle and he is the appropriate person to decide. However the Team strongly suggests that it would be of benefit to create a small "Management Board," consisting of the Division Director, the three Service/Branch Chiefs of AGA, the two non-chief Programme Entity Managers and the Project Coordinator. Board meetings might also include occasional participation of one or two people of at least Service Chief level from elsewhere in FAO outside AGA but in closely related activities (such as the Chief, Agricultural Sector in Economic Development Service, and the Director, Policy Assistance Division). 82. Such a Board would meet no more frequently than once every two months. Its job would be to support translation of project progress into 'impact', in particular impact on AGA. In formal terms, this would support the project at its "purpose" level in the log-frame ('develop the capacity of AGA/FAO to inform national and international decision-making in support of poor people's livestock- related livelihoods'). The concept is that these meetings should serve as 'mini-retreats' for reflection and discussion of strategic issues in the evolution of AGA and the project's role in that evolution. They should not resemble "management meetings" in the current sense of AGA's weekly management meetings. 83. The Management Board will need clear Terms of Reference (see suggested elements for this in Annex II) which define its role in creating and supporting the environment to promote the spread of knowledge, attitude, analysis and experience emerging from the project within AGA and FAO and beyond. 84. While budget-holder and day to day management responsibility would remain with the Project Coordinator, the Board could, if appropriate, provide approval of overall annual workplan and budget allocations. The Team also suggests that the Management Board might have direct control
16 over a small part of the project budget (say 10% of remaining funds) to allocate to activities directly contributing to the exchange and transfer of capacity between the project and FAO's regular programme staff and activities.
B. Beyond FAO: Role of the Focus Countries, Regional Hubs and Work at Global Level 85. In view of the differing views encountered, the Evaluation Team throughout its work was attempting to clarify how the work in focus countries, in regions and with international organisations at global level fit into the general programme of PPLPI's work, i.e. what its specific role is and how that role helps to achieve the log-frame's purpose.
Work in the Five "Focus Countries" 86. Following presentation of the Team's Aide-mémoire on 14 April 2005, a consolidated AGA response was prepared. On the issue of the importance of the country-level work, it stated: "For the message about the importance of support to livestock policy making and poverty to be taken seriously, there is a need of a "proof of concept". We always thought that this was PPLPF's primary task, i.e. to develop, through pilot schemes, the appropriate messages and processes, that would provide evidence that a suitable combination of technical and institutional/policy innovations can indeed make a sizeable poverty alleviation impact. Without that evidence, the above statement about the "importance of policy support" will not go very far." 87. Even before looking into the question of 'pilot schemes,' the Team would like to comment on the stated need of a "proof of concept". This need is, in the Team's view, somewhat overstated. While there are a few pockets of resistance, it seems that the importance of (livestock) policy to poverty has already been largely accepted, due mainly to general shifts of opinion in the 'development world' about policy matters generally. Rather than 'proof of concept,' the country work serves to provide concrete examples to illustrate PPLPI advocacy for the new approaches, as well as providing an element of credibility in the eyes of many developing country partners. 88. The above comments make reference to the "primary" importance of pilot schemes. The Team was unable to find evidence in past documentation (e.g. the Project Memorandum submitted to DFID, the Project Description on PPLPI's website, or the minutes of the Steering Committee meetings) that pilot schemes were ever generally viewed as PPLPI's primary task except by some staff of project itself. In the "Activities" section of the original logical framework, testing of stakeholder mechanisms, platforms and policy options makes up only three of 14 activities mentioned, and in the revised log- frame there is no similar activity mentioned at all. 89. This is not at all to suggest that country level activities involving pilot schemes should stop. They definitely have a useful and important role. It is the Team's observation, however, that the AGA statement quoted above, the discussions we have held with some PPLPI members, and the low priority which seems to have been given to some other activities, suggests that the PPLPI has become too absorbed in the country-level work.
Need for a Strategy 90. The Team could not find a strategy document clearly explaining the role of PPLPI in assisting the five focus countries. Initially, one country in each region/sub-region was selected for inclusion in PPLPI's country programme primarily by the criteria of the probability of an early start-up and the existence of favourable conditions for positive outcomes. Within country, the strategy was stated to be for PPLPI to acquire adequate experience and expertise in the content of policies and in the process of policy formulation and then to gradually move "up the policy ladder" with the approach changing from one of PPLPI "trying new approaches" to "rendering assistance in policy formulation" by itself participating in various stages of policy formulation. 91. As noted in Section III above, the Team believes there may be an issue of PPLPI seeking to take on too high a profile in the country programmes. This question came up during the field visit to Uganda, and it relates to the relationship between the pilot work in the five focus countries and the function of capacity strengthening in FAO (see para. 65 above). A clearer strategy linking the
17 different activities of the project would help to focus resources where the project wishes to achieve the greatest impact.
Links Between Field Work and Headquarters 92. After several discussions it appeared to the Team that there is as yet a very weak link tying the country work to the wider PPLPI activities in Rome. As indicated by the Project Coordinator, and rightly so in the view of the Team based on the revised Purpose statement discussed above, the main importance of the country experiences is to strengthen FAO/AGA capacity ("learning by doing"). The country activities are of course also intended to produce positive outcomes for the country itself, as this is a necessary condition if FAO is to use them as a basis for legitimacy in other situations. But in the context of the PPLPI, the learning aspect is the primary goal of this work. 93. Given this fact, the Team was surprised not to find more systematic learning from the fieldwork taking place. Very few people if any are aware or informed of the focus country activities outside of the PPLPI staff, and more particularly that staff member directly responsible for that country. In addition, it was pointed out to the Team that in spite of the obvious potential for immediate cross- fertilisation already available, there is little or no information exchange between the focus countries themselves. 94. A further area for improvement appeared to be for PPLPI to widen the country work to take in the experience of other organisations as well as its own as input to its work. The concentration of the project's experience gained in the current way will be too narrow to cover the range of circumstances in which policy formulation takes place.
Relative Importance of Country Activities 95. If the PPLPI is to focus more on developing FAO's capacity to support pro-poor policy-making in the livestock sector, then it also needs to be clear about the utility and relative importance of operational work at country-level in the project work programme. The relatively small number of pilot policy interventions being conducted at country level, while useful and of high quality, will in themselves not have impact at sufficient scale to justify the PPLPI investment, nor will they provide for the scale of change required to contribute usefully to the MDGs. They have to be part of a wider process of learning about, and having an influence on, policy processes at greater scale. The lessons deriving from these experiences need to be properly articulated and widely circulated with a vigorous campaign of dissemination of results both of PPLPI's own experience in the policy formulation field and of that of others. 96. The Team supports continuation of the focus country activities, as learning activities. It is recommended that PPLPI formulate a more solid strategy for providing assistance to individual member nations consisting of direct participation of PPLPI in the policy formulation process, but predominantly focussed on the vigorous dissemination of experiences and results through various means targeting particular information to different audiences, and in particular audiences within FAO/AGA. These could include publications and articles, but also visits, workshops and seminars, joint activities with others, etc. This is closely related to the recommendation in para. 57 above on the need for a communications strategy. The present recommendation applies to exchange of experiences between focus countries as well.
Regional Hubs and Work at Regional Level 97. There seems to be some uncertainty about the possibility of establishing other hubs besides the Horn of Africa one, with the Project Coordinator indicating that the days of actively seeking high levels of funding for hubs were past. The project has now decided to concentrate attention and efforts on the five focus countries rather than hubs. However the Team observed that there remains misunderstanding, both in the project team (at global and at country levels) and outside, with regard to future intentions in this area. 98. In spite of lack of funding, "proto"-hubs have been started on a very limited scale in West Africa, the Andean countries, in Southeast Asia and, though now apparently dormant, in South Asia.
18 Normally physically situated in one of the focus countries concerned but serving the other countries in that focus country's region, the "regional hub" is primarily concerned with raising awareness of pro- poor livestock policy issues. This has involved some regional studies, i.e. livestock sector reports in the Mekong, East African and West African regions, and regional meetings and workshops. 99. Particularly in Africa, there was discussion about the advantages of working through the Regional Economic Commissions (IGAD, SADC, UEMOA, etc.) as an effective entry point to support a pro- poor orientation in sub-regional policy agreements among neighbouring countries in the livestock sector. This may also be the case in other regions (as with the trade groups in Latin America, or ASEAN and it specialised committees in Asia), but it was most evident in Africa due to the rapidly rising importance and role of these RECs in sub-regional policy making. 100. PPLPI should (and does) recognise the importance of these institutions as potential fora for support to countries and to supra-national processes. The Informal Meeting of African Institutions on Livestock Policy Making for Africa which the project held in March 2005 is an example of a very positive initiative which was centred on this type of institution. The Team fully supports this approach. 101. The PPLPI needs to take a clear position on the future of the concept of regional hubs. If this approach is to continue, then a specific strategy will need to be developed.
Work at Global Level and with International Institutions 102. A key assumption in design of the PPLPI was that "the livestock / public goods interface is often determined by … policies, institutions and processes that require international agreement and cooperation, or are subject to international norms, conventions and protocols that need to be informed from a regional or global perspective." Hence FAO's comparative advantage in implementing a project of this kind - as the international organisation mandated with this function for agriculture. 103. During its first years, the PPLPI has tended to give more emphasis to its country-level work than to its interface with international institutions. This is where most demand has been, and there is a view that this is where it will have most impact. However, in deflecting the focus from the role that FAO could and should have in providing input to the debate at the international level, the PPLPI appears to the Team to be missing an opportunity, as well as questioning one of the project's fundamental assumptions. 104. FAO, through its Animal Health Service, now has good relations with OIE and some potential influence on its deliberations. It has not, so far, raised PPLPI issues at OIE, although FAO's mandate and commitment to the MDGs would make this appropriate. 105. FAO has rather little influence, with respect to pro-poor livestock policies, with other international agencies (notably the World Bank) or the big bilateral donors (other than in terms of funding PPLPI activities). This may change as FAO's expertise in PPLPI issues increases and its credibility in such forums is established, but this process is likely to be slow and may not progress very far: they are self-confident organisations which tend to rely on their own analyses. 106. FAO has few means of influencing the organisations in rich countries which represent those countries in trade negotiations or (except for OIE) standard-setting. This is unlikely to change. 107. The Team recommends continued activity by the project in the area of global decision making on livestock development and poverty, and in particular: PPLPI should continue to commission and publish studies on international trade and other PPLPI issues relating to international public goods. AGAH should take the initiative in raising pro-poor issues in OIE and should consult PPLPI about these. PPLPI should invest more time and resources in relationships with key international institutions (well identified in the project document) – both directly and through the mandated interlocutors (for example TCI in the case of the World Bank and AGAH in the case of the OIE).
19 PPLPI should examine the possibility of increasing its influence at the international level by appropriate briefing of civil society organisations and the mass media that have an increasing influence both on developed countries' development aid and on trade policies. PPLPI may achieve substantial indirect influence in this way. As recommended above in para. 37, PPLPI should sponsor a high-level discussion meeting between the key international institutions involved in the World Bank's ALive initiative (WB, OIE, EU, FAO/TCI) in order to better define complementary roles in the field of policy change for livestock to support poverty eradiction.
Creating a Strong PPLP Network - a neglected strategy 108. The concentration on pilot projects as the primary task is taking the resources of PPLPI away from other activities and resulting outputs. A specific and important example is in the original project memorandum (on pp. 10-11 and 17-19): the establishment of "a network or networks." One of the activities in the log-frame is "Electronic Pro-Poor Livestock Policies Network developed;" another activity in the revised log-frame is "Establish systems for dissemination." While there has been considerable effort made in developing AGA websites, no significant development of an electronic network to facilitate discussion of livestock policy issues has occurred. 109. PPLPI's primary purpose (as recommended also by this evaluation) is strengthening the capacity FAO so that it can better support member countries to formulate policies and implementation plans. Many more countries than can be reached by direct involvement in policy advice, and more organisations within any one country, can be involved if FAO undertakes the development of an international livestock policy network to enable that learning. It should be an information network with newsletter, distance-discussion forums (fora), and regional workshops. Such a network would have, as one of its objectives, dissemination of AGA's own results; but it needs to be much more than a place where FAO can preach. In practice people learn more from hearing from their peers (peers in terms of topic, type of employment, region, academic discipline and culture) about their experiences, and discussing these with each other, than they do from someone seen as an expert outsider. 110. FAO has a strong comparative advantage in establishing such a livestock policy information network. It has country offices with excellent in-country contacts to enable the drawing up of a "mailing-list" of suitable network members actually or potentially involved in livestock policy in each country. It has funding for the next three-and-a-half years that will enable PPLPI, by commissioning studies from developing country academics or consultants, to build up expertise among researchers- authors in the developing world rather than the developed. It has, through its working papers, information work and work in focus countries, a bank of its own experience to feed out to the network until the members of that network gain the experience and confidence to present their own to the other members. 111. It is highly recommended that the PPLPI establish and develop a global network (and possibly regional networks) in support of pro-poor livestock policy.
C. Cost-effectiveness and Use of Extra-budgetary Funding 112. The Evaluation Team is not seeking to make a quantitative cost-benefit analysis: it would neither be possible nor meaningful. However, qualitatively, there are important principles that need to be considered. The substantial investment in the project (£9m, or about $12m) was justified on the basis that it would create the capacity to leverage pro-poor outcomes in policy and institutional reform at a scale that makes a significant contribution to poverty reduction, in this case through equitable, safe and clean livestock farming. 113. The test, in terms of cost-effectiveness, is not therefore one of establishing whether the project's activities produce benefits or not - no doubt they all do - but whether they produce the kind of benefits that will, ultimately, cause or contribute to change at significant scale.
20 114. Closely related to this is the issue of comparative advantage: maximising cost-effectiveness by building on the organisation's unique mandate, position and strengths, rather than duplicating the efforts of others or engaging in activities that others may be in a better position to accomplish. 115. This raises a key issue in terms of the project's cost-effectiveness: the extent to which its studies and field activities are, in real and measurable ways, feeding into wider policy and institutional reforms through FAO's normative functions (or, indeed, whether FAO has a seat at all the right tables to achieve that). 116. The issues of the project's integration into the wider work of FAO, and of FAO's integration into external policy processes, are discussed elsewhere in this report, but are mentioned here because this is such a critical part of the project's logic: the project cannot possibly merely constitute a portfolio of interesting studies and small-scale policy interventions at country level. They have to be, and to be seen to be, part of a wider and higher-impact process of informing the policies and institutions (or policy and institutional processes) that determine the direction of poverty reduction much more widely. Although it may quite fairly be too early to tell, the Evaluation Team did not see overwhelming evidence that this is yet happening. 117. The fact that the PPLPI is a project with a finite lifespan that it is funded via extra-budgetary support9 also carries important considerations. Firstly, as discussed above, project funding of this kind can only be justified in transformational terms, i.e. the impact that it has in changing the way business is carried out. Were it to be valued in transactional terms, i.e. the value of the immediate, direct, benefits of its individual components (such as the country-level activities), then it could not possibly be seen as achieving enough. If the investment is justified because of its transformational properties, then the extra-budgetary resources must be targeted at achieving that change more quickly and robustly, and at institutionalising (mainstreaming) the change. 118. Were extra-budgetary resources merely to be used to cross-subsidise or enhance Regular Programme activities, then (a) the transformational impact of the investment would be minimal and (b) serious issues of sustainability would be introduced. 119. There is a fine line between achieving integration and mainstreaming of headquarters-based ('normative') project initiatives, and use of such a project to subsidize regular programme work. The Evaluation Team has some concerns that that line may be being crossed at times, and encourages AGA management to keep this under review. 120. Extra-budgetary resources can – almost certainly do – have a distorting effect on the balance of the work that FAO undertakes and may be accused of undermining the Programme of Work and Budget approved by the Governing Bodies. However the Evaluation Team found no evidence that the extra-budgetary resources were being used to support anything that was not already an explicit priority within the FAO's Strategic Framework, 2000-2015.
D. Sustainability of Project Impacts 121. While it is still very early to predict the sustainability of PPLPI impact (and in fact what those impacts are so far), the Team spent considerable time examining the sustainability implications of different scenarios for the next years of implementation. It was the opinion of the Evaluation Team that without adjustment at this mid-term stage, the PPLPI risked producing limited impact. Work at country level is on too small a scale to achieve significant, let alone sustainable, impact on national policy processes or the national livestock sectors of the countries concerned. Working papers, studies and other publications as well, while they will no doubt be relevant for some time to come, without the support of an active programme in FAO are unlikely to have more than the occasional user after the end of the project. 122. As such, many of the recommendations in preceding sections are oriented towards ensuring that project outputs achieve a lasting impact.
9 Donor funding that is granted and managed outside the organisation’s regular programme of work and budget.
21 123. The Evaluation's major recommendation with regard to narrowing the project Purpose statement to focus on development of capacity within AGA and FAO (section II.C) is specifically intended to ensure that the experience and learning coming from the project will produce a lasting change in AGA, which will thus carry on the work beyond project termination. Also, part of that analysis proposes the gradual transfer of posts and responsibilities from the project to the FAO Regular Programme (a process already contemplated by AGA management) over the project's remaining years. 124. Likewise, the recommendation for the creation of a Management Board and its role in strengthening the integration of the PPLPI inside AGA and FAO (section IV.A) supports the project's ability to carry out the capacity development effectively. 125. The development of a strong global network and possibly regional networks on pro-poor livestock policy change (under section IV.B) is another means to set in motion an activity which can be maintained by FAO/AGA at limited cost for as long into the future as it can be useful.
V. EVALUATION CONCLUSIONS
A. General Conclusions and Factors Which Will Affect Project Success 126. Several conclusions emerge from the analysis in this report with regard to the present accomplishments and future strategic orientations of the project. The project will ultimately be judged on the extent to which it has successfully led to creation of enhanced capacity in FAO, in member nations and in other international organisations to formulate livestock sector and related policies and implementation plans that reduce poverty. 127. Examples of the sorts of capacity needed includes the ability to understand, assess and appreciate the impact of policy on poverty, to participate in the policy debate, to ensure representation of all (and especially weaker groups) in the policy process, and to build coalitions for change. AGA/FAO staff need the appropriate knowledge and skills, as well as the internal mandate, structure and resources, to provide useful input for each of these aspects of policy support. 128. Each of these implies a raft of approaches and activities, and strategic repositioning of FAO and its PPLPI. Collectively, we can say that these are the activities involved in "informing" (see footnote Error: Reference source not found) the policies and institutions that determine poor people's livestock-related livelihoods – a phrase used widely in project documentation. 1. The Evaluation Team highlights the following major factors which emerged as affecting the likelihood of achieving some of these outcomes:
The log-frame 129. Despite a slow start, excellent progress has been made in generating new knowledge and partnerships, and in the establishment of solid, quality work in focus countries. Project activities and resources are well managed. However, the Team encountered an unexpected lack of common understanding - or agreement - about the project’s objectives and raison d’être. The (internally revised) project log-frame is not proving to be a sufficiently used management tool in this respect. It is in need of another review and rationalisation effort, preferably with the assistance of external expertise, as recommended below.
Impact on AGA 130. The primary focus of PPLPI activities remains strengthening the capacity, especially within FAO, better to inform the policies and institutions that determine poor people’s livelihoods. As noted in this report, both capacity and inform are used in their broad sense: capacity refers to the
22 institutional, structural and operational capacity, not just human resources and knowledge; inform means ‘giving form to’, determining the character of, and influencing. 131. The Pro-Poor Livestock Policy Initiative has clearly had impact on the work and configuration of AGA, contributing to the Division’s efforts to achieve a better balance between policy and technical work, and to place greater emphasis than before on social equity objectives (especially poverty eradication). Good progress has been made towards mainstreaming project activities, particularly through the evolution of staff positions. However, integration of the project with the rest of AGA, and FAO more widely, was found to be patchy. The reasons for this are complex, coming both from the project itself and from the surrounding environment. There is certainly more to do, and this should become a priority for the remainder of the project. An important priority is the more proactive and constructive management of the environment into which the project is expected to deliver and the fostering of joint objectives and shared 'ownership' of results.
Integration with wider processes and complementarity 132. Since the project design phase, significant shifts and developments have occurred in the project’s external environment – in particular with the emergence of new African institutions, and the interests of other international organisations. Even in this changing context, the project remains highly relevant to contemporary development priorities and approaches. However, PPLPI cannot achieve its ambitious objectives alone: it will achieve them by working with and adding value to wider policy and institutional processes. 133. There are encouraging examples of how PPLPI is successfully doing this at the level of national focus country work, but relatively less progress has been made in engaging with the international institutional architecture. This has led to FAO being marginalised in some of the international debate and activity being championed (e.g., by the World Bank and OIE). PPLPI and AGA more widely need to reflect on the likely scope and impact of these other programmes and seek to ensure complementarity as far as possible. As recommended, convening a high-level meeting to reflect on the changing institutional landscape and identify areas of mandate, comparative advantage and complementarity would be most useful at this stage.
Strategic engagement through pilot country initiatives 134. The Evaluation Team agrees, with some reservations, that the PPLPI should continue to invest in pilot projects in focus countries. The justification is the need for the project and its staff to establish credibility with member nations and to be able to speak from the experience of live country- level policy processes. The argument is made that the project/FAO needs to provide "proof of concept" that informed policy adjustments can make a tangible difference to poverty reduction. However, for these arguments to be valid, field work must, clearly, feed into wider policy processes at national, regional and international level, otherwise PPLPI field work represents little more than a supplementary technical cooperation programme. Greater and more visible consideration needs to be given to the role of the field projects in providing input to wider policy processes.
Fostering innovation 135. PPLPI needs to have the space and the energy to innovate, to bring fresh perspectives and big ideas to the table, and to relate to a new set of clients in new ways. The commissioning and management of studies and field programmes is a part of this, but must not become the only end of the project.
Communicating 136. The communication of PPLPI output to its diverse constituency of clients cannot be passive, and it certainly cannot be assumed that academic papers and websites are reaching or influencing the right people and organisations. The development of a coherent communication strategy is very important.
23 B. Recapitulation of the Evaluation's Main Recommendations for the Future of the PPLPI 137. The main recommendations emerging from the Evaluation Team's work are the following:
Refocusing the Project Purpose/Objective on Strengthening FAO Capacity 138. The most important overall recommendation of the Evaluation Team concerns the need to build on the strong body of experience and outputs accumulated over the past two and a half years and concentrate more deliberately on developing FAO/AGA's capacity to provide pro-poor livestock policy support to member nations and international organisations. 139. As a first action, AGA should sharpen the focus of the project's objective, as articulated in the Purpose statement of the log-frame. The Team recommends that AGA hold a professionally facilitated strategic planning and logframe revision workshop in this context (para. 36). The Evaluation Team is convinced that the achievement to a significant degree of the project's overall goal of poverty reduction through improved policy making in the livestock sector can only be reached in the long term by focusing more narrowly in the short term (life of the project) on strengthening FAO's capacity to provide effective support. In this context, the OVIs should all be reconsidered and amended appropriately, including recognition of the approved one-year extension to September 2008 (para. 46). 140. Suggestions as to how the project can contribute to strengthening FAO capacities include such initiatives as organising/sponsoring joint field visits both to PPLPI focus countries and to other AGA (or FAO) field activities; joint planning, proposal preparation, reviewing/auto-evaluating including bringing non-project staff into the project implementation process; reciprocal peer reviewing and discussion of each other's work; joint participation in international and regional meetings, workshops or training opportunities on related topics, and other means as determined during implementation. (para. 42) 141. Ensuring the sustainability of acquired project knowledge and experience, as well as databases, information systems, communications and other strategies, will also involve gradually transferring responsibility for PPLPI work to more permanent staff, e.g., through redefining existing AGA posts (ongoing to some extent), co-funding posts with the RP, seeking to place project staff on vacancies that open up in AGA, etc. (para. 43)
Improving Project Governance and Providing an Enabling Environment 142. The second most important recommendation concerns the environment in which the PPLPI (one of AGA's major investments) operates in AGA and in FAO. Management of this environment, which must be as receptive and supportive as possible, requires decision-making and configuration of resources beyond the project itself and beyond the Project Coordinator's mandate and ability to influence. The original project memorandum foresaw a "management task force" drawn from various parts of AGA and elsewhere in FAO but this was tried only briefly. (paras. 81-84) 143. This is a governance issue that the Division Director needs to tackle and he is the appropriate person to decide. However the Team strongly suggests that it would be of benefit to create a small "Management Board," consisting of the Division Director, the three Service/Branch Chiefs of AGA, the two non-chief Programme Entity Managers and the Project Coordinator. Board meetings might also include occasional participation of one or two people of at least Service Chief level from elsewhere in FAO outside AGA but in closely related activities (such as the Chief, Agricultural Sector in Economic Development Service, and the Director, Policy Assistance Division). 144. The Management Board will need clear Terms of Reference (see suggested elements for this in Annex II) which define its role in creating and supporting the environment to promote the spread of knowledge, attitude, analysis and experience emerging from the project within AGA and FAO. 145. Such a Board would meet no more frequently than once every two months. Its job would be to support translation of project progress into 'impact', in particular impact on AGA. The concept is that these meetings should serve as 'mini-retreats' for reflection and discussion of strategic issues
24 in the evolution of AGA and the project's role in that evolution. They should not resemble "management meetings" in the current sense of AGA's weekly management meetings. 146. While budget-holder and day to day management responsibility would remain with the Project Coordinator, the Board could, if appropriate, provide approval of overall annual workplan and budget allocations. The Team also suggests that the Management Board might have direct control over a small part of the project budget (say 10% of remaining funds) to allocate to activities directly contributing to the exchange and transfer of capacity between the project and FAO's regular programme staff and activities like those suggested in the previous recommendation. 147. One specific issue which may benefit from the Board's attention: There is a fine line between achieving integration and mainstreaming of headquarters-based ('normative') project initiatives, and use of such a project to subsidize regular programme work. The Evaluation Team has some concerns that that line may be being crossed at times, and encourages AGA management (through the Management Board) to keep this under review. (para. 119)
Working with Other Organisations: Global Mandates and Complementarity 148. The Team recommends continued activity by the project in the area of global decision making on livestock development and poverty, and in particular (para. 111): PPLPI should continue to commission and publish studies on international trade and other PPLPI issues relating to international public goods. The Animal Health Service (AGAH) should take the initiative in raising pro-poor issues in OIE in close consultation with the PPLPI. PPLPI should invest more time and resources in relationships with key international institutions (well identified in the project document) – both directly and through the mandated interlocutors (for example the FAO Investment Centre [TCI] in the case of the World Bank and AGAH in the case of the OIE). PPLPI should examine the possibility of increasing its influence at the international level by appropriate briefing of civil society organisations and the mass media, that have an increasing influence both on developed countries' development aid and on trade policies. Substantial indirect influence may be achieved in this way. As part of the process of clarifying its objectives and mandate, PPLPI should convene reasonably high level meetings of the development agencies and sector institutions now working on livestock-sector policy and standard-setting, and on policies relating to livestock's contribution to poverty reduction (OIE, World Bank, IFPRI, ILRI...), to examine and better define institutional mandates, competencies and comparative advantages and, above all, to remove any duplication and competition for resources. (para. 37) A specific meeting (at least Director-level) should be convened by the PPLPI between the key international institutions involved in the World Bank's ALive initiative (WB, OIE, EU, FAO/TCI) in order to better define complementary roles in this particularly closely related activity.
Country and Regional Work 149. The Team supports continuation of the focus country activities, as learning activities. It is recommended that PPLPI formulate a more solid strategy for providing assistance to individual member nations consisting of direct participation of PPLPI in the policy formulation process, but predominantly focused on the vigorous dissemination of experiences and results through various means targeting particular information to different audiences, and in particular audiences within FAO/AGA. These could include publications and articles, but also visits, workshops and seminars, joint activities with others, etc. This is closely related to the recommendation relating to the need for a communications strategy. Of course, this recommendation also applies to exchange of experiences between focus countries. (para. 96)
25 150. The PPLPI needs to take a clear position on the future of the concept of regional hubs, which is at present confused. If this approach is to continue, then a specific strategy will need to be developed. (para. 101)
Implementation Issues 151. Several recommendations refer to specific implementation issues. These include the following: 152. The livestock-dependent poor are very often women. The Team was surprised by how seldom women or gender more generally were present in discussions of PPLPI and in the documentation, both in Rome and in the focus country visits. As a general recommendation, the PPLPI should give more attention to gender issues. Greater gender balance in staff and the SC might help with this. (para. 46) 153. As Output 1 (developing a portfolio of policy interventions) is currently defined, it is not really possible to assess progress. The log-frame needs to be revised to incorporate both the need to build up a body of knowledge and the need to systematise the policy options, e.g. in terms of objectives and instruments - as is now being worked on by PPLPI. (para. 49) 154. Under Output 2 (increased awareness of the potential of PPLP), the Evaluation Team recommends that the emphasis on dissemination in the original project log-frame be restored. The PPLPI should focus on development of a Communications Strategy, as this will necessarily be a key element of future work in informing the policy process. It should also be clear that the stakeholders concerned are not just those of the focus countries. (para. 57) 155. With regard to Output 3 (effective information and decision support systems), the Team recommends that the Steering Committee consider whether the number of decision-making tools being simultaneously developed is too large for PPLPI adequately to supervise and/or to test and advise on. Early emphasis should be give to the question of M&E indicators where little progress seems to have been made. Also, someone unconnected with developing the systems needs to do some simple "market research" to see what kind of systems are in practice "demanded" (and likely to be used); and what difficulty, if any, users have had in getting acquainted with the PPLPI tools. (para. 61) 156. The activities under Output 4 (effective stakeholder representation) are behind schedule. PPLPI needs to rethink its approach in ensuring stakeholder representation and to equip itself with adequate skills in institutional matters. Serious consideration should be given to recruiting an Institutional Development Specialist as was originally foreseen. Only by doing that is the PPLPI likely to achieve the quality and quantity of activities needed under this Output. (para. 66) 157. It is highly recommended that PPLPI pursue the establishment and development of a global network (and possibly regional networks) in support of pro-poor livestock policy, as foreseen in the project log-frame. (para. 111)
26 ANNEX I: LOG-FRAME OUTPUTS AND PERFORMANCE
The log-frame of the PPLPI in relation to outputs Output OVI Comment by Evaluation Team 1 Portfolio of Endorsement of at least 5 Some excellent studies have been carried out livestock-related sets of generic pro-poor but the project has not achieved its selected interventions for livestock sector-related (OVI) target, and will not do so by a later reducing poverty policy options by the date, because PPLPI has defined "portfolio" in through policy and steering committee & a way that is incompatible with achieving it10 institutional change. external reviewers by and has not, in any case, requested the spring 2005. Steering Committee to give the endorsement stipulated in the OVI. However PPLPI are now developing the framework of and inventory' of policy options; and this could form the basis of a revised OVI with specified target dates. 2 Increased - Stakeholders are - The date for the achievement of the first OVI awareness of the actively promoting target set is not until 2006, and looks potential identified pro-poor achievable. The PPLPI's own assessment of contribution of livestock sector-related progress in terms of this OVI seems to under- livestock and the policy and institutional emphasise the importance of the words livestock sector to options in Poverty "actively promote" in this OVI. Some of the poverty reduction. Reduction Strategies organisations they cite have shown interest but and related processes in seem to be still a considerable distance away 5 countries by 2006. from "actively promoting". - Donor and - A much more ambitious target for the first government finance OVI by end-2008 in respect of national-level directed at livestock stakeholders should be set to verify the wider development in line dissemination effort that we recommend. with PPLPI policies and - The second OVI will probably be attained best practices in 5 cases provided a generous interpretation is given to by 2007. the words "directed at" so that it includes intentions and not just payments actually made.
10 The project decided to disaggregate the work under Output 1 into four thematic areas, namely: - The role of livestock in economic development and poverty alleviation. - Policies, applied policy instruments and the political economy of policy reform - Promotion of access to markets for livestock and livestock products - Promotion of access to livestock services. The redefinition by the PPLPI has simply set out a number of themes to study. The PPLPI has certainly done far more than 5 studies (their own report lists dozens of studies under this output) but few of these really look like a “generic pro-poor livestock sector-related policy option” which can be endorsed by the Steering Committee or anyone else. Working Papers 11, 12 and 13 seem to us to possibly fill the bill of discussing “a generic pro-poor livestock sector-related policy option” which could be endorsed or not. In the PPLPI’s own assessment of their performance under this output they list a number of “key findings (from their studies which are specifically listed under each theme) that may guide policy development/reform”. We do not feel that it is our task to assess whether each of these findings are sound or not.
27 Output OVI Comment by Evaluation Team 3 Effective systems - Policy makers and The OVI targets set for this output are in for livestock policy other key stakeholders terms of (1) the number (unspecified) of information, including non-livestock policy makers and other key stakeholders … analysis, decision- actors actively seek actively seeking information and using tools support and M&E information and use generated by the project. …and: (2) of the tools generated by the number (10) of organisations actively project in their policy contributing to the information analysis formulation and decision-support and M&E systems. preparation of poverty The first indicator's target level is undated. reduction strategy We can expect that a substantial number of papers. policy-makers and other key stakeholders will - At least 10 be actively seeking information and using the organisations actively tools by the end of the project's life. The date contributing to the set for achieving the second target levels is information analysis 2006 and is unlikely to be met, particularly as decision-support and PPLPI has not yet finished drawing up M&E systems by end appropriate M. & E indicators. 2005. 4 Mechanisms - Acceptance of at In the original version of the log-frame the established for least 5 identified fora as OVI equivalent to this one referred to effective stake- negotiating platforms "Establishment of appropriate negotiating holder for pro-poor policy platforms." The revised version refers to representation in the formulation by a "acceptance of fora as negotiating platforms negotiations… majority of stakeholders by a majority of stakeholders." The difference by spring 2005. in wording is crucial. AGA claims that - FAO/AA promoting "Policy negotiations are taking place in all 5 smallholder interests in focus countries in various platforms, which relevant committees of appear to have found acceptance and in most OIE, Codex and WTO. cases include government participation / have government endorsement." What we observed in Uganda, and the evidence reviewed in Table _ of Section III above make us doubt whether the interests of the livestock-dependent poor are effectively represented.
28 ANNEX II: ELEMENTS FOR TERMS OF REFERENCE FOR THE MANAGEMENT BOARD
Background
1 FAO is committed to poverty reduction and the achievement of the Millennium Development Goals. This implies an upscaling of development efforts, and ways of working that impact at scale. Increasingly, this means focusing AGA’s work on the broad underlying policies and institutions that determine poor people’s livestock-related livelihoods.
2 Livestock contribute to the livelihoods of many of the world’s rural poor. However, the nature of livestock farming is typically determined by policy and institutional frameworks that are rarely pro- poor and that sometimes encourage production that threatens environmental stability and public health. Characteristically the rules of the game surrounding trade in livestock and access to services are increasingly determined by national and international policies, institutions and processes that require supra-national agreement and cooperation and/or are subject to international norms, conventions and standards that need to be informed from a global or regional perspective. AGA’s ‘Pro-Poor Livestock Policy Initiative’ (PPLPI) aims to establish and institutionalise capacity to support and provide input into national and international policies and institutions in support of poor people’s attempts to make ‘A Living from Livestock’.
Purpose
3 While the project’s day-to-day management is delegated to a Project Coordinator, it is the wider AGA’s responsibility to ensure that the products of this significant FAO investment are mainstreamed into its Regular Programme, into the Field Programme where appropriate, and into work of FAO more generally, as well as into the work of AGA’s external partners. In order to assist in this mainstreaming, a Management Board will be constituted.
4 The purpose of the PPLPI Management Board is to: Manage the environment (within AGA) into which PPLPI’s outputs are delivered, in order to maximise the return on AGA’s investment in PPLPI; Ensure coherence and complementarity between the work of the PPLPI and AGA’s Regular Programme Entities; Ensure coherence and complementarity between the work of the PPLPI and AGA’s external interactions and relationships (with, for example, the World Bank and the OIE); Ensure the sustainability of PPLPI impacts by managing the mainstreaming over time of PPLPI project activities and staffing into the Regular Programme; Provide the Project Coordinator with guidance and support in the allocation and management of PPLPI resources, including staff recruitment and performance.
Operation
5 The Management Board will meet every 2 months. Its members will comprise: Director, AGA (Chair) Service / Branch Chiefs of AGAH, AGAL and AGAP other managers of AGA Programme Entities (the two who are not service chiefs) PPLPI Coordinator
29 6 The Management Board will have at its disposal approximately 10% of the PPLPI’s remaining financial resources to allocate in support of collaborative, cross-disciplinary work across AGA, and to assist the integration of PPLPI activities and approaches into the Regular Programme.
7 The Management Board will prepare a report to the PPLPI Steering Committee each year on its activities and recommendations.
30 ANNEX III: ITINERARY, PEOPLE MET AND SUMMARY OF FINDINGS OF VISITS TO COUNTRIES AND INSTITUTIONS
Date Itinerary Organisation People Met (First occasion only if multiple meetings)
March 2005
Wed 9 Travel to Rome Background reading
Thu 10 FAO HQ, Rome FAO Samuel Jutzi, Director AGA Joachim Otte, PPLPI Coordinator Achilles Costales, PPLPI East Asia Cheikh Ly (PPLPI West Africa) Katinka DeBalogh, PPLPI HQ Pius Chilonda, PPLPI HQ
DFID Jo Yvon, DFID Liz Nasskau, DFID
African Union Babagana Ahmadu, Director, Rural Economy & Agriculture
Fri 11 FAO HQ, Rome FAO Anni McLeod, AGAL Marc Moens, TCI [& other follow-up meetings]
Sat 12 Travel to Addis Background reading
Sun 13 Travel to Entebbe Background reading
Tim Robinson, PPLPI
Mon 14 Kampala / Entebbe PMA Secretariat Tom Mugisa, Programme Officer
Oxfam Jonathan Davis, Coordinator
Panos Sarah Ashanut Ossiya
Dairy Development Nathan Twinamasako, Executive Authority Director
FAO Representation Patrick Tesha, Representative a.i. Charles Owach James Okoth
31 Date Itinerary Organisation People Met (First occasion only if multiple meetings)
Tue 15 Kampala / Entebbe Ministry of Agriculture, William Olaho-Mukani, Director Animal Animal Industries & Resources Fisheries Jimmy Saamanya, Commissioner Animal Production & Marketing Nicholas Kauta, Commisisoner Livestock Health & Entomology Uganda Bureau of Statistics John Male Mukasa, Executive Director Tom Emwanu, Systems Analyst Emanuel Mnewa
Dairy Policy Task Florence Kasyire, Coordinator Force
Donor Group on Alan Tollervey, DFID (Chair) Agriculture + Danida, EC, FAO, France, Germany, Netherlands, Sweden, USAID, World Bank, & other representatives attending
Wed 16 Kampala / Entebbe Dairy Sector Workshop Representatives of government, parastatal, academic, private sector & civil society organisations Philip Thornton, Consultant Peter Thorne, Consultant
Travel to Nairobi ILRI Carlos Seré, Director General Nairobi DFID John Hansell, Livelihoods Adviser
Thu 17 Nairobi ILRI Brian Perry Christopher Delgado Ade Freeman Jeroen Djikman Steven Staal
IBAR Modibo Tiemoko Traore, Director Rene Bessin, Chief Animal Health Officer Berhanu Admasu, Institutional & Policy Support Team Tim Leyland, Adviser
FARM Africa Helen Altshul, Country Director Bonface Kaberia, Livestock Adviser
Food Security Analysis Cindy Holleran, Technical Manager Unit for Somalia
FAO Representation Bruce Isaacson, Representative Travel to Addis
32 Date Itinerary Organisation People Met (First occasion only if multiple meetings)
Fri 18 Addis FAO Representation Abdulkarim Ahmed Guleid, MP, Chairman Parliamentary Committee on Pastoralist Affairs Ali Haribou, FAO Liason Officer to AU and ECA Luciano Mosele, FAO Emergency Coordinator Tony Forman, Consultant Sintayehu (& others)
Pastoralist Patta Scott-Villiers (IDS) Communication Initiative, Participation Group
DFID Tim Robertson, Livelihoods Adviser
Sat 19 Addis Independent Mohammed Mussa, Consultant
Team round-up session
Travel to Europe
Mon 28 Travel to Rome
Tue 29 FAO HQ, Rome FAO Mafa Chipeta, Director Policy Assistance Division MariaGrazia Quieti, Agricultural Policy Support Officer
Henning Steinfeld, Chief Livestock Information, Sector Analysis & Policy Branch
Wed 30 FAO HQ, Rome FAO Carlos Eddi, Senior Officer, Animal Health Service
Thu31 FAO HQ, Rome FAO Louise Fresco, Assistant Director- General (Agriculture)
Carlos Tarazona Cervantes, Evaluation Officer Travel to Paris
April 2005
33 Date Itinerary Organisation People Met (First occasion only if multiple meetings)
Fri 1 Paris OIE Bernard Vallat, Director General
Travel to Brussels
Brussels European Commission Bernard Rey EuropAid Cooperation Enrico Pironio Office Rural Development and Environment Unit
Sun 3 Travel to Washington Background reading DC
Mon 4 Washington DC World Bank Kevin Cleaver, Director, Agriculture & Rural Development
John McIntire, Director ESSD-Africa Francois leGall Livestock Adviser Cees de Haan, Consultant
Neil MacPherson, DFID Rural Livelihoods secondee
USAID Joyce Turk, Snr Livestock Adviser Susan Thompson, Snr. Policy Adviser
Travel to Lima
Tue 5 Lima, Peru Condesan (CIP) Hector Cisneros, PPLPI Andes Victor Mares, CIP Judith Kuan, Consultant PPLPI Peru Musuq Briceno, InfoAndina Raul Ho, CARE Peru
Consejo Nacional de Group Camelidos Sudamericanos (CONACS), Ministry of Agriculture
SPAR Joselyn Valer Rojas
Wed 6 Travel to Puno
Juliaca Cecoalp alpaca Higinio Porto Huasco keepers’ cooperative General Manager
CARE Puno Woodro Andía Castelo Puno Regional Director
Thu 7 Puno District Field visit to alpaca keepers in high plateau Travel to Lima
34 Date Itinerary Organisation People Met (First occasion only if multiple meetings)
Fri 8 Lima Office of the President Alberto Gonzales-Luniga, Coordinator of the Council of General, Sierra Rural Development Ministers Project
Office of Strategy & Group Policy, General Office of Agrarian Planning, Ministry of Agriculture
FAO Representation Luis Castello, Representative
Sat 9 Travel to Europe
Mon 11 FAO HQ, Rome FAO Joseph Domenech, Chief, Animal Health Service / FAO Chief Veterinary Officer
Simon Mack, Senior Officer, Animal Production Service
Tue 12 FAO HQ, Rome FAO Meeting with PPLPI team Kostas Stamoulis, Chief, Agric. Sector in Economic Development Vinod Ahuja, PPLPI South Asia
DFID Matthew Wyatt, DFID Ambassador Vic Heard, Adviser Liz Nasskau, Programme Manager
Wed 13 FAO HQ, Rome FAO Follow-up meetings
Thu 14 FAO HQ, Rome FAO Presentation of main conclusions and recommendations / Aide Mémoire
Thu 5 May FAO HQ, Rome FAO Discussion of conclusions and recommendations with AGA Management group
Summary of Findings of Field Visits and Visits to Other International Institutions 158. In the course of the evaluation, the Team travelled to Uganda (Kampala and Entebbe), Kenya (Nairobi), Ethiopia (Addis Ababa), Paris, Brussels, Washington, D.C., and Peru (Lima and Puno). A brief description of the outcomes of these visits follows.
Visit to East Africa 159. In Uganda (one of the five 'focus countries') the PPLPI is providing assistance to improve the process through which a specific policy (dairy sector development) is being formulated. This involves such activities as helping constitute a Dairy Policy Task Force (DTF), the use of tools (e.g., those in
35 the table under Output 3, Section III.A above), stakeholder workshops, and technical assistance from FAO/PPLPI staff. 160. Made up mostly of government staff, the DTF has so far been focused mainly on collecting information, and not yet on examining policy options. The intention is to cover domestic consumption as well as marketed surplus, and pastoralists as well as non-pastoralists. At a Stakeholders' Workshop on Dairy Policy attended by the Team, attention was on learning to use a tool for policy analysis. In general, government staff met by the Team were all very positive about FAO’s potential and actual role in livestock policy formulation in Uganda, while the NGO representatives and the donor group were more critical. The NGOs found that no attempt was made to incorporate pastoralists into the dairy policy formulation process. The donor group wanted to be adequately informed about FAO’s initiatives. 161. In Nairobi, meetings with ILRI (the International Livestock Research Institute) and AU/IBAR (the African Union's Inter-African Bureau of Animal Resources) illustrated the problems involved in cooperation between international organisations posed by the perceived need to claim primary credit for any successful outcome. ILRI expressed some reservations about the nature of the partnership between ILRI and PPLPI (ILRI feels that it is being 'commissioned' by PPLPI rather than being a real partner). IBAR said it was essential that IBAR be the focal point of any PPLPI "policy hub" activities in Africa, especially East Africa, and was opposed to the decision to place the Horn of Africa hub in IGAD. 162. In Ethiopia the Evaluation Team met the Chairman of the Parliamentary Committee on Pastoralist Affairs, members of the FAO emergency team in Ethiopia, a private livestock policy expert, and staff of the Pastoralist Communication Initiative. In general, these people were unacquainted with the PPLPI, though they found the initiative interesting when introduced by the Team.
Visit to Peru 163. In Peru, another of the PPLPI's focus countries, the Team met with the PPLPI focal institution in Lima, CONDESAN (Consorcio para el Desarrollo Sostenible de la Ecorregion Andina), government officials from the Planning Office of the Ministry of Agriculture, the government agency for South American camelids, alpaca producers' organisations, a new large-scale government initiative for poverty reduction in the highlands and CARE-Peru. The Team travelled to the Juliaca- Puno area in the high Andes and met with CECOALP (alpaca marketing organisation whose members are alpaca keepers), CARE staff, and (too briefly) a group of alpaca-keepers in the high Andean plateau. 164. The work being undertaken by CONDESAN, using alpaca production as a model, involved a thorough exploration of possible channels for influencing policy in favour of poor highland alpaca producers. Other initiatives which had this kind of impact were also being studied, both in Peru and in neighbouring countries. The PPLPI was bringing together stakeholders in the policy process in meetings and workshops, and appeared to have made a certain name for itself in Peru.
OIE, the World Bank, USAID and the European Commission (EuropAid Cooperation Office) 165. The Evaluation Team met the Director General of the World Organisation for Animal Health (OIE). The principal point of discussion was the World Bank-OIE initiative “ALive” (A partnership for livestock development for poverty reduction and economic growth in Sub-Saharan Africa), which has overlapping and sometimes conflicting goals with PPLPI. In spite of this, the DG found that there was potential for complementarity, as ALive is a "platform" open to all. He described PPLPI’s publications as being “too academic” to be influential, and that simple prescriptive messages are the only ones likely to be effective. On questioning by the Team, he expressed mild interest in including a “pro-poor” element, to be devised by PPLPI, in the standard training courses OIE already carries out
36 for chief veterinary officers, but stressed that the technical aspects of animal health control must remain OIE’s first priority. 166. At the World Bank, the Team met with the Bank's livestock specialist (it has only one, in the Africa division) as well as his retired predecessor, their director and the Director of Agricultural and Rural Development. Again, discussion focused mostly on ALive, due to the similarity between that programme and the PPLPI. It became clear that different people in the WB have different expectations about ALive, some seeing it as a means to influence internal opinion to get more WB funding allocated to livestock development, others as a coordinating mechanism between donors which in will move from being Africa-oriented to having a global mandate. 167. At USAID, the Team met with a senior livestock adviser and a senior policy adviser. They briefed the Team on USAID’s activities in the livestock sector in Africa which have increased 10-fold in size (from a very low base) between 1994 and 2005. They observed that PPLPI’s work is very little known, but FAO has a global mandate and could do good work in consciousness-raising. 168. At the European Commission, the Evaluation Team met with the livestock specialist of the Rural Development and Environment Unit. The EC is far along in the approval process for a € 5 million project to fund the Horn of Africa hub of the PPLPI based in IGAD. This unit would also be an integral member of the ALive platform, with the EC acting in this case as a bridge between the ALive initiative and the PPLPI. The Hub's role would be feed into East African policy-makers, easing the integration of livestock policies and services at the regional level. As is evident from their decision to fund this Hub, the EC strongly backs FAO's role (through the PPLPI) as facilitator and broker for improved pro-poor policy in the livestock sector.
37 ANNEX IV: TERMS OF REFERENCE OF THE EVALUATION MID-TERM DFID OUTPUT-TO-PURPOSE REVIEW (OPR) AND FAO EVALUATION
10 MARCH - 15 APRIL 2005
Background
1. Livestock contribute to the livelihoods of an estimated 70% of the world’s rural poor, or about 750 million people. However, the nature of livestock farming is determined by policy and institutional frameworks that are rarely pro-poor and that sometimes encourage production that threatens environmental stability and public health. Characteristically the rules of the game surrounding trade in livestock and access to services are increasingly determined by national and international policies, institutions and processes that require supra-national agreement and cooperation and/or are subject to international norms, conventions and standards that need to be informed from a global or regional perspective. 2. DFID is providing £9m extra-budgetary support to FAO between October 2001 and November 2008 to help establish and institutionalise a capacity better to inform national and international policy- making in support of poor people’s attempts to make ‘A Living from Livestock’. 3. The project’s explicit objectives are to enhance the contribution of the livestock sector to poverty reduction (the goal) by strengthening capacity in FAO, member nations and international organisations to formulate livestock sector and related policies and implementation plans that reduce poverty, while managing environmental and public health risks (the purpose). 4. The principal outputs of the project currently comprise: i.) A portfolio of livestock-related interventions for reducing poverty through policy and institutional change; ii.) Increased awareness of the potential contribution of livestock and the livestock sector to poverty reduction; iii.) Effective systems for livestock policy information, analysis, decision support and M&E; and iv.) The establishment of mechanisms for effective stakeholder representation in the negotiation of policies and institutional changes that better support poor people’s livestock-dependent livelihoods. 5. At its inception, the project represented a significant shift in emphasis and direction, focusing on FAO’s unique comparative advantage in achieving the MDGs through supra-national, normative, work. The project therefore has an implicit role in supporting institutional reforms and new ways of working in FAO. 6. Previous Output-to-Purpose reviews (July 2003 and July 2004) noted good progress with few contentious issues at output level, although some questions and imperatives for future monitoring and review were identified. These have been incorporated in the issues to be considered in this Mid-Term OPR / Evaluation. 7. The project’s Steering Committee is meeting in June 2005. This mid-term OPR / evaluation will be submitted to the Steering Committee and will be influential in setting out the priorities for the remainder of the programme and, possibly, the allocation or reallocation of resources to it.
38 8. The exercise will be carried out jointly by DFID and FAO and is intended to satisfy, concurrently, both DFID’s requirement for a substantive Output-to-Purpose Review and FAO’s requirements for a Mid-Term Evaluation. 9. The original project memorandum and log-frame, work plans and progress reports, OPRs and Steering Committee recommendations have been well documented and will all be made available to the review/evaluation. An internal thematic evaluation of FAO's activities in support of livestock production, policy and information more generally has also recently been completed and will be made available.
Purpose of the Review / Evaluation 10. From DFID’s perspective, the function of Output-to-Purpose Reviews (OPRs) is to assess progress towards achievement of the project’s objectives (Purpose and Goal) and to check on whether the assumptions remain valid. It should concentrate on the higher levels of the logical framework, rather than on day-to-day technical or operational issues. The findings of an OPR will inform the consideration of whether any reorientation of the project is required, and/or the reallocation of resources. Data from the review will also be fed into DFID’s Performance Reporting Information System for Management (PRISM). 11. From FAO’s perspective, a mid-term evaluation is intended to make recommendations for any necessary changes in the overall design and orientation of the project and to make recommendations on the work-plan for the remainder of the project.
Scope of the Review / Evaluation
12. This, mid-term, review / evaluation will carefully examine progress towards the achievement of the programme’s higher-level objectives, which are ambitious and imply wide-ranging impact within FAO and externally. Given the long-term nature of the project in an evolving policy, institutional and development environment, the review / evaluation will, in particular, need to reflect on the assumptions made about the external players, forces and influences that constitute the programme’s raison d’etre. 13. The mission will assess the: a. Continued relevance of the project to development priorities and needs, given, in particular, shifts in external (other organisations’) perceptions and priorities about poverty and the importance of policy and institutional reform since this project was conceived. b. Clarity, and realism of the project's development and immediate objectives, including: The likelihood that the project purpose will be fulfilled, and the extent to which the likely achievement of purpose is attributable to the project outputs. Specification of targets and identification of beneficiaries and prospects for sustainability. The early indications about achievement of the project goral. c. Quality, clarity and adequacy of project design including: Clarity and logical consistency between, inputs, activities, outputs and progress towards achievement of objectives (quality, quantity and time-frame); Realism and clarity in the specification of prior obligations, prerequisites, assumptions and risks; o Have the risks, assumptions and undertakings specified changed, or new ones appeared?
39 Realism and clarity of external institutional relationships, and in the managerial and institutional framework for implementation and the work plan; Likely cost-effectiveness of the project design.
d. Efficiency and adequacy of project implementation including: availability of funds as compared with budget for both donor and FAO contributions; the quality and timeliness of input delivery by both FAO and partner organisations; managerial and work efficiency; implementation difficulties; adequacy of monitoring and reporting; the extent of national support and commitment and the quality and quantity of administrative and technical support by FAO.
e. Project results, including an assessment of outputs produced to date (quantity and quality as compared with work-plan and progress towards achieving the immediate objectives). What progress has been made for each output? To what extent are outputs likely to be achieved? What actions are recommended? Are there any unexpected outputs? The mission will, especially, review the status and quality of work on the interface between supra-national policy analysis and national policies and programmes, including the utility of the proposed ‘Hubs’ and sub-regional programmes. An implicit objective of the programme is to support FAO’s institutional development. What do we see in that respect and what are the issues?
f. The prospects for sustaining the ways of working introduced by the project, the analysis and information provided to internal and external partners, and its influence and impact on international policy and practice, after the termination of the project. The mission should examine in particular: The extent to which the implied incorporation of the PPLPI’s staffing and work-stream is being, or will be, incorporated into FAO’s regular programme of work and budget; The extent to which the implied shift in emphasis and direction conflicts with other work-streams, priorities or mandates within the FAO (at divisional, departmental or organisational levels), and the implication of any such conflict on sustainability, credibility and impact. What progress is being made in securing wider donor buy-in to the programme, in particular the Regional Hubs, and what are the issues here?
g. The cost-effectiveness of the project In particular, is the extra-budgetary support genuinely being used to promote and cover the costs of developing capacity, and not simply subsidising business as usual? 14. Based on the above analysis the mission will draw specific conclusions and make proposals for any necessary remedial or further action by FAO and/or DFID to ensure sustainable development, including Revisions to the logical framework; Any need for additional assistance and activities of the project prior to its completion. (Any proposal for further assistance should include precise specification of objectives and the suggested outputs and inputs.)
8 The mission will also reflect on and report the main lessons that should be learned from this project to date.
40 Composition of the Mission
9 The mission will comprise: i. A Team Leader (nominated by FAO’s Evaluation Service) (Daniel Shallon) with experience of evaluating complex international, normative, programmes; ii. An experienced reviewer/evaluator (nominated by DFID) (Peter Bazeley) familiar with the policy and institutional environment relating to the contribution of livestock to sustaining and promoting poor people’s livelihoods; iii. An external and independent reviewer/evaluator (Stephen Sandford) familiar with livestock policy processes and the roles of livestock in poverty alleviation, the international institutional architecture of aid and development, and the role of membership organisations such as FAO.
Timetable and Itinerary of the Mission
10 The mission will assemble in FAO headquarters in Rome on 10 March 2005 for initial briefings by both FAO and DFID.
11 The review / evaluation will then continue through Friday 15 April 2005 (breaking for the period 20-28 March) examining, in broad terms, the impact of the PPLPI on its three major constituents: FAO, member nations, and other international organisations.
12 The review/evaluation will comprise desk studies, workshops, interviews and field visits with/to: Representatives of relevant FAO Departments, Divisions, Services, decentralised offices and projects; DFID; Partners, stakeholder representatives and key informants in the countries visited; The OIE; The EC; The World Bank; The African Union and the Inter-African Bureau for Animal Resources; The International Livestock Research Institute; Examples of policy interface work at national/sub-regional level, in Uganda and Peru.
13 Discussion meetings will be held with stakeholders in Rome before the field missions and at mid- point of the evaluation to ensure that the direction and thrust of the review/evaluation broadly reflects FAO and DFID requirements.
14 Following the field work, sufficient time will be allowed for the proper analysis of, and reflection on, data and information gathered during the review, prior to presenting and discussing findings to a stakeholder workshop in FAO tentatively on 14 April 2005.
15 A draft report will be issued within two weeks of completing the evaluation, and in any case no later than 29 April 2005, which will be subject to comment and approval by FAO and DFID. The final report will then be submitted to members of the Steering Committee prior to its meeting in June 2005 for peer review. The FAO and DFID reviewers/evaluators will also make a presentation to that PPLPI’s Steering Committee meeting and take questions.
16 FAO and DFID will subsequently agree on follow-up to the review/evaluation recommendations.
41 Consultations
17 The mission will maintain close liaison with representatives of DFID and FAO, and other relevant organisations, as well as with national and international project staff. Although the mission should feel free to discuss with the authorities concerned anything relevant to its assignment, it is not authorized to make any commitments on behalf of the Government, DFID, or FAO.
Reporting
18 A succinct report is required of no more that 25 pages.
19 The mission is fully responsible for its independent report which may not necessarily reflect the views of DFID, FAO or partner organisations. The report will be written, in English, in general conformity with the FAO model for evaluation reports provided to mission members.
20 The report will be completed, to the extent possible, during the course of the mission and the findings and recommendations fully discussed with concerned parties and wherever possible consensus achieved.
21 The mission (in particular FAO’s nominee) will also complete the FAO Project Evaluation Questionnaire.
22 The mission (in particular DFID’s nominee) will also complete DFID’s PRISM OPR form.
23 The mission leader bears responsibility for finalisation of the report, a draft of which will be submitted to FAO within two weeks of mission completion as noted above. FAO will submit the report to DFID together with its comments.
Preparatory Inputs by FAO
24 Prior to the review/evaluation, FAO’s PPLPI team will: . Suggest an appropriate itinerary of visits and interviews and arrange meetings with relevant stakeholders, and make appropriate travel/accommodation arrangements; . Report on the implementation or otherwise of recommendations and requirements from previous OPR’s and Steering Committee meetings; . Complete missing OVIs etc from the log frame. . Compile sets of relevant documents and briefing materials for the reviewers/evaluators. . Catalogue and summarise project outputs and reports, prior to the review/evaluation, where such a catalogue/summary would help the reviewers/evaluators more efficiently understand the project and the achievement of its objectives. (The reviewers/evaluators should NOT be given an unrealistically large number of documents to read in order to do this.)
42