Evaluation and Aid Effectiveness 2

EVALUATION AND AID EFFECTIVENESS 2 (YDOXDWLQJ &RXQWU\ 3URJUDPPHV Vienna Workshop, 1999 DEVELOPMENT ASSISTANCE COMMITTEE © OECD 1999 1 O E C D FOREWORD Since the mid-1980s development agencies have increasingly targeted development assistance at the country level. This approach provides a rational framework that can be used to improve relevance, effectiveness and coherence of development assistance programmes. The increasingly widespread adoption of country assistance strategies has challenged evaluators to assess results at the country level. As documented in this publication, development agencies have employed a range of different approaches to carry out country programme evaluations. In order to review these experiences and to share lessons about strengths and weaknesses inherent in different approaches to country programme evaluation, the Working Party on Aid Evaluation organised a workshop in Vienna on 11 - 12 March 1999. This workshop principally focused on progress made since the first country programme evaluation workshop, also held in Vienna, in May 1994. This publication provides a unique overview of approaches and methods for country programme evaluation. It contains i) an analytical review of the workshop’s main findings; ii) an overview of the state- of-the-art approaches and methodologies used for country programme evaluations; and iii) the case studies, which were presented and discussed at the workshop. Special thanks are due to the Austrian Ministry of Foreign Affairs -- who hosted the workshop -- for their hospitality and assistance, and in particular to Brigitte Dekrout, Director of the Department for Audit and Evaluation. We are also grateful to DAC Members who contributed interesting case studies (France, Germany, the Netherlands, Switzerland, United States, European Commission, Inter-American Development Bank and the World Bank), and financial support (Austria, Denmark, the Netherlands and United Kingdom). The publication was prepared by the Development Co-operation Directorate of the OECD under the direction of Hans Lundgren and Maria Iarrera, with editing assistance by Carola Miras and Marcia Bystrom. Niels Dabelstein Chair of the Working Party on Aid Evaluation © OECD 1999 2 REPORT FROM THE COUNTRY PROGRAMME EVALUATION WORKSHOP, VIENNA, 11-12 MARCH 1999 TABLE OF CONTENTS FOREWORD.................................................................................................................................................. 2 GLOSSARY OF ACRONYMS AND DEFINITIONS .................................................................................. 4 CHAPTER 1 COUNTRY PROGRAMME EVALUATION: SYNTHESIS REPORT FROM THE WORKSHOP by Tim Conway and Simon Maxwell,* Overseas Development Institute (ODI).................. 6 CHAPTER 2 COUNTRY PROGRAMME EVALUATION: A STATE OF THE ART REVIEW A Desk Study Prepared by Tim Conway and Simon Maxwell*, Overseas Development Institute (ODI).. 38 CHAPTER 3 COUNTRY PROGRAMME EVALUATIONS IN THE NETHERLANDS’ DEVELOPMENT ASSISTANCE: THE CASE OF EGYPT By Jan Sterkenburg, Policy and Operations Evaluation Department Ministry of Foreign Affairs, Netherlands ............................................................... 85 CHAPTER 4 COUNTRY PROGRAMME EVALUATIONS WORKSHOP CASE STUDY TANZANIA By Camilla Lema, Elieshi Lema and Carin Salerno Swiss Agency for Development Co-operation Co- ordination Office, Tanzania.......................................................................................................................... 96 CHAPTER 5 COUNTRY ASSISTANCE EVALUATION IN THE MULTILATERAL DEVELOPMENT BANKS: APPROACHES, INSTRUMENTS AND EMERGING PRACTICES By the Multilateral Development Banks – Evaluation Co-operation Group ............................................................................. 114 CHAPTER 6 FRENCH EXPERIENCE WITH COUNTRY PROGRAMME EVALUATION By the French Ministry of Economy, Finance and Industry, and The Ministry Of Foreign Affairs, Paris........... 137 CHAPTER 7 ISSUES FOR COUNTRY EVALUATION THE CASE OF THE EU COUNTRY PROGRAMME IN RUSSIA By Dr. E. Caputo and Alexander Kotchegura for the European Commission, Brussels....................................................................................................................................................... 143 CHAPTER 8 COUNTRY ASSISTANCE STRATEGIES AS A MANAGEMENT AND EVALUATION INSTRUMENT FOR DONORS: SOME CONCLUSIONS DRAWN FROM GERMAN EXPERIENCE By Guido Ashoff, German Development Institute ..................................................................................... 150 CHAPTER 9 THE ANATOMY OF COUNTRY ASSISTANCE EVALUATION: CASE STUDY OF THE OED PHILIPPINES CAR by Gianni Zanini Operation Evaluation Department, World Bank........ 164 CHAPTER 10 COUNTRY PROGRAMME PLANNING AND COUNTRY PROGRAMME EVALUATION (CPE) WITHIN THE SWISS AGENCY FOR DEVELOPMENT AND CO-OPERATION (SDC) By Edita Vokral, Swiss Agency for Development Co-operation (SDC), Bern.............................. 171 CHAPTER 11 REAL PROGRESS: FIFTY YEARS OF USAID IN COSTA RICA By James W. Fox, Center for Development Information and Evaluation U.S. Agency for International Development (USAID) .................................................................................................................................................................... 184 © OECD 1999 3 GLOSSARY OF ACRONYMS AND DEFINITIONS 3M Macro, Meso and Micro (UNDP system of classifying projects by likely level of impact) ACP Asia, Caribbean and Pacific AsDB Asian Development Bank AfDB African Development Bank BoP Balance of Payments CAE Country Assistance Evaluation CAN Country Assistance Note (World Bank): CAR Country Assistance Review (World Bank) CCF Country Co-operation Framework (UNDP) CEE Central and Eastern Europe CPE Country programme Evaluation CSE Country Strategy Review (multilateral banks) CIR Country Impact Review (IFC) CIS Commonwealth of Independent States DAC Development Assistance Committee DFID Department for International Development ESW Economic and Sector Work (World Bank): EU European Union FDFA Political Affairs Division FDPE Federal Department of Public Economy FOFEA Federal Office of Foreign Economic Affairs HA/SDR Humanitarian Affairs and Swiss Disaster Relief HDI Human Development Index IDB Inter-American Development Bank IDR Institute for Development Research, Addis Ababa IDS Institute of Development Studies, University of Sussex IFC International Finance Corporation JICA Japanese International Co-operation Agency © OECD 1999 4 MDB Multilateral Development Bank MTR Mid-Term Review NGO Non-Governmental Organisation NPOs National Programme Officers ODA Official Development Assistance OECD Organisation for Economic Co-operation and Development SHD Sustainable Human Development (UNDP) SIDA Swedish International Development Agency TA / TC Technical Assistance / Technical Co-operation TOR Terms of Reference UNDP United Nations Development Programme USAID United States Agency for International Development WB World Bank © OECD 1999 5 CHAPTER 1 COUNTRY PROGRAMME EVALUATION: SYNTHESIS REPORT FROM THE WORKSHOP by Tim Conway and Simon Maxwell,* Overseas Development Institute (ODI) Introduction Since the mid-1980s, official development agencies have moved towards country programming. Many individual donors have made efforts to concentrate their aid upon a limited number of countries and to ensure that it is relevant and coherent with regard to the specific problems and opportunities of these countries. The shift towards country programming has been slower in some agencies than in others. Nonetheless, for many donors the partner country has become the logical unit of account and evaluation -- especially with the recent revival of interest in the role of the State as the key external determinant of aid performance2. Reflecting the shift towards country programming, from the late 1980s donors began to use country programme Evaluations (CPEs) to complement project or sectoral evaluations. A workshop on this topic was convened in Vienna by the DAC Expert Group on Aid Evaluation from 11th-12th March 1999, in order to review experience and to share lessons about what appear to be the strengths and weaknesses of different approaches to country programme evaluation. This workshop concentrated upon progress made since the first workshop on the subject, also held in Vienna, in May 1994. Twenty-one donor countries or multilateral aid organisations and seven partner countries were represented at Vienna in March 1999 (see Appendix 1.2). This paper provides an analytical overview of the presentations and discussions. Opening and closing with comments from the DAC and from the Austrian hosts, the workshop was structured as four plenary sessions in which delegates presented prepared papers. Between plenary sessions participants divided into four issue-based discussion groups. These small Working Groups met on three occasions over the two days, each group discussing a specific aspect of country programme evaluation. The conclusions of each of the groups were presented to the whole Workshop in the fourth and final plenary session on the afternoon of the second day. (See Appendix 1.1). The Overseas Development Institute (ODI) prepared a background paper for the Workshop. This reviewed a sample of the CPEs known to have been published between 1994 and 1999 and described the range of * This paper synthesises the proceedings of a workshop on country programme evaluation, held in Vienna in March 1999. It draws upon the presentations and discussions at that Workshop

Load more