United Nations Environment Programme Global Environment Facility (GEF) TERMINAL EVALUATION – FINAL REPORT “Gambia – Adoption of Ecosystem Approach for Integrated Implementation of MEAs at National and Divisional Levels” “Data Flow System and Indicators to Enhance Integrated ManagementFinal of Global Environmental Issues in Croatia” “Enhanced Regulatory and Information Systems for Integrated Implementation of MEAs in Kenya” Hugo Navajas Evaluation Office July 2016 CONTENTS Acronyms 5 Executive Summary 6 I. INTRODUCTION 11 A. Introduction of the Projects 11 B. Objectives, Approach and Limitations of the Evaluation 13 II. THE PROJECTS 14 A. Context 14 A.1 The Gambia 14 A.2 Croatia 15 A.3 Kenya 17 B. Project Goals, Objectives and Outcomes 19 C. Target Groups and Areas 21 D. Milestones 21 E. Implementation 21 F. Project Financing 21 G. Partners 21 H. Changes to Design during Implementation 21 I. Reconstructed Theory of Change 22 III. EVALUATION FINDINGS 32 A. Strategic Relevance 32 B. Achievement of Outputs 34 B.1 The Gambia 34 B.2 Croatia 40 B.3 Kenya 46 C. Effectiveness 51 C.1 The Gambia 52 C.2 Croatia 54 C.3 Kenya 56 D. Sustainability and Replication 59 2 D.1 The Gambia 59 D.2 Croatia 59 D.3 Kenya 60 E. Efficiency 61 E.1 The Gambia 61 E.2 Croatia 62 E.3 Kenya 62 F. Factors affecting Performance 63 F.1 Preparation and Readiness 63 F.1.1 The Gambia 64 F.1..2 Croatia 65 F.1.3 Kenya 65 F.2 Project Implementation and Management 66 F.2.1 The Gambia 66 F.2.2 Croatia 66 F.2.3 Kenya 67 F.3 Stakeholder Participation 67 F.4 Communications and Public Awareness 69 F.5 Country Ownership and Driven-ness 70 F.6 Financial Planning and Management 71 F.7 Supervision, Guidance and Technical Backstopping 72 G. Monitoring and Evaluation 73 III. CONCLUSIONS 73 IV. LESSONS LEARNED 78 V. RECOMMENDATIONS 83 FIGURES: 1. Causal Pathways linking Outputs to Outcomes: The Gambia 24 2. Causal Pathways linking Outputs to Outcomes: Croatia 27 3. Causal Pathways linking Outputs to Outcomes: Kenya 30 4. Gambia: Project Outcomes, Success Indicators and Outputs 35 5. The NEA Website Home Page: February 2016 37 3 6. Community Action Plan for Tumani Tenda Village: Matrix of Problems and Opportunities 39 7. Croatia: Project Outcomes, Success Indicators and Outputs 40 8. DFS: Selected National Environmental Indicators for UN Conventions 42 ANNEXES 1. Project Ratings 2. Terms of Reference 3. Project Costs and Co-financing 4. Terms of Reference 5. Persons Interviewed 6. Bibliography _____________________________________________________________________________________ This report is based on the evaluator’s findings and does not necessarily reflect the views of UNEP, national executing agencies or other project partners. I would like to thank project focal points and participants in The Gambia, Croatia and Kenya for generously contributing their time and ideas to the evaluation. In particular, the assistance given by Dr. Ndey Bakurin, Leroy Muhammad Gomez (The Gambia), Hana Mesic (Croatia), Dr. Kennedy Ondimu and Paul Nguru (Kenya) was essential for the country visits and highly appreciated. Thank you! 4 ACRONYMS ANRWG Agriculture & Natural Resources Working Group ANRE Agriculture, Natural Resource & Environment Sub-committee BSP Bali Strategic Plan CAEN Croatia Agency for Environmental Management CAP Community Action Plan CDM Clean Development Mechanism CSE Centre de Suivi Ecologique (Dakar) DFS Data Flow System EA Environmental Audit EEA European Environment Agency EIA Environmental Impact Assessment ENDA Environnement et Développement du Tiers Monde (Sénégal) ESA Environmentally Sensitive Area EU European Union GEAP Gambian Environmental Action Plan GEF Global Environment Facility GIS Geographic Information System GHG Greenhouse Gases IIMS Integrated Information Management System IMS Information Management System IUCN International Union for Conservation of Nature NCSA National Capacity Self-Assessment NEA National Environment Agency (The Gambia) NEMA National Environmental Management Agency (Kenya) NEPAD New Partnership for Africa’s Development NFP National Focal Point MEA Multilateral Environmental Agreement MENP Ministry of Environment & Nature Protection (Croatia) MSP Medium-size Project MTE Mid-term Evaluation NAPA National Adaptation Plan of Action NBSAP National Biodiversity Strategy and Action Plan NGO Non-governmental Organization NPC National Project Coordinator PIR Project Implementation Review PRA Participatory Rural Appraisal ROtI Review of Outcomes to Impacts TAC Technical Advisory Committee TE Terminal Evaluation ToC Theory of Change UNEP United Nations Environment Programme UNCBD United Nations Conference for Conservation of Biodiversity UNFCCC United Nations Conference for Climate Change UNCCD United Nations Conference on Desertification and Land Degradation VDC Village Development Committee 5 EXECUTIVE SUMMARY 1. This report presents the terminal evaluations of “Gambia – Adoption of Ecosystem Approach for Integrated Implementation of MEAs at National and Divisional Levels”; “Data Flow System and Indicators to Enhance Integrated Management of Global Environmental Issues in Croatia”; and “Enhanced Regulatory and Information Systems for Integrated Implementation of MEAs” in Kenya. Although this was not a thematic evaluation and country projects were assessed separately, there are a number of common findings and issues that lend themselves to comparative analysis, and are relevant to the implementation of the Rio Conventions, capacity building and environmental management. 2. The three projects were relevant to both national and global environmental priorities. Their design was based on the findings of National Capacity Self-Assessments (NCSAs) and supported the Bali Strategic Plan (BSP) for Technology Support and Capacity Building. All sought to integrate data collection, monitoring and/or reporting mechanisms for the main Rio Conventions (UNFCCC, UNCBD, UNCCD) at country levels through improved coordination between focal points and integrated information systems. They represent a first generation of UNEP-GEF projects that sought to implement NCSA priorities and in doing so, assist the BSP’s implementation at the country level. There were also linkages with national decentralization policies in The Gambia and environmental impact assessment legislation in Kenya. The inter- institutional dynamics of these projects supported the core environmental mandates of national executing agencies and broadened the range of interaction to local government and community levels. Project relevance in Croatia was offset by the country’s accession to the EU in 2013, bringing new environmental policies, indicators, data collection and compliance requirements. 3. All projects were able to deliver most of the planned outputs and deliverables by the end of their terms. To enable this, project extensions – often prolonged – were requested and approved by UNEP; for example, the project in Kenya was commenced in 2009 and remains open. A number of outputs and products have stood out for their quality and actual (or potential) impact. In The Gambia, the ecosystems approach to natural resource planning was successfully applied in two pilot villages, leading to the design of Community Action Plans and execution of conservation activities; the pilot plans have helped both villages mobilize additional cooperation. In Croatia, data flow systems were designed for 27 environmental indicators with the aim of strengthening vertical and horizontal institutional linkages to support MEA implementation. An integrated information management portal with clearinghouse functions in Kenya is facilitating the work of convention desk officers and has the potential to raise synergies and feedback during convention monitoring and reporting cycles. Training on environmental impact assessments, environmental audits, valuation of environmental services and indicator data flows was imparted at national and sub-national levels, feeding into pilot planning processes. 4. Outputs were delivered, yet project objectives and outcomes were only partially reached in relation to their indicators. Impact was undermined by late project starts and slow implementation, administrative delays and changing national contexts. Unrealistic expectations 6 were nurtured in project design by assumptions and indicators that were largely outside project control. As a result, overall effectiveness and impact were not significant and are rated as moderately satisfactory to moderately unsatisfactory by the evaluator. 5. Objectives and outcomes that envisioned permanent MEA coordination mechanisms or sought to integrate management and implementation practices were undermined by inconsistencies between the indicators, technical definitions, formats, and reporting cycles of the different Rio Conventions. Such incompatibilities cannot not be resolved nationally and need to be addressed by the convention Secretariats. In Croatia, only 1 of 27 selected environmental indicators was compatible with the three main conventions. Convention focal points and desk officers acknowledge the limited space they have to influence convention indicators or formats, yet consider that the priority is the quality and availability of data. Kenya’s National Environmental Management Agency (NEMA) has advanced towards streamlining data access and management through the integrated information management system (IIMS), which has been operational since 2015. Conversely, an integrated information system and portal contracted in The Gambia was contracted to