2014 Terminal Evaluation of the UNEP/GEF project “Continued Institutional Strengthening Support for CEITs to meet the obligations of the Montreal Protocol” Valery Smirnov Evaluation Office 15 September 2014 1 TABLE OF CONTENTS Page Acknowledgements ............................................................................................................................. 4 List of acronyms .................................................................................................................................. 5 Project Profile ..................................................................................................................................... 6 Pезюме на русском языке ................................................................................................................ 7 Executive summary ............................................................................................................................. 17 I. Introduction ..................................................................................................................................... 23 II. The Evaluation .............................................................................................................................. 24 A. Objectives of the evaluation ......................................................................................................... 24 B. Overall approach and methods ..................................................................................................... 24 C. Limitations ................................................................................................................................... 26 III. The Project ................................................................................................................................... 26 A. Context ......................................................................................................................................... 26 1. Historical overview .................................................................................................................. 26 2. Other GEF funded activities in CEITs 30 3. Status of NOUs at the starting point of the project 32 B. Objectives and components .......................................................................................................... 32 C. Target areas/groups ...................................................................................................................... 34 D. Milestones/key dates in project design and implementation ........................................................ 35 E. Implementation arrangements ...................................................................................................... 35 1. Implementation approaches and management framework ....................................................... 35 2. The interaction of NOU with members of national steering committee and higher level authorities ....................................................................................................................... 37 3. Response to directives and guidance ........................................................................................ 38 4. Operational and political / institutional problems and constraints ........................................... 38 5. GEF environmental and social safeguards and requirements .................................................. 39 F. Project financing ........................................................................................................................... 39 G. Project partners ............................................................................................................................ 40 H. Changes in design during implementation ................................................................................... 40 I. Reconstructed Theory of Change of the project ............................................................................ 41 IV. Evaluation Findings ..................................................................................................................... 42 A. Strategic relevance ....................................................................................................................... 42 B. Achievement of outputs................................................................................................................ 43 1. Results by categories of outputs ............................................................................................... 43 2. Rating of achievement of output .............................................................................................. 51 C. Effectiveness: Attainment of project objectives and results ......................................................... 52 1. Direct outcomes from reconstructed TOC ............................................................................... 52 2. Likelihood of impact using RoTI and based on reconstructed TOC ........................................ 59 3. Achievement of project goal and planned objectives............................................................... 62 D. Sustainability and replication ....................................................................................................... 63 1. Socio-political sustainability .................................................................................................... 63 2. Financial resources ................................................................................................................... 64 3. Institutional framework ............................................................................................................ 65 4. Environmental sustainability .................................................................................................... 66 5. Catalytic role and replication ................................................................................................... 67 2 Page E. Efficiency ..................................................................................................................................... 67 F. Factors affecting performance ...................................................................................................... 68 1. Preparation and readiness ......................................................................................................... 68 2. Project implementation and management ................................................................................ 69 3. Stakeholder participation and awareness ................................................................................. 69 4. Country ownership and motivation .......................................................................................... 70 5. Evaluation of financial planning .............................................................................................. 71 6. UNEP supervision and backstopping ....................................................................................... 74 7. Monitoring and evaluation ....................................................................................................... 74 8. Use of GEF tracking tolls ......................................................................................................... 75 G. Complementarity with UNEP strategies and programmes........................................................... 75 V. Conclusions and Lessons Learned ............................................................................................... 75 A. Conclusions .................................................................................................................................. 76 B. Lessons Learned ........................................................................................................................... 81 VI. Annexes ......................................................................................................................................... 84 Annex 1- Terms of Reference ........................................................................................................... 84 Annex 2 - Questionnaire .................................................................................................................. 95 Annex 3 - List of individuals consulted and interviewed during the main evaluation phase ............10 1 Annex 4 - 1999 – 2006 GEF funding and co-financing for ODS phase out .....................................10 5 Annex 5 - CEIT ODS consumption and duration of GEF IS projects (separate Excel file) .............1 06 Annex 6 - Assessment of the quality of project design .....................................................................1 07 Annex 7 - Project budget ..................................................................................................................12 2 Annex 8 - Reconstructed Theory of Change .....................................................................................12 5 Annex 9 - Brief resume of the consultant .........................................................................................1 26 3 Acknowledgements I would like to express my very great appreciation to Ms. Harriet Matsaert the Evaluation Officer in the UNEP Evaluation and Oversight Unit for her patient guidance and valuable and constructive critiques during the planning and development of this evaluation, and writing the report. I would like to extend my sincere gratitude to Ms. Elisa