Independent Terminal Evaluation Phase out of Hcfcs and Promotion
Total Page:16
File Type:pdf, Size:1020Kb
INDEPENDENT EVALUATION DIVISION OFFICE OF EVALUATION AND INTERNAL OVERSIGHT Independent Terminal Evaluation Phase out of HCFCs and promotion of HFC-free energy efficient refrigeration and air- conditioning systems in the Russian Federation through technology transfer UNIDO Project ID.: 105324 GEF Project ID.: 3541 INDEPENDENT EVALUATION DIVISION OFFICE OF EVALUATION AND INTERNAL OVERSIGHT Independent Terminal Evaluation Phase out of HCFCs and promotion of HFC-free energy efficient refrigeration and air-conditioning systems in the Russian Federation through technology transfer UNIDO Project ID.: 105324 GEF Project No.: 3541 Vienna, 2018 Distr. GENERAL ODG/EIO/IED/18/R.14 December 2018 Original: English This evaluation was managed by the responsible UNIDO Project Manager with quality assurance by the Independent Evaluation Division This document has not been formally edited. The designations employed and the presentation of the material in this document do not imply the expression of any opinion whatsoever on the part of the Secretariat of the United Nations Industrial Development Organization (UNIDO) concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Mention of company names and commercial products does not imply the endorsement of UNIDO. The views and opinions of the team do not necessarily reflect the views of the Governments and of UNIDO. TABLE OF CONTENTS Acknowledgements .................................................................................................................................. v Acronyms and Abbreviations ..............................................................................................................vi Glossary of evaluation-related terms ............................................................................................. vii Executive Summary .............................................................................................................................. viii I. Evaluation objectives, methodology and process .................................................................... 1 II. Country and Project background .................................................................................................. 4 2.1 Brief country context and project background................................................................. 4 2.2 Project summary ........................................................................................................................... 5 2.3 Project implementation arrangements and implementation modalities .............. 6 2.4 Major changes to project implementation ......................................................................... 9 2.5 Positioning of UNIDO project ................................................................................................ 11 2.6 Executing partner organisation ........................................................................................... 11 III. Project Theory of Change and progress to impact ............................................................ 13 IV. Project assessment .......................................................................................................................... 15 4.1 Project design .............................................................................................................................. 15 4.2 Relevance and ownership ...................................................................................................... 17 4.3 Effectiveness ................................................................................................................................ 19 4.4 Efficiency ....................................................................................................................................... 27 4.5 Sustainability of project outcomes/benefits .................................................................. 30 4.6 Progress to impact ..................................................................................................................... 31 4.7 Project coordination and management ............................................................................ 32 4.8 M&E design and implementation ........................................................................................ 34 4.9 Gender mainstreaming ............................................................................................................ 36 4.10 Performance of partners ...................................................................................................... 37 4.11 Overall project achievement ............................................................................................... 40 V. Conclusions, recommendations and lessons learned ........................................................ 43 5.1 Conclusions................................................................................................................................... 43 5.2 Recommendations ..................................................................................................................... 45 5.3 Lessons Learned ......................................................................................................................... 46 ANNEXES: ............................................................................................................................................. 47 Annex A Terms of Reference................................................................................................... 47 Annex B List of people met ...................................................................................................... 62 Annex C List of documents reviewed .................................................................................. 64 Annex D List of Project Stakeholders and Partners....................................................... 74 Annex E Summary of the main project achievements by outcome/output ........ 80 Annex F Summary on Project Identification and Financial Data ............................. 85 iii Table 1: Project Factsheet ................................................................................................................. 5 Table 2: Main Stakeholder Mapping ............................................................................................. 9 Table 3: Planned Stakeholders versus Effective Stakeholders ....................................... 10 Table 4: Distribution of budget among the different components ............................... 16 Table 5: Planned Project finance and co-finance ................................................................. 16 Table 6: RF’s consumption of HCFcs versus MP target from the baseline to 2015 25 Table 7: ODS consumption from the baseline to 2015 ..................................................... 26 Table 8: Comparative analysis planned budget and effective budget spending ..... 29 Table 9: Project co-finance ............................................................................................................ 38 Table 10: Overall assessment of Project achievements..................................................... 40 iv Acknowledgements Thanks are due to UNIDO’s Department of Environment and the Independent Evaluation Division for all the guidance provided during this Terminal Evaluation. Thanks, are also due to UNIDO´s Project Manager, Mr. Yury Sorokin and UNIDO country representative in the Russian Federation, Mr. Sergey Korotkov for all the support during the evaluation and assistance provided in the field mission. A special note of gratitude to former members of the Project Management Unit: Mr. Artem Kushnerev, Mr. Alexander Lyubeshkin and Mr. Vasily Tselikov for exceptional efforts put into support of the efficient organization of this Terminal Evaluation and for all the information shared. Sincere appreciation is also expressed to Mr. Sergey Vasiliev, Focal Point for Ozone Agreements, Department of International Cooperation of the Ministry of Natural Resources and Environment of the Russian Federation for his institutional memory of the Project implementation. The Evaluation team is also deeply grateful to all stakeholders who made their time available for providing information, discussing and answering questions and who have contributed with their views and experience to the present report. The evaluation team was composed of one senior international evaluation consultant acting as the team leader, Ms. Teresa Amador, and the national evaluation consultant, Mr. Maxim Eliseev. The tasks of each team member are specified in the job descriptions annexed to the Terms of Reference. v Acronyms and Abbreviations CIIC UNIDO Centre for International Industrial Cooperation in the RF CIS Commonwealth of Independent States FSP Full Size Project ET Evaluation Team PIF Project Identification Form GEF Global Environment Facility GDP Gross Domestic Product GHG Greenhouse Gas HCFC Hydrochlorofluorocarbons HFC Hydrofluorocarbons HVAC Heating, ventilation, and air conditioning ICSTI International Centre for Scientific and Technical Information ITPO Investment and Technology Promotion Offices (UNIDO) M&E Monitoring and Evaluation MIT Ministry of Industry and Trade MMT Million Metric Tones MNRE Ministry of Natural Resources and Environment of the RF MoP Meeting of the Parties MP Montreal Protocol on Substances that Deplete the Ozone Layer (1987) MTR Mid Term Review ODG/EVA UNIDO Office for Independent