Global Environmental Facility RF President’s Plenipotentiary United Nations Representative Office in the Development Program Northwest Federal District MID-TERM EVALUATION REPORT UNDP/GEF PROJECT BUILDING ENERGY EFFICIENCY IN THE NORTH WEST OF RUSSIA Atlas Award ID: 00059438 Project ID: 00074315 PIMS # 4131 Prepared by: Vesa Rutanen January, 2014 Acknowledgements Author of this mid-term evaluation would like to express his gratitude to all project stakeholders and other contributors met during the project evaluation mission in Arkhangelsk, Vologda, Pskov and Saint Petersburg in November 2013 and later in Moscow in December 2013, who provided valuable inputs and insights for conducting the evaluation as well as to the project management team in Saint Petersburg, the UNDP office in Moscow, the UNDP/GEF regional technical adviser in the Bratislava Regional Centre and the local co- ordinators in the cities visited for effectively organising the required meetings and other logistics of the mission, sharing in a timely manner the requested information and supporting the evaluation otherwise. 2 TABLE OF CONTENTS LIST OF ACRONYMS............................................................................................................................5 EXECUTIVE SUMMARY........................................................................................................................6 1. INTRODUCTION...........................................................................................................................11 1.1 Project background ............................................................................................................. 11 1.2 Purpose of the evaluation .................................................................................................. 12 1.3 Key issues to be addressed ............................................................................................... 12 1.4 The outputs of the evaluation and how will they be used............................................... 13 1.5 Methodology of the evaluation .......................................................................................... 13 1.6 Structure of the evaluation ................................................................................................. 15 2. THE PROJECT AND ITS DEVELOPMENT CONTEXT................................................................16 2.1 Project start and its duration ............................................................................................. 16 2.2 Problems that the project seeks to address ..................................................................... 16 2.3 Project objective .................................................................................................................. 17 2.4 Main stakeholders ............................................................................................................... 17 2.5 Expected key results and their status of implementation .............................................. 17 2.6 Status of Financial Delivery ............................................................................................... 23 3. FINDINGS AND CONCLUSIONS OF THE EVALUATION...........................................................24 3.1 Project formulation.............................................................................................................. 24 3.1.1 Project relevance ............................................................................................................... 24 3.1.2 Project design and implementation approach ................................................................... 24 3.1.3 Country ownership/driveness ............................................................................................ 27 3.1.4 Stakeholder participation ................................................................................................... 27 3.1.5 Replication approach ......................................................................................................... 27 3.1.6 Cost-effectiveness .............................................................................................................. 28 3.1.7 Sustainability ..................................................................................................................... 28 3.1.8 Linkages between project and other interventions within the sector ................................. 29 3.1.9 Management arrangements .............................................................................................. 29 3.2 Project implementation ....................................................................................................... 30 3.2.1 Project implementation and management approach and arrangements .......................... 30 3.2.2 Project administration and financial management ............................................................ 33 3.2.3 Monitoring, evaluation and adaptive management ........................................................... 34 3.2.4 Contribution of Implementing and Executing Agencies ..................................................... 36 3.2.5 Stakeholder involvement, outreach and co-ordination with other related initiatives ......... 37 3.2.6 Identification and management of risks ............................................................................. 38 3.3 Results .................................................................................................................................. 38 3.3.1 Attainment of project objective, outcomes and outputs ..................................................... 38 3.3.2 Project impact and prospects for sustainability ................................................................. 43 3.4 Summary of ratings ............................................................................................................ 45 3 4. RECOMMENDATIONS.................................................................................................................46 4.1 Corrective actions for the design, duration, implementation, monitoring and evaluation of the project ..............................................................................................................................................46 4.2 Actions to strengthen the ownership, manage potential risks and reinforce the benefits from the project with due emphasis on sustainability aspects ......................................................................... 49 5. LESSONS LEARNED............................................................................................................. .....50 ANNEXES.............................................................................................................................................51 Annex 1: Terms of reference of the evaluation ................................................................................. 52 Annex 2: Itinerary .............................................................................................................................. 61 Annex 3: Meetings during the project evaluation mission ............................................................... 63 Annex 4: Complementary post mission questions to the project management team ...................... 65 Annex 5: List of documents reviewed ............................................................................................... 74 Annex 6: PSC inception meeting report ........................................................................................... 75 Annex 7a: Stakeholder matrix and institutional coordination as elaborated in the ProDoc .............. 77 Annex 7b: Summary of international financial institutions consultations as presented in the project document ........................................................................................................................................... 81 Annex 7c: Russia energy efficiency umbrella project ........................................................................ 83 Annex 8: Disbursements in 2011 - 2013 and planned budget for 2014 - 2015 ............................... 85 Annex 9: Project co-financing on 2011-2013 ................................................................................... 86 Annex 10: Comments by Stakeholders (only in case of discrepancies with evaluation findings and conclusions) ....................................................................................................................................... 89 4 LIST OF ACRONYMS AOEEC Arkhangelsk Regional Energy Efficiency Center APR Annual Project Review CO Country Office EBRD European Bank for Reconstruction and Development EE Energy Efficiency GEF Global Environmental Facility ISO International Standards Organisation GHG Greenhouse gases kW kilowatt kWh kilowatt-hour Mt megatonne MTE Mid-term Evaluation NEFCO Nordic Environment Finance Corporation NIB Nordic Investment Bank OFP Operational Focal Point PIR Project Implementation Review PMT Project Management Team PSC Project Steering Committee SBAA Standard Basic Assistance Agreement ToR Terms of Reference UNDP United Nations Development Programme UNIDO United Nations Industrial Development Organisation USD United States Dollar VSTU Vologda State Technical University 5 EXECUTIVE SUMMARY Brief description of project The project objective is “to build local capacities for and demonstrate local solutions to improved energy efficiency in new and existing buildings in the North West of Russia: Pskov,