United Nations Development Programme Government of the Russian Federation Terminal Evaluation of UNDP/GEF Project: TRAnsforming the Market for Efficient Lighting (TRAMEL) (GEF Project ID: 3658; UNDP PIMS ID: 4160) Terminal Evaluation Report Mission Members: Mr. Roland Wong, International Consultant Mr. Alexei Zakharov, National Consultant May 2017 UNDP – Government of the Russian Federation Terminal Evaluation of Transforming the Market for Efficient Lighting TABLE OF CONTENTS Page SYNOPSIS .............................................................................................................................................................. III EXECUTIVE SUMMARY .......................................................................................................................................... IV ABBREVIATIONS ................................................................................................................................................. VIII 1. INTRODUCTION ........................................................................................................................................ 1 1.1 PURPOSE OF THE EVALUATION ................................................................................................................................ 1 1.2 SCOPE AND METHODOLOGY ................................................................................................................................... 1 1.3 STRUCTURE OF THE EVALUATION ............................................................................................................................. 3 2. PROJECT DESCRIPTION AND DEVELOPMENT CONTEXT ............................................................................ 4 2.1 PROJECT START AND DURATION .............................................................................................................................. 4 2.2 PROBLEMS THAT TRAMEL PROJECT SOUGHT TO ADDRESS .......................................................................................... 4 2.3 GOAL AND OBJECTIVE OF TRAMEL PROJECT ............................................................................................................. 4 2.4 BASELINE INDICATORS ESTABLISHED ......................................................................................................................... 4 2.5 MAIN STAKEHOLDERS ........................................................................................................................................... 6 2.6 EXPECTED RESULTS ............................................................................................................................................... 6 3. FINDINGS ................................................................................................................................................. 7 3.1 PROJECT DESIGN AND FORMULATION ....................................................................................................................... 7 3.1.1 Analysis of Project Planning Matrix ..................................................................................................... 8 3.1.2 Risks and Assumptions ........................................................................................................................ 9 3.1.3 Lessons from Other Relevant Projects Incorporated into TRAMEL Project Design .............................. 9 3.1.4 Planned Stakeholder Participation .................................................................................................... 10 3.1.5 Replication Approach ........................................................................................................................ 10 3.1.6 UNDP Comparative Advantage ......................................................................................................... 10 3.1.7 Linkages between TRAMEL Project and Other Interventions within the Sector ................................ 10 3.1.8 Management Arrangements ............................................................................................................. 11 3.2 PROJECT IMPLEMENTATION .................................................................................................................................. 12 3.2.1 Adaptive Management ...................................................................................................................... 13 3.2.2 Partnership Arrangements ................................................................................................................ 14 3.2.3 Feedback from M&E Activities Used for Adaptive Management ...................................................... 16 3.2.4 Project Finance .................................................................................................................................. 17 3.2.5 M&E Design at Entry and Implementation ........................................................................................ 18 3.2.6 Performance of Implementing and Executing Entities ...................................................................... 21 3.2.7 Overall Results ................................................................................................................................... 23 3.2.8 Outcome 1: Improved efficient lighting standards and policy framework ........................................ 24 3.2.9 Outcome 2: Supply chain for energy efficient lighting is strengthened ............................................. 31 3.2.10 Outcome 3: Penetration of energy efficient lighting increases in Moscow ....................................... 38 3.2.11 Outcome 4: Energy efficient street lighting is adopted in the Volga Federal District ........................ 43 3.2.12 Relevance........................................................................................................................................... 46 3.2.13 Effectiveness and Efficiency ............................................................................................................... 47 3.2.14 Country Ownership and Drivenness ................................................................................................... 47 3.2.15 Mainstreaming .................................................................................................................................. 48 3.2.16 Sustainability of Project Outcomes .................................................................................................... 48 3.2.17 Impacts……………………………………………………………………………………………………………………………. ............ 49 4. CONCLUSIONS, RECOMMENDATIONS AND LESSONS ............................................................................. 52 4.1 CORRECTIVE ACTIONS FOR THE DESIGN , IMPLEMENTATION , MONITORING AND EVALUATION OF THE PROJECT ........................ 53 4.2 ACTIONS TO FOLLOW UP OR REINFORCE INITIAL BENEFITS FROM THE PROJECT ................................................................. 53 Terminal Evaluation i May 2017 UNDP – Government of the Russian Federation Terminal Evaluation of Transforming the Market for Efficient Lighting 4.3 PROPOSALS FOR FUTURE DIRECTIONS UNDERLINING MAIN OBJECTIVES .......................................................................... 53 4.4 BEST AND WORST PRACTICES IN ADDRESSING ISSUES RELATING TO RELEVANCE , PERFORMANCE AND SUCCESS ........................ 54 APPENDIX A - MISSION TERMS OF REFERENCE FOR TRAMELPROJECT TERMINAL EVALUATION .......................... 57 APPENDIX B - MISSION ITINERARY (FOR MARCH 2017) ....................................................................................... 62 APPENDIX C - LIST OF PERSONS INTERVIEWED .................................................................................................... 65 APPENDIX D - LIST OF DOCUMENTS REVIEWED ................................................................................................... 67 APPENDIX E - GHG EMISSION REDUCTION CALCULATION REPORT ...................................................................... 68 APPENDIX F - COMPLETED TRACKING TOOL ......................................................................................................... 77 APPENDIX G - PROJECT LOGFRAME MATRIX FOR TRAMEL PROJECT (FROM APRIL 2010) ..................................... 79 APPENDIX H - EVALUATION QUESTIONS .............................................................................................................. 83 APPENDIX I - RESPONSES TO COMMENTS RECEIVED ON DRAFT TE REPORT ........................................................ 86 APPENDIX J - ENGLISH TRANSLATION OF DECREE #898 ........................................................................................ 90 APPENDIX K - EVALUATION CONSULTANT AGREEMENT FORM ............................................................................ 91 Terminal Evaluation ii May 2017 UNDP – Government of the Russian Federation Terminal Evaluation of Transforming the Market for Efficient Lighting SYNOPSIS Title of UNDP supported GEF financed project: Russia: Transforming the Market for Efficient Lighting UNDP Project ID: PIMS 4160 GEF Project ID: 3658 Evaluation time frame: April 8, 2010 to March 17, 2017 CEO endorsement date : April 7, 2010 Project implementation start date : April 8, 2010 Project end date : April 30, 2017 Date of evaluation report: May 28, 2017 Region and Countries included in the project: Russian Federation GEF Focal Area Objective: SP-1 (for GEF-4): Promoting energy efficiency in residential and commercial buildings Implementing partner and other strategic