Final Evaluation Report
Total Page:16
File Type:pdf, Size:1020Kb
FINAL EVALUATION REPORT of the UNDP/GEF Full Size Project Armenia Improving the Energy Efficiency of Municipal Heating and Hot Water Supply Atlas Project ID: 00035799 PIMS: 1273 This Final Evaluation Report was prepared for UNDP Armenia by: Jiří Zeman, International Consultant July 2012 Table of content Acknowledgements .......................................................................................................................................... 5 Abbreviations and acronyms .......................................................................................................................... 6 1. Executive summary ................................................................................................................................. 7 1.1 Brief description of project ................................................................................................................ 8 1.2 Context and purpose of the evaluation .............................................................................................. 8 1.3 Main conclusions, recommendations and lessons learned ................................................................ 9 2. Introduction ........................................................................................................................................... 13 2.1 Purpose of the evaluation ................................................................................................................ 13 2.2 Key issues addressed ....................................................................................................................... 13 2.3 Methodology of the evaluation ........................................................................................................ 14 2.4 Structure of the evaluation ............................................................................................................... 14 3. The Project and its development context............................................................................................. 15 3.1 Problems that the project seeks to address ...................................................................................... 15 3.2 Immediate and development objectives of the project .................................................................... 16 3.3 Project start and its duration ............................................................................................................ 17 3.4 Main stakeholders ............................................................................................................................ 17 3.5 Results expected .............................................................................................................................. 18 4. Findings and conclusions ...................................................................................................................... 22 4.1 Project Formulation ......................................................................................................................... 22 4.1.1 Project Relevance and Implementation Approach .................................................................. 22 4.1.2 Analysis of Logical Framework (project logic/strategy, indicators) ....................................... 23 4.1.3 Lessons from other relevant projects incorporated into project implementation .................... 23 4.1.4 Country ownership/driveness .................................................................................................. 24 4.1.5 Stakeholder participation in the design phase ......................................................................... 24 4.1.6 Replication approach and sustainability strategy..................................................................... 24 4.1.7 Cost-effectiveness .................................................................................................................... 25 4.1.8 UNDP comparative advantage ................................................................................................ 25 4.1.9 Linkages between the project and other interventions within the sector ................................. 25 4.1.10 Management arrangements ...................................................................................................... 26 4.2 Project Implementation.................................................................................................................... 28 4.2.1 Implementation approach ........................................................................................................ 28 4.2.2 The logical framework used during implementation as a management and monitoring and evaluation tool ......................................................................................................................................... 29 4.2.3 Effective partnerships arrangements established for implementation of the project with relevant stakeholders involved in the country ......................................................................................... 29 4.2.4 Feedback from M&E activities used for adaptive management .............................................. 30 2 4.2.5 Financial planning and management ....................................................................................... 30 4.2.6 Monitoring and evaluation ....................................................................................................... 32 4.2.7 Management by the UNDP country office .............................................................................. 33 4.2.8 Co-financing and in-kind contributions ................................................................................... 34 4.3 Results ............................................................................................................................................. 36 4.3.1 Attainment of objectives .......................................................................................................... 36 4.3.2 Cost-effectiveness of Pilot Projects ......................................................................................... 41 4.3.3 Project Impact .......................................................................................................................... 42 4.3.4 Prospects of Sustainability ....................................................................................................... 43 4.3.5 Contribution to upgrading skills of the national staff .............................................................. 44 5. Recommendations .................................................................................................................................. 45 6. Lessons Learned .................................................................................................................................... 46 7. Annexes ................................................................................................................................................... 48 Annex 1: Summary of Pilot Projects………………………………………………………………………48 Annex 2: Summary of key project deliverables, reports, and studies…………………………………...51 Annex 3: Original Project Document LogFrame with revisions from the Inception Report and Project Revision .......................................................................................................................................................... 53 Annex 4: Revised LogFrame with revisions from the Inception Report and Project Implementation.57 Annex 5: Final evaluation TOR ................................................................................................................... 59 Annex 6: Itinerary ......................................................................................................................................... 84 Annex 7: List of persons interviewed ........................................................................................................... 88 Annex 8: List of documents reviewed .......................................................................................................... 90 Annex 9: Comments by stakeholders (only in case of discrepancies with evaluation findings and conclusions) .................................................................................................................................................... 92 3 Profile of the consultant: This final term evaluation was performed by Mr. Jiří Zeman, who has 20+ years of professional experience in energy efficiency, renewables, climate change and energy utilities primarily in Central and Eastern Europe and Central Asia. He has served as a Deputy Director of a leading energy efficiency consulting organization SEVEn, The Energy Efficiency Center in Prague, the Czech Republic, as a Solution Architect at a Utility Competence Center of Hewlett Packard for Central and Eastern Europe, Middle East and Africa, and since 2009 he serves as a freelance consultant. Mr. Zeman has developed, implemented and evaluated a series of UNDP/GEF energy efficiency projects in the region of Central and Eastern Europe and Central Asia. Contact details: Mr. Jiří Zeman Murmanská 5 100 00 Praha 10 Czech Republic Email: [email protected] Tel: +420-776818363 4 Acknowledgements The author of the final evaluation would like to express his gratitude to all project stakeholders whom he has met and interviewed during the project final evaluation missions in Armenia in July 2012 and who generously provided their views and opinions on project