IN.464-17 20 December 2017 Performance Evaluation Report Municipal Services Development Project and Municipal Services Development Project Phase 2 (Georgia) (Loans 2441 and 2534) The attached report is circulated at the request of the Director General, Independent Evaluation Department. For Inquiries: Walter Kolkma, Independent Evaluation Department (Ext. 4189) Ari Perdana, Independent Evaluation Department (Ext. 1891) Performance Georgia: Municipal Services Evaluation Development Project and Municipal Report Services Development Project Phase 2 Independent Evaluation Raising development impact through evaluation Performance Evaluation Report December 2017 Georgia: Municipal Services Development Project and Municipal Services Development Project Phase 2 This document is being disclosed to the public in accordance with ADB's Public Communications Policy 2011. Reference Number: PPE: GEO 2017-19 Project Numbers: 41198 and 43171 Loan Numbers: 2441-GEO(SF) and 2534-GEO(SF) Independent Evaluation: PE-803 NOTES (i) In this report, “$” refers to US dollars. (ii) For an explanation of rating descriptions used in Asian Development Bank evaluation reports, see ADB. 2006. Guidelines for Preparing Performance Evaluation Reports for Public Sector Operations. Manila Director General Marvin Taylor-Dormond, Independent Evaluation Department (IED) Deputy Director General Veronique Salze-Lozac’h, IED Director Walter Kolkma, Thematic and Country Division, IED Team leader Ari Perdana, Evaluation Specialist, IED Team members Ma. Patricia Lim, Senior Evaluation Officer, IED Jennifer Llaneta, Evaluation Assistant, IED (until October 2017) Christine Grace Marvilla, Evaluation Assistant, IED (from October 2017) The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area. Abbreviations ADB – Asian Development Bank DMF – design and monitoring framework IED – Independent Evaluation Department MDF – Municipal Development Fund MSDP – Municipal Services Development Project O&M – operation and maintenance PCR – project completion report PPER – project performance evaluation report PPMS – project performance management system PVR – project completion report validation RRP – report and recommendation of the President UWSCG – United Water Supply Company of Georgia Currency Equivalents At Appraisal At Completion At Evaluation MSDP (1 August 2008) (5 September 2012) (24 October 2017) GEL1.00 = $0.710257 $0.604193 $ 0.400000 $1.00 = GEL1.407904 GEL1.65510 GEL2.500000 MSDP Phase 2 (29 June 2009) (5 April 2013) (24 October 2017) GEL1.00 = $0.605455 $0.603172 $ 0.400000 $1.00 = GEL1.651650 GEL1.65790 GEL2.500000 Contents Acknowledgments vii Basic Data ix Executive Summary xi Project Map xv Chapter 1: Introduction 1 A. Evaluation Purpose and Process 1 B. Expected Impacts, Outcomes, and Outputs 2 Chapter 2: Design and Implementation 3 A. Rationale 3 B. Time, Cost, Financing, and Executing Arrangements 4 C. Procurement, Construction, Consultants, and Scheduling 6 D. Safeguard Arrangements and Gender Action Plan 8 E. Design Changes 8 F. Loan Covenants and Monitoring and Reporting Arrangements 9 Chapter 3: Performance Assessment 12 A. Relevance 12 B. Effectiveness 14 C. Efficiency 18 D. Sustainability 20 Chapter 4: Other Assessments 23 A. Preliminary Assessment of Development Impacts 23 B. Performance of the Asian Development Bank 25 C. Performance of the Borrower and Executing Agency 26 Chapter 5: Overall Assessment, Lessons, and Recommendations 27 A. Overall Assessment 27 B. Issues 29 C. Lessons and Recommendations 29 Appendixes 1. Design and Monitoring Framework 33 2. List of Municipalities and Projects Visited 38 3. Economic Analysis of Selected Water Supply Subprojects 39 4. Household Survey Results 41 Acknowledgments This report was prepared by a team led by Ari Perdana, Evaluation Specialist, Independent Evaluation Department (IED) of the Asian Development Bank (ADB) under the supervision of Walter Kolkma, Director, Thematic and Country Division. The team members were Ma. Patricia Lim, Jennifer Llaneta (until October 2017) and Christine Grace Marvilla (from October 2017). The consultants for the report were Peter Choynowski, Irakli Kaviladze and Giorgi Gigauri. Matsatso Tepnadze from the Ministry of Regional Development and Infrastructure of Georgia participated in the independent evaluation mission as part of the evaluation capacity development component of the study. IED staff Tomoo Ueda and Garrett Kilroy provided comments on a draft. Ian Green, international consultant, and Shinjini Mehta, Urban Development Specialist at the South Asia Department, ADB, served as external peer reviewers for a mature draft. The team wishes to thank officials of the Municipal Development Fund who supported the evaluation, particularly Shalva Kokochashvili, and provided comments on a draft, were considered in finalizing the report. The officials of the Ministry of Regional Development, Ministry of Finance, United Water Supply Company of Georgia, and municipal governments, as well as representatives of the World Bank, European Investment Bank, European Bank of Reconstruction and Development, and Japan International Cooperation Agency provided valuable information during the independent evaluation mission. The team acknowledges the support and cooperation of household members, who took part in the survey and focus group discussions during the evaluation mission as part of the qualitative data collection for the evaluation The team is grateful to the ADB Central and West Asia Department management and staff in ADB headquarters and in the Georgia resident mission for their comments on the draft of the report and for facilitating the evaluation mission. IED retains full responsibility for this report. Basic Data LOAN 2441 (SF) - Municipal Services Development Project Key Project Data ($ million) ADB Loan Documents Actual Total Project Cost 63.25 51.15 Foreign Exchange Cost 63.25 51.15 Local Currency Cost 0.00 0.00 KEY DATES Expected Actual Appraisal Waived during 1 April 2008 Management Review Meeting Loan Negotiations Started 10 July 2008 Loan Negotiations Completed 15 July 2008 Board Approval 12 September 2008 Loan Agreement 11 November 2008 Loan Effectiveness 10 January 2009 12 December 2008 Loan Closing 30 June 2013 27 February 2013 DMC: Georgia Executing Agency: Municipal Development Fund MISSION DATA No. of Missions No. of Person-Days Type of Mission Fact-Finding 1 103 Inception 1 16 Special Project Administration 0 0 Project Administration Review 5 52 Mid-term Review 1 22 Project Completion Review 1 44 PROJECT PERFORMANCE REPORT RATINGS Implementation Period Development Objective Implementation Progress 1 January 2011–30 September 2012 On track DMC = developing member country LOAN 2534 (SF) - Municipal Services Development Project Phase 2 Key Project Data ($ million) ADB Loan Documents Actual Total Project Cost 41.5 37.28 Foreign Exchange Cost 41.5 37.28 Local Currency Cost 0.0 0.0 KEY DATES Expected Actual Appraisal Waived during 28 April 2009 Management Review Meeting Loan Negotiation Started 10 June 2009 Loan Negotiation Completed 10 June 2009 Board Approval 28 July 2009 Loan Agreement 20 August 2009 Loan Effectiveness 19 October 2009 28 October 2009 Loan Closing 30 June 2014 30 June 2014 DMC: Georgia Executing Agency: Municipal Development Fund MISSION DATA No. of Missions No. of Person-Days Type of Mission Fact Finding 1 14 Inception 1 30 Special Project Administration 0 Project Administration Review 3 62 Mid-term 1 15 Project Completion Review 1 40 PROJECT PERFORMANCE REPORT RATINGS Implementation Period Development Objective Implementation Progress 1 January 2011–30 April 2013 On track DMC = developing member country Executive Summary Georgia became a member of ADB in 2007. The Municipal Services Development Project (MSDP), approved in September 2008, was the first Asian Development Bank (ADB) operation in the country. A year after MSDP approval, at the government’s request, ADB approved MSDP Phase 2 in 2009, to provide additional financing to rehabilitate poorly maintained infrastructure due to lack of investment and maintenance post-independence from the Soviet Union. Both MSDP and MSDP Phase 2 rehabilitated many small-scale infrastructure projects across Georgia. IED found MSDP and MSDP Phase 2 responsive to Georgia’s need for urban infrastructure upgrading. However, design shortcomings limited the projects’ achievements and development impacts, and institutional capacity building of the executing agency and municipal governments. While the subprojects were executed in a timely manner, the limited progress in reforming the water tariffs raises concern on the project’s sustainability. This project performance evaluation report (PPER) governments, which used them to finance the presents the findings of an independent rehabilitation of water systems and roads evaluation of the Asian Development Bank (ADB) subprojects. MSDP funded 71 subprojects in 44 Municipal Services Development Project (MSDP) municipalities, while MSDP Phase 2 funded 38 and Municipal Services Development