ASIAN DEVELOPMENT BANK Operations Evaluation Department PROJECT PERFORMANCE EVALUATION REPORT FOR INDONESIA In this electronic file, the report is followed by Management’s response. Performance Evaluation Report Project Number: 26190 Loan Number: 1339-INO August 2006 Indonesia: Capacity Building Project in the Water Resources Sector Operations Evaluation Department ABBREVIATIONS ADB – Asian Development Bank BAPEDALDA – Badan Pengendalian Dampak Lingkungan Daerah (provincial environment agency) BAPPENAS – Badan Perencanaan Pembangunan Nasional (National Development Planning Agency) CPO – central project office DGWR – Directorate General of Water Resources IHA – in-house adviser IRM – Indonesia Resident Mission m3 – cubic meters MPW – Ministry of Public Works NAD – Nanggroe Aceh Darussalam NGO – nongovernment organization NSIASP – Northern Sumatra Irrigated Agriculture Sector Project OED − Operations Evaluation Department OEM − operations evaluation mission PAMST – Policy Analysis and Management Support Team PCR – project completion report PISP – Participatory Irrigation Sector Project PP – peraturan pemerintah (national regulation) PPTA – project preparatory technical assistance PWRS – provincial water resources service RBO – river basin organization Rp – rupiah RRP – report and recommendation of the President TA – technical assistance WATSAL – Water Sector Adjustment Loan WATSAP – water sector adjustment program WRDC – water resources data center WRDM – water resources development and management GLOSSARY balai – provincial river basin management units dinas – local government organizations hydromet – hydrological and metrological NOTE The fiscal year (FY) of the Government ends on 31 December. Key Words capacity building; capacity development; water resources sector; water resources management; training; sustainability; water law; results-oriented management; development effectiveness; evaluation; lessons; operations evaluation department; performance evaluation Director General Bruce Murray, Operations Evaluation Department (OED) Director R. Keith Leonard, Operations Evaluation Division 1, OED Team Leader Jocelyn Tubadeza, Senior Evaluation Officer, OED Team Member Caren Joy Mongcopa, Senior Operations Evaluation Assistant, OED Operations Evaluation Department, PE-689 CONTENTS Page BASIC DATA iii EXECUTIVE SUMMARY v MAP ix I. INTRODUCTION 1 A. Evaluation Purpose and Process 1 B. Expected Results 2 II. DESIGN AND IMPLEMENTATION 2 A. Formulation 2 B. Rationale 3 C. Cost, Financing, and Executing Arrangements 4 D. Procurement, Construction, and Scheduling 5 E. Design Changes 5 F. Outputs 6 G. Consultants 6 H. Loan Covenants 7 I. Policy Framework 7 III. PERFORMANCE ASSESSMENT 7 A. Overall Assessment 7 B. Relevance 8 C. Effectiveness 9 D. Efficiency 13 E. Sustainability 14 IV. OTHER ASSESSMENTS 14 A. Socioeconomic Impact 14 B. Asian Development Bank Performance 14 C. Borrower Performance 15 V. ISSUES, LESSONS, AND FOLLOW-UP ACTIONS 15 A. Issues 15 B. Lessons 17 C. Follow-Up Actions 19 APPENDIXES 1. Project-Related Statistics 22 2. Project Outputs 25 3. SP-1 and Master’s Program, Bandung 31 4. Chronology of Significant Water Resources Sector Events 33 The guidelines formally adopted by the Operations Evaluation Department (OED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. Jonathan Cook and Peddy Suhardi were the consultants. To the knowledge of the management of OED, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. Attachment: Management’s Response BASIC DATA Capacity Building Project in the Water Resources Sector (Loan 1339-INO) PROJECT PREPARATION/INSTITUTION BUILDING Person- Amount Approval TA No. TA Name Type Months ($) Date 1859 Human Resources PPTA 33 600,000 29 Mar 1993 Development Project in the Water Resources Sector As per ADB KEY PROJECT DATA ($ million) Loan Documents Actual Total Project Cost $46.20 $22.95 ADB Loan Amount/Utilization 27.72 18.34 ADB Loan Amount/Cancellation 9.38 KEY DATES Expected Actual Fact-Finding 2–20 May 1994 Appraisal 24 Jul–5 Aug 1994 Loan Negotiations 7–8 November 1994 ADB Board of Directors’ Approval 6 December 1994 Loan Signing 10 January 1995 Loan Effectiveness 3 April 1995 3 April 1995 First Disbursement 1 June 1995 1 June 1995 Project Completion 30 June 2000 30 June 2002 Loan Closing 30 June 2000 29 November 2002 Months (Effectiveness to Completion) 62 86 BORROWER Republic of Indonesia EXECUTING AGENCY Directorate General of Water Resources Type of Mission No. of Missions No. of Person-Days Fact-Finding 1 57 Appraisal 1 78 Project Administration Inception 1 5 Review 10 142 Midterm Review 1 36 Project Completion 1 36 Operations Evaluation 1 62 ADB = Asian Development Bank, PPTA = project preparatory technical assistance. EXECUTIVE SUMMARY The Capacity Building Project in the Water Resources Sector (the Project) was executed by the Directorate General of Water Resources (DGWR) of the Ministry of Public Works (MPW) over 1995–2002. It was intended to strengthen national and provincial institutional capacity for sustainable, multisectoral, and economic management and use of water resources on a river basin basis. The project included three components: (i) strengthening the national policy and coordination framework, (ii) strengthening the capacity of regional institutions for multisectoral water resources development and management, and (iii) strengthening the capacity of DGWR. Overall, Indonesia has abundant water resources, with per person availability around four times the Asian average. Average country demand is 1.7% of runoff, but in some river basins in Java this approaches 50%. At the time of design, the major river systems of Java and the larger islands were beginning to suffer from significant stress due to catchment clearing, rapid urbanization and industrialization, resulting in increasing pollution. Project cost was $23 million, half of the appraised level. The Asian Development Bank (ADB) provided a loan of $18.3 million or 66% of the approved loan amount. Government contributions were only 25% of the $18 million planned, mainly due to devaluation of the rupiah (Rp) and fiscal problems resulting from the 1997–1998 Asian financial crisis. Project implementation was severely delayed initially; at the time of the midterm review in 1998, only about 10% of the loan had been drawn. Although disbursement accelerated thereafter, political changes caused problems—particularly decentralization under the Wahid government. The Project was completed 24 months late, in mid-2002. Significant changes occurred during implementation. A number of activities, particularly relating to the private sector, were deleted, while significant support was provided to two other projects: the Northern Sumatra Irrigated Area Sector project (NSIASP) and the Water Sector Adjustment Loan (WATSAL), which were supported via ADB and World Bank loans, respectively. Assistance to NSIASP has not proven useful or sustainable, due to the limited linkage between the projects. However, World Bank staff consider the contribution to WATSAL to have been invaluable (inter alia) in laying the foundation for the 2004 Water Law. The Project comprised 49 outputs in the three components. While almost all outputs contributed to capacity development, it is not clear from the project framework whether they were necessary and sufficient to lead to the desired outcomes. Future capacity-building projects need to be more explicit regarding their intended outcomes and the outputs required to achieve them. Some project activities have been successful. The provision of support to the diploma and master’s degree program in Bandung was valuable. Project irrigation training materials continue to be used, although other modules are not. The two most innovative aspects of the project— the in-house advisers and results-oriented management—have not survived, highlighting the difficulty of introducing institutional change, unless strongly demand driven. The Project is considered to have been relevant, though relevance was constrained by decentralization, which commenced in May 1999. The centralized project implementation approach largely bypassed the structural provincial organizations, and reduced the potential to build the capacity of the provincial water resources services. Project objectives align closely with ADB’s Water-for-All policy, but are less directly linked to country objectives than they were at design. vi It is difficult to define project effectiveness, as no outcome indicators were identified at design or monitored during implementation. The project has had some impact on policy, legislation, management, and operation, but in all cases less than planned. The water law is a positive outcome to which the Project contributed, but it is still awaiting the passage of supporting regulations. In terms of the hydrology network and data management, outcomes are constrained by lack of management support and reinvestment. The project is rated less effective. OEM rated the project less efficient, due to substantial delays, and the limited cost- effectiveness of provincial activities. However, some aspects of the project were cost-effective, including the conduct of training courses and local consultant recruitment. Consultant inputs totaled around 140 person-years, double the appraisal estimate, due to changes in scope and under-budgeting. The failure of the in-house advisers to meet their objectives contributed to increased