Validation Report August 2017

People’s Republic of : Flood Management Sector Project

Reference Number: PVR: 2017-510 Project Number: 37641-013 Loan Number: 2244

ABBREVIATIONS

ADB – Asian Development Bank CEMP – consolidated environmental management plan CNY – Chinese yuan DMF – design and monitoring framework EAMF – environmental assessment and management framework EIRR – economic internal rate of return EMP – environmental management plan HPG – Hunan Provincial Government IEE – initial environmental examination LPMO – local project management office M&E – monitoring and evaluation NCB – national competitive bidding O&M – operation and maintenance PFD – Hunan Provincial Finance Department PPMO – provincial project management office PRC – People’s Republic of China PWRD – Provincial Water Resources Department TA – technical assistance

NOTE

In this report, “$” refers to US dollars.

Director General M. Taylor-Dormond, Independent Evaluation Department (IED) Deputy Director General V. Salze-Lozac’h, IED Director W. Kolkma, Thematic and Country Division, IED Team Leader G. Kilroy, Evaluation Specialist, Sector and Project Division, IED

The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. The final ratings are the ratings of IED and may or may not coincide with those originally proposed by the consultant(s) engaged for this report.

In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

PROJECT BASIC DATA Project Number 37641-013 PCR Circulation Date 26 Aug 2016 Loan Number 2244 PCR Validation Date Aug 2017 Project Name Hunan Flood Management Sector Project Sector and Agriculture, natural Rural flood protection—Rural water policy, Subsector resources, and rural institutional and capacity development development Strategic Agenda Environment sustainable growth Inclusive economic growth Safeguard Environment B Categories Involuntary Resettlement A Indigenous Peoples A Country People’s Republic of China Approved Actual ($ million) ($ million) ADB Financing ADF: 0.00 Total Project Costs 497.40 567.60 ($ million) OCR: 200.00 Loan 200.00 191.17 Borrower 297.40 376.40 Hunan Provincial 2.20 3.00 Government Municipal/County 295.20 373.40 governments Beneficiaries 0.00 0.00 Others 0.00 0.00 Cofinancier Total Cofinancing 0.00 0.00 Approval Date 29 Jun 2006 Effectiveness Date 2 Jan 2007 9 Jan 2007 Signing Date 4 Oct 2006 Closing Date 31 Mar 2013 31 Dec 2014 Project Officers Location From To R. Bolt ADB headquarters Jun 2006 Mar 2007 Y. Kobayashi ADB headquarters Apr 2007 Dec 2014 IED Review Director W. Kolkma, Thematic and Country Division Team Leader G. Kilroy, Evaluation Specialist, Sector and Project Division* ADB = Asian Development Bank; ADF = Asian Development Fund; IED = Independent Evaluation Department, OCR = ordinary capital resources; PCR = project completion report. *Team members: T. Ueda (Principal Evaluation Specialist), M.J. Dimayuga (Senior Evaluation Officer), I. Green (Consultant)

I. PROJECT DESCRIPTION

A. Rationale

1. A number of cities and industrial centers in the People’s Republic of China (PRC) are in the vicinity of major rivers, which increased the probability of risk from periodic flooding, and consequently adversely affected economic activities. Hunan Province borders the middle portion of the Yangtze River, which had major floods in 1998. This prompted a review of policies and practice on flood protection. The project supported the policies and plans for flood protection investments of the PRC and the Hunan Provincial Government (HPG).1 Despite the sector and subsector classification of Agriculture, Natural Resources and Rural Development, which cover natural resources and rural development, the project was primarily conceived to reduce flood risks

1 ADB. 2006. Report and Recommendation of the President to the Board of Directors: Proposed Loan and Technical Assistance Grant to the People’s Republic of China for Hunan Flood Management Sector Project. Manila.

2 in priority municipality and county-level cities. The recurrent flooding of Lishui, Xiangjiang, , and Zishui rivers and their tributaries and the effects on the Dongting Lake area were constraining socioeconomic development in Hunan Province, with 56.4 million people and with 84% of the provincial population affected. Strengthened flood management was needed and the proposed project aimed to achieve this by supporting the government’s shift from a flood-control approach to one that is based on integrated river basin flood management and with a focus on economic and environmental sustainability and social inclusion.

2. The project was designed to support the implementation of part of Hunan's integrated flood control program under the Hunan Provincial 11th Five-Year Plan (2006–2010).2 The Asian Development Bank (ADB) country strategy and program supported this innovation.3 A sector loan modality was adopted as an appropriate way to support the government’s sector development plan that included organizational changes, investments, and policy reforms—all aimed at making broad sector improvements. The provincial project management office (PPMO) had experience with past and ongoing flood-control projects financed by the World Bank (WB) and the Japan Bank for International Cooperation (JBIC), and sector experience in the PRC was being gained through ADB support to two other flood management projects in the PRC.4

B. Expected Impacts, Outcomes, and Outputs

3. The intended impact of the project was sustainable and inclusive socioeconomic growth in flood-prone areas of Hunan Province. The expected outcome was improved flood protection for strategic and priority flood-prone areas in the upper reaches of the four main river basins in Hunan Province. Outputs were described as (i) nonstructural flood management systems, which comprise operational flood warning and management systems for up to 35 municipalities and counties linked to the provincial flood warning and management system; (ii) structural flood protection, resettlement, and environment management, which cover flood protection works completed in priority locations as part of Hunan's River Basin Flood Control Plan and the 11th Hunan Provincial Five-Year Plan—in compliance with the PRC regulations and ADB safeguard policies; (iii) project management and capacity building, which cover operational and strengthened project management and monitoring systems; and (iv) flood management sector planning, which encompasses selected sector assessments and planning to support the development of integrated flood management plans. This additional output was to be implemented with a grant-financed technical assistance (TA).5

C. Provision of Inputs

4. The project was approved on 29 June 2006, with the loan and project agreements signed on 4 October 2006 and loan effectiveness was on 9 January 2007. The original loan closing date of 31 March 2013 was extended to 31 December 2014. Reasons for the delayed closing date included (i) delays in land acquisition and resettlement; (ii) flood protection designs needed modification to be consistent with urban development plans, and construction schedule had to be

2 See the Hunan Provincial 11th Five-Year Plan (2006–2010). 3 ADB. 2005. Country Strategy and Program Update (2006–2008): People’s Republic of China. Manila. 4 ADB. 2002. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the People’s Republic of China for the Songhua River Flood Management Sector Project. Manila; and ADB. 2001. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the People’s Republic of China for the Yellow River Flood Management Sector Project. Manila. 5 ADB. 2006. Technical Assistance to the People’s Republic of China for Strengthening Flood Management Sustainability in Hunan Province. Manila. (TA 4813-PRC. This technical assistance supported the development or assessment of priority aspects of the provincial-level strategic plan to support national policy developments in integrated flood management.) 3

synchronized with the implementation of urban development plans;6 (iii) contract variations were needed following the design changes, which increased material and labor costs; and (iv) inefficient contractor performance where some contractors that won contracts did not have sufficient resources, which slowed completion of the contracts.

5. At appraisal, the total project cost was estimated to be equivalent to $497.4 million— comprising $242.3 million in foreign exchange and $255.1 million in local currency. Of this amount, $200.0 million (40%) was provided by ADB loan and $297.4 (60%) by government funds. At loan closing, project costs amounted to $567.6 million (a 14% increase in total project cost) with $191.2 million provided by ADB loan and $376.4 million by government funds—a 26.6% increased contribution in US$.7 On completion, some $8.8 million of the $200 million ADB loan as approved was not used because some contracts were canceled near the end of the project.8 The actual total project cost was higher than the estimate at appraisal mainly due to the yuan’s appreciation against US dollar during implementation.

6. The PPMO was assessed as having the minimum required management and technical capacity to implement the proposed works. Accordingly, it was planned that the PPMO would use counterpart funds to hire domestic specialists or companies to assist with compliance monitoring and supervision of environmental, resettlement, and ethnic minority issues.

7. The project was assessed as category B on Environment, category A on Involuntary Resettlement, and category A on Indigenous Peoples.9 During project preparation the following were prepared: (i) an initial environmental examination (IEE), including an environmental management plan (EMP), for each of the eight core subprojects; (ii) a consolidated IEE, a consolidated summary IEE, and the consolidated environmental management plan (CEMP); and (iii) an environmental assessment and management framework (EAMF) that outlined the noncore subproject selection criteria, approval process, and institutional arrangements for environmental assessment. A resettlement framework was also prepared and resettlement plans were developed for the core subprojects with assistance from a design institute, project preparatory technical assistance consultants, and an ADB team. An ethnic minority development framework and ethnic minority development plans were prepared for nine subprojects, including one for the core subproject during project preparation. Eight ethnic minority development plans were prepared for the remaining subprojects during implementation.10

8. Under the project, 69 person-months of domestic consulting services were planned to be funded through counterpart funds. The actual number of person months funded is not clear, but the project completion report (PCR) lists a range of agencies and experts funded by the PPMO and the local project management offices’ (LPMOs) own funds (para. 27). The associated TA grant provided 54.6 person-months of consultant inputs, up from the planned 49 person-months, including providing the PPMO with 30 person-months of consultancy services to support the

6 These were city- and county-level urban development plans, which were modified to incorporate the subprojects. 7 In Chinese yuan terms, the project cost at completion was lower than the estimate at appraisal. 8 Five contracts were canceled: City lot 1, City lot 1, County lots 2 and 3, and City lot 1. The contracts were canceled due to delays in land acquisition and resettlement and the need to adjust designs to be consistent with urban development plans. The civil works were completed with local government funding. 9 Resettlement was rated significant at the time of appraisal, the same as for indigenous peoples. See Footnote 1, Appendix 13: Summary Poverty Reduction and Social Strategy. 10 These eight noncore subprojects were in City and the counties of Baojing, Cili, Fenghang, Jianghua, Jiangyong, Luxi, and Shimen. 4 project’s management, financial and environmental management, and the management of land acquisition and resettlement. 11

D. Implementation Arrangements

9. The project was implemented using the designed implementation arrangements. The HPG was the executing agency and 35 participating cities and counties were the implementing agencies. A project leading group headed by the vice governor of HPG was established to oversee project implementation. The PPMO established in the Provincial Water Resources Department (PWRD) included representatives from the Hunan Provincial Development and Reform Commission, Hunan Provincial Finance Department (PFD), and PWRD. Each implementing agency set up a local project leading group and an LPMO for preparing and carrying out its subproject. The PPMO coordinated and directed subproject preparation and implementation by helping LPMOs plan and manage their subprojects, coordinate the monitoring of land acquisition and resettlement, and environment management, and establish project monitoring systems. On behalf of the national government, the Ministry of Finance relent the ADB loan proceeds, under the same terms of the original loan, to the PFD, which received them on behalf of the HPG. The PFD onlent the loan proceeds to the municipal, city, and county finance bureaus of the participating local governments implementing the subprojects.

10. Under the sector loan, representative subprojects were selected based on the feasibility of project components. Eight of 35 draft subproject feasibility reports were selected and reviewed during project preparation. These covered technical, financial, economic, and social safeguards, with environmental aspects. Representative subprojects were selected based on (i) the size of investment, (ii) whether they represent the geographic conditions in the four mountain watersheds, (iii) the severity of environmental and resettlement impacts, (iv) the severity and types of flooding, (v) the types of proposed flood protection measures, (vi) the potential impact on indigenous peoples, and (vii) potential impact on poverty. The remaining 27 draft subproject feasibility reports were prepared by municipal and county governments with assistance from a consulting company.

11. The TA grant supported the project’s outcome by providing the following: (i) an analysis of Hunan’s flood warning system; (ii) an assessment of the feasibility of providing flood insurance in Hunan; and (iii) further development and capacity building, including project management support (footnote 11). The government complied with most loan and project covenants but only partially complied with covenants on the project leading group,12 counterpart funds for external resettlement monitoring and evaluation (M&E),13 and environmental monitoring and reporting.14

11 Attached to ADB. 2006. Report and Recommendation of the President to the Board of Directors: Proposed Loan and Technical Assistance Grant to the People’s Republic of China for Hunan Flood Management Sector Project. Manila. (TA 4813-PRC). 12The project leading group met once a year on average. The review found the covenant requiring a meeting twice a year to have been unrealistic at points when no specific issues existed for the group to discuss or resolve. The PPMO had meetings quarterly and regularly reported project progress to the group. 13 A requirement that external resettlement M&E reports be submitted to ADB semiannually was not met due to the LPMOs’ reluctance to pay the agencies that conducted the M&E. 14 Performance of contractors for environmental protection was not reported in project progress reports, although it was required by the project agreement. The reporting of environmental monitoring by the PPMO did not fully meet EAMF and CEMP requirements although the project agreement required monitoring to be conducted as described in the EAMF and CEMP. 5

These issues were identified as risks in the report and recommendation of the President,15 but in practice not sufficiently mitigated.

II. EVALUATION OF PERFORMANCE AND RATINGS

A. Relevance of Design and Formulation

12. The PCR rated the project relevant. At the time of appraisal and throughout the project duration, the intended outcome was aligned with the PRC sector development priorities and with the government’s shift from flood control to integrated flood management. The intended outcome was aligned with the ADB Country Strategy and Program Update for the PRC for 2006–2008 and to the ADB water policy. 16 Project design benefited from the application of other ADB and development partner experience in the sector. The sector loan modality was appropriate given its alignment with, and support for, a geographic area and time period of the Hunan integrated flood control program. This support of the government’s shift away from a flood control approach—from wholly reliant on structural works—to an integrated flood management approach across river basins, using both structural and nonstructural measures to manage flood risk, was innovative and transformative.

13. The design and monitoring framework (DMF) indicators and targets at outcome and output level adequately covered intended key project benefits, such as (i) reduced government expenditures on flood repairs and compensation; (ii) reduced private property losses, which would be preconditions for the intended impact of increased overall investment and employment, and consequent industrial and commercial growth; and (iii) reduced incidence of urban poverty in project areas. The performance indicators and targets were appropriate although baseline data was not included consistently in the DMF at appraisal stage. The project did not adequately consider the geographic spread of the subprojects when assessing the implementation arrangements and the level of consultancy support required. The geographic spread had been correctly designed to ensure that priority municipal and county growth centers in the mountain river basins would be included to reduce investor concerns over floods, thereby facilitating an increase in investment in the industrial and commercial enterprises. However, implementation support from the PPMO and the TA grant was inadequate to meet the demand for support from LPMOs. The project risk analysis was sound, including its reference to the lack of coordination between provincial and local government (an issue arising from the consistency between urban development plans and project investments plans) and insufficient counterpart budget funds for civil works, resettlement, and environment management at project start-up. However, mitigation strategies were not always as effective as planned. In particular, consultant services inputs designed to support the PPMO in managing 35 subprojects undertaken by 35 separate city and county governments—spread across Hunan Province—were acknowledged as insufficient (PCR, para. 12). On balance, given the project’s innovative and transformative nature and despite some design weaknesses, this validation considers the project relevant.

B. Effectiveness in Achieving Project Outcomes and Outputs

14. The PCR rated the project effective in achieving its intended outcome. At outcome level, the detailed achievements of annual flood damage and disaster relief cost in the project area was reduced from CNY120.8 million in 2005 to CNY82.0 million in 2015. Meanwhile, the annual average direct economic loss caused by floods and waterlogging in the project area was reduced from

15 Footnote 1, paras. 58–60. 16 ADB. 2001. Water for All: The Water Policy of the Asian Development Bank. Manila. 6

CNY802.4 million in 2005 to CNY234.8 million in 2015. The project output achievements will have ensured that this project contributed to such reductions.

15. Outputs achieved under the nonstructural flood management systems were in increased times of advanced warning against potential floods—from a baseline of less than 24 hours to 1–3 days before flooding. Also, the accuracy of manual flood forecasting increased from 70%–85% to 85%–90%. The project has greatly increased the extent of river flow and rainfall instrumentation of the Hunan river basins, thus, improving the effectiveness of monitoring and reducing the lag time for issuing early warnings. The achievements of the nonstructural flood management systems included effective early warnings and evacuations without injuries or loss of life during the 2012 landslide and flood events (PCR, para. 9).

16. Outputs achieved under the structural flood protection, resettlement, and environmental management component included flood control level of county-level cities improved to 1 in 20-year return flood from below 1 in 5-year return flood recurrence. Meanwhile, flood control level of municipal cities improved to 1 in 50- or 100-year return flood from below 1 in 20-year return flood. Incomes and livelihoods of most affected persons were also satisfactorily restored to pre- resettlement levels and selected EMP monitoring targets were improved. Structural outputs across the 35 subprojects included almost 400 kilometers of embankments, flood walls, sluice gates, pumping stations, and diversion channels. For the project management and capacity building component, output indicators were limited to 24 quarterly progress reports submitted to ADB, based on LPMOs’ reporting (although most reports lacked environmental monitoring results). A domestic project management and monitoring system was operationalized, except for environmental monitoring systems. Training on flood warning was effective, however, the PCR acknowledged that project management support “fell short of what was needed to help the PPMO manage 35 subprojects scattered all over Hunan” and implemented by multiple LPMOs. For the flood management sector planning component, outputs were achieved with support from the TA grant. This included a basin-wide flood warning system, and development needs were assessed, flood insurance was appraised, and next steps for inclusion in future flood management plan were agreed upon by key provincial authorities.

17. Safeguards were generally implemented satisfactorily. Environmental impacts were localized, temporary, and there were no significant negative impacts. Resettlement took longer than envisaged but impacted less number of people than planned, and resettled people were very satisfied with their new housing. Ethnic minority development plans were also tailored to each subproject. However, there were procedural shortcomings in the monitoring and reporting of safeguard activities, which was not carried out as required by the project agreement.17

18. Notwithstanding these safeguard deficiencies, overall, the project outputs have been substantially achieved and this has enabled the project to contribute to the achievement of overall sector outcome targets. This validation also assesses the project effective.

C. Efficiency of Resource Use

17The PPMO submitted 40 environmental monitoring reports to ADB. These covered 27 of the 35 subprojects and were all disclosed at the ADB website. This fell short of the compliance with the EAMF and the CEMP that was required under the project agreement. The EAMF required the PPMO to submit semiannual reports to ADB on the progress of implementation of subproject EMPs, and the CEMP required the PPMO to submit annual compliance and ambient monitoring program summary reports to ADB. Similarly, the external resettlement M&E reports were not submitted semiannually. 7

19. The PCR assessed the project efficient. For the component for which the economic internal rate of return (EIRR) analysis was applicable, the project’s overall EIRR at completion was16.4%. The 35 subprojects were economically viable, with EIRRs at completion between 12.3% and 26.1%. The completion EIRRs of 26 subprojects exceeded appraisal estimates and nine were lower, with deviations due to changes in project costs, changes in economic benefit estimates resulting from design revisions, and the extension of subproject construction periods. The reevaluation compared “with” and “without” project scenarios and used the same assumptions and methods as the appraisal economic analysis. The reevaluated EIRRs can be considered conservative as they did not include a number of additional indirect and unquantifiable benefits.18 A sensitivity analysis was undertaken using three scenarios: (i) a 10% benefit decline, (ii) a 10% operation and maintenance (O&M) cost increase, and (iii) a combination of both. The overall project and 28 of the 35 subprojects remained economically viable under these scenarios.

20. Process efficiency was less satisfactory. Completion was delayed by 21 months with delays in land acquisition and resettlement. Exchange rate fluctuations led to the need to increase counterpart funds, flood protection designs needed to be modified to be consistent with urban development plans, and construction scheduling needed to be synchronized with the implementation of urban development plans. Contract variations were also needed following the design changes, which increased material and labor costs, and from inefficient performance by contractors that won contracts by submitting low bids. The problem was exacerbated by the challenging, but necessary, geographic spread of the subprojects since each was being implemented independently by 35 separate city and county governments. Some of these implementation problems had been recognized as risks in project preparation but, as discussed in para. 13, mitigation strategies were ineffective. Given that the delays were not unusually long, this validation nevertheless considers the project efficient.

D. Preliminary Assessment of Sustainability

21. The PCR rated the project most likely sustainable. Financial sustainability was assessed if the participating local governments would have the financial resources or be able to raise revenues to make project loan repayments and cover the costs of O&M for flood protection structures over their economic life. The local governments have two main financing sources for the construction and O&M of the flood protection structures and loan repayments. These are (i) a flood control security fund financed by taxes,19 and (ii) an urban maintenance and construction tax.20

22. The capacity of the 35 subproject city and county governments to service the loan for the repayment period and fund adequate O&M was reassessed—by comparing annual collections from identified sources with financing required. Annual O&M costs were based on actual expenditures incurred during the initial operation period. Interest and principal repayments of the ADB loan were estimated based on the outstanding amount of the sub-loan to each of the 35 local governments at the end of 2015, the remaining repayment period of 16 years from 2016 to

18 Indirect benefits, such as the avoidance of injuries, hospitalizations, and deaths were omitted. So were benefits not quantified such as an improved environment. The economic benefits of the nonphysical improvements to the flood management system—e.g., the reduced need for large-scale evacuations and the mobilization of flood-fighting crews—were also not quantified. 19 According to HPG provisions (1994), businesses must pay 0.08%–0.20% of their sales and/or revenues to a flood control security fund. Insurance companies must pay 1% of annual property insurance premiums to the flood control security fund. Of the funds collected, 10% are allocated to the province and 90% to Hunan's cities and counties. 20The tax is levied on all enterprises and individuals paying value-added taxes, business taxes, and consumption taxes at rates of 7% of the value-added tax, 5% of the business tax, and 1% of the consumption tax. In most of the project cities and counties, 15% of the urban maintenance and construction tax is earmarked for flood protection. 8

2032, and the ADB loan interest rate in May 2016. The affordability analysis for the 35 subprojects ranged from 109% to 217%, which means that all the governments will have sufficient resources available to repay the ADB loan and pay O&M costs during the operation period. This apparent sustainability is dependent on the projected economic outputs in the subproject areas and tax revenues being maintained. The analysis was conservative and excluded the possibility of additional financing sources. Since the improved or constructed flood protection structures will protect lives and property of residents, this will facilitate economic growth and investments in the subproject areas. This growth and greater investment will expand the tax base for the flood control security fund and the urban maintenance and construction tax collections, providing the city and county governments additional revenues to support the O&M and ADB loan repayments. Therefore, from a financial perspective, the project is sustainable.

23. The sustainability of the project outcomes and outputs also largely relies on the extent of institutionalization of the project’s sector planning activities. Although the PCR reports that the PWRD and city and county water resources bureaus—which are responsible for the O&M of the systems, stations, and structures—are quite stable, implementation experience21 suggests that it may be difficult to ensure adequate levels of qualified human resources for effective management of environmental issues.22 These potential environmental management institutional weaknesses could inhibit the project’s medium- to long-term environmental effects. The evidence provided on how the project is accepted by the local communities and stakeholders is somewhat limited—the DMF indicated a household survey would be undertaken, however, the level of satisfaction is limited to a reference to those who were resettled (para. 17). This validation assesses the project likely sustainable.

III. OTHER PERFORMANCE ASSESSMENTS

A. Preliminary Assessment of Development Impact

24. The overall impact of the project was intended to be enhanced sustainable and inclusive socioeconomic growth in flood-prone areas of Hunan Province. Performance indicators and targets used in the project design and the achievements include the following: (i) increased number of industrial and commercial enterprises in the project area from 522,569 in 2006 to 750,950 in 2015, (ii) increased land values for commercial and industrial purposes in project areas by 65% on average (over the 2005 levels) by 2015, and (iii) reduced urban poverty incidence in the 35 cities and counties where the 35 subprojects were implemented from 3.6% in 2003 to 2.8% in 2013. These preliminary results are promising and it is likely that the flood-risk reductions achieved at output level will have contributed significantly to these results and to economic growth that is inclusive and environmentally sustainable, albeit with some risk of inadequate institutional capacity. The project has contributed to the government’s shift from a flood control approach to a more economically and environmentally sustainable and socially inclusive river basin-based integrated flood management approach.

21 For example, the inadequacy of environmental monitoring and reporting is of concern. The levels of monitoring and reporting capacity among the LPMOs and LEPBs are uneven and a gap exists between the required and actual financing for environmental management. 22 As an example, the CEMP required that the PPMO and each of the LPMOs have an environment and social management division, including a full-time environment specialist. However, most LPMOs established embankment construction divisions of 5 ̶ 6 staff members each, instead of establishing environment and social management divisions. The staff of these divisions supervised subproject implementation, including site inspections on environmental aspects. Each LPMO engaged a construction supervision company whose assigned tasks included supervision and monitoring of the environmental management activities of the contractors. However, these supervision companies generally lacked dedicated environmental engineers. 9

25. The project had significant impacts in regional socioeconomic development and poverty reduction by (i) reducing flood risks for about 16 million beneficiaries in the project areas, including poor and vulnerable people; (ii) improving living conditions among affected people through compensation and rehabilitation; and (iii) increasing income and employment opportunities, particularly for the poor.

26. This validation assesses the development impact satisfactory. The project is likely to have positive development impacts. No negative impacts have been identified but there is some risk that institutional weaknesses may inhibit the achievement of environmental impacts.

B. Performance of the Borrower and Executing Agency

27. The PCR rated the performance of the borrower and executing agency satisfactory. The Ministry of Finance provided high-level support to the HPG and the participating cities and counties assumed responsibility for the project. Counterpart funding revisions were managed promptly without delays. Project preparation was generally sound although identified risks did not have adequate mitigation strategies and implementation proved difficult given the geographic spread of the subprojects. Loan covenant compliance was not comprehensive and arrangements for safeguards management, including monitoring, were deficient with possible risk to environmental sustainability. This validation agrees with the rating of satisfactory.

C. Performance of the Asian Development Bank and Cofinanciers

28. The PCR rated the performance of ADB satisfactory. The project preparatory technical assistance was sound and ADB was persistent with the compliance to the loan agreement covenants, including the safeguards policies to which government found compliance difficult. ADB undertook 15 missions that addressed issues during implementation—an inception mission, seven review missions, five special project administration missions, a midterm review mission, and a project completion review mission. When unexpected problems occurred, ADB reacted promptly to facilitate solutions, such as the inclusion of an ADB environment specialist in field missions and provision of a staff consultant to support the PPMO and LPMOs in preparing monitoring reports (PCR, Appendix 10, para. 9). Supervision reports were generally timely and included key required information. This validation agrees with the PCR rating of satisfactory.

D. Others

29. Procurement. Some of the contractors that won civil works awards by submitting low bids had implementation difficulties due to the lack of resources. In July 2007, ADB revised a model national competitive bidding (NCB) document to increase the performance security requirement to cover the risk of default by contractors whose bids were too low. The revised model NCB document worked well.

30. Government assessment of the project. The project was rated “successful” in the government’s Implementation Completion Report.23 It concluded that the project objectives had been achieved. The implementation of the subprojects and flood management systems have contributed to flood risk reduction at the county-level cities and at the prefecture-level cities of the subprojects.24 The report recommended that the examination and approval procedures of ADB project should be simplified by empowering the PPMO to make implementation adjustments

23 Hunan Flood Management (Sector) Project. 2015. Implementation Completion Report: PPMO of Hunan Flood Management Sector Project Utilizing ADB Loans. Hunan. 24 Paras. 11 and 12 of this validation describe the impact realized from flood risk reduction. 10 independently. In addition, it recommended that ADB procurement bidding procedures should be simplified and adapted to local practice although it also agreed that low price bids need to be scrutinized to ensure that the contractor has the capacity to complete the contracts.

IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS

A. Overall Assessment and Ratings

31. Overall, the project was rated successful by the PCR. This validation also views the project successful. It is rated relevant because although the intended project outcome was aligned with the government’s sector development goals and with the ADB country strategy, and was transformative in supporting integrated flood management, the project design was somewhat lacking in some aspects. It notes that the provision of support from PPMOs and the TA grant for the LPMOs for implementing subprojects across a wide geographic area was not sufficient. The project is rated effective, acknowledging that the intended outcome was achieved through the substantial accomplishments in the output targets, notwithstanding inadequate safeguard reporting. It is rated efficient, considering high investment efficiency but with a less than efficient implementation process. The project’s likely sustainable rating acknowledges the robust financial sustainability of the project but raises some concerns about the extent of local-level institutional capacity, particularly for managing environmental risks.

Overall Ratings Reason for Disagreement and/or Validation Criteria PCR IED Review Comments Relevance Relevant Relevant Effectiveness Effective Effective Efficiency Efficient Efficient Sustainability Most likely Likely This validation recommends a rating sustainable sustainable of likely sustainable. Although the fiscal affordability results are positive, the extent of the institutional capacity at local level is uncertain, particularly for managing environmental risks. Overall assessment Successful Successful Preliminary assessment Satisfactory of impact Borrower and executing Satisfactory Satisfactory agency Performance of ADB Satisfactory Satisfactory Quality of PCR Satisfactory ADB = Asian Development Bank, IED = Independent Evaluation Department, PCR = project completion report. Note: From May 2012, IED views the PCR rating terminology of "partly" or "less" as equivalent to "less than" and uses this terminology for its own rating categories to improve clarity. Source: ADB Independent Evaluation Department.

B. Lessons

32. This validation agrees with the lessons identified in the PCR. Notably, it agrees that while the sector loan modality was appropriate, the geographic spread of the 35 subprojects presented an implementation challenge. At the same time, the designed consultant inputs to support project management were inadequate even before the problems of the geographic spread of the subprojects were recognized. Adequate loan proceeds could have been earmarked for these 11

services, considering the scale and complexity of the project and the capacities of the PPMO and LPMOs. Similarly, the capacity of the local environmental protection bureaus for programming and monitoring should have been better assessed during project preparation. ADB’s introduction of a stronger performance security requirement to the standard NCB document to eliminate unrealistically low bids was a successful intervention.

33. This validation does not necessarily agree that ADB financing of the external M&E of resettlement would fully address the problem of inadequate resettlement M&E. During project preparation, the LPMOs committed to pay independent agencies for external resettlement M&E but later became reluctant to do so. It was incumbent upon PPMO to ensure that this was done and for ADB to support the PPMO. Ensuring LPMO responsibility for this activity is part of capacity building and supports sustainability.

C. Recommendations for Follow-Up

34. This validation agrees that to ensure the project’s sustainability, the HPG and the local governments should continue to monitor the related O&M programs. It is also suggested that the HPG and the local governments assess the project’s effect on flood mitigation, and to determine its statistical impact on income and poverty.

35. This validation also agrees with the PCR recommendations for the design and implementation of future projects. The efficiency of sector loan projects with multiple noncore subprojects should be considered based on the co-location of subprojects if this is appropriate or, if not, on the matching of implementation support to geographic spreads of subprojects. Project readiness should put more emphasis on the ability of executing and implementing agencies to raise funds (even if commitments are made to deliver this funding) and on ensuring that consulting services for project management support are adequate for the scale, spatial extent, and complexity of a project.

V. OTHER CONSIDERATIONS AND FOLLOW-UP

A. Monitoring and Evaluation

36. The PPMO established a project management and monitoring system. 25 Project monitoring was based on annual plans and monitoring and reporting systems, including EMPs and resettlement and ethnic minorities’ frameworks. However, there is no detailed explanation of the M&E process and to what extent it was implemented. For example, the review, monitoring, and mitigation of the risks were required but there is no information available on these in the documents available for this validation. Given the complexity of this project, consultants engaged under the TA grant helped the PPMO and LPMOs establish an internal reporting system, but eventually the LPMOs were unable to maintain this. The external resettlement M&E required by ADB also encountered problems and fell short of expected monitoring frequency.

B. Comments on Project Completion Report Quality

25 The Project's proposed management and monitoring system was developed as part of the TA grant delivered through the ADB. 2004. Technical Assistance to the People’s Republic of China for the Hunan Flood Control Project. Manila; and through the Cooperation Fund for the Water Sector Pilot Demonstration Activity: Hunan Flood Management Project Management and Monitoring System. 12

37. The PCR is consistent with the ADB guidelines and is internally consistent and logically presented (including appendixes).26 Safeguard issues, fiduciary issues, and covenants were all adequately addressed and the report clearly explains arising issues. Shortcomings in safeguard monitoring and reporting are particularly well-articulated and candidly described. However, the sections on the core criteria are too concise and substantiation of the ratings requires referencing back into the main text and appendixes where the evidence is available. The method for EIRR recalculation is sound. The lessons are based on evidence and are useful as inputs into the design and implementation of future projects or in further studies, although not unique. This validation recommends a rating of satisfactory.

C. Data Sources for Validation

38. This validation is based on reviews of the PCR and the report and recommendation of the President, a very limited selection of back-to-office reports, loan review reports, internal memos, and the government’s Implementation Completion Report.

D. Recommendation for Independent Evaluation Department Follow-Up

39. It is recommended that a performance evaluation review be conducted in 2019 or later. This will allow at least 5 years from the physical completion of the project to properly evaluate its effect on flood mitigation, income, the wider environment, affected people, and poverty reduction using accumulated statistical data.

40. The PPMO project M&E quarterly reports could be made available for project performance evaluation report preparation, which could help to assess quality and data sources.

26 ADB. 2016. Guidelines for the Evaluation of Public Sector Operations. Manila.