Document of The World Bank
Report No: ICR00004056 Public Disclosure Authorized
IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-17383 & TF-17384)
ON A
GRANT
IN THE AMOUNT OF EUR 6.54 MILLION (US$8.95 MILLION EQUIVALENT)
Public Disclosure Authorized TO THE
REPUBLIC OF MOZAMBIQUE
FOR AN
ENHANCING SPATIAL DATA FOR FLOOD RISK MANAGEMENT PROJECT
June 16, 2017
Public Disclosure Authorized
Public Disclosure Authorized Water Global Practice Africa Region
CURRENCY EQUIVALENTS (Exchange Rate Effective at approval on June 16, 2014)
Currency Unit = Mozambique Metical (MZN) MZN 1.00 = US$0.0327 US$1.00 = MZN 30.52
Currency Unit = Eurozone Euro (EUR) EUR 1.00 = US$1.37 US$1.00 = EUR 0.73
(Exchange Rate Effective at the time of completion on December 29, 2016)
Currency Unit = Mozambique Metical (MZN) MZN 1.00 = US$0.0140 US$1.00 = MZN 71.29
Currency Unit = Eurozone Euro (EUR) EUR 1.00 = US$1.04 US$1.00 = EUR 0.96
FISCAL YEAR January 1 – December 31
ABBREVIATIONS AND ACRONYMS
ANE National Roads Administration (Administração Nacional de Estradas) ARA Regional Water Authority (Administração Regional de Águas) ARA Sul Southern Regional Water Authority (Administração Regional de Águas do Sul) ARA Zambezi Zambezi Regional Water Authority (Administração Regional de Águas do Zambeze) CENACARTA National Center for Cartography and Remote Sensing (Centro Nacional de Cartografia e Teledetecção) CPF Country Partnership Framework CPS Country Partnership Strategy CREE Committee on External Economic Relations (Comissão de Relações Econômicas Externas) CWRAS Country Water Resources Assistance Strategy for Mozambique DEM Digital Elevation Model DFID U.K. Department for International Development DNA National Directorate for Water (Direcção Nacional de Águas) DNGRH National Directorate for Water Resources Management (Direcção Nacional de Gestão de Recursos Hídricos) DRM Disaster Risk Management DSM Digital Surface Model DTM Digital Terrain Model
EIRR Economic Internal Rate of Return EWS Early Warning Systems GFDRR Global Facility on Disaster Reduction and Recovery GIS Geographic Information System GoM Government of Mozambique ICR Implementation Completion and Results Report IFR Interim Financial Report INGC National Institute for Disaster Management (Instituto Nacional de Gestão de Calamidades) ISR Implementation Status and Results Report LAS LASer format Lidar data Lidar Light Detection and Ranging ESDFRMP Enhancing Spatial Data for Flood Risk Management Project M&E Monitoring and Evaluation MOPHRH Ministry of Public Works, Housing and Water Resources (Ministério das Obras Públicas, Habitação e Recursos Hídricos) NASA National Aeronautics and Space Administration NWRDP National Water Resources Development Project PAD Project Appraisal Document PAMT Project Administration and Monitoring Team PDO Project Development Objective PPCR-HYDROMET Pilot Program for Climate Resilience - Climate Resilience: Transforming Hydrological and Meteorological Services Project PQG Government’s Five-year Program (Program Quinquenal do Governo) SRTM Shuttle Radar Topography Mission TA Technical Assistant WRM Water Resource Management
Senior Global Practice Director: Guang Zhe Chen Practice Manager: Jonathan S. Kamkwalala Project Team Leader: Shelley Mcmillan ICR Team Leader: Odete Duarte Muximpua
MOZAMBIQUE Enhancing Spatial Data for Flood Risk Management Project
CONTENTS
Data Sheet A. Basic Information ...... i B. Key Dates ...... i C. Ratings Summary ...... i D. Sector and Theme Codes ...... ii E. Bank Staff ...... iii F. Results Framework Analysis ...... iii G. Ratings of Project Performance in ISRs ...... ix H. Restructuring ...... ix I. Disbursement Profile ...... x
1. Project Context, Development Objectives and Design ...... 1 2. Key Factors Affecting Implementation and Outcomes ...... 4 3. Assessment of Outcomes ...... 9 4. Assessment of Risk to Development Outcome ...... 15 5. Assessment of Bank and Borrower Performance ...... 16 6. Lessons Learned ...... 18 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors ...... 18 Annex 1. Project Costs and Financing ...... 21 Annex 2. Outputs by Component ...... 22 Annex 3. Economic and Financial Analysis ...... 25 Annex 4. Grant Preparation and Implementation Support/Supervision Processes ...... 26 Annex 5. Beneficiary Survey Results ...... 28 Annex 6. Stakeholder Workshop Report and Results ...... 29 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR ...... 31 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ...... 34 Annex 9. List of Supporting Documents ...... 35 MAP - Surveyed Areas under the Project ...... 37
A. Basic Information Enhancing Spatial Data Country: Mozambique Project Name: for Flood Risk Management Project Project ID: P149629 L/C/TF Number(s): TF-17383, TF-17384 ICR Date: 06/24/2017 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: IPF Grantee: MOZAMBIQUE Original Total US$8.96 million Disbursed Amount: US$7.24 million Commitment: Revised Amount: US$7.14 million Environmental Category: C Implementing Agencies: Direcção Nacional de Águas / National Director of Water Instituto Nacional de Meteorologia / National Institute for Meteorlogy Cofinanciers and Other External Partners:
B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 03/17/2014 Effectiveness: 06/16/2014 Appraisal: 06/05/2014 Restructuring(s): 06/20/2016 Approval: 06/05/2014 Mid-term Review: 02/15/2016 02/15/2016 Closing: 06/30/2016 12/31/2016
C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Unsatisfactory Grantee Performance: Moderately Satisfactory
C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Moderately Satisfactory Unsatisfactory Implementing Quality of Supervision: Moderately Satisfactory Satisfactory Agency/Agencies: Overall Bank Moderately Overall Borrower Moderately Satisfactory Performance: Unsatisfactory Performance:
i
C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Quality at Entry Project at any time No None (QEA): (Yes/No): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status:
D. Sector and Theme Codes Original Actual Major Sector/Sector Public Administration Other Public Administration 15 15 Transportation Other Transportation 15 15 Water, Sanitation and Waste Management Other Water Supply, Sanitation and Waste Management 70 70
Major Theme/Theme/Sub Theme Environment and Natural Resource Management Water Resource Management 50 50 Water Institutions, Policies and Reform 50 50 Finance Finance for Development 13 13 Disaster Risk Finance 13 13 Urban and Rural Development Disaster Risk Management 39 39 Disaster Preparedness 13 13 Disaster Response and Recovery 13 13 Disaster Risk Reduction 13 13
ii E. Bank Staff Positions At ICR At Approval Regional Vice President: Makhtar Diop Makhtar Diop Country Director: Mark R. Lundell Mark R. Lundell Practice Manager: Jonathan S. Kamkwalala Magdolna Lovei Task Team Leader(s): Shelley Mcmillan Louise E. M. Croneborg ICR Team Leader: Odete Duarte Muximpua ICR Primary Author: Jaime Palalane
F. Results Framework Analysis
Project Development Objectives (from Project Appraisal Document) The Project Development Objective (PDO), as set out in the Project Appraisal Document (PAD), was to increase the capacity of Mozambique to prepare for and manage flood events in the Limpopo and Zambezi River basins.
Revised Project Development Objectives (as approved by original approving authority) Not revised.
(a) PDO Indicator(s)
Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Flood preparedness improved through updated & utilized Contingency Plan led Indicator 1: by INGC in the Zambezi and Limpopo. Improved Value contingency plan Low resolution limits (quantitative or utilized in DRM contingency planning qualitative) routines and decision making Date achieved 06/20/2014 06/30/2016 Not achieved: Indicator was dropped at restructuring, as it was out of the Comments project scope and activities. Hence, the indicator was dependent on actions taken (including. % by third parties, outside the implementing agencies and beneficiary institutions achievement) mandate. Flood risk identification improved through creation of probabilistic flood risk Indicator 2: hazard maps for the Limpopo & Zambezi survey areas. Value (quantitative or N Y Y qualitative) Date achieved 06/20/2014 06/30/2016 12/30/2016
iii Partially achieved: The indicator was revised at restructuring to ‘flood risk Comments identification improved through the incorporation of new geospatial data in the (including % Limpopo and Zambezi basin plans’ to better reflect the project scope; and was achievement) achieved at 90%. Flood risk reduced through use of new geospatial data to build-back-better Indicator 3: infrastructure. Example/evidence Value of Lidar data used (quantitative or Lack of spatial data in infrastructure qualitative) planning and analysis Date achieved 06/20/2014 06/30/2016 Not achieved: Indicator was dropped at restructuring as it was out of the project Comments scope and activities. Indicator achievement was dependent on actions taken by (including % third parties, outside the implementing agencies and beneficiary institutions achievement) mandate. Improved accuracy of areas & pop. at risk of 100-year flood in the Limpopo and Indicator 4: Zambezi with improvement to EWS Value (quantitative or 0% 100% 90% qualitative) Date achieved 06/20/2014 06/30/2016 12/30/2016 Comments Partially achieved: Progress was made in verifying new flood hazard map (including. % overlaid with population maps compared to 2014 in the Limpopo Basin and is in achievement) progress for the Zambezi Basin. Indicator was achieved at 90%. Indicator 5: Direct project beneficiaries (number). Value (quantitative or 0 320,800 2,063,286 qualitative) Date achieved 06/20/2014 06/30/2016 12/30/2016 Comments Achieved: The underestimation of the target beneficiaries has contributed to the (incl. % target being exceeded by more than 600%. achievement) Indicator 6: Direct female project beneficiaries (% of total direct project beneficiaries). Value (quantitative or 0 192,480 1,100,920 qualitative) Date achieved 06/20/2014 06/30/2016 12/30/2016 Comments Achieved: The underestimation of the target beneficiaries has contributed to the (including % target being exceeded by more than 600%. achievement)
iv (b) Intermediate Outcome Indicator(s)
Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1: Airborne survey data collected for the Limpopo. Value Pre- and post- Pre- and post- Limited Lidar data (quantitative or processing of data processing of data available qualitative) complete complete Date achieved 06/20/2014 12/30/2015 09/30/2016 Comments Achieved: Airborne survey data collection was 100% completed, including the (including % additional surveyed areas. achievement) Indicator 2: Data transferred from contractor to client for the Limpopo. Value (quantitative or N Y Y qualitative) Date achieved 06/20/2014 12/30/2015 12/30/2016 Comments Achieved: Data transferred to the client in June 2016 and data from the (including % extension phase in December 2016. Indicator was 100% achieved. achievement) Indicator 3: Horizontal resolution of DEMs used for Limpopo. Value (quantitative or 9,000 cm 500 cm 100 cm qualitative) Date achieved 06/20/2014 12/30/2015 12/30/2016 Comments Achieved: Higher resolution data were collected since the flying cost was the (including % same. This will allow a much wider use by other sectors. Indicator was achievement) exceeded by 400%. Indicator 4: Geospatial data stored and managed for the Limpopo. Value No storage and Data utilized Data utilized (quantitative or management system in through network through network qualitative) place across agencies across agencies Date achieved 06/20/2014 06/30/2016 12/30/2016 Comments Partially achieved: Hardware and software Lidar data processing in place at (including % ARA-Sul, CENACARTA, and DNGRH from the end of December 2016. 90% achievement) of target achieved. Utilization of data is still ongoing. Indicator 5: New geospatial Limpopo data available for public. Value (quantitative or N Y N qualitative) Date achieved 06/20/2014 12/30/2015 12/30/2016
v Comments Partially achieved: The implementing agency is currently working on a (including % platform to avail data for the public, but this has not yet been achieved though achievement) 40% progress was made with installation of data processing equipment. Indicator 6: Hydraulic & hydrological modeling capacity increased for the Limpopo. Value Free 2D hydraulic Hydraulic modeling (quantitative or and hydrological capacity limited qualitative) model operational Date achieved 06/20/2014 12/30/2016 Comments Not achieved: Indicator dropped at restructuring. The modeling is being carried (including % out as part of the NWRDP (P107350) as it would be more efficiently delivered achievement) through the overall flood management planning exercise. Multi-sector uptake of Limpopo Lidar data for applications other than WRM Indicator 7: and DRM. Value (quantitative or N Y Y qualitative) Date achieved 06/20/2014 12/30/2015 12/30/2016 Comments Partially achieved: Lidar data are being used for the Mapai dam feasibility (including % study and was made available for the National Roads Agency. 75% of target achievement) achieved. Data sharing and use by other agencies is still ongoing. Indicator 8: Staff trained in Lidar processing and data management. Value (quantitative or 0 25 29 qualitative) Date achieved 06/20/2014 06/30/2016 12/30/2016 Comments Achieved: The training was expanded to cover other sectors and other potential (including % Lidar data users. Indicator achievement was exceeded by 16%. achievement) Indicator 9: Number of which are women. Value (quantitative or 0 11 8 qualitative) Date achieved 06/20/2014 06/30/2016 12/30/2016 Comments Partially achieved: Ratio between women and men among technical staff was (including % overestimated, as this is a very specialized area where very few women engage, achievement) despite efforts to attract more women. 88% of target achieved. Indicator 10: Staff trained in hydraulic and hydrological modeling. Value (quantitative or 0 30 qualitative) Date achieved 06/20/2014 06/30/2016 Comments Not achieved: Indicator dropped at restructuring. Modeling and training are (including % being done through the NWRDP (P107350) as it would be more efficiently achievement) delivered through the overall flood modeling and management plan.
vi Indicator 11: Number of which are women. Value (quantitative or 0 15 qualitative) Date achieved 06/20/2014 06/30/2016 Comments Not achieved: Indicator dropped at restructuring. Modeling and training are (including % being done through the NWRDP (P107350) as it would be more efficiently achievement) delivered through the overall flood modeling and management plan. Indicator 12: Airborne survey data collected for the Zambezi. Value Pre- and post- Pre- and post- Limited Lidar data (quantitative or processing of data processing of data available qualitative) complete complete Date achieved 06/20/2014 06/30/2016 12/30/2016 Comments (including % Achieved: Airborne survey data collection completed. 100% target achieved. achievement) Indicator 13: Data transferred from contractor to client for the Zambezi. Value (quantitative or N Y Y qualitative) Date achieved 06/20/2014 12/30/2015 06/30/2016 Comments (including % Achieved: Data transferred to the client in June 2016. 100% target achieved. achievement) Indicator 14: Horizontal resolution of DEMs used for Zambezi. Value (quantitative or 9,000 cm 500 cm 100 cm qualitative) Date achieved 06/20/2014 12/30/2015 09/30/2016 Comments Achieved: Higher-resolution data were collected since the flying cost was the (including % same. This will allow a much wider use by other sectors. Target was exceeded achievement) by 400%. Indicator 15: Geospatial data stored and managed for the Zambezi. Value No storage and Data utilized Data utilized (quantitative or management system in through network through network qualitative) place across agencies across agencies Date achieved 06/20/2014 06/30/2016 12/30/2016 Partially achieved: Hardware and software Lidar data processing in place at Comments ARA-Zambezi, CENACARTA, and DNGRH and shared with ZAMCOM from (including % the end of December 2016. 75% of target achieved as flood modeling and achievement) utilization is still under way. Indicator 16: New geospatial Zambezi data available for public.
vii Value (quantitative or N Y N qualitative) Date achieved 06/20/2014 12/30/2015 12/30/2016 Comments Partially achieved: The implementing agency is currently working on a (including % platform to avail data for the public, but this has not yet been achieved. Indicator achievement) was achieved at 30% with data transfer to ZAMCOM and CENACARTA. Indicator 17: Hydraulic & hydrological modeling capacity increased for the Zambezi. Value Free 2D hydraulic Hydraulic modeling (quantitative or and hydrological capacity limited qualitative) model operational Date achieved 06/20/2014 12/30/2015 Comments Not achieved: Indicator dropped at restructuring. This activity is being carried (including % out as part of the PPCR-HYDROMET (P131049) as it would be more achievement) efficiently delivered through the river basing planning exercise. Multi-sector uptake of Zambezi Lidar data for applications other than WRM and Indicator 18: DRM. Value (quantitative or N Y N qualitative) Date achieved 06/20/2014 12/30/2015 12/30/2016 Comments Partially achieved: Lidar data are still in the process of being incorporated to (including % the models but have been made available to both national and regional basin achievement) authorities. 30% progress made toward achieving this indicator. Indicator 19: Staff trained in Lidar processing and data management. Value (quantitative or 0 10 14 qualitative) Date achieved 06/20/2014 06/30/2016 12/30/2016 Comments Achieved: The training was expanded to cover other sectors and other potential (including % Lidar data users. Indicator achievement was exceeded by 40%. achievement) Indicator 20: Number of which are women. Value (quantitative or 0 4 1 qualitative) Date achieved 06/20/2014 06/30/2016 12/30/2016 Comments Partially achieved: Ratio between women and men among technical staff was (including % overestimated, as this a very specialized area where very few women engage, achievement) despite efforts to attract more women. 25% of target achieved. Indicator 21: Staff trained in hydraulic and hydrological modeling. Value (quantitative or 0 10 qualitative)
viii Date achieved 06/20/2014 06/30/2016 Not achieved: Indicator dropped at restructuring. This activity is being carried Comments out as part of the PPCR-HYDROMET (P131049) as both training and modeling (including % it would be more efficiently delivered through the river basing planning achievement) exercise. Indicator 22: Number of which are women. Value (quantitative or 0 10 qualitative) Date achieved 06/20/2014 06/30/2016 Not achieved: Indicator dropped at restructuring. This activity is being carried Comments out as part of the PPCR-HYDROMET (P131049) as both training and modeling (including % it would be more efficiently delivered through the river basing planning achievement) exercise.
G. Ratings of Project Performance in ISRs
Actual Date ISR No. DO IP Disbursements Archived (US$, millions) 1 12/08/2014 Satisfactory Moderately Satisfactory 0.00 2 06/24/2015 Satisfactory Moderately Satisfactory 0.02 3 12/17/2015 Satisfactory Moderately Satisfactory 2.52 4 06/21/2016 Satisfactory Satisfactory 5.70 5 12/28/2016 Satisfactory Satisfactory 6.85
H. Restructuring
ISR Ratings Amount Board at Disbursed at Restructurin Approved Reason for Restructuring & Key Restructuring Restructurin g Date(s) PDO Changes Made g in US$, Change DO IP millions At level II project restructuring in June 2016, two key PDO-associated outcome indicators were dropped, and one was revised to better align it to the project activities and ensure better efficiency within the Water Resources 06/20/2016 N S MS 5.22 Development Program, as some of these activities were being carried out in the two other projects. Hence, with the extensive delays in implementation and strong appreciation of the dollar against the euro, the available resources and
ix ISR Ratings Amount Board at Disbursed at Restructurin Approved Reason for Restructuring & Key Restructuring Restructurin g Date(s) PDO Changes Made g in US$, Change DO IP millions time were not sufficient to complete the project activities.
I. Disbursement Profile
x 1. Project Context, Development Objectives and Design
1.1 Context at Appraisal
1. Mozambique is vulnerable to the impacts of natural hazards such as floods, cyclones, and droughts. With 2,740 km of coastline, where roughly 60 percent of the population lives, the country is located downstream nine transboundary rivers and is exposed to recurrent extreme climate related events, mainly cyclones and floods. Best estimates suggest that as much as 58 percent of the population is vulnerable to natural disasters and that annual economic growth is 1.1 percentage points lower than it otherwise would be, because of weather and water shocks.
2. In the last two decades, major floods have hit Mozambique in 2000, 2001, 2007, and 2013 (collectively resulting in over 1,200 deaths, displacement of 1.5 million people, and destruction of US$1.5 billion in physical infrastructure). In 2013, extreme floods hit Mozambique in the lower stretches of the Limpopo, Incomati, and Zambezi River basins. The impacts were felt in urban centers and rural communities in the provinces of Gaza, Maputo, Zambezia, and Sofala. Over 170,000 people were evacuated, 113 lives were lost, and 89,000 ha of crops were destroyed, making this the worst disaster to hit Mozambique since the severe floods of 2000.
3. High-resolution spatial/topographic data, also called digital elevation models (DEMs), are required for estimating potential extent and impacts of floods. At project appraisal in 2014, the resolution of spatial data and DEMs available for modeling in the Limpopo and Zambezi River basins were limited to 90 m-by-90 m grid pixel data that are sourced through the Shuttle Radar Topography Mission (SRTM) satellite and made available freely through National Aeronautics and Space Administration (NASA).1 In the low-lying areas of the lower stretches of the two flood- prone basins, this resolution is inadequate for hydrological and hydraulic modeling and risk analysis or risk management.
4. In line with the abovementioned details, the rationale for the World Bank’s assistance lies in the recognition that higher resolution of spatial data and DEMs are critical to an adequate hydrological and hydraulic modeling and the further application of the results from this exercise for water resources management (WRM) and disaster risk management (DRM). Furthermore, improved WRM and DRM will contribute to prevent and reduce deaths and economic losses associated with the occurrence of floods.
5. The World Bank’s support was aligned with the second pillar of Mozambique’s FY12–15 Country Partnership Strategy (CPS), decreasing vulnerability and increasing resilience, especially the objective related to improving resilience to natural disasters and the impacts of climate change. Both the CPS, and the project, were aligned with Mozambique’s own poverty reduction strategy, the Poverty Reduction Action Plan (Plano de Acção para a Redução da Pobreza) FY11–14, which called for broad-based and inclusive growth through reduction of vulnerability to natural disasters and the threat of climate change. In addition, the project was building on the World Bank Country Water Resources Assistance Strategy for Mozambique2 (CWRAS 2007) by enhancing
1 The models available in the two Regional Water Authorities (Administração Regional de Águas, ARAs), but in limited use, are HECRAS, MIKE11, Waflex WEAP, and FloodWatch. 2 World Bank, 2007. Mozambique Country Water Resources Assistance Strategy: Making Water Work for Sustainable Growth and Poverty Reduction. Washington, DC.
1 geomorphological, hydrological, and meteorological data for the core operation of water resources planning, infrastructure development, transboundary cooperation with neighboring countries, and DRM. Finally, the proposed small grants were closely aligned with the objectives of the portfolio of World Bank support in Mozambique, with focused efforts on the Limpopo and Zambezi River basins.
1.2 Original Project Development Objectives (PDO) and Key Indicators
6. The PDO, as set out in the Project Appraisal Document (PAD) and reflected in grant agreements, was to increase the capacity of Mozambique to prepare for and manage flood events in the Limpopo and Zambezi River basins.
7. The PAD specified the following six key performance indicators to measure progress toward the achievement of the PDO:
(a) Flood preparedness improved through updated & utilized Contingency Plan led by INGC in the Zambezi and Limpopo
(b) Flood risk identification improved through creation of probabilistic flood risk hazard maps for the Limpopo and Zambezi survey areas
(c) Flood risk reduced through use of new geospatial data to ‘build-back-better’ infrastructure
(d) Improved accuracy of areas and populations at risk of 100-year flood in the Limpopo and Zambezi with improvement to EWS
(e) Direct project beneficiaries
(f) Direct female project beneficiaries (60 percent of total direct project beneficiaries)
1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification
8. The PDO statement was not revised during the project implementation. However, during the level II project restructuring in June 2016, two key PDO-associated outcome indicators were dropped, and one was revised to better align it to the scope and project activities. Hence, with the extensive delays in implementation, and limited resources within the project, these activities would be more efficiently addressed by other parallel projects within the Water Resources Development Program, where flood modeling activities were being carried out as part of a more comprehensive river basin planning exercise. The dropped indicators were (a) flood preparedness improved through updated & utilized Contingency Plan led by INGC in the Zambezi and Limpopo and (b) flood risk reduced through use of new geospatial data to ‘build-back-better’ infrastructure. As per the Restructuring Paper, these two indicators were dropped since they were not well aligned to the actual scope of the project, which rendered them impossible to measure.
9. The indicator ‘flood risk identification improved through creation of probabilistic flood risk hazard maps for the Limpopo and Zambezi survey areas’ was revised to ‘flood risk
2 identification improved through the incorporation of new geospatial data in the Limpopo and Zambezi basin plans’, to better reflect the scope of the project in improving data accuracy for flood modeling and management. The flood modeling and subsequent use for flood preparedness related activities were being implemented and will be monitored and reported through the National Water Resources Development Project (NWRDP, P107350) and the Pilot Program for Climate Resilience - Climate Resilience: Transforming Hydrological and Meteorological Services Project (PPCR- HYDROMET, P131049).
1.4 Main Beneficiaries
10. The direct project beneficiaries were the two institutions mandated with WRM in the Limpopo and Zambezi River basins: the Southern Regional Water Authority (Administração Regional de Águas do Sul, ARA Sul) and the Zambezi Regional Water Authority (Administração Regional de Águas do Zambeze, ARA Zambezi). Besides these two institutions that were the immediate beneficiaries of the project, the PAD specified that the outputs of the detailed high- resolution Lidar (light detection and ranging) surveys will be available to the National Directorate for Water (Direcção Nacional de Águas, DNA); the National Institute for Disaster Management (Instituto Nacional de Gestão de Calamidades, INGC); the National Center for Cartography and Remote Sensing (Centro Nacional de Cartografia e Teledetecção, CENACARTA); the National Roads Administration (Administração Nacional de Estradas, ANE); and the National Meteorological Institute, among others.
11. The PAD states that indirect project beneficiaries included 164,200 people in the Limpopo and 156,600 in the Zambezi who live in the priority survey areas identified for the two basins. These target groups would benefit through improved information on flood-related hazards that can both lower the exposure to risks and increase productivity.
1.5 Original Components (as approved)
12. The project comprised two components that would support the acquisition of high- resolution spatial and topographic data through airborne Lidar; the subsequent production of derived products; the application of those products into hydrologic models, hydraulic models, and decision support systems; and the application of those models for flood and natural Disaster Risk Management, aiming at achieving the PDO of increasing the capacity of Mozambique to prepare for and manage flood events. Component A: Limpopo River basin high-resolution Lidar survey (ARA Sul) had grant financing of EUR 3.58 million provided by the U.K. Department for International Development (DFID) to the World Bank-managed trust fund, the Global Facility on Disaster Reduction and Recover (GFDRR). Component B: Zambezi River basin high- resolution Lidar survey (ARA Zambezi) had grant financing of EUR 2.96 million equally provided by DFID to the GFDRR.
13. Both components contemplated the following similar activities: (1) technical assistance for implementation of Lidar assignment; (2) determination of priority survey areas; (3) acquisition and completion of Lidar surveys; (4) processing and quality assurance review of acquired Lidar point data; (5) integration of Lidar data into existing models and decision support systems and establishment of new ones; (6) long-term training and capacity building for technical staff in ARAs, DNA, INGC, and associated agencies working in surveyed basin; (7) effective data
3 management systems and solutions to ensure Lidar data and derivative projects are easily and openly accessible; and (8) investments into physical information management systems to ensure long-term accessibility and use of Lidar data and derivative products.
1.6 Revised Components
14. As part of the project restructuring in June 2016, activities A5 and B5 on integration of Lidar data into existing models and decision support systems were cancelled. This change in project activities was implemented due to an overlap between those and ongoing actions under the NWRDP and PPCR-HYDROMET. The restructuring would ensure harmonization, prevent duplication, and result in more efficient use of resources. This revision was discussed with the financing agency DFID and approved by the Country Director on June 27, 2016.
1.7 Other significant changes
15. Following the cancellation of project activities A5 and B5, availed funds were directed to expand the surveyed area in Limpopo Basin to cover additional critical areas, with a total cost of US$1,012,960. This provision was also reinforced with savings from different activities. To secure time for the additional surveys, the project restructuring also included a no-cost extension of the closing date to December 31, 2016 (six months).
2. Key Factors Affecting Implementation and Outcomes
2.1 Project Preparation, Design, and Quality at Entry
16. Soundness of the background analysis. The background analysis undertaken during the project preparation was sound. Selected basins for the Lidar surveys, Limpopo and Zambezi, are the most critical areas considering the recurrence and magnitude of flood events that impose huge social and economic losses to this country. Experience from prior work in the country favored the identification of the right institutions to implement the project and to participate as stakeholders. In addition to that, lessons learned from past Lidar initiatives3 implemented in the country and abroad contributed to strengthening the design of the project. Lidar data previously acquired in the Limpopo were merged with the data collected through this project, and key lessons considered during project preparation were the need for a technical assistance component, extensive and inclusive training, capacity building, and maximizing the use of the Lidar data by beneficiary institutions and stakeholders.
17. Assessment of the project design. The project design was relatively ambitious about its activities and indicators. Some of the proposed activities were dependent on existence of good hydraulic models and decision support systems where Lidar data would be incorporated. However, these tools were still under development through parallel projects (NWRDP and PPCR- HYDROMET), and full achievement of the PDO was dependent on a good alignment between all
3 Two past Lidar surveys of the Limpopo financed by the World Bank were implemented by INGC in 2013 integrated in the actions of the Mozambique-Programmatic Support to Disaster Risk Management Phase I (P124755); and by ANE in 2014–2015 as part of the Mozambique Roads and Bridges Management and Maintenance Project Phase 2 - 2nd Additional Financing (P146402).
4 the three projects and effective coordination between the ARAs, which were beneficiary institutions, and INGC, which is responsible for DRM.
18. Hence, the project had a high number of indicators (six PDO indicators and 22 intermediate indicators), some of which were repetitive, which could have been more efficiently monitored if combined. Another shortcoming in the project design was the inability to identify investment needs and capacity-building activities targeting CENACARTA as the key institution for storage and dissemination of the raw Lidar data and derived products as well as INGC as a direct beneficiary institution, given the dependency on its capacity to achieve the original PDO indicator 1. However, it should be noted that the focus of the project was on improving the accuracy of spatial and topographic data to increase preparedness and capacity to manage flood events, and therefore the ARAs are the key institutions for water resources and river basin management.
19. On the positive side, the project had a good balance between the two river basins, and it was possible to achieve a good geographical coverage. Special remarks go for training actions that benefited technicians from Maputo City and Maputo and Gaza Provinces from Southern Mozambique as part of the Limpopo Basin component as well as technicians from Tete, Manica, and Sofala Provinces, from the central region of the country, which benefited from the Zambezi Basin component. Hence, these inclusive training actions equally benefited a number of institutions with interest on Lidar data. Also, the inclusion of a technical assistance to support implementing agencies with quality control and strategic guidance was a key factor considering the specificities and novelty of the Lidar data and technology.
20. Adequacy of the Government’s commitment and participatory process. The Government of Mozambique (GoM) through the former DNA collaborated effectively in the project preparation, despite having a tight time frame, which did not favor an ideal contribution. Besides DNA leading role, ARA Zambezi and ARA Sul were equally committed with the project preparation, with special inputs in defining priority areas to be surveyed. These two ARAs designated competent technicians to act as project component coordinators and facilitate its implementation. All the relevant policies, strategies, and plans were in place and facilitated the identification of priority areas and interventions.
21. Assessment of risks. The overall risk of the project was assessed as Low. However, given the dependency on other parallel projects, previous experiences with delays in procurement and contract approval processes, and donor-funded projects, the overall risk should have been assessed as Moderate and mitigation measures put in place to address those risks. Both implementing agency risks, capacity and governance, were classified as Moderate, and they were effectively mitigated with (a) the contracting of a technical assistant (TA) to increase the implementing capacity and (2) the project management and administrative support from the experienced project administration and monitoring team (PAMT) in the areas of procurement, financial management, and reporting on progress.
2.2 Implementation
5 22. Project implementation progress was initially slow, due to delays in finalizing the bidding documents to hire a Lidar survey firm, a much longer than expected procurement process following a complaint presented in August 2015, by one of the bidders, and a long approval process by the Committee on External Economic Relations (CREE) and the Administrative Tribunal. After the resolution of this complaint, and following the signing of the contract with the service provider in November 2015, the project entered a phase of satisfactory implementation progress throughout the remaining original and project extension period. Except for cancelled activities, all project components were completed by the extension closing date of December 31, 2016. The cumulative disbursements were significantly behind schedule during the first year of its implementation, 0.24 percent by the end of June 2015.4 However, they increased positively with the start of the surveys up to 100 percent for the Zambezi grant (TF-17383) and 99.87 percent for the Limpopo grant (TF- 17384).5
23. As detailed in sections 1.5 and 1.6, the project was restructured during implementation, to reduce its scope, given the significant delays in implementation and to improve efficiency on the water resources program financing. Two project activities were cancelled, and a no-cost extension was granted for six months, to ensure the completion of the extended survey areas. As part of the restructuring, the project Results Framework was also revised, and some indicators were dropped (see section 1.3). However, the restructuring was only done less than a month before project completion and the extension was not sufficient to ensure effective use of the Lidar products and incorporation into the models and adequate dissemination. Hence, the PDO had not been changed despite significant changes in the PDO indicators.
24. The project has provided high-resolution spatial and topographic data required for estimating potential extent and impact of floods. All priority areas identified for the surveys were covered for both Limpopo (9,460 km2) and Zambezi Basins (13,951 km2). Following the project restructuring, the Limpopo Basin benefited from an additional surveyed area of 4,000 km2. However, strong appreciation of the U.S. dollar, the execution currency, against the euro, the funding currency, imposed a financial cumulative loss of approximately 18.32 percent (US$742,411.51) for the Zambezi grant and 19.51 percent (US$956,544.02) of the original value for the Limpopo grant6 and limited the resources available to effectively implement all the project activities.
25. Besides the delays in finalization of the bidding document and selection process of the Lidar survey service provider, changes in team leadership on the World Bank side during implementation introduced further delays in the reviews of the procurement packages. The project had three different task team leaders in three years, including project preparation and implementation. Also, the scheduling of training sessions for the last month of the project extension, December 2015, a busy planning month in most institutions and a preferred period for vacancies, has resulted in underrepresentation of some institutions in the training sessions.
4 ESDFRM project quarterly progress report for April to June 2015. 5 Cumulative amount disbursed by the project closing as reported in the World Bank Client Connection platform, given in the ESDFRM final project implementation report for July to December 2016. 6 The Enhancing Spatial Data for Flood Risk Management Project (ESDFRM) final project implementation report for July to December 2016.
6 26. On the positive side, the following principal factors contributed to the successful implementation of the project:
The recruitment of a local and experienced TA who had good relations and easy access to all relevant institutions involved in the process
The selection of a professional and cooperative service provider, with experience in conducting Lidar surveys in Mozambique and financed by the World Bank
Strong involvement of stakeholders to get inputs about potential usages and needs of Lidar data in different sectors and refine the survey specifications and areas and their further training to increase the potential of Lidar data use beyond the direct beneficiary institutions.
Project extension that favored the completion of training activities for target groups in both basins and made it possible to extend the surveyed areas in the Limpopo Basin
Favorable weather conditions that enabled the completion of Lidar surveys during the rainy season (from October to March), as both basins were affected by a severe drought and atypical low levels of rainfall
2.3 Monitoring and Evaluation (M&E) Design, Implementation, and Utilization
27. Design. The project system for tracking and recording implementation progress was based on a Results Framework and Monitoring Tool. A shortcoming on the design was the high number of indicators, duplication, and dependency on activities that were out of the scope of this project. Consequently, PDO indicators 1 and 3 were dropped and PDO indicator 2 was revised to better reflect the project scope and the delays in project implementation (see section 1.3). Six intermediate indicators related to flood modeling were equally dropped following the cancellation of modeling activities, to be developed through the NWRDP and PPCR-HYDROMET. The core PDO indicators on direct project beneficiaries were not consistently presented in the PAD. These indicators appear as indirect beneficiaries in the main document, with the direct project beneficiaries being the two ARAs that benefited from capacity enhancement. However, these same indirect beneficiaries are presented as direct beneficiaries in the Results Framework. This has implications on the project performance assessment, given that the primary objective was to create capacity for flood preparedness and management, and the people residing in the surveyed areas are indirect project beneficiaries. These people could be potential beneficiaries once flood forecasting information is generated and directly disseminated to these communities.
28. Implementation. Besides the refinement of the indicator to better track the project performance and outcome, there were no key issues with the M&E implementation. The PAMT regularly collected all the relevant data on the outcome and intermediate indicators from different sources, including the ARAs, the service provider, and the TA. The PAMT equally safeguarded the quality of collected data.
29. Utilization. Collected data on indicators were used by the PAMT to prepare quarterly progress reports. These reports were the basis for discussion on implementation progress during
7 the implementation support missions. Collected data were equally used for internal reporting at National Directorate for Water Resources Management (Direcção Nacional de Gestão de Recursos Hídricos, DNGRH) to report to government plans and prepare the annual project implementation reports.
2.4 Safeguard and Fiduciary Compliance
30. Environmental and social safeguards. It was an Environmental Assessment Category C project and did not trigger any World Bank safeguard policy, as it involved only flights over surveyed areas. No environmental or other safeguards issues were identified during implementation.
31. Procurement. Because of its nature, the project had a reduced number of procurement processes (four). Nevertheless, there was a complaint submitted by one of the bidders for Lidar survey services in May 2015, following the clearing of the evaluation report in April 2015. This compliant was considered unfounded, and the case was closed in August 2015. Also, the long approval process required by the Government (the CREE and Administrate Tribunal) for major contracts significantly delayed the procurement of the service provider. Besides these, no other issues arose with respect to procurement, and the project was always found to comply with procurement requirements.
32. Financial management. The project complied with all financial management requirements, and no issues arose during the project implementation. Interim financial reports (IFRs) were satisfactory submitted as specified in the Financing Agreements. The external audit report related to the fiscal year ending on December 31, 2015, conducted by the Administrative Tribunal, did not identify any significant issue, and an unqualified (clean) opinion on the financial statements was issued. The external audit report for the fiscal year ended on December 31, 2016, is expected to be finalized by May 2017.
2.5 Post-completion Operation/Next Phase
33. An important achievement of the project was the refurbishment of ARA Sul, ARA Zambezi, DNGRH, and CENACARTA with appropriate equipment to store, manage, and manipulate Lidar data from this and future projects. This, together with the training actions that were extended to technicians from other institutions, equipped the recipients with adequate capacity to sustain the benefits of the project. Nevertheless, the following follow-up actions are recommended:
Support CENACARTA to start a dialogue about needs of institutional reforms and technical and financial support to maximize data sharing for both the Lidar data and derived products, free of charge, not only for Government institutions, but also for the public, using open and easily accessible platforms.
Monitor the uptake of the Lidar data by different institutions (DNGRH, ARA Sul and Zambezi, the National Road Administration, the National Railway Company, and the National Disaster Risk Management Institute, among others) and the development of Lidar products transferred for other ongoing projects, as these are crucial elements to
8 assess the success of the project, as reflected in some of the project indicators transferred to other projects.
Support the efforts to survey additional areas along the Zambezi Basin,7 as formerly requested by ARA Zambezi to DNGRH, and funding of a Lidar survey for the Licungo Basin, identified as a priority by DNGRH during the closing mission.
34. LiDAR data were shared with the team working on the Limpopo flood management plan, as part of the NWRDP (P107350), which started in May 2016. The first phase of this study focused on the evaluation of hydrological models to select a model that will be adopted for flood modeling in the Limpopo Basin. The report for this phase was delivered in March 2017. The data are also being used for the Kariba Dam Break Analysis Study by the Zambezi River Authority. This Dam Break Analysis is one of the components of the Kariba Dam Rehabilitation Project (P146515). This study will prepare risk management plans and flood hazard maps that will include the identification and quantification of people who can potentially be affected by a failure of the Kariba Dam. It is expected that results from this study, including the estimation of people potentially affected by dam break, will be available by the end of 2017. In addition to that the data will be used in the hydrological modeling of the Zambezi study, which will define the hydrological model to be used for flood simulation in this basin and assess areas and population at risk of flood events with different recurrence times. This consultancy is part of the PPCR-HYDROMET (P131049).
3. Assessment of Outcomes
3.1 Relevance of Objectives, Design, and Implementation
Relevance of Objectives Rating: High
35. The PDO and its key associated outcome targets, in their original and revised version, are strongly aligned with the current country’s conditions, its development priorities, and the World Bank’s strategies. Revised key associated outcome targets did not influence the relevance of the objectives as it was focused on data provision and uptake for flood management. However, these changes reduced the assessment of their uptake by different actors and Lidar data application for different purposes within the scope of this project.
36. Mozambique continues to be vulnerable to the impacts of floods and other natural disasters. Considering the severity of the losses associated with the occurrence of flood events, the acquisition of high-resolution spatial/topographic data that can be used to increase the capacity to prepare and manage flood events is a country and global priority. The Government’s Five-year Program (Program Quinquenal do Governo, PQG) 2015–2019 identified, as one of its five priorities, the securing of sustainable and transparent management of natural resources and the environment. The PDO at the design stage remains relevant, as one of the strategic objectives for the fifth PQG priority is aligned with it—reduce the vulnerability of communities, economy, and infrastructures to climate risks and natural disasters. To materialize this objective, the following
7 Additional surveys are envisaged under the Kariba Dam Rehabilitation Project (P146515), Dam Break Analysis sub-component funded by Sweden.
9 two priority actions that are strongly aligned with the objectives and actions of the ESDFRMP were identified, among others: (a) continue with mapping the risk areas to natural hazards at appropriate scale and (b) improve the national capacity for modeling, forecasting, and providing early warning information of floods and other natural hazards. Aligned with the PQG, the recently created DNGRH is currently establishing a dedicated Unit for Floods and Droughts Management.
37. The project is highly relevant to the World Bank’s priorities and development objectives in the country. The Country Partnership Framework (CPF) FY17–218 recognizes that Mozambique will need to take long-term actions to reduce its vulnerability to long-term climate change and that such actions need to be complemented by (a) measures to address shorter-term weather variability by ensuring that robust systems are in place for disaster preparedness and management and (2) integrating climate risk assessments into planning and infrastructure development. Aligned with these, one of the 11 CPF objectives is to improve management of climate risk and natural resources. This objective is aligned with one of the 13 Systematic Country Diagnosis priorities— improve DRM and reinforce social and economic resilience. The project is equally aligned with the last CWRAS9 by enhancing geomorphological, hydrological, and meteorological data for water resources planning and DRM, among others.
Relevance of Design Rating: Modest
38. Relevance of original design. The original project components and activities, as well as implementation arrangements, were reasonably designed and included activities not only to acquire high-resolution spatial/topographical data, but also to create the required technical capacity to store, process, and use the Lidar data. The direct beneficiary institutions were therefore equipped to better prepare for and manage floods, and the capacity generated remains relevant after project closure and will positively contribute to maximizing the present and future use of the Lidar data.
39. However, the PDO could have been better reflected in the Results Framework and activities for capacity enhancement for the institution responsible for DRM reflected in the project. The logical link between the project objective, inputs (including funding), outputs (acquired Lidar data, equipment, persons trained, and so on), and the outcome of improved flood preparedness and management capacity was somehow fragmented by the dependency on existence of adequate hydraulic and hydrological models, which were being developed through other projects. With the initial delays and late restructuring, there were challenges with regard to implementation of the original Results Framework. These challenges were minimized with the cancellation and revision of part of the indicators during the project restructuring (see paragraphs 26–28). However, challenges remain in addressing the core PDO indicators, which in fact were indirect project beneficiaries.
40. Relevance of revised design. With the project restructuring, one of the eight project activities, integration/establishment of Lidar applications with focus on WRM and DRM, was cancelled mainly due to limited availability of time and dependency on existence of models that
8 World Bank. 2017. Mozambique - Country Partnership Framework for the Period FY17–21. Washington, DC: World Bank. 9 World Bank. 2007. Mozambique Country Water Resources Assistance Strategy: Making Water Work for Sustainable Growth and Poverty Reduction. Washington, DC: World Bank.
10 were yet to be developed. This cancellation affected the development of day-to-day operational products that are important for the envisaged increased capacity to manage flood events. However, the potential negative effect of this restructuring is minimized considering that these same products are being developed through other projects implemented by the same institution and with the same beneficiaries. The cancellation of this activity also allowed for expansion of the Lidar survey. The percentage of funds used for Lidar data acquisition increased from 82 percent to 96 percent of the historical disbursed value10 in U.S. dollars. The results indicators were revised to make them better aligned with the project activities.
3.2 Achievement of Project Development Objectives (Efficacy)
Original PDO and Key Associated Outcome Targets: Modest
41. The project activities contributed to the acquisition of high-resolution data that will enhance flood modeling in two of the most critical river basins in Mozambique. This finding is justified by the fact that acquired Lidar data increased the area covered with high resolution spatial and topographical data required for estimating potential impacts of floods from 3,651 km2 to 17,111 km2 in the Limpopo River basin.11 For the Zambezi River basin, this was the first wide Lidar survey and it covered 13,951 km2. The resolution of DEMs for covered areas was therefore increased from the 90 m-by-90 m grid pixel12 data that are sourced through the SRTM satellite, to 1 m-by-1 m grid pixel data acquired using the Lidar technology.
42. However, the PDO was not fully achieved, as the Lidar data are yet to be transformed into flood management tools. Some of the original outcome indicators were outside the scope of the project, which were subsequently dropped. In addition, PDO indicator 2 was revised since the activity was covered by another project, and PDO indicator 4 was met 90 percent. Meanwhile, the estimated direct project beneficiaries (PDO indicator 5), defined as people residing in the surveyed areas, was greatly exceeded, to 2,063,286,13 against the 320,800 (approximately 164,200 in the Limpopo and 156,600 in the Zambezi) target specified in the PAD. The direct female project beneficiaries (PDO indicator 6) are estimated to be 1,100,92014 (53 percent of total direct project beneficiaries), which is also higher than the target value of 192,480. However, this achievement reflects the underestimation of the target beneficiaries, which might have contributed to the above 600 percent exceedance. Also, the benefits of the Lidar products and the capacity enhancement will only be achieved once all the flood modeling and flood preparedness activities are completed within the NWRDP.
Revised PDO and Key Associated Outcome Targets: Substantial
43. The project substantially achieved the revised key outcome indicators. After the project revision, the main PDO-level outcome target was flood risk identification improved through the incorporation of new geospatial data in the Limpopo and Zambezi Basin plans. Currently, the Lidar
10 Corresponding to an increase from US$5,928,500 (initial survey contract) to US$1,012,960 (extension contract) of the historical disbursed of US$7,251,242.88. 11 INGC Lidar survey (2013): 2,635 km2; ANE Lidar surveys (2014–2015): 1,016 km2. 12 30m-by-30m grid-pixel data were released by NASA in September 2014. 13 Values estimated by the updated version of the Results Framework following the project closing workshop. 14 Values estimated by the updated version of the Results Framework following the project closing workshop.
11 data collected are being incorporated in project design and WRM plans under development. Acquired data were successfully processed and stored in purchased equipment for both ARAs, DNGRH and CENACARTA. The Lidar data are ready to be used and shared for multiple purposes. By the time of the draft Implementation Completion and Results Report (ICR) (end of April 2017), the Lidar data for the Limpopo River basin were being used for the Pre-feasibility Analysis of the Mapai Dam and the Limpopo Basin Study on Flood Prevention and Control. For the Zambezi Basin, the Lidar data were shared with the Zambezi River Authority for the purposes of the Kariba Dam Break Analysis, with ANE for the Zambezi Basin roads network assessment, and for the improvement of the flood protection dykes system in the Lower Zambezi.
44. The achievement of the second PDO-level outcome target (indicator 4), improved accuracy of areas and population at risk of 100-year flood in the Limpopo and Zambezi with improvement to Early Warning Systems (EWS), must consider the uptake of the Lidar data by other projects, as the development of specific Lidar products activities was cancelled under this project. So far, the acquired Lidar data have been processed and are being used for the Limpopo Basin Study on Flood Prevention and Control (NWRDP), which will determine flood risk areas and population. For the Zambezi Basin, accuracy of areas and population at risk of 100-year flood will be derived from the hydrological modeling of the Zambezi (under procurement by the PPCR-HYDROMET– expected to be onboard by the end of June), using the available Lidar data from this project. Therefore, considering the present uptake, and the fact that the Lidar data are freely available for other studies and plans that will be initiated by different institutions in the near future to improve EWS and DRM, this target can be considered likely to be fully satisfied.
3.3 Efficiency Rating: Substantial
45. The efficiency of the project is rated Substantial, considering the achievement of satisfactory results with reasonable costs and its efficiency on different aspects. Delays in starting with the surveys did not affect the ability of the service provider to fly the priority survey areas identified for both basins. It was equally possible to fly all the additional priority areas in the Limpopo Basin within the six months of extension for project implementation. The percentage of funds applied for the acquisition of Lidar data increased from the 42 percent indicated in the PAD to about 96 percent of the historical disbursements. There were also savings on the implementation, as the same PAMT from the NWRDP assisted the this project without additional operational/staff costs. As stated in the PAD, though Lidar is expensive, ground-based surveys are not feasible for covering areas as large as the basins in question. Therefore, airborne Lidar surveys are regarded as the most practical and cost-effective method for generating DEMs at a large scale and with required high resolution to better prepare for and manage floods.
46. The procedure for estimating the economic benefits of the project described in the PAD is considered valid. Following such procedures, the ex post assessment of the economic benefits would require new analyses through the adopted hydrodynamic models to update the estimates of floodplain inundation areas15 using the acquired Lidar data. It should be noted that, from the initial estimate of US$4.38 billion, the economic benefits of the project are expected to increase over
15 Detailed explanation available from Moller, D., G. Schumann, and K. Andreadis. 2014. Report on Airborne LiDAR Acquisitions over the Zambezi and Limpopo River Basins, Mozambique.
12 time, given that the total population and its density16 are expected to grow in the next three decades. In addition, the high-resolution spatial and topographical data of Lidar, derived products, and information assets remain relevant for repeat use in the longer term in multiple sectors and for a wide range of applications. These factors support the general idea of high economic internal rate or return (EIRR) from Lidar interventions and support the substantial efficiency rating.
47. The unit costs of delivering Lidar data for this project was US$253.24 per km2. This value is slightly higher than the indicative value given in the PAD, US$240 per km2, but it can be taken as good value for money considering that training was included in this rate without additional costs for the client. Analyzing the unit cost of the two previous Lidar surveys for the Limpopo Basin, US$238.06 per km2 for both INGC and ANE surveys in 2013 and 2014–2015, respectively, there was a 6.4 percent increase in the unit price.
48. The following four different products were derived from the Lidar survey: (a) LASer format Lidar data (LAS); (b) orthophotos (images orthorectified); (c) digital terrain model (DTM), a model of the bare earth, that is, a type of DEM that does not contain buildings or trees; and (d) digital surface model (DSM), a type of DEM that includes the top of buildings and trees. Although, DEMs and DTMs are the key products for flood risk analysis and management, the provision of other types of data added value for the project, as they are important for other sectors such as forests, roads, and security, among others. Acquired data respected all service standards, including collection requirements and surveys specifications,17 and in some cases products were delivered with better resolution than required in the contract with the service provider (orthophotos with a pixel size of 10 cm against the specified 30 cm). This fact could lead to a discussion about a possible increase in the coverage of the survey by reducing the number of Lidar products or its resolution. However, as the main costs are linked with the flight operations, the savings with the shut-off of one equipment or reduction of its resolutions are in most of the cases insignificant.
3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory
49. The project has an overall outcome rating of Moderately Unsatisfactory. This rating results from a PDO that remains highly relevant to the priorities of the GoM and the World Bank CPF FY17–21 but was not fully achieved through the project. The design and implementation arrangements are substantially relevant but had shortcomings in the original Results Framework and indicators. This yields a modest pre- and post-restructuring rating for relevance of objectives and design. Despite some improvements in efficacy, with project restructuring, some of the indicators are yet to be completely achieved. The economic and financial benefits of the project will be substantial, once the data are fully utilized and the unit costs for delivering the Lidar data were around the indicative value.
16 Gridded Population of the World_v4: UN-Adjusted Population Density - 2015 and 2020, accessible from NASA Socioeconomic Data and Application Center - SEDAC (http://sedac.ciesin.columbia.edu/data/set/gpw-v4- population-density-adjusted-to-2015-unwpp-country-totals) 17 United States Geological Survey (USGS) Techniques and Methods 11–B4: Lidar Base Specification (https://pubs.usgs.gov/tm/11b4/pdf/tm11-B4.pdf)
13 50. The overall assessment considered the ratings against both the original and revised PDO and key outcome targets. Weights of the assessment against the proportion of funds disbursed before and after the restructuring are shown in table 1.
Table 1 – Overall outcome ratings. Pre-restructuring Post-restructuring Overall Outcome
(72%) (28%) Rating Relevance Substantial Substantial Relevance of objectives High High Relevance of design and Modest Modest implementation Efficacy Modest Substantial Efficiency Substantial Substantial Moderately Moderately Moderately Combined rating (rating value) Unsatisfactory (3) Satisfactory (4) Unsatisfactory (3) Note: Amount disbursed at time of project restructuring approval was US$5.23 million, corresponding to a weight of 72 percent of the original amount of US$7.29 million, as reported in the Implementation Status and Results Report (ISR) from June 21, 2016. 3.5 Overarching Themes, Other Outcomes and Impacts
(a) Poverty Impacts, Gender Aspects, and Social Development
51. Most frequently the poor population is more exposed and affected by floods and other natural hazards. The acquired higher-resolution spatial and topographical data will contribute to improving the accuracy of potential areas affected by flood events and strengthening the existing EWS. This would contribute to reducing the occurrence of deaths and losses of food production between the poor farmers and population that live and work in low-land areas prone to inundation.
52. The project tracked the potential number of female project beneficiaries, once the surveys are translated into flood risk analysis products. It was found that 53 percent of the project beneficiaries are women. In rural areas, the percentage of women benefiting from the project actions can be higher, as this is the group that dedicates more to agriculture and migrate less to urban centers. With regard to capacity-building activities, it was found that 21 percent of the technicians trained during the project implementation were women.
(b) Institutional Change/Strengthening
53. Institutional strengthening was Significant. First, the inclusive and rich (four different modules were offered) training actions conducted in Maputo and Tete benefitted 43 technicians from 22 different institutions and more than the two direct beneficiary institutions of this project actions. Second, this Project equipped ARA Sul, ARA Zambezi, and DNGRH with appropriate means to make an effective use of the acquired Lidar data. Another important aspect was the inclusion of CENACARTA in the group of equipped institutions, although it was not contemplated in the initial project design. This made it possible to create a capacity, inside a key government institution for cartography and remote sensing to store, process, and share Lidar data and derived products with potential users.
14 (c) Other Unintended Outcomes and Impacts (positive or negative)
54. The local TA specialist recruited to support the implementation of the project enabled a stronger interaction with different government institutions and public companies, with the collaboration of the service provider. One positive result of this interaction was an increased awareness about the potential and applications of Lidar products by different sectors. In addition, an informal network of stakeholders and interested parties was created, although it is a challenge keeping the network active as the project comes to its end.
3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops
55. A project closing workshop with the participation of stakeholders and beneficiaries was held at DNGRH in Maputo, on February 7, 2017. Overall, participants were satisfied with the progress of the project following a long initial period of delays and some uncertainty. The extension of the training activities beyond the direct beneficiary institutions was commended, as well as the dissemination meetings that contributed to increasing the awareness and interest on the data. It was highlighted that the challenge is to start using the data and derived products for DRM and other sectorial needs.
4. Assessment of Risk to Development Outcome Rating: Moderate
56. The overall risk to development outcome is rated Moderate for the following reasons:
Technical risk: Negligible to Low. Technicians from beneficiary institutions were trained to be able to handle the Lidar data. However, during the interviews conducted as part of this ICR, there was a general feeling among them that more training would be important to enable in-house use of the data, besides using them for specific studies to be conducted by external consultants. The equipment and technology acquired to store and share the data will allow future and safe use of the data. Backups were provided to restore the Lidar data in case of losses.
Government ownership/commitment risk: Negligible to Low. Beneficiary institutions do value the importance of the Lidar data. That is reflected through the provision of operation and maintenance budget for equipment and investment in data use for various applications including hydraulic and hydrological modeling.
Other stakeholder ownership risk: Moderate. There is potential for a low uptake of the Lidar data if flood modeling and information systems are not developed. Hence, most of the institutions lack the financial resources and technical capacity to use the data. To mitigate this risk, the implementing agencies have invested in strong interaction with other stakeholders and institutional dissemination of data, throughout the project.
Sensitivity of acquired spatial data risk: Moderate. There is a tendency of the implementing agencies to classify and restrict data sharing, given the high resolution
15 of the acquired Lidar data. This may result in a lower willingness to establish functional mechanisms for open and free sharing of the data.
5. Assessment of Bank and Borrower Performance
5.1 Bank Performance
(a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory
57. The strategic relevance of the project was well assessed, yielding a definition of the PDO, aligned with both the country and World Bank strategies. However, the project components and activities that would contribute to the achievement of the PDO were relatively ambitious, with the inclusion of activities that were dependent on existence of models where Lidar data would be incorporated. The inclusion of training activities for strengthening the institutional capacity to use the acquired Lidar data was well assessed. The need for technical assistance to support the project implementation was equally well assessed. The allocation of funds between activities was well balanced with a significant portion being dedicated to the acquisition and processing of the data. Processes for procurement and fiduciary management were well assessed and assigned to the existing PAMT in DNGRH. The risk assessment did not candidly cover all the potential risks, including stakeholders’ ownership risk for both the beneficiaries and donor, and therefore the proposed risk mitigation measures did not fully address the level of risks, especially with regard to potential low uptake. A significant shortcoming was with the design of the Results Framework with a high number of indicators, dependency on parallel projects, and inclusion of activities and indicators that would be best addressed through other program components. Hence, the late project restructuring did not allow for effective completion of project activities and use and dissemination of Lidar data.
(b) Quality of Supervision Rating: Moderately Satisfactory
58. The World Bank’s supervision was Moderately Satisfactory. Despite carrying out supervision missions on a regular basis with a consistent and relevant composition of the World Bank team, late project restructuring undermined the project, reducing its potential for a successful implementation. A strong focus on the development impact was noticeable with World Bank team being open and flexible to changes that could contribute to maximizing the project benefits, such as securing more time for the project implementation and the reallocation of funds to maximize the acquisition of Lidar data. These facts contributed to the high disbursements (close to 100 percent) reached for both grants. In addition, Aide Memoirs from the missions were proactive in identifying actions to revert the slow progress of the project implementation during its first year. ISRs were generally consistent, with realistic ratings. Supervision of fiduciary and safeguard aspects was also strong. However, the decision to undertake project restructuring to improve its efficiency and efficacy was only done a month before the original closure date.
16 (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory
59. The overall World Bank performance is rated Moderately Unsatisfactory, based on the Moderately Unsatisfactory rating for ensuring quality at entry and Moderately Satisfactory rating for quality of supervision.
5.2 Borrower Performance
(a) Government Performance Rating: Moderately Satisfactory
60. The GOM supported the implementation of the project. The Central Government, through the Ministry of Public Works, Housing, and Water Resources (Ministério das Obras Públicas, Habitação e Recursos Hídricos, MOPHRH) participated in the project design and later in the approval and signing of the final agreement through the extinguished Ministry of Planning and Development, for its first two years, and the Ministry of Planning and Finance, for the project extension. The restructuring of the government structure, with the start of a new governing cycle in January 2015, following the October 2014 elections, did not negatively affect the project implementation. A significant shortcoming in the Government performance was associated with the long waiting period for approval of the procurement package for the service provider,18 contributing to the delay in starting with surveys. The combined effect of the complaint and the waiting time for CREE approval contributed to an extension of the procurement process to select the service provider from an expected duration of 3 months to 12 months.
(b) Implementing Agency or Agencies Performance Rating: Satisfactory
61. Implementation was initially under the responsibility of the extinguished DNA and was later facilitated by DNGRH (following a restructuring of the MOPHRH on July 17, 2015), in strong collaboration and with active participation of ARA Sul and ARA Zambezi. This change did not affect the project implementation. Project implementation was initially slow due to the time spent preparing and approving the procurement documents to hire a Lidar survey firm. The management responsibilities of the overall fiduciary tasks were under the responsibility of the experienced PAMT. The PAMT succeeded in ensuring the compliance with the Grant Agreement and all fiduciary requirements, as well as project monitoring and reporting. DNGRH supported the wider positive involvement and consultation of different stakeholders, facilitated by the TA.
(c) Justification of Rating for Overall Borrower Performance
18 According to the Presidential Decree 12/96, CREE had the competence to approve procurement processes with external funding above US$1 million. CREE was extinguished and the requirement of procurement approval was cancelled in March 2016.
17 Rating: Moderately Satisfactory
62. The overall borrower performance is rated Moderately Satisfactory, based on the moderately satisfactory rating for the GOM and satisfactorily for the implementing agencies.
6. Lessons Learned
63. Indicators and targets should be better aligned with the project activities despite having a broader impact on programmatic results. The project Results Framework and Monitoring Tool included a set of indicators that would be more efficiently addressed by other projects in the program, were dependent on existence of tools under development in other projects, or would require external actions beyond the implementing and beneficiary institutions. This has led to poor assessment of the project performance and cancellation of project activities and indicators. Therefore, during the project design, the actual scope of the project in relation to integrated programs and the link between progress indicators and project components or activities should be carefully assessed.
64. The extension of training opportunities far beyond the direct beneficiary institution contributes to increasing the ‘value for money’ of the Lidar data. The project was implemented with wide consultation and training of technicians from 22 different institutions. This resulted in increased awareness and interest in Lidar data, leading to a higher uptake and use of Lidar data for core flood assessment exercises and infrastructure development plans. Thus, the adoption of this approach in future similar projects is recommendable.
65. Although sector driven, Lidar projects should contemplate the capacity building inside key government institution for remote sensing. The original project design did not assess the needs of CENACARTA as the custodian institution for all Lidar data and derived products to fully comply with its role. With the project progress, it was observed that CENACARTA was lacking the technical capacity to fulfill its exclusive role of providing remote sensing data for all potential users. This limitation was addressed by equipping CENACARTA, bringing this institution to the same level as other main project beneficiaries with regard to the capacity to store and process Lidar data.
66. Maximizing reach of data into the public domain remains a challenge. During the project closing mission, some reluctance was noticed to open and free-of-charge provision of the Lidar data for noninstitutional uses. Although the reasons behind this reluctance can be understandable (for example, cost-recovery need and perceived sovereign security), it is a position that can prevent more donors from investing/supporting data acquisition projects. Thus, special attention should be given to the assessment of needs and actions during the design phase for a successful reach of data into the public domain.
7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors
(a) Grantee/Implementing agencies
67. Grantee comment. “It is our considered opinion, therefore, that the Moderately Unsatisfactory rating assigned to the overall project outcome is not fair, as it fails to, in addition to the abovementioned, recognize the fact that the full realization of the benefits to be derived from
18 this project is indeed a progressive process that does not fit within the confines of the limited time frame of its implementation, but requires a continued effort beyond such period.”
68. World Bank response. While the team recognizes the efforts made in acquiring Lidar data, the capacity increase as originally intended was not fully achieved in this project due to the implementation challenges highlighted in the report. Products of Lidar survey alone without preparation of other flood management activities limits the utility of the outputs of surveys. Given that intended use of Lidar information in hydrological/hydraulic modeling to carry out flood forecasting was cancelled from this project and flood management plans (contingency plans) were also deferred outside of the scope of this project, we cannot attribute these achievements to the current project, but they will certainly be evaluated under the projects where they were delivered, at the evaluation stage.
69. Grantee comment. “This, we believe, results from a literal (excessive) focus on the PDO that ignores the fact that the measures taken during implementation by the implementing agencies, the World Bank, and the primary donor not only permitted the attainment of the best results that could be possibly achieved under the circumstances, but ensured that those that could not, would be achieved in due course.”
70. World Bank response. Notwithstanding the efforts of the implementing agencies and the World Bank as well as the results achieved, the World Bank follows an objectives-based evaluation and thus needs to assess the outcome against the PDO.
(b) Cofinanciers/Donors
71. Issue 1. The need for revisiting the narrative around the development impact of the project, particularly on the inclusion/exclusion of associated outcome indicators, the linkage with the NWRDP and PPCR-HYDROMET, and the extent to which application of the Lidar data is covered within this project.
72. Comment accepted. The ICR was revised to reflect the shortcomings in the project design and impacts of late restructuring. The objective of creating the base data and enhancing capacity for flood management was fulfilled, but the actual benefits from the use of this capacity to reduce the impacts of floods can only be achieved once the required infrastructure and services are developed, and this is not part of the scope of this project. However, effort was made within the project scope to carry out activities to promote extensive use of the Lidar data beyond the water sector and to maximize its benefits, including training and capacity building for the cartography center.
73. Issue. Full coverage and more tangible evidence in the report of planned or actual institutionalization and use of the Lidar data and products.
74. Comment accepted. The ICR was revised to add more evidence on the current use of Lidar data within the National Water Resources Development Program, in other sectors, and in the region.
(c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)
19 75. No major comments were received from other stakeholders.
20 Annex 1. Project Costs and Financing
(a) Project Cost by Component (in US$ Million Equivalent)
Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions)
A. Limpopo River Lidar Survey and Applications 4.90 3.72 75.92% B. Zambezi River Lidar Survey and Applications 4.05 3.08 75.49% Total Project Costs 8.95 6.80
(b) Financing
Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (US$ millions (US$ millions Appraisal ) ) GFDRR Grant 4.90 3.72 75.92% GFDRR Grant 4.05 3.08 75.49%
21 Annex 2. Outputs by Component
Component A. Limpopo River basin high-resolution Lidar survey (ARA Sul)
Activity A1. A TA was recruited for eight months, plus an extension of six months; the TA supported the definition of the areas to be flown in the Limpopo Basin and in post-survey quality control.
Activity A2. Taking as a starting point the World Bank preliminary selection of priority survey areas,19 a total area of 13,460 km2 (62 percent of the initial estimate) was selected to be surveyed in the Limpopo Basin (9,460 km2 in the initial contract, plus 4,000 km2 in the extension contract) from a combined analysis of priority ranking and available funds.
Activity A3. 9,460 km2 was surveyed in the Limpopo Basin between December 2015 and January 2016 and an additional 4,000 km2 was surveyed during the extension phase in November 2016.
Activity A4. Quality of acquired data from the Lidar surveys was verified by the TA; Lidar data were processed by the service provider, and the following products were generated for the Limpopo Basin: (a) LAS (Lidar); (b) orthophotos (images orthorectified); (c) DTM; and (d) DSM.
Activity A5. Cancelled.
Activity A6. The following training actions were delivered:
o First training action for 10 technicians from DNGRH and ARA Sul, from May 25 to June 3, 2016, in Maputo, with a total duration of 32 hours equally distributed over 8 working days. The following topics were covered: M1 - Fundamentals of Lidar technology and geospatial reference system; M2 - Treatment and manipulation of Lidar information in a geographic information system (GIS) environment (Esri ArcGis); and M3 - Potential for using the Lidar information for cartographic purposes.
o Second training action for 13 technicians from DNGRH, ARA Sul, and 5 other institutions, from November 22 to 25, 2016, in Maputo, with a total duration of 16 hours equally distributed during 4 working days. The following topics were covered: M1 and M2. In addition, module M4 - Advanced processing of Lidar information in Global Mapper environment was delivered from December 1 to 2, 2016, with a total duration of 8 hours equally distributed over 2 working days.
19 Detailed in “Technical Advice on the Selection of Potential Target Regions for Airborne Lidar Survey Acquisitions over the Zambezi and Limpopo River Basins, Mozambique”.
22 o In parallel with the previous, there was a special customized session of module M3, delivered for 6 technicians from CENACARTA, with a total duration of 16 hours equally distributed during 4 working days, from November 22 to 25, 2016, in Maputo.
Activity A7. Single machine licenses of Global Mapper v17 were purchased and installed in workstations acquired for ARA Sul, DNGRH, and CENACARTA.
Activity A8. The following items were acquired and installed in ARA Sul, DNGRH, and CENACARTA: (a) network-attached storages with 30 TB for ARA Sul and 50 TB for DNGRH and CENACARTA and (b) workstations to access and process the Lidar data.
Component B. Limpopo River basin high-resolution Lidar survey (ARA Zambezi)
Activity B1. A TA was recruited for eight months; TA supported the definition of the areas to be flown in the Limpopo Basin and in post-survey quality control.
Activity B2. Taking as a starting point the World Bank preliminary selection of priority survey areas,20 a total area of 13,951 km2 (62 percent of the initial estimate) was selected to be surveyed in the Zambezi Basin from a combined analysis of priority ranking and available funds.
Activity B3. 13,951 km2 was surveyed in the Limpopo Basin between December 2015 and May 2016.
Activity B4. Quality of acquired data from the Lidar surveys was verified by the TA; Lidar data were processed by the service provider, and the following products were generated for the Zambezi Basin: (a) LAS (LiDAR); (b) orthophotos (images orthorectified); (c) DTM; and (d) DSM.
Activity B5. Cancelled.
Activity B6. The following training actions were delivered:
o A training action for 14 technicians from ARA Zambezi and seven other institutions, from December 19 to 21, 2016, in Tete, with a total duration of 20 hours distributed over 3 working days, covering modules M1, M2, and M4.
Activity B7. Single machine licenses of Global Mapper v17 were purchased and installed in workstations acquired for ARA Zambezi.
20 Detailed in “Technical Advice on the Selection of Potential Target Regions for Airborne Lidar Survey Acquisitions over the Zambezi and Limpopo River Basins, Mozambique”.
23 Activity B8. The following items were acquired and installed in ARA Zambezi: (a) network-attached storages with 30 TB and (b) workstations to access and process the Lidar data.
24 Annex 3. Economic and Financial Analysis
1. No new ex post economic analysis was undertaken, but the findings of the economic analysis undertaken at the time of project design (and captured in the Appraisal Summary section of the PAD) were considered valid.
2. Following such procedure, the ex post assessment of the economic benefits would require a new run of the adopted hydrodynamic models to update the estimates of floodplain inundation areas21 using the acquired Lidar data. As this exercise was conducted by a team of experts in DEMs, Lidar, and economic valuation, a full update of the original analysis was not performed.
3. Nevertheless, it should be considered that, from the initial estimate of US$4.38 billion, the economic benefits of the project are expected to increase in time as the total population and its density22 are projected to keep growing for the next three decades. In addition, the high-resolution spatial and topographical data of Lidar, derived products, and information assets remain relevant for repeat use in the longer term in multiple sectors and a wide range of applications.
4. In addition to Lidar-derived flood management, other benefits from the use of Lidar will contribute to increasing the value of the project. Although not always possible to quantify, the list includes time and effort saved on surveying, higher precision maps, flexibility of areas surveyed, reduced environmental footprint, and reduced risks and costs of poorly designed and located infrastructure, as recognized in the PAD. The combination of these factors support the general idea of high EIRR from Lidar interventions.
5. See also paragraphs 43–46 of the main text, which assess project efficiency.
21 Detailed explanation available from Moller, D., G. Schumann, and K. Andreadis. 2014. Report on Airborne Lidar Acquisitions over the Zambezi and Limpopo River Basins, Mozambique. 22 GPWv4: UN-Adjusted Population Density – 2015 and 2020, accessible from NASA Socioeconomic Data and Application Center - SEDAC (http://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-density- adjusted-to-2015-unwpp-country-totals)
25 Annex 4. Grant Preparation and Implementation Support/Supervision Processes
(a) Task Team Members
Responsibility/Speci Names Title Unit alty Lending/Grant Preparation Antonio Chamuço Sr Procurement Specialist AFTPC Procurement Célia dos Santos Faias Team Assistant AFCS2 Administration Elvis Langa Financial Management Specialist AFTFM Financial Eric Foster-Moore Operations Analyst AFTN2 Water Resources Disaster Risk & Climate Change DRM, Climate Francis S. Nkoka AFTN2 Specialist Change George Ledec Lead Environment Specialist AFTN3 Environment Hrishikesh Patel GIS Specialist AFTN1 GIS José Janerio Sr Finance Officer CTRLA Financial Kristine Schwebach Sr Social Specialist AFTCS Social development Water Resources Management Hydro-Met Services, Louise E.M. Croneborg AFTN2 Specialist, Task Team Leader Water Resources Luz Meza-Bartrina Senior Counsel LEGAM Legal Marcus Wishart Sr Water Resources Specialist AFTN2 Water Resources Rafael Saute Communication Specialist AFRSC Communication GFDRR Coord. Roberto White Sr DRM Specialist GFDRR Mozambique Sofia Bettencourt Lead DRM Specialist GFDRR DRM Non-Bank Staff Guy Schumann Flood Hydraulics Specialist Kostas Andreadis Hydrologist/GIS Specialist Delwyn Moller Remote Sensing Specialist Donald McKeown Sr Researcher, Lidar Jan van Ardt Associate Professor/Director Jeff Lazo Sr Hydro-Met Economist Supervision/ICR Antonio Chamuço Sr Procurement Specialist AFTPC Procurement Clarisse Nhabangue Team Assistant AFCS2 Administration Elvis Langa Financial Management Specialist AFTFM Financial Disaster Risk & Climate Change DRM, Climate Francis S. Nkoka AFTN2 Specialist, Task Team Leader Change Water Resources Management Hydro-Met Services, Louise E.M. Croneborg AFTN2 Specialist, Task Team Leader Water Resources Odete Muxímpua Water and Sanitation Specialist AFTN2 Water Resources Water Resources Management Shelley Mcmillan AFTN2 Water Resources Specialist, Task Team Leader Non-Bank Staff Jaime Palalane Water Resources Specialist/Researcher
26 (b) Staff Time and Cost
Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$, thousands (including No. of Staff Weeks travel and consultant costs) Lending FY14 5 39.62 Total: 5 39.62 Supervision/ICR FY15 6 54.52 FY16 0.5 1.57 FY17 8 20.47 Total: 14.5 76.56
27 Annex 5. Beneficiary Survey Results
Not applicable.
28 Annex 6. Stakeholder Workshop Report and Results
1. A project closing workshop was held at DNGRH in Maputo on February 7, 2016. This workshop was part of a closing mission that included a visit to surveyed areas in the Limpopo Basin, Gaza Province, on February 8, 2017 and a visit to DNGRH, ARA Sul, and CENACARTA in Maputo on February 9, 2017.
2. The project closing workshop counted with the presence of representatives from the two recipient agencies, ARA Sul and ARA Zambezi, DNGRH and CENACARTA, and from eight other stakeholder institutions. The World Bank and DFID were equally represented in the workshop and visits to beneficiary institutions and survey areas.
3. Following the introductory presentation of the project actions by the PAMT, and complementary presentations of conducted surveys and training actions by the service provider and the TA, the participants had the opportunities to express their opinions and assess the actions and results of the project.
4. Overall, participants were satisfied with the good progress of the project following a long initial period of delays and some uncertainty. The following major achievements were highlighted by the different participants:
The high financial execution rate of the project, with disbursement rates above 95 percent
The extension of the training actions beyond the direct beneficiary institutions, which was commended
The dissemination meetings that contributed to increasing the awareness and interest in the project actions and its results and in the Lidar data provided
The acquisition of equipment for safe management of the Lidar data
The equipping of CENACARTA, which will enable this institution to better fulfill its role of guardian and provider of Lidar data
The wide application of surveyed data in different sectors
5. The following weakness were identified:
Noninvolvement of the National Institute of Statistics
Difficulties in directly accessing the impact of the project actions in poverty reductions
Training actions in November and December, a critical period for availability of technicians
29 6. Few challenges were mentioned by the participants to maximize the benefits of the acquired Lidar data:
Replicate the training actions inside the beneficiary institutions to sustain the benefits of the project
Start using the data and derived products for DRM and other sectorial needs
Support CENACARTA to enable the public and free-of-cost access of the acquired data at national and regional levels, maximizing the chances of getting additional support to extend the covered areas and for surveying new basins
The need to provide unit costs for the different interventions to help in the assessment of the value for money
Maintain the network of Lidar users inside the Government institutions and make use of the acquired data
Continue with the multisectoral dissemination of the availability and potential of Lidar data
7. The substantial effort made by the implementing agencies, and by the PAMT, during the second half period of the project implementation and the extension period for timely resolution of implementation issues was commendable. The flexibility and professionalism of the service provider were recognized, as well as the valuable support from the TA that positively contributed to the project success.
8. For future interventions, the Licungo Basin was proposed, as well as an extension of the surveyed areas in the Zambezi Basin.
30 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR
1. The following notes were translated from the summary notes of the grantee’s ICR:
There is strong consistency between the project objectives and results.
Despite the initial delays with procurement, good coordination among the key stakeholders and project management allowed the achievement of expected results and good quality of the products, within the project time frame, after extension.
Good knowledge of the areas covered, the institutional capacity, and available tools, together with the changes in the original project design and rigorous assessment of the costs, allowed the extension of the areas covered under the project by 18 percent, which improved the performance of the project.
However, the survey did not cover all the flood-prone areas in both basins and these will not benefit from the enhanced flood modeling using the Lidar data. These additional areas should be surveyed to ensure that the modeling exercise accurately simulates flood events and supports improved land use planning and infrastructure development.
The trainings and capacity building activities, initially covered by the project, had a positive impact among the beneficiaries and the institutions.
The quality of the data delivered by the service provider respects high-quality standards with regard to accuracy and accessibility for trained users.
The level of satisfaction among beneficiary institutions is very high regarding the quality of the information and its potential for use across the sector.
Data sharing needs to be enhanced, including progressive demand creation and dissemination, as well as data sharing with engineering firms that are usually engaged in design and implementation of projects.
2. The following comments on the ICR were received from the grantee.
S. Comments Team Response No. 1 The report is very comprehensive and Thank you captures fairly well the essence of what transpired during the implementation of the project. 2 On the ratings assigned to the overall Notwithstanding the efforts of the implementing project performance and outcome: The agencies and the World Bank as well as the results assessment takes due cognizance of the achieved, we would like to note that the World justified changes made in respect of Bank follows an objectives-based evaluation and activities A5 and B5, and their absorption thus needs to assess the outcome against the PDO.
31 S. Comments Team Response No. incorporation into the relevant activities of other projects in the programme namely, P107350 (NWRDP) and P131049 (PPCR - HYDROMET), where such activities could be more efficiently implemented. However, in our opinion, more than just enhancing efficiency, these changes enhanced the overall benefits to be derived from the investment made, even if some will only be materialized later than originally anticipated. We also note with concern that the ratings assigned to different aspects in the assessment of outcomes, seem too formalistic and unbalanced, for they privilege the project´s conceptual assumptions (items 3.1 and 3.2), while the achievements seem to be relegated to insignificance, through a rather ungenerous rating that seems to be rather heavy handed and ignoring the realities and challenges of implementation, as well as the immense effort put in by the implementing agencies and indeed the Bank´s team, to overcome them. This, we believe, results from a literal (excessive) focus on the PDO, that ignores the fact that the measures taken during implementation by the implementing agencies, the Bank and the primary Donor, not only permitted the attainment of the best results that could be possibly achieved under the circumstances, but ensured that those that couldn’t, would be achieved in due course. 3 It is our considered opinion, therefore, that While the team recognizes the efforts made in the Moderately Unsatisfactory rating acquiring Lidar data, the capacity increase as assigned to the overall project outcome is originally intended was not fully achieved in this not fair, as it fails to, in addition to the above project due to the implementation challenges mentioned, recognize the fact that the full highlighted in the report. Products of LiDAR realization of the be benefits to be derived survey alone, without preparation of other flood from this project is indeed a progressive management activities, limits the utility of the process that does not fit within the confines outputs of surveys. Given that intended use of of the limited time frame of its LiDAR information in hydrological/hydraulic implementation, but requires a continued modeling to carry out flood forecasting was effort beyond such period. cancelled from this project and flood management plans (contingency plans) were also deferred outside of the scope of this project, we cannot attribute these achievements to the current project, but they will certainly be evaluated under the projects where they were delivered, at the evaluation stage. 4 On procurement challenges: The report is Comment accepted. A sentence was added to silent on the impact of changes of leadership reflect the changes in team leadership and impact (Task Team Leaders) in the WB team. The on procurement. project had 3 different TTLs during the
32 S. Comments Team Response No. implementation period and, while the last change had a positive impact, the first did not help. This change had a significant negative impact during tender preparation, i.e. a critical stage in the project implementation cycle, and contributed to the significant delay experienced in the early stages of the procurement process. 5 On making the data accessible to the The report recognizes the effort made in availing public: The rational for considering the data for ongoing studies and its incorporation on achievement of indicator 4 at 40% and flood modeling. Furthermore, the effort made in indicator 16 at 30% is unclear and do not installing the data at CENACARTA is a seem to the consistent, particularly when significant step toward data sharing. However, the assessment on indicator 16 refers that, accessibility to the public is yet to be fulfilled over and above CENACARTA, the data under this project and will require significant was also availed to ZAMCOM. More specialized processing and development of importantly though, the report does not take products that can be easily disseminated and cognizance of the fact that protocols and communicated with the public. regulations apply on the release of such data, and that DNGRH, ARA Sul and ARA Zambeze do not possess a mandate to do so, as this is of the exclusive competence of CENACARTA. Therefore, by involving and capacitating CENACARTA to absorb, store and manage the data, a huge step was taken to make the data accessible to the public. While it not possible to make the data available on a freely accessible platform (i.e. online), at this point in time, we are confident that through continued engagement with CENACARTA, ways will be found to, within regulatory constraints, make the data progressively more easily and freely accessible to the general public. This should therefore be seen in the context of an evolving process, and not necessarily as a point action that could be reasonably achieved within the short lifespan of the project.
33 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders
Not applicable.
34 Annex 9. List of Supporting Documents
1. ICR Guidelines
Implementation Completion and Results Report Guidelines (Last updated on July 22, 2014).
Guidelines for Reviewing World Bank Implementation Completion and Results Reports. A Manual for Evaluators (Last updated: August 1, 2014).
Independent Evaluation Group World Bank Project Performance Ratings - Codebook (September 2015).
Enrique Pantoja and Christopher Nelson (2017). Preparing High Quality Implementation Completion and Results Reports (ICRs) (Presentation).
Christopher Nelson (2017). Getting my ICR Right and Other Things Useful to Know about IEG (Presentation).
2. Project Appraisal Document
World Bank (2015). Project Paper for the Enhancing Spatial Data for Flood Risk Management Project (Report No: 87328-MZ).
World Bank (2016). Restructuring Data Sheet for Enhancing Spatial Data for Flood Risk Management Project (P149629) (Report No: RES24344).
Implementation Plan for the Enhancing Spatial Data for Flood Risk Management Project (P149629): June to December 2016 (extension).
3. Financial Agreements and Disbursement Letters
Disbursement Letter for GFDRR Grant No TF017383 - LiDAR Zambeze (June 6, 2014).
Disbursement Letter for GFDRR Grant No TF017384 - LiDAR Limpopo (June 6, 2014).
Second Disbursement Letter for GFDRR Grant No TF017383 - LiDAR Zambeze (June 6, 2014).
Second Disbursement Letter for GFDRR Grant No TF017384 - LiDAR Limpopo (June 6, 2014).
Revised Disbursement Letter - Grant Agreement No. TF017383 - (April 22, 2015).
35 Revised Disbursement Letter - Grant Agreement No. TF017384 - (April 22, 2015).
Request for a no-cost extension of the Enhancing Spatial Data for Flood Risk Management Project (June 1, 2015).
Amendment to Trust Fund Agreement (TF017384) - Extension of Closing date (June 27, 2016).
Amendment to Trust Fund Agreement (TF017384) - Extension of Closing date (June 27, 2016).
4. Quarterly Progress Reports (including Interim Financial Reports)
Quarterly Progress Report. Period of activity: January to March 2015.
Quarterly Progress Report. Period of activity: April to June 2015.
Quarterly Progress Report. Period of activity: July to September 2015.
Quarterly Progress Report. Period of activity: October to December 2015.
Quarterly Progress Report. Period of activity: January to April 2016.
Annual Progress Report. Period of activity: June 2015 to May 2016.
Quarterly Progress Report. Period of activity: July to September 2016.
Final Project Implementation Report. Period of activity: July to December 2016 (draft version).
36 MAP - Surveyed Areas under the Project
37