Impact Evaluation of Component 1 of the Sustainability Project of the Rural Water and Sanitation Sector (Prosasr) Results 2015-2019
Total Page:16
File Type:pdf, Size:1020Kb
IMPACT EVALUATION OF COMPONENT 1 OF THE SUSTAINABILITY PROJECT OF THE RURAL WATER AND SANITATION SECTOR (PROSASR) RESULTS 2015-2019 Index SUMMARY � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 3 ACKNOWLEDGEMENTS � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 5 INTRODUCTION � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 7 THE STATE OF NICARAGUA’S RURAL WATER AND SANITATION SECTOR � � � � � � � � � � 13 CURRENT ACCESS TO WATER AND SANITATION, WATER DISTRIBUTION SYSTEMS, AND SAFELY MANAGED WATER � � � � � � � � � � � 14 ACCESS TO WATER AND SANITATION AS MEASURED BY SIASAR � � � � � � � � � � � � 15 WATER DISTRIBUTION SYSTEMS IN RURAL NICARAGUA—FACTS AND FIGURES � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 17 ACCESS TO WATER ACCORDING TO THE SUSTAINABLE DEVELOPMENT GOALS � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 21 THE SECTOR’S INSTITUTIONAL STRUCTURE � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 22 HOW AVAR TRAINING AND ARAS SUPPORT IMPROVEMENT OF INSTITUTIONAL CAPABILITIES � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 25 THE PROJECT GOAL: HIGHER SIASAR-BASED SCORES � � � � � � � � � � � � � � � � � � � � � 26 TRAINING THROUGH AVAR WORKSHOPS � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 29 INTERVENTIONS BY WATER AND SANITATION REGIONAL ADVISORS (ARAS) � � 32 IMPLEMENTATION OF AVAR WORKSHOPS � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 33 IDENTIFICATION STRATEGY � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 35 SAMPLE AND ASSIGNMENT TO TREATMENT� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 36 DATA � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 38 EVALUATION SURVEYS � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 38 SIASAR DATA � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 39 DESIGN PRESERVATION AND BALANCE TESTS � � � � � � � � � � � � � � � � � � � � � � � � � � � 40 ESTIMATING EQUATIONS � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 43 RESULTS � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 44 SUMMARY OF RESULTS � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 44 DESCRIPTIVE RESULTS AT THE UMAS LEVEL � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 47 CAUSAL RESULTS AT THE HOUSEHOLD LEVEL � � � � � � � � � � � � � � � � � � � � � � � � � � � � 53 CLEAN ENVIRONMENT AND HYGIENE � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 53 SAFELY MANAGED WATER ACCESS, SANITATION, AND DIARRHEA � � � � � � � � � � � 55 CONCLUSIONS AND LIMITATIONS � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 57 BIBLIOGRAPHY � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 59 ANNEXES � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 64 3 Summary This document presents results of a randomized control trial on the effects of Component 1 of the Sustainability Project of the Rural Water and Sanitation Sector (PROSASR)� The intervention’s main objective was to strengthen institutional and management capabilities of Water and Sanitation Municipality Units (UMAS) and Water and Sanitation Committees (CAPS)� UMAS provide technical assistance to CAPS, the formal or informal institutions that manage, operate, maintain, and repair water distribution systems in the rural communities of Nicaragua� Concerning its main objective, the intervention succeeded: Impact evaluation results show that institutional and management capabilities of CAPS improved, in particular in operation, financial stability, and support for water distribution systems� In addition to these effects, the intervention already shows positive effects on longer-term outcomes such as improvements in sanitation indicators, and decreases in diarrhea� The intervention concluded in 2017/18 and endline measurement took place in 2019� Because measurement followed shortly after the intervention ended, all documented effects are intermediate� Benefits have yet to appear in terms of the longer-term goal of increasing access to safely managed water access� It is likely that not enough time passed to allow improvements in institutional and management capabilities to bring rural Nicaragua closer to this key objective� Updated data from the Rural Water and Sanitations Information System (SIASAR) in 2020, as well as a potential new data collection effort two years ahead, could help researchers to detect the longer-term impacts of the intervention� 5 Acknowledgements This impact evaluation was prepared and guided by Christian Borja-Vega (Senior Economist, Water Practice) with support from Pavel Luengas-Sierra (Consultant, World Bank) and Jonathan Grabinsky (Consultant, World Bank). The firm Sistemas de Inteligencia en Mercados y Opinión (SIMO) provided fieldwork supervision and support in formatting and producing this report. We give special thanks to Fondo de Inversión Social (FISE), to the Municipal Water Authorities (UMAS, by its acronym in Spanish), Water and Sanitation Committees (CAPS, by its acronym in Spanish) and the Ministry of Finance for their aid in this study� Particular help in evaluation design was provided by Virgilio Bravo, Trustee Administrator, FISE, Uriel Pérez, General Director of Public Credit, Ministry of Finance, and Ruth Humphreys, liaison with the ministry� Joxan Icaza López, Eduardo Osuna Palma, Eric Estrada Gómez, Anthony Palma Zeledon, and Jimmy Antonio Díaz Parrales, all of FISE, provided timely help and insights� We are grateful to Martin Albrecht and María Elite Gonzales Pérez (Water Resources Management Specialist, World Bank) for providing detailed comments about PROSASR, helping protect the evaluation design, and ensuring the success of fieldwork activities, to Lilian Pena, Clementine Stip, Sophie Ayling, and Alex Spevack, and all of the World Bank who helped in the evaluation design� Finally, we thank Joshua Gruber and Paul Gertler (University of California, Berkeley), Alberto Montoya Pérez, Nelson Medina, Gustavo Perochena, Rita Cestti, and the participants of the First Congress of Rural Water and Sanitation (2019) in Managua� Aidan Coville, Antonio Rodriguez, Miguel Vargas, Arndt Reichert of World Bank provided peer review comments. 7 Introduction There is little reliable data on what contributes to the success or failure of water and sanitation services in rural areas (Andres et al� 2018)� Even less is known about how institutional, managerial, technical, and operational capabilities contribute to sustainable services in communities beyond cities� But the available evidence suggests that good support and maintenance are key if water distribution systems are to operate reliably for the long term (Foster et al� 2018, Foster and Hope 2017, Thomson and Koehler 2016, and Whittington et al� 2009)� Studies on this topic grapple with the reality that quality of water and sanitation service in rural areas and sustainability are influenced by a complex interaction between local and national institutions� In an effort to broaden understanding of this interaction, this study applied a rigorous impact evaluation design to identify the causal effects of an intervention intended to strengthen institutional and management capabilities of rural water providers� In the long-run, better institutional and management capabilities are expected to improve water and sanitation services in rural areas� Nicaragua’s National Plan to Promote Human Development 2018-20191 has a critical mandate to increase access to water and sanitation services (WSS)� In 2013, the Government of Nicaragua developed the Integral Water and Sanitation Sector Program, to be implemented by the Emergency Social Investment Fund (FISE), the government institution that oversees water and sanitation services in rural areas� In 2014, the World Bank and the Government of Nicaragua launched the Sustainability Project of the Rural Water and Sanitation Sector (PROSASR), with particular attention going to poor communities� PROSASR has sought to consolidate institutions working on WSS and create a sustainability chain of better operational, technical, and financial management capabilities among operators of water distribution systems� PROSASR’s main objectives are to increase sustainable access to water and sanitation services in rural areas and to improve institutional capabilities to effectively respond to emergencies� 1 For details about the scope and objectives of the plan, see https://observatorioplanificacion�cepal�org/sites/default/files/plan/files/Nicaragua�EJES%20DEL%20PROGRAMA%20NACIONAL%20 DE%20DESARROLLO%20HUMANO�pdf PROSASR comprised various components for implementation� The main goal of Component 12 was to strengthen the institutional and management capabilities of Water and Sanitation Municipality Units (UMAS) so