Impact Evaluation of Component 1 of the Sustainability Project of the Rural Water and Sanitation Sector (Prosasr) Results 2015-2019

IMPACT EVALUATION OF COMPONENT 1 OF THE SUSTAINABILITY PROJECT OF THE RURAL WATER AND SANITATION SECTOR (PROSASR) RESULTS 2015-2019

Index

SUMMARY �� 3 ACKNOWLEDGEMENTS �� 5 INTRODUCTION �� 7 THE STATE OF NICARAGUA’S RURAL WATER AND SANITATION SECTOR �� 13 CURRENT ACCESS TO WATER AND SANITATION, WATER DISTRIBUTION SYSTEMS, AND SAFELY MANAGED WATER �� 14 ACCESS TO WATER AND SANITATION AS MEASURED BY SIASAR �� 15 WATER DISTRIBUTION SYSTEMS IN RURAL NICARAGUA—FACTS AND FIGURES �� 17 ACCESS TO WATER ACCORDING TO THE SUSTAINABLE DEVELOPMENT GOALS �� 21 THE SECTOR’S INSTITUTIONAL STRUCTURE �� 22 HOW AVAR TRAINING AND ARAS SUPPORT IMPROVEMENT OF INSTITUTIONAL CAPABILITIES �� 25 THE PROJECT GOAL: HIGHER SIASAR-BASED SCORES �� 26 TRAINING THROUGH AVAR WORKSHOPS �� 29 INTERVENTIONS BY WATER AND SANITATION REGIONAL ADVISORS (ARAS) �� 32 IMPLEMENTATION OF AVAR WORKSHOPS �� 33 IDENTIFICATION STRATEGY �� 35 SAMPLE AND ASSIGNMENT TO TREATMENT �� 36 DATA �� 38 EVALUATION SURVEYS �� 38 SIASAR DATA �� 39 DESIGN PRESERVATION AND BALANCE TESTS �� 40 ESTIMATING EQUATIONS �� 43 RESULTS �� 44 SUMMARY OF RESULTS �� 44 DESCRIPTIVE RESULTS AT THE UMAS LEVEL �� 47 CAUSAL RESULTS AT THE HOUSEHOLD LEVEL �� 53 CLEAN ENVIRONMENT AND HYGIENE �� 53 SAFELY MANAGED WATER ACCESS, SANITATION, AND DIARRHEA �� 55 CONCLUSIONS AND LIMITATIONS �� 57 BIBLIOGRAPHY �� 59 ANNEXES �� 64

Summary

This document presents results of a randomized control trial on the effects of Component 1 of the Sustainability Project of the Rural Water and Sanitation Sector (PROSASR). The intervention’s main objective was to strengthen institutional and management capabilities of Water and Sanitation Municipality Units (UMAS) and Water and Sanitation Committees (CAPS). UMAS provide technical assistance to CAPS, the formal or informal institutions that manage, operate, maintain, and repair water distribution systems in the rural communities of Nicaragua. Concerning its main objective, the intervention succeeded: Impact evaluation results show that institutional and management capabilities of CAPS improved, in particular in operation, financial stability, and support for water distribution systems. In addition to these effects, the intervention already shows positive effects on longer-term outcomes such as improvements in sanitation indicators, and decreases in diarrhea.

The intervention concluded in 2017/18 and endline measurement took place in 2019. Because measurement followed shortly after the intervention ended, all documented effects are intermediate. Benefits have yet to appear in terms of the longer-term goal of increasing access to safely managed water access. It is likely that not enough time passed to allow improvements in institutional and management capabilities to bring rural Nicaragua closer to this key objective. Updated data from the Rural Water and Sanitations Information System (SIASAR) in 2020, as well as a potential new data collection effort two years ahead, could help researchers to detect the longer-term impacts of the intervention.

5 Acknowledgements

This impact evaluation was prepared and guided by Christian Borja-Vega (Senior Economist, Water Practice) with support from Pavel Luengas-Sierra (Consultant, World Bank) and Jonathan Grabinsky (Consultant, World Bank). The firm Sistemas de Inteligencia en Mercados y Opinión (SIMO) provided fieldwork supervision and support in formatting and producing this report.

We give special thanks to Fondo de Inversión Social (FISE), to the Municipal Water Authorities (UMAS, by its acronym in Spanish), Water and Sanitation Committees (CAPS, by its acronym in Spanish) and the Ministry of Finance for their aid in this study� Particular help in evaluation design was provided by Virgilio Bravo, Trustee Administrator, FISE, Uriel Pérez, General Director of Public Credit, Ministry of Finance, and Ruth Humphreys, liaison with the ministry� Joxan Icaza López, Eduardo Osuna Palma, Eric Estrada Gómez, Anthony Palma Zeledon, and Jimmy Antonio Díaz Parrales, all of FISE, provided timely help and insights� We are grateful to Martin Albrecht and María Elite Gonzales Pérez (Water Resources Management Specialist, World Bank) for providing detailed comments about PROSASR, helping protect the evaluation design, and ensuring the success of fieldwork activities, to Lilian Pena, Clementine Stip, Sophie Ayling, and Alex Spevack, and all of the World Bank who helped in the evaluation design�

Finally, we thank Joshua Gruber and Paul Gertler (University of California, Berkeley), Alberto Montoya Pérez, Nelson Medina, Gustavo Perochena, Rita Cestti, and the participants of the First Congress of Rural Water and Sanitation (2019) in Managua� Aidan Coville, Antonio Rodriguez, Miguel Vargas, Arndt Reichert of World Bank provided peer review comments.

Introduction

There is little reliable data on what contributes to the success or failure of water and sanitation services in rural areas (Andres et al. 2018). Even less is known about how institutional, managerial, technical, and operational capabilities contribute to sustainable services in communities beyond cities. But the available evidence suggests that good support and maintenance are key if water distribution systems are to operate reliably for the long term (Foster et al. 2018, Foster and Hope 2017, Thomson and Koehler 2016, and Whittington et al. 2009). Studies on this topic grapple with the reality that quality of water and sanitation service in rural areas and sustainability are influenced by a complex interaction between local and national institutions. In an effort to broaden understanding of this interaction, this study applied a rigorous impact evaluation design to identify the causal effects of an intervention intended to strengthen institutional and management capabilities of rural water providers. In the long-run, better institutional and management capabilities are expected to improve water and sanitation services in rural areas.

Nicaragua’s National Plan to Promote Human Development 2018-20191 has a critical mandate to increase access to water and sanitation services (WSS). In 2013, the Government of Nicaragua developed the Integral Water and Sanitation Sector Program, to be implemented by the Emergency Social Investment Fund (FISE), the government institution that oversees water and sanitation services in rural areas. In 2014, the World Bank and the Government of Nicaragua launched the Sustainability Project of the Rural Water and Sanitation Sector (PROSASR), with particular attention going to poor communities. PROSASR has sought to consolidate institutions working on WSS and create a sustainability chain of better operational, technical, and financial management capabilities among operators of water distribution systems. PROSASR’s main objectives are to increase sustainable access to water and sanitation services in rural areas and to improve institutional capabilities to effectively respond to emergencies.

1 For details about the scope and objectives of the plan, see https://observatorioplanificacion.cepal.org/sites/default/files/plan/files/Nicaragua.EJES%20DEL%20PROGRAMA%20NACIONAL%20 DE%20DESARROLLO%20HUMANO.pdf PROSASR comprised various components for implementation. The main goal of Component 12 was to strengthen the institutional and management capabilities of Water and Sanitation Municipality Units (UMAS) so that they could in turn improve the functioning of Water and Sanitation Committees (CAPS), the local groups that operate rural water and sanitation services. 8 CAPS are the focal point of contact with rural communities and the final link in achieving the goal of sustainable services in the Nicaraguan countryside.

The current study is a rigorous and independent evaluation of Component 1. Funding from the Strategic Impact Evaluation Fund (SIEF) helped design the evaluation, collect data, and support research activities. The evaluation used randomization at the community level and detailed characteristics of households, communities, technical assistance providers, and operators of water and sanitation services. It is Nicaragua’s first impact evaluation of an intervention seeking to improve institutional and management capabilities of water and sanitation service providers and, to the best of our knowledge, one of the first in the literature.

Component 1 of PROSASR worked through training (AVAR) provided by FISE and through continuous support from water and sanitation regional advisors (ARAS) and other regional staff from FISE. All municipalities in the country were required to send a two- or three-person team from their Water and Sanitation Municipality Units (UMAS) to the training. These sessions provided standardized institutional and managerial training to UMAS on how to improve Water and Sanitation Committees (CAPS). Participants were required to develop, implement, and update action plans with specific goals and deadlines. These plans pursued to strengthen the institutional capacity of CAPS and to improve environmental, sanitary and hygiene conditions in rural communities. 2 Other components focused on infrastructure. For instance, Component 2 aimed to bolster services through better maintenance and improvement of infrastructure. There was no overlap between communities in this evaluation’s sample and those affected by Component 2. The current study’s goal is to analyze the effects that this intervention had on local service providers and households. For this purpose, researchers devised two random groups of 150 communities each— one group received the interventions, and the other did not, acting as a control group. Each community was pre-selected, based on it meeting a minimum and maximum level of infrastructure, and the goal was to measure the impact on 9 communities that could potentially benefit the most from the intervention. The experiment followed a phased-in design: control communities did receive the interventions from Component 1, but did so at a later date, at the end of 2018 or the beginning of 2019.

As is customary of impact-evaluations, effects are differentiated between intermediate and longer-term outcomes. Intermediate outcomes are those related to the project’s objectives of strengthening institutional and management capabilities of UMAS and CAPS. Long-term outcomes are those expected to materialize over time, as the effects of the program materialize, and filter-down to the community. Long-term effects are primarily expected in increases in a household’s safely managed water. This is linked to the intervention’s aim of improving the operation and maintenance of the systems, and helping CAPS better protect and manage the water-source (Table 8). Moreover, community-level increases in access to clean water are expected to have positive, long-term impacts on health; mostly in reducing levels of household diarrhea. Although secondary, the intervention also included trainings on proper community hygiene, which are expected to have long-term impacts on a household’s access to clean hygiene and sanitation.

The evaluation found that the program succeeded in improving intermediate- level indicators: CAPS institutional capabilities, measured by an index with range from 1 to 4, reached a value of 2.99 points in treatment communities compared with 2.69 points in control communities (0.30 points, 11 percent, or 0.42 standard deviations). In particular, CAPS improved along the following indicators: formal operation (+0.36 points), financial stability (+0.42 points), and system operation and maintenance (+0.30 points). Construction of the index and its components were set and validated by FISE in the early 2010s, when the Rural Water and Sanitation Information System (SIASAR), a joint 10 initiative of eleven countries that collects detailed information of water distribution systems and their operators, began activities in Nicaragua.

Although additional time is required for some of the longer-term outcomes to materialize, the intervention found preliminary, positive effects on downstream, household-level, sanitation indicators. Open defecation decreased by 1.8 percentage points (equivalent to a change of -37 percent), access to improved sanitation increased 3.7 points (+8 percent), and use of non-shared facilities increased 3.2 points (+4 percent). Plausibly linked to improvements in sanitation, diarrhea decreased 2.2 percentage points (-16 percent).

On the other hand, the program has yet to show improvements in the long-term indicator for safely managed access to water, as defined by the Sustainable Development Goals (SDG)—households having water from an improved source, located on premises, available when needed, and free from fecal (and priority chemical) contamination. Results show improvements from baseline to endline in both treatment and control groups, with slightly higher but not statistically significant effects in the treatment group. Because measurement took place in 2019, shortly after the intervention ended in 2017/18, it is likely that insufficient time had passed for the improvements in institutional and management capabilities to filter-down to the household level.

In contrast with the evaluation sample, SIASAR data from baseline to endline from all rural communities in Nicaragua showed decreases in access to sanitation and deterioration of older water distribution systems. This is consistent with other studies that find that systems stop working properly three or five years after construction owing to low technical capacity to operate them and lack of institutional support (Borja-Vega et al. 2017). Thus the experimental sample is not representative of all communities in the country. Rather, it is representative of communities that were chosen for the study because they were deemed able to benefit the most from the intervention. SIASAR data from all rural communities show that lack of long- term sustainability of water and sanitation services is a serious problem. But the impact evaluation showed that the Component 1 intervention strengthened institutional capabilities of service providers and that this, in the short term, can redress some of the deterioration, and improve rural residents’ well-being. 11

Results so far, albeit short-term and intermediate, are promising. The use of updated data from SIASAR in 2020, as well as a potential new data collection effort in two years, could elucidate the long-term effects of this intervention. The document proceeds as follows. Section 2 describes the state of Nicaragua’s rural water and sanitation sector. Section 3 describes the intervention. Section 4 details the impact evaluation design and the identification strategy. Section 5 presents the results. Section 6 offers conclusions.

The State of Nicaragua’s Rural Water and Sanitation Sector Current Access to Water and Sanitation, Water Distribution Systems, and Safely Managed Water

On average, rural areas in Nicaragua have experienced an increase in access to water in recent years but a decrease in access to sanitation. The increase in 14 access to water took place in all but one of the country’s fifteen departments. The decrease in sanitation occurred in four of the fifteen and one of the country’s two autonomous regions.

A large rise in the number of water distribution systems explains the general increase in access. But when continuity of service, distance to users, treatment of water with chlorine, and water quality analysis are considered, most systems perform positively in just one or two of these attributes. Compared to newer systems (those less than four years old), older systems (five years or more) show fewer of these positive attributes. For example, a high proportion (23 percent) of older systems in the Pacific region, the region with the highest access to water services, shows none of the four positive attributes. This suggests that it is not enough to focus on building new systems. It is also critical to find ways to improve management and institutional capabilities, this to ensure that new systems continue to provide water safely and reliably to the country’s rural residents.

The Rural Water and Sanitation Information System (SIASAR) is a joint initiative established in 2011. To date, eleven Latin- American countries and Kyrgyzstan participate in it. SIASAR uses a set of common questionnaires to collect detailed information on rural communities, on local providers of water and sanitation services, on those who provide them with technical assistance (municipality technical staff, in most cases), and on characteristics of water distribution systems. Collected information gets uploaded to an information system that performs validity checks. The information system then calculates and publishes online indicators on the state of the water and sanitation sector. The system allows access to all collected information.

Nicaragua collected information on all rural communities for the first time in 2012/13, updated information in some departments and regions in 2015, and updated information for all rural communities in 2017/19. Access to Water and Sanitation as Measured by SIASAR

Between 2012/13 and 2017/19, access to water in rural Nicaragua increased from 41 to 51 percent, but access across regions was not equal. In 2017/19 the Pacific region had the highest level of access (65 percent) and the autonomous regions the lowest. In the autonomous regions, access increased from 11 to 25 percent. Access to sanitation, in contrast, decreased from 51 to 47 percent 15 across all rural areas. This decline was concentrated in the Pacific region (falling from 70 percent to 64 percent) and in the autonomous regions (30 percent to 19 percent). Access to sanitation in the Central region, meanwhile, increased from 44 percent to 48 percent (Table 1).

Access to improved 2012/13 2017/19 Increase (%) services

Water 41% 51% 24%

Pacific 51% 65% 29%

Central 47% 54% 17%

CCAR 11% 24% 117%

Sanitation 51% 47% -9%

Pacific 70% 64% -8%

Central 44% 48% 9%

CCAR 30% 19% -35%

Table 1. Access to Water and Sanitation according to the Millennium Development Goals (MDG)

Notes: Percentage of dwellings CCAR: Caribbean Coast Autonomous Regions. Water: Access to water distribution system (aqueduct or well with pump). Sanitation: Access to protected latrine or to toilet connected to sewage.

Annex I expands statistics on access to water and sanitation to the department level. Figure A1.1 shows access to water by department. At this level, access decreased only in Carazo (56 percent to 47 percent). By region, the highest increases were in the Pacific region in Managua (41 percent to 79 percent); Rivas (40 percent to 61 percent) the Central region in Rio San Juan (31 percent to 48 percent), and Boaco (41 percent to 61 percent). In the autonomous regions, the North region showed a large increase: from 4 to 21 percent, closing the gap with the South region, which experienced an increase from 19 to 26 percent. Figure A1.2 shows access to sanitation by department. It is noteworthy that of the fifteen departments and two autonomous regions, access decreased only in four departments and in one autonomous region. In the Pacific region, it decreased in Carazo (70 percent to 47 percent) and Managua (78 percent to 59 percent). In the Central region, it fell in Madriz (76 percent to 61 percent) 16 and Nueva Segovia (47 percent to 40 percent). In the autonomous regions, the North experienced a sharp decrease: from 42 to 19 percent whereas the South experienced an increase: from 16 to 20 percent.

These figures only reflect levels of access to water distribution systems and do not reflect how they function. With the move from the Millennium Development Goals (MDGs) to the Sustainable Development Goals (SDGs) came an expansion of the criteria that defines what is adequate access to water, sanitation, and hygiene (WASH). SDG 6.1 states that “by 2030, universal and equal access affordable to all will be achieved.” SDG 6.3 lays out a series of higher attributes, including access to high-quality water, to define adequate access to WASH. Table 2 presents the ladder for measuring SDG access to water indicators. The highest rung means that everyone has access to water from an improved source, located on premises, available when needed, and free from fecal or priority chemical contamination.

Water from an improved source, on premises, available when Safely managed needed, and free from fecal or chemical contamination

Water from an improved source located within a Basic 30-minute round trip (including waiting time)

Water from an improved source located more than a Limited 30-minute round trip away (including waiting time)

Unimproved Water from unprotected well or unprotected stream

Surface water Water from rivers, dams, lakes, or irrigation channels

Table 2. Ladder of Access to Water according to Sustainable Development Goals (SDG) 17

Water Distribution Systems in Rural Nicaragua—Facts and Figures

SDG water standards require the monitoring of service, distance to users, and water quality. Using information from SIASAR, this section analyzes water distribution systems in rural areas according to SDG benchmarks. Table 3 shows the number of systems in rural areas across the country. Rural water distribution systems increased from about 4,800 in 2012/13 to 6,000 in 2017/19, a rise of 26 percent. In the autonomous regions, the number doubled. In the Pacific region, it increased 41 percent, and in the Central region, which already had a large number of systems in 2012/13, it increased 16 percent.

2012/13 2017/19 Increase (%)

Rural 4,769 5,996 26%

Pacific 1,117 1,575 41%

Central 3,261 3,795 16%

CCAR 391 626 60%

Table 3. Number of Water Distribution Systems in Rural Areas

Notes: CCAR: Caribbean Coast Autonomous Regions. Aqueducts by gravity or pump, wells with pumps, and rainwater systems. Based on SIASAR information, Table 4 presents four positive attributes that water distribution systems should have. These are (1) that systems are located, on average, less than 100 meters away from most dwellings in the community, (2) that they work 24 hours per day every day, (3) that they treat water with chlorine, and (4) that they monitor water quality. These attributes are not necessarily complements. For example, SIASAR data shows that systems based on wells with pumps work with fewer interruptions than systems based on aqueducts but are less likely to treat water with chlorine. Owing to reliability, rural residents might prefer water from systems based on wells instead of aqueducts but would thus face a higher risk of disease3. The table also compares attributes of systems available in 2012/13 and in 2017/19.

As of 2017/19, few systems treated water with chlorine (27 percent) and only 18 about half worked 24 hours a day, monitored water quality, or were located less than 100 meters away from most dwellings in the community. Compared with 2012/13, three of these four attributes worsened, in particular, continuity of service. Only water quality monitoring improved (47 percent to 52 percent). Few systems had none of the four attributes (11 percent) and few had all four (4 percent). Most systems had one or two (65 percent). The comparison between new systems and old ones in 2017/19 shows that new systems were better in all attributes, yet few of them (37 percent) treated water with chlorine.

Rural 2017/19

2012/13 2017/19 Old (80%) New (20%)

System age (years) 11.3 12.7 15.5 2.2

Attributes

(1) Less than 100 meters 48% 41% 40% 41%

(2) Work 24 hours per day 62% 56% 54% 63%

(3) Treat water with chlorine 30% 27% 24% 37%

(4) Perform water quality tests 47% 52% 51% 55%

Systems have:

No attribute 7% 11% 12% 8%

One or two 68% 64% 66% 59%

Three 22% 20% 19% 25%

All four 4% 4% 4% 7%

Table 4. Characteristics of Water Distribution Systems

Notes: Old: 5 years or more since construction. New: 4 years or less Less than 100 meters: The community leader estimates that most dwellings are less than 100 meters away from the water distribution system.

3 For example, see P. Loebach and K. Korinek. 2019. “Disaster vulnerability, displacement, and infectious disease: Nicaragua and Hu- rricane Mitch;” Population and Environment, 40(4), 434–455. https://doi.org/10.1007/s11111-019-00319-4, and J. Wolf et al. 2019. “A Faecal Contamination Index for interpreting heterogeneous diarrhea impacts of water, sanitation and hygiene interventions and overall, regional and country estimates of community sanitation coverage with a focus on low- and middle-income countries.” International Journal of Hygiene and Environmental Health. https://doi.org/10.1016/j. Table 5 expands to regions the comparison between new systems and old ones in 2017-19. As measured by the attributes, new systems performed better than old ones, especially in the autonomous regions, where 70 percent of new systems monitored water quality. Deterioration of old systems in the Pacific region was cause for concern. There 23 percent of old systems had none of the four attributes, compared to just 8 percent of old systems in the 19 Central region and 11 percent in the autonomous regions.

Pacific Central CCAR Systems en 2017/19 Old New Old New Old New (71%) (29%) (83%) (17%) (76%) (23%)

Attributes

(1) Less than 100 meters 29% 41% 44% 40% 47% 46%

(2) Work 24 hours per day 46% 63% 55% 62% 70% 71%

(3) Treat water with chlorine 25% 33% 25% 38% 16% 43%

(4) Perform water quality tests 40% 50% 56% 56% 49% 70%

Systems have:

No attribute 23% 13% 8% 6% 11% 2%

One or two 62% 58% 68% 62% 59% 53%

Three 13% 22% 19% 25% 29% 34%

All four 3% 7% 4% 7% 1% 11%

Table 5. Characteristics of Water Distribution Systems by Region and System Age, 2017/19

Notes: CCAR: Caribbean Coast Autonomous Regions. Old: 5 years or more since construction. New: 4 years or less. Less than 100 meters: The community leader estimates that most dwellings are less than 100 meters away from the water distribution system.

Figure 1 shows in maps where rural water distribution systems were located in 2012/13 and in 2017/19. Red dots depict systems with none of the four attributes. Yellow dots show those with one or two, and green dots those with three or all four. The maps, alongside information in previous tables, demonstrate that old systems are worsening. Their declension is concentrated in the three departments in the Pacific region that have the highest access to rural water distribution systems in the country: Masaya, Managua, and León. The North Caribbean Coast Autonomous Region suffers this problem as well. (a) Systems in 2012/13

RACCN

NUEVA SEGOVIA JINOTEGA 20 MADRIZ ESTELI

CHINANDEGA MATAGALPA RACCS

BOACO

LEÓN CHONTALES MANAGUA SYSTEMS IN 2012/13 MASAYA CARAZO # of attributes (N=4,769, 100%) GRANADA None (N=315, 7%) RIVAS RÍO 1 - 2 (N=3,219, 68%) SAN JUAN 3 - 4 (N=1,235, 26%)

(b) Systems in 2017/19

RACCN

NUEVA SEGOVIA JINOTEGA

MADRIZ ESTELI

CHINANDEGA MATAGALPA RACCS

BOACO

LEÓN CHONTALES MANAGUA SYSTEMS IN 2017/19 MASAYA CARAZO # of attributes (N=5,997, 100%) GRANADA None (N=658, 11%) RIVAS RÍO 1 - 2 (N=3,866, 64%) SAN JUAN 3 - 4 (N=1,473, 25%)

Figure 1. Rural Water Distribution Systems According to Four Positive Attributes

Note: The four positive attributes are (1) On average, less than 100 meters away from dwellings, (2) Works 24 hours per day every day, (3) Treats water with chlorine, and (4) Performs water quality tests. Access to Water According to the Sustainable Development Goals

SIASAR information gets collected through surveys of community leaders and operators of water distribution systems. To provide a detailed overview and comparison at the household level, this section uses a baseline and endline derived from household surveys conducted in 2015/16 and 2019 in 21 communities that were part of the evaluation sample. The sample covered all the country but, as explained in Figure 5, focused on municipalities and communities with water distribution systems that had neither failed nor were in perfect condition. Trends presented here are restricted to household data from the 300 communities included in the impact evaluation, and are not comparable with those presented in the previous section, which cover all rural areas and all systems.

Table 6 demonstrates how indicators were constructed from information collected by these surveys to reflect the SDG ladder of access to water.4 Figure 2 provides SDG ladder estimates for 2015/16 and 2019, differentiating between dry and wet seasons. It shows that access to safely managed water increased in dry season from 17 to 27 percent and in wet season from 24 to 38 percent.

Information from household surveys Ladder Connected Source if not Minutes to Water is Continuity to system connected water source sufficient

No service Safely managed 0 interruptions Yes Protected Yes Basic stream 1 to 30

Limited More than 30 With or without interruptions Yes or No Unprotected Unimproved or No source or Any time Surface water surface water Table 6. Definition of Access to Water According to the SDG and Information from Household Surveys

Notes: The safely managed definition considers water quality. At the time of writing of this report, collected water quality tests were being validated. Water is sufficient: Self-estimate of whether available water is enough to satisfy household needs. The survey captures water consumed in liters but only the endline survey values captured them carefully and with validation checks.

4 At the time of writing of this report, collected water quality tests were being validated. Given the importance of water quality, Annex VIII presents a water quality analysis based on SIASAR information. 100% 17% Safely managed 29% 27% 24% 80 % 38% Basic

60 % 30% Limited 62% 54% 22 40 % 57% 46% Surface water or unimproved 20% 37% 19% 14% 19% 14% 0% JMP, 2017 2015/1620192015/16 2019 DRY SEASON WET SEASON

Figure 2. Access to Water According to the SDG and Information from Household Surveys (Percentage of Population)

Sources: JMP, 2017: Rural Nicaragua estimates for 2017 from the Joint Monitoring Programme for Water Supply, Sanitation and Hygiene by WHO and UNICEF.

Notes: In contrast to JMP 2017, results from the evaluation surveys show higher access to basic water and lower use of surface or unimproved water because, as shown in Figure 6, evaluation surveys focus on municipalities with functioning water distribution systems. Another difference is that estimates from the evaluation survey omit water quality as an attribute of safely managed water. Table 6 details the reasons for this.

Estimates use the baseline and endline surveys. These surveys cover about 300 communities and have a sample of between 4,500 - 5,000 households.

The Sector’s Institutional Structure

In 2003 the Government of Nicaragua developed a national strategic plan for the water and sanitation sector (Programa Integral Sectorial de Agua y Sanemiento, PISASH) that gave FISE the responsibility for sustainable water and sanitation services in rural areas (Figure 2). FISE formulates and coordinates policies and plans. It contracts for and executes infrastructure projects and develops institutional capabilities at the municipality and community levels. Staff at the regional and local levels provide technical, operational, and managerial assistance to water distribution systems in rural areas. Below the national level, Water and Sanitation Municipality Units (UMAS) provide technical assistance to Water and Sanitation Committees (CAPS), which are formal or informal institutions that manage, operate, maintain, and repair water distribution systems at the community level in rural areas.

Staff at the municipality and community levels receive support from regional water and sanitation advisors (ARAS). ARAS, who are usually water and sanitation engineers, help shepherd all components of PROSASR across communities. They communicate FISE’s regional policies and foster technical capabilities at UMAS. They also provide technical support to UMAS and CAPS on operation, maintenance, and corrective plans for water distribution systems. They help on community improvement and gender inclusion. ARAS spend about three quarters of their time in the field supporting UMAS and CAPS and the remainder in Managua, receiving training from FISE. Municipality advisors (AMU), social specialists, and environmental experts support ARAS in activities that FISE envisages5. 23

National level: Social Investment Fund - Rural entity WSS (FISE)

• Sectoral policies orientation

• Assigns priority and implement investments

• Support sustainability chain

Regional technical assistance of WSS (ARAS)

• Communication channel between national and local levels

• Support municipality units on capacity building

Municipality level: Water and Sanitation Municipality Units (UMAS)

• Provide technical assistance to Water and Sanitation Committees (CAPS)

• Monitor coverage

Local level: Water and Sanitation Committees (CAPS)

• In the project planning stage, social community audit. Once the project is completed, administration

• Service provision and tariff collection

• Operation and maintenance

Figure 3. Nicaragua’s Rural Water and Sanitation Sector A Hierarchical Order

5 According to conversations with FISE (Spring 2019).

How AVAR Training and ARAS Support Improvement of Institutional Capabilities 26

The intermediate objectives of the impact evaluation strive to assess the effect at the CAPS and household levels of Component 1 of the Sustainability Project of the Rural Water and Sanitation Sector (PROSASR). The longer-term goals of the evaluation aim to measure impact at the household level, such as improvements to access to safely managed water and sanitation.

PROSASR’s purpose is to improve and increase water and sanitation services in a sustainable way. Component 1 had as its main goal the strengthening of institutional and management capabilities of Water and Sanitation Municipality Units (UMAS) to enable them to improve the operations of Water and Sanitation Committees (CAPS), the local providers of water and sanitation services. Activities in this component consisted of training based on the Learning Linked to Results methodology (AVAR) and support from water and sanitation regional advisors (ARAS). The training was provided to technical staff from all UMAS in the country and their equivalent units, known as UTASH, in the Alto Wangki Bocay special regimen zone, located in the Jinotega department.

The Project Goal: Higher SIASAR-Based Scores

The intervention specifically aimed to improve the scores of UMAS, the scores of CAPS, and the clean environment and hygiene in individual communities as calculated using SIASAR data. SIASAR has three data collection sources in Nicaragua: a rural census in 2012/13, a data update for some departments in 2015, and a census in 2017/19. Table 7 shows the criteria used to grade UMAS and Table 8 those to grade CAPS. Each category on each table got a score from 1 to 4, with the overall score being the average of the categories. Overall scores were classified into four categories (A, B, C, and D) in which A and B denote the better-performing 27 units. The specific objective for UMAS and CAPS was to increase their overall scores and the number in categories A or B. No similar objective in terms of score existed for clean environment and hygiene indicators, however. Table 9 shows instead the community aspects that training participants analyzed and strived to improve. Annex II details the specific indicators that each category encompasses6.

CATEGORY CRITERIA

The UMAS having information from Data availability all communities in its area and the information is up to date

Communities visited during Number of communities in the the last twelve months municipality visited

Support to communities for Number of communities in the water quality monitoring municipality supported

Human resources Ratio of communities to technicians

Transportation capacity Ratio of vehicles to technicians

Equipment for water quality monitoring, computer, vehicles, Availability and condition of the equipment informative printed material

Has available assigned annual budget, funds for travel expenses Number available and fuel, and Internet service

Table 7. Requirements to Promote UMAS to a Higher Category in SIASAR

6 SIASAR employed improved questionnaires in the 2017/19 survey wave. These improvements, however, rendered some indicators constructed in 2017/19 no longer comparable with those based on previous waves. Before workshops started, FISE foresaw this compli- cation and adjusted the official indicators published on SIASAR’s webpage in order to estimate comparable indicators between waves. Annex II lists the indicators FISE used in the workshops. For the purpose of this evaluation, indicators were further refined to ensure strict comparability between SIASAR collection waves. Annex II also details these refinements. 28

CATEGORY CRITERIA

CAPS has been legalized.

Board of directors members are assigned and well defined. Formal operation Board of directors had at least four meetings over the last six months.

CAPS provides details on financial accounts every three months

Tariffs have been established.

Tariffs allow recovery of costs. Adequate tariff for water supplied Revenues cover eighty percent or more of costs.

Tariffs vary in relation to consumption.

CAPS has bank account.

Financial stability Up-to-date accounting records are kept.

Revenues are higher than costs.

System replenishment fund is adequate.

System operation and maintenance CAPS provides corrective and preventive maintenance.

There are personnel assigned only to operation and maintenance.

Community keeps water source clean and Water source protection has a reforestation plan in place.

Table 8. Requirements to Promote CAPS to a Higher Category in SIASAR CATEGORY CRITERIA

Open defecation

Clean environment Trash presence 29 Puddle presence

Handwashing

Hygiene Latrine use

Safe water storage

Table 9. Criteria or Environment and Hygiene Evaluates at the Community Level

Training through AVAR Workshops

AVAR training consisted of three rounds of workshops of two or three days each. AVAR specialists, with the help of the ARAS, oversaw and administered the trainings. Participants were teams of two or three persons from the technical staff of each UMAS or UTASH in the country. FISE divided the country into nine groups of departments and regions. For logistical reasons, AVAR started on different dates for each group but the content and training activities were identical. Each group included people from between 20 and 36 UMAS or UTASH. In total, 244 technical staff underwent training. Time between rounds was planned to be about four months but for most groups it was more than six months, and in some cases about 117.

Table 10 synthesizes the content covered in each of the workshops. Knowledge imparted was practical and focused on supporting actual work activities. Participants learned to use and analyze SIASAR information, in particular the criteria that evaluate UMAS and CAPS; to make and update plans to operate and maintain water distribution systems, and ensure their sustainability; to understand the legal process and regulations that CAPS require to properly function; to perform water quality tests; and to appreciate the transversal role that gender has in CAPS and in water and sanitation projects. All activities directly targeted improvement of the overall scores of UMAS and CAPS. Crucially, participants were taught the indicators that SIASAR uses for each category on each matrix.

Training content focused on improving water sustainability. To a lesser extent, as stipulated by the indicators outlined in Table 9, training content considered promoting clean environment and hygiene in the community. This included work on managing the sustainability of sanitation facilities, and sharing strategies for households to follow proper hygiene.8

7 According to conversations in 2019 with FISE staff, a series of difficulties—among them problems that ARAS experienced in reaching some communities and low institutional capability in others—prevented keeping the time between rounds to six months in all groups. 8 Additional information on the institutional landscape of the rural water and sanitation sector in Nicaragua, and the role of the CAPS in the community, can be found here: https://www.developmentbookshelf.com/doi/10.3362/1756-3488.2017.003. Activities and products by workshop participants can be classified in four:

1. Analysis of SIASAR data: In the first workshop round, participants assess the initial 30 situation of the UMAS and of all CAPS in the municipality according to SIASAR 2012/13. In the second round, they assess progress or lack thereof according to SIASAR 2015. In the third, they compare the initial situation with SIASAR 2017/19

2. Action plans: In the first workshop round, participants make three action plans: to improve the UMAS, to improve CAPS, and to improve clean environment and hygiene. In the second round, they adjust these plans aided by group discussions of lessons learned and of difficulties experienced. In the third, they make a plan to support CAPS that improved and a plan to improved CAPS that failed to improve.

3. Visits to communities: In the second round, all participants visit two communities to discuss how action plans were implemented. In the third round, they visit another two communities. These communities were selected because they improved to categories A or B. Best practices are discussed.

4. Document submission: In the third round, participants submit legal and administrative documents for all CAPS in the municipality. WORKSHOP 1 WORKSHOP 2 WORKSHOP 3

2 days (8 hours in the classroom, 2 days (8 hours in the classroom, 3 days (24 hours in the classroom) TIME 8 hours for field visit) 8 hours for field visit)

• SIASAR: Description and use • Understanding how SIASAR • Gender concerning action plans evaluates UMAS and CAPS and water and sanitation projects 31 • Key characteristics of an action • Legal process to administer CAPS • How to ensure sustainability plan to improve UMAS, CAPS, and water distribution systems of water and sanitation and communities. (Specific • SWOT methodology projects and investments objectives, dates, monitoring, (Strengths, Weakness, • How to make sustainability plans

KNOWLEDGE assigned resources, etc.) Opportunities, and Threats) • Action plans and • Water quality maintenance plans for water distribution systems

[R] Compare scores in SIASAR 2012/13 of the UMAS and of all [R] Analysis of scores in SIASAR CAPS in the municipality with 2012/13 of the UMAS and of updated scores in 2017/19. all CAPS in the municipality. [R] Compare scores in SIASAR [P] Develop a sustainability plan Clean environment and hygiene 2012/13 of the UMAS and all for CAPS and UMAS with final indicators of all communities CAPS in the municipality with category A and B and a corrective are also analyzed. updated scores in 2015. one for those in C and D. [P] Develop three action plans: [P] SWOT analysis of the actions [V] Visit to field: Two communities (1) Improve UMAS, (2) Improve plans to update them in which CAPS improved to A or B. CAPS in the municipality, (3) [V] Visit to field: Two communities [D] Submit legal and Improve clean environment and to discuss implementation administrative documents of all

TASKS AND PRODUCTS hygiene in the communities. of action plans CAPS in the municipality—plan [V] No visit to field of operation and maintenance of the system, legalization of CAPS, bank account opening, copies of accounting system.

Table 10. AVAR Workshops [R] Review of analysis of SIASAR data [P] Work on action plans based on [R], knowledge, and group discussion [V] Field visits to learn and discuss best practices [D] Submission of legal and administrative documents of all CAPS in the municipality Each UMAS has a score between 1 and 4 on each of six components (CAPS have only five). The mean score across components is the UMAS and CAPS overall score (see Annex II). Mean overall scores have the following categories: A [3.5 - 4.0], B [2.5 - 3.5), C [1.5 - 2.5), y D [1.0 - 1.5]

Central to the items covered during the first workshop was the development of a Municipal Action plan , which lays out a series of steps for the municipality to take in order to increase the sustainability score of the CAPS and UMAS. The feedback from these plans helped inform the content of subsequent trainings, and better-tailor the subsequent content administered to UMAS, as part of the AVAR trainings.9

In practice, is unlikely that knowledge acquired by workshop participants gets no dissemination to all CAPS in their municipality. For this reason, the experimental design works through the municipality action plan. During the training, FISE made available to all participants the names of the 150 communities in the control group. Action plans would be carried out in all communities in the study, including those in the control group, but participants were asked to postpone activities in the control communities until the last trimester of 2018 or the first of 2019, so as to allow

9 Annex VII includes an example of an outline given to UMAS representatives, as part of the first workshop, that details how to formulate a Municipal Action Plan. 32

comparisons between the two groups. The ARAS and the AVAR specialists verified that the research design was maintained and that action plans were applied according to the stipulated timetable.10

Interventions by Water and Sanitation Regional Advisors (ARAS)

Water and sanitation regional advisors (ARAS) are staff members from FISE, usually water and sanitation engineers. They communicate FISE’s regional policies and foster the technical capabilities of UMAS. During the study, they oversaw all components of PROSASR.11 For AVAR workshops, their fundamental task was to review and guide action plans. ARAS started support to all treatment communities around 2015. They also took part in SIASAR activities to ensure that workshop participants had the information they needed to make and update action plans. The ARAS were also in charge of helping ensure compliance with the experimental design.

To supplement AVAR workshops. ARAS had specific objectives:

- Provide information to guide action plans elaboration and to support their implementation. - Give technical support to develop operation, corrective, and maintenance plans for water distribution systems. - Support communities to improve clean environment and hygiene.

10 Due to emergency situations, 38 confirmed control communities received the treatment ahead of time. The ARAS and AVAR oversaw the experimental design, and helped prevent additional contamination. Moreover, since the capacity-building occurred at the level of the UMAS but the implementation was administered at the CAPS level additional contamination may have occurred beyond the thirty-eight communities reported. 11 They were supported by other FISE staff: AVAR specialists, municipality advisors (AMU), social specialists, and environment specialists. 33

Implementation of AVAR Workshops

AVAR training eventually took place across all municipalities in Nicaragua, with departments and autonomous regions organized into nine groups. Each municipality was required to send staff for the training. Figure 4 shows the departments and regions that comprised each group, sorted by the date in which the first workshop began. The poorest departments and regions got priority in the sequence. The first group was the autonomous regions on the Caribbean coast and the last were less poor departments in the Pacific region. This rollout by poverty status had no implication in the experimental design because assignment to treatment was stratified at the municipality level—each municipality, regardless of its group, had an equal number of treatment and control communities. Annex VI shows AVAR training dates for all municipalities in the experimental sample.

WORKSHOP DEPARTMENTS AND GROUPS AUTONOMOUS REGIONS [1] ATLÁNTICO SUR [2] ATLÁNTICO NORTE BOACO [3] CHONTALES RÍO SAN JUAN [4] JINOTEGA MATAGALPA [5] ESTELI NUEVA SEGOVIA [6] MADRIZ MANAGUA [7] 18.9 - 28.0 MASAYA CARAZO 28.1 - 41.1 [8] GRANADA 41.2 - 54.8 RIVAS 54.9 - 70.9 CHINANDEGA [9] LEÓN Figure 4. Training Workshops by Region and by Delivery Sequence Shaded areas in the map denote extreme poverty levels estimated by the National Statistics Institute (INIDE, in Spanish). Red areas denote high poverty; pink, medium-high; light blue, medium-low; and blue, low.

Identification strategy Sample and Assignment to Treatment

The experimental sample consisted of 300 communities—150 assigned to the treatment group and 150 to the control group. It covered 76 of the 153 municipalities in Nicaragua. Although the intervention targeted municipality-level staff, the experimental design and results analysis were 36 conducted at the community level. For logistical reasons, UMAS and FISE were in the best position to evaluate program effects at the community level. Using communities instead of municipalities proved useful in terms of statistical ability to detect effects.

Figure 5 details the process by which the 300 communities12 were selected from all rural communities in the country. To determine whether communities were eligible to be part of the sample, SIASAR information was used to estimate indexes based on existence and state of (1) water distribution systems, (2) service providers, and (3) access to water and sanitation in the community. These indexes were used to restrict the sample to communities that had minimal- level infrastructure, so that they could benefit well from the program; but which did not have the highest level of infrastructure, so that benefits could be measured. The next step was to exclude communities that were the subject of other interventions or were too small or big by number of households. After that, the sample was further restricted to municipalities that had at least four communities.13

These steps yielded a sample consisting of 100 municipalities and 1,600 communities. The final step was to randomly select 75 municipalities, each one represents a stratum. Within each of the 75 stratums, between two and four communities were randomly selected, with an equal number allocated to treatment and control groups. This yielded the study’s final line-up of 150 treatment communities and 150 control communities.

12 On the most part, CAPS are meant to supervise one community and one water system. Exceptions to this sometimes occur, especially among smaller communities, where one CAPS may have to supervise multiple communities. This may increase spillovers at the level of the community, potentially introducing bias to our household-level indicators. To the best of our knowledge, in the sample of communities in our research design, CAPS in charge of managing more than one community were few or nonexistent. But the threat of spillovers due to this, although minor, remains a source of concern. 13 The selection of rural communities was not conducted randomly. Instead, it was based on those communities which had a minimum level of access to water, sanitation, and infrastructure, and were thus deemed most likely to benefit from the program. The non-random selection of communities raises some challenges to the study’s external validity. Thus, the authors of this study thus urge caution before universalizing the findings presented here, or applying the findings to different contexts. 3,698 COMMUNITIES IN 149 MUNICIPALITIES WITH AT LEAST ONE SYSTEM

2,599 COMMUNITIES IN 141 MUNICIPALITIES WITH SYSTEMS WITH INFRASTRUCTURE INDEX 37 SCORE IN SIASAR GREATER THAN 0.40 AND WITH SUSTAINABILITY INDEX SCORE EQUAL OR LESS THAN 0.80

1,851 COMMUNITIES IN 132 MUNICIPALITIES NOT SHARING SYSTEMS OR CAPS WITH OTHER COMMUNITIES

1,792 COMMUNITIES IN 130 MUNICIPALITIES WITHOUT OTHER DEVELOPMENT INTERVENTIONS FROM THE CENTRAL AMERICAN BANK FOR ECONOMIC INTEGRATION (CABEI)

1,674 COMMUNITIES IN 130 MUNICIPALITIES WITH A TOTAL NUMBER OF HOUSEHOLDS GREATER THAN 20 AND LOWER THAN 1,000

102 MUNICIPALITIES WITH FOUR COMMUNITIES OR MORE

RANDOM SELECTION BY REGION OR DEPARTMENT OF MUNICIPALITIES TO COVER ALL DEPARTMENT AND AUTONOMOUS REGIONS IN NICARAGUA 75 MUNICIPALITIES

STRATIFIED AND RANDOM SELECTION: TWO OR FOUR COMMUNITIES ARE SELECTED BY MUNICPALITY, HALF ASSIGNED TO TREATMENT AND HALF TO CONTROL 300 COMMUNITIES, T=150, C=150

Figure 5. Selection Process of Communities in the Evaluation Sample

During baseline fieldwork, owing to logistical issues and interviewer safety, the evaluation team had to make minor adjustments to the community list. The team replaced (1) one municipality (four communities), (2) two communities from one municipality with two from another, and (3) one community for another. For this reason, the final sample after baseline work had 76 municipalities instead of the 75 planned. These replacements and substitutions used randomization to ensure that the initial experimental design was preserved. Data

Evaluation Surveys

Baseline fieldwork started in November 2015 and ended in January 2016. 38 Data collection consisted of surveys of households, community leaders, water system operators, and CAPS and UMAS personnel. These surveys assessed function and resilience of services, institutional capabilities of CAPS, and access to water and sanitation services in households in the communities.14The final baseline sample was 4,850 household surveys in 299 communities, 77 UMAS, and 197 CAPS.

Piloting and interviewer training for endline work took place between March and May 2019. Endline field work started in March 2019 and ended in October 2019. The rollout of the survey was significantly delayed due to issues importing water tests into the country, and because heavy rainfall affected the capacity of enumerators to reach some of the more remote areas of the country. The final endline sample was 4,527 household surveys15 in 299 communities, 69 UMAS, and 226 CAPS. Both baseline and endline surveys gathered water quality information in households and systems. They include E.coli presence, and pH and residual chlorine values.16

Figure 6 overlaps baseline and endline fieldwork periods to AVAR training periods by group. FISE and the evaluation team coordinated their activities to ensure that the intervention started after baseline fieldwork and ended before endline fieldwork. The gap between the third AVAR workshop and data collection was six to 18 months, depending on the community.

Data from these surveys were used to evaluate environmental conditions and hygiene because, unlike SIASAR data, all information was collected after the intervention ended. The evaluation surveys, however, missed some indicators required to evaluate CAPS and UMAS. In those cases, SIASAR data was used instead to estimate overall and category scores for CAPS and UMAS.

14 See Christian Borja-Vega, Joshua Gruber, and Alexander Spenack. December 2017. “Increasing the Sustainability of Rural Water Service: Findings from the Impact Evaluation Baseline Survey in Nicaragua.” 15 Return visits were required to complete planned fieldwork. They took place in November 2019 in 11 communities and allowed the recovery of 157 surveys. 16 As of the time of the writing of this study, the information on water-quality gathered as part of the impact evaluation’s end-line survey had not been fully processed. The authors hope to integrate the results of the water-quality exercise either as part of a future iteration of this report, or as part of a separate, standalone paper. SIASAR Data

Data from the 2012/13 and the 2017/19 national data collection efforts were used to estimate UMAS and CAPS scores according to the matrixes of indicators described in Annex II. Figure 7 overlaps both data collection periods in communities in the experimental sample to AVAR training periods by group. About 60 percent of the CAPS information in the 2017/19 wave was collected before 39 the third round of workshops ended. This could bias the results because part of measurement took place before the intervention could have effects. The results presented here probably have a downward bias—they are lower than they would be had all measurement occurred after the intervention ended. The same problem applies, albeit to a lower extent, to UMAS. For UMAS, 25 percent of the data was collected before the third round of workshops ended.

SURVEYS wks 2013 2015 2016 2017 2018 2019 Order 2014 Ja/Se No/Di Ja Ma JunJSe Di Ja Ma un Se Di Ja Ma JunJSe Di Ja Ma un Se Di [1] [2] [3] [4] [5] [6] [7] [8] [9] Figure 6. Implementation of Evaluation Surveys in Relation to AVAR Training Workshops Notes: Green shades denote baseline surveys implementation and blue show endline surveys. Horizontal bars denote training dates per departmental groups. The start of the bar marks the date of the first training; the line in the middle of the bar the date of the second training, and the end of the bar, the date of the third.

CAPS wks 2013 2015 2016 2017 2018 2019 Order 2014 Ja/Se No/Di Ja Ma JunJSe Di Ja Ma un Se Di Ja Ma JunJSe Di Ja Ma un Se Di [1] [2] [3] [4] [5] [6] [7] [8] [9] Figure 7. SIASAR Information on CAPS in Relation to AVAR Training Workshops Notes: Green shades denote SIASAR wave 2012/13 and blue the wave of 2017/19. Horizontal bars denote workshop dates per departmental groups. The start of the bar marks the date of the first workshop; the line in the middle of the bar the date of the second workshop, and the end of the bar, the date of the third. Design Preservation and Balance Tests Workshop activities followed the evaluation design, but of the 150 communities in the control group, 38 (26 percent) received the intervention ahead of schedule. According to FISE, emergency situations prompted FISE or ARAS to intervene in these communities. Table 11 describes the final 40 evaluation sample. Of the 300 communities, 298 are present in both baseline and endline. The table also details “contamination”—control communities that received the intervention early, negating their status as control group members. Contaminated communities were similarly distributed across groups.

Evaluation sample Contaminated Municipalities in AVAR Communities Mun controls Selected Interviewed Order Mun Evaluation T C T C n %

[1] 12 6 11 11 11 11 2 18%

[2] 8 1 2 2 2 2 0 0%

[3] 21 11 22 22 22 22 8 36%

[4] 8 7 14 14 14 14 9 64%

[5] 19 15 30 30 30 29 4 14%

[6] 20 15 29 29 29 29 6 21%

[7] 18 4 8 8 8 8 1 13%

[8] 22 5 10 10 9 10 2 20%

[9] 23 12 24 24 24 24 6 25%

All 151 76 150 150 149 149 38 26%

Table 11. Final Sample for Evaluation, with Contaminated Controls as Percentage of Interviewed Communities

Notes: Community Laguna #2, a treatment community, had no complete surveys on baseline. Households in control community Las Limas San Miguel could not interviewed for the endline. Both communities were dropped from the analysis. Finally, another type of contamination occurred in the legalization indicator, one of several used to evaluate CAPS. Owing to a nationwide mandate to legalize CAPS, neither workshop activities nor ARAS support followed plans to delay the legalization intervention in control communities. For this reason, the CAPS score presented here excludes the legalization indicator. As shown in the results presented in Table 14, this indicator is reported separately. 41 Randomization was effective. Balance tests showed that treatment and control groups were no different in observable characteristics. They also demonstrated that random emergency reasons, and no other reasons, caused treatment to take place ahead of schedule in the 38 contaminated control communities. Table 12 provides evidence of no systematic difference in socio-demographic characteristics between treatment and control groups. Annex III presents equivalent tables for outcomes and other characteristics and further supports this conclusion. Table 13 provides evidence that there was no systematic difference in socio-demographic characteristics between contaminated and non-contaminated controls. Tables in Annex II also support this conclusion.

In the comparison between contaminated and non-contaminated controls, the only difference is that contaminated ones were more likely to have CAPS. This supports the conclusion that emergency situations drove intervention. Communities without CAPS were unlikely to have a channel through which to ask FISE for help. All other indicators showed no systematic differences.

Treatment Control Difference Indicator N Mean Std.Dev N Mean Std.Dev Dif. p-value

Average HH size 2,383 4.67 2.13 2,454 4.72 2.20 -0.06 0.45

HH members, 5 and under 2,382 0.58 0.80 2,454 0.62 0.84 -0.04 0.17

HH members, 14-30 2,382 1.58 1.29 2,454 1.58 1.35 0.00 0.98

HH members 65+ 2,382 0.25 0.56 2,449 0.25 0.59 0.00 0.84

Head knows how to read 2,383 0.70 0.46 2,454 0.70 0.46 0.00 0.92

Reports active employment 2,383 0.82 0.38 2,454 0.79 0.41 0.03 0.12

Wealth index 2,383 -0.05 1.95 2,454 0.06 2.00 -0.10 0.55

Poorest quintile 2,383 0.22 0.42 2,454 0.22 0.41 0.00 0.92

Third quintile 2,383 0.19 0.40 2,454 0.17 0.38 0.02 0.15

Richest quintile 2,383 0.19 0.39 2,454 0.21 0.41 -0.03 0.33

Table 12. Socio-demographic Characteristics by Treatment and Control Groups 42

Contaminated Non-contaminated Difference Indicator N Mean Std.Dev N Mean Std.Dev Dif. p-value

Average HH size 649 4.88 2.13 1,805 4.67 2.23 0.21 0.06

HH members, age 649 0.60 0.77 1,805 0.63 0.87 -0.03 0.50 5 and under

HH members, age 14-30 649 1.63 1.40 1,805 1.56 1.34 0.07 0.27

HH members age 65+ 646 0.26 0.58 1,803 0.24 0.59 0.02 0.61

Head knows how to read 649 0.68 0.47 1,805 0.71 0.46 -0.02 0.45

Reports active 649 0.80 0.40 1,805 0.79 0.41 0.02 0.60 employment

Wealth index 649 0.15 1.99 1,805 0.02 2.00 0.13 0.61

Poorest quintile 649 0.19 0.39 1,805 0.23 0.42 -0.04 0.30

Third quintile 649 0.17 0.38 1,805 0.17 0.38 0.00 0.86

Richest quintile 649 0.23 0.42 1,805 0.21 0.41 0.02 0.71

Table 13. Socio-Demographic Characteristics by Contaminated and Non-Contaminated Controls Estimating Equations

Two similar specifications were used in the analysis. The first equation was for analysis of CAPS indicators using SIASAR data17. Of the 300 communities taking part in the study, only 218 (73 percent) had CAPS on baseline (2012/13). In endline (2017/19), only 158 communities (72 percent) reported information. 43 The sample was restricted to those 158 communities18. The equation used endline data and had as dependent variable y for CAPS a in community c. Controls were the following: a dummy variable Tc for communities in the treatment group, a variable with the mean of the outcome for CAPS at baseline in the community (most communities have only one CAPS), and municipality fixed effects. The stratification was at the municipality level (76 strata). Simulations by Bruhn and McKenzie (2008) show that regressions that include the variables used for stratification have more power to detect effects relative to regressions without them. Standard errors were clustered at the municipality level.

Baseline (1) ϒac=β0+β1Τc+β2ȳc +μm+εac

The second equation was used for household level indicators. It considered endline data from all households h in community c. It had the same structure as equation (1) and used equivalent controls. Standard errors were also clustered at the municipality level.

Baseline (2) ϒhc=β0+β1Τc+β2ȳc +μm+εhc

Including outcome baseline means pursues power increases to detect effects when they exist. If the outcome systematically increases or decreases across time, the baseline outcome will have a high predictive effect on its endline values. This will result in a lower regression error and in more precise estimates. Because we have showed that treatment and control variables were statistically similar in all variables at baseline, the inclusion of the baseline mean had no effect on the point estimate of the treatment effect.

17 SIASAR, instead of the CAPS module from the endline survey, was used to validate these outcomes because of: a) budget constraints, and b) because the community data gathered from SIASAR offers a more reliable, and bullet-proof, estimate of community-level estimates than does the data collected via our household surveys. Once the last wave of SIASAR becomes available, the researchers of this study will have access from additional communities in the evaluation sample, and will have full estimates, collected after the end of the trainings. This will significantly increase the sample size, and the robustness of the estimates presented. 18 To ensure comparability, this analysis focused on those 158 CAPS included at both baseline and endline. But it should be noted that part of what the institutional strengthening component of PROSASR is aiming to achieve is to increase the presence of CAPS throughout the country, to make them ubiquitous. Additional work, informed by the new, incoming waves of SIASAR data, could aim to study how successful PROSASR has been in increasing the presence of CAPS throughout the country. RESULTS

Summary of Results

Results demonstrate that the intervention was effective in improving the 44 institutional and management capabilities of CAPS. According to the overall CAPS score, measured on a scale from 1 to 4, the intervention caused an increase from 2.66 to 2.99 points (+0.30 points). Three of the five components of the overall CAPS score improved: (1) Formal operation (+0.36 points); (3) Financial stability (+0.42 points) and (4) System operation and maintenance (+0.30 points). Two components—(2) Adequate tariff for water supplied and (5) Water source protection— showed no change. For water source protection, owing to comparability reasons between baseline and endline, only one indicator was used, which could explain lack of results. The program seems to have had no effect on adequate tariffs for water supplied. Figure 8 shows small or no improvements for both treatment and control groups between baseline and endline and no trend difference between groups. Legalization, for reasons explained before, was analyzed under a separate regression indicator. Results show that the program had no statistically significant additional effect to the national legalization mandate.

The intervention had positive effects in the overall CAPS score and on three of its five components. The overall CAPS score was 2.99 in the treatment group vs. 2.69 in the control group (0.30 points, 11%, or 0.42 standard deviations).

The three components that improved are: [1] Formal operation: +0.36 points (13% or 0.41 standard deviations) [3] Financial stability: +0.42 points (18% or 0.39 standard deviations) [4] System operation and maintenance: +0.30 points (11% or 0.33 standard deviations)

Two components showed no statistically significant effects: [2] Adequate tariff for water supplied: +0.08 (3% or 0.08 standard deviations) [5] Water source protection: +0.32 (10% or 0.24 standard deviations)

Excluding contaminated controls results in an increase of the effects found. Overall CAPS score was 2.91 in the treatment group vs. 2.54 in the control group (0.38 points, 15%, or 0.54 standard deviations). Effects increased on all components but remained not statistically significant in components (2) and (5). Positive short-term impacts at the household level were concentrated in Household-level effects are concentrated on sanitation. Annex V shows that safely sanitation indicators and diarrhea reductions. managed water access increased only No other statistically significant effects were marginally, for treatment and control found in access to water and in other clean group alike. This suggests that CAPS 45 environment or hygiene indicators. improvements take longer- time to materialize, and reflect on downward Indicators with statistically significant outcomes at the household level. improvements were: Longer-term, household level outcomes, such as changes in water Open defecation: Decrease of 1.8 percent points and sanitation, take additional time (T=3.0%, C=4.8%; equivalent to a decrease of to materialize. -37%) Diarrhea: Decrease of 2.2 percent points (T=12.0%, Regarding sanitation outcomes, C=14.2%; equivalent to a decrease of -16%) impacts were concentrated in a Sanitation: Increase of 3.7 percent points decrease of open defecation, higher (T=50.2% , C=46.5%; equivalent to an increase of use of non-shared sanitation services, 8%) and increase of improved sanitation Non-shared sanitation facilities: Increase of 3.2 services. The intervention plausibly percent points (T=81.4% , C=78.2%; equivalent to had positive effects on sanitation an increase of 4%) because training emphasized safe environment and hygiene as a Excluding contaminated controls results, in some third component to improve. In a cases, in larger effects but the sample decrease preliminary analysis of all the actions also results in a loss of power; for example, plans, open defecation stood out as the effect found in diarrhea remains of similar a particular concern. The positive magnitude but no longer statistically significant. effects found in sanitation may also explain a statistically significant reduction in diarrhea. b. Excludes a. All sample contaminated controls Treatment Treatment Control effect Control effect Mean mean Effect p-value Effect p-value

46 CAPS

Overall score (Components 1 to 5) 2.69 0.30 0.005 2.54 0.38 0.002

1. Formal operation 2.72 0.36 0.003 2.52 0.48 0.001

2. Adequate tariffs for water supplied 2.54 0.08 0.657 2.39 0.18 0.360

3. Financial stability 2.29 0.42 0.006 2.15 0.47 0.003

4. Adequate operation and maintenance 2.82 0.30 0.060 2.62 0.50 0.012

5. Adequate protection of water source 3.08 0.32 0.132 3.02 0.28 0.287

Other: Legalization 0.65 0.10 0.183 0.62 0.10 0.244

Households: Clean environment and hygiene

Open defecation 0.05 -0.02 0.018 0.05 -0.02 0.050

Unsafe trash disposal 0.75 0.00 0.855 0.73 -0.01 0.770

Feces or trash in yard 0.45 0.00 0.985 0.47 -0.01 0.714

Handwashing 0.84 0.00 0.790 0.84 0.00 0.837

Latrine use 0.98 0.00 0.727 0.98 0.00 0.761

Safe water storage 0.55 0.01 0.763 0.53 0.01 0.513

Households: Water and sanitation

Safely managed water: Dry season 0.26 0.03 0.204 0.26 0.03 0.213

Safely managed water: Wet season 0.38 0.02 0.534 0.37 0.01 0.604

Improved sanitation 0.46 0.04 0.076 0.44 0.07 0.003

Non-shared sanitation service 0.78 0.03 0.013 0.78 0.04 0.011

Diarrhea in the household (last 7 days) 14.2% -2.2% 0.044 13.7% -1.6% 0.168

Table 14. Summary of Results Descriptive Results at the UMAS Level

Because the program intervened in communities in all of Nicaragua’s municipalities, it is not possible to measure causal impacts at the UMA level. It is possible, however, to measure the change on indicators between baseline and endline. Part of the change can be attributed to the program. Table 15 compares indicators at the UMAS level before and after the intervention. 47 The overall UMAS score increased from 2.0 to 3.0, an improvement of 1.0 point (50 percent or 1.3 standard deviations). All components with the exception of the number of technicians improved. It is worth noting that more personnel are needed to ensure proper maintenance and operation of water distribution systems. The two components that improved the most were (1) having equipment and (2) having an annual budget officially assigned, funds for travel and fuel, and Internet service.

Mean score Difference UMAS Baseline Endline In score In % In Std. Dev. p-value 2012-2013 2017-2019

UMAS Score 2.0 3.0 1.0 50% 1.3 0.00

By component

1. Share of communities visited during the last 1.5 2.3 0.8 52% 0.7 0.00 12 months

2. Share of communities supported for water 1.2 1.9 0.7 63% 0.7 0.00 quality monitoring

3. Human resources: Ratio of communities to 3.2 3.2 0.0 1% 0.0 0.93 technicians

4. Transportation capacity: Ratio of vehicles to 2.1 3.2 1.2 58% 0.9 0.00 technicians

5. Equipment: a. Water quality monitoring 1.8 3.3 1.5 87% 1.4 0.00 b. Computer c. Informative material printed

6. Has: a. Assigned annual budget 1.4 3.7 2.2 156% 1.7 0.00 b. Funds for travel expenses and fuel c. Internet service

Table 15. UMAS Before and After AVAR Training, Using SIASAR Data

Note: Only 57 of the 76 municipalities in the evaluation sample had data on SIASAR 2017/19 added after the end of the third workshop.

Annex IV presents statistics for UMAS from the evaluation surveys. With few exceptions, results were positive: the number of UMAS activities that support CAPS increased, the number of UMAS having training needs decreased, and the number of UMAS declaring short- and long-term needs fell. Concerning activities that UMAS carry out to support CAPS, Figure A.4.1 shows that the areas in which this support increased the most were revision and update of finances and revision of operation regulations. Moreover, the proportion of communities supported by UMAS increased from 35 to 60 percent. 48 Short- and long-term needs decreased (Figure A.4.2). Concerning short-term needs, unsatisfied technical capabilities decreased from 53 to 8 percent and the need to improve staff skills decreased from 71 to 19 percent. In spite of the improvements, the gap between actual and required skills and technical capabilities to adequately operate water systems remained large. Short- and long-term financial needs, although they decreased, remained the most important UMAS need. For example, even though the proportion of UMAS with assigned budgets increased from 47 to 72 percent, less than 20 percent of all UMAS declared that they had sufficient resources to fulfill their duties.

Training needs from FISE decreased (Figure A.4.3). Needs related to supporting CAPS that decreased the most were water quality analysis (72 percent to 33 percent of UMAS), which reflected the learning of skills at workshops. UMAS also declared themselves to have fewer training needs related to operation and maintenance of systems, tariff collection, and support to protect water sources. AVAR training covered all these topics. The most important training needs remaining after AVAR were how to provide technical support to CAPS (29 percent) and how to handle UMAS finances (25 percent).

UMAS also provided feedback on the work of ARAS and FISE (Figure IV.4). Concerning FISE, most UMAS (72 percent) wanted greater financial support. About half wanted more training, technical support, and equipment, while between 30 and 40 percent wanted more independence for UMAS, and institutional and logistical support. Only 25 percent stated that they wanted more visits from FISE. According to the feedback, UMAS have close relations with ARAS. Eighty-one percent of UMAS declared that they receive technical support from them, 72 percent supported CAPS legalization, and 66 percent favored training related to SIASAR. But in spite of the close relations, only 18 percent declared that ARAS need no improvement. Areas in which improvements were desired were financial, technical support, and support to CAPS. 49

Causal Results at the CAPS Level

Figure 8 shows trends between baseline and endline for the overall CAPS score and its five components. The figure buttresses findings in Annex III: at baseline no systematic differences existed between the treatment and control groups, in particular when contaminated controls were excluded. At endline, the treatment group had improved in all components while control group improvements were minor or non-existent.

Table 16 presents regression results for the overall CAPS score and for the legalization indicator. Column 1 shows results from Equation (1)19 without the baseline mean of all CAPS per community. Column 2 adds the variable, the point estimate being no different compared to Column 1. In Column 3, the contaminated controls are dropped. This is valid in terms of the experimental design. We found no systematic differences between contaminated and non-contaminated controls. Compared with Column 2, the effect increased from 0.297 to 0.382. Because contaminated controls received at least part of the treatment, the sample that includes them tends to dilute the intervention effects. Columns 4 and 5 used Equation (1) and whether the CAPS had been legalized as a dependent variable. Although point estimates were positive, the program had no statistically significant effects in addition to the national mandate to legalize CAPS.

19 See page 45. A. All Sample Overall CAPS score 1. Formal operation 2. Adequate tariffs for water supplied 4.0 4.0 4.0

50 3.0 3.0 3.0 3.1 3.0 2.7 2.7 2.6 2.5 2.0 2.0 2.0

1.0 1.0 1.0 Baseline Endline Baseline Endline Baseline Endline 3. Financial stability 4. Adequate operation and 5. Adequate protection maintenance of water source 4.0 4.0 4.0 3.4 3.0 3.0 3.1 3.0 3.1 2.8 2.7 2.3 2.0 2.0 2.0

1.0 1.0 1.0 Baseline Endline Baseline Endline Baseline Endline B. Excluding contaminated control communities Overall CAPS score 1. Formal operation 2. Adequate tariffs for water supplied 4.0 4.0 4.0

3.0 3.0 3.0 3.1 3.0 2.6 2.5 2.5 2.4 2.0 2.0 2.0

2.0 2.1 2.0 2.0

1.0 1.0 1.0 Baseline Endline Baseline Endline Baseline Endline T C

Figure 8. Comparison of CAPS Score and Its Components between Treatment and Control Groups a. CAPS score b. Share legalized

(1) (2) (3) (4) (5)

0.302*** 0.297*** 0.382*** 0.096 0.100 Treatment=1 51 [0.103] [0.104] [0.120] [0.071] [0.085]

0.107 0.036 0.238** 0.133 Baseline outcome (0.098) (0.106) (0.107) (0.133)

2.683*** 2.423*** 2.485*** 0.568*** 0.594*** Constant [0.074] [0.243] [0.274] [0.063] [0.069]

Obs. (CAPS) 183 183 159 183 159

Communities 158 158 138 158 138

Adj. R2 0.306 0.307 0.370 0.201 0.181

Municipality FE Yes Yes Yes Yes Yes

Exclude contaminated No No Yes No Yes

Control mean 2.7 2.7 2.5 65% 62%

% inc. over control 11% 11% 15% 15% 16%

Notes: Robust standard errors clustered at the municipality level in parentheses *** p<0.01, ** p<0.05, * p<0.1 FE: Fixed Effects

Table 16. Effect on CAPS Score—Regression Results

Figure 9 shows point estimates and their intervals using Equation (1) on standardized indicators for the overall CAPS score and its five components. The top panel is equivalent to Column 2 on Table 16 and the bottom one to Column 3. It shows that three of the five components improved: (1) Formal operation (0.41 standard deviations), (3) Financial stability (0.39), and (4) System operation and maintenance (0.33). These effects increased when contaminated controls were excluded from the sample. Two components showed no statistically significant effects: (2) Adequate tariff for water supplied (0.08) and (5) Water source protection (0.24). Excluding contaminated results leads to slightly higher but still non-statistically significant results. BENCHMARK EQ. (2) 0.42 CAPS Score 0.41 52 1. Formal operation 0.08 2. Adequate tariffs for water supplied 0.39 3. Financial stability 0.33 4. Adequate operation and maintenance 0.24 5. Adequate protection of water source

EXC. CONT. EQ. (3) 0.54 CAPS Score 0.55 1. Formal operation 0.18 2. Adequate tariffs for water supplied 0.45 3. Financial stability 0.53 4. Adequate operation and maintenance 0.22 5. Adequate protection of water source

-1 -0.5 00.5 1 TREAT. - CONT. STD. DEV.

Figure 9. Effect on CAPS Score per Component. Regression Results. Coefficients of Treatment Variables Causal Results at the Household Level

Clean Environment and Hygiene

Table 17 presents regression results for clean environment and hygiene indicators using the whole sample, while regressions in Table 18 exclude 53 contaminated controls. (Annex V shows trends between baseline and endline for these indicators.) Positive and significant effects are found only for open defecation. It decreased 1.8 percent points (equivalent to a decrease of 37 percent over the control group). A similar but less precise effect was found when contaminated controls were excluded. This could be the result of a lower power induced by a smaller sample size.

a. Clean environment b. Hygiene

OD TRASH YARD HANDW LATRINE SAF.STO

-0.018** -0.003 0.000 0.003 0.002 0.006 Treatment=1 [0.008] [0.016] [0.016] [0.012] [0.006] [0.019]

0.184*** 0.663*** 0.176*** 0.101** 0.118* 0.469*** Baseline outcome [0.052] [0.075] [0.055] [0.041] [0.070] [0.049]

0.031*** 0.197*** 0.370*** 0.774*** 0.866*** 0.413*** Constant [0.006] [0.062] [0.027] [0.028] [0.068] [0.019]

Obs. (Households) 4,513 4,513 4,513 4,146 3,489 4,513

Communities 298 298 298 298 297 298

Adj. R2 0.04 0.18 0.04 0.06 0.02 0.06

Municipality FE Yes Yes Yes Yes Yes Yes

Exclude contaminated No No No No No No

Control mean 4.8% 75.1% 45.4% 83.8% 98.0% 54.8%

% inc. over control -37% 0% 0% 0% 0% 1%

Notes: Robust standard errors clustered at the municipality level are shown in brackets� *** p<0.01, ** p<0.05, * p<0.1 FE: Fixed Effects

Percentage of population that: OD: Practice open defecation TRASH: Burns trash, throws it anywhere, or into bodies of water YARD: Live in dwellings with feces or trash in the yard HANDW: Has handwashing station with water and soap that is used daily by all household members LATRINE: Always use latrine (only for households with access to latrines) SAF.STO: Stores drinking water in small lid containers into which hands cannot enter or gets water from tap or bottled water Table 17. Effect on Clean Environment and Hygiene—Regression Results 54

a. Clean environment b. Hygiene

OD TRASH YARD HANDW LATRINE SAF.STO

-0.015* -0.005 -0.007 -0.003 -0.002 0.013 Treatment=1 [0.008] [0.017] [0.018] [0.014] [0.006] [0.020]

0.161*** 0.777*** 0.172*** 0.078* 0.151** 0.452*** Baseline outcome [0.054] [0.063] [0.059] [0.043] [0.071] [0.050]

0.031*** 0.095* 0.381*** 0.791*** 0.838*** 0.407*** Constant [0.006] [0.051] [0.030] [0.029] [0.069] [0.021]

Obs. (Households) 3,865 3,865 3,865 3,552 3,002 3,865

Communities 260 260 260 260 260 260

Adj. R2 0.04 0.21 0.04 0.06 0.01 0.06

Municipality FE Yes Yes Yes Yes Yes Yes

Exclude contaminated Yes Yes Yes Yes Yes Yes

Control mean 4.7% 72.6% 46.8% 83.7% 98.4% 53.5%

% inc. over control -32% -1% -1% 0% 0% 2%

Notes: Robust standard errors clustered at the municipality level are shown in brackets� *** p<0.01, ** p<0.05, * p<0.1 FE: Fixed Effects

Percentage of population that: OD: Practices open defecation TRASH: Burns trash, throws it anywhere, or into bodies of water YARD: Lives in dwellings with feces or trash in the yard HANDW: Has handwashing station with water and soap that is used daily by all household members LATRINE: Always use latrine (only for households with access to latrines) SAF.STO: Stores drinking water in small lid containers into which hands cannot enter or gets water from tap or bottled water Table 18. Effect on Clean Environment and Hygiene—Regression Results Excluding Contaminated Controls Safely Managed Water Access, Sanitation, and Diarrhea

Table 19 presents regression results for safely managed water access. The results, although positive, were not statistically significant. Point estimates were higher in the dry season. Annex V shows trends between baseline and endline for these indicators. These trends showed improvement for both 55 treatment and control groups. Improvements in the treatment group were slightly higher, consistent with positive but not statistically significant effects.

b. Excluded contaminated a. All sample controls

DRY WET DRY WET

0.030 0.016 0.030 0.014 Treatment=1 [0.024] [0.026] [0.024] [0.027]

0.566*** 0.541*** 0.551*** 0.521*** Baseline outcome [0.052] [0.050] [0.058] [0.057]

0.160*** 0.243*** 0.160*** 0.246*** Constant [0.017] [0.020] [0.019] [0.023]

Obs. (Households) 4,510 4,510 3,862 3,862

Communities 298 298 260 260

Adj. R2 0.17 0.18 0.18 0.20

Municipality FE Yes Yes Yes Yes

Excludes contaminated No No Yes Yes

Control mean 26.3% 37.6% 26.2% 37.3%

% inc. over control 11% 4% 11% 4%

Notes: Robust standard errors clustered at the municipality level in parentheses. *** p<0.01, ** p<0.05, * p<0.1 FE: Fixed Effects DRY: Dry season WET: Rainy season

Table 19. Effect on Safely Managed Water Access—Regression Results The intervention, however, did have effects on sanitation and diarrhea. Use of improved sanitation services increased 3.7 percentage points (equivalent to an increase of 8 percent over the control group), use of non-shared sanitation services increased 3.2 points (4 percent), and diarrhea decreased 2.2 points (-16 percent).20 Higher or similar effects resulted when contaminated controls 56 were excluded. But the power loss induced by a smaller sample size led to point estimates of similar magnitude for diarrhea that were no longer statistically significant.

b. Excludes contaminated a. All sample controls

SAN PRIV DIA SAN PRIV DIA

0.037* 0.032** -0.022** 0.068*** 0.035** -0.016 Treatment=1 [0.021] [0.013] (0.011) [0.023] [0.014] (0.012)

0.343*** 0.344*** 0.030 0.325*** 0.317*** 0.022 Baseline outcome [0.048] [0.053] (0.081) [0.050] [0.056] (0.090)

0.332*** 0.495*** 0.138*** 0.313*** 0.514*** 0.133*** Constant [0.022] [0.046] (0.011) [0.023] [0.049] (0.012)

Obs. (Households) 4,513 4,513 4,513 3,865 3,865 3,865

Communities 298 298 298 260 260 260

Adj. R2 0.11 0.05 0.0212 0.11 0.05 0.0181

Municipality FE Yes Yes Yes Yes Yes Yes

Excludes contaminated No No No Yes Yes Yes

Control mean 46.5% 78.2% 14.2% 44.3% 77.6% 13.7%

% inc. over control 8% 4% -16% 15% 5% -12%

Notes: Robust standard errors clustered at the municipality level in brackets. *** p<0.01, ** p<0.05, * p<0.1 FE: Fixed Effects SAN: Access to improved sanitation PSA: Has private sanitation facility DIA: Any member experienced diarrhea (last 7 days)

Table 20. Effect on Access to Improved Sanitation and Diarrhea—Regression Results

20 The increase in improved sanitation is not driven by the increase in use of non-share sanitation facilities. In the regressions presented, improved sanitation is defined as access to protected latrines or to toilets connected to sewage, whether the facility is shared or not. This because whether the facility is shared or not receives a separate regression analysis. When the improved sanitation indicator adds as requirement that the facility is not shared with other households, regression analysis shows a non-statistically significant increase of 2.6 percentage points (p.value 0.172) , equivalent to an increase of 7 percent over the control group. The same regression but excluding contaminated controls shows a statistically significant increase of 5.1 percentage points (p.value 0.015), equivalent to an increase of 14 percent over the control group. CONCLUSIONS AND LIMITATIONS

Results presented here suggest that Component 1 of the Sustainability Project of the Rural Water and Sanitation Sector (PROSASR) increased the institutional and management capabilities of water and sanitation committees (CAPS), the local institutions in charge of providing water and sanitation services in 57 rural areas.

The positive effects of the intervention, however, have yet to materialize into longer-term outcomes, such as access to improved water. The indicator on access to safely managed water access in rural areas showed improvements in both the treatment and the control groups, with slightly higher but not statistically significant effects in treatment. The intervention did show a positive effect on access to rural sanitation: although this is only a secondary component of the institutional strengthening component of PROSASR, the results are encouraging, and additional work is required to further flesh-out the precise nature of the factors driving the increases in sanitation.

The main limitation of this study concerned measurement. About 60 percent of CAPS information from SIASAR in the 2017/19 wave was collected before AVAR training ended. This likely resulted in an underestimation of treatment effects. Effects on CAPS could have been higher had all measurement occurred once the intervention concluded. Moreover, because measurement followed shortly after the intervention ended, these effects are all short term. Additional time is needed for longer-term results at the household-level to materialize.

In regards to households, about six to eighteen months passed between when AVAR training ended and endline. Since the AVAR training finished recently, the effects of the intervention, especially at the household-level, may have not fully materialized. The study’s authors recommend a follow-up questionnaire, in one or two-years time, to detect longer-term impacts, but particularly at the community or household level, where the effects of the intervention take longer to filter-down.

This impact evaluation study is the result of close cooperation between the Government of Nicaragua, through FISE, and the World Bank. Its main takeaway is that training and institutional support can improve management capabilities of local providers of water and sanitation services in rural areas. Next steps are to find out why the intervention had positive short-term effects in sanitation but not in safely managed water access. If an additional data collection effort takes place, a follow-up study must explore the long- term positive effects of the intervention.

Bibliography

Andres, L., et al. 2018. “Overview and meta-analysis of global water, sanitation, and hygiene (WASH) impact evaluations. Washington, DC: World Bank. https://doi. org/10.1596/1813-9450-8444 59 Bruhn, M., and D. McKenzie. 2009. “In pursuit of balance: Randomization in practice in development field experiments.” American Economic Journal: Applied Economics. 200-232.

Borja-Vega, C., L. Pena, and C. Stip. 2017. “Sustainability of rural water systems: Quantitative analysis of Nicaragua’s monitoring data.” Waterlines. Vol. 36. No.1 DOI: 10.3362/1756-3488.2017.003

Borja-Vega, C., J. Gruber, and A. Spenack. 2017. “Increasing the Sustainability of Rural Water Service: Findings from the Impact Evaluation Baseline Survey in Nicaragua.” World Bank Policy Research Working Paper No. 8283. Washington, DC: World Bank.

Cronk, R., and J. Bartram . 2018. “Identifying opportunities to improve piped water continuity and water system monitoring in Honduras, Nicaragua, and Panama: Evidence from Bayesian networks and regression analysis.” Journal of Cleaner Production. https://doi.org/10.1016/j.jclepro.2018.06.017

Giné, R., and A. Pérez-Foguet. 2008. “Sustainability assessment of national rural water supply program in Tanzania.” Natural Resources Forum, 32: 327–342.

Gross, B., C. van Wijk, and N. Mukherjee. 2001. “Linking Sustainability with Demand, Gender and Poverty: A Study in Community-Managed Water Supply Projects in 15 Countries.” IRC International Water and Sanitation Centre.

Harvey, P.A., and R.A. Reed. 2003. ”Sustainable rural water supply in Africa: Rhetoric and reality.” Proceedings of the WEDC 29th Conference. Pages: 115-118.

Hulland, K., et al. 2015. “What factors affect sustained adoption of safe water, hygiene and sanitation technologies?” 3ie Systematic Review Summary No. 2. http:// www.3ieimpact.org/media/ filer_public/2015/08/23/srs_2-_factors_for_sustained_ wash_adoption.pdf

Hutton, G., L. Haller, and J. Bartram. 2007. “Global cost-benefit analysis of water supply and sanitation interventions.” Journal of Water and Health, 5, 481-502.

Hutchings, P., et al. 2015. “A systematic review of success factors in the community management of rural water supplies over the past 30 years.” Water Policy 17: 963–983.

Jiménez, A., and A. Pérez-Foguet. 2010. “Challenges for water governance in rural water supply: Lessons learned from Tanzania.” International Journal of Water Resources Development 26(2), 235-248. Jiménez, A., and A. Pérez-Foguet. 2011. “The relationship between technology and functionality of rural water points: Evidence from Tanzania.” Water Science and Technology 63(5), 948–955.

Karamage, F., et al. 2016. “The Need for Awareness of Drinking Water Loss Reduction for Sustainable Water Resource Management in Rwanda.” Journal of Geoscience and 60 Environment Protection, Vol. 4, 74-87.

Kvarnström, E., et al. 2011. “The sanitation ladder–A need for a revamp?” Journal of Water, Sanitation and Hygiene for Development. Vol 1 No 1 pp 3–12 IWA Publishing.

Levy, K., et. al. 2008. “Following the water: A controlled study of drinking water storage in northern coastal Ecuador.” Environmental Health Perspectives, 116(11), 1533.

Lim, S.S., et al. 2013. “A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: A systematic analysis for the Global Burden of Disease Study 2010.” The Lancet, 380, 2224-2260.

Loebach, P., and K. Korinek. 2019. “Disaster vulnerability, displacement, and infectious disease: Nicaragua and Hurricane Mitch.” Population and Environment, 40(4), 434–455. https://doi.org/10.1007/s11111-019-00319-4

Lockwood, H. 2002. “Institutional support mechanisms for community-managed rural water supply & sanitation systems in Latin America.” Strategic Report No. 6. Environmental Health Project (EHP). Washington, DC, USAID.

Lockwood, H., and S. Smits. 2011. “Supporting rural water supply: Moving towards a service delivery approach.” UK: Practical Action Publishing.

Lockwood, H., A. Bakalian, and W. Wakeman. 2003. “Assessing sustainability in rural water supply: The role of follow-up support to communities.” Literature review and desk review of rural water supply and sanitation project documents. Washington, DC: World Bank.

Lundin, M., S. Molander, and G. Morrison. 1999. “A set of indicators for the assessment of temporal variations in sustainability of sanitary systems.” Water Science and Technology, 39(5), 235–242.

Lyer, P., et al. 2002. “Rural Water Supply, Sanitation and Hygiene: A Review of 25 Years of World Bank Lending (1978–2003). Water Supply & Sanitation Working Notes, World Bank.

Mandara, C.G., C. Butjin, and A. Niehof. 2013. “Community management and sustainability of rural water facilities in Tanzania.” Water Policy 15, 79-100.

Marks, S., K. Komices, and J. Davis. 2014. “Community Participation and Water Supply Sustainability: Evidence from Handpump Projects in Rural Ghana.” Journal of Planning and Education Research. Sage Publication. 04-041. Mayer, P.W., et al. 1998. “Residential End Use Study Progress Report: Year 2.” AWWA Annual Conference Proceedings, Dallas, Texas.

Mehta, L., and S. Movik. 2014. “Liquid dynamics: Challenges for sustainability in the water domain. “WIREs Water 1: 369–84 http://dx.doi.org/10.1002/wat2.1031.

Mitchell, D.L., and T.W. Chesnutt. 2013. “Evaluation of East Bay Municipal Utility 61 District’s Pilot of WaterSmart Home Water Reports.” Report prepared for the California Water Foundation and East Bay Municipal Utility District.

Montgomery, M.A., J. Bartram, and M. Elimelech. 2009. “Increasing Functional Sustainability of Water and Sanitation Supplies in Rural Sub-Saharan Africa.” Environmental Engineering Science. 26(5): 1017-1023. doi:10.1089/ees.2008.0388.

Mukherjee, N., and C. van Wijk. 2003. “Sustainability planning and monitoring in community water supply and sanitation.” IRC International Water and Sanitation Centre.

Nuevo FISE. 2015. Aprendizaje vinculado a Resultados (AVAR) en Agua y Saneamiento Rural. III Fase_ 2015_2017. Dirigido a Tecnicos de las UMAS o unidades homologas de las alcald.as del Pacifico, Centro y Norte. Managua, Diciembre, 2015.

Pullan, R.L., et al. 2014. “Geographical Inequalities in Use of Improved Drinking Water Supply and Sanitation across Sub-Saharan Africa: Mapping and Spatial Analysis of Cross-sectional Survey Data.” PLoS Med, 11, e1001626.

Ramesh, A., et al. 2015. “Evidence on the Effectiveness of Water, Sanitation, and Hygiene (WASH) Interventions on Health Outcomes in Humanitarian Crises: A Systematic Review.” PLoS One. 10(9) NCBI. https://www.ncbi.nlm.nih.gov/pubmed/26398228

Schmidt, W.P., and S. Cairncross. 2009. “Household water treatment in poor populations: Is there enough evidence for scaling up now?” Environmental Science & Technology, 43 (4), 986–992.

Schouten, T., and P. Moriarty. 2003. “Community water, community management: From system to service in rural areas.” London, UK: ITDG Publishing.

Schweitzer, R., and J. Mihelcic. 2012. “Assessing sustainability of community management of rural water systems in the developing world.” Journal of Water, Sanitation and Hygiene for Development. Vol. 2 (1) 20-30; DOI: 10.2166/ washdev.2012.056

Soley, F., and J. Thøgersen. 2003. “Monitoring and evaluation system for rural water supply.” Nigeria: 29th WEDC International Conference. 296-298.

Sommer, M., M. Kjellen, and C. Pensulo. 2013. “Girls’ and women’s unmet needs for menstrual hygiene management (MHM): The interactions between MHM and sanitation systems in low-income countries.” Journal of Water, Sanitation and Hygiene for Development, 3, 283-297. Snilstveit B., and H. Waddington. 2009. “Effectiveness and sustainability of water, sanitation, and hygiene interventions in combating diarrhea.” Journal of Development Effectiveness.” Vol 1 Issue 3: 295–335.

Spears, D. 2013. “How much international variation in child height can sanitation explain?” World Bank policy research working paper. Washington, DC: World Bank. 62 Stanton, B.F., and J.D. Clemens. 1987. “An educational intervention for altering water/sanitation behaviors to reduce childhood diarrhea in urban Bangladesh.” II. A randomized trial to assess the impact of the intervention on hygienic behavior and rates of diarrhea.” American Journal of Epidemiology. Feb. 1987, 125(2): 292-301.

Stocks, M.E., et al. 2014. “Effect of Water, Sanitation, and Hygiene on the Prevention of Trachoma: A Systematic Review and Meta-Analysis.” PLoS Med 11(2): e1001605. doi:10.1371/journal.pmed.1001605.

Strunz, E.C., et al. 2014. “Water, sanitation, hygiene, and soil-transmitted helminth infection: A systematic review and meta-analysis.” PLOS Medicine.11(3) NCBI. https:// www.ncbi.nlm. nih.gov/pubmed/24667810

Taylor, D.L., et al. 2015. “The Impact of Water, Sanitation and Hygiene Interventions to Control Cholera: A Systematic Review.” PLOS One. 10(8). https://www.ncbi.nlm.nih. gov/pubmed/26284367

Timmins, C. 2003. “Demand-side Technology Standards Under Inefficient Pricing Regimes: Are They Effective Water Conservation Tools in the Long Run?” Environmental and Resource Economics. 26: 107-24.

Trevett, A.F., R.C. Carter, and S.F. Tyrrel. 2004. “Water quality deterioration: A study of household drinking water quality in rural Honduras. International Journal of Environmental Health Research, 14(4), 273-283.

UNICEF. 1999. Water Hand Book: Water, Environment and Sanitation Technical Guidelines Series—No. 2.

USAID. 2009. “Environmental guidelines for small-scale activities in Africa.” Chapter 16, Water and Sanitation.

Water and Environmental Health at London and Loughborough (Well) 1998. London School of Hygiene and Tropical Medicine (LSHTM) and Water, Engineering and Development Centre (WEDC). DFID guidance manual on water supply and sanitation programmes. Loughborough Univeristy.

Whittington, D., et al. 1992. “Giving Respondents Time to Think in Contingent Valuation Studies: A Developing Country Application.” Environmental Economic Management 2:205-545.

Whittington, D., et al. 2009. “How well is the demand-driven, community management model for rural water supply systems doing? Evidence from Bolivia, Peru, and Ghana.” Water Policy. Vol 11 No 6. IWA Publishing. WHO/UNICEF. 2014. “Progress on Drinking Water and Sanitation—2014 Update.” New York.

WHO/UNICEF. Joint Monitoring Programme for Water and Sanitation. New York.

Wild, L., et al. 2012. “Common constraints and incentive problems in service delivery.” ODI Working Paper 351. U.K. 63

Wolf, J., et al. 2019. “A Faecal Contamination Index for interpreting heterogeneous diarrhoea impacts of water, sanitation and hygiene interventions and overall, regional and country estimates of community sanitation coverage with a focus on low- and middle-income countries.” International Journal of Hygiene and Environmental Health. Vol 222, Issue 2, pp 270-282. https://doi.org/10.1016/j.ijheh.2018.11.005

World Bank. 2014. “Project Appraisal Document—Sustainable Rural Water Supply and Sanitation Sector Project.” Washington, DC: World Bank.

Wright, J., S. Gundry, and R. Conroy. 2004. “Household drinking water in developing countries: A systematic review of microbiological contamination between source and point-of-use.” Tropical Medicine & International Health, 9, 106-17.

WSP 2011. “Rural Water Supply and Sanitation Challenges in Latin America for the Next Decade.” WSP-WB Washington D.C. http://www.wsp.org/sites/wsp.org/files/ publications/WSP-LAC-Rural-Water-Sanitation-Next-Decade.pdf

WSSCC. 2010. http://www.wsscc.org/sites/default/files/publications/wsscc_ hygiene_and_ sanitation_software_2010.pdf

Rural Water and Sanitation Information System, Nicaragua, (2013). www.siasar.org/ es/paises/nicaragua

Wutich, A., and K. Ragsdale. 2008. “Water insecurity and emotional distress: Coping with supply, access, and seasonal variability of water in a Bolivian squatter settlement.” Social Science & Medicine, 67, 2116-2125. ANNEX I

ACCESS TO WATER AND SANITATION BY DEPARTMENT

26% 19.6% South 19% 16.1% 64

CCAR 21% 19.3% North 4% 42.2% 69% 75.1% Estelí 65% 60.3% 66% 61.2% Madriz 66% 74.8% 61% 60.3% Boaco 41% 46.7% L 58% 57.2%

RA Matagalpa 53% 44.3% NT 58% 42.3% CE Nueva Segovia 50% 38.8% 52% 40.0% Chontales 41% 46.7% 48% 36.3% Río San Juan 31% 29.2% 44% 24.2% Jinotega 31% 21.8% 94% 82.0% Masaya 75% 80.6% 79% 78.3% Managua 41% 73.7% 64% 72.8% León 55% 72.3%

IFIC 61% 66.8%

C Rivas 40% 64.7% PA 53% 59.2% Granada 49% 78.7% 47% 48.6% Carazo 56% 48.9% 44% 47.0% Chinandega 45% 69.5% 51% 47% Rural 41% 51%

0% 20%40% 60%80% 100% 0% 20%40% 60%80% 100% 2017/19 2012/13 2017/19 2012/13 Figure A1.1. Access to Water by Department according to the Millennium Development Goals (MDG) Figure A1.2. Access to Sanitation by Department according to the Millennium Development Goals (MDG)

Percentage of dwellings. CCAR: Caribbean Coast Autonomous Regions. Water: Access to water distribution system (aqueduct or well with pump). Sanitation: Access to protected latrines or to toilets connected to sewage ANNEX II

MATRIXES OF INDICATORS USED TO EVALUATE UMAS, CAPS, AND COMMUNITIES

SCORE CONCEPT 4 3 2 1 65

70 percent 50 percent 1. Share of communities visited 90 percent Less than and less than and less than during the last 12 months or more 50 percent 90 percent 70 percent

70 percent 50 percent 2. Share of communities supported 90 percent Less than and less than and less than for water quality monitoring or more 50 percent 90 percent 70 percent

a. 80 or more Less than 50 communities 3. Human resources: Ratio of communities 50 to less 60 to less per communities to technicians per than 60 than 80 technician technician or b. No technicians

At least one Greater than 4. Transportation capacity: Ratio 0.50 to less vehicle per zero to less No vehicles of vehicles to technicians than 1 technician than 0.50

5. Equipment: a. Water quality monitoring All three Two One None b. Computer c. Informative material printed

6. Has: a. Assigned annual budget All three Two One None b. Funds for travel expenses and fuel c. Internet service"

Changes to matrix used in the workshops: Data availability: Component dropped to focus on UMAS characteristics Share of communities visited during the last 12 months: No modifications Share of communities supported for water quality monitoring: No modifications Human resources: Ratio of communities to technicians: Score of 1 adds municipalities without technicians as condition Transportation capacity: Ratio of vehicles to technicians: No modifications Equipment: Unlike SIASAR V2 (2017/19), SIASAR V1 (2012/13) captures no information on equipment status (good, bad and not working, etc.). This component changes to consider only whether the UMAS has the equipment. Number of vehicles is discarded as condition because the previous component focuses on transportation capacity. Has: No modifications Table A2.1. Matrix of Indicators Used to Evaluate UMAS

Each UMAS has a score between 1 and 4 on each of six components. The mean score across components is the UMAS overall score (see Annex II). Mean overall scores have the following categories: A [3.5-4.0], B [2.5-3.5], C [1.5-2.5], y D [1.0-1.5] SCORE CONCEPT 4 3 2 1

1. Formal operation a. All board of directors members are assigned b. Board of directors had at least four meetings 66 Three Two One None over the last 6 months c. Last board meeting minutes include documents of income and expenses

Tariffs Tariffs Tariffs established established established and revenues and revenues No tariffs 2. Adequate tariffs for water supplied /1 and revenues cover less cover 80 established cover all than 80 percent billing percent of or more billing

3. Financial stability a. Has bank account Three Two One None b. Keeps up to date accounting records c. Has available funds (savings)

4. Adequate operation and maintenance a. Did preventive and corrective activities (last 12 months) Three Two One None b. Has personnel responsible of operation and maintenance c. Has service provision guidelines

Service Service provider provider does 5. Adequate protection of water source promotes not promote a clean a clean environment environment

1/ No tariff: Mean declared tariff is not specified or total billing is zero

Changes to matrix used in the workshops: Formal operation: Because CAPS legalization was part of national mandate that overruled to postpone interventions on control communities, we omit CAPS legalization from the component.

Owing to differences across SIASAR V1 (2012/13) and SIASAR V2 (2017/19): Adequate tariffs for water supplied: Conditions are simplified to focus on indicators of whether the CAPS has established fees and on the relation of what they billed their clients to what their clients paid Financial stability: Conditions are simplified and add whether the CAPS has savings (available funds). Adequate operation and maintenance: Whether the CAPS has service provision guidelina replaces a component. Adequate protection of water source: No information on the matrix used in the workshops is strictly comparable. The stated condition here used is the only strictly comparable information across SIASAR versions.

Table A2.2. Matrix of Indicators Used to Evaluate CAPS

Each CAPS has a score between 1 and 4 on each of five components. The mean score across components is the CAPS overall score (see Annex II). Mean overall scores have the following categories: A [3.5-4.0], B [2.5-3.5], C [1.5-2.5], y D [1.0-1.5] CATEGORY CRITERIA INDICATOR

Percentage of the population Open defecation that practice open defecation

Percentage of the population that 67 Clean environment Unsafe trash disposal burns trash, throws it anywhere, or throws it to bodies of water

Percentage of the population Feces or trash on yard in dwellings with feces or trash in the yard

Percentage of the population with handwashing station Handwashing with water and soap that is used daily by all household members Percentage of the population that always use latrine (only Hygiene Latrine use for households with access to latrines) Percentage of the population that stores drinking water in Safe water storage small lid containers in which hands cannot enter or gets water from tap or bottled water Table A2.3. Indicators used to Evaluate Communities ANNEX III

BALANCE TESTS

68 Treatment Control Difference INDICATORS N Mean Std.Dev N Mean Std.Dev Dif. p-value

Community has caps 149 0.83 0.37 149 0.80 0.40 0.03 0.457

CAPS Score [1-4] 138 2.46 0.79 134 2.34 0.78 0.12 0.285

1. Formal operation 138 2.61 0.95 134 2.39 0.89 0.22 0.087

2. Adequate tariffs 138 2.30 1.16 134 2.22 1.18 0.08 0.612 for water supplied

3. Financial stability 138 1.98 1.01 134 1.96 1.01 0.02 0.865

4. Adequate operation 138 2.61 1.10 134 2.57 1.08 0.04 0.792 and maintenance

5. Adequate protection 138 2.83 1.47 134 2.57 1.50 0.26 0.204 of water source

The system provider 138 0.33 0.47 134 0.28 0.45 0.05 0.397 is legalized

Contaminated Controls Rest of Controls Difference INDICATORS N Mean Std.Dev N Mean Std.Dev Dif. p-value

Community has caps 38 0.92 0.27 111 0.76 0.43 0.16 0.029

CAPS Score [1-4] 38 2.39 0.65 96 2.32 0.83 0.08 0.633

1. Formal operation 38 2.50 0.83 96 2.34 0.92 0.16 0.385

2. Adequate tariffs 38 2.39 1.05 96 2.15 1.23 0.25 0.294 for water supplied

3. Financial stability 38 2.05 0.96 96 1.92 1.03 0.14 0.495

4. Adequate operation 38 2.61 1.05 96 2.55 1.09 0.05 0.826 and maintenance

5. Adequate protection 38 2.42 1.52 96 2.63 1.50 -0.20 0.537 of water source

The system provider 38 0.39 0.50 96 0.24 0.43 0.16 0.092 is legalized

Table A3.1. Balance tests – CAPS (SIASAR data) Treatment Control Difference INDICATORS N Mean Std.Dev N Mean Std.Dev Dif. p-value Water Safely Managed (dry season) 2380 0.16 0.37 2451 0.18 0.39 -0.02 0.562 Basic (dry season) 2380 0.62 0.48 2451 0.60 0.49 0.02 0.556 69 Limited (dry season) 2380 0.03 0.17 2451 0.03 0.16 0.00 0.627 Safely Managed (wet season) 2380 0.23 0.42 2451 0.24 0.43 -0.01 0.795 Basic (wet season) 2380 0.55 0.50 2451 0.54 0.50 0.01 0.759 Limited (wet season) 2380 0.03 0.17 2451 0.03 0.16 0.00 0.627 Hous. Connected to System 2380 0.60 0.49 2451 0.64 0.48 -0.04 0.355 Sanitation and Hygiene Unimproved 2382 0.08 0.28 2454 0.08 0.27 0.00 0.909 Open defecation 2382 0.11 0.31 2454 0.10 0.30 0.01 0.605 Has private sanitation facility 2382 0.82 0.39 2454 0.82 0.39 0.00 0.946 Reports Handwashing Station 2383 0.70 0.46 2454 0.71 0.46 0.00 0.903 Trash in Yard 2383 0.42 0.49 2454 0.39 0.49 0.03 0.399 Feces in Yard 2383 0.38 0.49 2454 0.33 0.47 0.05 0.093 Diseases Any member experienced diarrea (last 7 days) 2383 0.08 0.27 2454 0.09 0.29 -0.01 0.322 Any member experienced cuts or abrasions (last 7 days) 2383 0.01 0.12 2454 0.02 0.14 -0.01 0.145

Contaminated Controls Rest of Controls Difference INDICATORS N Mean Std.Dev N Mean Std.Dev Dif. p-value Water Safely Managed (dry season) 649 0.17 0.37 1802 0.19 0.39 -0.02 0.699 Basic (dry season) 649 0.61 0.49 1802 0.60 0.49 0.01 0.816 Limited (dry season) 649 0.04 0.19 1802 0.02 0.15 0.02 0.319 Safely Managed (wet season) 649 0.24 0.43 1802 0.25 0.43 -0.01 0.899 Basic (wet season) 649 0.54 0.50 1802 0.54 0.50 0.00 0.994 Limited (wet season) 649 0.04 0.19 1802 0.02 0.15 0.02 0.319 Hous. Connected to System 649 0.71 0.45 1802 0.61 0.49 0.10 0.103 Sanitation and Hygiene Unimproved 649 0.06 0.24 1805 0.09 0.29 -0.03 0.272 Open defecation 649 0.09 0.28 1805 0.11 0.31 -0.02 0.316 Has private sanitation facility 649 0.84 0.37 1805 0.81 0.39 0.03 0.261 Reports Handwashing Station 649 0.72 0.45 1805 0.70 0.46 0.02 0.604 Trash in Yard 649 0.37 0.48 1805 0.40 0.49 -0.03 0.646 Feces in Yard 649 0.34 0.47 1805 0.33 0.47 0.01 0.851 Diseases Any member experienced diarrea (last 7 days) 649 0.08 0.27 1805 0.10 0.30 -0.01 0.312 Any member experienced cuts or abrasions (last 7 days) 649 0.02 0.15 1805 0.02 0.14 0.00 0.950 Table A3.2. Balance tests – Household Indicators (Evaluation survey data) ANNEX IV

UMAS INFORMATION FROM EVALUATION SURVEY

CAPS received support on: % >30

93% Revise and update ﬁnance c. 51% In

85% Operation guidelines 46% 97% Tariff collection 69% 84% System inspection %% 57% 30 72% to Contact private providers to help 47% % 20

72%

c. Accompany household visits

In 47%

63% Training on hygiene 40% 87% Training on management 65% 96% Training on operation and maintenance 78% %% 20 99% Legalization of CAPS to 84% %

10 82% Protect Water Source c. 69% In

% 93% Solve technical issues with the system 81% 91% Accountability 82% 93% 0% Conﬂict resolution on water <1

87% c.

In 96% Creation of CAPS 91% 79% Collect water quality measures 75%

%% 61% % of communities supported (of assigned) 75 35% 0% 10%20% 30%40% 50%60% 70%80% 90%100% Endline Baseline

Figure A4.1. CAPS Support from UMAS Before and After AVAR Workshops using Survey Data Note: Only 68 of the 76 municipalities in the evaluation sample have survey data on both baseline and endline 71

16% D. Human Resources(number) 32%

6% C. Human Resources(technical ability) 46%

62% B. Financial

hort term need s 74%

16% A. Equipment 51%

22% D. Human Resources(number) 47%

9% C. Human Resources(technical ability) 54%

49% B. Financial

ong term need sS 81%

19% A. Equipment 63%

Has annual Budget assigned to support CAPS 72% 47%

22% If has budget, is it enough? 22% Resource sL

12% Do you have enough resources to support CAPS ? 22%

0% 10%20% 30%40% 50%60% 70%80% 90%100%

Endline Baseline

Figure A4.2. Needs and Resources of UMAS Before and After AVAR Workshops using Survey Data

Note: Only 68 of the 76 municipalities in the evaluation sample have survey data on both baseline and endline 72 Wants More Training On

29% Technical Support for them 54% 26% Tariff collection 57% 25% Water quality analysis 62%

24% Promote responsible water use 35% 24% Foster good practices on hygiene and sanitation 41% 22% Operate and mantain infrastructure 62% PS

CA 19% Protect water source 49%

19% Ensure good water quality 47% 18% Conﬂict resolution 32% 13% Address Complaints on Service 34% 12% Ensure reliable and continuous service 21%

9% Workplan establishment 6% 25% Finance 51% 16% AS Technical Ability

UM 40% 9% Operation 31%

0% 10%20% 30%40% 50%60% 70%80% 90%100% Endline Baseline

Figure A4.3. Training Needs of UMAS Before and After AVAR Workshops using Survey Data

Note: Only 68 of the 76 municipalities in the evaluation sample have survey data on both baseline and endline A. FEEDBACK ON FISE

Financial support 72% 73 Training 51%

Technical support 51%

Equipment 49%

Foster UMA independence 38%

Institutional support 31%

Logistical support 28% FISE can help UMAS by providing: More visits 25%

0% 20%40% 60%80% 100% B. FEEDBACK ON ARAS

ARAS does not need to improve 18%

Financial support 46%

More often visits 38%

Technicians support 38%

Training 35%

CAP support 35% ARAS should improve on: Logistical support 32%

Municipality support 28%

Technical assistance 81%

Advise on CAPS legalization 72%

ARAS provides: SIASAR training 66%

0% 20%40% 60%80% 100%

Figure A4.4. Feedback of UMAS about FISE and ARAS, 2019

Note: Restricted to 68 municipalities in the evaluation sample with survey data on both baseline and endline ANNEX V COMPARISON BETWEEN TREATMENT AND CONTROL IN HOUSEHOLD EVALUATION SURVEYS A. All Sample Open defecation Unsafe trash disposal Feces or trash on yard

74 12% 100% 60%

9% 90% 50% 45% 6% 5% 46% 80% 40% 3% 3% 77% 75% 0% 70 % 30% Baseline Endline Baseline Endline Baseline Endline Handwashing Latrine use Safe water storage

90% 100% 60% 84% 98% 98% 54% 84% 50% 55% 80% 90% 40% 70 % 80% 30%

60% 70 % 20% Baseline Endline Baseline Endline Baseline Endline B. Excluding contaminated control communities Open defecation Unsafe trash disposal Feces or trash on yard

12% 100% 60%

9% 90% 50% 47% 6% 46% 5% 80% 40% 3% 3% 77% 73% 0% 70 % 30% Baseline Endline Baseline Endline Baseline Endline Handwashing Latrine use Safe water storage

90% 100% 60% 84% 98% 98% 54% 84% 50% 53% 80% 90% 40% 70 % 80% 30%

60% 70 % 20% Baseline Endline Baseline Endline Baseline Endline T C Figure A5.1. Clean Environment and Hygiene A. All Sample Safely managed water: Safely managed water: Dry season Wet season 60% 60% 75

40% 40% 38% 28% 38% 20% 26% 20%

0% 0% Baseline Endline Baseline Endline Improved Sanitation Private sanitation facility Health: Any member had diarrhea (last week) 60% 90% 15% 14% 51% 46% 12% 40% 80% 81% 78% 10%

20% 70 % 5%

0% 60% 0% Baseline Endline Baseline Endline Baseline Endline B. Excluding contaminated control communities Safely managed water: Safely managed water: Dry season Wet season 60% 60%

40% 40% 38% 28% 37% 20% 26% 20%

0% 0% Baseline Endline Baseline Endline Improved Sanitation Private sanitation facility Health: Any member had diarrhea (last week) 60% 90% 15% 14% 51% 44% 12% 40% 80% 81% 10% 78%

20% 70 % 5%

0% 60% 0% Baseline Endline Baseline Endline Baseline Endline T C Figure A5.2. Safely Managed Water Access, Sanitation, and Diarrhea ANNEX VI

AVAR TRAINING DATES FOR THE 76 MUNICIPALITIES IN THE EVALUATION SAMPLE

76 MUNICIPALITY WORKSHOP 1 WORKSHOP 2 WORKSHOP 3

ACHUAPA 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

ACOYAPA 22,24 June 2016 22,24 Mayo 2017 21,23 March 2018

BLUEFIELDS 20,22 March 2016 14,15 December 2016 15,17 November 2017

CAMOAPA 22,24 June 2016 22,24 May 2017 21,23 March 2018

CÁRDENAS 28,29 ,30 September 2016 4, 5, 6 October 2017 3, 4,5 October 2018

CHINANDEGA 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

CINCO PINOS 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

CIUDAD ANTIGUA 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

CIUDAD DARÍO 10,12 August 2016 20,22 September 2017 26,27 July 2018

COMALAPA 22,24 June 2016 22,24 Mayo2017 21,23 March 2018

CONDEGA 10,12 August 2016 20,22 September 2017 26,27 July 2018

DIPILTO 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

EL ALMENDRO 22,24 June 2016 22,24 Mayo2017 21,23 March 2018

EL AYOTE 20,22 March 2016 14,15 December 2016 15,17 November 2017

EL CORAL 22,24 June 2016 22,24 Mayo2017 21,23 March 2018

EL CUA 3,5 August 2016 20,22 September 2017 7,9 March 2018

EL JICARAL 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

EL JíCARO 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

EL RAMA 20,22 March 2016 14,15 December 2016 15,17 November 2017

EL SAUCE 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

EL TUMA - LA DALIA 10,12 August 2016 20,22 September 2017 26,27 July 2018

EL VIEJO 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

ESQUIPULAS 10,12 August 2016 20,22 September 2017 26,27 July 2018

ESTELI 10,12 August 2016 20,22 September 2017 26,27 July 2018 MUNICIPALITY WORKSHOP 1 WORKSHOP 2 WORKSHOP 3

GRANADA 28,29 ,30 September 2016 4, 5, 6 October 2017 3, 4,5 October 2018

JALAPA 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

JINOTEGA 3,5 August 2016 20,22 September 2017 7,9 March 2018 77

JUIGALPA 22,24 June 2016 22,24 Mayo2017 21,23 March 2018

LA CRUZ DE RÍO GRANDE 20,22 March 2016 14,15 December 2016 15,17 November 2017

LA PAZ CENTRO 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

LA TRINIDAD 10,12 August 2016 20,22 September 2017 26,27 July 2018

LARREYNAGA 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

LEÓN 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

MACUELIZO 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

MATIGUAS 10,12 August 2016 20,22 September 2017 26,27 July 2018

MUELLE DE LOS BUEYES 20,22 March 2016 14,15 December 2016 15,17 November 2017

MURRA 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

NUEVA GUINEA 20,22 March 2016 14,15 December 2016 15,17 November 2017

POSOLTEGA 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

PUEBLO NUEVO 10,12 August 2016 20,22 September 2017 26,27 July 2018

QUILALI 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

RANCHO GRANDE 10,12 August 2016 20,22 September 2017 26,27 July 2018

RÍO BLANCO 10,12 August 2016 20,22 September 2017 26,27 July 2018

RIVAS 28,29 ,30 September 2016 4, 5, 6 October 2017 3, 4,5 October 2018

SAN DIONISIO 10,12 August 2016 20,22 September 2017 26,27 July 2018

SAN FERNANDO 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

SAN FRANCISCO DEL NORTE 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

SAN FRANCISCO LIBRE 21, 22, 23 September 2016 22,23, 24 November 2017 20, 21,22 November 2018

SAN ISIDRO 10,12 August 2016 20,22 September 2017 26,27 July 2018

SAN JOSÉ DE BOCAY 3,5 August 2016 20,22 September 2017 7,9 March 2018

SAN JOSÉ DE CUSMAPA 24,26 August 2016 27,29 September 2017 11,13 Abril 2018 MUNICIPALITY WORKSHOP 1 WORKSHOP 2 WORKSHOP 3

SAN JUAN DE LIMAY 10,12 August 2016 20,22 September 2017 26,27 July 2018

SAN JUAN DEL RÍO COCO 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

78 SAN JUAN DEL SUR 28,29 ,30 September 2016 4, 5, 6 October 2017 3, 4,5 October 2018

SAN LORENZO 22,24 June 2016 22,24 Mayo2017 21,23 March 2018

SAN LUCAS 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

SAN PEDRO DE LOVAGO 22,24 June 2016 22,24 Mayo2017 21,23 March 2018

SAN RAFAEL DEL NORTE 3,5 August 2016 20,22 September 2017 7,9 March 2018

SAN RAFAEL DEL SUR 21, 22, 23 September 2016 22,23, 24 November 2017 20, 21,22 November 2018

SAN SEBASTIÁN DE YALI 3,5 August 2016 20,22 September 2017 7,9 March 2018

SANTA LUCIA 22,24 June 2016 22,24 Mayo2017 21,23 March 2018

SANTA MARÍA DE PANTASMA 3,5 August 2016 20,22 September 2017 7,9 March 2018

SANTA TERESA 28,29 ,30 September 2016 4, 5, 6 October 2017 3, 4,5 October 2018

SANTO TOMÁS 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

SEBACO 10,12 August 2016 20,22 September 2017 26,27 July 2018

SOMOTO 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

TELPANECA 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

TERRABONA 10,12 August 2016 20,22 September 2017 26,27 July 2018

TEUSTEPE 22,24 June 2016 22,24 Mayo2017 21,23 March 2018

TIPITAPA 21, 22, 23 September 2016 22,23, 24 November 2017 20, 21,22 November 2018

VILLA CARLOS FONSECA 22,24 June 2016 22,24 Mayo2017 21,23 March 2018

VILLANUEVA 12,13,14 October 2016 22, 23, 24 February 2017 26, 27, 28 September 2018

WASLALA 3,5 August 2016 20,22 September 2017 7,9 March 2018

WIWILI DE JINOTEGA 3,5 August 2016 20,22 September 2017 7,9 March 2018

WIWILI DE NUEVA SEGOVIA 24,26 August 2016 27,29 September 2017 11,13 Abril 2018

YALAGUINA 24,26 August 2016 27,29 September 2017 11,13 Abril 2018 ANNEX VII

SAMPLE OUTLINE ON HOW TO DEVELOP A PLAN OF ACTION – ADMINISTERED TO UMAS REPRE- SENTATIVES AS PART OF THE FIRST AVAR TRAINING

QUE ES UN PLAN DE ACCIÓN 79 Es el momento en que se determinan y se asignan las tareas, se definen los plazos de tiempo y se calcula el uso de los recursos.

Un plan de acción es una presentación resumida de las tareas que deben realizarse por ciertas personas, en un plazo de tiempo específicos, utilizando un monto de recursos asignados con el fin de lograr un objetivo dado

El plan de acción es un espacio para discutir qué, cómo, cuando y con quien se realizaran las acciones.

COMO ELABORAR EL PLAN DE ACCIÓN

El plan de acción es un trabajo en equipo, por ello es importante reunir a los demás trabajadores comunitarios y a los miembros de la comunidad y formalizar el grupo llamándolo “Comité de planeamiento” u otra denominación.

El plan lleva los siguientes elementos.

• Que se quiere alcanzar (objetivo) • Cuánto se quiere lograr (cantidad y calidad) • Cuándo se quiere lograr (en cuánto tiempo) • En dónde se quiere realizar el programa (lugar) • Con quién y con qué se desea lograrlo (personal, recursos financieros) • Cómo saber si se está alcanzando el objetivo (evaluando el proceso) • Cómo determinar si se logró el objetivo (evaluación de resultados)

Los planes de acción solo se concretan cuando se formulan los objetivos y se ha seleccionado la estrategia a seguir.

Los principales problemas y fallas de los planes se presentan en la definición de los detalles concretos. Para la elaboración del plan es importante identificare las grandes tareas y de aquí desglosar las pequeñas.

Se recomienda utilizar un “cuadro de plan de acción” que contemple todos los elementos.

ACTIVIDADES CUANTO TIEMPO LUGAR RECURSOS SEGUIMIENTO LOGRO

El plan de acción es un instrumento para la evaluación continua de un programa. Es a su vez la representación real de las tareas que se deben realizar, asignando responsables, tiempo y recursos para lograr un objetivo ANNEX VIII

ACCESS TO SAFELY MANAGED WATER THAT CONSIDERS WATER QUALITY AND IS MEASURED BY SIASAR

Because SIASAR captures water quality information, its information can 80 be used to assess community level access to safely managed water. Table A8.1 presents adjustments the attributes of rural water distribution systems analyzed so far. These adjustments allow to map SIASAR information to the SDG ladder. The principal adjustment is that instead of considering water quality testing, now test results are considered. Another adjustment is to drop as attribute whether systems treat water with chlorine. A community has access to improved water sources if at least one water distribution system services it. Once attributes of rural water distribution systems are considered, a community has limited water access when at least one system servicing it is deemed to be more than 100 meters away from the households subscribed to it. In turn, a community has basic water access when no system servicing it is deemed to be more than 100 meters away.

Improved Unimproved Safely Limited Basic managed Access to water No Yes Yes Yes distribution system (1) Time to source: Distance A. Distance Yes No No more than 100m (2) Free from chemical B. Quality Yes and fecal contamination (3) Available 24hrs every C. Availability Yes day of the week Table A8.1. Definition of Access to Water According to the SDG and Information from SIASAR

Lack of water quality test information hinders SDG estimation. Water quality information in SIASAR Nicaragua consists of when water was tested and on whether results were satisfactory. Nicaragua’s National Health Ministry routinely executes water quality tests and provides a report to water distribution system providers. SIASAR interviewers ask for the report and sum-up its information into three indicators. The first is date of interview, a field left missing in the information system when no report is found. The second and third are the results for fecal and for chemical tests. This analysis focuses only on recent water quality assessments. Systems must have water quality test results carried-out in the year of interview or, at most, the year before. Systems with older tests are deemed as having no water quality information. As showed in the analysis of attributes of rural water distribution systems, only 47 percent of systems reported recent results in 2012/13. This lack of information leads the analysis to restrict only to communities in which at least one system reports water quality test. This sample restriction hinders SDG estimation. For the communities in the restricted sample, a community has basic water access when no system servicing it: (1) is deemed to be more than 100 meters away from the households subscribed to it, (2) reports fecal or chemical contamination, and (3) services the community less than 24hrs per day, every day. 81

Table A8.2 provides SDG ladder estimates for 2012/13. Even under the Millennium Development Goal (MDG) standard, access to water in rural areas is low. Access is only 41 percent and is as low as 11 percent in the Caribbean coast region. Once we account for distance to water services (basic water), adequate access drops to 27 percent. The basic water metric portrays a different regional rank vis-à-vis the MDG standard. The Pacific region, the region that encompasses Managua and the better-off municipalities in the country, has a higher access than the central region under the MDG standard but a lower one under the basic water standard. In the restricted sample, once we account for water quality and continuity of service, access drops from 20 to 3 percent. The Central and Caribbean coast regions exhibit lower decreases compared with the Pacific region. As showed in the analysis of water distribution systems, these results also signal how system characteristics across regions differ. The Caribbean coast region has fewer systems but the few systems there are closer to users, interrupt service less often, and show lower water pollution. In contrast, the Pacific region has more systems than the Central region, but these systems are farther away, interrupt service more often, and show higher water pollution.

Rural areas of main regions

Caribbean Rural Pacific Central Coast

MDG: Access to improved water 41% 51% 47% 11%

SDG Ladder

0. Unimproved 59% 49% 53% 89%

1. Limited 14% 24% 12% 2%

2. Basic 27% 27% 35% 9%

Only if has systems with water quality tests

2a Basic 20% 21% 30% 3%

3a Safely managed (2) 3% 2% 5% 0%

Table A8.2. Access to Water According to the SDG and Information from SIASAR 2012/13 (percentage of dwellings) This profile, however, is partial and likely biased. To construct safely managed water, communities with systems without water quality tests necessarily drop-out from the sample. Basic water access drops from 27 to 20 percent when we communities without water quality 82 information drop-out. Basic water access decreases are higher in the Pacific and Caribbean coast regions. Given poverty levels across regions, this suggest that better-off communities in terms of basic water drop-out. Estimates, overall, likely have a downward bias but the extent of the bias, and whether is downward or not, across regions is unclear.

Figure A8.1 expands to the community level the summary in Table A8.2. It shows that basic water access is high on the north-west areas of the country, in the Central region departments of Madriz, Esteli, Nueva Segovia, Matagalpa, and Jinotega. It is also high in the Pacific region departments of Carazo, Granada, Leon, and Managua. Once we account for water quality and continuity of service, very few pockets of communities with safely managed water remain. Most of the few that do remain are located in the central region departments of Matagalpa and Nueva Segovia. a) Access to Improved Water

b) Access to Basic Water Improved water within 100mts

c) Access to Safely Managed Water Improved water, within 100mts, free of contamination, and available 24hrs everyday (Restricted sample of communities)

% of Households Less than 50% 50-75% 75-100%

Figure A8.1. Access to Water in Rural Communities, 2012/13.

Notes: Dark blue dots denote rural communities with high access to water, light-blue medium-high access, and light-red low access. Demarcations for Nicaragua’s three regions are provided. Left to right, these regions are Pacific, Central, and Caribbean Coast. The safely managed access map, panel c, excludes communities without water quality information. MARCH 2020