Mapping the Evidence on WASH Promotion in Communities, Schools and Health Facilities

Evidence gap map Water, sanitation and hygiene

Mapping the evidence on WASH promotion in communities, schools and health facilities

Water, sanitation and hygiene (WASH) are Highlights fundamental human rights that underpin survival, dignity, productivity, reproductive The number of rigorous studies on WASH health and happiness. According to a 2017 promotion in households, schools and communities report by the World Health Organization has increased substantially. and UNICEF Joint Monitoring Programme, Very few rigorous studies exist on WASH 3 in 10 people worldwide (2.1 billion) lack promotion in medical facilities. access to safe, readily available water at Many new studies evaluate previously under-researched home, and more than 6 in 10 (4.5 billion) approaches, such as community-led total sanitation, and 1 lack safely managed sanitation. measure important sector outcomes, including school The Sustainable Development attendance and reducing open defecation. Goals – as well as other major policy More studies are needed to measure programme initiatives, such as the Swachh Bharat effects on the sustained use of technologies and slippage back to open defecation. Abhiyan (Clean India Mission) in India – go further than the Millennium Development A critical need exists for studies on promotional Goals with a call for ending open defecation, approaches for vulnerable populations, especially people living with disabilities, for whom no studies are available. which is still practised by more than 900 million people globally. The Sustainable New, high-quality systematic reviews are available, including ones on the effectiveness of Development Goals also set the target of hygiene and sanitation behavioural approaches universal access to WASH facilities by 2030. and menstrual hygiene management. Synthesis gaps remain for high-quality reviews on WASH interventions in schools and on decentralised delivery. 1 WHO/UNICEF JMP, 2017. Progress on drinking water, sanitation and hygiene – 2017 update and SDG baselines. Improved WASH services can consequences, poor 3ie’s sector-wide WASH lead to a wide range of health, conditions can also diminish evidence gap map provides an social and economic educational attainment and assessment of the evidence outcomes. Poor WASH cause danger and stress for base for behaviour change, conditions contribute to high vulnerable populations, health and socio-economic rates of diarrhoeal disease especially women and girls. outcomes resulting from WASH promotional approaches in and acute respiratory and To help achieve universal households, schools, health parasitic worm infections. targets and reach the most facilities and communities. This Chronic high infection rates disadvantaged populations, brief articulates the primary are a leading cause of decision-makers need and synthesised evidence undernutrition and death in access to high-quality available in the map and children in developing evidence on what works in what evidence is needed countries; UNICEF estimates WASH promotion in different across different low- and that diarrhoea alone kills contexts and for different 2 middle-income countries 480,000 children each year. groups of people. Beyond the serious health (L&MICs) and regions.

Sustainable Development Goal for water and sanitation

Sustainable Development Goal 6 aims to ‘ensure the availability and sustainable management of water and sanitation for all’ and includes these goals: End open defecation, ensuring everyone has access to at least a basic toilet and a safe waste disposal system; Provide universal access to safe and affordable drinking water; Pay attention to the specific needs of women and vulnerable populations; and Expand international cooperation and strengthen the capacity of local and national bodies to manage their water and sanitation systems.

Main findings WASH interventions have two against outcomes. There is then a Research has shifted to important components – the ‘what’ filter for the technology, which cover a broader set of and the ‘how’. The what describes allows easy comparison of the mechanisms. This reflects the the technology, service or practice evidence for providing, for change of focus to how a the participants end up with (for example, latrines via these technology is provided. In example, a latrine), and the how different channels. particular, there has been an describes the mechanism or increased focus on behaviour Impact evaluations promotional approach of the change communication that intervention (for example, whether In the last 10 years, there has uses psychosocial ‘triggering’ you give a toilet directly to a been huge growth in the and a shift away from simply household or use a subsidy to help number of impact evaluations providing a good (see Figure 3). them buy one for themselves, see being conducted. There are at In sanitation, this is most Figure 1). Before the early 2000s, least 367 completed or ongoing commonly community-led total the focus of the conversation was rigorous impact evaluations in sanitation, which aims to on what works, but in the last 15 L&MICs – nearly three quarters increase the use of latrines by years it has increasingly switched conducted since 2008. Although leveraging social cohesion to from not only what technology to studies are spread over 87 make collective behavioural provide but also the best way to do countries around the globe, there changes, but also can include so in order to have high uptake and is a high concentration of work in information campaigns focused sustained use. Given this changing Bangladesh, India and on disgust or being a good focus, the evidence gap map Kenya, each with more than parent, as opposed to presents intervention mechanisms 40 studies (see Figure 2). factual information.

2 UNICEF, 2018. Diarrhoeal disease – current status and progress. Figure 1: WASH promotional approaches

Direct hardware provision All interventions for which the required infrastructure is provided by an external authority. This includes, for example, boreholes, piped water systems, water filters, soap, handwashing stations, latrines and public sewer connections.

Behaviour change communication All informational campaigns including health messaging – an educational approach to increase participants’ knowledge or skills – and psychosocial approaches, which use social or emotional motivators and pressures to change behaviour.

Systems-based approaches Approaches that try to change people’s behaviour and how hardware is accessed by changing the wider system around them. This includes subsidies; microfinance; and working with the suppliers of a service, such as improving current providers’ performance and decentralisation.

Behaviour change communication combined Interventions for which direct hardware with other promotional approaches provision or systems-based approaches are combined with a behaviour change communication campaign. An example is community-led total sanitation with marketing to sanitation providers.

Figure 2: WASH evidence in L&MICs

Number of impact evaluations by country Up to 3 4–7 8–12 13–19 20–50 No data Figure 3: Change in intervention mechanisms studied since 2008

40%40.00%

30%30.00%

20%20.00%

10%10.00%

0%0.00% DirectDirect hardware hardware provision HealthHealth messaging messaging Psychosocial 'triggering' Systems-basedSystems based approaches provision Pre-2008 Post-2008 ‘triggering’ approaches Until 2008 2009 onwards

There has been a large increase in improved behaviours as accurately Despite the importance of the number of studies on as possible. The principal argument sustained use and the need to sanitation technologies, used by proponents of alternative prevent slippage back to open particularly latrines. This followed delivery mechanisms is that they defecation, only 18 studies the United Nations International Year are more effective at changing these examine whether the of Sanitation in 2008, which brought behaviours and therefore more interventions promoted these attention to the importance of effective at improving lives. There outcomes. Sustainability of sanitation technologies, an area has also been an increase in social behaviours (defined here as previously considered too difficult or and economic outcomes, such as being measured 12 or more costly for applying rigorous impact school attendance and labour months after implementation) is evaluations. Until 2008, only 6 market outcomes. Having said this, most commonly measured for studies had been conducted on by far the most commonly reported handwashing practices, rather promoting or providing latrines, but outcome is still carer-reported than latrine use and sustained now there are more than 50. diarrhoea amongst children, open defecation–free status. sometimes alongside more More generally, although there Behavioural outcomes, such as objective, observed measures of are some innovations towards whether individuals are actually disease incidence. Research in the more rigorous outcome washing their hands or using a sector also underutilises recent measurement based on objective water filter or latrine, are now advances, such as list experiments tests, most studies measuring being reported by the majority of and vignettes, which aim to elicit behaviour change do so through studies (see Figure 4). It is views from participants on topics self-reporting, which is important that studies measure that are difficult to measure. considered less reliable.

Figure 4: Changes in the percentage of studies reporting outcomes over time

100%100.00%

80%80.00%

60%60.00%

40%40.00%

20%20.00%

0%0.00% Behavioural outcomes HealthHealth and and nutrition nutrition outcomes SocialSocial and economic and outcomes SustainabilitySustainability and slippage and outcomes outcomes Pre-2008 Post-2008economic outcomes slippage Until 2008 2009 onwards Some new studies evaluate included in the map (19% of health facility, despite its interventions and outcomes impact evaluations and 20% of importance for reproductive that disproportionately affect systematic reviews) report any health and infection control. women and girls, but more sex-disaggregated outcomes. The World Health Organization studies evaluating In impact evaluations, some of estimates that nearly 40 per interventions in more the most commonly sex- cent of facilities in L&MICs lack contexts are needed. Women disaggregated outcomes are handwashing facilities and 20 and girls carry most of the psychosocial health, education per cent lack basic sanitation; burden of water collection – and cognitive development, research on what mechanisms including time, calories spent, open defecation, and time use. are the most effective in this musculoskeletal injuries, and Most studies are not taking sex context is a key to improving risk of assault or attack – and or gendered determinants into health outcomes and could have to use unsafe places to account or targeting women’s have secondary effects on defecate, where water and specific needs. practices at home. sanitation services are not The evidence base on the The majority of rigorous accessible. They also impact of WASH interventions studies that examine experience particular hardships on vulnerable populations is effectiveness are randomised where inadequate services sparse. There are no rigorous controlled trials. However, constrain menstrual hygiene studies examining WASH more opportunities exist to management. Studies promotional approaches that conduct rigorous evaluations of measuring time use (22 target people living with existing data on natural studies), psychosocial health disabilities or the success of experiments. This approach is outcomes (7 studies), and standard WASH interventions in used when the rollout of a safety and vulnerability (4 improving outcomes for them. programme was not controlled studies) have been available There are also very few studies by the investigator, but the since 2008, but there are still that look at the needs of people process governing who very few studies examining pain living with HIV or people living in received the intervention and musculoskeletal disorders refugee camps (see Figure 5). created comparable control and due to water collection (1 study) intervention groups. It is and menstrual hygiene In terms of place of use, the underused in the WASH sector management (5 studies). And vast majority of studies look at and could prove to be a very there are no studies of providing technologies for use cost-effective approach for sanitation for non-binary or at home or by the general future research. transgender individuals. community. There has also been a large increase in the A great number of studies More generally, gender number of studies looking at the continue to be conducted analysis is rarely used as part impact of providing WASH without a control group, of the framework for hardware in schools, with at least simply measuring outcomes understanding gendered 39 rigorous studies on the topic. before and after; these are programme effects. This is due, not usually able to attribute in part, to a lack of sex- The authors found only one changes to the intervention. disaggregation in the reporting. rigorous study of WASH Only a minority of studies improvements for use in a

Figure 5: Populations targeted by interventions studied

Rural households and communities (267 studies) Urban households and communities (92 studies) Slums or informal settlements (33 studies) Schools (39 studies) Medical facilities (1 study) Refugee camps or humanitarian crises (3 studies) People living with HIV (1 study) People with disabilities (0 studies) Systematic reviews There are 42 completed systematic behaviour. Another review confirmed ending open defecation found in reviews and 1 protocol that the limited extent to which village case studies. Finally, the summarise the findings of the impact implementers were engaging with evidence suggests that community- evaluations for policymakers, different technological needs across based approaches and social programme managers, researchers the human life cycle. marketing are particularly effective in and practitioners. Many of these are promoting sanitation behaviour Evidence from medium- and high- of high quality (12 systematic change, whilst participatory quality systematic reviews indicates reviews) and cover a breadth of communication works well in that introducing piped water supplies, technologies, mechanisms and handwashing promotion. point-of-use water treatment systems outcomes. A recent review of hygiene with safe storage containers, and However, there are still many gaps and sanitation behavioural handwashing with soap could be with significant bodies of evidence in approaches found that involving the particularly effective at improving need of synthesising. For instance, community in different stages of health outcomes. Community-led there are no high-quality systematic programme design and total sanitation also seems to reduce reviews on WASH in schools, water implementation was particularly open defecation, at least in the short use and treatment behavioural effective, whereas health promotion term, but the evidence does not outcomes, community-driven was ineffective in changing corroborate the widespread claims of approaches, and subsidies. © Jim Holmes / AusAID © Jim Holmes /

How to read an evidence gap map 3ie presents evidence gap maps larger the bubble, the greater the evidence for that cell. The using an interactive online platform volume of evidence in that cell. The hyperlinks for these studies lead to that allows users to explore the colour of each bubble represents user-friendly summaries in the 3ie evidence base of included studies the type of evidence and, for a evidence database. Users can filter and reviews. Bubbles appearing at systematic review, a confidence the evidence by type, confidence intersections between interventions rating (as indicated in the legend). rating (for systematic reviews), and outcomes denote the existence In the online version, hovering over region, country, study design of at least one study or review. The a bubble displays a list of the and population. WASH Evidence Gap Map

Impact evaluation High confidence Medium confidence Low confidence Protocol Conclusions

Ensuring everyone has access to Understudied outcomes (such as change, health and socio-economic appropriate WASH facilities is one of sustainability and slippage, time use, outcomes, whenever possible; the most fundamental challenges in musculoskeletal disorders, Sex and age disaggregation and international development. Rigorous psychosocial health, safety and explicitly employing gender analysis evaluations of WASH programmes vulnerability) and final outcomes to better understand not only have been conducted since at least (such as education, income and differential outcomes, but also the the 1970s, but the last decade has poverty); role of gendered norms and seen a revolution in the research Alternative mechanisms (such as discriminatory social and structural being conducted in the sector, with an community-led total sanitation or barriers facing vulnerable populations increasing focus on sanitation and community-driven approaches), that need to be addressed during hygiene behaviour change. which could provide better incentives intervention design; and Researchers and funders need to for communities to adopt beneficial Synthesising the evidence in areas consider carefully where there is the practices, and the extent to which with sufficient impact evaluations, need for new primary evidence, such they should be implemented such as WASH in schools and water as impact evaluations, and for new alongside systems-based use and treatment behavioural evidence syntheses, such as approaches (such as microfinance) in outcomes, using mixed methods systematic reviews. This evidence certain contexts; approaches to provide evidence on gap map suggests these priorities for Collecting data on objective effectiveness and implementation. future research: measures of gendered behaviour

What is a 3ie evidence gap map? 3ie evidence gap maps are interactive, online visualisation of the needed. Maps help collections of evidence from impact evidence base, displayed in a decision-makers target their evaluations and systematic reviews framework of relevant interventions resources to fill these important for a given sector or policy issue, and outcomes. They highlight where evidence gaps and avoid duplication. organised according to the types of there are sufficient impact They also facilitate evidence- programmes evaluated and evaluations to support systematic informed decision-making by making outcomes measured. They include an reviews and where more studies are existing research more accessible.

About this brief This brief is based on the Water, evidence gap map was produced efficiency of the screening, sanitation and hygiene (WASH) and conducted new searches of enabling savings of 90 per cent in evidence gap map: 2018 update by behavioural outcomes and WASH the number of studies screened. Hugh Waddington, Hannah provision in health facilities. The Suggested citation: 3ie, 2018. Chirgwin, Duae Zehra, John Eyers 2018 update includes 234 Mapping the evidence on WASH and Sandy Cairncross. The 2018 additional studies in L&MICs; promotion in communities, schools update was funded by the Water overall, 320 are controlled impact and health facilities, 3ie Evidence Supply and Sanitation evaluations, 42 are systematic Gap Map Brief 10, New Delhi: Collaborative Council. The authors reviews, 47 are impact evaluation International Initiative for Impact systematically searched for trial registries and 1 is a systematic Evaluation. Available at: doi: https:// published and unpublished studies review protocol. The authors used doi.org/10.23846/B/EGM/201810 since the original 2014 WASH machine learning to improve the

The International Initiative for Impact Evaluation (3ie) promotes evidence-informed, equitable, inclusive and sustainable development. We support the generation and effective use of high-quality evidence to inform decision-making and improve the lives of people living in poverty in L&MICs. We provide guidance and support to produce, synthesise and quality assure evidence of what works, for whom, how, why and at what cost.

For more information on 3ie’s evidence gap map, contact [email protected] or visit our website.

3ieimpact.org

@3ieNews /3ieimpact /3ievideos

international-initiative-for-impact-evaluation August 2018