Systems Approaches to Water, Sanitation and Hygiene: a Systematic Literature Review

Article Systems Approaches to Water, Sanitation and Hygiene: A Systematic Literature Review

Nicholas Valcourt1,3,*, Amy Javernick-Will1,3, Jeffrey Walters2,3, Karl Linden1,3

1 Department of Civil, Environmental, and Architectural Engineering, University of Colorado Boulder, Boulder, CO 80309 2 College of Engineering, George Fox University, Newberg, OR 97132 3 USAID Sustainable WASH Systems Learning Partnership * Correspondence: [email protected];

Received: date; Accepted: date; Published: date

Abstract: Endemic issues of sustainability in the Water, Sanitation and Hygiene (WASH) sector have led to the rapid expansion of ‘system approaches’ to for assessing the multitude of interconnected factors that affect WASH outcomes. However, the sector lacks a systematic analysis and characterization of the knowledge base for systems approaches, in particular how and where they are being implemented and what outcomes have resulted from their application. To address this need, we conducted a wide-ranging systematic literature review of systems approaches for WASH across the peer-reviewed, grey, and organizational literature. Our results show a myriad of methods, scope, and applications within the sector, but an inadequate level of information in the literature to evaluate the utility and efficacy of systems approaches for improving WASH service sustainability. Based on this analysis, we propose four recommendations for improving the evidence base including; diversifying methods that explicitly evaluate interconnections between factors within WASH systems, expanding geopolitical applications, improving reporting on resources required to implement given approaches, and enhancing documentation of effects of system approaches on WASH services. Overall, these findings provide a robust survey of the existing landscape of systems approaches for WASH and propose a path for future research in this emerging field.

Keywords: WASH; systems approaches; systematic literature review; grey literature

Supplementary Information

SI 1: PRISMA CHECKLIST

Table S1-1: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Checklist

REPORTED ON SECTION/TOPIC # CHECKLIST ITEM PAGE # TITLE Title 1 Identify the report as a systematic review, meta-analysis, or both. Title ABSTRACT 2 Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, Structured participants, and interventions; study appraisal and synthesis Abstract summary methods; results; limitations; conclusions and implications of key findings; systematic review registration number. INTRODUCTION 3 Describe the rationale for the review in the context of what is Rationale Introduction already known. 4 Provide an explicit statement of questions being addressed with 1.1 Study Objectives reference to participants, interventions, comparisons, outcomes, objectives and study design (PICOS). METHODS 5 Indicate if a review protocol exists, if and where it can be accessed Protocol and (e.g., Web address), and, if available, provide registration 2. Methods registration information including registration number. 6 Specify study characteristics (e.g., PICOS, length of follow-up) and Eligibility criteria report characteristics (e.g., years considered, language, publication 2.1 Literature ID status) used as criteria for eligibility, giving rationale. 7 Describe all information sources (e.g., databases with dates of Information coverage, contact with study authors to identify additional studies) 2.1 Literature ID sources in the search and date last searched. 8 Present full electronic search strategy for at least one database, 2.1 Table 1, search Search including any limits used, such that it could be repeated. string formula 9 State the process for selecting studies (i.e., screening, eligibility, 2.2 Selection of Study selection included in systematic review, and, if applicable, included in the articles meta-analysis). 10 Describe method of data extraction from reports (e.g., piloted Data collection 2.2 Data forms, independently, in duplicate) and any processes for process extraction obtaining and confirming data from investigators. 11 List and define all variables for which data were sought (e.g., 2.2.1; 2.2.2 Data items PICOS, funding sources) and any assumptions and simplifications Screening criteria made. 12 Describe methods used for assessing risk of bias of individual Risk of bias in studies (including specification of whether this was done at the 2.2.2.3 individual studies study or outcome level), and how this information is to be used in Application any data synthesis. Summary 13 State the principal summary measures (e.g., risk ratio, difference in N/A measures means). 14 Describe the methods of handling data and combining results of Synthesis of studies, if done, including measures of consistency (e.g., I2) for N/A results each meta-analysis. 15 Specify any assessment of risk of bias that may affect the Risk of bias across Conclusions / cumulative evidence (e.g., publication bias, selective reporting studies Limitations within studies). 16 Describe methods of additional analyses (e.g., sensitivity or Additional subgroup analyses, meta-regression), if done, indicating which N/A analyses were pre-specified. RESULTS 17 Give numbers of studies screened, assessed for eligibility, and Study selection included in the review, with reasons for exclusions at each stage, 3.1 Search Results ideally with a flow diagram. 18 For each study, present characteristics for which data were Study extracted (e.g., study size, PICOS, follow-up period) and provide SI Table characteristics the citations. Risk of bias 19 Present data on risk of bias of each study and, if available, any N/A within studies outcome level assessment (see item 12). 20 For all outcomes considered (benefits or harms), present, for each Results of study: (a) simple summary data for each intervention group (b) N/A individual studies effect estimates and confidence intervals, ideally with a forest plot. Synthesis of 21 Present results of each meta-analysis done, including confidence N/A results intervals and measures of consistency. Risk of bias across 22 Present results of any assessment of risk of bias across studies (see N/A studies Item 15). Additional 23 Give results of additional analyses, if done (e.g., sensitivity or N/A analysis subgroup analyses, meta-regression [see Item 16]). DISCUSSION 24 Summarize the main findings including the strength of evidence Summary of for each main outcome; consider their relevance to key groups 4. Discussion evidence (e.g., healthcare providers, users, and policy makers). 25 Discuss limitations at study and outcome level (e.g., risk of bias), Limitations and at review-level (e.g., incomplete retrieval of identified 4.4 Limitations research, reporting bias). 26 Provide a general interpretation of the results in the context of Conclusions 5. Conclusions other evidence, and implications for future research. FUNDING 27 Describe sources of funding for the systematic review and other Funding Funding support (e.g., supply of data); role of funders for the systematic Statement review.

From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097 . For more information, visit: www.prisma-statement.org.

SI 2: SEARCH STRATEGY

Table S2-1: Databases Included in Literature Search

Database Literature Search Database Name Description Rationale Type Types Results Articles from ASCE Preliminary reviews showed journals, papers that it contained many ASCE Research from conference sources of complex systems Organization Peer-reviewed 821 Library proceedings, and e- analysis for water and books and wastewater service delivery. standards. Includes documents from documents from the Gillings Repository of School of Global Public University of North Health at UNC and The Peer-reviewed Carolina Digital Carolina Academic Water Institute at UNC, Grey 13 Repository publications Institution conveners of the Water & Thesis (student, thesis, Health Conference, one of etc..). the largest annual WASH conferences. Database of Deutsche Deutsche Gesellschaft für One of the larger and most Gesellschaft für Internationale innovative bilateral Grey Internationale Zusammenarbeit organizations working in the Organization Organizational 109 Zusammenarbeit GmbH (GIZ), WASH sector worldwide. Open Source (GIZ) Library German Also, open source resource. development agency. Database of The Department for International A large funder of WASH Grey Development projects worldwide and early DFID Organization Organizational 1 (DFID), which adopter of systems Open Source manages the UK approaches to development Government's aid programs. Provider of 375 full- One of the largest and most text databases for e- comprehensive databases EBSCO Provider Peer-reviewed 4,129 journals, magazine covering WASH and systems subscriptions, research topics. ebooks and discovery service for academic libraries, public libraries. Eldis repository of Database focused specifically the Institute of on international Development development topics, Grey ELDIS (Int’l Studies managed including WASH, with a Provider Peer-reviewed 151 development) by Knowledge, range of peer-reviewed and Open Source Impact and Policy grey literature that is open team. source. Repository of The International Rescue Committee International IRC is a large donor and (IRC), a global Rescue implementer in the WASH humanitarian aid, Organization Organizational 44 Committee sector, including in systems relief, and Library approaches to WASH. development nongovernmental organization. Library of IRC WASH - formerly the WHO Database consists of International publications by IRC WASH IRC WASH Reference Center and others. One of the largest Organization Organizational 1,357 on Community repositories of WASH Water Supply systems-focused work. (abbreviated as I.R.C.). Includes documents from Loughborough’s Water, Repository of Engineering and Loughborough Loughborough Development Centre University University Academic Peer-reviewed (WEDC), an innovative 27 Institutional publications Institution Thesis WASH research center and Repository (student, thesis, convener of a large annual etc..). conferences on innovative directions in WASH. OECD iLibrary is the online library of OECD is a large funder of the Organisation innovative research and for Economic Grey OECD iLibrary projects in the infrastructure Organization 13 Cooperation and Organizational services sector, including Development water supply and sanitation. (OECD) featuring its books, papers and statistics and is the gateway to OECD's analysis and data. International coverage of public affairs, public and Provider of peer-reviewed- social policies, and reviewed and grey literature PAIS international from countries outside of Peer-reviewed Provider 518 International relations. It North America and Europe, Grey includes two focused on international databases, PAIS topics, including WASH. International and PAIS Archive. Used to specifically search for dissertations on WASH ProQuest ProQuest for systems topics which may Dissertations Provider Thesis 376 Dissertations have not been disseminated and Theses through peer-reviewed reviewed or grey sources. SciELO is a bibliographic database, digital Large database representing library, and non-English language cooperative publications, including non- SciELO electronic Provider Peer-reviewed 244 WASH topics, which may publishing model not appear in other journals of open access or databases. journals. Primarily with non-English publications Library, project database and Organizational and grey discussion forum literature specifically focused for the Sustainable Grey SuSanA on emerging sanitation Organization 46 Sanitation Alliance, Organizational topics, both from the SuSanA an informal and its members. network of people and organizations USF is home to the International Development Repository of USF University of Engineering Program, a publications Academic Peer-reviewed South Florida teaching and research center 24 (student, thesis, Institution Thesis Repository focused on new directions etc..). and approaches to international development. A multi-donor partnership, part of Leading producer of research Water and the World Bank and practice in water and Peer-reviewed Sanitation Group's Water sanitation topics. Early Organization Grey 105 Program (WSP) Global Practice adopter of systems Organizational Library focused on water discourses in WASH. and sanitation. A global, multi- stakeholder membership and partnership Water Supply & Similar to WSP, a large organization that Sanitation producer of research and works with poor Collaborative practice documentation for people, Organization Grey 35 Council WASH, including innovative organizations, (WSSCC) approaches to service governments and Resources delivery. local entrepreneurs to improve sanitation and hygiene at scale. Focus on multidisciplinary Premier research on WASH topics to multidisciplinary augment engineering- and Web of Science index covering the Provider Peer-reviewed 379 public health-focused journal literature of databases included in the the sciences. literature review World Bank is one of the largest legacy funders of WASH projects worldwide. Collection of World World Bank This database contains all Bank publications Organization Organizational 670 eLibrary project reports and since the 1990s. evaluations on WASH projects from the bank that have been made public.

Table S2-2: WASH and Systems Keywords and Search String Formula

WASH Keywords Systems Keywords Search String actors adapt "drinking water" OR "hand washing" OR drinking water alliance "hygiene" OR "latrine" OR "sanitation" OR "toilet" hand washing approach OR "WASH" OR "water access" OR "water hygiene coalition management" OR "water quality" OR "water latrine complex resource" OR "water service" OR "water source" sanitation connect OR "water supply" OR "water use" OR toilet dynamics WASH emergent AND water access factors water management feedback "actors" OR "adapt" OR "alliance" OR "approach" water quality holistic OR "coalition" OR "complex" OR "connect" OR water resource inter "dynamics" OR "emergent" OR "factors" OR water service network "feedback" OR "holistic" OR "inter" OR "network" water source scenario OR "scenario" OR "social" OR "stakeholder" OR water supply social "system" OR adapt* OR complex* OR connect* OR water use stakeholder dynam* OR inter* OR system* OR system Table S2-3: Initial Screening Exclusion Terms and Criteria

Exclusion Term Exclusion Criteria Non-WASH Study is not applicable to any aspect of the Water, Sanitation & Hygiene Sector. Study of water treatment or water quality including processes, technologies, or Water Treatment microbiology (e.g. evaluation of a ceramic filter design for use in rural communities). Study of wastewater treatment or water quality including processes, technologies, or Wastewater Treatment microbiology (e.g. evaluation of an eco-san toilet design). Study of physical hydrology of water resources including drinking water, wastewater Hydrology and non-consumption water. Water Resource Study related to the management of water, land and other natural resources for non- Management consumptive use including irrigation, stream by flows or hydropower.

Public Health Study focused on public health impacts of a WASH program / intervention but does not (WASH) investigate multiple factors or attributes of that impact.

Study represented reviews or a meta-analysis of existing literature on a WASH topic. Meta-Analysis Full text documents for references labeled as meta-analysis were obtained and searched for additional studies which may meet eligibility criteria.

Study is related to an aspect of WASH but does not describe an analysis or process that investigates two or more aspects, factors or attributes of the WASH context. (i.e. post- Non-Systems project report evaluating the effectiveness of a sanitation intervention program, cost- effectiveness analysis).

A study which analyzes the correlations between multiple variables against a chosen Descriptive Statistics WASH outcome, but which is decidedly not ‘systematic’ in nature

SI 3: WORKSHEET AND CODING FORM

Table S3-1: Coding Form

Attribute Descriptors Evaluation Criteria

Stated name of method of analysis or approach employed by the Method2 study (open response) and open-coding of methodologies Low - Non-computational tasks can be completed without specialized knowledge or training Analytical Medium - Some specific knowledge or training is required to Complexity1 complete computational tasks High - Process requires a high level of specialized knowledge or training to conduct the analysis Does the method explicitly consider interactions among factors? Interactions1 (Yes / No) Open coding of source of data for the study (e.g. surveys, Sources of data2 interviews, focus groups, observations, etc.) Open coding of factors included in the study and count (e.g. Factors2 Methodology finance, hardware, regulations) Analysis – Application of an established analytical method, presented without a broader theory on how it should be applied, or steps for applying it (e.g. system dynamics modeling of rural water functionality) Tool – A discrete, standalone activity or analysis presented with sufficient detail to be readily replicated; or, an analytical program Method or software (e.g. asset management tool for water utilities) Application1 Framework – A guiding outline of activities, analyses or procedures for applying the method (e.g. Sanitation Cityscape Conceptual Framework) Approach – A theoretical or conceptual construct, without discrete steps for implementation of the method (e.g. Livelihoods Approach) Aspects of Water, Sanitation and/or Hygiene WASH1 Emergent coding of study outcomes or dependent variable (e.g. Outcome Scope2 Scope behavior change, service sustainability) Rural / Urban Context1 Local (incl community), City, Regional (incl District), National, Sector Assessment of what stage, if any, the study was implemented in relation to a WASH infrastructure or services project: Planning – Used to plan future WASH service delivery project; Implementation – Method used to guide the implementation of a Project Application WASH project Application1 Evaluation – Post-project assessment of outcomes Case Study – Standalone analysis of a specific case(s), not related to a WASH project None – No project application conducted

Int. J. Environ. Res. Public Health 2019, 16, x; doi: FOR PEER REVIEW www.mdpi.com/journal/ijerph Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 2 of 40

Name and count of geographic locations where study was applied Location1

Stated outcomes or effects on WASH services that occurred as a result of the application of the study (if any) Reporting of Impacts2

Secondary Screening Criteria Coding Form including Attributes, Descriptors and Evaluation Criteria. Notes: 1: a-priori, deductive criteria; 2: open or emergent criteria

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 3 of 40

Table SI3-2: Literature Search Worksheet

Document Information Method Scope Application Resources

Understanding handpump sustainability: Social M. Fisher, K. Shields, T. Chan, E. Determinants of Rural Financial Christenson, R. Cronk, H. Leker, Water Resources Greater Afram Plains 2 Hand Search water source 2015 Peer X Bayesian Networks High Yes Secondary Data Technical Analysis Water Rural Regional Sustainability Case Study West Africa 1 D. Samani, P. Apoya, A. Lutz, J. Research region of Ghana functionality in the Environmental Bartram Greater Afram Plains Context region of Ghana Assessent of Institutional Sustainability of Rural M. Malik, L. Karangwa, A. Muzula, Surveys Social 5 Hand Search Water, Sanitation, and J. Sano, J.M. Rutaganda, G. 2016 RWSN Peer X Composite Scoring High No Interviews Tool WASH Rural Regional Sustainability Evaluation Rural Rwanda East Africa 1 4 months $85,000 Financial Hygiene Interventions in Musabyimana Secondary Data Rwanda Technical Governance Against the Current: How Institutional Interviews to Shape an Enabling Capacity Secondary Data Regional Environment for H. Hima, C. Santibanez, H. Kida, Global Delivery Qualitative Data Services Rural 6 Hand Search 2015 Grey X High No Primary Data Analysis Water National Sustainability Evaluation Nigeria West Africa 1 Sustainable Water K. Bain Initiative Analysis Social Urban Observation Sector Service Delivery in Financial Focus Groups Nigeria Political Coordination Institutional International Secondary data Rural Water Supply and AHP Social Symposium on the Primary Data Water Local 7 Hand Search Sanitation in Developing S. Bhattarai and M. Starkle 2005 Peer X Multi-Criteria Medium No Environmental Analysis Rural Sustainability Planning Nepal South Asia 1 Analytic Hierarchy Surveys Sanitation Regional Countries Decision Analysis Financial Process Expert Opinion Technical Technical Analytical Heirarchy International Expert Opinion Institutional AHP Water 13 districts of Nepal Process for Rural Micro Symposium on the Interviews Social Rural 8 Hand Search S. Bhattarai and B. Adhikari 2005 Peer X Multi-Criteria Medium No Tool Sanitation Local Sustainability Evaluation (4 terai, 9 hill and 3 South Asia 13 Project Sustainability Analytic Hierarchy Focus Groups Environmental Urban Decision Analysis Hygiene urban projects). Monitoring in Nepal Process Observation Financial Behavioral Building environments to Institutional support sustainability of Financial Lao improved sanitation Framework National 9 Hand Search N. Mukherjee 2016 Practical Action Grey X High No Secondary Data Capacity Framework Sanitation Rural Sustainability Case Study Vietnam South Asia 3 behaviours at scale: Theory of Change Sector Social Indonesia levers of change in East Asia Monitoring Cambodia Indonesia Lao Policy Philippines Going beyond mason Social Vietnam training: enabling, Infrastructure South Asia Market Based- Peru facilitating, and engaging Financial National Southeast Asia 10 Hand Search J. Dumpert and E. Perez 2015 Waterlines Peer Approach Low No Secondary Data Framework Sanitation Rural Sustainability Evaluation Ethiopia 22 Rural sanitation markets Coordination Sector Kenya East Africa Framework for the base of the Capacity Tanzania Central Asia pyramid Monitoring Uganda Private Sector Bangladesh India Pakistan

Using Causal Loop Donors Diagramming to Explore Surveys NGOs System Dynamics the Drivers of the Sustainability Interviews Technical 14 Hand Search K. Neely and J. Walters 2016 Peer X Causal Loop High Yes Analysis Water Rural Local Sustainability Case Study Timor-Leste Southeast Asia 1 Sustained Functionality Journal Observation O&M Diagramming of Rural Water Services Expert opinion Financial in Timor-Leste Social First International A Framework to Assess Technical Conference on Observation Sustainability of AHP Environmental Construction In Focus Groups Tool 16 communities in 15 Hand Search Community-based Water K. Panthi and S. Bhattarai 2008 Peer X Multi-Criteria Medium No Social Water Rural Local Sustainability Case Study South Asia 1 Developing Interviews Framework Nepal Projects Using Multi- Decision Analysis Institutional Countries Primary Data Criteria Analysis (ICCIDC–I) Financial Framework Technical A planning-oriented Social Network Economic sustainabiilty Analysis Focus Groups Networks assessment framework N. Starkl, M. Brunner, E. Lopez, J. Tool 19 Hand Search 2013 Water Research Peer Multi-Criteria High No Interviews Environmental Water Urban City Sustainability Planning Xochmilco, Mexico Central America 1 Coordination (POSAF) for peri-Urban Martinez-Ruiz Framework Decision Analysis Surveys Institutional water management in AHP Financial developing countries Statistics Users Technical Management Management of Rural Community Stakeholder International Qualitative Data water services in Social Focus group Journal of Analysis Nicaragua: a systemic Governance sessions 21 Hand Search J. Walters and A. Javernick-Will 2015 Sustainable Peer Social Network High Yes Focus Groups Analysis Water Rural Local Sustainability Case Study Terrabona, Nicaragua Central America 1 network approach to Technical lasting Development & Analysis Environmental evaluating stakeholder between 2 World Ecology System Dynamics Functionality alignment and 3 hours Financial Coordination Long-Term Functionality Functionality of Rural Water Services Governance in Developing Countries: Financial Environmental A System Dynamics System Dynamics Secondary Data Technical 22 Hand Search J. Walters and A. Javernick-Will 2015 Science & Peer High Yes Analysis Water Rural Local Sustainability Case Study Approach to Expert Opinion Environmental Technology Understanding the Energy Dynamic Interaction of Community Factors Management Exploring the use of Coordination Engineering social network analysis NGOs Project Social Network Interviews Water 23 Hand Search to inform exit strategies J. Walters 2016 Peer High Yes Networks Analysis Rural Local Sustainability Case Study Darío, Nicaragua. Central America 1 Organization Analysis Surveys Sanitation for Rural water and Technical sanitation NGOs Journal Functionality Governance Community Planning Rural water Management services in Nicaragua: A Graphic Modeling Interviews Rural communities in Environmental Financial 25 Hand Search systems-based analysis J. Walters and P. Chinowsky 2016 Peer Statistics High Yes Observation Analysis Water Rural Local Access Case Study Darío, and Terrabona Central America 1 Science & Policy Technical of impact factors using Network Analysis Primary Data in Nicaragua Environmental graphical modeling Energy Functionality Rethinking Sustainabilty, Scaling Up, and Enabling Environment: A U. Amjad, E. Ojomo, K. Downs, R. Implementation Rural 27 Hand Search 2015 Water Peer X Framework Low No Secondary Data Analysis Water Local Sustainability Case Study Framework for Their Cronk, J. Bartram Functionality Urban Implementation in Drinking Water Supply

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 4 of 40

The Political Economy of Brazil South America Sanitation: How can we Secondary Data Social S. Garbarino, J. Holland, S. Brook, Rural National India Southeast Asia 29 Hand Search increase investment and 2011 WSP Org X Framework Medium No Interviews Political Framework Sanitation Sustainability Evaluation 4 K. Caplan, A. Shankland Urban Sector Indonesia South Asia imporove service for the Focus Groups Economic poor Senegal West Africa Institutional Financial Expert Opinion Governance Framework Strengthening Enabling D. Tsetse, E. Kouassi-Komlan, C. Framework Primary Data Rural National 30 Hand Search 2016 UNICEF Org X Medium No Monitoring Approach WASH Sustainability Environment for WASH Scharp, G. Hutton Theory of Change Secondary Data Urban Sector Infrastructure Tool Interviews Planning Services

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 5 of 40

Document Information Method Scope Application Resources

staff Triple-S in

assigned to Ghana for the project IRC have and Triple-S been US$ project staff 3,469,924. The long hired This price Collective Impact Institutional time frame – specifically tag includes Whole system change: Whole Systems Primary Data Financial Local Planning at least six for the all capturing the change Approach Policy 32 Hand Search H. Lockwood and V. Duti 2015 IRC WASH Org X Approach Low No Secondary Data Coordination Water Rural National Sustainability Implementing Ghana West Africa 1 years and project and international process in the Ghana Framework Services Service Delivery Focus Groups Governance Sector Evaluation potentially hosted by staff support, Rural water sub-sector Approach Regulations up to ten CWSA. but does not years A much include other larger investments number of made by IRC people also Ghana provided through support to activities the project under on a part- time basis, WASHCost including and the both Ghana- Tripartite Partnership Services A qualitative Technical C. Chatterley, A. Javernick-Will, K. Interviews 16 case schools in comparative analysis of QCA Community 37 Hand Search Linden, K. Alam, L. Bottinelli, M. 2013 BioMed Central Peer X High No Focus Groups Analysis Sanitation Rural Local Sustainability Case Study Meherpur, South Asia 1 well-managed school Qualitative Governance Venkatesh Observation Bangladesh sanitation in Bangladesh User Committees O&M Institutional Capacity Community Assessment International Technical Model for Sustainable Council on Framework Expert Opinion Economic 41 Hand Search Municipal Sanitation G. Louis and M. Bouabid 2004 Peer Medium No Tool Sanitation Local Sustainability Evaluation Systems Composite Scoring Secondary Data Financial Services in low income Engineering Energy communities Environmental Cultural Functionality A decision model for Institutional selecting sustainable Capacity drinking water supply Technical Journal of Interviews and greywater reuse Economic Planning 42 Hand Search J. Henriques and G. Louis 2010 Environmental Peer X Composite Scoring High Yes Secondary Data Tool Water Urban Local Sustainability Cimahi, Indonesia Southeast Asia 1 systems for developing Financial Case Study Management Observation Energy communities with a case Environmental study in Cimahi, Social Indonesia Cultural Tanzania (10 Introducing SaniFOAM: Technical districts), Indonesia A Framework to Analyze Financial (East Java), and India East Africa Approach Rural Local Planning 43 Hand Search Sanitation Behaviors to J. Devine 2009 World Bank Grey X Framework Medium No Secondary Data Behavioral Sanitation Sustainability (in two states– South Asia 3 Framework Urban City Implementation Design Effective Regulations Madhya Pradesh and Southeast Asia Sanitation Programs Social Himachal Pradesh).

Applied in 30 districts in 5 countries. Results presented from; Services Kamwenge District, A Suite of Tools to Institutional Uganda Support a Systems- East Africa Interviews Financial Implementation Praganas District, Based Approach to Tool Rural South Asia 44 Hand Search K. Lemme, K. Latham, K. Kugler 2016 Water For People Org X Composite Scoring Medium No Primary data Environmental Water Regional Sustainability Evaluation India 30 Financial Sustainable Framework Urban Central America Secondary Data Management Case Study (District name?), Peru Management of Water South America O&M (rural district), Malawi Service Delivery Monitoring Sagar Island, India Rulindo, Rwanda San Rafael del Norte, Nicaragua El Negrito, Honduras Policy Synthesis of Four Coordination Country Enabling Institutional Peru South America Expert Opinions Planning Environment Capacity Sanitation Senegal West Africa Behavior Change 47 Hand Search F. Rosenweig 2008 World Bank Grey X Composite Scoring Medium No Observations Tool Rural National Behavior Change Implementation 4 Assessments for Scaling Technical Hygiene Tanzania East Africa Uptake Secondary Data Evaluation Up Handwashing Financial Vietnam Southeast Asia Programs Implementation Monitoring Governance Financial 3-10 Governence into Local Expert Water 50 Hand Search S. Ogden 2013 CARE Grey X Composite Scoring Medium No Management Tool Local Sustainability Evaluation community Functionality (GIFT) Tool Opinion Sanitation Users members Functionality Gender Technical Social Surveys Eighteen Framework Hygiene Methodology for Interviews Approach assessments across (no mention of R. Dayal, C. van Wijk, N. Statistics Participation Water Planning 52 Hand Search Participatory 2000 WSP, IRC WASH Org X High No Observation Tool Local Sustainability 18 communities in 15 locations in 18 Mukherjee Participatory Institutional Sanitation Evaluation Assessment (MPA) Primary Data Framework countries. (no report) Composite Scoring Users Secondary Data Social mention of locations) Community Policy Socioeconomic A model for assessing Services sustainability A web- Composite Scoring Environmental based decision support Tool Local Greater Afram Plains 53 Hand Search S. Takougan and J. Wethe 2016 Arima Journal Peer X Multi-Criteria Medium No Expert Opinion Technical Water Rural Sustainability Evaluation West Africa 1 system for predicting and Framework Regional region of Ghana Decision Analysis Financial evaluating sustainability O&M (Toppes) Institutional Access The SIASAR Initiative: Management Observation An Information System Functionality Local Nicaragua SIASAR Primary Data Water 54 Hand Search for More Sustainable L. Pena, D. Michaud, J. Biau 2013 World Bank Grey X Low No Hygiene Tool Rural Regional Sustainability Evaluation Panama Central America 3 Composite Scoring Interviews Sanitation Rural Water and Health Honduras Expert Opinion Sanitation Services Financial O&M

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 6 of 40

chairman of the informal settlement invited the Addressing WaSH research challenges in Pacific Technical team to

Island Countries: A Context visit the S. Saunders, D. Barrington, S. Two-person participatory marketing Habitat Participatory Interviews Networks Water community 56 Hand Search Sridharan, S. Meo, W. Hadwen, K. 2016 Peer X Low No Tool Urban Local Access Case Study Suva City, Fiji Oceania 1 research systems mapping International Systems Mapping Observation Private Sector Sanitation over a two- Shields, R. Souter, J. Bartram team approach to empower Socioeconomic week period informal settlement User Committees to community action specifically undertake the systems mapping activities.

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 7 of 40

Document Information Method Scope Application Resources

Document Document Open Method Analytical Interaction Data Source Method Aspects of Rural / Project Location Location Human Database Title Author(s) Date Publication Factors Context Outcome Scope Location Study Impacts Time Cost ID Type Access Analysis Complexity Analysis Analysis Application WASH Urban Application Region Count Resources Regulations Water and Sanitation Financial Hygiene Bottleneck UNICEF Local Expert Private Sector Tool Rural Sustainability Evaluation 57 Hand Search Analysis Tool 2016 WASH BAT Grey X Composite Scoring Medium No WASH National (Uncredited) Opinion O&M Framework Urban Access Planning (WASH BAT 2.0) User Social Manual Environmental Assessing Women’s Negative Sanitation International Social Experiences and Journal of Interviews Technical B. Caruso, T. Clasen, K. Youn, H. Framework Puri district, Odisha, 59 Hand Search Concerns: The 2017 Environmental Peer X High Yes Focus Groups Technical Tool Sanitation Rural Local Access Case Study South Asia 1 Cooper, C. Hadley, R. Haardörfer Statistics India Development of a Novel Research and Surveys Community Sanitation Insecurity Public Health Behavioral Measure Burkina Faso Dominican Republic Institutional Ethiopia East Africa WASH Sustainability Interviews Management Ghana $15-150k Rural Caribbean 60 Hand Search Index Tool (SIT) J. Annis and L. Moreland 2015 WASH Plus Org X Composite Scoring High No Focus Groups Financial Tool WASH National Sustainability Evaluation Kenya 9 Uptake 2-6 Months Avg - $50k / Urban West Africa Comparative Analysis Secondary Data Environmental Liberia country Southeast Asia Technical Niger Philippines Tanzania Financial Policy Planning Secondary Data Local Sanitation Monitoring UNICEF Composite Scoring Social Tool Access 63 Hand Search 2014 UNICEF Org X Medium No Expert Opinion Sanitation Rural Regional Evaluation Toolkit (Uncredited) Framework O&M Framework Sustainability Interviews National Institutional Private Sector Coordination Modeling the Complexities of Water, Environmental Environmental J. Mellor, J. Smith, G. Learmonth, (2) communities in 64 Hand Search Hygiene, and Health in 2012 Science & Peer X System Dynamics High Yes Secondary Data Technical Analysis Water Rural Local Health Case Study South Asia 2 V. Netshandama, R. Dillingha South Africa Limpopo Province, South Technology Technical Africa Community Networks Management Developing A Systems Social Network Interviews NGOs Understanding of Rural Analysis Analysis Access (5) Rural villages 65 Hand Search K. Neely and J. Walters 2015 Deaking University Peer X High Yes Observation Coordination Water Rural Local Case Study Southeast Asia 5 Water Supply in Timor- Causal Loop Approach Sustainability Timor-Leste Focus Groups Social Leste Diagramming Economic Environmental O&M O&M A systems approach to Private Sector Bugiri and sustainable water Market-Based Planning 66 Hand Search N. Mary 2016 RWSN Peer X Low Yes Governance Tool Water Rural Regional Sustainability Namayingo districts, East Africa 2 Services operation & maintenance approach Evaluation Regulations Uganda in Uganda User Committees Decision-making on International Financial shared sanitation in the S. Simiyua, M. Swilling, S. Journal of Qualitative Data Services Kisumu (informal 70 Hand Search 2017 Peer High No Interviews Analysis Sanitation Urban Local Sustainability Case Study East Africa 1 informal settlements of Cairncross Environmental Analysis O&M settlements) Kenya Kisumu, Kenya Health Research Socioeconomic Policy System Dynamics Management Tipping Points in Causal Loop Interviews Community 72 Hand Search Community-Based G. Guerrero 2016 TU Delft Peer X High Yes Tool Water Rural Regional Sustainability Case Study Kabarole, Uganda East Africa 1 Diagramming Secondary Data Financial Management Environmental Access Community Implementing Planning sustainable water and R. Barnes, N. Ashbolt, D. Roser, Qualitative Data Social Water 73 Hand Search sanitation projects in 2014 Waterlines Peer X Medium No Interviews Analysis Rural Local Sustainability Case Study P. Brown Analysis Implementation Sanitation Rural, developing Technical communities Governance Capacity The Role of Learning in Financial Water Supply Systems: System Dynamics Functionality Regional 74 Hand Search Analysing the Spread of F. Knipschild 2016 TU Delft Peer X Agent Based High Yes Secondary Data Coordination Tool Water Rural Sustainability Case Study Uganda East Africa 1 National Policies in Rural Uganda Modeling Education using Memetics Technical O&M Community Revealing causal Journal of Water, Technical 26 projects Qualitative pathways to sustainable Sanitation and Secondary Data User Committees implemented by EWB 77 Hand Search K. Gasparro and J. Walters 2017 Peer Comparative High No Analysis Water Rural Local Sustainability Case Study 26 water service delivery Hygiene for Interviews Financial across multiple Analysis using fsQCA Development Coordination countries Education Technical Agriculture Improving the IRC WASH Financial Uganda East Africa robustness of financial Symposium 2010: Planning 78 Hand Search J. Parkinson and S. Blume 2010 Org X Financial Analysis Low No Secondary Data Health Analysis Sanitation Local Sustainability South Africa South Africa 3 and economic analysis Pumps, Pipes and Case Study Implementation Burkina Faso West Africa of sanitation systems Promises Environmental Economic Whole Systems Financial Approach Focus groups Towards systemic Institutional Local Service Delivery Interviews Approach Planning Ghana West Africa Users 80 Hand Search change in urban G. Galli, C. Nothomb, E. Baetings 2014 IRC WASH Org X Medium No Environmental Sanitation Urban Regional Sustainability 2 Approach Primary data Framework Implementing Uganda East Africa Services sanitation Technical National Shit Flow Diagrams Secondary data Theory of Change Social Political Coordination Framework Institutional Local Ghana The Triple-S Theory of Service Delivery Monitoring Regional Planning Uganda East Africa 81 Hand Search T. Schouten and P. Moriarty 2013 IRC WASH Org X Low No Secondary Data Approach Water Rural Sustainability 4 Change Approach Capacity National Implementing Mozambique West Africa Theory of Change Policy Sector Burkina Faso Financial Planning Climate Systems Approach to Environmental Technical Climate, Water, and J. Mellor, E. Kumpel, A. Ercumen, Agent Based Primary data Tool Hubli-Dharwad in 82 Hand Search 2016 Science & Peer High Yes Environmental Water Urban City Health Case Study South Asia 1 Diarrhea in Hubli- J. Zimmerman Modeling Secondary data Framework Karnataka, India Technology Health Dharwad, India Services

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 8 of 40

Document Information Method Scope Application Resources

San Pedro, Bolivia work to work to

complete the complete the Tiraque, Bolivia Villa Rivero, Bolivia SSC. SSC. San Andres Timing Timing Sajcabaja, Guatemala Activity Activity San Antonio At least 2 At least 2 Institutional Illotenango, weeks weeks Financial South America Guatemala before the before the Sustainable Services Water For People Interviews Management Water Rural Central America 84 Hand Search 2018 Water For People Org Composite Scoring Low No Tool Regional Sustainability Evaluation San Bartolome, 26 interview interview Checklist (Uncredited) Secondary Data Monitoring Sanitation Urban East Africa Guatemala Country Country O&M South Asia Santa Cruz del Program Program Environmental Quiche, Guatemala staff should staff should Chinda, Honduras share share El Negrito, Honduras relevant relevant San Antonia, background background Honduras information information La Concordia, with the with the Nicaragua interviewer. interviewer. San Rafael del Norte, Background Background Nicaragua information information Asuncion, Peru could could Cascas, Peru include include Priority Addressment O&M Protocol: Understanding Environmental Interviews Technical A. Davis, A. Javernick-Will, S. Framework 20 slum communities 85 Hand Search the Ability and Potential 2019 Science & Peer High No Focus Groups Social Framework Sanitation Urban City Sustainability Evaluation South Asia 1 Cook AHP in India of Sanitation Systems to technology Primary Data Financial Address Priorities Community Financial DWA Sustainability Surveys Institutional Dutch Water Alliance, IRC Dutch Water Local Uganda 86 Hand Search Monitoring Framework – 2014 Org X Composite Scoring Low No Focus Groups Environmental Tool WASH Rural Sustainability Evaluation East Africa 2 (Uncredited) Alliance Regional Ghana The Guide Secondary Data Technical Social Characteristics of Stakeholder Networks Supporting Local (7) local governments Networks Government in South America D. McNicholl, A. McRobie, H. Social Network Institutional Local 88 Hand Search Performance 2017 Water Alternatives Peer High Yes Interviews Analysis Water Rural Coordination Case Study (2) Ghana East Africa 7 Cruickshank Analysis Governance Regional Improvements in Rural (2) Malawi West Africa Coordination Water Supply: Cases (3) Bolivia from Ghana, Malawi, and Bolivia Monitoring Bangladesh Coordination Ghana An evaluative framework Water and Regulations South Asia Two-day Workshop Framework National Evaluation Kenya Coordination 89 Hand Search for urban WASH sector S. Drabble, R. Renoug, J. Stokes 2018 Sanitation for the Org X Composite Scoring Low No Financial WASH Urban Policy West Africa 6 long Surveys Tool Sector Planning Madagascar Policy functionality Urban Poor Capacity East Africa workshop Mozambique Private Sector Policy Zambia Behavioral 20 communities (i.e., Education cases) in Karnataka The use of qualitative O&M or Tamil Nadu, India comparative analysis to Qualitative Interviews Community were selected to A. Davis, A. Javernick-Will, S. Science of the 90 Hand Search identify pathways to 2019 Peer Comparative High Yes Observations Infrastructure Analysis Sanitation Urban Local Functionality Case Study ensure variability South Asia 20 Cook Total Environment successful and failed Analysis Primary Data Governance between outcomes sanitation systems Planning and factors Financial (i.e., causal conditions) Policy Institutional Infrastructure Building strong WASH Monitoring Local systems for the SDGs: Environmental Expert Opinion Tool Rural Regional Evaluation 91 Hand Search Understanding the A. Huston and P. Moriarty 2018 IRC WASH Org X Building Blocks Low Yes Regulations WASH Sustainability Secondary Data Framework Urban National Planning WASH system and its Financial Planning Sector building blocks Education Health Management Access Infrastructure Financial Primary Data Implementation Rural Evaluation 92 Hand Search Sustainability Framework R. Carter, V. Casey, E. Harvey 2011 WaterAid Org X Framework Low No Framework WASH Local Sustainability Expert Opinion Monitoring Urban Planning Environmental Management Donors Financial Sustainability Snapshot Primary Data Technical Indicator Local Planning 93 Hand Search S. Sugden 2003 WaterAid Org X Framework Low No Surveys O&M Tool Water Rural Sustainability Malawi East Africa 1 Uptake Equity in Distribution Regional Implementation Observations Infrastructure Indicator Access

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 9 of 40

Document Information Method Scope Application Resources

preparation using the

meetings, Water for data Life collection framework NGOs and wrap-up. will cost Coordination Though a between Services Water and Sanitation Secondary Data week is not a $US 15,000- Education Accountyability Forum; Framework Focus Groups substantial 20,000 per Improve Hygiene Framework Water Evaluation 96 Hand Search Organizational F. Rojas and C. Chatterley 2011 Grey X Qualitative Data Medium No Observation Rural Local Sustainability Lempira, Honduras Central America 1 amount of evaluation International Technical Approach Sanitation Case Study Evaluation of Analysis Interviews time for a and require O&M COCEPRADIL Surveys field-based one week of Environmental study, field data Community collection independent Management and three to evaluators five months and Forum total participants including felt that preparation this was and sufficient reporting. time to (Triple-S respond to Summary) the main Thousands of handpumps have been installed across sub- Saharan Africa over the past four decades to

improve rural water

supplies. However, multiple studies have raised concerns over the extent to which these Technical sources are providing O&M safe and adequate Capacity quantities of water post- Environmental Elisabeth S. Liddle and Richard A. Systems Thinking Causal Loop Interviews Framework Access 97 Hand Search construction. A number 2019 Grey Medium Yes Services Water Rural Local Evaluation Uganda East Africa 1 Fenner in WASH Diagrams Expert Opinion Tool Sustainability of factors could be Institutional causing these problems, Financial either directly or Coordination indirectly. Understanding the interrelated nature of these factors and their inherent complexity is crucial if handpump failures are to be avoided in the future. This chapter demonstrates how some of this complexity can be WASH services are influenced by complex stakeholder interactions that need to be understood and

potentially influenced in

order to improve service quality and sustainability. Social network analysis provides an opportunity for mapping and Coordination Ghana East Africa rigorously interpreting Systems Thinking Social Network Institutions Water Rural Local Planning Tajikistan Central Asia 98 Hand Search these stakeholder Duncan McNicholl 2019 Grey High Yes Interviews Tool Coordination 4 in WASH Analysis NGOs Sanitation Urban Regional Evaluating Bolivia South America interactions in order to Networks Uganda West Africa develop and monitor strategic interventions. Studies using social network analysis can be scaled to fit the scope of any project, and the visual nature of social networks helps to make findings accessible to a wide range of stakeholders. This chapter presents

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 10 of 40

Document Information Method Scope Application Resources

assess a system that is

fundamentally in flux. This chapter will look at the challenges of current approaches and provide insights into how to Monitoring restructure the way we Governence Social Network measure our impact on Institutional Systems Thinking Analysis Tool 100 Hand Search whole systems, rather Chris Brown 2019 Grey High Yes Secondary Data Management Water Rural Regional Sustainability Evaluating Tanzania East Africa 1 in WASH Causal Loop Approach than just individual O&M Diagrams processes. This chapter will explain how to adopt a system view of your operating environment, how to map that system to understand its interconnectedness and inherent complexity, and then how to identify the true drivers of the system to allow for the creation of a range of Towards sustainable Social water, sanitation and S. Taylor, S. Asimah, R. Buamah, Economic Burkina Faso hygiene technology use K. Nyarko, S. Sekuma, Y. Environmental Access East Africa EBSCO AP 1 EBSCO 2017 Water Policy Peer X Collective Action High No Interviews Approach WASH Rural Evaluation Ghana 3 3 years in sub-Saharan Africa: Coulibaly, A. Wozuame, P. Jeffrey, Institutional Sustainability West Africa Uganda the Learning Alliance H. Parker Technical approach. Users Access A new approach to Economic nationwide sanitation EBSCO AP S. Kerstens, M. Spiller, I. Science of the Framework Secondary Data Financial Tool Rural EBSCO planning for developing 2016 Peer High Yes Sanitation National Access Planning Indonesia Southeast Asia 1 Uptake 1033 Leusbrock, G. Zeeman Total Environment GIS Expert Opinions Technical Framework Urban countries: Case study of Infrastructure Indonesia. Policy Technical Capacity A novel planning Financial approach for the water, Socioeconomic Water sanitation and hygiene R. Gine-Garriga, D. Requejo, J. Secondary Data 21 Rural districts in EBSCO AP 14 EBSCO 2018 ScienceDirect Peer X Bayesian Networks High Yes Access Tool Sanitation Rural Regional Sustainability Planning East Africa 1 (WaSH) sector: The use Molina, A. Perez-Foguet Expert Opinion Kenya Functionality Hygiene of object-oriented Coordination bayesian networks. Management O&M Access Assessing poverty- Framework Community Annals of the New Secondary Data EBSCO AP alleviation outcomes of Qualitative Data Networks Evaluation EBSCO T. London and H. Esper 2014 York Academy of Peer X High No Interviews Analysis Sanitation Urban City Health Nairobi, Kenya East Africa 1 1619 an enterprise-led Analysis Health Case Study Sciences Observations approach to sanitation. Case Study Hygiene Economic A Holistic Concept to Two slum Design Optimal Water Mathematical Context communities in EBSCO AP Supply Infrastructures L. Rausch, J. Friesen, L. Altherr, East Africa EBSCO 2018 Remote Sensing Peer X optimization High Yes Secondary data Financial Tool Water Urban Local Access Planning Dhaka, Bangladesh, 1 165 for Informal Settlements M. Meck, P. Pelz South Asia Remote sensing Technical Dar es Salaam, Using Remote Sensing Data. Tanzania The three- year project, which two urban slum Community was carried communities in Drinking Water Supply, Hygiene out between central Uganda; Sanitation, and Hygiene Survey Education 2010 and D. Musoke, R. Ndejjo, A. Halage, Journal of Water Planning Kikulu zone in Promotion Interventions Qualitative Data Primary Data Social Analysis Access Services 2013, had EBSCO AP 17 EBSCO S. Kasasa, J. Ssempebwa, D. 2018 Environmental & Peer X Low No Sanitation Urban City Implementation Kawempe division, East Africa 2 in Two Slum Analysis Focus Groups Behavioral Approach Behavior Change Behavior Change three phases: Carpenter Public Health Hygiene Evaluation Kampala district; and Communities in Central Observations Technical preparatory, Kikooza zone in Uganda. Health implementati Mukono Municipality, Capacity on, and evaluation as described below Context Technical Why Do Water and O&M 60 case studies Sanitation Systems for Environmental Hygiene across; Central America EBSCO AP the Poor Still Fail? Policy M. Starkl, N. Brunner, T.A. Case Study Primary Data Water Rural EBSCO 2013 Science & Peer High Yes Community Analysis Local Sustainability Case Study Mexico South Africa 60 2228 Analysis in Economically Stenström Statistics Secondary Data Sanitation Urban Technology Social South Africa South Asia Advanced Developing Financial India Countries. Governance NGOs The role of community participation for Socioeconomic Rufiji (as intervention sustainable integrated Demographic district) and S. Madon, M. Malecela, K. Qualitative Data Surveys neglected tropical Education Approach Planning Mkuranga (used as a EBSCO AP 30 EBSCO Mashoto, R. Donohue, G. 2018 ScienceDirect Peer X Analysis High Yes Interviews WASH Rural Local Health East Africa 1 Health diseases and water, Governance Framework Case Study control) in the Coast Mubyazi, E. Michael Statistics Focus Groups sanitation and hygiene Health (Pwani) Region of intervention programs: A Technical Tanzania pilot project in Tanzania. The social benefits of water and sanitation Municipality of projects in Northern Cost-Benefit Surveys Social EBSCO AP Water Moñitos,Department EBSCO Colombia: Cost-Benefit D. Azqueta, A. Montoya 2017 Wiley Peer Analysis Medium No Interviews Economic Tool Rural Regional Access Case Study South America 1 324 Sanitation of Córdoba, in Analysis, the Water Composite Scoring Secondary data Environmental Northern Colombia Poverty Index and beyond. Health Environmental A framework for planning Economic EBSCO AP of sustainable water and R. Tornqvist, A. Norstrom, E. Water Science & Technical Water EBSCO 2008 Peer Framework Medium No Primary Data Approach City Sustainability Planning 3313 sanitation systems in Karrman, P.A. Malmqvist Technology Social Sanitation peri-urban areas. Cultural NGOs Access Exploring a New Governance Regionalism-Based Policy Society & Natural Interviews Kootenay, British EBSCO AP 41 EBSCO Approach to Managing S.P. Breen 2018 Peer Framework Medium Yes Environmental Approach Water Rural Regional Sustainability Case Study North America 1 Resources Secondary Data Columbia, Canada Drinking Water Systems Economic in Rural Regions. Social Designing a handwashing station for infrastructure-restricted communities in K. Hulland, E. Leontsini, R. Mohammadpur Context EBSCO AP Bangladesh using the Dreibelbis, L. Unicomb, A. Afroz, Behavioral Model Framework Rural Planning (Urban) and EBSCO 2013 BMC Public Health Peer X Medium No Interviews Behavioral Hygiene Local Access South Asia 2 414 integrated behavioural N. Dutta, D. Nizame, S. Luby, P. Qualitative Coding Tool Urban Evaluation Kishoreganj (Rural), Technical model for water, Ram, R. Winch Bangladesh sanitation and hygiene interventions (IBM- WASH)

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 11 of 40

Inclusive water poverty Environmental index: a holistic Access Niabouri, Silly and To EBSCO AP approach for helping Secondary Data Framework Water EBSCO J. Kini 2017 Water Policy Peer Composite Scoring Medium No Management Rural Regional Access Case Study municipalities in West Africa 3 428 local water and Interviews Approach Sanitation Social Burkina Faso sanitation services planning. Financial

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 12 of 40

Document Information Method Scope Application Resources

Demographic A decision support Technical model to aid the Policy EBSCO AP management of crises in Urgan Water Governance Administrative Region EBSCO W. Da Silva, M. Souza 2017 Peer Framework High Yes Secondary Data Framework Water Urban Regional Sustainability Case Study South America 1 508 urban water supply Journal Financial of Brasilia, Brazil systems (the UWC- Context MODEL). Climate Socioeconomic Harnessing the plurality Health of actor frames in social- Interviews Environmental EBSCO AP M. Eelderink, J. Vervoort, D. Snel, Development in EBSCO ecological systems: 2017 Peer X Statistics High No Primary Data Economic Framework Sanitation Rural Local Access Evaluation San Pedro, Bolivia South America 1 511 F.D. Castro Practice ecological sanitation in Observation O&M Bolivia. Social Sustainable community- Technical based drinking water Journal of Water Environmental EBSCO AP M. Aslam, M. Adil, S. Mirza, EBSCO systems in developing 2016 Supply: Research Peer AHP High No Surveys Economic Analysis Water Urban Local Coordination Case Study Peshawar, Pakistan South Asia 1 774 D.Frigon countries: stakeholder and Technology Social perspectives. Institutional Functionality Multiple-use water Community Bashubehar, Magbari supply systems: do the Interviews Financial Access Case Study and Chandaikona; Services IRC 1070 IRC claims stack up? : M. Fontein, J. Webster, P. Trawick 2010 Waterlines Peer Framework Low No Observations Institutional Framework Water Rural Local South Asia 3 Sustainability Evaluation Bogra, north-west Users evidence from Secondary Data O&M Bangladesh. Bangladesh Management Capacity Context Technical Community Management Governance Institutional

Monitoring Between rural and Tanzania Regulations urban: towards Sudan West Africa P. Moriarty, T. Patricot, T.F. O&M Rural Access IRC 1073 IRC sustainable management 2002 IRC WASH Org X Case Study Medium Yes Secondary Data Analysis Water City Case Study Ghana East Africa 5 Bastemeijer, J. Smet, C. Voorden Financial Urban Sustainability of water supply systems Senegal North Africa Private Sector in small towns in Africa Benin Political Capacity Policy Donors Environmental Hygiene Planning Management Capacity Coordination Cultural Economic Donors

Assessing the “Plus” of Australian AID: Secondary Data Financial M. Snehalatha, S. Smits, S. Jasthi, Governance successful community- Department of Qualitative Data Interviews 20 communities in IRC 1075 IRC P. Hutchings,R. Poonia, C. Daniel, 2015 Grey X High No Technical Analysis Water Rural Local Sustainability Case Study South Asia 20 managed water supply Foreign Affairs Analysis Focus Groups India C. Dash Implementation programs in India and Trade (DFAT) Surveys Monitoring Planning Political Private Sector Services Regulations

Technical Regulations Norms and attitudes Economic Addis Ababa, Ethiopia towards ecosan and EcoSanRes Expert Opinion Health Cuernavaca, Mexico other sanitation systems Stockholm Qualitative Data Secondary Data Cultural Kisumu, Kenya East Africa IRC 1077 IRC J.-O. Drangert 2004 Grey X High No Analysis Sanitation Urban City Behavior Change Case Study 5 : desk study by a group Environment Analysis Observations Social Dar es Salaam, Central America Behavioral of experts on ecological Institute Interviews Tanzania Management sanitation Kabale, Uganda O&M Networks Community Assessing sustainability Journal of Water, Users of community Sanitation and Composite Scoring Primary Data Governance IRC 1089 IRC management of rural R. Schweitzer, J. Mihelcic 2012 Peer High No Tool Water Rural Local Sustainability Evaluation Dominican Republic Caribbean 1 Hygiene for Statistics Interviews Financial water systems in the Development O&M developing world Functionality Networks Hygiene Environmental Financial EcoSanRes Management Expert Opinion Open planning of Stockholm Multi-Criteria Technical Framework Rural IRC 1122 IRC R. Kvarnström, E. Petersens 2004 Grey X Low No Secondary Data Sanitation Local Sustainability Planning Vadsbro, Sweden North Europe 1 sanitation systems Environment Decision Analysis Social Approach Urban Focus Groups Cultural Institute Institutional O&M Monitoring Policy

O&M Social Research into financial Environmental 19 villages in 5 and institutional Financial Interviews districts in Nepal structures to support the Qualitative Data Functionality IRC 1201 IRC N.P. Kathmandu 2010 WaterAID Org X Medium No Secondary Data Analysis Water Rural Local Sustainability Evaluation Sankhuwa, Sabha, South Asia 19 functionality and Analysis Capacity Observations Institutional Dhading, Baglung, sustainability of rural hill Monitoring Kaski water systems Community Governance Capacity Four pilot regions in Coordination Ghana; Northern, Financial Brong Ahafo and Governance Volta Learning Alliances Secondary Data Implementation Three pilot districts in Learning alliance Qualitative Data Regional Coordination Coordination IRC 122 IRC J. Nkum 2014 IRC WASH Org X Medium No Interviews Policy Approach Water Rural Evaluation Ghana; Gonja East in West Africa 7 approach: final report Analysis National Sustainability Policy Focus Groups Networks the Northern region; Collective Action NGOs Sunyani West in the Participation Brong Ahafo region; Policy and Akatsi in the Political Volta region Economic Coordination Regulations Participation Governance, Implementation sustainability and Secondary Data Capacity IRC 1266 IRC decision making in water M.A. Iribarnegaray, L. Seghezzo 2012 Sustainability Peer X Composite Scoring Medium No Interviews Tool Water Urban City Sustainability Evaluation Salta, Argentina South America 1 Institutional and sanitation Observations Behavioral management systems Social Cultural Governance Networks

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 13 of 40

Document Information Method Scope Application Resources

Functionality Services Institutional Financial Environmental Project Towns in Social Ethiopia

Infrastructure Maksegnit ONEWASH Plus Water Primary Data Management Urban Evaluation Abomsa IRC 153 IRC Sustainability Checks M. Adank, E. Defere, J. Butterworth 2015 IRC WASH Org X Composite Scoring Medium No Tool Sanitation Local Sustainability East Africa 6 Surveys Capacity Rural Planning Sheno First Annual Report WASH Monitoring Welenchiti O&M Adishihu Policy Wukro Participation Economic Hygiene Institutional Access Behavioral Capacity Community Context Coordination

Education

Financial Behavior Change Making cities open Department, Maharashtra (State) 19 cities, 51 councils Surveys Health Behavior Change Uptake defecation free : Urban Development; University, Government of Sanitation Local Planning and 1 corporation in IRC 175 IRC 2016 Grey X Framework Low No Observations Hygiene Approach Urban Access South Asia 1 Services systematic approach in CEPT; Self-Government, All India Maharashtra, India Hygiene City Implementation Maharashtra District, Focus Groups Implementation Sustainability Financial Maharashtra Institute of Local India Management Policy NGOs O&M Participation Planning Political Services Social 30 villages across 8 countries; Bolivia Multiple-use services as South America Surveys Technical Colombia an alternative to rural East Africa S. Smits, B. van Koppen, P. Framework Interviews Institutional Access Ethiopia IRC 253 IRC water supply services : a 2010 Water Alternatives Peer X Medium No Analysis Water Rural Local Case Study South Africa 8 Moriarty, J. Butterworth Case Study Observations Financial Sustainability India characterisation of the South Asia Focus Groups Environmental Nepal approach Southeast Asia South Africa Thailand Zimbabwe Access A livelihood approach to International Agriculture East Africa water interventions in J.M. Faurès, G. Santini, A.N. Symposium of Context West Africa IRC 278 IRC rural areas and 2008 Peer X Framework Low No Secondary Data Framework Water Rural Local Access Framework Sub-Saharan Africa 4 Villemarceau Multiple-Use Economic Middle Africa implications for Multiple Water Services Climate South Africa Use Systems Technical Community Services Capacity A service delivery Monitoring Framework approach for rural water University of Policy Building Blocks Secondary Data IRC 330 IRC supply in Timor-Leste : J. Willetts 2012 Technology Grey X High No Technical Framework Water Rural Local Sustainability Framework Timor-Leste Southeast Asia 1 Qualitative Data Interviews institutional options and Sydney Regulations Analysis Education strategy Infrastructure Financial Governance

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 14 of 40

Access Agriculture Capacity Climate Community Context Coordination

Demographic

Donors

Economic Adopting a sustainable Education livelihoods approach to Overseas Sustainable Environmental Rural IRC 348 IRC water projects : A. Nicol 2000 Development Grey X Livelihoods Medium Yes Expert Opinion Framework Water Local Access Implementation Financial Urban implications for policy Institute Framework Gender and practice Governance Institutional Management NGOs Participation Policy Political Private Sector Regulations Services

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 15 of 40

Document Information Method Scope Application Resources

Technical A life-cycle approach to Environmental improve the Institutional Lifecycle Analysis Jhumka, Indrapur, sustainability of rural S.A. Jones, A. Anya, N. Stace, L. Community Framework IRC 356 IRC 2012 Challenges Peer X Framework Medium No Secondary Data Water Rural Local Sustainability Planning and Panchakanya in South Asia 3 water systems in Weir Management Approach Decision Trees Nepal resource-limited Users countries Financial Health A one-day interactive community workshop ensures that the

Technical communities’ Costa Rica views and Integrate at the top, Users Burkina Faso Central America knowledge is involve at the bottom – Social Expert Opinion Kenya West Africa recognised IRC 362 IRC the household-centred C. Lüthi, A. Morel, E. Tilley 2008 IRC WASH Peer X Framework Medium No Financial Approach Sanitation Urban Local Access Evaluation 6 Uptake Workshop Tanzania East Africa and forms a approach to Institutional Laos South Asia valuable environmental sanitation Access Nepal input to the Governance assessment report that follows in the third planning step.

Demographic Economic Cultural The study was The effectiveness and Social carried out in three Journal of Water, sustainability of two Case Study Surveys Community districts of Sanitation and Sanitation Behavior Change IRC 364 IRC demand-driven L. Whaley, J. Webster. 2011 Peer Qualitative Data Medium No Interviews Health Analysis Rural Local Evaluation Zimbabwe:Chiredzi East Africa 3 Behavior Change Hygiene for Hygiene Sustainability sanitation and hygiene Analysis Focus Groups Hygiene district, Chipinge Development Financial approaches in Zimbabwe district and Mutoko Climate Infrastructure district NGOs Services Institutional Capacity A life-cycle capacity- Technical based approach to Structure and Economic Manila, Philippines East Africa IRC 372 IRC allocating investments in G.E. Louis, L. Magpili 2007 Infrastructure Peer Composite Scoring High No Secondary Data Tool Sanitation Urban Local Sustainability Case Study 2 Financial Mombasa, Kenya Southeast Asia municipal sanitation Engineering Energy infrastructure Environmental Social Cultural Access Services From beneficiaries to Private Sector businesses to the big Community Interviews picture: Monitoring for IRC, Water For NGOs Rural Local IRC 373 IRC D. Sparkman 2013 Grey X Composite scoring Low No Observations Tool Sanitation Sustainability Case Study Blantyre, Malawi East Africa 1 sustainability in market- People Infrastructure Urban Regional Surveys based approaches to Financial sanitation Regulations Political

Social Economic Health Technical Flores revisited : Environmental sustainability, hygiene Focus Groups Community Method for and use of community- C. Sijbesma, K. Sari, N. Shatifan, Workshops Financial Water 63 Communities in WSP-EAP, IRC Participatory IRC 478 IRC managed water supply R. Walujan, I. Mukherjee, R. 2002 Grey X High Yes Primary Data Management Analysis Sanitation Rural Local Sustainability Evaluation Flores, NTT, Oceania 1 WASH Assessment (MPA) and sanitation and the Hopkins Secondary Data Access Hygiene Indonesia Statistics relationships with project Observations Hygiene approaches an... Behavioral Institutional Participation Gender

Planning The International NGOs Water Association Networks (IWA) Institutional The Swiss Federal Private Sector Institute of Aquatic Policy Science and Sanitation 21 : simple Secondary Data Regulations Technology approaches to complex Primary Data O&M IRC 539 IRC J. Parkinson, C. Lüthi, D. Walther 2006 (Eawag) Grey X Framework Low No Infrastructure Framework Sanitation Urban City Sustainability Planning sanitation : a draft Surveys The Deutsche Technical framework for analysis Focus Groups Gesellschaft für Financial Internationale Services Zusammenarbeit Social (GIZ) Monitoring Implementation Sustainable Capacity Sanitation Alliance External factors Social influencing the International Qualitative data Surveys Political sustainability of micro Journal of Nairobi, Mombasa IRC 575 IRC E. Mukulu, L. Oyugi, S. Mwarania 2011 Peer X analysis High No Interviews Regulations Analysis Water Urban City Sustainability Case Study East Africa 3 and small piped water Humanities and and Kisumu, Kenya Statistics Observations Economic enterprises in Kenya’s Social Science peri-urban areas Private Sector Hygiene Users Factors influencing Behavioral knowledge and practice Financial of hygiene in Water, 12 Villages across six Environmental Sanitation and Hygiene Rural and Remote Qualitative data Interviews Sanitation upazilas in BRAC IRC 601 IRC T. Akter, A. Mehrab 2014 Peer X Medium No Social Analysis Rural Local Behavior Change Case Study South Asia 6 (WASH) programme Health analysis Observation Hygiene programs in Education areas of Bangladesh Bangladesh Participation Rural Advancement Demographic Committee Services

Infrastructure Technical Mexico Promoting harmonized Financial Dominican Republic monitoring for the WASH O&M Costa Rica Central America sector : the Rural Water IRC WASH, World SIASAR Services IRC 63 IRC A. Rodriguez Serrano 2018 Grey X Medium No Survey Tool Water Rural Local Sustainability Evaluation Nicaragua South America 3 Uptake and Sanitation Bank Composite Scoring Access Panama Caribbean Information System Demographic Health Peru SIASAR as an example Hygiene Brazil Capacity

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 16 of 40

Context NGOs Institutional Capacity Sustainable replication Financial Four utilities in and scaling up of small Case Study Interviews Manila, Philippines University of Tokyo Civil Asian Regulations South Asia IRC 636 IRC piped water networks : 2008 Grey X Qualitative Data High No Observations Case Study Water Urban City Sustainability Case Study (2); Ahmedabad, 4 Engineering -Tokyo, JP, UTCE Development Bank Monitoring Southeast Asia technical assistance Analysis Secondary Data O&M India; Tien Giang, consultant’s report Management Vietnam Private Sector Social Services

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 17 of 40

Document Information Method Scope Application Resources

Environmental Factors influencing water Financial system functionality in Environmental Functionality Rural Local 82,503 systems in East Africa IRC 637 IRC Nigeria and Tanzania : a R. Cronk, J. Bartram 2017 Science & Peer Statistics High Yes Secondary Data Agriculture Analysis Water Sustainability Evaluation 2 Urban City Tanzania and Nigeria West Africa regression and Bayesian Technology Management network analysis Technical Context Institutional Organizing framework Capacity for functional national UNICEF, IRC Coordination Framework Rural IRC 70 IRC UNICEF; IRC; Akvo 2016 Grey X Framework Low No - WASH National Sustainability Evaluation WASH monitoring and WASH, Akvo Financial Tool Urban evaluation systems Implementation Monitoring NGOs Users Sustainable community Qualitative data Management Surveys water : managing supply analysis Participation Baglung and Kavre, IRC 879 IRC B.S. Bhandari 2005 Water Policy Peer High No Observation Analysis Water Rural Local Sustainability Evaluation South Asia 1 systems in the mid-hills Statistics Community Nepal Focus Groups of Nepal Composite Scoring Access Services Gender The The monitoring structure of exercise is the groups – scheduled an external during the volunteer,

dry season local partner,

in all Water For countries, as People staff,

we are and Finance: most university The majority Access interested in students of costs are Management Honduras seeing if (when Looking back to move Knowledge Central America borne by Interviews Financial Guatemala water available) forward in strength : Management for Rural South America external IRC 912 IRC K. Fogelberg 2009 Peer X Composite Scoring Low No Observation O&M Tool Water Regional Sustainability Evaluation Bolivia 3 Uptake systems are is created monitoring of water Development Urban West Africa participants, Primary Data Users India able to meet precisely to system sustainability Journal South Asia the World Hygiene Malawi year rounds, encourage Water Environmental needs and this sharing Corps thus, of information, volunteers visit when technical they are expertise, under the and creative most stress. thinking. The A typical two- last day of week the exercise exercise is is typically a structured as presentation follows: 1. Start-up – Policy design and Technical piloting Private Sector [2009-2011] Financial Strengthening the IRC WASH: All 2. Scale-up, Gender Local sanitation market system M.W. Jenkins, L. McLennan, G. Systems Go Market-Based Primary Data maturation, IRC ASG15 IRC 2019 Peer X Low No Capacity Approach Sanitation Rural Regional Access Implementation Cambodia Southeast Asia 1 Access : WaterSHED’s Hands- Revell, A. Salinger Conference approach Secondary Data replication & Governance National Off experience Preceedings consolidation Coordination [2012-2017] NGOs 3. Donors Sustainability Regulations [2018-2020] Two dedicated week-long capacity- building workshops Four locations in for the Coordination Ethiopia: facilitators Facilitating local IRC WASH: All Monitoring Mile, Afar Woreda M.E. Mussa, T.N. Michael, B.G. were held in strengthening of WASH Systems Go Infrastructure Approach Water Rural Local Planning South Ari SNPPR IRC ASG2 IRC Hailegiorgis, M.J. Morris, J.A. 2019 Peer X Learning Alliances Low No Primary Data Sustainability East Africa 4 Uptake September systems : whose Conference Financial Framework Sanitation Urban Regional Implementation Woreda Butterworth, L. Henry 2017 understanding counts? Preceedings O&M Woliso and April Management Debre Birhan 2018, with further capacity- building and support arranged subsequently At the time

of writing

this paper,

the SSI Institutional ‘upstream’ Technical component Capacity of the project Kick-starting WASH Environmental IRC WASH: All Surveys in Nepal had systems in newly Financial Systems Go Systems Approach Primary Data Chitwan District, only been IRC ASG20 IRC established local W. Tillett, P. Bastola, S. Gautam 2019 Grey X Medium No Services Approach WASH Rural Regional Sustainability Case Study South Asia 1 Policy Conference Composite Scoring Secondary Data Nepal running authorities : an Planning Preceedings Workshops for 10 experience from Nepal Implementation months, with Social a strong Institutional focus to date Monitoring on assessment phase activities. Private Sector Economic IRC WASH: All Local Developing consumer Market-Based Government Systems Go Water Regional IRC ASG22 IRC markets within rural A. Smith, J. Butterworth 2019 Peer X approach Medium No Secondary Data Hygiene Analysis Rural Sustainability Evaluation Conference Sanitation National WASH systems Case Study NGOs Preceedings Sector Policy Networks Private Sector Governance Users Management Planning Fostering WASH system IRC WASH: All O&M changes in Haiti using A. Montangero, V. Cavin, M. Systems Go Market-Based Focus Groups Rural National Policy IRC ASG24 IRC 2019 Peer X Medium No Monitoring Approach WASH Sustainability Planning Haiti Caribbean 1 principles from the Pierre, L. Blaser Conference approach Expert Opinion Coordination Urban Regional Coordination market systems sector Preceedings Financial Networks Institutional Policy Regulations

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 18 of 40

Institutional Policy The Sanitation Cityscape IRC WASH: All Capacity Conceptual Framework : Systems Go Financial One small town in IRC ASG26 IRC P. Scott 2019 Peer X Framework Low Yes Primary Data Framework Sanitation Urban City Sustainability Implementation East Africa 1 understanding urban Conference Planning Ethiopia sanitation systems Preceedings Monitoring Services Infrastructure

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 19 of 40

Document Information Method Scope Application Resources

Infrastructure Services Developing district IRC WASH: All District-Wide Capacity Six years WASH investment plans, Systems Go Primary Data Rulindo District, Policy IRC ASG5 IRC P. Kamuyumbu, M. Kwizera 2019 Peer X Approach Medium No Environmental Approach WASH Rural Regional Sustainability Planning West Africa 1 (project a cornerstone of the Conference Secondary Data Rwanda Services Systems Approach O&M duration) district wide approach Preceedings Monitoring Political Access Behavioral Capacity Community Education Positioning WASH Financial

services within a Gender

systems framework and IRC WASH: All Governance Pusa block of demonstrating Systems Go Technical IRC ASG7 IRC A. AUmar, B. Satviki Varma 2019 Grey X Systems Approach Low No Workshops Approach Sanitation Rural Local Access Approach Samastipur district, South Asia 1 Services achievement of open Conference Hygiene Bihar, India defecation free status at Preceedings Implementation scale : an example from Institutional the Indian State of Bihar Monitoring Participation Political Private Sector Social

Institutional Services Performance-based Infrastructure 20 districts in two financing and capacity IRC WASH: All Building Blocks Interviews Monitoring Planning provinces of building to strengthen Systems Go Qualitative Data Approach Uptake IRC ASG8 IRC L. Rudge 2019 Grey X Medium No Surveys Planning WASH Rural Regional Sustainability Implementation Mozambique East Africa 2 WASH systems in Conference Analysis Analysis Services Workshops Financial Evaluation (Nampula and Mozambique : early Preceedings Mapping Regulations Zambezia) findings Capacity Coordination Sarlahi and Rautahat Digging deep behind the IRC WASH: All districts in Terai Qualitative Data Primary Data Technical complexities of Systems Go and Nuwakot, and IRC ASG9 IRC A. Adhikari 2019 Grey X Analysis Medium No Interviews Financial Analysis Water Rural Regional Sustainability Evaluation South Asia 5 sustainable water supply Conference Gorkha and Dhading Financial Analysis Focus Groups Governance in Nepal Preceedings districts in the Hills region (Nepal) Enhancement of domestic financial Social support mechanisms for OECD Economic Water Rural OECD 6 OECD 2017 OCED Grey X Scenario Analysis Low No Expert opinion Analysis National Sustainability Planning Moldova East Europe 1 water supply and (Uncredited) Institutional Sanitation Urban sanitation in Moldova: Financial Scenario analysis Local and social Agricultura, management capacities Management Six rural communities sociedad y Interviews for domestic water J.L. Pimentel-Equihua, M.A. Governance in the municipality of desarrollo Observation SciElo 120 SciElo supply in rural Velázquez-Machuca, J. Palerm- 2012 Peer X Framework Medium No Capacity Analysis Water Rural Local Sustainability Case Study Zamora, in the Central America 6 (Agriculture, Primary Data communities of the Viqueira Financial northwest of Society and Secondary Data Zamora Valley, Monitoring Michoacá, Mexico Michoacán, Mexico Development) Management Proposition of a system Economic of performance Sanitary and Delphi Panel Secondary Data Financial SciElo 147 SciElo indicators to evaluate the T.L. von Sperling, N. von Sperling 2013 Environmental Peer X High No Framework Sanitation National Sustainability Evaluation Brazil South America 1 Framework Expert Opinions Environmental quality of sewage Engineering Capacity services Infrastructure Infrastructure Technical Water service delivery Governance challenges in a small Users South African Complex Social- Policy municipality: Identifying Ecological NGOs Makana Local Expert Opinion SciElo 221 SciElo and exploring key M.J.T. Weaver 2017 Water SA Peer X Systems Approach Medium Yes Social Approach Water Urban Regional Sustainability Case Study Municipality, South South Africa 1 Uptake Surveys elements and Causal Loop Economic Africa relationships in a Diagramming Political complex social- Financial ecological system O&M Capacity Management Networks Coordination Soft systems Financial methodology for Soft Systems Services Analysis Water WEDC 2 WEDC performance S. Kayaga 2008 IWA Peer X Methodology Low No Workshops Political National Sustainability Evaluation Uganda East Africa 1 Uptake 3 days Framework Sanitation measurement in the CATOWE Economic Uganda water sector Private Sector NGOs Donors

Livelihood factors,

explaining water I. Dominguez, W. Torres-Lopez, I. Agriculture Loughborough Primary data WEDC 7 WEDC consumption in a (de Restrepo-Tarquino, R. Oviedo- 2014 Peer X Statistics High No Socioeconomic Tool Water Rural Local Access Case Study Colombia South America 1 University Surveys facto) multiple uses Ocana, I. Smout Technical water system in Colombia

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 20 of 40

Bangladesh Benin Brazil China South Asia Ethiopia Central America Institutional Ghana Sustainability South America Financial Haiti Assessment of Rural Interviews Caribbean S. Smets, H. Lockwood, S. Management Regional- India World Bank 1 World Bank Water Service Delivery 2017 World Bank Group Grey X Composite Scoring Medium No Secondary Data Framework Water Rural Sustainability Case Study East Africa 16 Mansour, S. Smits Environmental National Indonesia Models Findings of a Expert Opinion West Africa Monitoring the Kyrgyz Republic Multi-Country Review Central Asia Regulations Morocco East Asia Nepal Southeast Asia Nicaragua the Philippines Tanzania Vietnam

Document Information Method Scope Application Resources

Document Document Open Method Analytical Interaction Data Source Method Aspects of Rural / Project Location Location Human

Database Title Author(s) Date Publication Factors Context Outcome Scope Location Study Impacts Time Cost ID Type Access Analysis Complexity Analysis Analysis Application WASH Urban Application Region Count Resources

China (Sichuan)

China (Shaanxi)

Ghana

East Asia India (Kerala)

Technical West Africa Kyrgyzstan

Financial South America Madagascar

World Bank World Bank Introducing FOAM Y. Coombes, J. Devine 2010 The World Bank Grey X Framework Medium No Secondary Data Approach Hygiene Rural Local Sustainability Planning 3 Behavioral Central Asia Peru

218 Framework Urban City Implementation Regulations East Africa Senegal

Social Southeast Asia Tanzania

Uganda

Vietnam Political

Institutional

Infrastructure

Community

Six cities in India:

Financial

Ahmedabad, Gujarat;

Regulations

Bhopal Madhya,

Health Enabling Surveys Planning Pradesh; Kalyani,

Participation World Bank Community Slum City Access World Bank World Bank Group 2016 The World Bank Grey X Environment Low No Primary Data Approach Sanitation Urban Implementation South Asia 6 West Bengal;

Gender 355 Sanitation in India Local Sustainability Framework Secondary Data Evaluation Mumbai, Maharashtr;

Coordination

Pune, Maharashtra;

Context

Tiruchirappalli (or

Management

Trichy), Tamil Nadu

Monitoring

Int. J. Environ. Res. Public Health 2019, 16, x; doi: FOR PEER REVIEW www.mdpi.com/journal/ijerph Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 2 of 40

NGOs

Capacity

Technical

Institutional

Services

Four districts in Private Sector

Expanding Access to Surveys Philippines around Financial

World Bank S. Saruta, A. Geaneotes, R. Gilles, The World Bank / Qualitative Data Analysis Improved Sanitation for Focus Groups metro Manila NGOs

World Bank R. Muana, F. Mills, T. Masuoka 2018 Grey X Medium No Sanitation Urban City Access Case Study Oceania 4 408 IFC Analysis Framework the Poor Interviews (Pampanga, Rizal, Governance

Laguna, Manila) Economic

Demographic

Social Functionality

Management

O&M

20 communities Users

Survey across 4 provinces Financial

Papua New Guinea Qualitative Data Water Interviews (Eastern Highlands, Community

World Bank World Bank K. Clark, N. Francis, G. Pigolo, M. 2017 The World Bank Grey X Medium No Analysis Rural Local Sustainability Evaluation Oceania 4 Rural WaSH Analysis Sanitation Observation Central, Morobe, Hygiene

587 Bond, P. Tyndale-Biscoe Sustainability Study Composite Scoring Hygiene Focus Groups Madang) in Papua Gender

New Guinea Monitoring

Governance

Policy

Community

Private Sector

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 3 of 40

NGOs Marching together with a Workshops

Access WSP 1 WSP N.T. Utomo, M. Listyasari 2010 WSP Grey X Framework Medium No Policy Approach Sanitation Urban City Planning Citywide Sanitation Surveys

Capacity Sustainability Stratergy Secondary Data

Regulations

Financial

Monitoring

Int. J. Environ. Res. Public Health 2019, 16, x; doi: FOR PEER REVIEW www.mdpi.com/journal/ijerph Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 2 of 40

SI 4: RESULTS SUMMARY

Table S4-1: Results Summary Table - Methodology

Methodology Criteria Descriptors

Method Peer Grey Org Total % Framework 21 12 8 41 31% Composite Scoring 12 10 8 30 23% Qualitative Data Analysis 11 10 2 23 17% Statistics 11 1 2 14 11% Social Network Analysis 8 2 10 8% Case Study 5 1 1 7 5% Systems Approach 3 2 2 7 5% AHP 6 6 5% Multi-Criteria Decision 5 1 6 5% Analysis System Dynamics 6 6 5% Market-Based Approach 5 5 4% Causal Loop Diagramming 4 2 6 5% Collective Action / Impact 1 3 4 3% Method for Participatory 2 1 1 4 3% Assessment Theory of Change 1 3 4 3% Building Blocks 2 1 3 2% Learning Alliances 1 2 3 2% Method2 Participatory Systems 2 1 3 2% Mapping Service Delivery Approach 3 3 2% SIASAR 3 3 2% Agent Based Modeling 2 2 2% Bayesian Networks 2 2 2% Financial Analysis 1 1 2 2% Lifecycle Analysis 2 2 2% Qualitative Comparative 2 2 2% Analysis Behavioral Model 1 1 1% CATOWE 1 1 1% Cost-Benefit Analysis 1 1 1% Decision Trees 1 1 1% Delphi Panel 1 1 1% District-Wide Approach / 1 1 1% Systems Approach Enabling Environment 1 1 1% GIS 1 1 1% Graphic Modeling 1 1 1% Institutional Mapping 1 1 1%

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 3 of 40

Mathematical optimization 1 1 1% Remote sensing 1 1 1% Scenario Analysis 1 1 1% Shit Flow Diagrams 1 1 1% Soft Systems Methodology 1 1 1% Sustainable Livelihoods 1 1 1% Framework

Peer Grey Org Total % Low 13 10 10 33 25% 23 16 10 49 37% Analytical Medium Complexity1 High 39 9 3 51 38%

Peer Grey Org Total % 26 5 4 35 26% Interactions1

Data Source Peer Grey Org Total % Secondary Data 34 18 19 71 53% Interviews 37 15 11 63 47% Primary Data 20 7 10 37 28%

Sources of Observation 22 10 3 35 26% data2 Surveys 16 11 5 32 24% Focus Groups 14 10 7 31 23% Expert Opinion 13 10 5 28 21% Workshops 1 5 0 6 5%

Factor Name Peer Grey Org Total % Financial 46 31 21 98 74% Technical 45 17 9 71 53% Institutional 21 19 17 57 43% Social 30 15 9 54 41% Factors2 Environmental 32 8 13 53 40% Capacity 17 18 9 44 33% Economic 30 6 4 40 30% O&M 20 13 7 40 30% Management 18 14 7 39 29% Governance 20 9 7 36 27% Community 21 10 3 34 26%

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 4 of 40

Monitoring 7 16 11 34 26% Coordination 13 11 6 30 23% Services 10 15 3 28 21% Policy 10 7 10 27 20% Regulations 6 15 4 25 19% NGOs 11 10 1 22 17% Access 10 8 2 20 15% Infrastructure 9 6 5 20 15% Private Sector 8 9 3 20 15% Hygiene 7 9 3 19 14% Political 6 7 5 18 14% Functionality 12 3 2 17 13% Health 9 6 2 17 13% Networks 11 4 2 17 13% User 14 2 1 17 13% Planning 4 5 7 16 12% Context 9 4 2 15 11% Implementation 3 7 4 14 11% Behavioral 7 6 0 13 10% Education 6 5 2 13 10% Participation 3 5 5 13 10% Cultural 7 3 1 11 8% Demographic 4 3 1 8 6% Gender 2 5 1 8 6% Donors 3 2 2 7 5% Socioeconomic 7 0 0 7 5% Climate 4 1 1 6 5% Agriculture 3 1 1 5 4% Energy 5 0 0 5 4%

Factor Peer Grey Org Total % Count 2 1 0 0 1 1% 3 5 1 1 7 5% 4 4 2 0 6 5% 5 14 4 5 23 17% 6 16 3 1 20 15% 7 9 2 4 15 11% 8 11 3 4 18 14% 9 5 3 0 8 6% 10 3 5 2 10 8%

Unless specified otherwise, each table represents counts of studies and % of total count of studies, n=133

Notes: 1: a-priori, deductive criteria; 2: open or emergent criteria

Note: Many descriptors represent non-exclusive criteria (i.e. studies can be classified under multiple attributes, % do not always sum to 100%)

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 5 of 40

Table SI 4-2: Results Summary Table - Scope

Scope Criteria Descriptors

Scope Peer Grey Org Total % Water (only) 37 10 8 55 41% Sanitation (only) 15 10 4 29 22% Aspects of Water and Sanitation 12 6 2 20 15% WASH1 WASH 5 4 7 16 12% Water, Sanitation, Hygiene 3 2 1 6 5% Hygiene 1 1 0 2 2% Hygiene and Sanitation 2 2 0 4 3%

Outcome Peer Grey Org Total % Sustainability 48 23 17 88 66% Access 13 3 0 16 12% Outcome Scope2 Health 4 0 0 4 3% Behavior Change 1 2 0 3 2% Coordination 2 1 0 3 2% Functionality 1 0 0 1 1% Policy 0 0 1 1 1%

Context Peer Grey Org Total % Urban and Rural 9 10 11 30 23% Urban (Only) 18 7 2 27 21% Urban (Any) 27 16 13 56 43% Rural (Only) 44 15 8 67 52% Rural (Any) 53 24 19 96 74% None 4 1 2 7 5% Context1

Application Peer Grey Org Total % Local 46 19 14 79 59% Regional 21 10 11 42 32% National 8 8 11 27 20% City 10 9 1 20 15% City-only 9 5 1 15 11% Sector 2 2 7 11 8% None / Unspecified 1 0 0 1 1%

Unless specified otherwise, each table represents counts of studies and % of total count of studies, n=133

Notes: 1: a-priori, deductive criteria; 2: open or emergent criteria

Note: Many descriptors represent non-exclusive criteria (i.e. studies can be classified under multiple attributes, % do not always sum to 100%)

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 6 of 40

Table SI 4-3: Results Summary Table - Application

Application Criteria Descriptors

Project Application Peer Grey Org Total % Evaluation 26 15 15 56 42% Project Case Study 38 9 4 51 38% Application1 Planning 16 13 11 40 30% Implementation 4 7 4 15 11% None 0 0 1 1 1%

Region Peer Grey Org Total % East Africa 23 10 13 46 40% South Asia 21 10 6 37 32% West Africa 12 5 11 28 24% South America 9 6 4 19 17% East Asia 8 6 2 16 14% Southeast Asia 8 6 2 16 14% Central America 8 6 2 16 14% South Africa 4 2 6 5% Caribbean 2 2 1 5 4% Oceania 1 3 4 3% Central Asia 1 3 4 3% North America 1 1 1% East Europe 1 1 1% Middle Africa 1 1 1%

Location1

Country Peer Grey Org Total % India 8 8 4 20 15% Ghana 5 3 10 18 14% Uganda 7 4 6 17 13% Tanzania 5 6 2 13 10% Nepal 7 3 1 11 8% Kenya 7 1 2 10 8% Nicaragua 3 4 2 9 7% Bangladesh 6 1 1 8 6% Burkina Faso 3 5 8 6% Ethiopia 4 2 2 8 6% Indonesia 3 4 1 8 6% Bolivia 4 2 1 7 5% Peru 1 4 2 7 5% Honduras 1 3 2 6 5% Malawi 2 1 3 6 5% Mexico 3 3 6 5%

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 7 of 40

Philippines 2 3 1 6 5% South Africa 4 2 6 5% Vietnam 1 5 6 5% Brazil 2 2 1 5 4% Colombia 3 1 4 3% Dominican Republic 1 2 1 4 3% Mozambique 1 3 4 3% Rwanda 2 2 4 3% Senegal 2 2 4 3% Costa Rica 1 2 3 2% Lao 2 1 3 2% Niger 1 1 1 3 2% Panama 3 3 2% Timor-Leste 2 1 3 2% Benin 1 1 2 2% Cambodia 2 2 2% China 2 2 2% Columbia 1 1 2 2% Fiji 2 2 2% Guatemala 1 1 2 2% Haiti 1 1 2 2% Madagascar 1 1 2 2% Nigeria 1 1 2 2% Pakistan 2 2 2% Zimbabwe 2 2 2% Argentina 1 1 1% Canada 1 1 1% Egypt 1 1 1% Jordan 1 1 1% Kyrgyz Republic 1 1 1% Kyrgyzstan 1 1 1% Laos 1 1 1% Liberia 1 1 1% Mali 1 1 1% Moldova 1 1 1% Morocco 1 1 1% Palestine 1 1 1% Papua New Guinea 1 1 1% Paraguay 1 1 1% Sudan 1 1 1% Sweden 1 1 1% Tajikistan 1 1% Thailand 1 1 1% Zambia 1 1 1%

Reporting of Study Impacts Peer Grey Org Total % Impacts2 Uptake 7 5 4 16 12% Services 4 3 2 9 7% Policy 2 2 4 8 6%

Int. J. Environ. Res. Public Health 2019, 16, x FOR PEER REVIEW 8 of 40

Coordination 2 3 5 4% Behavior Change 2 2 4 3% Financial 1 1 2 2% Users 1 1 2 2% Health 1 1 1% Access 1 1 1%

Unless specified otherwise, each table represents counts of studies and % of total count of studies, n=133

Notes: 1: a-priori, deductive criteria; 2: open or emergent criteria

Note: Many descriptors represent non-exclusive criteria (i.e. studies can be classified under multiple attributes, % do not always sum to 100%)