UC Berkeley UC Berkeley Electronic Theses and Dissertations Title Methods to Study Intervention Sustainability Using Pre-existing, Community Interventions: Examples from the Water, Sanitation and Hygiene Sector Permalink https://escholarship.org/uc/item/8db349mq Author Arnold, Benjamin Ford Publication Date 2009 Peer reviewed|Thesis/dissertation eScholarship.org Powered by the California Digital Library University of California Methods to Study Intervention Sustainability Using Pre-existing, Community Interventions: Examples from the Water, Sanitation and Hygiene Sector by Benjamin Ford Arnold A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Epidemiology in the Graduate Division of the University of California, Berkeley Committee in charge: Professor John M. Colford, Jr., Chair Professor Alan E. Hubbard Professor Kirk R. Smith Fall 2009 Methods to Study Intervention Sustainability Using Pre-existing, Community Interventions: Examples from the Water, Sanitation and Hygiene Sector c 2009 Benjamin Ford Arnold Abstract Methods to Study Intervention Sustainability Using Pre-existing, Community Interventions: Examples from the Water, Sanitation and Hygiene Sector by Benjamin Ford Arnold Doctor of Philosophy in Epidemiology University of California, Berkeley Professor John M. Colford, Jr., Chair This dissertation presents a quasi-experimental study design to evaluate non-randomized, pre-existing community interventions not originally designed to gather rigorous data about sustainability and impact. The core components of the design include selection of a control group using propensity score matching with pre-intervention (baseline) secondary data and post-intervention follow-up in the field. The main advantages of the design include measurement of interventions implemented under actual field conditions (independent of scientific research), as well as the design's ability to gather information about the long term impacts and sustainability of interventions without years of costly prospective follow-up. Studies of non-randomized, pre-exisiting interventions must address threats to validity, principal among them: unmeasured confounding and informative censoring. I outline the main strengths and weaknesses of the study design using simulation and empirical examples. I also apply the design in two sustainability field studies: a 3-year household water treatment and hygiene promotion intervention in rural Guatemala and a 5-year community led total sanitation, water supply and hygiene education intervention in rural India. In both studies, the design leads to samples of intervention and control groups with highly comparable baseline characteristics. A principal finding of both field studies is poor initial impact and sustainability of the behavioral components of the interventions. In Guatemala, I find a small, five per- centage point increase (8.7% vs. 3.3%) in the proportion of households that treat their water six months after the promotion intervention, but no differences in hygiene knowl- edge or practice, and no detectable differences in child health based on acute illness or growth. In India, I find a large, 33 percentage point increase (48% vs. 15%) in private toilet construction as a result of the intervention, but open defecation persists in 40% of households with functional private toilets. In India, diarrhea is rare in both intervention and control communities (1:8% over 14,259 child weeks), but most children show growth faltering by international standards (mean height-for-age Z-score: −1:98). Despite no signifiant differences in health between children living in intervention and control villages, 1 I observe important non-health benefits: private toilets increase privacy and safety during defecation for women and girls by 28 percentage points (81% vs. 53%), and private water taps reduce water collection time by a median of 25 minutes per day relative to public taps (50 vs. 75 minutes). I also find that hardware improvements are highly sustainable up to five years after implementation with more than 94% of private toilets and 96% of private water taps in use during repeated visits over one year. Studies of non-randomized, pre-existing interventions are a rapid, low-cost alternative to prospective intervention studies for evaluating intervention sustainability. The study design and methodology developed in this dissertation are applicable to evaluating a broad range of pre-existing, community interventions beyond the water, sanitation and hygiene sector. 2 For Ellis and Oliver © i Contents Contents Contents........................................ iii List of Tables.....................................v List of Figures..................................... vii Acknowledgements.................................. viii 1 Introduction and Background1 1.1 Introduction...................................2 1.2 Specific aims...................................3 1.3 Structure of the dissertation..........................3 1.4 Water, sanitation and hygiene interventions in developing countries....4 1.5 Sustainability of water, sanitation and hygiene interventions........8 1.6 Motivation and challenges of studying pre-existing interventions...... 15 Chapter Bibliography................................. 17 2 A Quasi-experimental Design to Evaluate Non-randomized, Pre-existing Community Interventions 23 2.1 Goals....................................... 24 2.2 Introduction................................... 24 2.3 Background: estimating treatment effects and matched designs....... 25 2.4 Necessary conditions for the intervention................... 31 2.5 A quasi-experimental design.......................... 35 2.6 Comments on analysis............................. 43 2.7 Didactic simulation studies........................... 45 2.8 Empirical example............................... 60 2.9 Discussion.................................... 67 Chapter Bibliography................................. 72 3 Evaluation of a pre-existing, 3-year household water treatment and handwashing intervention in rural Guatemala 77 3.1 Goals....................................... 78 3.2 Background................................... 78 3.3 Methods..................................... 80 ii Contents 3.4 Results...................................... 90 3.5 Discussion.................................... 115 Chapter Bibliography................................. 118 4 Evaluation of a pre-existing, combined sanitation, water and hygiene intervention in rural Tamil Nadu, India 123 4.1 Goals....................................... 124 4.2 Background................................... 125 4.3 Methods..................................... 126 4.4 Results...................................... 142 4.5 Discussion.................................... 193 Chapter Bibliography................................. 204 5 Integrated discussion 209 5.1 Compendium of scientific results........................ 210 5.2 Conclusions................................... 219 Chapter Bibliography................................. 223 A Additional background: current evidence for intervention efficacy 225 A.1 Water supply improvements.......................... 225 A.2 Water quality improvements.......................... 227 A.3 Handwashing and hygiene promotion..................... 228 A.4 Sanitation improvements............................ 228 A.5 Multiple interventions............................. 230 A.6 Hypotheses for synergy and antagonism between interventions....... 233 Appendix Bibliography................................ 235 B Targeted maximum likelihood estimation for point-treatment studies 240 Appendix Bibliography................................ 243 iii List of Tables List of Tables 1.1 JMP water supply and sanitation definitions.................7 2.1 Necessary conditions for studies of non-randomized, pre-existing commu- nity interventions................................ 34 2.2 Summary of steps in the quasi-experimental design............. 36 2.3 Simulation 1 results, random treatment.................... 50 2.4 Simulation 1 results, non-random (targeted) treatment........... 51 2.5 Comparison of univariate, bivariate and propensity score matching..... 59 2.6 Performance of PSM for different sample sizes................ 60 2.7 Community balance characteristics in different samples........... 64 2.8 Measurements needed to document intervention sustainability....... 70 3.1 Summary of study power estimates...................... 85 3.2 Covariates considered in targeted maximum likelihood estimation...... 88 3.3 Village selection results............................. 91 3.4 Post-intervention population characteristics.................. 92 3.5 Reasons for using household water treatment................. 95 3.6 Water practice outcomes............................ 97 3.7 Self-reported handwashing information sources................ 98 3.8 Handwashing and hygiene outcomes...................... 100 3.9 Covariates used in adjusted analyses..................... 101 3.10 Acute child health outcomes.......................... 105 3.11 Child growth outcomes............................. 107 3.12 Relative efficiency of unadjusted and adjusted estimates........... 108 3.13 Intra-cluster correlation estimates....................... 109 3.14 Water practice outcomes (alternate treatment)................ 110 3.15 Handwashing and hygiene outcomes (alternate treatment)......... 111 3.16 Acute child health outcomes (alternate treatment).............. 113 3.17 Child growth outcomes (alternate