Impacts of Community-Led Video Education to Increase Vaccination Coverage in Uttar Pradesh, India: a Mixed Methods Randomised Controlled Trial
Total Page:16
File Type:pdf, Size:1020Kb
Impacts of community-led video education to increase vaccination coverage in Uttar Pradesh, India: A mixed methods randomised controlled trial Nikki Gurley PATH Jessica Shearer PATH Yachna Srivastava PATH Sudip Mahapatra PATH Michelle Desmond PATH Grantee Final Report Accepted by 3ie: June 2020 Note to readers This impact evaluation has been submitted in fulfilment of requirements under grant TW10.1039 issued under Innovations in Increasing Immunisation Evidence Programme. This version is being published online as it was received. A copy-edited and formatted version will be available in the 3ie Impact Evaluation Report Series in the near future. All content is the sole responsibility of the authors and does not represent the opinions of 3ie, its donors or its board of commissioners. Any errors and omissions are the sole responsibility of the authors. All affiliations of the authors listed in the title page are those that were in effect at the time the report was accepted. Any comments or queries should be directed to the corresponding author, Nikki Gurley at: [email protected]. The 3ie technical quality assurance team comprises Monica Jain, Avantika Bagai, Ananta Seth, Kirthi Rao, Sayak Khatua, an anonymous external impact evaluation design expert reviewer and an anonymous external sector expert reviewer, with overall technical supervision by Marie Gaarder. Funding for this impact evaluation was provided by Bill & Melinda Gates Foundation. A complete listing of all of 3ie’s donors is available on the 3ie website. Suggested citation: Gurley, N, Shearer, J, Srivastava, Y, Mahapatra, S and Desmond, M, 2020. Impacts of community-led video education to increase vaccination coverage in Uttar Pradesh, India: A mixed methods randomised controlled trial, 3ie Grantee Final Report. New Delhi: International Initiative for Impact Evaluation (3ie). Acknowledgments This report was written by Nikki Gurley (M&E Associate, PATH, Seattle), Jessica Shearer (Senior Technical Officer, PATH, Seattle), Yachna Srivastava (M&E Officer, PATH, Lucknow), Sudip Mahapatra (Regional M&E Advisor, PATH, New Delhi), and Michelle Desmond (Senior M&E Officer, PATH, Seattle). Special thanks to Julie Rajaratnam (Director M&E, PATH, Seattle) for her technical feedback. Thank you to Emma Korpi for project management support to finalize this report. Thanks to the implementation team, Shri Anil Mishra (Field Manager, PATH, Lucknow) and Shiji Srivastava (Program Assistant, PATH, Lucknow), without whom this project would not have been possible. Also, we would like to thank Richard Anderson (Professor of Computer Science, University of Washington, Seattle) and Kiersten Israel-Ballard (Associate Director, Maternal, Newborn, Child Health & Nutrition, PATH, Seattle) for creating the idea of Projecting Health. We would also like to thank Nielsen India and their team of data collectors for their hard work in collecting the data informing these results. A special thanks to our local community-based partner, Nehru Yuva Sangathan-Tisi, which facilitated access to the evaluation villages and interviews with community leaders. Thank you to Karen Grepin and New York University for village-level shape files for Uttar Pradesh. Thank you to the 3ie staff involved in this grant window and other grantees who attended periodic workshops and from whom we learned much. Finally, thank you to all community members, accredited social health activists, auxiliary nurse midwives, and other stakeholders in Airaya and Hathgaon blocks who made this evaluation possible. This evaluation was made possible through support from the International Initiative for Impact Evaluation (3ie). The intervention was funded by the Bill & Melinda Gates Foundation [OPP1153158]. Contents Acknowledgments .................................................................................................................... 3 List of tables ............................................................................................................................. 6 List of figures ............................................................................................................................ 6 Abbreviations ........................................................................................................................... 7 Executive summary .................................................................................................................. 8 1. Introduction ........................................................................................................................ 10 2. Intervention, theory of change, and research hypotheses .............................................. 11 2.1 Intervention description .................................................................................................. 11 2.2 Theory of change ........................................................................................................... 14 2.3 Monitoring plan of the intervention relevant to the impact evaluation .............................. 16 3. Evaluation design and methods........................................................................................ 18 3.1 Evaluation questions ...................................................................................................... 18 3.2 Evaluation design ........................................................................................................... 18 3.3 Sampling and data collection .......................................................................................... 21 3.4 Model specifications ....................................................................................................... 26 3.5 Qualitative data .............................................................................................................. 29 3.6 Strategies to avoid bias .................................................................................................. 31 3.7 Ethics and transparency ................................................................................................. 31 4. Findings .............................................................................................................................. 33 4.1 Implementation fidelity .................................................................................................... 33 4.2 Descriptive statistics and balance table .......................................................................... 39 4.3 Impact ............................................................................................................................ 40 4.4 Impact among subgroups ............................................................................................... 45 4.5 Impact among exposed .................................................................................................. 51 4.6 Outcomes: Intention and knowledge ............................................................................... 57 4.7 Outcomes: ASHA knowledge and facilitation skills ......................................................... 63 4.8 Outcomes: Networks ...................................................................................................... 63 5. Cost analysis ...................................................................................................................... 66 5.1 Methods ......................................................................................................................... 66 5.2 Results ........................................................................................................................... 67 6. Challenges and lessons .................................................................................................... 73 6.1 Challenges during implementation.................................................................................. 73 6.2 Challenges during evaluation ......................................................................................... 74 7. Discussion .......................................................................................................................... 75 7.1 Household and community-level determinants of vaccination ......................................... 75 7.2 Barriers and enablers in the local health system ............................................................. 77 7.3 Barriers and facilitators to the impact of the video intervention ....................................... 80 7.4 Processes required for an improved intervention ............................................................ 81 7.5 Strengths and weaknesses of the evaluation design ...................................................... 82 7.6 Conclusions and recommendations ................................................................................ 83 Annex 1: Original theory of change ..................................................................................... 84 Annex 2: Revised theory of change ...................................................................................... 85 Annex 3: Map of project area ................................................................................................. 86 Annex 4: Full list of villages .................................................................................................. 87 Page 4 of 108 Annex 5: Socioeconomic class calculation .......................................................................... 89 Annex 6: Household survey instruments (attached) ..........................................................