Rapidsms in Rwanda
Total Page:16
File Type:pdf, Size:1020Kb
Evaluating the Impact of RapidSMS: Final Report A Report Commissioned by UNICEF Rwanda December 2, 2016 Hinda Ruton, MSc1,2 Angele Musabyimana, MD, MSc1 Karen Grépin, PhD3 Joseph Ngenzi1 Emmanuel Nzabonimana1 Michael R. Law, PhD1,2,4 1. School of Public Health, College of Medicine and Health Sciences, University of Rwanda, Kigali, Rwanda 2. Centre for Health Services and Policy Research, The University of British Columbia, Vancouver, British Columbia, Canada 3. Sir Wilfred Laurier University, Waterloo, Ontario, Canada 4. Department of Global Health and Social Medicine, Harvard Medical School, Boston, MA, USA 1 Table of Contents Table of Contents ..................................................................................................................................... 2 Index of Tables and Figures ...................................................................................................................... 4 Tables ...................................................................................................................................................... 4 Figures ..................................................................................................................................................... 4 List of Acronyms....................................................................................................................................... 6 Executive Summary .................................................................................................................................. 7 Introduction and Program Description ..................................................................................................... 9 The role of UNICEF and the Government of Korea ................................................................................. 10 Purpose of this Evaluation ...................................................................................................................... 11 Theory of Change ................................................................................................................................... 12 Objectives and Research Questions ....................................................................................................... 14 Program Relevance ............................................................................................................................... 14 Program Effectiveness .......................................................................................................................... 14 Program Impact .................................................................................................................................... 14 Program Efficiency & Coordination ...................................................................................................... 14 Program Sustainability .......................................................................................................................... 14 Equity Considerations ........................................................................................................................... 14 Evaluation Design ................................................................................................................................... 14 Overview ............................................................................................................................................... 14 Quantitative Methodology ................................................................................................................... 15 Analysis of HMIS data ....................................................................................................................... 15 Outcome Measures ........................................................................................................................... 16 Qualitative Methodology ...................................................................................................................... 17 Study Locations ................................................................................................................................. 17 Sampling procedures ........................................................................................................................ 17 Recruitment of study participants .................................................................................................... 18 Data collection procedures ............................................................................................................... 18 Data management and analysis ....................................................................................................... 19 Stakeholder Participation ....................................................................................................................... 19 Ethical Considerations ............................................................................................................................ 19 Informed Consent ................................................................................................................................. 19 Participant Confidentiality .................................................................................................................... 19 Research Ethics Board Review .............................................................................................................. 20 Evaluation Results: Quantitative Analysis ............................................................................................... 21 RapidSMS Use ....................................................................................................................................... 21 Overall Use ........................................................................................................................................ 21 Recording of Births ........................................................................................................................... 23 Data Quality ...................................................................................................................................... 24 Outcomes Analysis ................................................................................................................................ 24 Start Date for Health Facilities.......................................................................................................... 24 Number of Messages per capita ....................................................................................................... 25 Antenatal Care Outcomes ................................................................................................................. 26 Delivery Outcomes ............................................................................................................................ 33 2 Post-Birth Outcomes ......................................................................................................................... 35 Evaluation Results: Qualitative Analysis ................................................................................................. 45 Interviews and Focus Groups Conducted ............................................................................................. 45 Organization of Qualitative Results ...................................................................................................... 46 Problems intended to be addressed by RapidSMS ............................................................................... 46 The Impact of RapidSMS ....................................................................................................................... 47 The Relevance of RapidSMS .................................................................................................................. 48 Alignment with National Priorities and Strategies ........................................................................... 48 Acceptance of RapidSMS by the community .................................................................................... 48 The Effectiveness of RapidSMS ............................................................................................................. 49 The Efficiency of RapidSMS ................................................................................................................... 50 The Sustainability of RapidSMS............................................................................................................. 51 The Coordination of RapidSMS ............................................................................................................. 52 The Equity Focus of RapidSMS .............................................................................................................. 53 Perceived areas where RapidSMS is not currently having the intended impact .................................. 53 Perceptions about RapidSMS training .................................................................................................. 54 Issues with the RED Alert system .......................................................................................................... 55 Suggested improvements to the system .............................................................................................. 55 Increase Motivation of CHWs ........................................................................................................... 56 Improve technical