Performance Evaluation of the High-Impact Health Services (SSGI)This Document and Was Preparedthe Social for the USAID/Mali and Behavior As Part of Contract Change No
Total Page:16
File Type:pdf, Size:1020Kb
MALI Photo credit: Children watch as a woman and child practice hand washing in Mali. / WASHplus Performance Evaluation of the High-Impact Health Services This document was prepared for the USAID/Mali as part of contract No. 720-688-18-F-00002 – Evaluation (SSGI) and the Social and Behavior Change Communication of the High-Impact Health Services (SSGI) and Social and Behavior Change Communication (KJK) (KJK)Programs. Programs Final evaluation report January 16, 2019 This document was produced for review by the United States Agency for International Development. It was prepared by the Mitchell Group, Inc. (TMG). i Prepared by: Dr. Issakha Diallo, Evaluation Specialist/ Team Leader M. Mamadou F. Camara, Research Coordinator and Quality Assurance Advisor Dr. Arkia Diallo, MNCH/FP Technical Advisor Dr. Mohamed Amadou Traore, Infectious Disease Technical Advisor M. Sankaria Maiga, Behavior Change/Health, Communication Advisor Dr. Elvira Beracochea, Technical Manager Dr. Sarah Castle, Qualitative Data Specialist Dr. John McCauley, Quantitative Data Specialist KoitaConsulting: Local partner Main contact: M. Abou Kone, Program Manager, TMG, Inc., Washington, DC. Email: [email protected] The Mitchell Group, Inc. 1816 11th Street, NW Washington, DC 20001 Tel.: 202-745-1919 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ii ABSTRACT Background: Since December 2014, USAID has supported the High-Impact Health Services (Services de Santé à Grand Impact - SSGI) and Keneya Jemu Kan ((KJK) Social and Behavior Change Communication (SBCC) programs. This Mid-Term Evaluation explores whether these programs’ activities are making progress toward planned results, and to what extent financial, management, and M&E systems are affecting implementation and achievement of results. The evaluation relied on document and data review, Key Informant Interviews, Focus Group Discussions, observation of program teams, and a randomized, 30- cluster sample survey of health facilities (30) and individual women (320). Findings: Though designed to work together, the two programs function separately. Both experienced delays due to difficulties setting up administrative processes; hiring and retaining qualified staff; and contracting with and strengthening local NGOs. KJK is on track to meet its program targets; however, Jigi, the local NGO contracted to sustain the activities, is not yet implementing SBCC activities. SSGI is on track to meet training targets and has improved performance at some facilities, but not yet consistently or sustainably. Both programs will require additional time to achieve planned results. Recommendations: 1. USAID should extend both programs, creating a 3-month overlap with subsequent programs to ensure service continuity. 2. KJK should second technical staff to Jigi and the CNIECS in the remaining project period to build their capacity to continue SBCC after KJK ends. This should be prioritized over research. 3. SSGI should second its staff to regional and district offices in the remaining project period and focus on sustaining quality health service delivery and addressing barriers to access. 4. Both programs should work to strengthen the financial management capacity and accounting systems of the local NGOs with whom they work. iii ACKNOWLEDGMENTS The Evaluation Team would like to thank the KJK and SSGI teams who welcomed us and opened up their programs to us. We would also like to thank all of the hard-working health care providers – doctors, nurses, pharmacists, community health workers and others working to improve the quality of health care in Mali. The Evaluation Team also wishes to extend its sincere gratitude to the USAID/Mali health team who supported us throughout the implementation of the evaluation in-country and who has worked with us patiently and diligently to help us to ensure a report that answers their questions and meets their needs. iv TABLE OF CONTENTS ABSTRACT .................................................................................................................................................. iii ACKNOWLEDGMENTS ............................................................................................................................ iv TABLE OF CONTENTS ............................................................................................................................... 1 ACRONYMS ................................................................................................................................................. 2 EXECUTIVE SUMMARY ............................................................................................................................... 4 I. INTRODUCTION ................................................................................................................................... 10 II. PROGRAM BACKGROUND ................................................................................................................. 11 III. EVALUATION METHODS & LIMITATIONS ....................................................................................... 13 Table 1: Number and type of focus group discussion by region ........................................................ 14 IV. FINDINGS ............................................................................................................................................. 17 Figure 1. Percentage of Women Receiving Antenatal Care (ANC) Services at the Health Facility during Most Recent Pregnancy, by Region. ........................................................................................ 19 Table 2. IP Findings regarding the Share of Women who Take Part in Antenatal Care Visits .......... 19 Table 3. Correlation between ANC Services and Health Facility Deliveries (p = 0.000) ................. 20 Table 4. IP Findings Regarding the Share of Deliveries in Health Facilities ........................................ 21 Figure 2. Reported Use of FP by Region ............................................................................................ 21 Figure 3. Modern FP Methods Used by MTE Respondents. ............................................................... 22 Table 5. Relationship between Spousal Support and Making ANC Visits (P=0.042) ......................... 23 Table 6. Regions, Districts and Health Areas Covered by KJK and Partners at Year 4 of the Program Implementation ................................................................................................................................... 26 Table 7: Messages and Tools Developed under KJK .......................................................................... 26 Table 8. Results of the Distribution/Sale of Social Marketing Products. (Source KJK) ...................... 30 Figure 4. Referrals by Region .............................................................................................................. 38 Table 10. Evidence of Most- and Least-Successful Interventions ....................................................... 41 V. CONCLUSIONS AND RECOMMENDATIONS .................................................................................. 57 ANNEX I: Evaluation Scope of Work ANNEX II: List of Key Informants ANNEX III: Bibliography of Documents Reviewed ANNEX IV: Additional Tables: Summary of CSCOM MTE survey ANNEX V: List of Selected CSCOMs and Target population ANNEX VI. Data Collection Tools 1 ACRONYMS ACT Artemisinin-based combination therapies AMTSL Active management of third stage of labor ANC Antenatal Care ARI Acute Respiratory Infection ASACO Association de Santé Communautaire (Community Health Association) ASFM Association des Sages-Femmes ATN PLUS Assistance Technique Nationale Plus (National Technical Assistance Plus) BEmONC Basic Emergency Obstetric and Newborn Care CEmONC Comprehensive Emergency Obstetric and Newborn Care CAG Groupe d’action communautaire (Community Action Group) CHW Community Health Worker CLTS Community-led total sanitation CNIECS National Center for Health Information, Education and Communication COP Chief of Party CPR Contraceptive Prevalence Rate CSCOM Centre de santé communautaire (Community Health Center) CSREF Centre de Reference (Reference Health Center) CYP Couple Years of Protection DHIS2 District Health Information System 2 DHS Demographic and Health Survey DP Development Partner EBF Exclusive breastfeeding EMMP Environmental Mitigation and Monitoring Plan ET Evaluation Team FENASCOM Fédération Nationale des Associations de Santé Communautaire (National Federation of the Association of Community Health Groups) FGD Focus Group Discussion FM Finance Manager FP Family Planning GoM Government of Mali GP/SP Groupe Pivot Santé Population (Pivot Group for Health and Population) GSAN Groupe de Soutien à la Nutrition (Nutrition Support Group) HIHS High-Impact Health Services HIS Health Information System HIV Human Immunodeficiency Virus IEE Initial Environmental Examination IMCI Integrated Management of Childhood Illness IPPF International Planned Parenthood Federation IR Intermediate Result IUD Intra-Uterine Device INW Intensive National Nutrition Activity Week IYCF Infant and Young Child Feeding JHU/CCP Johns Hopkins University Center for Communications Programs JIGISIGI “Hope for Families” communication platform KII Key Informant Interview KJK Keneya Jému Kan KMC Kangaroo mother care LAM Lactational Amenorrhea Method 2 LARC Long Acting