EVALUATION REPORT End-Of-Project Evaluation of the PSI Social Marketing Project in Madagascar
Total Page:16
File Type:pdf, Size:1020Kb
EVALUATION REPORT End-of-project Evaluation of the PSI Social Marketing Project in Madagascar January 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by International Business & Technical Consultants, Inc. (IBTCI). The authors are Veronique Praz, Alice Morton, and Ian Matondo. Caption: A community-based distributor presents her stock of socially marketed products. This page is intentionally left blank. END-OF-PROJECT EVALUATION OF THE PSI SOCIAL MARKETING PROJECT IN MADAGASCAR January 31, 2013 Contract Number AID-RAN-I-00-09-00016, Task Order Number AID-687-TO-12-00001 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID/Madagascar: End-of-project Evaluation of the PSI Social Marketing Project – Evaluation Report TABLE OF CONTENTS ACKNOWLEDGEMENTS iv GLOSSARY v ACRONYMS vi EXECUTIVE SUMMARY i 1. INTRODUCTION 1 2. EVALUATION DESIGN AND METHODOLOGY 5 3. QUESTION ONE: PERFORMANCE IN KEY HEALTH AREAS 11 CONCLUSIONS ON DIARRHEA PREVENTION (MCH) 13 RECOMMENDATIONS FOR DIARRHEA PREVENTION (MCH) 14 FINDINGS: MCH—DIARRHEA TREATMENT 14 CONCLUSIONS ON DIARRHEA TREATMENT (MCH) 15 RECOMMENDATIONS FOR DIARRHEA TREATMENT (MCH) 15 FINDINGS: MCH—PNEUMONIA TREATMENT 16 CONCLUSIONS ON PNEUMONIA TREATMENT (MCH) 17 RECOMMENDATIONS FOR PNEUMONIA TREATMENT (MCH) 17 FINDINGS: MCH—INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESSES (IMCI) 17 CONCLUSIONS ON IMCI (MCH) 18 RECOMMENDATIONS FOR IMCI (MCH) 18 FINDINGS: MCH—UTILIZATION OF MCH RESEARCH RESULTS 18 CONCLUSIONS ON UTILIZATION OF MCH RESEARCH RESULTS 18 FINDINGS: FAMILY PLANNING AND REPRODUCTIVE HEALTH 19 CONCLUSIONS ON FP/RH 20 RECOMMENDATIONS FOR FP/RH 20 MALARIA PREVENTION AND TREATMENT FINDINGS 20 CONCLUSIONS ON MALARIA PREVENTION AND TREATMENT 21 RECOMMENDATIONS ON MALARIA PREVENTION AND TREATMENT 21 FINDINGS: HIV/STI PREVENTION 21 CONCLUSIONS ON HIV/STI PREVENTION 22 RECOMMENDATIONS FOR HIV/STI PREVENTION 23 USAID/Madagascar: End-of-project Evaluation of the PSI Social Marketing Project – Evaluation Report 4. QUESTION TWO: IMPLEMENTATION OF THE TOP RÉSEAU FRANCHISE 23 FINDINGS ON ACCESS AND USE OF SERVICE BY THOSE MOST IN NEED 25 CONCLUSIONS ON ACCESS AND USE OF SERVICE BY THOSE MOST IN NEED 26 RECOMMENDATIONS ON ACCESS AND USE OF SERVICE BY THOSE MOST IN NEED 26 FINDINGS ON THE SUSTAINABILITY OF TOP RÉSEAU WITHOUT PSI/M SUPPORT 27 CONCLUSIONS ON THE SUSTAINABILITY OF TOP RÉSEAU 27 RECOMMENDATIONS FOR THE SUSTAINABILITY OF TOP RÉSEAU 28 5. QUESTION THREE: PERFORMANCE OF SUPPLY CHAIN & DISTRIBUTION SYSTEMS 28 FINDINGS: ACHIEVEMENTS AND WEAKNESSES IN MAINTAINING THE SUPPLY CHAIN 29 CONCLUSIONS ON THE ACHIEVEMENTS AND WEAKNESSES IN MAINTAINING THE SUPPLY CHAIN 31 RECOMMENDATIONS ON THE ACHIEVEMENTS AND WEAKNESSES IN MAINTAINING THE SUPPLY CHAIN 31 FINDINGS: PSI/M’S CONTRIBUTION TO NATIONAL COVERAGE OF HEALTH PRODUCTS THROUGH THE PHARMACEUTICAL SUPPLY CHAIN 32 CONCLUSIONS ON PSI/M’S CONTRIBUTION TO NATIONAL COVERAGE OF HEALTH PRODUCTS THROUGH THE PHARMACEUTICAL SUPPLY CHAIN 33 RECOMMENDATIONS ON PSI/M’S CONTRIBUTION TO NATIONAL COVERAGE OF HEALTH PRODUCTS THROUGH THE PHARMACEUTICAL SUPPLY CHAIN 33 FINDINGS: PSI/M’S CONTRIBUTION TO NATIONAL COVERAGE OF HEALTH PRODUCTS THOUGH ITS DISTRIBUTION SYSTEM TO RURAL AREAS 34 FINDINGS: SUCCESSES AND OBSTACLES IN PSI/M’S SUPPLY CHAIN MODEL FOR COMMUNITY-BASED DISTRIBUTION 36 SUMMARY OF STOCK-OUT ISSUES IN PSI/M’S SUPPLY CHAIN SYSTEM 37 6. QUESTION FOUR: OTHER CROSS-CUTTING ISSUES 39 FINDINGS: CAPACITY BUILDING EFFORTS WITH LOCAL ORGANIZATIONS 39 CONCLUSIONS ON CAPACITY BUILDING EFFORTS WITH LOCAL ORGANIZATIONS 41 FINDINGS: HARMONIZATION WITH NATIONAL PROGRAMS 41 CONCLUSIONS ON THE HARMONIZATION WITH NATIONAL PROGRAMS 41 FINDINGS: GOM PERCEPTIONS OF PSI/M’S OUTCOMES AND FUTURE NEEDS 41 CONCLUSIONS ON GOM PERCEPTIONS OF PSI/M’S OUTCOMES AND FUTURE NEEDS 42 USAID/Madagascar: End-of-project Evaluation of the PSI Social Marketing Project – Evaluation Report FINDINGS: LEVERAGING PARTNERSHIPS WITH LOCAL AND INGOS VIS-À-VIS DISTRIBUTION SUPPORT 42 CONCLUSIONS ON LEVERAGING PARTNERSHIPS WITH LOCAL AND INGOS VIS-À-VIS DISTRIBUTION SUPPORT 44 FINDINGS: INFLUENCE OF THE POLITICAL SITUATION ON PSI/M’S PERFORMANCE 44 CONCLUSIONS ON THE INFLUENCE OF THE POLITICAL SITUATION ON PSI/M’S PERFORMANCE 45 ANNEXES 48 ANNEX A: PSI MADAGASCAR SUB-AWARDEES 48 ANNEX B: PSI/M MAPS OF COMMERCIAL COVERAGE AND TOP RÉSEAU CLINICS 51 ANNEX C: TABLE OF RESPONDENT CATEGORIES 53 ANNEX D: EVALUATION SCOPE OF WORK 54 ANNEX E: MAP OF REGIONS AND DISTRICTS VISITED BY THE EVALUATION TEAM 60 ANNEX F: TABLE OF DATA COLLECTION RESULTS 61 ANNEX G: LIST OF REFERENCE DOCUMENTS 68 ANNEX H: DATA COLLECTION INSTRUMENTS 71 ANNEX I: PSI/M TRAINING SCHEDULE FY 2008 – FY 2012 92 ANNEX J: SUPPLY CHAIN GRAPHIC 101 ANNEX K: AGGREGATED RESULTS OF SELECTED LSAT VARIABLES 102 USAID/Madagascar: End-of-project Evaluation of the PSI Social Marketing Project – Evaluation Report ACKNOWLEDGEMENTS The Evaluation Team would like to thank the personnel from USAID whose views helped form a more complete understanding of the context in which PSI/Madagascar (PSI/M) was designed and implemented. Special thanks are extended to: Mr. Jean Claude Randrianarisoa, Mr. Hery Zo Ramiandrisoa, Mr. Jacky Ralaiarivony, Ms. Lucie Raharimalala, Ms. Fabienne Ranjalahy, Ms. Cathy J. Bowes, Mr. David J. Isaak, Ms. Jackie Ramanitrera, Mr. Robert Kolesar, Mr. Benjamin Rakotondranisa, Ms. Vololontsoa Raharimalala, and Dr. Jocelyne Andriamiadana. Special thanks are given to Mr. Rudolf Thomas, USAID Mission Director and Acting Chief of Mission, United States Embassy, Madagascar, who kindly attended and spoke at the Evaluation Out briefing at the Mission, in Antananarivo on August 6, 2012. The Team is also very grateful to the staff of Population Services International (PSI), including Mr. Brian McKenna, Deputy Regional Director, and Daniella Fanarof, Senior Program Manager for Southern Africa and at PSI/Washington, as well as the very dynamic team at PSI/M Headquarters, led by Mr. Chuck Szymanski, Country Representative, and Monique Weiss and Henri Rabesahala, Deputy Country Directors, and including many others such as Ms. Ietje Reerink, Ms. Marie-Alix Valensi, Ms. Iarimalanto Rabary, Mr. Davy Robson, Mr. Maminirina Rakotoarisoa, Mr. Kenny Dakadisy, Dr. Mbolatiana Razafimahefa, Ms. Velonirina Andrianifahanana, Dr. Andry Nirina Rahajarison, Mr. Njakatiana Rajaonarivelo, Mr. Teddy Lala Ramaroson, Mr. Solofo Andriaherinosy, Mr. Kenny Dakadisy, Dr. Eric Ratsaravolana, and Dr. Phillipe Botralahy. We appreciate the valuable and transparent information they provided. Our thanks go also to PSI/M Inter-regional Directors in Antsiranana and Tamatave, as well as PSI/M Top Réseau Regional Coordinators at Toliara, Mahajanga, Antsirabe, Fianarantsoa, Tôlanaro, and Morondava for the valuable information and introductions they provided on PSI/M activities. We present our special thanks to all the program and project beneficiaries of PSI/M’s partner organizations, as well as their representatives and managers, and others whom participated in the evaluation. Finally, the Evaluation Team would also like to its great appreciation to the IBTCI home office Team in Vienna, Virginia, who provided technical and managerial support to the Team during field work. USAID/Madagascar: End-of-project Evaluation of the PSI Social Marketing Project – Evaluation Report GLOSSARY DELTA A marketing planning process developed by Population Services International (PSI) FARMAD Local pharmaceutical manufacturer and retailer Fokontany The smallest administrative subdivision in Madagascar ITEM A local non-governmental organization (NGO) Malagasy Of or from Madagascar MAHEFA Stands for Malagasy Heniky ny Fahasalamana, a project implemented by John Snow, Inc. (JSI) in Madagascar NIPHAR A local pharmaceutical manufacturer SALAMA A government agency which is the Central Medical Store of Madagascar SantéNet2 A project implemented by Research Triangle Institute, International (RTI) in Madagascar SIGMA A private manufacturer of Sur’ Eau SISAL A local NGO Supply Points For the purposes of this report, Supply Points are places in the supply chain at the community level from which to distribute social marketed health products directly to community health workers (CHWs) Top Réseau A franchised network of private, clinic-based providers established by PSI/Madagascar in 2000 with support from the Bill and Melinda Gates Foundation ZETRA A local NGO USAID/Madagascar: End-of-project Evaluation of the PSI Social Marketing Project – Evaluation Report ACRONYMS ACT Artemisinin-based Combination Therapy AFSPS Agence Française de Santé de Produits Sanitaires AGMED Agence du Médicament de Madagascar AIDS Acquired Immunodeficiency Disease Syndrome ARI Acute Respiratory Infection BCC Behavior Change Communication CDC Community Development Committees CHW Community Health Worker CROM Conseil Régional de l’Ordre des Médecins CSB Centre de Santé de Base CSW Commercial Sex Worker CU5 Children Under Five Years Old CYP Couple Years of Protection DAMM Direction de l’Agence des Médicaments de Madagascar DRS Directeur Régional de la Santé DTK Diarrhea Treatment Kit FBO Faith Based Organization FGD Focus Group Discussion FISA Fianakaviana Sambatra FP Family Planning GFATM Global Fund to Fight HIV/AIDS, Tuberculosis, and Malaria GOLD Groupements et Organisations Locales de Developpement GOM Government