A Ganar Alliance Impact Evaluation Synthesis Report Guatemala & Honduras
Total Page:16
File Type:pdf, Size:1020Kb
A GANAR ALLIANCE IMPACT EVALUATION SYNTHESIS REPORT GUATEMALA & HONDURAS JUNE 2018 This publication was developed by Social Impact, Inc. under contract No. AID-OAA-M-11- 00019 and produced for review by the United States Agency for International Development (USAID). A GANAR ALLIANCE IMPACT EVALUATION SYNTHESIS REPORT Guatemala & Honduras June 2018 Submitted to: Karen Towers, Education Team Leader Bureau for Latin America and the Caribbean USAID/Washington D.C. Prepared by: Social Impact, Inc. Contract No. AID-OAA-M-11-00019 2300 Clarendon Boulevard Suite 1000 Arlington, VA, 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 [email protected] DISCLAIMER This report was made possible by the support of the American people through the United States Agency for International Development (USAID). The contents of this report are the sole responsibility of Social Impact and do not necessarily reflect the views of USAID or the United States Government. iii ACKNOWLEDGMENTS The A Ganar evaluations could not have been completed without extensive collaboration between, and support from, many organizations and individuals. First and foremost, the evaluation team would like to thank USAID, both in Washington D.C. and the Missions in Guatemala and Honduras, for committing to such a long term and intensive evaluation, demonstrating an impressive commitment to learning and improving their workforce development programming. Specifically, we thank Karen Towers, Education Team Leader in the Office of Regional Sustainable Development in the Bureau for Latin America and the Caribbean in Washington, who has served as the Contracting Officer’s Representative for the A Ganar impact evaluation and provided excellent guidance on all matters to the team. We would also like to thank Partners of the Americas, the implementer of the A Ganar program, and its local partners in Guatemala and Honduras, including Centro Asesor para el Desarrollo de los Recursos Humanos (CADERH), CESAL, Centro Nacional de Educación para el Trabajo (CENET), Children International, Fundación Nacional para el Desarrollo de Honduras (FUNADEH), Fundación Paiz, Glasswing, Grupo Ceiba, Libre Expresión, and Organization of Ibero-American States. They all engaged in the evaluation in a spirit of learning and collaboration, despite the additional burden an impact evaluation entails. We particularly want to thank Paul Teeple, Director of the A Ganar Program, Nadia Moreno, Country Coordinator, and Ana Perdomo, Program Coordinator for A Ganar Honduras, for their extended collaboration and support, even after the program was completed. From the evaluation perspective, we would also like to thank our local data collection partners, Centro de Investigaciones Económicas Nacionales (CIEN) and in Guatemala and ESA Consultores in Honduras, and particularly Jorge Enrique, Lavarreda Gargollo, Pablo Pastor, and Claudia Regina Aguilar. They and their teams put forth tremendous effort, often at personal risk, to capture the high-quality data that is the basis for this report. Finally, we thank the youth and other stakeholders who participated in data collection. We hope that this report accurately represents their views and experiences and is useful in developing and improving future programs to better support them and their communities. This document was produced by Mike Duthie, Mateusz Pucilowski, Lisette Anzoategui, Braden Agpoon, of Social Impact; and Erin Murphy-Graham of the University of California, Berkeley. iv A Ganar Alliance Impact Evaluation Synthesis Report – Guatemala & Honduras ACRONYMS CI Children International CIEN Centro de Investigaciones Económicas Nacionales CO Coordinating Organization CV Curriculum Vitae DAP Developmental Assets Profile EDC Education Development Center ESA ESA Consultores Internaciónal FP Fundación Paiz GEM Gender Equitable Men GEI Gender Equitable Index Ho Hypothesis IDB Inter-American Development Bank IE Impact Evaluation INE Instituto Nacional de Estadística IO Implementing Organization IT Information Technology LAC Latin American and the Caribbean MDES Minimum Detectible Effect Size MIF Multi-Lateral Investment Fund PCA Principal Component Analysis RCT Randomized Control Trial SI Social Impact, Inc. USAID United States Agency for International Development v TABLE OF CONTENTS Acknowledgments ..................................................................................................................................................... iv Acronyms...................................................................................................................................................................... v Table of Contents ...................................................................................................................................................... vi Figures and Tables .............................................................................................................................................. viii Executive Summary .................................................................................................................................................... 1 Evaluation Objectives............................................................................................................................................ 1 Program background............................................................................................................................................. 1 Evaluation Design ................................................................................................................................................... 1 Synthesis Findings by Outcome .......................................................................................................................... 2 Conclusions ............................................................................................................................................................. 4 Recommendations ................................................................................................................................................. 5 Introduction ................................................................................................................................................................. 7 Purpose .................................................................................................................................................................... 7 Evaluation Objectives............................................................................................................................................ 7 Program Background ............................................................................................................................................ 7 Evaluation Design ..................................................................................................................................................... 10 Evaluation Questions ......................................................................................................................................... 11 Research Methodology ...................................................................................................................................... 11 Measurement ....................................................................................................................................................... 12 Sampling ................................................................................................................................................................ 14 Analytical Approach ........................................................................................................................................... 15 Qualitative Approach ......................................................................................................................................... 16 Limitations ............................................................................................................................................................ 16 Attrition ................................................................................................................................................................ 17 Impact Evaluation Synthesis by Outcome ..................................................................................................... 18 Conclusions ............................................................................................................................................................... 47 Recommendations ................................................................................................................................................... 49 Bibliography .......................................................................................................................................................... 52 Annex A: Balance Checks ................................................................................................................................. 55 Annex B: Program Differences ........................................................................................................................ 57 Annex C: Analysis of Attrition ........................................................................................................................ 59 Annex D Outcome Tables ..............................................................................................................................