INCLUSION for PEACE ACTIVITY PERFORMANCE EVALUATION Final Evaluation Report
Total Page:16
File Type:pdf, Size:1020Kb
MEL ACTIVITY INCLUSION FOR PEACE ACTIVITY PERFORMANCE EVALUATION Final Evaluation Report DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Contract Title: USAID/Colombia Monitoring, Evaluation and Learning activity Contract Number: 72051419C00001 Submitted: January 7, 2021 Contractor: Panagora Group, LLC Sub-contractor: Management Systems International COR: Omar López Authors: Manuel Fernando Castro, Bibiana Quiroga, Sonia Rodríguez, Alejandro Espitia, Mauricio Aguilar, Juana Oberlaender, María Angélica Sarmiento, Óscar Quiroz, and Camila Chavarría. The technical team at Panagora Group included Orlando Gracia, Juan Guillermo Bedoya, Luisa Fernanda Cardona, Jose Luis Astorquia, Javier Rodriguez, Yazmín Medina, and David Doncel. Substantive inputs during the evaluation process were provided by Andrew Dicello (Management Systems International), Juan Camilo Paillie, Alfredo Torres, and Martha Barriga. Data collection was supported by the firm IPSOS Colombia. Panagora Group Contacts: Ana María Rivera Chief of Party Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia Email: [email protected] Orlando Gracia Lead Evaluation Expert Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia Email: [email protected] Sarah Tisch, PhD Director, Monitoring, Evaluation, Research, Learning and Adapting Practice 8601 Georgia Avenue, Suite 905, Silver Spring, MD, USA Email: [email protected] CONTENTS ABSTRACT 1 EXECUTIVE SUMMARY 2 INSTITUTIONAL STRENGTHENING – ACCESS TO HIGHER EDUCATION 3 INSTITUTIONAL STRENGTHENING – GOC 4 INSTITUTIONAL STRENGTHENING – CSOS 4 INSTITUTIONAL STRENGTHENING – IMPLEMENTING THE PEACE AGREEMENT 5 ECONOMIC INCLUSION AND LIVELIHOODS 6 COMMUNICATION AND CHANGES REGARDING ETHNIC DIVERSITY 7 EVALUATION RECOMMENDATIONS 8 EVALUATION PURPOSE 11 ACTIVITY DESCRIPTION AND BACKGROUND 12 EVALUATION QUESTIONS ON METHODOLOGY 16 EVALUATION QUESTIONS 16 METHODOLOGY 17 LIMITATIONS TO THE EVALUATION 20 EVALUATION FINDINGS AND CONCLUSIONS 21 INSTITUTIONAL STRENGTHENING – ACCESS TO HIGHER EDUCATION 21 INSTITUTIONAL STRENGTHENING – GOC 26 INSTITUTIONAL STRENGTHENING – CSOS 32 IMPLEMENTING THE PEACE AGREEMENT 37 ECONOMIC INCLUSION AND LIVELIHOODS 40 COMMUNICATION AND CHANGES REGARDING ETHNIC DIVERSITY 47 CONTEXTUAL ELEMENTS 51 CROSS – CUTTING FINDINGS 55 EVALUATION RECOMMENDATIONS 56 INSTITUTIONAL STRENGTHENING – ACCESS TO HIGHER EDUCATION 56 INSTITUTIONAL STRENGTHENING – GOC 57 INSTITUTIONAL STRENGTHENING – CSOS 58 INSTITUTIONAL STRENGTHENING – IMPLEMENTING THE PEACE AGREEMENT 58 ECONOMIC INCLUSION AND LIVELIHOODS 59 COMMUNICATION AND CHANGES REGARDING ETHNIC DIVERSITY 60 CONTEXTUAL ELEMENTS 62 CROSS-CUTTING RECOMMENDATIONS 63 ANNEX I: RECOMMENDATIONS IMPLEMENTATION MATRIX 65 ANNEX II: SUPPLEMENTAL ANALYSIS AND DATA 108 ANNEX III: TABLA DE HALLAZGOS, CONCLUSIONES Y RECOMENDACIONES 178 ANNEX IV: DETAILED METHODOLOGY 207 WORK PLAN 207 EVALUATION DESIGN 225 ANNEX V: INSTRUMENTS 255 INSTRUMENTOS CUANTITATIVOS 255 INSTRUMENTOS CUALITATIVOS – FASE 1 275 INSTRUMENTOS CUALITATIVOS – FASE 2 Y 3 312 INSTRUMENTOS CUALITATIVOS – FASE 4 Y 5 328 INSTRUMENTOS CUALITATIVOS – FASE 6 362 ANNEX VI: SOURCES OF INFORMATION 377 ANNEX VII: EVALUATION SCOPE OF WORK 382 PURPOSE OF THE EVALUATION 382 BACKGROUND 382 EVALUATION QUESTIONS 387 EVALUATION DESIGN AND METHODOLOGY 388 DELIVERABLES AND REPORTING REQUIREMENTS 390 EVALUATION TEAM COMPOSITION 392 ACRONYMS CLA Collaboration, Learning, and Adaptation COP Colombian Peso CSO Civil Society Organization DANE National Administrative Department for Statistics EGM Media General Survey ELN National Liberation Army Guerrilla FARC Colombian Armed Revolutionary Forces Guerrilla GIS Geographic Information System GOC Government of Colombia ICAE Institutional Capacity for Ethnic Support Index ICFES Colombian Institute for the Promotion of Higher Education ICOE Ethnic Organizations Capacity Index ICT Information and Communications Technologies IICA Armed Conflict Impact Index IOM International Organization for Migrations IPA Inclusion for Peace Activity IRACA Comprehensive Rural Interventions Program MCA Multiple Correspondence Analysis MEL Monitoring, Evaluation, and Learning Activity MEN National Education Ministry MINTIC Ministry of Information and Communications Technologies MPI Multidimensional Poverty Index OCAD Management and Collegiate Body of the Science Ministry PATR Action Plans for Regional Transformation PDET Territorial-focused Development Programs PTA “Todos a Aprender” Program SOW Statement of Work SPE Public Employment Service SWOT Strengths, Weaknesses, Opportunities and Threats Analysis TBD To Be Determined Later TOR Terms of Reference UNAL Colombia’s National University USAID United States Agency for International Development USG United States Government ABSTRACT The IPA’s objective is to promote the social and economic inclusion of ethnic communities affected by the conflict in nine departments and 21 municipalities of the country. The IPA’s approach aims to: • Improve the capacities of the government and civil society organizations; • Increase income and access to economic opportunities; • Promote awareness and respect for ethnic diversity; and • Provide humanitarian assistance. The purpose of this evaluation is to examine the intervention’s results, effectiveness, and challenges, considering Colombia’s current context. The evaluation used mixed methods supplemented with collaboration, learning, and adapting (CLA) techniques to facilitate communication and geographic information system (GIS) visualization tools. The qualitative methodology included analyzing the phenomenology of perception of beneficiary groups and the strengths, weaknesses, opportunities, and threats (SWOT) analysis based on semi-structured interviews with various actors. The quantitative methodology included impact measurement techniques, social experiments of behavioral economics, statistical inference, descriptive statistics, and multiple correspondence analysis using administrative records and field surveys. The evaluation’s main conclusion is that the activity shows favorable results in institutional strengthening, social and economic inclusion, and improved recognition and respect for Colombia’s Afro-Colombian and indigenous communities’ cultural identity and heritage. However, the evaluation also identified several significant challenges and opportunities for improvement regarding the operational context and the design and implementation of its tools and interventions. We propose 34 distinct recommendations and implementation measures to address those challenges and opportunities. 1 | IPA PERFORMANCE EVALUATION FINAL REPORT USAID.GOV EXECUTIVE SUMMARY The primary findings and recommendations are described below, with key messages highlighted in bold type. Colombia has a wide ethnic diversity that encompasses indigenous, Afro-Colombian, Raizal, and Palenquero communities. According to 2018 data from the National Administrative Department for Statistics (DANE), ethnic groups represent 9.34 percent of the national population. In general, ethnic groups have been disproportionately affected by the armed conflict and have significant disadvantages in their living conditions. In 2016, 45.8 percent of the indigenous communities and 26.9 percent of black, Afro-Colombian,1 Raizal, and Palenquero communities experienced multi-dimensional poverty, according to the Multidimensional Poverty Index (MPI). The ethnic population’s illiteracy rate (15 years and older) was 8.5 percent, double that of Colombia's non-ethnic population (4.2 percent). Furthermore, of the number of victims reported in the Unified Victims’ Registry, 12 percent self- identified as members of these ethnicities. Due to these communities’ social and economic lag coupled with higher impacts from the armed conflict, the Colombian Government, through the Peace Agreement, set out to enhance indigenous and Afro-Colombian communities’ relevance in national development policies. The Inclusion for Peace Activity (IPA) is an initiative to promote the economic and social inclusion of ethnic communities afflicted by violence in Colombia. Implemented since November 2016 in the aftermath of the Peace Agreement, IPA focused on 21 municipalities in nine departments, aiming to achieve four outcomes: i) strengthening the institutional capacity of the Colombian Government (GOC) and civil society organizations (CSOs); ii) reducing poverty and increasing access to economic opportunities; iii) promoting inclusion and respect for ethnic diversity, and iv) providing humanitarian assistance to Afro-Colombian and indigenous populations.2 After almost (4) years of implementation, this evaluation seeks to identify IPA’s main results across its different components. The evaluation answers 24 guiding questions grouped in seven dimensions, examining if the delivered products and the achieved effects are relevant in the current national and local contexts. It also assessed whether institutional and community strengthening efforts were effective in improving capacities and policy implementation, if IPA made progress in the economic inclusion of ethnic communities, and if IPA achieved cultural changes that benefit ethnic awareness and respect and inclusion of indigenous and Afro-Colombian groups. To answer the guiding questions, a mixed-methods approach was used that combined quantitative and qualitative analysis techniques. These complementary approaches