Evaluation of the UNICEF Albania Country Programme 2017-2021 Final Report December 2020 Disclaimer: This is a publication by the independent evaluation team of IOD PARC. The analysis and recommendations of this report do not necessarily reflect the views of UNICEF. Readers are encouraged to use material from this report for their own publications. As copyright holder, UNICEF in Albania requests due acknowledgement and a copy of the publication. For further information, please contact: UNICEF Albania Country Office Skenderbej Street, UN House Bld, 3rd floor Tirana, Albania Telephone: +355 4 45 48 400 Email: [email protected] Suggested Citation of the Report: O’Callaghan, B. Duci, V. Kacapor-Dzihic, Z. (2020) “Evaluation of the UNICEF Albania Country Programme 2017-2021”. UNICEF / IODPARC / IMC Worldwide. Evaluation of the UNICEF Albania Country Programme 2017-2021 Final Report December 2020 Acknowledgements Sincere thanks to Ashley Wax at UNICEF Europe and Central Asia Regional Office, Dr Roberto De Bernardi, UNICEF Albania Country Representative, Elda Denaj and the staff members of the UNICEF Albania Country Office, as well as the members of the Evaluation Reference Group for providing the evaluation team with feedback, information and support throughout the evaluation period. The evaluation was undertaken by IOD PARC’s consortium consisting of IMC Worldwide, Child Frontiers, Development Analytics and M-Vector. The evaluation team members are Brian O’Callaghan (Team Leader, IMC Worldwide), Veronika Duci and Zehra Kacapor-Dzihic. Additional research was carried out by Joy McCarron. The evaluation was supported by Project Managers - Karolina Mclellan and Elisa Radaelli. The evaluation direction and quality assurance were provided by Jo Kaybryn. Contents Acronyms and Abbreviations vi Executive Summary viii Overview of the Country Programme viii Purpose of the Evaluation and its Audiences viii Evaluation Methodology ix Key Findings ix Relevance ix Effectiveness x Efficiency xii Sustainability xii Synergies and coherence xiii Conclusions xiv Lessons Learned xvi Recommendations xvi 1. Introduction 1 1. 1 Purpose, objectives and scope of the evaluation 1 1. 2 Country context 3 1.2.1 Geographic and demographic context 3 1.2.2 Political context and government priorities 4 1.2.3 Social development and gender profile 6 1.2.4 Children, youth and families 7 1.2.5 Governance and civil society 9 1. 3 Country Programme components and goals 9 2. Methodology 12 2.1 Evaluation framework 12 2.1.1 Departures from the Terms of Reference 12 2.2 Data collection, data analysis and sampling strategy 13 2.3 Limitations and constraints faced by the evaluation 14 2.4 Management of the evaluation 15 2.5 Ethical considerations 15 3. Findings of the Evaluation 17 3.1 Relevance 17 3.1.1 Evaluation Question 1 UNICEF’s strategic positioning 17 3.1.2 Evaluation Question 2 Implementation strategies 23 3.2 Effectiveness 26 3.2.1 Evaluation Question 3 Organizational structure 26 3.2.2 Evaluation Question 4 Achievement of results 30 3.2.3 Evaluation Question 5 Gender mainstreaming 52 3.3 Efficiency 55 3.3.1 Evaluation Question 6 Resourcing 55 3.4 Sustainability 61 3.4.1 Evaluation Question 7 Long-term alignment of programming 61 3.5 Synergies/Coherence 64 3.5.1 Evaluation Question 8 Internal coherence 64 3.5.2 Evaluation Question 9 Coordination 66 4. Conclusions 70 5. Lessons Learned 74 6.Recommendations 76 Annexes 80 1. Terms of Reference 2. Reconstructed Theory of Change/Evaluation Matrix 3. List of Interviewees 4. Documentary Evidence 5. Data Collection Instruments 6. Evolution of UNICEF Albania CPD Priority Results Areas 7. UNICEF Albania Programme Locations 8. Gender Integration and Targeting in AWPs, COARs, RAMs and SMQs 2017–2019 9. Results Reporting Analysis 10. UNICEF Albania Stakeholders 11. Evaluation Reference Group Members 12. Human Rights Analysis 13. Evaluation Team List of Tables Table 1: UNICEF Albania country programme outcomes and outputs 2017–2021 11 Table 2: Progress in realization of child rights reported by UNICEF stakeholders 31 Table 3: Summary of UNICEF Albania results assessment module reporting against indicators 32 Table 4: Selected UNICEF Albania results assessment module indicators 33 Table 5: UNICEF Albania expenditure by programme area (2017–2019) 56 Table 6: UNICEF Albania donor funding in 2017–2019 58 Table 7: UNICEF Albania financial breakdown by partners 60 List of Figures Figure 1: Map of Albania 3 Figure 2: Young persons’ perspectives on rights in Albania (U-Report Survey 2020) 35 Figure 3: UNICEF Albania planned vs. utilized annual budgets 2016–2019 56 Figure 4: UNICEF Albania regular resources vs other resources vs management/admin 56 Figure 5: UNICEF Albania budgets 2017–2019 by programme outcome 57 Figure 6: UNICEF Albania financial allocations by partner type 60 Acronyms and Abbreviations C4D Communication for Development CEDAW Convention on the Elimination of all Forms of Discrimination Against Women CFC Child Friendly City CFS Child Friendly Spaces CO Country Office (UNICEF) COAR Country Office Annual Report COVID-19 Coronavirus Disease 2019 CP Country Programme CPD Country Programme Document CPE Country Programme Evaluation CRC Committee on the Rights of the Child CRM Child Rights Monitoring CSO Civil Society Organization DAC Development Assistance Committee DaO Delivering as One DCM Decisions of the Council of Ministers DFID Department for International Development DHS Demographic and Health Survey DRR Disaster Risk Reduction ECARO Europe and Central Asia Regional Office (UNICEF) ECD Early Childhood Development ER Emergency Response ERG Evaluation Reference Group ESRC Economic and Social Research Council EU European Union EUROSTAT Statistical Office of the European Union EVAW End Violence Against Women FP Focal Point GAP Gender Action Plan GDP Gross Domestic Product GPR Gender Programmatic Review HACT Harmonized Approach to Cash Transfer HCT Humanitarian Cash Transfers HDI Human Development Index IFI International Financial Institution ILO International Labour Organization INSTAT Instituti Shqiptar i Statistikës (Albanian Institute of Statistics) IOM International Organization for Migration IR Inception Report JWP Joint Working Plan KAP Knowledge, Attitudes and Practices LNB Leave No One Behind M&E Monitoring and Evaluation MAPS Mainstreaming, Acceleration, and Policy Support MCH Maternal and Child Health MESY Ministry of Education, Sports and Youth MoHSP Ministry of Health and Social Protection of the Population MNCH Maternal, Newborn and Child Health Care NSDI National Strategy for Development and (European) Integration NGO Non-governmental Organization OECD Organisation for Economic Co-operation and Development OOSC Out-of-School Children OR Other Resources PF4C Public Finance for Children PoCSD Programme of Cooperation for Sustainable Development PSS Psychosocial Support RAM Results Assessment Module RO Regional Office RR Regular Resources SDGs Sustainable Development Goals SitAn Situation Analysis SMQ Strategic Monitoring Questions SMR Strategic Moment of Reflection SRH Sexual and Reproductive Health TB Tuberculosis ToC Theory of Change ToR Terms of Reference UMIC Upper Middle-Income Country UN United Nations UNDAF United Nations Development Assistance Framework UNEG United Nations Evaluation Group UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund WB World Bank WHO World Health Organization 10 EVALUATION OF THE UNICEF ALBANIA PROGRAMME 2017-2021 EXECUTIVE SUMMARY FINAL REPORT 11 12 EVALUATION OF THE UNICEF ALBANIA PROGRAMME 2017-2021 his report presents the findings of the contributions to realization of child rights in Albania, evaluation covering the entire UNICEF portfolio including inequities and gender; and during the first four years (2017- August 2020) 3. Identify good practices and lessons and draw T of the five-year programme as articulated in its forward-looking recommendations that can inform 2017–2021 Country Programme Document (CPD), and the development of the next country programme. successive implementation strategies derived from this. The intended primary users of this evaluation are the UNICEF Albania Country Office and the Government Overview of the Country of Albania, as well as other United Nations agencies, Programme UNICEF partners, and other UNICEF offices in the region, including the UNICEF Europe and Central Asia With the initial budget of US$ 16,345,000, and in Regional Office (ECARO). partnership with government ministries, agencies and departments at national, regional and district level, the overall goal of the Albania Country Programme by 2021 Evaluation Methodology is “the progressive realization of rights of all children in Albania, accompanied by the reduction of equity The evaluation Terms of Reference proposed nine key gaps, to be based on effective child-focused policies, evaluation questions (EQs), focusing on the OECD systematically enforced accountabilities with adequate Development Assistance Committee (DAC) criteria of public resources, and strengthened respect for and relevance, effectiveness, efficiency, sustainability and protection of children’s rights”. synergies and coherence. In the inception phase, some revisions were made to the evaluation sub-questions to With an equity focus on vulnerable and marginalized streamline data collection and analysis, with review and children (for example children with disabilities, from approval from the Evaluation Reference Group (ERG). ethnic minorities and/or out-of-school), the intended long-term