EVALUATION REPORT Evaluating the Effectiveness of PRM
Total Page:16
File Type:pdf, Size:1020Kb
EVALUATION REPORT Evaluating the Effectiveness of PRM Multilateral Partners in Ukraine to Assist Internally Displaced Persons and Prepare for the Eventual Transition from Relief to Development EVALUATION REPORT EvaluatingJune 2017 the Effectiveness of PRM Multilateral This publication was produced at the request of the United States Department of State, Bureau of Population, PartnersRefugees, and Migration in .Ukraine It was prepared independently to Assist by Zumrat Internally Salmorbekova, Erica Disp Holzaepfellaced, and Mariana Lopez-Davila for Social Impact, Inc. 1 Persons and Prepare for the Eventual Transition from Relief to Development COVER PHOTO Rural road in winter, Kharkiv Oblast. Photo Credit: Alex Shabotenko. 2 EVALUATING THE EFFECTIVENESS OF PRM MULTILATERAL PARTNERS IN UKRAINE TO ASSIST INTERNALLY DISPLACED PERSONS AND PREPARE FOR THE EVENTUAL TRANSITION FROM RELIEF TO DEVELOPMENT FINAL EVALUATION REPORT June 2017 IDIQ Contract Number: S-AQMMA-12-D-0086 Technical and Advisory Services for Program Evaluation Requirements Task Order Number: S-AQMMA-16-R-0600 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Department of State or the United States Government. 3 CONTENTS Acronyms ............................................................................................................................. vi Executive Summary ............................................................................................................. viii Evaluation Purpose and Evaluation Questions ....................................................................... 1 Program Background ............................................................................................................. 2 Evaluation Design, Methods, and Limitations ......................................................................... 5 Evidence and Findings ........................................................................................................... 8 Conclusions ......................................................................................................................... 31 Evaluation Question 1—Access to Services ............................................................................................ 31 Evaluation Question 2—Assistance ......................................................................................................... 32 Evaluation Question 3—Beneficiary Selection ........................................................................................ 32 Evaluation Question 4—Beneficiary Feedback ....................................................................................... 32 Recommendations .............................................................................................................. 34 Annexes .............................................................................................................................. 40 Annex I: Evaluation Statement of Work .................................................................................................. 41 Annex II: Evaluation Matrix ..................................................................................................................... 54 Annex III: Data Collection Instruments .................................................................................................... 56 Annex IV: Documents Reviewed ............................................................................................................. 94 Annex V: List of People Interviewed ....................................................................................................... 99 Annex VI: Monitoring Tools ................................................................................................................... 104 Annex VII: Country Context ................................................................................................................... 111 Annex VIII: HRP’s Strategic Objectives, and ICRC’s Emergency Appeal Objectives .............................. 115 Annex IX: Detailed Version of EQ1 ........................................................................................................ 117 Annex X: Contributing Factors for Effective Response, Successful Interventions, Challenges, and Gaps, EQ2 ........................................................................................................................................................ 128 Annex XI: List of NGO Implementing Partners of the Multilateral Partners ......................................... 139 Annex XII: Multilateral Partners’ Vulnerability Criteria, EQ3 ................................................................ 141 Annex XIII: Beneficiary Selection Process, EQ3 Part I ............................................................................ 143 Annex XIV: Beneficiary Feedback on Received Integration Assistance from UNHCR, ICRC, and IOM, EQ4 Part I ...................................................................................................................................................... 146 Annex XV: Beneficiary Feedback on Preferred Forms of Assistance, EQ4, Part II................................. 148 Annex XVI: Do PRM’s Partners Make Use of Best Practices in Their Programming and Engagement? EQ5, Part I, Detailed Version ................................................................................................................. 151 Annex XVII: Desk Review Report ........................................................................................................... 160 iv FIGURES Figure 1. Partner information; type of assistance, target group, and PRM funding .................................... 3 Figure 2. IOs identified contributing factors for successful response, successful interventions, challenges, and gaps ........................................................................................................ Error! Bookmark not defined. Figure 3. Use of best practices in IDP programming and engagement ...................................................... 24 v ACRONYMS ADRA Adventist Development and Relief Agency ATO antiterrorist operation area CAP Cash Assistance Project CBI Community-based initiative CBO community-based organization CSSFCY Center of Social Services for Family, Children, and Youth DDC Donbas Development Center DoS U.S. Department of State DPR Donetsk People’s Republic DRC Danish Refugee Council ECA PRM’s Europe, Central Asia, and the Americas ECHO European Commission Humanitarian Aid and Civil Protection EQ evaluation question ET evaluation team FLAC Free Legal Aid Center GBV gender-based violence GCA government-controlled area GI group interview GoU Government of Ukraine GP good practice HCT Humanitarian Country Team HH household HPV Housing Purchase Voucher HRP Humanitarian Response Plan IASC Inter-Agency Standing Committee ICRC International Committee of the Red Cross IDMC Internal Displacement Monitoring Center IDP internally displaced person II individual interview IMS information management system IO international organization (UNHCR, ICRC, IOM, UNFPA) IOM International Organization for Migration IP NGO implementing partner of UNHCR, IOM and UNFPA IPA individual protection assistance KII key informant interview LNGO local non-governmental organization LPR Luhansk People’s Republic MoSP Ministry of Social Policy MPTF Multi-Partner Trust Fund MSL minimum subsistence level MT mobile team MTOT Ministry of Temporary Occupied Territories and Internally Displaced Persons NFI non-food items NGCA non–government-controlled area vi NGO non-governmental organization NMS National Monitoring System NRC Norwegian Refugee Council OCHA Office for the Coordination of Humanitarian Affairs OSCE Organization for Security and Cooperation in Europe PCPM Polish Center for International Aid PDM post-distribution monitoring PIN People in Need PMP Performance Monitoring Plan (PMP) PRM Bureau of Population, Refugees, and Migration PRP PRM’s Policy and Resource Planning PSS psychosocial support PTM Post Transfer Monitoring RH reproductive health R2P Right to Protection SBU State Security Service SI Social Impact, Inc. SRH sexual and reproductive health UAH Ukrainian hryvnia (currency) UFPH Ukrainian Foundation for Public Health UN United Nations UNCT United Nations Country Team UNDAF United Nations Development Assistance Framework UNFPA United Nations Population Fund UNHCR United Nations High Commissioner for Refugees USG U.S. Government WHFP Women Health and Family Planning vii EXECUTIVE SUMMARY Evaluation Purpose and Evaluation Questions The U.S. Department of State (DoS) Bureau of Population, Refugees, and Migration (PRM) contracted Social Impact, Inc. (SI) to conduct a performance evaluation to examine the effectiveness of PRM’s multilateral partners (international organizations, or IOs) in assisting internally displaced persons (IDPs) in Ukraine and preparing for the eventual transition from relief to development. The evaluated IOs include the United Nations High Commissioner for Refugees (UNHCR), the International Committee of the Red Cross (ICRC), the International Organization for Migration (IOM), and the United Nations Population Fund (UNFPA). PRM’s Europe, Central Asia, and the Americas and Policy and Resource Planning offices will use the evaluation findings and recommendations to guide their programmatic and diplomatic decision making in planning for longer-term development regarding the local integration