EVALUATION REPORT EVALUATION OF THE UNFPA SUPPORT TO FAMILY PLANNING 2008-2013 VOLUME II – ANNEXES ____________________________________ Evaluation Office New York April 2016 Evaluation of the UNFPA Support to Family Planning Services 2008-2013 Evaluation Manager: Louis Charpentier Evaluation Office UNFPA Reference group members: Nestor Azandegbe UNFPA WCARO Elizabeth Benomar UNFPA Technical Division, HIV and AIDS Branch Rita Columbia UNFPA Technical Division, Commodity Security Branch Hugo Gonzalez UNFPA LACRO Ezizgeldi Hellenov UNFPA EECARO Desmond Koroma UNFPA Technical Division, Commodity Security Branch Laura Laski UNFPA Technical Division, Sexual and Reproductive Health Branch Benedict Light UNFPA Technical Division, Commodity Security Branch Shawn Malarcher USAID Office of Population and Reproductive Health Bureau for Global Health Selen Ors UNFPA Turkey Country Office Farah Usmani UNFPA Programme Division, Operational Support & QA Branch Kanyata Sunkutu UNFPA ESARO Team member, Burkina Faso and Zimbabwe case study Euro Health Group (EHG) and Royal Tropical Institute (KIT) Evaluation Team: team leader Lynn Bakamjian Team member, Burkina Faso and Zimbabwe case study team leader Meg Braddock Overall evaluation team leader, Ethiopia and Bolivia case study team leader Hermen Ormel Team member, Cambodia case study team leader and desk study team leader UNFPA Evaluation Office Research Team: Karen Cadondon Evaluation Research Consultant Melinda Elias Evaluation Research Consultant Natalie Raaber Evaluation Research Consultant Any enquiries about this Report should be addressed to: Evaluation Office, United Nations Population Fund Email: [email protected] – Phone number: +1 212 297 2620 ii TABLE OF CONTENTS Annex 1: Evaluation matrix....................................................................................................... 3 Annex 2: Terms of Reference ................................................................................................. 89 Annex 3: Bibliography for the evaluation matrix .................................................................. 124 Annex 4: Portfolio of UNFPA Family Planning Interventions 2008-2013 .............................. 138 Annex 5: Theory of Change .................................................................................................. 143 Annex 6: Country tables ....................................................................................................... 153 Annex 7: Data collection limitations and related implications ............................................. 158 Annex 8: List of people interviewed ..................................................................................... 159 Annex 9: In-country key informant interview topic guide .................................................... 173 Annex 10: Focus group discussion guide for in-country case studies ................................... 178 Annex 11: Interview guide for UNFPA headquarters, regional offices, country offices and international stakeholders ................................................................................................... 181 Annex 12: Online survey – Stakeholders .............................................................................. 184 Annex 13: Internet Survey – Country offices ........................................................................ 192 Annex 14: Methodological note on financial analysis .......................................................... 202 iii ABBREVIATIONS AND ACRONYMS APRO Asia and the Pacific Regional Office ASRH Adolescent Sexual and Reproductive Health ASRO Arab States Regional Office AWP Annual Work Plan BMGF Bill and Melinda Gates Foundation CHAI Clinton Health Access Initiative CIP Costed Implementation Plan CO Country Office CP Country Programme CPAP Country Programme Action Plan CPD Country Programme Document CPR Contraceptive Prevalence Rate CSB Commodity Security Branch CSO Civil Society Organisation DHS Demographic Health Survey DP Development Partner EC Emergency Contraception EECARO Eastern Europe and Central Asia Regional Office ESARO East and Southern Africa Regional Office FBO Faith Based Organisation FGD Focus Group Discussions FGM Female Genital Mutilation FMoH Federal Ministry of Health (Ethiopia) FP Family Planning GBV Gender-Based Violence GPRHCS Global Programme for Reproductive Health Commodity Security HDA Health Development Army HEW Health Extension Worker (Ethiopia) HIP High Impact Practices HQ UNFPA Headquarters HRBA Human rights-based approach ICPD International Conference on Population and Development IDP International Development Partner IEC Information, Education and Communication IPPF International Planned Parenthood Federation ISP Integrated Service Programme IUCD Intra Uterine Contraceptive Device KI Key Informant KII Key Informant Interview KM Knowledge Management LAC Latin America and Caribbean 1 LGBTI Lesbian, Gay, Bisexual, Transgender, Intersexed people M&E Monitoring and Evaluation MCH Mother and Child Health MDG Millennium Development Goal MIC Middle Income Country MISP Minimum Initial Service Package MoH Ministry of Health MoHCC Ministry of Health and Child Care (Zimbabwe) MoU Memorandum of Understanding MSI Marie Stopes International MSM Men who have Sex with Men NGO Non-Government Organisation PGH The Pledge Guarantee for Health PHC Primary Health Care PLHIV People Living with HIV PMTCT Prevention of Mother to Child Transmission (of HIV) RA Rapid Assessment RH Reproductive Health RHCS Reproductive Health Commodity Security RMNCH Reproductive, Maternal, Newborn and Child Health RO Regional Office SIKMB Strategic Information and Knowledge Management Branch SIKMB Strategic Information and Knowledge Management Branch SRH Sexual and Reproductive Health SRHR Sexual and Reproductive Health Rights SuR State under Review TA Technical Assistance TMA Total Market Approach ToT Training of Trainer UNDAF United Nations Development Assistance Framework UPR Universal Periodic Review VMG Vulnerable and Marginalised Group 2 Annex 1: Evaluation matrix Q1: To what extent has UNFPA supported integration of family planning with maternal health, HIV/STI and GBV services in health plans and at primary health care level, in services for adolescents, and in emergency and humanitarian situations? Assumptions for Indicators Sources of Methods and Verification information tools for the data collection 1.1: UNFPA HQ, RO and Knowledge generated Documents Literature CO staff and in-country and shared regarding International key review (global partners are working nature of and lessons informants and for country towards a common learned from External notes) understanding of the integration stakeholder survey International meaning and importance interventions. respondents key informant of service integration. UNFPA staff, partners’ UNFPA country interviews and users' (women's office survey Desk study key and men's) perception respondents informant of meaning and Desk study key interviews importance of service informants Focus group integration. Case study country discussions notes for Bolivia, Group Burkina Faso, discussions Cambodia, Ethiopia Site visits and Zimbabwe UNFPA developed and/or participated in the development several documents that provide guidance on the definition of sexual and reproductive health rights (SRHR) and integrated services for use by UNFPA staff and programmes include: UNFPA Framework on Reproductive health and rights, including priority components and a basic package of SRH services (UNFPA 2010c: 14) SRH-HIV Linkages (IPPF, UNFPA et al. 2009, IPPF, UNFPA et al. 2014) Reproductive Health for Communities in Crisis (UNFPA 2012l) Planning and Implementing an Essential Package of SRH services (Williams, Warren et al. 2010) SRH-HIV Linkages Compendium of Indicators and related assessment tools (IPPF, UNFPA et al. 2014) Programming Strategies for Postpartum Contraception (WHO 2013) Global key informants (KIs) indicated that UNFPA has made an important contribution to integration of sexual and reproductive health (SRH) with HIV and AIDS programmes, especially in partnership with other stakeholders. UNFPA collaborated with several partners including the International Planned Parenthood Federation (IPPF), WHO and Young Positives to develop the Rapid Assessment Tool for Sexual and Reproductive Health and HIV Linkages: A Generic Guide (IPPF, UNFPA et al. 2009). The tool is intended to support the development of country-specific action plans to forge and strengthen linkages between sexual and reproductive health and HIV at the levels of policy, systems and service delivery. UNFPA held five workshops to roll out the assessment guide attended by 66 countries from West Africa, the Arab States, the Caribbean, Asia and the Pacific, Eastern Europe and Central Asia (UNFPA 2008: 20). Global KIs point to the lack of collaboration and alignment of the technical branches within UNFPA headquarters (HQ), and hence a lack of consensus around the strategies for advancing family planning (FP). The SRH and Global Programme for Reproductive Health Commodity Security (GPRHCS) branches notably do not have a harmonised approach towards FP, with the SRH branch more focused on integration and rights and the Commodity Security Branch (CSB) focused on 3 supply side and commodity security with less of a holistic and integrated approach. The CSB is noted for being strong on the supply side, but requires capacity in areas where UNFPA has an important role to play, i.e., in rights