EVALUATION REPORT EVALUATION OF UNICEF’S CLUSTER LEAD AGENCY ROLE IN HUMANITARIAN ACTION EVALUATION OFFICE DECEMBER 2013 EVALUATION REPORT EVALUATION OF UNICEF’S CLUSTER LEAD AGENCY ROLE IN HUMANITARIAN ACTION EVALUATION OFFICE DECEMBER 2013 Evaluation of UNICEF’s Cluster Lead Agency Role in Humanitarian Action Final Report © United Nations Children’s Fund, New York, 2013 United Nations Children’s Fund Three United Nations Plaza New York, New York 10017 December 2013 UNICEF’s Evaluation Office manages evaluations and evaluative studies on a wide range of topics in order to contribute to learning about what makes for effective development, as well as supporting accountability for results. These evaluations seek to identify what works and what does not in terms of achieving sustainable and equitable development results, and to throw light on how and why interventions are successful or not under various circumstances. In assessing UNICEF’s support to governments and other development partners, these evaluations consider where, how and why progress is being made and the difference it is making in the lives of children, women and communities around the world. By publishing evaluation reports, the Evaluation Office makes evaluation findings, lessons and conclusions available to a wide audience. Lessons learned from evaluation are expected to inform operational improvements and, ultimately, to support the achievement of better results. The publication of evaluation reports also supports accountability, by providing all interested parties with independently determined evidence relating to UNICEF’s performance. This provides a basis for informed dialogue and discussion, and helps assure all stakeholders that the organization operates in an open and transparent manner. The contents of this report do not necessarily reflect the policies and views of UNICEF. The text has not been edited to official publication standards, and UNICEF accepts no responsibility for error. The designations in this publication do not imply an opinion on the legal status of any country or territory, or of its authorities, or the delimitation of frontiers. The copyright for this report is held by the United Nations Children’s Fund. Permission is required to reprint/reproduce/photocopy or in any other way to cite or quote from this report in written form. UNICEF has a formal permission policy that requires a written request to be submitted. For non-commercial uses, the permission will normally be granted free of charge. Please write to the Evaluation Office at the address below to initiate a permission request. For further information, please contact: Evaluation Office United Nations Children’s Fund Three United Nations Plaza New York, New York 10017 [email protected] PREFACE The cluster approach is an important vehicle to ensure adequate coordination of humanitarian response. Introduced in 2005 within the wider context of humanitarian reform, the cluster approach is organized along a two-tiered structure involving roles and responsibilities at both global and country levels. UNICEF serves as Global Cluster Lead Agency (CLA) for three Clusters: Nutrition, Water, Sanitation and Hygiene (WASH) and, as co-CLA with Save the Children, for Education. In addition, UNICEF is Focal Point Agency for Child Protection area of responsibility (AOR) and co-Focal Point Agency with the United Nations Population Fund (UNFPA) for the Gender-based Violence (GBV) AOR. UNICEF is the organization with the most clusters and AORs under its remit, and its cluster coordination accountabilities are enshrined in the organization’s Core Commitments for Children in emergencies (CCCs). The UNICEF Evaluation Office commissioned a global evaluation of UNICEF’s CLA role to generate credible evidence for results achieved or not achieved by UNICEF in exercising its CLA role. By taking a forward-looking view while drawing on lessons from the past, this evaluation serves the dual purpose of promoting learning and accountability. The evaluation focuses on UNICEF’s fulfilment of its CLA role, not the wider results achieved by the clusters and their partners. Rather, the evaluation focuses more narrowly on UNICEF’s management of its own CLA responsibilities. Conducted by Avenir Analytics and Everywhere Humanitarian Response and Logistics Services, the evaluation was staged in three phases: the inception phase, the data collection and analysis phase, and the reporting phase. In the inception phase, in addition to exploring scoping issues and an action plan for conducting the exercise, the evaluation team undertook a rapid evaluability type of assessment to develop a logic model to be used to measure UNICEF’s cluster performance. In the data collection and analysis phase, the evaluation employed a mixed-method approach to derive the best perceptual and documentary evidence. Data collection took place at global and country levels, with emphasis on eight country case studies: Burkina Faso, El Salvador, Haiti, Nepal, the Philippines, South Sudan, the State of Palestine and Tajikistan. The evaluation found that partners believe UNICEF-led clusters and AORs are broadly effective. Human resources, institutional support and leadership are the primary factors influencing UNICEF effectiveness as a CLA. There is evidence to credit UNICEF-led clusters for having positive effects on geographic and thematic coverage. Roles and responsibilities are reportedly clearer at country level than at global level. The evaluation found, however, that global and country cluster coordinators clearly make substantial efforts to establish shared responsibilities by developing workplans, and other mapping of capacities and activities. UNICEF’s policies and systems have increasingly integrated the CLA roles, but CLA performance monitoring systems remain limited. There are clear indications that consolidation under the Global Cluster Coordination Unit (GCCU) is beginning to yield positive results. Investments in UNICEF surge capacity arrangements have been positive and should be further enhanced. UNICEF has also implemented its CLA role in an increasing number of contexts in response to demand for coordination services, and the organization should maintain and build on its strengths in partnership and coordination. On behalf of the Evaluation Office, I would like to thank Brian Majewski for his leadership in conducting the evaluation, and the evaluation team consisting of Hetty van Doorn, Mark Slezak, Natalia Gomez-Tagle Leonard, Ian Heigh, Kurt Wilson and Paule Heredote. I would also like to express our sincere gratitude to our colleagues in UNICEF headquarters, in particular Ted Chaiban, Dermot Carty and Gwyn Lewis, and to colleagues in the UNICEF regional offices and country offices that were involved in the evaluation in Burkina Faso, El Salvador, Haiti, Nepal, the Philippines, South Sudan, the State of Palestine and Tajikistan, as well as their government partners, for their contribution to the data collection. In addition, I would like to extend our thanks to the many external partners and individuals that contributed their time and inputs to this evaluation. Lastly, I would like to thank my colleagues in the Evaluation Office, including Robert McCouch, Erica Mattellone, Tammy Smith, Mathew Varghese and Dalma Rivero for their support and guidance to the evaluation team, which has been exemplary throughout the process. Colin M. Kirk Director Evaluation Office UNICEF New York CONTENTS ABBREVIATIONS AND ACRONYMS ................................................................................. vi EXECUTIVE SUMMARY ................................................................................................... viii RÉSUMÉ ANALYTIQUE .................................................................................................. xvii RESUMEN ....................................................................................................................... xxix 1 EVALUATION QUESTION ............................................................................................ 1 1.1. Rationale for the evaluation ............................................................................................ 1 1.2. Scope of the evaluation .................................................................................................. 2 1.3. Background and context ................................................................................................. 2 2 EVALUATION APPROACH AND METHODOLOGY ..................................................... 6 3 KEY FINDINGS AND ANALYSIS ................................................................................ 13 3.1. External coordination performance ................................................................................... 15 3.2 Internal CLA performance ................................................................................................. 23 3.3 Human resources performance ......................................................................................... 29 3.4 Scope and boundary issues .............................................................................................. 36 3.5 Cost-effectiveness, value for money and efficiency ........................................................... 40 4 CONCLUSIONS ........................................................................................................... 45 4.1 Conclusions by key area ..................................................................................................