Evaluation of Unicef's Cluster Lead

EVALUATION REPORT

EVALUATION OF UNICEF’S CLUSTER LEAD

AGENCY ROLE IN

HUMANITARIAN ACTION

EVALUATION OFFICE DECEMBER 2013

EVALUATION REPORT

EVALUATION OF

UNICEF’S CLUSTER LEAD

AGENCY ROLE IN

HUMANITARIAN ACTION

EVALUATION OFFICE DECEMBER 2013

Evaluation of UNICEF’s Cluster Lead Agency Role in Humanitarian Action Final Report

United Nations Children’s Fund Three United Nations Plaza New York, New York 10017

December 2013

UNICEF’s Evaluation Office manages evaluations and evaluative studies on a wide range of topics in order to contribute to learning about what makes for effective development, as well as supporting accountability for results. These evaluations seek to identify what works and what does not in terms of achieving sustainable and equitable development results, and to throw light on how and why interventions are successful or not under various circumstances. In assessing UNICEF’s support to governments and other development partners, these evaluations consider where, how and why progress is being made and the difference it is making in the lives of children, women and communities around the world.

By publishing evaluation reports, the Evaluation Office makes evaluation findings, lessons and conclusions available to a wide audience. Lessons learned from evaluation are expected to inform operational improvements and, ultimately, to support the achievement of better results. The publication of evaluation reports also supports accountability, by providing all interested parties with independently determined evidence relating to UNICEF’s performance. This provides a basis for informed dialogue and discussion, and helps assure all stakeholders that the organization operates in an open and transparent manner.

The contents of this report do not necessarily reflect the policies and views of UNICEF. The text has not been edited to official publication standards, and UNICEF accepts no responsibility for error. The designations in this publication do not imply an opinion on the legal status of any country or territory, or of its authorities, or the delimitation of frontiers.

The copyright for this report is held by the United Nations Children’s Fund. Permission is required to reprint/reproduce/photocopy or in any other way to cite or quote from this report in written form. UNICEF has a formal permission policy that requires a written request to be submitted. For non-commercial uses, the permission will normally be granted free of charge. Please write to the Evaluation Office at the address below to initiate a permission request.

For further information, please contact:

Evaluation Office United Nations Children’s Fund Three United Nations Plaza New York, New York 10017 evalhelp@unicef.org

PREFACE

The cluster approach is an important vehicle to ensure adequate coordination of humanitarian response. Introduced in 2005 within the wider context of humanitarian reform, the cluster approach is organized along a two-tiered structure involving roles and responsibilities at both global and country levels. UNICEF serves as Global Cluster Lead Agency (CLA) for three Clusters: Nutrition, Water, Sanitation and Hygiene (WASH) and, as co-CLA with Save the Children, for Education. In addition, UNICEF is Focal Point Agency for Child Protection area of responsibility (AOR) and co-Focal Point Agency with the United Nations Population Fund (UNFPA) for the Gender-based Violence (GBV) AOR. UNICEF is the organization with the most clusters and AORs under its remit, and its cluster coordination accountabilities are enshrined in the organization’s Core Commitments for Children in emergencies (CCCs).

The UNICEF Evaluation Office commissioned a global evaluation of UNICEF’s CLA role to generate credible evidence for results achieved or not achieved by UNICEF in exercising its CLA role. By taking a forward-looking view while drawing on lessons from the past, this evaluation serves the dual purpose of promoting learning and accountability. The evaluation focuses on UNICEF’s fulfilment of its CLA role, not the wider results achieved by the clusters and their partners. Rather, the evaluation focuses more narrowly on UNICEF’s management of its own CLA responsibilities.

Conducted by Avenir Analytics and Everywhere Humanitarian Response and Logistics Services, the evaluation was staged in three phases: the inception phase, the data collection and analysis phase, and the reporting phase. In the inception phase, in addition to exploring scoping issues and an action plan for conducting the exercise, the evaluation team undertook a rapid evaluability type of assessment to develop a logic model to be used to measure UNICEF’s cluster performance. In the data collection and analysis phase, the evaluation employed a mixed-method approach to derive the best perceptual and documentary evidence. Data collection took place at global and country levels, with emphasis on eight country case studies: Burkina Faso, El Salvador, Haiti, Nepal, the Philippines, South Sudan, the State of Palestine and Tajikistan.

The evaluation found that partners believe UNICEF-led clusters and AORs are broadly effective. Human resources, institutional support and leadership are the primary factors influencing UNICEF effectiveness as a CLA. There is evidence to credit UNICEF-led clusters for having positive effects on geographic and thematic coverage. Roles and responsibilities are reportedly clearer at country level than at global level. The evaluation found, however, that global and country cluster coordinators clearly make substantial efforts to establish shared responsibilities by developing workplans, and other mapping of capacities and activities. UNICEF’s policies and systems have increasingly integrated the CLA roles, but CLA performance monitoring systems remain limited. There are clear indications that consolidation under the Global Cluster Coordination Unit (GCCU) is beginning to yield positive results. Investments in UNICEF surge capacity arrangements have been positive and should be further enhanced. UNICEF has also implemented its CLA role in an increasing number of contexts in response to demand for coordination services, and the organization should maintain and build on its strengths in partnership and coordination.

On behalf of the Evaluation Office, I would like to thank Brian Majewski for his leadership in conducting the evaluation, and the evaluation team consisting of Hetty van Doorn, Mark Slezak, Natalia Gomez-Tagle Leonard, Ian Heigh, Kurt Wilson and Paule Heredote. I would also like to express our sincere gratitude to our colleagues in UNICEF headquarters, in particular Ted Chaiban, Dermot Carty and Gwyn Lewis, and to colleagues in the UNICEF regional offices and country offices that were involved in the evaluation in Burkina Faso, El Salvador, Haiti, Nepal, the Philippines, South Sudan, the State of Palestine and Tajikistan, as well as their government partners, for their contribution to the data collection.

In addition, I would like to extend our thanks to the many external partners and individuals that contributed their time and inputs to this evaluation. Lastly, I would like to thank my colleagues in the Evaluation Office, including Robert McCouch, Erica Mattellone, Tammy Smith, Mathew Varghese and Dalma Rivero for their support and guidance to the evaluation team, which has been exemplary throughout the process.

Colin M. Kirk

Director Evaluation Office UNICEF New York

CONTENTS

ABBREVIATIONS AND ACRONYMS ...... vi EXECUTIVE SUMMARY ...... viii RÉSUMÉ ANALYTIQUE ...... xvii RESUMEN ...... xxix 1 EVALUATION QUESTION ...... 1

1.1. Rationale for the evaluation ...... 1 1.2. Scope of the evaluation ...... 2 1.3. Background and context ...... 2 2 EVALUATION APPROACH AND METHODOLOGY ...... 6 3 KEY FINDINGS AND ANALYSIS ...... 13

3.1. External coordination performance ...... 15

3.2 Internal CLA performance ...... 23 3.3 Human resources performance ...... 29 3.4 Scope and boundary issues ...... 36

3.5 Cost-effectiveness, value for money and efficiency ...... 40 4 CONCLUSIONS ...... 45

4.1 Conclusions by key area ...... 45 5 RECOMMENDATIONS ...... 50 6 KEY SUCCESS FACTORS ...... 63 ANNEXES ...... 65

ABBREVIATIONS AND ACRONYMS

AOR area of responsibility HCT humanitarian country team APSSC Asia Pacific Shared Services HRBA human rights-based approach Center IAHP Interagency and Humanitarian CAR Central African Republic Partnerships Unit CAP Consolidated Appeal Process IASC Inter-Agency Standing CC cluster coordinator Committee CCCs Core Commitments for ICRC International Committee of the Children in Humanitarian Red Cross Action ICVA International Council of CERF Central Emergency Response Voluntary Agencies Fund IFRC International Federation of CHF Common Humanitarian Fund Red Cross and Red Crescent Societies CEE-CIS Regional Office for Central and Eastern Europe and INGO international non- Commonwealth of governmental organization Independent States ISDR United Nations Office for CLA Cluster Lead Agency Disaster Risk Reduction CLARE Cluster Lead Agency Role IT information technology Evaluation MENARO Middle East and North Africa CP child protection Regional Office CPWG Child Protection Working MoU Memorandum of Group Understanding DFID Department for International NDMA National Disaster Development, United Kingdom Management Agency DHR Division of Human Resources NGO non-governmental organization EAPRO East Asia and the Pacific Regional Office OCHA Office for the Coordination of Humanitarian Affairs (United EMOPS Office of Emergency Nations) Programmes ORE Other Resources–Emergency ERC Emergency Relief Coordinator (resources directed to (United Nations) emergency programmes and ESARO Eastern and Southern Africa humanitarian relief) Regional Office ORR Other Resources–Regular GBV gender-based violence (resources earmarked to GCC Global Cluster Coordinator specific projects and GCCU Global Clusters Coordination initiatives) Unit PCA Programme Cooperation GenCap Gender Standby Capacity Agreement Project PD Programme Division HC Humanitarian Coordinator RC Resident Coordinator

REA Regional Emergency Adviser UNEG United Nations Evaluation RO Regional Office Group RRT rapid response team UNFPA United Nations Population Fund SCHR Steering Committee for Humanitarian Response UNHCR United Nation High Commissioner for Refugees SMART specific, measurable, attainable, realistic and time- UNICEF United Nations Children’s bound Fund SSA Special Service Agreement WASH Water, Sanitation and Hygiene TA Temporary Assignment WCARO West and Central Africa Regional Office TACRO The Americas and the Caribbean Regional Office WFP World Food Programme ToR terms of reference WHO World Health Organization UN United Nations UNDP United Nations Development Programme

vii

EXECUTIVE SUMMARY

Evaluation Question Rationale for the evaluation The evaluation of UNICEF’s Cluster Lead The IASC initiated the cluster approach in Agency (CLA) role in humanitarian action 2005 to enhance coordination during has primarily sought to address the emergencies in the context of broader following overarching question: humanitarian reform. The approach was intended to improve humanitarian How well does UNICEF carry out its CLA response by increasing predictability, role and what should it do for the future? accountability and partnerships across actors. The evaluation was commissioned by the UNICEF Evaluation Office (EO) and was Together, the five clusters and AORs – conducted from January through June of led or co-led by UNICEF – represent 2013 by independent evaluators from responsibility for approximately one third Avenir Analytics. Based on the terms of of the IASC cluster system. To undertake reference (ToR) (see Annex 1) the these responsibilities, UNICEF has evaluation sought to generate credible invested significant time and resources. evidence regarding the results achieved As an institution, UNICEF needs to or not achieved by UNICEF in undertaking understand how it is performing as a CLA its CLA role. and how it could improve. During the evaluation inception phase, six uses of the Objective evaluation output were identified through consultation with internal and external The objective of the evaluation was to stakeholders: assess, as systematically and objectively as possible, the performance of UNICEF 1. Provide a benchmark for measuring as a CLA and area of responsibility (AOR) CLA performance. focal point at global and country levels. 2. Contribute to ongoing UNICEF planning processes (office The evaluation sought to generate management plans, the medium-term evidence regarding the results achieved strategic plan and cluster work or not achieved by UNICEF in undertaking planning). these roles. It is primarily intended for use 3. Provide UNICEF with unfiltered by UNICEF senior management to make critiques and information on how improvements in how it undertakes its partners view its implementation of the CLA and AOR roles within the Inter- CLA role towards improving its Agency Standing Committee (IASC) approaches. cluster system. 4. Contribute to influencing the IASC on unresolved cluster system issues. The evaluation is anticipated to serve the 5. Share information with partners dual purposes of learning and regarding good practice and capacity accountability. Its primary emphasis is required. forward-looking – seeking to enhance 6. Provide evidence on UNICEF learning by drawing lessons, identifying performance to key internal and opportunities and providing information for external stakeholders, including UNICEF to improve its capacities, donors. systems and approaches for exercising its CLA and AOR roles.

viii

Evaluation approach and feedback, and it was approved before methodology data collection began.

Design Data collection Based on the terms of reference developed by UNICEF’s Evaluation Office, Data collection took place through six over 100 lines of enquiry were reviewed country visits and in-person visits to and further refined during the inception Copenhagen, Geneva and New York, as phase of the evaluation. Questions were well as remote interviews and desk organized to look at UNICEF’s ability to research over a period of three months meet performance criteria based on the with a team of eight evaluators. The Organisation for Economic Co-operation evaluators collected data from more than and Development’s Development 1,000 people, including 282 UNICEF staff Assistance Committee criteria, namely, and 779 staff from partner organizations, relevance and appropriateness, as well as 790 documents. Data were effectiveness (further split into captured in the form of original notes then predictability, accountability and summarized per interview, focus group or partnership), efficiency, coherence, observation against each of the evaluation coverage and sustainability. indicators. The output of this stage was a deep repository of data for each case Seventy-two indicators were developed to study and at global level. measure how well UNICEF is performing in its CLA roles and to identify patterns Data analysis suggesting areas for future improvement. The data from the more than 1,000 Eight countries were selected by the reference points were synthesized by Evaluation Office for case studies to group and location. The results were reflect the diversity of capacity levels and summarized for each case against the operating contexts where UNICEF indicators, consolidated into 33 outcome clusters are operating. These were and capability factors, and then a Burkina Faso, El Salvador, Haiti, Nepal, quantitative measure was applied to the Philippines, South Sudan, the State of provide an overall indication of how well Palestine and Tajikistan. UNICEF meets the key outcomes and evaluation criteria. Data collection instruments were designed to allow the evaluators to collect Per case findings and global findings were information from multiple sources for each analysed and synthesized to produce of the 72 indicators and to allow for overall preliminary findings. These triangulation. The data collection findings were then organized based on instruments supported face-to-face and emergent trends and major issues under remote interviews, focus groups, online the following five key areas related to how surveys, observation and formal well UNICEF carries out its CLA role and documentary research. what it needs to do for the future: Interview guides and surveys were translated into French and Spanish to  External coordination performance facilitate participation.  Internal CLA performance  Human resources performance The output of the evaluation design was  Scope and boundary issues an inception report. This was presented to  Cost-effectiveness, value for money the Evaluation Office, to internal and and efficiency external focus group members for

Findings, analysis and potential recent evaluations.1 Stakeholders recommendations have been presented to noted UNICEF generally does the best case study country offices and regional that can be expected within the offices, as well as global UNICEF staff complex country and overarching and partners in participatory workshops coordination contexts. (in person and online) before the report  Human resources, institutional was completed. support and leadership are the primary factors influencing UNICEF effectiveness as a CLA according to Findings, analysis data from interviews, focus groups, and recommendations surveys and the document review. The findings have been analysed against  UNICEF clusters are having positive the future likely requirements of the key effects on geographic and thematic stakeholders. coverage according to surveys, interviews and focus groups. Recommendations were then developed Subnational coordination structures related to improvements UNICEF can have been established, although make to strengthen implementation of concerns were noted about their CLA roles. financial and staffing sustainability. Clusters and AORs have had positive The outputs are (i) a set of points that effects on thematic coverage through describe how well UNICEF is carrying out increased operations research and its CLA role and what it is not doing so advocacy, particularly in the Education well, and (ii) a set of recommendations Cluster and Gender-based Violence that link directly to the identified (GBV) AOR. weaknesses that, if implemented, will  The partnership aspects of UNICEF- facilitate the required improvement. led clusters and AORs are stronger at country level than for Global Clusters. Country-level partners agree Findings UNICEF adheres to principles of partnership, while Global Cluster/AOR Findings for key area 1: External members raised concerns particularly coordination performance regarding transparency. Roles and External coordination performance was responsibilities are reportedly unclear reviewed against three essential among global survey respondents, elements: although documents and observation show Global Cluster Coordinators 1. Overall effectiveness (GCCs) making significant efforts to 2. Coverage develop shared responsibility. 3. Partnership Country-level partners had questions regarding UNICEF representation of The main findings were: clusters at humanitarian country teams (HCT) distinctly from its institutional  Partners believe UNICEF is broadly interests. effective at exercising its country cluster coordination responsibilities across surveys, interviews and focus groups. This is supported by other 1 UNICEF, UNICEF’s Response to the Emergency in the Horn of Africa, 2011–2012: Lesson-learning exercise, Final Report, Evaluation Office, June 2012. x

3 Findings for key area 2: Internal level. Based on previous evaluations 4 CLA performance and internal documents this may stem from a more systemic confusion within Internal CLA performance was reviewed UNICEF on roles, accountabilities and against four key elements: authorities between headquarters,

regional offices and country offices. 1. Establishment of the Global Cluster This is significant because regional Coordination Unit (GCCU) offices are the only formal link in 2. Policy and performance monitoring UNICEF’s accountability chain between 3. The role of regional offices country offices and headquarters. 4. The comparative advantages of  UNICEF’s CLA roles were found to UNICEF broadly match its comparative

advantages in programming. The main findings were: However, Global Cluster members

question UNICEF’s programme  The evaluation found clear indications strengths and ability to leverage its that the consolidation of Global coordination skills and capacities. Cluster staff under the GCCU is

beginning to yield positive results across all of UNICEF’s cluster Findings for key area 3: Human operations. More than 70 per cent of resources performance UNICEF staff surveyed and the Human resources performance was majority of staff interviewed in case- reviewed against two key elements: study countries stated that they had seen moderate or substantial 1. Surge capacity and recruitment improvements in global coordination systems and leadership as a result of the 2. Cluster coordination capacity GCCU’s creation, despite this development organizational change being only one year old. The main findings were:  UNICEF’s policies and systems have increasingly integrated the CLA  Surge capacity and recruitment roles but CLA performance systems for clusters have improved monitoring systems remain limited. since 2010, as evidenced by the Country office reporting under the difference in findings between Humanitarian Performance Monitoring evaluations of the Haiti5 earthquake Toolkit2 is voluntary and limited to a and Horn of Africa6 responses. series of “yes/no” questions regarding Management of surge capacity inputs and outputs, rather than quality systems remains fragmented, however, and outcomes. and there is no overarching strategy. The role of regional offices related to CLA responsibilities remains

unclear. While the Core Commitments 3 UNICEF’s Response to the Emergency in the for Children in Humanitarian Action Horn of Africa, 2011–2012. (CCCs) articulate a coordination 4 UNICEF, ‘Cluster Issues Matrix: Internal’, Internal mandate, they have not been Working Document, Programme Division, translated into clear responsibilities and December 2010. 5 UNICEF, ‘Independent Review of UNICEF’s accountability mechanisms at regional Operational Response to the January 2010 Earthquake in Haiti’, Evaluation Office, September 2011. 2 UNICEF, ‘Humanitarian Performance Monitoring – 6 UNICEF’s Response to the Emergency in the Toolkit and Guide’,June 2011. Horn of Africa, 2011–2012.

 Double-hatting (the practice of development coordination, with no performing several organizational formal IASC activation. functions) remains prevalent and  This is cluster “scope creep” and diminishes effectiveness. Even in significantly affects UNICEF’s ability ongoing emergency situations or active to perform well as a CLA in terms of clusters, only 28 per cent of staff work relevance, predictability, partnership, full time on cluster activities. efficiency and sustainability.  Cluster coordinator capacity development is limited to classroom- Findings for key area 5: based awareness training, and no Cost-effectiveness, value for money systematic efforts are in place to build awareness and consistency of practice and efficiency among country office representatives. Cost-effectiveness, value for money and efficiency were reviewed against three key Findings for key area 4: elements:

Scope and boundary issues 1. Cost-effectiveness and value for Scope and boundary issues were money reviewed against two key elements: 2. Linkages with other internal and external initiatives 1. The scope of UNICEF’s cluster 3. Co-leadership arrangements responsibilities 2. The operational role of UNICEF’s The main findings were: clusters and AORs  Staff and partners strongly perceive The main findings were: that the outcomes of cluster work justify their organization’s  The effects of clusters on lasting investment. In surveys, 83 per cent of coordination and national response UNICEF country staff and 84 per cent capacity systems is limited by a lack of country partners stated that cluster of guidance on transition, decentralized outcomes justify their investment. This authority and funding, and an unclear needs to be critically reviewed against role for clusters in capacity-building. the findings in key area 4.  Clusters are increasingly filling gaps  Cross-cluster coordination was beyond emergency coordination. found to be insufficient, and Indicatively, of the 34 UNICEF clusters integrated programming approaches 7 active in 2012 only 14 are operated remain the exception rather than the fully in line with the IASC cluster rule. objectives and guidelines (time-bound  Co-leadership arrangements have and emergency-focused). Ten of the generally improved since 2012, remaining 20 may have been correctly although some challenges remain due activated, but are currently operating to members’ expectations and capacity as long-term sectoral development imbalances. coordination mechanisms. The others were activated in place of sectoral

7 UNICEF, Internal Working Document and Database, ‘A Survey of the Operational Status and Staffing of UNICEF-led and Co-led Clusters and AORs in the Field’, Final Report and Annexes, July 2012. xii

Conclusions and to a global CLA strategic management recommendations effort.

Key area 1: External coordination Key area 3: Human resources performance performance UNICEF strengths in cluster coordination Investments in UNICEF surge capacity performance, coverage and country level arrangements have been positive and partnership practice should be maintained should be further enhanced. The main and replicated. The key areas for areas for improvement are (i) a improvement are (i) weaknesses in cluster fragmented human resources approach coordinators’ abilities to identify gaps and and the lack of a career path for solve problems, (ii) challenges in UNICEF coordination staff and (ii) the fact that staff representation of cluster members at capacity development for cluster work is HCTs and (iii) a lack of clarity on limited to classroom-based basic coordination roles and responsibilities awareness training across the externally with partners. organization.

The key recommendation is to develop a The major recommendation is to develop “cluster-ready” initiative with regional an integrated strategy for surge capacity offices and/or the GCCU, using globally and a UNICEF-wide effort for developing developed capacity assessment and coordination staff. enhancement tools. In short, this means increasing the preparedness of country Key area 4: Scope and boundary offices in high-risk areas so they can UNICEF has implemented its CLA role in activate and disengage from the an increasing number of contexts in appropriate cluster mechanism when response to demand for coordination necessary. This ensures that clusters services. The most important area for provide effective and efficient support to improvement is in the lack of organization- vulnerable communities within the wide cluster guidance and the inconsistent humanitarian system. application of IASC policies, which makes

it difficult to meet divergent stakeholder Key area 2: Internal CLA expectations. performance UNICEF’s positive efforts to consolidate The primary recommendation is to cluster support under the GCCU and establish UNICEF policy and guidance to integrate the CLA role into its policies clearly interpret and articulate CLA policy should be maintained and built upon. The (coherence) and disseminate it to ensure key areas for improvement are (i) a lack of consistent application (fidelity), with focus clarity on internal coordination on the most critical emergency situations, responsibilities and roles, (ii) unclear and management of stakeholder regional office roles and (iii) no systematic expectations. approach to ensure country representatives consistently implement Key area 5: Cost-effectiveness, CLA roles. value for money and efficiency

The key recommendation is to strengthen UNICEF should maintain and build upon management systems to support the CLA its strengths in perceived partner value role, including performance measurement, and co-leadership arrangements. The key strengthened regional office roles and area for improvement is over-investment better connecting country representatives in use of clusters at field level to fill gaps

xiii in long-term sectoral or humanitarian  ‘Independent Review of UNICEF’s coordination. Operational Response to the January 2010 Earthquake in Haiti’, Evaluation The key recommendation is to mitigate Office, UNICEF, 2011. the use of clusters in inappropriate  UNICEF’s Response to the scenarios by developing models and tools Emergency in the Horn of Africa, for non-cluster coordination, including 2011–2012: Lesson-learning exercise, transition points for country offices, and to Evaluation Office, UNICEF, 2012. establish clarity on the role of clusters for national capacity-building to ensure The following six actions can be taken to efficient and fit-for-purpose coordination support successful implementation of the approaches. evaluation recommendations:

Key success factors  Engage senior leadership across Some of the evaluation findings and UNICEF. recommendations are similar to those  Implement recommendations as a found in the following documents: package.  Engage partners in developing  Cluster Approach Evaluation 2 strategies for the future. Synthesis Report, GPPi and Groupe  Engage IASC bodies in discussions URD, 2010. based on the evaluation.  Synthesis of Cluster Lessons from  Organize discussions with donor UNICEF’s response to the Pakistan organizations to discuss how their Floods, 2011. funding shapes CLA implementation.  Implementing the Cluster Approach: A  Engage national governments before synthesis of documented lessons and at the beginning of cluster 2006–2010, Programme Division, operations to establish and agree clear UNICEF, 2011. scope and transition points.

xiv

Table 1: Summary of findings, conclusions and recommendations

Summary of findings, conclusions and recommendations

Findings - what Findings - what Key area UNICEF is doing UNICEF is doing Conclusions Recommendations well less well • High perception of ▪ Weaknesses in UNICEF is Develop a "cluster- overall effectiveness. global partnership increasingly effective ready" initiative to performance. at cluster increase country • Increasingly robust coordination, but it office preparedness cluster coordination ▪ Lack of clarity on needs to address for cluster activation services. coordination roles lack of clarity on in high-risk countries. and responsibilities: roles and • Strong country-level internally across partnership responsibilities with units and externally partners, variance in External performance. with partners. coordination partner expectations performance ▪ Gaps in cluster and gaps in coordinators’ abilities coordinator abilities to identify gaps and to ensure continued solve problems. improvement. ▪ Challenges in UNICEF representation of cluster members at HCTs. • CLA role well • Lack of clarity on UNICEF has Strengthen UNICEF- integrated at the roles and instituted positive wide management broad policy level. responsibilities changes in its global systems to support internally across approach to the CLA role, • Improvements in units. managing the CLA including coordination and roles, but has not yet strengthening the leadership due to • Unclear regional established the role of regional creation of GCCU. office roles and no clarity on roles and offices and better systematic approach Internal CLA • Initial efforts to responsibilities connecting country to ensure country across internal units representatives to a performance develop CLA strategy representatives and information and ensured a chain global CLA strategic consistently of accountability that management management implement CLA roles. function. would enable structure. consistency and • Broad match quality across between UNICEF country offices. comparative advantages and CLA roles. • Significant • Fragmented HR UNICEF's ability to Develop an improvement in approach and lack of rapidly deploy integrated strategy cluster surge a coordination career coordinators has for human resources capacity with path. improved but surge capacity and increased ability to continued UNICEF coordination • Continued Human deploy coordinators fragmentation, lack of staff development. prevalence of resources in a timely fashion. an integrated HR double-hatting performance strategy, continued reduces prevalence of effectiveness. double-hatting and • Capacity limited approach to development limited coordinator capacity to basic awareness development reduce

training. the consistency and quality of its cluster human resources.

• Active participation • Lack of Cluster “scope creep” Increase coherence in IASC and organization-wide significantly affects (interpretation and Transformative cluster guidance and UNICEF's ability to articulation) and then Agenda work and inconsistent perform well as a fidelity global efforts to application of IASC CLA in terms of (understanding and clarify cluster policies policies. relevance, consistent and guidance. predictability, application) through • Lasting effects on partnership, UNICEF CLA policy coordination and efficiency and and practice. national response sustainability. capacity limited by Scope and lack of guidance on boundary transition and issues national capacity building, decentralized authority and funding. • Clusters increasingly filling gaps beyond emergency coordination. • High by staff and • Much investment at Overall UNICEF Mitigate the use of partners that field level in long- investment in clusters in outcomes justify term sectoral implementing its CLA inappropriate investments. coordination role is not excessive scenarios by contexts. and staff and developing models • Investment in CLA partners agree and tools for non- Cost- role not excessive. • Cross-cluster cluster outcomes cluster coordination, effectiveness, coordination is justify their including transition • Co-leadership insufficient and value for arrangements have investments, but points for country money and integrated investments in offices, and establish generally improved programming efficiency since 2012. clusters to fill gaps in clarity on the role of approaches remain long-term sectoral clusters, if any, for the exception rather coordination offer national capacity- than the rule. opportunities for cost building to ensure savings or efficient and fit-for- realignment. purpose coordination approaches.

xvi

RÉSUMÉ ANALYTIQUE

L’interrogation de l’évaluation du système de clusters du Comité permanent inter-organisations (IASC). L’évaluation du rôle de l’UNICEF en tant qu’organisme chef de file pour les 8 L’évaluation devrait jouer un double rôle, Groupes sectoriels (CLA - Cluster Lead enseignement et responsabilisation. Sa Agency) dans l’action humanitaire a perspective dominante est prospective – il principalement cherché à répondre à la s’agit de chercher à renforcer principale question, qui peut se formuler l’apprentissage en tirant des leçons de de la manière suivante : l’expérience, d’identifier des perspectives favorables et de fournir à l’UNICEF les Avec quelle efficacité l’UNICEF remplit-il informations dont il a besoin pour son rôle de CLA et qu’est-ce que améliorer les capacités, les systèmes et l’organisation doit envisager à l’avenir ? les méthodes auxquels il fait appel dans ses rôles de CLA et d’AOR. Cette évaluation a été commandée par le Bureau de l’évaluation de l’UNICEF et a été menée de janvier à juin 2013 par des Raison d’être de l’évaluation évaluateurs indépendants du cabinet Le Comité permanent inter-organisations Avenir Analytics (voir Annexe 1). Se (IASC) a lancé la politique des clusters en fondant sur les termes du mandat qui 2005 pour renforcer la coordination avait été défini, l’évaluation a cherché à pendant les situations d’urgence et dans établir par le recueil de données factuelles le contexte d’une réforme plus globale de crédibles quels résultats l’UNICEF avait l’action humanitaire. Cette méthode était obtenus ou n’avait pas obtenus dans son destinée à améliorer les interventions rôle de CLA. humanitaires en renforçant l’étude prospective des catastrophes Objectif humanitaires pour les rendre plus prévisibles, ainsi qu’à renforcer les L’objectif de l’évaluation était de juger, partenariats entre les différents acteurs et aussi systématiquement et objectivement leur obligation de responsabilité mutuelle. que possible, les résultats obtenus par l’UNICEF comme CLA et point focal de Pris ensemble, les cinq clusters et les Zone de responsabilité (area of AOR – gérés ou cogérés par l’UNICEF – responsibility - AOR) au niveau mondial et assument environ un tiers des au niveau de différents pays. responsabilités dans le système de clusters du Comité permanent inter- L’évaluation a cherché à établir de organisations (IASC). Pour endosser ces manière factuelle et crédible les résultats responsabilités, l’UNICEF a investi temps obtenus, ou pas, par l’UNICEF en et ressources en quantités importantes. assumant ces rôles. Cette étude est En tant qu’institution, l’UNICEF a besoin destinée en priorité aux cadres supérieurs de savoir avec quelle efficacité elle joue le de l’UNICEF, afin de leur permettre rôle de CLA et comment il peut d’améliorer la manière dont l’organisation l’améliorer. Pendant la phase initiale de joue ces rôles de CLA et d’AOR au sein l’évaluation, des consultations avec des parties prenantes internes et externes ont permis d’identifier six usages qui 8 (Appelés aussi cluster, le mot que nous utiliserons dans ce document)

xvii pouvaient être éventuellement faits des efficience, cohérence, couverture et résultats de l’évaluation : viabilité (pérennité). Soixante-douze indicateurs ont été mis au 1. Fournir une base de référence pour point pour mesurer l’efficacité avec jauger l’action de la CLA. laquelle l’UNICEF joue ses rôles de CLA 2. Contribuer aux processus de et pour identifier des tendances planification en cours à l’UNICEF : susceptibles de suggérer de futures Plans de gestion des bureaux, Plan améliorations. Huit pays ont été stratégique à moyen terme et sélectionnés par le Bureau de l’évaluation planification du travail en cluster. pour mener des études de cas afin de 3. Fournir à l’UNICEF une évaluation refléter la diversité des niveaux de critique non filtrée et des informations capacités entre pays et des contextes sur la manière dont ses partenaires dans lesquels opèrent les clusters de voient la façon dont il joue son rôle de l’UNICEF. Ces pays sont le Burkina Faso, CLA afin de lui permettre d’améliorer le Salvador, Haïti, le Népal, l’État de ses méthodes de travail Palestine, les Philippines, le Soudan du 4. Contribuer à influencer le Comité Sud et le Tadjikistan. permanent inter-organisations (IASC) concernant les questions non résolues Les instruments de collecte des données qui se posent au système des ont été conçus pour permettre aux clusters. évaluateurs de recueillir des informations 5. Échanger avec les partenaires des de sources multiples pour chacun des 72 informations concernant les bonnes indicateurs et pour pouvoir procéder à des pratiques et les capacités requises. « triangulations ». Ces instruments de 6. Fournir des éléments factuels sur les collecte de données ont été utilisés pour résultats obtenus par l’UNICEF à des des entretiens en face-à-face et des parties prenantes clé, internes aussi entretiens indirects à distance, pour des bien qu’externes, dont les bailleurs de discussions en groupe de contrôle, des fonds. sondages en ligne, des observations et une recherche documentaire Technique d’évaluation et conventionnelle. Les guides d’entretien et méthodologie les questions d’enquête avaient été traduits en français et en espagnol pour Conception faciliter la participation. À partir des termes du mandat mis au point par le Bureau de l’évaluation de Le produit de la phase de conception de l’UNICEF, plus de 100 lignes de l’évaluation à été un rapport préliminaire recherche ont été examinées et encore qui a été présenté au Bureau de affinées pendant la phase préliminaire de l’évaluation et à des personnes ayant l’évaluation. Les questions ont été participé aux discussions en groupes de organisées de façon à examiner la contrôle, certaines faisant partie de capacité qu’a l’UNICEF de satisfaire à des l’UNICEF, d’autres venant de l’extérieur. critères de résultats basés sur ceux du Le rapport leur a été présenté afin de Comité d’aide au développement de recueillir leurs réactions et leurs l’Organisation de coopération et de contributions et pour approbation avant le développement économiques début de la collecte des données. (CAD/OCDE), c’est-à-dire pertinence et adéquation, efficacité (divisé entre questions de capacité de prévision, de responsabilité et de partenariat), xviii

Collecte des données CLA et ce que l’organisation doit faire à La collecte des données à été entreprise l’avenir : par une équipe de huit évaluateurs au cours de six visites dans les pays  Résultats concernant la concernés et de déplacements en coordination externe personne à Genève, New York et  Résultats concernant le rôle interne Copenhague, ainsi que par des entretiens en tant que CLA à distance et une recherche  Résultats dans le domaine des documentaire. Les évaluateurs ont ressources humaines recueilli des informations auprès de plus  Questions d’ampleur et de limites de de 1000 personnes, dont 282 membres l’action du personnel de l’UNICEF et 779  Rapport coût-efficacité, optimisation membres du personnel d’organisations des ressources et efficience partenaires, ils ont également consulté 790 documents. Les données ont été Les constatations, les analyses et les recueillies sous forme de notes originales propositions de recommandations ont été qui ont ensuite été synthétisées par présentées au Bureaux de pays des États entretien, groupe de contrôle ou ayant fait l’objet des études de cas ; aux observation en fonction des indicateurs personnels de l’UNICEF à travers le d’évaluation définis. Le produit de cette monde et aux partenaires impliqués dans phase du travail d’évaluation a été un les ateliers participatifs (en personne ou important fonds de données constitué en ligne) avant que l’étude ne soit pour chacune des études de cas ainsi que finalisée. pour le travail d’évaluation au niveau mondial. Constatations, analyses Analyse des données et recommandations Les données obtenues à partir des plus Les constatations ont été analysées en de 1000 points de référence ont été fonction des besoins que les principales synthétisées par groupe et par localisation parties prenantes éprouveront géographique. Les résultats ont été probablement à l’avenir. résumés pour chaque cas en fonction des indicateurs regroupés sous 33 facteurs de Des recommandations ont été ensuite réalisation et de capacité; une mesure mises au point en fonction des quantitative leur a ensuite été appliquée améliorations éventuelles que l’UNICEF pour fournir une indication globale sur le pourrait apporter à ses rôles de CLA. degré auquel l’UNICEF satisfait aux Les réalisations sont (i) un ensemble de critères clés de réalisations et points qui décrivent l’efficacité avec d’évaluation. laquelle l’UNICEF remplit son rôle de CLA ainsi que les domaines où son action n’est Les constatations tirées de chaque cas et pas totalement satisfaisante ; et (ii) un celles relatives au niveau mondial ont été ensemble de recommandations qui sont analysées et synthétisées pour obtenir directement liées aux faiblesses des constatations préliminaires globales. identifiées et dont la mise en œuvre Celles-ci ont été ensuite organisées sur la faciliterait les améliorations requises. base des tendances émergentes et des questions clés figurant dans les cinq rubriques principales suivantes qui indiquent la mesure dans laquelle l’UNICEF joue avec efficacité son rôle de

xix

Constatations coordination infra-nationales ont été er mises en place, mais on a noté Constatations concernant le 1 cependant des préoccupations domaine clé : résultats concernant concernant la capacité de leur assurer la coordination externe le financement et le personnel Les résultats concernant la coordination nécessaires de façon durable. Clusters externe ont été examinés en fonction de et AOR ont eu des effets positifs sur la trois critères : couverture thématique en stimulant la recherche opérationnelle et les 1. Efficacité globale activités de plaidoyer, particulièrement 2. Couverture pour ce qui concerne le cluster 3. Partenariats éducation et l’AOR relatif à la violence basée sur le genre. Les principales constatations sont les  Les aspects relatifs au partenariat suivantes : des clusters dirigés par l’UNICEF au niveau des pays sont plus forts  Les partenaires pensent que qu’au niveau des clusters mondiaux. l’UNICEF exerce globalement avec Les partenaires agissant au niveau efficacité ses responsabilités de d’un pays conviennent que l’UNICEF coordination du cluster dans les adhère fiablement aux principes de pays concernés, opinion exprimée partenariat, par contre les membres dans les enquêtes, les entretiens des clusters ou d’AOR de niveau comme dans les groupes de contrôle. mondial expriment des inquiétudes, Ce résultat est conforme à d’autres particulièrement en ce qui concerne la évaluations9. Les parties prenantes question de la transparence. Les ont noté que l’UNICEF faisait du parties prenantes du niveau mondial mieux qu’il était possible d’espérer ayant répondu à l’enquête ne dans les situations complexes sembleraient pas avoir une conception présentées par les pays concernés et parfaitement claire des rôles et des par les problématiques globales de responsabilités ; cependant, les coordination. documents et l’observation montrent  Les ressources humaines, le que les coordonnateurs de Cluster caractère institutionnel et le rôle mondial (Global Cluster Coordinators - moteur de l’UNICEF sont les GCC) font des efforts importants pour principaux facteurs qui expliquent mettre en place un partage des l’efficacité de l’organisation dans responsabilités. Les partenaires de son rôle de CLA selon les données niveau national ont exprimé des inquiétudes concernant le fait que recueillies dans les enquêtes, les entretiens comme dans la recherche l’UNICEF représente les clusters dans les Équipes humanitaires de pays documentaire. distinctement de ses intérêts  Les clusters de l’UNICEF ont des institutionnels. effets positifs sur la couverture

géographique et thématique, opinion exprimée dans les enquêtes, les entretiens comme dans les groupes de contrôle. Des structures de

9 UNICEF, UNICEF’s Response to the Emergency in the Horn of Africa, 2011–2012: Lesson-learning exercise, Rapport final, Bureau de l’évaluation, juin 2012. xx

Constatations concernant le 2e concernant les intrants et les extrants domaine clé : résultats concernant plutôt que la qualité et les réalisations.  Le rôle des Bureaux régionaux en le rôle interne en tant que CLA relation aux responsabilités des Les résultats concernant le rôle interne en CLA reste peu clair. Bien que les tant que CLA ont été examinés en Principaux engagements pour les fonction de quatre questions clés : enfants dans les situations humanitaires comportent une mission 1. Mise en place de la Cellule de de coordination, celle-ci n’a pas été coordination des clusters mondiaux traduite en mécanismes de niveau (Global Cluster Coordination Unit - régional assurant clairement le respect GCCU) des responsabilités et l’exigence de 2. Politique et suivi des résultats obtenus rendre compte de l’action engagée. Sur 3. Rôle des Bureaux régionaux la base d’évaluations précédentes11 et 4. Avantages comparatifs de l’UNICEF de documents internes12, on peut avancer que ceci provient d’une Les principales constatations sont les confusion plus systémique au sein de suivantes : l’UNICEF concernant la répartition des rôles, des exigences de compte rendu  L’évaluation a trouvé de claires et de l’autorité entre le siège, les indications que le regroupement du Bureaux régionaux et les Bureaux de personnel des clusters de niveau pays. Ce fait est significatif, car les mondial dans la cadre de la cellule Bureaux régionaux sont le seul lien GCCU commence à avoir des officiel dans la chaîne de responsabilité résultats positifs pour toutes les de l’UNICEF entre les Bureaux de pays opérations de l’UNICEF gérée par des et le Siège. clusters. Plus de 70 % du personnel de  Les rôles de l’UNICEF en tant que l’UNICEF ayant répondu à l’enquête et CLA sont d’après cette étude la majorité de ceux qui ont été globalement adaptés à ses interrogés dans le cadre des études de avantages comparatifs dans le cas de pays ont affirmé qu’ils avaient domaine des programmes. perçu des améliorations modérées ou Cependant les points forts de l’UNICEF bien substantielles dans la coordination dans ce domaine et son aptitude à tirer et la direction au niveau mondial en parti de son savoir-faire et des ses conséquence de la création de la capacités en matière de coordination cellule GCCU, et ce malgré le fait que sont mises mondiaux. cette réorganisation ne date que d’un an. Constatations concernant le 3e  Les politiques et les dispositifs de l’UNICEF ont de mieux en mieux domaine clé : résultats dans le intégré les rôles de CLA, mais les domaine des ressources humaines dispositifs de suivi des résultats Les résultats obtenus dans la sphère des obtenus par les CLA restent ressources humaines ont été examinés en insuffisants. L’utilisation du Manuel et fonction de deux questions clés : du guide de suivi des résultats de l’action humanitaire 10 par les Bureaux de pays est facultative et limitée à une série de questions « Oui/Non » 11 UNICEF’s Response to the Emergency in the Horn of Africa, 2011–2012. 12 UNICEF, ‘Cluster Issues Matrix: Internal’, Internal 10 UNICEF, ‘Humanitarian Performance Monitoring Working Document, Programme Division, – Toolkit and Guide’, juin 2011. décembre 2010.

xxi

1. Capacités de montée en puissance Constatations concernant le 4e (surge) et dispositifs de recrutement domaine clé : questions d’ampleur 2. Développement des capacités de coordination des Clusters et de limites de l’action Les questions d’ampleur et de limites de Les principales constatations sont les l’action des clusters ont été examinées en suivantes : fonction de deux aspects clés :

 Les dispositifs permettant d’assurer 1. L’ampleur des responsabilités de la montée en puissance (surge) et le l’UNICEF au sein des clusters recrutement pour les clusters se 2. Le rôle opérationnel des clusters et sont améliorés depuis 2010, comme des AOR de l’UNICEF le prouve la différence dans les faits constatés par les évaluations entre Les principales constatations sont les l’intervention menée en réponse au suivantes : tremblement de terre d’Haïti13 et celle organisée dans la corne de l’Afrique14.  Les effets durables des clusters sur Cependant, la gestion des dispositifs les dispositifs de coordination et sur de montée en puissance (surge) reste les capacités d’intervention au fragmentée et ne comporte pas de niveau national sont limités par un stratégie globale. manque de conseils d’orientation sur  Les « doubles casquettes » restent les questions de transition, d’autorité très fréquentes et affaiblissent décentralisée et de financement, ainsi l’efficacité de l’action. Même dans les que par un manque de clarté dans la situations d’urgence actuelles ou dans définition du rôle des clusters dans le les clusters actifs, 28 % seulement du renforcement des capacités. personnel est engagé à temps plein  Les clusters servent de plus en plus dans des activités concernant les à remédier à des carences qui ne clusters. concernent pas la coordination dans  Le développement des capacités les situations d’urgence. Il est révélateur que sur les 34 clusters actifs des coordonnateurs de Cluster est 15 limité à des sessions de sensibilisation en 2012 , 14 seulement opèrent en organisées en classe, et aucun effort complète conformité avec les objectifs n’est fait pour mieux sensibiliser les et les directives du Comité permanent représentants des Bureaux de pays et inter-organisations (IASC) sur les renforcer l’uniformité de leurs clusters qui demandent que leur pratiques. intervention soit limitée dans le temps et consacrée à une situation d’urgence ; 10 des 20 autres clusters ont sans doute été correctement activés, mais opèrent actuellement comme dispositifs de coordination à long terme du développement sectoriel. Le reste a été activé en lieu et place d’une coordination du développement

13 UNICEF, ‘Independent Review of UNICEF’s Operational Response to the January 2010 15 UNICEF, Internal Working Document and Earthquake in Haiti’, Bureau de l’évaluation, Database, ‘A Survey of the Operational Status and septembre 2011. Staffing of UNICEF-led and Co-led Clusters and 14 UNICEF’s Response to the Emergency in the AORs in the Field’, Final Report and Annexes, Horn of Africa, 2011–2012. juillet 2012. xxii

sectoriel sans que cette activation ait intégrées restent l’exception plutôt été officiellement le fait du Comité que la règle. permanent inter-organisations (IASC).  Les dispositifs de codirection se  Il s’agit là d’une « dérive » de sont généralement améliorés depuis l’ampleur des responsabilités 2012, bien que certaines difficultés assignées aux clusters qui a des subsistent en raison des attentes des conséquences importantes pour la membres des clusters et des capacité de l’UNICEF de remplir son déséquilibres de leurs capacités. rôle de CLA de manière satisfaisante, que ce soit en termes Conclusions and de pertinence, de capacité recommandations prévisionnelle, d’efficacité ou de viabilité. 1er domaine clé : résultats concernant la coordination externe e Constatations concernant le 5 Les bons résultats de l’UNICEF dans le domaine clé : rapport coût- domaine de la coordination de l’action des efficacité, optimisation des clusters, de la couverture qu’ils assurent et de leurs pratiques de partenariat au ressources et efficience niveau national doivent être préservés et Rapport coût-efficacité, optimisation des reproduits. Les points les plus importants ressources et efficience ont été examinés sur lesquels une amélioration est en fonction de trois critères : nécessaire sont (i) les faiblesses dans les capacités des coordonnateurs de cluster 1. Rapport coût-efficacité et optimisation d’identifier les carences et de résoudre les des ressources problèmes qui se manifestent, (ii) les 2. Liens avec d’autres initiatives internes problèmes que pose la représentation par et externes l’UNICEF des membres des clusters dans 3. Dispositifs de co-direction les Équipe de pays pour l'action humanitaire, (iii) un manque de clarté sur Les principales constatations sont les les rôles et les responsabilités dans la suivantes : coordination externe avec les partenaires

 Le personnel et les partenaires La recommandation principale est de considèrent clairement que les mettre au point une initiative de « prêt-à- réalisations issues du travail des organiser » sur les clusters pour laquelle clusters justifient l’investissement les Bureaux régionaux et/ou la cellule qu’y fait l’organisation. Les enquêtes GCCU pourront utiliser les outils indiquent que 83 % du personnel des d’évaluation et de renforcement des Bureaux de pays et 84 % de celui des capacités conçus au niveau mondial. En partenaires nationaux pensent que les bref, cela signifie améliorer la préparation réalisations des clusters justifient les des Bureaux de pays dans les zones à investissements qui y sont faits. Ce hauts risques de façon à leur permettre de résultat doit être évalué de manière s’engager et de se désengager des critique en le comparant aux mécanismes de cluster en fonction des constatations faites dans le domaine clusters o nécessités. Ceci assurera que les clé n 4. apportent dans le cadre du système  La coordination entre les différents d’action humanitaire un soutien efficace et clusters a été jugée insuffisante et efficient aux communautés vulnérables. les techniques de programmation

xxiii

2e domaine clé : résultats Les recommandations principales sont de concernant le rôle interne en tant mettre au point une stratégie intégrée de développement des capacités de montée que CLA en puissance et de faire un effort à Les efforts positifs faits par l’UNICEF pour l’échelle de tout l’UNCEF pour former un consolider le soutien fourni par les personnel de coordination spécialisé. clusters dans le cadre de la cellule GCCU et d’intégrer le rôle de CLA dans ses 4e domaine clé : questions politiques doivent être poursuivis et menés plus loin. Les points les plus d’ampleur et de limites de l’action importants sur lesquels une amélioration L’UNICEF a été amené à assumer son est nécessaire sont (i) un manque de rôle de CLA dans un nombre de situations clarté dans les questions de coordination croissant en réponse à la demande de et de responsabilité internes ainsi que sur coordination des services. Les points le les rôles, (ii) des rôles mal définis pour les plus importants sur lesquels une Bureaux régionaux, (iii) l’absence d’une amélioration est nécessaire sont de approche systématique pour assurer que remédier à l’absence dans toute les Représentants de pays remplissent l’organisation d’orientation et de conseils leur rôle de manière cohérente dans le concernant le travail de cluster, et au cadre des CLA. manque d’uniformité dans l’application des politiques du Comité permanent inter- La recommandation principale est de organisations (IASC) qui rend difficile de renforcer les systèmes de gestion pour répondre aux attentes divergentes des appuyer le rôle de CLA, y compris dans diverses parties prenantes. les domaines de la mesure des résultats obtenus, du renforcement du rôle des La recommandation principale est que Bureaux régionaux et d’une meilleure l’UNICEF définisse une politique et des liaison entre les Représentants de pays et directives permettant de formuler et un effort de gestion stratégique au niveau d’interpréter clairement les orientations mondial en faveur des CLA. sur le travail de CLA (cohérence) et de les diffuser afin d’assurer une application 3e domaine clé : résultats dans le uniforme (fidélité) en se concentrant sur les situations d’urgence les plus critiques domaine des ressources humaines et sur la gestion des attentes des parties Les investissements dans les dispositifs prenantes de montée en puissance (surge) de l’UNICEF ont eu des effets positifs et 5e domaine clé : rapport coût- doivent être encore augmentés. Les points les plus importants sur lesquels efficacité, optimisation des une amélioration est nécessaire sont ressources et efficience (i) une méthode opératoire fragmentée L’UNICEF doit préserver les points forts dans le domaine des ressources que représente sa perception de la valeur humaines et l’absence de parcours des partenaires et des dispositifs de professionnel défini pour le personnel codirection et en tirer le meilleur parti chargé de la coordination, (ii) une action possible. Le point le plus important sur de développement des capacités du lequel une amélioration est nécessaire est personnel qui est limité dans le domaine le surinvestissement dans l’utilisation des des clusters à la formation théorique clusters sur le terrain pour remédier aux donnée dans toute l’organisation sous carences de la coordination sectorielle et forme d’une sensibilisation de base. humanitaire à long terme.

xxiv

La recommandation principale est de L’application des six mesures suivantes modérer le recours aux clusters quand les facilitera la mise en œuvre des scénarios ne sont pas adaptés en mettant recommandations de l’évaluation et leur au point des modèles et des outils de succès : coordination ne faisant pas appel à ces clusters, y compris des points de  Impliquer les cadres supérieurs dans transition pour les Bureaux de pays ; il est l’ensemble de l’UNICEF. également nécessaire de clarifier le rôle  Appliquer l’ensemble du train de des clusters dans le renforcement des recommandations. capacités de façon à mettre en place des  Obtenir la collaboration des partenaires méthodes de coordination efficaces et pour mettre au point les stratégies adaptées aux objectifs déterminés. futures.  Inviter les organismes participants au Les principaux facteurs de succès Comité permanent inter-organisations Certaines des constatations et des (IASC) à débattre des résultats de recommandations de l’évaluation sont l’évaluation. similaires à celles que l’on peut trouver  Organiser des discussions avec les dans les documents suivants : organisations bailleuses de fonds sur la manière dont leur financement oriente  Cluster Approach Evaluation 2 l’action des CLA. Synthesis Report, GPPi and Groupe  Obtenir la collaboration des URD, 2010. gouvernements nationaux avant et au  Synthesis of Cluster Lessons from début de la mise en place des clusters UNICEF’s response to the Pakistan afin de définir de concert et clairement Floods, UNICEF, 2011. l’ampleur de leurs responsabilités et les  Implementing the Cluster Approach: A points de transition. synthesis of documented lessons  2006–2010, Division des programmes, UNICEF, 2011.  ‘Independent Review of UNICEF’s Operational Response to the January 2010 Earthquake in Haiti’, Bureau de l’évaluation, UNICEF, 2011.  UNICEF’s Response to the Emergency in the Horn of Africa, 2011–2012: Lesson-learning exercise, Bureau de l’évaluation, UNICEF, 2012.

xxv

Table 1: Résumé des faits constatés, des conclusions et des recommandations

Résumé des faits constatés, des conclusions et des recommandations Constatations : ce Constations : ce Domaine clé que l’UNICEF fait que l’UNICEF fait Conclusions Recommandations efficacement moins efficacement • Bonne perception • Faiblesses dans L’UNICEF joue un Mettre au point une de l’efficacité globale. les résultats des rôle de coordination initiative de « prêt-à- partenariats au de plus en plus organiser » pour • Services de niveau mondial. efficace dans les améliorer la capacité coordination des clusters, mais doit des Bureaux de pays clusters de plus en • Manque de clarté traiter la question de se préparer à plus solides. sur les rôles et les du manque de l’activation de clusters responsabilités de clarté sur les rôles dans les pays à hauts • Très bons résultats coordination : à en termes de et les risques. l’interne entre les responsabilités partenariats organismes nationaux. avec ses composant les partenaires, les clusters, à l’externe attentes Résultats avec les divergentes des concernant la partenaires. diverses parties coordination • Capacités limitées prenantes et les externe des coordonnateurs carences des à identifier les capacités des carences et à coordonnateurs résoudre les pour assurer des problèmes. améliorations continues. • Problème de la représentation par l’UNICEF des membres des clusters au sein des Équipes de pays pour l'action humanitaire. • Rôle de CLA bien • Manque de clarté L’UNICEF a Renforcer les intégré au niveau de sur les rôles et les introduit des systèmes de gestion la politique globale. responsabilités de changements globaux de coordination à positifs dans sa l’organisation pour • Améliorations dans l’interne entre les manière de traiter faciliter le rôle de CLA, la coordination et la organismes ses rôles de CLA y compris par le direction dues à la composant les au niveau mondial, renforcement du rôle création de la cellule clusters. mais n’a pas encore des Bureaux régionaux GCCU. clarifié les rôles et et en liant mieux le rôle Résultats • Manque de clarté les responsabilités des Représentants de • Premiers efforts sur le rôle des concernant le pour mettre au point internes entre les pays à un dispositif rôle interne en Bureaux régionaux organismes stratégique de gestion une stratégie de CLA et aucune tant que CLA et une fonction de composant les au niveau mondial du démarche clusters, ni mis en travail de CLA. gestion de systématique pour l’information. place une chaîne s’assurer que les • de responsabilités • Adéquation globale Représentants de qui garantirait entre les avantages pays remplissent uniformité et qualité comparatifs de leur rôle au sein des d’un Bureau de l’UNICEF et ses rôles CLA de manière pays à l’autre de CLA. cohérente.

Résultats dans • Améliorations • Traitement La capacité de Mettre au point une le domaine importantes dans les fragmenté des l’UNICEF de mettre stratégie intégrée pour

xxvi

des capacités de montée ressources rapidement en les ressources ressources en puissance (surge) humaines et place des humaines destinées à humaines avec un potentiel absence d’un coordonnateurs être mobilisées pour croissant de déployer parcours s’est améliorée, opérer une montée en des coordonnateurs professionnel défini mais la puissance (surge) et dans les délais dans le domaine de fragmentation pour la formation du requis. la coordination. persistante, personnel de l’UNICEF l’absence de travaillant à la • « Doubles stratégie intégrée, coordination. casquettes » restant la fréquence des très fréquentes et « doubles affaiblissant casquettes » et la l’efficacité de façon limitée de l’action. traiter le • Développement développement des des capacités limité capacités des à des sessions de coordonnateurs sensibilisation réduit l’uniformité et élémentaires. la qualité des ressources humaines investies dans les clusters • Participation active • Absence dans La lente expansion Améliorer la cohérence aux travaux de l’IASC toute l’organisation de l’ampleur des des orientations et sur le Programme de travail responsabilités des (formulation et de changement d’orientation et de clusters interprétation) ainsi transformationnel et conseils concernant affaiblissant la que sa fidélité à ces efforts au niveau le travail de cluster. capacité de orientations mondial pour clarifier l’UNICEF de jouer (compréhension et la politique et les • Manque de efficacement son application uniforme) directives pour les cohérence dans rôle de CLA en en référence à la clusters. l’application de la termes de politique et aux politique de l’IASC. pertinence, de pratiques de l’UNICEF • Effets durables sur capacité dans le domaine des les capacités de prévisionnelle, CLA. coordination et de d’efficacité et de Questions réponse nationales viabilité. d’ampleur et dus au manque de limites de d’orientation sur la l’action transition et le renforcement des capacités, ainsi qu’à la décentralisation de l’autorité et du financement. • Clusters de plus en plus utilisés pour remédier aux carences ne concernant pas la coordination des interventions d’urgence. • Le personnel et des • Investissements Globalement, les Modérer le recours aux Rapport coût- partenaires jugent importants sur le investissements de clusters quand les efficacité, nettement que les terrain dans les l’UNICEF dans son scénarios ne sont pas optimisation réalisations justifient contextes de rôle de CLA ne sont adaptés en mettant au des les investissements coordination pas jugés excessifs, point des modèles et ressources et réalisés. sectorielle à long personnel et des outils de efficience terme. partenaires pensent coordination ne faisant

xxvii

que les réalisations pas appel à ces • Investissements • Coordination entre justifient les clusters, y compris de dans le rôle de CLA clusters insuffisante investissements points de transition jugés non excessifs. et méthodes de réalisés, cependant pour les Bureaux de programmation les investissements pays ; clarifier le rôle • Dispositifs de co- intégrée restant direction dans les clusters des clusters, dans l’exception plutôt destinés à remédier l’hypothèse où ils en généralement que la norme. améliorés depuis à des carences de ont un, dans le 2012. la coordination renforcement des sectorielle à long capacités nationales terme offrent des de façon à mettre en possibilités de place des méthodes réaliser des de coordination économies ou de efficaces et adaptées procéder à des aux objectifs réalignements déterminés.

xxviii

RESUMEN

Pregunta de evaluación Se prevé que la evaluación cumplirá dos propósitos: aprendizaje y rendición de La evaluación del papel de UNICEF como cuentas. Se orienta esencialmente al organismo coordinador de grupos futuro, con miras a mejorar el aprendizaje temáticos sobre acción humanitaria por medio de las experiencias pasadas, la buscaba responder la siguiente pregunta identificación de oportunidades y la fundamental: generación de datos que permitan a UNICEF perfeccionar sus capacidades, ¿Cuán eficientemente desempeña sistemas y métodos en beneficio de su UNICEF su función como organismo labor como coordinador de grupos coordinador de grupos temáticos y qué temáticos y esferas de responsabilidad. debería hacer en el futuro?

La Oficina de Evaluación de UNICEF Justificación de la evaluación encargó la evaluación, que fue dirigida El Comité Permanente entre Organismos por evaluadores independientes de Avenir adoptó el enfoque de gestión por grupos Analytics entre enero y junio de 2013. temáticos en 2005, con el objeto de Sobre la base de los principios de mejorar la coordinación durante las referencia (ver el Anexo 1), se pretendía situaciones de emergencia, en el contexto que la evaluación generara información del proceso más amplio de reforma de la fidedigna sobre los resultados que asistencia humanitaria. Se pretendía que UNICEF había obtenido o dejado de este enfoque favoreciera la respuesta obtener en el desempeño de su función humanitaria reforzando la previsibilidad, la como organismo coordinador de grupos rendición de cuentas y las asociaciones temáticos. entre diferentes actores.

Objetivo Los cinco grupos temáticos y ámbitos de responsabilidad que dirige o codirige El objetivo de la evaluación era examinar UNICEF representan aproximadamente de la manera más sistemática y objetiva un tercio del sistema de grupos temáticos posible la actuación de UNICEF como del Comité Permanente entre organismo coordinador de grupos Organismos. Para hacer frente a esta temáticos y de ámbitos de responsabilidad, UNICEF ha invertido responsabilidad en los planos mundial y tiempo y recursos considerables. En su nacional. calidad de institución, UNICEF debe saber cómo está cumpliendo su función Se aspiraba a que la evaluación generara de organismo coordinador de grupos datos concluyentes acerca de los temáticos y cómo podría mejorar. Durante resultados que UNICEF había obtenido o la fase inicial de la evaluación se dejado de obtener en el ejercicio de esas identificaron seis aplicaciones de las funciones. La evaluación está destinada observaciones obtenidas, mediante básicamente a ayudar al personal consultas con interesados internos y directivo superior de UNICEF a mejorar externos: su labor de coordinación de grupos temáticos y ámbitos de responsabilidad, 1. Servir de referencia para medir el en el marco del sistema de grupos desempeño en materia de coordinación temáticos del Comité Permanente entre de grupos temáticos. Organismos.

xxix

2. Contribuir a los actuales procesos de Evaluación seleccionó ocho países que planificación de UNICEF (planes de pudieran reflejar en los estudios de casos gestión de oficinas, Plan Estratégico de los diversos niveles de capacidad y Mediano Plazo y planificación del contextos operativos en los cuales trabajo de grupos temáticos). funcionan los grupos temáticos de 3. Dar a conocer a UNICEF las opiniones UNICEF: Burkina Faso, El Salvador, positivas y negativas de sus asociados Filipinas, Haití, Nepal, Sudán del Sur, con respecto a su actuación como Tayikistán y el Territorio Palestino coordinador de grupos temáticos, con Ocupado. vistas a mejorar sus métodos. 4. Contribuir a la solución de problemas Se diseñaron instrumentos de relacionados con el sistema de grupos recopilación de datos, a fin de ayudar a temáticos que enfrenta el Comité los evaluadores a reunir información de Permanente entre Organismos. múltiples fuentes para cada uno de los 5. Intercambiar con los asociados 72 indicadores, así como también para información sobre las buenas prácticas facilitar la triangulación. y la capacidad requeridas. Esos instrumentos se utilizaron durante 6. Suministrar información objetiva sobre las entrevistas personales y a distancia, el desempeño de UNICEF a los los grupos de discusión, las encuestas en interesados internos y externos más línea, la observación y la investigación importantes, incluyendo a los documental. Para facilitar la participación, donantes. las guías para las entrevistas y las encuestas se tradujeron al francés y al Enfoque y metodología de la español. evaluación El producto del diseño de la evaluación Diseño fue un informe inicial que se presentó a la Con base en las instrucciones de la Oficina de Evaluación y a miembros de Oficina de Evaluación de UNICEF, se grupos de discusión internos y externos examinaron y perfeccionaron más de 100 con el ánimo de conocer su opinión. Este líneas de investigación durante la fase informe se aprobó antes de comenzar la inicial de la evaluación. Las preguntas se recopilación de los datos. formularon de manera que permitieran conocer la capacidad de UNICEF para Recopilación de los datos cumplir los criterios de desempeño La reunión de los datos se llevó a cabo basados en los parámetros del Comité de por medio de visitas a seis países y Asistencia para el Desarrollo de la visitas personales a Copenhague, Organización de Cooperación y Ginebra y Nueva York. Así mismo, Desarrollo Económico, a saber, mediante entrevistas a distancia e pertinencia e idoneidad, efectividad investigación documental efectuada (dividida en previsibilidad, rendición de durante tres meses por un grupo de ocho cuentas y asociación), eficiencia, evaluadores. Estos últimos reunieron coherencia, cobertura y sostenibilidad. datos procedentes de más de 1.000 personas –incluyendo a 282 funcionarios Se elaboraron 72 indicadores para de UNICEF y a 779 funcionarios de evaluar la eficiencia con la que UNICEF organizaciones asociadas– y 790 desarrolla su labor como organismo documentos. Las notas originales en las coordinador de grupos temáticos, y para que se registraron los datos se identificar patrones que indiquen la resumieron luego por entrevistas, grupos necesidad de introducir mejoras en de discusión u observaciones con determinadas esferas. La Oficina de xxx

respecto a cada uno de los indicadores de asociados en seminarios participativos la evaluación. El producto de esta etapa (presenciales y en línea). fue un depósito de datos por cada estudio de caso y a nivel mundial. Observaciones, análisis y recomendaciones Análisis de los datos La información de los más de 1.000 Las observaciones se analizaron en puntos de referencia se sintetizó con base función de los posibles requisitos futuros en grupos y ubicación. Los resultados de de los principales interesados. cada caso se resumieron en función de los indicadores y se consolidaron en A renglón seguido se formularon 33 factores de resultados y capacidad. recomendaciones sobre aspectos que Posteriormente se aplicó una medida UNICEF podría mejorar para fortalecer la cuantitativa para obtener una indicación coordinación de los grupos temáticos. general de la eficiencia con la que UNICEF cumple los criterios cruciales en Los productos son: (i) una serie de términos de resultados y evaluación. observaciones sobre los aspectos positivos y menos positivos de la Las observaciones globales y por casos actuación de UNICEF como coordinador se analizaron y sintetizaron para extraer de grupos temáticos y (ii) una serie de conclusiones preliminares de carácter recomendaciones asociadas directamente general. A continuación, esas con los aspectos menos positivos que, de observaciones se organizaron sobre la aplicarse, facilitarán la introducción de las base de nuevas tendencias y cuestiones mejoras necesarias. correspondientes a las cinco esferas clave relativas a la eficiencia con la que Observaciones UNICEF ejecuta su función como Observaciones relativas a la esfera organismo coordinador de grupos temáticos y a lo que debería hacer en el principal 1: Desempeño en materia futuro: de coordinación exterior Este tema se examinó sobre la base de  Desempeño en materia de tres aspectos esenciales: coordinación exterior  Funcionamiento interno del 1. Eficiencia general organismo coordinador de grupos 2. Cobertura temáticos 3. Asociación  Desempeño en materia de recursos humanos Las observaciones más importantes  Cuestiones referentes al alcance y fueron: los límites  Rentabilidad, eficiencia y relación  Los asociados consideran que entre calidad y precio UNICEF cumple con gran eficiencia sus responsabilidades como Antes de concluir la elaboración del coordinador de grupos temáticos a informe, las conclusiones, análisis y nivel de países, de acuerdo con recomendaciones se presentaron a las encuestas, entrevistas y grupos de discusión. Evaluaciones recientes oficinas de las regiones y los países 16 donde se habían realizado estudios de respaldan este concepto . Los casos, al igual que a funcionarios de UNICEF alrededor del mundo y a 16 UNICEF, UNICEF’s Response to the Emergency in the Horn of Africa, 2011–2012: Lesson-learning

xxxi

interesados señalaron que el documentos y la observación desempeño de UNICEF suele ser evidencian que los coordinadores del mejor de lo que se puede esperar en grupo temático mundial hacen los complejos contextos nacionales y grandes esfuerzos para compartir las de coordinación. responsabilidades. Asociados en los  Los recursos humanos, el apoyo países expresaron inquietud por la institucional y el liderazgo son los representación por parte de UNICEF factores que más influyen en la de los grupos temáticos en los efectividad de UNICEF como equipos humanitarios en los países, organismo coordinador de grupos aisladamente de sus intereses temáticos, según datos de institucionales. entrevistas, grupos de discusión, encuestas y exámenes de Observaciones relativas a la esfera documentos. principal 2: Funcionamiento interno  Los grupos temáticos de UNICEF tienen efectos positivos en la del organismo coordinador de cobertura geográfica y temática, de grupos temáticos acuerdo con encuestas, entrevistas y Este tema se examinó sobre la base de grupos de discusión. Se han creado cuatro aspectos esenciales: estructuras subnacionales de coordinación, a pesar de que existe 1. La creación de la Dependencia de preocupación por su sostenibilidad Coordinación de los Grupos financiera y de dotación de personal. Temáticos Mundiales. Los grupos temáticos y las esferas de 2. La supervisión de las cuestiones responsabilidad han repercutido normativas y de funcionamiento. favorablemente en la cobertura 3. El papel de las oficinas regionales. temática, gracias al incremento de las 4. Las ventajas comparativas de investigaciones operacionales y a la UNICEF. promoción, particularmente en el grupo sectorial sobre educación y en Las observaciones más importantes la esfera de responsabilidad sobre la fueron: violencia de género.  Los aspectos de asociación de los  En la evaluación se encontraron grupos temáticos y los ámbitos de claros indicios de que la responsabilidad dirigidos por consolidación del personal del UNICEF son más sólidos a nivel de grupo mundial bajo la Dependencia los países que a nivel de los grupos de Coordinación de los Grupos mundiales. Mientras que los aliados Temáticos Mundiales está en los países coinciden en que empezando a arrojar resultados UNICEF observa los principios de positivos en todas las operaciones de asociación, los miembros de los UNICEF relativas a los grupos grupos/ámbitos de responsabilidad temáticos. Más del 70% de los mundiales se han mostrado funcionarios de UNICEF encuestados preocupados especialmente por el y la mayoría de los funcionarios aspecto de la transparencia. Las entrevistados en países donde se funciones y responsabilidades no son realizaron estudios de casos afirmaron claras para quienes responden que la coordinación y la dirección encuestas mundiales, aunque los habían mejorado moderada o significativamente en todo el mundo exercise, Informe final, Oficina de Evaluación, junio como resultado de la creación de la de 2012. Dependencia de Coordinación de los xxxii

Grupos Temáticos Mundiales, a pesar  Se comprobó que las funciones de de que este cambio institucional se UNICEF como organismo introdujo hace apenas un año. coordinador de grupos temáticos  Las funciones de coordinación de corresponden ampliamente a sus grupos temáticos se han ventajas comparativas respecto de incorporado cada vez más en las la programación. No obstante, políticas y los sistemas de UNICEF; miembros de grupos temáticos sin embargo, los sistemas de mundiales ponen en duda las supervisión de estas funciones fortalezas de los programas de siguen siendo limitados. Con arreglo UNICEF y su capacidad para a las directrices del Sistema de aprovechar sus habilidades de Supervisión de la Actuación coordinación. Humanitaria17, la presentación de informes de las oficinas en los países Observaciones relativas a la esfera es voluntaria y se limita a un conjunto principal 3: Desempeño en materia de preguntas que se responden “sí” o “no” y tratan sobre los aportes y los de recursos humanos productos, y no sobre la calidad y los Este tema se examinó sobre la base de resultados. dos aspectos esenciales:  El papel de las oficinas regionales con respecto a las 1. Sistemas de apoyo adicional y responsabilidades del organismo contratación coordinador de grupos temáticos 2. Desarrollo de la capacidad de sigue siendo poco claro. Si bien los coordinación de los grupos temáticos Compromisos Básicos para la Infancia en la Acción Humanitaria prevén un Las observaciones más importantes mandato de coordinación, no se han fueron: traducido en responsabilidades precisas ni en mecanismos de  Los sistemas de apoyo adicional y rendición de cuentas a nivel regional. contratación para los grupos Sobre la base de evaluaciones temáticos han mejorado desde anteriores18 y documentos internos19, 2010, como evidenciaron las lo anterior podría obedecer a una diferencias entre las observaciones de confusión de índole sistémica en las evaluaciones del terremoto de 20 UNICEF en torno a las funciones, las Haití y las respuestas en el Cuerno 21 responsabilidades y la autoridad entre de África . Sin embargo, la gestión de la sede, las oficinas regionales y las los sistemas de apoyo adicional sigue oficinas nacionales. Lo anterior es siendo fragmentada y no existe una importante porque, en la cadena de estrategia general. rendición de cuentas de UNICEF, las  El régimen de dobles cargos sigue oficinas regionales constituyen la siendo común y disminuye la única vinculación oficial entre las eficiencia. Incluso en situaciones de oficinas nacionales y la sede. emergencia o de grupos temáticos activos, apenas un 28% del personal trabaja en régimen de jornada

17 UNICEF, ‘Humanitarian Performance Monitoring – Toolkit and Guide’, junio de 2011. 20 UNICEF, ‘Independent Review of UNICEF’s 18,UNICEF’s Response to the Emergency in the Operational Response to the January 2010 Horn of Africa, 2011–2012. Earthquake in Haiti’, Oficina de Evaluación, 19 Documento interno de trabajo de UNICEF, septiembre de 2011. ‘Cluster Issues Matrix: Internal’, División de 21 UNICEF’s Response to the Emergency in the Programas, UNICEF, diciembre de 2010. Horn of Africa, 2011–2012.

xxxiii

completa en actividades relacionadas Permanente entre Organismos (con con los grupos temáticos. plazos concretos y centrados en las  El desarrollo de la capacidad de los situaciones de emergencia). De los 20 coordinadores de grupos temáticos restantes, 10 pueden haberse se limita a cursos de sensibilización activado correctamente, pero hoy en en el aula, y no se han tomado día operan como mecanismos de medidas sistemáticas para asegurar la coordinación del desarrollo sectorial a coherencia en la práctica de los largo plazo. Los demás se activaron representantes en las oficinas de país. en lugar de la coordinación sectorial del desarrollo, sin la activación oficial Observaciones relativas a la esfera del Comité Permanente entre Organismos. principal 4: Cuestiones referentes al  Dar a los grupos temáticos un alcance y los límites alcance inapropiado afecta Este tema se examinó sobre la base de seriamente la capacidad de UNICEF dos aspectos esenciales: para ejercer de forma adecuada su función como organismo 1. El alcance de las responsabilidades coordinador de grupos temáticos de UNICEF con los grupos temáticos en cuanto a pertinencia, previsibilidad, 2. La función operacional de los grupos asociación, eficiencia y sostenibilidad. temáticos y las esferas de responsabilidad de UNICEF Observaciones relativas a la esfera

Las observaciones más importantes principal 5: Rentabilidad, eficiencia fueron: y relación entre calidad y precio Estos temas se examinaron sobre la base  Las repercusiones a largo plazo de de tres aspectos esenciales: los grupos temáticos sobre los sistemas de coordinación y la 1. Rentabilidad y relación entre la calidad capacidad de respuesta nacional y el precio son limitadas. Esto se debe a la falta 2. Relaciones con otras iniciativas de directrices sobre transición, internas y externas descentralización de la autoridad y la 3. Acuerdos de codirección financiación, así como también a la falta de precisión sobre el papel de los Las observaciones más importantes grupos temáticos desde el punto de fueron: vista del fomento de la capacidad.  Con creciente frecuencia, los grupos  Tanto los funcionarios como los temáticos subsanan deficiencias que asociados consideran que los van más allá de la coordinación en resultados del trabajo de los contextos de emergencia. De los 34 grupos justifican ampliamente las grupos temáticos de UNICEF activos inversiones de sus organizaciones. en 201222, solamente 14 actúan en En las encuestas, el 83% de los completa consonancia con los funcionarios de UNICEF en los países objetivos y las directrices de los y el 84% de los asociados en los grupos temáticos del Comité países afirmaron que los resultados de las actividades de los grupos temáticos justifican sus inversiones. 22 Documento interno y base de datos de UNICEF, Esto debe analizarse críticamente ‘A Survey of the Operational Status and Staffing of sobre la base de las observaciones UNICEF-led and Co-led Clusters and AORs in the Field’, Informe final y anexos, julio de 2012. relativas a la esfera principal 4. xxxiv

 Se determinó que la coordinación eficiente a las comunidades vulnerables, entre los distintos grupos en el marco del sistema humanitario. temáticos es insuficiente y que los enfoques integrados hacia la Esfera principal 2: Funcionamiento programación siguen siendo la interno del organismo coordinador excepción en vez de la norma.  En términos generales, los de grupos temáticos. acuerdos de codirección han Deben aprovecharse y mantenerse los mejorado desde 2012, aunque esfuerzos de UNICEF tendientes a persisten algunos problemas consolidar el apoyo a los grupos derivados de las diferencias en las temáticos, de conformidad con la expectativas y la capacidad de los Dependencia de Coordinación de los miembros. Grupos Temáticos Mundiales, y a integrar en sus políticas la función de coordinación de grupos temáticos. Los aspectos Conclusiones y fundamentales que deben mejorar son: recomendaciones (i) la falta de claridad sobre las funciones Esfera principal 1: Desempeño en y las responsabilidades en materia de coordinación interna, (ii) la falta de materia de coordinación exterior claridad sobre el papel de las oficinas Se deben mantener y replicar las regionales, y (iii) la ausencia de un fortalezas de UNICEF en lo que se refiere enfoque coherente que garantice el a la coordinación de los grupos temáticos, cumplimiento sistemático de las funciones la cobertura y las asociaciones en los de coordinación de grupos temáticos por países. Los aspectos fundamentales que parte de los representantes en los países. deben mejorar son: (i) las limitaciones en La recomendación primordial es fortalecer la capacidad de los coordinadores de los los sistemas de gestión en apoyo de la grupos para detectar deficiencias y labor de los organismos coordinadores de resolver problemas, (ii) las dificultades en grupos temáticos, incluyendo la medición la representación por parte de UNICEF de de los resultados, el fortalecimiento de la miembros de grupos temáticos en los función de las oficinas regionales, y el equipos humanitarios en los países, y mejoramiento del nexo entre los (iii) la falta de claridad sobre las funciones representantes en los países y la gestión de coordinación y las responsabilidades estratégica mundial de los organismos externas con los asociados. coordinadores de los grupos.

La recomendación primordial es diseñar Esfera principal 3: Desempeño en una iniciativa de “grupos temáticos en paralelo”, que permita a las oficinas materia de recursos humanos regionales y/o a la Dependencia de Las inversiones de UNICEF en acuerdos Coordinación de los Grupos Temáticos sobre aumento de la capacidad han sido Mundiales utilizar instrumentos de positivas y deben incrementarse. Los evaluación y mejoramiento de la aspectos fundamentales que deben capacidad elaborados en todo el mundo. mejorar son: (i) la fragmentación del En breve, esto implica reforzar la enfoque hacia los recursos humanos y la preparación de las oficinas de país en falta de perspectivas profesionales del zonas de alto riesgo, de modo que personal encargado de la coordinación y puedan activar o separarse del (ii) el hecho de que el desarrollo de la mecanismo de grupos temáticos cuando capacidad del personal para trabajar con sea necesario. Además, permite que los grupos temáticos se limite, en toda la grupos proporcionen asistencia eficaz y

xxxv organización, a impartir en el aula La recomendación primordial es reducir la capacitación básica sobre sensibilización. utilización de grupos temáticos en La recomendación primordial es formular contextos inapropiados, elaborando una estrategia integrada de aumento de la modelos e instrumentos para las capacidad y diseñar un programa actividades de coordinación no orientado a reforzar la preparación del relacionadas con estos grupos –por personal de coordinación en todo ejemplo, puntos de transición para la UNICEF. oficinas en los países–, y esclarecer la importancia de los grupos temáticos para Esfera principal 4: Cuestiones el fomento de la capacidad nacional, a fin de lograr métodos de coordinación referentes al alcance y los límites eficientes y adecuados a los propósitos En respuesta a las solicitudes que recibe que se persiguen. para prestar servicios de coordinación, UNICEF se desempeña como organismo Factores clave de éxito coordinador de grupos temáticos en un Algunas de las conclusiones y las número cada vez mayor de contextos. El recomendaciones de la evaluación son aspecto fundamental que debe mejorar es similares a los que se encuentran en los la falta de directrices sobre los grupos siguientes documentos: temáticos en toda la organización y la falta de aplicación sistemática de las políticas del Comité Permanente entre  Cluster Approach Evaluation 2 Organismos, lo que dificulta el Synthesis Report, GPPi y Groupe cumplimiento de las expectativas URD, 2010. divergentes de los interesados.  Synthesis of Cluster Lessons from UNICEF’s Response to the Pakistan La recomendación primordial es hacer Floods, UNICEF, 2011. que las políticas y las directrices de  Implementing the Cluster Approach: A UNICEF interpreten y definan con synthesis of documented lessons precisión las políticas del organismo 2006–2010, División de Programas, coordinador de grupos temáticos UNICEF, 2011. (coherencia), y darlas a conocer para  ‘Independent Review of UNICEF’s garantizar su aplicación sistemática Operational Response to the January (fidelidad), prestando especial atención a 2010 Earthquake in Haiti’, Oficina de las situaciones de emergencia más Evaluación, UNICEF, 2011. críticas y a la gestión de las expectativas  UNICEF’s Response to the de las partes interesadas. Emergency in the Horn of Africa, 2011–2012: Lesson-learning exercise, Esfera principal 5: Rentabilidad, Oficina de Evaluación, UNICEF, 2012. eficiencia y relación entre calidad y Tomar las seis medidas que se exponen a precio continuación contribuiría a aplicar UNICEF debe mantener y aprovechar las exitosamente las recomendaciones de la fortalezas que le atribuyen los asociados, evaluación: al igual que los acuerdos de codirección. El aspecto fundamental que debe mejorar  Lograr la participación del grupo es la excesiva inversión en el uso de directivo superior de todo UNICEF. grupos temáticos sobre el terreno para  Poner en práctica la totalidad de las subsanar deficiencias en la coordinación recomendaciones. humanitaria o sectorial a largo plazo.

xxxvi

 Promover la participación de los organismos coordinadores de grupos asociados en el desarrollo de temáticos. estrategias para el futuro.  Obtener la colaboración de los  Lograr la participación de los órganos gobiernos nacionales antes y al inicio del Comité Permanente entre de las operaciones de los grupos Organismos en debates basados en la temáticos, con el objeto de definir los evaluación. puntos de transición y convenir su  Celebrar reuniones con las alcance de forma inequívoca. organizaciones donantes para examinar la manera como sus aportes determinan la implementación de los

xxxvii

Tabla 1: Resumen de las observaciones, las conclusiones y las recomendaciones

Resumen de las observaciones, las conclusiones y las recomendaciones Observaciones - lo Observaciones - lo Esfera que UNICEF esta que UNICEF debe Conclusiones Recomendaciones principal haciendo bien mejorar • Existe una alta ▪ Las deficiencias en La labor de Diseñar una iniciativa percepción de el rendimiento de las UNICEF como de "grupos temáticos en efectividad general. asociaciones coordinador de paralelo" que mejore la mundiales. grupos temáticos preparación de las • Los servicios de es cada vez más oficinas de país para coordinación de los ▪ La falta de claridad efectiva, pero activar los grupos grupos temáticos sobre los roles y debe abordar la temáticos en los países muestran una responsabilidades falta de claridad de alto riesgo. creciente solidez. en materia de sobre las coordinación: a nivel • Los resultados en funciones y interno, entre las responsabilidade materia de dependencias y a asociaciones en los s con los nivel externo, con asociados, las Desempeño países son positivos. los asociados. en materia de diferentes coordinación ▪ La deficiente expectativas de exterior capacidad de los los asociados, y coordinadores de los las deficiencias grupos temáticos en la capacidad para detectar fallas y de los resolver problemas. coordinadores para mejorar ▪ Las dificultades en constantemente. la representación por parte de UNICEF de los grupos temáticos en los equipos humanitarios en los países. • La función del • La falta de claridad UNICEF ha Fortalecer los sistemas organismo sobre los roles y las introducido de gestión en todo coordinador de responsabilidades cambios positivos UNICEF para apoyar la grupos temáticos de las dependencias en su enfoque función de coordinación está bien integrada a a nivel interno. mundial hacia la de los grupos nivel normativo. gestión del papel temáticos, incluyendo el • La falta de claridad de los fortalecimiento del • Se han registrado sobre el papel de las organismos papel de las oficinas mejoras en la oficinas regionales y coordinadores de regionales y el coordinación y el la ausencia de un Funcionamien los grupos mejoramiento de la liderazgo gracias a la mecanismo to interno del temáticos, pero relación de los creación de la coherente para organismo aún no ha representantes en los Dependencia de garantizar el coordinador definido países con una Coordinación de los cumplimiento de grupos claramente las estructura mundial de Grupos Temáticos sistemático de las temáticos funciones y gestión estratégica de Mundiales. funciones de responsabilidade la coordinación de coordinación de los • Ha habido s de las grupos temáticos. grupos temáticos dependencias esfuerzos iniciales por parte de los para preparar la internas ni representantes en establecido una estrategia de los países. coordinación de los cadena de grupos temáticos y responsabilidad establecer la función que favorezca la de gestión de la coherencia y la xxxviii

información. calidad en las oficinas en los • Existe amplia países. concordancia entre las ventajas comparativas de UNICEF y las funciones de coordinador de grupos temáticos. • La capacidad de • La fragmentación La capacidad de Diseñar una estrategia refuerzo de los del enfoque hacia despliegue integrada referente al grupos temáticos ha los recursos rápido de apoyo adicional en el mejorado humanos y la falta coordinadores ámbito de los recursos notablemente, al de perspectivas por parte de humanos y al igual que la profesionales del UNICEF ha perfeccionamiento del capacidad para personal encargado mejorado, pero la personal de desplegar de la coordinación. continua coordinación de coordinadores de fragmentación, la UNICEF. manera oportuna. falta de una • La persistencia del estrategia régimen de dobles integrada sobre Desempeño cargos, lo que recursos en materia de reduce la humanos, la recursos efectividad. persistencia del humanos • El hecho de que el régimen de desarrollo de dobles cargos y capacidad se limite el limitado a impartir cursos desarrollo de la básicos de capacidad de los sensibilización. coordinadores reducen la coherencia y la calidad de los recursos humanos de sus grupos temáticos. • La participación • La falta de Dar a los grupos Mejorar la coherencia activa en el Comité directrices sobre los temáticos un (interpretación y Permanente entre grupos temáticos en alcance articulación) y la Organismos, en el toda la organización inapropiado fidelidad (comprensión trabajo de la Agenda y la aplicación afecta y aplicación Transformativa y en inconsistente de las seriamente la sistemática) por medio los esfuerzos políticas del Comité capacidad de de las políticas y las mundiales para Permanente entre UNICEF para prácticas de UNICEF en esclarecer las Organismos. ejercer de forma materia de coordinación políticas y las adecuada su de grupos temáticos. directrices de los • La falta de función como Cuestiones directrices sobre referentes al grupos temáticos. coordinador de transición, creación grupos temáticos alcance y los de capacidad límites en cuanto a nacional y pertinencia, descentralización de previsibilidad, la autoridad y la asociación, financiación eficiencia y repercute sostenibilidad. adversamente y a largo plazo sobre la coordinación y la capacidad de respuesta de los países.

xxxix

• El hecho de que los grupos temáticos subsanan, cada vez con mayor frecuencia, deficiencias que van más allá de la coordinación en contextos de emergencia. • Los funcionarios y • Demasiada La inversión Reducir el uso de los asociados inversión sobre el general de grupos temáticos en consideran que los terreno en contextos UNICEF en el contextos inapropiados, resultados justifican de coordinación ejercicio de su desarrollando modelos ampliamente las sectorial a largo función como e instrumentos para las inversiones. plazo. coordinador de actividades de grupos temáticos coordinación no • Las inversiones en no es excesiva, y relacionadas con estos la función del • La coordinación tanto los grupos (incluyendo organismo entre los distintos funcionarios puntos de transición coordinador de los grupos temáticos es como los para las oficinas en los grupos temáticos no insuficiente y los asociados países) y, cuando es excesiva. enfoques integrados coinciden en que proceda, esclarecer el Rentabilidad, hacia la los resultados de papel de los grupos eficiencia y • Por lo general, los programación los grupos temáticos en el fomento relación entre acuerdos de siguen siendo la justifican sus de la capacidad calidad y codirección han excepción en vez de inversiones. Sin nacional, logrando así precio mejorado desde la norma. 2012. embargo, las que los métodos de inversiones en coordinación se ajusten grupos temáticos a los propósitos que se para subsanar persiguen. deficiencias en la coordinación sectorial a largo plazo podrían llevar a un replanteamiento o a una disminución de los costos.

1 EVALUATION QUESTION

The evaluation of UNICEF’s CLA role in 1.1. Rationale for the evaluation humanitarian action (CLARE) has The IASC initiated the cluster approach in primarily sought to address the following 2005 to enhance coordination during overarching question: emergencies in the context of broader

humanitarian reform. The approach was How well does UNICEF carry out its CLA intended to improve humanitarian role and what should it do for the future? response by increasing predictability,

accountability and partnerships across The evaluation was commissioned by the humanitarian actors. UNICEF Evaluation Office and was conducted from January through June of Together, the five clusters and AORs – 2013 by independent evaluators from led or co-led by UNICEF – represent Avenir Analytics. Based on the terms of responsibility for approximately one third reference (see Annex 1), the evaluation of the IASC cluster system. To undertake sought to generate credible evidence these responsibilities, UNICEF has regarding the results achieved or not invested significant time and resources. achieved by UNICEF in undertaking its As an institution, UNICEF has a manifest CLA role. need to understand how it is performing

as a CLA and how it could improve. The objective of the evaluation was to assess, as systematically and objectively During the inception phase of the as possible, the performance of UNICEF evaluation, six uses of the evaluation as a CLA and AOR at global and country output were identified through consultation levels. It is primarily intended for use by with internal and external stakeholders. UNICEF senior management decision- These are: makers to make concrete improvements in how the organization undertakes its 1. Provide a benchmark for measuring CLA and AOR roles within the IASC CLA performance. cluster system. 2. Contribute to ongoing UNICEF

planning processes (office The evaluation is intended to serve the management plans, the medium term dual purposes of learning and strategic plan and cluster work accountability and, therefore, has planning). elements of both identifying lessons for 3. Provide UNICEF with unfiltered improvement and assessing performance. critiques and information on how It seeks to measure performance by partners view its implementation of the assessing the results achieved, or not CLA role in order to improve its achieved, to date by UNICEF in its lead approaches. roles. However, the primary emphasis of 4. Contribute to influencing the IASC on the evaluation is forward looking. It seeks unresolved cluster system issues. to enhance learning by drawing lessons 5. Share information with partners learned and identifying opportunities and regarding good practice and capacity options for UNICEF to improve its required. capacities, systems and approaches for 6. Provide evidence on UNICEF exercising its CLA and AOR roles. performance to key internal and

external stakeholders, including donors.

1.2. Scope of the evaluation the goal of improved predictability, the IASC designates CLAs for each of the The scope of the evaluation covered all of clusters at a global level. UNICEF is the UNICEF’s efforts to exercise its CLA role, CLA for the Water, Sanitation and focal point role for the two AORs, and Hygiene (WASH) Cluster and the Nutrition related cluster or AOR coordination roles. Cluster. UNICEF also serves as co-lead The evaluation covers the period from agency for the Education Cluster with 2010 to 2013, following the IASC Phase 2 23 Save the Children. Within the Protection Cluster Approach Evaluation in 2010. Cluster (led by UNHCR), UNICEF serves

as the focal point agency for two AORs, The focus is on UNICEF’s implementation Child Protection and Gender-based of its CLA role, rather than the impact of Violence (GBV). each cluster in changing the conditions of the affected population. Emphasis is Each cluster or AOR is led at a global placed on UNICEF’s effectiveness in level by a Global Cluster Coordinator serving as a CLA and facilitating (GCC) and supported by cluster teams, humanitarian sectoral coordination information managers and agency staff. through its clusters and AORs, and the On a day-to-day basis, the GCCs manage ways it could improve CLA functions and efforts to strengthen system-wide coordination performance in the future. preparedness, to build capacity and to

provide support to clusters at country The analysis explored general UNICEF- level. level issues common to all of the clusters and AORs it leads or co-leads. Global At the country level, clusters are activated data collection and eight country case by the IASC on the recommendation of studies allowed the evaluation team to the humanitarian Coordinator (HC), in collect comparable bottom-up data to consultation with the humanitarian country synthesize into a global picture for each team (HCT). When established, a cluster cluster and across clusters. coordinator (CC) is selected to lead and

1.3. Background and context coordinate the country-level cluster in providing operational coordination, and The Inter-Agency Standing Committee sometimes, emergency planning and (IASC) initiated the cluster approach in preparedness. Country-level CCs often, 2005 to enhance coordination during but not always, come from the emergencies in the context of broader organization with global CLA responsibility humanitarian reform24. The approach was for the clusters, and may serve in their intended to improve humanitarian positions on a full-time basis, or part-time response by increasing predictability, while fulfilling other organizational accountability and partnerships across the responsibilities (‘double-hatting’). United Nations (UN), Red Cross/Red Cluster/AOR staff members are also Crescent and non-governmental drawn from standby partnerships, organizations (NGOs). including as members of rapid response teams (RRTs). Today there are nine sectoral clusters and two common service clusters. To further Within UNICEF management of the CLA roles has shifted over time. From 2005 to early 2012 the GCCs were managed by 23 Groupe U.R.D. and GPPi, Cluster Approach UNICEF Programme Division sector Evaluation 2 Synthesis report, commissioned by managers, under two separate Deputy IASC, April 2010. Directors. In early 2012 UNICEF decided 24 United Nations, Humanitarian Response Review, August 2005. to consolidate the CLA functions and

created the Global Cluster Coordination other elements of humanitarian reform. Unit (GCCU) with new reporting lines to a These efforts have established cross- Deputy Director in the Office of cluster tools and systems while clarifying Emergency Operations (EMOPS) and co- expectations of CLAs and cluster located all GCCs and their staff in participants. UNICEF’s Geneva office. In 201026 a review of the co-leadership The cluster system is inherently dynamic arrangement for the Education Cluster at and continues to evolve. The timing of this the global level was conducted. evaluation is opportune given the current implementation and field-testing of The review states that the Education elements of the IASC Transformative Cluster was achieving its objective by Agenda. Furthermore, UNICEF and other adding value through appropriate CLAs have been rapidly adapting to this information, tools and training. It noted changing environment through internal unresolved challenges in the structure, structural changes, mainstreaming of staffing and resourcing of the Education budgets and enhanced internal support. Cluster. The review found that roles were not mainstreamed into the co-lead Previous evaluations agencies, exacerbated by "a lack of clarity Previous reviews, lessons-learned over the nature of the arrangement as a exercises and evaluations relevant to the full and equitable partnership between the CLARE fall into three categories: two agencies." i) overarching IASC cluster system and 27 CLAs; ii) responses to major A 2011 review of Save the Children's emergencies; and iii) a donor review of experience of co-leading the Education multilateral aid. Cluster with UNICEF revealed significant benefits for all actors stemming from this Overarching reviews of the IASC cooperation. These benefits include: the cluster system and individual clusters effective division of co-leadership Since its introduction, the cluster responsibilities between two capable approach has continuously evolved based organizations; greater capacity and on informal and formal learning. In 2005, accountability to speak out on behalf of the IASC established an evaluation children and young people who are process to review implementation of the denied access to education; and system. The first phase of the evaluation demonstrating that humanitarianism works was completed in 2007 and the second best when based on a partnership phase in 2010. Together, these between UN and non-UN actors. evaluations have examined the processes and performance of the cluster system as In 2011 and 2012, the Global Logistics Cluster undertook the first comprehensive a whole, providing valuable insights that 28 served as a basis for continued evaluation of a single cluster, spanning adjustments.

26 25 The Partnering Initiative, Review of the Global Building on the Cluster 2 Evaluation and Education Cluster Co-Leadership Arrangement, lessons from Haiti and Pakistan, in 2011 Save the Children and UNICEF, October 2010. the IASC initiated an organized effort, 27 Save the Children, Lessons in Leadership: Save known as the Transformative Agenda, to the Children's experience of co-leading the improve the cluster system and to further Education Cluster, 2011. 28 World Food Programme, Joint Evaluation of the Global Logistics Cluster, Vol. II – Annexes, commissioned by WFP, UNICEF and the 25 Cluster Approach Evaluation 2 Synthesis Report. Government of the Netherlands, August 2012.

3 global and country efforts from 2005 to A synthesis of cluster lessons learned 2012. UNICEF served in a from UNICEF’s response to the Pakistan co-management capacity for the floods in 201030 also identified a number evaluation, which has been widely of issues related to cluster coordination disseminated and embraced by the World including: Food Programme (WFP), the lead  Further clarity on cluster/CLA/partner agency, and external stakeholders. accountabilities is critical.  Weak communication with donors on Reviews of major emergency performance and progress, and poor operations management of expectations can The response to the 2010 Haiti result in the undermining of earthquake and the Pakistan floods confidence. served as a major test of the cluster  Underestimation of the staffing system in large, sudden onset numbers and staff skills set required to emergencies in complicated contexts. carry out effective cluster coordination, including interpersonal and technical The 2011 evaluation of UNICEF’s skills and seniority to ‘lead’ clusters. 29 response to the Haiti earthquake  Inadequate appreciation and support contained a number of key for information management causes recommendations and findings related to delays in the compilation of and its CLA role, including: sharing of information, maps, etc. to enable effective coordination.  UNICEF‘s surge staffing infrastructure  Incomplete separation of cluster had been disbanded leading to a flurry coordination functions from of activity to quickly deploy staff for too programme implementation. short periods of time with unclear  The co-locating of coordination teams objectives and reporting lines. was effective, creating ‘economies of  UNICEF is credited for actively scale’ but more emphasis on common engaging the Government of Haiti in cross-cluster services. select clusters, which proved vital for a coordinated response. The 2012 Horn of Africa lessons learned  Weak leadership capacity (except for provided insight into UNICEF’s corporate WASH) in the first three months due emergency response, with specific to: the lack of role clarity between relevance for cluster leadership.31 heads of programmes and cluster coordinators; inadequate resourcing This internal review noted the following (information management), and junior key findings relevant to this evaluation: coordinators; the lack of orientation of key Haiti Country Office staff on  Cluster/sector coordination UNICEF‘s cluster lead role and how arrangements vary between countries standby partnership on clusters in the Horn of Africa and do not follow worked; and the need to develop a the IASC standard cluster approach in cadre of highly trained information any country. managers who can be deployed  Subnational coordination, although not rapidly in any emergency to support always strong, has the potential for programmes or clusters.

30 UNICEF, Synthesis of Cluster Lessons from UNICEF’s Response to the Pakistan Floods, February 2011. 29 ‘Independent Review of UNICEF’s Operational 31 UNICEF’s Response to the Emergency in the Response to the January 2010 Earthquake in Haiti’. Horn of Africa, 2011–2012.

improving coordinated service to determine the value for money delivery. Its improvement requires demonstrated. Overall the review found adaptation at the subnational level to that the quality of leadership and intentionally merge groups of partnership behaviour within the cluster clusters.32 system was highly variable.  UNICEF country offices in the region aspire to more intersectoral working The UNICEF-specific aid review within UNICEF. There is potential for assessment report highlighted a number improving the effectiveness and value of issues that are relevant to this for money of UNICEF emergency evaluation including: interventions by integrating services at the point of delivery, backed up by  Effectiveness of UNICEF’s response integrated support from regional and in acute emergencies is a concern. headquarters levels, rather than the  The importance of human resources individual vertical sector support that and, particularly, the ability of UNICEF tends to prevail currently. This implies to deploy the right staff to a more cohesive approach to UNICEF humanitarian emergencies. and cluster support to local  UNICEF provides critical support and government and communities. coordination in humanitarian and emergency situations, and the scale of A 2012 independent real-time evaluation its operations cannot be delivered by of UNICEF's response to the 2011–2012 others. 33 Sahel Crisis found that UNICEF's role as  There are still concerns over Global Cluster Coordinator for Nutrition UNICEF’s capacity to effectively lead gave the organization a central role in humanitarian clusters on the ground. building resilience to food security and nutrition crises at the household The various reviews, evaluations and and government levels. The evaluation lessons-learned exercises suggest a found that cluster and inter-cluster mixed trend with variable CLA coordination systems were weak across performance across levels and the response, with the notable exception emergency situations. While these of the Nutrition Cluster in Niger. The exercises provide valuable insights and evaluation recommended that UNICEF inform particular lines of inquiry, none of improve cluster and inter-cluster them constitute an overall systematic coordination arrangements to assessment of UNICEF’s CLA role. enhance service delivery, access and coverage.

Donor review of multilateral aid The United Kingdom Department for International Development (DFID) conducted a Multilateral Aid Review in 2011, assessing 43 multilateral organizations on a wide range of criteria

32 Ibid., p. 39. 33 UNICEF, ‘UNICEF West and Central Africa Regional Office Real-Time Independent Assessment (RTIA) of UNICEF’s Response to the Sahel Food and Nutrition Crisis, 2011–2012’, Assessment Report, August 2012.

2 EVALUATION APPROACH AND METHODOLOGY

The methodology for the evaluation is The evaluation team used a combination rooted in the objectives, criteria and sub- of data collection tools and approaches, questions found in the ToR developed by including key respondent interviews, focus the Evaluation Office. The following groups, observation, online surveys and overview of the methodology is detailed in desk research and review. Data collection Annex 2. tools were translated into French and Spanish. Over 1,000 people provided During the inception phase an evaluability information in the data collection phase. type of assessment was conducted to examine stakeholder relationships, The evaluation examined eight criteria determine data availability and develop a related to UNICEF’s implementation of its theory of change. CLA roles. These are relevance and appropriateness, effectiveness (split into Data collection took place at global and predictability, accountability and country levels, with emphasis on eight partnership), efficiency, coherence and country case studies. Case selection was connectedness, coverage and conducted by the Evaluation Office using sustainability. Data collection was a statistical analysis to select a diverse set designed based on 72 indicators, which of country cases that illustrate situations were drawn from over 100 questions in where UNICEF has led clusters or cluster- the evaluation ToR and were linked to the like coordination (see Annex 3). Ethical eight criteria. These indicators were safeguards were instituted by ensuring further consolidated into 33 outcome and confidentiality of data sources and non- capability factors to allow for meaningful attribution of input. analysis.

Country case study context These factors are summarized below in The eight country cases studied for this Table 2. Further background and context evaluation represent a diversity of information can be found in the ToR in capacity and contextual factors. Annex 1.

Table 2: Country case studies

Country Context Description Risk Profile

Number Country of Presence Qualitative Country Office Emergency UNICEF- of Sub- Country CO / CLA Type Description of Office Size by Risk Led National CO Budget Staff # Clusters Clusters or AoR Burkina CO with In Burkina Faso Faso Cluster-like some "groupes Type sectoriels" serve "cluster-like" functions. Clusters were proposed in the context of the 4 Sahel crisis in Child 2011/2012 but Protection the decision was Large Medium Moderate Yes Education made to keep Nutrition pre-existing WASH coordination structures that link donors, the national government and humanitarian agencies in place. El Small CO with The cluster Salvador Active Clusters approach was activated in El Salvador in 2011 in response to national disasters in conjunction with a CERF 4 allocation. El Child Salvador has Protection Small Low Low No one of the Education highest rates of GBV government WASH funding of cluster coordination posts. A good example of a small CO with modest resources in which

government participation and capacity is significant.

Haiti CO with Cluster In Haiti the Deactivation clusters were Experience activated in response to the 2010 earthquake and subsequently the cholera outbreak. Haiti represents an example of a sudden on-set response with a very high number of new actors 4 requiring Child coordination Moderate to Protection Large High Yes support, High Education transitioning Nutrition slowly into WASH recovery. Clusters are slowly phasing- out (the last being the WASH Cluster) with hopes of transfering coordination leadership to the government or national mechanisms. Nepal CO Partnering Nepal is graded with medium/high for Government on emergency risk Clusters (but low frequency and recently small scale). Nepal is an example of clusters co-led in partnership with 4 government. Child Clusters were Moderate to Protection Large Medium Yes activated in High Education conjunction with Nutrition a UN special WASH political mission and the RO is also located in Nepal. OCHA is scaling down its presence, and the HC has transitioned to an RC.

Philippines CO Supporting In the Philippines Government- the cluster Led Clusters approach has been institutionalized and it is fully led 5 by government Child with high Protection capacity. It is Large Medium Moderate Education Yes also used to GBV bridge Nutrition emergency and WASH development programmes under a more specific DRR framework. South Large CO with One of the Sudan Complex Active largest UN Clusters operations, South Sudan has a robust inter- cluster coordination and pooled funding system in which UNICEF leads four clusters in partnership with INGOs. The country and its government are less than 2 years 4 old and remain Child dependent on Protection Large High High Yes international Education support. In such Nutrition a large scale, WASH protracted and chronic humanitarian situation, the clusters are large, decentralized and central to humanitarian response. Sudan is also a pilot country for the IASC Transformative Agenda.

State of CO Partnering The cluster Palestine with INGOs on approach was Clusters introduced in 2008/2009 due to conflict. Ongoing activation is primarilly serving protection oriented advocacy needs. 3 A classic Child example of Large Medium Low Protection Yes clusters led by Education UNICEF and co- WASH led by INGOs that aim to transition to promote national ownership. In addition, it has a high proportion of full time staff dedicated to cluster coordination. Tajikistan CO with Semi- In Tajikistan Dormant clusters were Clusters introduced in 2008 in response to cold weather. Currently the clusters in Tajikistan are considered 4 dormant or Medium Low Moderate Education No seasonal. They WASH are active during an emergency response and are involved in contingency planning and preparedness during the off season.

After examining the data gathered across Cross-case analysis was then carried out cases and types, the evaluation team to identify patterns and trends in the data, used a set of assessment criteria to and average scores across cases were triangulate qualitative and quantitative calculated along with descriptive finding findings. These unified assessment statements. criteria also served as the basis for assigning a performance rating for each Draft findings were presented to key case of the 33 outcome and capability stakeholders during five workshops and statements. A descriptive justification briefings in mid-May 2013 using webinars, statement supplemented each rating. video conferences and face-to-face

11 meetings. UNICEF regional and country took place during the time frame of the offices, headquarters units and external evaluation. partners were included in this process,  Interviews and focus groups were and have contributed significantly to largely scheduled by UNICEF country shaping the recommendations. offices within the protocols developed during the inception phase. The one- Limitations week schedule for country visits The evaluation was conducted within precluded most direct observation of specific parameters related to scope, cluster activities, and also precluded timing, resources and data availability that independent confirmation of findings unavoidably create limitations. The overall originating from interviews. UNICEF time frame for the evaluation was six offices provided contact lists for the months, which is significantly shorter than online surveys. The lists were drawn up similarly scoped global humanitarian based on different interpretations of evaluations. The ToR for the evaluation what qualifies as a cluster participant established a preference for a broad and cluster member in different clusters evaluation of many topics, for many and countries. Therefore, the clusters in multiple country contexts rather distribution lists for the surveys sought than deeply examining a few topics, to maximize the response of clusters or countries. Given the desired organizations, not individuals. breadth rather than depth of the  Elapsed time between some cluster evaluation scope, and the resources activities and the evaluation meant that available to carry it out, the results of the detailed feedback by some important evaluation are meant to provide strong informants was limited. Some indications of performance and influencing interviews were carried out by factors, rather than conclusive causal telephone, rather than in person, which impact statements. may have limited candour or detail in the information provided. This was the Taking this factor into account, the case in the Philippines and Tajikistan conclusions of this evaluation must be case studies, where operational understood in the context of the following: limitations necessitated remote contact.  A lack of consistent understanding  The need to direct data collection and about the CLA role among internal and analysis to questions related to the external stakeholders sometimes broad CLA role precluded in-depth hampered the evaluators in discussing examination of specific cluster key concepts and in collecting performance. meaningful data.  Country case studies were selected to  No pre-existing global monitoring and illustrate different situations and levels reporting system is in place to allow for of capacity, rather than to emphasize measuring performance against a the largest and possibly most cluster- baseline. Similarly, few quantifiable relevant humanitarian situations. The data on cluster activation and result was that the majority of case performance are available through the studies illustrate cluster activities IASC. outside emergency response  Some limitations of data availability coordination. were encountered in instances where  Focus of resources on country case record-keeping is not systematized and studies precluded broad interview consolidated at a global or country follow-up with Global Cluster members. level and is difficult to trace due to staff And only one Global Cluster meeting turnover.

3 KEY FINDINGS AND ANALYSIS

Following data collection, the evaluation patterns and trends in the detailed results team analysed the data gathered from five key areas were identified to global and case study interviews, focus communicate the most strategic findings groups, documents reviewed and surveys of the evaluation. These are: for each evaluation indicator and developed performance ratings and 1. External coordination performance justification statements for each. The 2. Internal CLA performance result of this analysis was compared and 3. Human resources performance contrasted across cases to identify 4. Scope and boundary patterns in the triangulated data. Based 5. Cost-effectiveness, value for money on the most frequent and significant and efficiency

3.1. External coordination The evaluation of UNICEF’s 2011/2012 performance response to the Horn of Africa crisis found that cluster leadership received a high External coordination performance was approval rating from 78 per cent of survey assessed based on three key elements: respondents.34

1. Overall effectiveness Interviews and focus groups in case 2. Coverage countries showed complementary high 3. Partnership degrees of partner satisfaction with UNICEF’s overall coordination Overall effectiveness performance at country level. While The evaluation found that partners believe partners interviewed identified various UNICEF-led clusters and AORs are points requiring improvement, in general broadly effective. Across case studies and UNICEF’s cluster leadership was clusters, 87 per cent of partners reported consistently rated well, often with in the online surveys that UNICEF is accompanying comments that UNICEF “somewhat” to “very effective” in its cluster does the best that can be expected within coordination responsibilities at country the complex country and overarching level. coordination contexts.

The range across clusters and AORs at Around 80 per cent of partners also stated country level was between 71 and 90 per that UNICEF country clusters provide cent. These positive views of UNICEF’s “somewhat” effective to “very effective” cluster leadership are supported in other operational support through needs recent evaluations. assessment, advocacy and resource mobilization.

Figure 1: Effectiveness of cluster coordination

34 UNICEF’s Response to the Emergency in the Horn of Africa, 2011–2012.

In focus groups, cluster members also Focus group and interview participants gave a positive rating to UNICEF in regard consistently noted similar themes, to needs assessment and advocacy. including strength, e.g., knowledge, Reflections on the resource mobilization experience and soft (coordination) skills of efforts of UNICEF on behalf of cluster coordinators; UNICEF’s organizational members varied by country context, infrastructure; and support of country however, with greater satisfaction in office leadership for cluster work. places where overall humanitarian funding The document review further supports the was more stable and where cluster importance of these issues. A 2011 coordinators were able to help members UNICEF synthesis from implementing the access funding. cluster approach35 identified a series of internal and external factors that influence Qualitative data from open-ended survey CLA performance. responses and interviews illustrate some of the explanatory factors that most Among the internal factors, this synthesis influence UNICEF’s CLA performance. stressed the importance of establishing clear internal understanding of leadership Analysis of open-ended qualitative survey accountabilities, effectively managing the responses shows that human resources, separation between programme and institutional support and leadership are cluster coordination responsibilities and most frequently cited as influencing increasing internal knowledge of CLA overall effectiveness – and each of these responsibilities to successfully three factors is noted 8 to 10 times more communicate context and results to help frequently than technical expertise. manage expectations, not underestimating the staffing requirements for effective cluster coordination and ensuring an adequate pool of staff with “interpersonal skills, technical competence, experience and seniority to Figure 2: Factors influencing CLA ‘lead’ clusters.” effectiveness

35 UNICEF, ‘Implementing the Cluster Approach: A synthesis of documented lessons (2006–2010)’, Programme Division, December 2011.

Coverage that “Subnational coordination, though not The evaluation found – across case always strong, has the potential to studies, interviews, survey data and improve coordinated service delivery. Its documentary evidence – that UNICEF-led improvement, however, will require the clusters are having positive effects on cluster system to be adapted at the subnational level to intentionally merge geographic and thematic coverage. 36 Two thirds of partners responding to groups of clusters.” country surveys said that cluster Transition from and to sectoral Figure 3: Gap identification and problem coordination mechanisms or other forms solving of humanitarian coordination is problematic. In some cases, the clusters have overwhelmed or competed with pre- existing coordination structures and weakened them: for example, when the cluster system was activated in the State of Palestine at the time of the Gaza conflict in late 2008.

Prior to activation of the clusters in the State of Palestine a national WASH sectoral coordination mechanism called “e-WASH” was in place. According to numerous internal and external interview participants the WASH Cluster did not manage to complement the activities of coordinators succeed in identifying gaps, the e-WASH and instead resulted in a and only half said that coordinators parallel mechanism. Donors reallocated effectively facilitate problem-solving. funding to the WASH Cluster, lessening In about half of the case studies, the the capacity of the e-WASH mechanism evaluation team found that UNICEF-led and reducing the potential to handover clusters have established subnational coordination if the clusters are coordination structures. In South Sudan deactivated. all UNICEF-led clusters/AORs have subnational structures in 7 to 10 States, In other cases, clusters were replacing usually with a double-hatting NGO sectoral development and humanitarian coordinator. In other countries such as contingency planning coordination by Haiti and the State of Palestine just a few filling gaps in long-term coordination critical subnational clusters are needs rather than effectively establishing maintained. Key informants note that capacity and transferring coordination sustaining subnational coordination responsibilities. This was particularly structures is difficult as it significantly evident in the Nepal case study where increases costs, there is high turnover of clusters have been primarily replacing NGO staff assigned to lead subnational sectoral development coordination since clusters, and often subnational presence their activation with additional roles for and capacity of cluster members are humanitarian contingency planning. Such limited, leaving fewer organizations to dilemmas with long-term coordination coordinate. needs were also evident to some extent in

The evaluation of UNICEF’s 2011/2012 36 UNICEF’s Response to the Emergency in the response to the Horn of Africa crisis noted Horn of Africa, 2011–2012.

17 the later stages of the activation in Haiti understanding of the need for clusters to where identifying or building suitable long- maintain independence. term coordination structures for cluster handover has proved difficult. According to global survey respondents and country-level key informants, The Burkina Faso case study provided an UNICEF-led and co-led clusters and example of UNICEF, the national AORs have had positive effects on government and partners resisting cluster thematic coverage. activation. Many months into the nutrition crisis, some international actors The evaluators observed particularly advocated cluster activation. As long-term focused efforts to increase knowledge and coordination mechanisms were already in awareness of thematic issues related to place, and donors, the national gender-based violence and education in government and UNICEF believed these emergencies through operations research mechanisms were able to meet and evaluation, as well as advocacy and coordination needs, cluster activation was educational efforts. In South Sudan the rejected. UNICEF Country Representative effectively advocated throughout 2012 to UNICEF leadership has recognized the raise the standing and priority of the need for better systems for transition and Education Cluster in the Consolidated investment in sectoral coordination Appeal Process (CAP) and Common mechanisms and have noted to the IASC Humanitarian Fund (CHF) process, and that this gap needs to be addressed within the country office has undertaken focused the IASC and with donors. operational research to demonstrate the effect of education in emergency work. Overall, the evaluation found consistent Additionally, the evaluation found that evidence that UNICEF-led clusters include UNICEF-led clusters and AORs have national authorities in their work. In South promoted gender mainstreaming and Sudan government ministries co-chair Figure 4: Gender mainstreaming and most cluster meetings. In El Salvador, human rights Nepal and the Philippines the government is considered the official cluster lead with UNICEF cluster coordinators providing support or co-leading clusters.

UNICEF’s strong relationships with national authorities were observed in most case study countries, although these relationships are often focused on long- term development interests. In some cases, the involvement of national authorities has also created some risks to independence. For example, in one case, the evaluation found that the likelihood of civil unrest and conflict scenarios was markedly absent from contingency plans and cluster discussions, reportedly due to government sensitivities. In other cases, however, country representatives and human rights in their work. cluster coordinators demonstrated a clear Interviews and focus groups consistently stated that demonstrating accountability to

affected populations is restricted by Partnership information management and reporting Roles and responsibilities are reportedly system limitations, including shortages of clearer at country level than at global capable information management staff, level: 85 per cent of country partners and difficulties in collecting information from 61 per cent of global partners responding partners, and difficulties in sustaining the to surveys reported that responsibilities costs of a robust monitoring and reporting and roles are clearly agreed. However, system. Interview participants noted that the evaluation team observed and found joint accountability with national documentary evidence showing that governments is more difficult as it global and country cluster coordinators depends on governments’ capacities and clearly make substantial efforts to their concepts of accountability. establish shared responsibility by developing workplans, who, what, where matrices, and other mapping of capacities and activities.

In survey responses, country-level partners also rate UNICEF better than its global partners on adherence to the Global Humanitarian Platform’s principles of partnership.

Figure 5: Adherence to partnership principles according to partners

Figure 6: Effectiveness of UNICEF’s representation of its clusters at HCTs

Across Global Clusters, between 49 and interest, as well as competing priorities, of 63 per cent of partners stated that country representatives. UNICEF adheres to each of the principles of partnership. The lowest combined UNICEF representatives and country score for global and country-level partners office staff reported different reporting was for transparency.37 structures for cluster coordinators – with some coordinators reporting directly to a At global level, the evaluation also found representative or deputy representative that UNICEF has increased its level of and others reporting to a chief of section. participation, representation and Staff frequently noted that direct reporting coherence at the IASC and through lines to a representative can help ensure collaboration with other CLAs on cluster that cluster issues are clearly initiatives since clusters were consolidated communicated to a representative so they under the GCCU. can articulate issues to an HCT. Staff also noted that a representative’s ability and At country level, the evaluators observed willingness to represent these issues and heard in interviews that UNICEF’s depended on their understanding of the representation of its clusters at CLA role, and that with five clusters or humanitarian country teams (HCTs) varies sub-clusters to represent sometimes only widely depending on the knowledge and the most critical issues can be raised.

External survey respondents questioned UNICEF’s ability to represent the interests 37 Transparency is defined by the Global of cluster members rather than its own Humanitarian Partnership as, “Transparency is organizational interests. This is further achieved through dialogue (on equal footing), with an emphasis on early consultations and early confirmed by the variance shown in the sharing of information. Communications and survey responses from country partners transparency, including financial transparency, regarding the representation of cluster increase the level of trust among organizations.” members in HCTs. This definition was used in evaluation questionnaires.

Figure 7: UNICEF’s participation as a member of clusters

Some 39 per cent of partners across UNICEF’s strong relationships with country case studies said UNICEF is “very governments were found to be an asset in effective” in representing their concerns at the early stages of an emergency, but HCTs as opposed to UNICEF’s own pose challenges when it comes to institutional perspective; on average, 41 deactivation and transition. per cent said UNICEF is only “somewhat effective” and 20 per cent said UNICEF is In many country cases, governments “not effective.” demonstrated focus on the resource mobilization benefits of clusters, rather Of Global Cluster members surveyed than how they can complement and build across clusters, 81 per cent said UNICEF existing national disaster management participates in the clusters or AORs systems. It takes time to reorient pre- equally as a member in addition to its existing government relationships to work coordination roles. However, 19 per cent in a cluster system – and convincing said UNICEF does not, with particular governments of the need to transition out concerns for the Education, Nutrition and of clusters can be difficult. WASH Clusters.

Despite this, documentary evidence38 supported by interviews, shows that UNICEF is well represented in Global Clusters, though some UNICEF staff noted that their participation has shifted towards playing a more “defensive role” to monitor cluster activities and growth in scope and advocate against growth of clusters beyond their core purpose and UNICEF capacities.

38 Global Cluster and AOR Meeting Minutes and Workplans.

Summary findings: External thematic coverage, but transition to and coordination performance from other coordination mechanisms and nationally led coordination is problematic. UNICEF is reasonably effective at exercising its country cluster coordination While the partnership aspects of UNICEF- responsibilities. This is most influenced by led clusters and AORs are stronger at human resources, institutional support country levels than for Global Clusters, and leadership. UNICEF is not perceived to effectively

represent cluster members at country UNICEF-led clusters and AORs have had HCTs, in most cases. positive effects on geographical and

3.2 Internal CLA performance The evaluation team also observed some economies of scale (in representation and Internal CLA performance was assessed information management), and sharing of on the basis of four key elements: good practice across clusters, although this is still ad hoc. Since cluster 1. The establishment of the Global coordination staff were relocated and Cluster Coordination Unit (GCCU) consolidated in Geneva, the physical 2. Policy and performance monitoring distance between staff in the Programme 3. The role of regional offices Division in New York and cluster 4. Comparative advantages of UNICEF coordination staff in Geneva has reportedly reduced day-to-day interactions Establishment of the Global Cluster between some programme staff and their Coordination Unit (GCCU) cluster counterparts. The consolidation of the Global Clusters under the GCCU in Geneva in 2012 has demonstrated certain benefits. Over 70 per cent of UNICEF country staff members who were surveyed stated that these changes have “moderately” or “substantially” improved cluster coordination and leadership in the first year of the GCCU.

Figure 8: Effects of the GCCU on cluster leadership

A few UNICEF headquarters staff also Consideration of CLA roles in strongly cited concerns that the GCCU is policy and performance monitoring now sometimes operating independently, The 2010 revised Core Commitments for and this may reinforce cluster growth Children40 (CCCs) explicitly include beyond its intended mandate. organizational commitments for cluster

and inter-agency coordination. The The evaluators found that cross-cluster evaluation found that cluster coordination management approaches for the GCCU broadly contributes to UNICEF’s were initially under-resourced. Early achievement of the coordination elements decision support memos39 leading up to in the CCCs. However, performance the transition provided two options for measurement systems linked to CCCs are putting a dedicated cross-cluster currently output focused rather than management structure in place to remedy outcome focused, and country office identified weaknesses in the previous reporting is voluntary, so there is no fragmented structure within the regular global analysis of coordination Programme Division. When the GCCU performance. was established, a third approach was implemented with Global Cluster The UNICEF cluster/sector coordination Coordinators reporting directly to the milestone monitoring tool contained within Deputy Director of EMOPS in Geneva, the Humanitarian Performance Monitoring who is supported by an Inter-cluster Toolkit41 focuses on a series of simple Coordinator. “yes/no” questions such as whether a

coordinator position is filled and if not for This has resulted in less than adequate how long it was vacant. time and attention being devoted to global cross-cluster management. The GCCs The series of indicators does not seek to meet together with the Inter-cluster measure the quality of personnel, Coordinator for basic information sharing, capacities and tools used, or the success and they attend EMOPS staff meetings. of implementation (e.g. resolution of According to interviews with staff, problems, duration of coordinator’s observation and review of GCCU deployments vs. turnover frequency, and documents have not yet been partner ratings of coordination complemented with focused management performance). There is only a checklist of of the five global coordinators to ensure whether they are in place. sound performance, identifying opportunities to create synergies and Within UNICEF, a capacity development sharing good practice. The planned effort initiative linked to the CCCs42 has been to develop a common GCCU, or perhaps launched to provide advice to country CLA, strategy is intended to strengthen offices on implementation, including these practices. Recently proposed lessons-learned components on good changes to the role and level of the Inter- coordination practice. An ongoing cluster Coordinator position in the 2012 UNICEF-wide effort to strengthen EMOPS Office Management Plans humanitarian performance monitoring also process are designed to address these challenges.

40 UNICEF, Core Commitments for Children in Humanitarian Action, May 2010. 41 Humanitarian Performance Monitoring – Toolkit 39 UNICEF Internal Working Document, ‘Position and Guide. Paper by the UNICEF Global Cluster Coordinators: 42 UNICEF, ‘Capacity Development for the Core Management of the UNICEF-led clusters/AORs’, Commitments for Children in Humanitarian Action: February 2011. A technical note’, July 2011.

seeks to address some of these role have been documented. Evidence weaknesses. suggests, however, that UNICEF has not absorbed these lessons through UNICEF-led and co-led clusters and implementing necessary adjustments.44 AORs each have a workplan and results In 2012 and 2013, an IASC working group framework and have periodically developed a country-level coordination sponsored operational lessons-learned performance monitoring tool45 with exercises and reviews. The evaluation UNICEF input that is currently being found, however, that these are not piloted in two areas (South Sudan and the standardized within or across clusters, State of Palestine). This performance and their results are not systematically monitoring tool focuses on the collective consolidated and synthesized. UNICEF- performance of individual clusters based wide operations reviews, lessons-learned on the cluster functions outlined in the exercises and evaluations often touch on IASC Reference Module for Cluster issues related to cluster coordination and Coordination at the Country Level. CLA practice. A summary of these lessons was created by UNICEF’s The tool is primarily based on qualitative Programme Division in 2011.43 self-assessment of minimum standards and does not include critical CLA UNICEF knowledge management measures related to partnership systems were found to be insufficient to effectiveness, cost-effectiveness, and support CLA roles at many levels, with no relevance and appropriateness. A concept central archival systems, and significant note on CLA performance measurement loss of knowledge and information when is included in Annex 6. turnover or organizational change takes place. For example, when end-of-mission Other than this evaluation of UNICEF’s notes and handover documents were CLA role, no other overarching CLA requested at headquarters and country performance monitoring system and office level the evaluation team was results framework was found within mostly told that such documentation was UNICEF. not systematically collected and that the only way to obtain such documentation The evaluation also found that no global would be to request it from each individual performance indicators have been defined coordinator. since the initial 2006 IASC guidance note on implementing the cluster approach, Although individual clusters and the which articulated the responsibilities of GCCU were found to be investing in global CLAs in a few paragraphs. development of information management practices and systems, reported The role of regional offices weaknesses in UNICEF-wide knowledge The evaluation team found that the role of management systems may limit the the regional offices and regional potential of these efforts. emergency advisers remains unclear.

Many individual clusters and AORs have While the CCCs articulate a coordination demonstrated their commitment to mandate, it has not been translated into learning by following up lessons learned clear responsibilities and accountability and other assessments of their work, and broad lessons on implementing the CLA 44 ‘Cluster Issues Matrix: Internal’. 45 IASC Draft Working Document, ‘Cluster 43 ‘Implementing the Cluster Approach: A synthesis Coordination Reference Module: Coordination of documented lessons (2006–2010)’. performance monitoring’, August 2012.

25 mechanisms at regional level. Many that questions of accountability and roles interviewees mentioned that they were between headquarters, regional offices unaware of a specific role and that there and country offices arose early in the was no particular engagement on cluster operation but were left unresolved, issues at regional level. Regional office resulting in a “chain of consultation” rather staff members also noted ambiguity about than a clear chain of command.48 their role, accountability and authority to provide support and oversight for cluster In 2010 the UNICEF Programme Division work. undertook an analysis of internal and external cluster issues. The internal The evaluation found some examples in issues analysis noted that the “role of the country case studies of regional regional office in terms of strategic emergency advisers (REAs) and other oversight and quality assurance for CLA regional technical staff providing ad hoc implementation at country office level is support to country-level clusters, such as currently unclear, and each regional office help in identifying tools and contacts at has engaged with a different approach.”49 headquarters. The Global WASH Cluster has also engaged UNICEF regional The evaluation found only one example of emergency WASH advisers “to support organized and systematic regional roll-out of global WASH cluster tools and support: in the temporarily combined implementation of the cluster approach.”46 UNICEF Asia Pacific Shared Services Center (APSSC). The APSSC combined The lack of clarity on regional office roles technical emergency staff from the South may stem from a general confusion on Asia and East Asia and Pacific regional roles, accountabilities and authorities offices and developed cluster-related between UNICEF headquarters, regional capacity assessment and development offices and country offices. tools and approaches. Key informants cited these as providing valuable support The 2012 evaluation of UNICEF’s to country offices when implementing the response to the Horn of Africa crisis47 CLA role. noted that “uncertainties regarding accountability are intrinsic to UNICEF’s A 2011 exercise to document the lessons decentralized management structure…. learned from the experience of the Authority is given to the country APSSC50 found “An emergency unit representative, with regional office and uniquely structured at a regional level headquarters having some – not quite around UNICEF’s Global Cluster clear – measure of accountability for Leadership Accountabilities is an country programme outcomes.” The report innovative model which has proven goes on to state that this confusion effective in supporting UNICEF’s country becomes more obvious in emergencies offices across regions to deliver on their because of the “pressure to act quickly Core Commitments for Children in and get results.” Humanitarian Action (CCCs).”

The 2011 evaluation of UNICEF’s While these tools and approaches have, response to the Haiti earthquake noted to some extent, been adopted by Global

46 The Global WASH Learning Project, 48 ‘Independent Review of UNICEF’s Operational ‘Implementation of the WASH Cluster Approach: Response to the January 2010 Earthquake in Haiti’. Good practice and lessons learned’, date not 49 ‘Cluster Issues Matrix: Internal’. provided. 50 UNICEF, ‘Lessons from the Experiences of 47 UNICEF’s Response to the Emergency in the UNICEF’s Asia-Pacific Shared Services Centre’, Horn of Africa, 2011–201. Internal exercise, February 2011.

Clusters, the APSSC was disbanded in The comparative advantages 2012, and the two regions no longer of UNICEF provide the same type of cluster support The evaluation found that UNICEF’s to country offices. programme strengths tend to match each

of its CLA and AOR focal point roles. That The lack of clarity in the role of regional said, UNICEF is self-confident about its offices is considered by internal key programme strengths and believes it is informants to be a significant gap in the most suitable organization to serve as UNICEF’s internal CLA performance. a CLA in each of its sectors. In survey Lines of authority between country offices responses, significantly more global and headquarters run through the regional partners than country partners questioned offices directly to UNICEF’s Executive UNICEF’s strength in programming and Director. Key respondent interviews, coordination. This reserved reaction at the documentary evidence and observations global level significantly differs from suggest that many UNICEF country UNICEF’s own perceived strengths. offices and representatives operate in a highly autonomous way, with unclear Across clusters, 66 per cent of country oversight or performance management partners stated that UNICEF’s programme from above. The Programme Division strengths are stronger than other analysis of internal cluster issues noted humanitarian actors in the sectors where it that “At CO level, UNICEF has a CLA role; while only 40 per cent of Representatives have a high degree of global partners said its programme autonomy.”51 strengths are stronger.

As a result, various headquarters divisions and units – including the GCCU – have no direct line of authority to country representatives and their offices. This means that all GCCU and EMOPS efforts to establish consistent and predictable country office approaches to implement the CLA role must currently rely on persuasion, deployments and written guidance to influence a highly decentralized country-driven system. Such efforts have little influence if country office managers are not supportive or open to guidance.

51 ‘Cluster Issues Matrix: Internal’.

Figure 9: UNICEF programme strength compared to other actors

Figure 10: UNICEF’s strength in programming supports its CLA roles

Overall, 47 per cent of global partners Summary findings: responding to the surveys stated that Internal CLA performance UNICEF’s strength in programming The consolidation of Global Cluster staff supports its CLA roles across clusters, under the GCCU is beginning to yield while 66 per cent of country partners positive results. This includes economies agreed. of scale, for example in information

management, quality and coherence of Across clusters, 61 per cent of country- external representation, and informal level survey respondents agreed that sharing of good practice across clusters. UNICEF is effectively harnessing its

coordination skills and capacities to fulfil The CLA role is increasingly incorporated its CLA roles, while only 36 per cent of into broad UNICEF policies and initiatives. global partners agreed. Most key Nonetheless, their interpretation is informants stated in interviews and focus inconsistent, and performance groups that cluster coordinators need both measurement, monitoring and knowledge technical and coordination skills, although management remain weak for CLA roles coordinators who have the necessary within UNICEF. ‘soft’ (coordination) skills can succeed

without deep technical experience, while a The role of regional offices in relation to coordinator with only technical skills is CLA responsibilities remains unclear, yet unlikely to succeed. At country level, regional offices are the only formal link in programme and coordination strengths UNICEF’s accountability chain between were found to be both an asset and a country offices and headquarters. limitation as they were often described by

key informants as based on sector UNICEF’s CLA roles broadly match its development-oriented programmes and comparative advantages in programming, relationships, which can detract from though UNICEF’s programme strengths cluster coordination independence. and ability to leverage its coordination

skills and capacities are both questioned by global cluster members.

Figure 11: UNICEF harnesses internal coordination skills and capacities for CLA roles

3.3 Human resources However, improvements in surge capacity performance systems are evidenced by the positive findings from the evaluation of UNICEF’s Human resources performance was response to the 2011/2012 Horn of Africa assessed based on two key elements: crisis. This evaluation found that UNICEF successfully provided surge capacity to 1. Surge capacity and recruitment strengthen the clusters and sectors it systems leads. 2. Cluster coordination capacity development

Surge capacity and recruitment systems Overall, the evaluation found that UNICEF has made significant improvements in its surge capacity systems since 2010.

The evaluation of UNICEF’s response to the 2010 Haiti earthquake52 found that leadership capacity in UNICEF-led clusters was weak in the first three months of the response with the exception of the WASH Cluster. This evaluation pointed out “slow regular recruitment processes and lack of protocol for scaling up meant that the surge was greatly protracted (up to six months).”

Table 3: 2012 Global Cluster deployment statistics by mechanism

Global mechanism # of positions # of countries Average Range of deployment filled/ deployment duration duration deployments DHR 8 6 188 days Low 80 days High 364 days Globally managed standby 28 15 117 days Low 15 days partner rosters High 279 days

Cluster/AOR RRTs 59 21 37 days Low 4 days High 95 days

52 ‘Independent Review of UNICEF’s Operational Response to the January 2010 Earthquake in Haiti’.

The survey conducted for the evaluation The evaluation found that DHR and found that 81 per cent of respondents globally managed standby partner rosters agreed that UNICEF successfully combined are providing approximately 50 deployed its own staff, and 77 per cent per cent of the demand for country-level agreed that standby partner staffers were cluster coordination staff. The other 50 per deployed successfully and all categories cent of country-level cluster coordination of personnel deployed “were seen to be of positions are being filled by country offices high quality.”53 through direct recruitment. The following pros and cons were identified for each Since the GCCU was established in 2012, type of recruitment mechanism. UNICEF has worked to expand the rapid response teams (RRTs) for clusters that did not previously have a dedicated group of staff available for deployment and operational support. Each cluster and AOR has five RRT members with the exception of the Education Cluster, because of resource constraints. The Division of Human Resources (DHR) emergency unit was re-established in 2010 after being dismantled. It has increased its recruitment of requested cluster positions and started working with the Programme Division to review coordination qualifications for UNICEF global roster members. The evaluation found that three global mechanisms are in place to provide surge capacity and recruitment support to country-level clusters: DHR emergency unit’s systems, globally managed standby partner rosters and the rapid response teams for the Global Cluster and AOR. The 2012 deployment statistics for each of these three mechanisms is shown in Table 3. DHR and globally managed standby partner rosters are the most often-used mechanisms for recruitment and deployment of longer-term cluster staff based on the average duration of deployments. Interview responses show that RRTs are reportedly deployed more often for first-wave deployments until a longer-term candidate can be identified, as well as for technical assessments and advice on setting up coordination systems.

53 UNICEF’s Response to the Emergency in the Horn of Africa, 2011-2012.

Table 4: Pros and cons of global deployment systems

Mechanism Pros Cons DHR  Access to experienced UNICEF staff  Available staff are often retired and less  DHR staff’s skills and competencies familiar with clusters  Ability to recruit new staff based on  No single integrated database available to needed skills support rapid deployment (qualifications,  Access to staff with knowledge and UNICEF history, performance records) experience of UNICEF systems  Recruits not necessarily pre-trained  Higher cost for international staff  Lack of career path and incentives for UNICEF staff limits current DHR pool

Stand-by partner  Deployments often fully funded by  High turnover among roster members limits rosters partners cluster experience  Contractual arrangements allow for rapid  Contractual status limits privileges and deployment authority  Credibility with partners due to links with  Higher training-to-deployment ratio to ensure NGOs availability  Shared cluster coordination  Some partners keep their best staff for their accountability with partners own deployments (quality)  Standby staff recruited primarily for technical vs. soft skills

RRTs  Many RRTs funded by partners  Only able to cover needs as a first-wave  Full-time role assures availability deployment and technical support mechanism  More experience in cluster coordination  High reliance on external resources for and linked to GCCU guidance and UNICEF’s CLA responsibilities resources  Contractual status limits privileges and  Shared cluster coordination authority accountability with partners

Country office  Ability to recruit from national staff (local  External recruits and national staff not direct knowledge and lower cost) necessarily pre-trained recruitment  Recruitment more informed by local  Lack of connection with global clusters context and needs  Often recruited for double-hatted roles  Speed and cost of deployment

The three global mechanisms are deployment in emergencies, except during currently managed by three separate a short period from 2006–2008 when the groups, and there is no overarching DHR emergency unit existed.”54 strategy for cluster surge capacity. This Demand prediction – the number of fragmentation and lack of strategy has people and skills needed – is difficult persisted since at least 2010, as given different uses of clusters in many evidenced by findings in the 2011 places for prolonged periods of time. As a evaluation of UNICEF’s response to the result, the decision on which mechanism Haiti earthquake, which noted: “There has been no centrally coordinated process to 54 ‘Independent Review of UNICEF’s Operational develop the roster as a strategic tool for Response to the January 2010 Earthquake in Haiti’.

to use to fill a cluster coordination position and stated there remains a preference for is ad hoc, and often depends on who roles in direct programming rather than in takes action first based on country office coordination. preference (or lack of action). An internal UNICEF effectiveness and efficiency Double-hatting remains prevalent among initiative has recently identified that staff who are recruited or assigned directly fragmented human resources efforts are a by country offices across countries. Based potential area for change. on analysis of data from the 2012 EMOPS survey of cluster activity,56 only 23 per Another factor influencing surge capacity cent of UNICEF staff who reported is that the overall pool of expertise for working with clusters or AORs were full some sectors is quite limited. Key time. While the percentage of full-time informants reported in interviews that staff is higher for countries in the midst of UNICEF is generally challenged by an ongoing emergency, even then the industry-wide gaps in emergency nutrition, figure only reaches 28 per cent. A third of education and gender-based violence countries with ongoing emergencies have personnel, and this affects cluster two or fewer full-time staff dedicated to all recruitment and roster capabilities. of the UNICEF-led clusters or AORs. In interviews, key internal and external Some internal process and procedural informants agreed that double-hatting is impediments also exist in rapid surge not satisfactory during emergencies as capacity deployment. The fast-track cluster coordination is a full-time job, and recruitment procedures developed by members expect a focused and UNICEF in recent years to expedite independent coordinator. emergency recruitment and deployment reportedly only apply to fixed-term The evaluation found that double-hatting positions.55 Medical clearance and is not an appropriate, relevant or effective laissez-passer processing times are noted staffing solution for key country-level as impediments to deploying new cluster coordination positions in active temporary assignment staff. emergency settings. Partners and staff noted that these positions have full-time Global surge capacity systems primarily responsibilities and need to be dedicated recruit people from outside UNICEF on to focus on independently supporting temporary and short-term consulting cluster coordination. Analysis of the contracts. This is because cluster reported workload demands on cluster coordination is seen as a temporary coordinators confirms the need for function, regardless of cluster longevity. dedicated coordinators. These contracts have limited time spans and job security. Fixed-term staff can Cluster coordination capacity potentially deploy quickly, but do not have development an incentive to serve as cluster Virtually all key informants agreed that coordinators – currently, there is no clear predictability of consistent high-quality coordination career path within UNICEF, cluster coordination performance depends and giving up job security for a temporary highly on the cluster coordinator. Similarly, assignment is rare. Fixed-term UNICEF the evaluation found that the predictability staff also noted cultural challenges, such and consistency of quality CLA as being labelled as emergency focused,

55 UNICEF, Review of the Fast Track Recruitment 56 ‘A Survey of the Operational Status and Staffing Process, Draft Report, Division of Human of UNICEF-led and Co-led Clusters and AORs in Resources, January 2013. the Field’.

Figure 12: Cluster coordinators’ technical and coordination skill levels

performance depends on the level of A review of cluster coordinator capacity understanding of the CLA role, interests enhancement was conducted in 2013 for and related skills of country the UNICEF Middle East and North Africa representatives and deputies. Regional Office (MENARO).57 This review found that “no overall strategy for capacity Capacity development for cluster enhancement for cluster coordinators coordinators is focused on a limited four- exists” at global or regional level. or five-day classroom-style training – with an emphasis on awareness-building Recent efforts – launched by the GCCU to rather than e-learning, interactive create a capacity development strategy – simulation, mentoring and performance are still relatively focused on training, but management. Paradoxically, most people collaboration with other CLAs in this who are trained are not deployed often, initiative and recent adjustments suggest due to turnover, availability and poor the GCCU is now seeking to establish a linkages between training lists, rosters more comprehensive approach. and country offices that make decisions on the selection of coordinators. According to country survey responses, approximately 25 per cent of people serving in cluster coordination roles have never received cluster training.

57 Richard Luff, ‘Review of MENARO’s cluster- coordinators capacity enhancement actions 2008– 2012’, March 2013.

Across country surveys, 62 per cent of Summary findings: Human partners stated that UNICEF cluster resources performance coordinators have the necessary technical Surge capacity and recruitment systems skills, and 63 per cent said they have the have improved since 2010, but their required coordination skills. This suggests management remains fragmented and room for improvement in both the there is no overarching strategy for cluster selection and capacity development of human resources. There is no career path cluster coordinators. that would encourage UNICEF staff to serve in coordination roles. Double-hatting The evaluation found virtually no remains prevalent and diminishes systematic efforts across UNICEF to build effectiveness, even in ongoing emergency the understanding, interest and skills of situations, where only 28 per cent of staff country representatives and their deputies working on cluster activities are full time. to fulfil CLA roles. Coordination performance depends highly The evaluation of UNICEF’s response to on the skills and capacities of cluster the Haiti earthquake noted that “many coordinators, and overall CLA senior managers, section chiefs and performance depends highly on the representatives were not trained in understanding and approaches of country clusters and had no experience working representatives. Capacity development 58 with them.” The induction training and has been limited to classroom-based orientation for representatives and awareness training. Paradoxically, many deputies is, reportedly, only required once people trained are not deployed, and in their career. many cluster coordinators have not been trained. The evaluation did not find any A high percentage of current systematic efforts to build understanding representatives and deputies first went and consistency of approaches among through the induction before the cluster country representatives and their system was developed. While the deputies, despite the critical role they play induction course has been revised to in implementing the CLA role. include some information on the cluster system, key informants estimated that, with the current system, it would be decades before all deputies and representatives receive the updated training.

UNICEF staff, including representatives, noted that there is no current systematic interaction across country representatives in different locations. No global meetings are held to bring country representatives together to discuss organizational priorities and learning about key responsibilities, such as the CLA role.

58 ‘Independent Review of UNICEF’s Operational Response to the January 2010 Earthquake in Haiti’.

3.4 Scope and boundary understand IASC and cluster-specific issues tools and guidance, and only 50 per cent said that coordinators use such tools and Scope and boundary issues were guidance. assessed on the basis of two key elements: Key informants noted that this is partly related to the utility of existing tools and 1. The scope of UNICEF’s cluster guidance and partly due to the volume of responsibilities existing guidance (the number of 2. The operational role of UNICEF’s documents and their overall length). clusters and AORs During the evaluation – in focus groups and workshops – stakeholders noted they do not need more guidance but, instead, The scope of UNICEF’s cluster need to better understand how to use and responsibilities implement existing guidance. Overall, the evaluation team found that UNICEF faces challenges in implementing Although UNICEF was found to be making its CLA roles due to an evolving and significant efforts to influence the IASC inconsistently understood and applied and to push for resolution on long- IASC operating framework. outstanding gaps in clarity, very little evidence was found of UNICEF-wide Policies and guidance are perceived as efforts to interpret and guide being ambiguous, while inconsistent use implementation of IASC policy by country of the cluster system to fill an increasing offices. number of needs beyond emergency coordination have created an environment The evaluation found that the IASC where expectations are difficult to Reference Module for Cluster manage, and new roles and uses of Coordination at the Country Level,60 clusters are replicated in other contexts. developed in 2012, provides greater clarity but is still being field tested. The While UNICEF-led clusters are broadly reference module still leaves decisions on aligned with IASC policy, these perceived “the most appropriate coordination ambiguities mean that much is left open to mechanisms” – not necessarily clusters – interpretation at country level. This has led up to HCs, Resident Coordinators (RCs) to a wide variance in practice across and HCTs. countries by UNICEF country offices, HCs and HCTs. This document stresses, however, that IASC principals have called for cluster The evaluation UNICEF’s 2011/2012 activation to be more strategic, less response in the Horn of Africa found that automatic, and time limited. They clearly “Cluster/sector coordination arrangements define the activation of clusters as a vary between countries in the [Horn of temporary coordination solution linked to Africa] and do not follow the IASC international emergency response. The standard cluster approach in any reference module also calls for reviews 59 country.” every six months to assess the relevance of activation and for supporting nationally Just 59 per cent of country-level partners stated that UNICEF cluster coordinators

60 IASC, ‘Reference Module for Cluster 59 UNICEF’s Response to the Emergency in the Coordination at the Country Level’, November Horn of Africa, 2011–2012. 2012.

led coordination mechanisms as the The Haiti case study provided the clearest preferred option. examples of attempted handover and transition planning yet shows the Cluster staff and partners have a widely challenges for UNICEF of doing so. Due differing understanding of what the to gaps in the capacity of other actors, the ‘provider of last resort’ concept entails. WASH Cluster coordination was Many internal and external key informants prolonged as a transitional capacity- see UNICEF as sometimes playing a building and preparedness mechanism ‘provider of first resort’ role when while the Education Cluster invested in resources are available, especially in efforts to hand over coordination to clusters where the vast majority of national counterparts that did not continue participants are also UNICEF cluster initiatives. implementing partners. In Nepal the HC has now transitioned to a The original ‘provider of last resort’ Resident Coordinator and some clusters concept was found, in the evaluation, to have closed but UNICEF led (and co-led) be largely meaningless given the 2008 clusters are awaiting a clear agreement revisions61 to its definition – "depending on a cluster transition strategy that has on access, security and availability of been discussed for over two years, partly funding" – which can be used to explain due to government reluctance to expedite almost all operational gaps. the transition.

Yet, ambiguities in partner and staff The operational role of understanding of the concept may result UNICEF clusters and AORs in a significant divergence of expectations The team found that the perceived and may also challenge partnership ambiguity of IASC policy and an management. inconsistent country-driven, decentralized

or bottom-up approach has fuelled cluster UNICEF's ability to ensure that its CLA “scope creep”; this takes the form of an responsibilities will result in long-term, increasing number of roles, response to lasting, sectoral development and/or smaller and smaller emergencies, and humanitarian coordination and enhanced extended activation timelines. response capacity was found to be limited by a lack of clear guidance on transition This includes: and handover, decentralized authority and funding approaches, and a lack of clarity  growing use of the cluster system to on the role of clusters in national capacity- serve new purposes and fill gaps building. where long-term coordination systems do not exist, or are perceived to be Country-level clusters are taking different inadequate; approaches and positions regarding  activation in areas where there are no transition and handover, while attempts to large-scale emergency needs; hand over to sectoral or other  prolonged activation after emergency humanitarian coordination mechanisms coordination requirements have been and national authorities have had limited met; and success.  reluctance and inability to transition coordination responsibilities to governments and sectoral coordination mechanisms.

61 IASC, Operational Guidance on the Concept of ‘Provider of Last Resort’, June 2008.

The basic drivers of this extended role perceived influence over funding seem to be primarily linked to external decisions makes maintaining a neutral stakeholder interests and include: role more difficult. The administrative burden of these roles, particularly where a  a lack of standard or funded sectoral CHF is in place, was also found to take coordination mechanisms for time and resources away from operational development and protracted coordination and problem-solving. situations; In 2012, 34 UNICEF country offices  gaps in effective recovery reported cluster activity, of which 14 have coordination mechanisms; clear, ongoing relevance as active  donor support for a single clusters. Analysis based on the IASC coordination vehicle and unified ‘Reference Module for Cluster funding conduits; Coordination’ activation criteria of all  national governments’ expectations country offices reporting cluster activity in that an emergency designation linked 2012, suggests that 59 per cent of these to clusters will result in additional contexts no longer have emergency financial resources and support; coordination needs. Because the clusters  the realignment of national are being used to fill gaps in sectoral governments’ expectations during development coordination and to respond emergencies to expect international to emergencies that could be managed by cluster support, which makes it national coordination structures, there is a difficult to shift operating modes when broad risk of creating dependency and an emergency ends; and undermining national capacity.  the role of clusters in national capacity-building before, during and This trend is seen by some stakeholders after emergencies, which remains as a deliberate effort to ‘maximize’ unclear. clusters and by others as the result of an unregulated bottom-up approach. The evaluation found evidence in Nepal Regardless of its origins, the evaluation and the State of Palestine that prolonged found that the activation of clusters use of international clusters creates beyond emergency coordination dependency and undermines the significantly affects UNICEF’s global assumption of responsibility by nationally ability to perform well as a CLA in terms of led coordination mechanisms and national relevance, predictability, partnership, authorities (competition and replacement). efficiency and sustainability. Furthermore, Some key informants in South Sudan long-term development coordination for noted concerns that the presumed development work requires different permanence of clusters there would also coordination skills and approaches than enable the government to delay building those needed for emergency coordination. its own capacity for coordination. It was found that, where country offices Donor support for unified funding conduits have staff with the skills to meet these has influenced HCT and global long-term coordination needs, UNICEF is approaches for pooled funding, resulting not necessarily better equipped for actual in a growing administrative and quality emergency coordination, although the control role for clusters in vetting and assumption that an ongoing cluster is in preparing submissions for the CAP and place may lead others to believe that the pooled funding requests through CHFs. system is sufficiently prepared. This trend was found to put cluster Summary findings: Scope and coordinators in a delicate position, as their boundary issues

An inconsistent understanding and Clusters are increasingly filling gaps application of IASC policies and guidance beyond emergency coordination. within UNICEF and HCTs has led to an This cluster “scope creep” significantly increasing scope and scale of partner affects UNICEF’s global ability to perform expectations that are difficult to manage well as a CLA in terms of relevance, and support. While UNICEF is predictability, partnership, efficiency and increasingly working to improve clarity sustainability. Clusters are not designed to through the IASC at global level, no fulfil all of these roles over a prolonged UNICEF-wide efforts were found to have period of time, and doing so detracts from increased the consistent country office their focus on readiness for emergency interpretation and application of policies coordination. and guidance. The impact of clusters on long-term, lasting coordination and national response capacity is a lack of guidance on transition, decentralized authority and funding, and an unclear role for clusters in national capacity-building.

Figure 13: Current rationale for clusters at country level

3.5 Cost-effectiveness, value While these survey data, and for money and efficiency complementary interview data, are inherently perceptual, this is nonetheless Cost-effectiveness, value for money and a significant finding. efficiency were assessed on the basis of three key elements: Observation during the evaluation showed that, in most cases, UNICEF-led clusters 1. Cost-effectiveness and value for have a diverse and large membership, money and belief in value for money in a 2. Linkages with other internal and partnership is critical to sustain the external initiatives partnership. 3. Co-leadership arrangements In addition to the high overall average Cost-effectiveness and value for ratings of value for money, there is a money notable variance across evaluation case Overall, the evaluation found that UNICEF studies; partners rate value higher in has identified cost-effective means of places where the clusters are focused on implementing its CLA roles and that the emergency coordination. coordination services provided through its clusters and AORs are widely perceived UNICEF has also leveraged internal and to provide good value for money. external resources to fulfil its CLA roles effectively. Its clusters and AORs have In surveys, 83 per cent of UNICEF country successfully engaged partners in sharing staff and 84 per cent of country partners cluster costs at global level. stated that cluster outcomes justify their Approximately 54 per cent of staff and investment. However, a true measure of infrastructure costs, and 33 per cent of value for money would need to compare workplan activity costs, are being covered cost to the operational impact of by partners. With the creation of the coordination, which was beyond the scope GCCU in Geneva, UNICEF has also of this evaluation.62 mainstreamed the cost of cluster coordinator positions, demonstrating a financial commitment to its CLA role.

Table 5: Annual cost of UNICEF-led global clusters and AORs

CLA Annual Global Running Costs Total % UNICEF funded % Partner funded Staff costs for 3 clusters and 2 AORs $9,914,205 39% 61% (including RRTs) Other GCCU staff, equipment and $1,331,275 100% 0% infrastructure

Subtotal $11,245,480 46% 54% Global Cluster/AOR Workplan $6,627,385 67% 33% Activity Costs Grand total $17,872,865 54% 46%

62 The evaluators searched for cross-sectoral operational impact data from different stages of major operations to measure value-for-money measure. No sound sources that cover entire sectors of work across UNICEF-led clusters could be identified.

Basic global running costs per UNICEF establish where, when and how to use the cluster appear to be lower than those of cluster system were found to pose the other CLAs based on a 2012 analysis of greatest challenge to the financial cluster coordination costs by the Office for sustainability of the cluster system. The the Coordination of Humanitarian Affairs global costs and functions of the GCCU, (OCHA),63 although discrepancies in however, are largely perceived as different CLA’s cost frameworks make financially sustainable as long as partners comparison difficult. continue their significant support for RRT capacity and other initiatives. At country level, the 2012 EMOPS 64 survey estimated that UNICEF-led and Linkages with other internal and co-led clusters are costing approximately external initiatives US$35 million per year at country level, Cross-cluster coordination and integrated US$10 million of which is being covered programming approaches remain the by partners. While many UNICEF exception rather than the rule among headquarters staff perceive this total as UNICEF clusters and AORs, and with high, it averages only approximately US$7 those sectors or clusters led by other million per cluster and AOR, and covers a organizations. total of 34 countries – for less than US$1 million per year per country. Cluster workplans and response

strategies are generally siloed. There Based on the full cost of a UN appears to be little awareness of other international staff post, this equates to vertical interventions in emergency approximately four to five professional response that target the same affected staff per country. populations.

In general, the evaluation found that The evaluation of UNICEF’s 2011/2012 UNICEF’s investment in its CLA role at response to the Horn of Africa crisis global and field level is not excessive and noted: “There is potential for improving the is shared well by partners. effectiveness and value for money of

UNICEF emergency interventions by Up to 33 per cent of estimated field integrating services at the point of coordination costs were found to be in delivery, backed up by integrated support long-term sectoral development from regional and HQ levels, rather than coordination contexts rather than large- the vertical sector support that tends to scale emergency coordination, suggesting prevail currently. This implies a more some potential cost savings for UNICEF if cohesive approach to UNICEF and cluster IASC and UNICEF activation and support to local governments and deactivation clarity can be established, as communities.”65 stakeholders frequently noted that clusters are a “heavier” and more costly form of A few limited instances of integrated coordination. cross-sectoral programming approaches

were identified, including the Pakistan Unrealistic stakeholder expectations and a Inter-Cluster Child Survival Strategy, joint lack of consistent global efforts to sections of workplans and cross- references in contingency plans, but most

evidence suggests little change over time 63 OCHA, Cluster Coordination Costs: 2011 Requirements and Contributions, November 2012. 64 ‘A Survey of the Operational Status and Staffing of UNICEF Led and Co-Led Clusters and AORs in 65 UNICEF’s Response to the Emergency in the the Field’. Horn of Africa, 2011–2012.

41 to improve integrated response follow the same methodology as if approaches. they were (implementing) partners”

The focus group discussions brought Key informants, nonetheless, report some together partners and staff from different continued confusion about the roles clusters. Stakeholders noted that such between the Programme Division and meetings rarely take place and that such EMOPS following the consolidation of dialogue across clusters was considered cluster staff under EMOPS. This very useful. confusion exists about sectoral versus cluster coordination, and who is The IASC Transformative Agenda responsible for emergency preparedness. integrated-programme cycle, the multi- Staff and partners noted that UNICEF cluster initial rapid assessment tool and makes some effort to participate in inter- the inter-agency rapid response cluster coordination, but that the systems mechanism were developed, with are weak and reliant on individual UNICEF’s input, to address challenges relationships, both among UNICEF-led related to integrated cross-cluster clusters and across all clusters. programming approaches, but these have yet to be tested in an operation. Within UNICEF, country office practices vary for inter-cluster coordination. Within Most staff at global and country level some, an emergency coordination unit reported that clusters and programmes or plays a de facto role in convening cluster sectors interact without duplication, coordinators, although the emphasis though often due to double-hatting. seems to be on basic information-sharing Seventy-four per cent of UNICEF staff at and reporting deadlines (for UNICEF’s country level stated that clusters enhance Humanitarian Action for Children reports, programmes, while 11 per cent said they CHF, CAP, etc.). In others, this role falls have no effect on programmes, and 15 to representatives or deputy per cent said they detract from representatives, or is not systematically programmes. Reasons cited for the ensured. positive effect on programme implementation included: At global level, UNICEF increasingly demonstrates proactive efforts to  Clusters attract and help leverage contribute to the IASC Transformative additional resources Agenda and inter-cluster concepts and  Clusters serve as an entry point for ideas. Within the GCCU, however, inter- future programmes cluster coordination remains somewhat ad  Staffers view “clusters as one of the hoc due to weaknesses to date in the development interventions” and, cross-cluster management system. therefore, are not in competition  The “cluster coordination function Co-leadership arrangements enables better programme Co-leadership arrangements in the implementation of all partners, many Education Cluster were found to have of whom are UNICEF implementing improved, while agreements for co- partners” leadership of the Gender-based Violence  Clusters allow UNICEF “leverage on AOR offer potential for improvements. activities done by non-implementing partners as non-PCA (Programme At field level, co-leadership generally Cooperation Agreement) partners will works well and adds significant value. Co-leadership approaches increase

cluster credibility and help elicit permanent (and important) thematic contributions of information from partners. coordination mechanisms than many other clusters. Child protection and Key informants reported significant gender-based violence are cross-cutting improvements since the 2010 review in issues that pertain to emergency and non- the Education Cluster co-leadership with emergency environments (in contrast to Save the Children.66 While not all specific emergency education, emergency recommendations have been nutrition or emergency WASH). As such, implemented, the underlying and critical the characteristics of the two AORs before issues were found to have been and after an emergency phase of addressed. The Education Cluster was activation may differ less than those of also found to have strong links with the other clusters, although it is unclear if this International Network for Education in is understood within the broader Emergencies, which plays a significant Protection Cluster. normative guidance role in support of the cluster. Key informants at global and country levels reported that the United Nations Arrangements between the Protection Population Fund (UNFPA) co-leadership Cluster and the two UNICEF-led/co-led with UNICEF of the GBV AOR has been Protection AORs were found to be more challenging, but agreements now seem to problematic. There are differences be in place that should lead to UNFPA’s between AoRs and clusters in terms of taking a more active lead role at global how they work. On an official level, the level. Challenges remain in this various Protection Cluster AORs serve arrangement, however, as UNFPA’s field effectively as sub-clusters. In practice, the presence/capacity is reportedly weaker two AORs that are led or co-led by than UNICEF’s, and expectations of UNICEF have somewhat comparable UNICEF may remain high unless other systems and capacities – just like their potential co-leaders step forward. fully-fledged cluster counterparts – and their members expect the same support. Summary findings: Cost- AORs operate, therefore, under a more effectiveness, value for money and complex operating structure, having to efficiency maintain their own systems and UNICEF has found cost-effective means coordination practices, while also of implementing its CLA roles, including coordinating within the Protection Cluster. through leveraging significant partner contributions to cluster work. Staff and At global and country level, this can result partners strongly believe that the in AORs having less visibility and outcomes of cluster work justify their influence when relationships between organization’s investment. Although coordinators and agencies are weak. The significant, a true value-for-money two UNICEF-led or co-led AORs are measure would require cross-sectoral better capacitated, with full-time global impact data correlated to coordination. coordinators, than the overall Protection Global and country-level costs were not Cluster, further complicating this found to be excessive, but sustainability is relationship. threatened by unrealistic stakeholder expectations. The two UNICEF-led and co-led AORs also demonstrate more characteristics of Clusters, AORs and UNICEF programmes interact without duplication and have 66 ‘Review of the Education Cluster Co-Leadership neutral-to-positive effects on programmes. Arrangement’.

Cross-cluster coordination was found to the GBV AOR. Relationships between the be insufficient, while integrated Protection Cluster and the two related programming approaches remain the UNICEF AORs are more problematic as exception rather than the rule. member expectations of the AORs are the Co-leadership arrangements have same as for other clusters, yet they have improved within the Education Cluster, more complicated management and agreements between UNFPA and arrangements and less visibility and UNICEF should improve co-leadership of influence.

4 CONCLUSIONS

Based on the findings of the evaluation, system has employed clusters the team concluded that examining how (deliberately and through non-action), well UNICEF has carried out its CLA role UNICEF’s own lack of consistency in depends on two interconnected factors: advocating for cluster rationalization at country level is also a factor. 1. Is UNICEF implementing its CLA role well? (doing things right) External coordination performance 2. Is UNICEF doing the right things? UNICEF cluster coordination practices are (applying clusters appropriately, in the generally strong and enhance collective right places, at the right time, with the accountability and shared responsibility. right activities) Partner perceptions that cluster coordinators are not capable of identifying This section shows analysis based on the gaps and facilitating problem-solving five key areas and concludes with a suggest a weakness, but this is probably summary of the analysis. driven by the use of international clusters in non-emergency contexts. 4.1 Conclusions by key area Overall, the evaluation found evidence The partnership aspects of UNICEF-led that UNICEF has invested significantly in clusters seem to be sound at country implementing its CLA role since the level, but there are indications of partner beginning of the IASC cluster system and concerns about UNICEF partnership is increasingly implementing its CLA roles practices across many Global Clusters well. that need to be examined further. Recent global organizational changes to The evaluation concluded, however, that consolidate cluster staff in Geneva have UNICEF’s ability to carry out its CLA role improved the representation of clusters at well is limited because it is undertaking its global IASC meetings, but concerns about activities in more situations and over a country-level representation of members’ prolonged period of time above and interests suggests a lack of understanding beyond the role that activated clusters are by country representatives and a poor designed to play. This cluster “scope appreciation of the CLA role. creep” stretches resources by spreading efforts and resources over more countries; UNICEF's equal participation as a there are no clear, established priorities member of Global Clusters is rated lower that are based on risk; and this limits the by partners. Country-level clusters with ability of global clusters to provide high- double-hatting coordinators are also less quality support. likely to experience distinct UNICEF participation as a member. This cluster “scope creep” is driven by critical internal and external stakeholders, While partners believe that roles and including donors and national responsibilities are unclear at global level, governments, their competing interests, the evaluation team found that cluster and and contextual factors such as gaps in AOR coordinators are making significant non-cluster systems for preparedness and efforts to establish clear roles, and sectoral development coordination. While partners bear some of the responsibility a significant amount of cluster “scope for achieving such clarity. Country-level creep” is linked to how the overarching partners rate UNICEF better for adherence to principles of partnership

45 than global partners, although the CCCs, to some extent enhancing overall principle of transparency is often cited as accountability. However, monitoring a weakness at both levels. UNICEF systems do not assess quality or effectively engages national and, appropriateness, and reporting systems sometimes, subnational authorities in are voluntary for UNICEF country offices. cluster work, but this can pose challenges The evaluation can serve to inform a to independence that should be potential approach for future UNICEF CLA consistently monitored. performance measurement. A concept note on CLA performance measurement UNICEF-led clusters and AORs have had is included in Annex 6. positive effects on both geographic and thematic coverage. Its clusters and AORs UNICEF's CLA approach is increasingly frequently participate in joint well linked to other internal initiatives and assessments, but joint operational partners. Organizational initiatives, strategies remain rare. policies and procedures developed in the past few years have taken the CLA role While subnational mechanisms have been into account. put in place in many countries, they are limited due to resource constraints and With UNICEF as a Cluster Lead Agency, staff turnover. the evaluation team believes it is important that UNICEF leaders throughout UNICEF has improved accountability to the organization provide cohesive and affected populations by increasing gender coherent leadership. However, the mainstreaming and human rights potentially critical role of UNICEF regional approaches in its cluster work. offices in supporting country clusters and ensuring consistency of CLA approaches Nevertheless, limitations in information remains undefined. At senior management and reporting capacity, as management level, the evaluation found well as in national government capacities, that only the Deputy Executive Director have somewhat limited the ability to with responsibility for emergency demonstrate cluster accountability to operations is regularly engaged in CLA affected populations. issues, while no evidence of a CLA strategic leadership group was found. Internal CLA performance The consolidation of clusters and AORs Confusion remains over sector versus under the GCCU in EMOPS is beginning cluster coordination roles at country level, to pay dividends, and may create and organizational knowledge economies of scale and consistency of management systems are weak and too good practice. often reliant on individuals to systematically support the CLA role. UNICEF's comparative advantages in programming and coordination mostly match its designated CLA roles, but questions raised by Global Cluster members about UNICEF’s comparative advantages need further examination.

UNICEF-led clusters and AORs have broadly contributed to meeting the coordination objectives noted in the

Figure 14: Current active organizational components of UNICEF CLA leadership

Human resources performance UNICEF's human resources and surge High-quality cluster coordination depends rosters to support CLA roles have on the technical and coordination significantly improved since 2010, but strengths of cluster coordinators. The efforts remain fragmented and there is no capacity development of cluster overall strategy. Surge capacity systems coordinators has been limited to basic remain challenged by difficulties in awareness training, with no investment in predicting and meeting demand when mentoring, e-learning or simulations. cluster activation and deactivation Based on best-practice protocols are inconsistently applied.

47 research,67 the elements of a Scope and boundary issues comprehensive cluster capacity UNICEF is broadly aligned with IASC development system are shown in Figure policy and guidance but ambiguities in 16. A concept note on capacity IASC documents and inconsistency in enhancement is included in their application have allowed the cluster Annex 7. system to drift in many directions. UNICEF’s organization-wide policies and There is no coordination career path guidance to operationalize the CLA role within UNICEF, and cluster coordination is are missing, leaving important decisions seen as a temporary function. As a result, about interpretation up to country offices there is little incentive in the current and individual cluster coordinators. As a system for UNICEF career staff to result, up to 59 per cent of country offices consider serving in cluster coordination reporting cluster activity appear to be roles. operating with a questionable activation rationale and are probably using the term The quality and coherence of UNICEF’s ‘cluster’ to mean sectoral coordination for implementation of its overall CLA roles more development-oriented work. depends on the understanding, skills and approaches of UNICEF country The concept of ‘provider of last resort’ has representatives and deputies. No become an increasingly irrelevant systematic investment has been made in accountability concept as understanding developing such capacity. varies widely among internal and external Figure 15: Elements of comprehensive stakeholders, while caveats related to capacity development access, security and funding provide an escape clause applicable in many of the most critical humanitarian situations where gaps and bottlenecks limit assistance. The variance in partner understanding can complicate the management of partner expectations.

While UNICEF is not the final responsible actor for these overarching cluster system issues, given UNICEF’s responsibilities for five clusters and AORs, it does have the responsibility to influence IASC efforts to resolve them. The evaluation team believes UNICEF has an opportunity to develop clear organizational practices that operationalize IASC policy in a way that would lead to greater consistency and help country offices interpret ambiguous system issues.

At global level, UNICEF has increased its contributions to IASC initiatives to clarify and strengthen cluster approaches. Until 67 Capacity development systems in place in the UNICEF clarifies its institutional positions United Nations Food and Agriculture Organization on the appropriate use of the cluster FAO and the United Nations Development Programme (UNDP) were reviewed along with system at country level, it will not be able research from People in Aid.

to ensure consistency and stakeholder led clusters have been sustained with expectations through advocacy at HCTs. relatively modest investment, and UNICEF has leveraged partner The evaluation team does not believe that contributions to support key capacities cluster “scope creep” is simply an issue of and initiatives. terminology; using an international cluster tool for long-term (mostly development) Cost-effectiveness is called into question, coordination and in situations where it is however, due to questionable activation not needed creates demand for support at relevance in 59 per cent of countries global level that is difficult to meet. It also reporting UNICEF-led cluster activity. This further confuses actors on what the shows a lack of central direction and clusters are meant to do, and raises control over costs, and it suggests that myriad expectations about what UNICEF significant costs are actually directed is committed to provide to partners. towards ongoing sectoral development coordination needs rather than emergency While coordination is necessary before, coordination. during and after emergencies – and UNICEF has a responsibility to help Inter-cluster coordination remains weak develop and sometimes facilitate sectoral and ad hoc among UNICEF-led clusters development and other humanitarian and across clusters led by other coordination – it is not bound to do so in organizations. There is still little the same way as when clusters are awareness and understanding of the activated because of its CLA Transformative Agenda at country level, commitments. Criteria for cluster although elements of these reforms are activation and deactivation, under designed to improve inter-cluster refinement by the IASC, should lead to coordination. less cluster “scope creep”, assuming that CLAs apply practices and criteria UNICEF-led clusters are often connected consistently. to line ministries and sometimes to disaster management agencies, but Cost-effectiveness, value for money linkages to disaster risk reduction and and efficiency preparedness actors remain unclear and guidance does not exist. Co-leadership Internal and external stakeholders arrangements have improved and offer strongly agreed that their investment is advantages in managing the partnership justified by the coordination outcomes of aspects of clusters and AORs. UNICEF-led clusters and AORs. UNICEF has increasingly harnessed the resources at its disposal to fulfil its CLA responsibilities. At global level, UNICEF-

5 RECOMMENDATIONS

In a system as decentralized and with as the “unbalanced force” to effect change many diverse stakeholders as the cluster within the IASC and HCTs at country system, inertia and fragmentation can be level. major challenges. The following recommendations have “An object in motion stays in motion with been developed by the evaluation team the same speed and in the same direction based on careful consideration of the unless acted upon by an unbalanced evaluation findings and additional force.” stakeholder consultations on potential Newton’s First Law of Motion areas for improvement.

There are some positive trends in The categories for each recommendation UNICEF performance as a CLA, and the complement the themes covered in the efforts behind these areas of improving findings and conclusions sections of this performance should be continued. In the report, and follow the same order, as the areas where UNICEF is doing less well as evaluation team believes they are all of a CLA additional efforts are required. equal importance. Some are fully within UNICEF’s control, while others require UNICEF to serve as

Recommendation 1: External rapidly, and improve the quality of coordination performance coordination staff.

Develop a ‘cluster-ready’ initiative to Recommendation 4: Scope increase country office preparedness for cluster activation in high-risk countries. and boundary issues Increase coherence (interpretation and This recommendation is meant to articulation) and then fidelity increase understanding of country offices, (understanding and consistent application) improve contingency planning with through UNICEF CLA policy and practice. partners in high-risk countries and better enable the GCCU to predict surge This recommendation is meant to focus capacity demands. GCCU operational support on the most relevant emergency situations and help manage stakeholder expectations. Recommendation 2: Internal CLA performance Recommendation 5: Strengthen UNICEF-wide management systems to support the CLA role, including Cost-effectiveness, value for strengthening the role of regional offices money and efficiency and better connecting country Mitigate the use of clusters in representatives to a global CLA strategic inappropriate scenarios by developing management structure. models and tools for non-cluster coordination, including transition points for This recommendation is meant to country offices, and establish clarity on increase coherence and consistency, the role of clusters, if any, for national improve performance management and capacity-building to ensure efficient and monitoring and improve cross-divisional fit-for-purpose coordination approaches. coordination support. This recommendation is meant to help Recommendation 3: Human maintain partner satisfaction with UNICEF resources performance coordination leadership, improve transition to and from sectoral development and Develop an integrated strategy for human humanitarian coordination mechanisms resources surge capacity and UNICEF and help avoid duplication with other coordination staff development. disaster preparedness and risk reduction actors. This recommendation is meant to improve targeted training of coordinators, further increase the ability to deploy the right staff

6 KEY SUCCESS FACTORS

UNICEF has the resources and capacity  inertia caused partly by a lack of a to be a coherent, relevant and fully sense of urgency effective CLA across its five clusters and AORs if the organization fully embraces The following six actions can be taken to and implements the recommendations in support successful implementation of the chapter 5. evaluation recommendations:

Some of the evaluation findings and Senior leadership across UNICEF recommendations are similar to those should be completely engaged. found in the following documents: This includes fully briefing the headquarters and regional leadership  Cluster Approach Evaluation 2 teams and involving them in the Synthesis Report, GPPi and Groupe process of developing a management URD, 2010. response. As explained in the findings  Synthesis of Cluster Lessons from and recommendations, the lack of an UNICEF’s response to the Pakistan integrated leadership approach across Floods, 2011. levels of the organization risks  Implementing the Cluster Approach: A perpetuating disconnects in synthesis of documented lessons accountability, with negative effects on 2006–2010, Programme Division, relevance, effectiveness, coherence UNICEF, 2011. and, potentially, sustainability.  ‘Independent Review of UNICEF’s Operational Response to the January 2010 Earthquake in Haiti’, Evaluation  Implement recommendations as Office, UNICEF, 2011. a package.  UNICEF’s Response to the Implementing one or just a few of the Emergency in the Horn of Africa, recommendations while ignoring 2011–2012: Lesson-learning exercise, others is unlikely to significantly affect Evaluation Office, UNICEF, 2012. UNICEF’s CLA performance. The areas for improvement are interlinked, The reason these key recommendations and the underlying performance have only been partly implemented factors influence other areas of appears to be related to: performance. For example, leadership without coherence results in continued  the absence of an overall responsible confusion and inconsistent practice. entity Coherence without leadership has the  limited senior management ownership same incomplete results. and follow-through  recommendations not being translated  Partners should be engaged in into clear and accountable plans for developing strategies for the execution future.  challenges in effecting change in the The most active members of each broader IASC system Global Cluster, and the most frequent  difficulty in balancing decentralized contributing organizations at country authority with required global level, should be engaged in consistency developing future strategies based on

the evaluation results. As the the cluster system, both at global and evaluation illustrates, however, individual country levels. UNICEF stakeholder expectations vary widely, should organize discussions with key and UNICEF has the responsibility to donors to present the findings and use this as an opportunity to lead in recommendations of the evaluation, shaping partner perspectives. This and to discuss different concepts and begins partly by determining how to means of addressing funding articulate what UNICEF will and will approaches that help ensure that not do as a CLA in the future. these approaches reinforce the IASC principals’ agreement on cluster roles  UNICEF should engage the IASC and relevance, while identifying ways principals and subsidiary bodies to support coordination needs beyond the scope of cluster work. in discussions on the evaluation findings and recommendations,  UNICEF should engage national as well as their implications. governments before and at the As the overall architects of the cluster beginning of cluster operations system, IASC principals and emergency directors should be briefed to establish and agree clear and engaged by UNICEF leadership in scope and transition points. discussions around the evaluation Discussions with national findings and recommendations. In governments should ensure clarity on some cases, it will strengthen the character and expected lifespan of UNICEF’s advocacy if it first acts to cluster work and how transition will create coherence across UNICEF-led take place. Although the timing and clusters and AORs. Also, it may help approach may differ by country lay the groundwork for such future context, national governments play an advocacy by briefing and engaging essential role in shaping expectations these groups early and often. and influencing potential transition strategies. Discussions with national  UNICEF should engage donor governments should consistently build organizations to discuss how appreciation for clusters as a temporary construct, ideally with their influence shapes the shared responsibility between the CLA patterns and findings of the and government, and planned evaluation. transition to full national government Donor policies and practices have ownership over time. significant bearing on the direction of

ANNEXES

The Evaluation Office has been supplied with the following appendices that contain supporting information for the report. These are:

1. Terms of reference 2. Evaluation methodology description 3. Selecting country case studies: Technical note 4. List of documentary data sources 5. List of stakeholders consulted 6. CLA performance measurement: Concept note 7. Cluster coordination capacity development: Concept note