Lessons from an Evaluation of the Emergency Capacity Building (ECB) Project September 2014
Total Page:16
File Type:pdf, Size:1020Kb
A DECADE OF LEARNING Lessons from an Evaluation of the Emergency Capacity Building (ECB) Project September 2014 CONTENTS ACKNOWLEDGMENTS III 3.2.2. Capacity Building Approaches Prioritized ABBREVIATIONS AND ACRONYMS V by the ECB Project 23 EXECUTIVE SUMMARY 1 3.2.3. Alignment of ECB and HEA Community 1. INTRODUCTION 5 Capacity Building Approaches 24 1.1. Challenges and Trends in the HEA Sector 5 3.2.4. Approach Prioritization and Capacity Building Success 24 1.2. The Emergency Capacity Building Project 5 3.3. Whose Capacity? Stakeholder Groups Targeted by the ECB Project 38 2. EVALUATION METHODOLOGY 7 3.3.1. Stakeholder Groups and System Levels 2.1. Conceptual Frame work 7 Targeted by the HEA Community 38 2.2. Evaluation Questions 7 3.3.2. Stakeholder Groups and System Levels 2.3. Data Collection 7 Prioritized by the ECB Project 41 2.3.1. Desk Review Analysis 7 3.3.3. Alignment of ECB and Broader HEA Priorities 41 2.3.2. HEA Stakeholder Survey 8 3.3.4. Relevance of ECB Project and HEA Community Priorities 41 2.3.3. Materials Use Survey 8 3.3.5. Stakeholder Targeting and Capacity Building Success 41 2.3.4. Key Informant Interviews 9 3.3.6. Stakeholder Targeting and Sustainability 42 2.3.5. Constraints and Limitations 9 3.4. What Worked and Didn’t Work and Why 43 3. FINDINGS 11 3.4.1. ECB Project Governance 43 3.1. What Competencies Were Prioritized? 11 3.4.2. The Consortium Model 43 3.1.1. Capacity Building Competencies 3.4.3. Continuation of Activities at Country Level 44 Prioritized by the HEA Community 11 4. CONCLUDING REMARKS 47 3.1.2. Competencies Prioritized by the ECB Project 13 4.1. WHAT Emergency Response Competencies 3.1.3. Alignment of ECB Project and HEA Community Priorities 13 Did the ECB Project Build? 47 3.1.4. Critical but Neglected Competencies 15 4.2. HOW Did the ECB Project Build Capacities 3.1.5. Competency Prioritization and Capacity Building Success 15 for Emergency Response? 47 3.2. How Did the ECB Project Approach Capacity Building? 22 4.3. WHO Did the ECB Project Target for Capacity Building? 49 3.2.1. Capacity Building Approaches Prioritized 4.4. Looking to the Future 49 by the HEA Community 22 REFERENCES 51 I 4 Acknowledgments This study would not have been possible without the support and involvement of various ECB Project stakeholders, beneficiaries and partners at national, regional and global levels who acted as respondents and facilitated the study to successful completion. Many thanks to everyone who provided feedback. Their positive re- sponse ensured that this study reflects a wide variety of perspectives. The evaluation team is grateful to the Bill & Melinda Gates Foundation, in partic- ular, Ms. Valerie, Bemo, Ms. Amanda Lanzarone and Ms. Daya Levin, as well as the ECB Project secretariat for their support and constructive feedback in executing this study. This report does not necessarily reflect the views of the commissioning Bill & Melin- da Gates Foundation and has not been edited to its official publication standards. Feedback or questions should be addressed to Ky Luu at [email protected]. Disaster Resilience Group Evaluation Team members Ky Luu, JD: Team Leader Apollo M. Nkwake, PhD, CE: Lead Evaluator Margie Ferris Morris, MSc: Evaluator Steven Hansch, MA: Evaluator Matesa Gago, MPH: Coordinator Eric Corzine, MBA: Evaluator Justin Marcello, MA: Reviewer Graduate Research Assistants Tim White, MA Bonnie Haskell Editor Wendy Hammond III Abbreviations ACAPS Assessment Capacities Project PAHO Pan American Health Organization AIM Accountability and Impact Measurement PDRA Participatory Disaster Risk Assessment ALNAP Active Learning Network for Accountability and PROCOSI Programa de Coordinación en Salud Integral (Integral Performance in Humanitarian Action Health Coordination Program), a Bolivian network of BMGF Bill & Melinda Gates Foundation non- profit organizations CBHA Consortium of British Humanitarian Associations SCF Save the Children Federation CDMP Comprehensive Disaster Management Project, Bangladesh UN United Nations CEP R&R Country Engagement Plans Review and Revision UNDP United Nations Development Program CLR Consortium Learning Review UNOCHA UN Office for the Coordination of Humanitarian Affairs CRS Catholic Relief Services USAID United States Agency for International Development CSAS Consortia Self- Assessment Study VIDECI Vice Minister of Civil Defense, Bolivia DFID UK Department for International Development (UK aid) WASH Water, sanitation and hygiene DRG Disaster Resilience Group WFP World Food Program DRM Disaster risk management WHO World Health Organization DRR Disaster risk reduction WVI World Vision International ECB Emergency Capacity Building Project ECHO European Commission Humanitarian Organization ENHAnce Expanding National Humanitarian Ability training program Other Terms GHP Global Humanitarian Platform GEG Impact Measurement and Accountability in Emergencies: CARE International The Good Enough Guide International humanitarian and development organization and ECB Project lead HAP Humanitarian Accountability Project Caritas Internationalis HEA Humanitarian emergency assistance Confederation of 164 Catholic relief, development and social service organizations HPN Humanitarian Practice Network, Overseas Development Institute IFRC International Federation of the Red Cross and Red Crescent Societies ECBinter- ‐active IASC Inter- Agency Standing Committee Regional Learning Conferences hosted by the ECB Project in Bangladesh, Bolivia, ICT Information and communication technology Horn of Africa, Indonesia and Niger for project agencies to share collaborative work INGO International non-governmental organization and training experience, focusing on the project’s core themes of accountability IRC International Rescue Committee, an IWG member and impact measurement, disaster risk reduction and national staff development ISDR UN Office for Disaster Risk Reduction IWG Inter- Agency Working Group (ECB agencies plus IRC) Nethope JNA Joint needs assessment New- generation information technology collaboration of 33 international NGOs KIRA Kenya Initial Rapid Assessment and ECB collaborator LNGO Local non- governmental organization People in Aid MCI Mercy Corps International Membership organization that improves organizational effectiveness in the hu- M&E Monitoring and evaluation manitarian and development sectors M&L Management and leadership (core skills for ENHAnce) NGO Non- governmental organization Sphere Project NSDP National Staff Development Program Voluntary initiative of humanitarian response practitioners to improve the quality ODI Overseas Development Institute of humanitarian assistance, author of the Sphere Handbook, Humanitarian Charter OFDA USAID Office of Foreign Disaster Assistance and Minimum Standards in Humanitarian Response V Executive Summary Over the past decade, the global humanitarian agenda has moved from saving learning on accreditation systems, best practices, staff retention and surge lives and providing basic services in emergencies toward building resilience to cri- capacity with humanitarian networks, platforms and fora. The National Staff ses. In 2005 the United Nations launched a process to improve the effectiveness Development Program (NSDP) and other training initiatives were created to of the humanitarian response. As a result, in 2011, the United Nations (UN) Inter- increase national agency staff capacity to lead and manage emergency pro- Agency Standing Committee (IASC) agreed on a set of actions focusing on stron- grams. Evaluation respondents rated the quality of the materials and training ger leadership, more effective coordination and improved accountability. This highly and reported remarkable improvement in staff and member agency transformation requires new competencies and approaches to capacity building. surge capacity and staff confidence. In 2003, seven relief and development organizations conceived a project to Although the ECB Project did not articulate partnerships/collaboration/coor- strengthen the capacity of the humanitarian and emergency assistance (HEA) dination as a theory of change for capacity building, it drew on the strengths community to deliver humanitarian aid. The resulting Emergency Capacity Build- of global humanitarian organization partnerships and collaborated with local ing (ECB) Project aimed to improve the quality and effectiveness of emergency agencies, governments, international organizations and the UN. Local partner- preparedness and response by improving field-level capacity, collaborating with ships were a cornerstone of the project in Phase II to build the capacity of other partners and organizations and enhancing resources for field set- up, com- national governments and local agencies to respond to emergencies. The trust munication and training. These initiatives focused on the themes of (1) staff ca- and respect the consortia built in each country made them effective agents pacity building, (2) accountability and impact measurement and (3) disaster risk for change, accomplishing more than any one agency could do on its own. reduction and climate change adaptation. In Phase I, the project conducted re- search and produced field tools and guidelines. In Phase II, interventions were im- Disaster risk reduction (DRR) was integrated into project proposals in Phase I. plemented through consortia in Bolivia, Bangladesh, the Horn of Africa, Indonesia In Phase II, DRR training and joint needs assessments brought groups together and Niger. to address a common problem, strengthening relationships