ACKNOWLEDGEMENT
This report is dedicated to all conference participants from outside the Emergency Capacity Building Project who took the time to read the papers and travel so far to share their knowledge and perspectives on accountability to disaster-affected populations and impact measurement. We thank you.
ACRONYMS
ALNAP Active Learning Network for Accountability and Performance in Humanitarian Action
CEO Chief Executive Officer
DFID UK Department for International Development
ECB Emergency Capacity Building Project
ECHO European Commission Humanitarian Aid Department
HAP-I Humanitarian Accountability Partnership International
IDP Internally Displaced Person
INGO International Non-Governmental Organization
IWG Interagency Working Group on Emergency Capacity
NGO Non-Governmental Organization
OCHA UN Office for the Coordination of Humanitarian Affairs
URD Groupe Urgence, Réhabilitation, Développement
ii Contents
Acknowledgement ...... ii Acronyms ...... ii EXECUTIVE SUMMARY ...... 2 SECTION I: INTRODUCTION ...... 4 About the Conference ...... 4 The Purpose and Layout of the Report ...... 4 SECTION II: RECOMMENDATIONS FROM PARTICIPANTS ...... 5 SECTION III: BACKGROUND ON THE ECB ...... 7 Q&A: About the Emergency Capacity Building Project ...... 7 How Seven Agencies Tackled Accountability and Impact Measurement ...... 8 SECTION IV: PERSPECTIVES FROM THE SPEAKERS ...... 12 Liberating the Humanitarian Spirit, Mukesh Kapila ...... 12 Accountability: Self Assessments, Validation by Others, Brendan Gormley ...... 16 Impact Measurement: Impact Assessment is an Intervention, Mary Anderson ...... 18 Coordination: A Logical Step in the Process, Mamadou Ndiaye ...... 20 Collaboration: Lessons from the Private Sector, Gib Bulloch ...... 22 Sit Down with the Person who has Suffered, Nelly Badaru ...... 24 The Right Relationships, Hugo Slim ...... 25 Questions to Speakers ...... 27 SECTION V: GROUP DISCUSSSIONS AND RECOMMENDATIONS ...... 29 Accountability ...... 29 Impact Assessment ...... 31 Collaboration and Coordination ...... 33 Topics from “Open Space” Discussions ...... 36 Helping Beneficiaries to Hold Us Accountable ...... 36 Upward Accountability ...... 37 Integration of Different Sets of Standards ...... 37 Certification and Accreditation ...... 38 Field Level Capacity ...... 38 Collaboration ...... 39 Learning Beyond the ECB Agencies ...... 39 SECTION VI: CONCLUSIONS ...... 40 ANNEXES ...... 41 Annex 1: Agenda ...... 41 Annex 2: Participants’ Evaluation of the Conference ...... 42 Annex 3: List of Participants ...... 44 Annex 4: The Basic Elements of Accountability and Impact Measurement ...... 45
1 EXECUTIVE SUMMARY
INTRODUCTION
In December 2006, the seven member agencies of the Emergency Capacity Building Project (ECB) hosted a workshop called Translating Standards into Practice: NGO Accountability and Impact Measurement in Emergencies. The seven agencies had worked together since 2005 to improve their practice of accountability (to disaster-affected populations) and impact measurement and wanted to hear the views of experienced actors in the humanitarian sector on how they could do this work better. The specific objectives of the conference were to share lessons and challenges and develop recommendations for action, primarily by the ECB member agencies, within the thematic areas of accountability, impact measurement, collaboration, and coordination. The conference was for a day and a half and involved seventy-eight participants including donors, U.N representatives, international and local NGOs, and humanitarian network representatives. It began with presentations and continued on the second day with workshop debates to formulate and prioritize recommendations. The purpose of this report is to record the presentations and the rich discussions of the conference so they can be used for advocating for better NGO accountability to disaster-affected populations and impact measurement practice in emergencies.
What is ECB?
The Emergency Capacity Building (ECB) Project is a collaborative effort of the seven agencies of the Inter-agency Working Group on Emergency Capacity: CARE International, Catholic Relief Services, the International Rescue Committee, Mercy Corps, Oxfam GB, Save the Children, and World Vision International. Over a two-year period from March 2005, these agencies and their partners are jointly addressing issues of staff capacity, accountability and impact measurement, risk reduction, and the use of information and communication technologies (ICTs) in emergencies, with the objective of improving the speed, quality and effectiveness of emergency response. For further information, please visit www.ecbproject.org or email [email protected].
RECOMMENDATIONS
The top three recommendations developed and prioritized by conference participants were:
N Prioritize systematic listening to disaster-affected people as the key way to assess impact and develop staff competencies and methods to do so.
N Build a culture of accountability to meeting the needs of disaster-affected populations as expressed by those populations, from the field all the way up to senior management, and provide resources accordingly.
N Undertake a formal process led by CEOs to consider practical steps for greater integration of existing quality and accountability initiatives and their standards in light of the costs of the current fragmentation.
Other recommendations from participants are in Section V of the report.
DISCUSSIONS
There were many issues discussed at the conference. All concerned how NGOs could deliver better services to disaster- affected populations. Discussion topics related to the three prioritized recommendations and to the main conference themes are summarized below.
ACCOUNTABILITY A culture of accountability: Accountability involves judging individuals and agencies on the appropriateness of their actions for meeting the needs of disaster-affected populations. How can agencies move away from a preoccupation with spending money allocated for emergencies, bidding for funding for interventions outside of their expertise, jockeying for visibility, etc., to an overriding focus on whether their responses are meeting the needs of disaster-affected people in the most effective way? This will require a change in the way agency staff think. In short, it will require the creation of a culture of accountability that recognizes the importance of developing relationships with disaster-affected populations that promote respect, uphold dignity and share power.
2 Executive Summary A common framework for accountability: There are several quality and accountability initiatives, each with its own standards for accountability. The humanitarian sector needs better integration of these initiatives and standards. This will re - quire leadership from NGO senior executives. The Interagency Working Group (IWG) on Emergency Capacity could come up with recommendations as to which standards should be universally adopted. (Note: ECB has started this work through creation of “The Basic Elements of Accountability and Impact Measurement. See Annex 4).
Upward accountability: NGOs are not sufficiently accountable to their donors. Their task is complicated by too many different demands and requirements from donors. Additionally, current requirements from donors do not encourage NGOs to measure impact.
Certification and accreditation: There is growing pressure for certification within the humanitarian sector. Obstacles to certification include inadequate mechanisms to measure performance of organizations and lack of agreement of what standards to use. However, certification is not a threat and may not be as difficult as we think. Much progress has already been made.
IMPACT MEASUREMENT Systematic listening: NGOs need to be more purposeful in how they collect and use information that measures impact. If we first answer the question “Who will use the data and how?” this will then guide us in what we collect, how we collect it, and who we involve in this process. We must also demonstrate our commitment to measuring impact by integrating it into our performance management systems. We need to be accountable for what we do with the information provided us. Most of all, we need to be truly listening to the people we claim to be helping and doing so systematically.
Mapping and building staff knowledge and skills to measure impact: Our staff do not have all the skills and knowledge needed to measure the impact of our programs. At the same time, we are not sure how much knowledge and capacity already exists. We must be more purposeful about providing training along with other ways of addressing these capacity gaps, setting up knowledge management systems, and learning from the experiences and methodologies of those who measure impact within the humanitarian sector and other sectors.
COLLABORATION AND COORDINATION Collaboration and competition can exist together if we have an understanding of when and how each is needed. Because we are investing so much of our resources and expectations into collaboration, it would be helpful to have a cost-benefit analysis and evaluation of coordination and collaboration efforts. We need to understand how the end products and the process itself are beneficial. As we seek to understand the nature and practice of collaboration, we must build on past experiences and develop models for collaboration and coordination that work both at headquarters and at field levels. We must build the skills of our staff in collaboration and coordination and urge donors to support collaborative processes. There are several lessons from collaboration during the first phase of the ECB project that can be applied to the second phase. For example, we learned that collaboration models that work well at headquarters do not necessarily work well in the field. The collaboration model used by the ECB project has helped the seven agencies better understand the practice of accountability and impact measurement.
CONCLUSIONS
Although they are doing much good work, the quality of services NGOs provide to disaster-affected populations needs to improve. Both organizations and the individuals that constitute them have a responsibility to turn standards into practice in the field.
Executive Summary 3 SECTION I: INTRODUCTION
ABOUT THE CONFERENCE
In December 2006, the seven member agencies of the Emergency Capacity Building Project 1 (ECB) hosted a workshop called Translating Standards into Practice: NGO Accountability and Impact Measurement in Emergencies. The objectives were to share lessons and challenges and come up with recommendations for action, primarily for ECB member agencies, within the following thematic areas: accountability (to disaster-affected populations), impact measurement, collaboration, and coordination. The conference was for a day and a half and involved seventy-eight participants including donors, U.N representatives, international and local NGOs and humanitarian network representatives. It began with presentations on day one and continued with workshop debates to formulate and prioritize recommendations on day two. The conference was planned as a catalyst for action for two main reasons: First, the ECB group designing the conference realized that though the seven agencies had achieved some success in testing and advancing good practices in impact measurement and accountability in emergencies, these isolated instances needed to be applied widely across their organizations. This would require not only the commitment and action of their staff in the field, but also that of managers and leaders throughout their organizations, all the way up to their CEOs. For this, they needed the knowledge and experience of a diverse group of practitioners working in the humanitarian sector to help them identify some of the next steps that the ECB agencies and their partners could take to ensure good practice is applied. Second, there are external pressures forcing agencies to improve their humanitarian action. Emergencies continue to increase in number and scale, causing profound political, social, and environmental impacts. NGOs are becoming more significant players in the humanitarian landscape, commanding a third of financial resources for the Tsunami response. With this has come increased responsibility and raised expectations for professionalism and certification against a universally agreed set of standards. The conference was well attended by representatives of the seven agencies, as well as donors, U.N. representatives, international and local NGOs, and representatives of networks deeply engaged in the humanitarian sector. Participants were purposely invited for their known commitment to quality humanitarian action and experience of working in the humanitarian sector. A full list of participants is in Annex 3. Proceedings began with a series of presentations from experienced humanitarians: Mukesh Kapila, Mary Anderson, Mamadou Ndiaye, Gib Bulloch, Brendan Gormley, and later Hugo Slim, who had all been asked to address specific conference topics from their perspective. There was also a presentation by Charlie Ehle on ECB’s work on accountability and impact measurement. A panel discussion chaired by Denis Caillaux followed, with responses from the speakers as well as from Nelly Badaru, who had at one time been a refugee in the Congo. Throughout the conference, Nelly’s responses helped keep discussions grounded in the realities faced by disaster-affected people. During the morning of the second day, a series of parallel workshops addressed the conference issues of accountability, impact measurement, coordination, and collaboration. Workshop groups were asked to produce actionable recommendations for use by ECB agencies, which all participants then voted on. In the afternoon, organizers set up a series of parallel discussion groups on other outstanding key issues participants wanted to address. Topics included upward accountability, helping beneficiaries to hold us accountable, certification and accreditation, field level capacity, collaboration, and learning out.
THE PURPOSE AND LAYOUT OF THE REPORT
The purpose of this report is to record the presentations and the rich discussions of the conference so they can be used for advocating for better NGO accountability to disaster-affected populations and impact measurement practice in emergencies. Main conference messages have been highlighted and some contextual background and interpretation of discussions is provided. This is to facilitate use of the report by participants for this advocacy purpose. Section I introduces the conference, describing the reasons why it was held and the processes followed; Section II sets out the top three recommendations from conference participants; Section III describes ECB and its work on accountability to disaster- affected populations and impact measurement; Section IV captures the speaker presentations; Section V summarizes the small group discussions and recommendations; and Section VI sets out conclusions.
1 The Emergency Capacity Building (ECB) Project is a collaborative effort of the seven agencies of the Inter-agency Working Group on Emergency Capacity: CARE International, Catholic Relief Services, the International Rescue Committee, Mercy Corps, Oxfam GB, Save the Children, and World Vision International. Over a two-year period from March 2005, these agencies and their partners are jointly addressing issues of staff capacity, accountability and impact measurement, risk reduction and the use of information and communication technologies (ICTs) in emergencies, with the objective of improving the speed, quality and effectiveness of emergency response. For further information, please visit www.ecbproject.org or email [email protected].
4 SECTION I: INTRODUCTION SECTION II: RECOMMENDATIONS FROM PARTICIPANTS
THE TOP THREE RECOMMENDATIONS
Below we list the top three recommendations produced by conference participants for consideration and action by the ECB agencies. These recommendations have implications for field staff, managers, and leaders throughout each organization. They are listed in order of the number of votes given by participants. For the discussions related to these recommendations, please see Section V of the report.
RECOMMENDATION 1: SYSTEMATIC LISTENING (See Impact Measurement, page 31)
Because: N Agencies have improved on delivering and monitoring outputs but have not listened enough to people’s views on impact; N Humanitarian action has impacts in multiple dimensions; and N People we intend to help know which dimensions are most important to them,
we recommend that:
Agencies prioritize systematic listening to disaster-affected people as the key way to assess impact and develop staff competencies and methods to do so.
RECOMMENDATION 2: CULTURE OF ACCOUNTABILITY (See Accountability, page 30)
Build a culture of accountability to meeting the needs of disaster-affected populations as expressed by those populations, from the field all the way up to senior management, and provide resources accordingly.
This could include training of field staff, developing systems that support accountability, incorporating accountability requirements into performance reviews, changing the language that senior management uses, and changing donor expectations.
RECOMMENDATION 3: INTEGRATING QUALITY AND ACCOUNTABILITY STANDARDS (See Accountability, page 30)
Undertake a formal process led by CEOs to consider practical steps for greater integration of existing quality and accountability initiatives and their standards in light of the costs of the current fragmentation
Other recommendations produced by participants are in Section V of the report, along with discussion notes from each of the parallel workshops.
SECTION II: RECOMMENDATIONS FROM PARTICIPANTS 5 REFLECTIONS FROM A SENIOR LEADER Adapted from closing remarks by Carolyn Miles, Chief Operating Officer, Save the Children US
Someone in my position spends a lot of time thinking about the topics we discussed here. I personally think they are also things that our sector needs to get a lot better at. Particularly on accountability and impact measurement, there is going to be a lot of pressure on us all to improve; at this conference, we have talked about getting ahead of that pressure and driving these things ourselves before they are forced upon us.
These are some of the things I heard during this event:
Systematic listening: This idea of basing our accountability on listening to the people we are serving (Recommendation 1) is absolutely different from the way most of us organize our accountability measures, if we have them, and I think it’s absolutely the right thing to do.
External pressure: The trends are all there—there will be more emergencies, more people affected, more funding, and more pressure on us all to perform better.
Collaboration: Somebody said it is 10 times harder to do something together than to do it by ourselves. I think we can be much more effective when we collaborate. However, we have to do a much better job of talking of the added value of collaboration.
Impact Measurement: We acknowledged that there are many dimensions of impact measurement and that it starts with understanding what the needs are and talking to the people we are serving.
I was in the group that worked on Recommendation 2 (Culture of Accountability). We talked specifically about things like training staff, performance appraisal processes, reporting systems, key messages that we want senior management to use, and donor education. I personally am very committed to this and will commit to you that within Save the Children, I can and will drive this agenda. I think you need to get those kinds of commitments from others within other organizations.
6 SECTION II: RECOMMENDATIONS FROM PARTICIPANTS SECTION III: BACKGROUND ON THE ECB
Q&A: ABOUT THE EMERGENCY CAPACITY BUILDING PROJECT
Q: What is the ECB Project?
A: ECB is a seven-member collaboration focused on research, piloting, and adoption of good practice in the areas of staff capacity, accountability and impact measurement, disaster risk reduction, and information and technology requirements. It was founded by emergency directors from seven agencies—CARE International, Catholic Relief Services, International Rescue Committee, Mercy Corps, Oxfam GB, Save the Children, and World Vision International—that collectively form the Inter-Agency Working Group (IWG) on Emergency Capacity. Over a two-year period from March 2005, the Project aims to “improve the speed, quality, and effectiveness of the humanitarian community in saving lives, improving the welfare, and protecting the rights of people in emergency situations.” Senior executives at the seven agencies endorsed the project proposal along with each agency’s commitments to improve its emergency capacity.
Q: How is ECB funded?
A: ECB is funded by the Bill & Melinda Gates Foundation and Microsoft Corporation. Each agency provides some funding and much staff time to ensure project success.
Q: What is ECB’s value for the seven agencies?
A: ECB was designed to address emergency capacity gaps that are common among large international NGOs and where collaboration could accomplish more than agencies acting alone. Members have benefited from having additional resources to research and implement activities that would not otherwise have been possible. They have also gained from sharing knowledge and good practices, peer accountability, and the inter and intra-agency networking that has happened as a result of the project.
Q: What are ECB’s contributions to the humanitarian sector?
A: Rather than duplicating the work of others, the ECB Project has been working with important existing sector initiatives, such as Sphere, People in Aid, ALNAP, HAP International and Nethope. The Project is committed to transparency and sharing of knowledge and learning. All Project publications are freely available at our website 2, and many are disseminated widely through our partners and other humanitarian networks. A regular newsletter provides updates on progress.
Q: Can other organizations join ECB?
A: This collaboration has been deliberately limited to seven agencies in light of the large amount of time, energy, and commitment it takes to collaborate even with a relatively small group of like-minded agencies. However, in its second phase, the project will work more extensively with partners, particularly at field level.
Q: What happens when the Project ends?
A design process for a second phase of the Project is currently underway. However, it is expected that the seven agencies will continue the work being done on improving emergency capacity, whether through collaboration through ECB and other networks, or through internal change processes.
For further information about the ECB Project, visit www.ecbproject.org or email [email protected].
2 www.ecbproject.org
SECTION III: BACKGROUND ON THE ECB 7 HOW SEVEN AGENCIES TACKLED ACCOUNTABILITY AND IMPACT MEASUREMENT Adapted from a presentation by Charlie Ehle
BACKGROUND Our story started in 2003 when a group of humanitarian directors came together to discuss their frustrations with emergency work. They were from CARE International, Catholic Relief Services, the International Rescue Committee, Mercy Corps, Oxfam GB, Save the Children, and World Vision International, international NGOs of different sizes and strengths. Together they lamented the difficulty of getting funding for forgotten emergencies, of quickly finding qualified staff for emergencies, of measuring program quality. This group of strangers quickly became friends as they jointly imagined a future in which their agencies worked together to overcome these common problems. Fast forward to a conference room in Rome, Italy, in December of 2006. By now that initial spark of collaboration has spawned something bigger, known as the The ECB Project Goal: Emergency Capacity Building Project (the group of directors continues to be known Improved speed, quality, as the Interagency Working Group (IWG) on Emergency Capacity). Equipped with and effectiveness of the an agreement for collaboration, $5.2 million from the Bill & Melinda Gates humanitarian community in Foundation, another $1 million from Microsoft Corporation, and a work plan, this saving lives, improving the project has an ambitious goal: to change the way the ECB agencies and the welfare, and protecting the humanitarian sector respond to emergencies. rights of people in At this event, donor representatives, academics, and representatives from local emergency situations. and international NGOs and humanitarian networks have gathered to hear about the achievements of one of the four strands of the project—the Accountability (to disaster-affected populations) and Impact Measurement Initiative (ECB2). But primarily, they are gathered to discuss accountability and impact measurement issues at large, and make recommendations for how the seven agencies can take forward this work. The name of the conference is Translating Standards into Practice . It harkens back to the approach adopted by members of this initiative at the start of the project. Though the seven agencies were members of various quality and accountability networks and had signed on to several humanitarian standards, actual practice in the field was limited and inconsistent. Therefore, this initiative decided to develop simple tools and approaches that would help busy field staff in an emergency to practice what their agencies had committed to on paper.
WHAT THEN? What have the seven agencies done in accountability and impact measurement?
1. We defined good practice. In the beginning, we tripped over terms. What was the definition of accountability? Of impact measurement? We were getting nowhere with these debates. But we all had some idea of what these concepts look like in practice. So a group of about 30 of us from all seven agencies came together to define what we called “Basic Elements” of accountability and impact measurement. Agreeing the Basic Elements was a major achievement, as they set out the minimum “must dos” for living up to our objectives of keeping disaster-affected people at the center of our work. For accountability, these were practices like providing public information on our organization and project plans, establishing consultation and feedback mechanisms, and modifying projects in response to feedback. For impact measurement they were about getting basic descriptions of the communities we work with at the start of the emergency, tracking what we receive and what we give out as assistance, asking people about their expectations and views on our programs, explaining our methods for gathering feedback to disaster affected-populations—and most importantly, using all this information to improve our programs (see Annex 4 for a complete list of the Basic Elements).
2. We created a “Good Enough” approach. THE ECB2 GOAL: Our intentions in creating the Basic Elements were bold. We envisioned that one day, Enhanced agency each of our agencies would practice them consistently. But we would have to start accountability to modestly by introducing them to a few field offices. The challenge was to explain how humanitarian sector these Basic Elements synthesized the core minimum out of the wide array of standards and standards, principles, and tools that field staff were already struggling with, and to improved practices in identify how they could be put into practice. impact measurement of Given the wide difference in emergency contexts, a “Good Enough” Approach humanitarian action. was adopted. This was about identifying practical approaches to ensure that the each of the Basic Elements could be implemented, and emphasizing the importance of
8 SECTION III: BACKGROUND ON THE ECB choosing simple approaches over elaborate ones. We didn’t mean this to be a “dumbing down” or a lowest common denominator exercise, but rather a way of ensuring that staff could implement the Basic Elements in any emergency setting.
3. We created a Good Enough Guide. We also used the Good Enough approach to create our most significant product to date— Impact Measurement and Accountability in Emergencies: A Good Enough Guide. 3 First drafted in January 2006 and field-tested by various agencies in Guatemala, Indonesia, and Sri Lanka, the guide is based on existing standards and adapted tools such as informal interviewing, focus group discussions and rapid surveys. These have been made as simple and user-friendly as possible for the busy field worker. So popular was this step-by-step guide that field staff themselves translated it into French, Spanish, and Bahasa Indonesian. In addition to the considerable time and effort agency staff have put into developing and reviewing the guide, we are indebted to contributions and guidance from HAP-I, ALNAP, and Sphere, as well as several other agencies. While the guide can be used without training, we are also developing a short module that will make it a more powerful learning tool. We believe this guide has great potential to enhance community engagement. The guide will be published in February 2007.
GOOD PRACTICE NOTE
Field tests of the guide have shown how simple methods can have significant impact. In Indonesia, one of the agencies used an example of a complaints mechanism in the guide and set up a telephone service where people could provide feedback. The phone number was published together with information about the relief program and signed by the local government. Small changes like this made a big difference in the level of trust between local people, the agency, and the government. People did use the phone service; often calling on behalf of others whom they felt should have been eligible for assistance. As a result, the beneficiary lists were modified from the first month to the second month.
4. We formed an interagency standing team. Very early on in our work, we acknowledged that a major hindrance to practicing accountability and measuring impact was that our agencies’ staff lacked the knowledge and skills to do so. In addition to creating tools like the guide, we decided that we needed a standing team whose members could be sent to country offices to help train staff, monitor and evaluate programs, support agencies in collaborative efforts, and build awareness of the importance of accountability and impact measurement. A standing team consisting of 14 members from field and headquarters locations was formed early in 2006. The team has helped to develop and test the guide. Such a standing team is the first of its kind among ECB member agencies, and it means that the staff involved operate on behalf of other agencies, evaluate each other’s work and share lessons.
5. We conducted multi-agency evaluations, trainings, and learning events. Often with standing team involvement, we have also carried out or supported multi-agency evaluations, learning events, trainings, and coordination meetings in the Tsunami-affected countries, Niger, Guatemala, and Pakistan. Standing team members have also run events in their work locations, including Sudan, Kenya, Bolivia, and Indonesia. During all of these events, we have been proactive in encouraging other agencies to join us so we can share what we have learnt and learn from others. We are now getting requests for the team to assist other consortiums of agencies at field level to conduct trainings and share our knowledge.
INTERPRETING OUR WORK Most of these activities happened over the space of a year. Our learning curve has been steep, particularly as inter-agency collaboration is a process that is new to many of us. As you can imagine, we have several examples of lessons and good practice, most of which may not come as a surprise, but are nonetheless hard-earned realities for us.
3 Available at http://publications.oxfam.org.uk/
SECTION III: BACKGROUND ON THE ECB 9 Good Practices
Collaboration This project has shown that collaboration can happen between agencies with significant outputs being created in a relatively short period. Building trust is a vital element, which was, in our case, aided by having consistent faces around the table. As a result, an unprecedented level of information sharing and peer learning has occurred. For instance, Oxfam’s real time evaluation in Indonesia was based on guidance from CARE. Collaboration at headquarters levels has also given a sense of permission to our staff in the field to work together rather than see each other only as competitors. For example, in Niger, advisors encouraged agencies in the field to conduct a multi-agency evaluation after the drought response in 2005. The IWG agencies on the ground shared program documents with one another. They then took the opportunity to reflect together upon the recommendations, most of which centered on improving NGO coordination, as each agency was working independently and this limited the effectiveness of their emergency response as well as of their advocacy work. In response, the agencies began an informal coordination group even extending beyond the IWG members to include eight other NGOs operating in the area. In fact, during one of these meetings, three agencies decided to negotiate a contract with World Food Program as a small group. This manifestation of coordination had never been seen before in Niger. However, these occurrences at the field level are still the exception rather than the norm.
Peer support and pressure Working with sister agencies within the Emergency Capacity Building Project has shown each of us how collaboration of this nature can speed up progress in our agencies, whereas individually we simply would not have had the staff or the motivation to undertake even a fraction of what has been achieved. Collaboration has also allowed us to cite the good practices in sister agencies and create momentum for change within our own.
The standing team as change agents Establishing and supporting a standing team has taught us a great deal. We are proud to have a functioning group whose members can work on each other’s behalf, share knowledge, and leave prejudices behind. This group has been instrumental in spreading the word on the importance of accountability and impact measurement in emergencies. We still need to refine the roles and responsibilities of team members and set up a mechanism so it can become sustainable outside of ECB. The example has been set, and we hope it can serve as a model to other groups of agencies with similar skill sets.
Working with quality and accountability networks Working closely with quality and accountability networks has been another example of good practice. It has ensured that we do not suffer too much from groupthink, and that we are constantly reminded of the excellent work that has been going on for many years and the need to reinforce this in all of our activities.
Developing the Good Enough Guide The creation, testing, and editing of The Good Enough Guide is another example of good practice. Not only have agencies come together to agree on its foundational Basic Elements of accountability and impact measurement, but the guide has been written by field staff for field staff. The process used to develop the guide was a very important step to getting genuine input and ownership from the field.
Lessons
For every good practice we still have many lessons we are learning.
Focus on the goal From the beginning the message from our field colleagues was loud and clear—no more new initiatives! “Of course not,” we replied, we will simply consolidate and simplify existing work. But in due course the project took on a life on its own and soon our staff were being asked to come to ECB meetings, test ECB work, and attend ECB trainings. This was in addition to all the other ongoing endeavors such as involvement in HAP-I testing, Sphere trainings, and responding to emergencies. Through this we have learned that many of these initiatives are talking about the same thing: improving the quality of emergency work through enhanced accountability. Therefore, we need to take a thematic approach and clarify how these initiatives and tools interlink to promote accountability and impact measurement.
10 SECTION III: BACKGROUND ON THE ECB Let the field drive it We said from the start that this project is for the field—and we wanted the field to lead it. Then we promptly started deciding things on the field’s behalf! Initiatives such as multi-agency evaluations only work if they are actively initiated by the field. However, this process can take a very long time, and even if the field drives an activity, the change of one or two key staff members can derail the whole thing. We know now that we have to allow more time for results to be seen, and that engagement with field offices needs to be all about a long process and not about the product.
Have a lead agency in-country Connected to this, we have learnt the hard way the vital role a lead agency plays in the field. Once a country has self- selected or been persuaded to engage, there is a need for one of the agencies in country to take the lead. The role of the lead includes communicating the objectives, organizing logistics, and being a champion for the work. When this does not happen, we end up with a cycle of misunderstandings and inappropriate outputs.
Make time for collaboration We have also learnt that to achieve what we have done takes a huge amount of time and effort. Everything takes 10 times as long as expected, and needs consistent and repeated communications. We are yet to find a solution to how we can maintain this level of workload over the long term without adding more grey hairs.
Remember that organizational change is a long process We learnt that just because change is happening in one part of the agency, organizational change cannot be presumed. Sustainable change is a long-term process and while ECB can contribute to it, it is just one player on a crowded stage. We need to be very clear about what kind of change we are looking for and how this incorporates all the actors and builds leadership commitment to ensure quality improvements of our emergency programs.
Build capacity at the preparedness stage We have also learnt, again, that trying to build capacity or introduce a new concept at the onset of an emergency response is not a smart move. We need to put much more emphasis on working at the preparedness stage so that the principles of accountability, impact measurement and coordination are in place and can be passed on to new staff going into the response.
WHAT NEXT? Over the coming months we will be finishing the first phase of the ECB project (through June 2006) and designing the second phase. We are proud of the success we have achieved so far. Yet we aware of the giant question marks hovering over the future. Will the seven agencies continue to build upon the progress achieved in the ECB project’s first phase in the coming months? How can the seven agencies proactively handle the growing pressure for certification of humanitarian agencies using universally agreed standards? When are collaborative activities like multi-agency evaluations effective and when is it best to work alone? Despite these concerns, we believe that collaboration will continue to have lasting benefit for our agencies in seeking improved program quality, and that in the months ahead, our agencies will push forward more forcefully to address issues of accountability and impact measurement.
SECTION III: BACKGROUND ON THE ECB 11 SECTION IV: PERSPECTIVES FROM THE SPEAKERS
Editor’s note: This section summarizes the presentations from the keynote speaker, the panel speakers, the dinner speaker, and an aid recipient. The section ends with a question and answer discussion. The speakers were not actively involved with the Emergency Capacity Building Project but were recognized experts in their subject areas.
LIBERATING THE HUMANITARIAN SPIRIT Adapted from the keynote presentation by Mukesh Kapila
Editor’s note: As keynote speaker, Dr. Mukesh Kapila was asked to give the context for the four conference themes— accountability to local people, impact measurement, coordination, and collaboration—and to identify likely trends in these areas in the future.
HUMANITARIAN TRENDS
Some of the trends that will influence our current and future humanitarian work are commonly talked about. For example, catastrophic events with wide-scale and complex human, social, economic, and political impacts are expected to increase. On the response side, the numbers of actors may be expected to continue to increase too, along with a widening of the sources and channels of funding. Stakeholders will exercise greater scrutiny and hold higher expectations over the speed, quality, and accountability with which results are achieved. In this dynamic environment, which organizations will flourish? According to the Humanitarian Futures Programme, they will be the ones that show the following characteristics: 1. Those that are better informed, i.e. they give space for internal learning and knowledge management to link policy and strategy with operations; 2. Those that proactively scan trends and draw from them the right response strategies; 3. Those that can adapt management strategies to achieve results in fast-changing environments; 4. Those that are agile by exercising decentralized leadership through the creative empowerment of staff at all levels; and, 5. Those that collaborate effectively across a range of networks.
ACCOUNTABILITY
Countering the tyranny of indicators and logframes While the infusion of a culture of accountability will be desired by any well-functioning humanitarian organization, a single method for accountability is probably impossible. However, an accountability framework could have a number of components concerned on the one hand with the usual accountability to donors based on “the tyranny of indicators and logframes,” and on the other, with the new drive for accountability to clients based on eliciting beneficiaries’ own views of the “most significant changes” that have occurred. There is an evident contradiction in that donor accountability demands pre-set, quantifiable targets, while the latter seeks to extract outcomes retrospectively from the experiences of beneficiaries. Both are necessary and, indeed, enrich each other. But in practical terms they will need to be conducted separately; they should be conducted separately.
Individual accountability However, the bigger question for accountability systems is whether they ascribe responsibility for actions taken or not taken. Just consider for a moment: in our recent professional lifetimes, there have been three major incidents of mass murder of genocidal proportions: in Rwanda, in Srebrenica, and in Darfur. Good enquiries have been done to detail what caused the failure of the international responsibility to protect. It is striking, however, that nobody has had their knuckles rapped, let alone lost their jobs in the chanceries of powerful capitals or along the corridors of power at the United Nations, despite plenty of evidence of what might be called serial offences of omission or commission by individuals. Accountability systems have meaning only when someone is held personally responsible for what they have done or should have done. Far too many people hide behind the anonymous facade of so-called institutional positions or policies. But these are formulated or shaped by individuals—often very few of them—with the mass of workers in institutions following or hiding behind their leaders. The implications for NGOs of this perspective on accountability are twofold. First, you must look internally to see if this linkage is taken seriously at the level of the individual. Second, you might wish to consider the advocacy responsibilities you have to push for accountability and responsibility from the great global institutions that have such profound influence on shaping the environment in which you work.
12 SECTION IV: PERSPECTIVES FROM THE SPEAKERS IMPACT MEASUREMENT
Assessing impact starts with assessing needs Turning to the issue of impact measurement, my starting point is on the question of how needs are assessed and financing decisions made. It must be self-evident that to judge the impact of programming there has to be a clear understanding on needs and how resources are applied. But this remains a most contentious area. It is extraordinary that there is still no internationally agreed means of measuring humanitarian need. Economists can make comparisons between countries through measures such as GNP. We also have the Millennium Development Goals and a good debate has occurred on what they cost and how to foot the bill. Yet though we have Sphere, there is no equivalent measure to guide those who are trying to provide basic human succor. The absence of quantitative tools masks extraordinary inequities with, sometimes, an 80-fold difference in resources spent per beneficiary depending on their location. This apparent prejudice, which is, no doubt, unintended, does little to counter the claim by some that the current humanitarian enterprise is essentially a Western plot. I noted during a previous speech that,
….Of all the major worldwide public endeavors of welfare, education, transport, environment, and so on, the financing of the global humanitarian system is the most primitive, based on little rationality. It relies on media attention, political self-interest, the goodwill of rich world citizens, and the sacrifices of dedicated humanitarian workers. The result is that, for millions of victims of catastrophe and conflict, their life chances depend on a lottery. But this is not even an ordinary lottery where the natural laws of chance prevail. The humanitarian dice often spins in strangely whimsical directions.
Not surprising when we still argue about what constitutes humanitarian action, who needs humanitarian help, how much should we give, when do we know when the job has been well done or even when enough has been done, and how do we move on?
Seeking objective quantitative tools What are the implications of this irrationality for NGOs? They should think of setting the example by separating the functions of objective needs assessment from the functions concerned with maximizing agency income and programming, so as to demonstrate due integrity and impartiality. Earlier this year, while working as director for emergency response at the World Health Organization and serving as the first chair of the new Inter-Agency Steering Committee Health Cluster, I postulated a concept for a “life and death measurement service,” or to call it by its official name, a “Humanitarian Tracking Service.” The Humanitarian Tracking Service came from the concern that the allocation of attention and resources to humanitarian crises is not based on objective and impartial evidence; and furthermore, there is no agreed system for monitoring trends and “I have no doubt that measuring the effectiveness of humanitarian action. Addressing this is essential to sooner or later, such an improving accountability, which is at the core of the international humanitarian system objective approach to reform. measuring needs and Human survival and health are commonly perceived to be the key dimensions by which the severity of a crisis and the success of humanitarian assistance are judged. The allocating resources will tracking service was therefore intended to provide an impartial evidence base for happen, as the present making resource allocation decisions by tracking trends in mortality, morbidity, and arrangements will not be malnutrition in populations of humanitarian concern. Humanitarians would thus be defensible forever in a enabled to direct, focus, coordinate, and prioritize their assistance and protection efforts world that will continue in line with validated needs, trends, and gaps. For those involved in humanitarian to demand greater and assessment and monitoring, there would be greater efficiency and synergy of effort. greater transparency and Everyone agreed that this was a good idea whose time was long overdue. accountability.” However, it ran into the heavy waters of the rivalries within and between agencies, mostly UN agencies, over roles and responsibilities. I have no doubt that sooner or later, such an objective approach to measuring needs and allocating resources will happen, as the present arrangements will not be defensible forever in a world that will continue to demand greater and greater transparency and accountability. NGOs could also think of working collectively to establish a genuinely independent and trustworthy “life and death tracking service” to provide the basis for informed resourcing and programming decisions, and for holding decision-makers accountable accordingly.
SECTION IV: PERSPECTIVES FROM THE SPEAKERS 13 COLLABORATION AND COORDINATION
Unchanging human behaviors Let me start with various Tsunami, Rwanda and other evaluation reports that are on the ALNAP database. It is comforting to note that the findings and prescriptions of these studies are so reassuringly similar, despite the years that separate them. This is not intended to be an ironic or cynical observation. It simply shows again the persistence of certain behaviors that are inherent in the way that humans set themselves up in tribal organizations, and that will probably continue forever. Thus, I suggest that it is not worth expending too much energy on making marginal and temporary improvements through yet more elaborate coordination structures that require more and more servicing while possibly reducing flexibility and responsiveness. This is not a criticism of the Inter-Agency Steering Committee machinery, including its new clusters, or a criticism of the seven agencies of the Interagency Working Group, but a realistic recognition of limitations that are inherent in an unequal system where size and resources equate to power and influence. Neither should this be misunderstood as a sanction to return to isolated or silo working. Collaboration and coordination are essential means towards achieving certain effective outcomes —but they should not become an end in themselves, nor an ever-growing industry. It has been said that there are situations nowadays where there seem to be more coordinators than actors!
A post-coordination world My vision is for a world where actors are inspired not by coordination structures, but by common objectives, and disciplined not by standards but by an understanding of an accountable division of labor. Elsewhere, I have called this moving to a “post-coordination world.” What does this post-coordination world look like? First, it will be a world in which there is much more trust within the international humanitarian system. This will only happen if we give primacy to the moral duty to get the best humanitarian good out of each humanitarian dollar. The test of this is to analyze not just what goods and services are bought with each dollar, but what is the benefit for each dollar spent on one type of humanitarian action, compared to another type. This implies having to make trade-offs and choices, and a different way of formulating appeals for donor resources. In turn, this will require, as mentioned already, a degree of separation between the assessment of needs, performance, and impact on one hand, and financing of agencies on the other hand. “My vision is for a world In such a new humanitarian order, we might have collaboration and where actors are inspired coordination architecture that is different from now—and possibly radically different. not by coordination Imagine this different world: where our great humanitarian organizations are only structures, but by common concerned with achieving the most benefit for the maximum number of people, objectives and disciplined where donors are motivated only by the desire to do good and not to pursue not by standards but by an narrow national interests, and where there is fairness and predictability of response. understanding of an Coordination has a different meaning in such a world—that is why I have called accountable division of it the post-coordination world—where we do not need such elaborate and costly bureaucratic arrangements to police the mutual paranoia that characterizes the labor. Elsewhere, I have current humanitarian system. This includes the donors who need to take an honest called this moving to a self-examination of their own accountability and their behavior, in terms of whether ‘post-coordination world.’” or not they are providing the right incentives or sending the right signals to the agencies they fund.
14 SECTION IV: PERSPECTIVES FROM THE SPEAKERS LIBERATING THE HUMANITARIAN SPIRIT
In the “post coordination world,” the debate will be more sophisticated than the technicalities of accountability, performance and coordination, and we will not complain about (but instead, welcome) the ever-increasing numbers of humanitarian actors. This is because the humanitarian instinct—the urge to help others—is one of the oldest and most noble of human characteristics. It is practiced in many ways in many parts of the world. No single culture or institutional system has the monopoly on humanitarian wisdom. In the new humanitarian order, our institutions would not be prisons to box us in or fortresses to keep others out, but launch pads for yet more creative and ambitious humanitarian enterprise. Indeed, the challenge for the modern humanitarian is to liberate the humanitarian spirit in all who wish to join the humanitarian enterprise. The greatest danger may yet be that we do the opposite, i.e. we suffocate the humanitarian idea by over-institutionalizing it. This way of thinking can be summarized very easily—it is a return to the most fundamental basics of finding the simplest way to organize all who wish to make common cause for one shared purpose: to help, in the most direct way possible, all those afflicted by crisis and catastrophe. Not here and there, not now and then, but everywhere and always.
ABOUT MUKESH KAPILA With a background in medicine, public health, and international development, and with qualifications from the Universities of Oxford and London, Dr Mukesh Kapila has worked extensively in relation to HIV/AIDS, crisis and conflict management, humanitarian aid, and post-conflict recovery and development in many countries in Africa, Asia, and Europe. He is currently the Special Representative of the Secretary General for HIV/AIDS, International Federation of the Red Cross and Red Crescent Societies.
SECTION IV: PERSPECTIVES FROM THE SPEAKERS 15 ACCOUNTABILITY: SELF ASSESSMENTS, VALIDATION BY OTHERS Adapted from the panel presentation by Brendan Gormley A new challenge for the DEC The Disasters Emergency Committee (DEC) 4 has taken up the challenge of driving improvements in accountability and determining to what extent putting beneficiaries in the driving seat might be the best way to do it.
The DEC’s 13 members, along with its Secretariat, are revisiting its accountability framework and have set three goals:
N We want performance on accountability to be validated independently and publicly. N We want to validate it cumulatively, moving away from our current one-off “The DEC wants evaluation snapshot, to a more comprehensive and long-term picture of performance on accountability. We will continue to widen the toolkit with methods such as accountability beneficiary surveys and internal and external audits. to be validated N We want to use accountability as a means of building trust, rather than decreasing it. independently and publicly.” The central criteria of our new framework are:
N Be cost effective. N Show that funds were used as stated or give reasons for any differences. N Review performance against agreed sector standards such as the Red Cross/NGO code of conduct. N Demonstrate that beneficiaries were actively involved and their wishes respected. N Review performance on the basis of previous lessons having been learnt.
Members rate themselves Our approach is to shift away from the DEC secretariat managing evaluations and monitoring missions to having our members driving improvement. We say to them, “You tell us how you and your chief executives are sleeping well at night and that evidence can ensure that the DEC Trustees can sleep well at night.” For example, our members can rate themselves on how well they are involving or being accountable to beneficiaries against a list of indicators (see Figure 1). The country manager could then score that country office’s programs, supply the evidence, and decide what improvements will be made. DEC would sign off on it, investigating only where it feels evidence might not be substantiated. This is a reversal of the current roles.
No agreed model for accountability One of the things holding us back is that we still do not have an agreed framework by which to model accountability to beneficiaries. There is a plethora of competing “We need to be frameworks, which makes it harder to find an outside entity that would give independent ready as individuals sign-off on our progress. Auditors such as Pricewaterhouse Coopers ask what they are to and organizations evaluate us against, and what performance is good enough to ensure we achieved what we to show we are set out to do. We struggle to define these things in such a way that a third party could give being accountable an independent opinion. to beneficiaries. The ECB model talks of information sharing, participation and grievance mechanisms We don’t need to and there are other models to choose from such as the Global Accountability Project Framework (One World Trust) or HAP-I frameworks. Can’t we just decide on one and do it perfectly but start using it? we have to show we are trying.”
4 The DEC brings together 13 leading UK international aid organizations at times of major crisis to launch national fundraising appeals and is active in promoting high standards of accountability in humanitarian work.
16 SECTION IV: PERSPECTIVES FROM THE SPEAKERS Figure 1
Requirement Evaluation Comments / Evidence Planned / Proposed actions
Key Principles and Standards – Red Cross/NGO Code
No 7. Involvement of/ accountability 5 Evidence to support score to beneficiaries. Excellent capability established Indicate the rationale for the score ------selected (e.g. if ‘3’ is selected, what Indicative leading practice: 4 evidence can be provided to Embedded and improving support this score?) N Evidence of meetings being held with community 3 Planned / proposed actions leaders Implemented in all key areas Indicate actions (planned or potential) to bridge any gap which N Local input to agencies’ 2 exists between current state and strategy/projects Implementation planned and in desired score. Indicate owner and progress timescales for actions identified N Published in country budgets and plans 1 Sign off /comments of internal or Awareness and understanding external reviewer/auditor N Complaints mechanism in place
Start somewhere with accountability I think NGOs should be encouraged that they have real skills to get this work done. One area for concern may be whether we are creating a bureaucracy and rulebook that will squeeze out those very local players and partners we say are so crucial for effective humanitarian action. If DEC raises the bar, will some international NGOs stop working with good local partners that struggle to demonstrate they are meeting the desired standards? In response to this, I come back to the simple question, “What allows you to sleep at night?” We need to be ready as individuals and organizations to show we are being accountable to beneficiaries. We don’t need to do it perfectly, but we have to show we are trying. Also of note, the DEC is in discussion with DFID to harmonize and simplify upwards accountability and other reporting processes, which should make it easier for us to focus on downward accountability. Finally, I should note that trickle down capacity building doesn’t seem to work. We need to be empowering and building the skills of our partners and staff in the countries where we work if we are to be successful in being accountable to beneficiaries.
About Brendan Gormley Brendan Gormley became the Chief Executive of the UK’s Disaster Emergency Committee (DEC) in September 2000. After working as an “unattached social worker” with a motorcycle gang in Birmingham, he joined the White Fathers Monastic Order and worked and studied in Ireland and France. He continued his interest in social issues working with travelers and immigrants in both countries, and with teenage drug abusers in London. Leaving the Order, he went on to study at Cambridge and then worked in the late seventies and early eighties on environmental and pastoral issues while living in Africa and on urban problems in the Middle East based in Cairo. During the 1990s, he was Oxfam’s Africa Director. He finished his career with Oxfam supporting the establishment of Oxfam UK’s global network of regional offices.
SECTION IV: PERSPECTIVES FROM THE SPEAKERS 17 IMPACT MEASUREMENT: IMPACT ASSESSMENT IS AN INTERVENTION Adapted from the panel presentation by Mary Anderson The many dimensions of impact The dictionary says an impact is the influence, effect, or impression of one thing on another. Impacts happen all the time in whatever we do. In the humanitarian field, impacts have “Impact is about multiple dimensions. They are positive and negative; immediate and subsequent (later); asking the ‘So what?’ short-term and lasting; direct and side effect (indirect); intended and unintended; tangible question. ‘What has and intangible; and individual and cumulative. Every time we undertake an emergency response, we are having impacts in all these actually happened to dimensions on the people and in the societies where we are intervening. Accountability people’s lives as a requires that we track all of these dimensions—not just those we intend to have in the result of our immediate short-term—and assume responsibility for them. efforts?’” Experience shows that, in general, humanitarian agencies are better at tracking intended, positive, direct, short-term, and tangible impacts. We often ignore or miss out unintended, negative, indirect, subsequent, long-lasting, and intangible impacts. ECB uses the term impact measurement. I find the word measurement puts emphasis on quantitative over qualitative, tangible over intangible impacts. Thus, I prefer the term assessment, because it can include both numbers and non- quantitative dimensions of impacts. An impact is not an output. It is not about the number of times we conduct a nutritional survey or how much water we have delivered. Impact is about asking the “So what?” question. “What has actually happened to people’s lives as a result of our efforts?”
Assessing impact How do we assess impact? First, what is it we are assessing? We should be assessing whether our arrival, our effort, has or has not been helpful to people’s own efforts as they deal with their emergency. Further, has it been helpful enough to warrant the effort? Has it been the very best we can do or are we getting by just being a little bit helpful? The people who can answer these questions are the people we are supposed to be helping. ECB’s Good Enough Guide asks, “How can we involve the men, women, and children affected by an emergency in planning, implementing, and judging our response?” The fact is that they are judging our response whether we have involved them or not. They are the ones who can answer the question of whether or not we are being or have been helpful. How do we listen to their judgments, tap into their analyses? In their Standard 6 on implementing complaints procedures, the Humanitarian Accountability Partnership International calls for agencies to establish and implement effective systems for hearing complaints that are accessible and safe for the intended beneficiaries. This system focuses on direct, immediate impacts that can be traced to a specific intervention, and on negative, unintended, tangible, and intangible impacts on intended beneficiaries. We might ask how we can expand on processes such as this and create broader systems to listen to the judgments of many people across all of the impact dimensions.
What we know about our impacts We have asked enough questions of enough beneficiaries, both happy and disgruntled, to know a lot about the “what” of aid delivery. We know people need the things we have brought, and that we make mistakes in delivery and timing. We get mixed reviews on distribution systems—who gets what and who doesn’t, and whether things get to places on time. Everyone knows what those assessments say and what the problems are. Evidence we are missing is adequate understanding of how people assess our management of “insider” and “outsider” relationships. We get positive and negative feedback on how we handle information and how we communicate—we are told we do too much explaining about what we are doing and not enough listening. People also give us mixed reviews on participatory methods, saying they feel they are participating in our processes. We get a lot of negative feedback on the frequency of contact between outsiders and insiders—people feel they do not see us enough to develop the kind of relationships that would be rewarding. People tell us they are afraid to complain about our interventions for fear that we will withdraw aid from them. Finally, we get a lot of negative feedback on whether people feel respected as we provide assistance.
18 SECTION IV: PERSPECTIVES FROM THE SPEAKERS Impact assessment is an intervention One reason we fail to communicate respect may be because we do not realize that “To deliver things impact assessment is, itself, an intervention. This takes us back to the basic question without ‘delivering’ an of whether we are being helpful or not. Not to ask this question of the people who assessment—not can answer it, or to do so casually or intermittently, or to limit what we will and will listening to people’s not listen to, conveys disrespect. Impact assessment is not an optional part of views—is to do only humanitarian action. To deliver things without “delivering” an assessment—not listening to people’s views—is to do only half of the job. half of the job.”
ABOUT MARY ANDERSON Mary B. Anderson, Executive Director of CDA Collaborative Learning Projects and President of CDA, Inc. has worked in international development assistance for over forty years. CDA Collaborative Learning Projects, a non-profit entity established by Dr. Anderson in 2003, broadens the scope of collaborative training, learning, and new project development with many international governments and NGOs. Named “the most influential theorist in the world of humanitarianism” (NY Times Magazine, Feb.11, 2001), Dr. Anderson has built CDA’s reputation for pragmatism and a grounded approach to solving problems faced by the staff of international agencies.
SECTION IV: PERSPECTIVES FROM THE SPEAKERS 19 COORDINATION: A LOGICAL STEP IN THE PROCESS Adapted from the panel presentation by Mamadou Ndiaye Coordination is about appraising critical needs There is growing criticism of the lack of coordination among NGOs. I can cite several examples of poor shelters being built, wrong medicines administered, etc., “We have to realize that and we all have stories of how NGOs fail to be complementary. when an emergency Sometimes people in the field say, “We are busy saving lives, we don’t have time happens, no single agency for anything else.” Yet we have to realize that when an emergency happens, no single can solve the problem.” agency can solve the problem. To be credible, to meet the quality criteria that are important in accountability to disaster-affected populations, NGOs need to find a way to make coordination happen. The Tsunami and Rwanda evaluations have shown the negative impacts of poor or no coordination on the quality of services delivered by NGOs in these crises. Unfortunately, the word coordination is often misused. For many, coordination is interpreted as a policing tool. This misuse can be an excuse for not coordinating. We need to look at coordination as a tool to enhance our credibility, and the quality of service we provide to beneficiaries in light of our objective to save lives. Coordination is how actors get together and decide ways to address critical needs in an emergency. Don’t be sidetracked by coordination for coordination sake, spending time without any tangible results. Go through the process in order to assess the gaps of the response, share key information, and find appropriate remedies. The first step of efficient coordination is to jointly discuss capacities and resources. Then, you can look at a possible division of labor to avoid duplication and waste of scarce resources.
Who should ensure coordination happens? In addition to NGOs, various actors have different roles to play to ensure that coordination happens:
Consortia and networks can play a significant role because many organizations subscribe to several of these. These networks and consortia can help guide coordination in the field by putting together agreed guidelines. The problem is that they have limited power in enforcing compliance to coordination guidelines like the NGO field cooperation protocol. Are we ready in the sector to give enforcement power to networks or consortia to which we subscribe?
OCHA, UNHCR, and other UN institutions can be complementary to NGO coordination. I believe NGOs should not be coordinated by OCHA because NGOs have their own personality. The UN is trying to build coordination efficiency and effectiveness through the cluster approach. NGOs are only marginally involved; we need to engage more and learn from this initiative.
Donors can help in making coordination happen, but they may have their own political interests that do not help coordination.
Host countries cannot have a high level of involvement in coordination but they need to be informed. We usually say that host governments are not transparent, but I’m not sure that we are more transparent.
Barriers to coordination Competition/visibility: NGOs want high visibility for their bag of rice and logo. Competition for funds is strong in emergencies. This competition does not help coordination.
Financial dependence: Most NGOs depend on donor funding, limiting the resources available for coordination. Donors often have political interests linked to their support. Consequently, donors ignore many real crises.
Diversity: There is no single way of thinking or doing things. Many NGOs have different backgrounds, ages, functions, and expertise. This diversity can be a strength. But some diversity, in goals, for example, can be a hindrance.
Poor representation: Many times, NGOs send individuals without much decision-making authority to coordination meetings. If serious about coordination, NGOs should send people that have the power to make decisions and share key information.
20 SECTION IV: PERSPECTIVES FROM THE SPEAKERS Poor or no use of coordination platforms: The “raison d’être” of networks is to create conditions of working together. Network or consortia should not just act like unions to protect their members. These platforms should have power to enforce what we agree on coordination.
Donor policies: Although donors ask us to be accountable, they themselves are not always accountable, as public and political agendas often come first.
No or poor local representation: INGOs tend to leave out key players in coordination efforts—local actors who are active in the field, and as said earlier, are the first to save lives.
What does coordination take? Coordination needs a new vision of its importance in our actions. It requires a change of mindset and behavior so that coordination is considered as a duty and not as an option. Agencies must use shared and/or agreed values to guide their actions. You cannot come to disaster-affected populations and clients and have different messages. It is also about respect. Respect to disaster-affected populations starts by trying to find out what their needs are. Coordination also requires resources and commitment for NGOs engaged in an emergency.
Ways forward N Staff will need to be trained on coordination. There are great written “Coordination needs a documents in drawers at headquarters; field staff often ignore their new vision of its existence. importance in our actions.
N It requires a change of NGOs need to invest in coordination to meet quality delivery criteria. mindset and behavior so that coordination is N NGOs need coordination focal points that are chosen on our behalf to work with the government and with other coordination bodies. This considered as a duty and usually doesn’t happen, or the lowest ranking person, without expertise in not as an option.” the area, is chosen.
N We also need to improve coordination within the broader sector. Networks and consortia have to provide increased regulation and enforcement of principles we sign up to. Additionally, we need to better link up with OCHA on coordination.
ABOUT MAMADOU NDIAYE Mamadou Ndiaye is the Director General of Office African pour le Developpement et la Cooperation (OFADEC), an NGO operating in the humanitarian and development sectors in Senegal. Mr. Ndiaye previously worked as an Agriculture Project Director in Senegal, and with UNHCR Refugees programs in West Africa for more than 10 years.
SECTION IV: PERSPECTIVES FROM THE SPEAKERS 21 2 2 par c s The L W W C S Dr s c wo n s H i har iz ome ta omp ol ome ot R ea ere are a few OL e hat e h labo iver r
s both tner se N N N N N N N k to easy! But rni a and ng i p W I n t t lexit d of LA h & Deav rc i r r e is ng s vate o frequen he p cti ge ati d t P N T e Cli T N R w c c in an hap w e m e p re m o al we BO eas co u r o ver xp ngi tn ar ar y s iver he dr her he esar fr o i ther on f o e e e p l kn th d of th lab col ents al. ul w d eti mbl o tic b p sec llabo lo om ivat r on b NGO e ia. T is yo Ser t r e i RAT and c d ens mean gencis wil incr emer es wi s er r lend ngs , es o Pr ular o id heal ation on laborati e wan c
tor go ne h ing w r s di on , are colla c h o vi ate w it d his e s
o thi elop pecial y in a comp y . with f ratio can d in i
th c ther and
ng t ly ION: LES bloc erent example omplement th he uc has their genci of emer e S ec i n and beyond n fro s can lear ec b as heir for goin to get fo at to s at of colabor ment at y colabor lear ts t p alt sec in t t D to bo ben t s
co or, c r ro c ks. Develo t eti n an o b h
ing h on ar o b o th comp e act o t evelop m ? tor tor c r la o n il an Bu v ser
e b e es a of ion e Ne w P ola r f th a bor ake u ivte tio ig a ing . I : T he r se f auto t the s to e shor
om M t joint ackling d g S ice r pm e fo bo s; y ar ter n is a jo ON tenci o com tio s a in r ea wo har ion? mo a n info a r one UK y we ent: e s t ion s. Ge re b fo skil m s int m s! t f ch w n is no nd nt: t he the at d e ti Ea ea S hat ie petior co on happens tive k r so h u der Ho w go el When wo in ro FR ma r om s w ld .se r a Divs another l e p per hum re w t c wi o-p ur c i- duct
atio labor ine h wo ve t ind us . D sign withou t a gan ea ny o b he ctual e ho At the s ould be e o ce her OM TH SEC TION IV: PER SPEC TIVES F ROM THE SP EAKRS t ri tion sy fo th o i nme r olitca ins lion d o is anit s isater ea ivate pr s to m va s than s ch a Dev ion o le ma r r f e a b r u ne sy and is im it its m the te s tr iskr l d ca f i r n dif I pp rop r count nce ings it y. Som k r elop stems go thr ough b idd ing p r t.or ” What quite t w T f la an se at var am n s r s.r e fe l ector and em E lar, multi st istakes se com for de im be re ation al bor and colab or a But tabilty w PRIVA ment ants e t te ar t t t s ap pr oach to the go ply infor ctor are r egular “ joint ventur e” nt fr om ctor. e car m caus able anlog he ious places iat ime of it pan y. S o t fr they t to digt d in t ,e e clients peded by pe and fr ca
o t -year ef or er e t , NGOs w
dist to one anothe de m p ape M s com pe TE Ma .g mul a For ge il lead to more “ he ou h sim mation. a s, made ple cordinat he nufact e ncie ingus om y want
nu ize ’ d em up s s SECTOR he n com e to the huma r into d igt alo ilar factu p at its s are g tr ti to t he il s ur onm e v er ng t ands rce ap eam ient sea or oces to g ing r er s it ion? The on t til One t s ived he v proaches nty hos s r t t A
o m a from oil ind mles ver et for re be s o s
d g al so o es apte e et . he
of mistak es and discov co comp
M alue cha n man-mad ke S nt req w hare s nit in h m s eamle r .increas e ale
ds cor d int it ia ar ent ha d t us r uir he i h gr t
y definto ctionar s & fro a egr t s t et o co y r ed to he ia humanitr n and v car dinato is tha s m it in sector oups is r io a Ma
:
e” em
tion tha b in s agobout 20 y,ear no th l Cl n, ation,abor some ed the find s itself incre t a colabor ks and cost e lack eg r to w “Cl se bec ev ble of comp e
keting imate cha pan ation.r in am e ge er nds i r
aus ents ust of tr e y sy up into sector count r a o le l : n sector
o n eme pre rk ncie the be lends the be e the ne ha ss a n of colabor ered
M te ar w D i it is more about s n s a tior with a broad s. e ib istr nteg g asocited with , i a ntai The scale e fo gencyr . e wil impact the th o tha it is y asing s nd ategic and str
wa to br st o v ution & Ser r o rc e a the nd n go tio l nt
i com p y having to by ng st of what f w b . Bu r ing.” n tainly cer and id for s Gib tio us hat that er s t n is “T any Bu ice et of lo ch o A value chain for NGOs So what are the critical success factors and where might collaboration take place on the humanitarian “value chain”? We may compete for access to scarce resources but do we all have to do our own monitoring and evaluation individually? Can we present a united voice to all the important media when reporting on progress? NGOs could also agree upfront who has the strongest capacity in which region or within which sector and cede responsibilities accordingly. Can we as an international NGO community get our act together “Collaboration is really and introduce a more unified front with which to participate in the UN cluster about understanding system? where you have a core NGOs could collaborate through a range of models from collaboration on competency and where support infrastructure, such as logistics, up to more complex models that would you might want to involve regional “operators,” as well as joint ventures between NGOs similar to collaborate to bolster the examples of the upstream oil industry collaboration. perceived skills gaps.” Collaboration is really about understanding where you have a core competency and where you might want to collaborate to bolster perceived skills gaps. These partnerships don’t have to be forever. You may partner on a one-off, one product, and one country basis. You may also partner in one area and compete in another.
ABOUT GIB BULLOCH Gib Bulloch is a business professional with over 15 years of experience in a variety of large private sector companies and in the international development sector. He currently heads Accenture Development Partnerships (ADP), a new not-for-profit consulting group whose clients include Oxfam, CARE International and The World Bank. ADP’s main focus is on bringing business and technology expertise to the international development sector and on promoting private sector engagement in sustainable development. ADP, and Mr. Bulloch’s role in helping to create it, is featured in a new book called “Everyday Legends,” which highlights the stories of 20 leading social entrepreneurs in the UK. In his role as Director of ADP, Mr. Bulloch travels and works extensively in developing countries.
SECTION IV: PERSPECTIVES FROM THE SPEAKERS 23 SIT DOWN WITH THE PERSON WHO HAS SUFFERED Adapted from the presentation by Nelly Badaru I was once a recipient of aid. In the fifteen years since then, I have seen a lot of difference in the way aid is being provided to recipients and I thank NGOs for the work that they do. “What many recipients are not aware of is that It is hard to ask for accountability resources are raised in The way I understand accountability is simple. Let us say your neighbors have gone to church and come back to find their house burnt down. People will come to the their name. Hence it is aid of these people that have lost everything. They will bring money, food, whatever their right to demand they have. Just imagine how hard it would be for the recipients to ask these people, accountability!” “Where did you get these things from?” “How much did they cost?” That is the way disaster-affected people see it! They view NGOs as those who have come to help, and it is usually hard for them to ask NGOs about accountability. What many recipients are not aware of is that resources are raised in their name. Hence it is their right to demand accountability! Disaster-affected people do give feedback in many ways. They might sing a song, dance for you, but they will never ask you directly for accountability. Why? Because if you have a problem—if someone’s house burns down—you will sit down with the person who has suffered and together you will decide what each has to do. And that is what disaster-affected people look for, assistance that upholds their dignity. In this spirit of solidarity, then, the issue of who is accounting to whom will not come up because each party will account to the other. If NGOs want to be honest and transparent and help somebody out of their problem, the practical thing to do, then, is to treat disaster-affected people with dignity and give them all the information that they need, rather than talking about the NGO’s mandate. And from this process of sitting down together, then you can measure your impact, because you would already have agreed on what to do and who is doing what.
A common goal drives coordination and collaboration I think the burden of collaboration and coordination rests with disaster-affected people. Take for example, grievance mechanisms. If you sat with disaster-affected people to agree on what to do, you would also agree on how you would address problems together. If this initial consultation process was a joint exercise, then complaints mechanisms should be an avenue for the NGO to complain also, because we are working together towards one common goal.
Accountability is everyone’s business Ultimately, accountability is everyone’s business. Donors, NGOs, disaster-affected people are all interested in the business of aid and everyone has a stake. In that case, everyone should be interested in accountability. This is why we should not use the word “beneficiary”—the reality of aid is that we are all beneficiaries at some place along the aid chain. If those of us who have been recipients of aid are to keep our dignity, it would be nice to use a different word.
ABOUT NELLY BADARU As a child in Uganda, Ms. Badaru and her family were driven out of their village when it was attacked during a civil war. Fleeing to the Congo, Ms. Badaru lived for a number of years in a refugee camp administered by the World Food Program, the United Nations High Commissioner for Refugees, and NGOs. She was deeply affected by this experience and the lack of dignity and information provided to refugees. Since returning to Uganda, she has committed herself to ensuring improvement in the quality of aid delivery. She currently works as a consultant with NGOs with this purpose in mind.
24 SECTION IV: PERSPECTIVES FROM THE SPEAKERS THE RIGHT RELATIONSHIPS Adapted from the dinner presentation by Hugo Slim Accountability and transparency St. Francis of Assisi, like several other founders of religious orders, used to be a posh, rich bourgeois boy, big on war and a good fighter. But St. Francis’ conversion was “The whole question of unique in that it was gradual, not dramatic. At one stage in his conversion, he went accountability and to Rome and saw beggars begging for alms. “I need to know what that’s like,” he transparency is really thought. So this very rich person exchanged clothes with the beggar for money and about relationships, and spent the whole day on the streets as a beggar. I find this an interesting idea to get us thinking about accountability. Suppose we about getting exchanged clothes with IDPs? If the IDPs wore our clothes, it would be quite relationships right.” interesting, because to some extent accountability is about power and switching of power. I have also had my own personal experiences of practicing accountability. One of the first was as a humanitarian worker in Ethiopia in 1985. I was in a place called Korem, at the heart of the famine. We were doing wet feeding and health care as well as massive dry feeding for the villages all around. Then the trucks stopped coming up the road from the port and we had about 12 to14 villages waiting for food. I was 24 at the time and I decided to be very transparent. Turning to my Ethiopian colleague, I said, “We have to tell people the trucks have stopped coming so they can decide whether to wait or leave.” So we called all the villagers out into this plain, I got up on the roof of the Land Rover and said, “Sorry, there is no food and we can’t tell when it’s coming.” The crowd looked angry. But it got happier and started to calm down as my Ethiopian colleague translated what I was saying. He went on at length in Amharic and then told me everything was fine. Later on I asked him what he said to them, because he spoke much longer than I did. He said, “I told them that everything was fine and that the food was coming in the morning. If I had said what you said, you would have been beaten up and we would still be out there looking for bits of you!” The moral of the story is that there is a moment for transparency and a moment when it is not practical! Another experience with practicing accountability was after the first Iraq war when I was working with Save the Children. Saddam Hussein struck really hard against the Kurds and hundreds of thousands of them fled into the mountains in eastern Turkey at a very cold time of year. Most had come with nothing. When I arrived at the camps, there were airdrops going on and people running around. It was the usual terrible mess. I said to myself, “I’ve got to do something participatory, I’ve got to talk to people about what’s happening.” Walking into a tent, I found myself in the company of a very old man and some younger women. I told him I was interested in hearing what was going on and sat down with my notepad. The old man looked tired. He stopped me and said, “You’re my guest, would you like a drink? Would you like tea?” He sent one of the women out and she melted snow and made one cup of tea. We shared it. I tell that story because the whole question of accountability and transparency “If we are to measure is really about relationships and about getting relationships right. Many of us know our impact, we just have how to do it but don’t have the time, or we think we’ve got other pressures. It is about keeping dignity on both sides of the relationship. That man was not going to to step out and ‘do it.’ let me make him into a beneficiary. He was going to tell me who he was, and that I We have to find simple, was in his house, and that he was having a tough time but he was still there. And commonsense ways of accountability is really also about power sharing, about us letting go of power and making judgments about letting others take hold of it in this humanitarian relationship. our performance and of Whatever words we use, whether delivery, service, discipleship, solidarity, or being judged.” empowerment, they have to capture that right relationship and the dignity that is so important to getting that relationship right. Using business language, it is about customer care—respecting the customer and responding to the customer as an equal, someone you are very dependent on as well. But is the customer always right? I think it is always right to tell the customer that he or she is always right. But I’m not sure the customer is always right. For example, I personally am not the sole authority on my life and probably genuinely do need help in some areas. Humanitarian agencies do have expertise, we do have oversight that an individual doesn’t necessarily have, and we do have principles. The relationship should not be seen as an abdication and submission of rights to claimholders but a balance of power and a complementarity of offer and need.
SECTION IV: PERSPECTIVES FROM THE SPEAKERS 25 Impact Impact is a complicated, mysterious, terrible, and wonderful thing because we don’t know what impact we make in any of our lives as individuals, parents, professionals, or people on the street. We do have some useful tools and ways of measuring impact; but our actions are always going to be ambiguous, even paradoxical. A bad response may lead to a good outcome. For example, NGOs can do such a bad job that people decide to do it themselves—and this empowers them. We can be sure that our work will always be ambivalent—and we shouldn’t try to hide it, because people are bound to have mixed feelings about the human experience. Life isn’t all good or bad.
Measurement If we are to measure our impact, we just have to step out and “do it.” We have to “Using business language, find simple, commonsense ways of making judgments about our performance and of accountability is about being judged. We must use simplicity as a guide and the tools we have, such as ECB’s customer care— Good Enough Guide, Sphere, HAP-I, and other quantitative and qualitative respecting the customer approaches. and responding to the customer as an equal, Practice someone you’re very Practice encompasses all of these topics. It is about the art of doing accountability, dependent on as well.” transparency, and impact measurement. It is important to think of practice on three levels—the individual, the team and the organization. We need to help the individual at the point of contact with the field practice all these things that are important to us. We need to think of teams and make sure they are able to practice, accountability, transparency, and impact thinking; and we need our organizations to do all these things as well and want to do them. All these three parts of our agencies need to embody the right relationship, that knowledge and thinking about impact and those sound judgments on what we are doing and how we are doing it. Right now, we have a chance to think about impact, methods, standards, and other important concepts, and to get them right. If we don’t build our houses on firm foundations, we are building them on sand, and as we know, houses on sand don’t stand.
ABOUT HUGO SLIM Hugo Slim is Chief Scholar at the Centre for Humanitarian Dialogue (HDC) in Geneva. He has a degree in Theology from Oxford University and a PhD in Humanitarian Ethics from Oxford Brookes University. Before joining the HDC he was Reader in International Humanitarianism at Oxford Brookes University (1994-2003), where he led the highly successful Masters program in Development Practice which won a Queen’s Anniversary Prize in 2001 for “its pioneering contribution to the education of humanitarian aid workers.” Before becoming an academic, he was a humanitarian worker for Save the Children UK and the United Nations in Morocco, Sudan, Ethiopia, and the Palestinian Territories (1983-1994). He has published widely in academic journals on questions of humanitarian action in war. He was on the Council of Oxfam GB, an International Adviser to the British Red Cross between 1998 and 2004, and is currently a Patron of Merlin and an Academic Advisor to the Aegis Trust for the Prevention of Genocide.
26 SECTION IV: PERSPECTIVES FROM THE SPEAKERS QUESTIONS TO SPEAKERS Editor’s note: At the end of the keynote and panel presentations, there was a question and answer session in plenary chaired by Denis Caillaux, CARE International’s Secretary General. To facilitate reading, questions have been grouped according to theme.
ACCOUNTABILITY
Q: To what extent should individuals be held accountable for what they do?
Answer (Mukesh Kapila): I have learned one thing in 15 years—it boils down to individual responsibility. Having worked in government, with UN agencies, and with NGOs, I can say that there is no such thing as institutional policy. It is always about the prejudices or beliefs of a few individuals. For example, concerning the Rwanda genocide, we know that one ambassador said one thing. The Security Council did not fail; the Security Council is just a table with chairs around it. The question is, how can we institutionalize this individual responsibility?
Answer (Nelly Badaru): People do many things under cover of policy. In one instance, UN troops in the Congo entered Uganda and raped a 14-year-old girl. Three men were identified but they were transferred and no justice was served. Individuals should be made to pay, and organizational policies should be evaluated. Make sure somebody is held responsible within an organization.
Q: How do communities define accountability and are their definitions taken into account when we design our systems?
Answer (Mary Anderson): If we say we are there to help, we have to listen to what they say they want. In Aceh, when we were conducting the Listening Project, many people said they held the international community accountable for a system of corruption that had lasted a long time. We pondered this: the system had existed before, so how could we have done otherwise? They said, “Talk to village leaders, and the military. But if you go to them and work out a system for distribution and then find out it didn’t happen as planned, go back and point out the difference. Work within the system, but help us hold local leaders accountable.” In dealing with this particular issue, the communities were more interested in having international staff than local staff because international staff had no local political affiliations.
IMPACT MEASUREMENT AND EVALUATION
Q: What is good enough in terms of impact measurement given the Do No Harm framework?
Answer (Mary Anderson): “Good enough” is trying to listen systematically and carefully to people’s judgments and analyses.
Q: In the humanitarian sector, there is rapid staff turnover and programs are short, yet measuring impact is a long-term endeavor. What can we measure and what can’t we measure, and who should do this?
Answer (Mary Anderson): Many agencies continue to exist after staff leave. If the agency is still there, other staff can still ask the questions. If the agency leaves too, communicate with agencies that are still there. From time to time, go back and see what people are telling us about the impact of the work we did previously.
Q: You have given mixed message about evaluations. Is evaluation being devalued in the new DEC framework?
Answer (Brendan Gormley): No, evaluation is not being devalued but should be seen instead as part of a bigger toolkit. There should be a way of triangulating formal monitoring and evaluation with a wider range of accountability tools. There is a tension between learning and public accountability and evaluations have a role in both, but might be better suited to learning.
SECTION IV: PERSPECTIVES FROM THE SPEAKERS 27 COORDINATION AND COLLABORATION
Q: How should we interpret [Mukesh Kapila’s] vision for a post-coordination future? How different is this from a competitive NGO community?
Answer (Mukesh Kapila): I wish we lived in a world where we did not need coordination. I wish people would simply agree to work under a common framework, negotiate a division of labor, agree on an acceptable level of transaction costs, and then get on with it. There are enough IT tools for sharing information. Abolish meetings and communicate on websites. Everybody does not have to be on board.
Answer (Nelly Badaru): If everyone has a common goal and understands why they are there, we do not need coordination meetings. Coordination is an extra cost, an extra burden, and not one of the targets for which we measure performance. What does this mean? We should return to the basics of why are we out there. What has gone wrong that we feel the need to fix through coordination?
Q: What are the risks of not collaborating?
Answer (Gib Bulloch): Everyone doing everything is not sustainable in the long term, nor is it an efficient use of resources. The free market does brutally drive efficiency in the private sector and has lead to massive consolidation in the past two decades. However, the number of humanitarian NGOs has expanded six-fold in the same period, leading to large-scale duplication of effort. As something of a surrogate for the free market, donors, and the public will continue to be more demanding, which will drive greater efficiencies and inter-agency collaboration. Organizations that do not adapt risk being left behind as more complex coalitions evolve, drawing on collaboration both within and outside the humanitarian sector.
ECB
Q: ECB still takes a “top down” approach. How can ECB move to an approach that puts claim holders in the driving seat? Accountability is more than just provision of information, it is about building communities’ capacity to analyze information.
Answer (Charlie Ehle): ECB was developed by headquarters-based staff of the member organizations. We tried to adjust this top-down approach by doing more to engage field staff; for example, we created a field reference group to help develop the Good Enough Guide.
Answer (Ton van Zutphen): From the very beginning, we did not consult enough with the field. In the second phase of ECB, the plan is to have much more field input, focusing on five countries. People in the field will drive decision-making.
28 SECTION IV: PERSPECTIVES FROM THE SPEAKERS SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS
Editor’s note: Following the presentations and panel discussions, participants formed groups based on conference themes they wanted to debate in order to produce actionable recommendations for ECB agencies. There were four groups in total: one on accountability, two on impact measurement, and one on collaboration and coordination. Each group was led by an external facilitator. There were only a few hours for discussion before groups were pressed into defining recommendations to be presented in plenary. Once all the recommendations had been presented, all participants then voted on the four recommendations they considered the most important.
Some contextual information has been provided for each of the major discussion topics.
ACCOUNTABILITY
Introduction As the scale and complexity of humanitarian crises grew in the 1980s and early 1990s, NGOs and the Red Cross Movement developed and agreed standards and principles of performance. These included the Red Cross Code of Conduct and the Sphere standards, and later the Humanitarian Accountability Partnership principles. However, the extent to which performance has changed remains unclear, and mechanisms are not in place to determine how well these standards are applied in different emergency settings. The Tsunami Evaluation Coalition report states “…many international agencies did not live up to their own standards with regard to respect and support for local and national ownership…and they frequently failed in the modest objective of informing disaster-affected people in a timely and comprehensive manner.” 5 Many NGOs are clearer now about the practical steps to ensure accountability to people affected by disasters. These are:
N Provide people they serve in emergencies with appropriate information about an agency’s response; N Ensure their participation in the planning, implementation and evaluation of humanitarian response; and N Establish grievance mechanisms for them to complain if the response is falling short of what was promised or not reaching those entitled to assistance.
Agencies, however, face constraints in implementing these accountability practices. These include a lack of wider organizational commitment, resulting in little funding to deliver accountability measures, and to train staff on them. In order to improve performance, the NGO Impact Initiative recommended that NGOs undertake regular audits to measure their accountability to local populations. — From the “Big Issues” briefing paper, Translating Standards into Practice
A Culture of Accountability Editor’s note: The group on accountability started off by discussing individual versus agency accountability. Early on in the discussion, participants shifted the focus to developing a culture of accountability within organizations. The group developed two recommendations.
Accountability is a broad term. We could spend much time dissecting its legal, ethical, and managerial implications. Easier, perhaps, is to think of the purpose of accountability as a way of ensuring that we are meeting the needs of people. That being the case, we need to look at how to judge individuals and agencies on the appropriateness of their actions for meeting the needs of disaster-affected populations, even if it means making tough decisions about the way we deliver aid. Agencies must be able to depart when they are not needed or when other entities could do the job better. For example, in Lebanon, there was one large INGO that could have left after a couple of weeks, because local organizations were quite capable of taking over at that point. But the agency stayed on because money for the response was still coming in from the public and the agency’s senior leaders wanted it to be visible. How then can agencies move away from a preoccupation with expending money allocated for emergencies, bidding for funding for interventions outside of their expertise, jockeying for visibility, and reporting to donors on the quantity and scale of the response, etc., to an overriding focus on whether their responses are meeting the needs of disaster-affected people in the most effective way? This will require a change in the way agency staff think. In short, it will require the creation of a culture of accountability. Senior executives will need to lead this cultural change. We should say to CEOs, “How do you know that we are
5 Tsunami Evaluation Coalition report: pages 17-18
SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS 29 delivering on all the things that we have signed up to?” At present, it is not that our leaders are unwilling to think and talk about this; they have not been Senior executives will provided with much evidence that their agencies are not systematically need to lead this cultural addressing needs in the most effective way. This in turn requires a real change. We should say to commitment from the field to collect this type of information and send it upward. CEOs, “How do you know For example, field staff need to collect information on things that their that we are delivering on organization doesn’t have the capability to do and how these needs are being all the things that we have met by other agencies. Peer reviews could also help agencies pinpoint weaknesses in their service signed up to?” delivery. For example, they could critique one another as to when their interventions become redundant and it is time for local actors to take over.
ACCOUNTABILITY: RECOMMENDATION 1
Build a culture of accountability to meeting the needs of disaster-affected populations as expressed by those populations, from the field all the way up to senior management, and provide resources accordingly.
This could include training of field staff, developing systems that support accountability, incorporating accountability requirements into performance reviews, changing the language that senior management uses, and changing donor expectations.
A Common Framework for Accountability There are several quality and accountability initiatives, each with its own standards for accountability. The humanitarian sector needs better integration of these initiatives and their standards. This will require the leadership of senior executives. The IWG could come up with recommendations for which standards should be universally adopted. If the seven IWG agencies drive this, it will have an impact on the humanitarian sector. One possible opportunity to further this dialogue with senior leaders could be at the ECB conference in April 2007.
ACCOUNTABILITY: RECOMMENDATION 2:
Undertake a formal process led by CEOs to consider practical steps for greater integration of existing quality and accountability initiatives and their standards in light of the costs of the current fragmentation.
Beneficiary-Centered Accountability (Quality Assurance) Editor’s note: Beneficiary-centered accountability was one of the topics discussed by the accountability group. The discussion addressed questions like, “How can we promote communities’ definition of what aid agencies should be accountable for?” “How do we build the capacity of claim holders to hold us accountable?”
Groupe URD has done a lot of work to define quality for beneficiaries. It uses 12 criteria to define the quality of interventions. URD went through exercises with refugees to find out what they considered to be important in a humanitarian context. The refugees said respect came first. The Humanitarian Accountability Partnership International has done similar work over the past six years and come up with almost exactly the same results. There should be opportunities to share findings from these initiatives with ECB. People in the field are asking for quality—that we respond to their needs without doing harm. Other sectors, such as manufacturing and healthcare, have started to work on quality assurance decades before us. We are still struggling to translate our commitment to quality into practice in the field. This is not just a question of tools or standards but of institutional change. First, you need the commitments at the highest level, and then the resources to execute the commitments, and then you conduct pilot training. But it takes a certain level of organizational readiness for standards and tools to be mainstreamed. Organizational change of this dimension is an individual journey for each of the agencies and collaboration may not be useful for some aspects of it. However the ECB collaboration has helped the organizations be more accountable to one another and reduce prejudices.
30 SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS IMPACT ASSESSMENT Editor’s note: There were two groups on impact measurement. One focused on the importance of systematic listening and the other on setting boundaries and building skills and knowledge of staff. Each came up with recommendations related to their discussions.
Introduction Impact measurement approaches that NGOs can realistically apply are being developed although field application is limited. This is partly due to funding and technical constraints. It is also partly due to media and donor pressure to respond quickly in emergency settings where information is far from complete and time to understand program outcomes is lacking. Over the last few years, NGOs have become increasingly interested in the use of customer survey methods and other ways of asking aid recipients to judge the effectiveness of emergency interventions. These approaches do not rely on baselines, except those that can be constructed after the fact by people served in an emergency, nor indicators or rigorous quantitative systems. Instead, they rely on a periodic assessment of recipients, who are asked if the emergency response programs made any difference to their lives. Their views are recorded and analyzed, and the findings shared publicly. — From the “Big Issues” briefing paper, Translating Standards into Practice
Systematic Listening to Disaster-Affected People Discussions around impact measurement can be difficult and debates have been going on for years about what are we measuring, for whom, and for what purposes? Are we using the information immediately, to improve our current programming? Are we gathering it to be able to report to donors what we have done with their money? There are several dimensions of impact, e.g. short- versus long-term, intended versus unintended. Which will we prioritize? How can we separate the impact of one agency’s work from that of the overall response? Will the need for quantitative rigor and universally agreed indicators hinder the gathering of intuitive, qualitative knowledge? Some things are certain. Agencies do already collect a lot of information. Yet it is not necessarily the right type of information for understanding impact. We have to do better than providing information on outputs such as how many wells we have dug and how much food we have distributed. NGOs need to be more purposeful in how they collect and use information. If we first answer the questions on who will use the data and how, this will then guide us in what we collect, how we collect it, and who we involve in this process. We must go to field staff with a simple approach, explaining clearly what we mean by impact, what information we are asking them to gather and why. We must also demonstrate our commitment to measuring impact by integrating it into our performance management systems. Donors are also our clients. What type of information do they want from us and how can they help us in addressing constraints to impact measurement? They can help by providing resources to get impact measurement done. They can begin to demand more than reports on outputs. Agencies can influence and educate donors about the need for better impact measurement. Above all, are we truly listening to the people we claim to be helping? We need to be doing so systematically. Perhaps we are making them feel better by asking them questions that show concern for what they think.
IMPACT MEASUREMENT: RECOMMENDATION 1
Because:
N Agencies have improved on delivering and monitoring outputs but have not listened enough to people’s views on impact; N Humanitarian action has impacts in multiple dimensions; and N People we intend to help know which dimensions are most important to them,
we recommend that:
Agencies prioritize systematic listening to disaster-affected people as the key way to assess impact and develop staff competencies and methods to do so.
SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS 31 Mapping and Building Staff Knowledge and Skills to Measure Impact
It is true that there is limited time for data gathering during an emergency response. But more significantly, our staff do not have all the skills and knowledge needed to measure the impact of our work. At the same time, we are not sure how much knowledge and capacity already exists. We must be more purposeful about providing training along with other ways of addressing these capacity gaps and of setting up knowledge management systems for sharing what we know about our programs and about impact measurement itself. The impact debate is not new. What can we learn from what has been gathered? We must be better at learning from the experiences and methodologies of those who have expertise in impact measurement within the humanitarian sector and within other sectors.
IMPACT MEASUREMENT: RECOMMENDATION 2
Set boundaries to the use of impact measurement by:
N Mapping the spectrum of impact measurement (the “what,” “why,” and “how”); N Exploring the applicability of other sectors’ models; and N Defining “why” to do impact measurement, “how” and “for whom” at each level.
IMPACT MEASUREMENT: RECOMMENDATION 3
Build knowledge and skills for measuring the impact of emergency programs by:
N Using the impact measurement “map” (Recommendation 2) to identify knowledge, skills and attitudes required; N Mapping existing capacities to determine gaps, and prioritizing which to address; and N Doing something about it!
32 SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS COLLABORATION AND COORDINATION Editor’s note: There was one discussion group that combined collaboration and coordination.
Introduction
Collaboration: NGOs often act independently in their humanitarian and development work and at times compete for the same pool of public resources. While there are benefits to autonomous action such as speed, agility, and innovation, there are drawbacks as well. Criticisms include duplication of effort, and misuse of scarce resources. Yet to collaborate effectively takes much commitment, time, and effort, resulting in high transaction costs. Its importance was highlighted in the NGO Impact Initiative consultation process where participants “…cited as best practice those programs in which international NGOs, local NGOs and local governments worked together.” 6 Yet in reality, collaboration difficulties plagued the tsunami response, and collaborative mechanisms between government, UN, donors, and NGOs were far from ideal.
Coordination: NGOs are frequently criticized for failing to coordinate during emergencies. Mechanisms for sharing information are rare and the information is often redundant. There are many stories of local people being asked the same set of questions by multiple NGOs and of NGOs setting up programs in the same geographical areas though they may not actually be needed. The Tsunami Evaluation Coalition report said that this lack of coordination led to a “fragmented response.” Many tsunami studies and relief workers “…identified lack of coordination as a major, if not the most crucial factor that contributed to problems including critical gaps in service delivery and duplication of efforts; …with the biggest gap in coordination being between the international NGOs and local NGOs.” 7 Such studies advocate for more effective coordination as a route to better use of resources and fairer and more impartial delivery of aid. — From the “Big Issues” briefing paper, Translating Standards into Practice
Strengthening Collaboration and Coordination Editor’s note: This group produced ten recommendations. They are listed in the order of importance set out by the group.
Collaboration and coordination can be engines for improvement or a drain on resources. They can bring us new ideas, better ways of doing things, and new relationships. They can also mire us in bureaucracy, endless meetings, and slower decision-making. The key is to understand when and how to work together. There are times when competition is good. For example, it makes for better proposals to choose from, and thus better program quality. The need for agencies to have their logos visible on camera during an emergency response cannot be ignored either. We feel that collaboration and competition can exist together if we have an understanding of when and how each is needed. Because we are investing so many of our resources and expectations into collaboration, it would be helpful to have a cost-benefit analysis and evaluation of coordination and collaboration efforts. We need to understand the benefits of not only the end products but the process itself. How can we ensure that we collaborate and coordinate effectively? As we seek to understand the nature and practice of collaboration, we must build on experiences. For example, we know that there are times when we need a strategic vision for collaboration and coordination and times when we must simply do what seems expedient for the task at hand, e.g. sharing information. We know collaboration and coordination thrive where there is trust. At the same time, because we work in emergencies, we know that it makes sense to build trust in the preparedness phase. Also, we know collaboration and coordination happen differently at the field and headquarters levels. We need to build on this knowledge to develop models for collaboration and coordination that work both at headquarters and at field levels. ECB should incorporate lessons from collaboration from its first phase into its second phase. If the IWG agencies can create models for collaboration, they can share these elsewhere. There are calls to collaborate throughout the humanitarian sector. As IWG agencies, we must have a clear idea of how we will work with the larger humanitarian community. We must build the skills of our staff in collaboration and coordination, starting with how to share key information. We must also urge donors to support collaborative processes.
6 Clinton/NGO Impact Initiative: page 5. 7 Clinton/NGO Impact Initiative: page 3.
SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS 33 COLLABORATION AND COORDINATION: RECOMMENDATION 1
With respect to incorporating local actors (NGOs, local government, etc.) with a focus on emergency preparedness, we recommend using InterAction’s NGO Field Cooperation Protocol for field coordination with next steps to include:
N Reviewing the existing protocol N Reviewing the tools N Updating, rolling out
COLLABORATION AND COORDINATION: RECOMMENDATION 2
With respect to cost-benefit assessment of collaboration and coordination, we recommend developing outcome-driven performance indicators (hard and soft) to allow tracking of their added value (e.g. creativity, learning, flexibility).
COLLABORATION AND COORDINATION: RECOMMENDATION 3
With respect to emergency preparedness, we recommend that donors invest in supporting collaborative approaches/processes/skills within the humanitarian community.
COLLABORATION AND COORDINATION: RECOMMENDATION 4
With respect to ECB’s Phase II, we recommend that IWG principals adopt and articulate a dynamic collaborative framework that can be applied at each level of ECB and shared externally for free adaptation, adoption, and discussion.
COLLABORATION AND COORDINATION: RECOMMENDATION 5
With respect to building trust and respect among partners, we recommend that this needs to be done in the emergency preparedness phase as much as possible. Since response happens, IWG agencies must take the initiative to assess and reinforce pre-existing collaboration and coordination mechanisms and supplement where necessary so that information on capacity and plans can be shared transparently.
COLLABORATION AND COORDINATION: RECOMMENDATION 6
With respect to modes and models of collaboration, we recommend exploring the conditions where creating a vision for collaboration is necessary and /or desirable, and where it is best to just “go with the flow.”
COLLABORATION AND COORDINATION: RECOMMENDATION 7
With respect to competition, we recommend that competition be recognized for its value to the sector in raising the quality programming bar. We can be competitors while engaging in positive collaboration.
34 SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS COLLABORATION AND COORDINATION: RECOMMENDATION 8
With respect to coordination and collaboration, we recommend actively developing teams with collaborative capacity.
COLLABORATION AND COORDINATION: RECOMMENDATION 9
With respect to field versus headquarters collaboration, we recommend developing a process that is inclusive of different levels.
COLLABORATION AND COORDINATION: RECOMMENDATION 10
With respect to collaboration progress to date on ECB’s Phase I, we recommend that Phase II clearly articulates and incorporates lessons learned.
SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS 35 TOPICS FROM “OPEN SPACE” DISCUSSIONS Editor’s note: During the final two hours of the conference, participants discussed topics that had emerged during the workshop in small groups with rotating membership. These notes capture perspectives and recommendations on issues of concern to participants, all of whom work within the humanitarian sector. Main messages are highlighted to aid follow up action by participants.
HELPING BENEFICIARIES TO HOLD US ACCOUNTABLE
Where in the ECB process is the beneficiary? It all seems very agency and donor-led. Even in situations where there is a common goal of alleviating suffering, there are inherent tensions. Beneficiaries may want NGOs to stay a long time; a donor like ECHO may want them to quickly leave, or do local purchasing, even if it impacts quality. An NGO’s work can be better evaluated when it is measured against others’. However, people affected by emergencies often have no alternative to the services we give. And we have a superiority complex with beneficiaries. We don’t always think they are ready to participate. We could just give beneficiaries money instead of services and let them choose. Or better yet, we could give them vouchers, because these can be tracked. Oxfam did a study on cash transfers to community groups and it was very successful. We need to buy locally; we need to enhance monetization so we can bring more cash/vouchers into emergency settings. We don’t trust beneficiaries, and we need to. The bottom line is that NGOs are afraid to change and want control. Even the term “beneficiary” comes with assumptions, when in reality aid benefits many, including NGO staff, government, and donors. People affected by disasters are not always right. But they have a right to demand quality services from us. How can we help them hold us to account? Training and other types of capacity building may not be necessary, as the ability to hold others accountable is normal. However, we have to educate the beneficiaries about what we do and manage their expectations. It is good when donors go out and do unannounced evaluations. However, donors tend to evaluate differently than beneficiaries would. We can do things like having beneficiaries double-check evaluations before we publish them. We could have beneficiaries evaluate our work, instead of the outside evaluators we currently use. We could use the UN evaluation of NGOs as a model of how we should go to our stakeholders and offer them our resources for an evaluation led by them with methods chosen by them. It would represent a comprehensive 360˚ approach.
What are the disincentives to practicing accountability to beneficiaries? People affected by 1. It costs substantial time and money. disasters are not always 2. It exposes NGOs failings and our risk-averse nature. 3. It demands organizational change, and agencies may not be sure how to right. But they have a manage change. right to demand quality 4. We are not sure how to do it. services from us. 5. It might necessitate the effort of an entire sector and be hard for individual organizations to do on their own. 6. Leadership may not be sufficiently committed to it. 7. The incentives are not clear enough. 8. Traditional systems of beneficiary accountability are not suitable for emergency response; we have to find simpler ways. 9. Beneficiary accountability is more long-term and loosely defined—somewhat contrary to the emergency mentality. 10. It threatens personal job security within the status quo. 11. There are no explicit rewards for doing it well. 12. There are many other stakeholders to whom to be accountable. 13. The pressure to be visible and fast surpasses the pressure to be accountable.
The challenge for IWG agencies is to address these disincentives so that people affected by disasters can hold them to account for the emergency services they provide.
Main Message: NGOs should find ways to cede some control to people affected by disasters, such as allowing them to evaluate the services we provide to them.
36 SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS UPWARD ACCOUNTABILITY
NGOs find upward accountability confusing. There are many different demands from donors, particularly European and American donors, and their reporting requirements frequently change. It would be helpful for NGOs to be less focused on donors and on spending money and be more focused on talking with people affected by disasters. At the same time, upward accountability is important and NGOs have new donors to whom we are not being accountable. We don’t think enough about making information available to the public. When we accept money from an institutional donor, that makes us accountable to the taxpayers of that donor. Thus, politicians request information from us because they have to provide answers to their citizens. How can impact measurement relate to upward accountability? This is a question with added resonance for NGOs as they increasingly become donors. But the indicators we use, such as those expressed in logframes, are not capturing the importance of what we are doing. Can logframes be modified to reflect this? They should be living documents informed by our projects. Because donors and NGOs are trying to accomplish similar things, it is likely that donors would be amenable to revising indicators during our projects. The important thing is to agree on processes of working together with donors, rather than on the indicators to be used. This process could include defining questions from donors that would help NGOs to be accountable and better capture the importance of what we are doing. If donors ask for NGO accountability, they should be flexible in giving NGOs the resources to be more accountable to all stakeholder groups, particularly people affected by disasters.
Main Message: NGOs are not being sufficiently accountable to their donors. At the same time, their task is complicated by the many different demands from donors and the fact that current donor requirements are not conducive to assessing impact. Because donors and NGOs have common goals, they need to agree on the processes, rather than the indicators, that would help NGOs be more accountable to all.
INTEGRATION OF DIFFERENT SETS OF STANDARDS
It is a familiar quandary: the emergency worker wondering which quality assurance tools and standards to use when. There are HAP-I, Sphere, and ALNAP tools and standards, plus internal agency requirements, to name some. The problem is illustrative of the fact that though agencies have signed on to several standards, realistically, there is little hope of them being able to adhere to all. How can agencies help to integrate these standards and make it easier for their end users—the field workers? It should start with better collaboration among these initiatives. Co-location should be considered—for example, quality and accountability initiatives could share a building in Geneva, thus facilitating information sharing and learning. They could also share a virtual space such as a common website or portal. In many ways, it is natural that there are so many initiatives at present. A new industry often gives rise to many small companies. Over time, there is a selection process and market forces decide which stay. But do NGOs exert enough “market pressure”? “Do their staff constantly ask the question, “Why are we incurring the costs of multiple initiatives?” “Why do we attend so many meetings?” The extra value gained from multiple perspectives may outweigh the costs, but it is important to keep asking the question. Everyone agrees that the current situation is untenable. We have enough, even too many standards—so how can we go about selecting which are the priorities for our agencies. What can we do to better attain them in the field? Some standards are relevant at some moments in time and in some contexts. How can we distill common standards as Sphere has done in creating simple principles that apply across contexts? If we define these common elements, can we create some indicators? We do not have to present the details of this integration to the field; these integrated initiatives could instead be incorporated into an agency’s standard operating procedures. Country directors could have a more comprehensive knowledge of the standards and assign different responsibilities to various units.
Main Message: There are currently too many standards and tools for ensuring program quality. They should be prioritized and common principles distilled. (Note: ECB has started this work by defining “The Basic Elements of Accountability and Impact Measurement.” See Annex 4).
SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS 37 CERTIFICATION AND ACCREDITATION
Certification is a common practice in other sectors such as industry. It is meant to The growing trend build the trust of the client and thus increase business. It also protects the public, e.g. towards certification for in healthcare, doctors must be accredited to practice. In all sectors, quality arose first, other sectors means that then a few years later certification and accreditation followed. It is part of the it will be impossible to process of professionalization. There is a difference between certification and accreditation: you certify that someone is doing what they were trained, or prevent it in the accredited, to do. humanitarian sector. The growing trend towards certification for other sectors means that it will be impossible to prevent it in the humanitarian sector. Certification is not a threat—it can be done, it gives credibility, and brings improvement. As a start, it can highlight the gaps between our commitments and our ability to carry them out. For example, we can say that we have the intention to be accountable to beneficiaries in our policy documents, but we may not necessarily have the capacity in the field. NGOs are already undergoing some levels of certification. For example, InterAction certifies child sponsorship schemes. People in Aid has a code, and member agencies complete a social audit to certify they are practicing the code. HAP-I has started a certification process. HAP-I members must agree on how many standards they will apply, what these standards mean, and how they will fulfill them over a given period. This process is at an early stage and a number of HAP-I members, including OFADEC and World Vision Sri Lanka, have agreed to do a certification trial. Note that NGOs’ financial systems are also certified when they complete financial audits. A challenge with NGO certification is that we do not have agreed systems in place to measure our program quality. Another challenge is making sure that certification has impact in the field, as something determined at headquarters does not necessarily ensure capacity in the field to carry it out. For certification to have an impact at field level, there will have to be a clear agreement on standards. They may be internal or external and may change depending on the context, i.e. first phase of emergency, etc. There are dangers to a one-size-fits-all approach. There should also be clarity on which standards are being applied to an individual and which to an organization. Individual standards can be put in individual performance plans with associated consequences if they are not fulfilled. Accreditation may not be as difficult as we think. As there are a limited range of tasks within the humanitarian sector, we should be able to create a foundational program with a curriculum such as “understanding humanitarian principles,” and “basic project management.” What we then need is a body that will assess the quality of training offerings and perhaps even allocate points to different programs that count towards an eventual overall qualification.
Main Message: There is growing pressure for certification within the humanitarian sector. Obstacles include lack of mechanisms to measure program performance and lack of agreement on the standards to use for certification. However, certification is not a threat and may not be as difficult as we think. Much progress has already been made in this direction.
FIELD LEVEL CAPACITY
How do we strengthen and make use of the human resources that exist in the countries we work in, such as our staff, local and national governments, local partners, and networks? We need to look beyond workshops and trainings to find creative ways of building skills and empowering the people we are working with to think strategically. One way to do this is to create effective partnerships at the local level. We should start planning handover to local partners from the very beginning. We can use disaster mitigation to build the capacity of local partners along the continuum from preparedness to relief to development. External drivers such as the Tsunami evaluation are helping us make the case for capacity building. We need to include others in ECB. For example, in Indonesia, where we work on disaster mitigation, we can move beyond the IWG agencies. We need to ask those who we think need to have their skills and knowledge developed what they want, recognizing that we can learn as much as we can teach. We need to help local NGOs scale-up so that they have more options after they have enhanced their capabilities and resources. Individuals also move on to new things when they have new skills. The opportunity for people to develop themselves is what keeps them there. In general, we need to put our words into action. If we are serious about building local capacity then we need to do it!
Main Message: NGOs need to find creative ways of building capacity in the field, such as planning early handover to local partners. If we are serious about building local capacity then we need to do it!
38 SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS COLLABORATION
What does collaboration require? And what are the lessons on collaboration for Phase II of the ECB project? Certain aspects of the ECB collaboration have worked well in Phase I. For example, there is trust between the advisors and they call one another. However, this collaboration does not necessarily exist in the field. ECB could replicate this model of collaboration at headquarter levels, within the regions and in the field. ECB could also connect with local collaborations in the field. For collaboration to be successful in Phase II, it must be properly resourced. For example, there should be full time advisors within agencies, or two agencies could share an advisor.
Main Message: There are several lessons from collaboration during the first phase of the ECB project that can be applied to the second phase. For example, collaboration models that work well at headquarters do not necessarily work well in the field and will need to be adapted to meet field realities.
LEARNING BEYOND THE ECB AGENCIES
The ECB project had an objective to create tools that would be shared with the humanitarian community. To date, there have been attempts to share the tools and products beyond the seven agencies and most are available on the ECB website 8. However, the project is young—several of the products and studies have only just been developed, and some are not yet ready to share. ECB should become more inclusive in its design processes. It would be good if others outside the seven agencies have a chance to influence the development of tools, as those not involved in the process are less likely to buy into the product. The primary challenge is However, to what extent will being more inclusive slow down the collaboration to get ECB learning or cause it to disintegrate? ECB could avail of some networks like ALNAP and shared among the InterAction to share and gain learning. There will also be different audiences and thousands of workers different distribution channels depending on the product. However, the primary within the seven challenge is to get ECB learning shared among the thousands of workers within the agencies. seven agencies. ECB has gained an important experience about the process of collaboration itself. This may be a different experience for Phase II, as there will be greater involvement of non-ECB members. The quality of the products themselves will help in their distribution. People continue to use tools like CARE’s commodity manual or World Vision’s global positioning catalog because they find them useful.
Main Message: One objective of the ECB project was to create tools that would be shared with the humanitarian community. This needs to be done more widely.
8 www.ecbproject.org/publications.htm
SECTION V: GROUP DISCUSSIONS AND RECOMMENDATIONS 39 SECTION VI: CONCLUSIONS
Editor’s note: These conclusions were developed by the editor to assist those seeking to use this report for advocacy purposes.
The Challenge Participants discussed many issues at this workshop, including impact measurement, upward accountability, accountability to disaster-affected populations, quality assurance standards, certification, collaboration, and coordination. Though they are discrete topics, they all relate to helping humanitarian agencies deliver better services to disaster-affected populations. NGOs know that they are not fully applying the humanitarian standards they signed on to, and though they are doing much good work, the quality of services they provide to disaster-affected populations needs to improve. The time has come for more decisive action. The core business of the humanitarian sector is to help people. Our standards and principles help to make us professional and effective. But we must not focus on the technical aspects of their application at the expense of their underlying purpose—human interactions that promote respect, uphold dignity, and facilitate the sharing of power.
Next Steps Both organizations and the individuals that constitute them have a responsibility to turn standards into practice in the field. The seven agencies of the Emergency Capacity Building Project are large humanitarian agencies that combined, work in over a hundred countries. They have used the ECB project’s collaboration model with relative success to make progress on accountability and impact measurement. If these agencies create changes within themselves, they have the ability to influence the way the humanitarian sector provides assistance. But they must lead through action. This will require their leaders to commit to and mirror the accountability changes that they expect to see from their field staff.
40 SECTION VI: CONCLUSIONS ANNEXES
ANNEX 1: AGENDA
OBJECTIVES:
1. Share learning from the inter-agency work completed so far on accountability (to local people), impact measurement, coordination, and collaboration. 2. Identify lessons and challenges within these four themes. 3. Make recommendations for future work by the Interagency Working Group agencies and the humanitarian sector.
December 4
13:00 Buffet Lunch
14:00 Opening remarks & presentation on ECB2
14:20 Keynote speaker – Mukesh Kapila, Special Representative of the Secretary General and Director of the Policy and Planning Division, IFRC
15:00 Four-Member Panel discussion Panelists: Brendan Gormley – Accountability to local people Mamadou Ndiaye – Coordination Mary Anderson – Impact Measurement Gib Bulloch – Collaboration
16:15 Plenary – Question and Answer Session
17:15 Aid Recipient Response – Nelly Badaru
17:30 Group Formation and Identification of Issues for following day
19:00 Cocktail and Dinner with Guest Speaker – Hugo Slim, Centre for Humanitarian Dialogue
December 5
8:30 Working Groups: Review of issues and identification of top 3 recommendations for IWG consideration.
11:15 Coffee Break
11:45 Plenary Debate: Representatives from each of the four working groups form a panel and debate their top recommendations.
12:45 Plenary: Small “buzz groups” to identify outstanding questions for afternoon session.
13:00 Buffet Lunch
14:30 Open space discussion groups on outstanding questions
16:30 Conclusions in plenary
17:00 Closing Remarks
ANNEXES 41 ANNEX 2: PARTICIPANTS’ EVALUATION OF THE CONFERENCE
Twenty-seven participants completed an evaluation. The following is a summary of what they wrote about the conference. We would like to thank them for their feedback as it allows us to be accountable to all participants.
1. WHAT PARTICIPANTS LIKED MOST ABOUT THIS CONFERENCE Most rated all aspects of the conference as ‘good enough’ (please see page 43). Aspects rated the highest were conference organization and the relevancy of the presentations. Participants wrote words like, “I liked most the ‘presentations from senior well informed people about key recent developments.’” “Interesting, and some excellent speakers.” “I liked the input from experts including the private sector.” Another wrote “the provocative, challenging, experienced panel on the first day set the tone for later discussions and recommendations.” Most rated the workshop either excellent or good enough on inviting the right people. Words written on what they particularly liked were “the variety and diversity of people participating,” “the knowledge and experience that was present,” plus “the diversity of opinions and perspectives” though “yet with the shared goal of making our services better.” Other comments written on what participants liked most varied and included: “The time for real intellectual debate and discussion which underpins practice”’; “having the forum and the time to discuss and debate interesting issues in our field,” “the chance to talk about critical issues with key people in the sector,” and the “openness to critical debate.” A few specifically mentioned liking the whole content of the conference especially the impact assessment part of it. Another liked the accountability and quality discussions and another the group discussions around accountability, impact assessment, and coordination/collaboration, saying “these were very helpful and could have possibly used more time.” Some wrote that the event was good from a networking perspective. A few liked the open space and outside space to meet up and discuss with friends and colleagues. Comments written by only one person included: “it expanded the circle of ECB knowledge and learning,” “the engagement with those involved in accountability and impact measurement,” and “the opportunity to influence and learn from a wide forum.” On the process side, comments under what was liked most by one person included the “facilitated sessions,” “work groups,” “serious workshop,” “catching up on the status of ECB2,” and “enjoyed the simplicity and location of the venue.”
2. WHAT PARTICIPANTS SAID COULD BE DONE BETTER NEXT TIME Nineteen out of the twenty-seven respondents wrote views on what could be done better next time. Most comments related to the conference’s objectives, outcomes, and recommendations. Some said “clarify even more the objectives and expected results. We were a bit lost time to time. Recommendations for whom? For ECB? For a wider audience? Provide a ‘clearer focus on objectives of discussions as they tended to ramble a bit,” “a bit more clarity on intended outcomes and the parameters within which we should make recommendations,” “drive to more specific recommendations on next steps,” “an opportunity to develop recommendations further than was possible in the time allowed,” and “make recommendations more specific and SMART.” Another wrote that “next time recommendations should be explained better to those making them. A recommendation needs to be specific and actionable and within our realm of control. There should be a clear recipient of that recommendation. I was frustrated with some of the recommendations as they were too ‘fluffy’ in nature, weren’t very clear, didn’t have a recipient or weren’t actionable by those in this room. We always need to be realistic.” One wrote extensively on the above saying, “the three objectives articulated in the materials weren’t achieved or really followed, at least in the manner I had expected. I wanted to know how the seven agencies had come together as ECB, what the benefits and pitfalls were. This would have been useful for making recommendations for the future. There was not enough evidence-based discussion on what worked or didn’t in Phase I so it was difficult to suggest useful recommendations to address existing issues and priorities recommendations…. I would suggest in future that the intended result of the workshop be clearly articulated – what kind of recommendations are you hoping for? Recommendations for whom? ECB staff or IWG members? The larger humanitarian community? What will these recommendations be used for? Objectives for ECB for Phase II? Background info for use in a future planning process?” All other views varied. Statements related to content included “lessons learnt got missed out,” “provide more chance to learn about the bigger picture of ECB: goals, objectives, etc. of other initiatives, perhaps an optional session for those who know less about it,” “maybe more specific analysis on certain topics.” Provide “greater cohesion between beneficiary accountability and impact measurement,” and “ensure good ideas that got one less vote and didn’t make it on to the flipcharts don’t get lost.” Statements related to process included “I’m not sure that a consensus-driven workshop is the best way of tackling complex organizational and conceptual issues though they are very good for dialogue and enhancing ownership. Would more structure/breakdown of key conceptual issues help?” “Bring the spirit and energy of day one on to day two,” “build on presentations on day one,” “more rigorous facilitation in some groups during day two,” and “more variety within each day, i.e. mix up speakers and small group sessions.”
42 ANNEXES Statements related to participants included a question about whether the senior executives of the IWG agencies were present, plus a question on “Why are non-ECB folks here? What did you want of us? Clearer purpose, e.g. endorse Good Enough Guide, comments on ECB Phase II, etc., would have provided more specific outputs, though maybe not impact.” Other statements were “involve more southern NGOs,” “please next time invite more local NGO representatives especially from Africa and Asia,” and “hold outside Europe/North America to allow greater field staff and southern NGO involvement’.” A few wondered if this had advanced the ECB agenda and whether it was useful to those who organized the event. Another said that next time, provide “a sense that what the participants come up with will be seriously considered. Report back to agencies on responses to the meeting as evidence that what we’ve come up with has been considered—accountability.”
3. OTHER VIEWS EXPRESSED Eleven participants expressed other views. A few were common, such as “well done,” ‘‘good enough,” “thanks to all” “really enjoyed thinking hard about accountability for two days. Thanks for the huge work involved in bringing this group together. Please keep the whole initiative simple and focused which it has been to date but feel it may become overcomplicated and hijacked by purists,” “appreciated effort behind workshop even if it was unclear,” and “well timed with the ALNAP event.” Other views were about the speakers and participants and included “the diversity of the speakers was remarkable,” “not all the speakers offered useful insights, some were excellent, some were not,” “wish we had heard directly from donors,” “would have been good to have more field representatives and less from HQ,” “more chances to share actual learning among participants and what people in the other agencies are doing around a certain issue.” And “having a competent ‘beneficiary voice’ provided a valuable reality check throughout.” One wrote “take into account the Francophone/Anglophone gap on these issues, e.g. the word accountability does not exist in French; and perhaps behind this take into account cultural differences on these issues. If not, the existing gap will increase.” One said that the venue was great and another that it was quite difficult to get to, there was a mosquito in their room and it was noisy outside during the night. Finally, one said the “food was very good” and another “the food was bad.”
4. HOW YOU RATED THE WORKSHOP ON
Achieving objectives Providing relevant background Inviting the right people Excellent 11% information Excellent 44% Good Enough 78% Excellent 48% Good Enough 52% Not Good Enough 11% Good Enough 48% Not Good Enough 4% Not Good Enough 4% Meeting your expectations Facilitating sessions Excellent 15% Making relevant presentations Excellent 30% Good Enough 74% Excellent 27% Good Enough 63% Not Good Enough 11% Good Enough 73% Not Good Enough 7%
Organizing all aspects of the event Excellent 56% Good Enough 44%
Thank you so much for your feedback. We will use it when planning subsequent events and in the follow-up from this meeting.
ANNEXES 43 ANNEX 3: LIST OF PARTICIPANTS
UN LOCAL NGOS Jeffrey Marzilli World Food Program Fr. Anthony Fom Development and Peace, Nigeria Olivier Cossee Food and Agriculture Heng Aik Cheng Mercy Malaysia Organization of the UN Hilda de Bojorquez A-BRAZO (El Salvador) Susanne Frueh UN Office for the Coordination of Humanitarian Affairs NETWORKS AND OTHERS Wayne MacDonald United Nations Children’s Fund Alex Jacobs MANGO Desiree Bliss Fritz Institute GOVERNMENT AND DONORS Hani Eskandar SPHERE Anette Haug Norwegian Agency for Jonathan Potter People in Aid Development Cooperation Kathrin Schick VOICE David Hajjar Office of U.S. Foreign Disaster Linda Poteat InterAction Assistance Maurice Herson ALNAP Frank Thomson AusAid Nicholas Stockton Humanitarian Accountability Harriet Spanos United States Agency for Partnership - International International Development Rachel Houghton Tsunami Evaluation Coalition (and Magdalena Lesjak Swiss Government Overseas Development Institute) Michael Ahrens Germany Government Tom Longley Aid Workers Network Michel Van Bruaene Consultant, European Véronique de Geoffroy Quality Compass/URD Commission Humanitarian Aid Department SPEAKERS Petra Hofs Dutch Ministry Brendan Gormley Disasters Emergency Committee Carolyn Miles Save the Children US NGOS AND IWG AGENCIES Denis Caillaux CARE International Beatrice Teya World Vision International Gib Bulloch Accenture Development Ben Nicholson Tearfund UK Partnership Benoit Miribel Action Contre la Faim Hugo Slim Center for Humanitarian Dialogue Carlotta Sami Save the Children US Mamadou Ndiaye Office African pour le Caroline Loftus Save the Children US Developpement et la Cooperation Charles-Antoine Hofmann British Red Cross Mary Anderson CDA Collaborative Learning Charlie Ehle Catholic Relief Services Projects Clare Smith CARE Mukesh Kapila International Federation of the Red Dane Fredenburg Catholic Relief Services Cross and Red Crescent Societies Daryl Martyris Save the Children US Nelly Badaru Access Development Associates Eleanor Monbiot World Vision International Emma Roberts Save the Children UK FACILITATORS AND STAFF Erynn Carter Mercy Corps Ana Hiltunen ECB – Staff Capacity Initiative Guy Sharrock Catholic Relief Services Ana Urgoiti Facilitator Ivan Scott Oxfam Christophe Lanord Facilitator Jock Baker CARE International Emma Jowett Facilitator Liz Stone Caritas International Greg Brady ECB Mark Janz World Vision International Jim McAllister World Vision International Mary Pack International Medical Corps Malaika Wright ECB – Accountability and Impact Monica Rodriguez Catholic Relief Services Measurement Initiative Nan Buzard American Red Cross Mark Hammersley ECB – Staff Capacity Initiative Pat Johns Catholic Relief Services Matthew Bannerman ECB Paul M. Majarowitz Mercy Corps Paul Currion ECB – Information and Technology Peter Giesen Médecins Sans Frontierès - Holland Requirements Initiative Rigoberto Giron CARE Pauline Wilson ECB - Accountability and Impact Robert Schofield Medair Measurement Initiative Sara Smith International Rescue Committee Sarah Gerein ECB - Accountability and Impact Saskia Carusi Oxfam Guatemala Measurement Initiative Steve Claborne Mercy Corps Sean Lowrie Facilitator Teresa Murray Action Contre la Faim Ton Van Zutphen World Vision International
44 ANNEXES ANNEX 4: THE BASIC ELEMENTS OF ACCOUNTABILITY AND IMPACT MEASUREMENT
The Basic Elements of Accountability and Impact Measurement are the foundation on which Impact Measurement and Accountability in Emergencies: The Good Enough Guide was developed. The Basic Elements listed below were drawn up by representatives of the seven agencies of the Emergency Capacity Building Project at a workshop in Nairobi in February 2006.
BASIC ELEMENTS OF ACCOUNTABILITY
At a minimum, humanitarian project staff should:
1. Provide public information to beneficiaries and other stakeholders on their organization, its plans, and relief assistance entitlements.
2. Conduct ongoing consultation with those assisted. This should occur as soon as possible at the beginning of a humanitarian relief operation, and continue regularly throughout it. ‘Consultation’ means exchange of information and views between the agency and the beneficiaries of its work.
The exchange will be about:
N The needs and aspirations of beneficiaries N The project plans of the agency N The entitlements of beneficiaries N Feedback and reactions from beneficiaries to the agency on its plans and expected results
3. Establish systematic feedback mechanisms that enable: N Agencies to report to beneficiaries on project progress and evolution N Beneficiaries to explain to agencies whether projects are meeting their needs N Beneficiaries to explain to agencies the difference the project has made to their lives
4. Respond, adapt, and evolve in response to feedback received, and explain to all stakeholders the changes made and/or why change was not possible.
BASIC ELEMENTS OF IMPACT MEASUREMENT
Impact measurement means measuring the changes in people’s lives (outcomes) that result from a humanitarian project, striking a balance between qualitative and quantitative data. At a minimum, humanitarian project staff should:
1. Establish a basic description (profile) of affected people and related communities.
2. Identify desired changes, in negotiation with affected people, as soon as possible.
3. Track all project inputs and outputs against desired change.
4. Collect and document individual and community perspectives through participatory methods in order to: N Increase understanding of what change they desire N Help establish a baseline and track change
5. Explain methodology and limitations to all stakeholders, honestly, transparently, and objectively.
6. Use the information gathered to improve projects regularly and proactively.
ANNEXES 45