Accountability and Aid Effectiveness Research in Global Health 11

Elvira Beracochea

Contents Introduction ...... 262 Why Does Aid Effectiveness Research Matter in Global Health? ...... 262 What Is Accountability and Aid Effectiveness Research in Global Health? ...... 269 What Are the Principles of Aid Effectiveness and Accountability that Guide Research? ...... 270 What Kind of Accountability and Aid Effectiveness Research Is Needed to Advance the SDG Agenda? ...... 272 What Can AAE Research Do to Contribute to Global Health Practice in the SDG Era? ...... 274 How to Conduct AAE Research ...... 277 AAE Research Methodology ...... 280 How Would an AAE Research Department Work? ...... 281 Effective Steps to Conduct AAE Research ...... 286 What to Do with AAE Research Results? ...... 289 Conclusion ...... 290 References ...... 291

Abstract Accountability and aid effectiveness (AAE) research accounts for three main global health outcomes: first, the delivery of effective global health solutions; second, the efficiency or “value for money” of these solutions to donors and recipient countries; and third, the sustainability of these effective solutions to continue working towards the country’s health goals after the aid ends. AAE research is based on five principles: country ownership, harmonization, alignment, management by results, and mutual accountability. AAE research is difficutl because of the same reason that effective global health practice is difficult, that is, the global health and aid architecuture are complex and fragmented. However, it is possible to focus on

E. Beracochea (*) Realizing Global Health, Fairfax, VA, USA e-mail: [email protected]

research questions at country level, measure effectiveness, efﬁciency, and sustainability, reduce fragmentation and duplication, and aim for cross-national effectiveness comparisons. This chapter will discuss why this type of research matters and describe examples of how effective and accountable global health practice has demonstrated to be so far. Next, we will discuss mixed research methods used to measure accountability and effectiveness. If the HIV/AIDS (Esser, Glob Public Health 9:43–56, 2014), Ebola, and 2020 COVID-19 pandemics have served for a greater purpose, it is to show that AAE research is an essential part (or at least should be) of global health practice. This chapter will show you how to design and implement this type of research and use it to improve your own performance as a global health practitioner, as well as contribute to improve the results of global health practice.

Keywords Global health research · Aid effectiveness · Accountability · Sustainability · Universal health coverage · Health systems

Teaching and Learning Objectives The three main objectives of this chapter are: (1) to explain why accountability and aid effectiveness matters and why there is need to have a global research agenda to systematically conduct AAE research, (2) to describe what is studied in AAE research, and (3) to teach you how to conduct this type of research. The speciﬁc objectives you will achieve after studying this chapter are:

1. Understand the deﬁnition and principles of Accountability and Aid Effectiveness 2. Discuss the rationale and why these principles matter in global health 3. Discuss various approaches to AAE research in global health 4. Learn how to design and conduct research in the ﬁeld of AAE 5. Learn practical application of AAE research in global health practice

Introduction

Why Does Aid Effectiveness Research Matter in Global Health?

Aid is effective when it achieves the desired effect or outcome (Beracochea 2015). The desired outcome of global health practice is a sustainable improvement in the health status of the population that receives the aid. If the aid does not help the country deliver better services that achieve better outcomes and the country’s health system has not been strengthened to keep delivering better services after the aid 11 Accountability and Aid Effectiveness Research in Global Health 263 ends, then aid has not been effective. I am referring to development aid. The outcome of effective humanitarian aid is to save lives and prevent further worsening of the country’s health status. Effective development aid, along with stable governance (Kabir 2019), helps countries to develop. Aid solves health service delivery and management problems, improves quality of care, and expands coverage of health services. It is about hitting the bull’s eye (Wickremasinghe et al. 2018) and doing so that you can explain what worked to achieve what outcomes (Beracochea 2018). For example, aid is effective when a HIV/AIDS project increases the detection of HIV+ persons and improves the quality of the health care delivery model so patients can sustain viral suppression. In addition, the country where these persons live is able to sustain the improvement and keep delivering better HIV/AIDS quality services. In short, the project can account for improved performance of the country’s HIV/AIDS program and of the country’s healthcare delivery system. In addition, the country continues doing it after the aid ends. The aid was effective. If not, the job is not done, and aid needs to continue and demonstrate that it is showing improvement in the right direction. On the other hands, effective humanitarian aid is aid that is provided during an emergency such as an Ebola outbreak, civil unrest, war, or an earthquake or Tsunami. This aid helps implement the emergency response faster and better, reaching larger numbers of victims, and preventing negative outcomes such as diarrhea and other diseases related to lack of drinking water, food, or lack of access to medical care. Thus, effective humanitarian aid should provide quality healthcare to victims and provide basic services to those affected, such as mosquito nets to prevent malaria, children’s vaccination to prevent vaccine preventable diseases, and family planning services to prevent unwanted pregnancies. Effective humanitarian aid provides lifesaving aid without which people would die or suffer unnecessary and preventable hardship or disease. Effective aid is evidence-based by design and requires effective planning and management to deliver the maximum return on the investment by a governmental or private donor that are affected by so many factors including rate exchange (Chansa et al. 2018). In short, it is not only what a donor-funded project does but also how it is done that leads to effective results. This matters, because effective aid needs to achieve three objectives:

Overview: Threefold Goal of Global Health Practice 1. Deliver effective global health solutions that solve the identiﬁed problems 2. Be efﬁcient, that is, demonstrate “value for money” of these solutions to donors and recipient countries 3. Be sustainable so that these effective solutions do continue working and helping the country achieve its health goals event after the aid ends

Effective aid accounts for its results and is subject of evaluation research. However, not all aid is effective and there is evidence of selective practices (Adedokun and 264 E. Beracochea

Folawewo 2017). Aid that is not evaluated cannot be determined to be effective, not matter how good the original intentions of the donor or the implementing stakeholders. Effective results also help fundraising and attract more support from donors (Karlan and Wood 2017; Metzger and Günther 2019a, b). Aid is funded by a donor and provided by global health experts in various ﬁelds. These global health experts usually come from more developed and richer nation. Their goal is to help counterparts in developing ones achieve one or all of the three objectives above. This aid is essential especially in the poorest and war-torn countries such as Iraq and Afghanistan (Dalil et al. 2014). Therefore, global health practice must account for solving the challenges for which aid is provided and deliver effective development and/or humanitarian aid. AAE research is part of the practice of global health and helps demonstrate that the global health practice and practitioners have indeed solved the problem at small scale at least. In addition, the effectiveness of the role of the global health experts, consul- tants, facilitators, trainers, coaches, researchers, and/or evaluators in global health needs to be questioned and measured as a matter of professional accountability. The continuous and transparent assessment of the performance of those of us working in global health will ensure our work achieves the desired outcomes. Although altruistic intentions to help those in need are usually the motivation for providing aid, foreign policy and political alliances are also underlying motivations for countries to be involved in giving and receiving aid. Understanding the ultimate purpose helps explain why some countries with smaller populations receive more aid than large countries with more people and bigger needs. Finding ways to advocate for equitable distribution and allocation of aid is the job of international organizations such as the UN, WHO, UNICEF, etc., and the professionals that work in them. How transparent and effective these organizations are is also part of AAE research. This chapter will show you why you need to gather evidence of effectiveness and account for the results of aid in global health. You will also learn what aid outcomes to account for and how to research and ﬁnd evidence of effectiveness and accountability. Also, you will learn how to demonstrate effectiveness in your global health work and to advocate for accountability of health outcomes of every global health interventions or projects you are involved. Global health is a great career that carries also the professional responsibility for effectiveness and accountability.

Why Does Accounting for Aid Effectiveness in Global Health Matter? International laws and treaties are the legal reason for accounting for aid effectiveness, for the results of our global health work, that is, effective and ineffective results and everything in between. In fact, accounting for the effectiveness of using aid funds is necessary to demonstrate the fulfillment of the right to development of the receiving nation. AAE research in global health measures how well aid has helped fulfill the right to health of the people in the recipient country. Accounting for aid and its results in global health includes accounting for the result of financial aid, as well as the sharing of technology or knowledge and expertise provided through various mechanisms, usually humanitarian or development projects that provide technical assistance to improve the health status of the people in the receiving country. Therefore, multidisciplinary teams are usually 11 Accountability and Aid Effectiveness Research in Global Health 265 needed to conduct AAE research and make use of various research tools form their fields. It is essential that everyone on the team be aware of the purpose of AAE research and the legal basis. Having a common foundation will help you apply epidemiological and social science tools as well as financial tools to conduct AAE research studies that make use of the right mix of qualitative and quantitative methods. Quantitative research methods help quantify and account for the effectiveness of the aid while qualitative methods help us explain why aid was or was not effective in fulfilling human rights treaties and conventions, as well understand the cultural differences and expectations and perceptions of those receiving and giving aid. In short, every global health organization and the work we do is to contribute to fulfill these international legislations.

What Is the Legal Basis for Global Health Practice and for Accountability and Aid Effectiveness? The legal basis for the provision of global health aid was first created through the Universal Declaration of Human Rights (UN 1948). The rights-based nature of our work in global health was further defined in subsequent declarations and international covenants that most countries have endorsed with some notorious exceptions. In short, there is large body of international laws and treaties that your global health work will help put into practice and fulfill. Aid is provided in the context of these legal documents. Therefore, you need to remember that all you do either global health practice or research is rights-based, that is, it is based on the rights of every human being on the planet, particularly the right to health, and contributes to fulfill those rights in the most effective way. I suggest you create a folder in your computer for these documents, and even print them and keep them in a binder near your desk. You will need to refer to these documents many times in the course of your global health practice and while conducting AAE research. Global health aid is rights-based and also provided in the context of the right to health as defined by article 25 in the Universal Declaration of Human Rights (UN 1948). The Committee on Economic, Social, and Cultural Rights (CESCR) further defined the right to health in more clear terms in the year 2000 in its General Comment 14 (UN 2000), which despite its unappealing name, it states that “Health is a fundamental human right indispensable for the exercise of other human rights. Every human being is entitled to the enjoyment of the highest attainable standard of health conducive to living a life in dignity.” General Comment 14 matters because the CESCR includes a body of 18 independent experts that monitors the implementation of the International Covenant on Economic, Social, and Cultural Rights by the UN country members. This includes the right to health. You need to read General Comment 14, print it, and keep a copy with you because it is the legal basis that will guide your global health work. Global health aid is provided in the context of humanitarian or development assistance through various treaties and agreements between countries and is – or should be – based on the Declaration on the Right to Development (UN 1986). This declaration was endorsed by most countries and established the right to 266 E. Beracochea development as a collective human right and it is centered on the right to self- determination of every human being. In just 10 articles, this declaration established the duty of all states to cooperate to help realize the right to development and to sustain their effort to ensure “effective international cooperation” to provide “countries with appropriate means and facilities to foster their comprehensive development.” With a sound right-based foundation, aid effectiveness and accountability are now the basis of your global health career. Now you know why we do what we do and why we account for our work. There is not a universally accepted standard code of practice in global yet. However, I hope in my lifetime, the principles and the tools of aid effectiveness and accountability you will learn in this chapter become common practice and routine in the work of the international health and development industry complex. Please keep a folder in your computer or print the documents and keep a binder to refer to them. Each is a few pages long, but you will discover every word counts and will guide your path towards more effective and accountable global health aid.

What Has Been Done About Improving Accountability and Aid Effectiveness in Global Health? You are now aware of the legal basis for working and doing in global health and for conducting AAE research in global health. There has been progress towards more effective and better accounting of global health results. However, despite the legal basis and international treaties, and the many conferences and meetings, progress towards global consensus on accounting for aid effectiveness has not been moving as fast and transparently as desired. There are a number of lessons to be learned so mistakes of the past are not repeated and what works is preserved and improved. Therefore, it is essential that you be aware of the main milestones and the lessons learned in the road towards more effective and from the past to avoid mistakes and learn from the lessons of the past 40 years as you prepare to conduct AAE research. The purpose for listing these milestones is to show you that global health professionals have been working to improve the effectiveness of our work for over 40 years with varied degree of success. The important message here is to learn from the past and not repeat its mistakes but ﬁx them.

Main Aid Effectiveness and Accountability Milestones in Global Health 1948 – UDHR. The 25th article of the Universal Declaration of Human Rights to which most countries are signatories established the right to health of every human being on our planet. How countries have gone to fulfill this right differs and for that reason, each country has different and more or less effective health system and each country’s people have different health status. Fortunately, research to compare the effectiveness of various health systems is a growing field. 1978 – HFA2000. In 1978, international conference on Primary Health Care (PHC) was held in Alma-Ata, USSR. The Alma-Ata Declaration established that PHC was essential to realize the right to health of every person and that PHC was urgent to achieve health for all by the year 2000. This strategy became known as HFA2000. However, the year 2000 came and PHC had not been yet implemented by 11 Accountability and Aid Effectiveness Research in Global Health 267 most countries. Many excuses and explanations were attributed to this failure, but not formal accountability structures were in place, except the ability of the UN to ask member countries to report. 1979 – CEDAW. The Convention on the Elimination of Discrimination Against Women addressed gender inequality and most of the social determinants that affect the health of women in our planet. Despite renewed emphasis on gender, most CEDAW articles still need to be enforced and many countries have not endorsed it, including the USA. In 1994, UNFPA held a conference on Cairo that brought 113 countries and set an agenda. Research to determine the effective implementation of this agenda has not been conducted despite several meeting held since then. The health status of women was partially addressed in the Millennium Declaration in 2000 (see below) because it called for countries to reduce by 50% maternal mortality. Research has shown that this target was not met and many disparities within counties disproportionally affect certain ethnic and racial minorities. More AAE research is needed to find effective solutions to women’s health. 1986 – DRD. Declaration on the Right to Development. I believe this declaration rules our global health work. More developed countries have the duty to help developing nations develop faster and better. Research on the effective implementation of the declaration needs to be further explored. 1989 – CRC. The Convention on the Rights of the Child is another international human rights document that guides our global health work. As with the CEDAW, many countries have not ratified it, including the USA, because it implies accepting the duty of fulfilling the right to health of every child. The USA has a federal program to protect children’s access to health care: the Child Health Insurance Program (CHIP) provides low-cost health care to children who are not poor enough to be covered by Medicaid, the free healthcare program. Research has shown that there are disparities across states and racial and ethnic groups that question the effectiveness of these programs. AAE research within and across countries is still needed to determine how best to fulfill the right to health of every child. 2000 – MDGs. The Millennium Declaration was a good document that included important commitments, especially the eight Millennium Development Goals, of which three goals were specifically health-related and others also related to improving gender equality and saving lives such as reducing hunger and increasing access to clean water. The MDGs also had a list of “Quick Wins” that would help accelerate their achievement, but these were not implemented consistently to achieve impact and most countries did not meet the MDG 48 targets. Research on the effectiveness of the interventions designed to meet the MDGs did not start until after 2005, the year of the first monitoring report on the progress of the MDGs and the year of the Paris Declaration on Aid Effectiveness and Accountability. As you can see, the international machinery moved slowly and it took 5 years to start monitoring the progress of a 15-year agenda. Fortunately, the international organizations associated with the Sustainable Development Goals (SDGs) learned from that and the UN created a monitoring toolkit and the SDGs are monitored annually. Here is the latest report: https://unstats.un.org/sdgs/report/2019/The-Sustainable-Development- Goals-Report-2019.pdf 268 E. Beracochea

2005 – PD. The Organization for Economic Cooperation and Development (OECD) (www.oedc.org) convened a meeting in Paris that led to the Paris Declara- tion on Aid Effectiveness and Accountability was a turning point in the practice of AAE research. For the first time, principles for the provision of overseas development assistance (ODA) were agreed upon. Applying these principles is at the core of AAE research in aid effectiveness and in global health effectiveness (OECD 2012). As it will be shown below, ensuring that donors and recipient countries apply these principles account for effective results has not been a straightforward mandate and there are still many global health organizations and professionals that do not practice them making research even more urgent (ADB 2011). See the section on the Paris Declaration below for more detail. 2007 – IHP+ and 2016 UHC2030. The International Health Partnership was created to improve cooperation to achieve the health MDGs. In 2016, the IHP+ was transformed into UHC2030 (www.UHC2030.org) to help achieve SDG3 and focus on strengthening health systems and achieve Universal Health Coverage (UHC). UHC is the main strategy to achieve all the other health targets of improve maternal and child health and preventing and controlling diseases. UHC2030 is an advocacy and coordination stakeholder organization (Bigsten and Tengstam 2015). It is to facilitate accountability and integration of interventions to improve UHC. AAE research to find ways to monitor the expansion of coverage is needed and well as definition of what the effective ways to do so. The UHC2030 has a vision document that lists a number of principles similar to those of the PD and a number of health system performance dimensions focused on improving service delivery, financing, and governance. The principles are:

1. Leaving no one behind: a commitment to equity, nondiscrimination, and a human rights-based approach 2. Transparency and accountability for results attract beneﬁciaries (McGee 2013) 3. Evidence-based national health strategies and leadership as the foundations for HSS 4. Making health systems everybody’s business – with engagement of citizens, communities, civil society, and private sector 5. International cooperation based on mutual learning across countries and development effectiveness principles

There are no indicators or data regarding the impact of the UHC2030 on its stakeholder’s performance or a selected number of countries is not reported in their website, nor is there a mechanism for coordinating coverage expansion activities. Therefore, the UHC2030 and its partners need to develop AAE research branch to monitor their progress towards 2030. 2010 – AAA. The Accra Agenda for Action called for accelerating progress and strengthening country ownership, a core principle of the PD. Having ownership over development by strengthening country capacity to manage development is an area of research that has not been developed in global heath yet. How effective is aid at strengthening a country’s health system, and its government 11 Accountability and Aid Effectiveness Research in Global Health 269 ownership and leadership of what happens in the health sector is a question that needs answer in many countries as the study about “Hitting the Bulls Eye” (Wickremasinghe et al. 2018) described below has shown. The AAA agenda raised the costly problem of aid fragmentation (Gehring et al. 2017)andthe need to increase value for money and accounting for results and recommended a number of practical actions. All these topics still require research 10 years later. In fact, global health would really benefit from have a list of evidence-based effective “Quick Wins” and effective ways to account for value for money and results. The meeting in Accra showed the limitations in health and other development sectors (Cabral 2008). There is evidence of a number of interventions that we know do work and save lives that need to be scaled up and included in every primary health care program in every country. These include vaccination, antenatal care, active management of the third stage of labor, antiretrovirals, breastfeeding, handwashing, mosquito nets, etc. 2011 – 4thHLF. Fourth High Level Forum on Aid Effectiveness. In a meeting in Busan, Korea, OECD also convened and helped reaffirm the PD principles and AAA again. The Busan meeting also produced a list of additional actions that were agreed for more effective international cooperation, particularly through South-South Cooperation and Partnerships. These actions among others included: improving the quality of cooperation, increasing the focus on results, and making effective aid part of effective and sustainable development, promoting theroleofCSOsandprivatesector,and combating corruption (Quibria 2017). There is still need of AAE research to measure the effectiveness of the Busan commitments. 2015 – AAAA. The Addis Ababa Action Agenda was the result of the Third International Conference on Financing for Development, which called for a Global Partnership implement the Sustainable Development Agenda. The COVID-19 pandemic has made the need for an effective global partnership even more urgent. 2015 – SDGs. The 2030 Agenda for Sustainable Development included 17 Sustainable Development Goals (SDGs) approved by the UN General Assembly in 2015. The UN monitors the SDGs and while progress has not been significant in the first 5 years, there have been improvements. This progress is at risk given the current COVID-19 pandemic that might cause high mortality in itself and for other causes due to reduced access to care for other patients for all other conditions. The 2019 SDG report (UN 2019) is a general report that does show gaps in coverage but does not account for disparities within countries.

What Is Accountability and Aid Effectiveness Research in Global Health?

AAE research is research based on the principles of the Paris Declaration on Aid Effectiveness and Accountability of 2005. Let’s see these principles and what type of research has been and is being conducted in the SDG era. 270 E. Beracochea

What Are the Principles of Aid Effectiveness and Accountability that Guide Research?

The Paris Declaration. In the year 2005, a number of donors, countries, and organizations gathered in Paris and drafted ﬁve principles of Aid Effectiveness and Account- ability. The principles are interrelated and are focused on accounting for effective results. These principles are the focus on AAE research. The principles are (Fig. 1):

1. Ownership. This means that receiving countries need to lead the development process. They must be in the driver’s seat and be empowered to put in place national development strategies with clear strategic priorities. This principle is the number one requirement for increased effectiveness (Martinez-Alvarez 2018). Bypassing the government does not lead to effective results (Chasukwa and Banik 2019). 2. Harmonization. This principle means that aid is provided through harmonized programs that are coordinated among donors. Coordination structures became a must with the proliferation of donors (Pallas and Ruger 2017), all trying to help in different ways. It is agreed that donors must conduct their field missions and country analytical work together with recipient countries and demonstrate effectiveness themselves (Minasyan et al. 2017). 3. Alignment. This principle requires that countries develop reliable national fidu- ciary systems or reform program to achieve them and that donors must align their aid with national priorities and provide the information needed for it to be included in national budgets, allowing for gender-specific needs (Holvoet and Inberg 2014). Coordinated programs aligned with national development strategies are to provide support for the country’s capacity development. Donors are to avoid creating parallel systems and are to use the systems that already exist in

Fig. 1 Principles of the Paris Declaration 11 Accountability and Aid Effectiveness Research in Global Health 271

recipient countries: for example, education and health systems, information, procurement, government administrative, ﬁnancial, and HR systems. Effective public procurement can have an important impact on health outcomes (La Chimia and Trepte 2019). Aid is to be predictable and reliable so the country can count on the aid to be released according to agreed schedules, and that bilateral aid is not tied to services supplied by the donor. 4. Management by Results. Countries have transparent, measurable assessment frameworks to measure progress and assess results 5. Mutual Accountability. Regular reviews assess progress in implementing aid commitments. Research can focus on those that need help the most such as postconﬂict areas (Ssengooba et al. 2017)

The Organization for Economic Cooperation and Development (OECD) is a policy making partnership of countries with over 60 years of experience. Their motto is “better polices for better lives” and was the leader in AAE, particularly in health since convening the Paris conference in 2005 and subsequent meetings. A number of targets and indicators have been used to measure progress of implementation of the ﬁve PD principles (Fig. 2). However, by 2010, that is, 5 years after the PD, progress had been limited. And even, since Busan in 2011, OECD and the international aid industry does not seem to focus on the PD principles as the basis for cooperation and development. OECD no longer seems to spearhead emphasis on the PD indicators, and although, as an organization, its website does not show data of how much lives have been improved by its policies, it does have a health data base (https://www. oecd.org/health/health-data.htm) that is available for researchers to use. Since 2014, the OECD produces a report on what makes aid effective. The 2019 report (https:// www.oecd.org/dac/making-development-co-operation-more-effective-26f2638f-en. htm), which presents the 2018 results, showed that progress is not too good with the

12. Mutual accountability 2010 targets 11. Results-oriented frameworks 10b. Joint country analytic work 10a. Joint missions 9. Use of common arrangements or procedures 8. Aid is untied 7. Aid is more predictable 6. Strengthen capacity by avoiding Parallel PIUs 5a. Use of country PFM systems 4. Strengthen capacity by co-ordinated support 3. Aid flows are aligned on national priorities 2a. Reliable Public Financial Management (PFM) systems 1. Operational Development Strategies

-0.1 1

Fig. 2 Indicators of the Paris Declaration 2010 (Source: Elizabeth Sandor, OECD 2008) 272 E. Beracochea exception of improve mutual accountability and country planning, all other indicators have worsened:

1. Reaching the Sustainable Development Goals (SDGs) requires urgent action on effective partnerships, as called for in SDG 17. 2. Development partners’ alignment to partner country priorities and country- owned results frameworks is declining 3. Partner country governments have made significant progress in strengthening national development planning. 4. Forward visibility of development co-operation at country level is weakening 5. Strengthened public financial management (PFM) systems have not been matched with significantly increased use by development partners. 6. More systematic and meaningful consultations with development actors are needed both by partner country governments and development partners. 7. The enabling environment for civil society organizations is deteriorating. 8. Improving the quality of public-private dialogue (PPD) in partner countries requires increased capacity, strengthened relevance, and the inclusion of a wider range of private sector actors. 9. There is mixed progress in making development co-operation more transparent. 10. In response to the evolving development landscape and the ambition of the 2030 Agenda, mutual accountability mechanisms are becoming more inclusive.

In short, there is a lot of room for AAE research and the ﬁeld needs to develop to demonstrate what really works. In the context of the current pandemic, innovative approaches are now urgent. Fulﬁlling the PD principles in global health practice brings up a number of research questions:

• What is the purpose of Aid in global health? • Does Aid work to achieve global health goals or just to improve public health programs? • Does it matter how it is used and who takes ownership? • How can one organization harmonize with what others are doing?

These and other questions require a new paradigm to design and implement aid in global health, a paradigm that integrates the ﬁve principles and one that is systemic. Systemic means that aid helps develop and expand the coverage of the country’s health system and public and private health sectors.

What Kind of Accountability and Aid Effectiveness Research Is Needed to Advance the SDG Agenda?

The global health agenda has been reset by SDG3, which calls for ensuring healthy lives at all ages through the achievement of Universal Health Coverage (UHC) and a 11 Accountability and Aid Effectiveness Research in Global Health 273 number of measurable reductions in preventable morbidity and mortality by 2030. However, the fragmented architecture of global health practice and weak international coordination among countries has not allowed the development of a coordinated strategy and plan to achieve these targets. Most countries do not know what their baseline coverage rate is, how many of their people are not covered, and how they will be covered or with that services. Some countries have defined a basic or essential healthcare package or basket. How effectively is such package delivered is known in most developing nations. In conclusion, we do not know what the baseline is for UHC and we do not have a plan to achieve it. In addition, we are in the midst of a pandemic that has disrupted and weakened most health systems. Through its articles of constitution, the WHO is the international organization charged with coordinating and monitoring SDG3, but its funding and role have been put in question during the Ebola and Covid-19 pandemics. Strengthening the WHO might require political will and commitment to the mission of the WHO as it was defined in its constitution. In addition, more strategic AAE and transparent reporting of performance indicators of the assistance provided by WHO and other agencies of the UN facility to member countries. In addition to hundreds of bilateral and multilateral organizations working in global health, there are hundreds if not thousands of foundations and nongovernmental, faith- based, and civil society organizations that provide global health services and healthcare services. Lack of effective partnerships and stakeholder coordination makes it difficult to measure the effectiveness of various global health programs, projects, and interventions. Therefore, it is now almost impossible to attribute outcomes to either one of them or measure their contribution unless they are organized and coordinated to do and account for their contribution. For example, the outcomes of the effective coordination role of WHO with other UN agencies such as UNPD, UNAIDS, UNFPA, and UNCEF is not well understood or known yet. The evidence of the effectiveness of the World Bank’s health projects usually relies on the effectiveness of the country’s health information system, which sometimes tends to be inaccurate and incomplete. Therefore, it is confusing in this fragmented scenario how best to measure what is effective and how to account for health outcomes. If more effective coordination could be achieved across all stakeholders, AAE studies on the effectiveness of various global health programs could be done using secondary data collected in WHO’s Global health observatory (https://www.who.int/ data/gho). The GHO also depends on the quality of the data that countries’ heath information systems can produce, but with better coordination data quality problems (Breitwieser and Wick 2016) could be diagnosed and solved with the advent of digital health technology. There are probably indicators that measure hundreds for services and programs in the GHO that would allow hundreds of AAE studies. Despite data quality and system inefficiencies, the GHO and other existing databases are untapped sources of data for meta-analysis in AAE research. If aid is to help achieve SDG3, it has to be by supporting the development of effective health systems. The country’s health system must be the focus on global health aid efforts because health systems do not happen by chance. A health system is a complex organization of public and private healthcare delivery, academic and 274 E. Beracochea government organizations that need to work in a coordinated manner to meet the health needs of the country’s people. There is not a globally accepted simple paradigm that helps keep global health work simple and focused on its ultimate goal: UHC. The paradigm I have been using in my work in the last 10 years was presented in a book a number of colleague and I put together entitled “Improving Aid Effectiveness in Global Health” (Springer 2015). The paradigm is deceptively simple: project > program > system.It means that if effective health systems are the vehicle to deliver quality healthcare to every person, then, every project, initiative, or donor-funded organization working in a country must ensure that what they do contribute to designing and improving the effectiveness of the country’s health system. In short, every aid-funded project must demonstrate to have effectively improved how one or several of the country’s programs work so that the health system can deliver better quality and more services to cover as many people as possible. In addition, the project should be implemented in a manner that the health system, that is, a number of public and/or private health facilities and the local health authorities can continue to do so after the project ends. The effect should be measurable at least in the project-supported geographic areas.

What Can AAE Research Do to Contribute to Global Health Practice in the SDG Era?

Let’s see a couple of hypothetical examples of global health practice to show how research would contribute to improve outcomes. First, let’s imagine an effective family planning project. It would be one that demonstrates having improved the country’s national family planning program performance, as demonstrated through a number of improved indicators of the country’s family planning program. And consequently, the country’s health system is able to deliver better and more family planning services in the project areas. In our family planning example, a signiﬁcant number of healthcare providers, health facilities, districts, or even whole provinces report to deliver family planning services in accordance to the improved standard and evidence-based procedures and are able to do so after the project ends. The project was then effective and accounted for its results in terms of the improvement of the family planning program and the health system. The Project-Program- System paradigm has worked well when I conduct aid effectiveness research and helps show any gaps that may have not been addressed and that will need to be solved by the next project or another stakeholder, usually by the recipient of the aid. Here is another example. There is evidence from MDG and SDG reports that aid does work to reduce malnutrition. However, it matters how aid is used because it can also have negative effects. There are missed opportunities, wastages, unsustainable results, and increased coordination and management costs for an aid-recipient country that has to manage the aid provided by hundreds of different donors or donor-funded organizations, each with their own agenda that is not aligned with the country’s policies or systems. 11 Accountability and Aid Effectiveness Research in Global Health 275

Let’s imagine a country is suffering a severe drought and food shortages with the resulting famine and high acute infant malnutrition. A donor decides to help by supplying a large number of nut packets and delivers them to a local NGO for distribution. However, the donor does not provide funding for other activities except distribution of nut packets to malnourished infants. How effective is this aid? Here are some questions AAE research would answer based on the PD principles:

• Ownership and Governance: Did the donor ask the local NGO to ensure that the nut packets be distributed in accordance with country’s nutrition policy guidelines? Did it happen and how well was it? How much of the time of the National Nutrition Program staff needs to be involved to make this happen? What about if this donor is one of the 47 other donors supporting the Nutrition Program and there are only ﬁve staff in this program? Should the donors adapt to the MOH or the other way around? • Ownership and Harmonization: Was the Director of the National Nutrition Program involved in the design of the distribution plan so this supply can complement what other donors and the government have bought? What about the other children in the family that may be also malnourished? How will the health staff that are not familiar with the nut treatment be able to learn to use it? • Alignment: Is the NGO working with the health facilities in the districts where the project is being implemented to ensure that their staff are trained in the country’s policies and malnourished children are rapidly detected? • Management by results: Is the NGO monitoring that the number of children that are detected and the rate of acute malnutrition is going down? What about chronic malnutrition and stunting? • Mutual Accountability: Is the NGO accounting for the results obtained to the National Nutrition Program Directorate? Was the government able to account for the nut supplies in the country’s supply chain? • Effective sustainability: What will happen after the infant recovers from the malnutrition and goes home? Will he or she be back in a few weeks? How will the country continue the program after the donor’s supply ends? What is the donor and the NGO exit strategy? Did the donor help the country’s government include the nut supply in the next year’s procurement plan? Did the donor help the government create pool resources and build a local manufacturing plant that produces nuts and other commercial peanut-based products? Was the prevention and management of acute malnutrition included in the preservice training in all the country’s academic institutions so in the future every healthcare provider is able to promptly detect and manage infant malnutrition?

Progress in global health must be based on the evidence of effective global health aid and medical and health science. As the UN SDG2019 report showed, there is evidence that over the last 20 years or so, life expectancy has increased in most developing countries, infant and child mortality have decreased mostly due to community-based programs and expansion of immunization program. Access to quality health services and medicines has increased, along with poverty reduction 276 E. Beracochea

(Bourguignon and Platteau 2017), although the rate at which these had been improved has slowed down. The connection between health and poverty is well known so for that reason, the 17 SDGs matter to achieving health outcomes. There has also been a modest decrease in maternal mortality and in mortality due to new and chronic conditions, but the national average hide inequality in access and life expectancy within a country. Inequity and lack of access to surgical and specialized and emergency care are still a barrier most developing countries and rural and urban areas of middle and high income countries. In 2020, the COVID-19 pandemic is raging across the world and making the ineffectiveness of health systems to meet existing and new health needs has become more evident in many countries. Aid is likely to be reduced and be focused on humanitarian assistance and/or on achieving short term results and not on UHC. Research is likely to be stopped or delayed despite the fact that now is the time to really use AAE research to invest on aid programs that do both, deliver the desired outcomes of meeting the healthcare needs of COVID-19 patients and the rest of the population, while building resilient and sustainable health systems and not bypassing them. The goal should be to conduct research on how to sustain the current systems and consistently expand its coverage to meet the needs of COVID- 19 patients and the effective epidemic control measures. AAE research is based on the principles of the Paris Declaration and starts with a question. Here are some research questions that ideally would need to be answered during and after the COVID-19 emergency is over:

• Ownership: Is the MOH leading and coordinating the national COVID-19 prevention and control program? Do the aid-funded programs have a clear role in the program and are contributing? • Harmonization: Are all the aid-funded stakeholders using the same harmonized strategies and healthcare delivery interventions? • Alignment: Are all the aid-funded stakeholders using the same systems and reporting using the same indicators? • Results: How much has the aid-funded project or a donor improved the quality of care? Was aid used to improve coverage? Where and by how much? Has aid improved the capacity of healthcare providers to respond to COVID-19 cases? Who is performing better where? What percentage of the workforce remain to be trained? • Accountability: Are aid-funded stakeholders accountable for results in their respective coverage area? Are they sustainably strengthening the country’s institutions?

Despite pandemics and especially because of them, effective healthcare coverage and its expansion also need to be accounted for by every country, not only those receiving aid. There are two main barriers to achieving UHC and delivering effective aid and health services: first, there is the limited capacity of the health system management structures and second, there is the shortage of facilities and human resources that has not been addressed despite the call for UHC. A 10% annual 11 Accountability and Aid Effectiveness Research in Global Health 277 expansion would be required to achieve UHC by 2030. Accounting for effective coverage of quality healthcare has not been implemented yet and WHO does not monitor that yet. Lack of manpower (O’Sullivan 2015) is a barrier to expansion of coverage. There is already evidence that inefficient production and distribution and management of human resources has caused shortages of healthcare providers in most countries. The surge for paid and unpaid Community Health Workers to fill the gaps due to lack of facilities and trained staff has been the focus of research. The evidence shows that CHWs do help deliver health promotion and prevention services and a selected number of home-based care treatment and care services. The well-trained, super- vised, and supported CHW has proven to be effective but few countries have integrated the CHW in the health system sustainably or developed a plan to train and recruit the required health workforce in adequate numbers. How effectively CHWs are sustained has not been researched in most countries yet.

How to Conduct AAE Research

Now that you know why you need to conduct AAE research and what this kind of research entails, let’s see two practical scenarios so you can see how to conduct AAE research as part of your work as a global health practitioner. These are imaginary cases and any coincidence is just a coincidence. I have created these scenarios from a composite of projects from various sources. Please put yourself in the role of the lead evaluator or if that is too hard to imagine, pretend you are observing how the aid is provided. Your perspective as external research is very important to focus the research on answering the most important research questions.

Scenario 1 Setting: A donor designed a project to help improve the health system of a country. The donor considers this its flagship project and expects the project to have significant impact on improving the health of the people in a number of districts. The donor has allocated several million dollars to implement this project and hired an international organization to implement the project. Three years into thelifeofthefive-year project, the donor hired a team to conduct a performance evaluation and find out how well the project is being implemented and if anything needs to improve. AAE Research Findings: The evaluation found evidence that the project team has met all its milestones to date. The project has trained healthcare providers in a selected number of districts in the country on how to improve maternal and child health and HIV/AIDS and TB treatment and has worked with the Ministry of Health (MOH) authorities to develop a new health system improvement strategy and a new quality improvement (QI) policy. The evaluation also found that these documents are very comprehensive and meet international standards, but there is not an implementation plan for either document was not part of the project’s design. The NGO was only to help draft these documents. The QI and the new health improvement strategy 278 E. Beracochea are not part of the country’s annual health plan and budget yet, despite the documents having been approved a year ago. In fact, the new health system improvement strategy and the QI plan are not being implemented in the project-supported districts or anywhere else in the rest of the country. In terms of improving the health system performance, per their contract, the project had hired and trained over 3000 community health workers (CHW) to follow up HIV+ patients until their viral load was reduced and trace TB contacts. However, there was not a plan for how these CHWs would continue doing the job for which they had been trained after the life of the project. The project had met the target number of patients diagnosed HIV+. How- ever, although there was an increase in the number of HIV patients diagnosed and under treatment, the percentage of patients lost to follow up was almost 40%. It had been harder to follow up HIV+ children, who almost 50% had a high viral load in their last follow-up visit. There was an average of 3% increase in the number of children immunized and a 5% increase in the number of mothers that use antenatal and family planning services but not an increase in the number of mothers that deliver in the facilities in the project-supported districts. The interviewed healthcare providers reported to appreciate the training received by the project team, but when asked how they had applied the training, they reported not have the resources to put it into practice. When asked about the lack of implementation by the trained health staff, the project said that the government supervisors were in charge of the follow- up. The project team reported they had strengthened the capacity of the health workforce through training, but it was out of their SOW to help supervisors implement the training in all the facilities. They also said that the project’s scope of work (SOW) was to help the government develop the new policies, but the government had failed to implement them due to lack of funding.

Scenario 2 Setting: A project to improve the delivery of maternal and child health (MCH) services is being implemented by an international organization also funded by a donor. The project had achieved less than expected targets in the ﬁrst 2 years, so an AAE taskforce comprised of external evaluation experts, and donor and Ministry of Health (MOH) representatives has been created to monitor and evaluate how well the project has been performing and decide if corrections are necessary. AAE Research Findings: The project has created a partnership with the MOH and helped selected MOH staff review and update the National Safe Motherhood and the Child Health policies and program guidelines, proposed updates to the data collected by the DHIS21 system and helped develop next year’s annual work plan and budget to include the new MCH priorities. The MOH MCH program managers have worked along the project’s MCH experts to design and implement a baseline study of the country’s MCH program effectiveness to ﬁnd out what works and what is missing in the light of the new evidence-based MCH program

1DHIS2 is a web-based open source health information system used by many developing countries, and the main source of health statistics. 11 Accountability and Aid Effectiveness Research in Global Health 279 guidelines. The AAE taskforce studied this baseline study and sampled a number of facilities and worked with the Medical Faculty Research Center to conduct data collection and analysis to find out the reasons for the slow progress of the project. The baseline study findings along with the DHIS2 data had showed a large number of women completed ANC1 but only 22% of them had ANC4. The ANC1/ANC4 ratio showed that some districts were better performing than others, so the project focused on conducting research to find out what caused the difference and how to address the performance problems. With these findings, the donor-funded project’s annual workplan was jointly developed by the project team and the country’s MCH program managers at national and district levels to solve the problems identified. The project accepted to work on the worst performing districts to bring them up to par and that explained why the progress of the project-supported districts had been lower than the others. These districts had started well below the average national ANC1/ANC4 ratio. The sustainability of the new MCH program was safeguarded because the MOH had agreed to use the same improvement strategy as the project and adapted their annual workplan to jointly and progressively expand the new MCH program to 10 new districts every year and thus reach the rest of the country’s districts. The project also worked out an agreement with the National Medical Association, the Medical Faculty, and the Nursing Council to revise their training program to include the new MCH interventions, and also in collaboration and with the support of the project experts, they developed and implemented an ongoing MCH refresher program that would annually include 20% of the nation’s workforce and in that way update the whole workforce every 5 years. This target had been achieved in the past year of the project and it was on track for the next year.

Analysis These two scenarios show different aid approaches and AAE research, respectively, and how differently the PD principles were applied as well as different models of global health practice. The first scenario shows a donor-driven project that delegates implementation to a consulting organization that does not observe the ownership of the MOH or help strengthen its leadership. The project also did not align itself with the existing MOH programs and involved the country’s training institutions in their effort to develop CHW. In addition, there was no coordination with other organizations supporting the same programs with funding from the World Bank or Global Fund. The project showed to be focused on meeting all its deliverables and thus continue getting funds from the donor. The project had developed a new QI policy and health system strategy and had them approved by MOH and reached the numbers of people trained and the numbers of people tested and diagnosed, etc. However, the project was not also focused on the outcomes of those project inputs and outputs. The evaluation research did show the project had done what it was contracted to do and had the evidence to show it did meet it deliverables. In addition, the evaluation research also demonstrated there were a number of project design flaws that had prevented it from achieving the desired outcomes of those deliverables. When asked why they had not brought up these problems with the donor, they 280 E. Beracochea said a contract modification was very complex and had not pursued it. In summary, this was not an effective project. Despite the high number of patients treated, their care was not likely to be sustained after the end of the project, so the donor will have to continue providing this kind of aid for a long time or correct the design flaws. Thanks to the research findings, the design flaws were corrected, and a contract modification was signed that allowed the project to account for effective and sustainable outcomes. The second scenario shows a project that was designed to be accountable and effective. It was implemented in alignment with the MOH structures and leadership, was in harmony with other donors improving MCH, and had become a valuable partner helping the MOH improve its performance and do its leadership job better. Although the project had not met its targets in the first 2 years, it was on track to do it in the third year. In addition, impact of the project interventions had been found to be beyond the planned targets in the districts where it worked and this success had had an impact throughout the health system. The project had been accountable for its performance to the MOH and the donor. It was able to explain why it had not me all its targets and was able to show what it had done to respond to the country’s needs and get on track. The alignment with the MOH had shown the project’s impact beyond the districts it was supposed to focus and this would increase the sustainability of the project’s new MCH interventions to be sustained after the aid ended. In short, this was an effective project and the country’s MCH program is likely to continue performing well after the end of the project.

AAE Research Methodology

As the scenarios above have shown and the evidence from research in the last 20 years have shown (WHO 2001), AAE research is complex requires that address numerous challenges. For that reason, you will need to use qualitative and quantitative research so that your research can account for three main global health outcomes: ﬁrst, the delivery of effective global health solutions, second, the cost- effectiveness of these solutions to donors and recipient countries, that is, the value for money, and third, the sustainability of the effective solutions which led to sustainable and measurable outcomes and progress towards global health goals. This is my personal approach. There is no international consensus on the most effective way of practicing global health and conducting AAE research yet. In fact, there is no consensus on applying the principles of the Paris Declaration either. Global health experts are working on this. As you already know, AAE research is an essential part (or at least should be) of global health practice. It is the way to demonstrate that global health practice actually improves the health of everyone in our planet. In fact, I believe that every global health organization must endorse AAE principles and have an AAE research department that ensures the effectiveness of their work and accounts for their results. 11 Accountability and Aid Effectiveness Research in Global Health 281

How Would an AAE Research Department Work?

First, an AAE research department or tram should focus on assessing and accounting for aid to the country’s health system. The Project-Program-System paradigm must be demonstrated where aid is provided. This means that a global health project needs to improve the performance of the respective country’s programs and to improve the performance of the health system in their geographic area of responsibility in terms of the four main areas of performance improvement: health outcomes, quality of care, continuity of care and coverage. Most countries have a health system to fulﬁll their duty and achieve varied outcomes with varied degrees of accountability and effectiveness. Internationally, the World Health Organization helps UN country members through various programs to improve their health systems in order to protect and fulﬁll their citizens’ right to health. In addition to WHO, there are also hundreds of donors and organizations that have their own programs to help countries improve outcomes. Therefore, the PD principles of harmonization matters. AAE research needs to measure how well each donor is contributed to improve the various building blocks of the health system (WHO 2000) (see Fig. 3) as well as improving the coordination and organization of various sectors and levels of the health system. The main sectors are: government, public and private sector providers, academia, and professional asso- ciations. The four levels are: primary, secondary, tertiary, and quaternary, depending

Fig. 3 Health system building blocks (WHO 2000) 282 E. Beracochea on various degrees of medical technology and specialization that is required. The primary levels serve the needs of the whole population and the other levels serve increasingly smaller sectors of the population. An efﬁcient system allows for patients to move up and down the system according to their needs and ensures the continuity of care throughout the patient’s life (Rutherford et al. 2019). As the HIV/AIDS, Ebola, and COVID-19 pandemics have shown, there is no formal global coordination and accountability mechanism and no global consensus on the most effective way to practice global health, how to improve health systems, control epidemics and account for all the effort and investment made by all parties involved yet. Evidence-based AAE research is also hampered by the lack of effective health information systems and the lack of sharing of information across borders. This lack of an effective global health information system requires that AAE research gather data from primary sources which is more expensive and time- consuming than use of secondary data to analyze trends, predict disease patterns, and assess effectiveness of various programs and interventions. Second, AAE research must “follow the money.” Research needs to ﬁnd out where the aid funding is being spent and how allocation decisions are made (Easterly and Pfutze 2008). Equity is the ideal, so everyone receives what they need in proportion to their need. Again, follow the PD principles and ask: Is funding allocation donor-driven or in partnership with the country’s MOH and other government agencies? Economic aid helps countries grow (Arndt et al. 2015; Maruta 2019) even when not linked to good polices in the recipient country and even in “unfavorable policy environment.” The evidence shows that there are various ways to help the health sector improve and save more lives. In order to increase effectiveness, they are:

(a) Aid as an increment to government efforts. This is not part of the national income (outside the funding pie or in parallel pie) but hopes to pick up what the government cannot do (what they don’t know or cannot afford to do). Aid is seen as humanitarian assistance and the donor is in control. This aid is usually time-limited. The purpose is to help the poorest or most affected to get pie. (b) Aid as a complement to government efforts. This aid is invested in selected programs and geographic areas of country (still aid is managed in a parallel pie, some get pie, some don’t, because aid is not usually based on need). The purpose is to demonstrate how to “do it” right to the government. The idea is that the government does not perform well because they do not know how, and they need to be shown how to do it correctly. (c) Aid to sustain and develop government efforts. This aid is an investment with unknown return (expanding the pie). The focus is on getting results, with various degrees of coordination and success. (d) Aid to develop the civil society, faith-based organizations (FBOs) and NGO sector but not the government sector, which is supposed to be working well by now. The purpose is to expand the pie for coverage expansion and improve the effectiveness of the nonproﬁt private sector. 11 Accountability and Aid Effectiveness Research in Global Health 283

Aid is invested in the management of the health system, its health workforce, its facilities at various levels, and the coordination of the various sectors to advance the country’s health goals and policies. Aid funding is sometimes invested as part of the government basket and sometimes is provided in kind in the form of technical assistance or consulting services (Annen et al. 2016). In AAE research, our job is to show the results of various types of aid received and their effectiveness. Third, focus on gathering and analyzing existing monitoring and evaluation (M&E) data and data from the health management and information systems. Most AAE research is conducted in the context of monitoring and evaluation (M&E) studies that are funded by the donors that also provided the aid. The purpose of these studies is to measure the performance and/or the impact of a global health project (Haque et al. 2017). The donor that is funding the project and the recipient government through the MOH have different degrees of involvement in these M&E studies. Someday, hopefully not too long, data from health information systems and data collected through systems embedded in all we do in global health will be gathered by AAE research departments and AAE findings will flow as fluidly and transparently into dashboards in real time just as well as credit card records can be tracked online. But we are not there yet. There are not globally accepted AAE standards of practice yet or a central repository of consolidated results where global health professionals can find the answers to the questions that keep us awake at night: is this intervention working? Is it the most cost-effective use of aid? Is it scalable and sustainable? M&E research is usually conducted to answer a number of evaluations questions. These questions are usually developed by the donor or organization that funds the research with input from various stakeholders and not necessarily comply with PD principles. It is the evaluators’ job to ensure that. Most questions are aimed at finding out what is effective and how effective the implementing team and their project have been and not about outcomes, cost-effectiveness or sustainability. When we design and evaluation, there are, therefore, questions of effectiveness of how well the project is or has been at achieving the objectives for which it was designed, whether the original project design was effective, how cost-effective and efficient the project has been in the use of the funds received; questions of accountability and of who is accountable to whom, and questions of sustainability beyond the life of the project. In addition, like any human organization, projects have human relationship and performance issues that range from intercultural differences to personality clashes between donor and project staff, between the project and the MOH staff, and between staff within these organizations that if not detected and addressed effectively, they may hinder the project’s performance and the evaluation. Confidentiality of in-depth informant interviews will need to be maintained at all times. For this reason, the AAE team needs to be not only proficient in research methodology, be sensitive to personality and management styles, and also play the role of good brokers between donor and recipient country, and effectively facilitate the discussion of the findings and their interpretation to ensure the maximum benefit of the investment in the evaluation or AAE research. 284 E. Beracochea

Effective and continuous communication between all parties involved in and affected by AAE research is much of a science as an art and requires maturity of the AAE team to not let AAE discussions derail into a blame and shame game. Objectivity and a focus on learning from successes and mistakes needs to be maintained and modeled by the AAE team at all times, from the research design stage, through the finding’s presentation and application stage. This is where transparency, integrity, and professional ethics come in. I will not dwell on these as they are beyond the scope of this chapter but remember that 20 or 30 years from now when you retire, you want to look back on every AAE assignment and have no ethical regrets. You may have lessons learned of things you would have done differently, but not regrets having been transparent and done the right thing for all involved, especially the people of the country that received the aid or were supposed to benefit from it. AAE is easier to be enforced through governmental organizations such as USAID or through international ones such as UNICEF or UNDP. However, the websites of most foundations or NGOs will not show evidence of how effective they have been in improving health in a country where they may have been working for decades or show evidence that the interventions they promote have actually achieved improved patient outcomes and are sustained after the life of their projects. Most donors do not fund “after the project” impact evaluation. I know of only one in my 30 years of global health practice, and it was a very useful evaluation that showed important design flaws that only could have been detected in hindsight. The Global Fund (https://www.theglobalfund.org/en/) is one of the few organizations that having been designed for performance, awards performance-based grants, and rates them on their website, where data and graphs on every country can be accessed. It would be great to be able to integrate these data with the countries’ data bases and conduct AAE research. In addition, these countries receive aid from other donors and foundations and NGOs that do show how they have been able to “graduate” a certain number of facilities, districts, and even whole countries. Some of these countries have been able to become donors themselves, such as Brazil, India, Russia, and China, and share their lessons learned. There is need of a coordinating mechanism and a repository of AAE research evidence. Then, the evidence could be used to inform global health practice, maybe in real time too. In health, WHO would be the natural place for such integrated AAE repository. After all, that is the mission for which WHO was created as stated in its article of constitution (WHO 1946–2006): “the attainment by all peoples of the highest possible level of health” (article 1), and “to direct and coordinate international health work” (article 2a). In fact, all the functions listed in article 2 of the WHO constitution would require the evidence gathered through well-coordinated and focused AAE research teams. M&E research is the closest we have to AAE research we have so far and there is a lot that can learned from M&E research if more meta-studies and cross- country studies were done regularly and strategically. It is good practice to monitor and evaluate the performance and impact global health projects even if 11 Accountability and Aid Effectiveness Research in Global Health 285

AAE research is not conducted. Most evaluations are performance evaluations, impact evaluations are few. In my experience, most end-of-project evaluations have the objective of gathering data to ascertain results that will serve as baseline to design the next project. In any case, M&E research is a good practice and most donor government organizations conduct systematic evaluations. Private foundations and NGOs might also conduct M&E studies, but their findings are used for internal decision-making and not shared to add to the body of global AAE evidence. For example, you can go to www.dec.usaid.gov, the USAID development clearing house and find the report of every evaluation and other documents that has been conducted of the projects in any country where USAID works. But you cannot usually find the same in the website of foundations and NGOs. Most articles are informative, share successes, and are posted for marketing purposes. It is my hope that this book you hold in your hands will set a new standard in global health practice and that AAE research be systematically coordinated and conducted and its findings implemented to help countries achieve universal health coverage by 2030. In 2009, the US government joined other countries in the G20 summit and endorsed Aid Transparency. Realizing that transparency is difficult when no one is ultimately responsible for accounting for it and there are no tools to ensure it, the US Congress passed the Foreign Aid Transparency and Accountability Act in 2016. The act led to the creation of the foreign assistance dashboard that can be found at https:// www.foreignassistance.gov/ in an effort to coordinate the development work more than 20 agencies. For some time, there were two dashboards and confusion as what each would do (Ingram and Sally 2018), but in 2020 they were consolidated. The website shows every appropriation and what is was for but not results or value for money. One still wonders: Are countries better because of these investments? And what percentage of the population benefited and what will happen to them when the funding ends? Does this funding complement what hundreds of private foundations also invest in developing nations? If yes, how? These are more questions can only be answered thought research, which will eventually and hopefully inform new policies, acts and practice. Fourth, stay up to date with AAE research reports and analyze the evidence. In the absence of a global repository of AAE research, you will need to keep your own database of evidence and bibliography, at least in your area of expertise. In most donor organizations and large NGOs, it is good practice to have a Knowledge Management Unit and that helps to keep track of what projects your organization has implemented or is implementing and what is effective. It is not good to keep doing the same thing in a country over 20 or 30 or more years without having the evidence of what does work. Someday, there will be effective knowledge management and AAE research tools that allow us to analyze large databases and compare the effectiveness of various programs and interventions across countries. Unfortunately, we are not there yet. You need to stay informed and use your judgment when reading published and unpublished AAE research findings. Read published articles in your area of expertise. When working in another country, I suggest you review the evidence from M&E studies in that country. Not all donors share reports transparently. 286 E. Beracochea

Effective Steps to Conduct AAE Research

1. Background documentation and desktop review

Before conducting an evaluation or study in a speciﬁc country, you will need to review available background documents including online searches for published articles, evaluations, and reports in that country. Below is a list of questions to review M&E reports. You will need to adapt it to what you are looking for, but this list will help you get started. Mine the documents and reports and look for evidence that allow you to answer these questions:

1. What was the purpose of the study or project? 2. What were the outcomes of the study or the project? 3. Did it demonstrate accountability? 4. Did it demonstrate effectiveness? 5. Was the project’s implementation methodology reproducible and sustainable? 6. Did the study/project show value for money? 7. What would you have done differently?

2. Select the research questions

Most donors have already decided on the evaluation or research questions by the time the research or evaluation team is gathered. However, it is important to discuss them with the donor and all the stakeholders and make adjustments as necessary. Keeping the focus on the PD principles, the results on the health system, and various human rights treaties, the countries have been signatories is a way to get started and ensure the research is asking the minimum right questions. Below are a number of sample questions

Sample Selected Research Questions • Are the objectives of the donor’s overall development agenda for the health sector being met progressively? • How effective has the project been at meeting its targets and objectives? • Has the project improved the performance of the related country’s health programs? • How effective has project’s contribution been to the growth and development of the country’s health system? • Has the project improved the performance of the health providers? • Have patient outcomes improved? • Has the project improved the efﬁcient management of medicines, technol- ogies, and ﬁnances?

(continued) 11 Accountability and Aid Effectiveness Research in Global Health 287

• Does/Did the project work as it was designed? Did the implementers have to make adjustments to the design? • Are the results achieved scalable and repeatable in other geographic areas of the country? • Are the results likely to be sustained by the country’s health system? • How was the project monitored? Who was the project accountable? How well did the project monitored and used the monitoring data to make decisions? • Has the project’s information contributed to improve the country’s health information system?

3. Design an effective AAE research proposal

Proposal Outline 1. Background: In this section, you will summarize what is known and the current knowledge of status of health outcomes in terms of health indicators, quality of care and coverage 2. Problem Statement: This section describes what is not known and needs to be studied (a) Sample general objective: to measure the effectiveness of a certain intervention or improvement and account for results and coverage (b) Sample speciﬁc objectives: (i) To determine the effectiveness of the project and help design future improvements (ii) To identify the causes of effective or ineffective interventions (iii) To determine the degree of compliance with PD principles (iv) To make recommendations for improving effectiveness and accountability for health outcomes 3. Effectiveness Evaluation questions: (a) How effective was the project in meeting its objectives? (b) How effective was the project design to meet the desired outcomes? (c) How well was the project implemented? (d) How effectively did the project identify and solve implementation problems? (e) How sustainable are the project interventions? 4. Methodology: In this section, you will describe how the research or evaluation will be conducted. You will include the following topics. (a) Type of study: descriptive, cross-sectional, retrospective, operational research, time series, etc. (b) Population whose outcomes will be studied: if target population or denominator is available, you will be able to calculate proportions and measure progress and coverage rates.

(continued) 288 E. Beracochea

(c) Sampling: sample size estimation and how records or patients will be selected for study, how health worker to be interviewed will be selected, etc. If randomization is possible, it is desirable. In most cases, givent time and funding limitations, a convenience sample is used to contrast top and bottom performers or document case studies. (d) Variables are selected based on the evaluation or research questions and their definition is usually included in a table to ensure all the right variables are collected and data from various sources can be triangu- lated for analysis. The variable table usually also include the source of data for each variable (e) Data collection methods: The methods section of your proposal will include a mixture of qualitative and quantitative methods. The right mix will allow you to find the answers to the questions above (f) Sources of data: In this section, you will identify primary and secondary sources of data that are usually used in AAE research. Primary data are collected to answer specific questions such as How satisfied are mothers with the new maternity? Secondary sources make use of data already available, usually collected for M&E or management purposes. Having access to the country’s HIS is a great advantage, or better yet, working side by side with the HIS and HMIS teams in the MOH will help uncover the evidence and detect information quality or complete- ness problems to be solved. (g) Instruments: The instruments you will design to conduct your research will include: document review checklist, interview guides to interview key informants, guides for conducting and analyzing focus group discussions, and survey tools and data collector training tools may be necessary. (h) Analysis plan: AAE research and project evaluations are complex exercises. The way to avoid problems is to have a clear analysis plan for analyzing qualitative (interview notes, focus groups notes) and quantitative surveys (facility surveys, health staff interviews, community, or patient interviews) (i) Results dissemination, publication, and archiving: I suggest you plan the outline of the final report and how the results will be discussed with all stakeholders before starting the research so you can plan the enough time for everyone to get involved in this most important task. (j) IRB approval: Each country has a different approval process so becoming familiar with the process as early as possible will save time and ensure the IRB has all the documentation to make the right decision. (k) Timeline: a week by week Gantt chart helps keep everyone on the team on track and complete all the tests within time and budget. 5. References: Include previous studies with similar methodology and/or that involved the same population or health outcomes. 11 Accountability and Aid Effectiveness Research in Global Health 289

4. Conduct the research

Gathering the data and analyzing the findings requires planning to ensure logistics are in place, flight, and hotel or car rental arrangements have been made, interview appointments are made, and focus group participants are invited in a timely manner. Data collection quality control throughout the research needs to be maintained by the senior members of the research team. Also ensure you keep all stakeholders informed of the progress and any challenges along the way. Nobody likes surprises. The credibility of the research findings may be put in question if quality control is not enforced.

5. Data analysis

By the time you get to this phase, you will already have planned what data will be analyzed by whom and even anticipated how the data will be presented in tables, graphs or charts. As with everything in life, less is more. Make sure that you use all the data collected and present the findings in a prioritized manner so they conclusions and recommendations you make can be easily understood and acted upon. It helps to validate with your findings with senior project and donor and MOH staff to ensure your conclusions make sense. A presentation of preliminary findings helps get all stakeholders on board and guides a more in-depth analysis.

6. Presentation of ﬁnal report, discussion of recommendations, and planning of next steps

Researchers usually prepare a long and a short PowerPoint presentation for the project and donor to disseminate the results, in particular to the MOH and staff in the project districts or provinces.

What to Do with AAE Research Results?

Health is a science and as professional scientists we need to demonstrate that what we do is effective. AAE research shows how effective aid money is in global health. In most cases, the main issue is whether resources and efforts were invested in the right interventions. With the COVID-19 pandemic in 2020, the question about the effectiveness of our global health work is even more important and pressing. Asking why we conduct AAE research and finding out where money is being well used or wasted, who is or is not performing and who needs to be held accountable as well as where the gaps in the health system is more important than ever. You might be wondering what happens to those responsible for doing their job and who failed in their responsibility and who delivered ineffective solutions or implemented projects that wasted valuable resources and failed to save more lives. The answer is to ensure we all learn and share our mistakes so that others do not repeat them. Except in the case of willing corruption, most professionals just did not 290 E. Beracochea know what they were doing was not effective. AAE is not about investigating criminal charges. The truth is that we are all responsible for doing our best and admitting when we did not. This is how we learn. We all make mistakes and in hindsight, wish we had done things differently, the author of this chapter included. What matters is transparency (Arkedis et al. 2019) and learning from our not so effective experiences and not make those mistakes again. Taken together, evidence matters and our performance as global health professionals needs to be evidence-based. Effective global health professionals search for evidence and do not stop until they have figured out how to fulfill everyone’s right to health and the right to development (UN 1986) of every nation in this planet. This, any time you might be asked “Why account for aid effectiveness in global health?” you will be able to respond: “Because effective global health practice helps fulfill human rights: the right to health, that is, to receive the highest quality of health care, and the right to develop one’s country.”

Conclusion

Global health practice needs to be evidence-based. Research findings need to be applied to improve results and effectiveness of global health aid projects. Some bilateral donors choose to give aid to countries based on need, some on policy partnerships. Other donors give untied aid, define specific conditionalities, or contribute to the country’s “financial basket” and provide technical assistance to improve the management of health budgets including monitoring and health information systems. Some donors choose to give the money to implementing organizations that manage the funding in alignment with the country’s health programs and priorities. Some donors invest in the country’s medical and nursing training institutions to improve the quality of their training programs and increase production, others provide the training themselves, or through NGOs or CSOs they give grants to. Some donors have an interest in supporting vertical programs such as family planning, malaria, cervical cancer, TB or HIV/AIDS, and others focus on expanding an integrated patient-centered healthcare delivery model that responds to their health needs according to their life cycle. Whatever kind of aid is provided it needs to be accounted for and the effectiveness of the investment needs to demonstrate to ensure aid helps fulfill the human rights of every citizen that is to benefit from it. AAE research is based on the principles of the Paris Declaration, improves the performance of the recipient country’s health program and health system. In this way, we ensure that every global health project contributes to improve the performance of the country’s health programs and the functioning of the country’s health system to deliver better services and achieve better patient outcomes. In the SDG era, AAE research is essential to achieve universal health coverage by 2030 and overcome the impact of the COVID-19 pandemic. Improvement in health status is essential for all the other development sectors, because no country can develop and grow without healthy people. AAE research should be conducted 11 Accountability and Aid Effectiveness Research in Global Health 291 routinely by every donor organization, starting with the Paris Declaration signatories, to improve the effectiveness of the global health architecture.

References

Adedokun AJ, Folawewo AO (2017) Aid selectivity practice and aid effectiveness in sub-Saharan Africa. Rev Innov Compet 3(2):43–64 Annen K, Batu M, Kosempel S (2016) Macroeconomic effects of foreign aid and remittances: implications for aid effectiveness studies. J Policy Model 38(6):1136–1146 Arkedis J, Creighton J, Dixit A, Fung A, Kosack S, Levy D (2019) Can transparency and accountability programs improve health? Experimental evidence from Indonesia and Tanzania. SSRN Electron J. https://doi.org/10.2139/ssrn.3399124 Arndt C, Jones S, Tarp F (2015) Assessing foreign aid’s long-run contribution to growth and development. World Dev, Aid Policy and the Macroeconomic Management of Aid 69(May):6– 18. https://doi.org/10.1016/j.worlddev.2013.12.016 Asian Development Bank (2011) Aid effectiveness report 2011: overall achievements on Paris Declaration Commitments. Asian Development Bank, Manila. https://www.adb.org/sites/ default/files/institutional-document/33038/files/aid-effectiveness-report-2011.pdf Beracochea E (2015) Improving aid effectiveness in global health. Springer, New York. http:// search.ebscohost.com/login.aspx?direct¼true&AuthType¼ip,sso&db¼edsebk& AN¼1060619&site¼eds-live&custid¼s9001925 Beracochea E (2018) Effective aid for hitting the bull’s eye: comment on ‘it’s about the idea hitting the bull’s eye’: how aid effectiveness can catalyse the scale-up of health innovations’. Int J Health Pol Manag 12:1155. https://doi.org/10.15171/ijhpm.2018.90 Bigsten A, Tengstam S (2015) International coordination and the effectiveness of aid. World Dev 69 (May):75–85 Bourguignon F, Platteau J-P (2017) Does aid availability affect effectiveness in reducing poverty? A review article. World Dev 90(Feb):6–16 Breitwieser A, Wick K (2016) What we miss by missing data: aid effectiveness revisited. World Dev 78(Feb):554–571 Cabral L (2008) Accra 2008: the bumpy road to aid effectiveness in agriculture. Nat Res Perspect 114(Apr):1 Chansa C, Sundewall J, Östlund N (2018) Effect of currency exchange rate fluctuations on aid effectiveness in the health sector in Zambia. Health Policy Plan 33(7):811–820. https://doi.org/ 10.1093/heapol/czy046 Chasukwa M, Banik D (2019) Bypassing government: aid effectiveness and Malawi’s local development fund. Polit Gov 7(2):103–116. https://doi.org/10.17645/pag.v7i2.1854 Dalil S, Newbrander W, Loevinsohn B, Naeem AJ, Griffin J, Salama P, Momand FM (2014) Aid effectiveness in rebuilding the Afghan health system: a reflection. Glob Public Health 9(Jul): S124–S136 Easterly W, Pfutze T (2008) Where does the money go? Best and worst practices in foreign aid. J Econ Perspect 22(2):29–52 Gehring K, Michaelowa K, Dreher A, Spörri F (2017) Aid fragmentation and effectiveness: what do we really know? World Dev 99(Nov):320–334 Haque H, Hill PC, Gauld R (2017) Aid effectiveness and programmatic effectiveness: a proposed framework for comparative evaluation of different aid interventions in a particular health system. Glob Health Res Policy 2(1):1–7. https://doi.org/10.1186/s41256-017-0029-8 Holvoet N, Inberg L (2014) Gender responsive budgeting and the aid effectiveness agenda: experiences from Mozambique. J Int Women’s Stud 2:61 Ingram G, Sally P (2018) US foreign aid transparency: how to fix dueling dashboards. Brookings (blog), 13 Jun 2018. https://www.brookings.edu/blog/future-development/2018/06/13/us-foreign- aid-transpare 292 E. Beracochea

Kabir AHM (2019) Development aid in stable democracies and fragile states. Palgrave Macmillan, Cham. http://search.ebscohost.com/login.aspx?direct¼true&AuthType¼ip,sso&db¼edsebk& AN¼1855061&site¼eds-live&custid¼s9001925 Karlan D, Wood DH (2017) The effect of effectiveness: donor response to aid effectiveness in a direct mail fundraising experiment. J Behav Exp Econ 66(Feb):1–8 La Chimia A, Trepte P (2019) Public procurement and aid effectiveness: a roadmap under construction. Hart Publishing, Oxford, UK. http://search.ebscohost.com/login.aspx? direct¼true&AuthType¼ip,sso&db¼edsebk&AN¼2179171&site¼eds-live& custid¼s9001925 Martinez-Alvarez M (2018) Ownership in name, but not necessarily in action: comment on ‘it’s about the idea hitting the bull’s eye’: how aid effectiveness can catalyse the scale-up of health innovations’. Int J Health Policy Manag 11:1053. https://doi.org/10.15171/ijhpm.2018.72 Maruta AA (2019) Can aid for financial sector buy financial development? J Macroecon 62(Dec). http://search.ebscohost.com/login.aspx?direct¼true&AuthType¼ip,sso&db¼edselp& AN¼S016407041830123X&site¼eds-live&custid¼s9001925 McGee R (2013) Aid transparency and accountability: ‘build it and they’ll come’? Dev Policy Rev s1:107. https://doi.org/10.1111/dpr.12022 Metzger L, Günther I (2019a) Making an impact? The relevance of information on aid effectiveness for charitable giving. A laboratory experiment. J Dev Econ 136(Jan):18–33 Metzger L, Günther I (2019b) Is it what you say or how you say it? The impact of aid effectiveness information and its framing on donation behavior. J Behav Exp Econ 83(Dec). http://search. ebscohost.com/login.aspx?direct¼true&AuthType¼ip,sso&db¼edselp& AN¼S2214804318304336&site¼eds-live&custid¼s9001925 Minasyan A, Nunnenkamp P, Richert K (2017) Does aid effectiveness depend on the quality of donors? World Dev 100(Dec):16–30 O’Sullivan SL (2015) Funding conditions for aid effectiveness: a mixed blessing for the sustainable development of host-country-national employees. Can J Adm Sci (John Wiley & Sons, Inc.) 32 (3):189–202 Organisation for Economic Co-operation and Development, and Development Assistance Com- mittee (2001) Evaluation feedback for effective learning and accountability. OECD Develop- ment Assistance Committee, Paris, France. https://www.oecd.org/dac/evaluation/2667326.pdf OECD (2008) The Paris Declaration on aid effectiveness and the Accra Agenda for Action. https:// www.oecd.org/dac/effectiveness/34428351.pdf OECD (2012) Better aid aid effectiveness in the health sector: progress and lessons (Complete edition) – ISBN 9789264178014. Source OECD Development 2012 (16):1 OECD and United Nations Development Programme (2019) Making development co-operation more effective: 2019 progress report. OECD. https://doi.org/10.1787/26f2638f-en. http://www.oecd. org/publications/making-development-co-operation-more-effective-26f2638f-en.htm. Accessed 22 Jun 2020 Pallas SW, Ruger JP (2017) Effects of donor proliferation in development aid for health on health program performance: a conceptual framework. Soc Sci Med 175(Feb):177–186 Quibria MG (2017) Foreign aid and corruption: anti-corruption strategies need greater alignment with the objective of aid effectiveness. Georgetown J Int Aff 18(2):10–17 Rutherford C, King MT, Butow P, Legare F, Lyddiatt A, Souli I, Rincones O, Stacey D (2019) Is quality of life a suitable measure of patient decision aid effectiveness? Sub-analysis of a Cochrane systematic review. Qual Life Res Int J Qual Life Asp Treat Care Rehab 28(3):593– 607. https://doi.org/10.1007/s11136- Ssengooba F, Namakula J, Kawooya V, Fustukian S (2017) Sub-national assessment of aid effectiveness: a case study of post-conflict districts in Uganda. Glob Health 13(1):1–12. https://doi.org/10.1186/s12992-017-0251-7 UN (1948) Universal Declaration of Human Rights. https://www.un.org/en/universal-declaration- human-rights/ 11 Accountability and Aid Effectiveness Research in Global Health 293

UN (1986) Declaration on the Right to Development. https://www.un.org/en/events/rightto development/declaration.shtml UN (2000) General comment no. 14: the right to the highest attainable. n.d., 21. http://health-rights. org/index.php/cop/item/general-comment-14-the-right-to-the-highest-attainable-standard-of- health UN (2019) The sustainable development goals report. https://unstats.un.org/sdgs/report/2019/The- Sustainable-Development-Goals-Report-2019.pdf WHO (1946 and 2006) Who_constitution_en.Pdf. n.d. https://www.who.int/governance/eb/who_ constitution_en.pdf. Accessed 1 Sept 2020 WHO (2000) World health report. https://www.who.int/whr/2000/en/whr00_en.pdf?ua¼1. Accessed 1Sept2020 Wickremasinghe D, Gautham M, Umar N, Berhanu D, Schellenberg J, Spicer N (2018) ‘It’s about the idea hitting the bull’s eye’: how aid effectiveness can catalyse the scale-up of health innovations. Int J Health Policy Manag 8:718. https://doi.org/10.15171/ijhpm.2018.08