Health for the Money and Its Partners Fund Global the for Work to Puttingincentives

A Report of the Center for Global Development Development Global for A Report Center the of Health Global in Money for Value on Working Group Over Victoria with Mead and Fan chair, Amanda Glassman, MoreHealth for the Money and Its Partners Fund Global the for Work to PuttingIncentives

More Health for the Money Putting Incentives to Work for the Global Fund and Its Partners Center for Global Development ” ISBN 978-1-933286-80-8

” ” ” ” ” We have a moral imperative to ensure that every kwacha, dollar, rand, andrupee moral a imperative ensure every wisely is to that lives have spentsave to kwacha, We dollar, This reportThis It funders—both challenges sets financing. agenda health new for a global and government The approach outlined in this report the calls for commitment full and cooperation funders global health of This document contribute help the about will to ongoing dialogue efficiency improving in donor for funding Funds for health are scarce, so we have to deploy our money efficiently deploy our money to soandeffectively. have we scarce, report are health Funds for This on value A value for money agenda for health financing could achieve significant health gains onhealth ground. the significant Our achieve could agenda financing money health for for A value health. It shedsIt onpossible health. light solutions andaddresses the complexities inherent these solutions present. health partners—tohealth a fresh take look their investments at and consider what can greater be done achieve to Dean the of School Public of Health at the University California–Berkeley of and former HIV Program, Director, & Melinda Bill Gates Foundation David Serwadda David Donald KaberukaDonald Eric Goosby impact health. and improve of serviceof delivery. It will no doubt be of use as PEPFAR continues to advance to continues its agenda on smart doubt no be use will of PEPFAR as It investments. global health financing institutionsglobal financing health can betterresourcesleverage attaintheir to the best possible health outcomes. Funders shared a and responsibility implementers have ensure to funding that health for accomplishes maximum impact. and change the course epidemics. of This report combines strong recommendations health on improving outcomes with practical consideration the implementers that of challenges face. K.T. Li Chair Professor Economics, of HarvardK.T. School Public of Health Makerere University School Public of Health, Kampala, Uganda Minister Health, of Rwanda President, African Development Bank U.S. Global AIDS CoordinatorU.S. to respectto country specificity forand implementers respectto everyat the for money principleof value step for money tells us how we can we us how do tells it. A must money for leaders health for read and funders. Stefano Bertozzi William C. Hsiao Agnes BinagwahoAgnes “ “ “ “ “ “ For an electronic version of this book, visit www.cgdev.org Center for Global Development

More Health for the Money

Putting Incentives to Work for the Global Fund and Its Partners

A Report of the Center for Global Development Working Group on Value for Money in Global Health

Amanda Glassman, chair, with Victoria Fan and Mead Over c Center for Global Development. 2013. Some Rights Reserved. Creative Commons Attribution-NonCommercial 3.0

Center for Global Development 1800 Massachusetts Ave NW, Floor 3 Washington DC 20036 www.cgdev.org

ISBN 978-1-933286-80-8

Editing, design, and production by Communications Development Incorporated, Washington, D.C., and Broadley Design, Bristol, UK Working Group on Value for Money in Global Health

Working Group chair Josh Salomon, Harvard School of Public Health Amanda Glassman, Center for Global Development Nalinee Sangrujee, U.S. Centers for Disease Control and Prevention Working Group members Nina Schwalbe, GAVI Alliance David Barr, Pangaea Global Aids Foundation Bernard Schwartlander, Joint United Nations Programme on Joseph Brunet-Jailly, Institut de Recherche pour le HIV/AIDS Développement and Sciences-Po Paris David Serwadda, Makerere University School of Public Health Kalipso Chalkidou, National Institute for Health and Care Agnes Soucat, African Development Bank Excellence International Yot Teerawattananon, Health Intervention and Technology Karl Dehne, Joint United Nations Programme on HIV/AIDS Assessment Program in Thailand Alan Fairbank, Independent Damian Walker, Bill & Melinda Gates Foundation Victoria Fan, Center for Global Development Brenda Waning, UNITAID Kara Hanson, London School of Hygiene and Tropical David Wilson, World Bank Medicine Prashant Yadav, University of Michigan Iain Jones, U.K. Department for International Development Gavin Yamey, University of California, San Francisco Jason Lane, U.K. Department for International Development Bruno Meessen, Institute of Tropical Medicine Staff Mead Over, Center for Global Development Denizhan Duran, Center for Global Development Nancy Padian, Independent Kate McQueston, Center for Global Development Mark Rilling, U.S. Agency for International Development Rachel Silverman, Center for Global Development iv

Table of contents

Preface vii

Acknowledgments viii

Executive summary ix

Chapter 1 The Global Fund and value for money 1 Why this Working Group? Why now? 2 Notes 4

Chapter 2 What is value for money? 5 Generating incentives for value for money: a framework for funders 6 Note 9

Chapter 3 Planning allocation 11 Overview 11 Opportunities and limitations 14 Recommendations 15 Summary 22 Notes 22

Chapter 4 Designing contracts 25 Overview 25 Opportunities and limitations 26 Recommendations 28 Summary 32 Notes 32

Chapter 5 Tracking and using cost and spending data 33 Overview 33 Opportunities and limitations 37 Recommendations 39 Summary 41 Notes 41 v T able ofable contents

Chapter 6 Verifying performance 43 Overview 43 Opportunities and limitations 46 Recommendations 48 Summary 49 Notes 50

Chapter 7 Conclusions 51 Reflect value for money in key performance indicators 51 Build better accountability with technical partners 52 Connect countries with scarce (but essential) expertise to inform allocation 52 Create synergies in data collection and analysis 53 Assess and share best practices among principal recipients, country coordinating mechanisms, and other partners 54 Wrapping up 54 Notes 55

Appendix 1 Seque ncing and division of responsibility for the value for money agenda 57

Appendix 2 Innovations in the design of “contract-like” grant agreements 59

Appendix 3 Working Group on Value for Money for Global Health Funding Agencies 67

References 73

Boxes 1.1 Statement by African Ministers of Finance and Ministers of Health on value for money 2 1.2 Global Fund aspirations meet aid realities 3 1.3 Global Fund leadership and value for money 4 2.1 Definition of value for money and its components 5 2.2 Selected global health funders 7 3.1 Redirecting resources to hot spots of infection and transmission 18 3.2 Data requirements to optimize for impact 20 3.3 Modeling as a success 21 4.1 Statement by the Technical Evaluation Reference Group on performance-based financing 26 4.2 Comment on country coordinating mechanism incentive structure 27 4.3 Suggested core indicators for performance-based financing 29 4.4 Frequently asked questions about performance-based financing 30 4.5 Innovations in grant design can improve recipient efficiency and enhance the donor’s cost-effectiveness while economizing on information 31 4.6 Health Results Innovation Trust Fund–participating countries and Global Fund high-impact countries 31 5.1 The Price and Quality Reporting system 34 5.2 What determines the unit cost of a first-line antiretroviral drug? 35 vi Table of contents of Table

5.3 The U.S. Government Accountability Office on the Global Fund 36 5.4 Strengthening supply chains: a new initiative at GAVI Alliance 36 5.5 Perspective from Liberia 39 7.1 Excerpt from the new funding model 51 7.2 Indicative menu of services provided by the proposed Decision Support Network 53 7.3 Statement by Ambassador Eric Goosby 53

Figures 1 Value for money domains for global health funders x 1.1 Life-years saved for $1,000 of tuberculosis interventions 1 1.2 The new funding model (as of April 2013) 3 2.1 Value for money domains for global health funders 8 3.1 Misalignment between men who have sex with men’s share of disease burden and funding level 12 4.1 Grant performance does not predict disbursement levels 27 5.1 Variation in reported cost per patient-year for Ritonavir 100 milligrams, September 2010–April 2013 34 5.2 Trend in 25th, 50th, and 75th percentile per-patient unit prices for Efavirenz 200 milligrams 35 5.3 Heterogeneity in the unit cost of antiretroviral treatment across 45 Zambian facilities, 2009 37 5.4 PEPFAR expenditure analysis pilot in Mozambique, mean and range of non–antiretroviral unit spending per patient-year 37 5.5 Variation in cost per patient-year of HIV treatment in five African countries, 2012 38 6.1 Boy stands in front of insecticide-treated bed net used as soccer net in Wassini Island, Kenya 45 6.2 Approaches to assessing grant performance 46 A2.1 Efficiency-enhancing response of a recipient granted residual claimant status and paid the previous year’s average cost for every current year unit of output 61 A2.2 Maximum payment of donor to recipient under the two-part payment contract of table A2.1 64

Tables 1 Value for money: summary of domains, key problems, and recommendations xi 7.1 Value for money: summary of domains, key problems, and recommendations 52 A2.1 Worked example of payments for above-threshold output during a single year of a two-part price agreement 63 vii

Preface

The Global Fund to Fight AIDS, Tuberculosis and Malaria, set up ownership while making the Global Fund itself far more account- in 2002, was meant to implement a new model of financing aid, able to its own funders for ensuring, as the report title says, more under which it would eschew the usual donor bureaucracy and red health for the money. tape and simply provide financing to countries needing support for My colleagues at the Center for Global Development have expanded health programs. The idea was that priorities in fighting closely followed the Global Fund over the years, from Steven Rade- the three major diseases should be set at the country level by con- let’s working group for incoming leadership in 2006, to Nandini sortia of government, civil society actors, and health providers. The Oomman’s HIV/AIDS Monitor between 2006 and 2010, to Ruth assumption was that the resulting country ownership, along with Levine’s analyses of the quality of health aid, allocation, and effec- periodic evaluation of countries’ performance in implementing tiveness. Mead Over’s work on the missing AIDS “transition” in programs they proposed, would translate into good health outcomes the absence of incentives to invest in prevention built on our cash- at reasonable costs. on-delivery model to recommend aligning incentives in the U.S. In the decade since a tremendous amount has been accomplished President’s Emergency Plan for AIDS Relief, the Global Fund, and in the fight against AIDS, tuberculosis, and malaria in countries bilateral programs to reduce AIDS by rewarding the outcome of benefiting from Global Fund financing. But relying on country- reduced HIV incidence not just inputs to treatment. based consortia, usually with government in the lead, did not create This report, however, raises deeper and tougher challenges for the incentives for maximizing what has come to be called “value for Global Fund and indeed for all the major funders of global health money”—that is, the maximum health benefit for the minimum programs. Our global health policy team supporting the Working cost. Most recipient countries faced political pressures to distribute Group analyzed all available data and placed the Global Fund’s funds to many different constituencies, independent of the coun- policies and practices in the context of the broader global health tries’ different exposures to the diseases, as well as social and other ecosystem. As a result this report has proposals that are relevant to pressures at times not to spend—for example, in the case of local all health funders concerned with getting the most health impact religious objections to working with prostitutes and other popula- for every dollar invested. tions most at risk to HIV/AIDs. The Global Fund has navigated significant changes in recent While well intentioned and aligned with the Global Fund’s core years. These changes include new leadership, a new funding model, principle of country ownership, the model thus suffered from what and renewed commitment by donors that is focused more clearly the Global Fund’s High Level Panel in 2011 labeled the “free-for- on incentives, outcomes, and greater accountability of country con- all phenomenon,” wherein countries “[had] every incentive to seek sortia. The changes at the Global Fund and among its contributors as much money as possible”—despite actual need, other funding make this an opportune time to move to a new model that buys sources, or a strategic assessment of their most pressing priorities much more health for the money. I hope the recommendations when facing a budget constraint. in this report will be adopted by the Global Fund’s Board and This report deals head-on with the resulting challenge to the implemented by its Secretariat and partners, and look forward to Global Fund going forward. It sets out a new model of financing reporting on progress in implementation. more closely related to actual outcomes at a given cost, and builds Nancy Birdsall in better measures of recipient accountability for their performance President on outcomes not just inputs. It retains the sensible focus on country Center for Global Development viii

Acknowledgments

This report is based on the deliberations of a working group com- The authors are grateful to Till Barnighausen, Gemma Beren- prising government officials, aid agency staff, technical partners, guer, Salal Humair, Ananth Iyer, and Jomkwan Yothasamut for civil society, and thematic experts. The Working Group members their contributions. We thank Ida Hakizinka, David Kim, Aida served as volunteers representing their own views and perspec- Kurtovic, and David Logan for their input to the report in Febru- tives, and were at the center of the messages and recommendations ary 2013. We thank Stefano Bertozzi, Simon Bland, Chris Collins, included in this report. A list of the Working Group members and Geoff Garnett, Robert Hecht, Jason Lawrence, Jeff Sturchio, and their biographies is in appendix 3. Mitchell Warren for their feedback during the early stages. This report was written by Amanda Glassman, Victoria Fan, and We appreciate individuals from the Global Fund who have Mead Over with Rachel Silverman, Kate McQueston, and Den- offered their input to the report, including Heike Allendorf, Jac- izhan Duran. Kate McQueston coordinated the Working Group queline Bataringaya, Nicolas Bidault, Matthew Blakley, Michael and its meetings, and Amanda Glassman served as its chair. Jenny Borowitz, Ruwan De Mel, Carol D’Souza, Tarek Elshimi, David Ottenhoff coordinated policy outreach around the report, and John Kim, Ryuichi Komatsu, Osamu Kunii, Daniel Low-Beer, Martina Osterman coordinated the production of the report. All the work- Niemeyer-Riviere, and Susanne Reichelt. ing and background papers, data, and presentations that informed The authors are grateful to the participants who attended the this report can be found on the Working Group’s website (www. consultation meeting held at the Rockefeller Bellagio Center in cgdev.org/initiative/value-money-agenda-global-health-funding- April 2013, including Robert Brinckman, Kieran Daly, Mireille agencies). The Working Group meetings and staff time dedicated Guigaz, Jason Lane, Julia Martin, Susan Nazzaro, Todd Summers, to this research were supported by grants to the Center for Global Nertila Tavanxhi, Thomas Teuscher, Rajan Varatharajan, Monique Development from the Bill & Melinda Gates Foundation, the Rock- Vledder, and Diana Weil. We also thank Rob Garris, Managing efeller Foundation (Bellagio Center), and individual donors. Director of Bellagio Programs, and Nadia Gilardoni for their sup- Beginning in April 2012 the Center for Global Development— port in arranging the meeting, and Pilar Palaciá for hosting on an independent think tank working on financing, economics, and behalf of the Rockefeller Foundation. global public goods issues in development—convened a working We thank Rifat Atun, Kanika Bahl, Chandrani Banerjee, Ties group of policymakers, funders, and experts, and conducted policy Boerma, Jesse Bump, Bradley Chen, David Collins, Kate Con- research to define value for money, set out the challenges facing dliffe, Wouter Deelder, Christopher Fitzpatrick, Tim Hallett, funders and recipients, and build a practical agenda to enhance the Vicky Hausman, William Hsiao, Judith Kallenberg,Jennifer Kates, value for money of Global Fund investments. Broader consulta- Itamar Katz, Sonali Korde, Eline Korenromp, Steve Kuo, Ben tions were also undertaken with low- and middle-income country Leo, Geir Lie, Lisbeth Loughran, Robert Marten, John Monahan, government policymakers, Global Fund staff, principal recipients, Justin O’Hagan, Marjorie Opuni, Liesbet Peeters, Rosalia Rodri- Board constituencies, advocacy partners, and others. While deeply guez-Garcia, Oliver Sabot, Catherine Severo, Bakhuti Shengelia, informed by the Working Group members and consultations (named Karin Eva Elisabet Stenberg, Daniel Thornton, Saba Waseem, Paul in this acknowledgments section), the content of the report is the Wilson, and Debrework Zewdie who were consulted during the responsibility of the authors, and does not represent the opinions drafting of the report. and positions of the entire Working Group or the Global Fund itself. We apologize for any omissions. All errors in the report remain At different stages in developing this report, many individuals our own. offered comments, critiques, and suggestions. We are very apprecia- tive of these contributions. ix

Executive summary

The Global Fund to Fight AIDS, Tuberculosis and Malaria (the positioned to lead, and its new funding model offers an opportu- Global Fund) is one of the world’s largest global health funding nity for quick and flexible adoption of “value for money” principles agencies. From 2002 to 2011 the Global Fund disbursed about $15.5 and practices. billion to support programs aiming to prevent and treat these three The Global Fund Board has already identified value for money diseases, to care for people suffering from them, and to strengthen as a priority, and has taken steps within the new funding model health systems in more than 150 countries. Although it is difficult to improve the health impact of their funding. However, current to systematically track the Global Fund’s health outcomes, the sheer efforts stop short of realizing their full potential. scope of its activities suggest that many millions of people are alive today because of its efforts. In 2013 the Global Fund requested an How to get more health for the money additional $15 billion from donors to support grant-making activities through 2016. Global health programs operate within a complex funding architec- While the Global Fund has made important contributions to ture, where competing mandates and sometimes perverse incentives fight HIV/AIDS, tuberculosis, and malaria over the past decade, can stand between money and health impact—the latter being the the organization and its partners could save many more lives with appropriate measure of success. the same amount of money by allocating it in ways that maximize For example, a standard grant agreement between donor and the positive impact on health. recipient often contains health goals and objectives, a description This is what we call “value for money.” of activities, a budget of inputs required to carry out the activities, Value for money is not about reducing costs or cutting budgets, and requirements for routine reporting and financial audits. Yet but rather about maximizing the health impact of every available this design contains no explicit incentive for efficiency, offers few peso, pound, or pula to reduce human suffering and save lives. Or, incentives for effectiveness, and may create perverse incentives to simply put, getting more health for the money. over-report results to meet agreements. Ensuring more health for the money is especially urgent in the These forces make up an incentive environment unlikely to current austere budget environment. Governments and global be aligned with maximizing health. Getting more health for the health donors are making tough decisions on how to invest scarce money thus requires re-examining explicit and implicit incen- resources and demanding that their investments in health yield tives for funders and recipients, and adjusting them in ways higher returns. that encourage the allocation of funds to highly cost-effective This report describes practical steps for the Global Fund and its interventions. partners to see these demands become reality. The recommendations are straightforward, if not uniformly easy to implement. Still, Getting more out of the grant cycle the moral imperative that drove the Global Fund’s creation more than a decade ago also motivates them to ensure that the billions of This report identifies four domains within the Global Fund’s dollars raised and disbursed reduce the disease burden as much as grant cycle where value for money can be improved: allocation, possible. By applying modest changes to its grant-making process, contracts, costs and spending, and performance verification (fig- the Global Fund could save hundreds of millions of dollars that ure 1). Decisions in each domain affect the availability and quality could then be reprogrammed to save even more lives. of services provided to people at risk of or suffering from disease, Achieving more health for the money is the core business of and ultimately the Global Fund’s ability to reduce suffering and all global health funders. But the Global Fund is particularly well save lives. x Executive summary

Figure 1 Value for money domains for global At the contract stage, a country may include bed nets in the health funders budget, but no incentives—financial or nonfinancial—are built into the grant agreement to ensure their availability and use in the How can resources How can contracts be be allocated structured to create most affected areas. ex ante to maximize stronger incentives for value for money? value for money? To get more health for the money, contracts between the Global Fund and a recipient country should connect some funding to incre- Allocation Contracts mental progress on a few important indicators, like the number of children sleeping under bed nets. This progress should be rigorously measured in a simple, objective way to ensure accuracy. At the costs and spending stage, supply chains may be slow or Performance Costs and veriﬁcation spending subject to leakage in moving bed nets from warehouses to front-line providers, and program managers may not have data or leverage in How can performance How can costs of be veriﬁed rigorously, real time to solve these problems. The cost to distribute each net to and spending be better to generate greater tracked to improve incentives for value for the right population is unknown, so money may be wasted. value for money? money? To get more health for the money, the Global Fund should increase and expand reporting to commodity price tracking sys- Source: Authors. tems to ensure that the lowest prices for best value products are obtained, track and use cost information on supply chains and service delivery, and create financial and accountability incen- More health for the money in action tives to ensure bed nets arrive in the right place at the right time for the right price. To illustrate this idea, think of common problems that a Global At the performance verification stage, neither the Global Fund Fund–supported bed net distribution program might face during nor recipients rigorously collect data on the use and distribution of implementation, and how better decisions and incentive structures bed nets at the household or facility level, and the Global Fund and in each domain could help solve them. country programs must rely on incomplete or unreliable self-reports At the allocation stage, funding might be directed to more than from recipients. Without accurate information on performance, it 200 different types of bed nets—including those with custom label- is difficult for the Global Fund and country governments to know ing or nonstandard sizes—despite a lack of evidence that these speci- if the bed net program should continue to receive funding, or how fications improve outcomes. A program may purchase too many nets its management and delivery strategy could be adjusted to improve or too few due to inaccurate demand forecasting, unknown program effectiveness and efficiency. efficiency, or an inability to assess which mix of interventions will To get more health for the money, the Global Fund should verify achieve the greatest reduction in disease incidence. performance in a rigorous and representative manner, and use data To get more health for the money, the Global Fund should ask to contract better, allocate better, and strengthen the overall impact countries to purchase their bed nets from a menu of proven, cost- of the bed net distribution program over time. effective interventions and commodities for malaria. The Global As all this shows, opportunities exist within each domain for the Fund could also require that recipients describe the distribution Global Fund and its partners to generate more or less health for the of malaria in their respective countries, to help target bed nets to money depending on decisions made and incentives put in place. the most-at-risk groups and geographic locations, and to decide on the mix of interventions—bed nets or otherwise—that will Recommendations maximize disease impact. And the Global Fund should make this information available to other donors and country programs to Table 1 summarizes key problems in each domain and opportuni- reduce gaps in coverage. ties for improving incentives to generate more health for the money. xi E xecutive summary

Table 1 Value for money: summary of domains, key problems, and recommendations

Domain Key problem Recommendation Allocation. How can National and donor funding Choose from a menu of effective and cost-effective interventions resources be allocated is not consistently supporting and commodities. to maximize impact on best practice, despite Identify and target key populations with appropriate HIV/AIDS, tuberculosis, and substantial evidence on what interventions. malaria? works most cost-effectively to reduce disease. Optimize investments for the greatest health impact. Improve ex ante budgeting and transparency on spending. Contracts. How can Current agreements provide Directly connect a portion of funding to incremental progress on contracts and agreements only weak incentives for performance. between the Global Fund and impact. Link performance payments to incremental progress against the its recipients be structured to most important indicators. create stronger incentives? Support performance incentives between the principal recipient and service providers. Cost and spending. How Cost, price, and spending on Continue to improve the scope, completeness, and timeliness of can costs of and spending on commodities varies widely reporting to commodity price tracking systems. commodities, supply chains, between countries; this Benchmark and use supply chain costs and outputs. and service delivery be better variation is unexplained. tracked and used? Identify core services for more extensive analysis and use of service delivery costs and spending. Share costing data with partners and the public. Develop a strategy to use unit-cost data throughout the new funding model grant cycle. Performance verification. The Global Fund relies on Define a subset of core indicators to receive strengthened How can performance be weak instruments to verify performance verification. verified and evaluated the accuracy of self-reported Independently verify the accuracy and quality of principal rigorously, to generate performance measures. recipients’ self-reported results using rigorous, representative greater incentives and measurement instruments. accountability? Complement output verification with population-based measurement and formal impact evaluation for interventions and service delivery strategies of unknown efficacy.

Source: Authors.

The challenge ahead While this report focuses mainly on practical technical policies and practices to improve impact, it is also worth anticipating More health for the money cannot be an afterthought, a checklist, the political challenges posed by the agenda. For example, shifts in or an extra obligation—it is the very essence of ethical and respon- allocations by intervention mix—while favoring those at risk for sible global health funding. or affected by disease—may threaten the interests of those invested The Global Fund Board has already identified a subset of these in a less optimal intervention mix. recommendations as priorities, particularly those on market shaping Further, beyond the incentive environment, increasing the and optimization of commodities. But other areas have attracted less availability and use of economic analysis and expertise is critical attention, such as reforming and redesigning the performance-based to improving value for money. Using unit-cost benchmarking financing system, strengthening performance verification, and using and variability analysis more in different health systems and epi- cost and spending data to improve the efficiency of procurement, demiological contexts could help, as could increasing technical supply chains, and health care delivery. assistance capacity to build recipient economic and financing xii Executive summary

analysis. But mandate and funding for these analyses still lack and integrating the report’s recommendations into its operations, scale and support. and systematically implementing and evaluating the agenda along The challenges for the Global Fund are recognizing that this with the new funding model. The Working Group hopes this report systematic agenda of more health for the money can influence all can prompt and guide the Global Fund and its partners to greatly aspects of its business, obtaining higher priority for the agenda from enhance their contribution to reducing disease burden and improv- the Secretariat, Board, and key strategic partners, adapting, adopting, ing health and well-being—the heart of the Global Fund’s mission. More Health for the Money Putting Incentives to Work for the Global Fund and Its Partners

Chapter 1 The Global Fund and value for money

With the same amount of money spent today, the Global Fund to fantastic value for money. The United States spends upward of Fight AIDS, Tuberculosis and Malaria (the Global Fund) and its $20,000 a year per patient on life-sustaining antiretroviral treat- partners could save more lives—if they are willing to create stron- ment,1 a service provided in low- and middle-income countries ger incentives for evidence-based resource allocation and proven for an estimated average of $768,2 less than 4% of the cost of U.S. health impact. treatment. Likewise, about $200 buys enough bed nets to save a Contrary to misconceptions and misuses, value for money is not child’s life from malaria3—or to fund a routine pediatrician visit about reducing costs or cutting budgets, but rather about maximiz- in wealthy countries. ing the health impact of every peso, pound, or pula spent. Yet these Nonetheless, getting better value for money is still imperative, as transactions occur within a complex funding architecture, where well as a moral and human rights issue. The least effective interven- competing mandates and sometimes perverse incentives can stand tion in HIV/AIDS produces less than 0.001% of the value generated between health financing and health impact—the ultimate measure by the most effective strategies.4 Tuberculosis interventions may of collective success. be universally cost-effective when compared with a gross domestic And we’ve come a long way. For all who value the inherent product per capita threshold, yet they range enormously in their dignity and worth of human life, investing in global health is cost per disability-adjusted life year saved (figure 1.1). Further, some

Figure 1.1 Life-years saved for $1,000 of tuberculosis interventions

First-line treatment under DOTS (patients with smear-positive tuberculosis)

Isoniazid preventive therapy (people living with HIV, infected with tuberculosis)

Diagnosis of tuberculosis using Xpert MTB/RIF as an add-on to smear (people in whom tuberculosis is suspected)

First-line treatment under DOTS (patients with smear-negative or extrapulmonary tuberculosis)

First-line drugs under DOTS plus antiretroviral therapy (people living with HIV, with tuberculosis)

18–24 months of second-line treatment under WHO guidelines (patients with multidrug-resistant tuberculosis)

0 50 100 150 200

DOTS is directly observed treatment, short-course; WHO is World Health Organization. Note: Estimates of cost-effectiveness do not take into account positive externalities on other disease conditions. Source: WHO (2012a). 2

commonly funded interventions have never been rigorously evalu- Box 1.1 Statement by African Ministers of ated, and may not produce health benefits. Where evaluation has Finance and Ministers of Health on value for

The Global Fund and value for money occurred, results are decidedly mixed—35 of 45 trials evaluating money HIV/AIDS prevention interventions found no statistically signifi- “We recommend [taking] concrete measures . . . to enhance cant effect.5 Ignoring cost-effectiveness in resource allocation can value for money, sustainability, and accountability in the thus imply huge losses relative to the maximum achievable health. health sector . . . to accelerate progress toward the health In practical and ethical terms this translates to hundreds, thousands, [Millennium Development Goals].” or millions of avoidable infections and deaths due to a failure to Source: Joint Declaration by Ministers of Finance and Ministers of Health prioritize on value for money. of Africa, July 5, 2012. This does not mean that all decisions about allocation and spending should be made based on cost-effectiveness—equity, ethics, feasibility, and other factors also play a role. But the missed oppor- Global Fund is a public-private partnership mandated to invest tunities to achieve shared HIV/AIDS, tuberculosis, and malaria “the world’s money to save lives” and create “a world free from the goals may be substantial. So value for money should be among the burden of AIDS, tuberculosis, and malaria.”6 The Global Fund major factors considered in decision-making. is a “financial instrument, not an implementing agency,”7 so it relies on its recipients and its partners, particularly the World Why this Working Group? Why now? Health Organization, the Roll Back Malaria Partnership, the Stop TB Partnership, and the Joint United Nations Programme The Working Group was motivated by three windows of opportu- on HIV/AIDS, among others, to provide technical guidance and nity. First, the world has rallied around ambitious global HIV/AIDS, support. tuberculosis, and malaria goals, pledging unprecedented funds to The Global Fund also plays a central role in the complex ecosys- combat the “big three” diseases. Yet in recent years budgets have tem of global health funding agencies, requiring extensive coopera- plateaued even as effective new technologies and interventions have tion with the U.S. President’s Emergency Plan for AIDS Relief, become available, requiring tough choices on the use of resources the U.S. President’s Malaria Initiative, UNITAID, and the World to maximize impact. Bank. Having emerged from a period of transition, the Global Second, low- and middle-income country governments are Fund’s new leadership and new funding model (figure 1.2)—both spending more on health given rising domestic economic growth put in place in 2013—offer an opportunity for adopting the value and stagnating global health funding, suggesting that donors for money agenda quickly and flexibly (box 1.2). For these reasons, should focus more on leveraging their money where recipients and though the underlying analysis and principles can be extended are spending smartly. Global health funders have set increasingly to other funders and disease-control priorities, this report’s main stringent co-financing requirements for governments that receive audience is the Global Fund and its partners—country govern- their support, creating an even greater imperative to ensure that ments, recipients, Secretariat, Board constituencies, and technical funds are spent on the best possible uses for health in each recipi- partners. ent country. Recent scientific progress, paired with global investments over And third, more governments and global health funders are the past 10 years, has created a unique moment for global health demanding that their investments yield “value for money” returns (box 1.3). Whereas pioneers in the fight against HIV/AIDS, (box 1.1). Yet the practical steps needed to fulfill those demands tuberculosis, and malaria had to rely on intuition, trial and error, have remained vague up to now. and their own perseverance to aid afflicted communities, today’s While these global trends affect all health systems and global practitioners can base their work on an expanding, rigorous tool- health agencies, the Global Fund’s mandate, resources, partner- box for “what works.” Value in global health can now be planned, ships, and flexible model offer a unique opportunity for leadership implemented, and documented according to established best in—and partnership for—value for money. Created in 2002, the practice. 3 T he Global F und a nd value for money

Figure 1.2 The new funding model (as of April 2013)

1 2 3 4 5 6 7

Determination of split National between diseases and Unfunded strategic health and community quality plan support systems strengthening Technical demand Review Panel Board review approval National Determine and Country Concept Grant-making strategic plan; approve adjusted dialogue note (different grants) investment case funding amount

Indicative funding Incentive funding

Band allocation

Allocation formula

Source: www.theglobalfund.org/en/activities/fundingmodel/process/.

Box 1.2 Global Fund aspirations meet aid realities The 2002 creation of the Global Fund was motivated by dis- through national plans and country dialogue, it does not yet satisfaction with the “mainstream aid industry” and its re- create clear incentives to ensure good results. And while sponses to HIV/AIDS. The new model of aid—to be embodied the Global Fund has been a leader on transparency, it was by the Global Fund—was intended to: “be evidence based, not able to link its spending with outputs or outcomes, thus sharing cutting-edge technology and good practice globally; missing opportunities to understand costs and incentivize show quantifiable results and provide performance-based greater efficiency. This report presents recommendations financing to help achieve them; have the ability to bring off to help close the gap between the Global Fund’s original massive short-term change; be nimble and adaptable; serve intent and the operational challenges of reality. as a financing agency and rely on partner agencies for help Amid a difficult economic climate and ever-increasing in-country; and set a high standard of transparency,”1 among demands for accountability, the Global Fund’s progress other goals. Yet competing mandates in the global health on value for money will be its best justification for a strong field and in the governance of the Global Fund have played replenishment—planned for later this year—and for ensur- out in ways that did not always support founders’ aspirations. ing that those resources have the desired impact on the This report suggests that the Global Fund’s grants have global fight against HIV/AIDS, tuberculosis, and malaria. As not consistently supported evidence-based interventions or noted by Executive Director Mark Dybul in a recent blog best value for money technologies, and rarely share good post: “[W]e must make our money count. . . . Great in- practice globally. Quantified performance is documented, vestments are effective, and efficient. In order to raise the but, as chapters 4 and 6 discuss, the measures used may money we need for global health, we need to demonstrate create perverse incentives, and rigorous verification of these to everyone that this money is put to excellent use.”2 measures has been missing. Relying on partner agencies to ensure that grants are designed and implemented ade- Notes quately has been problematic. While the new funding model 1. Isenman and Shakow (2010). is intended to structure partner agency contributions better 2. Dybul (2013).

Box 1.3 Global Fund leadership and value for money

The Global Fund and value for money “Every era offers something special. I think the most special thing about our current time is the incredible opportunity that scientific advances have provided in the field of global health, giving us the ability to completely control highly dangerous infectious diseases such as AIDS, tuberculosis, and malaria. . . . Timing is critical. If we do not start to act this year, we may miss that opportunity. . . . As a financing institution, the Global Fund will continue investing in programs that support national health strategies, and will expect that implementers increasingly engage and focus on high value for money and high-impact programs.” —Executive Director Mark Dybul “Value for money is a challenging, but essential and highly collaborative, process. . . . There is no other alternative.” —Board Vice-Chair Mireille Guigaz

Notes 1. www.cdc.gov/hiv/topics/preventionprograms/ce/index.htm. 2. PEPFAR (2012b). 3. WHO (2007). 4. Ord (2013). 5. Padian and others (2011). 6. www.theglobalfund.org/en/about/whoweare/. 7. The Global Fund (2001). 5

Chapter 2 What is value for money?

“Value for money” has different meanings in global health. Some Box 2.1 Definition of value for money and its use the term to refer to the cost-effectiveness of a health technol- components ogy, such as a vaccine. Others use it to refer to efficiency as cost Value for money in the health sector is defined as creat- minimization—as described in the Global Fund’s Value for Money ing and complying with rules or procedures for allocating Information Note. The World Bank defines value for money as effi- resources that elicit the production and use of the health- ciency and effectiveness, while the U.S. President’s Emergency Plan maximizing mix of services for the available donor, nation- for AIDS Relief refers only to efficiency and effectiveness and does al, and private resources. In keeping with this definition, not use the specific term “value for money.” At times the term is also achieving value for money entails high levels of “technical used to characterize an agency’s overall value proposition relative to efficiency” and “allocative efficiency,” which can only be other agencies, as in the United Kingdom’s Multilateral Aid Review. reached by ensuring “incentive compatibility.” These terms The definition of value for money here builds on all these views are defined as: but focuses on the broader goal of maximizing health impact, given disease-control goals and a health funder’s resource constraint. This Technical efficiencyimplies producing as much quality- Working Group’s definition implies that health funders will increase adjusted output as possible with a given set of inputs, value for money if they create and apply incentives for recipients to or, conversely, producing a given output with a minimum allocate resources to an optimal mix of cost-effective interventions amount of inputs. For example, measures of technical ef- that maximizes impact toward a disease-control priority within a ficiency would be expressed as “antiretroviral treatment given budget (box 2.1). person-years gained per $1,000.” Value for money measures such as cost-effectiveness and efficiency are standard, quantitative indicators used in health economics Allocative efficiency implies the distribution of resources to to compare the health results of alternative health spending choices. maximize health or minimize selected diseases across coun- Cost-effectiveness, measured as the cost per quality-adjusted out- tries, across subpopulations, across diseases, and across come achieved, informs both allocative and technical efficiency. interventions. A measure of allocative efficiency would be Technical efficiency is achieved by minimizing the cost per quality- expressed as “malaria cases averted per $1,000.” adjusted unit of intervention or service, while allocative efficiency is achieved when technically efficient interventions or services in Incentive compatibility implies creating and complying with the health sector produce the highest health gain given a resource rules or procedures that align incentives to achieve technical constraint.i and allocative efficiency based on the disease-prevention In thinking about the value for money of investments, a start- and -control goals set by the global health community. ing point is to consider the relevant perspective—that is, whose i. The term “allocative efficiency” can also be used to describe the socially investments are under question? From a country’s perspective its optimum allocation of resources between the health sector and other ambition to optimize its overall strategy for a given disease through sectors of the economy, such as education or transport. In this report the a range of interventions is often described as achieving “value for concept applies to allocations within the health sector. money” for that disease. This perspective is a simplification of a very 6 What is value for money? complex problem, since a low-income country is not a “unitary” incentives not necessarily aligned with maximizing health improve- decision-maker. With many actors in a country, value for money ment. An incentive environment can never be perfect, and there requires all internal and external actors to coordinate their invest- is no single best approach. Further, introducing new incentives ments around a single coherent strategy. affects how current incentives operate, and thus requires constant Meanwhile, global health funding agencies must optimize their monitoring and continuous adjustment to keep them aligned with investments not only for a given country—but also across coun- overall health objectives. For these reasons the incentive environ- tries, particularly because infectious diseases can create cross-border ment is an important starting point for discussing value for money. spillovers and externalities. And global health funding agencies Improving value also requires adequate information, and the are also not unitary decision-makers. They must be accountable ability to link information on costs and spending with data on out- to many different constituencies, including country governments, puts and outcomes. Measures of cost-effectiveness and efficiency are beneficiaries, their own governance structures, and a diverse group comparative. The maximum achievable value for money is based on of donors—each with distinct priorities. Likewise, country gov- historical data and comparisons across recipients and providers, con- ernments are accountable to both donors and their own citizens. trolling for other factors that might affect costs and performance. As A country thus embarks on planning its disease-control strategy, a result, data are most useful to funders or program managers when potentially conditional on simultaneous decisions from multiple they are comparable across time and context, and when spending actors with competing interests and with uncertainty on the year- is related to outputs and outcomes. Databases of this scope, detail, to-year availability of budgetary and donor funds. and quality remain scarce, however. The definition here implies that incentives are essential to manag- In conducting the analyses for this report, the Working Group ing this complex landscape, and thus that the explicit and implicit has faced challenges in obtaining data on donors’ actual costs and incentives operating between funders and recipients should be struc- spending, and sometimes even in obtaining ex ante budgets. Where tured to enhance value for money. To illustrate, consider a standard available, budgets themselves are of limited use because they are grant agreement between a bilateral donor and a recipient country organized by input type, rather than by interventions, outputs, or government. These agreements usually contain health goals and outcomes. The inability to link money to outputs and outcomes, thus objectives, a description of activities, a budget of the inputs required clearly defining and prioritizing “what we are buying,” is reflected to carry out the specified activities, and requirements for routine in the difficulty of measuring value for money in typical grants and reporting and financial audit. The standard grant agreement thus contracts. This is a serious limitation of the analysis, and represents creates incentives to spend along the approved budget, and to report a value for money priority in its own right. activities as specified in the grant agreement. But resource-constrained governments often have other press- Generating incentives for value for money: a ing priorities and, given the fungibility of government resources, framework for funders may reasonably prefer to shift scarce funds away from health sector activities already funded by an external donor. Should government As an international group committed to achieving the global com- funds be shifted away, the recipient would produce less health and munity’s ambitious health goals amid a plateau in donor spend- services, but that variation in performance would be hidden from ing, this Working Group focused on value for money from the the donor and thus would not affect remuneration or subsequent perspective of a global health funder (box 2.2). An external funder funding allocations. With this design an agreement limits spend- of health policies and interventions, such as a bilateral donor or ing through a hard budget constraint, but otherwise contains no global health partnership, is just that—a funder. The funder does explicit incentives for efficiency, offers few incentives for effective- not set national policies, produce health commodities, or provide ness, and may create perverse incentives to over-report results in health services, all of which must be optimized to achieve health order to meet agreed targets. goals. Indeed, with few exceptions, external funders usually pay These forces comprise the “incentive environment,” where the only a small portion of the costs for purchasing commodities or conditions governing grants may create financial and nonfinancial providing services in a country. 7 What is value for money? for value is What

Box 2.2 Selected global health funders multilateral development banks, or civil society organizations, • The Global Fund to Fight AIDS, Tuberculosis and Malaria each of which can exercise more direct control over program • GAVI Alliance implementation. • World Bank Thus while an international nongovernmental organization or a • U.S. President’s Emergency Plan for AIDS Relief bilateral donor agency can directly hire and supervise doctors and • U.S. President’s Malaria Initiative managers, an organization like the Global Fund must take a more • Bill & Melinda Gates Foundation indirect approach. The Global Fund can award grants to attain • UNITAID predefined health objectives. It can measure the performance of • European Commission its grantees against these objectives. It can give or deny approval of • Bilateral Development Assistance Committee donors procurement proposals by principal recipients. And, importantly, it can offer or withhold payments, bonuses, rewards, and other incentives based on measured performance.1 Further, because of Rather, a funder must work with a very limited toolbox to lever- its standards-setting authority, the Global Fund can ensure that age or create value for money incentives among recipient country its principal recipients have and exercise analogous authorities over governments and implementing agencies.ii Global health funders subrecipients such as facility managers, program managers, and can exercise six authorities to achieve their objectives: subcontractors. • The authorityto grant money. Keeping the limited authorities in mind, this report sets out a • The authorityto set standards for allocating or disbursing funds. framework of value for money domains for decision-making. There • The authorityto verify the performance of recipients or suppliers are four domains within the grant or funding cycle where value for against those standards. money can be improved: allocation, contracts, costs and spending, • The authorityto finance or facilitate technical assistance in support and performance verification (figure 2.1). The approach to each of standard adherence and performance verification. domain is necessarily collaborative and starts with country policy- • The authorityto convene stakeholders in order to improve stan- makers in partnership with funders. dard adherence or performance verification. • The authorityto iterate the exercise of these authorities in a How can resources be allocated ex ante to maximize impact on project cycle, making adjustments to improve value for money.iii HIV/AIDS, tuberculosis, and malaria? Despite expanding evidence These authorities derive from the charter or founding doc- on what works most cost-effectively to reduce disease, national and uments, from established precedent, and from the consent of donor funding does not consistently support best practices. Using an organization’s board members. Note that these authorities stronger evidence thresholds for funded interventions through a differ substantially from the tools available to sovereign states, fair process, shifting funds to best value commodities, and encouraging the use of economic evaluation and modeling to inform ii. Definitions of “value” also depend on the type of intervention that national and Global Fund resource allocation can help drive value a global health funder supports. While this report focuses on funders for money. that support the Global Fund–related global health goals—where value is defined as incidence, prevalence, and access to quality services How can contracts and agreements between the Global Fund and with equity—funders with other goals such as market shaping, as with its recipients be structured to create stronger incentives for value for UNITAID, will define value differently. money? Current agreements provide only weak incentives for impact. iii. The Working Group’s background paper entitled “Value for Money Directly connecting performance to a portion of funding, linking in Health: A Framework for Global Health Funding Agencies” defines performance payments to progress on the most important indica- “value for money” as it applies to a health funding agency and enumerates tors of quality and impact, and supporting performance incentives the limited number of policy instruments or “tools” available to the Global between principal recipients and service providers are ways to get Fund, which are derived from these basic authorities. more value for money. 8 What is value for money? Figure 2.1 Value for money domains for Decisions in each domain directly affect the availability and global health funders quality of services provided to those at-risk for and suffering from disease, and ultimately the collective impact. Consider why a bed How can resources How can contracts be be allocated structured to create net distribution program might run into trouble in implementation ex ante to maximize stronger incentives for value for money? value for money? and how better decisions and incentive structures in each domain might help solve these problems. Allocation Contracts At the allocation stage, a lower value for money net can be eligible for purchase rather than a long-lasting insecticide-treated bed net, as recommended by the World Health Organization. Too many or too few nets may be purchased due to the inability to assess the Performance Costs and veriﬁcation spending mix of interventions that will have the most impact on disease incidence, or due to inaccurate demand forecasting or unknown How can performance How can costs of be veriﬁed rigorously, program efficiency. and spending be better to generate greater tracked to improve incentives for value for At the contract stage, nets may be budgeted, but no incentives— value for money? money? financial or nonfinancial—will guarantee their availability and use in the most-affected areas. During implementation, supply chains may be slow to move product from warehouses or out to front-line providers, but program managers may have no data or leverage in Source: Authors. real time to deal with these problems. At the cost and spending stage, the cost to distribute each net to the right population is unknown, and money may be wasted. How can costs of and spending on commodities, supply chains, and And at the performance verification stage, no household or facil- service delivery be better tracked and used to improve value for money? ity data are collected in a representative way, and the funder and the Improving the scope, completeness, and timeliness of reporting to government program must rely on principal recipients’ self-reports commodity price tracking systems, identifying a core package of of unknown accuracy to determine whether to continue funding, services for more extensive analysis of costs, sharing cost and spend- decide on how to adjust the management and delivery strategy to ing data with partners and the public, and developing a strategy to be more effective, or provide feedback to the allocation process to use cost and spending data to drive value for money improvements improve efficiency. throughout the grant cycle are main value for money agenda items. At each stage, within each domain, funders—both countries and international actors—can successively lose value from their How can performance be verified and evaluated rigorously, to gener- investments. ate greater incentives and accountability for value for money? Despite Together all four decision-making domains reflect a system- well-known discrepancies between self-reported administrative atic agenda for value for money that previously may not have been data and actual performance, the Global Fund has relied on weak interpreted as essential to impact. Chapters 3–6 each track one of instruments to verify the accuracy of self-reports. Defining a subset the four domains, analyzing current practices and offering recom- of essential indicators to receive strengthened performance verifica- mendations to enhance value for money at every stage of the Global tion, independently validating the accuracy and quality of principal Fund grant cycle. recipient’s self-reported results using rigorous and representative The four domains also make implicit what is not value for money measurement instruments, and complementing output verifica- or efficiency. One such example is that the term “value for money” tion with impact evaluation for interventions of unknown efficacy has been misused to refer to cutting country budgets by 10 percent are essential to aligning incentives and creating accountability for or more amid financial uncertainty. Efficiency is not achieved by impact and value for money. reducing budgets without considering health outcomes or outputs. 9 What is value for money? for value is What

While the Global Fund Board has already identified a subset obtaining higher priority for the agenda from the Secretariat, Board, of the recommendations as priorities (particularly those on mar- and key strategic partners, adapting and adopting the report’s rec- ket shaping and optimization of commodities), other areas have ommendations into operations, and implementing and evaluating attracted less attention, including the need to reform and redesign the value for money agenda with the new funding model. the performance-based financing system, strengthen performance Value for money is not merely a checklist, a principle, or another verification, and use cost and spending data to improve the efficiency task on the to-do list—it is the core business of any health funder. of procurement, supply chains, and health care delivery.iv This report The Working Group hopes that this report can contribute to the identifies actions to take now—and build into the business model effort. over the next two to five years. The challenges are recognizing that a systematic value for money Note agenda can influence all aspects of the Global Fund’s business; 1. The Global Fund (2011c).

iv. Indeed, some of these recommendations are not new. Variants of recommendations have been issued by the Global Fund’s High-Level Independent Review Panel on Fiduciary Controls and Oversight Mechanism in 2012, the Fund’s Technical Evaluation Reference Group Five-Year Evaluation in 2010, and the Fund’s Office of the Inspector-General in recent years. Fur- ther, a recent U.S. Institute of Medicine evaluation of the U.S. President’s Emergency Plan for AIDS Relief highlighted similar issues, reinforcing the universality of the value for money challenge.

Chapter 3 Planning allocation

Recommendations • Choose from a menu of effective and cost-effective interventions and commodities • Identify and target key populations with appropriate interventions • Optimize investments for the greatest health impact • Improve ex ante budgeting and transparency on spending

A value for money agenda for any funding agency must start at the Overview beginning: Which programs and interventions are eligible for the agency’s financial support? And how can the agency ensure that its The Global Fund adopted a passive approach to grant allocation, allocations best achieve objectives for disease control and health driven by the belief that its “demand-driven approach ensures that improvement? the money is going where it is needed most,”2 albeit with reviews These questions lie at the heart of “allocative efficiency,” or “doing for technical merit by the Technical Review Panel (TRP). Coun- the right things.”1 Allocative efficiency means selecting interven- tries were expected to optimize their portfolios, without a clear tions that achieve maximum health impact within a given budget resource constraint. The Global Fund relied on countries to do constraint. It typically requires tailoring interventions to geographi- this despite their often limited capacity to do so (lack of tools and cal and epidemiological contexts, and can be achieved either for a expertise in costing, epidemiologic surveillance, and modeling) single disease area or for a population’s health more broadly. Any and potentially without addressing political or economic conflicts funding agency must allocate its resources along a set of allocation of interest (political or religious objections to working with most- criteria. Such criteria may be explicit, as in the U.S. President’s at-risk populations, or political imperative to distribute funds to Emergency Plan for AIDS Relief’s (PEPFAR) 10 percent earmark many different constituencies). for programs targeting orphans and vulnerable children, or implicit, While well intentioned and consistent with the Global Fund’s as in the Global Fund’s historic approach to grant-making, where core principle of country ownership, this model suffered from seri- allocations were based on expressions of country demand. ous limitations. By failing to provide countries with a clear budget Despite the expanding evidence base on intervention effective- constraint, predictable funding windows, or rewards for efficiency, ness, both global and national funders continue to allocate resources the Global Fund created strong incentives for countries to maximize to interventions and intervention packages that do not provide their funding requests—often without considering actual need and the best value for money. To maximize the value for Global Fund other funding sources, or assessing their most pressing priorities given investments, this Working Group believes that the Global Fund scarce resources. As the Global Fund never had enough resources to must make good on its commitment to take a more active approach meet the full demand of all countries for all diseases—and arguably to grant allocations, including using stronger evidence thresholds never will because country demand is always increasing and also for funded interventions, shifting funds to best value commodities, incorporates a desire to build general health systems and address and encouraging economic evaluation and modeling as part of the health challenges outside the Global Fund’s “big three” purview— proposal process. Together, a deliberate and coordinated approach demand was insufficient as a mechanism to ensure an effective and to resource allocation will enable countries and funders to drive efficient response to HIV/AIDS, tuberculosis, and malaria. While better value for money from their shared investments. the Global Fund would reject the most incomplete or inappropriate 12 Planning allocation Planning

proposals, it rarely pushed countries to select the most effective or Figure 3.1 Misalignment between men who cost-effective interventions and commodities. Some interventions have sex with men’s share of disease burden that could be effective were nonetheless proposed in a manner that and funding level either disregarded the dynamics and distribution of disease nation- Share of preventive expenditure on MSM ally and subnationally or ignored national implementation capacity. Percent Share of AIDS cases among MSM Given the overwhelming public health evidence on tailoring 60 responses to the characteristics of an epidemic in a country, the

Global Fund is increasingly recognizing that efficient allocation 40 is an essential part of the value for money agenda. With the new funding model, the allocation formula represents a way to distribute global resources across many countries more methodically, but 20 much work remains on optimizing investments within each country. Amid severe budgetary constraints in a difficult economic climate, any spending on interventions—that are poorly coordinated, not 0 Peru Chile Brazil

targeted to key populations, not cost-effective, or, worse, not even Bolivia Mexico Panama Uruguay Paraguay Argentina Costa Rica Guatemala known to be effective—represents a missed opportunity to improve El Salvador

health through high-impact interventions. Dominican Rep. A widespread failure to explicitly target and reach popula- MSM is men who have sex with men. tions most at risk has had serious consequences, both for stop- Source: Forsythe, Stover, and Bollinger (2009). ping transmission of a disease (prevention) and reducing mortality and morbidity from that disease (treatment). This failure has been particularly acute for HIV/AIDS (and arguably much less so for While these misalignments speak to aggregate inefficiencies tuberculosis and malaria) because of the political sensitivity of the in the global response, the Global Fund itself (alongside most relevant populations at risk (sex workers, injecting drug users, and other donor agencies and national governments) shares only lim- men who have sex with men [MSM]), where political barriers to ited information on its investments, particularly on the mix of acknowledging these populations are coupled with fear of further interventions and commodities financed by its grants and on the discrimination against them if prioritized for outreach. populations targeted by those interventions. For each country a Underinvestment in high-risk populations appears to be obvi- donor agency may be aware of its own distribution of funding by ous and widespread. MSM in Latin America tend to receive scant intervention. But this information is rarely available to the public, resources for HIV prevention relative to their central role in the other donor agencies, and often even the country itself. Without region’s epidemic (figure 3.1).3 In Costa Rica, the most extreme case, such information on the distribution of investments—both by an estimated 60 percent of all infections occur among MSM, yet intervention and targeted population—it is not possible to assess this high-risk group receives only about 1 percent of the country’s the overall allocative efficiency of a country’s intervention mix. As spending on HIV prevention.4 A similar misalignment occurs in noted in the Institute of Medicine’s PEPFAR evaluation report, Ghana, where more than 99 percent of HIV/AIDS funding failed even recipient countries are at a loss for “where (geographically) the to specifically reach high-risk populations, despite that 76 percent of money is going and what services are being supported so that they HIV transmission in Accra has been driven by the commercial sex can identify unmet needs.”6 Because allocative efficiency requires industry.5 While correlating funding proportions with the distribu- that actors and donors share information to optimize a country’s tion of disease burden provides only a crude assessment of alloca- intervention mix, suboptimal allocations are likely to prevail with- tive efficiency, such extreme misalignments suggest a joint failure out full transparency. of donors and country governments to deploy strategic investment This limited sharing of information seriously constrains both and target populations most at risk. achieving and analyzing allocative efficiency. After all, how can 13 P lanning allocation lanning

we determine whether resource allocation is efficient if we do not Fund subsidizes second- and third-line antiretroviral (ARV) and even know what the allocation was? Even so, some sources suggest tuberculosis medications in several low-income countries where a misalignment of donor financing given current evidence on high- first-line coverage and, perhaps, quality remains low. In the last impact and cost-effective interventions. For the Global Fund’s HIV round of commodity spending reported to the Price and Qual- grants, for example, observed misalignments include low uptake of ity Reporting (PQR) system, spending on second- and third-line medical male circumcision in Global Fund grants (despite its large ARV and tuberculosis medications represented the majority of and proven potential to reduce HIV transmission)7 and large alloca- total medication spending.i Spending on second- and third-line tions to prevention interventions of questionable efficacy (behavior treatment is likely to increase as more patients fail first-line treat- change communication and various training interventions) that ment, which may imply tradeoffs in reaching the Global Fund’s often lack evidence from rigorous evaluations. expressed disease goals and in achieving equitable access or (still The chosen interventions (even those determined “cost-effec- implicit) disease goals specific to drug-resistant strains.ii The Global tive”) must be determined jointly with their intended key popu- Fund likely offers second- and third-line regimes for reasons of lations. For example, despite a progressive official policy in place “gap-filling” role as countries take on (though incompletely) raising (the Sexual Orientation and Gender Identities Strategy8), whether coverage and quality of first-line treatment. Or because patients this policy influences disbursement decisions is unclear: “of the failing on first-line treatment that stay on first-line can develop $1.5 billion in funding allocated to these six countries since 2001, resistant strains of the virus. Or for ethical reasons of continu- only 0.07 percent was for programs specifically targeting [gay men, ing treatment for those already being treated,14 though there are other MSM, and transgender individuals]. Moreover, the majority also reasons for doing otherwise—for example, based on the “fair of this support is concentrated in just one of these six countries innings” principle.15 Ethical arguments on equitable access aside, (Namibia).”9 A recent amFAR report notes that Global Fund pro- the Global Fund risks pursuing an ad hoc approach with an unclear posals often take a “tokenistic approach” to MSM and other key disease-control objective without a systematic policy to tackle the populations in which these groups are mentioned in passing but spread of drug resistance and subsequently the use of second- and do not receive specifically targeted (let alone budgeted) activities.10 third-line or other newer treatment regimens. Moreover, there is evidence of underinvestment in harm reduction in countries where the epidemic is fueled by injecting drug users.11 i. Analysis of the PQR indicates that the majority of tuberculosis com- HIV interventions are not always clearly or appropriately tar- modity funding in the last available round (round nine) went to second-line geted to high-risk groups. In a sample of grant agreements from drugs (85 percent), though in six of the nine rounds the majority of com- five countries with varying epidemiologic profiles (Ethiopia, India, modity funding went to first-line drugs. For HIV commodity purchases, Nigeria, the Philippines, and South Africa), most funding over the majority of purchases were spent on first-line drugs until the last round, 2002–12 was either earmarked for the general population, or did when second-line spending was higher (88 percent). Note, however, that not indicate a specified target group.12 While this finding does not only half of total spending on PQR-reportable commodities is actually necessarily imply that the Global Fund itself did not tailor inter- reported and reflected in the PQR, so this may not be a representative ventions to specific populations, it suggests that other funders such sample of spending. as PEPFAR are unlikely to know what populations Global Fund ii. In March 2013, for example, the government of Zimbabwe announced grants reach, and to respond accordingly. that it would be financing third-line ARVs as part of its approach to HIV Further, there are still opportunities to scale up the most cost- treatment services. But there are still an estimated 238,000 people liv- effective interventions in the pursuit of disease-control objectives. ing with HIV in Zimbabwe that have not yet accessed first-line treat- For example, first-line regimens are more cost-effective than secondment, and retention rates on first-line treatment are around 70 percent and third-line regimens. In countries that have not fully scaled up (HEALTHQUAL International 2012). Similarly, in 2011 Zambia the coverage and quality of first-line treatment to the entire eligible announced that that it would be providing free third-line ARVs to more population, investing in second- and third-line treatment is not than 200 people that need them. Uganda similarly has been offering sec- likely the most cost-effective intervention.13 However, the Global ond-and third-line ARVs on a limited basis since 2010. 14 Planning allocation Planning

Even within the same categories of medication, shifting resources diagnostic, and treatment interventions are now accessible, and to more cost-effective formulations can yield health gains and sav- epidemiological and economic models and available data can be ings. For example, research in South Africa found that the most used to estimate the ex ante optimal mix of interventions and target commonly used first-line ARV combination (stavudine, lamivudine, populations to prevent disease or reduce mortality. Here, the Global and nevirapine) was among the least cost-effective and efficacious.16 Fund and countries share a common interest in optimizing their Shifting to another World Health Organization (WHO)–approved investments—and a common challenge that can only be addressed first-line regimen would thus be a win-win, both in improving through collaboration and mutual support. patient outcomes and in saving money. The extent of these poten- Although the Working Group advocates greater use of cost- tial gains will likely increase in the coming years due to the growing effectiveness criteria in investment decisions, it recognizes that need for second-line treatment.17 cost-effectiveness, particularly in a clinical setting, is but one fac- Among nonmedical health commodities such as condoms and tor in decision-making. Other factors are the overall efficiency and bed nets, there are also likely to be savings from shifting investments effectiveness of the delivery and implementation, and the context to better value for money commodities. The Global Fund along with in which a commodity or intervention is purchased—for example, the U.S. President’s Malaria Initiative has been the main purchaser the supply, acceptability, durability, or user-friendliness of a prod- of insecticide-treated bed nets (ITNs), standardizing to some extent uct—which could in turn influence its ex post cost-effectiveness. ITN purchases through better reporting of prices and specifications Moreover, slavish devotion to static cost-effectiveness can ignore (durability, acceptability, usability). But countries receive funding relevant long-run dynamics. For example, standardizing procure- for more than 200 different types of ITNs—including requests for ment could reduce prices (see chapter 5) and competition, which customized labeling or nonstandard sizes. Some have hypothesized may affect the long-run entrance of competitors and the long-run that diverse ITN specifications could be critical for uptake, but to value for money of products. date no evidence supports or refutes this hypothesis, and choosing The Global Fund and PEPFAR each have distinct goals and to fund different kinds of ITN comes at higher cost with no evi- objectives, but some need clarifying. For example, both the Global dence of marginal benefit. Under some assumptions on ITN uptake Fund and PEPFAR fund programs to mitigate the adverse effects of conditional on observed specifications, this inefficiency appears AIDS on orphans and vulnerable children (OVC). But the relative substantial. One report finds that more than $340 million could cost-effectiveness of different OVC interventions should be judged be saved worldwide by purchasing more cost-effective long-lasting not on their ability to prevent HIV/AIDS per dollar of investment, ITNs over the next five years.18 but on their ability to improve OVC well-being per dollar. Cost- In short, better incorporation of cost-effectiveness criteria for effectiveness still applies, but the measure of effectiveness will differ. procurement decisions could produce effectiveness and efficiency With unclear objectives and intended outcomes of OVC pro- gains—though cost-effectiveness must be balanced by acceptabil- grams, there has been a consequent lack of consensus on indicators ity, usability, timeliness, market stability, quality, and other local used to measure the effectiveness of these programs. However, recent considerations. work in Kenya and South Africa—supported by the United Nations Children’s Fund—illustrates that rigorous measurement and evalu- Opportunities and limitations ation of OVC interventions is eminently feasible,19 while recent efforts by PEPFAR to better define OVC program outputs and With stagnating support for global health funding, there is a moral outcomes can also be adopted by Global Fund–supported efforts.20 imperative to spend Global Fund money on interventions and com- Goals and objectives also need clarifying in health systems modities that are effective and cost-effective, reach those most at strengthening (HSS) investments by the Global Fund.21 HSS invest- risk, and realize disease-control objectives. In the 10-plus years of ments require clarity on and links to the expected outcomes, such the Global Fund, the epidemiological knowledge base for preventing as increased access, quality of care, efficiency, financial risk protec- and treating HIV/AIDS, tuberculosis, and malaria has expanded. tion, responsiveness, and patient satisfaction. The goals of HSS Systematic reviews evaluating the effectiveness of prevention, investment are not mutually exclusive from that of disease-specific 15 P lanning allocation lanning

investments: each disease-specific investment can and should be subsequent steps of the grant proposal process (country dialogue, classified as having an HSS characteristic or building block. Given concept note, TRP review, approval). This is appropriate since alloca- the lack of clarity on expected goals of HSS investment, many HSS tive efficiency is most relevant to the total spending on disease investments focus on the WHO building blocks, which emphasize control in a country. system inputs (service delivery, health workforce, health informa- The new funding model—and the Working Group—envi- tion system, drugs, financing, and leadership and governance), but sion a country’s NSP as a foundational document, where value for less so their links to outcomes, however defined, as well as current money recommendations on “doing the right things” should be first incentives affecting each input. addressed. However, given the focus on the Global Fund and some For example, for human resources for health (under the building of the challenges around current NSPs,iii the four recommendations block of “health workforce”), a recent study examined the invest- below are more closely linked to the country dialogue, concept note, ments in human resources for health by three donor agencies—the and TRP review steps in the new funding model. GAVI Alliance, the Global Fund, and the World Bank.22 This study found that most GAVI Alliance and Global Fund grants finance Choose from a menu of effective and cost- health worker remuneration, largely through supplemental allow- effective interventions and commodities ances, with little information on how payment rates are determined, how any negative consequences are mitigated, and how payments The Global Fund’s country dialogue and TRP review of concept are to be sustained at the end of the grant period. Only a third of notes are important opportunities to shape allocation in accord with GAVI proposals and less than a tenth of Global Fund proposals evidence-based funding criteria. The TRP review has included an considered health workforce policies, despite a median share of explicit value for money component since 2011,25 and the TRP is 27 and 22 percent of grants devoted to human resource activities. now empowered to rank individual components within the grant Finally, cost-effectiveness as a criterion for decision-making is proposal according to its value for money review criteria.26 While sometimes critiqued as “unfair” to non-biomedical interventions, important, these changes will not be enough without explicitly particularly for HIV prevention. The approaches proposed by coun- recognizing the importance of cost-effectiveness, not just effective- tries should, for the most part, exclude interventions not effective ness and efficiency. in some dimension of the disease response. But there are examples A key recommendation of this report is to invest mainly in of non-biomedical—social or behavioral—interventions that have effective and cost-effective interventions and commodities, pro- been rigorously evaluated and can be cost-effective, though not vided as guidance to recipients through a predetermined “menu” against impact measures of HIV incidence. For example, peer sup- of options. The TRP should be mandated to ensure compliance port for ARV adherence and nutrition was found to increase the with this requirement during its review of concept notes, while timeliness of clinic and hospital visits in South Africa. And peer encouraging countries to innovate and experiment in the delivery mentoring for HIV counseling and testing was effective in increasing of these interventions as well as when there is an absence of proven testing of a HIV+ partner in Senegal.23 Despite the lack of evidence on many social and behavioral interventions on health outcomes, iii. A cursory review of current NSPs finds that most do not include any conditional cash transfers are an important and unique category significant analysis of choice or mix of intervention given disease dynamics, of non-biomedical intervention for which there have been statisti- most lack accurate and updated information on and scenarios of budgets cally significant declines in the incidence or prevalence of sexually and spending from different revenue sources, and most follow different transmitted infections (including HIV) and pregnancy.24 time periods (five to six years) than the Global Fund three-year grant cycle. Further, the donor coordination that would be required to address Recommendations these challenges is itself a difficult task, at least in the past. For example, a 10-country study on coordination for HIV/AIDS programs found that A country’s national strategic plan (NSP), the starting point for “incentives for coordination are weak and practice falls far short of policy the Global Fund’s new funding model, is intended to frame the intent” (WHO 2008, p. 1). 16 Planning allocation Planning

interventions or an apparent failure to slow the transmission of affordable, and critical for achieving certain disease-control goals. diseases. For example, one justification for expanding to second- and third- If the menu lacks proven interventions, if recipients prefer to line treatment is that adding such patients can be a marginal cost, invest in interventions not included on this menu, or if recipients whereas expanding first-line treatment may involve larger capital recognize a need to innovate and experiment, recipients can “opt investments, such as in infrastructure and outreach. The costs and out” of the intervention and commodity menu if their proposed impact of different paths and strategies must be carefully weighed. intervention is better value for money. Countries should provide This recommendation is consistent with the Board-approved local analysis showing that the proposed product or intervention market-shaping strategy focused on commodities (and not interven- would be more cost-effective within the local context to justify tions more broadly),28 and the Working Group recommends that their decision. If the country chooses not to provide such analysis, the Global Fund fully implement this strategy. The Global Fund’s it should be asked to pay the differential between its selected prod- Market Dynamics Committee identified opportunities for efficien- uct or intervention and the most cost-effective option. Further, cies through product optimization, incentives to use cost-effective recipients that deploy other interventions should be required to products, and expedited reprogramming. As these recommendations assess their effectiveness and cost-effectiveness through rigorous have already been developed and approved by the Board for imple- evaluation (see chapter 6). mentation by the Secretariat, these changes should be expedited to For grants seeking renewal, the Global Fund should assess the ensure that efficiency gains are realized as soon as possible. Specifi- cost-effectiveness of each grant’s intervention and commodity mix. cally, the Global Fund Secretariat should implement the following To speed the reprogramming of grants, the Global Fund could recommendations as suggested by the Market Dynamics Committee enable countries to retain the savings generated by shifting to a and approved by the Board in May 2011: more efficient intervention and commodity mix. • Optimize commodity purchases using cost-effectiveness analy- Among interventions in treating and preventing the big three sis. Identify gaps in product quality assurance for procurement diseases, the evidence on proven interventions for HIV preven- guidance, and identify partners or processes to fill those gaps. tion (beyond male circumcision and conditional cash transfers), • Require that recipients opt out of purchasing cost-effective particularly for the key populations of MSMs and sex workers, is products. Develop credible and reliable process to assess opt- still developing.27 So interventions in these areas will likely require out requests. ex ante justification of plausibility and epidemiologic importance, • Expedite reprogramming processes to allow principal recipi- combined with rigorous evaluation on key outcome or impact mea- ents to absorb new cost-effective technologies or respond to sures (see chapter 6). These requirements should not be seen as oner- new evidence. ous but rather a means to document ongoing experimentation and • Ensure principal recipients have first right to savings from adopt- innovation in the search for effective interventions. ing higher cost-effective products or increasing cost-effective To encourage countries to pursue the most efficient and equitable deployment. strategy, the Global Fund needs to develop a systematic policy on These recommendations also create financial incentives by enti- prioritizing high-quality first-line treatment of HIV/AIDS and tling recipients that switch to lower cost commodities of comparable tuberculosis in countries that are still scaling up first-line services quality to a “right of first use” and the opportunity to reinvest freed and have poor results on retention and completion of treatment. resources.29 Similarly, shared savings programs, being piloted in the Specifically, the Global Fund needs to articulate a clearer vision to United States, reward health care providers for keeping per-unit realize goals on disease control and equitable access and to articulate spending below targets while maintaining quality.30 A proportion a policy on how to respond to drug-resistant disease transmission. of those cost savings are then allocated back to successful programs. Some members of the Working Group strongly agree that the Global Within the U.S. health care system, this tool is used to encourage Fund should offer funding for second- and third-line treatment only strengthened coordination and to reduce unnecessary or high-cost if countries have fully scaled up or quality-assured first-line treat- care. Likewise, the Global Fund could create incentives to reduce ment or if they demonstrate that second-line treatment is equitable, costs while improving value for money. Savings from improved 17 P lanning allocation lanning

efficiency could be returned to recipients, the country coordinating health insurance recommended “a policy and institutional mecha- mechanism, or other implementers as appropriate (and as agreed nism . . . to assess the cost-effectiveness of new health technology on before implementing the program). and make recommendations for inclusion or not in [insurance-] A critical issue for this recommendation is how the Global Fund funded services.”34 In addition, with the Tunis Value for Money will obtain a menu of eligible interventions and commodities for declaration, many countries plan to build capacity to conduct cost- each disease area. The Working Group recognizes that the Global effectiveness analysis and carry out health technology assessments of Fund is constrained by a lack of appropriately helpful technical new interventions as a tool to rationalize scarce national resources guidance on the cost-effectiveness of commodity purchases. While for health. Indeed, more countries are willing to address the financial the WHO has helped inform medicine purchases through its treat- sustainability of HIV programs through strategies that prioritize ment guidelines, WHO guidance tends to focus on quality assur- interventions and improve the efficiency of service delivery.35 ance rather that cost-effectivenessiv (with exceptions for specific To draft the terms of reference for such an exercise, the Global categories, such as multidrug-resistant tuberculosis31), and on clini- Fund will need to agree on the key principles and methods of health cal treatment rather than prevention or population interventions technology assessment. For example, the National Institute for (such as bed nets and condoms). Moreover, WHO guidance on cost- Health and Care Excellence published the 2013 edition of the Guide effectiveness is often deemed “weak” under the Grading of Recom- to Methods of Technology Appraisal, which reviews the principles and mendations Assessment, Development and Evaluation methodology methods of health technology assessment and appraisal within the for not relying on randomized controlled trials.32 Because there is institute’s appraisal process.36 The menu should be regularly updated limited guidance and quality assurance for nonclinical commodities, to reflect emerging evidence, new innovations, and evolving cost and because current guidance rarely incorporates affordability, the structures for existing interventions. Global Fund often lacks the technical expertise to inform its product In addition to obtaining a menu of cost-effective interventions, purchases. Moreover, given a dynamic environment with shifting the Global Fund could better house and share the results of health commercial demand and emerging scientific evidence, guidance technology assessments with country coordinating mechanisms will require regular modification to reflect changing conditions. and principal recipients, which in turn should use such information Many members of the Working Group recommend that the in writing their concept note. The country dialogue process and Global Fund formally request such a menu from its key technical subsequent TRP review should also encourage the incorporation partners such as the WHO. If the technical partners are unable to of cost-effectiveness analysis (both guidance used by the Global provide such a menu, the Global Fund must commission it from an Fund and by independent research) as an indicator for well-designed independent expert body. A 2011 Results for Development report and actionable NSPs. The TRP should disseminate relevant cost- prepared for the Market Dynamics Committee also suggested that effectiveness research, which should be used in all parts of program the Global Fund commission expert guidance for key commodities, design, particularly during the country dialogue phase. such as from the UK National Institute for Health and Clinical The Global Fund’s new release of guidance through “strategic Excellence, “to conduct robust comparative cost-effectiveness analy- investment guidance and information notes” developed by techni- ses of two or more WHO-recommended products and provide that cal partners is one important but limited step to ensure the value information to the Global Fund and its recipients.”33 for money of new concept notes and renewal of grants.37 Each note Further, this is an area of growing interest to recipient country needs to better reflect value for money and cost-effectiveness criteria. governments. In South Africa an analysis of the first 18 months of For example, the recent “Strategic Investments for HIV Programs” note describes several basic programs that have high impact while iv. See, for example, the WHO Prequalification program (WHO 2013) referring applicants to review “the most recent technical and norma- and the WHO Pesticide Evaluation Scheme that primarily aims to “[study] tive guidance related to these high-impact interventions.” Rather the safety, efficacy, and operational acceptability of public health pesticides than refer grant applicants to technical partners for guidance, the and developing specifications for quality control and international trade” Global Fund should make technical and normative guidance more (www.who.int/whopes/en/). explicit to its applicants based on the above process. 18 Planning allocation Planning

Countries have innovated—and will continue to innovate— Box 3.1 Redirecting resources to hot spots of in a dynamic epidemiologic and economic context with Global infection and transmission Fund support. This recommendation ensures value for money of “These interventions, however, would be misdirected and investments by largely investing in proven interventions that are used inefficiently if we did not understand what drives already effective and cost-effective. Countries should nonetheless be the HIV epidemic. Fortunately, we know more about the encouraged to experiment, innovate, and learn, particularly when epidemic today than ever before and the insights we are the evidence base is still developing.38 Indeed, the Global Fund’s gaining present major opportunities to sharpen the im- Affordable Medicines Facility for Malaria represents a unique and pact of our interventions. . . . So, in many settings HIV large-scale experiment that contributed to the evidence base on exists in clumps—or hot spots—amid a sea of much lower malaria treatment and that countries may choose to draw on when levels of infection. . . . In fact, there are hot spots within developing their proposals.39 hot spots. Within the highest prevalence corner of South Africa, a study has found that up to a third of infections Identify and target key populations with may occur within just 6 [percent] of the area. And, within appropriate interventions those hotspots, we see that the risk of infection is piled upon specific small groups. . . . Now that we have the The investment case in each concept note should reflect an under- tools and resources, we can leverage this new intelligence standing of the key populations driving new infections, address to squeeze even more impact out of the resources we the country’s strategy to better reach these populations, and target have. Our computer models suggest that impact could “hot spots” of disease transmission with appropriate interventions. increase by 20 [percent], just by redirecting the same re- Targeting is essential, as a nominally cost-effective intervention sources to the populations at greatest risk of infection package may not be effective or cost-effective if it is not appropriately and transmission.” tailored to reach key populations. In its most recent replenishment Source: Hallett (2013). the Global Fund has prioritized better targeting, particularly for using geographic and epidemiologic data to identify (and target) the foci of HIV transmission. Likewise, new Global Fund Executive facility capacity for stock-outs, diagnosis, and service readiness. Director Mark Dybul has publicly stated that using hot spots to As necessary parts of the micro-targeting approach endorsed by improve targeting is a critical disease-control strategy. Executive Director Mark Dybul, these reviews should be systemized, As countries strive toward a generation free from HIV/AIDS, scaled up, and integrated in the grant-making process to improve tuberculosis, and malaria, optimizing national interventions to allocative efficiency and drive value for money. Moreover, facil- reflect the subnational diversity of disease transmission takes on ity surveys will have only limited impact in identifying hot spots added importance, and new concept notes should reflect tailored beyond populations already seeking care. New measurements and subnational approaches (box 3.1). In a study with minimal data surveys will be required. inputs, the optimal HIV intervention mix in two different prov- To operationalize this strategy, the Global Fund should require inces of South Africa (Kwazulu-Natal and Western Cape) vastly that concept notes use available data (even if out of date) to describe differed.40 the distribution of new infections within a country across key popu- To better target high-risk populations and identify geographi- lations—not only MSMs, sex workers, and injecting drug users, but cal hot spots, the Global Fund has undertaken mapping exercises also key epidemiologic parameters of gender, age, place of residence, within its “impact reviews.” The first two reviews were conducted in and geographic region (such as province or district)—that represent Cambodia and Thailand, mapping the geography and characteristics the main modes of transmission and infection. of the epidemics. In addition to these impact reviews, Burkina Faso The Global Fund should require that applicants collect data, and the Democratic Republic of the Congo have conducted Service when they are not already available, and submit evidence for geo- Availability and Readiness Assessments, in-depth reports measuring graphic locations and establishments that can be identified as hot 19 P lanning allocation lanning

spots, in conjunction with technical partners. Moreover, the Global Moreover, designing an intervention mix will differ depend- Fund can encourage applicants to designate part of the grant for ing on the stated budget constraint, as an optimized investment collecting such data of key populations that can help shape future case under “full expressions of demand” will differ greatly from an investments. Innovation is needed in identifying and characteriz- investment case given actual budget constraints. Funding interven- ing hot spots, particularly in countries that have not surveyed key tions at some level of target coverage under “fully funded demand” populations for political reasons, or where the quality and rigor scenarios does not optimize impact on disease under the actual of data collection has been weak. Such information on the aggre- budget constraint, suggesting that planning under a range of budget gates of key populations should be regularly shared and reported scenarios will be important. to the Joint United Nations Programme on HIV/AIDS database The Global Fund and the TRP should require that all con- as a public good, which in turn will further drive value for money cept notes and proposals justify their program design based on a among all donors, not just the Global Fund.41 comprehensive assessment of epidemiologic and cost-effectiveness data, focusing on identifying key populations within a country and Optimize investments for greatest health impact selecting cost-effective interventions for them. Better budgeting by intervention mix will be essential to achieving such optimization. Beyond ensuring that funded interventions are effective and cost- While the Working Group recognizes the importance of models effective, and with an eye on the key populations driving incidence and tools to optimize investments, these tools can range in simplic- of the big three diseases, the Working Group urges the Global ity and complexity, and the technical expertise required to conduct Fund to ensure that the funded intervention mix is tailored to such analyses will also vary. While adopting higher sophistication local disease epidemiologic dynamics and implementation capac- should be encouraged by the Global Fund, there will be a need for ity. To align incentives with disease-control goals, the Global Fund technical assistance, supported through multilateral partners such should clarify its institutional objectives—that is, the outcomes it as the WHO or bilateral assistance such as PEPFAR or other Global hopes to attain through its grant-making, and the time horizons Fund monies allocated specifically for country use. for those objectives. The Global Fund has envisioned in its strategy Encouragingly, the Joint United Nations Programme on framework a “world free of the burden of AIDS, tuberculosis, and HIV/AIDS and World Bank are working together to develop 15–20 malaria,” and targeted saving 10 million lives and preventing 140– country “investment cases” that will rely on detailed epidemio- 180 million new infections by 2016.42 But focusing on preventing logic and economic modeling of priority countries for the Global new infections will yield a very different intervention mix from a Fund. For priority countries and diseases (tuberculosis and malaria) portfolio optimizing for persons enrolled in treatment within the not covered in this process, the Global Fund should request that next five years. For example, whether the Global Fund optimizes such analyses be conducted by technical partners. These analy- for person-years on ARV treatment, lives saved (within 1 year or ses will build on the WHO’s standardized cost-effective analysis 20), or number of HIV infections averted will result in dramatically model, “Choosing Interventions that are Cost Effective” (WHO- different optimal portfolios.43 CHOICE), which reports on costs and effects of health interven- While reaching those targets would greatly improve the health tions in 14 epidemiologic subregions using a generalized cost-effec- and welfare of millions of people worldwide, reaching them would tiveness analysis method.44 But because WHO-CHOICE results not by itself be enough to free the world of HIV/AIDS, tuberculo- are standardized and not updated regularly, within-country varia- sis, and malaria. A world free of HIV/AIDS will require targets on tion and flexible country parameters are not taken into account.v infections averted, which the Global Fund has not yet established. So, the Global Fund could commission key technical partners for To achieve both the strategy framework’s vision and targets, each each disease to adapt WHO-CHOICE so that countries can then recipient will need to design an intervention mix tailored to key populations, all while abiding by the Global Fund’s guiding prin- v. In recent years WHO-CHOICE has been adapted, however, to develop ciples of country ownership, human rights, value for money, and country-specific estimates for three to four countries each year depending performance-based funding. on country interest. 20 Planning allocation Planning

use it to input country-specific information and in writing their Box 3.2 Data requirements to optimize for concept note. impact To encourage countries to optimize their intervention package • Length of time over which optimal allocations are to be for key populations, the Global Fund could make funding available determined. to countries for conducting such modeling work to be reflected in • Available budget or expected budget scenarios over the concept note. Such modeling is dually beneficial. It simulates time. the impact of different intervention mixes and targeted populations • Set of interventions for prevention and treatment. to triangulate an optimal investment portfolio given local institu- • Production functions for different interventions, speci- tional and resource constraints. And it generates an ex ante model fying the coverage achievable for an intervention as a of feasible performance targets and program impacts. The Global function of total resources allocated to that intervention. Fund could allocate this funding to countries wishing to conduct • Epidemiological profile of the population including these analyses with support from technical partners or experts. prevalence of HIV by gender, men who have sex with The simplest (and back-of-the-envelope) approach to optimizing men, sex workers, injecting drug users, and other key investments is identifying the key populations that represent the populations. major source of infections or new cases and then shifting resources to Source: Barnighausen, Bloom, and Humair (2013). reduce transmission in these populations. The Global Fund should encourage applicants to use analytics and tools, adapted for country- specific needs and data. Country applicants can develop over time Selected applicants who wish to go beyond what can be gleaned from using simple prefabricated tools to developing more complex, from existing data and information or the use of prefabricated tools tailor-made models, all with country-specific data. or models supported by technical partners (as described above) Several prefabricated modeling tools offer basic and simple alloc- can develop more nuanced guidance on their plans including their ative guidance at low costs. Most have been tested and applied in intervention portfolios and targeting strategy and investment levels. various settings, though not usually for resource allocation plan- For selected applicants the Global Fund should encourage com- ning in NSPs or specific grants. For example, Maude and others missioned expertise that can conduct the detailed, more complex (2011) offer a “free, Internet-based, user-friendly, and interactive economic evaluations for the concept note. As noted above, this can model of malaria elimination” to guide short- and medium-term be supported by technical partners from ongoing modeling efforts decision-making. However, they caution that the model’s “sim- or can be specially commissioned. ple” calculations should be supplemented by “more complex and Although models can vary in their complexity and data require- detailed models” to inform long-term strategies.45 Likewise, the ments, simpler models can provide clearer insights and recommen- online Malaria Tools software, developed by researchers at Impe- dations.49 As Barnighausen, Bloom, and Humair (2013) suggest, rial College London, attempts to optimize an intervention mix for the data requirements do not need to be more onerous than the due malaria control based on data inputs such as intervention coverage, diligence required to understand the geography and epidemiology of parasite prevalence, and seasonality.46 The Futures Institute has a disease and the costs of prevention and treatment in the country also developed modeling tools for needs assessment and the cost- (box 3.2). Applicants would not be required to plan their programs ing and planning of HIV interventions, all offered on its website.47 exclusively based on results from economic evaluation and model- And the U.S. government has recently piloted an HIV prevention ing. But they would be expected to articulate their application of resource allocation modeling pilot (HIV RAMP) to achieve the the optimization analysis and modeling to program design, and to U.S. National HIV/AIDS strategy “to focus efforts in communi- explain any major deviations (related to cultural, legal, and other ties where HIV is concentrated and to target resources on tailored implementation constraints, or concerns about human rights, eth- combinations of effective, evidence-based strategies.”48 The project ics, and equity). will develop a model that will help jurisdictions decide how best to In general these tools should be seen as a means for gaining more invest in HIV prevention. information and insight, and certainly not as a cure-all silver bullet 21 P lanning allocation lanning

or a binding straightjacket. Rather, it is because situations are so Box 3.3 Modeling as a success dynamic and complex that models can help broadly inform the “Based on my experience in Myanmar—a country where reasonableness of allocations to drive value for money or whether many believe there is no local information available and significant shifts in investments and strategies need to be considered no capacity to use modeling itself—I’ve found if we work to slow the transmission of diseases. As an epidemic changes, so too with local government staff, if we are transparent on the must strategies be altered. Indeed, recent modeling helped highlight modeling, and if we get all stakeholders involved to inform the importance of targeting hot spots in achieving greater impact, a us about the likely information to be, then we are able to strategy with little traction until the new leadership of the current use modeling despite limitations. executive director (see box 3.1). With more data on the effective- “We use these results to talk to decisions-makers, ex- ness of different interventions as well as strategies to deliver those plaining to them step by step. I’ve found that they really interventions, models can be updated and adapted accordingly. like the modeling results and they like to learn about the likely impact of what they are going to do. The program Improve ex ante budgeting and transparency on is implementing there. I would like to tell everybody that spending it’s possible.” —Yot Teerawattananon, HITAP, The lack of transparency on both planned and actual spending Ministry of Public Health, Thailand by intervention mix and target population, a major challenge to achieving overall value for money, prevents coordinated investment across donors and national governments. The Working Group urges Tool, the Resource Needs Model, and the Excel-based Marginal the Global Fund and other donors to increase transparency on the Budgeting for Bottlenecks costing tool, which enables countries detailed planned and actual distribution of funds through improved to plan and forecast “the potential cost and impact of scaling up budgeting and spending reporting. Such sharing of information is investments to increase the intake, coverage, and quality of high- needed to maximize health gains; sharing will make it possible to impact health interventions” for HIV and malaria (among other analyze the consistency of actual spending with a value for money health priorities).51 knowledge base and allow for decisions by a single actor to be made As noted above, the overall value for money in a country depends with full knowledge of what other actors are doing. on knowing the full portfolio of investments within a country, Pre‑existing budgeting and resource allocation platforms, and such as through national health accounts. Indeed, the Global Fund current data and information to explicitly cost the proposed inter- has supported national health accounts through the WHO since vention mix by the target population, are available but not necessar- 2012. To date there are 47 funded national health accounts, and ily used. Guidance on concept notes should encourage applicants to the Global Fund continues to work on extending guidance on disuse such platforms. As greater emphasis is given to the International ease subaccounts. To complement disclosure of budget data, sev- Health Partnership (IHP+) and its potential to harmonize report- eral countries have begun to fully institutionalize the System of ing to multiple donors, using the IHP+-supported tools will be Health Accounts 2011 framework, an internationally comparable more important. For example, the OneHealth Tool, a joint United methodology for comprehensive tracking of health spending as the Nations tool developed by multiple partners for the IHP+, supports standard platform facilitated through the Health Accounts Produc- strategic “planning, cost analysis, impact analysis, budgeting, and tion Tool (HAPT). The Global Fund has adopted a concern for the financing of strategies” for a national health plan, including all major HAPT at the aggregate national level and for counterpart financ- diseases and health system components.50 The software uses recent ing largely because of concerns of additionality and fungibility. But available data and epidemiologic and health systems models to help this neglects the multiple benefits of HAPT when disaggregated, countries set priorities within a national budget envelope (box 3.3). though these other potential functions of the HAPT have yet to be The OneHealth Tool’s cost estimation approach expands on pre- recognized or used systematically. When disaggregated the HAPT vious costing tools, such as Roll Back Malaria’s Malaria Costing could also be used to assess the interventions and key populations 22 Planning allocation Planning

supported by other financiers in the country. Moreover, the HAPT interventions. The Working Group urges the Global Fund to define when disaggregated can also help funders and program managers a set of effective and cost-effective interventions that is eligible for to understand the costs of service delivery of different delivery funding; clearly articulate its own institutional objectives; demand models and channels, such as integrated or vertical programs and high-quality, strategic proposals that justify the selected interven- community or hospital-based programs. tion mix within the national program, and with respect to cost- Encouragingly, the Global Fund has signed a memorandum of effectiveness; and disclose a detailed profile of its own investments understanding with the WHO to use the HAPT to assess aggre- to enable a coordinated, efficient joint response. gate counterpart financing for a given disease. The HAPT will be conducted in the 75 priority countries of the United Nations Notes Commission on Information and Accountability for Women’s and 1. Fan and Glassman (forthcoming). Children’s Health. The Working Group recommends that this work 2. www.theglobalfund.org/en/about/principles/. be prioritized for the Global Fund’s high-impact countries as well. 3. Forsythe, Stover, and Bollinger (2009). The OneHealth Tool and the HAPT should be tested and used 4. Forsythe, Stover, and Bollinger (2009). for assessing both aggregate and detailed counterpart spending and 5. Forsythe, Stover, and Bollinger (2009). for understanding the distribution of spending by intervention mix 6. National Research Council (2013), p. 470. and key populations. Countries can use these tools to seek better 7. Kinyanjui (2013). value for money in the context of sustainability and increasing 8. The Global Fund (2007b). national ownership of programs. Both tools, when combined with 9. amFAR and Johns Hopkins Bloomberg School of Public the NSPs and the Joint Assessment of National Health Strategies Health (2013), p. 2. tool,52 could reduce duplication in planning and reporting in most 10. amFAR and Johns Hopkins Bloomberg School of Public countries. If explicit about the interventions used and targeted to Health (2013). key populations, these exercises could enhance the value for money 11. Bridge and others (2012). of investments. 12. Fan and others (forthcoming). While the OneHealth Tool can inform strategic planning for 13. Resch and others (2006). a national health plan and improve the transparency of spending 14. The Global Fund (2011b). to countries and donors, countries need to test its effectiveness and 15. Williams (1997). appropriateness within the new funding model. These tools repre- 16. Bendavid and others (2011). sent supported approaches to budgeting and costing, various other 17. Stover and others (2011). tools developed with the OneHealth Tool53 and by such organiza- 18. Bahl and Shaw (2012). tions as Management Sciences for Health and Abt Associates that 19. Handa and others (2012); Heinrich and others (2012). may be as or more adaptable and flexible to specific needs.54 20. PEPFAR (2012a); www.cpc.unc.edu/measure/tools/child As is true for optimizing investments for the greatest health -health/child-status-index. impact, improving budgeting and expenditure transparency will 21. Fan and Glassman (forthcoming). also require technical assistance—for example, through technical 22. Vujicic and others (2012). partners and bilateral assistance or Global Fund monies allocated 23. Rodriguez-Garcia and others (2013). to countries for this purpose. 24. Over (2011). 25. The Global Fund (2011j). Summary 26. The Global Fund (2013d). 27. WHO (2011b, 2012b). In recognition of suboptimal portfolio allocations from its past 28. The Global Fund (2011d). demand-driven approach, the Global Fund is embracing a more 29. The Global Fund (2011i). active role in directing its resources toward the highest impact 30. Weissman and others (2012). 23 P lanning allocation lanning

31. WHO (2011a). 46. www1.imperial.ac.uk/publichealth/departments/ide/ 32. Fitzpatrick and Floyd (2012). research_groups/malaria/malariatools/. 33. Results for Development Institute (2011), p. 7. 47. http://futuresgroup.com/resources/software_models. 34. Matsoso and Fryatt (2013), p. 31. 48. Kouzouian and Forsyth (2013). 35. Katz and others (2013). 49. Lasry, Richter, and Lutscher (2009). 36. National Institute for Health and Care Excellence (2013). 50. www.internationalhealthpartnership.net/en/tools/ 37. www.theglobalfund.org/en/accesstofunding/notes/. one-health-tool/. 38. Fan and Silverman (2013). 51. www.who.int/pmnch/topics/economics/costing_tools/en/ 39. Bump and others (2012). index12.html. 40. Barnighausen, Bloom, and Humair (2013). 52. www.internationalhealthpartnership.net/en/key-issues/ 41. Fan and Glassman (forthcoming). national-health-planning-jans/. 42. The Global Fund (2011k). 53. www.who.int/pmnch/topics/economics/costing_tools/en/ 43. For recent work, see Barnighausen, Bloom, and Humair (2013). index12.html. 44. Edejer and others (2003). 54. www.msh.org/health-care-financing/costing-of-health-services; 45. Maude and others (2011). www.healthsystems2020.org/section/topics/hiv/hapsat.

Chapter 4 Designing contracts

Recommendations • Directly connect a portion of funding to incremental progress on performance • Link performance payments to incremental progress on the most important indicators • Support performance incentives between the principal recipient and service providers

Ensuring accountability for performance is crucial for health sys- PBF, but the design, structure, and implementation of their PBF tems worldwide, regardless of disease burden or income. Among systems vary widely. For example, the World Bank’s Health Results high-income countries, for example, the United States has unimpres- Innovation Trust Fund (HRITF) supports a portfolio of PBF projects sive health outcomes despite sky-high health spending—a dynamic within country health systems, where facilities and providers are paid created at least in part by a long-standing failure to align provider conditional on coverage and quality of health services.1 Similarly, the and patient incentives for better health at lower cost. With economic Inter-American Development Bank’s Salud Mesoamerica 2015 initia- growth, health spending will inevitably increase, but health or health tive seeks to close Central America’s health equity gap by conditioning system efficiency will not necessarily improve. a portion of funding on independently measured progress toward To increase the impact of each health dollar, the incentive envi- predefined coverage goals, health status gains, and policy changes. ronment must receive careful attention. One key tool for aligning Both the GAVI Alliance and the Global Fund have applied vari- incentives and promoting accountability is performance-based ants of PBF. Between 2002 and 2007 the Alliance’s Immunization financing (PBF), where an agency structures its payments to coun- Services Support program paid countries $20 for each additional tries so that they are at least partly conditional on demonstrable child vaccinated with three doses of the diphtheria-tetanus-pertussis improvements in health care coverage and health outcomes. Through vaccine.i Likewise, the Global Fund establishes PBF as a guiding PBF, country recipients gain incentives to achieve maximum health principle in its strategy framework.2 Historically, the Global Fund impact, while donor agencies are ensured that their investments has given performance ratings to its grants throughout the grant contribute to real health improvements. PBF is thus a contract period, in which ratings are constructed from “country-owned between two parties, with explicit expectations of progress that objectives and targets,”3 countries choose their performance indica- ensure accountability for results in the short term. tors and targets, and principal recipients report their own progress, Since its creation in 2003 the Global Fund has been a leader of subject to verification by local fund agents. innovation in PBF. Yet the details of its contracts and grant agreements Nonetheless, many donor agencies—the Global Fund included— dilute performance incentives, squandering an opportunity to improve have yet to realize PBF’s full potential.4 Contracts with PBF work results and health outcomes. This chapter describes the limitations of best when payment is linked clearly and directly with performance the Global Fund’s approach and suggests adjustments to unite funders, and when performance is measured in a simple, objective way. The implementers, and national governments around shared health goals. Global Fund’s approach is quite different. It correctly recognizes that performance should be one of several important factors in determin- Overview ing funding allocations. Other considerations could include country

Over the past decade several aid agencies have deployed innovations i. As reported by the WHO/UNICEF Joint Reporting System. See www. in PBF as part of contracts with countries. These agencies all practice gavialliance.org/results/evaluations/iss/. 26 Designing contracts Designing

capacity, predictability, ethical commitments, and continuity of ser- Box 4.1 Statement by the Technical vices. Yet because there is no clear link between performance and at Evaluation Reference Group on performance- least a portion of overall funding, the Global Fund does not effec- based financing tively transmit performance incentives to its implementing partners. “[P]erformance-based financing, a key tenet within the Guid- Perhaps more importantly, a lack of clear and consistent criteria for ing Principles, has evolved into a complex and burdensome PBF-related disbursements can lead to unpredictable and subjective system that has thus far focused more on project inputs and funding decisions. The complexity of the Global Fund’s PBF system outputs than on development outcomes, departing from has been described in previous work.5 The Global Fund relies on local the vision of an outcome-based model. Most importantly, fund agents, who adhere to a multistep rating process at each disburse- there remain inadequate information system and monitor- ment request. Each grant includes indicators—primarily inputs and ing and evaluation capacities in countries critically limiting outputsii—that the principal recipient reports as a percentage achieved the feasibility of the performance-based funding approach of a predefined target. Local fund agents check principal recipient espoused by the Global Fund. . . . [M]any countries found results, aggregate them into numeric progress scores, and convert the the system burdensome, rigid, and fixed exclusively on aggregate scores into an alphabetic performance rating. This rating short-term outputs rather than on longer term outcomes, can be changed for a number of reasons, including poor financial results, and capacity building.” management or data quality issues.6 The final grant rating informs Source: Technical Evaluation Reference Group (2009), p. 30. an “indicative disbursement range” for the next period, which can be adjusted further at the discretion of Secretariat staff. Several studies have identified challenges arising from this com- the applicable “indicative funding ranges” that correspond to the plex, multistep design. A year after the first Global Fund grants were ratings assigned to each grant (figure 4.1). Finally, ratings for HIV rated in 2006, a Center for Global Development report noted several and malaria grants were not significantly associated with changes problems with the process, including input-oriented targets that in disease prevalence or incidence, demonstrating that grant ratings did not measure impact and weak recipient country data systems.7 often fail to predict the overall impact of Global Fund resources for More issues emerged with the Technical Evaluation Review Group’s achievement of disease-control objectives. 2009 report, with particular attention paid to input-based indicators Perhaps most important, the perceptions of principal recipients that provided a poor metric of performance (box 4.1).8 The report appear to confirm the results of statistical analyses. According to recommended a comprehensive examination of the system’s goals a 2013 Aidspan survey, only 34 percent of principal recipients feel and procedures, a consensus on core indicators, and strengthen- that “the grant rating system accurately reflects performance.”10 ing of data quality. Two years later the High Level Independent If principal recipients do not feel that performance is accurately Review Panel’s final report pointed out the need to “hold [principal measured or tied to future disbursements, PBF incentives will not recipients] accountable against measurable results previously agreed improve health outcomes. through clearly defined long-term roadmaps for each disease, and provide incentives for good performance.”9 Opportunities and limitations Fan and others (2013) reveal further challenges in the system. To understand and replicate the Global Fund’s PBF process, they find In normal competitive markets efficiency is ensured by the inter- little statistical relationship between Phase 1 performance ratings play of supply and demand. Providers must fight for their market and Phase 2 disbursement levels—though higher grant scores did share: either cut costs while maintaining high quality or see cus- increase the likelihood of a successful grant renewal. Further analy- tomers flee to a different supplier. By contrast, the Global Fund’s sis for a sample of grant scorecards also showed large discrepancies core “suppliers”—the country coordinating mechanisms and their between actual Phase 2 funding and what would be expected given principal recipients—bear little risk of losing their privileged positions (with rare exceptions; box 4.2). Within the constraints of ii. See chapter 6 for a more detailed discussion of indicators. this single-buyer, single-seller relationship, performance incentives 27 Designing contracts

Figure 4.1 Grant performance does not Box 4.2 Comment on country coordinating predict disbursement levels mechanism incentive structure “Current incentives don’t encourage country coordinating Share of Technical Review Panel– Actual disbursement: Phase 2 incremental amount adjusted proposal amount (%) Expected range, lower and upper bounds mechanisms to actively look for the most cost-effective re- 125 cipients [or] providers, and in some cases country coordinating mechanism governance and membership structures 100 can act as a barrier to entry for providers who could deliver services at lower costs. Further work to develop models 75 of efficiency will have limited impact until these incentives

50 are aligned.” —Prashant Yadav, Working Group member

25 and outcomes. And it has applied the term “PBF” to many core 0 Global Fund functions, including grant monitoring and disburse-

IndiaIndia India Russia Malawi NigeriaNigeriaNigeria EthiopiaEthiopia RwandaTanzania PakistanPakistan Ethiopia Uganda ment; management of the central balance sheet; stabilization of cash IndonesiaIndonesiaIndonesia Bangladesh flow; assessment of country capacity for implementation; identifica- HIV/AIDS Tuberculosis Congo,Congo, Dem.Congo, Dem. Rep.Malaria Dem. Rep. Rep. tion of “potential risk of fraud during assessments;” oversight, fidu-

Source: Fan and others (2013). ciary controls, and financial management in a risk-mitigating environment; and support for fraud identification through a “bottom-up are one way to restore the most important characteristics of free audit trail.”11 Indeed, the word “risk” appears 376 times in the Global markets in an attempt to ensure similar efficiencies. This chapter Fund’s manual for local fund agents, with “audit” not far behind, suggests feasible improvements to the PBF mechanism while assum- at 279. While the accounting, financial management, and fiduciary ing that the basic country coordinating mechanism and principal control work done by the local fund agents is important—and under- recipient structures will remain. Nonetheless, it is important to standable given recent media attention to charges of fraud—the recognize the limitations of the country coordinating mechanism conflation of performance with risk avoidance challenges the Global model and perhaps, in the long term, to consider other options to Fund’s ability to ensure that its programs are effective. promote competition. Although available documents on the new funding model do The Working Group recommends that the Global Fund redesign not mention PBF, the Global Fund has already moved toward PBF in accord with three design principles: greater emphasis on downstream indicators in its use of PBF. Spe- • Reduce the number of key performance indicators by excluding cifically, Phase 2 grant renewals now include “impact assessments,” input and output indicators, such as number of bed nets distrib- through which an “impact rating” is assigned—a process that reflects uted, while refocusing measurement on key outcomes and coverage. country-level trends in disease prevalence.iii While the Working • Set aside a tranche of funding for which payments are directly connected to performance, without deference to discretionary iii. The Global Fund (2013a). AIDS treatment prevents AIDS mortality or contextual factors. and thus increases HIV prevalence, so disease prevalence is a flawed indica- • Use independent third-party measurement to verify self-reported tor of HIV prevention unless it is restricted to the youngest age groups, say results (see chapter 6). ages 15–20, where it is a useful proxy for HIV incidence among women. In embarking on these reforms, the Global Fund faces an uphill Measuring HIV incidence among older groups will be greatly facilitated by battle. The perspective of risk management, driven by an accounting the new “limiting-antigen avidity assay,” which can reliably estimate HIV and audit perspective, is dominant at the Global Fund and has been incidence in older age cohorts (Incidence Assay Critical Path Working emphasized in recent years. Further, the Global Fund has made per- Group 2011; Duong and others 2012). See Hallett and Over (2010) for a formance a catch-all term, encompassing processes, inputs, outputs, discussion of how such an assay could be used to incentivize HIV prevention. 28 Designing contracts Designing

Group applauds this shift, the Global Fund can go further, as this base level of funding provided to ensure continuity of care, which adaptation of the existing performance-based financing system does would be administered through a traditional grant management not represent a significant redesign, nor does it address any of the approach. Over time the proportion of funding linked directly to three design principles. performance could increase; high-achieving countries could also A final consideration: while this chapter emphasizes PBF compo- choose to have a higher portion of overall funding linked to per- nents within a given grant agreement between the Global Fund and formance, perhaps in exchange for an increase in the overall grant a country, there are other important considerations when designing ceiling. For higher income countries the tranche could be used to contractual agreements (including the importance of ensuring well- either reward performance or penalize failure (through a reduction tailored ex ante allocation and planning, as described in chapter 3). in the total grant amount). More evaluation and piloting is needed While PBF often focuses on short-term gains and performance, there to identify the optimal approach. will be important long-term financial considerations once the Global Fortunately, the basic structure necessary for this approach is Fund commences a contract or agreement. These financial consider- already outlined in the new funding model, where the Global Fund ations include ethical commitments related to the maintenance of has set aside indicative funding and incentive funding for each already-enrolled patients on antiretroviral medication and continuity country. Indicative funding is determined by the allocation for- of services policy. The long-term horizon of these agreements and mula and represents the “fair share” of what a country should be exit strategy are thus important aspects that should be considered allocated based on country disease burden and income. Incentive carefully if not made explicit in a contract or grant agreement. funding represents additional funds for “ambitious” proposals. The Global Fund could deploy the incentive funding tranche to Recommendations reward ambitious and successful programs that aggressively pursue core objectives for disease control and health improvement as one To create stronger incentives for coverage, quality, and impact, the strategy to implement this recommendation. Global Fund should redesign its PBF procedures to ensure that at least a portion of funding is consistently and transparently disbursed Link performance payments to incremental against strong performance in health outcomes and coverage. Under progress on the most important indicators the leadership of the new executive director, active discussions are under way on the potential use of social impact bonds, such as for The Global Fund should drastically reduce the number of key per- malaria, as suggested by the Roll Back Malaria Partnership.12 Such a formance indicators by keeping only those that are closely related bond represents an application of PBF well aligned with the recom- to health care coverage and outcomes (for example, coverage of and mendations that follow in this chapter. At a minimum the Working retention on antiretroviral therapy) while eliminating the consid- Group recommends that the Global Fund should: eration of most input and output indicators in making payments (such as condoms distributed; box 4.3). Thus, the Global Fund will Directly connect a portion of funding to no longer need to amalgamate indicators into a single grant rating incremental progress on performance on which basis payments are made. Instead, the Global Fund should work with countries during grant negotiation to identify one or The Working Group recognizes that tying all program support more key performance indicators to be linked to performance-based directly to performance is neither feasible nor desirable. Nonetheless, disbursements. The Global Fund, local fund agents, and princi- the ability to transmit performance incentives to recipients, and thus pal recipients can continue to monitor financial management and to create opportunities for accountability, is contingent on money implementation progress through input and output indicators, but following and rewarding improvements in coverage and outcomes. such indicators should not be used as the basis for PBF. The Global For each grant the Global Fund should thus set aside a dedicated Fund should use independent third-party verification and rigorous, tranche of funding that would be linked directly to verified per- representative measurement approaches to complement self-reported formance. This tranche could be provided on top of a guaranteed progress (see chapter 6). 29 Designing contracts

Box 4.3 Suggested core indicators for In most cases the Global Fund should link PBF payments with performance-based financing incremental progress in achieving high-quality service coverage or Good indicators directly contribute to or quantify a change health outcomes—for example, a fixed amount for each additional in health status. Below is a selection of useful indicators person initiated and retained on antiretroviral therapy (box 4.4). recommended by the Working Group: The Global Fund’s complex architecture stands in stark contrast to • Change in disease prevalence and incidence is the ulti- the GAVI Alliance’s streamlined (and now eliminated) immuniza- mate outcome of interest and should be rigorously mea- tion services support, which paid $20 per additional child covered. sured through household surveys. The latest Joint United Immunization services support is being phased out in favor of a Nations Programme on HIV/AIDS data date to 2010; graduated approach based on pre-existing coverage levels.14 By pay- more frequent monitoring and evaluation is needed. ing based on marginal progress, the Global Fund could also help • Antiretroviral therapy retention rate, a main determinant mitigate countries’ perverse incentives to set easily achievable targets of the effectiveness of treatment, should be measured rather than ambitious goals. instead of the simple number of people on treatment. As In practice the Global Fund would need to clarify and pilot more average antiretroviral retention is only 80 percent in the specific design features before settling on an approach. For example, first 6 months and 75 percent in the next 18, it is crucial the payment scheme within the incentive funding stream could take to carefully monitor this indicator. several forms, such as a fixed price per unit (say, $400 per additional • Tuberculosis case detection and treatment completion antiretroviral therapy person-year above a threshold) or a varying rates can be measured easily and cheaply with a sputum price depending on the degree of success (say, a pay scale based on the test. number of additional antiretroviral therapy person-years). Box 4.5 • Facility stock-out, a crucial indicator in many tuberculosis and appendix 2 suggest pricing alternatives that are modeled on grants, could be expanded for artemisinin-based com- the contracts that have long been used by government regulators in bination therapy availability to treat malaria. The prob- Europe and North America to improve the value for money achieved lem is particularly acute in tuberculosis, with 45 percent by regulated private or parastatal providers of critical public services of central facilities in high-burden countries reporting like electricity or water. Moreover, the price offered is expected to stock-outs. While no studies aggregate the impact of vary by country given the variation in costs of service delivery (see stock-outs for antiretroviral drugs, many studies point to chapter 5) and countries’ ability to pay. Where the principal recipi- an effect of stock-outs on retention and death in some ent is not part of the national government, it might make sense to high-burden contexts. Further, earlier work has shown a split performance payments between the principal recipient and direct connection between artemisinin-based combina- either the country coordinating mechanism (which nominates and tion therapy stock-outs and child mortality from malaria oversees the principal recipient) or a government ministry (which in Kenya. can provide a key source of support and facilitation, and to some extent oversight, for principal recipient activities). Such distribution Source: Stop TB Partnership (2010); Hamel and others (2011). could incentivize stronger performance and accountability across a broader range of actors, from which collaboration is needed to In some settings the core indicator could measure lasting achieve- achieve maximum program impact. ments in disease control, prevention, or even elimination. This will be especially useful where malaria has been eliminated but a constant Support performance incentives between the budgetary and programmatic effort must be maintained, or in geo- principal recipient and service providers graphic “hot spots,” where at-risk populations are concentrated but where a substantial up-front investment of time and money must In recognition of the enormous potential for performance incen- be invested to identify and approach high-risk groups to enable tives to improve the quality and responsiveness of national health necessary service provision.13 systems, several donors are supporting within-system results-based 30 Designing contracts Designing

Box 4.4 Frequently asked questions about performance-based financing What if the country does not meet the performance targets? of their intergovernmental transfer schemes. In addition to The best designed programs do not set targets at all. countries, many donors mentioned in this chapter, such as Rather, payments are set proportional to the degree of the World Bank’s Health Results Innovation Trust Fund and success. For example, an amount is given for each ad- the Inter-American Development Bank’s Salud Mesoamerica ditional course of tuberculosis treatment that is success- 2015, are financing projects that condition funding on health fully completed. So, countries cannot “fail”—they can only outcomes. Most of these projects are being tracked and show more or less success. This reduces the anxiety over evaluated. Impact evaluations are finding a positive effect meeting a particular threshold and facilitates financial plan- on health coverage rates. ning by reducing the risk of losing a big disbursement. Another way to mitigate the variability in performance Who receives the incentive, and why should this work dif- payments is to use this mechanism for only part of total ferently compared with traditional funding mechanisms? funding, as a performance “bonus” on top of guaranteed The incentive could be received either by the principal re- base disbursements. cipient, who is the chief implementer of Global Fund grants, or by country coordinating mechanisms, which choose prin- What if countries over-report their achievements, and how cipal recipients. Both parties should be accountable to the expensive is measurement? Global Fund, as money would be directly conditioned to To mitigate over-reporting, grants should incorporate regu- specific health outcomes. lar independent verification of key performance measures. Experience elsewhere suggests that independent verifica- If there are multiple funders, is it necessary to reward only tion is not prohibitively expensive, and has considerable “attributable” performance improvements? spillover benefits for improving routine data collection and Programs supported by global health funders usually re- service quality (see chapter 6). ceive financial resources from multiple sources. While it is not always possible to measure attributable program im- Has funding ever been conditioned to performance before? provements, doing so may be neither necessary nor desir- Yes, both by donors and by governments. Liberia and Rwan- able. Rather, performance-based funding can be viewed da have both structured their postconflict health systems as an incentive for the program as a whole to reach its to include performance-based financing. They are joined goal, thus aligning multiple sources of funding around a by many other countries that use conditional grants as part common objective.

financing initiatives. The World Bank’s HRITF has been a pioneer of some services has been observed, and the project has helped in “support[ing] the design, implementation, monitoring, and incentivize more responsive and proactive behavior among health evaluation of results-based financing mechanisms” at the country workers, including reduced absenteeism.16 An impact evaluation level.15 Early collaboration between HRITF and the Global Fund will report findings in 2014. appears promising. In 2012, for example, a Global Fund princi- Given this apparent success (and especially if these preliminary pal recipient and the GAVI Alliance partnered with the Trust findings are confirmed by the upcoming impact evaluation), the Fund in Benin to reward health facility performance based on Working Group recommends continued multidonor collaboration 18 quantitative indicators and a quality dimension. Overcoming with HRITF initiatives, with emphasis on the Global Fund’s five apparent fiduciary obstacles to joint implementation, all three “high-impact” countries, which also receive Trust Fund support partners were able to pool their funds in a single basket. Already, (the Democratic Republic of the Congo, Nigeria, Tanzania, Zam- the experiment has produced promising results: increased use bia, and Zimbabwe; box 4.6). Where appropriate, the Global Fund 31 Designing contracts

Box 4.5 Innovations in grant design can improve recipient efficiency and enhance the donor’s cost-effectiveness while economizing on information Two designs for an efficiency-enhancing, “contract-like” this benchmark unit cost, and then declines to less grant agreement differ on how much information they re- than it. quire about the recipient’s cost of operation. One design, Neither proposed contract mechanism can achieve the Vogelsang-Finsinger mechanism (1979), could encour- optimal efficiency within a year of implementation. Over age improvements in efficiency if the recipient submits its a period of years, through successive adjustments, both previous year’s total cost to the donor every year. The other mechanisms can improve value for money for both donor design, the two-part tariff, could work even without such and recipient. Both can motivate the recipient to achieve information. efficiency gains. And both can reduce the average cost to Suppose that, for at least some of the activities funded the donor per unit of service output. in an agreement, a quality-adjusted unit of service output The Vogelsang-Finsinger mechanism requires more in- is agreed upon during the initial negotiation between the formation on the recipient’s previous year’s cost of pro- donor and recipient. That output would then subsequently duction but offers substantial efficiency improvements be counted and independently verified during each year because it reveals how much the recipient’s average cost of program implementation. The two mechanisms can be of service production declines over time and leverages that briefly described as: information to reduce the donor’s average cost over suc- • Vogelsang-Finsinger mechanism. For each unit of output cessive years. Although the two-part tariff requires only during the current year, up to a an output ceiling larger an educated guess at a benchmark unit cost, it too can than the provider produced the previous year, pay the achieve considerable improvements in efficiency over time recipient an amount equal to its average cost the previ- as it motivates recipients to explore ways to expand service ous year. delivery at lower incremental cost and passes part of this • Two-part tariff. Pay the “benchmark unit cost” for ev- cost savings on to the donor. ery unit of output up to a threshold number of units. Appendix 2 provides details and examples of the two After reaching that output threshold, for each addi- contracting mechanisms. The mechanisms are intended tional unit of output during the current year, up to an only as illustrative examples, to outline the improvements output ceiling larger than the provider produced the in value for money that could be gained by exploiting the previous year, pay an amount that starts higher than literature on optimal regulation of public sector utilities.

Box 4.6 Health Results Innovation Trust Fund–participating countries and Global Fund high- impact countries

Health Results Innovation Trust Opportunities Global Fund Fund–participating countries for collaboration high-impact countries Afghanistan, Benin, Burkina Faso, Democratic Republic of the Congo, Bangladesh, China, Côte d’Ivoire, Burundi, Central African Republic, Nigeria, Tanzania, Zambia, Zimbabwe Ethiopia, Ghana, India, Indonesia, India, Lao People’s Democratic Kenya, Mozambique, Myanmar, Pakistan, Republic, Rwanda, Armenia1, Ethiopia1, Philippines, South Africa, Sudan, Haiti1, Kyrgyz Republic1, Lesotho1, Tanzania, Uganda Liberia1,Tajikistan1, Vietnam1

1. Pilot under preparation. Source: www.rbfhealth.org/rbfhealth/about; The Global Fund (2012b). 32 Designing contracts Designing

should encourage these countries to prepare grant applications that 3. www.theglobalfund.org/en/performancebasedfunding/. incorporate HRITF collaboration and to support joint fiduciary or 4. Birdsall and Savedoff (2010); Eichler, Levine, and the other implementation arrangements between the principal recipient Performance-Based Incentives Working Group (2009). and partner organizations. 5. The Global Fund (2008, 2011l). 6. The Global Fund (2011g), pp. 157–59. Summary 7. Center for Global Development (2006). 8. Technical Evaluation Reference Group (2009). The opportunity for the Global Fund to redesign its PBF system 9. The Global Fund (2011l), p. 78. is ripe. By simplifying measures of performance, focusing perfor- 10. Wafula, Marwa, and McCoy (2013). mance on health coverage and outcomes, and connecting a portion 11. The Global Fund (2011a), pp. 75, 112, 115, and 119. of disbursements directly to additional coverage achieved, the Global 12. Porcher and Kerouedan (2011); WHO (2012c); Roll Back Fund can regain its position as a PBF leader and innovator. Malaria (2011); Task Force on Innovative Financing Resource Mobilization Sub-Committee (2011). Notes 13. Over (2010); Zanzibar Malaria Control Programme (2009). 14. Hansen, Eriksson, and Stormont (2011) 1. www.rbfhealth.org/rbfhealth/about. 15. www.rbfhealth.org/rbfhealth/about. 2. The Global Fund (2011k). 16. Lemière (2012). 33

Chapter 5 Tracking and using cost and spending data

Recommendations • Continue to improve the scope, completeness, and timeliness of reporting to commodity price tracking systems • Benchmark and use supply chain costs and outputs • Identify core services for more extensive analysis and use of service delivery costs and spending • Share costing data with partners and the public • Develop a strategy to use unit-cost data throughout the new funding model grant cycle

Chapters 3 and 4 have discussed how decisions made during alloca- Overview tion planning and contract negotiation can leverage greater health impact throughout the funding cycle. Chapter 5 now turns to Commodities. The Global Fund records and tracks the prices paid opportunities for efficiency gains during grant implementation— for commodities through the Price and Quality Reporting (PQR) particularly the collection, analysis, and use of cost and spending system (box 5.1). The Global Fund has complemented the PQR data to drive improvements in procurement, supply chains, and with a voluntary pooled procurement mechanism, to reduce the service delivery. cost of inputs through bulk purchasing and streamlined procure- All health funders require information on costs, spending, ment.2 The Global Fund offers a voluntary pooled procurement use, and quality of care to manage programs, identify waste, and mechanism for most core health commodities, including ARVs, improve value for money. But data on their own do not generate rapid diagnostic tests, and artemisinin-based combination thera- such improvements. It is using the data to improve performance pies, which has lowered purchasing costs and raised procurement that is important. High-priority uses of data for policy at the efficiency among participating grants.3 Global Fund might include identifying high-cost outliers for fur- Despite substantial progress in improving price transparency, ther analysis, adjusting incentives embedded in grant agreements, standardizing procurement costs, and reducing the average price providing feedback to program managers to adjust cost structures paid for core health commodities, the limited available data sug- and implementation strategies, and informing the country dia- gest that persistent variability remains in the cost of some inputs— logue and requests for technical partner support. The U.S. Presi- though strong convergence is apparent over time for others. Con- dent’s Emergency Plan for AIDS Relief (PEPFAR), for example, sider the variation in the price paid for Ritonavir 100 milligrams (a has used spending analysis “to better understand cost structures second-line ARV) among transactions reported to the PQR database within their programs and to identify program outliers, to provide (figure 5.1). South Africa paid $66.83 per person-year for Ritonavir decision-makers with data on which interventions provide the in November 2010. Less than a month later the West Bank and greatest value for money in terms of impact on the epidemic, and Gaza paid $1,216.62—almost 18 times more than the price South to inform country-level harmonization of expenditure tracking Africa paid. And prices for this drug have not converged over time. for governments.”1 In some cases merely disclosing average unit PQR reporting suggests a fairly constant 25th–75th percentile range costs has resulted in cost savings, though the mechanisms of such between 2009 and 2011 (about $80–370). changes are still being understood. For other commodities, prices reductions have been quick and Thus, a key recommendation of the Working Group to improve widely shared, suggesting that further transparency, standardiza- efficiency is to measure and use cost, price, and spending data on tion, and consolidation of commodity purchases represents an commodities, supply chains, and service delivery. opportunity for substantial and rapid gains. The trend in the price 34

Box 5.1 The Price and Quality Reporting Figure 5.1 Variation in reported cost per system patient-year for Ritonavir 100 milligrams, The Price and Quality Reporting (PQR) system is set up to com- September 2010–April 2013 municate market information to principal recipients, improve

Tracking and using cost and spending data West Bank and Gaza transparency, and enable the Global Fund and its partners to Iran better understand and influence the market for pharmaceutical Albania products. It is “a web-based system used by the Global Fund Moldova to collect transaction-level procurement information from [re- Guatemala Ukraine cipients] on key health products.” First called for by Vasan and Jamaica others in 2006, the system now contains almost $6 billion in Dominican Rep. transactions and covers bed nets, condoms, HIV/malaria tests, Thailand antituberculosis and antimalarial medicines, and antiretroviral Benin (ARV) therapies. The disclosure of this information is likely one Burundi Angola of several drivers of falling ARV prices in recent years. The PQR Ethiopia reports that the median price of common first-line ARVs was Cuba $127 in 2012, a steady decline from $144 in 2007. Similarly, Lao PDR the average unit cost for long-lasting insecticide-treated bed Bolivia nets purchased by Global Fund–supported programs fell from Georgia Armenia $5.10 in 2009 to $3.03 in 2012. Belarus Source: www.theglobalfund.org/en/procurement/pqr/; DFID (2012). Nicaragua Morocco Eritrea paid for Efavirenz 200 milligrams (a first-line ARV) reflects this Nepal Gabon pattern (figure 5.2); in only two years, the median purchase price Cambodia was roughly halved, while the space between the 25th and 75th Myanmar percentiles shrank to about a quarter of its original size. Egypt Nonetheless, outliers persist even where overall convergence has Guinea occurred. Kazakhstan paid $1,636.48 per patient-year for Efavi- Guinea-Bissau Mali renz (including freight costs) during a February 2012 transaction. Mauritania To determine the driving force behind such outliers, the Working South Sudan Group analyzed the PQR dataset and explored factors correlated Burkina Faso with unit costs of one ARV (box 5.2). South Africa Finally, Global Fund grants finance the purchase of commodi- 0 250 500 750 1,000 1,250 US$ ties beyond medications and bed nets, such as computers, vehicles, Note: This graph was replicated from information in the price reference and office supplies. The Enhanced Financial Reporting system, report, but could not be fully replicated using data from the public version introduced in 2008, is intended to include such commodities, and Price and Quality Reporting (PQR) database. In addition, the timeframe of 2010–13 may not reflect differences in lower prices achieved over time, local fund agents are required to report to it. But these costs are differences in price due to the volume of procurement, or country-specific grouped in general spending categories, and its various limitations patent law—all of which are expected to affect price. Nonetheless, this shows variation in prices paid, and while this may be an extreme example, it 4 have led to the system’s disuse. reflects variation common across many drugs identified in the PQR database. Source: Adapted from the Global Fund’s PQR Price Reference Report, http:// bi.theglobalfund.org/analytics/saw.dll?Dashboard&nqUser=PQRExternalUser Supply chains. Once commodities are purchased, the Global Fund &PQRLANGUAGE=en&PortalPath=/shared/PQR%20External%20Users/ and its partners must ensure that the right medicines and products _portal/PQR%20Public&Page=Price%20list. 35 T racking and using cost and spending data

Figure 5.2 Trend in 25th, 50th, and 75th Box 5.2 What determines the unit cost of a percentile per-patient unit prices for first-line antiretroviral drug? Efavirenz 200 milligrams Many factors can contribute to how much a first-line antiretroviral (ARV) drug costs. Using the Global Fund’s publicly US$ 300 available Price and Quality Reporting (PQR) dataset, the Working Group chose to analyze one first-line fixed-dose combination ARV drug—Lamivudine, Nevirapine, Zidovu- dine (LNZ), the most frequently purchased drug in this 200 dataset, for a 150, 200, and 300 milligram tablet dosages. LNZ purchases totaled $380 million for 839 total transactions for 57 countries over 2007–12. Most countries purchased this drug from a manufacturer, at an average cost 100 of $11 a pack. Those who purchased it from the Global Fund’s voluntary pooled procurement mechanism paid $9 a pack. The Working Group analyzed the relationship of pack cost and other factors including the number of people 0 2009 2010 2011 on ARV therapy, lags between purchase dates and delivery dates, and the share of the market controlled by the largest Source: The Global Fund’s PQR Price Reference Report, http://bi.theglobalfund .org/analytics/saw.dll?Dashboard&nqUser=PQRExternalUser&PQRLANGUAGE manufacturer (in a given country and year). =en&PortalPath=/shared/PQR%20External%20Users/_portal/PQR%20Public& An important mediating factor is whether freight cost was Page=Price%20list. included, excluded, or unknown in the pack cost. In the PQR dataset freight costs fit three categories: in 42 percent of reach the target population at the right places, in the right amounts, transactions freight cost is included in pack cost, in 25 per- and at the right price.5 Procurement processes do not end after cent it is excluded, and in 33 percent it is unknown whether it purchasing, and the logistical challenges of placing these commodi- is included or excluded. Results suggest that the more people ties in the hands of front-line providers remains challenging. For who are on ARV therapy or the more people with HIV/AIDS in example, despite bulk purchasing contracts in Kenya for malaria a country (a larger potential market), the lower the LNZ pack treatment, stock-outs and drug shortages remain due to production cost. The longer the lead time between scheduled delivery and distribution issues.6 The average availability of drugs at public date and purchase date, the lower the price paid. Results on health facilities in low- and middle-income countries is estimated the concentration of markets and pack cost are sensitive to at less than 25 percent.7 Stock-outs have important health conse- whether the freight cost is included, excluded, or unknown. quences, and stock-outs for essential antimalarial drugs have been For transactions in which pack cost includes freight cost, more associated with increased transmission and disruption of services.8 A concentrated markets are correlated with higher pack costs, review of 16 supply chains in seven PEPFAR partner countries also with no correlation of pack cost and concentrated markets found poor record keeping and insufficient controls for monitoring when freight cost is excluded or unknown. drug supply.9 In Zambia as much as 9 percent of all ARVs in one These results suggest that better cost data are impera- drug facility, totaling $265,000, could not be located, partly due tive to understanding the factors behind drug costs. The to inadequate inventory controls.10 Global Fund has been purchasing drugs and commodities The reasons behind these suboptimal results are many. During for 11 years, yet there is no rigorous published analysis of delivery from global suppliers to countries there can be long lead the factors affecting the prices that countries pay. While times and delays in getting shipment clearance as well as a lack of the PQR dataset is a good start, it should be improved to transparency of shipment data. During storage and distribution make in-depth analyses possible. 36

Box 5.3 The U.S. Government Accountability Box 5.4 Strengthening supply chains: a new Office on the Global Fund initiative at GAVI Alliance “PEPFAR has . . . provided technical assistance to the Global Powerful new vaccines have been introduced that protect Fund to improve its procurement system, with the goal of against the biggest killers of children, but there has been Tracking and using cost and spending data reducing the need for further emergency support from little investment or attention paid to the supply chain that PEPFAR. According to USAID officials, in September 2012 moves vaccines reliably and efficiently from manufacture PEPFAR helped the Global Fund develop a proposal for to immunization. The strategy will encompass many ap- its own emergency procurement mechanism. As of March proaches, but one strategy—the use of barcodes to capture 2013 the Global Fund had not notified PEPFAR whether it the data needed to track vaccines through the supply chain, had established this mechanism.” streamline inventories, and improve vaccine targeting in developing countries—is under serious consideration at the Source: GAO (2013), p. 9. GAVI Alliance, and may suggest similar directions for some Global Fund–purchased commodities. Barcode technology there can be inefficient management of inventories, poor equipment is a robust, scalable technology used in many industries. repair and maintenance, a lack of well-documented supply chain A supermarket can track a banana across the world, but processes or poorly implemented processes, and ad hoc delivery vaccines cannot currently be tracked. Barcodes on vaccine schedules leading to unreliable distribution. And during provision packaging can improve: there can be limited information on the frequency, size, and loca- • Stock management and logistics, including shipment tion of demand and use that then limits the efficiency of demand and receipt tracking. forecasting and procurement upstream. In general the limited incen- • Vaccine safety by improving access to insert information tives for efficiency in service delivery also apply to supply chains. or lot traceability. Analyses from the U.S. government demonstrate that supply chains • Counterfeit and fraud detection. are only as effective as the systems that mobilize them. A recent U.S. • Patient management, insofar as barcodes can link pa- Government Accountability Office report found that PEPFAR’s tient records with information about the vaccine that emergency procurement mechanism was used in five countries to was administered. procure emergency ARVs due to Global Fund disruptions, and six The GAVI Alliance and World Health Organization are countries almost experienced shortages following Global Fund together considering requiring barcodes on packaging, by delays (box 5.3).11 Effective grant management and forecasting the end of 2014, and barcodes on vaccine vials at a later efforts are needed to minimize stock-outs, and better mechanisms date. A pathway program in Tanzania is under way, and once should be established for emergency procurement (box 5.4). the standard is established, other countries will be able to Supply chain costs, like commodity and service delivery costs, invest in systems to better capture data and strengthen can also be highly variable. In a review of logistics costs for several vaccine supply chains. global health agencies, supply chain costs as a share of total stock value were found to range from 4.8 percent for ARVs in Nigeria to 44 percent for bed nets in Liberia.12 Performance evaluation and Service delivery. Beyond commodity purchases and supply chain benchmarking analysis of the efficiency of supply chains for con- expenses, substantial cost variation is also observed for other traceptive orders and shipments found that only about a quarter of elements of service delivery, where total expenses may be highly 37 Sub-Saharan countries are efficient.i The rest experience a large malleable and dependent on the environment and contractual excess of spending given actual supply chain output.13 relationships.

i. Benchmarking is used here as “the ongoing activity of comparing one’s activity so that challenging but attainable goals can be set and a realistic own process, practice, product, or service against the best known similar course of action implemented efficiently” (Balm 1996, p. 28). 37 T racking and using cost and spending data

Data from a sample of 45 Zambian facilities show the relation- Figure 5.4 PEPFAR expenditure analysis pilot ship between the cost per person-year of treatment and a selection in Mozambique, mean and range of non– of cost determinants, including aspects of service quality, envi- antiretroviral unit spending per patient-year ronmental factors, and the scale of operation (figure 5.3).14 The Patient-years of antiretroviral Unit expenditure per patient-year (US$) reference facility (on the far left of the figure) represents a relatively therapy (thousands) Upper and lower bounds 100 1,000 costly type of service delivery—a public primary-care facility in an urban setting, which began offering ARV treatments only within the past 24 months and has fewer than 300 enrollees a year—but 75 750 that has poor adherence. Here, a person-year of treatment, including $497 of facility-level spending and $423 of “above facility” spending, would cost $920. Holding all other factors constant but 50 500 improving adherence, the cost would rise to $1,020 per patient- year (illustrated at the second stop). Further to the right along the graph, one facility “trait” is altered at a time, with corresponding 25 250 incremental changes to the indicative cost of treatment, both at the facility level and in aggregate, which includes above-facility costs of management, oversight, and procurement. 0 0 To better understand the sources of such variation in costs, several 2009 2011 global health funding agencies have embarked on exercises to better Source: PEPFAR (2012c). measure unit costs of service delivery, such as PEPFAR’s expenditure analysis, unit costing by the Clinton Health Access Initiative, and measurement of the costs of integrating sexual and reproductive health services with HIV/AIDS.15 Findings from such exercises show that the savings from reducing variation in service delivery costs could Figure 5.3 Heterogeneity in the unit cost of be substantial. For example, a PEPFAR expenditure analysis shared antiretroviral treatment across 45 Zambian findings on variation in unit expenditures on facility-based care and facilities, 2009 treatment with the PEPFAR country team in Mozambique, and ii Average cost per patient-year Average total cost documented reduced costs, up to 45 percent in some cases (figure 5.4). (point estimate in 2010 US$) Average on-site cost 1,250 Upper and lower bounds Opportunities and limitations 1,000 Processes to better measure, analyze, and incorporate cost and 750 spending analysis into budgeting and management processes could 500 improve value for money, particularly among commodity purchases or supply chain processes that are readily comparable across national 250 and subnational contexts. But costing of service delivery can be 0 challenging and resource-intensive, and a naïve approach to costing

Rural Private and benchmarking could lead to perverse incentives that ultimately Hospital Reference Early start detract from public health objectives.

Adherence over 91% More than 24 months

More than 300More patient-years than 800 patient-years ii. While figure 5.4 shows a reduction in PEPFAR spending, note that it does not claim causality in reducing unit spending, as there are many fac- Source: Marseille and others (2012). tors that may improve efficiency fluctuations from year to year. 38

Benchmarking service delivery costs is challenging, and highly to inefficiencies in service delivery. Reasons why service delivery sensitive to methodological choices. Such costing exercises—to would entail higher unit costs include a rural or urban location; a be comparable—require standard spending categories, as well as new facility (with attendant capital costs) or an established clinic; clear definitions of activities and related costs. Figure 5.3 presented or a sicker group of patients at treatment initiation versus a healthier Tracking and using cost and spending data results from one costing exercise in a sample of Zambian facilities. pool of patients. Thus, pressuring all such facilities to abide by a Figure 5.5, likewise, is drawn from a random sample of facilities standard cost structure may be extremely inefficient in some cases. in five countries, including Zambia.16 But within this sample, the Facilities could be incentivized to turn away sicker patients who average facility-level cost in Zambia is about a third smaller than it require additional care, and would face disincentives in outreach appeared in the figure 5.3 data. There could be several reasons for to hard-to-reach, but potentially high-risk populations. the observed discrepancy, including changes in input costs over time, For these reasons one-size-fits-all benchmarking—assuming measurement error, or that figure 5.3 was based on a convenience that all facilities should have the same cost structure as measured sample that may not be nationally representative. Nonetheless, these across multiple costing exercises—is inadequate and can cause harm. two figures prove that even expensive and time-consuming efforts Still, a more nuanced approach can help all stakeholders understand to collect cost data are imprecise, even when merely attempting to their respective costs structures and cost drivers, and thus squeeze assess the unit cost of a single service in a single country. substantial efficiencies throughout implementation, including for The actual unit costs of service delivery are highly variable, both procurement, supply chains, and service delivery. The recommen- across and within a single country. Some facilities are indeed asso- dations here thus provide broad guidance on the collection and ciated with much higher costs, yet it is not always readily apparent uses of cost and spending data, while leaving space for the Global whether the higher costs stem from immutable characteristics of the Fund partnership to craft and iterate an appropriate management facility or catchment population, or whether they can be attributed response to the findings of such exercises.

Figure 5.5 Variation in cost per patient-year of HIV treatment in five African countries, 2012

Number of facilities 40 Ethiopia Malawi South Africa

40 Rwanda Zambia Total

0 100 300 1,000 100 300 1,000 100 300 1,000 Average total cost of antiretroviral therapy per patient-year (US$)

Source: Over, Schneider, and Velayudhan (2013). 39 T racking and using cost and spending data

Recommendations and create incentives for supply chain entities to improve performance. Optimization analysis—for inventory and distribution—is Continue to improve the scope, completeness, well developed as an approach but still underused in Global Fund and timeliness of reporting to commodity price countries. Findings from these efforts can also feed back into the tracking systems performance-based funding approach, to align incentives throughout the system (box 5.5).19 Tools that track the prices of commodities, such as the Global Fund’s PQR, have provided the information and leverage to help drive Identify core services for more extensive analysis down the costs of commonly funded health commodities. Even so, and use of service delivery costs and spending this dataset remains incomplete, and the PQR database only offers partial coverage, despite previous goals to achieve full coverage of A large portion of Global Fund resources is spent on delivering a grants.17 The Global Fund might consider expanding coverage of few core services. The Global Fund should identify the most fre- these commodities through incentives, including those leveraged quent and costly health interventions financed with its resources during the grant review processes. In addition, the Global Fund that would be good candidates for collecting, analyzing, and using should consider expanding the PQR or improving the quality of cost data. The Global Fund has already reported estimated unit the Enhanced Financial Reporting system to cover other com- costs of service delivery for first- and second-line ARV therapy at monly purchased high-cost items, such as computers and vehicles. the national program level.20 Building off previous experience in Benchmarked unit costs for these items could be used to leverage costing ARV therapy per person-year or contracting out to a special- better pricing, as well as to identify comparable products purchased ized organization, the Global Fund can expand its costing exercises at higher cost. to other services, such as tuberculosis and malaria treatment. The In the medium term, procurement reporting should be extended Global Fund should deploy rigorous methods that relate a unit of beyond commodity purchases to encompass the supply chains and the product’s end use. Incorporating the costs throughout the sup- Box 5.5 Perspective from Liberia ply chain into cost databases could ensure that countries get best In Liberia value for money discussions center on achieving value both for commodities and for the systems used to deliver those higher national program coverage, pursuing efficiency ob- commodities to patients and facilities. jectives, and fostering country ownership in Global Fund grant management responsibilities. Implementation of Benchmark and use supply chain costs Global Fund grants has focused on increasing efficiency— and outputs ascertaining unit cost per contact and costs per capita, introducing expenditure tracking systems to determine Comparing relative costs per output can be used to assess perfor- whether resources reach the beneficiaries, and, where mance and improve efficiency in the supply chain. For example, blockages exist, working to eliminate them. PEPFAR saved an estimated $38.9 million over four years by tweak- Further, efforts have been under way to build strong ing its supply chain design and management, including using the partnerships and improve coordination for maximum re- more efficient ground and sea routes over pricey air freight.18 Com- sults. For example, large grants handled in the Ministry of paring cost–performance ratios among peer organizations with Health and Social Welfare, especially the Pool Fund and in- similar challenges can provide key information on performance at cluding the Global Fund and Fixed Amount Reimbursement the different stages of the supply chain, from upstream demand and Agreement funds, use a newly developed spending tacking supply forecasting, through ordering, production, and delivery to tool to gauge resource flow directed to the county health countries, to in-country delivery. Data envelopment analysis tools, teams—where a bulk of health service provision takes place. used to determine an efficient frontier of performance, could be Source: David Logan, Global Fund Project Manager, Ministry of Health better applied to determine the highest possible level of efficiency and Social Welfare, Liberia. 40

service delivery to its quality, thus ensuring that lower costs do implementation, reduce duplication, and strengthen and standard- not reflect (or incentivize) poor quality care. Initially, unit costs ize costing methods. Expanding such costing exercises represents could be disclosed to recipients and subrecipients as collaborative additional costs but has potential benefits to further reducing costs feedback, as with PEPFAR’s Mozambique expenditure analysis. and increasing efficiencies that cannot be underestimated. As more Tracking and using cost and spending data Managers could then be supported to analyze the data and define global health organizations adopt open data standards, the Global and implement strategies to improve efficiency. Fund should also work to maintain its reputation for transparency While better spending tracking is an important goal, doing so and collaboration through increased availability of data to a broader can be a challenge when health services and finances are decen- audience, including the general public. tralized.iii Only the vertical program elements may be tracked Beyond sharing data the Global Fund and its partners should separately, and many diagnosis and treatment costs (depending leverage more open lines of communication to identify research on how drugs are bought) are embedded within general facility “gaps” where further investigation is required, and for which part- costs, which may be unknown. For example, Indonesia has 498 ners can agree to an efficient division of labor to evaluate differ- autonomous local government units that do not report health ent aspects of the shared agenda. Ideally, methods for evaluation program spending. While these subnational entities are asked to would follow similar inputs and processes, enhancing comparability submit reports on tuberculosis spending once a year, less than half across organizations. The Global Fund has already begun collabo- actually report, and most are inaccurate. In such a system, tracking ration studies with PEPFAR23—and such collaborations, if found expenditures accurately in the short-term is not feasible, except effective, should be continued and expanded. Understanding the with special studies. Even in countries that are not decentralized, nuances that inhibit organizations and programs from using the inaccuracies can be common. While the Global Fund should work same benchmarks, the Global Fund and its partners should strive to promote better expenditure tracking, simple and rough estima- for comparability on at least the most commonly measured items. tions can still be useful for making decisions. Unit costs can be estimated based on inputs in different regions and multiplied by Develop a strategy to use cost and spending data the numbers of treatments, generating a rough total cost that can throughout the new funding model grant cycle be used for policy dialogue. In the long run the Global Fund’s new funding model provides an Share costing data with partners and the public opportunity to better integrate and use supply chain and service delivery output data with cost and spending data. The strategy As a member of an information-sharing network, the Coordinated can clarify which data are required by whom to drive real-time Procurement Initiative, the Global Fund participates in efforts to improvement in program performance. Aside from continuing to support procurement practices of the Office of the U.S. Global benchmark commodity costs with data from the PQR, the Global AIDS Coordinator, the World Bank, the U.S. Agency for Interna- Fund should encourage proposals to include unit costs from previ- tional Development, United Nations entities, and nongovernmental ous grants to inform subsequent budgeting exercises (see chapter 3). organizations.21 In addition, as a signatory to the International The Global Fund should support the calculation of unit costs, Aid Transparency Initiative, the Global Fund has pledged to share which could inform future budgeting, help benchmark principal financial and programmatic data with external stakeholders, par- recipient or subrecipient performance, provide feedback to manage- ticularly recipients, local fund agents, and other external funders.22 ment on benchmarking results, inform implementation strategies, Building on this commitment, relevant information from costing and feed into evaluation. Unit costs from previous grants can be and commodity price tracking systems should be distributed among used to determine cost estimates for countries’ national strategic partner organizations to gather knowledge on the costs of program plans and to justify budgets in the concept note. In addition, a con- comitant strategy to strengthen the capacity of program managers iii. This paragraph is based on comments to the report provided by David (down to the facility and program levels) to use the data to drive Collins, Management Sciences for Health. performance can be undertaken. For example, recipients can be 41 T racking and using cost and spending data

encouraged to incorporate unit costs in their own domestic pro- nuanced approach to cost control, with flexible benchmarks as one gram management systems, using information to drive efficiency tool, can align incentives for technical efficiency among the Global gains and reprogramming if necessary. The Global Fund could also Fund, principal recipients, and subrecipients. There are many dif- reward, either financially or through visibility, principal recipients ferent options for deploying unit costs in the pursuit of value for and local fund agent partners that match their cost and spending money, many of which can be driven at the recipient or project data with outputs to generate efficiencies, bringing down input level. The Global Fund should ensure that its funds are structured costs (as suggested by current Global Fund guidance on value for to support creative efforts to improve efficiency, and to continue money) and reducing output unit costs while maintaining quality. supporting efforts to improve supply chains and generate procure- In moving forward the Global Fund could better use cost and ment efficiencies. spending data to explore alternate grant management styles over the coming years. It is understood that programs that pay on a fixed Notes cost per input are not desirable due to their limited incentives for 1. PEPFAR (2012c), p. 40. efficiency. But the problems with a simplistic input- or output-based 2. www.theglobalfund.org/en/procurement/vpp/. approach do not invalidate the use of benchmarking as a tool for 3. www.theglobalfund.org/en/procurement/vpp/. project management. As chapter 4 argued, this Working Group 4. The Global Fund (2007a). believes that the Global Fund can create high-level incentives for 5. Kraiselburd and Yadav (2012). efficiency gains by conditioning a portion of payment on verified 6. Tren, Hess, and Bate (2009). improvements in population health or service coverage. But because 7. Kraiselburd and Yadav (2012). population-level measurement is time-consuming (as is input track- 8. Hamel and others (2011). ing), it can only be conducted infrequently. In the interim and 9. GAO (2013). planning stages principal recipients and local fund agents can use 10. GAO (2013). indicative input and output benchmarks to help negotiate lower 11. GAO (2013). prices, identify cost and performance outliers, and otherwise shape 12. Yadav, Tata, and Babaley (2011); Sarley, Allain, and Akkihal a more responsive and timely approach to performance manage- (2009). ment. At the project level, recipient program managers can use 13. Berenguer, Iyer, and Yadav (2013). benchmarking to continuously improve their efficiency, and the 14. Marseille and others (2012). Secretariat should support them in this endeavor. 15. GAO (2013); Over, Schneider, and Velayudhan (2013); Lon- don School of Hygiene and Tropical Medicine (2012). Summary 16. Marseille and others (2012). 17. The Global Fund (2010b). The Global Fund, mainly by design, is limited in its ability to directly 18. PEPFAR (2012c). improve program efficiency. Even so, the Global Fund can use its 19. USAID (2013). high-level authorities to incentivize and facilitate improved program 20. The Global Fund (2010a). efficiency, such as by using unit costs to determine appropriate price 21. GAO (2013). ranges and induce cost savings. While a one-size-fits-all approach to 22. The Global Fund (2011e). benchmarking is likely to fail and perpetuate perverse incentives, a 23. Garmaise (2012).

Chapter 6 Verifying performance

Recommendations • Define a subset of core indicators to receive strengthened performance verification • Verify the accuracy and quality of principal recipients’ self-reported results using rigorous, representative measurement instruments • Complement verification with population-based measurement and formal impact evaluation for interventions and service delivery strategies of unknown efficacy

Chapter 5 highlighted the importance of efficiency and the incen- Overview tive environment for realizing cost savings and stretching limited resources—that is, for minimizing the costs of any program. This The Global Fund prioritizes data and information systems as central chapter now shows that such measures are, however, fundamentally components of its model in four key areas. The first area is program incomplete without a strong mechanism to verify the value created sustainability and efficiency—that is, strengthening national health by programs and thus ensure that value for money is achieved. information systems and other data collection to improve program Consider a program to treat malaria with artemisinin-based management by the principal recipient and build sustainable health combination therapies (ACTs). Through benchmarking, proactive systems. To this end the Global Fund recommends that recipients allocation, and efficient supply chain management, the Global Fund allocate 5–10 percent of their budgets to monitoring and evalua- could ensure that the “right” amount of ACTs are purchased at the tion (M&E) activities.2 lowest cost, and delivered to the country efficiently. Yet all would be The second area is resource allocation, within and across coun- for naught if the pills sat unused in a warehouse or a rural clinic, or tries, and for Global Fund grants and for the national strategy plans if they were improperly prescribed to children suffering from non- that provide a starting point for grant negotiations. The Global malarial febrile illnesses. Robust performance verification is thus Fund’s 2012–2016 strategy calls for “strategic investment” in “the essential to ensure that funded commodities and high-quality health highest impact interventions and technologies suitable to the coun- services reach their intended beneficiaries—and to hold recipients try situation,” and for “appropriate targeting of most-at-risk popu- of funding accountable for achieving health impact. lations.”3 This can only be done with robust data on the efficacy of The Global Fund has taken aggressive steps to verify fiscal per- interventions (including efficacy for particular subgroups) and on formance through strengthened fiduciary controls and financial the size and characteristics of a country’s epidemics, including high- oversight of principal recipients. These steps have helped win back risk groups and geographic “hot spots” of transmission. donor confidence, and may deter financial misconduct, two essen- The third is grant management by the Global Fund, encompass- tial elements for the Global Fund’s long-term stability and funding ing risk mitigation, regular oversight, performance incentives, and outlook. But this approach addresses only the first half of the Global iterative reprogramming as challenges or opportunities for greater Fund’s core mission—to “[invest] the world’s money to save lives”1— impact arise. Previous chapters discussed how performance data are without the necessary rigor in documenting the health returns of intended to determine later disbursement amounts, and suggested those global investments. The Working Group thus recommends strengthening the direct connection between grant performance that the Global Fund adopt a more robust and rigorous approach and funding decisions. But this is only one aspect of the Global to performance verification, measurement, and evaluation. Fund’s reliance on data for grant management purposes. Beyond 44 Verifying performance Verifying

performance-based funding (PBF) the Global Fund uses data and the Global Fund contracts to “independently oversee program per- measurement to detect and deter fraud, assess overall epidemiologi- formance” and “verify results.”8 For most periodic reports LFA cal trends, revise its funded activities, and coordinate funding with “verification” is conducted through a desk review of data sources, in other partners and national governments. All require real-time which aggregate results are compared with the underlying documen- measurement of financial flows, implementation progress, and other tation from facilities and program managers.9 LFAs also conduct aspects of grant performance. annual site visits for each disease area and principal recipient to The fourth is accountability—between principal recipients and verify data sources and to assess the quality of health services, both the Secretariat, between the Secretariat and the Board, and between as described in official policy (usually at the Ministry of Health) the Board and donor governments. Just as the Global Fund is respon- and as followed in practice (at health facilities).10 sible for preventing misuse of its funds, it also provides implicit In sum, these methods represent a good faith effort by the Global promises to its stakeholders about what it will achieve with those Fund to monitor grant performance and assess data quality in the resources—“[save] 10 million lives and [prevent] 140–180 million absence of an on-the-ground presence. Even so, the Working Group new infections from HIV/AIDS, tuberculosis, and malaria between recognizes several limitations of the Global Fund’s verification and 2012 and 2016.”4 To be accountable to its Board and donors (and to measurement policies in the absence of more robust methods. Given mobilize additional resources in future replenishments), the Global how essential reliable data are to the value for money agenda, these Fund must track progress on those goals and document the appro- weaknesses require urgent attention by the Global Fund’s leaders. priate use of its resources to achieve health impact. Three concerns stand out. First, there are several inherent rea- Given the system-wide reliance on data and measurement as an sons to question whether recipients’ self-reporting reflects genuine integral input to core Global Fund and country-level health objec- improvements in health, including general capacity constraints and tives—and given that much of the underlying data come from prin- data-quality concerns. Yet the credibility and rigor of self-reported cipal recipients themselves—it is not surprising that “data quality” data is of considerable concern “when information is used to reward is a recurring concern in Global Fund policies. The Global Fund has performance or quality.”11 In such cases administrative data may be responded to this challenge with procedures to assess and improve distorted by the recipients’ (and thus facilities’) clear incentive to the accuracy and reliability of the information on which it bases report the “right” results to meet output targets, particularly when many key decisions. For example, the Global Fund has adopted a results are implicitly or explicitly tied to future funding. “risk management approach” to implementing data quality audits For example, a Global Fund Office of Inspector General audit among its grants, to “provide an in-depth assessment of data qual- report for a malaria grant in Madagascar found that “net results ity and M&E systems” for grant recipients.5 The Global Fund has reported to the Global Fund included UNITAID [long-lasting also planned “country reviews” for recipients of its largest grants, insecticidal nets] (and yet the indicator results were tied to fund- designed to “evaluate disease outcome and impact, review pro- ing).”12 This challenge is not unique to the Global Fund. Lim and gram progress, and provide practical recommendations on where to others (2008) found that the GAVI Alliance’s results-based immu- achieve the greatest impact,” which are expected to inform program nization services support program (currently being phased out) design under the new funding model.6 caused countries, on average, to inflate their official immunization Beyond these initiatives, routine performance validation by local statistics—an effect neither prevented nor predicted by the GAVI fund agents (LFAs) has long been part of Global Fund oversight Alliance’s use of data quality audits.13,i Similarly, more health ser- practice. Principal recipients provide the Global Fund with periodic vices do not necessarily equal better health. For example, the “fee- reports on grant implementation, including progress on country- chosen indicators and targets. These indicators often emphasize i. According to personal communications with the GAVI Alliance staff, easily documented inputs and outputs (people trained, condoms the GAVI Alliance was aware of the likelihood of discrepancies between distributed)7 rather than downstream health effects (outcomes, administrative and survey data at the time the immunization services impacts). Once submitted these reports are forwarded to the Global support program was launched. However, the initial program design was Fund’s designated LFA, typically an audit or consulting firm, which borne from a conscious decision to endow countries with responsibility for 45 V erifying performance erifying

for-service” model common in the U.S. health care system incentiv- Figure 6.1 Insecticide-treated bed net used izes providers to perform unnecessary but costly procedures with as soccer net in Wassini Island, Kenya little to no health benefit (and possibly net harm).14 Such perverse incentives, when unchecked, undermine three of the four areas of measurement described above. They undermine attempts to establish performance incentives and reward high- performing grants—a core Global Fund principle and essential tool in achieving value for money. They undermine the accountability of principal recipients to use funds appropriately and the responsibility to improve the health of populations served. And they can degrade national health information systems, with adverse spillovers for the entire health sector. Because of these well-documented dynamics, self-reported data should be treated with caution and robustly verified to manage and mitigate perverse incentives. But despite their experience validating country coordinating mechanism’s and recipient’s financial performance, the Working Group assessed that LFAs lack the mandate, Source: Jessica Brinton. resources, and staff capacity to ensure representative, credible, and rigorous verification of results reported by recipients, mostly due to constrained resources and a lack of technical expertise on health and diseases.15 Beyond regular desk reviews of program and financial that recipients’ outputs lead to better health services and population documents, LFAs’ annual on-site data verification and rapid services health. A principal recipient could accurately report (and thus be quality assessment provide the only routine on-the-ground spot verified) as having distributed a given number of insecticide-treated checks of program performance. Yet these two procedures, while bed nets to households in high-transmission areas, yet omit (or be helpful in spotting or signaling egregious problems, are mostly unaware) that nets had been misappropriated as fishing equipment18 limited to documentation review rather than independent, obser- or soccer nets (figure 6.1). vational verification of intervention coverage or quality. Given their A third concern relates to innovative programs or interven- limited scope (eight or more site visits by one staff member over tions of unknown efficacy. In insecticide-treated bed net distri- about 6–12 days for at least three indicators), they are also unable bution, more robust verification of household use would likely to offer a representative sample for all but the smallest programs, provide enough documentation of program effectiveness for two even if sites are selected through random sampling (as recommended reasons. First, the outcome of interest (correct use) can be easily in Global Fund guidance, though not commonly implemented). observedii by an independent evaluator—bed nets are clearly vis- Further, selected sites are notified of the LFA visit a week before, ible in households, and the target population is not stigmatized or giving time to prepare data sources.16 In contrast Rwanda’s highly hard to reach. Second, extensive biomedical literature demonstrates successful PBF scheme also uses an audit approach, but auditors verify results at all facilities once each quarter.17 ii. One caveat relates to the quality of bed nets, which may not be read- Second, because of the portfolio-wide emphasis on documenta- ily observable to the naked eye. If insecticide efficacy has been degraded tion review and verification, these procedures cannot assess interven- despite nets’ pristine appearance (a lack of holes or tears) and correct use, tion coverage and outcomes at the population level—and thus ensure the nets may not offer the expected protection. Performance verification may need to incorporate an element of quality assurance to assess the type measurement in line with principals of country ownership and health sys- of net (a traditional bed net or a long-lasting insecticide-treated bed net) tem strengthening, and as an effort to avoid creating new parallel systems. and the time passed since its distribution or manufacture. 46 Verifying performance Verifying

a clear relationship between the outcome (correct use) and impact recommends that the Global Fund move rightward along the spec- (prevention of malaria transmission). Yet many other Global Fund trum. In the short term this entails a more rigorous and represen- interventions will face difficulties on both outcome and impact, as in tative approach to verifying self-reported results. In the long term implementing behavioral interventions to prevent HIV among high- the Global Fund could consider shifting from “verification” toward risk groups such as commercial sex workers or men who have sex payment based on independently measured outcomes and impact. with men, or with social marketing programs to encourage condom The Global Fund should thus immediately strengthen its verifi- use. In such situations, impact evaluation is needed to isolate a clear cation of recipient performance through a more robust approach to causal relationship between the intervention and health impact. measuring the quantity and quality of health services delivered with its support. The Working Group remains committed to strengthen- Opportunities and limitations ing national health information services, and urges the Global Fund to continue investing in this crucial element of health systems. Even Different approaches to assessing grant performance can be rep- so, to ensure that the Global Fund makes decisions based on accu- resented as a continuum between hierarchical self-reporting and rate and reliable information, robust independent verification and purely independent measurement (figure 6.2). At one extreme, the measurement must complement recipients’ self-reported data and grant recipient does all measurement, without external checks to domestic verification. Independent verification and measurement verify accuracy. At the other extreme, self-reporting is required, but could potentially be undertaken by a polling or consulting firm, grant performance is assessed based on independently conducted national statistics office (if independent), United Nations agency, population-based measurement. Few funding agencies adopt either research group, or nongovernmental organization, among others. extreme approach. Rather, they choose among hybrid approaches in The benefits of robust, independently verified data are fourfold: the middle of the spectrum, where self-reported results are subject to • First, as a recent World Bank report notes, “the very existence of increasingly rigorous verification and supplemented by population- the verification process is a key improvement in the governance based measurement to assess the coverage, outputs, and impacts of of the health system” through its ability to both promote health supported programs. system accountability and encourage national dialogue on health The Global Fund’s verification approach is toward the left end service results.19 of the spectrum, with principal recipients’ self-reports and cursory • Second, independent data sources and rigorous verifica- independent checks for accuracy and data quality. While there is tion improve the quality of administrative data, critical to no correct approach to verifying performance, the Working Group promoting sustainable M&E systems in recipient countries

Figure 6.2 Approaches to assessing grant performance

Self-reports and Population-based Self-reports and rigorous, representative measurement and data quality audit independent veriﬁcation ﬁnancial audit only

Hierarchical Self-reports and Rigorously veriﬁed Independent, self-reporting cursory independent self-reports supplemented by population-based veriﬁcation independent measurement measurement

TODAY TOMORROW THE FUTURE?

Source: Authors. 47 V erifying performance erifying

and improving in-country program management. Even the country-owned processes, the Working Group takes a different view. best-performing countries will gain if they can regularly test Indeed, independent verification is valuable largely thanks to its their administrative reporting systems against independent ability to validate and strengthen the country-owned measurement robust and reliable data. For the lowest capacity countries, architecture, as in Cameroon. Further, independent verification such independent verification may be the only way to have need not be conducted by foreign entities. Local nongovernmental accurate data until the substantial time and investment in organizations or research groups are often well-equipped to serve this reporting systems begins to pay off. Indeed, when programs role. And in some countries there may even be independent govern- financed by the performance-based Health Results Innova- ment agencies with a mandate and demonstrated ability to do so tions Trust Fund implemented verification in participating (for example, independent statistical offices or inspectors general). facilities (at times alongside penalties for over-reporting), the A second stream of objections stemmed from worries about World Bank observed a clear and rapid jump in the accuracy adding additional checks and paperwork to the already-extensive of self-reported data on quantity of services delivered.20 In Global Fund grant management framework—a real concern. But Cameroon, for example, independent verification helped sig- robust performance verification is perhaps the most essential check nificantly reduce over-reporting of outpatient consultations.21 of all risk management controls. After all, how can the Global Fund Still, there remains much to learn about the optimal strategy ensure that funds are used properly without convincingly dem- for measuring and verifying service quality. onstrating improvements in the results established by its strategy • Third, robust performance verification is critical to informed framework? program management by the Secretariat. Without on-the-ground A separate but related concern is “attribution”—whether results staff who can regularly interact with beneficiaries and observe need to be assigned to an individual funder or program, rather than program implementation, independent data are crucial for ensur- to the joint efforts of all stakeholders. Global Fund staff described ing that the Secretariat has an accurate assessment of the returns the organization’s current momentum away from “project-based to its financial investments. In turn accurate data ensure that aid” toward “investment in the national program”—that the Global performance-based payments reward real improvements, rather Fund was moving away from attribution of outputs and impacts. than administrative reporting errors or intentional manipula- Even so, the Global Fund is committed to using its funds properly tion. For this reason alone verification of programmatic data from an audit perspective, meaning that at the very least it will deserves substantial investment by the Secretariat, likely equal- continue to require attribution of inputs. This speaks to the differ- ing or exceeding the amounts spent on LFAs. ent purposes that attribution can serve, purposes closely related to • Fourth, high-quality data are global public goods that can be the areas of measurement described above—particularly resource coordinated with other stakeholders and inform the work of allocation, program management, and accountability. national governments, donors, and independent researchers. Attribution is important for determining what does and does To improve accountability around the Family Planning 2020 not work—whether an intervention is effective. Even if the overall commitments, the Bill & Melinda Gates Foundation and oth- national program is seeing strong epidemiological progress, it is still ers will support direct data collection in 69 countries, including wasteful to invest scarce resources in an ineffective component. baselines and annual follow-ups to estimate modern contracep- This is at the core of impact evaluation, which seeks to isolate tive users. Such large data collection efforts merit joint support, the causal effect of an intervention from overall trends and other and connections with funders like the Global Fund. confounding factors, to “measure the net change in outcomes for In meetings and consultations with Global Fund staff and other a particular group of people that can be attributed to a specific stakeholders, the Working Group heard concerns about adopt- program,” as defined by the International Initiative for Impact ing a greater focus on independent verification and measurement. Evaluation.22 This definition is thus quite distinct from that of Some worried about the implications of independent measure- the Global Fund’s Technical Evaluation Reference Group, which ment for country ownership and health systems strengthening. emphasizes “the importance of contribution and assessing causa- While the term “independent” is often interpreted as detrimental to tion and competing explanations rather than narrow attribution to 48 Verifying performance Verifying

one source of financing and single intervention.”23 The core ques- Verify the accuracy and quality of principal tion remains quite difficult, however. At what level is attribution recipients’ self-reported results using rigorous, required to ensure accountability of funds, identify effective and representative measurement instruments ineffective program components, and enable active grant management? While attribution of impact may not be necessary in most The Global Fund should design efficient mechanisms to rigorously cases, the Working Group believes that attribution of outputs verify a few of the most essential self-reported program indicators. remains important for accountability and management. Where To do this, the Global Fund should create terms of reference for the Global Fund finances only one part of the health production a local performance agent in each country, an entity independent function, it can assess its contribution on a set of outputs—but it of the LFA but conceived as the LFA analogue for grant perfor- should still be able to draw a direct line between its investment mance. LFA nonfiduciary responsibilities (such as the on-site data and service delivery. verification and the rapid services quality assessment) should be A final consideration for implementation—closely related to the gradually scaled back, and in their place local performance agents question of attribution—is the difficulty of constructing a robust would provide independent verification of recipients’ self-reported baseline against which to measure future progress. In chapter 4 the results. Verification should be at least annual, to align with the Working Group advocates for an explicit link between funding and Global Fund’s annual disbursement cycle under the new funding incremental progress on a subset of the most important indicators. model. While the verification would vary by country and program Yet without knowing the initial coverage or retention rates, it will type, it should abide by four minimum requirements: not be possible to assess marginal improvements. Further, in many • Verification must be technically sound and produce robust,iii countries it may not be possible to measure a baseline before grant representative results of the facilities and people involved or implementation begins for the Global Fund’s first grant cycle under targeted in a Global Fund–sponsored intervention. the new funding model. The Working Group thus recognizes that a • Verification visits must be unannounced. baseline may not be immediately feasible in some contexts. Where • Verification must cover the relevant core indicators—all indica- this is the case measurement during the first round of the new tors possibly tied to performance disbursements. funding model can serve as the baseline for the second three-year • Verification must be conducted by an independent third party grant cycle, when PBF recommendations can be fully implemented. (the local performance agent). Verification will fall into two broad categories depending on the Recommendations program: clinic-based services and community- and population- based services. For clinic-based services verification should include Define a subset of core indicators to receive unannounced on-site data audits at a representative sample of facili- strengthened performance verification ties, assessments of service readiness and quality (stock-outs, absenteeism), and interviews with reported program beneficiaries. Where As chapter 4 outlined, the Global Fund should reduce the set of possible, the Global Fund should “piggy-back” on current verifica- M&E and PBF indicators as much as possible and shift to a small tion efforts, such as the President’s Malaria Initiative’s deployment set of core indicators that measure the most important outputs of end-use verification in facilities that receive joint support. and outcomes that define value. Ideally, these core indicators will For community-based programs (that is, bed net or condom be linked and aligned closely with country data systems, the U.S. distribution, behavior change, and programs targeting orphans and President’s Emergency Plan for AIDS Relief, the President’s Malaria vulnerable children) verification should require a representative Initiative, and other significant external funders. These indicators annual “mini-survey” within the target population to assess service should be based on a clear relationship between the intervention coverage and effects, such as whether distributed bed nets were being measured by the indicator and its health impact, incorporating the quality of care. Progress on these indicators should be regularly iii. A robust statistic is resistant to errors in the results; a robust estimator verified across all relevant grants. will be reasonably efficient, with reasonably small bias. 49 V erifying performance erifying

used correctly in the targeted community. Mini -surveys—possibly Complement verification with population-based through mobile phone interviews,24 where appropriate—would be measurement and formal impact evaluation for less precise but nonetheless be representative and document pro- interventions and service delivery strategies of gram implementation and effectiveness. Measuring community- unknown effectiveness based programs will require greater consensus about their core objectives and corresponding indicators of outputs and outcomes. Regular output verification should be complemented by representa- Many programs will include multiple clinical- and community- tive, population-based measurement of the target population once based components. In such cases, verification should occur for any per three-year grant, timed to coincide with grant negotiations activity that represents a significant chunk of Global Fund support for the next funding cycle. The Global Fund should take advan- in dollar terms—that is, an activity costing at least $250,000, or tage of current population-based measurement exercises, such as some other threshold to be set by the Secretariat. the Demographic and Health Survey and the Multiple Indicator Independent verification, particularly at the facility level, creates Cluster Survey. But the Global Fund will also need to commission incentives for investing in accurate, complete, and routine monitor- tailored surveys to meet verification needs. In both cases the goal is ing systems. These systems can be supplemented by explicit financial to measure the coverage of key health services among the targeted or reputational incentives to reward high-quality routine reporting population and to assess trends in health that can be connected to or penalize inaccuracies, as has been done in a pay-for-performance Global Fund investments. The target population may be defined by scheme in Rwanda.25 Many other design features of verification are geographic boundary, age, gender, other high-risk behaviors, or some being evaluated by the World Bank’s Health Results Innovations combination—for example, key populations within geographically Trust Fund, and the Global Fund should incorporate evolving defined hot spots. In the long run the Global Fund may consider evidence into its terms of reference for local performance agents.iv moving to more frequent population-based measurement, where Performance verification should be implemented across the coverage and outcomes are linked to payment, instead of output Global Fund grant portfolio. Such additional checks would have verification (further to the right on the performance assessment financial implications, though they would be mediated by scaling continuum; see figure 6.2). back LFA responsibilities, eliminating the current rapid services In cases where a country opts out of the Global Fund interven- quality assessment, data quality audit, and on-site data verification tion list (see chapter 3)—where it funds interventions of unknown procedures, and coordinating measurement with other donors. efficacy or cost-effectiveness—the end-of-cycle measurement should To ensure that recipient and Global Fund investments achieve a be part of a broader strategy to facilitate causally attributable impact strong return in service coverage and health impact, the Board evaluation that links impacts to specific interventions, and that should authorize all requisite resources for this endeavor. In turn builds on a baseline established at the beginning of the grant cycle the Secretariat should draft clear, standardized guidance on the (ideally with randomized intervention and control groups, and coor- expectations for verifying grant performance. Over time the Global dinated with other partners to avoid confounding or duplication). Fund could also work to cut costs and increase the frequency of Where impact evaluation shows new or unproven interventions verification through new monitoring and survey technologies. For to be effective and cost-effective, the Global Fund can expand its example, recent analysis by Croke and others (2012) suggests that eligible interventions list to reflect the evolving evidence base. In high-quality, representative panel data could be collected by mobile doing so the Global Fund can both support new ideas and innova- phone interviews in several African countries, at a cost of about tive delivery strategies, while also ensuring that its resources mainly $2,500 per survey round.26 fund interventions proven to save lives or prevent new infections. iv. A fuller discussion of different approaches to verifying performance Summary in performance-based incentive schemes is available on the World Bank’s Results-Based Financing for Health website (www.rbfhealth.org) and in The efficacy of the Global Fund’s core model (and thus its ability to a forthcoming cross-country analysis on verification. implement a value for money agenda) depends on its having accurate 50 Verifying performance Verifying

and reliable data for the programs it is funding. While LFAs have a 5. www.theglobalfund.org/en/me/documents/dataquality/. long history of validating grant performance, which has been sup- 6. The Global Fund (2013d). plemented by other exercises such as data quality audits, program 7. The Global Fund (2012b). reviews, and impact assessments, the Working Group believes that 8. www.theglobalfund.org/en/lfa/. current checks on data accuracy are insufficient. Because ensuring 9. The Global Fund (2013b). results is a key component of the Global Fund’s core mandate and 10. The Global Fund (2011f). new strategy of “investing for impact”—and because of clear evidence 11. World Bank (2013a). that administrative data are unreliable and subject to distortion by 12. The Global Fund (2011a), p. 2. perverse incentives—the Working Group recommends that the 13. Lim and others (2008). Global Fund adopt a systematic framework for using independent, 14. Abelson and Creswell (2012). representative sampling and rigorous measurement instruments, both 15. Glassman and Silverman (forthcoming). for performance verification and impact evaluation. Verification must 16. The Global Fund (2011f). include first-hand observation at the facility and beneficiary levels. 17. Basinga and others (2010). Such measures will require sustained Global Fund investment and 18. Martins, Zwi, and Kelly (2012). possibly a new dedicated entity, but they should nonetheless be a core 19. World Bank (2013b), p. 18. element of the Global Fund’s value for money agenda. 20. Cashin and Vergeer (2013). 21. World Bank (2013b). Notes 22. International Initiative for Impact Evaluation (2008). 1. www.theglobalfund.org/en/about/whoweare/. 23. The Global Fund (2012d). 2. The Global Fund (2011h). 24. Croke and others (2012). 3. The Global Fund (2012c). 25. Loevinsohn (2008). 4. The Global Fund (2012c). 26. Croke and others (2012). 51

Chapter 7 Conclusions

In April 2013 the Global Fund released four documents to motivate Box 7.1 Excerpt from the new funding model its upcoming replenishment, which aims to raise $15 billion over “To adapt to a new economic reality, new technologies, sci- 2014–16.1 The documents highlight the Global Fund’s achieve- entific advances, and a better understanding of epidemio- ments thus far, including its contribution to lowering disease bur- logical patterns, the Global Fund needed to make changes, den from HIV/AIDS, tuberculosis, and malaria in more than and move . . . toward sustainable programs.” 100 countries. They further showcase the advantages of the new Source: The Global Fund (2013c), p. 1. funding model, such as flexibility, simplicity, and a more active approach to between-country allocation and engagement (box 7.1). on how the Global Fund can move forward in adopting this But despite references to themes in this report, “value for money” agenda. is not explicitly mentioned as a core Global Fund objective for the replenishment. Reflect value for money in key performance Since its inception in 2001 the Global Fund has undergone mul- indicators tiple comprehensive reviews, generating many recommendations. The Global Fund has proven dynamic and agile in its response, The Global Fund’s key performance indicators orient the action of transforming in an effort to manage risk, maintain donor confi- the Secretariat, and should provide a benchmark to assess Secretariat dence, and increase its health impact. But it acknowledges that performance and Global Fund leaders’ success. But in the past the transitioning to sustainable results is not yet complete. Global Fund’s performance indicators have been mainly process- This report seeks to complement the Global Fund’s progress oriented: “percentage of funds allocated to civil society organizations with a practical value for money agenda. It targets four value as implementers”; Global Fund “operating expenses as a percentage for money domains: allocation, contracts, costs and spending, of total expenditures”; and “percentage of well-performing grants.”2 and performance verification. This report’s recommendations, While the performance indicators included a value for money indica- discussed in chapters 3–6, vary in urgency and immediate fea- tor, it was narrow in scope, low in rigor, and an amalgamation of three sibility. Some are more pressing and require immediate action, non-comparable indicators.ii The extent to which the Board judged such as defining an eligible commodity list. Others necessitate Secretariat performance against the indicators is also unknown. It is long-term attention, such as measuring and applying unit costs unclear whether the performance indicators represented an explicit and more robustly verifying performance. Table 7.1 summarizes standard for Secretariat accountability. the recommendations, and appendix 1 offers suggestions on the With the performance indicators under revision, the new indica- sequencing and division of responsibility among the partners.i In tors represent an important opportunity to set clear value for money addition to the recommendations made in each substantive area, expectations for the Secretariat. this report offers five final “how” recommendations—thoughts ii. A simple arithmetic average of percent change in the median price i. Short-term recommendations are those that the Global Fund can imple- paid for antiretroviral drugs per patient-year; the median price paid for ment with few staffing and capacity needs, within the next year. Medium- insecticide-treated bed nets; and the “proportion of countries with a DOTS and long-term recommendations are those that will need new institutions unit cost per patient successfully treated within reference range” (The and staffing, and could be implemented within the next three years. Global Fund 2012a, p. 17). 52 Conclusions

Table 7.1 Value for money: summary of domains, key problems, and recommendations

Domain Key problem Recommendation Allocation. How can National and donor funding Choose from a menu of effective and cost-effective interventions resources be allocated is not consistently supporting and commodities. to maximize impact on best practice, despite Identify and target key populations with appropriate interventions. HIV/AIDS, tuberculosis, and substantial evidence on what malaria? works most cost-effectively to Optimize investments for the greatest health impact. reduce disease. Improve ex ante budgeting and transparency on spending. Contracts. How can Current agreements provide Directly connect a portion of funding to incremental progress on contracts and agreements only weak incentives for performance. between the Global Fund and impact. Link performance payments to incremental progress against the its recipients be structured to most important indicators. create stronger incentives? Support performance incentives between the principal recipient and service providers. Cost and spending. How Cost, price, and spending on Continue to improve the scope, completeness, and timeliness of can costs of and spending on commodities varies widely reporting to commodity price tracking systems. commodities, supply chains, between countries; this Benchmark and use supply chain costs and outputs. and service delivery be better variation is unexplained. tracked and used? Identify core services for more extensive analysis and use of service delivery costs and spending. Share costing data with partners and the public. Develop a strategy to use unit-cost data throughout the new funding model grant cycle. Performance verification. The Global Fund relies on Define a subset of core indicators to receive strengthened How can performance be weak instruments to verify performance verification. verified and evaluated the accuracy of self-reported Independently verify the accuracy and quality of principal rigorously, to generate performance measures. recipients’ self-reported results using rigorous, representative greater incentives and measurement instruments. accountability? Complement output verification with population-based measurement and formal impact evaluation for interventions and service delivery strategies of unknown efficacy.

Source: Authors.

Build better accountability with technical Global Fund can seek external support when needed, as it did when partners it contracted for the impact evaluation of the Affordable Medicines Facility–malaria with the London School of Hygiene and Tropical While responsibility for implementing many of the recommendations Medicine. Alternatively, the Global Fund’s donors may wish to fund falls on the Global Fund’s Secretariat, Board, and Board committees, technical partners directly for certain deliverables, as part of their value for money must be a shared agenda among Global Fund part- technical assistance funding to the Global Fund. ners. In the past the loose accountability between the Global Fund and its technical partners has sometimes failed to deliver key inputs Connect countries with scarce (but essential) to grant proposals and implementation that would enhance value for expertise to inform allocation money. The Global Fund may require more formal contracts with technical partners to obtain needed data, skills, and support. Instead Most countries (and many global health funders) lack the in-house of a memorandum of understanding, the Global Fund Secretariat capacity to apply cost-effectiveness, modeling, and other health eco- can contract with partners for specified deliverables, following the nomics tools in grant applications and national planning processes. example of the GAVI Alliance and its relationships with the United Spread over many agencies, universities, and companies, with few Nations Children’s Fund and the World Health Organization. Or the specialized institutions or departments, health economics expertise 53 Conclusions

is both scarce and diffuse—and thus rarely applied to routine plan- Box 7.2 Indicative menu of services provided ning for resource allocation and management. Greater partnership is by the proposed Decision Support Network thus needed to connect countries with this expertise. In the interim, • Economic modeling (ex ante): input into design and ad- recipients and the Global Fund could obtain technical assistance justment of intervention mixes to realize specific disease from partners with institutional modeling capacity, such as the goals. World Health Organization, the Joint United Nations Programme • Country or payer cost-effectiveness and budget impact on HIV/AIDS, the World Bank, and the U.S. President’s Emergency analysis for specific technologies and commodities, ben- Plan for AIDS Relief. Yet, given competing demands for their time, efits plans, or negative lists. other sources will be needed in the long run. • Impact evaluation (ex post). Inspired by the work of the HIV Modelling Consortium in mar- • Financing and sustainability frameworks and analysis. shaling applied health economics research to the fight against HIV, • Analysis of fiscal and budgetary issues: risk adjustment, the Working Group thus proposes establishing a dedicated network federal-state transfers, conditional block grants, perfor- to connect countries and donors with health economics expertise mance-based financing, and so on. on HIV, tuberculosis, and malaria. Tentatively called the “Decision • Costing and efficiency analysis. Support Network,” this nonprofit resource netw ork—based at an • Behavioral economic analysis: interventions for adher- existing entity—could mobilize expertise across organizations offer- ence, preventive care seeking, and healthy behaviors. ing a menu of services to provide demand-based analysis to support • Expenditure analysis: uses, budgets, benchmarking, evidence-based and efficient resource allocation and management benefit incidence, and so on. (box 7.2). While the network would not be a formal arm of the • Assessment and provision of data and information needs Global Fund or any other funding agency, the Global Fund and for the above services. other financing institutions could help its creation by promising to commission a substantial quantity of its analysis once created. Once such a network is created, the Global Fund could both Box 7.3 Statement by Ambassador Eric contract its services to inform institution-wide priority-setting and Goosby policy, and also encourage (or fund) countries to commission its “With country leadership the new paradigm for the future expertise for national planning processes. Over time, uptake of net- response entails more joint planning [and] cognizance of work services would allow countries and donor agencies to overcome shared responsibility to people who need services, to donor barriers to applying health economics to national policy, enabling countries, and to the U.S. taxpayers to be assured of effec- such analysis to be routinized into both Global Fund grant-making tive and efficient use of their resources. and national strategic plans. —PEPFAR Ambassador Eric Goosby, 2012 Given the President’s Emergency Plan for AIDS Relief’s proposed increased contribution to the Global Fund in 2014, a larger amount of technical cooperation funds will be available to support and population-based surveys once per three-year grant cycle. While Global Fund operations (box 7.3). This added funding may be an the Global Fund can independently contract these tools for its opportunity to design and deploy a Decision Support Network as grants, it can also achieve substantial efficiencies by coordinating recommended by the Working Group. data collection and analysis with countries’ statistical agencies and other funders and stakeholders—and then by distributing data Create synergies in data collection to maximize its value and use. Coordinating data collection can and analysis reduce the duplication of efforts and facilitate comparability across surveys and funders, helping realize the full potential of data as a In chapter 6 the Working Group recommended that the Global global public good and empowering countries to use measurement Fund commission more rigorous performance-based verification for program management and planning. 54 Conclusions

Some progress has been made in harmonizing indicators and whom it interacts. . . . [Fund portfolio managers] should be data collection across countries and agencies. The Joint United systematically exchanging knowledge with in-country play- Nations Programme on HIV/AIDS’s Monitoring and Evaluation ers—not only [with country coordinating mechanisms, local Reference Group, which includes the Global Fund, has defined 30 fund agents, and principal recipients], but also with [United core progress indicators for global HIV control.3 Likewise, as part Nations] agencies, the World Bank, regional development of a broader agenda to harmonize donors and national governments, banks, and bilateral donors, especially those that are provid- the International Health Partnership (to which the Global Fund is ing funding for related fields such as health systems strength- a signatory4) works “to increase the use of shared mechanisms for ening and the management of pharmaceutical supply chains.”6 reporting on progress and reviewing performance.”5 Building on these efforts, the Working Group recommends that And while this concern about knowledge-sharing may seem global health funders continue to pursue measures that improve lofty, it has many concrete and practical applications for program survey coordination and data sharing with each other, with national development. Anecdotal evidence suggests that more success- governments, and with the public. Funders and technical partners ful grants have fewer subrecipients and clearer contractual and could establish a joint database of all funded data collection efforts accountability relationships between the principal recipient and worldwide, such that Secretariat and headquarters staff could easily subgrantees. assess data sources. They could evaluate the need for further invest- The Global Fund should help foster a learning community that ment. And, where relevant data already exist, they could request could better share best practice in program implementation, and access from other agencies to inform planning and grant negotiation. thus cultivate more effective methods to increase efficiency and improve service delivery. Such a culture should start by sharing more Assess and share best practices among information within the Secretariat. This would allow fund portfolio principal recipients, country coordinating managers to be better appraised of the pros and cons of compara- mechanisms, and other partners tive practices and thus able to communicate those lessons to their country’s principal recipients, country coordinating mechanisms, Despite many common experiences—and thus many opportuni- and other partners when appropriate. The Global Fund could also ties to learn from each other—principal recipients and country connect principal recipients and country coordinating mechanisms coordinating mechanisms rarely interact with their counterparts with each other through remote communication (Skype, phone in other countries. Such national “silos” are problematic, as they calls, or email), or through short study trips to enable first-hand impede assessing and sharing best practices of grant implementa- observation. And where implementation arrangements appear sub- tion and evaluation. Without greater interaction the two may optimal and the current recipient shows little interest in addressing be unaware of alternative implementation arrangements or more the problem, the Global Fund could incubate alternative recipients efficient practices. as a strategy to encourage innovation and find the most efficient These silos may also extend to the Secretariat. Fund portfolio channel for its resources. managers are knowledgeable about the implementation successes and challenges in countries under their purview. But it is unclear Wrapping up whether their individual assessments are systematically translated into institutional knowledge about “what works” that is then shared Achieving value for money—the greatest health impact with avail- with the Global Fund’s principal recipients and country coordinat- able resources—is the core business of any global health funder. ing mechanisms. Value for money cannot be an afterthought, a checklist, or an extra As a high-level independent review panel noted: obligation—it is the very essence of ethical and responsible global “The Global Fund acknowledges that it does not make the health funding. best use of the vast store of knowledge, evidence, and insights From the Global Fund Secretariat in Geneva, to the Office of available from the wide range of people and institutions with the U.S. Global AIDS Coordinator in Washington, DC, to the 55 Conclusions

Department for International Development in London, to the hos- Notes pital in Nigeria where hundreds are on antiretroviral treatment, to 1. www.theglobalfund.org/en/donors/replenishment/fourth/. civil society organizations like the Center for Global Development, 2. The Global Fund (2012a). everyone bears some responsibility for improving value for money. 3. UNAIDS (2011). And everyone will benefit from the ensuing gains in efficiency, 4. www.theglobalfund.org/en/about/partnership/development/. quality, and health. The Working Group hopes this report can 5. www.internationalhealthpartnership.net/en/key-issues/ prompt and guide the Global Fund and its partners’ ongoing value monitoring-evaluation/. for money transformation. 6. The Global Fund (2011l), p. 35.

Appendix 1 Sequencing and division of responsibility for the value for money agenda

Domain Recommendation Division of responsibility Outlook Allocation Choose from a menu of effective and cost-effective Global Fund Secretariat, Short term interventions and commodities principal recipients Identify and target key populations with appropriate Global Fund Secretariat, PEPFAR, Short term interventions partner country governments Improve ex ante budgeting and transparency on Global Fund Secretariat, Short to spending partner country governments, medium term country coordinating mechanisms Optimize investments for the greatest health impact Global Fund Secretariat, Medium term Technical Review Panel Contracts Link performance payments to incremental progress Global Fund Secretariat Short term against the most important indicators Directly connect a portion of funding to incremental Global Fund Secretariat, Medium term progress on performance principal recipients Support performance incentives between the principal Global Fund Secretariat, Medium term recipient and service providers principal recipients Costs and Continue to improve the scope, completeness, and Global Fund Secretariat, Short term spending timeliness of reporting to commodity price tracking principal recipients, subrecipients systems Identify core services for more extensive analysis of Global Fund Secretariat, Market Short term service delivery costs and spending Dynamics Advisory Group Benchmark and use supply chain costs and outputs Global Fund Secretariat Short term Develop a strategy to use unit-cost data throughout Global Fund Secretariat, Short term the new funding model grant cycle PEPFAR Share costing data with partners and the public Global Fund Secretariat, Long term principal recipients, subrecipients Performance Define a subset of core indicators to receive Global Fund Secretariat, Short term verification strengthened performance verification technical partners Verify the accuracy and quality of principal recipients’ Global Fund Secretariat, Medium term self-reported results using rigorous, representative local performance agents measurement instruments Complement output verification with population- Global Fund Secretariat, Medium term based measurement and formal impact evaluation local performance agents for interventions and service delivery strategies of unknown efficacy

Appendix 2 Innovations in the design of “contract-like” grant agreements

The current practice of the Global Fund and most other health This appendix proposes alternative contract designs for that donors is to issue grant agreements that specify the donor’s dis- output-contingent part of a grant agreement. “Agreements” and bursement schedule in terms of the recipient’s spending on inputs “contracts” are used interchangeably here to refer to these hypotheti- and conditional mainly on the recipient’s timely submission of cal “contract-like” agreement structures, which the Global Fund or documents to support this spending. Some grant agreements only another donor could choose to issue. The payments for these outputs allude in passing to the health service processes, outputs, and out- could be from the Global Fund to the principal recipient or from comes to which these inputs are intended to contribute. Others pre- the principal recipient to a subrecipient. If the principal recipient scribe a target number of processes, outputs, or outcomes, without is a federal government like those of Brazil, India, or Nigeria, the conditioning payment on a count of any of them. By reimbursing payment could be from the federal government to a subnational incurred expenses rather than paying a predetermined amount per government, such as a province, state, or municipality. unit of output, the traditional grant agreement perversely rewards Regulatory regimes in Europe and North America are designed higher expenses per unit of output. Since any cost reduction the to improve the value for money that the public receives for its pur- recipient achieves saves money only for the Global Fund, this kind chases of critical services from private or parastatal providers. Regu- of agreement provides no incentive for the recipient to economize. lators charged with designing regulatory regimes draw on a large The incentives in the traditional grant design have little power to literature on optimal regulation and optimal procurement, which motivate the recipient either to improve efficiency or to save money explores many design alternatives.i Some alternatives can achieve for the Global Fund. exemplary value for money under the assumption that the regula- More ambitious agreements condition payment on a count of tor knows the producer’s entire cost function, for both the past processes, outputs, or outcomes but depend mainly on the grant and the future.ii With this knowledge a global health donor could recipient’s own report, with limited or weak third-party verification. Such an agreement violates a principle of efficient contract design i. For textbook treatments, see Laffont and Tirole (1993) and Armstrong, first enunciated by the philosopher Charles Babbage in the 1830s: Cowan, and Vickers (1994). The first chapter of Laffont and Martimort “That every person connected with [an enterprise] should (2002) concisely reviews the intellectual history of incentives, contracts, derive more advantage from applying any improvement he and mechanism design starting with the work of Adam Smith. might discover [to improving the efficiency of the enterprise] ii. A cost function is defined as a function that estimates the total annual than he could by any other course.”1 cost of an operating facility as a function of how much of each of its outputs it produces, the prices of its factor inputs (like labor, utilities, and capi- Chapter 4 argues that pursuing efficiency in health financing tal), and an array of environmental and policy determinants. By making requires that part of each grant agreement be reserved for disburse- strong assumptions about the interactions among the various outputs, ment within a more contract-like agreement. The first requirement this total cost function can be divided by one of the outputs to construct of such a contract or agreement is that a quality-adjusted unit of a function relating average cost to the same variables: quantities of all the service output be mutually agreed by the donor and recipient during outputs, prices paid for all the factors of production, and social and eco- negotiation and subsequently that the number of produced units nomic determinants. Health economists and health service researchers of this output be counted during the agreement’s implementation have been estimating cost functions for various categories of health services and be verified independently by a mutually agreed third party. for decades. See Meyer-Rath and Over (2012) for a detailed review and 60 Appendix 2

be ensured of paying the recipient the minimal cost for its efficient probably because it does not itself know, how costs would change production of any verified number of quality-adjusted units of out- with any of several variables that might change from one year to the put. If the regulator has perfect knowledge of the producer’s cost next. For example, assume that no one knows how average cost will function, technology changes would be routinely absorbed into the change with the number of units produced—that is, with the “scale payment amounts, so that cost-saving technological improvements of production.” Since many production processes in the health sec- would appropriately reduce the donor’s average payment per unit tor involve a substantial fixed cost (such as the cost of the building of output and quality-enhancing improvements would, if they pass and of personnel salaries), these processes benefit from “economies a cost-effectiveness test, appropriately increase the donor’s average of scale” and can produce at a smaller average cost if they expand payment. their output. Suppose all parties are sure that economies of scale But the nascent literature on the determinants of the cost of apply but are not sure how much average costs will decline with HIV/AIDS service delivery (see chapter 5) already reveals how diffi- scale. The uncertainty might be because the managers and their cult it is to know the entire cost function for antiretroviral treatment staff will need to experiment with management arrangements to (ART) delivery. And the way the costs of tuberculosis or malaria handle the expanded number of clients. Or it might be due to the services vary across all facility types, ownership types, geographi- unknown cost of a proposed quality-enhancing new drug or other cal locations, sizes, and scopes are even less understood. With these technological innovation. technical obstacles to knowing the cost function, it is reasonable to A very simple but surprisingly powerful contract design is the assume that much less than perfect cost information will typically Vogelsang-Finsinger (VF) mechanism.2 Suppose that the average be available to the donor. This appendix therefore groups contract cost per quality-adjusted unit of output last year was $600 and that design alternatives into two broad categories, depending on whether 600 units were produced. (This might be the facility-specific cost of the recipient is able to learn and willing to reveal its total cost for ART, for example.) The VF contract design is simply an agreement producing last year’s output. First, consider a contract design that to pay the recipient $600 for every unit of quality-adjusted output could encourage efficiency improvement if the recipient reveals its it produces this year, with the understanding that if it is able to pro- previous year’s total cost to the donor every year. Second, consider duce at a lower average cost than $600 it can retain the difference a contract design that could work even without the recipient reveal- between its receipts and costs, to be redeployed for investments in ing its previous year’s total cost. building maintenance, new equipment, and so on.iii This last provision establishes the recipient as the “residual claimant” on any excess Contract designs that assume knowledge of of donor payments over costs. By guaranteeing residual claimant last year’s total cost status, the grant agreement creates an incentive to the recipient. Because of economies of scale, the incentive will be for the recipient Suppose that the recipient is able to learn, and willing to reveal to to expand output (without sacrificing quality). Figure A2.1 shows the donor every year the number of quality-adjusted units of output how such a contract would work with a hypothetical average cost produced the previous year—and the total cost in doing so. Since during the first two years it is applied. the ratio of the previous year’s total cost to its output is the previous Under this contract, as long as the recipient can expand output, it year’s average cost (or “unit cost”), the recipient can thus reveal its has an incentive to do so until average cost no longer declines, which previous year’s average cost to the donor and this cost report can in figure A2.1 appears to happen at an output level of 1,200 units. be a condition of contract continuation. At that point, when average costs have flattened out or begun to However, in contrast to the perfect information assumption rise, the recipient will stabilize its output level unless it can discover where the entire cost function is known by both the donor and technology improvements that reduce its quality-adjusted average the recipient, let’s assume that the recipient is not able to reveal, cost. Should it find such improvements, its entire average cost curve

critique of cost functions used in several papers to model the scale-up of iii. In practice the donor would impose an upper bound on the number antiretroviral therapy. of units to be paid at this rate. 61 A ppendix 2

Figure A2.1 Efficiency-enhancing response of a recipient granted residual claimant status and paid the previous year’s average cost for every current year unit of output

Unit cost and revenue Unit cost and revenue

600 600 Net revenue 500 500 Net revenue 400 400

200 200

600 800 1,000 1,200 600 800 1,000 1,200 Quantity Quantity

Source: Authors (Mead Over). will shift downward, and it can again benefit from expanding out- can then redeploy the resources saved to other program needs in put, year by year, until average costs flatten out. the same country or in different ones. While endowing the recipient with residual claimant status A recognized weakness of the VF mechanism is that grant recipi- generates an incentive for the recipient to expand output, by doing ents may be reluctant to reduce their average cost relative to the so the recipient reveals a new lower average cost to the donor. Now previous year because they know that the donor would use this that the donor knows the recipient’s lower average cost, it can reduce lower average cost in the following year as the basis for a reduced the amount it pays this recipient for all future units of output. payment per unit.3 In the Global Fund context the VF mechanism This consequence, if replicated year after year in many grant agree- may be less vulnerable to this weakness for three reasons. ments around the world, has the potential of generating two types First, the regulated natural monopolies to which the VF mecha- of cost-savings for the donor. First, the recipient’s revelation of its nism was originally applied are accountable only to their purely previous year’s average cost provides the donor with an estimated profit-motivated stockholders. In contrast the government and “benchmark unit cost” specific to a country and even to the recipi- nongovernmental organizations that are the “principal recipients” ent within the country. Without this information the donor might of the envisioned agreements are rarely privately owned or account- have to spend millions of dollars to estimate benchmark costs that, able to stockholders, but they are always accountable to some degree because of the time such studies take, would be several years out of to constituencies who expect these recipients to pursue the public date (see chapter 5). Second, and more important, over several years through sequential adjustments the donor’s average spending per the current year and then increases the recipient’s net revenue. This incen- unit of output falls lower and lower, until the donor is paying no tive encourages the recipient and the recipient government’s health system more than the minimum average unit cost achievable.iv The donor to take full advantage of the cost savings attainable from joint production and the attendant economies of scope. This assumption that the recipient iv. To the extent that the recipient can progressively “mainstream” its will reveal to the donor its previous year’s average cost suggests that the service delivery within the nations’ health care system, it can shift part of cost-saving benefits of mainstreaming, like those of scaling up, will be its fixed costs to the system. That lowers the recipient’s average total cost in shared by the donor over time. 62 Appendix 2

interest. The constituencies include their clients, the recipient coun- by the original authors. In response to Sappington’s critique, the try’s government and its citizens, and, especially for international mechanism’s authors proposed that the regulator (in this case the nongovernmental organizations, a global constituency of public- donor) sweeten the deal for the recipient by offering a periodic spirited supporters. These features of the intended recipients imply lump-sum payment. Other adjustments envisioned by the authors that they share many of the donor’s nonmaterial objectives. For include expanding the scheme to encompass all the outputs a recipi- example, to the extent that the recipients represent the interests of ent produces, allowing the recipient to set the payment it receives their public stakeholders, they should share the donor’s interest in for each unit of current output subject only to the constraint that expanding health service delivery in the recipient’s country, improv- its total payment from the donor not exceed the previous year’s total ing the efficiency and cost-effectiveness of its own operations, and cost for all these outputs. With these modifications the mechanism even ensuring the continuing viability of the donor agency, through is a plausible contract design for use with any recipient willing to improvements in the donor’s value for money.v systematically collect and report its cost of production.vii Second, since the donor will still be auditing the recipient’s spending, the recipient that inflates current year’s spending to sustain next Contract designs that assume knowledge year’s per unit payment is forgoing the receipt of fungible net revenues only of a benchmark average cost this year in favor of inflated expenses next year, all of which must be justified against previously authorized budget lines. Recipients that Many recipients do not have the capacity to collect and report their prefer to receive the fungible net revenue this year instead of receiv- operating costs with enough detail to reliably compute the average ing payment the following year for higher inflated expenses, will be cost of each output.viii For some of these recipients it might never- willing to expand their output this year and reap the gains. The V-F theless be feasible for the donor to count and verify the number of mechanism will thus achieve the anticipated efficiency improvements. units of output they produce in a year and to estimate a “bench- Third, suppose that a substantial part of the recipient’s manage- mark” average cost for each unit, knowing that the benchmark is ment and staff changes every year. Those who expect to leave before only an approximation of the true average total cost of production. next year have an incentive to earn net revenue this year, because this Suppose that in the first year of the application of this agree- flexible resource can immediately improve their working conditions. ment, the anticipated total cost will be $400,000 and the target The departing staff will not benefit from the higher future total output will be 800 quality-adjusted person-years of treatment. So, revenue that would result from inflating current year expenses.vi the assumed benchmark unit cost is $500 per unit. The donor and Thus under a global health donor’s agreement with a recipient recipient both aspire to achieve more than 800 units of output, but agency, the VF mechanism may be less vulnerable to the recipi- they are even more uncertain about the cost of producing more than ent’s strategic manipulation than if the mechanism were applied to 800 units than about the cost of the first 800. European and North American regulated monopolies. If a global A “two-part tariff” or two-part price agreement would establish health donor pilots the mechanism, and discovers that recipients two prices, the first being at $500 per unit for the first 800 units.4 resist revealing their true total cost for the previous year’s output, it will be possible to fine-tune the mechanism following suggestions vii. A contract design that is related to the VF mechanism is called the “shared saving contract.” Like the VF mechanism, the shared saving con- v. This is not to assume that the donor’s and recipient’s interests are iden- tract confers residual claimant status on the grant recipient and states that tical. For example, the recipient may attach more importance to fixing any cost savings achieved by the recipient be shared with the donor, with a its clinic’s roof than would the donor, if only because of the cost entailed previously agreed proportion X going to the recipient and the proportion in documenting the need for the roof repair to the donor’s satisfaction. (1 – X) reverting to the donor. Depending on the acceptability of the idea vi. Rapid staff turnover is a common feature of health service delivery that net revenue be shared with the global donor, this sharing provision organizations in recipient countries and justifies assuming these recipi- could be added to the VF mechanism (Weissman and others 2012). ents are more “present-oriented” than would be the owners of the typical viii. The audits performed by the Global Fund’s local fund agents only regulated monopoly in Europe or North America. verify spending. They do not reveal the true cost of production. 63 A ppendix 2

The second would be paid per unit for units produced above the This table reveals that the incentive to the recipient to produce 800-unit threshold. Various types of two-part price mechanisms cor- the 801st through the 900th unit averages $550 per unit (given respond to various rules to determine the second of the two prices. in the second row of column 4), which exceeds the per-unit incen- Suppose that neither the donor nor the recipient is certain tive for producing the first 800. Now suppose that the recipient whether the 801st unit of output will cost more or less than $500. strives to maximize its current year net revenue and can approxi- For example, the attempt to expand program output sometimes mately estimate its incremental or marginal cost of producing a encounters difficulties, meaning higher cost per unit beyond some single additional unit of output during the current year. These threshold of output. In this case the incremental or marginal cost costs include not only its additional direct operational expenses of the 801st unit of output might be $550, $600, or more. But if the per unit, but also the cost it incurs in outreach and additional program, like that depicted in figure A2.1, benefits from economies managerial efforts to attract additional patients and increase the of scale at 800 units and beyond, the incremental or marginal cost demand for its services. As the year progresses and the recipient of the 801st unit might be $400, $350, or less. accumulates verified units of delivered services, two things may In a situation where less is known about cost than was assumed occur. The recipient may find that it cannot reach the threshold above in analyzing the VF mechanism, the second price in the two- of 800 units during the year. In this case it is reimbursed $500 part price mechanism can be used both to enhance the recipient’s per unit for each of the units it has managed to produce and the incentive to be efficient and to reveal the recipient’s marginal cost second part of the contract is inoperative. The count would begin to the donor. at zero again the following year. Or the recipient may find that To see how this would work, consider the example in table A2.1. expansion is difficult and encounters rising costs, but its marginal Suppose the second part of the two-part price agreement specifies cost only exceeds $500 after it has passed the 800-unit threshold. that, in addition to the $400,000 to be paid when the recipient In this case it will expand output into the second part of the two- achieves a verified and quality-adjusted output of 800, the amount part contract until it finds the additional cost is no longer worth paid for all verified and quality-adjusted units of output above 800 the additional payment (or until it encounters the upper bound are given by the entries in table A2.1 and depicted in figure A2.2. of the grant agreement).

Table A2.1 Worked example of payments for above-threshold output during a single year of a two-part price agreement

Amount of the second Payment per unit part of the two-part Marginal revenue per Units of output above for units above the payment (the amount paid unit of output above the the threshold of 800 thresholda above $400,000)b threshold of 800c (1) (2) (3) (4) 1 $600 $600 $600 100 $550 $55,000 $550 200 $500 $100,000 $450 300 $450 $135,000 $350 400 $400 $160,000 $250

a. Entries in column (2) are calculated from the formula: 600 – X/2, where X is the amount of output above the threshold, given in column (1). This formula is designed to be decreasing in above-threshold output. In practice each grant agreement would need its own individually designed and negotiated formula. b. Entries in column (3) are computed as the product of columns (1) and (2). c. Entries in column (4) are computed as the increment in above-threshold revenue from column (3) divided by the increment in output from column (1). For example, the last entry in column (4) is calculated as: (160,000 − 135,000) / 100 = $250, which is the average of the marginal revenue over the interval from 300 to 400 units of output. Source: Authors (Mead Over). 64 Appendix 2

Figure A2.2 Maximum payment of donor should not be used to justify raising the unit payment for the first to recipient under the two-part payment part of its subsequent contract from $500 to $550. Conversely, contract of table A2.1 the observation that the recipient stopped production at 1,200

Unit revenue units should not be used to justify lowering unit payment for the first part of its following year’s contract from $500 to $250. The first of these mistakes would be likely to overpay the recipient and thus be wasteful, while the second might underpay the recipient

Maximum revenue and drive it into bankruptcy. in one year: Although the two-part price contract is a less dependable 600 $560,000 guide to the donor’s payment per unit for the following year 500 than the VF mechanism would be, it still provides substan- 400 tial benefits to both the donor and recipient that would not be Revenue for Revenue available under the traditional grant structure or a flat per-unit ﬁrst 800 units for next 200 $400,000 400 units price contract. For the recipient, the two-part contract offers $160,000 the chance to earn additional revenue while serving additional patients and provides the innovative service manager with the 800 1,000 1,200 Quantity incentive to experiment with attracting and providing quality services to incremental patients at lower costs. For the donor, Source: Authors (Mead Over). with insufficient resources to fund all demand or to estimate accurately the marginal cost of service in all client countries, the The recipient that expands into the second part of its contract two-part contract offers the chance to expand services in any receives additional revenue as its reward. To the extent that it country at a lower unit cost, thus improving the donor’s overall responds to this incentive, it will reveal the incremental or marginal value for money. cost of service delivery. For example, if it stops production at 900 Given that the two-part price contract reveals only the marginal units (or 100 above-threshold units) it presumably does so because cost, and not the average cost, how can the donor and recipient use at that scale of output its marginal cost is above the $550 it receives this information to improve their sequential adjustment toward on average for those the 801st through the 900th unit of output. more efficiency? Over several years of operation under the two-part If it stops production at the upper bound of the contract, which is price contract the donor and recipient will come to understand 1,200 units (or 400 above-threshold units), it does so because its more about the costs of service delivery, including the cost of marginal cost at that scale is below the $250 it receives on average attracting additional patients. This improved understanding can for the 1,100th through the 1,200th unit of output. lead to gradual adjustment of all the dimensions of the two-part The information revealed to the donor by the recipient’s output contract. For example, the threshold amount could be gradu- under this two-part price contract is valuable—but limited. In ally reduced from year to year, to give the recipient more leeway particular, even if all the assumptions apply, the recipient has only for controlling both its output and the price it receives per unit. revealed its marginal cost for the last unit of its annual output, Or the donor and recipient could negotiate a payment schedule not its average cost for producing all that year’s output.ix So the with a steeper downward slope, which would provide the recipi- observation that the recipient stopped production at 900 units ent greater rewards for improved efficiency. The entire payment schedule could be shifted to a higher scale of production with a ix. This is in contrast to the VF mechanism discussed above, which is higher threshold and a higher upper bound each year, as scale-up more costly to administer but has the advantage of revealing the average progresses. All these possible adjustments to the design can be cost, not just the marginal cost. considered part of the sequential adjustment process intended to 65 A ppendix 2

continuously improve value for money in donor financing of these Notes health service delivery organizations. These two ideas, the VF mechanism and the two-part tariff, are 1. Babbage (1835), as quoted in Laffont and Martimort (2002), p. 11. intended only as examples to illustrate the potential improvements 2. Vogelsang and Finsinger (1979). in a donor’s value for money to be gained by exploiting the large 3. Sappington (1980). existing literature on the optimal regulation of public sector utilities. 4. Laffont and Tirole (1993).

Appendix 3 Working Group on Value for Money for Global Health Funding Agencies

Amanda Glassman is the Director of Global Health Policy and a clients include the Ford Foundation, New York City Department research fellow at the Center for Global Development. She has 20 of Health, New York State AIDS Institute, Open Society Institute, years of experience working on health and social protection policy UNAIDS, and UNDP. and programs in Latin America and elsewhere in the developing world. Prior to her current position, Glassman was the principal Joseph Brunet-Jailly is an economist. He has been a teaching assis- technical lead for health at the Inter-American Development Bank, tant at the University of Strasbourg (1962–68), Professor at Uni- where she led health economics and financing knowledge products versity of Aix-Marseille (1968–86), and then Senior researcher at and policy dialogue with member countries. From 2005 to 2007 Institut de Recherches pour le Développement (French Research Glassman was deputy director of the Global Health Financing Institute for Development Studies), living in Mali and Côte d’Ivoire Initiative at Brookings and carried out policy research on aid effec- (West Africa) from 1986 to 2004. After retiring as senior researcher tiveness and domestic financing issues in the health sector in low- (emeritus), he is now lecturer at Sciences-Po Paris and an indepen- income countries. Before joining Brookings, Glassman designed, dent consultant. His field of specialty is health economics in West supervised, and evaluated health and social protection loans at African countries. the Inter-American Development Bank and worked as a Popula- tion Reference Bureau Fellow at the U.S. Agency for International Kalipso Chalkidou is the founding director of NICE’s interna- Development. Glassman holds a MSc from the Harvard School tional program, advising governments overseas on building tech- of Public Health and a BA from Brown University, has published nical and institutional capacity for using evidence and values to on a wide range of health and social protection finance and policy inform health policy. She is interested in how local information, topics, and is editor and co-author of the books From Few to Many: local expertise, and local institutions can drive decisions on scien- A Decade of Health Insurance Expansion in Colombia (IDB and tific and legitimate health care resource allocation. She is involved Brookings 2010) and The Health of Women in Latin America and in the Chinese rural health reforms and also in national health the Caribbean (World Bank 2001). reform projects in Georgia, Turkey, the Middle East, and Latin America. She holds a doctorate on the molecular biology of prostate David Barr began working on HIV/AIDS issues in 1985. The cancer from the University of Newcastle, and has an MD (Hons) scope of David’s work has included treatment access and clinical from the University of Athens. She is an honorary lecturer at the research, addressing stigma and discrimination, HIV prevention London School of Hygiene and Tropical Medicine, a senior advi- policy, HIV funding structures, drug policy, strategic planning, sor on international policy at the Center for Medical Technology facilitation, and program evaluation. In 2003 David coordinated Policy, and visiting faculty at the Berman Institute for Bioethics, the creation of the HIV Collaborative Fund, a partnership of the at Johns Hopkins. International Treatment Preparedness Coalition and the Tides Foundation, which provides small grants for community-based Karl Dehne is the acting Chief of the UNAIDS Economics, Evalu- HIV treatment awareness, literacy, community mobilization, and ation and Program Effectiveness Division. This is a newly established advocacy projects. He was a founding member of the Treatment division that provides leadership on policies and approaches for Action Group and the ACT UP Treatment and Data Group. He achieving the High Level Meeting goals on efficiency and financing currently consults as part of the Fremont Center. His consulting of HIV responses. Previously Dehne was the Team Leader, System 68 Appendix 3

Integration, UNAIDS. He was also instrumental, together with col- cash transfers, and child health interventions. Fan has worked at leagues in PEPFAR and UNAIDS, in developing the Global Plan various nongovernmental organizations in Asia and different units for the Elimination of New Child Infection by 2015 and Keeping at Harvard University and has served as a consultant for the World Their Mothers Alive. He has worked on HIV prevention, treatment Bank and WHO. She was born and raised in Hawaii. care, and support for more than 25 years, in various positions in the WHO, UNAIDS, NGOs, and the government of Zimbabwe. Kara Hanson is Reader in Health System Economics at the London From 1998 to 2000 he was a lecturer at the University of Heidel- School of Hygiene and Tropical Medicine. She holds degrees from berg, Germany, where he led the UNAIDS Collaborating Centre McGill University, University of Cambridge, and Harvard Univer- on AIDS Strategic Planning and Operational Research. He holds sity. She has nearly 25 years of experience researching health systems an MD from the University of Heidelberg, and a PhD and MPH in low- and middle-income countries, providing policy advice and from the University of Leeds. input, and teaching health economics and supervising PhD projects. Her interests in the health sector were first developed during her Alan Fairbank is an applied research economist, lecturer, budget/ time as a health economist in the Ministry of Health, Swaziland, cost analyst, and policy advisor, who has applied his varied exper- as a fellow of the Overseas Development Institute (1988–90). At tise on issues of financing the organization and delivery of medical the end of her fellowship she returned to the United Kingdom to care and health services in diverse settings and conditions around a research position at the London School of Hygiene and Tropical the world. Extensive experience includes assignments as execu- Medicine. She completed her doctorate at the Harvard School of tive director, consultant team leader, principal analyst, program Public Health in 1999, and has worked at the London School of manager, trainer and lecturer, and project design and evaluation Hygiene and Tropical Medicine since then. She has been involved specialist. Assignments have involved design and implementation in the management of the Health Economics and Systems Analy- of health systems financing reform efforts in developed, transition, sis group for a number of years and in 2011 became Head of the and developing countries. Among consultancies for the World Bank, Department of Global Health and Development. Her research USAID, and the Inter-American Development Bank, among oth- focus is on the financing and organization of health services, and ers, he has costed public, preventive, and primary health programs, has included research on scaling up health services, the impact of estimated National Health Accounts, performed economic model- community-based health insurance, equity consequences of user ing for costing alternative health policies and scenarios, and advised fees and their removal, and expanding domestic fiscal space. She on decentralized health management, on reviewing social health has worked extensively on the role of the private sector in health insurance plans, and on resource imbalances created by increased systems, identifying the opportunities and limits of the private and targeted global health funding. In the United States he was a sector in improving the efficiency, quality, and responsiveness of Principal Analyst at the Congressional Budget Office, and later health systems. She has published widely in health economics and served as Executive Director of the Office of Health Care Access in public health journals, and was Editor of Health Policy and Plan- Connecticut. He has a PhD in economics from Boston University, ning from 2001 to 2008. and a MPA in development economics from Princeton Woodrow Wilson School of Public and International Affairs. Iain Jones is an economist for the Development Financing Team at the U.K. Department for International Development. Victoria Fan is a research fellow at the Center for Global Develop- ment. Her research focuses on the design and evaluation of health Jason Lane is a medical doctor with postgraduate qualifications in policies and programs, and since joining CGD, development assis- tropical medicine and international public health. He has worked tance for health and global health aid architecture. Fan joined for DFID for 15 years in a variety of posts including overseas in CGD after completing her doctorate at Harvard School of Public Bolivia, the Kyrgyz Republic, and Malawi. Following a secondment Health where she wrote her dissertation on health systems in India, to the EU in Brussels on health policy work, he is now senior health focused on government-sponsored health insurance, conditional adviser in DFID’s Global Funds Department. 69 A ppendix 3

Bruno Meessen is an economist. He is based at the Department Mark Rilling is Chief of the Commodities Security and Logistics of Public Health, at the Institute of Tropical Medicine, Antwerp, Division in the Office of Population and Reproductive Health, Belgium. His main domain of expertise is health sector reform, Bureau for Global Health, U.S. Agency for International Devel- health care financing, performance-based financing, social health opment. He oversees three USAID programs to improve the protection, and pro-poor strategies in low- and middle-income availability of essential medicines, diagnostics, and other health countries. His current regions of focus are Sub-Saharan Africa and supplies in developing countries over the short and long term South-East Asia. He is one of the “fathers” of the Performance-Based through improved forecasting and procurement, improved per- Financing strategy, as designer, theorizer, and evaluator of the first formance of national supply chains, and improved global coordi- experiences in Cambodia (2000–03) and Rwanda (2002–06). He nation. Prior to that, he worked in USAID’s Office of Education is the lead facilitator of the Performance-Based Financing Com- to improve and expand basic education in developing countries, munity of Practice and an editor of the blog Financing Health in especially for girls. Before joining USAID, he worked in legislative Africa (www.healthfinancingafrica.org/). affairs for a small grass-roots educational organization successfully advocating for the creation of the United States Institute Mead Over is a senior fellow at the Center for Global Development, of Peace. He graduated from Wheaton College and Cambridge researching economics of efficient, effective, and cost-effective health University with degrees in ancient languages and religious and interventions in developing countries. Much of his work since 1987, theological studies. first at the World Bank and now at CGD, is on the economics of the AIDS epidemic. After work on the economic impact of the AIDS epi- Joshua Salomon is Associate Professor of International Health demic and on cost-effective interventions, he co-authored the Bank’s at the Harvard School of Public Health. His research focuses on first comprehensive treatment of the economics of AIDS in the book, priority-setting in global health, within three main substantive Confronting AIDS: Public Priorities for a Global Epidemic (1997). areas: measurement and valuation of health outcomes; modeling His most recent book is Achieving an AIDS Transition: Preventing of patterns and trends in major causes of global mortality and dis- Infections to Sustain Treatment (2011) in which he offers options, for ease burden; and evaluation of health policies and interventions. donors, recipients, activists, and other participants in the fight against A recent emphasis in his work has been to combine techniques of HIV, to reverse the trend in the epidemic through better prevention. simulation modeling with decision analysis to inform policies on Recruited to the Bank as a health economist in 1986, Mead Over using current health interventions and priorities for developing new advanced to the position of Lead Health Economist in the Develop- technologies. Salomon received a PhD from Harvard University in ment Research Group, before leaving the Bank to join CGD in 2006. Health Policy and Decision Science.

Nancy Padian, PhD, MPH, is an internationally recognized leader Nalinee Sangrujee is an economist with the U.S. Centers for Dis- in the epidemiology and prevention of sexually transmitted infec- ease Control and Prevention. She is the head of the Health Econom- tions including HIV. She is a senior technical advisor at the Office of ics and Finance Team within the Division of Global HIV/AIDS, the Global AIDS Coordinator, a consultant for the Bill & Melinda which conducts economic research on HIV/AIDS programs to Gates Foundation, and a faculty member at the University of Cali- inform the development of HIV/AIDS policymaking. She has more fornia at Berkeley in the Department of Epidemiology. For more than 15 years of experience in conducting international policy devel- than two decades Padian has developed and directed a range of opment and economics research in the fields of HIV/AIDS, mater- research and intervention projects on HIV, sexually transmitted nal health, vaccine preventable diseases, child survival, and avian infections, and contraception in high-risk populations in the United influenza. She is a co-chair of PEPFAR’s Finance and Economics States and across the world. Her research also addresses the broader Technical Working Group. She received her PhD in Agriculture and context of economic development, empowerment, and gender-based Resource Economics from the University of California at Berkeley violence. In addition, she has expertise in the rigorous design and and a Masters in Public Health from the Tulane School of Public evaluation of public health interventions. Health and Tropical Medicine. 70 Appendix 3

Nina Schwalbe is the Managing Director, Policy and Performance David Serwadda, infectious disease epidemiologist, is a Professor at the GAVI Alliance secretariat. In this capacity she is responsible of Disease Control and the former Dean of the School of Public for policy development, market shaping, performance management, Health at Makerere University in Kampala. He received his medical and monitoring and evaluation of Alliance efforts. She has spent degree and masters in internal medicine from Makerere University more than 20 years in international health. She came to GAVI from and an MPH and honorary doctorate from Johns Hopkins Bloom- the Global Alliance for TB Drug Development where she served berg School of Public Health. He was among the first researchers to as the policy director. Prior to that she spent seven years direct- report on the presence of HIV/AIDS in Uganda (Lancet, 1985) and ing the Soros’ Foundations global public health program, which has worked continuously on HIV-related research and prevention focused on a range of critical issues, including strengthening health since the mid-1980s. He has been a senior investigator on the Rakai systems, tuberculosis, HIV/AIDS, and programs for vulnerable Program since its inception in 1988, and is the Ugandan principal populations. She also worked in maternal and child health, first with investigator on the ongoing National Institute of Health–funded the Population Council and then with AVSC International (now “Trial of Male Circumcision for HIV Prevention.” He has been Engender Health), focusing on the introduction of new programs instrumental in the scientific design and management of the proj- and technologies. She holds degrees from Harvard and Columbia ect and has provided critical liaison among the project, the local universities, is member of the Council on Foreign Relations, and community, Ugandan political and policy decision-makers, the has served on the faculty of the Department of Population and Ugandan Ministry of Health, and international agencies including Family Health at Columbia’s Mailman School of Public Health. UNAIDS, the WHO, and the World Bank.

Bernhard Schwartländer currently holds the position of Director Agnes Soucat is the Director for Human Development at the for Evidence, Strategy and Results at UNAIDS. He took up this African Development Bank. She is responsible for health, educa- position in May 2010 when he joined UNAIDS at headquarters tion, and social protection for 54 countries in Africa, including in Geneva from his assignment as the United Nations Country Sub-Saharan Africa and the Maghreb. She is currently develop- Coordinator on AIDS in Beijing. Prior to these assignments, He ing the first Human Capital Development Strategy for the AfDB held a number of senior international positions including as the along with a New Model of Education for Africa. Previously, she Director for Performance Evaluation and Policy at the Global Fund was the World Bank’s Lead Economist, and Advisor for Human to Fight AIDS, Tuberculosis and Malaria, Director of the WHO’s Development in the Africa region. She led the Health Systems HIV Department, and as the Director of Evaluation and Strategic For Outcomes program of the World Bank, a program focused on Information at UNAIDS. In 2000 he undertook a special assign- health systems strengthening and reaching the MDGs. She has ment to the World Bank to perform economic analyses on the cost more than 25 years of experience in international health covering and impact of the HIV/AIDS epidemic and the responses to it. Prior more than 30 countries in Africa, Asia, and Europe. She holds an to joining the United Nations, he was the Director of the national MD and Masters in Nutrition from the University of Nancy in AIDS program in Germany and the Director of the Division of France as well as a Master of Public Health and PhD in Health Infectious Disease Epidemiology at the Robert Koch-Institut in Economics from Johns Hopkins University. She is a public sec- Berlin, the central biomedical and infectious disease research and tor and public finance specialist and has worked extensively on reference laboratory of the federal Ministry of Health, Germany. designing and implementing community-based financing programs, He has published widely in scientific journals and books and taught poverty reduction strategies, social services decentralization, and applied epidemiology in Berlin. He brings extensive experience in performance-based financing. She was responsible for multisectoral development policies as well as infectious disease epidemiology and and results-based budget support programs covering sectors such programming at the global and national levels. He is a medical doc- as agriculture, education, health, water, and energy, and focusing tor and holds a doctorate in medical epidemiology. He received his on reaching the MDGs in several countries, particularly Rwanda. education and professional training in Germany and in the United She co-authored the Poverty Reduction Strategy Paper toolkit and States at the Centers for Disease Control and Prevention. the World Development Report 2004: Making Services Work for 71 A ppendix 3

Poor People. She was the main author of the background reports to responsible for monitoring trends in HIV/AIDS, tuberculosis, the High Level Task Force on Innovative financing. She was also a and malaria markets and assessing the public health and market member of the Global Expert Team on Health Systems of the World impact of UNITAID’s interventions. She has more than 15 years Bank. She also worked for UNAIDS, UNICEF, and the European of experience in teaching, research, and consulting in global phar- Commission. She is a French national. maceutical policy and access to medicines in developing countries. Prior to joining UNITAID, she served as Director of Pharma- Yot Teerawattananon is a medical doctor and economist, and cur- ceutical Policy at Boston University School of Medicine where rently serves as Program Leader and Senior Researcher at Health she authored numerous peer-reviewed studies on pharmaceutical Intervention and Technology Assessment Program in India. He policy at local, national, and global levels. She has considerable previously served as a director of Pong District Hospital Phayao regional expertise in the Central Asia region where she has sup- Province in northern Thailand where he developed an intense inter- ported pharmaceutical reform initiatives, including medicines est in Health Economics and Policy. Since 2000 he has worked as insurance schemes and public-private partnerships. She serves on a health system researcher at Senior Research Scholar Program in many advisory groups and technical panels, including the Global Health Financing and Policy (which later became the International Fund to Fight AIDS, Tuberculosis and Malaria’s Market Dynamics Health Policy Program), where he gained experience in project Committee, Medicines for Malaria Venture Access and Delivery evaluations at the grass-root and national levels. From 2001 to 2002 Advisory Group, WHO’s Vaccine Product, Price, and Procurement he worked as principal investigator in a project that evaluated the Steering Committee, and chair of the UNITAID Artemisinin national program for prevention of mother-to-child HIV transmis- Forecasting Steering Committee. sion in Thailand that subsequently led to the revision of the national protocol. In 2003 he received the WHO Fellowship Award to study David Wilson is the World Bank’s Global AIDS Program Direc- at University of East Anglia, England, where he completed his PhD tor and was previous the Bank’s Lead HIV Specialist. His work on in Health Economics. He has a number of publications on health HIV/AIDS spans almost 25 years. During his career he has worked economic evaluation in leading international journals, such as Value as a scientist and program manager in more than 50 countries and in Health and Pharmacoeconomics. His current research focuses on published approximately 100 scientific papers. His interests lie in health technology assessment, decision-making in health care, and HIV epidemiology, HIV prevention science, and program evalua- reproductive health including HIV/AIDS. tion. He has developed prevention programs that have been recognized as best practice by the World Bank, WHO, and DFID, and Damian Walker is a Senior Program Officer, Integrated Delivery have been influential in international HIV prevention science. In at the Bill & Melinda Gates Foundation. He is a health economist addition, he has served as technical consultant and adviser to many with more than 13 years of experience in international health eco- international agencies, including USAID, DFID, EU, AUSAID, nomics, with a focus on the economic evaluation of public health SIDA, NORAD, UNAIDS, UNICEF, and WHO. programs in low- and middle-income countries. Prior to the Gates Foundation, he was an Associate Professor in the Department of Prashant Yadav is a Senior Research Fellow and Director of International Health, Bloomberg School of Public Health, Johns Healthcare Research at the William Davidson Institute at the Hopkins University. University of Michigan. He also holds faculty appointments at the Ross School of Business and the School of Public Health at the Brenda Waning is Coordinator of Market Dynamics at UNI- University of Michigan. His research explores the functioning of TAID/WHO in Geneva. She received a bachelor’s degree in phar- health care supply chains using a combination of empirical, analyti- macy at the Massachusetts College of Pharmacy & Health Sciences, cal, and qualitative approaches. a master’s in public health degree from Boston University’s School of Public Health, and a PhD in Pharmaceutical Sciences from Gavin Yamey is Lead of the Evidence-to-Policy Initiative at the Utrecht University. At UNTAID she leads the technical team UCSF Global Health Group. He has undergraduate and master’s 72 Appendix 3

degrees in physiological sciences (medicine) from Oxford University. While at Middlebury, he wrote his honors thesis on evaluating the He did his medical training at Oxford University and University regional and gender-specific impact of the conditional cash transfer College London, qualifying as a physician in 1994. He became a program in Turkey. He also spent a semester studying French and Member of the Royal College of Physicians in 1997. He worked European Politics in Sciences Po, Paris. He is now a health financ- for five years in a variety of London teaching hospitals, followed ing analyst with the Clinton Health Access Initiative. by a fellowship in medical journalism and editing at the BMJ. In 2001 he moved to San Francisco to be the Deputy Editor of The Kate McQueston is a program coordinator to the global health Western Journal of Medicine while remaining an Assistant Editor policy team. Before joining CGD, she received her MPH from at the BMJ. In 2004 he was appointed as a founding Senior Edi- Dartmouth College, where she researched cost-effectiveness and tor of PLOS Medicine, published by the Public Library of Science, quality improvement in both clinical and global health settings. an international nonprofit organization dedicated to making the Additionally, she interned at the WHO Regional Office for Europe, biomedical literature a freely available global public good. He was where her work focused on quality improvement techniques for use the Principal Investigator on a $1.1 million grant from the Bill & in HIV prevention. Previously, she worked as program assistant Melinda Gates Foundation to support the launch of PLOS Neglected with the World Justice Project in Washington, DC. She received Tropical Diseases, the world’s first journal devoted specifically to the her B.A. from the University of Virginia. neglected infections of poverty. In 2009 he was awarded a Kaiser Family Mini-Media Fellowship in Global Health Reporting, which Rachel Silverman worked at CGD from August 2011 to 2013 as a took him to Kenya, Sudan, and Uganda. His fellowship led to a research assistant for the global health team. Prior to joining CGD series of published articles addressing the barriers to scaling up in August 2011, Silverman spent two years with the National Demo- low-cost, low-tech health tools. He was invited to discuss his series cratic Institute, where she worked on democratic development ini- in the U.K. Parliament, by the All-Party Parliamentary Group on tiatives in Kosovo and a program to increase political participation Malaria and Neglected Tropical Diseases. among Europe’s Roma minority. Previously, Silverman also served as a research assistant at Stanford University’s Center on Democracy, Center for Global Development staff Development, and the Rule of Law, where she investigated the effect of international assistance on democratic transitions. Originally Denizhan Duran worked at CGD from June 2011 to 2013 as a from the New York area, she graduated from Stanford University research assistant for the global health team. Originally from Istan- in 2009 with a B.A. in International Relations and Economics. She bul, Duran graduated cum laude from Middlebury College with is currently pursuing an MPhil in Public Health at the University a B.A. in economics and minors in political science and French. of Cambridge with support from the Gates Cambridge Trust. 73

References

Abelson, Reed, and Julie Creswell. 2012. “Hospital Chain Global Health Supply Chains.” Unpublished note for the Cen- Inquiry Cited Unnecessary Cardiac Work.” The New York ter for Global Development, Washington, DC, April. Times, August 6. www.nytimes.com/2012/08/07/business/ Birdsall, Nancy, and William D. Savedoff. 2010.Cash on Delivery: hospital-chain-internal-reports-found-dubious-cardiac-work. A New Approach to Foreign Aid. Washington, DC: Center for html?hp&_r=0. Global Development. amFAR and Johns Hopkins Bloomberg School of Public Health. Bridge, Jamie, Benjamin M. Hunter, Rifat Atun, and Jeffrey V. 2013. Achieving an AIDS-Free Generation for Gay Men and Lazarus. 2012. “Global Fund Investments in Harm Reduction Other MSM in Southern Africa. New York. from 2002 to 2009.” International Journal of Drug Policy 23 Armstrong, Mark, Simon Cowan, and John Vickers. 1994. Regu- (4): 279–85. doi:10.1016/j.drugpo.2012.01.013. latory Reform: Economic Analysis and British Experience. Cam- Bump, Jesse B., Victoria Y. Fan, Heather E. Lanthorn, and Elif bridge, MA: MIT Press. N. Yavuz. “In the Global Fund’s Court: Experimentation, Babbage, Charles. 1835. On the Economy of Machinery and Manu- Evaluation, and the AMFm.” The Lancet 380 (9858): 1989. factures. London: Charles Knight. doi:10.1016/S0140-6736(12)62123-0. www.thelancet.com/ Bahl, Kanika, and Pooja Shaw. 2012. Expanding Access to LLINs: journals/lancet/article/PIIS0140-6736(12)62123-0/fulltext. A Global Market Dynamics Approach. Results for Development Cashin, Cheryl, and Petra Vergeer. 2013. Verification in Results- Institute: Washington, DC. Based Financing (RBF): The Case of the United Kingdom. Wash- Balm, Gerald J. 1996. “Benchmarking and Gap Analysis: What Is ington, DC: World Bank. https://openknowledge.worldbank. the Next Milestone?” Benchmarking for Quality Management org/handle/10986/13567. & Technology 3 (4): 28–33. Center for Global Development. 2006. Challenges and Opportu- Barnighausen Till, David E. Bloom, and Sala Humair. 2013. “A nities for the New Executive Director of the Global Fund: Seven Model for Optimizing Intervention Mix for HIV.” Interim Essential Tasks: Report of The Global Fund Working Group. report to assist The Center for Global Development’s Working Washington, DC. www.cgdev.org/files/10948_file_CGD_ Group on Value for Money for Global Health Funding Agen- Report_sprd_ENG.pdf. cies, April 17. Croke, Kevin, Andrew Dabalen, Gabriel Demombybes, Marcelo Basinga, Paulin, Paul J. Gertler, Agnes Binagwaho, Agnes L. B. Giugale, and Johannes Hoogeveen. 2012. “Collecting High Soucat, Jennifer R. Sturdy, and Christel M. J. Vermeersch. Frequency Panel Data in Africa Using Mobile Phone Inter- 2010. “Paying Primary Health Care Centers for Performance views.” Policy Research Working Paper 6097, World Bank, in Rwanda.” Policy Research Working Paper 5190, World Washington, DC. Bank, Washington, DC. DFID (U.K. Department for International Development). 2012. Bendavid, Eran, Philip Grant, Annie Talbot, Douglas K. Owens, “Annual Review (Multilateral Orginisation Core Funding) for and Andrew Zolopa. 2011. “Cost-effectiveness of Antiretrovi- Global Fund for AIDS, Tuberculosis and Malaria.” London, ral Regimens in the World Health Organization’s Treatment October 2012. Guidelines: A South African Analysis.” AIDS 25 (2): 211–20. Duong, Yen T., Maofeng Qiu, Anindya K. De, Keisha Jackson, Berenguer, Gemma, Ananth Iyer, and Prashant Yadav. 2013. “A Trudy Dobbs, Andrea A. Kim, John N. Nkengasong, and Benchmarking Methodology for Sustaining Efficiencies of Bharat S. Parekh. 2012. “Detection of Recent HIV-1 Infection 74 References

Using a New Limiting-Antigen Avidity Assay: Potential for Reduce Risks. Washington, DC, April. www.gao.gov/ HIV-1 Incidence Estimates and Avidity Maturation Studies.” assets/660/654247.pdf. PLoS ONE 7 (3): e33328. doi:10.1371/journal.pone.0033328. Garmaise, David. 2012. “New PEPFAR Blueprint Outlines Dybul, Mark. 2013. “The Big Push to Defeat AIDS, TB and Areas of Collaboration with the Global Fund.” Global Malaria.” The Huffington Post, January 21, 2013. Accessed Fund Observer Newsletter, Issue 205, December 5, 2012. August 8, 2013, at www.huffingtonpost.com/amb-mark-dybul/ Accessed July 18, 2013, at www.aidspan.org/gfo_article/ the-big-push-to-defeat-ai_b_2516889.html. new-pepfar-blueprint-outlines-areas-collaboration-global-fund. Edejer, Tessa Tan-Torres, Rob Baltussen, Taghreed Adam, Ray- Glassman, Amanda, and Rachel Silverman. Forthcoming. “Mea- mond Hutubessy, Arnab Acharya, David B. Evans, and Chris- surement and Evaluation at the Global Fund.” Center for topher J.L. Murray, ed. 2003. Making Choices in Health: WHO Global Development, Washington, DC. Guide to Cost-effectiveness Analysis. Geneva: World Health Hallett, Timothy. 2013. “Historic Gains In the Fight Against Organization. www.who.int/choice/publications/p_2003_ HIV Are Within Reach and Within Budget.” The Global Fund generalised_cea.pdf. blog, May 27, 2013. www.theglobalfund.org/en/blog/32206/. Eichler, Rena, Ruth Levine, and the Performance-Based Incen- Hallett, Timothy B., and Mead Over. 2010. “How to Pay ‘Cash- tives Working Group. 2009. Performance Incentives for Global on-Delivery’ for HIV Infections Averted: Two Measurements Health: Potential and Pitfalls. Washington, DC: Center for Approaches and Ten Payout Functions.” Working Paper 210, Global Development. Center for Global Development, Washington, DC. Fan, Victoria, Denizhan Duran, Rachel Silverman, and Amanda Hamel, Mary, Kubaje Adazu, David Obuor, Maquins Sewe, John Glassman. 2013. “Performance-Based Financing at the Global Vulule, John M. Williamson, Laurence Slutsker, Daniel R. Fund to Fight AIDS, Tuberculosis and Malaria: an analysis Feikin, and Kayla F. Laserson. 2011. “Reversal in Reductions of grant ratings and funding, 2003–12.” The Lancet Global in Child Mortality in Western Kenya, 2003–2009.” American Health (1): e161–68. Journal of Tropical Medicine and Hygiene 85 (4): 597–605. ———. Forthcoming. “Getting Value for Money in the Global Handa, Sudhanshu, Carolyn Halpern, Audrey Pettifor, and Har- Fund’s Portfolio of HIV/AIDS Interventions.” CGD Policy sha Thirumurthy. 2012. “Impact of the Kenya Cash Trans- Paper, Center for Global Development, Washington, DC. fer for Orphans and Vulnerable Children Program on HIV Fan, Victoria, and Amanda Glassman. Forthcoming. “Value for Risk Behavior.” Presentation at IAC 2012, Washington, Money: A Framework for Global Health Funders.” CGD Pol- DC. Accessed April 29, 2013, at www.cpc.unc.edu/projects/ icy Paper, Center for Global Development, Washington, DC. transfer/countries/kenya/kenya-ct-ovc-aids-2012-v2.pdf. Fan, Victoria, and Rachel Silverman. 2013. “Experimentation for Hansen, Peter, Par Eriksson, and Laura Stormont. 2011. “Per- Better Health: Lessons from the US for Global Health.” Blog formance Based Funding.” Report to the GAVI Alliance post, May 6, 2013. Accessed June 20, 2013, at www.cgdev.org/ Board, November 16–17, 2011. www.gavialliance.org/about/ blog/experimentation-better-health-lessons-us-global-health. governance/gavi-board/minutes/2011/16-nov/minutes/ Fitzpatrick, Christopher, and Katherine Floyd. 2012. “A System- performance-based-funding/. atic Review of the Cost and Cost Effectiveness of Treatment HEALTHQUAL International. 2012. “Using Early Warning Indi- for Multidrug-Resistant Tuberculosis.” Pharmacoeconomics 30 cators to Improve HIV care & treatment Services.” Kampala, (1): 63-80. Uganda. www.healthqual.org/sites/default/files/Zimbabwe Forsythe, Steven, John Stover, and Lori Bollinger. 2009. %20Country%20Presentation.pdf. “The Past, Present and Future of HIV, AIDS and Heinrich, Carolyn, John Hoddinott, Michael Samson, Kenneth Resource Allocation.” BMC Public Health 9 (Suppl 1): S4. MacQuene, Ingrid Niekerk, and Bryant Renaud. 2012. The doi:10.1186/1471-2458-9-S1-S4. South African Child Support Grant Impact Assessment: Evidence GAO (U.S. Government Accountability Office). 2013.Drug from a Survey of Children, Adolescents, and Their Households. Supply Chains Are Stronger, but More Steps Are Needed to Pretoria: UNICEF South Africa. 75 R eferences

Incidence Assay Critical Path Working Group. 2011. “More Allocation Models.” BMC Public Health 9 (Suppl 1): S8. and Better Information to Tackle HIV Epidemics: Towards doi:10.1186/1471-2458-9-S1-S8. www.ncbi.nlm.nih.gov/pmc/ Improved HIV Incidence Assays.” PLoS Med 8 (6): e1001045. articles/PMC2779510/. doi:10.1371/journal.pmed.1001045. www.plosmedicine.org/ Lemière, Christophe. 2012. “Result-Based Financing (RBF) article/info%3Adoi%2F10.1371%2Fjournal.pmed.1001045. in Benin and Senegal: Some (First) Lessons Learnt.” www. International Initiative for Impact Evaluation. 2008. “Founding rbfhealth.org/rbfhealth/system/files/event_doc/Lessons%20 Document for Establishing the International Initiative for RBF%20Benin%20Senegal.pdf. Impact Evaluation.” Washington, DC. www.3ieimpact.org/ Lim, Stephen S., David B. Stein, Alexandra Charrow, and Christo- media/filer/2012/05/17/3iefoundingdocument30june2008. pher J. L. Murray. 2008. “Tracking Progress towards Universal pdf. Childhood Immunisation and the Impact of Global Initiatives: A Isenman, Paul, and Alexander Shakow. 2010. “Donor Schizophre- Systematic Analysis of Three-dose Diphtheria, Tetanus, and Per- nia and Aid Effectiveness: The Role of Global Funds.” IDS tussis Immunisation Coverage.” The Lancet 372 (9655): 2031–46. Practice Paper 5, April 2010. Institute of Development Stud- Loevinsohn, Benjamin. 2008. Contracting for Health Services in ies, Brighton, UK. Developing Countries: A Toolkit. Washington, DC: World Bank. Katz, Itamar, Douglas Glandon, Wendy Wong, Brima Kargbo, http://siteresources.worldbank.org/INTHSD/Resources/ Regina Ombam, Shanti Singh, Leslie Ramsammy, Anta topics/415176-1216235459918/ContractingEbookAppC.pdf. Tal-Dia, Ibrahima Seck, and John S. Osika. 2013. “Lessons London School of Hygiene and Tropical Medicine. 2012. “Ben- Learned from Stakeholder-driven Sustainability Analysis of efits and Costs of Integrating Sexual-Reproductive Health and Six National HIV Programmes.” Health Policy and Planning. HIV Services in Kenya and Swaziland.” http://clinicaltrials. doi:10.1093/heapol/czt024 http://heapol.oxfordjournals.org/ gov/show/NCT01694862. content/early/2013/04/23/heapol.czt024.abstract. Marseille, Elliot, Mark J. Giganti, Albert Mwango, Angela Kinyanjui, Karanja. 2012. “Africa: The Global Fund Should Chisembele-Taylor, Lloyd Mulenga, Mead Over, James G. Actively Promote Male Circumcision.” Global Fund Observer, Kahn, and Jeffrey S. A. Stringer. 2012. “Taking ART to Scale: Nairobi, July 12. Accessed June 20, 2013, at http://allafrica. Determinants of the Cost and Cost-Effectiveness of Antiretro com/stories/201207130100.html All Africa. viral Therapy in 45 Clinical Sites in Zambia.”PLoS ONE 7 Kouzouian, Oskian, and Andrew D. Forsyth. 2013. “HIV Pre- (12): e51993. doi:10.1371/journal.pone.0051993. vention Resource Allocation Modeling Pilot (HIV RAMP).” Martins, João S., Anthony B Zwi, and Paul M Kelly. 2012. AIDS.gov, April 10, 2013. Accessed June 20, 2013, at http:// “Did the First Global Fund Grant (2003–2006) Contrib- blog.aids.gov/2013/04/hiv-prevention-resource-allocation ute to Malaria Control and Health System Strengthening in -modeling-pilot-hiv-ramp.html. Timor-Leste?” Malaria Journal 11: 237. www.malariajournal. Kraiselburd, Santiago, and Prashant Yadav. 2012. “Supply Chains com/content/11/1/237#B17. and Global Health: An Imperative for Bringing Operations Matsoso, Malebona Precious, and Robert Fryatt. 2013. “National Management Scholarship into Action.” Production and Opera- Health Insurance: The First 18 Months.” InSouth African tions Management 0 (0): 1–5. http://wdi.umich.edu/research/ Health Review 2012/13, ed. Ashine Padarath and Rene Eng- healthcare/resources/poms1315.pdf. lish. Durban, South Africa: Health Systems Trust. www.hst. Laffont, Jean-Jacques, and David Martimort. 2002.The Theory of org.za/sites/default/files/Chapter2_National%20Health%20 Incentives: The Principal-Agent Model. Princeton, NJ: Prince- Insurance-The%20first%2018%20months.pdf. ton University Press. Maude, Richard J., Sompob Saralamba, Adrian Lewis, Dean Sher- Laffont, Jean-Jacques, and Jean Tirole. 1993.A Theory of Incentives wood, Nicholas J. White, Arjen M. Dondorp, and Lisa J. White. in Procurement and Regulation. Cambridge, MA: MIT Press. 2011. “Online Malaria Elimination Modeling Project.” www. Lasry, Arielle, Anke Richter, and Frithjof Lutscher. 2009. “Rec- tropmedres.ac/departments-units/mathematical-modelling/ ommendations for Increasing the Use of HIV/AIDS Resource malaria-elimination. 76 References

Meyer-Rath, Gesine, and Mead Over. 2012. “HIV Treatment Porcher, Damien, and Dominique Kerouedan. 2011. “Health as Prevention: Modelling the Cost of Antiretroviral Treat- Financing: Evolving Context, Evolving Methods: A Review on ment—State of the Art and Future Directions.” PLoS Med 9 Innovative Financing for Health.” French Ministry of Foreign (7): e1001247. doi:10.1371/journal.pmed.1001247. and European Affairs, Paris. http://leadinggroup.org/IMG/ National Institute for Health and Care Excellence. 2013. “Guide pdf/Review_on_innovative_financing_for_health_final.pdf. to the methods of technology appraisal.” London. Accessed Resch, Stephen C., Joshua A. Salomon, Megan Murray, and Milton June 20, 2013, at www.nice.org.uk/aboutnice/howwework/ C. Weinstein. 2006. “Cost-effectiveness of Treating Multidrug- devnicetech/guidetothemethodsoftechnologyappraisal.jsp. resistant Tuberculosis.” PLOS Med 3 (7): e241. doi:10.1371/ National Research Council. 2013. Evaluation of PEPFAR. Wash- journal.pmed.0030241. ington, DC: The National Academies Press. Results for Development Institute. 2011. Market Dynamics Study: Ord, Toby. 2013. “The Moral Imperative towards Cost-effective- Phase II Report for the Global Fund Market Dynamics Commit- ness in Global Health.” Center for Global Development, Wash- tee (MDC). Washington, DC. ington, DC. www.cgdev.org/publication/moral-imperative Rodriguez-Garcia, Rosalia, Rene Bonnel, David Wilson, N’Della -toward-cost-effectiveness-global-health, accessed August 5, N’Jie. 2013. Investing in Communities Achieves Results: Find- 2013. ings from an Evaluation of Community Responses to HIV and Over, Mead. 2010. “Using Incentives to Prevent HIV Infections.” AIDS. Washington, DC: World Bank. Center for Global Development Essay, Washington, DC. www. Roll Back Malaria. 2011. “Report of the Taskforce on Innova- cgdev.org/files/1424161_file_AIDS_Transition_Essay_2_ tive Financing (Pre-read for decision).” RBM/BOM/2011/ clean_final.pdf. SUB.3, November 3, 2011. www.fightingmalaria.org/pdfs/ ——— . 2 011. Achieving an AIDS Transition: Preventing Infections taskforceinnovativefinancingreport.pdf. to Sustain Treatment. Washington, DC: Center for Global Sappington, David. 1980. “Strategic Firm Behavior under a Development. Dynamic Regulatory Adjustment Process.” The Bell Journal of Over, Mead, Matthew T. Schneider, and Tejaswi Velayudhan. Economics 11: 360–72. 2013. “Abstract: Explaining the Variation in On-site AIDS Sarley, David, Linda Allain, and Anup Akkihal. Estimating the Treatment Costs: The MATCH Study of 161 Facilities Global In-country Supply Chain Costs of Meeting the MDGs by from Five Countries.” The Lancet 381: S106. doi:10.1016/ 2015. Arlington, VA: U.S. Agency for International Develop- S0140-6736(13)61360-4. ment. http://pdf.usaid.gov/pdf_docs/PNADP080.pdf. Padian, Nancy S., Sandra I. McCoy, Salim S. Abdool Karim, Nina Stop TB Partnership. 2010. Falling Short: Ensuring Access to Simple, Hasen, Julia Kim, Michael Bartos, Elly Katabira, Stefano M. Safe, and Effective First-Line Medicines for Tuberculosis. Geneva. Bertozzi, Bernhard Schwartländer, and Myron S. Cohen. 2011. Accessed July 18, 2013, at www.stoptb.org/assets/documents/ “HIV Prevention Transformed: The New Prevention Research resources/publications/technical/Falling_Short.pdf. Agenda.” The Lancet 378 (9787): 269–78. Stover, John, Eline L. Korenromp, Matthew Blakley, Ryuichi PEPFAR (The U.S. President‘s Emergency Plan for AIDS Relief). Komatsu, Kirsi Viisainen, Lori Bollinger, and Rifat Atun. 2012a. “Guidance for Orphans and Vulnerable Children Pro- 2011. “Long-Term Costs and Health Impact of Continued Global gramming.” Washington, DC. Fund Support for Antiretroviral Therapy.” PLoS ONE 6 (6): ———. 2012b. “Report on Costs of Treatment in the Presi- e21048. doi:10.1371/journal.pone.0021048. www.plosone. dent’s Emergency Plan for AIDS Relief.” www.pepfar.gov/ org/article/info:doi/10.1371/journal.pone.0021048. documents/organization/188493.pdf, accessed August 5, 2013. Task Force on Innovative Financing Resource Mobilization Sub- ——— . 2 012 c . Report on Pilot Expenditure Analysis of PEPFAR Committee. 2011. “Development of a Business Plan Malaria Programs in Six Countries. Washington, DC. www.pepfar.gov/ Bond.” www.fightingmalaria.org/pdfs/malariabondbusinessplan. documents/organization/195700.pdf. pdf. 77 R eferences

Technical Evaluation Reference Group. 2009. The Five-Year Eval- ———. 2011i. “Report of the Market Dynamics and Commodi- uation of the Global Fund to Fight AIDS, Tuberculosis, and ties Ad-hoc Committee the Global Fund.” Geneva, May 2011. Malaria: Synthesis of Study Areas 1, 2 and 3. Geneva. www. ———. 2011j. “Terms of Reference for the Technical Review theglobalfund.org/en/terg/evaluations/. Panel.” Geneva, June 2011. www.theglobalfund.org/ The Global Fund. 2001. “The Framework Document of the Global documents/trp/TRP_TechnicalReviewPanel_ToR_en/. Fund to Fight AIDS, Tuberculosis and Malaria.” Geneva. ———. 2011k. “The Global Fund Strategy 2012–2016: Invest- ———. 2007a. “Guidance for Completion of the Enhanced Finan- ing for Impact.” Geneva. Accessed June 20, 2013, at www. cial Reporting Template.” Geneva. theglobalfund.org/en/about/strategy/. ———. 2007b. “The Global Fund Strategy in Relation to Sexual ——— . 2 011 l . Turning the Page from Emergency to Sustainability: Orientation and Gender Identities.” Geneva. The Final Report of the High-Level Independent Review Panel ———. 2008. “Technical Evaluation Reference Group Five Year on Fiduciary Controls and Oversight Mechanisms of the Global Evaluation Study Area 2 Results: Global Fund Partner Envi- Fund to Fight AIDS, Tuberculosis and Malaria. Geneva. www. ronment.” Evaluation Brief 7, Geneva. www.theglobalfund.org/ theglobalfund.org/en/highlevelpanel/report/. en/terg/evaluations/5year/sa2/. ———. 2012a. “Key Performance Indicators: End-Year Results for ——— . 2 010 a . Financial and Health Impacts of Continued Support 2011.” Geneva. to the Three Diseases: Long-term Estimates. Geneva. ——— . 2 012 b . S t rategic Investments for Impact: Global Fund ——— . 2 010 b . Improving Value for Money in Global Fund-Sup- Results Report 2012. Geneva. ported Programs. Geneva. ———. 2012c. “Strategy, 2012–2016: Investing for Impact.” ———. 2011a. “Audit of Global Fund grants to Population Ser- Geneva. Accessed June 20, 2013, at http://theglobalfund.org/ vices International Madagascar.” Audit and Investigation Documents/core/strategies/Core_GlobalFund_Strategy_en reports issued by the Global Fund’s Office of the Inspector -4294929871/. General, Geneva, November 1, 2011. ———. 2012d. “Technical Evaluation Reference Group (TERG) ———. 2011b. “Continuity of Services.” Operational Policy Note, Terms of Reference, Membership and Procedures.” Geneva, July 11, 2011. Geneva. April 2012. ———. 2011c. “Funding Model.” In Governance Handbook, Sec- ——— . 2 013 a . Global Fund Operational Policy Manual. Geneva. tion 3. Geneva. ———. 2013b. “Progress Update/Disbursement Request ———. 2011d. “Global Fund Market-Shaping Strategy and Market- Guidelines.” Geneva. Accessed June 20, 2013, at www. Shaping Interventions for ARVs.” Decision Point GF/B23/ theglobalfund.org/documents/core/guidelines/Core_ DP21, Twenty-Third Board Meeting, Geneva, May 11–12, 2011. PUDR_Guidelines_en/. ———. 2011e. “Global Fund Signs International Aid Trans- ———. 2013c. The Global Fund’s New Funding Model. Geneva. parency Initiative.” Press Release, June 23, 2011. Accessed ———. 2013d. “Transition Manual for the New Funding Model June 20, 2013, at www.theglobalfund.org/en/mediacen- of the Global Fund.” Geneva. www.theglobalfund.org/ ter/announcements/2011-06-23_Global_Fund_signs_ Documents/core/newfundingmodel/Core_NewFunding International_Aid_Transparency_Initiative/. ModelTransition_Manual_en-4294930165/. ———. 2011f. “LFA Guidelines for On-site Data Verification Tren, Richard, Kimberly Hess, and Roger Bate. 2009. “Drug Pro- (OSDV) and Rapid Services Quality Assessment (RSQA) curement, the Global Fund and Misguided Competition Poli- Implementation.” Geneva. cies.” Malaria Journal 8: 305. doi:10.1186/1475-2875-8-305. ———. 2011g. “LFA Manual.” Geneva. UNAIDS (Joint United Nations Programme on HIV/AIDS). ———. 2011h. “Monitoring and Evaluation Toolkit.” Geneva. 2011. Global AIDS Response Progress Reporting 2012. Guide- Accessed June 20, 2013, at www.theglobalfund.org/en/me/ lines: Construction of Core Indicators for Monitoring the 2011 documents/toolkit/. Political Declaration on HIV/AIDS. Geneva. 78 References

USAID (U.S. Agency for International Development). 2013. Public Health Approach. Geneva. http://whqlibdoc.who.int/ “Commercial Sector Performance-Based Financing Offers publications/2011/9789241501750_eng.pdf. Lessons for Public Health Supply Chains in Developing Coun- ——— . 2 012 a . Global Tuberculosis Report 2012. Geneva. tries.” Washington, DC. http://deliver.jsi.com/dlvr_content/ ——— . 2 012 b . Prevention and Treatment of HIV and Other Sexu- resources/allpubs/guidelines/PBFComSec.pdf. ally Transmitted Infections for Sex Workers in Low- and Mid- Vogelsang, Ingo, and Jorg Finsinger. 1979. “A Regulatory Adjust- dle-income Countries: Recommendations for a Public Health ment Process for Optimal Pricing by Multiproduct Monopoly Approach. Geneva. http://apps.who.int/iris/bitstream/10665/ Firms.” Bell Journal of Economics 10 (1): 151–71. 77745/1/9789241504744_eng.pdf. Vujicic, Marko, Stephanie E. Weber, Irina A. Nikolic, Rifat Atun, ———. 2012c. “Resource Mobilization Strategy for the 2012– and Ranjana Kumar. 2012. “An analysis of GAVI, the Global 2015 Phase of Implementation of the Global Malaria Action Fund, and World Bank Support for Human Resources in Devel- Plan (May 2012).” Geneva. www.rbm.who.int/gmap/GMAP oping Countries.” Health Policy and Planning 27 (8): 649–57. FinancialStrategy.pdf. Wafula, Frank, Charles Marwa, and David McCoy. 2013. “Global ———. 2013. “Prequalification of Medicines by WHO.” Geneva. Fund Principal Recipient Survey: An Assessment of Opinions www.who.int/mediacentre/factsheets/fs278/en/. and Experiences of Principal Recipients.” Aidspan Working Williams, Alan. 1997. “Intergenerational Equity: An Explora- Paper 01/2013, Aidspan, Nairobi. tion of the “Fair Innings” argument.” Health Economics 6 (2): Weissman, Joel S., Michael Bailit, Guy D’Andrea, and Meredith B. 117–32. Rosenthal. 2012. “The Design and Application of Shared Sav- World Bank. 2013a. Information to Improve Value for Money ings Programs: Lessons from Early Adopters.” Health Affairs in Health. Africa Health Forum: Finance and Capacity for (Millwood) 31 (9): 1959–68. Results. Washington, DC. Accessed April 22, 2013 at http:// WHO (World Health Organization). 2007. “Insecticide-treated siteresources.worldbank.org/INTAFRICA/Resources/AHF Mosquito Nets: A WHO Position Statement.” www.who.int/ -information-to-improve-value-for-money-in-health.pdf. malaria/publications/atoz/itnspospaperfinal.pdf. ———. 2013b. Using Results-Based Financing to Achieve Mater- ———. 2008. “Briefing Sheet 3: The Challenge of Coordination.” nal and Child Health—Progress Report 2013. Washing- Geneva. www.who.int/alliance-hpsr/researchsynthesis/Alliance ton, DC. Accessed August 5, 2013, at www.rbfhealth.org/ HPSR_GHIN_ChallengeCoordination_BS3.pdf. progressreport2013. ———. 2011a. “Guidelines for the Programmatic Management Yadav, Prashant, Helen Lega Tata, and Magali Babaley. 2011. of Drug-resistant Tuberculosis: 2011 Update.” Geneva. http:// World Medicines Report 2011: Storage and Supply Chain Man- whqlibdoc.who.int/publications/2011/9789241501583_eng. agement. Geneva: World Health Organization. pdf. Zanzibar Malaria Control Programme. 2009. Malaria Elimina- ——— . 2 011b . Prevention and Treatment of HIV and Other tion in Zanzibar: A Feasibility Assessment. www.soperstrategies. Sexually Transmitted Infections among Men who Have Sex com/countries/pemba/tanzania-library/files/Elimination with Men and Transgender People: Recommendations for a Zanzibar.pdf. For an electronic version of this book, visit www.cgdev.org A Report of the Center for Global Development Development Global for A Report Center the of Health Global in Money for Value on Working Group Over Victoria with Mead and Fan chair, Amanda Glassman, MoreHealth for the Money and Its Partners Fund Global the for Work to PuttingIncentives

More Health for the Money Putting Incentives to Work for the Global Fund and Its Partners Center for Global Development ” ISBN 978-1-933286-80-8