Complete Issue

Dedicated to what works in global health programs

GLOBAL HEALTH: SCIENCE AND PRACTICE

2016 Volume 4 Number 3 www.ghspjournal.org

Editor-in-Chief James D. Shelton, MD, MPH, Johns Hopkins Center for Communication Programs

Deputy Editor-in-Chief Stephen Hodgins, MD, DrPH, Senior Technical Advisor, Save the Children, Newborn Health

Associate Editors Matthew Barnhart, MD, MPH, Senior Science Advisor, United States Agency for International Development (USAID), Bureau for Global Health Cara J. Chrisman, PhD, Biomedical Research Advisor, USAID, Bureau for Global Health Digital Health: Margaret d’Adamo, MLS, MS, Knowledge Management/Information Technology Advisor, USAID, Bureau for Global Health Malaria: Michael Macdonald, ScD, Consultant, World Health Organization, Vector Control Unit, Global Malaria Programme Maternal Health: France Donnay, MD, MPH, Consultant Nutrition: Bruce Cogill, PhD, MS, Consultant

Managing Editors Natalie Culbertson, Johns Hopkins Center for Communication Programs Ruwaida Salem, MPH, Johns Hopkins Center for Communication Programs

Editorial Board

Al Bartlett, Save the Children, USA Vinand Nantulya, Uganda AIDS Commission, Uganda Zulfiqar Bhutta, Aga Khan University, Pakistan Emmanuel (Dipo) Otolorin, Jhpiego, Nigeria Kathryn Church, London School of Hygiene and Tropical James Phillips, Columbia University, USA Medicine, United Kingdom Yogesh Rajkotia, Institute for Collaborative Scott Dowell, Centers for Disease Control and Development, USA Prevention, USA Suneeta Singh, Amaltas, India Marelize Görgens, World Bank, USA David Sleet, CDC, USA Lennie Kamwendo, White Ribbon Alliance for Safe John Stanback, FHI 360, USA Motherhood, Malawi Lesley Stone, USAID, USA Jemilah Mahmood, Malaysian Medical Relief Society, Douglas Storey, Johns Hopkins Center for Malaysia Communication Programs, USA

Global Health: Science and Practice (ISSN: 2169-575X) is a no-fee, open-access, peer-reviewed journal published online at www.ghspjournal.org. It is published quarterly by the Johns Hopkins Center for Communication Programs, 111 Market Place, Suite 310, Baltimore, MD 21202. The journal is made possible by the support of the American People through the United States Agency for International Development (USAID). The Knowledge for Health (K4Health) Project is supported by USAID's Office of Population and Reproductive Health, Bureau for Global Health, under Cooperative Agreement #AID-OAA-A-13-00068 with the Johns Hopkins University. GHSP is editorially independent and does not necessarily represent the views or positions of USAID, the United States Government, or the Johns Hopkins University.

Global Health: Science and Practice is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of this license, visit: http://creativecommons.org/licenses/by/3.0/.

For further information, please contact the editors at [email protected]. Table of Contents September 2016 | Volume 4 | Number 3

EDITORIALS

Effective LARC Providers: Moving Beyond Training (Republication)

Effective and productive providers are the key to successful provision of long-acting reversible contraceptives (LARCs). But LARCs demand more of providers than short-acting resupply methods. In addition to sound training, key elements to developing highly productive providers of LARCs include a thorough understanding of the service delivery system context; selecting providers with the most potential, especially from mid-level cadres; strong mentoring and supportive supervision; and attention to the supply chain and to demand-side support.

James D Shelton, Anne E Burke

Glob Health Sci Pract. 2016;4(3):356–358 http://dx.doi.org/10.9745/GHSP-D-16-00258

VIEWPOINTS

Moving Medicine, Moving Minds: Helping Developing Countries Overcome Barriers to Outsourcing Health Commodity Distribution to Boost Supply Chain Performance and Strengthen Health Systems

Senegal and other developing countries are improving access to health commodities by outsourcing supply chain logistics to private providers. To achieve broader, lasting reform, we must support further adoption of the outsourced model; assist country-led cost- benefit analyses; and help governments build capacity to manage contracts and overcome other barriers.

Priya Agrawal, Iain Barton, Roberto Dal Bianco, Dana Hovig, David Sarley, Prashant Yadav

Glob Health Sci Pract. 2016;4(3):359–365 http://dx.doi.org/10.9745/GHSP-D-16-00130

COMMENTARIES

Accessible Contraceptive Implant Removal Services: An Essential Element of Quality Service Delivery and Scale-Up

Use of contraceptive implants has surged in recent years, yet emerging data show a deficit of service delivery capacity and coverage for implant removals. The number of projected removals needed in the 69 FP2020 focus countries in 2018 (4.9–5.8 million) is more than twice that estimated for 2015 (2.2 million). We must proactively plan and execute high-quality implant removal services in order to fulfill the exceptional promise of implants in meeting client needs and advancing toward FP2020 goals.

Megan Christofield, Maryjane Lacoste

Glob Health Sci Pract. 2016;4(3):366–372 http://dx.doi.org/10.9745/GHSP-D-16-00096

Global Health: Science and Practice 2016 | Volume 4 | Number 3 Table of Contents www.ghspjournal.org

Using Qualitative Methods to Validate and Contextualize Quantitative Findings: A Case Study of Research on Sexual Behavior and Gender-Based Violence Among Young Swazi Women

Nesting qualitative data collection methods within quantitative studies improves results by assessing validity and providing depth and context. Using data from 3 sources from Swaziland, we triangulate qualitative and quantitative findings to highlight how different methodologies produce discrepant data regarding risky sexual behaviors among young women. We found that women reported similar numbers of lifetime sex partners in all sources, but the proportion reporting multiple and concurrent partnerships was several times higher in qualitative interviews. In addition, qualitative data can provide deeper understanding of how participants, such as those experiencing gender-based violence, understood the experiences behind the quantitative statistics.

Allison Ruark, Rebecca Fielding-Miller

Glob Health Sci Pract. 2016;4(3):373–383 http://dx.doi.org/10.9745/GHSP-D-16-00062

Vouchers: A Hot Ticket for Reaching the Poor and Other Special Groups With Voluntary Family Planning Services

Vouchers can be a highly effective tool to increase access to and use of family planning and reproductive health services, especially for special populations including the poor, youth, and postpartum women. Voucher programs need to include social and behavior change communication with clients and quality assurance for providers, whether in the private or public sector. In the longer term, voucher programs can strengthen health systems capacity and provide a pathway to strategic purchasing such as insurance or contracting.

Elaine P Menotti, Marguerite Farrell

Glob Health Sci Pract. 2016;4(3):384–393 http://dx.doi.org/10.9745/GHSP-D-16-00084

ORIGINAL ARTICLES

Successful Implementation of a Multicountry Clinical Surveillance and Data Collection System for Ebola Virus Disease in West Africa: Findings and Lessons Learned

Despite resource and logistical constraints, International Medical Corps cared for thousands at 5 Ebola treatment units in Liberia and Sierra Leone between 2014 and 2015 while collecting hundreds of data points on each patient. To facilitate data collection and global reporting in future humanitarian responses, standardized data forms and databases, with clear definitions of clinical and epidemiological variables, should be developed and adopted by the international community.

Reshma Roshania, Michaela Mallow, Nelson Dunbar, David Mansary, Pranav Shetty, Taralyn Lyon, Kacey Pham, Matthew Abad, Erin Shedd, Anh-Minh A Tran, Sarah Cundy, Adam C Levine

Glob Health Sci Pract. 2016;4(3):394–409 http://dx.doi.org/10.9745/GHSP-D-16-00186

Global Health: Science and Practice 2016 | Volume 4 | Number 3 Table of Contents www.ghspjournal.org

Safety and Acceptability of Community-Based Distribution of Injectable Contraceptives: A Pilot Project in Mozambique

Trained community health workers, including traditional birth attendants (TBAs), safely and effectively administered injectables in northern Mozambique; two-thirds of the women choosing injectables had never used contraception before. Including TBAs in the Ministry of Health’s recent task sharing strategy can improve rural women’s access to injectables and help meet women’s demand for contraception.

Ana Jacinto, Mahomed Riaz Mobaracaly, Momade Bay Usta´b, Cassimo Bique, Cassandra Blazer, Karen Weidert, Ndola Prata

Glob Health Sci Pract. 2016;4(3):410–421 http://dx.doi.org/10.9745/GHSP-D-16-00133

Scheduled Follow-Up Referrals and Simple Prevention Kits Including Counseling to Improve Post-Discharge Outcomes Among Children in Uganda: A Proof-of-Concept Study

Post-hospital discharge is a vulnerable time for recurrent illness and death among children. An intervention package consisting of (1) referrals for scheduled follow-up visits, (2) discharge counseling, and (3) simple prevention items such as soap and oral rehydration salts resulted in much higher health seeking and hospital readmissions compared with historical controls.

Matthew O Wiens, Elias Kumbakumba, Charles P Larson, Peter P Moschovis, Celestine Barigye, Jerome Kabakyenga, Andrew Ndamira, Lacey English, Niranjan Kissoon, Guohai Zhou, J Mark Anserminoa

Glob Health Sci Pract. 2016;4(3):422–434 http://dx.doi.org/10.9745/GHSP-D-16-00069

Intensive Group Learning and On-Site Services to Improve Sexual and Reproductive Health Among Young Adults in Liberia: A Randomized Evaluation of HealthyActions

Combining intensive group learning and provision of on-site reproductive health services through an existing alternative basic education program increased use of contraception and HIV testing and counseling among young out-of-school Liberians.

Rebecca Firestone, Reid Moorsmith, Simon James, Marilyn Urey, Rena Greifinger, Danielle Lloyd, Lisa Hartenberger-Toby, Jewel Gausman, Musa Sanoeg

Glob Health Sci Pract. 2016;4(3):435–451 http://dx.doi.org/10.9745/GHSP-D-16-00074

A Randomized Controlled Trial of a Trauma-Informed Support, Skills, and Psychoeducation Intervention for Survivors of Torture and Related Trauma in Kurdistan, Northern Iraq

Providing survivors of torture, imprisonment, and/or military attacks with a counseling program that includes support, skills and psychoeducation by well-trained and supervised community mental health workers can result in moderate yet meaningful improvements in depression and dysfunction.

Judith Bass, Sarah McIvor Murray, Thikra Ahmed Mohammed, Mary Bunn, William Gorman, Ahmed Mohammed Amin Ahmed, Laura Murray, Paul Boltone

Glob Health Sci Pract. 2016;4(3):452–466 http://dx.doi.org/10.9745/GHSP-D-16-00017

Global Health: Science and Practice 2016 | Volume 4 | Number 3 Table of Contents www.ghspjournal.org

Progress in Harmonizing Tiered HIV Laboratory Systems: Challenges and Opportunities in 8 African Countries

Countries have had mixed results in adhering to laboratory instrument procurement lists, with some limiting instrument brand expansion and others experiencing substantial growth in instrument counts and brand diversity. Important challenges to advancing laboratory harmonization strategies include: 1. Lack of adherence to procurement policies 2. Lack of an effective coordinating body 3. Misalignment of laboratory policies, treatment guidelines, and minimum service packages

Jason Williams, Farouk Umaru, Dianna Edgil, Joel Kuritsky

Glob Health Sci Pract. 2016;4(3):467–480 http://dx.doi.org/10.9745/GHSP-D-16-00004

REVIEWS

Postabortion Care: 20 Years of Strong Evidence on Emergency Treatment, Family Planning, and Other Programming Components

Twenty years of postabortion care (PAC) studies yield strong evidence that: N Misoprostol and vacuum aspiration are comparable in safety and effectiveness for treating incomplete abortion. N Misoprostol, which can be provided by trained nurses and midwives, shows substantial promise for extending PAC services to secondary hospitals and primary health posts. N Postabortion family planning uptake generally increases rapidly–and unintended pregnancies and repeat abortions can decline as a result–when a range of free contraceptives, including long-acting methods, are offered at the point of treatment; male involvement in counseling–always with the woman’s concurrence–can increase family planning uptake and support.

Douglas Huber, Carolyn Curtis, Laili Irani, Sara Pappa, Lauren Arrington

Glob Health Sci Pract. 2016;4(3):481–494 http://dx.doi.org/10.9745/GHSP-D-16-00052

FIELD ACTION REPORTS

Improving the Quality of Postabortion Care Services in Togo Increased Uptake of Contraception

The quality improvement approach applied at 5 facilities over about 1 year increased family planning counseling to postabortion clients from 31% to 91%. Of those counseled provision of a contraceptive method before discharge increased from 37% to 60%. Oral contraceptives remained the most popular method, but use of injectables and implants increased. The country-driven approach, which tended to use existing resources and minimal external support, has potential for sustainability and scale-up in Togo and application elsewhere.

Stembile Mugore, Ntapi Tchiguiri K Kassouta, Boniface Sebikali, Laurel Lundstrom, Abdulmumin Saad

Glob Health Sci Pract. 2016;4(3):495–505 http://dx.doi.org/10.9745/GHSP-D-16-00212

Global Health: Science and Practice 2016 | Volume 4 | Number 3 Table of Contents www.ghspjournal.org

SHORT REPORTS

Use of the World Health Organization’s Medical Eligibility Criteria for Contraceptive Use Guidance in sub-Saharan African Countries: A Cross-Sectional Study

The revised 2015 World Health Organization guidance expanded the recommended contraceptive options available to breastfeeding women during the early postpartum period to include progestogen-only pills and implants, but a substantial number of surveyed country representatives indicated that as yet their national policies did not allow such women to use these methods at that time. Countries may benefit from support to incorporate MEC guidance into national service delivery guidelines.

Melissa J Chen, Mary E Gaffield, James Kiarie Glob Health Sci Pract. 2016;4(3):506–510 http://dx.doi.org/10.9745/GHSP-D-16-00216

Global Health: Science and Practice 2016 | Volume 4 | Number 3 EDITORIALS

Effective LARC Providers: Moving Beyond Training (Republication)

James D Shelton,a Anne E Burkea

Note: This is a republication of an editorial first published in GHSP, Volume 4, Supplement 2, ‘‘Long-Acting Reversible Contraception Crucial to Meeting Unmet Need Goals by 2020: Key Papers From the 2016 International Conference on Family Planning.’’

he special issue of GHSP, focused on long-acting provide than short-acting methods. It is essential to T reversible contraceptives (LARCs) based on papers address the local context, systems, and infrastructure presented at the 2016 International Conference on Family through which LARCs are provided. Key situational Planning, provides testimony to the remarkable rise in the issues include physical resources, staffing, organization popularity of implants and intrauterine devices (IUDs), of work, and cultural context. as well as some limited evidence on permanent methods. Providers are central to the success of any health As these articles report, we see substantial uptake of program. The global health field tends to focus on LARCsinawidevarietyofsituationswhentheyare training to assure providers’ capabilities. But training provided in a quality fashion, including provision of a wide alone is not enough. Drawing largely on articles in this choice of methods. These situations include: issue, we offer observations on some program elements N Public and private sectors that are necessary to help providers be highly productive in LARC programming. N Postpartum N Postabortion Selecting Providers With the Most Potential N Difficult crisis-affected settings Inserting and removing LARCs requires particular skills N Peri-urban slums and more effort than providing many other family planning services, such as pills or injectables. Thus, providers N With vouchers for those particularly in need N of LARCs must be not only technically competent but Via a range of provider cadres. also motivated to provide the service again and again. So, Moreover, these articles provide insights specifi- one selection criterion is whether a provider is likely to cally into the acceptability and provision of IUDs, be able to provide LARCs often enough to keep her which have long been recognized as underutilized. or his skills and have confidence in them. And that calls for selecting staff who are already motivated to WHAT IS NEEDED FOR GOOD LARC PROVISION? provide, and even be champions for, family planning in general. Also, many nonphysician providers may For any health service to be successful, going beyond find satisfaction from ‘‘task sharing’’—being able to the technology or basic intervention alone is crucial. provide a service that was previously reserved for This lesson is particularly important for LARCs and physicians—and so may be enthusiastic about provid- permanent methods, which are more complicated to ing LARCs. Providers such as midwives and qualified nurses often suit this role. In addition, it appears crucial a Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. Correspondence to James D Shelton ([email protected]). to select providers who are likely to remain in their

Global Health: Science and Practice 2016 | Volume 4 | Number 3 356 LARC Providers – Beyond Training (Republication) www.ghspjournal.org

posts for a number of years rather than staff who reported by Boddam-Whetham and colleagues7 LARC providers are going to retire or could be transferred soon. and Bajracharya and colleagues.8 must be not only technically Mentoring and Supervising CONTRAST WITH A LESS SUCCESSFUL competent but It is telling that many of the articles in this PROJECT also motivated to special issue that describe full programming provide the service The project in Bangladesh reported by Rahman efforts emphasize ongoing support to providers, again and again. et al.9 resulted in some appreciable increase in use variously referred to as mentoring, supportive but less than in comparison districts, which had no supervision, or coaching (Samuel,1 White,2 Gueye,3 such intensive intervention. The project included Good mentoring Muthamia,4 Pleah5). A common strategy is to substantial training, with some focus on the supply benefits any begin by recruiting, training, and deploying chain and demand-side support. However, it service but is a core cadre of mentors, who then train and appears the intervention was operating in a much especially mentor other providers. Good mentoring greatly more challenging system context than that in the important for the benefits any service, providing technical sup- comparison districts. For example, there were provision of port, accountability, and motivation. But it is substantially more provider vacancies and fewer LARCs. especially productive for a potentially challenging clients were contacted by community health intervention such as provision of LARCs, which workers in the program districts than in the Provision of LARCs calls for a higher level of skill, commitment, and comparison districts. The high vacancy rate of can be the linchpin motivation, and for which skills are easily lost providers speaks to the possible importance of in the effort to if not practiced. Good mentoring also provides a stronger selection process from the outset. attain FP2020 broader support for provision of all family plan- Notably, an appreciable ongoing mentoring or goals. ning methods, including counseling and practical supportive supervision activity was absent. Would problem solving. the Bangladesh project have been more successful had it had included stronger provider selection and Assuring the Supply Chain mentoring components? Yes, we believe so. A provider without the proper commodities cannot provide the service. And a disrupted supply chain undermines confidence in the CONCLUSION entire service. Thus, several of the articles in this In the context of availability of a wide range issue focus on building and maintaining a of other methods, provision of LARCs can be reliable supply chain. In addition to the contra- the linchpin in the effort to attain FP2020 goals ceptives themselves, LARC provision requires to meet the contraceptive needs of millions of equipment and supplies such as gloves, anti- people. The global health community must make septics, and local anesthetics. While external the necessary investment to foster the skills and donors may supply the implants and IUDs, motivation of LARC providers and give them especially at the beginning of a project, dispo- sufficient system support to facilitate program sable supplies cannot be overlooked or assumed. success. A reliable source of ongoing supply is crucial to sustain continuous services, particularly once Competing Interests: None declared. programs leave the shelter of donor funding. REFERENCES

Supporting the Demand Side 1. Samuel M, Fetters T, Desta D. Strengthening postabortion family Several of the articles emphasize demand planning services in Ethiopia: expanding contraceptive choice and promotion activities, especially outreach to the improving access to long-acting reversible contraception. Glob community, including religious leaders, and Health Sci Pract. 2016;4 Suppl 2:S60–S72. CrossRef some mass media communication. Promotion of 2. White JN, Corker J. Applying a total market lens: experience of Population Services International in increasing IUD service delivery IUDs, in particular, benefits from such activities. through complementary public- and private-sector interventions Misperceptions about IUDs persist among both in 4 countries. Glob Health Sci Pract. 2016;4 Suppl 2:S21–S32. providers and the general public, as documented CrossRef by Twesigye and colleagues.6 Addressing such 3. Gueye B, Wesson J, Koumtingue D, Stratton S, Viadro C, Talla H, misperceptions is essential. In addition, the use et al. Mentoring, task sharing, and community outreach through TutoratPlus: increasing use of long-acting reversible contraceptives of vouchers can increase demand and service in Senegal. Glob Health Sci Pract. 2016;4 Suppl 2:S33–S43. seeking among clients particularly in need, as CrossRef

Global Health: Science and Practice 2016 | Volume 4 | Number 3 357 LARC Providers – Beyond Training (Republication) www.ghspjournal.org

4. Muthamia M, Owino K, Nyachae P, Kilonzo M, Kamau M, Otai J, contraception in Yemen and Pakistan. Glob Health Sci Pract. et al. The Tupange Project in Kenya: a multifaceted approach to 2016;4 Suppl 2:S94–S108. CrossRef increasing use of long-acting reversible contraceptives. Glob 8. Bajracharya A, Veasnakiry L, Rathavy T, Bellows B. Health Sci Pract. 2016;4 Suppl 2:S44–S59. CrossRef Positive impact of a reproductive health voucher program 5. Pleah T, Hyjazi Y, Austin S, Diallo A, Dao Bk, Waxman R, et al. on uptake of long-acting reversible contraceptives in Increasing use of postpartum family planning and the postpartum Cambodia: evidence from a difference-in-difference IUD: early experiences in West and Central Africa. Glob Health analysis. Glob Health Sci Pract. 2016;4 Suppl 2: Sci Pract. 2016;4 Suppl 2:S140–S152. CrossRef S109–S121. CrossRef 6. Twesigye R, Buyungo P, Kaula H, Buwembo D. Ugandan women’s 9. Rahman M, Haider MM, Curtis SL, Lance PM. The Mayer Hashi view of the IUD: generally favorable but many have large-scale program to increase use of long-acting reversible misperceptions about health risks. Glob Health Sci Pract. 2016; contraceptives and permanent methods in Bangladesh: 4 Suppl 2:S73–S82. CrossRef explaining the disappointing results. An outcome and 7. Boddam-Whetham L, Gul X, Al-Kobati E, Gorter AC. Vouchers in process evaluation. Glob Health Sci Pract. 2016;4 Suppl 2: fragile states: reducing barriers to long-acting reversible S122–S139. CrossRef ______Cite this article as: Shelton JD, Burke AE. Effective LARC providers: moving beyond training (republication). Glob Health Sci Pract. 2016;4(3):356-358. http://dx.doi.org/10.9745/GHSP-D-16-00258.

ß Shelton and Burke. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi. ______org/10.9745/GHSP-D-16-00258. ______

Global Health: Science and Practice 2016 | Volume 4 | Number 3 358 VIEWPOINT

Moving Medicine, Moving Minds: Helping Developing Countries Overcome Barriers to Outsourcing Health Commodity Distribution to Boost Supply Chain Performance and Strengthen Health Systems

Priya Agrawal,a Iain Barton,b Roberto Dal Bianco,a Dana Hovig,c David Sarley,c Prashant Yadavd

Senegal and other developing countries are improving access to health commodities by outsourcing supply chain logistics to private providers. To achieve broader, lasting reform, we must support further adoption of the outsourced model; assist country-led cost-benefit analyses; and help governments build capacity to manage contracts and overcome other barriers.

INTRODUCTION These gains were achieved during a 3-year program supported by MSD for Mothers (known as Merck for Until recently, women in Senegal who went to their Mothers in the United States and Canada), The Bill & local public health clinics seeking a means to Melinda Gates Foundation, IntraHealth International, prevent pregnancy had about a 3 or 4 in 5 chance of 1 and other partners. During the program, other compo- leaving empty-handed. Implants, pills, injectables, and nents developed by the private sector were implemented, ’ other contraceptives, while promoted as part of Senegal s including an innovative inventory management system commitment to improving maternal health and access to and new methods of data tracking to support mobile family planning, were routinely out of stock. The problem, warehousing. These made stock replenishment more all too common in developing nations, was improper responsive to real consumption. Contracted workers are supply chain management, specifically inventory manage- able to accurately track and verify consumption and stock ment and delivery. To address this, the government of levels and plan and customize shipments; regular data Senegal embarked on a major overhaul of its supply chain uploads allow them to quickly address potential issues. for 9 different contraceptives (with 2 more added later). 1,2 Qualitative analysis and cost comparisons between Within 6 months, stock-outs ceased almost entirely. regions being served by 3PLs and the one region that ’ Key in the success of Senegal s reform was the was not served by 3PLs helped garner support within ’ government s strong commitment to, and effective imple- the government for the outsourced model. In sum- mentation of, hiring private third-party logistics providers mer 2016, the new system began transitioning from (3PLs) to manage orders and handle deliveries from donor-supported partners to government control under district warehouses to local health facilities, with clear Senegal’s National Supply Pharmacy (PNA), a process benefits for service levels and costs. One analysis com- that is expected to take several months. At the launch pared cost and service through the use of private operators of the handover process in August, the Director of versus those of government employees performing the same the PNA pledged to continue contracting with pri- activity in a different region of the country. The analysis vate operators for last-mile deliveries to all 1,400 of found that outsourcing decreased the proportion of facilities the country’s service delivery points, and to expand the experiencing stock-outs from over 80% to less than 2%1 3 system to cover approximately 100 products by the end while reducing distribution costs by 36% annually. of 2017. Senegal’s Minister of Health and Social Action has endorsed the outsourced model as a way to help a Merck & Co., Inc., Kenilworth, NJ, USA. Merck & Co., Inc. is known as MSD ensure access to essential medicines and health com- outside the United States and Canada. modities for all citizens. This political support is both b Imperial Health Sciences, Gauteng, South Africa. c The Bill & Melinda Gates Foundation, Seattle, WA, USA. encouraging and necessary, as long-term success will d University of Michigan, Ann Arbor, MI, USA. depend on the government’s ability to institutionalize Correspondence to Priya Agrawal ([email protected]). the approach.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 359 Moving Medicine, Moving Minds www.ghspjournal.org

By outsourcing A handful of other developing countries are also to do so risk termination of their contracts. supply chain pursuing outsourcing and reaping the benefits. (For these incentives to work, of course, govern- logistics to private In the Western Cape Province of South Africa, ments must be willing to hold those service operators, for example, the storage, handling, and trans- providers accountable.) Senegal portation of vaccines has markedly improved in Private 3PLs are better-equipped to manage decreased the the hands of a 3PL while costs have also been 4 growth. Government agencies typically lack proportion of lowered. Kenya outsources the transport of med- the flexibility and capacity to scale quickly to facilities ical supplies to all its health care facilities, having address seasonal issues or outbreaks, due to experiencing determined that managing a fleet of trucks was rigid budgets and bureaucratic hiring processes. neither a core competency of government nor stock-outs from an effective use of government resources.5 In 3PLs typically add stronger information and over 80% to less Nigeria, direct deliveries from an outsourced pro- data management systems, enabling greater than 2%. vider have increased the availability of vaccines stock management and resource efficiency. in Kano and Lagos,6 and, despite constraints from (Data, and data visibility, are central to supply the economic downturn and weaknesses in manage- chain performance.) Success of direct- ment capacity, the direct delivery model is gradually For all these reasons and more, outsourcing is delivery programs being expanded nationally. In Mozambique and standard practice across Canada, Europe, and the managed by the Zimbabwe, direct deliveries of a range of products United States, and in the rest of the developed private sector have increased stock availability, enhancing system world. In fact, no country in the Organisation for 7,8 depends on strong performance. In other places, however, such Economic Co-operation and Development (OECD) government as Togo, similar programs have not been as suc- has a government-run public health supply 10-12 commitment, clear cessful. The success of direct-delivery programs chain. The U.S. Centers for Disease Control, roles and managed by the private sector ultimately depends for example, has achieved significant cost efficiencies outsourcing storage and distribution for accountability on strong government commitment; clear roles and accountability mechanisms; a robust techni- its multibillion dollar Vaccines for Children pro- mechanisms, cal design of the processes and key performance gram13; the UK National Health Service, which robust technical indicators to be measured; and effective multi- outsources procurement, storage, and distribu- design, and stakeholder collaboration. tion to DHL, has projected a savings of a billion effective multi- 14,15 We urge other developing nations—in Africa pounds over 10 years. stakeholder and elsewhere where public-sector supply chains Developing countries stand to benefit from collaboration. have been underperforming for years—to look to outsourcing in two other critical ways. First, with these countries’ experiences as examples of what private contractors handling deliveries and other can be achieved and reminders of the issues that on-the-ground logistics, doctors, nurses, and other Outsourcing will need to be addressed (Table). clinical staff can concentrate on their primary logistics responsibility and core competency: patient care. operations to In a government-run system, it is the medical private THE ADVANTAGES personnel, already in short supply across the contractors frees When deciding whether to outsource, there are developing world, who are often saddled with up doctors, many benefits to consider. For example: part-time supply chain duty, filling out inventory forms, traveling to storage sheds to collect stock, nurses, and other Outsourcing improves performance through and performing other tasks for which they were clinical staff to competition. It brings market forces into play, concentrate on not originally trained. When outsourced, skilled with competitive bidding and performance- 3PL workers make the deliveries and perform patient care. based incentives. To stay competitive, 3PLs the inventory control. This task shifting is a invest to acquire the technical expertise and key benefit of Senegal’s outsourced distribution Outsourcing capital assets they need to provide top service model. It has allowed government employees improves while keeping prices in line. Top performers to be redeployed to new clinical support and performance earn contracts. If they fail to perform, another management positions, not downsized, as is often through company gets the job. Poor performance by feared. competition. public-sector personnel is more difficult to Second, government spending through out- rectify. sourcing stimulates the local economy by creating Private operators are highly incentivized to business opportunities for local entrepreneurs comply with the strict storage requirements that in turn create new private-sector jobs. In governing health commodities. Those who fail Senegal, local operator Cabit SA has doubled its

Global Health: Science and Practice 2016 | Volume 4 | Number 3 360 Moving Medicine, Moving Minds www.ghspjournal.org

TABLE. Outsourcing Supply Chain Logistics in Low- and Middle-Income Countries: The Advantages, the Hurdles, and Some Recommendations on How to Move Forward

Advantages Hurdles Recommendations

Outsourcing saves money: Using private There is general reluctance by in-country Evaluate what lies within government’s operators to distribute medicines and other leadership to give up government core competency, and what lies outside health commodities can reduce costs by ownership of delivery systems and of it. Through oversight and enforcement as much as one-third or more.a infrastructure. of strict service-level agreements, governments can maintain control over their supply chains even while outsourcing the logistics; government still owns and manages the process, even when non-government drives the trucks. Built-in performance incentives: There is insufficient capacity and expertise Assist governments with building effective Logistics operators are motivated within governments to write and effectively contract management and enforcement to provide effective, efficient service manage contracts with private operators. teams. and prevent stock-outs in order to win and keep their contracts. Medical staff at service delivery points Outsourcing the logistics of health Governments may need help restructuring are no longer responsible for restocking commodity distribution could threaten to their human resources and retraining and inventory and can focus on patient care; eliminate some public-sector jobs. reassigning employees for oversight new jobs are created in the local private and supervisory roles; local private sector as local companies hire more operators may need training and employees and otherwise invest in their other support. own growth to secure government contracts. In a government-run distribution system, In some markets, it can be difficult to find Private-sector donors or businesses, acting employee working hours and staffing private-sector operators trained and as fourth-party providers, can assist in levels are fixed; private operators can be equipped to use state-of-the-art the professional development of local more flexible, so government pays only for approaches to inventory management private-sector operators. what it needs and uses. and to comply with strict requirements regarding storage and transport of medicines. Contracting with multiple operators can Concerns exist about long-term financing Private-sector expertise can help with the help spread risk, so that the failure of and sustainability of the new health necessary costing analysis, but top-level, one is far less likely to bring a whole distribution system given the temporary in-country leadership must take ownership system down. nature of donor support. over the process—and the solutions.

a Outsourcing in Senegal reduced distribution costs by an estimated 36%,3 and outsourcing the distribution of vaccines in South Africa yielded signiﬁcant savings, from 28% of the vaccine cost to 6%.9

staff from 30 to 60 since being contracted by THE CHALLENGES Outsourcing IntraHealth International to handle logistics. In stimulates the Malawi, contraceptives and some essential med- Given all the upside, why has outsourcing failed local economy by icines (including antimalarial drugs) are distrib- to catch on in the developing world? Here, creating business uted by Cargo Management Logistics (CML), a we provide a few explanations. First, outsourcing opportunities local private company that has grown from 46 to requires a sharp break with an old tradition of for local 112 employees in the 4 years since taking on this moving medicine through government-owned entrepreneurs. work. In Nigeria, Imperial Health Sciences manages centralized stores. This colonial practice was later warehousing services and deliveries by 7 locally reinforced when Ghana gained independence and owned and financed transport companies, which tried to emulate the supply chain model used in together employ more than 270 people to deliver the former Soviet Union. The Soviets owned and products to 5,000 points of care. operated the medicine supply chain, so Ghana

Global Health: Science and Practice 2016 | Volume 4 | Number 3 361 Moving Medicine, Moving Minds www.ghspjournal.org

logistics operators may only serve urban areas. For example, Beale et al.16 and VillageReach17 highlight the lack of transport providers in rural Mozambique. Private companies require long- term contracts to invest in transport capacity in an underserved region where they do not currently serve a market. Yet governments are hesitant to make long-term contracts as for many this is not yet a proven model. For transporters, especially smaller local players, the timeliness of government payment is also critical for their cash flow. Third, concerns that outsourcing threatens public health sector jobs can dampen political support. Nobody wants the prospect of laying people off, particularly in countries where the public sector is the primary source of stable employment. In addition, while outsourcing can reduce product diversion and pilferage, a robust, transpar- ent, and rigid tender process for the outsourced contract award is a prerequisite for its success. Finally, there is the question of sustainability. Donor-funded programs are often the way that outsourcing is introduced and tested in a country. Yet it is often the pattern that such efforts prove initially successful, only to revert to insourcing when the program transitions back to government. Kenya, for example, reverted back to insourcing some of its distribution activities when Employees of Cabit, a 3PL hired to handle last-mile distribution of an outsourcing project supported by the U.S. contraceptives and other health commodities, record consumption Agency for International Development (USAID) and delivery data in a stock room of a health center in the Fatick ended. Concerns about long-term financing—and region of Senegal. the temporary nature of donor support—have made many country leaders reluctant to take the necessary steps toward change, even as they readily acknowledge a pressing need for it. The and many other countries in Africa did the same. benefits of outsourcing and direct delivery are However, this model has long proved inefficient based on the combination of technical design and fragile, hobbled by weak performance incen- (who manages inventory control, information tives and the inability to flex capacity as needed. flow), source of financing (donor financed vs. By the 1990s, the model was already struggling domestic financing), and commitment from across Africa and it buckled when the HIV crisis political leaders or heads of public agencies. The hit. Subsequent efforts, with international assis- benefits can start declining when one of these tance, to sustain this model have done little to ingredients is missing or becomes diluted over alter the fundamental structure, which continues time. to prove incapable of handling the ever-widening Many developing scale and diversity of products required to meet nations lack the patient needs. THE WAY FORWARD mechanisms Second, many developing nations lack the To overcome these barriers, we need to support and capacities mechanisms and capacities for contract manage- country leaders as they determine which seg- for contract ment and oversight. And it can be difficult to find ments of their supply chains are best suited for management local private operators capable of handling health outsourcing, and as they seek to identify what and oversight. product distribution. In some countries, private truly lies within the core competency of government

Global Health: Science and Practice 2016 | Volume 4 | Number 3 362 Moving Medicine, Moving Minds www.ghspjournal.org

and what lies outside it. They will need support employees on the new inventory management to adequately assess risks, costs, and potential techniques adopted as part of Senegal’s newly savings, as well as their own abilities to manage revamped distribution system.) Engaging a contracts with outsourced partners. And they 4PL represents an additional cost layer, how- may need help restructuring internal human ever, and can be a disincentive to outsourcing resources for new supervisory roles within the unless the benefits are clearly understood. ‘‘ new system. The 2015 Gavi study, Outsourcing Build in a transition plan from the start to the Distribution Component of Vaccine and Medi- ensure sustainability after donors pull out. ’’ 18 cine Supply Chains, carried out by Transaid, The main challenges facing Senegal during its offers a useful framework for these strategic transition will be managing the higher costs discussions. and greater data collection and demand For those governments considering a move planning requirements posed by a tenfold toward outsourcing their supply chains and to all increase in the number of 3PL-distributed — supporting partners and stakeholders public products. and private—we recommend the following: Explore how existing government staff might Conduct an in-depth, cost-risk-benefit analy- be retrained and reassigned to new roles in sis of the existing government-run, insourced contract management and oversight. In an out- distribution system. Have an honest discus- sourced system, government is still in control sion about what changes are feasible and who of the public supply chain, so it is government is going to pay for it. The private sector knows that must enforce service-level agreements how to do the kind of granular costing anal- and ensure that performance targets are ysis that is required, and does it well. achieved. Look at pharmacists and others Top-level country leadership should drive the who have been pulled away from their field to Top-level country process from the beginning, so that govern- work the supply chain and allow them instead leadership should ment—including the finance ministry—takes to return to what they originally studied. drive the ownership of both the analysis and the Develop sufficient government capacity for outsourcing solutions. contract management and other mechanisms process from the Take full account of the many hidden costs for effective private-sector engagement. Rig- beginning. of a government-run distribution system, such orous government oversight is essential to as asset depreciation and staff time spent secure compliance, prevent corruption, and away from their clinic post. If not exposed, ensure that drugs are not misused or mis- these hidden costs can make an outsourced handled. Contracts will also need to include model, in which costs are more explicit, seem the proper incentives to ensure that 3PLs expensive by comparison. Spending more may distribute to rural and remote areas, not just make sense if it means dramatically improved the major towns and other densely populated ’ product availability on shelves. regions. (USAID s experience through its DELIVER Project highlights the importance Make clear the return on investment. of building a strong contract management Procure 3PL services through a competitive team,19 and VillageReach’s experience in process to keep costs in line. Mozambique highlights similar challenges.8) — Consider having a commercial or NGO partner Key performance indicators the ability of serve as a fourth-party provider (4PL) to manage outsourced partners to manage costs, quality — subcontracts with several 3PLs—which of service, responsiveness, etc. must be care- Mozambique, Nigeria, Senegal, and Zimbabwe fully designed and monitored. Strong contract all have done. A 4PL adds value by streamlin- language by itself is not enough; overseers ing the government’s role in the nitty-gritty must be willing and able to enforce the terms logistics and in performance management, and conditions. by creating a single point of engagement. Select individuals with prior private-sector The 4PL can find, train, and manage all local logistics and supply chain experience to serve transport partners, and it has the incentive as top-level supply chain managers. Those to make sure they do well. (In Senegal, with a commercial background are more likely IntraHealth International, acting as a 4PL, to make the bold (and sometimes politically provided training for Cabit SA and other 3PL unpopular) decisions necessary to implement

Global Health: Science and Practice 2016 | Volume 4 | Number 3 363 Moving Medicine, Moving Minds www.ghspjournal.org

change. Progress in Kenya has been widely risk mitigation approaches and bring on board attributed to the fact that an executive with government stakeholders who will strongly advo- rich private-sector experience was brought in cate for this. to run the Kenya Medical Supplies Authority (KEMSA). Under his leadership, KEMSA has CONCLUSION become less bureaucratic, more independent, and more competitive and a good model for Demands on in-country supply chains will only other countries looking to outsource and intensify going forward. The sheer volume of otherwise streamline their operations. drugs and other products flowing into the public Engage the public in the debate over supply health pipeline increases year after year, as new chain reform, by communicating what’sat treatment options become available and govern- stake for consumers—in other words, how a ing bodies establish more ambitious health well-functioning supply chain helps protect, targets. The newly adopted Sustainable Develop- improve, and save lives. One way to do this is ment Goals call for universal access to reproduc- ‘‘ ’’ to present performance metrics in terms that tive health care. The new 90-90-90 targets from are relevant and relatable to patients. Are the the Joint United Nations Programme on HIV/ — medicines and other supplies available when AIDS (UNAIDS) by 2020, 90% of people living they need them? Are they affordable and with HIV will know their HIV status, 90% of accessible? How far do patients have to travel people diagnosed with HIV will receive sustained to get to them? Activists and civil society can antiretroviral therapy, and 90% of people receiv- play an important catalytic role by shining a ing antiretroviral therapy will have viral sup- — spotlight on poor performance and advocat- pression will essentially double the number of ing for more reliable access. Sweden, the patients who will require treatment over the next last industrialized nation to embrace private 5 years. The need to ensure continuous access to sector-run medicines distribution (in 2009), maternal and child health commodities, as well tapped a bipartisan, independent agency to as the rising prevalence of hypertension, diabetes, monitor and review the successes and defi- and other non-communicable diseases in devel- Supply chains will ciencies resulting from the privatization. By oping countries, will add further strain. Supply have to be even highlighting issues important to the general chains will have to be even stronger, more efficient, and more effective to enable governments to meet stronger, more public (not just supply chain experts), the the needs of their citizens. The recent Ebola out- efﬁcient, and more agency allowed government to build public 10 break is a stark reminder that supply chains must effective in the confidence in the project. If we can similarly be agile and responsive—and a reminder of the future to enable generate public interest in supply chain issues — tragic consequences when they are not. governments to in developing countries and engage civic groups and other community organizations, We believe that to get public health supply meet the needs of chains—and patient access—to where they need their citizens. which can often sway the course of public health policies and programs and bring about to be, private-sector outsourcing needs to be part greater transparency—we can galvanize pub- of the equation. Health ministries can and should lic support for the solutions. learn from the Senegal experience, and from the successful efforts of other countries to outsource Global stakeholders should build a stronger parts of their supply chains. We believe that many evidence base for where, when, and how out- of the obstacles are surmountable and that in sourcing works in a long-term, sustainable way in most cases, with the proper conditions and support, developing country health systems. This will outsourcing can improve health commodity avail- create a global public good in terms of knowledge ability and help strengthen health systems. of what any country starting to consider out- The women of Senegal are already benefitting sourcing should watch out for, and how to from their country’s supply chain overhaul in remedy failures and create the right prerequisites family planning. Since the reform, a young wife for success. Before embarking on such projects, trying to space her pregnancies to avoid compli- global agencies must also pay close attention to cations, the mother of 6 children who is in no the complex political economy surrounding such condition to have a seventh, and all the other projects that is present in many low- and middle- women who require contraceptives, have been income countries. They must develop a clear path able to depend on the availability of contraceptive to more effectively communicate its benefits and products in a way that was unheard of just a few

Global Health: Science and Practice 2016 | Volume 4 | Number 3 364 Moving Medicine, Moving Minds www.ghspjournal.org

years ago. This is what a top-performing sup- Niassa 2015. Seattle (WA): VillageReach; 2015. Available from: ply chain can do, and it is what every person, http://www.villagereach.org/wp-content/uploads/2016/03/ everywhere, deserves. All-Provinces_PerformanceReport_2015_FINAL.pdf 9. PATH. Outsourcing vaccine supply chain and logistics to the private sector. Seattle (WA): PATH; 2012. Available from: https:// Acknowledgments: We thank Mark Allen, Maryanne Buechner, www.path.org/publications/ﬁles/TS_opt_outsourcing_br.pdf Jeffrey Jacobs (Merck & Co., Inc., Kenilworth, NJ, USA), and Adam Williams (Rabin Martin, New York, NY, USA) for their editorial support 10. Yadav P. Transition from state monopoly to managed markets for in preparing this article. delivering pharmaceutical products in Sweden: a case study in managing markets for health. Washington (DC): World Bank Competing Interests: Dr. Agrawal reports that she is an employee Institute; 2016. of Merck & Co., Inc., Kenilworth, NJ, USA, and may hold stock or stock options in the company. Dr. Barton reports that he is an executive of 11. Organisation for Economic Co-operation and Development a logistics company that provides outsourced services to governments, (OECD). Competition issues in the distribution of pharmaceuticals. donors, and pharmaceutical companies; he also serves on the advisory Paris: OECD; 2014. Available from: http://www.oecd.org/ board of Merck for Mothers. competition/competition-distribution-pharmaceuticals.htm#docs 12. Yadav P. Health product supply chains in developing countries: REFERENCES diagnosis of the root causes of underperformance and an agenda for reform. Health Systems & Reform. 2015;1(2). CrossRef 1. Daff BM, Seck C, Belkhayat H, Sutton P. Informed push 13. Vaccines for Children. Vaccines Management Business distribution of contraceptives in Senegal reduces stockouts and Improvement Project (Internal). Atlanta (GA): U.S. Centers for improves quality of family planning services. Glob Health Sci Disease Control and Prevention; 2014. Pract. 2014;2(2):245–252. CrossRef. Medline 14. NHS Supply Chain. Customers on track to unlock d150m savings 2. IntraHealth International. Expanding the Informed Push Model: through NHS Supply Chain [Internet]. 2015 Sep 21 [cited 2016 October progress report (Internal). Chapel Hill (NC): IntraHealth Mar 15]. Available from: https://www.supplychain.nhs.uk/ International; 2015. Home/News/Company/On%20Track%20To%20Unlock% 3. Dal Bianco R. IPM cost-effectiveness of private vs public sector 20150m%20Savings distribution. Presented at: International Conference on Family 15. DHL Supply Chain. NHS Supply Chain: innovative procurement Planning; 2016 Jan 27; Nusa Dua, Bali. and logistics model drives substantial savings for UK’s National 4. Lydon P, Raubenheimer T, Arnot-Kru¨ger M, Zaffran M. Outsourcing Health Service. Milton Keynes (UK): DHL Supply Chain; [date vaccine logistics to the private sector: the evidence and lessons unknown]. Available from: http://www.dhl.co.uk/content/dam/ learned from the Western Cape Province in South-Africa. Vaccine. downloads/uk/Logistics/nhs_dsc_lifescience_case%20study_v2.pdf 2015;33(29):3429–3434. CrossRef. Medline 16. Beale J, Mashiri M, Chakwizira J. Prospects for leveraging 5. Yadav P. Kenya Medical Supplies Authority (KEMSA): a case private sector logistics ﬁrms to support rural access to healthcare: study of the ongoing transition from an ungainly bureaucracy to a some insights from Mozambique. In: Proceedings of the 34th competitive and customer focused medical logistics organization. Southern African Transport Conference (SATC 2015); 2015 Washington (DC): World Bank; 2012. Available from: http:// Jul 6-9; Pretoria, South Africa. p. 329. documents.worldbank.org/curated/en/2012/04/20330086/ kenya-medical-supplies-authority-kemsa-case-study-ongoing- 17. VillageReach. Evaluation of health system transport capacity and transition-ungainly-bureaucracy-competitive-customer-focused- demand: Mozambique case study. Seattle (WA): VillageReach; medical-logistics-organization 2014 Available from: http://www.villagereach.org/wp-content/ uploads/2009/08/062014-TSS-Assessment-Report-FINAL.pdf 6. Department of Logistics and Health Commodities. Review of the vaccines supply chain transformation activities in 2015 and plans 18. Transaid. GAVI study -- outsourcing the distribution component of for 2016 (Internal). Abuja (Nigeria): National Primary vaccine and medicine supply chain. London: Transaid; 2015. Healthcare Development Agency; 2015. Available from: http://www.transaid.org/knowledge-centre/ 7. Rosen JE, Bem J, Wolf K. Evaluation of the Zimbabwe Assisted gavi-study-outsourcing-the-distribution-component-of-vaccine- Pull System (ZAPS): baseline report. Arlington (VA): USAID | and-medicine-supply-chains/ DELIVER PROJECT, Task Order 4; 2015. Available from: http:// 19. USAID | DELIVER PROJECT. Technical brief: logistics outsourcing deliver.jsi.com/dlvr_content/resources/allpubs/countryreports/ and contract management in public health settings. Washington ZWAssiPullBaseRepo.pdf (DC): John Snow, Inc.; 2014. Available from: http://deliver.jsi. 8. VillageReach. Mozambique Dedicated Logistics System (DLS) com/dlvr_content/resources/allpubs/logisticsbriefs/ performance report: Cabo Delgado, Gaza, Maputo and LogiOutsContMana.pdf

Peer Reviewed

Received: 2016 Apr 21; Accepted: 2016 Aug 26

Cite this article as: Agrawal P, Barton I, Dal Bianco R, Hovig D, Sarley D, Yadav P. Moving medicine, moving minds: helping developing countries overcome barriers to outsourcing health commodity distribution to boost supply chain performance and strengthen health systems. Glob Health Sci Pract. 2016;4(3):359-365. http://dx.doi.org/10.9745/GHSP-D-16-00130.

& Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., Kenilworth, NJ, USA. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi.org/10.9745/GHSP-D-16-00130.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 365 COMMENTARY

Accessible Contraceptive Implant Removal Services: An Essential Element of Quality Service Delivery and Scale-Up

Megan Christoﬁeld,a Maryjane Lacosteb

This article was drafted on behalf of the Implants Access Program Operations Group, comprised of representatives from the Bill & Melinda Gates Foundation, the United States Agency for International Development (USAID), the United Nations Population Fund (UNFPA), and the Clinton Health Access Initiative (CHAI). It was established as a working group to proactively and reactively address operational and service delivery issues arising from increased country-level availability of implants. The overall goal of this group is to improve coordination around LARC service delivery, including investigating and addressing country operational issues arising from wider availability of implants in-country and making recommendations for the way forward to appropriate stakeholders.

enewed investment in scaling up contraceptive Criteria for Contraceptive Use.4 Recently, implant manu- Rimplants has resulted in a dramatic increase in facturers Merck and Bayer announced plans to sustain their use since 2012. The surge is due in part to the their current reduced implant pricing for an additional reduction in price and increases in donor investments 5 years, through 2023, creating price parity for all the made through the Implants Access Program (a colla- currently available implant products and further paving boration between public and private organizations to the road for potential continued scale-up.5 make implants accessible to women in the world’s However, emerging data show that service delivery poorest countries) and ministerial prioritization and capacity for implant removals has not kept pace with support to facilities and providers, as well as user that for insertion. For example, in Kenya, among preference. Among the 69 Family Planning 2020 Ministry of Health facilities offering family planning (FP2020) focus countries, prevalence of injectables services in 2015, 86% provided contraceptive implants and implants is growing faster than all other contra- while only 67% provided removals.6 Furthermore, ceptive methods; in Ethiopia, Kenya, Malawi, Senegal, clients who access removal at private-sector (and some and Zimbabwe, the percentage of women ages 15–49 public-sector) facilities can encounter user fees,7 and using implants is growing by over 1 percentage point those who receive their method from a mobile outreach per year.1 Implants are reaching more women than ever campaign or a community health worker are at times before,2 including those who have traditionally been without clear or accurate, up-to-date information on underserved.3 Implants also now have the potential to how and where to seek follow-up services and removal. meet the needs of postpartum women who are While there is a paucity of evidence regarding access to breastfeeding immediately after birth as a result of removal in the peer-reviewed literature, ministries and the World Health Organization’s recent decision to program managers increasingly cite reports of clients’ allow their use among this important population, which failed attempts in obtaining the removal procedure.8 is reflected in the fifth edition of the Medical Eligibility With the rapid expansion of implants services, the family planning community—donors, implementers, — a Jhpiego, Baltimore, MD, USA. ministries, advocates, and health care providers has b Bill & Melinda Gates Foundation, Seattle, WA, USA. reached a critical point at which it needs to assure the Correspondence to Megan Christoﬁeld (megan.christoﬁ[email protected]). availability of convenient, quality removal services for

Global Health: Science and Practice 2016 | Volume 4 | Number 3 366 Accessible Implant Removal Services www.ghspjournal.org

clients who want removal for any reason 2-rod implant technologies (compared with Prevalence of throughout the use of their implant, including Norplant’s 6 rods) has made the removal proce- implants (and 7 those discontinuing contraceptive use, switching dure much easier, it has not freed the method injectables) is 16 to another method, or removing the implant to from technical difficulties. growing faster have a subsequent implant inserted. The family The sheer volume of anticipated removals in than all other planning community has a responsibility to sup- the coming years should give us pause. This contraceptive port method continuation as well as access to unprecedented growth in the availability of methods in the — quality removal when desired commensurate to implants will result in an equal growth in the 69 FP2020 focus ’ — the attention paid to the method s initiation need for implant removals in the near future countries. such that clients’ reproductive intentions can be because currently available implants have a realized.9 We need more data visibility into 3-to-5-year lifespan. Using publically available implant removals and adoptable approaches to data from RHInterchange,2 which has contra- Service delivery expanding access to removal services—and it is ceptive procurement data from major donors capacity for imperative that we act urgently. and international organizations for more than implant removals 140 countries, we modeled the approximate time- has seemingly not WHY THE FOCUSED ATTENTION ON line and magnitude of this upcoming removal kept pace with burden (Figure). The bars in the Figure show REMOVALS? that for insertion. procurement of implants by year, over the past Implant removal is an essential component of 10 years, in the 69 FP2020 focus countries. contraceptive implant scale-up, critical to offering We calculated the lines projecting number of Implant removal high-quality services and continuity of care for implants due for removal by disaggregating pro- is an essential family planning. Inadequate removal services leave curement data by implant product, adding a component of some clients on contraception when they would 12-month period from receipt in-country to inser- implant scale-up prefer not to be, whether the intention is to tion in a client, and then assuming that once and high-quality conceive or discontinue the method for other inserted, each product was used for its couple-years service delivery. reasons. This inability to access removal within a of protection (CYP) unit—2.5 years for Implanon, reasonable time frame consistent with access to 3.2 years for Sino-implant (II), and 3.8 years for other services compromises clients’ rights and Jadelle.17 For example, a shipment of Jadelle that choice. As the literature on rights-based family arrived in a country in July 2008 was modeled for planning has made clear, ensuring access to implant removal in April 2013. A second scenario is also removal helps fulfill the aim of voluntary family presented in the Figure to account for the possible planning such that it extends into the method’s shift of Implanon’s qualified effectiveness from discontinuation as well.10,11 For example, the 3 years to 5 years (represented as 3.8 CYP in the FP2020 ‘‘Rights and Empowerment Principles for model). This was applied beginning with clients Family Planning,’’ within the tenet of availability, who had an Implanon implant inserted in 2014, states clearly, ‘‘Health care facilities, trained provi- with the assumption that current users will be ders and contraceptive methods are available to notified that they may keep their implant inserted ensure that individuals can exercise full choice from longer than 3 years. Although the model uses a full range of contraceptive methods. y Avail- procurement data as a proxy for use, it echoes the ability of services includes follow-up and removal previously acknowledged trend of increasing services for implants and IUDs.’’11 growth in use of implants and conveys overall that There is precedent for serious concern about with either scenario we will experience a growing lack of quality removal services. The learning number of removals in years to come, as current generated from the global scale-up effort of implant users age out of their implants or remove Norplant implants (beginning in the early 1990s) for other reasons. According to this model, the The number of blames the method’s low uptake, in part, on in- number of estimated removals needed in 2015 estimated implant 12-15 adequate access to and quality of removals. (2.2 million) was less than half the number pro- removals needed Furthermore, issues of access to quality removal jected for 2018 by either scenario (4.9 million for in 2015 in the 69 services had repercussions; Frost and Reich noted the first scenario, 5.8 million for the second FP2020 focus ‘‘ — that for several reasons, removal problems be- scenario) a worrisome figure if removal issues countries was less came major barriers to Norplant access in some have already begun to emerge. than half the ’ countries, with negative implications for the Of note, the model s limitations include: number projected ’ product s reputation, appropriate use, and customer (1) use of procurement data as a proxy for use, for 2018. satisfaction.’’12 And while the advent of 1- and (2) the assumption that all implants are used for

Global Health: Science and Practice 2016 | Volume 4 | Number 3 367 Accessible Implant Removal Services www.ghspjournal.org

FIGURE. Number of Implants Procured by Year in the 69 FP2020 Focus Countries, 2005–2015, and Projected Number of Implant Removals, 2010–2019

Abbreviation: CYP, couple-year of protection; FP2020, Family Planning 2020. * Scenario 1 assumes that each implant is used for its current CYP unit—2.5 years for Implanon, 3.2 years for Sino-implant (II), and 3.8 years for Jadelle. Scenario 2 accounts for the possible shift in the approved length of effectiveness for Implanon, from 3 years to 5 years, which would change its CYP unit to 3.8 CYP. Source of data: RHInterchange.2

their CYP unit to calculate the removal projection, independent settings. For example, with USAID (3) use of the general estimate of 12 months to funding, Marie Stopes International (MSI) – represent the time between the implant ship- Tanzania partnered with the Ministry of Health ment’s arrival in country and the product’s and Social Welfare to build capacity of public-sector insertion into a client, and (4) the exclusion of providers to provide voluntary long-acting rever- direct procurement by governments and some sible contraceptive services, including removal of other third-party procurement (for example, implants and intrauterine devices (IUDs). They Indonesia’s procurement) from RHInterchange supported these facility-based providers by includ- procurement numbers. ing them in a 3-week tour with an MSI mobile outreach team where they received on-the-job training with many opportunities for insertion Some country PROMISING EFFORTS programs are and removal practice due to the high client load.19 implementing Access to implant removal has not been neglected And in Ethiopia, where health extension workers innovative entirely. In addition to the presence of removal (HEWs) provide implants, helping the nation strategies to guidance and training within national guidelines achieve great gains in the modern contraceptive address implant and curricula (and the global implants learning prevalence rate, the Federal Ministry of Health and 18 removal. resource package ), many programs are address- Pathfinder International (under the USAID-funded ing implant removal proactively within their Integrated Family Health Program) developed a

Global Health: Science and Practice 2016 | Volume 4 | Number 3 368 Accessible Implant Removal Services www.ghspjournal.org

coordinated strategy to expand access to implant has already shared lessons learned and identified removal through strong referral and support a learning agenda. In addition, in each of the task mechanisms—Ethiopia’s task-shifting policy allows force’s 4 subgroups (capacity building and service HEWs to insert but not remove implants.20,21 When delivery; data and monitoring; research; and HEWs identify clients in need of implant removal, difficult removals), action plans have been devel- they notify the linked health center, which res- oped and new tools, approaches, and analyses are ponds by sending a skilled service provider to the in development to meet the needs of ministries, community level to provide the service.20 In this providers, partners, and donors. As this task force way, clients who receive their implant from a HEW matures, it expects to deliver clear evidence and best are equally supported to get the implant removed practices and offer tangiblesolutionstothosewho when desired. Furthermore, research is underway need them. As a starting point, the task force has in Nigeria to assess the capability of community developed a short list of requirements of quality, health extension workers to insert and remove client-centered implant removal services (Box). implants, an assessment that has the potential to inform other community provision models. Cost WHAT WE CAN DO NOW solutions are also being tested, including the use of Now is the right vouchers that capture the insertion, follow-up care, Now is the right time to tackle this issue. In this time to tackle the and removal fees within 1 voucher so that the user period of rapid scale-up of contraceptive implants, issue of ensuring 22 fee is only levied upon uptake. the opportunities for advocacy and action sur- ready access to At a global level in 2015, several partners, round us. Whether in the development of costed high-quality including UNFPA and CHAI, developed a standar- implementation plans, in updates to heath implant removal dized consumables kit for contraceptive im- management information systems, in the actions services. plant services. The kit includes supplies for inser- associated with national FP2020 commitments, tion and removal, offering an easily procurable or in so many other efforts, each offers the option for places where supply planning has been opportunity to address preparedness for, and an issue.5 With Bill & Melinda Gates Foundation delivery of, implant removals when desired. funding, the Performance Monitoring and Account- First and foremost, more data are needed. The ability 2020 (PMA2020) project and FHI 360 great success in implant scale-up has been developed a series of implant removal access ques- measured almost exclusively by its uptake, yet tions for piloting in PMA2020 surveys in Ethiopia monitoring removal is one way to support ac- and Kenya. In part, the questions aim to collect countability in providing the full range of an information on why and to what extent clients implant service (an area where donors could add attempt to have their implant removed but fail to pressure by requiring reporting on revisits and do so. Recently published data from this effort discontinuation). To better assess the extent to show that 4% of current implant users in Kenya which this full range of service—including and 7.2% in Ethiopia have attempted but failed to follow-up and removal—has been scaled, pro- have their implant removed.23,24 grams can employ both traditional tactics, such as Coordinated and systematic efforts to high- including follow-up and removal indicators in light and implement best practices in expanding facility registers and survey instruments, and new access to implant removal services could greatly ways, such as innovative monitoring of client benefit situations in which implant removals access through technology solutions (which could have not received commensurate attention or potentially also capture incidence of difficult are only now emerging as a problem area. At removals and indicators of quality). Although the global level, the Implants Access Program an increasing number of countries track implant Operations Group partnered with Jhpiego to sup- uptake through DHIS 2 and other health man- port 2 technical consultations on implant remov- agement information systems, few routinely als. In late 2015, the group initiated the Implant track removals; yet systematic capture of data Removal Task Force to bring together implement- on removals is integral to developing plans ing partners and donors to identify existing best and scaling up access to this essential service. practices and call attention to research and So too, we need to better understand reasons programming gaps for future action. The task for method switching and discontinuation.25 force also aims to bring awareness and ensure Altogether, these data could offer ministries and adequate attention to the issue of implant program managers visibility into the performance removal. While only in its infancy, the task force of their family planning programs, which will

Global Health: Science and Practice 2016 | Volume 4 | Number 3 369 Accessible Implant Removal Services www.ghspjournal.org

BOX. What Would Quality Implant Removal Services Look Like?

Supplies for implant removal are available at the point of service. Provider is competent and conﬁdent. Systems are in place for managing difﬁcult removals. Counseling, side-effect management, and resupply and switching are offered. Client knows when and where to go for removal. Service is available when client wants it, within a reasonable distance. Service is affordable (or free). Removal data are collected and monitored.

Source: Compiled from the Implant Removal Task Force of the Implants Access Program Operations Group.

help them align volumes of insertions with implant removals, through traditional and volumes of removals, and could reflect the quality non-traditional approaches. Currently, oppor- of care within their programs. With this is mind, tunities for clinical practice are limited because the data and monitoring subgroup of the Implant demand for removals at training events is Removals Task Force is developing an adaptable relatively low or nonexistent during this phase tool to assist countries in identifying and address- of introduction or rapid scale-up. This means ing implant removal trends and issues. Addition- few providers get the chance to practice in the ally, where possible globally managed surveys, supportive learning environment of a training. such as those conducted by PMA2020 and the Those who do leave training competent may Demographic and Health Surveys, should incor- face low client load upon return to their facility, porate questions on capacity to provide removal which leaves them with few opportunities to services, number of implant removals, their timing maintain their skill.7 These factors may also of removal in relation to the 3-to-5-year life-of-use, affect the quality of the removal and capacity the reasons for removal, and whether the client to identify and manage difficult removals. elected to use another implant or any other Reviewing training plans (including plans for method after removal of an implant. client mobilization) and expanding session Ultimately all inserted implants will need to times on removal practice is a starting point be removed, and thus the ability to offer quality while additional solutions are explored. removal services on demand is of inevitable im- 3. Plan ways to include implant provision portance. This readiness rests on our ability to: (insertion, follow-up, and removal) within the total health system. At a planning level, Ensure clients are well-informed. 1. Not only strategies, budgets, and costed implementa- should women be counseled at the time of tion plans should accommodate the equip- insertion on when, where, and how to access 26 ment and consumables, human resources, follow-up care and removal, but programs and trainings required to ensure availability should also explore how to make certain this of removal services, including ensuring that information is available on an ongoing basis services are free for the poor or that any fees and clients are aware when it is time to have levied are affordable. While it may not be the implant removed. Community mobilization, feasible for every level of the health system including use of community health workers, as or provider offering implants to provide re- well as social and behavior change communica- movals, planning will be needed to establish tion efforts may be effective here. removal services through a referral system or 2. Support providers to maintain compe- other approaches. Additionally, service deliv- tence and confidence in the removal ery approaches that expand access to im- procedure. We must find ways to overcome plants should include contingencies for access the barriers to providers’ ability to perform to removal services. For example, task shifting

Global Health: Science and Practice 2016 | Volume 4 | Number 3 370 Accessible Implant Removal Services www.ghspjournal.org

and mobile outreach have helped implants 3. Duvall S, Thurston S, Weinberger M, Nuccio O, meet demand for family planning and pene- Fuchs-Montgomery N. Scaling up delivery of contraceptive trate farther into rural and other un- implants in sub-Saharan Africa: operational experiences of Marie Stopes International. Glob Health Sci Pract. 2014;2(1): derserved communities, but these programs 72–92. CrossRef. Medline must think ahead when expanding access 4. World Health Organization (WHO). Medical eligibility for through means that do not guarantee con- contraceptive use. 5th edition. Geneva: WHO; 2015. Available tinuous and universal access to a competent from: http://www.who.int/reproductivehealth/publications/ provider for follow-up and removal. An opti- family_planning/MEC-5/en/ mal approach to expanding access to implant 5. Implant Access Program. Implant Access Program: expanding family planning options for women. [place unknown]: Implant removals likely involves a total market ap- Access Program; 2016. Available from: http://www. proach that leverages private and NGO familyplanning2020.org/resources/13386 partners and uses their comparative advan- 6. Kenya Health Information System (Kenya DHIS2) [Internet]. tages to ensure that the needs of all segments Nairobi (Kenya): Kenya Ministry of Health. [cited 2016 Mar 1]. of the population are met and that available Available from: https://hiskenya.org/dhis-web-commons/ security/login.action resources are maximized so the poor and 7. Ramchandran D, Upadhyay UD. Implants: the next generation. vulnerable are not left out. In: Population Reports, Series K, No. 7. Baltimore (MD): Johns Hopkins Bloomberg School of Public Health, INFO Project; The current scale-up of implants is the result of 2007. Available from: https://www.k4health.org/toolkits/info- coordinated efforts on the part of many groups publications/implants-next-generation and individuals, including providers, seeking to 8. Implants Access Program Operations Group. Meeting report: sustain the expanded choice of methods presented consultative roundtable on access to implant removals in the by implants. To prevent the ongoing scale-up from context of responsible scale-up. Oslo (Norway): IAP Operations being undermined, further investment and efforts Group; 2015. to measure and expand access to implant removal 9. Hubacher D, Dorflinger L. Avoiding controversy in international provision of subdermal contraceptive implants. Contraception. are needed, and the timing is right. We also call on 2012;85(5):432–433. CrossRef. Medline advocates to aid in focusing efforts and refining 10. World Health Organization (WHO). Ensuring human rights in our ‘‘asks’’ at the global, national, and subnational the provision of contraceptive information and services: guidance levels, such that change can be achieved thought- and recommendations. Geneva: WHO; 2014. Available from: fully and efficiently. http://apps.who.int/iris/bitstream/10665/102539/1/ 9789241506748_eng.pdf The Implant Removal Task Force is poised to 11. Family Planning 2020 (FP2020). Family Planning 2020: play an important role in collecting and dis- rights and empowerment principles for family planning. seminating knowledge on this topic; we encou- Washington (DC): United Nations Foundation; FP2020; 2014. rage all parties engaged in contraceptive implants Available from: http://www.familyplanning2020.org/ provision to look carefully at all aspects of pro- resources/4697 viding quality removal services—the client, the 12. Frost LJ, Reich MR. Chapter 6. Norplant: access to contraceptives. In: Frost LJ, Reich MR. Access: how do good provider, and the system—and take swift action health technologies get to poor people in poor countries? to ensure the availability of quality implant re- Cambridge (MA): Harvard Center for Population and moval services. Development Studies; 2008. p.115–140. Available from: http://www.accessbook.org/download_access.asp Acknowledgments: Critical contributions and review and were 13. Harrison PF, Rosenfield A. Research, introduction, and use: provided by the following individuals (in alphabetical order): advancing from Norplant. Contraception. 1998;58(6):323–334. Elaine Charurat (Jhpiego), Lila Cruikshank (Global Impact Advisors), CrossRef. Medline Ricky Lu (Jhpiego), Patricia MacDonald (USAID), Julia McDowell 14. Sivin I, Nash H, Waldman S. Recommendations for introducing (CHAI), Elaine Menotti (USAID), Sandra Novo (UNFPA), and Renee Van de Weerdt (UNFPA). Additional support was provided by Lindsay Jadelle into developing-country family planning programs: Breithaupt (Jhpiego) and Elizabeth Thompson (Jhpiego). lessons learned from the Norplant experience. In: Sivin I, Nash H, Waldman S. Jadelle levonorgestrel rod implants: a summary Competing Interests: None declared. of scientific data and lessons learned from programmatic experience. New York: Population Council; 2002. p. 19–21. Available from: http://pdf.usaid.gov/pdf_docs/ REFERENCES PNACP693.pdf 1. Family Planning 2020 (FP2020). FP2020 measurement annex: 15. Hull T. The challenge of contraceptive implant removals in East November 2015. Washington (DC): United Nations Foundation, Nusa Tenggara Indonesia. Int Fam Plan Perspect. 1998;24(4): FP2020; 2015. Available from: http://progress.familyplanning 176–205. Available from: https://www.guttmacher.org/pubs/ 2020.org/uploads/03/00/FP2020_MeasurementAnnex_ journals/2417698.html 2015_PrinterFriendly.pdf 16. Pillai M, Gazet AC, Griffiths M. Continuing need for and 2. RHInterchange [Internet]. New York: United Nations Population provision of a service for non-standard implant removal. Fund (UNFPA). 2000 - [cited 2016 Mar 1]. Available from: J Fam Plann Reprod Health Care. 2014;40(2):126–132. https://www.myaccessrh.org/rhi-home CrossRef. Medline

Global Health: Science and Practice 2016 | Volume 4 | Number 3 371 Accessible Implant Removal Services www.ghspjournal.org

17. USAID. Couple Years Protection [Internet]. Washington (DC): here: http://www.xcdsystem.com/ICFP2013/abstract/ USAID; 2011 - [cited 2016 Mar 1]. Available from: https:// pdfpresentations/Prg219.pdf www.usaid.gov/what-we-do/global-health/family-planning/ 22. Burke E, Gold J. Increasing access to voluntary family planning couple-years-protection-cyp and STI services for young people: the youth voucher program in 18. Jhpiego. Providing Contraceptive Implants Learning Resources Madagascar. London: Marie Stopes International; 2015. Package. Baltimore (MD): Jhpiego; 2015. Available from: Available from: https://mariestopes.org/data-research/ http://reprolineplus.org/resources/implants-LRP resources/increasing-access-voluntary-family-planning-and-sti- 19. Komwihangiro J. Expanding FP access for national impact: services-young-people scaling up mobile outreach in Tanzania. Presented at: The 23. Performance Monitoring and Accountability 2020 (PMA2020). Support for International Family Planning Organizations (SIFPO) PMA2015/Kenya: implant use and removal in Kenya. Baltimore End of Project Meeting; 2015 Sep 18; Washington, DC. (MD): PMA2020; 2016. Available from: http://pma2020.org/ Available from: http://mariestopes.org/sites/default/files/ sites/default/files/Kenya_Implants-Use-Removal-Memo-v7- Joseph%20Komwihangiro%20Tanzania%20ppt_External.pptx 2016.04.26-web.pdf 20. Pathfinder International. Strengthening health systems to ensure 24. Performance Monitoring and Accountability 2020 (PMA2020). equitable access to implant removal services in Ethiopia. Watertown PMA2016/Ethiopia: implant use and removal in Ethiopia. (MA): Pathfinder International; 2013. Available from: http://www. Baltimore (MD): PMA2020; 2016. Available from: http:// pathfinder.org/publications-tools/pdfs/Strengthening-Health- www.pma2020.org/sites/default/files/Ethiopia_Implants- Systems-to-Ensure-Equitable-Access-to-Implant-Removal-Services- Use-Removal-Memo-Draft-v7-2016.08.31-WEB.pdf in-Ethiopia_July-2013.pdf?x = 136&y = 27 25. Castle S, Askew I. Contraceptive discontinuation: reasons, 21. Yewondwossen T, Kidest L, Keseal D, Tariku N. Providing challenges, and solutions. New York: Population Council; Implanon services at the community level through training of 2015. Available from: http://www.familyplanning2020.org/ low-level care providers: a national scale-up program microsite/contraceptive-discontinuation experience. Presented at: The International Conference on Family 26. Bruce J. Fundamental elements of the quality of care: a simple Planning; 2013 Nov 12-15; Addis Ababa, Ethiopia. Available framework. Stud Fam Plann. 1990;21(2):61–91. CrossRef. Medline

Peer Reviewed

Received: 2016 Mar 30; Accepted: 2016 Jun 15; First Published Online: 2016 Aug 30

Cite this article as: Christoﬁeld M, Lacoste M. Accessible contraceptive implant removal services: an essential element of quality service delivery and scale-up. Glob Health Sci Pract. 2016;4(3):366-372. http://dx.doi.org/10.9745/GHSP-D-16-00096.

& Christoﬁeld and Lacoste. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi. org/10.9745/GHSP-D-16-00096.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 372 COMMENTARY

Using Qualitative Methods to Validate and Contextualize Quantitative Findings: A Case Study of Research on Sexual Behavior and Gender-Based Violence Among Young Swazi Women

Allison Ruark,a* Rebecca Fielding-Millerb*

Nesting qualitative data collection methods within quantitative studies improves results by assessing validity and providing depth and context. Using data from 3 sources from Swaziland, we triangulate qualitative and quantitative findings to highlight how different methodologies produce discrepant data regarding risky sexual behaviors among young women. We found that women reported similar numbers of lifetime sex partners in all sources, but the proportion reporting multiple and concurrent partnerships was several times higher in qualitative interviews. In addition, qualitative data can provide deeper understanding of how participants, such as those experiencing gender-based violence, understood the experiences behind the quantitative statistics.

Dr. Allison Ruark and Dr. Rebecca Fielding-Miller contributed equally to the writing of this article.

INTRODUCTION debate.3-7 From a post-positivist point of view, these phenomena are subjective by their very nature, M ost modern public health researchers in the making them impossible to precisely define and behavioral and social sciences situate their measure, especially in a way that is meaningful research within a post-positivist framework, either 1,2 across all contexts. explicitly or implicitly. Researchers working within a In this commentary, we consider the challenges of post-positivist framework assume that while objective collecting and interpreting data on sexual behavior and ‘‘ ’’ truths of human behavior and experience exist, gender-based violence (GBV). We present a case study measuring and defining these realities is at best an that illustrates challenges and potential solutions to approximate science. A physician or clinical researcher maximize data validity and describe these behaviors can measure blood pressure or CD4 count using and experiences as closely as possible. The comparisons precisely calibrated instruments and feel confident in and concepts come from our experience conducting the accuracy of the measurements, but quantifying 2 separate research studies in Swaziland in 2013–2014, aspects of human health and well-being is not so both of which characterized sexual behavior among simple. Swazi women in their 20s and 30s. We did not set out Social scientists and public health practitioners to collect comparable data, but we noticed that our face multiple challenges in determining how best to research studies (1 qualitative and 1 quantitative using measure social phenomena and various behaviors audio computer-assisted self-interviewing [ACASI]) relevant to public health and how to define precisely produced very different findings about sexual behav- what to measure. Efforts to conceptualize and assess ior in research populations that seemed to be quite important constructs such as self-efficacy, stigma, similar. Our data also differed markedly from the latest social norms, sexual identity, violence, and sexual Swaziland Demographic and Health Survey (DHS).8 behavior have generated a great deal of research and These observations led to further consideration of how different data collection methodologies and various a Brown University, Department of Medicine, Providence, RI, USA. sources of bias may influence the story that participants b University of California, San Diego, Division of Global Public Health, tell in a research interview, and how closely our San Diego, CA, USA. research findings reflect the ‘‘true’’ nature of behav- *Co-ﬁrst authors. Correspondence to Allison Ruark ([email protected]). iors in our study populations. We believe that frank

Global Health: Science and Practice 2016 | Volume 4 | Number 3 373 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

Measuring and consideration of the strengths and weaknesses of Comparing the Validity of Different deﬁning human data collection methodologies is critical to max- Methods of Collecting Data on Sexual behavior and imizing the validity of collected data and the Behavior and Gender-Based Violence experience is value of research. at best an Various studies provide evidence that study inter- ’ approximate viewers can influence participants reports of 15,16 17,18 science. COLLECTION OF BEHAVIORAL DATA sexual behavior or experiences of violence. In South Africa, respondents reported more con- The collection of behavioral data in public health servative sexual behavior (fewer lifetime sexual research rests on 2 assumptions. First, that re- partners and more condom use) to older inter- Frank search participants have life experiences and viewers, and men were especially likely to report consideration of engage in behaviors that influence their health fewer lifetime sexual partners to male inter- the strengths and risks and outcomes. Second, that data collected viewers.19 In Uganda, women were more likely to weaknesses of through behavioral research can measure the report sexual activity and willingness to use ‘‘ ’’ data collection true nature of these experiences and beha- condoms to male interviewers compared with methodologies viors with enough accuracy to be useful to female interviewers.20 In Malawi, adolescent girls is critical to interventions designed to mitigate health risks. were more likely to report having had sex when maximizing the The challenge of behavioral research is to asked by a nurse before testing for sexually validity of minimize the degree of error and bias, which is transmitted infections, compared with face-to-face inevitable in all research studies, but especially 10 collected data interviews or ACASI. In qualitative research, the in studies that use self-reported behaviors on 21 and the value of researcher is the instrument used to collect data, sensitive topics. research. and rapport between the researcher and partici- Collecting data through self-report is often pant is a critical aspect of data collection. necessary for sensitive topics such as sexual ACASI has frequently been employed to in- Using qualitative history, experience or perpetration of violence, or crease confidentiality and data validity in research methods to collect other phenomena for which observation is pro- on sexual behavior. ACASI has generally been found data on sensitive blematic or impossible. Self-reported data on to yield higher reports of some sensitive sexual topics, such as sensitive topics are subject to a number of well- behaviors, but not others, compared with face-to- sexual history or recognized potential biases, including social desir- face interviews,15,16,22,23 differing in some cases by violence, can ability bias, item response bias, reporting bias, respondent gender.24 Two 2010 reviews yielded 9 provide critical and recall bias. People may report their sexual somewhat different conclusions. Langhaug and 10 insight and behavior inconsistently over time, and self- colleagues concluded that there was ‘‘strong evi- context, resulting reports of sexual behavior have been found to dence’’ that computer-assisted interviewing increa- be inconsistent with biological data11 and reports in better sed reports of sensitive sexual behaviors in develop- of sexual partners.12 Many types of bias derive 25 interventions. ing countries. Phillips and colleagues conducted a from the data collection activity itself and are in- meta-analysis of data from low- and middle-income In qualitative fluenced by the methodology used and the skill countries (LMICs). They concluded that compared research, the and identity of the data collector. with face-to-face interviews, other methods did not researcher is the There are many reasons why research parti- consistently yield higher reports of ever having sex, instrument used to cipants may choose to represent their stories in non-condom use, or number of sexual partners, but 2 26 collect data, and research settings in a certain way. The interview did produce higher reports of forced sex. ‘‘ ’’ therefore rapport is a situated, social activity in which the person We identified few studies that compared the ‘‘ between the being interviewed produces, reproduces, and validity of various methodologies for collecting articulates’’ an identity, largely in response to rap- self-reported data on GBV experiences. Inter- researcher and 13 participant is a port with and perceptions of the interviewer, viewer training and skill is likely an important and in response to ‘‘situational, cognitive, social, factor, and without specific training on the nature critical aspect. and psychological factors’’.14 While there is likely of GBV and sexual assault, even highly trained no research methodology that can consistently research assistants may struggle with how to deliver data that perfectly represent reality, there categorize a particular event.27 Evaluations from are many good reasons to strive to improve the the United States and Canada suggest that ACASI validity of the data we collect. For example, using may capture more reports of intimate partner qualitative data collection methods to understand violence than face-to-face interviews,28 and that sexual risks and experiences of sexual violence in many women prefer disclosing these experiences a population can result in better interventions. to a computer rather than to another person.29

Global Health: Science and Practice 2016 | Volume 4 | Number 3 374 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

However, very little evidence exists on the relative findings about sexual behavior.35-39 While the The value of validity of using ACASI to measure self-reports of value of combining quantitative and qualitative combining violence in LMICs. In low-income communities in methods in research of violence has been recog- quantitative and 27 Bangalore, India, young married women reported nized, few studies using such a mixed-method qualitative significantly fewer experiences of domestic vio- approach in LMIC contexts were found in the methods in lence via ACASI than they did in face-to-face literature. Schatz and Williams note that many research of 30 interviews. In these contexts, face-to-face inter- researchers have called for mixed-methods re- violence has been views may have had higher disclosure because of search on topics related to gender, and issue a recognized, but their perceived cathartic value, or because of the specific call for qualitative studies to validate and few studies using potential of being connected to services in other- contextualize DHS data on gender inequality.40 this approach in wise low-resource settings.30 Qualitative research can inform the development lower- and Qualitative methods can increase opportunities of structured, quantitative questionnaires,41 esta- middle-income for building trust between an interviewer and a blish which words or phrases are locally under- countries were participant and contain the flexibility to enable the stood to refer to acts of violence such as rape or participant to co-construct the interview and intro- coerced sex,42 or aid researchers in navigating found in the duce new topics of inquiry.21 These attributes of complex cultural minefields as they ask sensitive literature. qualitative research may produce data that are questions about sex and violence.42 Qualitative richer, more nuanced, and more valid than data methods also provide context. For example, a collected through quantitative means. Studies of study of GBV in the Democratic Republic of the individual sexual behavior and sexual violence Congo used FGDs with women who had survived typically use in-depth interviews (IDIs) rather than violence to further explore topics addressed in a focus group discussions (FGDs). The privacy and quantitative survey. The open-ended nature of confidentiality of IDIs encourages participants to the FGDs enabled women to voice concerns and share their personal opinions on sensitive topics, priorities that had not been addressed in the whereas FGDs are more likely to capture data on quantitative survey instrument, resulting in sug- community norms, or what FGD participants believe gestions for further research.43 is socially acceptable to say in front of others.21,31,32 Despite the frequency with which qualitative Survivors of violence may construct their and quantitative methods are used in the same experiences in a variety of ways depending on project, we identified only 1 study that used quali- GBV studies may their cultural context, current life circumstances, tative data to validate quantitative sexual behavior particularly and the interview scenario itself, and therefore data. In Malawi, a qualitative study nested within a beneﬁt from a GBV studies may particularly benefit from a larger quantitative project found that more young qualitative qualitative approach that allows space for nuance women and men reported having ever had sex in approach that 33 and flexibility. Survivors may be reluctant to IDIs compared with face-to-face surveys; 39% of allows space for discuss experiences of violence out of shame, young women and 17% of men gave discrepant nuance and 13 particularly with survey interviewers with whom answers in the 2 interview modalities. In an ﬂexibility. they have little rapport.17,18 Survivors may not analysis of the IDIs, Poulin concluded that they recall their experiences, or they may reconstruct allowed for ‘‘flexibility and reciprocal exchange’’ what occurred in a way that distances them from that did not exist in the surveys, thus producing stigmatized identities (such as that of a rape or trust between the interviewer and participant and sexual assault victim).18 A qualitative exploration resultinginmoreaccuratereporting.13 Repeated in South Africa found that women who described IDIs may be especially effective in increasing non-consensual, coerced, or violent sexual experi- rapport, and may also be useful for collecting ences with intimate partners would frequently longitudinal data on the experiences of participants describe these experiences as disappointing, emo- over time.35,44-46 tionally hurtful, or traumatic, but rarely categorized them as rape and often attributed them to men’s CASE STUDY COMPARING DATA ON ‘‘ ’’ 34 natural sexual drives and entitlement. SEXUAL BEHAVIOR AND GBV FROM DIFFERENT SOURCES The Importance of Using Mixed Methods in Research on Sexual Behavior In this case study, we compare data from 3 sources: the Swaziland DHS 2006–2007; a quantita- It is not uncommon for large research trials to use tive ACASI survey of young women’s sexual qualitative data to contextualize quantitative histories that included questions on GBV; and

Global Health: Science and Practice 2016 | Volume 4 | Number 3 375 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

a qualitative interview-based study of young research, the survey instrument was tested for women’s sexual partnerships that also elicited face validity with content experts and Swazi data about GBV. We provide this case study as a colleagues, translated into siSwati, and back- practical example for public health researchers translated into English to check translation and practitioners who wish to integrate qualita- accuracy. It was then piloted in siSwati using tive methods into a quantitative study. We believe cognitive interviewing50 at an urban public health this approach can lead to better research and clinic. outcomes. Data collection was carried out using ACASI, with a young Swazi female research assistant Swaziland Demographic and Health narrating the audio track in siSwati. Participants Survey were systematically sampled from women await- ing antenatal services at public clinics. A young The 2006–2007 Swaziland DHS was a large, female Swazi research assistant coached partici- nationally representative survey carried out by pants on the use of laptop computers for the the Swaziland Central Statistics Office in partner- initial set of demographic questions and then ship with Macro International and the first and withdrew unless a participant requested assis- only DHS to be conducted in Swaziland.8 Trained tance. The final survey took about 45 minutes to Swazi data collectors administered face-to-face complete, with an approximately 54% response interviews with participants from all 4 regions of rate. Per the request of the Swaziland Scientific Swaziland between July 2006 and February 2007. and Ethics Committee (SEC), no incentives were The DHS report does not mention any effort to offered, which may have resulted in the low match interviewers to respondents by age or response rate. Participants were offered food, gender. The Woman’s Questionnaire took an drink, and childcare. Further details are available average of 2 hours to complete and included elsewhere.48 The SEC and Emory University questions about demographic characteristics and Institutional Review Board reviewed and ap- attitudes and behaviors related to fertility and proved the study protocol. health, including a sexual partner history extending to the 3 most recent sexual partners. Nearly 5,000 women ages 15 to 49 were included in the Qualitative Interviews DHS (a 94% response rate). In this case study we From June 2013 to September 2014, the first present weighted data for 2,767 women ages 20 author (AR) carried out a qualitative ethno- to 39 who reported ever having sex to increase graphic study of the transitions and trajectories comparability to other data presented. We had no of young Swazi adults’ sexual partnerships.51,52 role in collecting these data, but the first author Data presented here are from repeated in-depth (AR) extracted age-specific data from datasets life-course interviews with 14 Swazi women made available at www.dhsprogram.com. between the ages of 20 and 39. Participants were recruited from a central location in Mbabane, the Audio Computer-Assisted Self-Interview capital of Swaziland, and were purposively Survey sampled to provide variation in education level, Between February and June 2014, the second marital and relationship status, and place of author (RFM) conducted a quantitative survey residence (urban, peri-urban, and rural). Inter- with 406 pregnant women ages 18 to 42 accessing views were carried out in siSwati or in a mixture antenatal care in 1 rural and 1 urban public of English and siSwati by trained Swazi inter- health clinic.47,48 In this case study, we present a viewers who were themselves young women in sub-sample of 340 women ages 20 to 39. The their 20s and 30s. survey was part of a larger mixed-methods project Each woman was interviewed 3 to 5 times, designed to operationalize and measure how with the total average interview time per woman Swazi women conceptualize transactional sex. being over 3 hours and the average time between The survey instrument included questions about first and last interview being 9 months. Inter- lifetime and 12-month sexual partner history as views were semi-structured and addressed family well as questions adapted from the World Health backgrounds and sexual partnership history, with Organization’s violence against women instru- emphasis given to the chronology and overlap ment based on previous work conducted in of sexual partnerships. Each woman was en- South Africa.17,49 After formative and qualitative couraged to tell the story of each of her sexual

Global Health: Science and Practice 2016 | Volume 4 | Number 3 376 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

partnerships in as much detail as she was willing interviews did so. The proportion of women to divulge. Further details about the methodology who reported 2 or more sexual partners in the of this study are provided elsewhere.51,52 The SEC past 12 months among qualitative interview and the Institutional Review Board of the Miriam participants was an order of magnitude greater Hospital (Providence, RI, USA) approved this than the proportion reporting multiple partners study. among ACASI and DHS participants. Similarly, participants in the qualitative interviews were USING QUALITATIVE DATA FOR several times more likely to report having concurrent partners in the past 12 months than were TRIANGULATION: SEXUAL BEHAVIOR participants in the ACASI survey. In Figure 1, Figure 2, and Figure 3, we present We do not argue that these data are directly sexual history data derived from the 3 sources comparable, and we have intentionally pre- described. Triangulating research findings from sented them visually rather than numerically so different sources provides a validity check to all as not to invite statistical comparison. Calculat- In all 3 data data sources. We noted that women reported ing the magnitude and significance of differ- sources, women similar numbers of lifetime sexual partners in all ences between data from these discrepant reported similar surveys, with a very similar proportion of parti- sources would be epistemologically and statis- numbers of cipants reporting 1 or 2 lifetime sexual partners in tically inappropriate. These data were drawn lifetime sexual ACASI and DHS data, and only a small minority from different populations at different points partners, but the in all 3 surveys reporting 5 or more lifetime sexual over a decade, and with somewhat different proportion of partners. The proportion of women reporting inclusion criteria. By definition, all women women reporting multiple and concurrent sexual partnerships in participating in the ACASI study had reported multiple and at least 1 sexual partner in the preceding qualitative interviews was several times that concurrent observed in the quantitative surveys, however. 12 months. The ACASI and qualitative studies partnerships in A substantial minority of women reported only also used convenience samples while the DHS qualitative 1 lifetime sexual partner in both ACASI and DHS data attempted to create a nationally repre- interviews was data, but no participants in the qualitative sentative sample. several times that reported in quantitative FIGURE 1. Number of Sexual Partners, Lifetime surveys.

Abbreviations: ACASI, audio computer-assisted self-interviewing; DHS, Demographic and Health Survey.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 377 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

FIGURE 2. Number of Sexual Partners, Past 12 Months

Abbreviations: ACASI, audio computer-assisted self-interviewing; DHS, Demographic and Health Survey.

FIGURE 3. Concurrent Sexual Partners, Past 12 Months

Abbreviation: ACASI, audio computer-assisted self-interviewing.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 378 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

All data describe young, sexually experi- who reported coerced or forced first sex. While we The differences enced women between the ages of 20 and 39 in deliberately present these as a diagram to dis- between the Swaziland. We believe the observed differences courage direct statistical comparisons that would 3 sources strongly between the 3 sources are striking and strongly be inconsistent with the nature of the data, we do suggest that suggest that qualitative methods may produce note that just over one-third of women in both qualitative higher reports of sensitive sexual behaviors samples described their first sexual experience methods may than do standard quantitative surveys. We as- as forced or coerced. This suggests that a well- produce higher sume that Swazi women will be highly unlikely implemented ACASI survey can produce similar reports of to over-report socially stigmatized behaviors levels of disclosure as same-gender, face-to-face sensitive sexual (such as multiple and concurrent sexual part- interviews with strong rapport—the suggested behaviors than nerships), and therefore that data showing best practice for collecting data on violence against quantitative higher levels of socially stigmatized behaviors women.17,53 surveys. are more accurate. In addition to this validity check of the The qualitative methods may have produced quantitative ACASI data, triangulating survey Triangulating higher reports of multiple and concurrent sexual findings with qualitative data provides a deeper quantitative partners for 2 reasons. First, an in-depth interview understanding of how participants understood ﬁndings with enables a conversation between an interviewer and the experiences behind the statistics, and how qualitative data a participant that elicits a detailed story rather their understanding may have shifted over time. provides a deeper than isolated points of data, reducing the possibi- Just less than half of ACASI respondents reported understanding of 21 ‘‘ ’’ lity of misunderstanding. The longitudinal and acquiescing to sex after a partner persuaded or how participants iterative nature of the research allowed inter- ‘‘ ’’ begged them (Figure 4). However, from the understood the viewers to probe and confirm information over ACASI survey alone we do not know precisely experiences behind multiple interviews (in some cases gently challen- how women may have experienced an encounter the statistics, and ging reports that seemed inconsistent or lacking in that they later labeled as ‘‘persuasion.’’ Restric- how their credibility), and to detect circumstances of risk tive cultural norms may lead participants to select understanding (such as concurrent sexual partners) that may the ‘‘persuaded’’ option to describe a fully may have shifted not have emerged in a once-off interview. Second, consensual and enthusiastic encounter if they repeated interviews and the prolonged nature of feel it is culturally unacceptable for women to over time. the relationship between interviewer and partici- express strong sexual desire.54 Conversely, pant created trust and rapport, which we believe women who reported being ‘‘persuaded’’ by a In-depth interviews increased participants’ willingness to reveal sen- partner could also be revising traumatic events elicit detailed sitive information. In many cases, additional because they feel shame admitting to experiences stories that can interviews increased frankness and disclosure as of sexual violence. In the qualitative data, some reduce the interviewers built rapport with participants over women recast violent or coercive events into acts possibility of mis- time, resulting in reporting of additional sexual of love or desire, such as this account of a understanding, partners. woman’s first sexual partner: and repeated He used to overpower me, to be honest. We didn’t interviews USING QUALITATIVE DATA FOR have sex because we were in love y He took can increase ’ CONTEXTUALIZATION advantage of me and I could see that he wanted to participants willingness to In the qualitative and ACASI studies, we asked have sex with me and I refused. He said that I ’ y reveal sensitive women how they would describe their first sexual couldn t refuse now and he carried on I got used y information. experience. The qualitative study focused on to him even though I was scared of him He saw participants’ own interpretation of their experi- that so he tried to bring me closer by apologizing ence, whereas the ACASI study asked them to and the relationship was okay from there. select one of multiple preexisting options: ‘‘Iwantedto,’’ ‘‘I was persuaded,’’ ‘‘Iwastricked,’’ While this participant might choose the ‘‘I was forced’’,or‘‘Iwasraped.’’ Acomparisonof category ‘‘I was persuaded,’’ given limited results is shown in Figure 4, with each bubble response options in a quantitative survey about plotted on the vertical axis according to the pro- her first sexual encounter, her account suggests portion of women who reported each outcome, the difficulty of subsuming complex experiences and with the size of each bubble representative of into a single descriptor or category. We suggest the absolute number of women. For the qualitative there is a need for nested qualitative research to data, a brief quote is presented for each woman build context and ‘‘thick’’ description55—rich,

Global Health: Science and Practice 2016 | Volume 4 | Number 3 379 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

FIGURE 4. Experiences of Young Swazi Women During First Sex, Qualitative Research Findings Compared With Quantitative ACASI Study Findings

Abbreviation: ACASI, audio computer-assisted self-interviewing. Each quote is plotted on the vertical axis according to the proportion of women who reported each outcome. For the qualitative data, 5 of 14 women reported coerced or forced ﬁrst sex, and a single quote represents an individual woman’s account. The size of each bubble represents the absolute number of women reporting each outcome.

contextualized description of human behavior— sequential explanatory design,57 which gives prior- into larger observational and survey-style studies ity to quantitative data but uses qualitative data to on subjects such as GBV. provide validation and insight into the meaning of the quantitative data (contextualization).

RECOMMENDATIONS Based on a comparison of multiple data sources Validation from Swaziland, we suggest that qualitative meth- For topics that may benefit from better rapport We recommend ods have an important role to play in research between an interviewer and participant, and the nesting qualitative studies, including surveillance, observational, and opportunity to probe or revisit topics over the course data collection experimental studies. Formative qualitative work of an interview, we recommend systematically within before and during a quantitative survey may sampling participants from the quantitative survey quantitative identify potentially unclear questions and language, and inviting them to participate in a qualitative studies of improving the quality of the survey questions and interview on the same topic. Although qualitative 56 sensitive topics final interpretation of the data. We also recom- research is not traditionally used to generate such as sexual mend, whenever possible, nesting qualitative data statistics, data from a systematically sampled, rep- behavior and GBV. collection within quantitative studies of sensitive resentative subsample could provide a useful vali- topics such as sexual behavior and GBV, in a dation check on the larger quantitative project.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 380 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

Contextualization requiring intense and specific training to assure For sensitive or ambiguous topics, in addition to data quality.58 As we discuss in this commentary, Purposively rigorous qualitative formative work to build valid quantitative methods also have substantial weak- sampling a survey instruments, we recommend purposively nesses; they lack the flexibility and iterative subsection of sampling a subsection of participants who have approach of qualitative research and cannot informants who participated in the quantitative data collection detect or correct for the distance between what participated in a ‘‘ ’’ process, or from a similar population, to better a participant reports and the truth. Mixed- quantitative data understand the context and potential shifting method approaches have the potential to enable collection process meanings within a survey item. Both the meth- qualitative and quantitative methods to work can help together in complementary and synergistic ways, odology and underlying philosophy of qualitative researchers better research provide the flexibility to understand and resulting in higher-quality research. understand report the sometimes ambiguous data that result as context and participants construct and reconstruct traumatic or potential shifting sensitive experiences. The goal in this case is not to CONCLUSION meanings. compare and contrast qualitative and quantitative In this commentary, we present a case study findings, but rather to continue with qualitative comparing 3 sources of data on sexual behav- investigation until the qualitative data have pro- ior and GBV experiences of young women in vided as rich, nuanced, and complete an under- Swaziland. We highlight discrepant findings not standing of the quantitative data as possible. for the purpose of statistical comparison, but as a means of discussing the importance of data Rigor collection methodology and the unique strengths of qualitative methods in providing validation The process of designing a qualitative study, or and contextualization for quantitative data. The conducting qualitative interviews or focus groups, higher frequency of multiple and concurrent requires specific skill sets and explicit training. sexual partnerships and the rich description of Acquiring credible, dependable, and confirm- GBV provided in the qualitative study suggest able58 qualitative data to complement quantita- that qualitative methods may more closely tive data requires careful thought and an approach the ‘‘truth’’ of certain behaviors and understanding of why and how a given qualita- experiences. Our objective in this commentary is tive method (i.e., IDIs, FGDs, observation) is not to offer definitive answers regarding sexual best suited to the question. It is also critically behavior and GBV among young women in important to select interviewers whose age, Swaziland, but to raise questions—and offer gender, social background, and life experiences suggestions—about how research might better enable them to create the right kind of rapport capture sensitive behaviors and experiences. We with interview participants. Qualitative inter- Qualitative argue that qualitative methods are critical and viewers require training specific to qualitative interviewers underused in validating and contextualizing approaches and methods to help them build require training data collected through quantitative methods. rapport with a participant, feel confident deviat- speciﬁc to ing from interview guides when appropriate, and qualitative probe deeply to draw out participant stories. Acknowledgments: We thank the Swazi women who participated approaches and in our research and contributed their experiences and insights. We also thank Nonhlanhla Mazibuko, Lunga Dlamini, Mphumi methods to Ncongwane, and Cynthia Vilakati for their assistance. Dr. Ruark successfully draw LIMITATIONS OF QUALITATIVE AND acknowledges support from National Institute on Drug Abuse (NIDA) grant T32DA13911 and from the New Paradigm Fund. out participant QUANTITATIVE METHODS Dr. Fielding-Miller was supported ﬁnancially and materially by a stories. US Fulbright Grant, the USAID-funded Swaziland Health Despite the importance of qualitative methods, Community Capacity Collaborative, and the Center for AIDS particularly IDIs, we note that they are not ap- Research at Emory University (P30AI050409). Support in preparing this manuscript was provided by NIDA grants propriate for all research objectives nor are they T32DA023356 and R21DA039782. the panacea for all data quality issues. Qualitative Competing Interests: None declared. methods are not intended to produce generalizable statistical inferences, and they are time and energy intensive, making qualitative studies with large REFERENCES numbers of participants impractical. The iterative 1. Clark AM. The qualitative-quantitative debate: moving from nature of data collection and analysis is also inher- positivism and confrontation to post-positivism and reconciliation. ently dependent on the researcher-as-instrument, J Adv Nurs. 1998;27(6):1242–1249. CrossRef. Medline

Global Health: Science and Practice 2016 | Volume 4 | Number 3 381 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

2. Tebes JK. Community science, philosophy of science, and the 20. Schopper D, Doussantousse S, Orav J. Sexual behaviors relevant practice of research. Am J Community Psychol. 2005;35(3-4): to HIV transmission in a rural African population: how much can 213–230. CrossRef. Medline a KAP survey tell us. Soc Sci Med. 1993;37(3):401–412. 3. Yang LH, Kleinman A, Link BG, Phelan JC, Lee S, Good B. Culture CrossRef. Medline and stigma: adding moral experience to stigma theory. Soc Sci 21. Creswell JW. Qualitative inquiry and research design: choosing Med. 2007;64(7):1524–1535. CrossRef. Medline among five approaches. 3rd edition. Thousand Oaks (CA): 4. Schutt RK. Conceptualization and measurement. In: Investigating SAGE; 2013. the social world: the process and practice of research. 4th ed. 22. van der Elst EM, Okuku HS, Nakamya P, Muhaari A, Davies A, Thousand Oaks, CA: Pine Forge Press; 2004. p. 85–126. McClelland RS, et al. Is audio computer-assisted self-interview 5. Weinstein ND. Misleading tests of health behavior theories. (ACASI) useful in risk behaviour assessment of female and male Ann Behav Med. 2007;33(1):1–10. CrossRef. Medline sex workers, Mombasa, Kenya? PLoS One. 2009;4(5):e5340. CrossRef. Medline 6. Maibach E, Murphy DA. Self-efficacy in health promotion research and practice: conceptualization and measurement. 23. Kelly CA, Soler-Hampejsek E, Mensch BS, Hewett PC. Social Int Q Community Health Educ. 1995;10(1):37–50. CrossRef desirability bias in sexual behavior reporting: evidence from an interview mode experiment in rural Malawi. Int Perspect Sex 7. Link BG, Phelan JC. Conceptualizing Stigma. Annu Rev Sociol. Reprod Health. 2013;39(1):14–21. CrossRef. Medline 2001;27(1):363–385. CrossRef. 24. Dolezal C, Marhefka SL, Santamaria EK, Leu CS, Brackis-Cott E, 8. Central Statistical Office [Swaziland]; Macro International Inc. Mellins CA. A comparison of audio computer-assisted self- Swaziland Demographic and Health Survey 2006-07. Mbabane, interviews to face-to-face interviews of sexual behavior among Swaziland: Central Statistical Office; 2008. Co-published by perinatally HIV-exposed youth. Arch Sex Behav. 2012;41(2): Macro International Inc. 401–410. CrossRef. Medline 9. Fenton KA, Johnson AM, McManus S, Erens B. Measuring sexual behaviour: methodological challenges in survey research. Sex 25. Langhaug LF, Sherr L, Cowan FM. How to improve the validity of Transm Infect. 2001;77(2):84–92. CrossRef. Medline sexual behaviour reporting: systematic review of questionnaire delivery modes in developing countries. Trop Med Int Health. 10. Palen LA, Smith EA, Caldwell LL, Flisher AJ, Wegner L, 2010;15(3):362–381. CrossRef. Medline Vergnani T. Inconsistent reports of sexual intercourse among South African high school students. J Adolesc Health. 2008; 26. Phillips AE, Gomez GB, Boily MC, Garnett GP. A systematic 42(3):221–227. CrossRef. Medline review and meta-analysis of quantitative interviewing tools to investigate self-reported HIV and STI associated behaviours 11. Minnis AM, Steiner MJ, Gallo MF, Warner L, Hobbs MM, in low- and middle-income countries. Int J Epidemiol. 2010; van der Straten A, et al. Biomarker validation of reports of 39(6):1541–1555. CrossRef. Medline recent sexual activity: results of a randomized controlled study in Zimbabwe. Am J Epidemiol. 2009;170(7):918–924. 27. Testa M, Livingston JA, VanZile-Tamsen C. Advancing the study of CrossRef. Medline violence against women using mixed methods: integrating qualitative methods into a quantitative research program. Violence Against 12. Helleringer S, Kohler HP, Kalilani-Phiri L, Mkandawire J, Women. 2011;17(2):236–250. CrossRef. Medline Armbruster B. The reliability of sexual partnership histories: implications for the measurement of partnership 28. Rhodes KV, Lauderdale DS, He T, Howes DS, Levinson W. concurrency during surveys. AIDS. 2011;25(4):503–511. ‘‘Between me and the computer’’: increased detection of intimate CrossRef. Medline partner violence using a computer questionnaire. Ann Emerg Med. 2002;40(5):476–484. CrossRef. Medline 13. Poulin M. Reporting on first sexual experience: the importance of interviewer-respondent interaction. Demogr Res. 2010;22(11): 29. MacMillan HL, Wathen CN, Jamieson E, Boyle M, McNutt L, 237–288. CrossRef. Medline Worster A, et al. McMaster Violence Against Women Research 14. Gribble JN, Miller HG, Rogers SM, Turner CF. Interview mode Group. Approaches to screening for intimate partner violence in and measurement of sexual behaviors: Methodological issues. health care settings: a randomized trial. JAMA. 2006;296(5): J Sex Res. 1999;36(1):16–24. CrossRef. Medline 530–536. CrossRef. Medline 15. Mensch BS, Hewett PC, Gregory R, Helleringer S. Sexual 30. Rathod SD, Minnis AM, Subbiah K, Krishnan S. ACASI and behavior and STI/HIV status among adolescents in rural Malawi: face-to-face interviews yield inconsistent estimates of domestic an evaluation of the effect of interview mode on reporting. Stud violence among women in India: The Samata Health Study Fam Plann. 2008;39(4):321–334. CrossRef. Medline 2005-2009. J Interpers Violence. 2011;26(12):2437–2456. CrossRef. Medline 16. Hewett PC. Consistency in the reporting of sexual behaviour by adolescent girls in Kenya: a comparison of interviewing methods. 31. Maxwell JA. Qualitative research design: an interactive Sex Transm Infect. 2004;80(Suppl 2):ii43–ii48. CrossRef. approach. 3rd edition. Thousand Oaks (CA): SAGE; 2013. Medline 32. Hennink M, Hutter I, Bailey A. Qualitative research methods. 17. Ellsberg M, Heise L. Researching violence against women: Thousand Oaks (CA): SAGE; 2011. a practical guide for researchers and activists. Washington (DC): 33. Campbell C, Mannell J. Conceptualising the agency of highly World Health Organization; 2005. Co-published by PATH. marginalised women: Intimate partner violence in extreme settings. Available from: http://apps.who.int/iris/bitstream/10665/ Glob Public Health. 2016;11(1-2):1–16. CrossRef. Medline 42966/1/9241546476_eng.pdf 34. Stern E, Buikema R, Cooper D. South African women’s 18. Cook SL, Gidycz CA, Koss MP, Murphy M. Emerging issues in the conceptualisations of and responses to sexual coercion in relation measurement of rape victimization. Violence Against Women. to hegemonic masculinities. Glob Public Health. 2016;11(1-2): 2011;17(2):201–218. CrossRef. Medline 135–152. CrossRef. Medline 19. Houle B, Angotti N, Clark SJ, Williams J, Go´mez-Olive´ FX, 35. Jewkes R, Wood K, Duvvury N. ‘I woke up after I joined Stepping Menken J, et al. Let’s talk about sex, maybe: interviewers, Stones’: meanings of an HIV behavioural intervention in rural respondents, and sexual behavior reporting in rural South Africa. South African young people’s lives. Health Educ Res. 2010; Field Methods. 2016;28(2):112–132. CrossRef 25(6):1074–1084. CrossRef. Medline

Global Health: Science and Practice 2016 | Volume 4 | Number 3 382 Using Qualitative Methods to Improve Quantitative Findings www.ghspjournal.org

36. Maman S, van Rooyen H, Stankard P, Chingono A, Muravha T, Available from: http://www.nswp.org/sites/nswp.org/files/ Ntogwisangu J, et al.; NIMH Project Accept (HPTN 043) study Zalwango%20et%20al.%20Parenting%20and%20Money% team. NIMH Project Accept (HPTN 043): results from in-depth 20Making%20Sex%20Work%20and%20Women%E2%80%99s% interviews with a longitudinal cohort of community members. 20Choices%20in%20Urban%20Uganda.pdf PLoS One. 2014;9(1):e87091. CrossRef. Medline 47. Fielding-Miller R, Dunkle KL, Cooper HLF, Windle M, Hadley C. 37. Wight D, Plummer M, Ross D. The need to promote behaviour Cultural consensus modeling to measure transactional sex in change at the cultural level: one factor explaining the limited Swaziland: scale building and validation. Soc Sci Med. impact of the MEMA kwa Vijana adolescent sexual health 2016;148:25–33. CrossRef. Medline intervention in rural Tanzania. A process evaluation. BMC Public 48. Fielding-Miller R, Dunkle KL, Jama-Shai N, Windle M, Hadley C, Health. 2012;12(1):788. CrossRef. Medline Cooper HLF. The feminine ideal and transactional sex: navigating 38. Higgins JA, Mathur S, Eckel E, Kelly L, Nakyanjo N, respectability and risk in Swaziland. Soc Sci Med. 2016;158: Sekamwa R, et al. Importance of relationship context in HIV 24–33. CrossRef. Medline transmission: results from a qualitative case-control study in 49. Dunkle KL, Jewkes RK, Nduna M, Levin J, Jama N, Khuzwayo N, Rakai, Uganda. Am J Public Health. 2014;104(4):612–620. et al. Perpetration of partner violence and HIV risk behaviour CrossRef. Medline among young men in the rural Eastern Cape, South Africa. AIDS. 39. Mojola SA, Williams J, Angotti N, Go´mez-Olive´ FX. HIV after 40 2006;20(16):2107–2114. CrossRef. Medline in rural South Africa: A life course approach to HIV vulnerability 50. Beatty PC, Willis GB. Research synthesis: the practice of cognitive among middle aged and older adults. Soc Sci Med. 2015;143: interviewing. Public Opin Q. 2007;71(2):287–311. CrossRef 204–212. CrossRef. Medline 51. Ruark A, Dlamini L, Mazibuko N, Green EC, Kennedy C, Nunn 40. Schatz E, Williams J. Measuring gender and reproductive health A, Flanigan T, Surkan PJ. Love, lust and the emotional context of in Africa using demographic and health surveys: the need for multiple and concurrent sexual partnerships among young Swazi mixed-methods research. Cult Health Sex. 2012;14(7):811–826. adults. Afr J AIDS Res. 2014. 13(2):133–43. CrossRef. Medline CrossRef. Medline 52. Ruark A, Kennedy CE, Mazibuko N, Dlamini L, Nunn A, 41. Jewkes R, Vundule C, Maforah F, Jordaan E. Relationship Green EC, Surkan PJ. From first love to marriage and maturity: dynamics and teenage pregnancy in South Africa. Soc Sci Med. a life-course perspective on HIV risk among young Swazi adults. 2001;52(5):733–744. CrossRef. Medline Cult Health Sex. 2016;18(7):812–25. CrossRef. Medline 42. Swiss S, Jennings PJ, Aryee GV, Brown GH, Jappah-Samukai RM, 53. Ellsberg M, Heise L, Penã R, Agurto S, Winkvist A. Researching Kamara MS, et al. Violence against women during the Liberian civil domestic violence against women: methodological and ethical conflict. JAMA. 1998;279(8):625–629. CrossRef. Medline considerations. Stud Fam Plann. 2001;32(1):1–16. CrossRef. Medline 43. Kelly JT, Betancourt TS, Mukwege D, Lipton R, VanRooyen MJ. 54. Schippers M. Recovering the feminine other: masculinity, femininity, Experiences of female survivors of sexual violence in eastern and gender hegemony. Theory Soc. 2007;36(1):85–102. CrossRef Democratic Republic of the Congo: a mixed-methods study. 55. Geertz C. Thick description: toward an interpretive theory of Confl Health. 2011;5(1):25. CrossRef. Medline culture. In: The interpretation of cultures: selected essays. 44. Harrison A, O’Sullivan LF. In the absence of marriage: long-term New York: Basic Books; 1973. p. 3–30. concurrent partnerships, pregnancy, and HIV risk dynamics 56. Nichter M, Nichter M, Thompson PJ, Shiffman S, Moscicki AB. among South African young adults. AIDS Behav. 2010; Using qualitative research to inform survey development on 14(5):991–1000. CrossRef. Medline nicotine dependence among adolescents. Drug Alcohol Depend. 45. Gysels M, Pool R, Nnalusiba B. Women who sell sex in a 2002;68(Suppl 1):41–56. CrossRef. Medline Ugandan trading town: life histories, survival strategies and risk. 57. Creswell J. Research design: qualitative, quantitative, and mixed Soc Sci Med. 2002;54(2):179–192. CrossRef. Medline methods approaches. 2nd edition. Thousand Oaks (CA): SAGE; 46. Zalwango F, Eriksson L, Seeley J, Nakamanya S, Vandepitte J, 2003. Grosskurth H. Parenting and money making: sex work and 58. Lincoln YS, Guba EG. Naturalistic inquiry. 1st edition. Thousand women’s choices in urban Uganda. Wagadu. 2010;8:71–92. Oaks (CA): Sage; 1985.

Peer Reviewed

Received: 2016 Feb 24; Accepted: 2016 Jun 8

Cite this article as: Ruark A, Fielding-Miller R. Using qualitative methods to validate and contextualize quantitative ﬁndings: a case study of research on sexual behavior and gender-based violence among young Swazi women. Glob Health Sci Pract. 2016;4(3):373-383. http://dx.doi. org/10.9745/GHSP-D-16-00062.

& Ruark et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi.org/ 10.9745/GHSP-D-16-00062.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 383 COMMENTARY

Vouchers: A Hot Ticket for Reaching the Poor and Other Special Groups With Voluntary Family Planning Services

Elaine P Menotti,a Marguerite Farrell a

WHAT ARE VOUCHERS? systems and processes; identify and accredit providers and outlets to participate in the program; ouchers are a form of results-based financing that and provide training on voucher components and Vhave been used in many sectors, including the health 1,2 requirements to key players including providers sector, in low-, middle-, and high-income countries. and voucher distributors. In addition, the voucher Vouchers work as both financing mechanisms to ensure management agency is engaged in monitoring and equity and programmatic tools to reduce barriers to access oversight of voucher distribution and service deliv- and increase use of critical health services. They are ery on an ongoing basis. paper or electronic tickets that are distributed or sold to segments of the population who exchange them for 2. The voucher management agency provides vouchers health services at accredited facilities. To be accredited, to trained distributors, such as NGOs and commu- a provider or outlet is generally reviewed against certain nity health workers. facility requirements and quality standards. When accom- 3. Voucher distributors counsel prospective clients and, panied by social and behavior change activities and if appropriate, give or sell vouchers (for a nominal quality assurance approaches, including training, mon- amount) to a defined segment of the population, itoring, supportive supervision, and site improvements, for example, the poor, youth, and pregnant or post- voucher programs can increase uptake of health services partum women. and improve service quality. A voucher program can 4. Clients visit preapproved, quality-assured health care also prepare a health system for strategic purchasing providers for products or services covered by the (e.g., insurance and contracting), engage the private voucher program. sector, and protect the poor and other special groups. In the last decade, many countries in Asia and Africa have 5. Providers give clients products or services free of introduced medium- and large-scale voucher programs. charge in exchange for a voucher. We have an opportunity to learn from their experiences. 6. Providers then submit their claims to the voucher management agency for processing and reimbursement of services, according to a defined verification HOW DO VOUCHERS WORK? process. Once established, a voucher program involves a series of 7. The voucher management agency reimburses pro- – transactions between key players.1,3 6 Key transactions viders after verifying service provision. include the following, as illustrated by the numbered 8. The voucher management agency monitors, reviews, arrows in Figure 1: and examines data and submits reports to the donor/ 1. A government or donor provides funding to the governance structure. voucher management agency to establish and run Some voucher programs also use an external verifi- a United States Agency for International Development, Washington, DC, USA. cation body as an additional layer of monitoring out- Correspondence to Elaine P Menotti ([email protected]). side of the role of the voucher management agency.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 384 Vouchers and Family Planning Services www.ghspjournal.org

FIGURE 1. How Does It Work? A Flow Diagram of Voucher Transactions

Source: Adapted from the World Bank 2005,3 Islam 2006,4 and Grainger et al. 2014.1

This external verification body conducts audits design features of a voucher system cover 4 main at regular intervals to ensure vouchers are areas: A voucher program being used appropriately and provider claims are should ideally Foundational accurate. elements include the source have the resources The objective of a voucher program is to of funding, generally donor or loan funds but to operate at a address key barriers to accessing and using health could also be the government; program objec- medium to large services, especially among vulnerable popula- tives and time frame; and the governance level of scale, tions. It should have the resources to operate at structure, often an advisory board comprised of given the systems a medium to large level of scale, given the government, donors, and other stakeholders. investments substantial systems investments required to set Management begins with identifying a capable required to set it it up and operate. As outlined in Figure 2, the key voucher management agency (government, up and operate.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 385 Vouchers and Family Planning Services www.ghspjournal.org

these) of health care facility to include; recruiting, accrediting, and retaining qualified pro- FIGURE 2. Key Features of a Voucher System viders to participate in the voucher program; and identifying and establishing service packages and pricing. The design of the voucher program should incorporate ongoing quality assurance inputs and support to providers so that the services covered by the program achieve quality standards and revenue from vouchers can be reinvested into quality improvements. Client factors include identifying key segments of the population and their need for services, how much to charge for a voucher (always below the market rate) or whether to distribute the vouchers for free, how to distribute vouchers, which services to include, how to promote the services and the program to the intended population to generate demand, and how to deliver services. Vouchers may focus on a single health area, such as family planning, but could include multiple services in that area such as counseling, method provision, a follow-up visit, and method removal (as relevant). Alternatively, vouchers can include a package of services such as antenatal care, institutional delivery, postpartum services, child health, and family planning. NGO, or commercial entity) that establishes Design of systems and processes for selection and THE ADVANTAGES OF A WELL- the voucher accreditation of health care facilities; provider IMPLEMENTED VOUCHER PROGRAM program should quality assurance; designing, printing (if using Voucher programs for family planning services incorporate paper vouchers), and delivering vouchers to have been implemented successfully in a range of ongoing quality distributors; claims processing and reimbursement; mitigation and control of fraud; and ongoing settings where financial, information, and other assurance inputs monitoring and reporting to the donor/ barriers impede access to and use of modern and support to 7,8 governance structure. Alternatively, these func- contraceptives. For example, Box 1 illustrates providers so that tions could be executed by multiple agencies how a voucher program in Uganda increased covered services to leverage existing local capacity. Manage- uptake of family planning services, particularly achieve quality ment of the voucher system may include of voluntary long-acting reversible contraceptives, standards and external verification to ensure transparency in private franchised clinics by removing bar- revenue from and fraud mitigation. This is particularly com- riers to access. Box 2 highlights two examples vouchers can mon when the voucher management agency of voucher programs that provided family plan- be reinvested participates in service provision and imple- ning and reproductive health services for youth. into quality mentation, for example, when social franchise A review of available literature and implementa- improvements. organizations (typically NGOs) serve as the tion experience shows that vouchers can be an voucher management agency while using effective programmatic tool as well as a financing their franchised health clinic networks and mechanism for family planning and reproductive 2,7,9,13,14 existing contracts with health care providers health. Key benefits and comparative to deliver services in voucher programs. advantage of voucher programs are summarized in the Table and outlined in more detail below. Provider factors include determining the type, level, location, and sector (public, private Target health care subsidies toward the for-profit, private not-for-profit, or a mix of poor and other special groups. Vouchers,

Global Health: Science and Practice 2016 | Volume 4 | Number 3 386 Vouchers and Family Planning Services www.ghspjournal.org

BOX 1. Voucher Program in Uganda Increases Access to and Use of Long-Acting Reversible Contraceptives From 2011 to 2014, Marie Stopes Uganda implemented a voucher program in private franchised clinics to provide family planning services to more than 325,000 clients; 66% were not using a contraceptive method before the program and nearly 80% had no education or only primary education (a proxy indicator for low income).9 The voucher program offered all methods of contraceptives; however, the majority (94.9%) chose long-acting reversible contraceptives, which have limited availability in the public sector and may be too expensive for some clients. Estimates show that the voucher program increased modern contraceptive prevalence by nearly 1.5 percentage points. It is clear that reducing ﬁnancial and other access barriers successfully increased uptake of family planning services.

BOX 2. Spotlight on Voucher Programs for Youth In Madagascar, a voucher program implemented between July 2013 and December 2014 provided family planning and reproductive health services for more than 43,000 youth who faced ﬁnancial and other barriers in accessing a full range of services; 78% chose an implant or intrauterine device, and just over half received screening or treatment for sexually transmitted infections.10 In Nicaragua, a youth voucher program running from September 2000 to July 2001 that provided a total of 28,771 vouchers, found that youth participating in the program were 3 times more likely to use family planning and reproductive health services, 2 times more likely to use modern contraception, and 2.5 times more likely to report condom use at last sexual contact compared with youth not participating in the program.11 Researchers also found that providers participating in the Nicaragua voucher program had better knowledge, improved practices, and some attitudinal changes in support of provision of family planning and reproductive health services to youth compared with providers not involved in the program.12

TABLE. Key Advantages and Challenges of Voucher Programs

Advantages Challenges

Reduce financial and other client barriers to accessing health Require establishment of systems and processes to execute and services. monitor program. Allow governments and donors to target subsidies Can be complex to target to defined groups, particularly for populations in need, such as poor, youth, for groups new to vouchers. Requires measures to mitigate and pregnant or postpartum women. fraud and leakage to general population or non-poor. Allow resources to be directed toward key or high-impact health Unclear how they may affect other health services and the overall interventions and can stimulate demand for health services and health system; could result in provider overload if the system is behaviors. not prepared. Create a network of quality-assured health care providers, which Require quality inputs to service providers to improve quality of can enhance access to services in the short term, and a platform care; quality inputs also prevent driving up service use without for strategic purchasing in medium to longer term. improving health outcomes and client satisfaction. Voucher revenue can flow directly to health care providers, Program funding may or may not be used to invest in areas that which the providers can then reinvest in facilities and improve services or client satisfaction. services to further improve them.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 387 Vouchers and Family Planning Services www.ghspjournal.org

Voucher programs a demand-side financing mechanism by design, and make their clinic more of a ‘‘one-stop help to protect enable governments and donors to target sub- shop’’ for clients.22 Because vouchers subsi- vulnerable sidies directly for populations in need, includ- dize the poor’s purchasing power, this can groups, increase ing poor, youth, and pregnant or postpartum facilitate the poor’s access to the private sector uptake of health women, rather than spreading limited resources without spending money out of pocket, thereby services, improve among the general population. Vouchers reducing inequities in access to health services 12,23 service quality, make sense as a solution when there are including family planning. engage the financial and other access barriers to seeking Engage and leverage the existing health private sector, and using key high-impact health services. system to maximize service delivery. Voucher Most programs use means testing or geo- and strengthen programs can be designed to fit the existing graphic targeting to identify poor or otherwise health systems. health system and mix of providers, although vulnerable clients to channel resources toward most programs engage both private- and those most in need of financial subsidy.1 some public-sector providers to ensure max- Recent improvements in standardized equity The private imum reach and to leverage capacity.1 Enga- measurement can simplify this process and sector serves ging the private sector in voucher provision reduce the cost in voucher programs.15 an especially is particularly important in contexts where Promote demand for and use of family important role quality may be weak in the public sector, planning and reproductive health services where public- where people seek health care in the pri- while enhancing client choice. Voucher sector quality may vate sector, and use of preventive services is programs can be a promotional tool to expand 12 be weak, where limited. Increasing the provider mix in a access to and choice of health care services.16 people already network also extends the reach of social Programs can and should accompany voucher seek private care, protection and creates more provider choice distribution with social and behavior change 12,24,25 and where use for clients. In addition, voucher pro- communication efforts to promote demand grams can establish a network of health care of preventive for and improve knowledge of the key health services is limited. 5 providers from fractionalized separate entities behaviors and services. Promotion of the that were not linked before, but are now accred- voucher program helps raise client awareness 13 ited voucher service providers. A voucher of what services are offered and where, program in Yemen, implemented by an NGO, particularly if they are new or underused kept primary health services, including family services; without promotion efforts, clients planning, running in public clinics or offered may not be aware of and use the voucher.6,16 the choice of private clinics when the govern- In one voucher program area, only 25% of ment was unable to flow resources due to women from the communities had heard of active conflict.26 A successful voucher pro- the family planning voucher, versus 82% who gram implemented by private franchised had heard of the safe motherhood voucher; clinics in Pakistan expanded access to family corresponding family planning service uptake planning services, as none of the providers was lower than for safe motherhood. Research- offered these services before participating in ers suggest that low uptake may have been 13 due in part to the absence of adequate com- the voucher program. Providers often like munication with clients and communities participating in a voucher program because about the family planning voucher pro- they can increase the number of clients served gram.17,18 Vouchers can particularly address and attract new clients while offering a range access barriers for clinical services, such as of services, potentially making clinics more 3,5,12 long-acting and reversible contraceptives and profitable. permanent methods, in areas where poor Assure and improve the quality of family women are less likely to use them, by reduc- planning and reproductive health care. ing out-of-pocket costs for clients and ensur- Implicit in a voucher program is that partici- ing providers get reimbursed for these methods pating health care facilities must be accredited that are relatively costlier to provide than and/or provide services according to estab- – short-acting methods.5,13,19 21 Often, provi- lished quality standards. Programmatic inputs ders are motivated to participate in a voucher are often required to get a facility accredited program as it gives them an opportunity to and enrolled in the voucher program and gain new skills, offer new services to clients, fully functional within the system to process

Global Health: Science and Practice 2016 | Volume 4 | Number 3 388 Vouchers and Family Planning Services www.ghspjournal.org

voucher claims. Ongoing quality assurance insurance capability in the health sector and and provider support can also be a feature of to increase government’s familiarity with the design, and is a key component when purchasing services from the private sector.1 implemented by private social franchise clinic Successfully contracting with the private providers. Reimbursements given to providers sector in a voucher program resulted in in exchange for provision of services can then a government plan for how to scale up be channeled toward facility-level improve- the program nationally.1 Voucher programs ments (e.g., supplies and commodities, staff, that work with social franchise clinics can equipment, infection prevention, and training). network fractionalized private providers In Kenya, both public and private providers with a third-party intermediary organiza- invested voucher revenue into facility-based tion (usually an NGO) that addresses quality improvements.17 Finally, providers quality inputs, simplifies reimbursement, know that clients can choose any provider and lends itself to future financial transaction participating in the voucher program, thereby flows from governments to multiple private driving competition and encouraging provi- providers. ders to make quality improvements, such as privacy, cleanliness, and the addition of other 27 services. In Kenya, postnatal clients at facil- THE CHALLENGES OF IMPLEMENTING ities participating in the voucher program VOUCHER PROGRAMS received more comprehensive counseling on fertility, healthy birth spacing, and available The potential for voucher programs are substan- contraceptive methods than postnatal clients tial, but challenges in their implementation do at comparable non-participating facilities.28 In exist. As with most complex interventions, imple- the longer term, vouchers can institutionalize menters must focus on the specifics and get performance in a clinical setting by focusing immersed in the details to ensure a well imple- attention on quality services rendered, and for mented program. With experience, the design public-sector clinics this can improve produc- and function of the voucher system improves and tivity and efficiency.29 implementation becomes easier. Vouchers can Regular Provide a pathway for health care provi- work in a range of contexts, but program imple- monitoring, mentation should be iterative and dynamic, with ders to participate in insurance. Most including course monitoring and regular opportunities to adjust voucher programs are financed by donors or corrections, is a designs, service packages, pricing, methods of loan funds, but these could eventually be key component of means testing to identify populations in need, financed by governments with domestic resources. voucher program and voucher distribution approaches to ensure Vouchers can be a precursor to health insur- success. optimal functioning and to reach the defined ance or government contracting, familiarizing client population.1 The following questions are providers with processes like accreditation, important to consider: standardization of service packages and pricing, billing, verification, and reimbursement Programs require substantial adminis- after service provision.24,30 Vouchers demand tration and oversight—do the benefits provider-level accountability and, when imple- outweigh the challenges? Voucher pro- Voucher programs mented well, they create disincentives for grams are somewhat complex to set up and can organize fraudulent behavior. In Armenia, a voucher manage, and they require ongoing oversight program was developed to prevent informal and management inputs. Because of this, health care payments and strengthen accountability.1 voucher programs should be implemented facilities, including By linking qualified providers under a pro- at a medium to large scale to maximize invest- a fractionalized gram umbrella, voucher programs can orga- ment in these systems. Costs are high at the private sector, into nize a fractionalized private sector or a early stages of setting up systems, but they a quality-assured mix of public and private health care facili- should decrease over time as systems are network, which ties into a quality-assured network, which established and expertise is strengthened.6 could then provide could then provide services for government Using local organizations to manage voucher services for health programs or insurance. In Cambodia, programs can lower administrative costs.24 government Kenya, Tanzania, and Uganda, voucher pro- In some countries, vouchers relied on existing health programs grams were introduced to build social health ways to identify the poor and to target benefits, or insurance.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 389 Vouchers and Family Planning Services www.ghspjournal.org

Costs may be such as in India’s Below Poverty Line card or privacy measures; training on clinical stan- 1,5 higher at the early Cambodia’s Health Equity Fund. dards and guidelines; ensuring availability of stages of setting How can public-private partnerships be high-quality commodities, instruments, and up voucher encouraged? The majority of voucher pro- consumables; providing job aids, counseling systems but grams to date have been designed to facilitate guides, and client informational materials) should decrease contracting of health services through the to ensure that providers have sufficient quality over time as private sector, including social franchise clinics,1 standards to participate in the program, systems are to leverage private-sector reach, expand pro- which may have spillover effects on improving the quality of other health services (e.g., established and vider choice, and improve quality service infection prevention or client-centered care). expertise is provision. Voucher programs can, however, In addition, revenue from voucher program- strengthened. function as a type of public–private partnership, with specific roles and functions carried ming may enable health care facilities to hire out by the government and private sector. For staff or purchase supplies and equipment example, in India a formal public–private to add or improve services, potentially improv- partnership approach to the Sambhav voucher ing specific services, the client experience, or program was considered a success.5 In many health care provision overall. However, it is countries, public-sector health care services also possible to increase uptake of services but are mandated as free, so governments may not see any improvements in outcomes or face difficulty in making the rationale to service quality, as shown by researchers examining maternal health demand-side financ- include public-sector services in a voucher 31 program. Without concerted efforts to address ing programming, including vouchers. For operational challenges, governments may not family planning, this may not be the case, have the infrastructure, staff, or systems in as any use of services can improve overall place to adequately implement, oversee, and health outcomes, such as maternal and infant manage all aspects of a voucher program to mortality reduction, but it is important to function optimally.25 However, even if public- provide high-quality services and continuity sector health care provision and facilities are of care with vouchers. In addition to quality not included, governments can play impor- assurance inputs and training of providers, tant roles in oversight, planning, and priority a family planning voucher program in Uganda setting with the longer-term view of transi- aimed for continuity of care by offering a tioning a voucher program to government single voucher that included 4 services with financing, contracting, or insurance. While separate, reimbursable barcodes: family plan- vouchers should not compete with other social ning counseling, method provision, a follow- protection mechanisms, such as social health up visit to address any side effects, and a removal visit (for long-acting and reversible insurance, they should be positioned as com- 8 plementary and a partnership opportunity contraceptives). to stimulate uptake of health services and incen- Should family planning be integrated tivize high-quality service provision. Vouchers into a multi-service, integrated voucher Single-service can also fill a gap when family planning program or have its own voucher? In family planning methods or services are not covered, or only Uganda, single-service family planning vou- vouchers have partially covered, by insurance packages. chers have operated with success, including 9,13,14,21 been successful, How does voucher programming influ- alongside other types of health vouchers. but most voucher ence service provision as a whole? While Most voucher programs, however, offer inte- programs offer a vouchers may play an important role in grated services, such as a program in India that package of ensuring access to and delivery of key health included a package of antenatal, institutional services to populations in need, it is not clear delivery, postpartum, and family planning integrated 1 services. how vouchers affect service provision as a services. In Zimbabwe, a voucher program whole or how they affect services not covered for youth initially included only family plan- by vouchers or clients without vouchers.25 ning services, but later added screening and Voucher programs provide facility-level sup- treatment for sexually transmitted infections port (e.g., improving infection prevention after receiving feedback from clients who measures; ensuring adequate clean water/ were interested in and needed more services sanitation facilities; installing adequate client (personal communication with Anna Mackay,

Global Health: Science and Practice 2016 | Volume 4 | Number 3 390 Vouchers and Family Planning Services www.ghspjournal.org

Deputy Director, Support for International of key health services to meet the health needs of Family Planning Organizations [SIFPO-MSI] the poor, vulnerable, and other special popula- Project, Feb. 2016). It is unclear exactly the tions, in part because of their aim and perhaps best way to include family planning services ability to facilitate change at both client and in a bundled package so that the family plan- provider levels. Voucher programs may also get ning services are still promoted and provided. populations familiar with using key preventive Experience from two different voucher pro- and essential primary health care services and grams in different areas of Pakistan suggests thus may change behavior at a social normative that a separate family planning visit may yield level, another way of ensuring sustainability. more uptake.32,33 In one of these programs, As we work to reaching the Family Planning Vouchers can 79% of voucher clients visited clinics for 2020 (FP2020) goals by addressing gaps in access support the family planning services when the services to family planning, vouchers offer an opportunity aims of FP2020-- were offered through a separate postnatal care to target donor and government resources to addressing visit. In comparison, 62% of clients in the those who most need family planning services gaps in family second program visited clinics for family and products. Those with the highest unmet need planning access planning as part of a package of postnatal may not seek family planning services without by targeting care services. subsidies or support and are therefore vulnerable resources to How can we prepare providers to handle to unintended pregnancy and adverse maternal those who want health outcomes. As we look to improve access to an increase in demand for services? Akey voluntary family family planning for 120 million more women and design feature of voucher programs to ensure planning services girls, we should focus donor and government success is the appropriate selection and reten- but who may not resources on ensuring that the poor and other tion of qualified health care providers. Retain- otherwise seek special groups can achieve their reproductive ing qualified providers is particularly important services without for programs that provide resource-intensive intentions. While universal health coverage may be a a subsidy or quality monitoring and supervision. Voucher support.* programs may stimulate rapid uptake of services, vision for the future, there is less discussion in and if providers are not prepared (e.g., with the literature on which short- or medium-term adequate staff, commodities, and supplies; steps we need to take to get there. Vouchers could sufficient operating hours; efficient manage- serve as an intermediate step for public financing ment of services) the increased service volume to begin to target subsidies to address inequitable may have negative consequences on provider rates of maternal, child, and infant mortality job satisfaction.22 The increased demand for among the poor, versus traditional supply-oriented health financing that faces challenges in reaching services may also result in providers not 3,4,16 being able to provide comprehensive or client- the poor and underserved. Voucher programs centered care due to time constraints. The may strengthen capacity and readiness in the possibility of overload may be more acute health system for implementing universal health in public-sector health care facilities, which coverage that prioritizes the health needs of the may have less ability to hire staff, extend poor, engages the private sector, and has a service hours, or use the reimbursement revenue to package that does not leave out key primary make changes, unless the program is designed health interventions and preventive care, such as to address these challenges.25 Private-sector family planning. In Cambodia, a voucher program ’ clinics may have more autonomy over the became better aligned with the government s revenue to direct it toward quality improve- Health Equity Fund, a safety net program for the poor,1 and even helped to identify new benefici- We need partner ments, whereas public-sector clinics may organizations to not have the autonomy to organize service aries. However, such experiences of transitioning document their provision or direct access to resources flowing voucher programs into larger health insurance experiences with into the facility.1 programs are limited. The community of practice on health voucher vouchers, share implementation in developing countries is grow- their results, and ing and has experiences to share, as many voucher demonstrate NEXT STEPS programs must undergo midcourse corrections impact through When implemented at a medium to large scale, and adaptions to function optimally. We need research when voucher programs can achieve results, including partner organizations to document their experi- possible. reducing barriers to access and improving uptake ences and share their results, and conduct research

Global Health: Science and Practice 2016 | Volume 4 | Number 3 391 Vouchers and Family Planning Services www.ghspjournal.org

when possible to further demonstrate impact and Available from: http://siteresources.worldbank.org/ contribute to the body of knowledge to address HEALTHNUTRITIONANDPOPULATION/Resources/Peer- common challenges and issues in implementation. Reviewed-Publications/AGuidetoCompetitiveVouchersinHealth.pdf We have more to learn about voucher programs, 4. Islam M. Vouchers for health: a focus on reproductive health and family planning services: a primer for policy makers. Bethesda and we encourage the implementation and research (MD): Abt Associates Inc, Private Sector Partnerships-One communities to collaborate to shed light on the project; 2006. Available from: http://pdf.usaid.gov/pdf_docs/ following areas: PNADI574.pdf 5. IFPS Technical Assistance Project (ITAP). Sambhav: vouchers 1. Understand key differences in voucher pro- make high-quality reproductive health services possible for India’s grams that are implemented by public-sector, poor. Gurgaon (Haryana): Futures Group, ITAP; 2012. Available private-sector, or mixed public- and private- from: http://pdf.usaid.gov/pdf_docs/pnadz573.pdf sector health care facilities, and ways to 6. Bellows N. Vouchers for reproductive health care services in Kenya and Uganda. Frankfurt: KfW Entwicklungs-bank maximize performance, particularly how best Competency Center for Health, Education and Social Protection; to strengthen implementation and quality in 2012. Available from: http://www.nairobi.diplo.de/ public-sector voucher programs. contentblob/4306428/Daten/4532580/FZ_ EVoucherProgramm.pdf 2. Gain efficiencies in voucher program opera- 7. High Impact Practices in Family Planning (HIPs). Vouchers: tions including: addressing inequities in access to contraceptive services. Washington (DC): United States Agency for International J Best practices for provider reimbursement Development; 2015. Available from: https://www. J Opportunities for leveraging mobile health fphighimpactpractices.org/resources/vouchers technology (e.g., electronic vouchers, provi- 8. Mackay A. Increasing access and choice in FP through private sector vouchers in Uganda. Presented at: The International der reimbursement using mobile money, Conference on Family Planning; 2016 Jan 25–28; Nusa Dua, applications for tablets or mobile phones to Indonesia. https://www.xcdsystem.com/icfp/program/index. facilitate provider supervision and oversight cfm?pgid=288&search=1&qtype=Session&sid=16529& techniques) submit=Go 9. Bellows B, Mackay A, Dingle A, Tuyiragize R, Nnyombi W, J Optimal strategies to target voucher sub- Dasgupta A. Increasing contraceptive access with vouchers and sidies toward the populations that can most franchising in Uganda. Forthcoming 2016. benefit from them while minimizing fraud 10. Burke E, Gold J. Increasing access to voluntary family planning and STI services for young people: the voucher program in J Strengthened monitoring of programs with Madagascar. London: Marie Stopes International; 2014. aligned key outcomes of interest Available from: https://mariestopes.org/data-research/ resources/increasing-access-voluntary-family-planning-and- 3. Document more experiences of medium- to sti-services-young-people large-scale voucher programs that provide infor- 11. Meuwissen LE, Gorter AC, Knottnerus AJA. Impact of accessible mation on performance (e.g., client uptake, sexual and reproductive health care on poor and underserved new or lapsed family planning clients, poverty adolescents in Managua, Nicaragua: a quasi-experimental status of voucher clients, and observed and intervention study. J Adolesc Health. 2006;38(1):56.e1–56.e9. CrossRef. Medline reported quality) and cost. 12. Meuwissen LE, Gorter AC, Kester ADM, Knottnerus JA. Can a 4. Test and document the transition of a donor- comprehensive voucher programme prompt changes in doctors’ funded voucher program to government knowledge, attitudes and practices related to sexual and reproductive financing. health care for adolescents? A case study from Latin America. Trop Med Int Health. 2006;11(6):889–898. CrossRef. Medline Acknowledgments: The views expressed in this article are those of the 13. Khurram Azmat S, Tasneem Shaikh B, Hameed W, Mustafa G, authors and do not necessarily reflect the views of the U.S. Agency for Hussain W, Asghar J, et al. Impact of social franchising International Development or the U.S. Government. on contraceptive use when complemented by vouchers: Competing Interests: None declared. a quasi-experimental study in rural Pakistan. PLoS One. 2013;8(9):e74260. CrossRef. Medline REFERENCES 14. Azmat SK, Hameed W, Hamza HB, Mustafa G, Ishaque M, Abbas G, et al. Engaging with community-based public and 1. Grainger C, Gorter A, Okal J, Bellows B. Lessons from sexual private mid-level providers for promoting the use of modern and reproductive health voucher program design and function: contraceptive methods in rural Pakistan: results from two a comprehensive review. Int J Equity Health. 2014;13(1):33. innovative birth spacing interventions. Reprod Health. 2016; CrossRef. Medline 13(1):25. CrossRef. Medline 2. Bellows NM, Bellows B, Warren C. The use of vouchers for 15. Chakraborty NM, Fry K, Behl R, Longfield K. Simplified asset reproductive health services in developing countries: systematic indices to measure wealth and equity in health programs: review. Trop Med Int Health. 2011;16(1):84–96. CrossRef. Medline a reliability and validity analysis using survey data from 3. Sandiford P, Gorter A, Salvetto M, Rojas Z. A guide to 16 countries. Glob Health Sci Pract. 2016;4(1):141–154. competitive vouchers. Washington (DC): World Bank; 2005. CrossRef. Medline

Global Health: Science and Practice 2016 | Volume 4 | Number 3 392 Vouchers and Family Planning Services www.ghspjournal.org

16. Bhatia MR, Gorter AC. Improving access to reproductive and African Strategies for Health Project; 2015. Available from child health services in developing countries: are competitive https://www.msh.org/sites/msh.org/files/technical_ voucher schemes an option? J Int Dev. 2007;19(7):975–981. brief_cbhi_rhv_cea_june_2015.pdf CrossRef 25. Okal J, Kanya L, Obare F, Njuki R, Abuya T, Bange T, et al. 17. Obare F, Warren C, Njuki R, Abuya T, Sunday J, Askew I, et al. An assessment of opportunities and challenges for public sector Community-level impact of the reproductive health vouchers involvement in the maternal health voucher program in Uganda. programme on service utilization in Kenya. Health Policy Plan. Health Res Policy Syst. 2013;11:38. CrossRef. Medline 2013;28(2):165–175. CrossRef. Medline 26. Boddam-Whetham L, Gul X, Al-Kobati E, Gorter AC. Vouchers 18. Bajracharya A, Veasnakiry L, Rathavy T, Bellows B. Increasing in fragile states: reducing barriers to long-acting reversible uptake of long-acting reversible contraceptives in Cambodia contraception in Yemen and Pakistan. Glob Health Sci Pract; through a voucher program: evidence from a difference-in- 2016;4 Suppl 2. CrossRef differences analysis. Glob Health Sci Pract. 2016;4 Suppl 2: 27. Njuki R, Abuya T, Kimani J, Kanya L, Korongo A, Mukanya C, S109–S121. CrossRef. Medline et al. Does a voucher program improve reproductive health 19. Arur A, Gitonga N, O’Hanlon B, Kundu F, Senkaali M, service delivery and access in Kenya? BMC Health Serv Res. Ssemujju R. Insights from innovations: lessons from designing and 2015;15(1):206. CrossRef. Medline implementing family planning/reproductive health voucher 28. Watt C, Abuya T, Warren CE, Obare F, Kanya L, Bellows B. programs in Kenya and Uganda. Bethesda (MD): Abt Associates Can reproductive health voucher programs improve quality of Inc., Private Sector Partnerships-One project; 2009 Available postnatal care? A quasi-experimental evaluation of Kenya’ssafe from: http://pdf.usaid.gov/pdf_docs/PNADP185.pdf motherhood voucher scheme. PLoS One. 2015;10(4):e0122828. 20. Ugaz JI, Chatterji M, Gribble JN, Banke K. Is household wealth CrossRef. Medline associated with use of long acting reversible and permanent 29. International Development Association (IDA). A review of the use methods of contraception? A multi country analysis. Glob Health of output-based aid approaches. Washington (DC): IDA; 2006. Sci Practice. 2016;4(1):43–54. CrossRef. Medline Available from: http://siteresources.worldbank.org/IDA/Resources/ 21. Kemplay M, Neggaz M, Mani N. Madagascar program profile. Seminar%20PDFs/73449-1164920192653/IDANETOBA.pdf Bethesda (MD): Abt Associates, Strengthening Health Outcomes 30. Sochas L, Grainger C, Gorter A, Griffith D, Boddam-Whetham L. through the Private Sector Project; 2013. Available from: http:// What can vouchers do for universal health coverage? shopsproject.org/sites/default/files/resources/Madagascar% London: Options Consultancy Services; 2013. Available from: 20Program%20Profile.pdf http://www.options.co.uk/sites/default/files/what-can- 22. Brody C, Irige JM, Bellows B. Burnout at the frontline: the effect vouchers-do-for-universal-health-coverage.pdf of a reproductive health voucher program on health workers 31. Hurst TE, Semrau K, Patna M, Gawande A, Hirschhorn LR. in Uganda. Int Archives Nursing Health Care. 2015;1:001. Demand-side interventions for maternal care: evidence of more Available from: https://www.rbfhealth.org/sites/rbf/files/ use, not better outcomes. BMC Pregnancy Childbirth. 2015; The%20Effect%20of%20a%20Reproductive%20Health% 15(1):297. CrossRef. Medline 20Voucher%20Program%20on%20Health%20Workers%20in% 20Uganda.pdf 32. Agha S. Changes in the proportion of facility-based deliveries 23. Mustafa G, Azmat Khurram S, Hameed W, Ali S, Ishaque M, and related maternal health services among the poor in rural Hussain W, et al. Family planning knowledge, attitudes and Jhang, Pakistan: results from a demand-side financing practices among married men and women in rural areas of intervention. Int J Equity Health. 2011;10(1):57. CrossRef. Pakistan: findings from a qualitative needs assessment study. Medline Int J Reprod Med. 2015;2015:190520. CrossRef. Medline 33. Agha S. Impact of a maternal health voucher scheme on 24. Saya U. Cost-effectiveness of reproductive health vouchers and institutional delivery among low income women in Pakistan. community-based health insurance in Uganda. Arlington (VA): Reproductive Health. 2011;8:10. CrossRef. Medline

Peer Reviewed

Received: 2016 Mar 22; Accepted: 2016 Jun 8; First Published Online: 2016 Sep 26

Cite this article as: Menotti EP, Farrell M. Vouchers: a hot ticket for reaching the poor and other special groups with voluntary family planning services. Glob Health Sci Pract. 2016;4(3):384-393. http://dx.doi.org/10.9745/GHSP-D-16-00084.

& Menotti and Farrell. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi. org/10.9745/GHSP-D-16-00084.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 393 ORIGINAL ARTICLE

Successful Implementation of a Multicountry Clinical Surveillance and Data Collection System for Ebola Virus Disease in West Africa: Findings and Lessons Learned

Reshma Roshania,a Michaela Mallow,a Nelson Dunbar,b David Mansary,c Pranav Shetty,a Taralyn Lyon,a Kacey Pham,a Matthew Abad,a Erin Shedd,a Anh-Minh A Tran,a Sarah Cundy,a Adam C Levinea,d

ABSTRACT Background: The 2014 outbreak of Ebola virus disease (EVD) in West Africa was the largest ever recorded. Starting in September 2014, International Medical Corps (IMC) managed 5 Ebola treatment units (ETUs) in Liberia and Sierra Leone, which cumulatively cared for about 2,500 patients. We conducted a retrospective cohort study of patient data collected at the 5 ETUs over 1 year of operations. Methods: To collect clinical and epidemiological data from the patient care areas, each chart was either manually copied across the fence between the high-risk zone and low-risk zone, imaged across the fence, or imaged in the high- risk zone. Each ETU’s data were entered into a separate electronic database, and these were later combined into a single relational database. Lot quality assurance sampling was used to ensure data quality, with reentry of data with high error rates from imaged records. Results: The IMC database contains records on 2,768 patient presentations, including 2,351 patient admissions with full follow-up data. Of the patients admitted, 470 (20.0%) tested positive for EVD, with an overall case fatality ratio (CFR) of 57.0% for EVD-positive patients and 8.1% for EVD-negative patients. Although more men were admitted than women (53.4% vs. 46.6%), a larger proportion of women were diagnosed EVD positive (25.6% vs. 15.2%). Diarrhea, red eyes, contact with an ill person, and funeral attendance were significantly more common in patients with EVD than in those with other diagnoses. Among EVD-positive patients, age was a significant predictor of mortality: the highest CFRs were among children under 5 (89.1%) and adults over 55 (71.4%). Discussion: While several prior reports have documented the experiences of individual ETUs, this study is the first to present data from multiple ETUs across 2 countries run by the same organization with similar clinical protocols. Our experience demonstrates that even in austere settings under difficult conditions, it is possible for humanitarian organizations to collect high-quality clinical and epidemiologic data during a major infectious disease outbreak.

a International Medical Corps, Los Angeles, CA, USA. INTRODUCTION b Ministry of Health, Monrovia, Liberia. c International Medical Corps Sierra Leone, Freetown, Sierra Leone. he outbreak of Ebola virus disease (EVD) in d T The Warren Alpert Medical School of Brown University, Providence, RI, USA. WestAfricathatbeganin2014 is the largest since Correspondence to Adam Levine ([email protected]). the Ebola virus was first discovered in 1976. Nearly

Global Health: Science and Practice 2016 | Volume 4 | Number 3 394 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

30,000 people were infected and almost 12,000 METHODS died in the hardest-hit countries: Liberia, Sierra Leone, and Guinea.1-3 The World Health Organi- Study Design zation (WHO) formally declared the Ebola epi- This retrospective cohort study includes patient demic in West Africa a public health emergency of data collected at 5 ETUs operated by IMC in international concern on August 8, 2014.4 Days Liberia and Sierra Leone between September 15, ’ later, International Medical Corps (IMC), which 2014, and September 15, 2015, as part of IMC s had already begun its own assessment, launched comprehensive response to the West African EVD its initial response to the outbreak. epidemic. Ethical approval for this study and exemption from informed consent was provided Starting in September 2014, IMC opened and IMC opened and by the Sierra Leone Ethics and Scientific Review managed 5 Ebola treatment units (ETUs) in managed 5 Ebola Committee, the University of Liberia–Pacific Insti- Liberia and Sierra Leone. These 5 ETUs cumula- treatment units tute for Research and Evaluation Institutional tively cared for more than 2,500 patients. IMC in Liberia and used a comprehensive approach to EVD preven- Review Board, and the Lifespan Rhode Island Hos- pital Institutional Review Board. Sierra Leone, tion and management, which included direct which together health care within ETUs; water, sanitation, and cared for hygiene interventions; psychosocial support; sup- Program Setting more than port for infection, prevention, and screening in In cooperation with local health ministries, IMC 2,500 patients. local health facilities; and social and behavior operated 5 ETUs in Sierra Leone and Liberia change elements within affected and at-risk between September 15, 2014, and December 31, communities. 2015. The first 2 ETUs to open, in September and From the 5 ETUs in Liberia and Sierra Leone, November 2014, were located in Bong County IMC amassed more than 25,000 pages of clinical, and Margibi County, respectively, in Liberia, epidemiological, psychosocial, and operational where the epidemic first peaked in the summer ’ data over the course of the epidemic. IMC esta- and fall of 2014. As the epidemic began to peak in The goal of IMCs blished an Ebola Research Team in March 2015 neighboring Sierra Leone, 2 additional ETUs were Ebola Research with the goal of collecting, aggregating, cleaning, established there: in Lunsar, Port Loko District, Team was to quality checking, and analyzing this data to better and in Makeni, Bombali District. In April 2015, collect and inform the scientific and humanitarian response IMC assumed management of a fifth ETU in analyze quality to future epidemics. Kambia, Kambia District, Sierra Leone. data to better Several prior reports have documented the inform responses experiences of individual ETUs in Sierra Leone to future and Guinea. However, no prior studies have pre- Patient Triage Procedures epidemics. sented data from multiple ETUs across multiple Individuals experiencing symptoms consistent countries run by the same organization with with EVD arrived at IMC’s 5 ETUs in 3 ways: similar clinical protocols.5-9 In addition, prior transported in an IMC ambulance, transported in published studies have focused on demographic a government or private ambulance, or via their and outcome data for patients with EVD, and own means of transportation (private car, taxi, or have not presented a comprehensive picture of walking). The Liberia ETUs received all patients the details involved in both providing and from the ETUs’ catchment areas. In Sierra Leone, documenting clinical care for patients with EVD however, there were multiple agencies operating in resource-limited settings. in the ETUs’ districts, and the government-run This study presents IMC’sEVDcaseman- District Ebola Response Center determined where agement operations across Liberia and Sierra patients were sent. Leone, including numbers and trends of patient A minority of patients, tested in the com- admissions to our ETUs; key demographic infor- munity or at government-managed holding cen- This study mation and outcomes among admitted patients; ters before arriving at the ETU, presented with and geographical and longitudinal displays of laboratory-confirmed EVD. Most patients, how- provides detailed patient admissions, EVD positivity, and mortal- ever, presented to the ETU with 1 or more symp- information on ity. In addition, we provide detailed informa- toms consistent with EVD but without laboratory ways to both tion, within this article and the supplemental confirmation. Upon arrival, all patients without a provide and appendices, on the clinical care provided to previously confirmed test for EVD were brought document clinical patients and the methods of data collection through triage. In triage, patients were screened care in a resource- within our ETUs. by trained ETU clinical staff to ensure that they limited setting.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 395 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

met the clinical case definition for EVD. IMC crea- admitted to the Lunsar and Makeni ETUs, while ted guidelines for this process (see supplementary the Nigerian laboratory in Kambia District (sup- material) based on WHO and Médecins Sans ported by the European Union Mobile Laboratory Frontières (MSF) guidelines and in consultation Consortium) provided RT-PCR testing for patients Patients arriving with local health authorities.10-13 Patients who admitted to the Kambia ETU. The processes were at an ETU who met the case definition were admitted to the ETU, similar to those used by the NMRC laboratory, met the case while those who did not were referred to another except that the PHE and Nigerian laboratories deﬁnition for operating public or private health care facility, tested only a single EVD gene target (EBOV locus) Ebola virus when available, for necessary care. as opposed to 2 targets. In addition, the PHE disease were After triage at the ETU, patients without laboratories switched from using the commer- admitted, while previously confirmed EVD but who met the case cially available Altona real-time RT-PCR assay to definition were brought to the ward for either using the in-house Trombley assay in February those who did not 14-16 were referred suspect or probable disease. There they had a 2015. blood sample drawn for initial EVD testing, with- The PHE and Nigerian laboratories in Sierra elsewhere for in 24 hours. Patients with an initial negative test Leone performed malaria tests in addition to EVD care. result who had had symptoms for fewer than tests. However, no other laboratory diagnostics 3 days were held for repeated testing until were consistently available for any of the 5 ETUs. 72 hours had passed since the onset of their symptoms. Patients with a second negative test result after having symptoms for more than Clinical Management All patients at 3 days were considered EVD-negative (EVD-) All IMC ETU patients were treated according to IMC ETUs were and were discharged home from the suspect or standard treatment protocols that were based on treated according probable ward or were transferred to another guidelines developed by WHO and MSF during to standard health care facility for further care as soon as prior outbreaks and adapted by IMC, in consul- treatment logistically possible. Patients with a positive test tation with local Ministry of Health officials, + protocols adapted result were considered EVD-positive (EVD ) and to the needs and resources of Liberia and Sierra ’ 12,13 to the needs of the were moved to the ETU s confirmed ward for Leone. Briefly, the standard clinical protocol host country. further management, as were patients who included empiric antimalarial treatment; broad- presented to the ETU with laboratory-confirmed spectrum antibiotics; oral rehydration solution EVD. (ORS); medications to prevent gastritis; vitamins and nutritional supplementation; and symptomatic treatment for fever, pain, nausea, and Laboratory Testing delirium. Those who presented with or devel- For both the Bong and Margibi ETUs, laboratory oped moderate to severe dehydration or inability diagnosis of EVD was performed at the United to drink sufficient ORS independently were States Naval Medical Research Center (NMRC) treated with boluses of crystalloid solution. The Patients were Mobile Laboratory in Bong County, Liberia. Diagno- standard clinical and psychosocial procedures cared for by sis was confirmed with the 1-step quantitative Ebola manual provided with the supplementary materials trained hygienists, Zaire real-time reverse transcriptase–polymerase includes detailed information on all treatments nurses, physician chain reaction (RT-PCR) (TaqMan) assay (NMRC, provided. assistants, Frederick, MD). Briefly, Qiagen Buffer AVL and Throughout their inpatient course, patients physicians, or ethanol-inactivated blood samples were extracted were cared for by trained hygienists, nurses, psychosocial with QIAamp Viral RNA Mini Kit. Extracted ribo- physician assistants, physicians, or psychosocial support staff. nucleic acid was tested for 2 EVD gene targets support staff. In general, patients were rounded (Zaire ebolavirus [EBOV] locus and minor groove on 1 to 2 times per day by a physician or physi- binding locus), using the Applied Biosystems Step cian assistant, who documented clinical signs OnePlus instrument. A sample was confirmed to and prescribed treatments, and 3 to 6 times per All staff entering be positive for EVD if both targets were detected, day by either a nurse, who provided treatment, the high-risk but was considered indeterminate if only 1 target or a hygienist, who disinfected the environment patient care areas was detected. An indeterminate result led to retest- to prevent spread of the disease in the wards. were required to ing the patient. Because the suspect, probable, and confirmed wear full personal In Sierra Leone, the Public Health England wards were all located in the high-risk zone of the protective (PHE) laboratories in Port Loko and Bombali ETU, all staff entering this area were required to equipment. districts performed EVD testing for patients wear full personal protective equipment (PPE),

Global Health: Science and Practice 2016 | Volume 4 | Number 3 396 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

including scrubs, boots, Tyvek or Tychem suits, masks, hoods, goggles, aprons, and double latex gloves. This limited clinician rounds to 1 to 2 hours at a time due to the heat stress caused by the PPE. The limited rounds meant that all clinical care had to be provided within those periods of rounding, and that for the majority of the day, patients in the wards were unsupervised by clinical staff. At times, however, patients were supported by fellow ETU patients or by EVD survivors who served as caregivers for the very sick.

Clinical Documentation and Data Collection In consultation with local and national ministries of health, WHO, and other international organizations, IMC developed forms to capture demographic, clinical, and psychosocial support data on patients admitted to our ETUs. Examples of these forms, which varied slightly by ETU, are provided ’ as supplementary materials. Triage, laboratory, and Dr. Adam Levine of IMC s Ebola research team trains staff at the Bong discharge forms captured information on patient County Ebola treatment unit in Liberia on how to don personal protective demographics, presenting symptoms, laboratory equipment. test results, and final outcomes and were kept in patient files in the low-risk (nonclinical) area of waterproof camera. The camera was then decon- All forms used in the ETU. Patient rounding and treatment forms, taminated by soaking in a chlorine solution for the high-risk zone which were filled out in the high-risk zone of the 30 minutes before being transferred to the low- were considered ETU at the patient’s bedside, included data on risk zone, where the images were downloaded contaminated, patient vital signs and symptoms and on treat- onto a laptop computer. All patient files were and various ments given. Psychosocial support forms included eventually scanned into PDF, JPEG, or TIFF methods were data on mental health symptoms and family and format within the low-risk zone of each ETU. used to transfer caregiver support, and were filled out once patients All data were collected as part of routine clinical data out of the were well enough to receive psychosocial support care and for epidemiologic purposes. ETU. in either the low- or high-risk zone. The data forms Patient data, from paper forms or scanned were designed to track information on all patients images, were entered into separate electronic IMC developed admitted into our ETUs at each critical juncture in databases at each ETU by local data officers and forms designed to their stay from admission through discharge and were later combined into a unified database. The track information community follow-up. Forms were also specifically combined database was relational in structure on all patients designed with a system of check boxes to minimize and included 10 separate tables encompassing admitted to the the time clinicians spent charting while wearing patient demographic, triage, rounding, treatment, ETUs, from full PPE. laboratory, psychosocial support, outcome, and admission through Because all forms in the high-risk zone were follow-up data. discharge and ’ considered contaminated, various methods were At the conclusion of IMC s ETU program, community employed to transfer data from these forms out of scanned images of patients’ paper records were follow-up. the ETU. In some cases, the information was read stored in IMC’s secure network drive, and the across the fence between the high- and low-risk hard copies were transferred to the Ministry zones after rounds were completed each day. In of Health in each country. Camera images from these cases, data was read by one staff member, high-risk zones in Liberia were stored in this copied onto an identical chart by another staff same IMC network drive. Laboratory data, includ- member, and then placed in the patient’s file in ing EVD RT-PCR cycle thresholds, as well as malaria the low-risk zone. In other cases, charts from the test results from Sierra Leone, were obtained high-risk zone were either imaged from across from the NMRC, PHE, and Nigerian laboratories the fence between high- and low-risk zones or and linked to patient data in IMC’s unified imaged inside the high-risk zone using a database.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 397 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

data were reentered only for EVD+ patients, to prioritize limited resources. We took the following steps to ensure minimal errors during data reentry: (1) using data validation settings in Excel reentry documents, (2) using a codebook to ensure that patient data from various types of patient charts were standardized, (3) conducting additional audits by data entry research assistants, and (4) discussing data entry concerns with the principal investigator. Once reentry was complete, we conducted another data quality audit using LQAS. From each ETU, we selected 19 patient IDs from 2 substrata, EVD+ and EVD- (except in Margibi). Staff at the Lunsar Ebola treatment unit in Sierra Leone transfer patient We then compared data on scans of EVD+ and data from the high-risk zone to the low-risk zone. EVD- triage, EVD+ rounding, and EVD+ treatment in patient charts with data in the unified database. Each discrepancy was recorded as an TABLE 1. Quality of Data Entered From Original Patient Charts error. The number of errors per patient chart was Into ETU-Speciﬁc Databases: Results of LQAS Audit, Liberia and divided by the total number of data points for the Sierra Leone, November 2015 (N = 627 Patient Forms) specific patient, which depended on the patient’s length of stay. The total percentage of errors was Patient Form % Data Entered Correctly then calculated. With the results from this audit, we concluded that approximately 99% of the data Demographic 99.5% in IMC’s unified database were consistent with Triage 99.4% information from scans of patient charts. Table 1 summarizes the results of the LQAS. Rounding 98.1% Treatment 99.1% Discharge 99.8% Data Analysis The primary outcome variables of interest for Overall 99.8% patients admitted to the ETUs were final diagnosis (confirmed Ebola, probable Ebola, or other), Abbreviations: ETU, Ebola treatment unit; LQAS, lot quality assurance sampling. disposition (survived, deceased, or transferred), and length of stay in the ETU. Length of stay was calculated as the number of days from date of admission to date of discharge, inclusive of the Steps were taken Data Quality Audit and Reentry date of admission. Other variables of interest to ensure data In November 2015, we used lot quality assurance included demographic variables such as country quality, including sampling (LQAS), a random sampling methodol- of origin, sex, and age. Age was categorized based manually ogy, to assess the quality of the data entered from on WHO identification of infants under 1 and reentering data original patient charts into the ETU-specific children under 5 as particularly vulnerable and 17,18 from scanned databases. A random sample of 19 patient then into 10-year blocks. We analyzed ETU + ﬁles. ID numbers from 2 substrata, EVD and EVD-, admission trends by categorizing the date of were selected from each ETU (except Margibi, admission into epidemiological weeks consistent where 19 total patient ID numbers were ran- with WHO usage (Monday through Sunday). domly selected because only 5 EVD+ patients Clinical variables at triage, including fever, were were admitted) for this data quality audit. self- or-family-reported and categorized as yes (1) Due to a high number of discrepancies found or no (0). among triage, rounding, and treatment patient We used geographic information system charts and data entered in the unified database, (GIS) software to visualize the geographic dis- we reentered data using scanned files of original tribution of total admitted patients and total patient charts. Triage data were reentered for all EVD+ patients by subregion (e.g., chiefdom or admitted patients; daily rounding and treatment district in which a patient’s home village was

Global Health: Science and Practice 2016 | Volume 4 | Number 3 398 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

located). The maps generated for this analysis total patient admissions (Figure 3A) and con- display the sum of records per subregion. For the firmed EVD+ patient admissions (Figure 3B). map of total confirmed cases, a defined interval The supplemental figures in the GIS supplemen- classification method with an interval size of 15 tary material provide additional information on was used to display the number of confirmed the proportions of EVD+ and deceased patients Ebola diagnoses across the subregions included in by geographic origin as well as on the numbers of the EVD data set; the map of total patients uses a patients EVD+ by the location where they first modified natural breaks (Jenks) classification became ill. method. More information about the GIS methods used is provided as supplementary material. Demographic Data Basic descriptive statistics were calculated for the primary outcome variables as well as for Of the 2,351 patient admissions analyzed, 53.4% demographic, clinical, epidemiologic, geographic, were men and 46.6% were women. The median and time-dependent variables. We used chi- age was 30 years (interquartile range: 18, 43). square analysis to compare clinical and epide- Almost 10% of admitted patients were under age miologic variables present on the patient’s arrival 5, and 11% were 55 years or older, while nearly against the patient’s final diagnosis. We con- 25% were 25 to 34 years old (Table 2). ducted bivariate logistic regression analyses to examine differences in outcomes by age, sex, and country of origin, presenting odds ratios (ORs) Clinical and Epidemiologic Data with 95% confidence intervals (CIs). To assess differences in length of stay by subgroup, we Table 3 lists the clinical symptoms and epidemio- conducted independent samples t tests and 1-way logic characteristics of patients admitted to the analysis of variance (ANOVA) tests, as appro- 5 ETUs for triage, stratified by their final diagnosis. priate. Statistical significance was established at Clinical symptoms, including fever, were self- or .05. Data analyses were conducted in R version family-reported and endorsed. While fever (75.3%, 3.2.1 and ArcGIS for Desktop 10.3.1. P = .11), weakness (71.9%, P = .76), and loss of appetite (68.4%, P = .20) were the most common At triage, diarrhea symptoms in patients with EVD, they were RESULTS and red eyes were equally common in patients without EVD. Among more common The full IMC data set contained information clinical symptoms, only diarrhea (54.0%, Po.001) among patients on 2,768 total patients presenting to our 5 ETUs. and red eyes (27.5%, Po.001; a variable that diagnosed To ensure full follow-up data was available, included both conjunctivitis and conjunctival with EVD than we excluded from the analysis 88 patients whose hemorrhage) were more common in patients those without. data were either missing the date of triage or who with EVD than in those with other diagnoses. Abdominal pain, o were triaged outside the selected 1-year time Abdominal pain (54.0% vs. 43.5%, P .001), shortness of period. Patients who were declared dead on shortness of breath (30.9% vs. 23.5%, P = .002), breath, and arrival (n = 24) and those who were not admitted and non-ocular bleeding (11.3% vs. 5.5%, Po.001) non-ocular because they did not meet the predefined case were actually more common at triage among bleeding were definition (n = 260) were also excluded from patients without EVD than those with EVD. more common analysis. Finally, 45 patients with missing data Among epidemiologic variables, any contact on EVD outcomes (final diagnosis and/or disposi- with a sick person (82.1% vs. 21.5%, Po.001) and among patients tion) were also excluded, leaving 2,351 separate attendance at a funeral (39.7% vs. 8.8%, Po.001) without EVD. patient admissions for analysis (Figure 1). were far more common among patients with EVD. Eating bush meat and working in health care were both uncommon at triage and were Longitudinal and Geographic Data not associated with a final EVD diagnosis, while Figure 2 shows total patient admissions by ETU recent travel was somewhat more likely in patients by epidemiological week between September 15, without EVD. 20% of the 2014, and September 15, 2015. Overall, 1,524 (64.8%) patients were admitted to the 3 ETUs in patients admitted Outcome Data Sierra Leone, and 827 (45.2%) were admitted to IMC ETUs for to the 2 ETUs in Liberia. Figure 3 shows the Among all patients admitted for triage, 470 (20%) triage tested distribution of patients’ geographic origin by both tested positive for EVD, including 14 patients positive for EVD.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 399 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

FIGURE 1. Study Flow Diagram, Liberia and Sierra Leone ETUs, September 15, 2014, to September 15, 2015

Abbreviations: EVD, Ebola virus disease; ETU, Ebola treatment unit.

with probable EVD who died before laboratory Average length of stay was 4.7 (standard testing. Of these 470 patients, 197 recovered to deviation [SD] = 3.9) days for all admitted patients, discharge, 5 were transferred to other facilities although this differed greatly based on diagnosis and (final outcome unknown), and 268 died. The outcome (Table 4). The average length of stay for overall case fatality ratio (CFR) was 57%. Among EVD+ patients who recovered was 14.7 (SD = 5.5) EVD- patients, 156 of 1,881 admissions also died days; the average for EVD+ patients who died was during their stay in the ETU, for an overall CFR of 5.6 (SD = 3.2) days (Po.001). For EVD- patients, 8.1%. Figure 4 shows the total number of patients average length of stay was 3.6 (SD = 1.6) days for admitted per week across all 5 ETUs, by diagnosis those who survived and 2.3 (SD = 1.4) days for those and outcome. who died (Po.001).

Global Health: Science and Practice 2016 | Volume 4 | Number 3 400 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

FIGURE 2. Patient Admissions by ETUa and Epidemiological Week, Liberia and Sierra Leone, September 15, 2014, to September 15, 2015 (N=2,351)

Abbreviation: ETU, Ebola treatment unit. a ETUs were located in Bong County and Margibi County in Liberia, and in Kambia, Lunsar, and Makeni in Sierra Leone.

Outcome by Age, Sex, and Location patients over 55 had a mortality odds ratio of More men than 4.0 (95% CI, 1.9 to 8.7; Po.001). No significant women were Figure 5A and Figure 5B show the proportion of differences were found among EVD+ patients in patients diagnosed with EVD by age and by sex. admitted to IMC the odds of death based on sex (OR = 1.0l; 95% CI, Although a higher absolute number of patients ETUs. However, 0.7 to 1.4; P = .09) or country of origin (OR = 0.8; admitted to the 5 ETUs were men (53.4% vs. a greater 95% CI, 0.5 to 1.1; P = .15). 46.6%; see Table 2), a much larger proportion of proportion of the women admitted were actually diagnosed with women admitted EVD (25.6% vs. 15.2%, Po.001). Figure 5C and were diagnosed DISCUSSION Figure 5D show outcomes for EVD+ patients with EVD. (i.e., deceased or recovered) by age and sex. With nearly 30,000 cases of EVD, the 2014 Among patients Table 5 summarizes the CFR among EVD+ outbreak in West Africa dwarfed all prior epi- patients by age, sex, and location. As shown in demics of viral hemorrhagic fever.1 At its height, with EVD, those Figure 5 and Table 5, EVD+ patients ages 15 to this Ebola epidemic completely overwhelmed the ages 15 to 24 had 24 had the lowest CFR (38.0%), while the limited response capacity of the 3 most-affected the lowest case youngest and oldest patients had the highest countries and led to an untold number of fatality rate; the CFRs (89.1% for patients under 5; 71.4% for secondary deaths due to breakdowns in the local youngest and patients over 55). As compared with patients aged health care and public health systems.19,20 How- oldest patients 15 to 24, children under 5 had a mortality odds ever, as horrific as this epidemic has been for the had the highest ratio of 12.8 (95% CI, 4.8 to 41.3; Po.001), while people of West Africa, it likely could have been far case fatality rates.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 401 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

1 month earlier could have averted an additional 12,500 cases.21 FIGURE 3. Geographic Distribution of Patients (A) Admitted and IMC, in collaboration with local ministries of (B) Conﬁrmed EVD Positive, Liberia and Sierra Leone, health, contributed to the 2014 Ebola response September 15, 2014, to September 15, 2015 in West Africa by opening 5 ETUs to manage suspected and confirmed Ebola cases. IMC trained health workers in infection prevention and control, EVD clinical management, and ETU operations and worked in communities to build awareness around Ebola response activities. While the efforts of IMC and other humanitarian organizations to provide direct medical care to patients and limit the spread of the epidemic has been well recorded by the lay media, another critical activity IMC engaged in has gone almost completely unnoticed: Staff at IMC ETUs collected clinical and epidemiologic data in the most austere and difficult of circumstances.22,23 When IMC first launched its Ebola response in Liberia in September of 2014, very little was known about optimal prevention, treatment, or management strategies for a large-scale EVD outbreak. Despite more than 2 dozen prior outbreaks over the past 4 decades, little empirical evidence existed to guide response operations at the start of this outbreak.22 While recent reports have documented the experiences of individual ETUs in Guinea and Sierra Leone, no prior studies have presented data from multiple ETUs across multiple countries run by the same organization with similar clinical protocols.5-9 The lack of empiric evidence made it difficult to develop standardized clinical protocols for patient care. More importantly, the lack of evidence made it nearly impossible to prioritize our limited available resources for those who might benefit the most, especially early in the response. As a result of the data collected by our teams in Liberia and Sierra Leone, we know far more about this disease now than when we began our 2014 response. We now have a better understanding of the demographics of patients affected, the expected length of stay in an ETU based on patient diagnosis and outcome, and the expected mortality, for patients both with and with- Abbreviation: EVD, Ebola virus disease. out EVD, in a resource-limited environment. We found, for instance, that although fewer women will visit an ETU, a larger proportion of them worse had the international community failed will have EVD. Where prior publications gener- entirely to respond. Recent mathematical model- ally stratified patients into 2 age groupings ing from Sierra Leone suggests that the introduc- (young vs. old), we chose to look at patients in tion of treatment beds for patients with EVD 10-year blocks to get a more granular understand- between June 2014 and February 2015 averted an ing of the effect of age on diagnosis of EVD and estimated 56,600 cases, while opening ETUs just patient mortality. This more granular presentation

Global Health: Science and Practice 2016 | Volume 4 | Number 3 402 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

TABLE 2. Outcomes Among All Admitted Patients by EVD Status, Sex, Age, and Location, Liberia and Sierra Leone, September 15, 2014, to September 15, 2015

EVD+ EVD+ EVD+ EVD+ EVD- EVD- Recovered Deceaseda Transferred Total Discharged Deceased EVD- Total All Patients

All patients 197 (8.4) 268 (11.4) 5 (0.2) 470 (20.0) 1,725 (73.4) 156 (6.6) 1,881 (80) 2,351 (100) Sexb Men 76 (6.1) 107 (8.5) 3 (0.2) 190 (15.2) 959 (76.6) 97 (7.7) 1,062 (84.8) 1,252 (53.4) Women 118 (10.8) 159 (14.6) 2 (0.2) 279 (25.6) 754 (69.1) 58 (5.3) 812 (74.4) 1,091 (46.6) Age, yearsb 0too 1 1 (1.9) 13 (24.5) 0 (0.0) 14 (26.4) 36 (67.9) 3 (5.7) 39 (73.6) 53 (2.3) 1 to 4 4 (2.4) 28 (16.5) 0 (0.0) 32 (18.8) 127 (74.7) 11 (6.5) 138 (81.2) 170 (7.3) 5 to 14 36 (15.5) 29 (12.4) 1 (0.4) 66 (28.3) 158 (67.8) 9 (3.9) 167 (71.7) 233 (10.0) 15 to 24 44 (10.8) 27 (6.7) 1 (0.2) 72 (17.7) 312 (76.8) 22 (5.4) 334 (82.3) 406 (17.4) 25 to 34 45 (8.1) 42 (7.6) 2 (0.4) 89 (16.1) 434 (78.5) 30 (5.4) 464 (83.9) 553 (23.6) 35 to 44 33 (8.8) 53 (14.2) 1 (0.3) 87 (23.3) 260 (69.5) 27 (7.2) 287 (76.7) 374 (16.0) 45 to 54 17 (5.8) 36 (12.3) 0 (0.0) 53 (18.1) 214 (73.0) 26 (8.9) 240 (81.9) 293 (12.5) Z 55 16 (6.2) 40 (15.5) 0 (0.0) 56 (21.7) 175 (67.8) 27 (10.5) 202 (78.3) 258 (11.0) Country Sierra Leone 114 (7.5) 173 (11.4) 5 (0.3) 292 (19.2) 1,122 (73.6) 110 (7.2) 1,232 (80.8) 1,524 (64.8) Liberia 83 (10.0) 95 (11.5) 0 (0.0) 178 (21.5) 603 (72.9) 46 (5.6) 649 (78.5) 827 (35.2)

Abbreviation: EVD, Ebola virus disease. All data reported as No. (%). a Patients with suspected EVD (n = 14) included in EVD+ deceased. b Missing values not included. of age revealed some interesting nuances, such as lessons about the many challenges to collecting highlighting the age groups with the lowest CFR high-quality data during an epidemic, and the (15–24 years) and the highest mortality (children various ways in which these challenges can be under 5 and adults over 55). Our data confirmed overcome. Our data conﬁrmed that high-risk epidemiological factors for contract- that high-risk ing EVD include contact with a sick person and factors for attendance at a funeral, but found that eating Major Challenges to Data Collection and contracting EVD bush meat was not a significant risk factor for Lessons Learned include contact Lack of Data Standardization EVD, and that recent travel was more likely in with a sick person patients without EVD. One of the greatest challenges in building our and attendance In addition, prior analyses of our data set have data was the lack of standardization in the at a funeral. helped us to develop better tools for EVD screen- data collected across different countries and dif- ing.24 We have planned further analyses of our ferent ETUs. Despite being managed by the same data set as well, which we expect will continue to organization, the various ETUs collected differ- help us learn about the natural history of this dis- ent types of clinical and epidemiologic data in ease and the best ways to diagnose and manage somewhat different formats, and in some cases it in resource-limited settings. Our experience in the types of data collected changed over time. West Africa, however, also taught us important This was due to a variety of factors, including the

Global Health: Science and Practice 2016 | Volume 4 | Number 3 403 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

collection forms; and the lack of prior empiric evidence on which data elements were most important to collect in the context of EVD. Some variables, such as eating bush meat or recent travel, were collected in only 1 of the 2 countries. In some cases, subtle differences An aerial view of the Lunsar Ebola treatment unit in Sierra Leone. between the types of clinical variables collected made it difficult to compare data. For example, we had to group ‘‘red injected eyes,’’ ‘‘conjuncti- need to comply with local guidelines and use vitis,’’ and ‘‘hemorrhagic eyes’’ into a single government-approved triage forms; the emergent variable (‘‘red eyes’’) for analysis. In the future, nature of the epidemic and the lack of time to standardized data forms with clear, consensus- agree upon and disseminate standardized data based definitions of clinical and epidemiologic

TABLE 3. Chi-Square Analysis of Symptoms Reported at Triage by Patients With and Without EVD, Liberia and Sierra Leone, September 15, 2014, to September 15, 2014

EVD+ Patients, No (%) EVD- Patients, No. (%) P Value

Clinical symptom Fever 353 (75.3) 1,481 (78.7) .11 Asthenia (weakness) 337 (71.9) 1,338 (71.1) .76 Loss of appetite 321 (68.4) 1,228 (65.3) .20 Headache 273 (58.2) 1,109 (59.0) .77 Myalgia or arthralgia (muscle or joint pain) 273 (58.2) 1,093 (58.1) .97 Nausea or vomiting 225 (58.0) 947 (60.4) .38 Diarrhea 235 (54.0) 662 (37.4) o .001 Abdominal pain 204 (43.5) 1,016 (54.0) o .001 Red eyesa 129 (27.5) 189 (10.1) o .001 Sore throat or difﬁculty swallowing 112 (23.9) 440 (23.4) .82 Dyspnea (shortness of breath) 110 (23.5) 581 (30.9) .002 Hiccups 57 (12.2) 246 (13.1) .59 Jaundice 24 (5.1) 108 (5.7) .60 Bleeding, non-ocular 26 (5.5) 212 (11.3) o .001 Epidemiologic variable Had contact with someone ill 340 (82.1) 367 (21.5) o .001 Attended funeral 144 (39.7) 146 (8.8) o .001 Had recent travel outside of home district 18 (11.7) 122 (20.0) .02 Worked in health sector 9 (4.2) 62 (6.5) .20 Had contact with bush meat 0 (0.0) 13 (2.5) .06

Abbreviation: EVD, Ebola virus disease. a Includes red injected eyes, conjunctivitis, and hemorrhagic eyes.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 404 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

FIGURE 4. Final Diagnosis and Outcome of Admitted Patients by Epidemiological Week, Liberia and Sierra Leone, September 15, 2014, to September 15, 2015 (N=2,351)

Abbreviation: EVD, Ebola virus disease.

variables should be developed and adopted in virulent disease such as Ebola cannot be over- Standardized data advance by the international community to sup- emphasized. Aside from the initial triage data, collection forms port data collection during outbreaks of viral which we collected by asking patients or family should be hemorrhagic fever and other diseases of epidemic members questions from across a barrier fence at a developed and potential. Such standardized forms will allow for distance of 2 meters, all of the daily symptom data adopted in more consistent and comparable data. Further- and treatment information had to be collected at advance of ’ ’ more, clinical staff working with these clinical the patient sbedsideintheETUshigh-riskzone. disease variables should receive training to ensure that Theproviderscollectingthedataweredressedin outbreaks. knowledge of these variables is properly used in full PPE, which limited both their movements and the field. Evidence collected during this epidemic the time they could spend at the bedside. Clinical by our organization and others on the symptoms, data forms thus had to be simple to fill out; it was signs, and tests that are most predictive of EVD easiest to use mostly check boxes or circles. More- diagnosis and outcomes will likely be helpful in over, the paper forms that the data was collected on developing these new tools; in the end it will could themselves easily become contaminated with require substantial coordination by the interna- highly infectious body fluids, making each form a tional humanitarian community and local gov- biohazard that could not safely be removed from ernments to put these tools into practice. the high-risk zone. Because clinicians washed their (gloved) hands with chlorine in between each Logistical Constraints patient, but did not dry their hands, the paper The severe logistical constraints related to collecting forms also became degraded by chlorine over time. data in the setting of a highly contagious and Extraction of the data from these contaminated and

Global Health: Science and Practice 2016 | Volume 4 | Number 3 405 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

TABLE 4. Average Length of Stay (in Days) in ETU by EVD Status, Outcome, Sex, Age, and Location, Liberia and Sierra Leone, September 15, 2014, to September 15, 2015

EVD+a EVD-

Recovered Deceasedb Discharged Deceased All Patients

n ALOS n ALOS n ALOS n ALOS n ALOS

All patients 197 14.7 268 5.6 1,721 3.6 156 2.3 2,346 4.7 Sexc Men 77 13.6 109 5.6 963 3.6 98 2.2 1,250 4.2 Women 118 15.4 159 5.6 751 3.7 58 2.4 1,088 5.2 Age, yearsc 0too 1 1 18.0 13 6.0 36 4.2 3 1.7 53 4.8 1 to 4 4 23.0 28 6.6 127 3.8 11 1.9 170 4.6 5 to 14 36 15.4 29 5.5 158 3.6 9 3.2 233 5.7 15 to 24 44 14.4 27 4.7 311 3.5 22 2.0 405 4.7 25 to 34 44 13.5 42 5.8 433 3.5 30 2.6 551 4.4 35 to 44 33 15.0 53 5.4 260 3.6 27 1.9 374 4.8 45 to 54 17 13.4 36 5.8 213 3.4 26 2.1 292 4.2 Z 55 16 15.3 40 5.3 174 3.9 27 2.4 257 4.7 Country Sierra Leone 113 14.6 173 5.6 1,119 4.0 110 2.4 1,520 4.9 Liberia 83 14.9 95 5.7 602 2.9 46 2.0 826 4.3

Abbreviations: ALOS, average length of stay; ETU, Ebola treatment unit; EVD, Ebola virus disease. a EVD+ patients who were transferred (n = 5) not included. b Patients with suspected EVD (n = 14) included in EVD+ deceased. c Missing values not included.

In the future, degraded forms required either the laborious and Electronic data could then be transmitted via additional error-prone process of reading the data over the Wi-Fi to a computer located outside of the high- solutions for fence between risk zones and copying it onto new risk zone. Yet while ideal from a data collection collecting data in forms after each set of rounds, attempting to image standpoint, this method would present chal- high-risk zones the forms from across the fence, or sending in lenges in very resource-limited settings: it would safely and additional personnel in full PPE for the sole purpose be necessary to get the tablets prepositioned and efﬁciently should of imaging the forms with a submersible camera configured before the start of an outbreak, and be considered, that could be decontaminated and removed from they would need a working Wi-Fi network. such as the use of the high-risk zone. While each of these methods Another option would be a dual system, using were used at different ETUs, no method was ideal. paper charts that could be scanned and down- electronic medical In the future, other solutions could be consid- loaded onto a laptop within the high-risk zone records. ered for more efficient means of collecting data. after each set of rounds, and then electronically Electronic medical records, accessed through transmitted via Wi-Fi when available or even via a handheld tablets kept in the high-risk zone of the simple Ethernet cable to another computer in the ETU, might offer an easier way to collect data. low-risk zone.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 406 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

FIGURE 5. Final Diagnosis Among Admitted Patients (N=2,351) and Mortality Outcomes Among EVD+ Patients (N=465) by Age and Sex, Liberia and Sierra Leone, September 15, 2014, to September 15, 2015

Abbreviation: EVD, Ebola virus disease.

Data Entry Constraints using Microsoft Access and others using Micro- The final challenge involved entering data from soft Excel. This led to substantial challenges and a paper charts or scanned images into an electronic great deal of extra work to combine the different format that could be analyzed. While local data data sets. Moreover, our initial quality assurance officers entered data into electronic databases in check demonstrated unacceptably high error rates in real time at each of the 5 ETUs, the data entry some of the data, and it had to be reentered from methods and quality control varied by ETU and scanned images of the patient charts. Although we over time depending on the severity of the were eventually able to ensure a low error rate of epidemic (meaning more patients and more data about 1% for our data set overall, much time and to enter) and availability of staff. This led to slight effort could have been saved through closer oversight differences in the way variables were coded and and real-time quality assurance assessments in the the formats in which they were coded. Just as the field. In addition, just as standardized data forms data forms used varied from ETU to ETU, the would have helped, a standardized database with database software also varied, with some ETUs data entry controls (such as preset ranges for certain

Global Health: Science and Practice 2016 | Volume 4 | Number 3 407 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

TABLE 5. Case Fatality Ratios Among Patients With EVD by Gender, Age, and Location, September 15, 2014, to September 15, 2014

EVD+ Recovered, EVD+ Deceased,a Total,b EVD Fatality, No. (%) No. (%) No. (%) OR (95% CI) P Value

All patients 197 (42.4) 268 (57.6) 465 (100) Sexc Men 78 (41.7) 109 (58.3) 187 (40.3) - - Women 118 (42.6) 159 (57.4) 277 (59.7) 1.0 (0.7, 1.4) .09 Age, yearsc 0too1 1 (7.1) 13 (92.9) 14 (3.1) 18.3 (3.3, 463.5) o .001 1 to 4 4 (12.5) 28 (87.5) 32 (7.0) 10.8 (3.7, 40.8) o .001 5 to 14 36 (55.4) 29 (44.6) 65 (14.0) 1.3 (0.7, 2.6) .49 15 to 24 44 (62.0) 27 (38.0) 71 (15.3) - - 25 to 34 45 (51.7) 42 (48.3) 87 (18.8) 1.5 (0.8, 2.9) .20 35 to 44 33 (38.4) 53 (61.6) 86 (18.5) 2.6 (1.4, 5.0) .004 45 to 54 17 (32.1) 36 (67.9) 53 (11.4) 3.4 (1.6, 7.4) .001 Z 55 16 (28.6) 40 (71.4) 56 (12.1) 4.0 (1.9, 8.7) o .001 Country Sierra Leone 114 (39.7) 173 (60.3) 287 (61.7) - - Liberia 83 (46.6) 95 (53.4) 178 (38.3) 0.8 (0.5, 1.1) .15

Abbreviations: CI, conﬁdence interval; EVD, Ebola virus disease; OR, odds ratio. a Patients with suspected EVD (n = 14) included in EVD+ deceased. b EVD+ patients who were transferred (n = 5) not included. c Missing values not included.

Time and effort variables and drop-down lists for other variables, data collection in future epidemics, leading in turn to could be saved with limited ability to enter free-form text), coupled an improved overall humanitarian response. with standardized perhaps with a standardized training for data entry officers, would have simplified the process greatly Acknowledgments: We would like to thank the governments of Liberia, data forms, Sierra Leone, and Guinea for contributing to IMC’s humanitarian standardized and led to less data errors in the field. response. We would also like to thank all of our generous institutional, corporate, foundation, and individual donors, who placed their databases, and confidence and trust in IMC and made our work during the Ebola closer oversight epidemic possible. We thank the United States Naval Medical Research CONCLUSION Center, Public Health England, the European Union Mobile Laboratory and quality Consortium, and the Nigerian laboratory in Kambia District for assurance while Despite the challenges faced, IMC was able to collect providing laboratory data to our ETUs. We acknowledge the support of members of our Research Review Committee and other technical teams data are being roughly 25,000 pages of clinical and epidemiologic who contributed to this research, including Rabih Torbay, Ann collected in the data on more than 2,500 patients in the midst of the Canavan, Dennis Walto, Daniel Rodman, Yoav Rappaport, Samuel largest epidemic of viral hemorrhagic fever to date. Grindley, Syed Hassan, Erin Shedd, Ryan Burbach, Saikrishna field. Madhireddy?, Benedict Adjogah, Melody Xie, Nadezda Sekularac, This data is already helping to improve our opera- Inka Weissbecker, Sean Casey, Farrah Zughni, Natalie Sarles, and tional guidelines and preparedness for the next August Felix. Finally, we also wish to thank medical directors Vanessa Wolfman and Kassahun Gebrehiwot; monitoring and evaluation staff major outbreak and could help a national Ministry of including Annie Abbate, Razia Laghari, Allison Stewart, Alex Tran, Health launch a response to a future Ebola outbreak. Matthew Siakor, David Mansaray, Lamin Bangura, Sorie Sesay, and Hopefully the lessons learned from this experience Joseph Fangawa; and all other data collection officers at our ETUs. can also lead to higher-quality and better-coordinated Competing Interests: None declared.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 408 Lessons Learned From Collecting Clinical Data for Ebola Virus Disease www.ghspjournal.org

REFERENCES 14. Trombley AR, Wachter L, Garrison J, Buckley-Beason VA, Jahrling J, Hensley LE, et al. Comprehensive panel of real-time 1. World Health Organization (WHO). Ebola situation report: 30 March TaqMan polymerase chain reaction assays for detection and 2016. Geneva: WHO; 2016. Available from: http://apps.who.int/ absolute quantification of filoviruses, arenaviruses, and New ebola/current-situation/ebola-situation-report-30-march-2016 World hantaviruses. Am J Trop Med Hyg. 2010;82(5):954–960. 2. World Health Organization (WHO). Ebola virus disease. CrossRef. Medline Geneva: WHO; 2014. Fact Sheet No. 103. Available from: 15. Weller SA, Bailey D, Matthews S, Lumley S, Sweed A, http://www.who.int/mediacentre/factsheets/fs103/en/ Ready D, et al. Evaluation of the Biofire FilmArray BioThreat-E 3. World Health Organization (WHO). Statement on the Test (v2.5) for rapid identification of ebola virus disease 3rd meeting of the IHR Emergency Committee regarding the in heat-treated blood samples obtained in Sierra Leone and 2014 Ebola outbreak in West Africa. Geneva: WHO; 2014. the United Kingdom. J Clin Microbiol. 2016;54(1):114–119. Available from: http://www.who.int/mediacentre/news/ CrossRef. Medline statements/2014/ebola-3rd-ihr-meeting/en/ 16. Broadhurst MJ, Kelly JD, Miller A, Semper A, Bailey D, Groppelli 4. World Health Organization (WHO). Statement on the 1st meeting of E, et al. ReEBOV Antigen Rapid Test kit for point-of-care and the IHR Emergency Committee on the 2014 Ebola outbreak in West laboratory-based testing for Ebola virus disease: a field Africa. Geneva: WHO; 2014. Available from: http://www.who. validation study. Lancet. 2015;386(9996):867–874. int/mediacentre/news/statements/2014/ebola-20140808/en/ CrossRef. Medline 5. Bah EI, Lamah MC, Fletcher T, Jacob ST, Brett-Major DM, Sall AA, 17. Biedron C, Pagano M, Hedt BL, Kilian A, Ratcliff A, Mabunda S, et al. Clinical presentation of patients with Ebola virus disease in et al. An assessment of Lot Quality Assurance Sampling to evaluate Conakry, Guinea. N Engl J Med. 2015;372(1):40–47. CrossRef. malaria outcome indicators: extending malaria indicator surveys. Int Medline J Epidemiol. 2010;39(1):72–79. CrossRef. Medline 6. Lado M, Walker NF, Baker P, Haroon S, Brown CS, Youkee D, et al. 18. Mitsunaga T, Hedt-Gauthier B, Ngizwenayo E, Farmer DB, Clinical features of patients isolated for suspected Ebola virus disease Karamaga A, Drobac P, et al. Utilizing community health worker at Connaught Hospital, Freetown, Sierra Leone: a retrospective cohort data for program management and evaluation: systems for data study. Lancet Infect Dis. 2015;15(9):1024–1033. CrossRef. Medline quality assessments and baseline results from Rwanda. Soc Sci 7. Hunt L, Gupta-Wright A, Simms V, Tamba F, Knott V, Tamba K, Med. 2013;85:87–92. CrossRef. Medline et al. Clinical presentation, biochemical, and haematological 19. Kirigia JM, Masiye F, Kirigia DG, Akweongo P. Indirect costs parameters and their association with outcome in patients with associated with deaths from the Ebola virus disease in West Ebola virus disease: an observational cohort study. Lancet Infect Africa. Infect Dis Poverty. 2015;4(1):45. CrossRef. Medline Dis. 2015;15(11):1292–1299. CrossRef. Medline 20. Walker PGT, White MT, Griffin JT, Reynolds A, Ferguson NM, 8. Schieffelin JS, Shaffer JG, Goba A, Bgakie M, Gire SK, Ghani AC. Malaria morbidity and mortality in Ebola-affected Colubri A, et al. Clinical illness and outcomes in patients with countries caused by decreased health-care capacity, and the Ebola in Sierra Leone. N Engl J Med. 2014;371(22):2092–2100. potential effect of mitigation strategies: a modelling analysis. CrossRef. Medline Lancet Infect Dis. 2015;15(7):825–832. CrossRef. Medline 9. Dallatomasina S, Crestani R, Sylvester Squire J, Declerk H, 21. Kucharski AJ, Camacho A, Flasche S, Glover RE, Edmunds WJ, Caleo GM, Wolz A, et al. Ebola outbreak in rural West Africa: Funk S. Measuring the impact of Ebola control measures in Sierra epidemiology, clinical features and outcomes. Trop Med Int Leone. Proc Natl Acad Sci U S A. 2015;112(46):14366–14371. Health. 2015;20(4):448–454. CrossRef. Medline CrossRef. Medline 10. Kelly JD. Make diagnostic centres a priority for Ebola crisis. 22. Bausch DG, Sprecher AG, Jeffs B, Boumandouki P. Treatment of Nature. 2014;513(7517):145. CrossRef. Medline Marburg and Ebola hemorrhagic fevers: a strategy for testing 11. Dhillon RS, Srikrishna D, Sachs J. Controlling Ebola: next steps. new drugs and vaccines under outbreak conditions. Antiviral Res. Lancet. 2014;384(9952):1409–1411. CrossRef. Medline 2008;78(1):150–161. CrossRef. Medline 12. Me´decins Sans Frontie`res (MSF). Filovirus haemorrhagic fever 23. Glennerster R, M’cleod H, Suri T. How bad data fed the Ebola guideline. Barcelona: MSF; 2008. Available from: https://www. epidemic. New York Times. 2015 Jan 30. Available from: http:// medbox.org/ebola-guidelines/filovirus-haemorrhagic-fever- www.nytimes.com/2015/01/31/opinion/how-bad-data-fed- guideline/preview the-ebola-epidemic.html?_r = 0 13. World Health Organization (WHO). Clinical management of 24. Levine AC, Shetty PP, Burbach R, Cheemalapati S, Glavis-Bloom patients with viral haemorrhagic fever: a pocket guide for the J, Wiskel T, et al. Derivation and internal validation of the Ebola front-line health worker. Geneva: WHO; 2014. Available from: prediction score for risk stratification of patients with suspected http://apps.who.int/iris/bitstream/10665/130883/2/ ebola virus disease. Ann Emerg Med. 2015;66(3):285–293.e1. WHO_HSE_PED_AIP_14.05.pdf CrossRef. Medline

Peer Reviewed

Received: 2016 Jun 3; Accepted: 2016 Aug 23

Cite this article as: Roshania R, Mallow M, Dunbar N, Mansary D, Shetty P, Lyon T, et al. Successful implementation of a multicountry clinical surveillance and data collection system for Ebola virus disease in West Africa: ﬁndings and lessons learned. Glob Health Sci Pract. 2016;4(3): 394-409. http://dx.doi.org/10.9745/GHSP-D-16-00186.

& Roshania et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi.org/ 10.9745/GHSP-D-16-00186.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 409 ORIGINAL ARTICLE

Safety and Acceptability of Community-Based Distribution of Injectable Contraceptives: A Pilot Project in Mozambique

Ana Jacinto,a Mahomed Riaz Mobaracaly,a Momade Bay Usta´b,b Cassimo Bique,b Cassandra Blazer,c Karen Weidert,c Ndola Pratac

ABSTRACT Mozambique has witnessed a climbing total fertility rate in the last 20 years. Nearly one-third of married women have an unmet need for family planning, but the supply of family planning services is not meeting the demand. This study aimed to explore the safety and effectiveness of training 2 cadres of community health workers—traditional birth attendants (TBAs) and agentes polivalentes elementares (APEs) (polyvalent elementary health workers)—to administer the injectable contraceptive depot-medroxyprogesterone acetate (DMPA), and to provide evidence to policy makers on the feasibility of expanding community-based distribution of DMPA in areas where TBAs and APEs are present. A total of 1,432 women enrolled in the study between February 2014 and April 2015. The majority (63% to 66%) of women in the study started using contraception for the ﬁrst time during the study period, and most women (over 66%) did not report side effects at the 3-month and 6-month follow-up visits. Very few (less than 0.5%) experienced morbidities at the injection site on the arm. Satisfaction with the performance of TBAs and APEs was high and improved over the study period. Overall, the project showed a high continuation rate (81.1%) after 3 injections, with TBA clients having signiﬁcantly higher continuation rates than APE clients after 3 months and after 6 months. Clients’ reported willingness to pay for DMPA (64%) highlights the latent demand for modern contraceptives. Given Mozambique’s largely rural population and critical health care workforce shortage, community-based provision of family planning in general and of injectable contraceptives in particular, which has been shown to be safe, effective, and acceptable, is of crucial importance. This study demonstrates that community-based distribution of injectable contraceptives can provide access to family planning to a large group of women that previously had little or no access.

INTRODUCTION 2011, although knowledge of at least one contraceptive method was universal.1 Modern contraceptive use ap- Mozambique has witnessed a climbing total fertility pears to be strongly correlated with higher wealth and rate in the last 20 years despite declining fertility education, as well as urban residence. For instance, 3% of rates in the East African region. For example, the fertility women in the lowest wealth quintile were currently rate increased from 5.2 in 1997 to 5.9 at the time of the using modern family planning methods at the time of most recent Demographic and Health Survey (DHS) in 1,2 the survey, compared with 30% in the highest quintile, 2011. Just over 11% of women in union and 30% of and only 5% of women with no education were using unmarried women were using modern contraception in modern contraception compared with 31% of those who had reached the secondary level or more. Only 7% of a Pathﬁnder International, Watertown, MA, USA. women in rural areas were using modern contraception b Mozambican Association of Obstetricians and Gynaecologist, Maputo, compared with 21% of women in urban areas. Further- Mozambique. more, nearly one-third of married women in Mozambique c University of California, Bixby Center for Population, Health and Sustainability, Berkeley, CA, USA. have an unmet need for family planning. The supply Correspondence to Ndola Prata ([email protected]). and distribution of family planning services and

Global Health: Science and Practice 2016 | Volume 4 | Number 3 410 CBD of Injectables in Mozambique www.ghspjournal.org

contraceptives did not meet demand, as 40.1% In December 2013, the MOH trained 2,270 APEs In 2003, the of women surveyed expressed a desire for any on this new family planning package.16 preferred method method of contraception, with only 29% of Mozambique’s Family Planning Strategy for future 1 demand met. The preferred method for future 2010–2014 was released at the same time the contraceptive use contraceptive use—according to DHS data from APE program restarted. The MOH recognized the was injectable — 2003 when this indicator was last measured need for community involvement and participa- contraceptives. In was injectable contraceptives, for more than tion to improve universal access to family plan- 2011, however, 3 42% of women of reproductive age. However, only ning services and committed to improving access only 4.3% of 4.3% of all women were using injectable contra- through CHWs. The strategy also addressed women were ceptives in 2011, suggesting an unmet need.1 the potential role of traditional birth attendants using this method, Access to contraception itself increases use.4 (TBAs) in the provision of family planning coun- suggesting an In Mozambique, rural women find themselves seling and select methods.18,19 This was an impor- unmet need. cut off from access to family planning services due tant inclusion because TBAs often live in poor, to a shortage of health facilities.5 In 2011, 77% of rural areas that are far from health facilities, and family planning services were delivered through they have direct access to women during labor, Access to the public health system.1 Family planning ser- delivery, and the postpartum period. They may be contraception vices and counseling are typically provided through therefore uniquely suited to providing injectables itself will increase health facilities, and skilled providers are histori- in communities. They can serve as a bridge to the use. In cally the gatekeepers of those services. Evidence formal health system and effectively convey infor- Mozambique, 20 from the past decade, however, shows that com- mation to women in culturally appropriate ways. however, rural munity health workers (CHWs) are making key Very few studies have assessed TBAs as family women lack advances to increase access to injectables in other planning providers in sub-Saharan Africa, and 6-11 21-23 access to sub-Saharan African countries. Although im- these studies showed mixed results. Apilot contraception due plants recently surpassed injectables as the fastest- study conducted in Senegal, however, successfully to a shortage of growing method of contraception in sub-Saharan included matrones, or trained TBAs, to distribute health facilities. Africa,12 injectables are still the most commonly injectables.22 Evidence from used method among married women in the Although policy makers are supportive of CBD other countries region13; the ease of use and convenience of access in Mozambique, there is limited country-specific at the community level is likely a contributing experience on best practices, so stakeholders have shows that factor. called for operations research in Mozambique. community health Community-based distribution (CBD) of In response to the call for more research, we con- workers may help injectables by trusted CHWs may be an effective ducted this study to determine whether APEs and bridge this gap. approach to increasing family planning use in TBAs could safely and effectively administer the in- rural Mozambique quickly, given the success of jectable contraceptive depot-medroxyprogesterone CHWs in countries with similar barriers to access. acetate (DMPA), with high client acceptability, The Mozambique Ministry of Health (MOH) ap- among women in 2 rural districts. proved the revitalization of the national CHW program in 2010 in recognition of the critical importance of CHWs to expand access to basic INTERVENTION AND SETTING primary health care services to communities.14 Some CHWs, called agentes polivalentes elementares From February 2014 to April 2015, Pathfinder (APEs) (polyvalent elementary health workers), International implemented a pilot study on the focus on improving the health of the community distribution of DMPA by both APEs and TBAs—in Recognizing the primarily through health promotion and preven- partnership with the Mozambican Society of important role of tion activities. They serve as a linkage between Obstetricians and Gynecologists and the Bixby CHWs, communities and health facilities, and they pro- Center for Population, Health, and Sustainability. Mozambique vide community case management for HIV/AIDS, The study was conducted in 2 districts in north- revitalized a maternal care, nutrition, and acute illness among ern Mozambique, Chiure and Montepuez, which national program children (diarrhea, malaria, and respiratory infec- are located in the Cabo Delgado Province, where in which CHWs tions). Historically, APEs have not provided in 2011, only 2.9% of married women were using will offer family planning services, although some APEs contraceptives, with just 0.8% using injectable contraception provided contraceptives informally. In 2016, APEs contraceptives.1 The intervention was designed so beginning in will offer a new package of services that includes that TBAs served clients in Montepuez and APEs 2016. provision of pills, condoms, and injectables.15 served clients in Chiure.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 411 CBD of Injectables in Mozambique www.ghspjournal.org

Traditional birth Selection, Training, and Supervision of The providers also distributed condoms and attendants may be Community Health Workers received training on promoting dual protection uniquely suited to All 25 APEs that had worked in the Chiure district against both unintended pregnancy and sexually providing since 2009 were selected to participate in the transmitted infections (STIs) including HIV. injectables in rural study. Only 8 of the 25 APEs were women. The After completing the classroom training, pro- communities predominance of male APEs in this study reflects viders enrolled in stage 2, a practicum on injec- because they the APE program in general, which is 71% male.17 tion administration. All providers practiced giving often live in poor, This gender imbalance is likely a result of the injections until they felt confident. They were rural areas and government requirement that APEs should have considered qualified to administer injections after passing a test in which they provided an injection have direct access completed at least grade 7 to participate in the to a volunteer. to women during program. APEs in Mozambique receive general Stage 3 of the training involved a 1-week labor, delivery, training on health care prevention. Their roles practicum at a health facility in their catchment and the and responsibilities in the communities they serve include health promotion, provision of vitamin A, area. During the practicum, providers were res- postpartum ponsible for administering informed consent to period. deworming, malaria testing and administration of malaria medication, and antenatal care sensitiza- clients; completing medical screening for DMPA; tion. APEs are not officially part of the MOH, enrolling participants and assigning a client num- although they receive a stipend of 1,200 meticais ber; completing the enrollment questionnaire; (US$19) per month from the government as com- providing contraceptive counseling, and adminis- pensation for their participation in the program. tering DMPA with a plan for reinjection. This was External partners provide all funding for the done at the health facility under supervision of program including the stipend.15,17 the physicians leading the training and nurses in the facilities before the study began. The 34 TBAs who participated in the study were There were 3 levels of supervision provided registered with the MOH and worked collabora- during the study implementation: tively with the health facilities in the Montepuez district. All 34 TBAs were women. TBAs participate 1. In the first level of supervision, 18 trained in a nationally recognized training program that nurses (1 per facility in the catchment area) includes hygienic management and infection con- oversaw data collection and clinical super- trol; recognition of danger signs for referral, post- vision. Nurse supervisors were responsible for partum, and umbilical cord care; and mobilizing confirming completion of the enrollment communities to use general preventive health care questionnaire filled out by the study providers services.20 The TBAs were not paid for their parti- and for addressing any clinical issues that cipation in the study and in general do not receive arose during the course of the study. any formal compensation for the work they per- 2. The province coordination team provided a form in the community. However, they do receive second level of supervision on a monthly basis informal monetary and in-kind contributions from to ensure that nurse supervisors performed women in exchange for their services. All partici- their responsibilities as detailed in the study pating TBAs were literate, which was necessary for protocol, including timely administration of study recordkeeping. the 3-month and 6-month follow-up ques- CHWs participated Both APEs and TBAs received standardized tionnaires to clients. in a 10-day training before the study began. Five physicians 3. The study investigators provided a third level standardized conducted a 10-day training composed of 3 stages. of supervision on a quarterly basis by review- training program Stage 1 included classroom training on topics such ing all components of the project including that covered all as family planning methods and counseling, study data entry and management. contraceptive protocol, recruitment and screening requirements, injection administration, infection prevention, and methods, METHODS recruitment and reporting procedures. APEs and TBAs were trained screening on how to counsel women on all family planning The study was a prospective non-randomized methods to promote informed choice. However, requirements, and community intervention trial designed to assess because the providers (APEs and TBAs) would injection the safety, acceptability, and effectiveness of the administer only injectables, the clinical aspects of provision of DMPA by APEs and TBAs as well as administration of the training focused on determining clients’ eligi- continuation outcomes among clients of both DMPA. bility for DMPA and administering the injection. providers. The study design was based on earlier

Global Health: Science and Practice 2016 | Volume 4 | Number 3 412 CBD of Injectables in Mozambique www.ghspjournal.org

successful pilot studies in Ethiopia and Uganda.7,9 We entered the data in Epi Info (version Building on the By building upon these study designs, we hoped 7.1.4.0) and conducted the analysis with Stata study design of to provide a mechanism for standardizing data (version 13). The results presented in this article two successful across countries. In the case of clinical research, include information generated through fre- pilot studies, we standardization has been shown to increase data quency and cross-tabulations. We assessed differ- sought to provide quality, improve data integration and reusability, ences in responses between the APE clients and a mechanism to and enable facilitation of data exchange with TBA clients using chi-square tests for association standardize data 24 partners. among categorical variables and t tests for inde- across countries. Safety, acceptability, effectiveness, and con- pendent samples to determine differences be- tinuation rates were the outcomes of interest. tween the client group means. Discontinuation We compared these outcome rates among clients and continuation rates overall and by provider who received DMPA from APEs and clients who were estimated over time from the first injection to received DMPA from TBAs. We based the sample the second injection and from the first injection to size of 1,000 women on the need to test for non- the third injection. We estimated continuation, inferiority of the services provided by the 2 types discontinuation, and lost-to-follow-up rates using of providers. We assumed a continuation rate of data from the questionnaires at 3 months and 65% after first injection among APE clients and a 6 months. Women who reported receiving their continuation rate of at least 55% among TBA second and third injections represent continuation clients (for a maximum difference of 10% between at 3 months and at 6 months, respectively. Discon- groups) as being equivalent. We also assumed a tinuation during this same period was estimated if loss to follow-up of 20%, a design effect of 2.0, and a woman reported in her 3-month and 6-month similar recruitment rates in all districts. questionnaire that she did not receive her second All women of reproductive age in the com- or third injection. Lost to follow-up was estimated munity were made aware of the project through for women who completed the enrollment ques- existing community meetings led by CHWs and tionnaire, but for whom both the 3- and 6-month given the opportunity to request DMPA from a questionnaires were absent. community provider assigned to their district. Ethical approval for this project was grant- Those who wished to receive DMPA from an APE ed by the Committee for Protection of Human or TBA were recruited for the study. Members of Subjects at the University of California, Berkeley the study team met with potential participants (CPHS # 2012-06-4460) and from the Mozambique and obtained informed consent prior to enroll- Ministry of Health Comite de bioetica para a saúde ment in the study. Providers screened each (IRB00002657, Ref: 197/CNBS/13). participant for eligibility to receive DMPA using a tool and manual developed by FHI 360 and RESULTS based on World Health Organization recommendations. 25 Women who were medically ineligible A total of 1,432 eligible women enrolled in the or did not want to participate in the study were study between February and November 2014. referred to the nearest health facility. Participants TBAs recruited 782 women and APEs recruited did not receive any compensation, but they 649 women for the study. The summary of enroll- received DMPA free of charge during the study. ment and follow-up data is illustrated in Figure 1. Participants were asked to complete 3 orally At the 3-month follow-up visit, 1,242 women The 1,432 women administered questionnaires: 1 at enrollment (first participated in the questionnaire; 48 women who enrolled in injection), 1 at a follow-up visit after 3 months refused to respond to the questionnaire, resulting the study had a (13 weeks, second injection), and 1 at a follow-up in a response rate of 96%. At the 6-month visit, high response rate visit after 6 months (26 weeks, third injection). 1,264 women responded to the questionnaire, to questionnaires The questionnaires were first developed in with a response rate of 98.6%; this included given at the English and then translated into Portuguese. 22 women who refused the 3-month question- 3-month visit and The data collected included sociodemographic naire, and were therefore assumed lost to follow- the 6-month visit— characteristics, satisfaction with DMPA as a up. These 22 women were asked about their 96% and 98.6%, method, satisfaction with the provider, quality second injection; if they received it, they were respectively. of the services provided, knowledge of and added to the continuation rates. experience with side effects, reasons for discon- Table 1 outlines demographic and other key tinuing the injections, and willingness to pay for indicators for the 1,432 women who enrolled in the injectables. study. Clients of APEs and TBAs were statistically

Global Health: Science and Practice 2016 | Volume 4 | Number 3 413 CBD of Injectables in Mozambique www.ghspjournal.org

FIGURE 1. Summary of Client Enrollment and Follow-Up

similar on most variables. Although most clients use of DMPA. Figure 2 provides details on previous in both groups reported no education, clients of types of methods used. APEs had significantly less education than clients More than 60% of both APE and TBA clients of TBAs. reported duration of effectiveness as the reason Table 2 describes the rates of discontinuation why they preferred DMPA. About 29% of TBA and loss to follow-up by group–APE clients and clients and 20% of APE clients reported that they TBA clients. Discontinuation rates were estimated chose to use DMPA because their husbands perfor 2 time frames: (1) between enrollment and mitted this method. Additionally, nearly 10% of the second injection; and (2) between enrollment TBA clients reported that they chose DMPA for The continuation and the third injection. Overall, the continuation the convenience, and 24.2% of APE clients liked rate was high rate was high (81.1%) in both groups after DMPA because there were ‘‘fewer side effects’’ (81.1%) in both 3 injections, with TBA clients having significantly (data not shown). groups after higher continuation rates both at 3 months and APE clients reported receiving more counseling 3 injections, with at 6 months. Loss-to-follow-up rates were 13.8% than TBA clients on both side effects and STIs TBA clients having for the entire project period between enrollment including HIV. However, the percentage of clients signiﬁcantly and the third injection at 6 months; however, who reported receiving counseling on side effects higher APE clients had 20.8% loss to follow-up compared and STIs improved between the 3-month and the continuation rates with 7.8% among TBA clients. For APE clients, 6-month follow-up visits in both groups (Figure 3). the most common reason given for discontinua- The percentage of APE clients who reported being both at 3 months tion at 3 months (50.6%) and 6 months (53.6%) offered condoms in addition to DMPA was also and at 6 months. was that the woman was planning to get her higher than TBA clients. At 3 months, 31% of TBA injection but had not yet reached the provider to clients and 48% of APE clients reported being offered receive it. Both TBAs and APEs provided injec- condoms. At 6 months, those percentages were APE clients tions at their own home, their client’s home, or 21.3% and 54.8%, respectively (data not shown). reported receiving another place in the community as determined by Most clients reported no side effects from DMPA more counseling the provider and client. TBA clients were more at both the 3-month and 6-month follow-up visits on side effects and likely to state ‘‘other reasons’’ without explaining (Figure 4). At 3 months, less than 10% of women STIs including HIV why they missed injections at both 3 months and reported experiencing amenorrhea, spotting, heavy than TBA clients. at 6 months (data not shown). bleeding, or irregular bleeding. Fewer side effects The majority of women enrolled in the study were generally reported at 6 months than at the had never used a contraceptive method before 3-month time point. Less than 0.5% of women re- (63% of TBA clients and 66% of APE clients). ported any morbidities at the injection site, including Approximately 30% of all clients reported previous induration or abscess (data not shown).

Global Health: Science and Practice 2016 | Volume 4 | Number 3 414 CBD of Injectables in Mozambique www.ghspjournal.org

TABLE 1. Background Characteristics of Enrolled Women, by Provider (N = 1,431)

TBA Clients APE Clients (n = 782) (n = 649)

Age at enrollment, years, mean (SD) 29.3 (6.9) 29.9 (7.6) No. of living children, mean (SD) 4.2 (2.1) 4.8 (2.6) Marital status, No. (%) Married/living together 655 (83.8) 539 (83.1) Single, never married 52 (6.7) 52 (8.0) Divorced/separated/widowed 64 (8.2) 40 (6.2) Education, No. (%) None 488 (62.4) 472 (72.7)* Only read and write 38 (4.9) 49 (7.6) Primary 246 (31.5) 177 (18.0)* Secondary or higher 6 (0.8) 6 (0.9) Husband supportive of using DMPA, No. (%) Yes 614 (78.5) 526 (81.1) No 47 (6.0) 46 (7.1) Husband not aware 28 (3.6) 16 (2.5) Not married/does not know 70 (9.0) 48 (7.4)

Abbreviations: APE, agente polivalente elementare (polyvalent elementary health worker); DMPA, depot- medroxyprogesterone acetate; SD, standard deviation; TBA, traditional birth attendant. Note: Percentages include missing, not shown. One client of the total recruited was missing provider information. * Po.05 for comparison of TBA vs. APE.

The majority of women were satisfied with women were willing to pay was about 39 meticais both the method and their provider. Among TBA (US$1.07) and 5 meticais (US$0.13) among TBA and APE clients, 74.7% and 88.2%, respectively, clients and APE clients, respectively. At the were satisfied with DMPA at 3 months. At 6-month visit, women were willing to pay 40 6 months, 90.1% of TBA clients and 89.2% of meticais ($1.10) and 7 meticais ($0.19), respec- APE clients reported satisfaction with the method tively (data not shown). (data not shown). At 3 months, 73.7% of TBA clients and 89.1% of APE clients reported satisfaction with their provider. At 6 months, reported DISCUSSION client satisfaction with the provider improved to Although CBD of injectable contraceptives is no 89.8% among TBA clients and 94.1% among APE longer a novel idea in sub-Saharan Africa, the The majority of clients (data not shown). process of translating community-based health women in the Overall, 64% of women in the study reported care policies into effective programs is not always study were willing that they were willing to pay for DMPA, but TBA straightforward. For example, policy makers in to pay for DMPA clients were much more willing to pay than APE Mozambique were supportive of CBD, but there (64%), with TBA clients (data not shown). For women in both was limited country-specific experience and no clients much more client groups, the mean amount they were willing country-level data to support the policy. Conse- willing to pay than to pay was approximately 34 meticais (US$0.93). quently, this study generated evidence to support APE clients. At the 3-month visit, the mean amount that CBD of DMPA in northern Mozambique and

Global Health: Science and Practice 2016 | Volume 4 | Number 3 415 CBD of Injectables in Mozambique www.ghspjournal.org

TABLE 2. Discontinuation and Loss to Follow-Up, by Provider (N = 1,432)

Total After 3 Second Injection Third Injection Injections

TBA Clients APE Clients TBA Clients APE Clients All Clients

Received injection, No. (%) 627 (80.2) 442 (68.1)* 716 (91.6) 445 (68.6)* 1,161 (81.1) Discontinued, No. (%) 11 (1.4) 89 (13.7)* 5 (0.006) 69 (10.6)* 74 (5.2) (did not receive injection) Lost to follow-up, No. (%) 144 (18.4) 118 (18.2) 61 (7.8) 135 (20.8)* 197 (13.8) (includes missing data) Total number of clients at 782 649 782 649 1,432 enrollment Abbreviations: APE, agente polivalente elementare (polyvalent elementary health worker); TBA, traditional birth attendant. Note: Percentages include missing, not shown. One client of the total recruited was missing provider information. * Po.05 for comparison of TBA vs. APE.

FIGURE 2. Previous Use of Contraception Among Study Population, by Provider Type (N=1,432)

Abbreviations: APE, agente polivalente elementare (polyvalent elementary health worker); DMPA, depot- medroxyprogesterone acetate; IUD, intrauterine device; TBA, traditional birth attendant.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 416 CBD of Injectables in Mozambique www.ghspjournal.org

FIGURE 3. Percentage of Women Counseled on Side Effects and STIs Including HIV at Follow-Up Visits, by Provider (N=1,432)

Abbreviations: APE, agente polivalente elementare (polyvalent elementary health worker); STI, sexually transmitted infection; TBA, traditional birth attendant.

found that provision of injectable contraceptives for sexual and reproductive health in many rural by APEs and TBAs was feasible, safe, effective, communities in Mozambique. The high continua- and acceptable among women. Very few morbid- tion rates among TBA clients might indicate that ities at the injection site and no deaths were TBAs can conduct more rigorous client follow-up reported. The study demonstrated that APEs and compared with APEs, or it may be that women TBAs can improve contraceptive access and use in interact with their TBAs more regularly. rural Mozambican communities; in fact, the Women may also place more trust in their majority of women in the study started using TBAs, or have higher satisfaction with TBAs as a contraception for the first time during the study family planning provider. Nearly 17% of births in period, and satisfaction with community-based the country in the 5 years preceding the most providers was high and improved over the entire recent 2011 DHS were assisted by a TBA,1 which study period. is up from 11% in 2003.3 This suggests that Continuation rates in both clients groups women and TBAs have an important relationship were high overall and similar to rates found in in Mozambique. other pilot studies on CBD of DMPA in Kenya and This study confirms previous findings that Ethiopia.7,11 Interestingly, TBA clients had sig- TBAs are uniquely poised to address critical gaps in nificantly higher continuation rates than APE postpartum contraceptive uptake.22,26 This is an clients. This difference could indicate a gender important finding given that postpartum women barrier in the program because the majority of are among those with the greatest unmet need for APEs were men and all TBAs were women; family planning and often do not receive family however, we believe it suggests that TBAs have a planning services.27 In fact, a previous analysis of critical role in linking women to reproductive DHS data from 27 low- and middle-income coun- health services at the community level. For tries found that although 95% of postpartum example, TBAs are involved in initiation rituals women wanted to avoid a pregnancy within the that take place before women become sexually next 2 years, only 30% were using contraception.28 active in Cabo Delgado, the province included in Our study generated pivotal evidence to support this study, and TBAs are also considered advisors the inclusion of TBAs in delivery of injectable

Global Health: Science and Practice 2016 | Volume 4 | Number 3 417 CBD of Injectables in Mozambique www.ghspjournal.org

FIGURE 4. Reported Side Effects Following Second and Third Injections of DMPA, by Provider (N=1432)

Abbreviations: APE, agente polivalente elementare (polyvalent elementary health worker); DMPA, depot- medroxyprogesterone acetate; TBA, traditional birth attendant.

contraceptives. However, future research is needed TBAs may require additional support to provide to understand the important linkage between these services consistently and adequately. Find- TBAs as family planning providers and high con- ings from a recent systematic review found a traceptive continuation rates among their clients. moderate increased risk of HIV among women Future qualitative research into women’s gender using DMPA in the general population.29 With an preference among providers may also illuminate HIV prevalence of 11.1% in Mozambique,30 HIV some of the differences between continuation counseling and dual protection should be prior- results of APEs and TBAs. itized in future trainings, and TBAs should receive In addition to recognizing the advantages of regular refreshers that emphasize the importance including TBAs in family planning, it is important of these issues. to address the challenges surrounding limited STI APE clients reported receiving more STI counseling related to TBA provision of services. counseling than TBA clients; however, the low TBAs provided substantially less STI counseling rates of reported side effects among all clients (36%) than APEs (70%) did at the 3-month follow- may indicate the quality of TBAs’ counseling on up visit, and only slightly improved at 6 months side effects, and may also have influenced con- (45%). TBAs were also less likely to offer condoms tinuation rates. Typical side effects associated in addition to DMPA for dual protection. Consider- with DMPA use were reported at lower rates (10% ing their lower education levels compared with at 3 months) in this study than in similar studies APEs, and relative inexperience with trainings, in Ethiopia and Uganda, where approximately

Global Health: Science and Practice 2016 | Volume 4 | Number 3 418 CBD of Injectables in Mozambique www.ghspjournal.org

20% to 30% of women reported side effects.7,9 It is 3-month questionnaire and 1 woman for the difficult to draw conclusions without additional 6-month questionnaire). research into the quality of services and perceptions of clients. Results of research on side effects and continuation have been mixed and both CONCLUSION individual characteristics of the users and ser- Given Mozambique’s largely rural population and vice delivery factors have been shown to affect critical health care workforce shortage, a reliance 31 both reporting of side effects and continuation. on safe, effective, and acceptable community- Nonetheless, several studies have found that both based family planning provision is of crucial providers’ attitudes and the information they importance. Evidence-based strategies should provide may influence a woman’s perception of guide programmatic implementation. In the case 31-33 bleeding. of Mozambique, 2 strategies could be adopted Women’s overall willingness to pay for DMPA based on evidence generated through this study: (64%) is also noteworthy and highlights the (1) TBAs should be included as community-based demand for injectable contraceptives among distributors of family planning services; and Mozambican women as well as an opportunity for (2) public-sector programs should include client scaling up CBD in Mozambique. Although donors fees based on willingness to pay and/or allow for contribute more than 50% of the country’s total revitalization of private-sector distribution of budget for health, with the United Nations contraceptives in rural areas. This study demon- Population Fund (UNFPA) and the U.S. Agency strated that CBD of injectables can provide access for International Development (USAID) purchas- to a large group of women that previously had 19 ing Mozambique’s contraceptive supplies, this little to none, considering the tremendous pro- may not always be the case. Cost recovery ap- portion of women using contraception for the proaches for contraceptive delivery may therefore first time during this study. More importantly, it be necessary in the future, even though they are also highlighted the central role that TBAs can not currently in place in the public sector. TBA play in a context similar to rural Mozambique. In clients were more willing to pay for injectable regions where the fertility rate is high, births are contraceptives than APE clients, which may be too closely spaced, and women face innumerable related to women’s expectations. Trained TBAs obstacles in reaching health facilities, TBAs can have reported informal compensation, both in- provide family planning services that promote kind and monetary, for births that they have healthy timing of pregnancies with high con- 20 attended, whereas APE services are paid for by tinuation rates. To meet the contraceptive desires the government and free to the client. Conse- of vulnerable women, policy makers and program quently, TBA family planning clients might be staff should consider CBD of injectables, espe- more comfortable with family planning user fees. cially models where TBAs are trained and trusted to provide safe and acceptable ongoing care.

Limitations Acknowledgments: We wish to express our gratitude to USAID/ In this study, 197 women (14%) were lost to Washington for providing the financial support to conduct this study and those at the USAID Mission in Mozambique for their guidance follow-up, and therefore it cannot be determined and technical oversight during the development of this pilot study. We whether these women discontinued use of DMPA. also wish to acknowledge the Provincial Health Directorate and Provincial Medical Chief Officer of Cabo Delgado. Additionally, verification of DMPA injection was completed at the 3- and 6-month follow-up visits. Competing Interests: None declared. Even among women who were not lost to follow- up, the response rate was not 100%. Women who REFERENCES refused to respond to the questionnaire were not captured in the continuation and discontinuation 1. Ministe´ro da Sau´de (MISAU); Instituto Nacional de Estatı´stica (INE); ICF International. Moçambique inque´rito demogra´fico rates, with the exception of the 22 women who edesau´de 2011. Maputo (Mozambique): MISAU; 2011. refused the 3-month questionnaire but accepted Co-published by ICF International. Available from: http:// the 6-month questionnaire (Figure 1). The lost to dhsprogram.com/pubs/pdf/FR266/FR266.pdf follow-up total does include women who were 2. Instituto Nacional de Estatı´stica (INE); Ministe´ro da Sau´de; interviewed, but did not have recorded data about MEASURE DHS+/ORC Macro. Moçambique inque´rito demogra´fico e de sau´de 1997. Maputo (Mozambique): INE; whether they received DMPA. The number of 1997. Available from: http://dhsprogram.com/pubs/pdf/ women without data was small (9 women for the FR92/FR92.pdf

Global Health: Science and Practice 2016 | Volume 4 | Number 3 419 CBD of Injectables in Mozambique www.ghspjournal.org

3. National Institute of Statistics (INE) [Mozambique]; ORC Macro. Available from: http://www.reachoutconsortium.org/media/ Moçambique inque´rito demogra´fico e de sau´de 2003. Maputo 1834/mozambiquecountryanalysisjuly2014compressed.pdf (Mozambique): INE; 2003. Co-published by ORC Macro. Available 18. Ministry of Health (MOH) [Mozambique]. Estrate´gia de from: http://dhsprogram.com/pubs/pdf/FR161/FR161.pdf Planeamento Familiar e Contracepçaõ 2011–2015 (2020). 4. Campbell MM, Prata N, Potts M. The impact of freedom on Maputo (Mozambique): MOH; 2010. Available from: fertility decline. J Fam Plann Reprod Health Care. 2013; http://republic-of-moldova.unfpa.org/webdav/site/global/ 39(1):44–50. CrossRef. Medline shared/procurement/10_supply_chain/ 5. Schwitters A, Lederer P, Zilversmit L, Gudo PS, Ramiro I, Cumba L, Mozambique_RHCS_Portugese.pdf et al. Barriers to health care in rural Mozambique: a rapid 19. Curtin L, Kantner A, Alleman P, Thatte N. USAID/Mozambique: ethnographic assessment of planned mobile health clinics for Family planning assessment. Washington (DC): Development and ART. Glob Health Sci Pract. 2015;3(1):109–116. CrossRef. Training Services, Inc.; 2012. Available from: http://pdf.usaid. Medline gov/pdf_docs/PA00HVJ7.pdf 6. Malarcher S, Meirik O, Lebetkin E, Shah I, Spieler J, Stanback J. 20. Gloyd S, Floriano F, Seunda M, Chadreque MA, Nyangezi JM, Provision of DMPA by community health workers: what the Platas A. Impact of traditional birth attendant training in evidence shows. Contraception. 2011;83(6):495–503. CrossRef. Mozambique: a controlled study. J Midwifery Womens Health. Medline 2001;46(4):210–216. CrossRef. Medline 7. Prata N, Gessessew A, Cartwright A, Fraser A. Provision of 21. Mamman-Daura F, DanMusa S, Mai M. Using traditional birth injectable contraceptives in Ethiopia through community-based attendants to increase access to family planning services in select reproductive health agents. Bull World Health Organ. 2011; communities in northern Nigeria. Nigeria: Pathfinder 89(8):556–564. CrossRef. Medline International; 2007. Available from: http://uaps2007.princeton. 8. Skiles MP, Cunningham M, Inglis A, Wilkes B, Hatch B, Bock A, edu/papers/70716 et al. The effect of access to contraceptive services on injectable 22. FHI 360. Senegal: community health workers successfully provide use and demand for family planning in Malawi. Int Perspect Sex intramuscular injectable contraception. Research Triangle Park Reprod Health. 2015;41(01):20–30. CrossRef. Medline (NC): FHI 360; 2013. Available from: http://www.fhi360.org/ 9. Stanback J, Mbonye AK, Bekiita M. Contraceptive injections by sites/default/files/media/documents/community-health- community health workers in Uganda: a nonrandomized workers-intramuscular-depo-senegal.pdf community trial. Bull World Health Organ. 2007;85(10): 23. Phillips JF, Greene WL, Jackson EF. Lessons from community- 768–773. CrossRef. Medline based distribution of family planning in Africa. New York: 10. Hoke TH, Wheeler SB, Lynd K, Green MS, Razafindravony BH, Population Council; 1999. Available from: http://chwcentral. Rasamihajamanana E, et al. Community-based provision of org/sites/default/files/Lessons%20from%20Community- injectable contraceptives in Madagascar: ‘task shifting’ to based%20Distribution%20of%20Family%20Planning%20in expand access to injectable contraceptives. Health Policy Plan. %20Africa.pdf 2012;27(1):52–59. CrossRef. Medline 24. U.S. Institute of Medicine. Chapter 5: standardization to enhance 11. Olawo AA, Bashir I, Solomon M, Stanback J, Ndugga BM, data sharing. In: Sharing clinical research data: workshop Malonza I. ‘‘A cup of tea with our CBD agent ...’’: community summary. Washington (DC): National Academies Press; 2013. provision of injectable contraceptives in Kenya is safe and 25. Mueller MP, Lasway C, Yacobson I, Tumlinson K. Training and feasible. Glob Health Sci Pract. 2013;1(3):308–315. CrossRef. reference guide for a screening checklist to initiate DMPA (or Medline NET-EN). Research Triangle Park (NC): FHI 360; 2008. Available 12. Sheldon WR, Durocher J, Winikoff B, Blum J, Trussell J. How from: https://www.k4health.org/sites/default/files/ effective are the components of active management of the third DMPAguide_0.pdf stage of labor? BMC Pregnancy Childbirth. 2013;13(1):46. 26. Naanyu V, Baliddawa J, Peca E, Karfakis J, Nyagoha N, Koech CrossRef. Medline B. An examination of postpartum family planning in western 13. United Nations, Department of Economics and Social Affairs, Kenya: ‘‘I want to use contraception but I have not been told how Population Division. Trends in contraceptive use worldwide 2015. to do so.’’ Afr J Reprod Health. 2013;17(3):44–53. Medline New York: United Nations; 2015. Available from: http://www. 27. Gaffield ME, Egan S, Temmerman M. It’s about time: WHO un.org/en/development/desa/population/publications/pdf/ and partners release programming strategies for postpartum family/trendsContraceptiveUse2015Report.pdf family planning. Glob Health Sci Pract. 2014;2(1):4–9. CrossRef. 14. Ministry of Health (MOH) [Mozambique]. Programa de Medline revitalizacao dos agentes polivalentes elementares. Maputo 28. Ross JA, Winfrey WL. Contraceptive use, intention to use and (Mozambique): MOH; 2010. unmet need during the extended postpartum period. Int Fam Plan 15. JSI Research & Training Institute, Inc., Advancing Partners and Perspect. 2001;27(1):20–27. CrossRef Communities. Country profile: Mozambique community health 29. Polis CB, Curtis KM, Hannaford PC, Phillips SJ, Chipato T, Kiarie programs. Arlington (VA): JSI Research & Training Institute, Inc.; JN, et al. Update on hormonal contraceptive methods and risk of 2013. Availabe from : https://www.advancingpartners.org/ HIV acquisition in women: a systematic review of epidemiological sites/default/files/landscape/countries/profiles/ evidence, 2016. AIDS. 2016 Aug 5. Epub ahead of print. country_profile_mozambique_0.pdf CrossRef. Medline 16. Chilundo BGM, Cliff JL, Mariano ARE, Rodrı´guez DC, George A. 30. Ministe´ro da Sau´de (MISAU); Instituto Nacional de Estatı´stica Relaunch of the official community health worker programme in (INE); ICF International. Inque´rito nacional de prevaleˆncia, riscos Mozambique: is there a sustainable basis for iCCM policy? comportamentais e informaçaõ sobre o HIV e SIDA em Health Policy Plan. 2015;30 Suppl 2:ii54–ii64. CrossRef. Moçambique (INSIDA) 2009. Maputo (Mozambique): INE; Medline 2010. Available from: http://dhsprogram.com/pubs/pdf/ 17. Sidat M, Ndima S, Taegtmeyer M, Ormel H. McCCollum R, Give AIS8/AIS8.pdf C. Context analysis: close-to-community providers in 31. Tolley E, Loza S, Kafafi L, Cummings S. The impact of menstrual Mozambique. Maputo (Mozambique): REACHOUT; 2014. side effects on contraceptive discontinuation: findings from a

Global Health: Science and Practice 2016 | Volume 4 | Number 3 420 CBD of Injectables in Mozambique www.ghspjournal.org

longitudinal study in Cairo, Egypt. Int Fam Plan Perspect. 33. Lei ZW, Chun Wu S, Garceau RJ, Jiang S, Yang QZ, Wang WL, 2005;31(1):15–23. CrossRef. Medline et al. Effect of pretreatment counseling on discontinuation rates 32. Bruce J. Fundamental elements of the quality of care: a simple in Chinese women given depo-medroxyprogesterone acetate framework. Stud Fam Plann. 1990;21(2):61–91. CrossRef. for contraception. Contraception. 1996;53(6):357–361. Medline CrossRef. Medline

Peer Reviewed

Received: 2016 Apr 28; Accepted: 2016 Jul 5; First Published Online: 2016 Sep 20

Cite this article as: Jacinto A, Mobaracaly MR, Usta´b MB, Bique C, Blazer C, Weidert K, et al. Safety and acceptability of community-based distribution of injectable contraceptives: a pilot project in Mozambique. Glob Health Sci Pract. 2016;4(3):410-421. http://dx.doi.org/10.9745/ GHSP-D-16-00133.

& Jacinto et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi.org/ 10.9745/GHSP-D-16-00133.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 421 ORIGINAL ARTICLE

Scheduled Follow-Up Referrals and Simple Prevention Kits Including Counseling to Improve Post-Discharge Outcomes Among Children in Uganda: A Proof-of-Concept Study

Matthew O Wiens,a Elias Kumbakumba,a Charles P Larson,a Peter P Moschovis,b Celestine Barigye,c Jerome Kabakyenga,c Andrew Ndamira,c Lacey English,d Niranjan Kissoon,a Guohai Zhou,a J Mark Anserminoa

ABSTRACT Background: Recurrent illness following hospital discharge is a major contributor to childhood mortality in resource- poor countries. Yet post-discharge care is largely ignored by health care workers and policy makers due to a lack of resources to identify children with recurrent illness and a lack of cohesive systems to provide care. The purpose of this proof-of-concept study was to evaluate the effectiveness of a bundle of interventions at discharge to improve health outcomes during the vulnerable post-discharge period. Methods: The study was conducted between December 2014 and April 2015. Eligible children were between ages 6 months and 5 years who were admitted with a suspected or proven infectious disease to one of two hospitals in Mbarara, Uganda. A bundle of interventions was provided at the time of discharge. This bundle included post-discharge referrals for follow-up visits and a discharge kit. The post-discharge referral was to ensure follow-up with a nearby health care provider on days 2, 7, and 14 following discharge. The discharge kit included brief educational counseling along with simple preventive items as incentives (soap, a mosquito net, and oral rehydration salts) to reinforce the education. The primary study outcome was the number of post-discharge referral visits completed. Secondary study outcomes included satisfaction with the intervention, rates of readmission after 60 days, and post-discharge mortality rates. In addition, outcomes were compared with a historical control group, enrolled using the same inclusion criteria and outcome-ascertainment methods. Results: During the study, 216 children were admitted, of whom 14 died during hospitalization. Of the 202 children discharged, 85% completed at least 1 of the 3 follow-up referral visits, with 48% completing all 3 visits. Within 60 days after discharge, 22 children were readmitted at least once and 5 children (2.5%) died. Twelve (43%) readmissions occurred during a scheduled follow-up visit. Compared with prospectively enrolled historical controls, the post-discharge referral for follow-up increased the odds of readmission (odds ratio [OR], 1.92; 95% conﬁdence interval [CI], 1.14 to 3.23) and care sought after discharge (OR, 14.61; 95% CI, 9.41 to 22.67). Overall satisfaction with the bundle of interventions was high, with most caregivers strongly agreeing that the discharge kit and post-discharge referrals improved their ability to care for their child. Conclusions: Interventions initiated at the time of discharge have the potential to profoundly affect the landscape of care during illness recovery and lead to signiﬁcantly improved outcomes among children under 5 years of age.

a University of British Columbia, Vancouver, Canada. BACKGROUND b Massachusetts General Hospital, Boston, MA. c Mbarara University of Science and Technology, Mbarara, Uganda. n resource-poor countries, in-hospital death rates for d I University of North Carolina at Chapel Hill, Chapel Hill, NC, USA. children hospitalized for a serious infection are simi- 1 Correspondence to Matthew O Wiens ([email protected]). lar to death rates in the weeks after they return home.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 422 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

Health care workers, policy makers, and caregiv- Children’s Hospital, both in Mbarara, Uganda. The ers are often unaware of the high vulnerability Mbarara Regional Referral Hospital is a public during this post-discharge period and are poorly hospital funded by the Uganda Ministry of Health equipped to identify, triage, and provide definitive and is associated with the Mbarara University of care for the children. An effective strategy is there- Science and Technology Faculty of Medicine. The fore required to address recurrent illness following pediatric ward admits approximately 5,000 patients hospital discharge in order to reduce overall child- per year. Holy Innocents Children’sHospitalisa hood mortality. Catholic children’s hospital offering subsidized fee- This study builds on an earlier observational for-service outpatient and inpatient care in Mbarara study in Uganda conducted between 2012 and and admits approximately 2,500 patients annually. 2014, in which we observed children who had been Thesestudysiteswerechosentoreflecttherela- admitted to the hospital with proven or suspected tively high proportion and use of both private and infections, for the purpose of developing prediction public hospitals in Uganda, and to improve external models for post-discharge mortality.2 The predictive validity by comparing 2 types of institutions. models included up to 5 variables, easily collected at The earlier observational study was approved admission, that identified children at high risk of by the institutional review boards at the Uni- mortality during the critical post-discharge period. versity of British Columbia (Vancouver, Canada) These models can help direct resources to the most and the Mbarara University of Science and Tech- vulnerable children, but any interventions must be nology (Mbarara, Uganda), the details of which evaluated to determine their impact on morbidity have been published.2 This proof-of-concept study and mortality before implementing at scale. was separately reviewed and approved by the The objective of this proof-of-concept study institutional review boards at the University of was to determine the effectiveness of a discharge British Columbia (Vancouver, Canada) and the package—consisting of a discharge kit (including Mbarara University of Science and Technology educational counseling and simple preventive (Mbarara, Uganda). The study was funded by items as incentives) and a post-discharge referral Grand Challenges Canada. for scheduled follow-up visits—in improving families’ health-seeking behavior from a qualified provider, and ultimately in improving mortality Eligibility during the post-discharge period. Using the same criteria as the observational study,2 children who were eligible to enroll in METHODS the proof-of-concept study were between the ages of 6 months and 5 years, and were admitted to Design the hospital with a proven or suspected infection. This proof-of-concept study builds upon an earlier Subjects already enrolled in the study were not This study builds observational study in Uganda in 2012–2014, which eligible to enroll again for subsequent admissions, upon an earlier had the primary objective to derive models to predict nor were subjects residing outside of the official observational 2 post-discharge mortality. The current study descri- catchment area of the hospital (10 surrounding study, in which bed in this article was conducted between December districts). modeling 2014 and April 2015 and represents the interven- predicted post- tional continuation of the observational study. This discharge study included an intervention aimed at improving Study Procedure mortality among outcomes following hospital discharge, whereas the The proof-of-concept study used the same research children. This earlier study was purely observational. The earlier nurses, field officers, and all equipment as in the proof-of concept study did not implement any systematic post- earlier observational study. Following enrollment, study provided a discharge policy. All children received routine care a research nurse obtained and recorded clinical post-discharge during enrollment and the post-discharge period. signs and symptoms as described in the observa- intervention to It is common practice to discharge children with 2 tional study. The clinical care provided during the improve mortality. instructions to return to a health center or hospital study was in accordance with local and national in the event of recurrence or worsening of illness. guidelines and reflected the in-hospital procedures used in the observational study (with the Population exception of the intervention itself). This was This study was conducted at 2 sites—the Mbarara done to ensure a high degree of consistency when Regional Referral Hospital and the Holy Innocents comparing the 2 cohorts.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 423 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

Interventions at Discharge Approximately 60 days after discharge, field During enrollment in the study, children received officers visited all subjects at their homes to deter- routine care according to the Uganda National mine vital status and assess whether care had Guidelines until the point of discharge. At been sought at a health center or from a commu- discharge, the children received a bundle of nity health worker. Field officers also adminis- interventions. The bundle consisted of (1) post- tered a short survey to caregivers of the children discharge referral for follow-up visits organized to solicit feedback on the discharge intervention by the research nurse at the time of discharge, and the post-discharge referral for follow-up. and (2) a discharge kit consisting of counsel- Field officers were trained, prior to the earlier ing and simple preventive items to reinforce the observational study, in patient tracking and admin- counseling. istering questionnaires, and they had some health- During the discharge counseling, the research related training. The procedures used to ascertain nurse provided the child’s caregiver with a paper outcomes were the same in both the observa- referral form for follow-up with either a commu- tional study and this study. nity health worker or at a nearby health center on Study data were collected electronically using days 2, 7, and 14 following discharge. These days tablet computers and managed using the Research were chosen as the highest-risk times during the Electronic Data Capture (REDCap) tool hosted early post-discharge period.1,2 Before this study, at the Child and Family Research Institute in we collected information on community health Vancouver, Canada.4 REDCap is a secure, web- workers and health centers in the catchment area based application designed to support data cap- (10 districts) at the parish level. Caregivers could ture for research studies. choose either a community health worker or a health center for the child’s follow-up visits, based on preference and proximity to their home. For Analysis Outcomes caregivers who chose a follow-up visit at a health center, nurses provided a list of health centers The primary study outcome was the proportion (private and public) from which the caregivers of children who successfully completed at least could choose. The referral form was then addressed 1 post-discharge referral for a follow-up visit at a to either the health center or the community health health center or with a community health worker. worker, and caregivers received relevant informa- Secondary study outcomes included caregiver tion as written instructions. satisfaction with the interventions (the discharge The caregivers also received a discharge kit, kit and post-discharge referral) and comparisons which included brief educational counseling in post-discharge mortality, readmission, and care paired with simple preventive items as incentives sought between this study and the earlier obser- 2 to reinforce the education. The educational coun- vational study. seling consisted of a storyboard-style, laminated cardwritteninthelocallanguage(supplementary Statistical Analysis At discharge, the material). Using this card, the nurse explained We performed descriptive analyses of the primary nurse explained the child’s vulnerability during the discharge outcome and of caregiver satisfaction. Using logi- the child’s period and discussed 3 main themes of action: stic regression, we also performed comparative vulnerability (1) prevention through hygiene and other health analyses between the earlier observational study during the behaviors (e.g., mosquito net use), (2) recogni- and this study. In this analysis, the outcomes discharge period tion of signs of early illness recurrence, and of readmission and post-discharge health seek- and 3 main (3) prompt care sought at a health center or ing were adjusted for both site of enrollment themes of action: from a community health worker. In our earlier and the post-discharge mortality risk score. The prevention, research, these themes featured prominently in post-discharge mortality risk score is a 5-item the children who died following discharge.3 Care- recognition, and composite score including mid-upper arm cir- givers received the card along with 3 preventive cumference, oxygen saturation, time since most seeking care. items meant to reinforce the education (a mos- recent hospitalization, HIV status, and coma score. quito net, 1 kg of soap, and 5 sachets of oral We also conducted a secondary univariate logistic rehydration salts). The value of these household regression analysis examining factors associated incentives was approximately US$6.50. The time with completing post-discharge referrals. For the it took hospital staff to give the bundle of inter- secondary analysis, a sample of 200 children would ventions was less than 30 minutes. provide 80% power to detect a 10% absolute

Global Health: Science and Practice 2016 | Volume 4 | Number 3 424 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

difference in health seeking compared with the diagnosed with pneumonia, while 45 (20%) were historical cohort, at an alpha of .05. All analyses were diagnosed with malaria and 17 (8%) with diarrhea. conducted in SAS 9.4 (Cary, North Carolina, USA). The rate of in-hospital mortality was 6.5%.

RESULTS Post-Discharge Referrals A total of 216 children were enrolled in the study. The number of children who survived to dis- An additional 93 children were screened but charge was 202 (93.5%). Of these, 170 (84%) 84% of children excluded from the study, mostly because they pre- completed at least 1 scheduled post-discharge surviving to sented with a non-infectious illness (Figure). The follow-up visit, 143 (71%) completed 2 follow-up discharge median age was 16.1 months (IQR 10.2–29.1), visits, and 96 (48%) completed all 3 follow-up attended at least and the sample was nearly evenly split between visits. Among those children who did not com- 1 follow-up boys and girls (107 children, or 49.5%, were boys) plete all 3 follow-up visits, the 3 most commonly referral visit after (Table 1). Forty-two children (20%) were referred reported reasons for non-completion were discharge, but for the initial hospital admission, mostly from (1) health system factors, such as a closed health health centers. Fifty-three (24.7%), 59 (27.4%), less than half center; (2) family factors, such as lack of trans- completed all 3. and 56 (26.2%) children were underweight, stun- portation or high cost; and (3) the child’s family ted, or wasted (defined as height-for-age z score did not consider the visit important (Table 2). For less than -2). One hundred and four (48%) were the first post-discharge follow-up visit, 115 (68%)

FIGURE. Study Flow of Subjects Enrolled and Excluded

Global Health: Science and Practice 2016 | Volume 4 | Number 3 425 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

TABLE 1. Baseline Characteristics of Study Subjects (N = 216)

Variable Value

Male sex, No. (%) 107 (49.5) Age, months, median (IQR) 16.1 (10.2, 29.1) Prior care sought for illness, No. (%) 160 (74.1) Referred for the initial hospital admission 42 (19.4) Referral source: hospital 3 (7.0) Referral source: health center 33 (78.6) Referral source: untrained health worker 6 (14.3) MUAC o115 mm, No. (%) 14 (6.5) MUAC 115–125 mm, No. (%) 21 (9.7) Underweight (WAZ o-2), No. (%) 53 (24.7) Severely underweight (WAZ o-3), No. (%) 24 (11.2) Wasted (WHZ o-2), No. (%) 56 (26.2) Severely wasted (WHZ o-3), No. (%) 28 (13.1) Stunted (HAZ o-2), No. (%) 59 (27.4) Severely stunted (HAZ o-3), No. (%) 31 (14.4) HIV positive, No. (%) 15 (7.0) Maternal education, No. (%) No school 18 (8.3) Less than primary 3 17 (7.9) Primary 4 to primary 7 91 (42.1) Secondary 1 to secondary 6 60 (27.8) Post-secondary 30 (13.9) Discharge diagnosis, No. (%) Malaria 43 (19.9) Pneumonia 104 (48.2) Diarrhea 17 (7.9) Discharged against medical advice, No. (%) 17 (8.0) In-hospital mortality, No. (%) 14 (6.5) Referred to higher level of care, No. (%) 4 (1.9)

Abbreviations: HAZ, height-for-age z score; IQR, interquartile range; MUAC, mid-upper arm circumference; WAZ, weight- for-age z score; WHZ, weight-for-height z score.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 426 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

TABLE 2. Post-Discharge Referral Completions and Outcomes Among Discharged Children

No. (%)

Referral program completions (N = 202) At least 1 visit 170 (84) At least 2 visits 143 (71) All 3 visits 96 (48) Outcome for visit 1 (n = 171) No intervention 111 (65) Outpatient-based intervention 54 (32) Admission 2 (1) Referral to higher level of care 4 (2) Outcome for visit 2 (n = 141) No intervention 88 (62) Outpatient-based treatment 42 (30) Admission 8 (6) Referral to higher level of care 3 (2) Outcome for visit 3 (n = 95) No intervention 60 (63) Outpatient-based treatment 31 (33) Admission 2 (2) Referral to higher level of care 2 (2) Reasons for missed referral visits (n = 104) Child not sick/visit not considered important 22 (22) Child away 12 (12) Child admitted 7 (7) Child died 3 (3) Forgot to go 15 (15) Visit not possible (health system factorsa) 16 (16) Visit not possible (family factorsb) 23 (23)

a Examples of health system factors include closed health center and unavailable community health worker. b Examples of family factors include cost barriers, no transportation available, and husband denied permission.

were completed at a local health center and (N = 407), 12 (2.9%) resulted in admission, 9 (2.2%) 56 (33%) were completed by a local community resulted in referral to a higher level of care, and health worker. For the second and third visits, 127 (31.2%) resulted in an outpatient-based inter- the same provider was used in 84% and 80% of vention. Further details of the breakdown of these cases, respectively. Among the total referral visits outcomes according to first, second, and third visit

Global Health: Science and Practice 2016 | Volume 4 | Number 3 427 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

are detailed in Table 2. Outpatient interventions were were not seen at follow-up visits or any occasion; defined as disease-specific treatments. While general no care was sought from any health care provider health advice was commonly provided, this was not before death. considered an outpatient intervention. Outpatient interventions were accepted in 121 (95%) cases. There were no statistically significant findings Satisfaction With the Interventions in the secondary analysis of predictors of follow- Approximately 60 days after discharge, field officers up, although trends toward increased referral administered a short survey with 5 questions for compliance were noted among those children caregivers to solicit feedback on the bundle of with increasing mid-upper arm circumference or interventions (Table 5). Overall, more than 75% weight-for-age z score, who had used a mosquito of caregivers strongly agreed that the education net consistently, and who had a length of stay of provided at discharge improved their ability to less than 5 days during the initial admission. Age, care for their child during the post-discharge sex, household crowding, sibling deaths, and age period; 65% of caregivers strongly agreed that the of the child’s mother were not associated with simple preventive items provided were helpful in follow-up success (Table 3). caring for their child. Among those who completed at least 1 referral (n = 170), 62% found that the referrals were very helpful in caring for their Post-Discharge Readmission and Mortality child and 75% found that the referrals were Within 60 days During the 60-day post-discharge period, a total neither difficult nor inconvenient. Overall, 75% after discharge, of 22 (11%) children were readmitted at least were very satisfied with the interventions. When 11% of children once, for a total of 28 admissions (Table 4). Of asked what else could have been done to help were readmitted. those children who were readmitted, 12 were with their child, the responses varied. However, admitted through a scheduled referral visit, and 3 responses occurred often, with 82 (41%) respon- 75% of caregivers the remaining 10 through self-referral. During dents stating they wished the program had provided were very the post-discharge period, 5 children died (2.5%); transportation for the post-discharge referrals, satisﬁed with the 4 died at home and 1 during a readmission. The 47 (23%) stating they wished the program pro- interventions. child who died in the hospital was seen twice at vided either food in the hospital or following referral follow-up visits. The 4 who died at home discharge, and 41 (20%) stating they wished the

TABLE 3. Analysis of Factors Associated With Post-Discharge Referral Completion (N = 202)

Variable OR (95% CI) P Value

Sex (female) 1.03 (0.48, 2.21) .93 Age (for each month increase) 1.00 (0.97, 1.02) .77 Referral at initial admission 0.85 (0.32, 2.60) .74 MUAC (for each 1 mm increase) 1.01 (1.00, 1.04) .10 WAZ (for each 1 SD increase) 1.20 (0.96, 1.50) .11 HIV positive 2.52 (0.32, 19.96) .38 Crowding (for each additional household member) 1.09 (0.90, 1.31) .38 Sibling death 1.03 (0.39, 2.72) .95 Maternal age (for each 1-year increase) 1.01 (0.94, 1.08) .81 Mosquito net use (always vs. no/sometimes) 2.00 (0.90, 4.47) .09 Length of stay 4 5 days 0.53 (0.24, 1.19) .12

Abbreviations: CI, conﬁdence interval; MUAC, mid-upper arm circumference; OR, odds ratio; SD, standard deviation; WAZ, weight-for-age z score.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 428 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

TABLE 4. Post-Discharge Mortality and Readmission Details (N = 202)

No. (%)

Mortality within 60 days post-discharge 5 (2.5) Location of death Home 3 (60.0) Home of a relative or friend 1 (20.0) Hospital 1 (20.0) 60-day post-discharge readmission 22 (10.9) Once 16 (72.7) Twice 4 (18.2) Three times 2 (9.1) Source of readmission (n = 28) Self-referral 16 (57.1) Scheduled post-discharge referral 12 (42.8)

educational component was administered at the in order for 1 additional child to receive in-person time of admission. follow-up care who otherwise would not have received follow-up care after discharge. The characteristics of these 2 samples were similar. Comparing the Two Cohorts However, the earlier observational cohort had a In the earlier observational study, 1,307 children higher proportion of patients with severe malnu- were enrolled using the same enrollment criteria trition. The interventional cohort had a substan- as this study. Of these, 65 died during hospita- tially higher proportion of pneumonia and lower lization (5%), resulting in 1,242 live discharges proportion of malaria (and thus a lower mean (Table 6). During the first 60 days of follow-up, oxygen saturation). The proportion of children 41 (3.3%) children died, 72 (5.8%) were readmitted, with HIV and of those with an abnormal Blantyre and 383 (30.8%) sought care with a community Coma Scale score was slightly higher in the inter- health worker, health center, or hospital. During ventional cohort (Table 7). When applying our the current study that included a bundle of previously derived post-discharge mortality predic- interventions, the rates of mortality, readmission, tion model,2,5 the 2 cohorts had similar risk pro- Compared with and health seeking were 2.5%, 10.9%, and 87.6%, files for predicted 6-month mortality, suggesting the earlier respectively. We found a non-significant 25% that the differences in outcomes are unlikely to observational lower odds of death at 60 days (odds ratio [OR], be related to differing risk profiles among the cohort study, 0.75; 95% confidence interval [CI], 0.29 to 1.92), 2 cohorts of children. In both the earlier observa- we found a a nearly twofold higher adjusted odds of read- tional study cohort and the current study cohort 25% lower odds mission (OR, 1.97; 95% CI, 1.14 to 3.23), and a with interventions, the modeled median risk of of death at 14-fold higher adjusted odds of seeking post- death at 6 months post-discharge was 5.1% (IQR 60 days, a nearly discharge care (OR, 14.61; 95% CI, 9.41 to 22.67). in earlier observational study, 2.8 to 9.3; IQR in 2-fold higher The absolute increase in care seeking post- current study, 2.6 to 9.5). adjusted odds of discharge increased from approximately 30% in readmission, and the earlier observational study (with no interven- a 14-fold higher tion) to nearly 90% in the current study (with the DISCUSSION adjusted odds of intervention to promote health-seeking behavior post-discharge). This indicates that less than This proof-of-concept study evaluated the effective- seeking post- 2 children would need to receive this intervention ness of using a discharge bundle, consisting of a discharge care.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 429 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

TABLE 5. Caregiver Satisfaction With Interventionsa

No. (%)

Did the education provided at discharge improve your ability to take care of your child? (n = 191) Yes, strongly 147 (76.9) Yes, somewhat 44 (23.0) No 2 (1.0) Were the soap, oral rehydration salts, and mosquito net helpful in better caring for your child after discharge? (n = 189) Yes, strongly 123 (65.1) Yes, somewhat 66 (34.9) No 1 (0.5) Did you feel that the referrals were helpful in caring for your child after discharge? (n = 170) Yes, very helpful 105 (61.7) Yes, somewhat helpful 54 (31.8) Not sure 5 (2.9) No 6 (3.5) Did you find the referrals difficult/inconvenient? (n = 170) Yes, very difficult/inconvenient 3 (1.8) Yes, somewhat difficult/inconvenient 32 (18.8) Not sure 7 (4.1) No, not difficult/inconvenient 128 (75.2) Overall satisfaction with discharge kit and post-discharge referral (n = 195) Very satisfied 72 (36.9) Somewhat satisfied 117 (60.0) Not satisfied 4 (2.1)

a Sample size for the satisfaction indicators are slightly different, reﬂecting that not all children discharged (such as most who were discharged against medical advice) received the counseling and incentives, and not all caregivers participated in the satisfaction survey.

Two, instead of discharge kit (education and preventive items as follow-up visit, with nearly 50% completing all three, follow-up incentives) and post-discharge referrals for follow- 3 follow-up visits. The observed falling compliance visits post- up visits, in a hospital environment in southwestern in referral completion, however, and the fact that discharge may be Uganda to improve health-seeking behavior and 40% of missed visits were due to circumstances that more practical in post-discharge mortality. Results showed that this appeared to be beyond the control of the caregiver, bringing this type intervention was both feasible and effective. After suggest that 2 visits may be more practical in of intervention to administering this intervention to the caregivers bringing this type of intervention to scale. scale. of some 200 children, we found that 89% of chil- Post-discharge care was community focused, dren successfully achieved at least 1 post-discharge with referrals being directed to community health

Global Health: Science and Practice 2016 | Volume 4 | Number 3 430 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

TABLE 6. Comparison of Outcomes Between Earlier Observational Cohort (N = 1,242) and Current Intervention Cohort (N = 202)

Earlier Observational Cohort, Current Intervention Cohort, Outcome No. (%) No. (%) OR (95% CI)

Readmission 72 (5.8) 22 (10.9) 1.97b (1.14, 3.23) Any visit 383 (30.8) 177a (87.6) 14.61b (9.41, 22.67) Death 41 (3.3) 5 (2.5) 0.75 (0.29, 1.92)

Abbreviation: CI, conﬁdence interval; OR, odds ratio. a Also includes non-referral visit; therefore, the number in this table is higher than the 170 indicated in Table 2. b Adjusted for site of enrollment and post-discharge mortality risk score.

TABLE 7. Characteristics of Discharged Subjects, Comparison Between Earlier Observational Cohort (N = 1,242) and Current Intervention Cohort (N = 202)

Earlier Observational Cohort Current Intervention Cohort

Male sex, No. (%) 682 (54.9) 103 (51.0) Age, months, median (IQR) 18.1 (10.8, 34.6) 16.2 (10.0, 29.0) MUAC o115 mm, No. (%) 96 (7.7) 12 (5.9) MUAC 115–125 mm, No. (%) 87 (7.0) 19 (9.4) Severely underweight (WAZ o-3), No. (%) 188 (15.1) 20 (10.0) Severely wasted (WHZ o-3), No. (%) 232 (18.7) 24 (11.9) Severely stunted (HAZ o-3), No. (%) 187 (15.0) 28 (13.9) Mean SpO2 at admission 94.0 (90.0, 96.0) 91.0 (85.5, 97.0) Percent with abnormal BCS score (o5) 133 (10.7) 31 (15.4) HIV positive, No. (%) 58 (4.7) 14 (7.0) Maternal education, No. (%) Less than primary 3 250 (20.1) 29 (14.4) Primary 4 to primary 7 630 (50.7) 85 (42.1) Secondary 1 to secondary 6 269 (21.6) 58 (28.7) Post-secondary 93 (7.5) 30 (14.9) Discharge diagnosis, No. (%) Malaria 418 (33.6) 39 (19.3) Pneumonia 390 (31.4) 98 (48.5) Diarrhea 96 (7.7) 17 (7.4) Discharged against medical advice, No. (%) 120 (9.6) 17 (8.4)

Abbreviations: BCS, Blantyre Coma Scale; HAZ, height-for-age z score; IQR, interquartile range; MUAC, mid-upper arm circumference; WAZ, weight-for-age z score; WHZ, weight-for-height z score.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 431 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

workers and nearby health centers. Three-quarters the scalability of effective post-discharge care. of caregivers reported that these visits were neither At the community level, the capacity to complete inconvenient nor difficult. Most strongly agreed follow-up visits is sufficient and does appear to that the education and referral were important exist in most areas, but the hospital resources to components of care for their child’s recovery. assess children for risk and to implement the We propose that Compared with prospective observational data referral process are likely to be insufficient. We discharge nurses collected before this study, this package of inter- propose the establishment of discharge nurses should assess ventions was associated with increased hospita- to oversee this role, as has been suggested for children for risk lizations (representing increased health seeking), improving post-stroke outcomes in sub-Saharan 7 and implement a with over 40% of readmissions being directly Africa. Further, engagement with other key referral process to linked to a referral visit. Overall health seeking stakeholders (e.g., ministries of health, hospital improve health from a community health worker or a health administrations, and community health worker outcomes after center post-discharge was only 30% in the pre- training programs) is critical in the development discharge. vious observational study with no intervention of a scalable intervention. compared with nearly 90% in the current study The identification of the most vulnerable chil- that included the package of interventions to dren presents an important strategy to improve improve health seeking. This reflects an impor- the cost-effectiveness of post-discharge care, as tant achievement given that modeling in the limited resources can be used to target high-risk observational study predicted that more than children. Our group recently derived a clinical 1 of every 30 children discharged is likely to die prediction tool that uses 5 easily collected vari- within the first 2 months and that health-seeking ables to identify such children.2 This tool has behavior is poor for the most common causes of been incorporated into a mobile application to childhood death.2,6 enable frontline health workers to rapidly identify Post-discharge Post-discharge mortality is a neglected but vulnerable children before discharge.5 The admini- mortality is a important issue in the field of pediatric global stration of the discharge bundle evaluated in this neglected but health. Prior research has shown that, in some study could be an inexpensive and effective strategy important issue in areas, mortality after discharge exceeds mortality to improve care following discharge, as it is well 1 the ﬁeld of in the hospital. Improved post-discharge mor- recognized that caregivers could benefit from pediatric global tality is an important contributor to overall additional education surrounding illness recogni- 6,8 health. childhood mortality; however, policy and practice tion and health seeking. have not recognized its importance, and a system- Prior research has shown that a signifi- atic approach to post-discharge care is lacking in cant proportion of child deaths occur outside of most resource-poor countries. While resources health facilities.9 Our study found that 4 out of such as the Integrated Management of Childhood 5 children who died after discharge were outside Illness strategy and the Emergency Triage Assess- of the health system, reflecting similar findings ment and Treatment guidelines provide standar- from our work during the observational study. dized approaches to the acute phase of infectious Critical to addressing post-discharge care, there- illness, no resources or guidelines currently exist fore, is improved community-level care during to provide recommendations during the vulner- the vulnerable post-discharge period, which was able recovery phase. an important focus of our work. Although post-discharge care should be an important component in the care of all discharged children, limited resources and an already strained Limitations health system are likely to be major barriers in This proof-of-concept study is subject to several its implementation. The current context among important limitations. First, the small sample size hospitals in Uganda and elsewhere is that the limits the ability of this study to make robust limited post-discharge care that does occur comparisons with the earlier observational study. focuses primarily on the ongoing treatment of Although comparisons of study outcomes such as specific diseases such as HIV and tuberculosis. We health seeking and readmission provide some are not aware of any hospitals in resource-poor insight into the potential value of a post-discharge settings that have adopted general discharge intervention, a primary study outcome based on policies. In addition to a lack of robust policies mortality is preferable and will be a primary for post-discharge care, the larger issue is the avail- component of future research. The results of this ability of resources, which is a major limitation to study, however, serve an important role in guiding

Global Health: Science and Practice 2016 | Volume 4 | Number 3 432 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

the design and sample size calculations for sub- discharge to improve post-discharge mortality sequent research, as does the feedback provided and will explore bringing such interventions to by the participants in the study. A further limi- scale in Uganda. tation of this study, and similar studies, is that the incorporation of post-discharge follow-up in a Acknowledgments: This study was funded by Grand Challenges Canada. Dr. Wiens is supported by a fellowship from Mitacs Canada. research context may not be easily replicated in a Dr. Moschovis is supported by the National Institutes of Health non-research context. The discharge education (5-F32-HL124951) and the Thrasher Research Fund. We gratefully given to caregivers, highlighting the vulnerability acknowledge the contributions of Annet Twinomuguni, Justine Kamazima, Clare Komugisha, Solome Kobugyenyi, Brenda of their children during the post-discharge period, Kembabazi, Alexander Mutungi, and Hassan Bariahikwa. Without likely played an important role in motivating their dedication, this study could not have been completed. them to sacrifice time and money to complete the Competing Interests: None declared. follow-up referral visits. Follow-up interventions, therefore, must be strongly linked to education at REFERENCES the time of discharge. A final limitation of this 1. Wiens M, Pawluk S, Kissoon N. Pediatric post-discharge study is the relatively narrow age range (6 months mortality in resource poor countries: a systematic review. to 5 years) of children. The reason this age range PLoS One. 2013;8(6):e66698. CrossRef. Medline was chosen was based on design considerations 2. Wiens MO, Kumbakumba E, Larson CP, Ansermino JM4, for the earlier observational study, which we used Singer J, Kissoon N, et al. Postdischarge mortality in children with to develop the prediction model. Our research acute infectious diseases: derivation of postdischarge mortality prediction models. BMJ Open. 2015;5(11):e009449. group is currently working toward the expansion CrossRef. Medline of these prediction models to eventually include 3. Wiens MO. Childhood mortality from acute infectious diseases in both younger (o6 months) and older (45 years) Uganda: studies in sepsis and post-discharge mortality [thesis]. children. Finally, because this intervention con- Vancouver (Canada): University of British Columbia; 2015. sisted of a bundle of interventions, it is impossible Available from: http://hdl.handle.net/2429/53787 to identify which components were critical in the 4. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—A metadata-driven results that were observed. The simple preventive methodology and workﬂow process for providing translational items that complemented the education (a mos- research informatics support. J Biomed Inform. 2009;42(2): quito net, soap, and oral rehydration salt) were 377–381. CrossRef. Medline relatively expensive (approximately US$6.50) and 5. English LL, Dunsmuir D, Kumbakumba E, Ansermino JM, Larson may impact scalability. However, it is unlikely that CP, Lester R, et al. The PAediatric Risk Assessment (PARA) mobile app to reduce postdischarge child mortality: design, usability and these particular incentives are required in order feasibility for health care workers in Uganda. JMIR mHealth to observe these benefits. The mosquito net was uHealth. 2016;4(1):e16. CrossRef. Medline the most expensive item in the bundle (about 6. Geldsetzer P, Williams TC, Kirolos A, Mitchell S, Ratcliffe LA, US$5.75); it could easily be replaced with a less Kohli-Lynch MK, et al. The recognition of and care seeking expensive item that complements the education behaviour for childhood illness in developing countries: a systematic review. PLoS One. 2014;9(4):e93427. while still providing value to the caregiver, and CrossRef. Medline could serve as a means to reinforce the instruc- 7. Ovbiagele B. Tackling the growing diabetes burden in tions provided by the discharge nurse. Sub-Saharan Africa: a framework for enhancing outcomes in stroke patients. J Neurol Sci. 2015;348(1-2):136–41. CrossRef. Medline CONCLUSION 8. Sandberg J, Odberg Pettersson K, Asp G, Kabakyenga J, Agardh A. Inadequate knowledge of neonatal danger signs In conclusion, we found that a simple bundle of among recently delivered women in southwestern rural interventions at discharge, including brief educa- Uganda: a community survey. PLoS One. 2014;9(5):e97253. tional counseling and a post-discharge referral, CrossRef. Medline can improve post-discharge care and outcomes 9. Nair H, Simo˜es EA, Rudan I, Gessner BD, Azziz-Baumgartner E, among children discharged from the hospital. Zhang JSF, et al. Global and regional burden of hospital admissions for severe acute lower respiratory infections in young New research is currently being planned to estab- children in 2010: a systematic analysis. Lancet. 2013; lish the benefit of a bundle of interventions at 381(9875):1380–90. CrossRef. Medline

Global Health: Science and Practice 2016 | Volume 4 | Number 3 433 Referrals and Prevention Kits to Improve Post-Discharge Outcomes www.ghspjournal.org

Peer Reviewed

Received: 2016 Feb 29; Accepted: 2016 Jul 25; First Published Online: 2016 Sep 14

Cite this article as: Wiens MO, Kumbakumba E, Larson CP, Moschovis PP, Barigye C, Kabakyenga J, et al. Scheduled follow-up referrals and simple prevention kits including counseling to improve post-discharge outcomes among children in Uganda: a proof-of-concept study. Glob Health Sci Pract. 2016;4(3):422-434. http://dx.doi.org/10.9745/GHSP-D-16-00069.

& Wiens et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi.org/ 10.9745/GHSP-D-16-00069.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 434 ORIGINAL ARTICLE

Intensive Group Learning and On-Site Services to Improve Sexual and Reproductive Health Among Young Adults in Liberia: A Randomized Evaluation of HealthyActions

Rebecca Firestone,a Reid Moorsmith,b Simon James,c Marilyn Urey,d Rena Greiﬁnger,a Danielle Lloyd,d Lisa Hartenberger-Toby,e Jewel Gausman,f Musa Sanoeg

ABSTRACT Introduction: Young Liberians, particularly undereducated young adults, face substantial sexual and reproductive health (SRH) challenges, with low uptake of contraceptive methods, high rates of unintended pregnancy, and low levels of knowledge about HIV status. The purpose of this study was to assess the impact of a 6-day intensive group learning intervention combined with on-site SRH services (called HealthyActions) among out-of-school young adults, implemented through an existing alternative education program, on uptake of contraception and HIV testing and counseling (HTC). Methods: The intervention was implemented among young women and men ages 15–35 who were enrolled in alternative basic education learning sites in 5 counties of Liberia. We conducted a randomized evaluation to assess program impact. Baseline data were collected in January–March 2014, and endline data in June–July 2014. Key outcomes of condom use, contraceptive use, and HTC were estimated with difference-in-difference models using ﬁxed effects. All analyses were conducted in Stata 13. Results: We assessed outcomes for 1,157 learners at baseline and 1,052 learners at endline, across 29 treatment and 26 control sites. After adjusting for potential confounders, learners in the HealthyActions intervention group were 12% less likely to report never using a condomwitharegularpartnerover the last month compared with the control group (P = .02). Female learners who received HealthyActions were 13% more likely to use any form of modern contraception compared with learners in control sites (Po.001), with the greatest increase in the use of contraceptive implants. Learners in HealthyActions sites were 45% more likely to have received HTC (Po.001). Conclusion: Providing intensive group learning in a supportive environment coupled with on-site health services improved SRH outcomes among participating learners. The focus of HealthyActions on participatory learning for low- literacy populations presents an adaptable solution for health programming across Liberia and the region.

a Population Services International, Washington, DC, USA. INTRODUCTION b Population Services International, Yangon, Myanmar. c Education Development Center, Inc., Washington, DC, USA. ven before the recent Ebola crisis, Liberia has faced d E Population Services International, Monrovia, Liberia. intersecting health and development challenges. e Education Development Center, Inc., Pasig City, Philippines. Fourteen years of conflict disrupted preexisting social f Independent Consultant, Harvard T.H. Chan School of Public Health, structures, resulting in young people being forced to Boston, MA. g take on adult responsibilities without experiencing Education Development Center, Inc., Advancing Youth Project, Monrovia, 1 Liberia. healthy developmental milestones. Liberians value edu- Correspondence to Rebecca Firestone (rﬁ[email protected]). cation highly as the path to better employment, and

Global Health: Science and Practice 2016 | Volume 4 | Number 3 435 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

particularly appreciate the positive impact that the armed conflict led to an overall collapse of the educating girls can have on families and com- health care system characterized by the flight of munities.2 However, due to war, poverty, and health workers, looting, and destruction of health family disruption, many young people in Liberia facilities and the desecration of roads.7 Even where were forced to drop out of school and never health services do exist, young people often face participated in community leadership structures. additional barriers. As in many countries in the In a sense, they were caught between youth and world, young people in Liberia face financial adulthood. barriers in accessing services, unfriendly providers As a result, in Liberia the government’s (particularly if they are seeking SRH services and definition of youth—15–35 years—is unique to are unmarried), and lack of privacy and con- a post-conflict environment. This broad definition fidentiality.8 Unequal gender norms mean that speaks to delayed social and cognitive develop- young women are less likely than young men to ment across generations, warranting creative solu- receive an education and make autonomous tions to reach young people and provide access to decisions.9 Social acceptance of intimate part- much-needed services.3 ner violence reflects a culture of male domi- Liberia’s legacy of Liberia’s legacy of conflict, orphanhood, lack nance.10 Intergenerational and transactional sex conﬂict combined of social cohesion, and school drop-out, combined is widespread across wealth quintiles. Many with early sexual with early sexual debut, place young Liberians at young women engage in transactional relation- debut places increased risk of negative sexual and reproductive ships to secure school fees, clothing, and other 4,11,12 young Liberians at health (SRH) outcomes. According to Liberia’s commodities. increased risk of 2013 Demographic and Health Survey, the me- Young people in Liberia transition into adult- – negative sexual dian age at first sex among women 25 49 is hood in an environment in which there is perva- and reproductive 16.2 years. Nearly one-quarter (24%) of women sive distrust of all institutions connected with the – health outcomes. ages 25 49 had sexual intercourse by age 15; government, which includes the health system. about three-quarters (78%) by age 18; and over This may be in large part due to the fact that 90% by age 20. Among men ages 25–49, the quality of life has increased minimally since the median age at first sex is slightly older at end of the conflict in 2003, despite donor money 18.3 years. By age 20, 75% of men have had and support from the United Nations.13,14 Many sex.4 By age 24, nearly 80% of women have communities lack social cohesion, and in some begun childbearing.4 Only 25% of sexually cases, young people are actively marginalized active girls and young women (ages 15–29) use a from existing organizational structures within modern contraceptive method, and about 30% their communities.15 The lack of formal train- have an unmet need for contraception.4 Over- ing opportunities and/or education,1 along with all, HIV prevalence is low in Liberia at 1.9% limited supportive family structure, prevent for adults 15–49. Among women, peak preva- many young people from becoming actively lence (3.6%) occurs in the 25–29 age group; among involved in their communities. The inability to men, prevalence is highest (3.6%) among rationalize appropriately within a group and/or those 40–44. While nearly all Liberians have to socialize normally during their formative heard of HIV/AIDS and the majority know years creates enormous barriers for young people where to get tested, only 37% of 15–24-year- to conform to conventional social norms.16 Fur- old women and 11% of 15–24-year-old men ther, there are widely respected taboos around have ever been tested and have received their discussing sex and sexuality, particularly for results.4 young women. Yet young women face greater Low education levels exacerbate this situa- pressure to have sex earlier, due to their tion; in 2013, 33% of women 15–49 and 13% of desirableness to older men and the fact that men 15–49 reported they had never been to the exchange system of young women’ssexu- school.4 Primary school gross enrollment is only ality for material goods has become normal- 52%, and nearly a third of all students enrolled in ized.12 These normalizations, however, have not school drop out in the first grade.5 Low rates of relieved the heath system actors of their judg- school enrollment and poor quality of teaching ment about young people, and young women in result in few Liberians gaining actionable knowl- particular.17 edge and skills.6 Supportive family, school, and peer environ- Youth in Liberia face several barriers that limit ments are often associated with improved health their access to formal health services. To begin, outcomes among young people.18 Connectedness

Global Health: Science and Practice 2016 | Volume 4 | Number 3 436 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

to schools, community, and family can have a METHODS protective effect on SRH outcomes, such as age of sexual debut, number of sexual partners, and con- Intervention traceptive use.19 Programs that aim to strengthen HealthyActions was structured as a 6-day intensive social ties through participation in basic educa- ‘‘burst’’ of participatory and action-oriented SRH tion can promote positive engagement in learning, education activities packaged with on-site health socialization, and recreation while also improv- services, designed to be easily integrated into ing access to SRH information and promoting an alternative basic educational setting; in this healthy behavior.20 case, the U.S. Agency for International Develop- Uptake of SRH services is strongest when ment (USAID) Advancing Youth Project,25 an interventions designed to bolster demand for alternative basic education project implemented services are linked directly to supply, and where by the Education Development Center (EDC). there is community and social support21;for The USAID Advancing Youth Project, which example, an evaluation of a program imple- beganinOctober2011andwillendinJune mented in Ghana and Nigeria that provided 2017, provides basic literacy and numeracy skills, both contraceptives within a school setting and social and leadership development, and liveli- referrals for other SRH services as part of a hoods training for out-of-school Liberian youth comprehensive SRH program targeting youth, ages 15–35 who have no or marginal literacy showed improved contraceptive uptake among skills. The first round of HealthyActions implemen- participants.22 Educational settings, whether tation took place from August 2012 to July 2013 Formal and formal or informal, can provide ample opportu- in Nimba and Montserrado counties as a pilot to informal nities for improving access to SRH information gain feedback on the curriculum and approach. educational and services for young people. Information and The second round of larger-scale implementa- settings can services are brought to learners, rather than tion took place from November 2013 to May 2014 provide learners needing to seek them out. Additionally, in Bong, Grand Bassa, Lofa, Montserrado, and opportunities for learners can access services at the same time Nimba counties. improving access as one another, reducing the stigma associated Over three-quarters of Advancing Youth 23 to sexual and with seeking contraceptive and HIV services. In learners are women, most of whom have never reproductive Liberia, reaching young adults with reproductive been to school before. The median age is 29. health information health information and services through an exist- Most learners report that they are living with a and services for ing platform could help overcome barriers to partner or spouse and that they have an average young people. accessing services and ensure that they receive of 3 children. Only about 5% are adolescents high-quality, youth-friendly services while also under the age of 18; these adolescents take the establishing relationships with local health ser- same classes as older learners and are integrated vice providers. into all activities including youth clubs and skills Evidence of effective SRH strategies remains training.26 The main difference between older limited in West Africa, despite the efforts of the and younger participants in the program is their Ouagadougou Partnership to expand access to experience of war; however, both groups have family planning.24 We aimed to assess whether issues with identity formation and positive role an intervention package consisting of intensive models, and poverty puts learners at risk for group learning and provision of on-site SRH transactional sex and other risky behaviors. services could increaseuseofmoderncontra- The USAID Advancing Youth Project works ception and HIV testing and counseling (HTC) through the Liberian Ministry of Education, services among out-of-school young adults in using government schools and teachers, with the Liberia. We hypothesized that participation in classes scheduled in the evening to make learn- an alternative basic education project where ing accessible to young people who have left the this program, called HealthyActions,wasimple- formal school system. In the Advancing Youth mented would be associated with greater use of Project’s approach to alternative basic education, modern contraception among women, greater Level 1 is for students who have never been to condom use among young women and men, school at all or have dropped out in grade 1 or 2. and greater use of HTC services by women and Level 2 is roughly equivalent to grades 3 and 4, men, compared with participation in an alternative and level 3 to grades 5 and 6. Students are placed basic education project where HealthyActions was in the appropriate level based on the results of a not implemented. short literacy and math test at intake.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 437 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

HealthyActions was implemented through a part- Government’s key priorities for youth initiatives.28 nership between Population Services Interna- In line with the policy, HealthyActions: tional (PSI) Liberia and EDC, and was originally Used an assets-based approach that focused funded in partnership between the 2 organi- on building internal assets (e.g., self-esteem, zations; 50% from EDC via the Advancing Youth communication and assertiveness skills, gender Project and 50% from a PSI innovation fund. The transformative attitudes) as well as external Advancing Youth Project’s life-skills module has assets (e.g., access to education and social been one of the most popular among learners cohesion) because of its relevance and applicability; however, teachers are often uncomfortable address- Involved communities by engaging learners, ing SRH topics with learners.27 HealthyActions their parents, and their community members provided the opportunity to bring in expert in health service delivery and promotion on trainers to reinforce learning and to link learners community celebration days directly to health services. Since the majority of Created opportunities for youth participa- learners are young women who never attended or tion particularly through insight gathering, dropped out of school due to pregnancy, topics piloting the curriculum, and peer education related to contraception and parent and partner opportunities communication are of extreme interest. Worked cross-sectorally by integrating SRH HealthyActions HealthyActions was conceived as a package of education and services within formalized (albeit consisted of educational and empowering SRH activities that nontraditional) education programming educational would motivate learners to use health services, activities to which were brought to them to try on-site. The To begin, PSI conducted an analysis of pub- motivate learners program consisted of 15 hours of classroom-based lished and gray literature with a particular eye ’ to use health modules delivered over the course of 5 sequential toward any discourse on Liberian youth s assets services, which days, focused on building knowledge and improv- and behavioral drivers. Results were used to sup- were brought to ing attitudes and skills related to SRH. The port the design of a focus group discussion guide them to try package then culminated in a community-based that was fielded through 2 group discussions with celebration on the 6th and final day of the USAID Advancing Youth Project learners to glean on-site. program, during which HealthyActions participants insights about their knowledge, attitudes, and and their families and communities were invited perceptions of SRH. Key insights from those for a day of music, food, and health services. The discussions—such as misconceptions about the celebrations were held on-site, with government safety and efficacy of certain contraceptive health workers providing contraceptive counsel- methods, harmful gender norms, and negative ing and services, HTC, and referrals to local clinics attitudes toward condom use—were used to for HealthyActions participants and other community design curriculum content. Other insights related members. At sites where classrooms were avail- to young people’s hopes, aspirations, and curiosity able, health care providers used the classrooms were used to design curriculum delivery approaches. as private and confidential spaces to provide The curriculum was grounded in the informa- services. At sites where private indoor space tion, motivation, behavioral skills, and resources was limited, tables and curtains were used to (IMBR) model, a health behavior theory often create private and clean areas for service delivery. used in peer-based learning contexts that places Participants who required clinical follow-up (e.g., equal emphasis on teaching young people knowl- for a positive HIV test, insertion of an intrauterine edge and skills while also increasing access to device [IUD], continued injections) were provided products and services.29 Adult learning approaches with referrals to the nearest government health were incorporated that included games and trivia facility. to build knowledge; values exercises to address The HealthyActions curriculum was designed attitudes and norms; and role-play and small through an iterative and youth-centered process group discussions to drive critical thinking, self- that included (1) insight gathering with learners; esteem, and skill-building. Many of the curriculum’s (2) curriculum development; (3) pilot testing activities drew on existing resources developed with learners; and (4) adaptation and imple- by international organizations, adapted for the mentation. Additionally, PSI looked to USAID’s Liberian context. recently released Youth in Development Policy to On Day 1, learners were introduced to the ensure the program design aligned with the U.S. program and to one another, and they set group

Global Health: Science and Practice 2016 | Volume 4 | Number 3 438 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

norms, took a pretest, and learned about ‘‘Healthy fit the needs of participants and facilitators.19,21 Competition’’—a competitive element to the pro- The curriculum was originally developed for gram. On that first day, the class was split into 15–24-year-olds—the age range defined by the 2 teams. Every day thereafter, there was at least United Nations as ‘‘youth.’’ However, during pilot one competitive exercise whereby teams com- testing we learned that the likely age range of peted for points. At the end of the week, the team participants would skew older and that many with the most points won a small prize, presented would already be parenting. Although the Advan- at the community celebration day. The competi- cing Youth Project is open to younger partici- tion provided a fun and engaging way to keep pants, it tends to draw older participants who learners motivated and to improve teamwork. were denied schooling and now see that they Day 2 focused on puberty, sex, and reproduc- need literacy and numeracy skills in order to tive health. Activities included body mapping better support themselves and their families, and exercises, true/false trivia games, role-playing exer- to gain respect in the community. However, levels cises to practice talking about sex (with parents, of SRH knowledge remain very low, regardless of health care workers, sexual partners, and their age. We therefore added content to the curricu- own children), and values exploration exercises lum that included positive parenting and parent- to explore feelings about sexuality. child communication about SRH.31 Day 3 focused on HIV/AIDS and included a Skilled facilitators, called County Health Leads, game of tag to explain how HIV works in the were trained to deliver the curriculum. County body, true/false trivia, small group discussions Health Leads were PSI employees, trained as about HIV stigma, and creating songs and dances health educators, and chosen for their ability to to promote HTC. communicate well with young adults in a fun, Day 4 focused on contraception and pregnancy. nonhierarchical, and yet still authoritative fash- Activities included a trivia game to learn about ion. County Health Leads worked in pairs of one the available contraceptive methods, condom man and one woman. They were not teachers demonstration and practice, storyboard discus- from the Advancing Youth Program—who were sions about early pregnancy and community uncomfortable with and lacked knowledge to norms surrounding pregnancy outside of mar- provide in-depth information about SRH—but riage, and values exploration surrounding use rather knowledgeable outsiders brought into the of contraception. program to engage topics that ‘‘insiders’’ usually Day 5 focused on communication skills and cannot broach. County Health Leads visited a taking action. Activities included a storyboard dis- different USAID Advancing Youth Project site cussion on gender and power, small group discus- each week, in pairs, to implement the curriculum. sions about coerced vs. voluntary sex, defining During the course of the HealthyActions week, and practicing assertiveness skills, and a game to the County Health Leads elected, with their per- understand the negative effects of alcohol and mission, 2 learners to serve as Peer Health drugs on decision making.30 Each day ended with Educators. The learners were selected based on a call to action (such as getting an HIV test or their demonstrated leadership and grasp of the talking to a health care provider about contra- curriculum, as well as eagerness to remain involved ception) and a condom demonstration, including with the project after the 6-day intervention. practice putting condoms on models. Peer Health Educators volunteered to sustain the Day 6 was a clinic celebration day to mark the teachings of the program among their peer end of the curriculum and to introduce on-site learners and other community members. Follow- health services to HealthyActions participants and ing HealthyActions, they received a formal 3-day members of the community. The aim in linking training from the County Health Leads and PSI health service provision in a ‘‘burst’’ with the cur- staff on delivery of the HealthyActions curriculum. riculum was to enable participants to act quickly However, because of low literacy levels, PSI and on what they had learned and share their learning EDC first created 8 recordings that could be with family and community members to create played through basic mobile phones. The record- further motivation for health service utilization, ings, produced in the style of popular Liberian thus building trust in government health services. radio shows and in Liberian English, feature a The curriculum was pilot tested with a group host who guides the Peer Health Educator of 15 learners during a weeklong facilitator through a facilitation of that recording’s lesson. training so that adaptations could be made to For instance, the host says, ‘‘My friend [referring

Global Health: Science and Practice 2016 | Volume 4 | Number 3 439 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

to the Peer Health Educator], please ask the group The total number of eligible sites was strati- what they have heard about the oral contracep- fied by county, and sites were sampled probability tive pill.’’ After a 2-minute pause, the host proportionate to the number of eligible sites per provides a brief explanation of the oral contra- county. Site names were written on paper and ceptive pill. A sample recording can be found on stacked in separate piles. One county at a time, PSI’s website.31 Peer Health Educators convened site names were placed in a box. Names of small gatherings within their communities to sites were selected from the box, with the first listen to and discuss the recorded lessons on the draw for treatment sites, the second for control phones. Peer Health Educators also provided sites, etc. One member of the research team vouchers for family planning referrals to locally selected treatment sites; another selected control accessible clinics. Peer Health Educators used sites; and a third held the box. Blinding of sites FrontlineSMS to send text messages to PSI and and participants to intervention status was not EDC on the number of participants, disaggre- possible. The University of Liberia Institutional gated by gender, in their meetings and the Review Board provided ethical approval for this number of vouchers distributed. study. HealthyActions was implemented twice between 2012 and 2014. The first round of implementation took place from August 2012 to July 2013 in Data Collection Nimba and Montserrado counties as a pilot to Baseline data collection occurred in January– gain feedback on the curriculum and approach. March 2014 and was conducted on a rolling basis The second round of larger-scale implementation across all 60 selected sites. Teams of data col- took place from November 2013 to May 2014 in lectors and quality control supervisors were Bong, Grand Bassa, Lofa, Montserrado, and assigned by county, due to transportation chal- Nimba counties. This second round included the lenges. At intervention sites, baseline data were Peer Health Educator component of the program. collected on day 2 of the HealthyActions week, The evaluation for the program was undertaken while at control sites baseline data were collected during the second round of implementation. during standard USAID Advancing Youth Project sessions. Eligible study participants—current USAID Advancing Youth Project learners between Evaluation Design the ages of 15–35—were randomly sampled from We used the Consolidated Standards of Reporting the group of learners available on the day that the Trials (CONSORT) statement to structure report- data collection team visited a selected learning ing of this evaluation. A randomized design was site. After informing all learners about the purpose used to evaluate the intervention during the and objectives of the evaluation, a minimum of second phase of HealthyActions. Eligible USAID 20 eligible learners per site were selected by Advancing Youth Project learning sites were iden- asking learners to pull blocks out of a bucket. The tified as clusters to be randomized. Learning sites number of blocks in the bucket corresponded to were randomly assigned to receive HealthyActions the number of learners at the site that day, with during the evaluation period or to serve as control blocks coded for selection to participate in the sites. Learning sites were not eligible to partici- evaluation. Learners could not see into the bucket pate if they were private education institutions when they selected a block. Identities of selected and/or had participated in the first phase of participants were cross-checked against a site HealthyActions. Out of 150 sites assessed for roster after sampling. eligibility, 34 sites were assigned to the treatment After ensuring informed consent, trained group and 26 sites to the control group using a female data collectors administered a 20–25 lottery process executed by PSI/Liberia research minute questionnaire on sociodemographic char- staff (Figure). Sample size was calculated using acteristics, parent-child and partner communica- G*Power software version 3.1.6. The follow- tion, gender norms, sexual history, contraception ing parameters were used: difference between knowledge and use, HIV knowledge and testing 2 independent means statistical test, a priori type experience, and, for female respondents, preg- of power analysis, one-tailed, .005 probability nancy history. The questionnaire was developed of Type I error, 95% power, 1/1 allocation ratio. using Liberian English terms when possible, and Resulting sample size was 1,144, rounded to data collectors were trained to administer the 1,200. questionnaire consistently. Respondents above

Global Health: Science and Practice 2016 | Volume 4 | Number 3 440 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

FIGURE. Flow Diagram of Enrollment, Allocation, and Analysis Process

the age of 18 were administered a verbal consent standard USAID Advancing Youth Project ses- script, and enumerators recorded their consent to sions at both intervention and control sites in participate via tablet. A paper consent form was June–July 2014. Because of expected difficulties administered to eligible respondents under the in contacting learners sampled at baseline and age of 18, with the minor’s informed assent, and confirming their identities, the survey was admin- consent provided by the learning site adminis- isteredtoanewlyselectedsampleofUSAID trator in lieu of a parent. Data were collected Advancing Youth Project learners using the same electronically using SurveyToGo on Android sampling techniques as at baseline. tablets. Questionnaire administration occurred For this analysis, we removed cases from away from other learners, typically in an empty 5 learning sites (4 sites assigned to the control classroom. group received the intervention; 1 site was not Endline data collection occurred after all inter- included in the initial sampling frame). The final vention sites completed HealthyActions and during analytical sample included 55 sites.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 441 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

Measures demographic variables (learning level [1, 2, or 3], Outcome measures used for this analysis were as income, marital status, age, number of children) follows: were included in the model to account for dif- Among women: ferences between the samples at endline. Inter- actions with age and gender were tested for all 1. Contraceptive use (modern or traditional outcomes. Standard errors were clustered at the methods)32 learning-site level to account for clustering due Among women and men: to randomization. All analyses were conducted using Stata 13. 2. Condom use with regular partners and with casual partners (never; intermittently, including sometimes or most of the time; or always) RESULTS 3. HIV knowledge At baseline, a total of 1,255 learners were recruited 4. Ever been tested for HIV and know the result to participate in the evaluation across 34 inter- 5. Intention to seek HIV counseling and testing in vention sites and 26 control sites (Figure). At the next year endline, 1,142 learners were recruited across the same number of learning sites. For this analysis, Treatment was defined as assignment to a learn- we assessed outcomes for 1,157 learners at base- ing site where HealthyActions was implemented. line and 1,052 learners at endline in 55 learning Other measures assessed included sociodemo- sites. HealthyActions treatment and control groups graphic factors of age, gender, daily income shared similar demographic profiles at baseline (no income; 1–100 Liberian dollars [LRD]; 100– and endline, indicating that randomization pro- 500 LRD; 600–1,000 LRD; or 41,000 LRD, where duced a balanced sample (Table 1). At baseline 85 LRD = US$1), alternative basic education learn- and endline, the sample consisted primarily of ing level (1, 2, or 3), number of living children, women with an average age of 28 years. The and marital status (single/never married, married/ majority of respondents had a very low level of living together, divorced, or widowed). We also education (corresponding to Alternative Basic assessed women’s reproductive history including Education level 1), were married or living with number of pregnancies and live births, currently a partner, and had children. At baseline, there pregnant or breastfeeding, and reproductive were no statistically significant differences between intentions. the treatment and control groups on any demographic variables. At endline, the control group Statistical Analysis had slightly more women than the treatment Descriptive statistics were calculated at baseline group, and the control group was slightly more and endline. Balance between treatment and con- educated and had a slightly higher average trol was assessed at baseline using chi-square income. We saw modest tests and t tests to assess quality of randomiza- We saw modest improvements in condom use improvements in tion. For all outcomes, a difference-in-difference with a regular sexual partner after taking part in condom use with a logistic regression model with fixed effects was HealthyActions (Table 2). No statistically signifi- regular sex used to assess the effect of HealthyActions partici- cant differences by age (15–24 vs. 25–35) or partner after pation on observed changes in the outcomes of gender were found. After adjusting for potential taking part in interest. The general model was specified as: confounders, individuals in the treatment sites HealthyActions. were 12% less likely to report never using a Y ¼ a þ b treatment þ b time þ b covariates condom with a regular partner over the last 1 2 3 month when compared with the control group þ b time treatment = 3 (P .02). Positive changes in condom use were also noted among learners reporting that they use This model adjusted for both observed and condoms always or intermittently, although these unobserved differences between treatment and findings were not statistically significant. Fre- control at baseline to minimize possibility of quency of condom use was slightly higher among confounding. The interaction term between treat- learners reporting condom use with a casual ment (coded 0 = control site and 1 = treatment partner, but at endline only 38% of those in the site) and time (coded 0 = baseline and 1 = end- control group reported always using a condom line) served as the intervention effect. Other key with their casual partners compared with 44% of

Global Health: Science and Practice 2016 | Volume 4 | Number 3 442 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

TABLE 1. Demographic Proﬁle of Sampled Learners at Baseline (January–March 2014) and Endline (June–July 2014), Selected Counties of Liberia

Baseline (N = 1,157) Endline (N = 1,052)

Control Treatment Control Treatment (n = 518) (n = 639) P Value (n = 503) (n = 549) P Value

Female, No. (%) 368 (71.04) 466 (72.93) .48 327 (65.01) 400 (72.86) .01 Age, years, mean (SD) 28.41 (5.93) 28.09 (5.81) .35 28.03 (5.86) 28.86 (6.24) .41 No. of children, mean (SD) 2.91 (2.08) 2.83 (2.16) .55 2.73 (2.17) 2.79 (1.93) .65 No. of children, No. (%) 65 (12.55) 77 (12.05) .80 75 (14.91) 86 (15.66) .73 Marital status, No. (%) .70 .08 Single/never married 187 (36.10) 241 (37.72) 167 (33.2) 180 (32.79) Married/living together 313 (60.42) 379 (59.31) 320 (63.62) 340 (61.93) Divorced 14 (2.70) 12 (1.88) 6 (1.19) 20 (3.64) Widowed 4 (0.77) 7 (1.10) 10 (1.99) 9 (1.64) Educational level, mean (SD) 1.72 (0.82) 1.65 (0.73) .12 1.78 (0.81) 1.60 (0.75) o.01 ABE level 1, No. (%) 268 (51.74) 324 (50.70) 232 (46.12) 306 (55.74) ABE level 2, No. (%) 127 (24.52) 216 (33.80) 148 (29.42) 156 (28.42) ABE level 3, No. (%) 123 (23.75) 99 (15.49) 123 (24.45) 87 (15.85) Income, No. (%) .10 .08 0 74 (14.29) 127 (19.87) 98 (19.48) 137 (24.95) LRD 1–100 80 (15.44) 83 (12.99) 52 (10.34) 70 (12.75) LRD 100–500 234 (45.17) 257 (40.22) 210 (41.75) 237 (43.17) LRD 600–1,000 68 (13.13) 77 (12.05) 79 (15.71) 57 (10.38) More than LRD 1,000 58 (11.20) 94 (14.71) 61 (12.13) 47 (8.56) Don’t know/no answer 4 (0.77) 1 (0.16) 3 (0.60) 1 (0.18)

Abbreviations: ABE, alternative basic education; LRD, Liberian dollars; SD, standard deviation.

those in the treatment group. None of these ages 15–19 we saw a 13 percentage point increase changes was statistically significant. (P = .005). Specifically, participation in HealthyActions Young women Among women who were not pregnant or wasassociatedwithan8percentagepointparticipating in breastfeeding at the time of the survey, substan- increase in the probability of using oral con- HealthyActions tial increases were seen in use of contraception traceptives (P = .06) and a 9 percentage point were 13% more for those in the treatment group (Table 3). Young increase in the probability of using implants likely to report = women participating in HealthyActions were 13% (P .007), and there were no differential effects using a modern more likely to report using a modern method of by age. In all groups before and after the inter- contraceptive contraception than women in the control group vention, injectables and pills remained the first method than o (P .01). Increases in modern contraceptive use and second most popular methods, respectively. women in the were greatest among unmarried women (33.7 per- Among the poorest women, at endline those in control group. centage points; P = .002), and among adolescents the treatment group were 31% (P = .009) more

Global Health: Science and Practice 2016 | Volume 4 | Number 3 443 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

TABLE 2. Self-Reported Condom Use With Regular and Casual Partners Among Sampled Learners at Baseline (January–March 2014) and Endline (June–July 2014), Selected Counties of Liberia

Baseline Endline Average Marginal Control Treatment P Value Control Treatment P Value Effects (95% CI) P Value

With regular partner .62 .01 Sample size 356 427 312 304 Never 229 (64.33) 288 (67.45) 199 (63.78) 159 (52.30) -0.12 (-0.22, -0.02) .02 Intermittently 102 (28.65) 114 (26.70) 94 (30.13) 113 (37.17) 0.08 (-0.02, 0.17) .13 Always 25 (7.02) 25 (5.85) 19 (6.09) 32 (10.53) 0.04 (-0.01, 0.09) .10 With casual partner .24 .85 Sample size 89 121 80 84 Never 34 (38.20) 56 (46.28) 25 (31.25) 24 (28.57) -0.05 (-0.18, 0.07) .41 Intermittently 33 (37.08) 31 (25.62) 24 (30.00) 22 (26.19) -0.4 (-0.16, 0.08) .48 Always 20 (22.47) 33 (27.27) 30 (37.50) 37 (44.05) 0.10 (-0.07, 0.28) .25 Don’t know 2 (2.25) 1 (0.83) 1 (1.25) 1 (1.19) – –

Abbreviation: CI, conﬁdence interval. Data reported as No. (%) except when otherwise noted.

likely to report using an implant than those in the age of 19 in HealthyActions sites were 32% more control group. likely to know where to get an HIV test than Substantial Substantial improvements were seen in their counterparts in the control sites (Po.001). improvements HIV counseling and testing after participating A 3-way interaction term between assessment were seen in HIV in HealthyActions (Table 4). At endline, 42% of period, intervention exposure, and age was also counseling and individuals in the control group reported having found to be highly significant (P = .02), indicat- testing after been tested for HIV compared with 88% in ing that the intervention had a positive, differ- – participating in the treatment group. After adjusting for base- ential effect on adolescents 15 19 years old HealthyActions. line probability and other potential confounders, compared with the general population of learn- participation in HealthyActions resulted in a 45% ersinthestudy.Girlsundertheageof19also increase in the probability that an individual had showed increased knowledge of where to obtain ever been tested for HIV and knew the result an HIV test as a result of HealthyActions: ado- (Po.001). There was no differential effect of the lescent girls in HealthyActions sites were 40% intervention for HIV testing among men versus (Po.001) more likely to report knowing where women, nor by age. Intention to test was high to obtain an HIV test than their counterparts in Beneﬁts of in both groups at baseline; however, learners in control sites; in comparison, adolescent boys participating in treatment sites had a 6% higher probability were 26% (P = .02) more likely to know where to HealthyActions than those in the control sites to report plan- obtain a test than their counterparts in control were concentrated ning to get an HIV test in the upcoming year sites. A 4-way interaction term was also tested among youth (Po.001). between assessment period, intervention expo- under the age Benefits of participating in HealthyActions were sure, age, and gender, which was also found to = of 19. concentrated among youth under the age of 19, be statistically significant (P .01), indicating particularly related to knowledge of HIV and that HealthyActionshadastrongereffectamong where to get tested. On average, youth under the girls than boys.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 444 nesv ru erigadO-ieSrie oIpoeSHwww.ghspjournal.org SRH Improve to Services On-Site and Learning Group Intensive lblHat:SineadPatc 06|Vlm ubr3 Number | 4 Volume | 2016 Practice and Science Health: Global

TABLE 3. Current Contraceptive Use Among Non-Pregnant, Non-Breastfeeding Women at Baseline (January–March 2014) and Endline (June– July 2014), Selected Counties of Liberia

Baseline (N = 546) Endline (N = 470)

Control Treatment Control Treatment Average Marginal (n = 232) (n = 314) P Value (n = 215) (n = 255) P Value Effects (95% CI)a P Value

Short-acting methods Injectable 56 (24.14) 76 (24.20) .99 65 (30.23) 62 (24.31) .15 -0.05 (-0.13, 0.04) 0.27 Male condom 19 (8.19) 29 (9.24) .67 18 (8.37) 28 (10.98) .34 0.03 (-0.0, 0.93) .37 Pill 53 (22.84) 53 (16.88) .08 33 (15.35) 60 (23.53) .03 0.08 (0.00, 0.17) .06 Female condom 1 (0.43) 0 (0.00) .24 0 (0.00) 1 (0.39) .21 – – Long-acting methods Implant 9 (3.88) 17 (5.41) .41 16 (7.44) 39 (15.29) .01 0.09 (0.03, 0.16) o.01 Female sterilization 1 (0.43) 1 (0.32) .83 0 (0.00) 0 (0.00) 4.99 – – Male sterilization 0 (0.00) 0 (0.00) 4.99 0 (0.00) 0 (0.00) 4.99 – – IUD 0 (0.00) 0 (0.00) 4.99 0 (0.00) 0 (0.00) 4.99 – – Any long-acting or permanent 10 (4.31) 18 (5.73) .63 16 (7.44) 39 (15.29) .01 0.10 (0.05, 0.16) o.01 method Any modern method 123 (53.02) 159 (50.64) .58 119 (55.35) 170 (66.67) .01 0.13 (0.04, 0.22) o.01 Traditional methods Rhythm 4 (1.72) 4 (1.27) .67 8 (3.72) 7 (2.75) .55 – – Other traditional 4 (1.72) 0 (0.00) .02 1 (0.47) 3 (1.18) .40 – –

Abbreviations: CI, conﬁdence interval; IUD, intrauterine device. Data reported as No. (%) except when otherwise noted. a Estimations for some methods couldnot be calculated because of the small number of users in the sample. 445 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

DISCUSSION ** ** ** We sought to assess the effectiveness of an intensive sexual and reproductive health interven-

Average tion embedded in an alternative basic education Marginal program for Liberian out-of-school young adults. Effects (95% CI) Combining Combining intensive group learning with provi- intensive group sion of on-site health services increased use of con- .001 0.06 (0.03, 0.09) .001 0.45 (0.38, 0.53) .001 0.19 (0.13, 0.26) learning with Value traception and HTC in this setting. The HealthyActions o o o provision of P group setting, educational components, and on-site on-site health access to services were designed, implemented services increased and tested as a package; thus, we cannot tease 1,052) 549)

= out whether some components had greater use of =

(n impact than others nor whether some compo- contraception and Treatment HIV testing and nents could have been removed. We hypothesized that the components interacted with each other counseling to achieve impact, but this could be formally services among 503) tested in future research. Liberian youth. = Control

(n Increases in use of contraceptive implants observed among the HealthyActions sites are likely related to the program’s emphasis on providing

Value information and counseling to young women

P about long-acting reversible contraceptive methods as well as improving access to the products themselves. Implants are generally used less fre- 1,157) Endline (N 639) = quently than other methods in Liberia: 11% of = currently married women in Liberia use injectable (n Treatment contraception and 2% use implants, while 22% of sexually active unmarried women use inject- 4 Baseline (N ables and 4% use implants. We suspect that

518) contraceptive implants were less familiar to =

Control HealthyActions learners. Many young women likely (n did not have sufficient information about implants from other sources, so supportive information about the method, provided in the context of informed choice, and with ready access to services, was an attractive offer. Further, contraceptive stock-outs are often widespread in Liberia, limiting access to the full range of methods. Easy access to a long- acting method likely helped to motivate women to choose implants. We suspect that assurances that the commodities would be available increased Increased use of women’s confidence in and willingness to use the implants services provided on clinic celebration day. Coun- corresponded seling on removal of implants was also provided. with a decline in Despite the substantial increase in use of use of injectables implants, HealthyActions did not appear to have among an impact on use of long-acting or permanent .001. methods overall. Increase in the use of implants HealthyActions o P HIV Knowledge, Attitudes, and Testing Behaviors Among Sampled Learners at Baseline (January–March 2014) and Endline (June–July corresponded with a decline in use of injectables participants, ** suggesting a among HealthyActions participants, although the .05; decline was not statistically significant. It was not possible o P possible in this study to measure method switch- Do you intend to get an HIV test within the next year? 474 (91.51) 560 (87.64) .01 457 (90.85) 533 (97.09) Have you ever been tested for HIV and received the result? 237 (45.75) 241 (37.72) .001 211 (41.95) 482 (87.80) TABLE 4. Have you heard of HIV or AIDS?Do you know where to access HIV testing services?Abbreviation: CI, conﬁdence interval. Data reported as No.* (%) except when otherwise noted. 366 (70.66) 423 (66.20) 510 (98.46) .07 629 (98.44) 357 (70.97) .99 494 (89.98) 498 (99.01) 547 (99.64) .38 0.00 (-0.01, 0.02) substation effect. 2014), Selected Counties of Liberia ing, but it is possible that the availability of

Global Health: Science and Practice 2016 | Volume 4 | Number 3 446 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

implants along with comprehensive information partner was statistically significant, but increases about all contraceptive methods contributed to a in use were not. Respondents may have struggled small substitution effect by which young women with distinctions between regular and casual using injectables switched to implants. As implants partners given that regular partners are often provide a longer-term solution for women who loosely defined in a Liberian context, and regular want to avoid a pregnancy than do injectables, may not equate with monogamous. According this finding is still consistent with program goals to the 2013 Liberian Demographic and Health and the value of informed choice. Survey, 7% of women reported 3.4 sexual part- Continuation rates may ultimately improve ners in the past year, and 18% of men reported among these women given that implants do not 8 partners.4 HealthyActions did not touch on poten- require resupply, which would also be a positive tial risks of concurrent partnerships, but the result of the study, although it is not possible to program encouraged consistent condom use and assess continuation with the data available. Longer- supported development of skills in partner com- term follow-up would need to be conducted to munication and condom negotiation. An inter- assess whether continuation rates improved after vention of longer duration may be needed to HealthyActions, which was beyond the scope of this address condom use with Liberian young adults. study. Further, additional research is needed to gain The large proportion of HealthyActions learners insight into sexual partner types and processes who were tested for HIV is indicative of the exist- of partner formation in Liberia, including the ing barriers to accessing HTC in Liberia. The influence of economic factors. Available evidence national average for HIV testing is 23% and 45% suggests that these processes are complex.12 for men and women, respectively,4 whereas 88% We implemented HealthyActions within a speci- of respondents in HealthyActions learning sites had fically targeted environment of learning sites pro- been tested for HIV at endline and knew their viding alternative basic education. The selec- results. The HealthyActions curriculum delivered tion of this setting was deliberate in order to lever- factual information and sought to discredit myths age a group setting that facilitated the formation and misconceptions about HIV in addition to of trusted peer networks. However, descriptive supporting confidentiality of services. Due to the statistics of sampled learners were generally interconnectedness of many Liberian commu- comparable to the rural Liberian population.4 nities, protocols around confidentiality are not HealthyActions’ approach may have broader applic- always respected. Many young Liberians don’t ability to other settings where young adults, parti- access health services, particularly those for HIV, cularly those outside the formal school system, for fear of being gossiped about by staff and older gather in Liberia or in similar settings, such as patients.33 HealthyActions may have also succeeded women’s associations, farmer groups, or trade in forging a connection between learners and associations. However, some specific subgroups of their local clinics or hospitals. Despite Liberia’s out-of-school youth, such as male ex-combatants, low HIV prevalence, HealthyActions’ large effect on may need different intervention strategies. use of HIV testing and counseling services holds As implemented in this setting, HealthyActions promise for replication in other settings with was designed as a short, intensive intervention to higher HIV prevalence. test the approach. However, the implementation The Condom use was featured throughout the model could be readily adapted to increase its HealthyActions HealthyActions curriculum, including activities that scalability. HealthyActions could be delivered over model could be highlighted (1) the dual protective benefits of the course of several weeks—with one activity per readily adapted to condoms (to prevent HIV and other sexually trans- week for instance—rather than in 5 consecutive increase its mitted infections [STIs] as well as unintended days. The curriculum could also be delivered as scalability. pregnancy); (2) how to negotiate condom use one intensive burst, with follow-up ‘‘booster’’ particularly by young women; and (3) condom sessions every few months to sustain the mes- demonstrations and practice with models. There sages. Additional studies could be used to measure are a few possible explanations to account for the effectiveness of the short-term, intensive the changes in condom use with regular part- ‘‘burst’’ of education versus a longer-term, lower ners that we observed. Trends were suggestive of dosage program. Linkage to service delivery could improvements in intermittent or consistent use also be adapted. In this setting, linking motiva- of condoms with regular partners. The decline in tions and skills built into the curriculum to use reporting never using condoms with a regular of readily accessible services was important to

Global Health: Science and Practice 2016 | Volume 4 | Number 3 447 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

enhance participants’ and community members’ that tracking individuals over time would not be trust in the health system. In other settings, feasible due to challenges in ensuring that unique delivery of the curriculum could be timed to identifiers would remain unique. We therefore coincide with regularly scheduled mobile health recruited cross-sectional samples at baseline and units, or the curriculum could provide vouchers or endline, using the same sampling procedure to referrals to static health clinics, in either the ensure comparability. It is possible that our design public or private sector. Again in this setting, was under-powered, since initial sampling plans clinic celebration days included provision of did not fully account for clustering of learning contraception and HTC services. Antenatal and/ sites. However, substantial effect sizes and narrow or postnatal care could potentially be added. confidence intervals, even accounting for clustering The components of the program promote ready of standard errors during analysis, suggest that we adaptation and investigation to determine what a did have sufficient statistical power. ‘‘lighter touch’’ yet still effective implementation Due to the rolling nature of data collection, model might be. intervention sites received differing recall periods Our findings contribute to filling a gap in following implementation of HealthyActions,which evidence for how to reach the large population may have attenuated estimates. Recall of some of out-of-school youth and young adults in sub- outcomes, such as condom use, may have been Saharan Africa, who are otherwise engaged in a subject to social desirability bias. group learning setting, with comprehensive SRH The timing of endline data collection coin- information and services. A market-based inter- cided with harvest season, rainy season, and the vention in Nigeria found similar increases in timing of tribal bush schools. As data collection contraceptive use, but no change in condom took place during the early evening hours, the use among sexually active youth.34 Community- onset of the harvest season meant that some based and structural interventions can contribute learners were needed on the farms until after to attitudinal changes around sexual risk and dark. This was problematic because some learners partnership formation, and there is some evi- would not be able to attend class until much later dence of reductions in risk of STIs.35 However, than usual and missed the sampling process. In these interventions have tended to focus on STIs/ an effort to address this issue, a small number of HIV, without necessarily combining access to sites were resampled after those working on contraceptive services. Youth-friendly services the farms arrived. In many parts of rural Liberia, can successfully provide HTC but are challenged participation in tribal bush schools marks the to retain youth in follow-up care, a limitation of transition from childhood to adulthood. Often, HealthyActions as well.36 However, we have not the young people participating in the bush schools been able to locate programs that show compar- will be away for weeks at a time; this happened to able increases in levels of HIV testing uptake coincide with endline data collection at one site, among African youth. and the site was therefore resampled. Training health service providers in youth- friendly services was outside the scope of this Limitations project, and therefore we could not guarantee that This study had several limitations. Because Healthy- the health services we brought on the community Actions was designed as a package that combined a celebration day would be aligned with standards ‘‘burst’’ of learning with rapid provision of services, for youth-friendly health services.37 PSI had since we cannot assess the relative effects of the com- undertaken a training program for health care ponents. However, the learning component was providers in the private sector in Monrovia to designed to motivate use of services, and thus offer youth-friendly health services, particularly attempting to tease out effects attributable to each with adolescents and younger, unmarried youth component may not have been appropriate. in mind, and this program ran concurrently to Further, we were not able to assess any potentially the second round of HealthyActions. Funds were beneficial ‘‘spill-over’’ of HealthyActions in uptake of originally allocated for scale-up of youth-friendly services by community members, since only Advan- health services; however, they were redirected to cing Youth learners were sampled. We initially Ebola efforts. The County Health Leads did meet intended to recruit and follow a sample of learners with the government health care providers before in intervention and control sites over time to control the program to prepare them for the community for individual preferences. However, we determined celebration day and confirm their participation

Global Health: Science and Practice 2016 | Volume 4 | Number 3 448 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

and mitigate against the risk of unfriendliness to acknowledge the data collectors who administered surveys in the during the intervention. field and USAID Advancing Youth Project and HealthyActions staff who implemented the program and facilitated data collection with learning sites. Lee-Ann Gallarano provided research assistance and technical support in completing the manuscript. Jon Silverstone and CONCLUSION Christopher Ying facilitated writing and data analysis. Deep appreciation goes to USAID Advancing Youth Project learners who Despite a range of obstacles to progress for Liberia’s participated in this evaluation. young people, the desire for opportunities for learn- Competing Interests: None declared. ing and growth is ever-present. HealthyActions brought together key components of established social ties, intensive learning, and convenient SRH REFERENCES services, presenting a new opportunity for many 1. Walker G, Millar Wood JC, Allemano E. Liberia youth fragility people in need. Based on our findings, this com- assessment. North Bethesda (MD): Aguirre Division of JBS prehensive package targeted to young adults appears International, Inc.; 2009. Co-published by Associates for Global to have produced substantial results; expanding Change. Available from: http://pdf.usaid.gov/pdf_docs/ access resulted in increased uptake of both modern Pnadq258.pdf contraceptives, particularly implants, and HIV 2. Black M, Pain D. The situation of children and women in Liberia: from conflict to peace. Monrovia (Liberia): United Nations counseling and testing. Children’s Fund (UNICEF); 2012. Available from: http://www. The concentration of positive family planning unicef.org/liberia/SITAN_LBR2012-2017.pdf results among youth under 19 and the success 3. Ministry of Health and Social Welfare (MOHSW), Republic of in increasing uptake of contraceptive implants Liberia. National sexual and reproductive health policy 2010. suggests that the HealthyActions model is success- Monrovia (Liberia): MOHSW; 2010. Available from: http:// liberiamohsw.org/Policies%20&%20Plans/National%20Sexual% fully affecting the portion of the target audience 20&%20Reproductive%20Health%20Policy.pdf that is most difficult to reach and that stands to 4. Liberia Institute of Statistics and Geo-Information Services gain the most from using contraception. (LISGIS); Ministry of Health and Social Welfare [Liberia]; HealthyActions targets youth who are unlikely National AIDS Control Program [Liberia]; ICF International. to be reached by traditional health outreach efforts, Liberia 2013 demographic and health survey key findings. Rockville (MD): ICF International; 2014. Co-published by LISGIS . and holds promise to enact substantial behavior Available from: https://dhsprogram.com/pubs/pdf/SR214/ change over a short period of time. In a context SR214.pdf where SRH knowledge and youth-friendly services 5. Ministry of Education, Government of the Republic of Liberia. are nearly nonexistent, getting the conversation Education statistics for the Republic of Liberia. Monrovia (Liberia): started and making services available were the Ministry of Education; 2013. Available from: http://fhi360bi. org/user/liberia/yearbook/2013_LIBStatBooklet2.pdf highest priorities of the program. Follow-on funding was sought and is considered to be crucial 6. World Bank. Liberia education country status report: out of the ashes--learning lessons from the past to guide education recovery to institutionalizing the program and sustaining in Liberia. Africa Education Country Status Report. Washington longer-term behavior change. In Liberia, this (DC): World Bank; 2010. Available from: http://documents. became challenging due to staff turnover and the worldbank.org/curated/en/257321468057236139/pdf/ 636350WP0P117600Liberia0CSR000Edit4.pdf need to respond to the Ebola epidemic. However, the program’s focus on sexual and reproductive 7. Kruk ME, Rockers PC, Williams EH, Varpilah ST, Macauley R, Saydee G, et al. Availability of essential health services in post- health integration within an established learn- conflict Liberia. Bull World Health Organ. 2010;88(7):527–534. ing environment, particularly among low-literacy CrossRef. Medline populations, presents an adaptable solution for 8. Glinski A, Sexton M, Petroni S. Adolescents and family planning: health programming across the country and the what the evidence shows. Washington (DC): International Center region. for Research on Women (ICRW); 2014. Available from: http:// www.icrw.org/sites/default/files/publications/140701% 20ICRW%20Family%20Planning%20Rpt%20Web.pdf Acknowledgments: Support for HealthyActions, including this evaluation, was made possible by internal Population Services 9. Chandra-Mouli V, Svanemyr J, Amin A, Fogstad H, Say L, Girard International (PSI) and Education Development Center (EDC) F, et al. Twenty years after International Conference on Population resources as well as by the generous support of the American people and Development: where are we with adolescent sexual through the United States Agency for International Development, and reproductive health and rights? J Adolesc Health. 2015; USAID/Liberia Cooperative Agreement No. 669-A-11-00001 to 56(1 Suppl):S1–S6. CrossRef. Medline Education Development Center. Manuscript preparation was made possible under the terms of USAID Cooperative Agreement No. 10. Allen M, Devitt C. Intimate partner violence and belief systems AID-OAA-A-10-0030 to Population Services International. in Liberia. J Interpers Violence. 2012;27(17):3514–3531. The content and opinions expressed within do not necessarily reflect CrossRef. Medline the views of USAID or the United States Government. The authors would like to thank the monitoring and evaluation teams of PSI and 11. Okigbo CC, McCarraher DR, Chen M, Pack A. Risk factors for EDC in Monrovia and Washington, DC, for their assistance in transactional sex among young females in post-conflict Liberia. collecting, cleaning, and analyzing data. We would particularly like Afr J Reprod Health. 2014;18(3):133–141. Medline

Global Health: Science and Practice 2016 | Volume 4 | Number 3 449 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

12. Atwood KA, Kennedy SB, Barbu EM, Nagbe W, Seekey W, 26. Education Development Center (EDC). USAID Advancing Youth Sirleaf P, et al. Transactional sex among youths in post-conflict Project: livelihoods and work readiness report. Washington (DC): Liberia. J Health Popul Nutr. 2011;29(2):113–22. Medline EDC; 2014. 13. U.S. Agency for International Development (USAID). Fiscal year 27. Kallon TA, Lloyd D, Mooresmith R. Assessments of Healthy- 2016 Title II development food assistance projects for Democratic Actions phase 1. Monrovia (Liberia): Populations Services Republic of Congo, Ethiopia, and Liberia. Washington (DC): International; 2013. USAID; 2016. Available from: https://www.usaid.gov/sites/ 28. U.S. Agency for International Development (USAID). Youth in default/files/documents/1866/2016%20Final%20RFA.pdf development policy: realizing the demographic opportunity. 14. Kruk ME, Rockers PC, Tornorlah Varpilah S, Macauley R. Washington (DC): USAID; 2012. Available from: https:// Population preferences for health care in Liberia: insights for www.usaid.gov/sites/default/files/documents/1870/ rebuilding a health system. Health Serv Res. 2011; Youth_in_Development_Policy_0.pdf 46(6pt2):2057–2078. CrossRef. Medline 29. FHI 360, Youth Peer Education Network (Y-PEER). Youth peer 15. Maclay C, O¨ zerdem A. ‘‘Use’’ them or ‘‘lose’’ them: engaging education toolkit: training of trainers manual. New York: Liberia’s disconnected youth through socio-political integration. United Nations Population Fund; 2005. Available from: Int Peacekeeping. 2010;17(3):343–360. CrossRef. Medline https://www.fhi360.org/sites/default/files/media/documents/ 16. Dominguez S, Borba CPC, Fatima B, Gray DA, Stinehart C, Youth%20Peer%20Education%20Toolkit%20-%20The% Murphy G, et al. Mental health and adaptation of young 20Training%20of%20Trainers%20Manual.pdf Liberians in post-conflict Liberia: a key informant’s perspective. 30. Penny A. Methods handbook for youth social work: a collection Int J Culture Mental Health. 2013;6(3):208–224. CrossRef. of games, exercises and techniques for the moderation of training 17. Ruiz Abril, Elena M. Girl’s vulnerability assessment. Background and planning events with young people. Eschborn (Germany): study for ‘‘Economic Empowerment of Adolescent Girls in Deutsche Gesellschaft fur Techniche Zusammenarbeit (GTZ); Liberia.’’ Washington (DC): World Bank; 2007. 2006. Available from: https://www.k4health.org/sites/default/ files/teambuildinggames.pdf 18. Viner RM, Ozer EM, Denny S, Marmot M, Resnick M, Fatusi A, et al. Adolescence and the social determinants of health. Lancet. 31. Population Services International (PSI) [Internet]. Washington 2012;379(9826):1641–1652. CrossRef. Medline (DC): Population Services International; c2016. Healthy Actions curriculum tools. Washington (DC): Population Services 19. Gavin LE, Catalano RF, David-Ferdon C, Gloppen KM, Markham International; 2016 [cited 2016 Jun 14]. Available from: http:// CM. A review of positive youth development programs that www.psi.org/publication/healthy-actions/ promote adolescent sexual and reproductive health. J Adolesc Health. 2010;46(3 Suppl):S75–S91. CrossRef. Medline 32. Family Planning and Reproductive Health Indicators Database [Internet]. Chapel Hill (NC): UNC Carolina 20. Svanemyr J, Amin A, Robles OJ, Greene ME. Creating an Population Center, MEASURE Evaluation; [2014]. Contraceptive enabling environment for adolescent sexual and reproductive prevalence rate (CPR); [cited 2015 Feb 15] . Available health: a framework and promising approaches. J Adolesc from: http://www.cpc.unc.edu/measure/prh/rh_indicators/ Health. 2015;56(1 Suppl):S7–S14. CrossRef. Medline specific/fp/cpr 21. Denno DM, Hoopes AJ, Chandra-Mouli V. Effective strategies to 33. Y Care International. Neglected health issues facing young provide adolescent sexual and reproductive health services and people in Liberia. London: Y Care International; 2012. to increase demand and community support. J Adolesc Health. Available from: http://1c8puy1lylrov7ssf1oz3o22.wpengine. 2015;56(1 Suppl):S22–S41. CrossRef. Medline netdna-cdn.com/wp-content/uploads/2013/09/Liberia-youth- 22. Brieger WR, Delano GE, Lane CG, Oladepo O, Oyediran KA. led-research-summary.pdf West African youth initiative: outcome of a reproductive health 34. Odeyemi KA, Onajole AT, Ogunowo BE, Olufunlayo T, education program. J Adolesc Health. 2001;29(6):436–446. Segun B. The effect of a sexuality education programme among CrossRef. Medline out- of- school adolescents in Lagos, Nigeria. Niger Postgrad 23. Foundation for Women’s Health, Research & Development Med J. 2014;21(2):122–127. Medline (FORWARD); Planned Parenthood Association of Liberia; 35. Jewkes R, Nduna M, Levin J, Jama N, Dunkle K, Puren A, et al. International Planned Parenthood Federation (IPPF). In their own Impact of stepping stones on incidence of HIV and HSV-2 and words: girls from Liberia on sexuality, pregnancy and services. sexual behaviour in rural South Africa: cluster randomised London: FORWARD; 2012. Co-published by IPPF; 2012. controlled trial. BMJ. 2008;337:a506. CrossRef. Medline Available from: http://www.forwarduk.org.uk/wp-content/ uploads/2014/12/Girls-from-Liberia-on-Sexuality-Pregnancy- 36. Nkala B, Khunwane M, Dietrich J, Otwombe K, Sekoane I, and-Services.pdf Sonqishe B, et al. Kganya Motsha Adolescent Centre: a model for adolescent friendly HIV management and reproductive health for 24. Ouagadougou Partnership. Family planning: Francophone West adolescents in Soweto, South Africa. AIDS Care. 2015; Africa on the move, a call to action. Dakar (Senegal): 27(6):697–702. CrossRef. Medline Ouagadougou Partnership; 2012. Available from: http://www. prb.org/pdf12/ouagadougou-partnership_en.pdf 37. Population Services International (PSI). Making your health services youth-friendly: a guide for program planners and 25. Education Development Center, Inc. (EDC) [Internet]. Washington implementers. Washington (DC): PSI; 2014. Available from: (DC): EDC; c2016. USAID Advancing Youth Project; [cited 2016 http://www.psi.org/publication/making-your-health-services- Aug 23]. Available from: http://www.edc.org/usaid-advancing- youth-friendly-a-guide-for-program-planners-and- youth-project implementers/

Global Health: Science and Practice 2016 | Volume 4 | Number 3 450 Intensive Group Learning and On-Site Services to Improve SRH www.ghspjournal.org

Peer Reviewed

Received: 2016 Mar 2; Accepted: 2016 Jun 30

Cite this article as: Firestone R, Moorsmith R, James S, Urey M, Greiﬁnger R, Lloyd D, et al. Intensive group learning and on-site services to improve sexual and reproductive health among young adults in Liberia: a randomized evaluation of HealthyActions. Glob Health Sci Pract. 2016;4(3):435-451. http://dx.doi.org/10.9745/GHSP-D-16-00074.

& Firestone et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi.org/ 10.9745/GHSP-D-16-00074.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 451 ORIGINAL ARTICLE

A Randomized Controlled Trial of a Trauma-Informed Support, Skills, and Psychoeducation Intervention for Survivors of Torture and Related Trauma in Kurdistan, Northern Iraq

Judith Bass,a Sarah McIvor Murray,a Thikra Ahmed Mohammed,b Mary Bunn,b,c William Gorman,b Ahmed Mohammed Amin Ahmed,d Laura Murray,a Paul Boltone

ABSTRACT Supportive counseling type interventions are frequently provided to meet the mental health needs of populations in emergency and post-conflicts contexts, but it has seldom been rigorously evaluated. Existing evaluations from low- and middle-income countries provide mixed evidence of effectiveness. While Iraqi Kurdistan experienced relative stability following the fall of Saddam Hussein’s government, the population in the northern Dohuk region has continued to experience periodic violence due to conflicts with neighboring Turkey as well as more recent ISIS-associated violence. We evaluated the impact of a trauma-informed support, skills, and psychoeducation intervention provided by community mental health workers (CMHWs) on depressive symptoms and dysfunction (primary outcomes) as well as post-traumatic stress, traumatic grief, and anxiety symptoms (secondary outcomes). Between June 2009 and June 2010, 295 adults were screened; 209 (71%) met eligibility criteria (trauma exposure and a symptom severity score indicating significant distress and functional impairment, among others) and consented to participate. Of these, 159 were randomized to supportive counseling while 50 were randomized to a waitlist control condition. Comparing average symptom severity scores post-treatment among those in the intervention group with those in the waitlist control group, the supportive counseling program had statistically and clinically significant impacts on the primary outcomes of depression (Cohen’sd,0.57;P = .02) and dysfunction (Cohen’sd, 0.53; P = .03) and significant but smaller impacts on anxiety. Although studies by the same research team of psychotherapeutic interventions in other parts of Kurdistan and in southern Iraq found larger effects, this study adds to the global research literature on mental health and psychosocial support and shows that a well-trained and supervised program of trauma-informed support, skills, and psychoeducation that emphasizes the therapeutic relationship can also be effective.

INTRODUCTION problems.1 Physical torture has been shown to be a robust predictor of mental distress in prisoners of war raumatic events, including torture, put people and other conflict-affected or displaced populations T – at increased risk for a range of mental health years, and even decades, after torture occurred.1 3 To a Johns Hopkins Bloomberg School of Public Health, Department of Mental address these problems, the most frequent type of Health, Baltimore, MD, USA. mental health intervention provided in emergency or b Heartland Alliance International, Chicago, IL, USA. post-conflict contexts has been basic supportive counsel- c University of Chicago, School of Social Service Administration, Chicago, IL, USA. ing from trained community workers.4 This is consistent d Trauma Rehabilitation and Training Center; Iraq-Kurdistan Region-Sulaimani and Department of Community Health, Sulaimani Polytechnic University, with the Guidelines on Mental Health and Psychosocial Support Technical College of Health, Qirga, Iraq. in Emergency Settings, from the Inter-Agency Standing e Johns Hopkins Bloomberg School of Public Health, Center for Refugee and Committee (IASC), which recommends that basic sup- Disaster Response and Department of International Health, Baltimore, MD, USA. portive counseling components be provided by commu- 5 Correspondence to Judith Bass ([email protected]). nity health workers to support psychosocial well-being.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 452 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

The content and activities provided in interventions traumatic experiences, conflict resolution, and stress labeled as basic counseling vary widely. Commonly management experienced small mean declines in used strategies for trauma-affected adult popula- trauma symptoms, while those enrolled in basic tions include problem solving, conflict resolution, skill-building activities and waitlist controls experi- psychoeducation (information provided to the enced small mean increases in trauma symptoms participant about their condition), relaxation, and over the study period.14 sharing of traumatic experiences, as well as teaching The population in Iraqi Kurdistan has experi- skills in coping, stress management, and basic enced significant trauma in the past several communication.6–14 While the literature suggests decades. In the late 1980s, the regime of then- that evidence-based treatments, such as Cognitive Iraqi President Saddam Hussein conducted a Behavioral Therapy, are effective for mental health violent campaign in Iraqi Kurdistan, known as problems in confict and post-conflict settings (see, the Anfal, that included genocide, chemical weapon for example, Weiss et al. 201615), it is possible that attacks, and torture.16 Within Kurdistan’snorth- some of the commonly seen depression and anxiety ernmost governorate, Dohuk, hundreds of villages symptoms in such settings may be made more were razed leading up to and during the Anfal severe due to environmental stress, and these in the name of ‘‘Arabization.’’16 While Kurdistan symptoms may be amenable to a more generally experienced relative stability following the fall of supportive counseling approach. Saddam Hussein’s government, conflict between These types of basic supportive counseling Kurdish groups and neighboring Turkey continued Basic supportive programs, as opposed to more structured and to produce periodic violence in the Dohuk border counseling 17,18 manualized pyschotherapies (i.e., therapies that region. Recenly, the brutal advance of ISIS programs have follow a series of prescribed goals and techniques (the Islamic State of Iraq and Syria) has made seldom been to ensure uniformity across therapists), have Dohuk a major site of refuge for displaced Iraqis rigorously 4 19 seldom been rigorously evaluated, and existing and refugees. evaluated. evaluations from low- and middle-income coun- Even prior to the current ISIS conflict, the tries provide mixed evidence of effectiveness for population in the Dohuk region of Iraqi Kurdistan survivors of torture and related traumas. For has had little access to community-based mental example, a group problem-solving counseling health services for trauma. As part of a larger program delivered in conflict-affected areas of study to evaluate mental health services for Aceh, Indonesia, showed an effect on improving torture and trauma survivors throughout Iraqi daily functioning in men and coping in both men Kurdistan, in 2010 we evaluated a counseling and women, but showed no effect on depression program that is based on psychotherapies that are or anxiety symptoms.7 In a study among torture designed for trauma-affected populations and survivors in Nepal, a multidisciplinary counseling includes specific skills and psychoeducation program similarly improved functioning and components to improve the participants’ symp- decreased participants’ somatic symptoms, but toms of depression, anxiety and trauma, and did not reduce mental health symptoms.10 In a dysfunction. The intervention was provided by trial in Uganda, the vast majority of Sudanese community mental health workers (CMHWs) at refugees were found to still meet criteria for Ministry of Health clinics in the Dohuk governor- post-traumatic stress disorder (PTSD) following a ate of Kurdistan, Northern Iraq. The study aimed 4-session supportive counseling program that to assess the impact of the intervention on included problem solving, conflict resolution, primary outcomes of depressive symptoms and and psychoeducation components.8 dysfunction, and secondary outcomes of post- In contrast, in a treatment trial of former traumatic stress, traumatic grief, and anxiety Palestinian political prisoners, participants in an symptoms. Based on a preliminary qualitative individual counseling program that contained study20 conducted with a similar population, de- trauma-specific elements, such as desensitization, pressive symptoms were identified as the most stress inoculaton, coping skills, and emotion regula- important mental health problem affecting this tion, and that addressed broader social problems population, with anxiety, traumatic stress, and saw small but significantly greater improvements in traumatic grief as secondary outcomes. Heartland traumatic stress symptoms than waitlist controls.10 Alliance International (HAI), an international Female survivors of conflict and associated sexual NGO based in the United States with offices violence in Liberia participating in a supportive and programs in Iraq, developed this intervention counseling program that included sharing of past as part of a broader program of integrating

Global Health: Science and Practice 2016 | Volume 4 | Number 3 453 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

mental health into the health care system in Iraqi problem solving, as well as core tasks such as Kurdistan. medication management, providing psychoeducation, working in the community, and advocacy. INTERVENTION DESCRIPTION In addition to practice evidence derived from working with survivors of torture in the United Program Development States,23 trauma theory and conceptual frameworks From 2005 to 2007, HAI implemented an inte- considered important for contexts with historical grated mental health program into the primary and ongoing political violence were integrated health care system in Iraqi Kurdistan, including a throughout the curriculum. This included a phase- comprehensive mental health curriculum for the based orientation to work with survivors of trauma paraprofessional level, CMHWs. CMHWs were that emphasized the importance of the therapeutic recruited through a joint selection process by the relationship and clinical principles of safety and Department of Health in the Dohuk governorate, stability when working with survivors of trauma.24 the Health Staff Association of Kurdistan (a non- The curriculum used a multisystem or person-in- governmental professional group of health staff), environment approach25 to understanding problems and staff of HAI including a study coauthor (AA). resulting from violence and trauma exposure and a The main selection criteria were clinical staff from strengths-based orientation to working with cli- the local primary clinics who had time and ex- ents.21 All HAI training included a train-the-trainer pressed an interest in gaining skills in mental manual and a participant workbook. The basic health and psychosocial support and had experi- training, together with a series of advanced train- ence working in rural areas with people who had ings, in total took place over the course of 2 years experienced torture and trauma. The goal was to andincluded30days(240hours)ofin-person identify 1 male and 1 female health staff to be training and monthly field supervision. The community trained as CMHWs, although the lack of female The curriculum development team consisted of mental health staff in the health centers in Dohuk meant that U.S-based adult learning experts and mental workers recruited there were more male CMHWs. These staff, who health technical staff as well as Iraqi program for our study would become the CMHWs, included pharmacists, staff with diverse expertise in curriculum develop- included nurses, and physician assistants, and were perma- ment, trauma-focused mental health practice and pharmacists, nent employees of the Ministry of Health. None of Iraqi culture and society. The project used an nurses, and the CMHWs had any formal mental health iterative, participatory action model for curriculum physician training prior to the HAI project. development, which took several months to assistants without At the time of the program, there were limited complete and included (1) identifying learning mental health services in the country and those any prior formal needs in collaboration with Iraqi staff and that did exist were concentrated in the major mental health CMHWs; (2) gathering information via interviews urban areas, with a focus on medication treat- training. with Iraqi staff and CMHWs to map curriculum ment. The HAI program’s first step was establish- content; (3) drafting the curriculum; (4) testing ing a CMHW role within the health centers and the curriculum during pilot train-the-trainer ses- The project used providing them with foundational knowledge to sions; (5) gathering post-pilot evaluative informa- an iterative, deliver care and support. At the time of the tion to revise training materials; (6) implementing participatory current study in 2009, HAI had trained approxi- the revised training with CMHWs; and (7) action model to mately 50 CMHWs in Kurdistan region. ongoing evaluation and further refinement. This develop the process resulted in locally informed training supportive Training Curriculum materials and allowed for ongoing quality counseling The basic skills curriculum was developed as a improvement. U.S-educated, licensed clinical social curriculum. 2-week training program that emphasized a workers with expertise working cross-culturally social work model of helping and support.21 It in- and with torture survivor communities facilitated cluded information on mental health and illness22 the train-the-trainer program, while the HAI and the skills necessary to provide psychosocial program staff in Iraq, mainly physicians, facili- support to individuals, with a particular focus tated the CMHW trainings. on depression, anxiety, and post-traumatic stress. The curriculum emphasized basic knowledge and skills of a helping professional including the thera- Adaptation for the Trial peutic relationship, compassionate care, maintain- In preparation for this trial, the original ing confidentiality, active listening, empathy, and basic skills curriculum was adapted into a

Global Health: Science and Practice 2016 | Volume 4 | Number 3 454 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

time-limited trauma-informed support, skills, The CMHW agreed the participant no longer and psychoeducation intervention so that it required counseling as evaluated by a review couldbecomparedwith2othertrauma-focused of whether the participant was no longer manualized interventions that were selected for experiencing many symptoms. study. This intervention was designed to include The intervention Standards of professional practice were incor- 6–12 sessions for each client depending on porated into the original training as well as into was designed to particular client needs and to include techni- the refresher training prior to trial initiation. These include 6–12 ques and skills analogous to what is done at the included the need for completing accurate treat- sessions for each beginning of phase-oriented treatment for ment monitoring records, having regular super- client depending trauma with significant emphasis on the vision meetings, adherence to ethical standards on the client’s importance of the therapeutic relationship.24 (e.g., do no harm, maintain professional relations needs. This process of adaptation for the study was led and boundaries), and reporting problems to the by the HAI clinical director (BG), a clinical supervisor. The CMHWs were advised to monitor psychologist with extensive expertise in torture their own feelings in the counseling process, rehabilitation. Together with the Kurdish study follow effective self-care strategies, make use of psychiatrist in Dohuk (TM), who acted as professional consultation and other support, and the clinical supervisor throughout the study, maintain a working balance between their clinical they conducted a refresher training for the experience and their personal lives. These advise- 11 CMHWs who were part of the evaluation ments were made to try to avoid CMHWs develop- study in the Dohuk region. This training pre- ing secondary trauma, becoming overly involved in sented a much-shortened version of the original or distanced from the client’s torture experience HAI program that was specific to survivors of (enmeshment or detachment), or otherwise burn- torture and imprisonment. CMHWs were pre- ing out. sented with an array of 9 techniques to use in counseling clients; for each technique, the CMHWs were taught 4 to 6 activities (Table 1). Supervision and Monitoring Theseactivitiesweretobeusedaccordingtothe Fidelity to the treatment model was promoted by The community individual needs of each client. The refresher monthly on-site group supervision by a psychia- mental health training also emphasized core clinical skills of trist (TM) as well as weekly check-ins via mobile workers were empathic reflection, building trust, emotional phone. If CMHWs had questions during the supervised expression and regulation, and the conveying of week, they could contact TM directly via phone. monthly on-site hope and meaning. To monitor adherence to the counseling protocol and weekly via CMHWs were trained to organize interactions during the on-site meetings, TM reviewed clinical mobile phone. with clients into (1) a preparatory first session notes, which included how the CMHW responded that set the stage for the development of a to the client’s needs and checklists of the trusting relationship and engaged the client in different activities the CMHW could have pro- the work; (2) a series of 4 to 10 ‘‘response’’ vided. The client monitoring form also included a sessions in which difficulties related to the brief checklist of common mental health symp- principal concerns of PTSD, depression, anxiety, toms that was used to review client progress and traumatic grief, and impaired functioning were help the CMHW and supervisor decide, together assessed and strategies were taught to address with the client, when treatment would be them; and (3) a concluding session that focused completed. on exploring progress made in treatment, con- solidation of work and skills learned, and plan- METHODS ning for the future. The counseling process was expected to require 6–12 sessions depending on The study sample from Dohuk governorate was the presenting problems and progress of the originally planned to be part of a larger trial client. For counseling to be considered completed, evaluating 3 different mental health interven- 3 criteria must have been met: tions (including the one described in this article) across 3 different regions of Kurdistan, Iraq. The study sample size was calculated for all The participant attended at least 6 sessions. 3 intervention groups and control participants The participant expressed no longer feeling combined. However, prior to initiating the trials, the need to attend counseling. it became clear that there were substantial

Global Health: Science and Practice 2016 | Volume 4 | Number 3 455 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

TABLE 1. HAI Refresher Training Techniques and Activities for CMHWs

Techniques Activities

Psychoeducation Give clients, families, or communities information on psychological problems. Reduce stigma about problems and treatment. Teach how thoughts, behaviors, and feelings can influence each other positively. Explain how talk therapy can help. Treatment planning Make arrangements with the client to begin treatment (e.g., confidentiality). Agree on how to continue treatment (e.g., weekly sessions, involving family if needed). Explain the way treatment will end. Describe follow-up assistance if needed after sessions end. Empowerment Help clients develop skills and use positive actions and attitudes. Start with small changes and help them focus on better parts of life, not only problems. Grow from a view of themselves as dependent to better able to care for themselves. Reduce feelings of helplessness by being more active and involved with family and community. Motivation Encourage clients to come to treatment regularly and make recommended changes in their behavior and thinking. Normalize their problems. Emphasize the progress they are making. Use the treatment relationship for emotional support with empathic listening and reflective techniques. Crisis management Assess for suicide or self-injury. Use safety plan if needed. Be more directive if needed. Involve family or other resources if needed. Get more consultation and supervision if needed. Change the balance between strengths and supports vs. stresses to manage the crisis. Medication Explain how drug therapy can combine with talk therapy to help reduce negative management feelings and improve sleep and other problems. Advise against the use of alcohol or illegal drugs, which can worsen problems. Consult with the physician about a combined therapy plan. Monitor for side effects and encourage daily use for later improvement. Strength building Identify the skills clients already have. Remind them how they have solved problems before. Find new ways to feel better, like talking about what is inside. Express concern for the negative parts of the client’s life but focus more on the positive (e.g., love of God or their children). Emphasize client’s ways of taking care of themselves (e.g., time with friends).

Global Health: Science and Practice 2016 | Volume 4 | Number 3 456 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

Table 1 (continued).

Techniques Activities

Stress reduction Assess and encourage client’s interests in positive activities (e.g., praying, exercising). Teach relaxation techniques like deep breathing and focusing inside. Practice relaxation regularly in counseling and have clients use it at home daily. Help clients use relaxation techniques any time they are upset, worried, or cannot sleep. Advocacy Identify resources in the family or community that can be used for additional client support. Help the client get additional needed services (e.g., medical or legal assistance). Promote human rights with equal protection, respect, and benefits for everyone. Try to end domestic abuse or child abuse and gender-based violence. Connect with other government offices, community programs, and NGOs to increase public awareness about mental health problems and find solutions.

Abbreviations: CMHW, community mental health worker; HAI, Heartland Alliance International.

population-based differences across the regions, Presenting with significant depressive sym- with the population in the Dohuk governorate ptoms speaking a different Kurdish dialect, tending Not being currently psychotic or actively toward greater religiosity and political conserva- suicidal tiveness, and experiencing different types of trauma exposure given proximity to the Turkish Being mentally competent to give consent border compared with the other study governor- Experiencing torture was defined as person- ates. We thus made the decision to separate the ally experiencing or witnessing physical torture, trials into one trial focusing on 2 of the imprisonment, and/or military attacks. Signifi- interventions in the Erbil and Sulaimaniyah 26 cant depression was defined as reporting a total governorates and one trial in Dohuk concen- score of at least 20 on the 20-symptom, adapted trating only on the supportive counseling Hopkins Symptom Checklist (HSCL) depression program. scale and meeting both of the following specific criteria necessary for a DSM-IV (Diagnostic and Study Design Statistical Manual of Mental Disorders, 4th edition)27 This randomized controlled trial was conducted diagnosis of a major depressive episode: crying or through the primary health clinics staffed by the feeling depressed most or all of the time in the study CMHWs. Potential trial participants were last 2 weeks, and loss of interest in sex or loss of identified through referral by doctors in the interest in things generally (as evidenced by clinics and by referrals from former prisoner being unable to enjoy festivals and celebrations organizations. All adults (ages 18 or older) most or all of the time) in the last 2 weeks. referred to the CMHWs were administered the If eligible, CMHWs read the study consent form study instrument as part of the standard intake that explained if a person agreed to participate they process for the HAI mental health services. This would be randomly assigned to receive the counsel- screening interview also served as the baseline ing service beginning immediately or to wait assessment for eligible participants. Trial eligibil- approximately 3 to 5 months before receiving ity criteria comprised: treatment. Persons who did not meet criteria or Being 18 years or older refused to be in the study were still able to receive services provided by the CMHW but were not Residing in the Dohuk governorate included in the trial. Supervisors reviewed all Reporting experiences of torture completed assessments and contacted all eligible

Global Health: Science and Practice 2016 | Volume 4 | Number 3 457 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

clients who declined to join the study to confirm scale. In total, we included 70 items in the full their refusal to participate. mental health instrument (see Bolton et al., 201426 Random allocation of participants to inter- for a list of all mental health symptom items). For all vention or waitlist control was done at the items, participants rated how frequently they participant level. Study CMHWs were provided experienced each symptom in the prior 2 weeks with a set of prenumbered consent forms with using an ordinal scale of 0 (never) to 3 (always). the designation of intervention or waitlist control Functionality was defined based on a series of status on a piece of paper that was folded and tasks and activities, identified during a prior qual- stapled to the back. ID numbers were randomly itative study,32 regularly done by adults in Dohuk allocated to study condition by study author (JB) to take care of themselves and their families, and using Stata’s randomization function with a ratio to participate in their community. We developed of 3 intervention participants to 1 waitlist control separate measures for men and women. Respon- participant, based on the original study design dents were asked to report how much difficulty that included all of Kurdistan. they had completing each task and activity. The study was approved by the Johns Hopkins Dysfunction for each activity was rated on a Bloomberg School of Public Health Internal Review Likert scale ranging from 0 (no more difficulty Board and by the Ethical Committee of the College than most other men/women of the same age) to of Medicine at the University of Sulaimani in 4 (frequently unable). Kurdistan, Northern Iraq. We conducted a brief validation study of the mental health and measure of dysfunction, 26 Mental Health and Functioning Assessments which is described elsewhere in detail. Briefly, Cronbach’s alpha scores for mental health and A mental health assessment instrument was dysfunction scales ranged from 0.73 to 0.93, in- developed based on initial qualitative data col- dicating adequate to good internal reliability. lected in 2008 in the Suleimaniyah governorate of Pearson correlation coefficients for combined Kurdistan with women and men who had been inter-rater/test-retest reliability (repeat by differ- subjected to torture and/or prison during Saddam ent interviewer) ranged between 0.73 and 0.86. Hussein’s regime.20 Symptoms of mental distress Intraclass correlation coefficients (ICC) similarly described by study respondents included feeling ranged from 0.80 to 0.87. sad and depressed, ruminating on the past, lone- We compared mean scores for the mental health liness, being withdrawn, fear, anger, anxiety, measures between individuals identified as having insomnia, remembering past traumatic events, each syndrome with those identified as not having and avoiding reminders of the past. Based on the syndromes to evaluate whether our measures these results, we selected the following mental could adequately discriminate between them. Dis- health measures to be adapted for the trial: criminant validity was identified for all mental The Hopkins Symptom Checklist-25 (HSCL- health syndromes in men but only for PTSD in 2528,29 (a 25-item version of the HSCL) for women. Because of the generally good performance symptoms of depression and anxiety of the symptom and dysfunction scales on other The Harvard Trauma Questionnaire (HTQ)30 tests of validity and reliability for both sexes, we for symptoms of post-traumatic stress suspected that the poor validity among women was due to testing methods in which we relied on The Inventory of Traumatic Grief31 for symp- husbands to report the existence of problems among toms of traumatic grief their wives, which may have resulted in misidenti- We adapted and validated these instruments for fication due to husbands being less skilled at the local context using methods described else- assessing the occurrence of these problems among where.32 The adaptation process included adding 22 their wives. We therefore concluded that the locally relevant symptoms identified during the symptom measures were likely adequate for use previous qualitative study,20 including 19 that among both men and women in this population. described features of depression, anxiety, trauma, or traumatic grief but were not already represented on these measures. Of these items, 5 were added to Waitlist Control Condition the depression measure, 7 to the post-traumatic CMHWs contacted waitlist control participants stress measure, 1 to the traumatic grief measure, and monthly, usually by telephone, for a brief check if 3 to the assessment but not a particular symptom they were experiencing substantially greater distress

Global Health: Science and Practice 2016 | Volume 4 | Number 3 458 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

or had become a danger to themselves or others. program) in the original study design, but be- Control participants were also instructed to contact cause of the decision to evaluate the Dohuk study the CMWHs at any time during the study if their separately from the other 2 interventions, we symptoms worsened for assessment and referral if continued to enroll participants using the same necessary, including transportation to a psychiatrist allocation ratio until our control group reached or the Trauma Rehabilitation and Training Center in 50 participants to increase our statistical power. Suleimaniyah. A post-hoc power analysis comparing mean outcomes at follow-up with unequal sample sizes at Post-Intervention Assessment alpha of 0.05 resulted in 52% power for the depression symptoms outcome and 88% for the We aimed to readminister the study instrument dysfunction outcome. within 1 month of completing counseling for the intervention participants and the equivalent (between 3 to 5 months after baseline) for the Statistical Analysis waitlist controls. The majority (82%, n = 154) of For each study participant, average baseline and the follow-up interviews were implemented by follow-up syndrome-specific and dysfunction scale CMHWs who were blinded to the participant’s scores were generated. Our main intent-to-treat treatment status, whereas 18% (n = 34) were analyses used multilevel models with a robust implemented by CMHWs or study supervisors variance estimator and 2 random effects: participant who were unblinded. This latter group included and CMHW. Assessment of treatment effects was persons who dropped out of the trial but were based on comparing differences by study arm in the subsequently located by a member of the research change in average syndrome-specific and dysfunc- team who confirmed that they did not want to tion scale scores from baseline to follow-up (inter- continue or be seen again. Rather than risk losing action of 2 fixed effects: study arm and time). We them to follow-up, the research supervisors and controlled for age, sex, marital status (currently staff (instead of CMHWs) did these follow-up married vs. not married), and employment status interviews despite the lack of blinding. Analyses (any current work vs. not working) based on theory were done with and without the 34 participants and literature indicating these factors may be who were assessed unblinded to evaluate the potential confounders. We also included variables impact of the unblinded subjects. that differed between treatment and control at Trial recruitment ran from June 2009 through baseline or predicted change in outcome as covari- June 2010. Due to logistic challenges and difficulty ates: number of children, disability (yes or no), and keeping track of study participants (i.e., many often length of time between assessments. turned off their phones or left the area for work), Multiple imputation by chained equations was the average time between baseline and follow-up used to account for item-level missingness in scales assessments was 6.4 months for waitlist control for any participant as well as missing post-assessment participants (range, 3.1 to 10.7) and 5.9 months for scale scores, employment status, disability status, intervention participants (range, 2.8 to 13.1). and length between assessments for those lost to Although this difference was not statistically follow-up.33 Marital status, years of education com- significant we did control for length of time between pleted, and number of children were carried forward assessments in the analyses. for those lost to follow-up. Multiple imputation was also used to estimate missing scale scores for Sample Size 2 participants whose baseline questionnaires were We calculated the sample size required to detect a lost. For these 2 participants, demographic informa- 20% greater reduction in depression symptom tion was carried back from follow-up. All analyses 34 scores among intervention participants compared were conducted using Stata version 12.0. with controls, with 80% power and an alpha of 0.05. The choice of 20% represents our estimate of RESULTS a likely meaningful change. This yielded a sample size of 85 per arm. Estimating a dropout rate of Participant Flow Including Losses and 25%, we increased the recruitment target to 106 in Exclusions each arm. The 106 participants in the waitlist A total of 295 adults living in the Dohuk control condition were to be spread across the governorate seeking services from a study CMHW 3 intervention programs (i.e., about 26 per treatment were screened for eligibility. CMHWs found

Global Health: Science and Practice 2016 | Volume 4 | Number 3 459 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

82 participants to be ineligible, and 4 others re- activities (e.g., going to coffee, shopping, or on fused participation (Figure). A total of 209 men a picnic) and even accompanied them in some and women were randomized to either the cases. intervention (n = 159) or a waitlist control condi- = tion (n 50). Of the 159 allocated to the counsel- Baseline Characteristics ing intervention, 5 never initiated counseling. Of Table 2 presents the demographic characteristics these 5, 3 dropped out of the trial stating they of the supportive counseling and waitlist control desired financial assistance, 1 was assigned to a participants. The average study sample age was CMHW who quit due to increased administrative 40 years and ranged from 18 to 82. About a third responsibilities in the hospital, and 1 failed to of the sample was female, and about 20% self- initiate counseling for an unknown reason but reported disability. The majority of the sample remained under follow-up. was married, approximately half reported being During the course of the trial, 7 (14.0%) unemployed, and more than 40% reported no individuals were lost to follow-up in the control education. Demographic characteristics of the arm, and 10 (6.3%) individuals who initiated participants across the 2 arms were comparable therapy were lost to follow-up in the intervention with no differences reaching statistical signi- arm. In total, 188 individuals completed follow- ficance. up (90.0%). Individuals lost to follow-up were significantly more likely to be female (P = .04), self-employed (P = .007), and unmarried (P = .04). Trial Effects Estimates of treatment effects across the primary outcomes of depression and dysfunction are The intervention Treatment Completion and Content presented in Table 3. The intervention had a had a statistically Among those who initiated treatment (n = 154), statistically significant and moderate-sized effect signiﬁcant and the rate of completion was 95.5% (n = 147). on depression symptoms (Cohen’sd,0.57;P = .02) moderate-sized (Some participants completed treatment by defi- and dysfunction (Cohen’s d, 0.53; P = .03). Although effect on nition but did not complete a follow-up assess- the intervention also appeared to decrease symp- depression and ment.) The mean number of sessions attended toms of the secondary outcomes (post-traumatic dysfunction. was 11.29 (range, 7 to 12) among completers and stress, traumatic grief, and anxiety), these effects 1.86 (range, 0 to 3) among non-completers who were small. Of the secondary outcomes, the effect initiated treatment. Neither treatment comple- was statistically significant for anxiety (Cohen’sd, tion nor mean number of sessions attended 0.41; P = .01) and marginally significant for post- differed significantly by participant sex. Among traumatic stress (Cohen’sd,0.35;P = .07) and those who initiated treatment, a higher average traumatic grief (Cohen’s d, 0.26; P = .08). Exclusion baseline depression scale score (P = .04) and of baseline covariates resulted in slightly smaller higher average baseline post-traumatic stress effects across outcomes (-0.04 to -0.05 difference in symptoms (P = .03) were both related to treat- Cohen’s d) but did not meaningfully change results. ment completion. There was also variation across Analyses conducted removing the 34 participants CMHWs in treatment completion among their who were assessed unblinded to their treatment clients (P = .01); however, all but 3 CMHWs had status resulted in smaller effect sizes for depression greater than 90% completion rates among their (Cohen’s d, 0.45; P = .12), dysfunction (Cohen’sd, clients who initiated treatment. 0.47; P = .08), and anxiety (Cohen’s d, 0.36; P = .06) The community Based on CMHW monitoring forms and and larger effect sizes for trauma (Cohen’s d, 0.43; mental health supervisory reports, the activities most com- P = .11) and traumatic grief (Cohen’s d, 0.28; workers most monlyusedbyCMHWsincludedrelaxationand P = .11). commonly used psychoeducation on mental health symptoms, treatment, and prognosis. If a client was relaxation and DISCUSSION psychoeducation comfortable with their family having knowl- techniques with edge of their problems, CMHWs visited their We assessed the impact of a trauma-informed their clients. families to provide psychoeducation. In 5 in- support, skills, and psychoeducation intervention stances, CMHWs helped clients find employ- on depression symptoms and dysfunction scores ment. The study intervention emphasized the among survivors of torture and related traumatic importance of social support, and CMHWs com- experiences living in the Dohuk governorate of monly encouraged clients to engage in social Kurdistan. We found moderate effect sizes for the

Global Health: Science and Practice 2016 | Volume 4 | Number 3 460 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

FIGURE. Flow Chart of Study Participants

a 295 individuals were screened at baseline and 209 randomized. However, data at baseline were missing for 2 individuals randomized to the intervention and followed-up; thus, we had data for only 293 people screened and 207 randomized. b Of the 5 people allocated to the study intervention who did not receive it, 3 opted for ﬁnancial support, 1 was assigned to a counselor who quit, and the last person’s reasons were unknown.

study intervention on the primary outcomes, effects on the secondary outcomes of post- which is consistent with the 16 studies of mental traumatic stress, anxiety, and traumatic grief health treatments for survivors of torture and were smaller, with the study intervention having trauma that reported effect sizes included in the the greatest impact on anxiety and the smallest review by McFarlane and Kaplan.35 Treatment impact on traumatic grief.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 461 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

TABLE 2. Baseline Characteristics of Intent-to-Treat Sample, Dohuk Governorate, Kurdistan, June 2009–June 2010 (N = 207)

Counseling Intervention Waitlist Control (n = 157)a (n = 50)

Age, years, mean (SD) 40.30 (15.3) 40.76 (12.82) Female, No. (%) 54 (34%) 15 (30%) No. of children, mean (SD) 4.80 (4.09) 4.86 (3.91) Disabled, No. (%) 32 (20%) 9 (18%) Marital status Married, No. (%) 116 (74%) 43 (86%) Single/divorced/widowed, No. (%) 41 (26%) 7 (14%) Employment Not working, No. (%) 87 (55%) 26 (52%) Regular work, No. (%) 27 (17%) 11 (22%) Self-employed, No. (%) 23 (15%) 8 (16%) Irregular work, No. (%) 20 (13%) 5 (10%) Education None, No. (%) 68 (43%) 24 (48%) Primary, No. (%) 53 (34%) 14 (28%) Secondary, No. (%) 29 (18%) 11 (22%) Bachelors/institutional degree or certiﬁcate, No. (%) 7 (4%) 1 (2%)

Abbreviation: SD, standard deviation. a 159 were allocated to the counseling intervention, but 2 participants’ paperwork at baseline was lost.

These findings are consistent with what average improvement over time may be due to we would expect to see from a trauma-informed regression to a population mean. Improvement may approach that emphasizes building a sense of also be a result of participants experiencing some internal safety and regulation by coping with and relief in symptoms because an interviewer is taking managing reactions to symptoms. In a progressive the time to ask them about their problems. It is also treatment model, this type of work is character- possible that control participants accessed other ized as setting the stage for later, more in-depth supportive services in the community unrelated to narrative work that focuses on addressing under- the clinical services offered. Regardless, a study lying trauma experiences and associated PTSD without a control sample would have likely over- symptoms, which the HAI intervention did not estimated the intervention effectiveness. include.8,29,30 Given the nature of the intervention and the Similar to other randomized controlled trials of context of historical and ongoing trauma expo- psychological interventions in low- and middle- sure, the reductions in depression symptoms and income countries,7,8,26 we observed average symp- improvements in functioning seem particularly tom improvement among the control sample. Study notable. While depression manifests in culturally eligibility was based in part on having severe distinct ways, a core feature typically includes enough symptoms to warrant a mental health negative beliefs about oneself and one’s life. intervention. Given that mental health symptoms It seems plausible that building skills in symp- andaccompanyingdysfunctioncanvaryovertime, tom identification, stress reduction, and emotion

Global Health: Science and Practice 2016 | Volume 4 | Number 3 462 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

TABLE 3. Adjusted Treatment Effects on Primary and Secondary Study Outcomes,a Dohuk Governorate, Kurdistan, June 2009–June 2010 (N = 209)

Counseling Waitlist Control Adjusted Intervention (n = 159) (n = 50) Net Effect Effect Size Score (95% CI) Score (95% CI) Score (95% CI) Estimateb P Value

Primary Outcomes Depression Baseline 1.61 (1.51, 1.71) 1.59 (1.44, 1.74) Follow-up 0.78 (0.67, 0.89) 0.97 (0.75, 1.20) Pre-post change -0.83 (-0.98, -0.69) -0.62 (-0.86, -0.37) -0.22 (-0.39, -0.04) 0.57 .02 Functional impairment Baseline 1.92 (1.69, 2.15) 1.86 (1.56, 2.16) Follow-up 1.16 (0.95, 1.38) 1.49 (1.15, 1.83) Pre-post change -0.76 (-1.06, -0.45) -0.37 (-0.83, 0.09) -0.39 (-0.74, -0.03) 0.53 .03 Secondary Outcomes Post-traumatic stress Baseline 1.34 (1.22, 1.46) 1.35 (1.17, 1.52) Follow-up 0.73 (0.64, 0.83) 0.86 (0.69, 1.04) Pre-post change -0.61 (-0.74, -0.48) -0.48 (-0.68, -0.29) -0.13 (-0.27, 0.01) 0.35 .07 Anxiety Baseline 1.30 (1.17, 1.43) 1.25 (1.08, 1.41) Follow-up 0.66 (0.53, 0.80) 0.81 (0.59, 1.03) Pre-post change -0.64 (-0.83, -0.44) -0.44 (-0.67, -0.21) -0.19 (-0.35, -0.04) 0.41 .01 Traumatic grief Baseline 0.87 (0.74, 1.01) 0.86 (0.68, 1.03) Follow-up 0.38 (0.31, 0.44) 0.47 (0.32, 0.62) Pre-post change -0.50 (-0.59, -0.40) -0.38 (-0.50, -0.27) -0.11 (-0.24, 0.02) 0.26 .08

Abbreviation: CI, conﬁdence interval. a Model-estimated differences after adjusting for age, sex, employment status, time between assessments, number of children, and marital status in all models. All models include multiple imputation by chained equations for missing data and for missing outcomes due to loss to follow-up. Robust standard error estimators are used to account for clustering by counselor. b Measured using Cohen’s d statistic and pooled baseline variances.

regulation resulted in a greater sense of self- the Iraqi context, a strong client-provider relation- efficacy for participants and reductions in depres- ship is emphasized as a core therapeutic factor sion symptoms. The HAI intervention also across many different treatment approaches and emphasized the centrality of the therapeutic has been found to be an important element in alliance in trauma-informed work, one that is predicting positive client outcomes.31 While there is characterized by compassion, positive regard, and good conceptual rationale for these ideas, future fostering a sense of hope. Although not specific to studies looking at these particular intervention

Global Health: Science and Practice 2016 | Volume 4 | Number 3 463 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

elements—self-efficacy, therapeutic alliance—and as well as impacts on other problems that we did A strong client- client outcomes are needed to test these hypotheses. not assess. The moderate impacts, while significant, provider The results of this study are particularly rele- do not approach the effect sizes seen when relationship has vant as many organizations working in human- evidence-base psychotherapy interventions have been found to be itarian contexts are moving forward with the been implemented for the same outcomes with 2,32 an important development and implementation of what are similar populations in Iraq, suggesting that for ‘‘ element in commonly referred to as low-intensity interven- organizations that want to specifically target ’’ predicting positive tions, i.e., interventions delivered and/or supported common mental disorders, evidence-based treat- client outcomes. by community-based providers without formal mental ment can have more impact for similar levels of health training and provided with limited support training and supervision resources. by formal health care institutions. These interventions fit within the larger WHO model for global Limitations – mental health services the Mental Health Gap Several limitations must be acknowledged. As 36 Action Programme (mhGAP). Like the HAI part of the informed consent process, the waitlist intervention evaluated in this study, these low- control group was informed that they would intensity interventions tend to be based on the be offered treatment regardless of their scores on techniques used in cognitive behavioral therapy the follow-up interview to reduce the possibility (CBT) that support relieving distress and improv- that they might try to report severe symptoms. It 37 ing daily functioning. For example, the World is still possible, however, that controls may have Health Organization (WHO) has developed such an seen an incentive to report more distress at + intervention, Problem Management Plus (PM ), follow-up than intervention participants. In that, similar to the HAI program, is aimed at addition, as controls did not meet with CMHWs treating a wide range of presentations of distress. routinely, we are uncertain as to how much of the + The PM intervention integrates problem-solving intervention’s impact was due to counseling and behavioral techniques in a program that can content as opposed to the act of meeting weekly be implemented by community providers with with CMHWs. We also did not explore the cul- 38 ongoing supervision. Ongoing trials of these low- tural understanding of what counseling might intensity, supportive counseling programs will be mean in this community and how the experience important to understand their effectiveness in of counseling may differ based on participant sex reducing the burden of mental health problems or other cultural factors. Another limitation is among trauma-affected populations. that the participants were chosen on the basis of Our choice to use depression symptoms as significant levels of depressive symptoms. The criteria for entry into this trial was based on our pre- HAI program was originally designed to be a vious qualitative study that found that depression- referral program for clients with a wider range of like symptoms were described most frequently in presenting problems, so the results may not discussions of mental distress with survivors of represent the typical target population for this 20 torture in Kurdistan. This is in contrast to other program. Our use of unblinded assessors for ap- mental health intervention studies for torture proximately 15% of the post-intervention assess- survivors that have generally included only those ments resulted in our interviewing a larger 35 suffering from PTSD-related symptoms. The choice proportion of the study participants but did to use depression symptoms for screening was to introduce some bias into our findings; while the identify those survivors who expressed a high degree general conclusions were not affected, the sig- of distress with respect to a group of symptoms that nificance of the findings were somewhat affected. were clearly important locally (depression). However, it is not clear if the variation in signi- While depression symptoms were the main ficance was due to unblinding or due to the change entry criteria, the study intervention was originally in sample size when the unblinded sample was designed to assist survivors with a wide range of excluded. Finally, we also are unable to assess if the mental and psychosocial issues. Therefore, our effects seen following counseling were sustained, finding of small to moderate effects across multiple due to a lack of long-term follow-up. mental health outcomes suggests that the intervention produced a wide-ranging, although limited, CONCLUSION effect on these specific outcomes. The moderate reduction in dysfunction may reflect the cumulative The study intervention was developed as a impact of reduction in the symptoms we measured trauma-informed treatment approach that could

Global Health: Science and Practice 2016 | Volume 4 | Number 3 464 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

be provided by CMHWs for torture and trauma 2. Lien L, Thapa SB, Rove JA, Kumar B, Hauff E. Premigration survivors living in rural areas of Northern Iraq. traumatic events and psychological distress among five immigrant This intervention was well defined, based on groups: results from a cross-sectional study in Oslo, Norway. Int J Ment Health. 2010;39(3):3–19. CrossRef. extensive practice experience with torture survi- 3. Park CL, Pless Kaiser A, Spiro A III, King DW, King LA. Does vors and trauma-affected communities, and the wartime captivity affect late-life mental health? a study of CMHWs were provided with training on the Vietnam-era repatriated prisoners of war. Res Hum Dev. 2012; content of the program as well as when and 9(3):191–209. CrossRef. Medline how to implement the different activities and 4. Tol WA, Barbui C, Galappatti A, Silove D, Betancourt TS, therapeutic skills. The intervention was adapted Souza R, et al. Mental health and psychosocial support in humanitarian settings: linking practice and research. Lancet. by a clinician with experience providing trauma- 2011;378(9802):1581–1591. CrossRef. Medline focused care and the CMHWs received regular 5. Inter-Agency Standing Committee (IASC). IASC guidelines on clinical supervision from a local psychiatrist. mental health and psychosocial support in emergency settings. Well-supervised The results suggest that this type of well- Geneva: IASC; 2007. Available from: http://www.who.int/ supportive mental_health/emergencies/guidelines_iasc_mental_health_ supervised, trauma-informed intervention can psychosocial_june_2007.pdf counseling can provide moderate improvement for depression 6. Ayoughi S, Missmahl I, Weierstahl R, Elbert T. Provision of mental provide moderate and anxiety symptoms and functional impair- health services in resource-poor settings: a randomised trial improvement for ments in torture- and trauma-affected commu- comparing counselling with routine medical treatment in depression and — North Afghanistan (Mazar-e-Sharif). BMC Psychiatry. nities. The current WHO model Mental Health anxiety symptoms 36— 2012;12:14. CrossRef. Medline Gap Action Programme (mhGAP) requires and functional that moderate to severe cases of mental health 7. Bass J, Poudyal B, Tol W, Murray L, Nadison M, Bolton P. A controlled trial of problem-solving counseling for war-affected impairments in problems be treated via referral to specialist adults in Aceh, Indonesia. Soc Psychiatry Psychiatr Epidemiol. trauma services. Under such a model, supportive services 2012;47(2):279–291. CrossRef. Medline communities. that have a moderate effect and are widely 8. Neuner F, Schauer M, Klaschik C, Karunakara U, Elbert T. accessible at the community level, such as the A comparison of narrative exposure therapy, supportive counseling, and psychoeducation for treating posttraumatic stress one studied here, can play an important role as disorder in an African refugee settlement. J Consult Clin Psychol. the first line of treatment for less severe cases. 2004;72(4):579–587. CrossRef. Medline However, it is not clear at this time how good 9. Psychosocial Centre. Lay counselling: a trainer’smanual. access to specialist services in low- and middle- Copenhagen (Denmark): International Federation of Red Cross income countries is to be achieved, leaving and Red Crescent Societies; [date unknown]. Available from: http:// community-level services as the only accessible pscentre.org/wp-content/uploads/Lay-counselling_EN.pdf option. Service providers will need to decide 10. Salo J, Punama¨ki RL, Qouta S, El Sarraj E. Individual and group treatment and self and other representations predicting whether this type of supportive intervention is posttraumatic recovery among former political prisoners. sufficient for communities with a high burden of Traumatology. 2008;14(2):45–61. CrossRef moderate to severe cases of common mental 11. Scholte WF, Verduin F, Kamperman AM, Rutayisire T, disorders, or whether more effective community- Zwinderman AH, Stronks K. The effect on mental health of a large scale psychosocial intervention for survivors of mass violence: based interventions will also be needed. a quasi-experimental study in Rwanda. PLoS One. 2011;6(8): e21819. CrossRef. Medline Acknowledgments: This study was solely funded by the USAID Victims 12. Tol WA, Komproe IH, Jordans MJD, Thapa SB, Sharma B, of Torture Fund (VOT) under grant #101978. USAID/VOT was not De Jong JTVM. Brief multi-disciplinary treatment for torture involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, survivors in Nepal: a naturalistic comparative study. Int J Soc review, or approval of the manuscript. We particularly thank USAID/ Psychiatry. 2009;55(1):39–56. CrossRef. Medline VOT for their commitment to evidence-based programming as the 13. Yeomans PD, Forman EM, Herbert JD, Yuen E. A randomized basis for effective services. Dr. Murray was supported in part by trial of a reconciliation workshop with and without PTSD an NIMH Training grant for Global Mental Health (T32MH103210). psychoeducation in Burundian sample. J Trauma Stress. 2010; We also wish to thank Heartland Alliance, the Kurdistan Ministry of Health, and the Dohuk Mental Health Center for their essential role in 23(3):305–312. CrossRef. Medline the training and implementation of the intervention. 14. Lekskes J, van Hooren S, de Beus J. Appraisal of psychosocial interventions in Liberia. Intervention. 2007;5(1):18–26. Competing Interests: None declared. Available from: http://www.interventionjournal.com/sites/ default/files/de%20beus.pdf References 15. Weiss WM, Ugueto AM, Mahmooth Z, Murray LK, Hall BJ, Nadison M, et al. Mental health interventions and priorities for 1. Steel Z, Chey T, Silove D, Marnane C, Bryant RA, van Ommeren research for adult survivors of torture and systematic violence: M. Association of torture and other potentially traumatic events a review of the literature. Torture. 2016;26:17–44. Available with mental health outcomes among populations exposed to mass from: http://www.irct.org/Admin/Public/DWSDownload.aspx? conflict and displacement: a systematic review and meta-analysis. File = %2FFiles%2FFiler%2FTortureJournal%2F26_01_2016% JAMA. 2009;302(5):537–549. CrossRef. Medline 2F1018_8185_2016-1_17-44.pdf

Global Health: Science and Practice 2016 | Volume 4 | Number 3 465 RCT of Counseling Intervention in Northern Iraq www.ghspjournal.org

16. Watch HR. Genocide in Iraq: the Anfal campaign against the a four-week period. Br J Psychiatry. 1984;144(4):395–399. Kurds. New York: Human Rights Watch; 1993. Available from: CrossRef. Medline http://www.refworld.org/docid/47fdfb1d0.html 30. Mollica RF, Caspi-Yavin Y, Bollini P, Truong T, Tor S, Lavelle J. 17. Hamsici M. Is Turkey’s peace process with the Kurds collapsing? The Harvard Trauma Questionnaire. Validating a cross-cultural BBC [Internet]. 2013 Sep 9 [cited 2013 Sep 27]. Available from: instrument for measuring torture, trauma, and posttraumatic http://www.bbc.co.uk/news/world-europe-24014555 stress disorder in Indochinese refugees. J Nerv Ment Dis. 18. Spolar C. Iraq’s Kurds live in cross hairs. Chicago Tribune 1992;180(2):111–116. CrossRef. Medline [Internet]. 2007 Oct 29 [cited 2013 Sep 27]. Available from: 31. Prigerson HG, Maciejewski PK, Reynolds CF III, Bierhals AJ, http://articles.chicagotribune.com/2007-10-29/news/ Newsom JT, Fasiczka A, et al. Inventory of complicated grief: 0710280197_1_pkk-ambush-kurdish-iraq-border a scale to measure maladaptive symptoms of loss. Psychiatry Res. 19. Borger J. Iraq humanitarian crisis has reached highest level, 1995;59(1-2):65–79. CrossRef. Medline UN aid ofﬁcials warn. The Gaurdian [Internet]. 2014 Aug 14 32. Johns Hopkins Bloomberg School of Public Health, Center for [cited 2014 Oct 22]. Available from: http://www.theguardian. Refugee and Disaster Response, Applied Mental Health Research com/global-development/2014/aug/14/iraq-humanitarian- Group. Design, implementation monitoring, and evaluation of crisis-highest-level-un-aid mental health and psychosocial assistance programs for trauma 20. Bolton P, Michalopoulos L, Ahmed AMA, Murray LK, Bass J. survivors in low resource countries: a user’s manual for The mental health and psychosocial problems of survivors of researchers and program implementers (Adult version). Module torture and genocide in Kurdistan, Northern Iraq: a brief 2: developing quantitative tools. Baltimore (MD): Johns Hopkins qualitative study. Torture. 2013;23(1):1–14. Medline Bloomberg School of Public Health; 2013. Available from: http://www.jhsph.edu/research/centers-and-institutes/center- 21. Hepworth DH, Rooney RH, Rooney GD, Strom-Gottfried K, for-refugee-and-disaster-response/response_service/AMHR/ Larsen JA. Direct social work practice: theory and skills. dime/VOT_DIME_MODULE2_FINAL.pdf 8th edition. Belmont (CA): Brooks Cole; 2009. 33. Azur MJ, Stuart EA, Frangakis C, Leaf PJ. Multiple imputation by 22. Pratt CW, Gill KJ, Barrett NM, Roberts MM. Psychiatric chained equations: what is it and how does it work? Int J Methods rehabilitation. Academic Press; 2006. Psychiatr Res. 2011;20(1):40–49. CrossRef. Medline 23. Marsh JC. Theory-driver versus theory-free research in empirical 34. StataCorp. Stata Statistical Software: Release 12. College Station social work practice. In: Briggs HE, Rzepnicki TL, editors. Using (TX): StataCorp LP; 2011. evidence in social work practice: behavioral perspectives. Chicago: Lyceum Books; 2004. p. 20–35. 35. McFarlane CA, Kaplan I. Evidence-based psychological interventions foradultsurvivorsoftortureandtrauma:a30-yearreview.Transcult 24. Herman JL. Trauma and recovery. Basic Books; 1997. Psychiatry. 2012;49(3-4):539–567. CrossRef. Medline 25. Bronfenbrenner U. The ecology of human development: experiments 36. World Health Organization (WHO). mhGAP intervention guide by nature and design. Harvard University Press; 1979. for mental, neurological, and substance use disorders in non- 26. Bolton P, Bass JK, Zangana GAS, Kamal T, Murray SM, specialized health settings. Geneva: WHO; 2010. Available Kaysen D, et al. A randomized controlled trial of mental from: http://www.who.int/mental_health/publications/ health interventions for survivors of systematic violence in mhGAP_intervention_guide/en/ Kurdistan, Northern Iraq. BMC Psychiatry. 2014;14:360. 37. National Collaborating Centre for Mental Health (UK). Low- CrossRef. Medline intensity psychological interventions. In: Generalised anxiety 27. American Psychiatric Association. Diagnostic and statistical disorder in adults: management in primary, secondary and manual of mental disorders. Fourth edition. Arlington (VA): community care. Leicester (UK): British Psychological Society; 2011. American Psychiatric Association; 2000. Available from: http://www.ncbi.nlm.nih.gov/books/NBK83456/ 28. Hesbacher PT, Rickels K, Morris RJ, Newman H, Rosenfeld H. 38. Dawson KS, Bryant RA, Harper M, Kuowei Tay A, Rahman A, Psychiatric illness in family practice. J Clin Psychiatry. 1980; Schafer A, et al. Problem Management Plus (PM+): a WHO 41(1):6–10. Medline transdiagnostic psychological intervention for common mental 29. Winokur A, Winokur DF, Rickels K, Cox DS. Symptoms of health problems. World Psychiatry. 2015;14(3):354–357. emotional distress in a family planning service: stability over CrossRef. Medline

Peer Reviewed

Received: 2016 Jan 28; Accepted: 2016 May 27; First Published Online: 2016 Sep 8

Cite this article as: Bass J, Murray SM, Mohammed TA, Bunn M, Gorman W, Ahmed AMA, et al. A randomized controlled trial of a trauma- informed support, skills, and psychoeducation intervention for survivors of torture and related trauma in Kurdistan, Northern Iraq. Glob Health Sci Pract. 2016;4(3):452-466. http:/dx.doi.org/10.9745/GHSP-D-16-00017.

& Bass et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi.org/ 10.9745/GHSP-D-16-00017.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 466 ORIGINAL ARTICLE

Progress in Harmonizing Tiered HIV Laboratory Systems: Challenges and Opportunities in 8 African Countries

Jason Williams,a Farouk Umaru,b Dianna Edgil,c Joel Kuritskyc

ABSTRACT In 2014, the Joint United Nations Programme on HIV/AIDS released its 90-90-90 targets, which make laboratory diagnostics a cornerstone for measuring efforts toward the epidemic control of HIV. A data-driven laboratory harmonization and standardization approach is one way to create efficiencies and ensure optimal laboratory procurements. Following the 2008 ‘‘Maputo Declaration on Strengthening of Laboratory Systems’’—a call for government leadership in harmonizing tiered laboratory networks and standardizing testing services—several national ministries of health requested that the United States Government and in-country partners help implement the recommendations by facilitating laboratory harmonization and standardization workshops, with a primary focus on improving HIV laboratory service delivery. Between 2007 and 2015, harmonization and standardization workshops were held in 8 African countries. This article reviews progress in the harmonization of laboratory systems in these 8 countries. We examined agreed-upon instrument lists established at the workshops and compared them against instrument data from laboratory quantification exercises over time. We used this measure as an indicator of adherence to national procurement policies. We found high levels of diversity across laboratories’ diagnostic instruments, equipment, and services. This diversity contributes to different levels of compliance with expected service delivery standards. We believe the following challenges to be the most important to address: (1) lack of adherence to procurement policies, (2) absence or limited influence of a coordinating body to fully implement harmonization proposals, and (3) misalignment of laboratory policies with minimum packages of care and with national HIV care and treatment guidelines. Overall, the effort to implement the recommendations from the Maputo Declaration has had mixed success and is a work in progress. Program managers should continue efforts to advance the principles outlined in the Maputo Declaration. Quantification exercises are an important method of identifying instrument diversity, and provide an opportunity to measure efforts toward standardization.

INTRODUCTION people living with HIV/AIDS, placing 90% of those with HIV/AIDS on antiretroviral therapy, and ensuring that In 2014, the Joint United Nations Programme on 90% of those on antiretroviral therapy are virally sup- HIV/AIDS (UNAIDS) released targets of testing 90% of pressed.1 These 90-90-90 targets made laboratory diagnostics a cornerstone for national efforts toward the a Partnership for Supply Chain Management (PFSCM), Supply Chain Management System (SCMS), Arlington, VA, USA. Now with the U.S. Agency epidemic control of HIV. A data-driven laboratory harmo- for International Development (USAID), Bureau for Global Health, Ofﬁce of nization and standardization approach is one way to create HIV/AIDS, Supply Chain for Health, Washington, DC, USA. efficiencies and ensure optimal laboratory procurements. b PFSCM, SCMS, Arlington, VA, USA. In 2008, a consensus meeting on clinical laboratory c USAID, Bureau for Global Health, Ofﬁce of HIV/AIDS, Supply Chain for Health, Washington, DC, USA. testing, harmonization, and standardization was held in Correspondence to Jason Williams ([email protected]). Maputo, Mozambique. Representatives of governments,

Global Health: Science and Practice 2016 | Volume 4 | Number 3 467 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

multilateral agencies, development partners, pro- technical and operational guidance for strategic fessional associations, and academic institutions planning for responsive laboratory development. sought to address overarching laboratory challenges Participants recognized a need to address the that had limited the scale-up of services for tuber- challenges limiting the uptake of diagnostic serv- culosis, malaria, and HIV diagnosis and care.2 ices in resource-limited settings. These challenges The meeting was organized by the World Health included lack of or insufficient leadership and Organization | Regional Office for Africa (WHO- advocacy, human resources, national laboratory AFRO) and the U.S. President’s Emergency Plan policies, strategic and financial planning, physical for AIDS Relief (PEPFAR), with the support of infrastructure, supply chain management, and the World Bank, The Global Fund to Fight AIDS, quality management systems.2 A tiered Tuberculosis and Malaria, the Bill & Melinda Gates To address these issues, participants recom- laboratory Foundation, the Clinton Health Access Initiative mended that countries adopt a tiered laboratory system, within (CHAI), and the Partnership for Supply Chain system strategy within a harmonized network. a harmonized Management (PFSCM). The outcome of this meet- A tiered laboratory system features stratified network, is critical ing was the ‘‘Maputo Declaration on Strengthening levels of laboratories (national, central/regional, ’’ to strengthening of Laboratory Systems : a call to governments to provincial, district, and health center) based upon public health take leadership in harmonizing tiered laboratory agreed testing services, with each level offering 2 laboratory networks and standardizing testing services. increased technical testing complexity and capa- — services. Those attending the meeting 120 experts city (Figure 1). This tiered laboratory scheme is and policy makers from 33 countries, including critical to strengthening public health laboratory representatives from 28 sub-Saharan African services and informing effective national labora- countries—were invited to reach a consensus on tory policy.3,4

FIGURE 1. Example of a Hierarchical Tiered Laboratory System

Adapted from the 2008 Maputo Declaration.2

Global Health: Science and Practice 2016 | Volume 4 | Number 3 468 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

In addition to the call for government leader- of addressing access to critical HIV-related labo- ship in the Maputo Declaration, the meeting ratory services. PEPFAR, The Global Fund, CHAI, resulted in technical and operational recommen- and others have led efforts to expand coverage of dations to guide the harmonization and standard- diagnostic instrumentation as part of the global ization of clinical laboratory testing in developing response to the HIV epidemic. Between 2007 and countries. Key recommendations from group break- 2016, the United States Agency for International out sessions included the following: Development's (USAID's) primary PEPFAR pro- The 2008 Maputo curement mechanism was PFSCM's Supply Chain Prioritize laboratory system coordination by Declaration Managment System. USAID's financial contribu- developing national laboratory policies, estab- recommended tion through this mechanism toward instrument lishing departments of laboratory systems prioritizing procurements and laboratory commodity require- within ministries of health, and calling upon coordination ments increased from US$33,759,096 (2008) to donors and partners to support national gov- of national $82,152,562 (2015) following the Maputo Decla- ernments in this effort. laboratory ration, for a total contribution of $511,475,320 systems. Define and establish the minimum test offer- across 43 countries.7 Countries have introduced ings required at each level of an integrated, hundreds of diagnostic instruments to reach patients tiered laboratory network, as well as the asso- within their laboratory networks, as well as at the ciated diagnostic instruments, equipment, health center level with the introduction of point- and human resources required to provide of-care (POC) instrumentation. such services. Now 8 years after Maputo, we review in this Prioritize supply chain systems and mainte- article how this financial and technical support has nance and service contracts for laboratory- enhanced efforts to implement harmonization based equipment at all levels of the laboratory strategies as part of scale-up efforts. The terms network. ‘‘harmonization’’ and ‘‘standardization’’ are often used interchangeably among laboratory practi- Following the 2014 Ebola outbreak in West- tioners and policy makers. Here we define ‘‘labora- ern Africa, a follow-on harmonization meeting tory harmonization’’ as a process of coordinating Laboratory was held in Freetown, Sierra Leone, in October host country governments and stakeholders in 2015. This meeting extended the call for interna- harmonization the procurement and placement of laboratory tional and local laboratory partners to increase is the process products within a defined tiered laboratory net- capacity and further emphasized the need to of coordinating work. This process is informed through consulta- develop tiered laboratory networks. The Freetown governments and tion with key stakeholders, such as physicians, meeting brought together the African Society for stakeholders in program leads, laboratory professionals, and pro- Laboratory Medicine, WHO-AFRO, and ministry a deﬁned tiered curement officers to develop technical policies. of health officials from more than 20 countries laboratory We define ‘‘standardization’’ as the process of in Africa. The meeting resulted in the ‘‘Freetown network. implementing and adhering to the established Declaration on Developing Resilient Laboratory technical policies. Standardization Networks for the Global Health Security Agenda in Africa,’’ which announced the need to effectively Harmonization and standardization efforts is the process of integrate tiered laboratory networks into disease offer considerable benefits. In South Africa, an implementing surveillance and public health institutes.5 The dec- integratedandstandardizedtieredservicedeliv- and adhering laration also emphasized the need to regularly ery model for CD4 (cluster of differentiation 4) to established measure progress with a standardized scorecard.6 testing could improve turnaround times by en- technical policies. The recent Ebola outbreak clearly demonstrates the suring appropriate placement and integration critical need to reduce vulnerabilities in health care of POC technologies within the conventional facilities and the laboratory system interface. tiered laboratory structure. These efforts demonstrated a reduction of R125 million (US$8.8 million) in HIV/AIDS program costs annually.8 ESTABLISHING A STRATEGY FOR Harmonization and standardization also offer LABORATORY HARMONIZATION AND the following broader benefits to laboratory 9 STANDARDIZATION service delivery : From 2008 to 2015, expenditures for laboratory Establishing minimum diagnostic test offer- instruments and commodities have increased ings and standardized testing methods within substantially among all countries, with the aim the tiered health network.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 469 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

Laboratory Reducing variation in laboratory products offerings, diagnostic instruments, and ancillary harmonization across different facilities, thereby improving equipment, as well as human resource templates, and commodity logistic systems, standardized qual- that countries can use as a starting point to guide 11 standardization ity control practices, and quality assurances. their laboratory harmonization efforts. These offer many Simplifying the identification and quantifica- lists were generalized to better serve multicountry beneﬁts, including tion of laboratory-based products. planning efforts, with detailed recommendations relevant to workshop participants. reduced costs Training laboratory staff more efficiently. and improved We view facilitation of harmonization and Improving coordination in laboratory instru- turnaround times. standardization workshops as a 2-step process. ment procurement, maintenance, and place- Phase 1, at the start, is for policy stakeholders, ment practices. implementers, clinicians, and key program, pro- Over the past 9 years, PFSCM’s Supply curement, and laboratory staff to define what Chain Management System, a project funded testing services are required within the health by PEPFAR and administered by USAID, facili- system and at what tier of the laboratory system tated laboratory harmonization and standardiza- they should be offered (Figure 2). tion workshops in 7 African countries at the In Phase 2, laboratory experts establish the request of the respective ministries of health. appropriate diagnostic methods to be used for testing services at each tier. The experts then Facilitated The USAID | DELIVER PROJECT and U.S. Centers develop a proposed harmonized list of diagnostic laboratory for Disease Control and Prevention (CDC) with instruments by tier, the necessary ancillary equip- harmonization PEPFAR provided direct assistance for an eighth ment, and the staffing required for the defined and workshop. These workshops were held in a mix of testing menu. These lists are then translated into standardization PEPFAR-supported sub-Saharan African countries (Eastern [3], Western [2], and Southern [3] African a national harmonization and standardization workshops were policy for implementation. The instrument har- held in 8 sub- countries). In general, requests for workshops were initiated to address HIV supply chain challenges monization approach is informed by existing Saharan African coverage of diagnostic instruments and the de- countries. (e.g., quantification, procurement, commodity diversity, and logistics) as well as suboptimal instrument gree of instrument diversity. Standardized instru- placement and high levels of instrument diversity ment lists should not be limited to one particular related to increasing numbers of donated instru- brand, but should include several brands to ments for rapid scale-up of HIV programs.10 disperse risk across diagnostic specialties and Each harmonization and standardization work- eliminate the potential for monopolization. shop had these overall objectives: METHODOLOGY FOR REVIEWING 1. Arrive at consensus on the methodology for PROGRESS IN HARMONIZATION AND harmonization and standardization. STANDARDIZATION 2. Establish national minimum test offerings and methodologies to be employed at each Past evaluations associated with implementa- laboratory tier for each test. tion of the Maputo Declaration have been limited. A review of previous work on labora- 3. Derive an evidence-based list of harmonized tory harmonization implementation focused on diagnostic instruments to support the required selection of the most appropriate tests and laboratory services. equipment types within the clinical cascade, as 4. Establish the minimum ancillary equipment well as on how tiered networks are defined.12 We sought to requirements at each tier. Other evaluations have reviewed published measure progress 5. Define the staffing complement required at reports, interviewed donors, and assessed coor- in harmonization each tier to support the recommended labora- dination efforts, with implementation of na- and tory services. tional laboratory plans found to be inconsistent standardization and frequently problematic.13 Past evaluations 6. Determine a strategic implementation plan, over time in have not targeted instrument brand diversity as a with defined roles and responsibilities. 8 countries by measure of adherence to standardized procure- analyzing annual A technical consultation document released ment policies. HIV laboratory following the 2008 Maputo meeting was used as Recognizing these limitations, we sought to quantiﬁcation a reference standard to initiate workshops. This measure implementation progress over time in data. document details a notional list of tiered test the 8 countries in which harmonization and

Global Health: Science and Practice 2016 | Volume 4 | Number 3 470 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

FIGURE 2. An Evidence-Based 2-Phase Approach to Developing a Harmonization and Standardization Proposal

standardization workshops were held. We did and standardization instrument policies in the this by analyzing available annual HIV laboratory countries where we held workshops. quantification data. These data include instru- All quantification data used in this compar- ment types and brands as a component of com- ison were collected through site visits, imple- modity forecasting over time. We organized menting partner data collection efforts, national standardized data import templates from ForLab equipment inventory lists, and commodity distri- (http://www.forlabtool.com), which is a multi- bution data from national logistics systems. The method laboratory forecasting tool, developed in final instrumentation network was validated partnership with USAID and CHAI. Laboratory in coordination with each country’snational instrument data were extracted by country for laboratory leadership, as well as by PEPFAR multiple forecasting periods. These data would implementing partners and U.S. Government mis- help measure adherence to instrument procure- sions (USAID and CDC), before we initiated ment practices against established harmonization the national forecasting exercises within ForLab.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 471 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

FIGURE 3. Instrument Counts in 8 African Countries, by Diagnostic Area

CD4 Chemistry Hematology Molecular

800 744

700 664

580 600 562 567 566

514 502 500 465 447

400 366 317 313 295 288

Instrtument Counts 300 264257

200 175 147 117 88 100 67 43 55 30 18 22 22 3 8 11 5 0 Country A Country B Country C Country D Country E Country F Country G Country H 2014 2015 2015 2014 2014 2015 2015 2015

We performed additional validation at the con- FINDINGS clusion of each national laboratory quantification exercise for commodity budgeting and procure- Instrument Counts and Increased Capacity ment purposes. Additionally, we assessed initial Figure 3 provides a summary of recent instru- harmonization and standardization proposals for ment counts by country. These numbers were potential instrument reductions by comparing extracted from national HIV laboratory forecast- existing diagnostic instrument variety against ing exercises conducted in 2014 and 2015. proposals that were developed at the time of Figure 4 represents the percentage growth by each harmonization and standardization work- diagnostic area over time in the 3 countries where shop (where data were available). When our consecutivedatapointswereavailable(CountryB, harmonization workshops were held, attendees Country E, and Country F). These 3 countries In workshops, identified recurring challenges to implementing have had marked increases in instrument counts: attendees harmonization proposals. These challenges were Country B has increased CD4 instrumentation by identiﬁed reviewed to identify obstacles to address as part of 155% (from 288 to 447) since 2012 by introducing challenges to implementing harmonization and standardiza- the Becton Dickinson FACSPresto in 2015 to address when tion efforts. replace aged FACSCounts machines and to implementing The intent of this analysis is (1) to illustrate expand CD4 testing to lower-level health facil- harmonization the efforts made to conduct harmonization and ities. From 2011 to 2014, Country E’s chemistry and standardization workshops to influence labora- and hematology coverage increased more than standardization tory development; (2) to determine how well 450% (from 124 to 567, chemistry, and from 111 efforts. these countries have done; and (3) to describe to 502, hematology) due to program scale-up. what potential underlying challenges must be And Country F reached an 820% growth in overcome to advance laboratory harmonization CD4 instrumentation (from 81 to 664) since and standardization efforts. 2009, primarily due to national deployment of

Global Health: Science and Practice 2016 | Volume 4 | Number 3 472 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

FIGURE 4. Growth in Instrument Counts in 3 African Countries, by Diagnostic Area, 2009–2015

700 CD4 Chemistry Hematology PCR (VL/EID) 664

600 567 566

502 500 447

400 366

300 288 264

Instrument Counts Instrument 257

207 200

150 144 124 124 111 100 81 67 53 18 7 2 3 7 8 0 Country B Country B Country E Country E Country F Country F 2012 2015 2011 2014 2009 2015

Abbreviations: EID, early infant diagnosis; PCR, polymerase chain reaction; VL, viral load.

Alere Pima CD4 testing machines as a POC solu- contribute to high instrument counts, but unlike tion to address CD4 sample referral challenges. CD4 instrumentation, these devices also contribute to high levels of instrument diversity, with ministries, donors, implementing partners, and Instrument Brand Diversity other stakeholders procuring many brands. High The highest levels of instrument diversity, in brand diversity further adds to higher levels manufacturer brand or unique diagnostic instru- of unique commodity types. For example, a CD4 A CD4 test run on mentation types, were found in chemistry and test run on a FACSCount requires a minimum of a FACSCount hematology (Figure 5). Chemistry and hematol- 6 different items: CD4 reagents, a control kit, requires a ogy instruments are used in general care and clean solution, rinse solution, FACSFlow sheath minimum of 6 clinical patient management services, but also fluid, and thermal paper. Thus, if a country has different items; play a key role in monitoring those on lifesaving 5 different types of CD4 instruments, more than thus, if a country HIV treatment. It should be noted that CD4 30 different commodities may be required for has 5 types of CD4 monitoring and molecular diagnostic instrumen- CD4 testing alone. instruments, more tation are predominantly procured through donor Many countries are using multiple open than 30 different mechanisms. These procurements are influenced systems (e.g., systems in which reagents and commodities may by the WHO prequalification process for introdu- consumables are nonproprietary) for chemistry be required for cing new diagnostic technology and therefore instrumentation, which helps reduce commodity CD4 testing alone. appear more harmonized due to fewer choices of variation by allowing for sharing of reagents and approved brands.14,15 general consumables, but it introduces challenges As with CD4 testing instrumentation, glu- in training and variation with instrument main- cometers and hemoglobinometers (POC devices) tenance. Hematology is a closed system market

Global Health: Science and Practice 2016 | Volume 4 | Number 3 473 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

FIGURE 5. Diversity of Instrument Types in 8 African Countries, by Diagnostic Area

(e.g., the systems require proprietary reagents), implementation and adherence to harmonization Different countries and these systems require many commodity types strategies (Figure 7). had different to keep instruments operational. Country B and Country G have been success- levels of ful at limiting instrument brand expansion. ’ implementation Proposed Reductions in Instrument Diversity Country B s success is partly due to PEPFAR, which has historically provided funding for and and adherence to Following HIV harmonization and standardiza- coordinated closely with partners and the minis- harmonization tion workshops, participants proposed substantial try of health around procuring laboratory instru- strategies. reductions in instrument diversity, ranging from mentation with the goal of complying with a 17% reduction in Country D’s existing CD4 standardized instrument lists. Country G has instruments to a high of 88% for Country H’s succeeded by limiting commodity availability chemistry testing instruments (Table). As men- and supporting procurement and distribution of tioned earlier and demonstrated in Figure 6 and commodities only for approved instrumentation. Figure 7, chemistry and hematology account for Both of these countries have improved national the most diagnostic instrument diversity, hence laboratory forecasting efforts, which are now led have the largest potential for instrument reduc- by national quantification committees and have tion. Potential reduction for both CD4 instru- directly influenced commodity availability and mentation and polymerase chain reaction–based improved laboratory logistic system proficiency molecular instrumentation is between 1 and 2 due to reduced commodity counts. For example, from an absolute count perspective. as part of its harmonization efforts, Country G designed an initial laboratory logistics system in Efforts to Implement Harmonization and 2007 that reduced commodity types from more Standardization than 400 down to 185 HIV-specific products. We compared the earlier adopters of the Maputo Once the national logistics system was estab- Declaration—Country B (2012), Country C (2007), lished and institutionalized, the commodity pro- Country F (2009), and Country G (2007)—and file was later expanded to include a full array of found notable differences between countries in diagnostic products (more than 380), further

Global Health: Science and Practice 2016 | Volume 4 | Number 3 474 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

TABLE. Proposed Instrument Reductions by Instrument Type

Proposed Instrument Reduction

Country CD4 Chemistry Hematology Molecular

Country A -50% -82% -56% -33% Country B 0% -22% -33% 0% Country C N/A N/A N/A N/A Country D -17% -55% -71% -50% Country E 0% -76% -67% 0% Country F 0% -20% -75% 0% Country G N/A N/A N/A N/A Country H 0% -88% -80% 0%

Note: Percentage reduction cannot be calculated for Country C and Country G due to lack of data on instrument diversity in these countries before harmonization efforts began.

FIGURE 6. Comparison of Current and Proposed Instrument Diversity in 6 African Countries With Harmonization Proposals

40 CD4 Chemistry Hematology Molecular

35 34 32

21 20 20 17 15 No. of Platform of No. Types 15 12 11 10 9 9 7 6 6 6 6 6 5 5 5 555 5 4 4 444 4 444 4 3 33 33 3 2 22222222 1 0 Country A Country A Country B Country B Country D Country D Country E Country E Country F Country F Country H Country H 2014 Harmonizaon 2012 Harmonizaon 2014 Harmonizaon 2014 Harmonizaon 2009 Harmonizaon 2015 Harmonizaon Proposal 2014 Proposal 2012 Proposal 2014 Proposal 2014 Proposal 2009 Proposal 2015

Global Health: Science and Practice 2016 | Volume 4 | Number 3 475 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

FIGURE 7. Shifts in Instrument Diversity in 4 African Countries Following Their Harmonization Proposals

CD4 Chemistry Hematology Molecular

50 47

40 37

20 No. of Instrtument Instrtument of No. Types

15 13

10 10 8 7 5 5 4 44 4 4 4 5 33 3 3 3 333 3 2 2 2 22 2 1 0 0 0 Country B Country B Country C Country C Country F Country F Country G Country G Harmonizaon 2015 Harmonizaon 2015 Harmonizaon 2015 2007 2015 Proposal 2012 Proposal 2007 Proposal 2009

Note: Country C has 75 additional instruments for chemistry and 55 additional instruments for hematology that are not included in the bar chart.

expanding the breadth of services offered and have developed harmonization and standardiza- improving the overall planning and procurement tion proposals only within the last 3 years, so it is practices associated with laboratory service deliv- difficult at this point to determine how well these National ery overall, not just HIV-related services.16,17 countries will succeed at advancing their procure- laboratory Conversely, Country C and Country F appear to ment practices to better align to their proposed quantification have had less success in implementing an strategies. committees can approach that would reduce instrument diversity, directly influence and instead have experienced substantial growth commodity in brand diversity. Country F has seen marked Common Challenges and Critical Success availability increases in chemistry and hematology instrument Factors to Advancing Harmonization and improve diversity, much of which was driven by POC Throughout the harmonization and standardiza- ’ laboratory logistic technology, with molecular instrumentation being tion efforts facilitated by PFSCM s Supply Chain the only diagnostic area that has remained Management System, in-country program leads, systems. constant. As stated earlier, in many countries the stakeholders, and workshop participants expressed coordination efforts associated with implementing recurring challenges in advancing harmonization Across countries, national laboratory plans and adherence to har- strategies. Overall, 10 recurring challenges were 10 recurring monization and standardization policies have been identified across countries: the structure of and 13,18 challenges to inconsistent and frequently problematic. We lack of adherence to existing procurement poli- laboratory believe that this may play an important role in the cies, misalignment of service delivery policies and harmonization ability of Country C and Country F to achieve guidelines, lack of defined laboratory tiers, lack have been leveled or decreasing brand diversity. of an effective coordinating body responsible for — identified. Another 4 of the evaluated countries laboratory harmonization, and issues with equip- Country A, Country D, Country E, and Country H— ment maintenance, data availability, managing

Global Health: Science and Practice 2016 | Volume 4 | Number 3 476 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

frequent shifts in technology, human resour- and vendor performance, but also to assess the ces, competing priorities, and political agendas. potential for introducing new technology. To advance the harmonization and standardization agenda at a national level, we believe that Lack of an Effective Coordinating Body 3 of the challenges identified by participants are Many countries do have a national laboratory the most important to address: (1) lack of adher- directorate or coordinating body responsible for ence to procurement policies (i.e., instrument guiding laboratory development, but they are diversity), (2) lack of an effective coordinating unable to prioritize laboratory harmonization and body, and (3) misalignment of laboratory policies, move the harmonization agenda from proposal to treatment guidelines, and minimum services. actual policy. This may be due to national priorities or political agendas, with laboratory tech- Lack of Adherence to Procurement Policies nical working groups operating without a formal There are high levels of instrument diversity in mandate or authority. Formalizing laboratory laboratory diagnostic instruments, equipment, technical working groups with specific terms of and services across health facilities. The lack of reference, authority, and accountability would coordination in the procurement and deployment support advocacy efforts and help finalize harmo- of laboratory equipment contributes to different nization and standardization proposals, as well as levels of compliance with service delivery stan- guide implementation. dards across facilities that provide similar levels of Additionally, technical working groups should Technical working care. be charged with monitoring instrument pro- groups should be Compounding this challenge, procurement curement and placement, as well as laboratory charged with agents and national governments may see redu- technology development. This will ensure that monitoring cing instrument diversity as reducing competi- ministries of health define processes for evaluat- instrument tiveness. They may assume that limiting the ing new technologies before they are deployed. procurement and number of types of diagnostic instruments will The working groups should also guide ministries placement, as well lead to sole and single sourcing of instruments in developing policy for laboratory network devel- as laboratory and reagents, creating monopolies and restricting opment and other national laboratory interests. technology competition, which contradicts most country- Once policy is finalized by the technical working development. level procurement regulations. groups, it is critical to ensure stakeholder adher- If harmonization and standardization policies ence to standardization of procurement practice, were static, this argument could hold true, but as well as instrument placement. policies must be dynamic and based on instru- Harmonization ment and vendor performance. In addition, over Misalignment of Laboratory Policies, Treatment and time, systems and clinical demands shift, tech- Guidelines, and Minimum Services standardization nology advances, and existing instruments age In many countries, laboratory policies, minimum policies must and become obsolete. Countries thus must update packages of care, and national HIV/AIDS care and be dynamic standardized instrument lists to align service treatment guidelines are not aligned at the time of and based on delivery expectations and ensure that laboratories harmonization and standardization workshops. instrument can provide the necessary services with instru- Laboratory and treatment policies and guidelines and vendor ments that perform well, with reliable vendor may have been updated with differing frequencies performance. support. Monitoring instrument and vendor and without coordination between laboratory staff performance should be continuous. These perfor- and clinicians. Many laboratories may have been mance measures have traditionally been linked to providing tests that were outdated or not aligned commodity logistics systems, which are based on with the minimum care needs by tier, or tests that Laboratories may procurement lead times and instrument repair were not clearly defined within existing laboratory provide tests that response times that inform replenishment of policy and/or strategy documents. are outdated or reagents. Historically, these systems have had For example, the harmonization and standar- not aligned with challenges, but with meaningful technical sup- dization effort in Country A began with a review the appropriate ’ port and investment from PEPFAR, commodity of the country s laboratory strategic plan, the tier’s minimum logistics systems are improving. Harmonization essential health service package, the integrated care needs. and standardization policy reviews should occur health service plan, and the HIV/AIDS care at a minimum of every 2 years, and should use and treatment guidelines. In Country E, a policy annual laboratory quantification and logistics review was initiated with the country’s norms and data to measure progress and general instrument standards for medical laboratories, the laboratory

Global Health: Science and Practice 2016 | Volume 4 | Number 3 477 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

strategic plan, and the HIV/AIDS prevention and and equipment types within the clinical cascade, treatment guidelines. When long-standing strate- as well as on how tiered networks are defined and gic plans cover many years and HIV treatment implemented.12,13 Ideally, a combination of mea- guidelines are more dynamic, these documents can sures associated with testing availability within quickly diverge in regard to priorities, implementa- the laboratory network, instrument brands, and tion planning, and overall expectations associated potential placement of instruments within the with laboratory service delivery. tiered laboratory network would provide a more Additionally, budget growth and scale-up efforts complete picture of harmonization and standar- for laboratory, program, and clinical needs increas- dization success. This could even provide oppor- ingly diverge, further widening the gap between tunities to identify optimization strategies that clinical needs and laboratory service capacity. build upon laboratory standardization efforts. In most cases, the harmonization and stan- Additional research could provide greater under- dardization workshops were the first time clin- standing of which components of harmonization icians and program staff had met in a large forum efforts have achieved success and which are to discuss laboratory service delivery challenges, works in progress or more challenging to imple- to define minimum test offerings by tier, and to ment more broadly. advance a coordinated and aligned way forward. Implementing a harmonization and standar- To address these challenges, it is important to dization policy and demonstrating alignment to ensure that when treatment guidelines or mini- a standardized instrument list can take many mum packages of care are updated, laboratory years. Half of the countries included in this analysis personnel have the opportunity to inform deci- had completed harmonization workshops before sion makers of existing laboratory capacity and 2013, the other half completed their workshops scalability. Laboratory, clinician, and program more recently. A follow-up evaluation in a few staff should be engaged frequently to ensure that years could identify additional challenges not the laboratory network evolves to meet clinical considered here, or could demonstrate further and program needs. Laboratory investigations compliance and success. should be fully integrated into clinical and preventive protocols and programs, with the rational CONCLUSION use of essential tests relevant to the level and type of facility. The Maputo Declaration and, more recently, the Tests offered at As consumers of laboratory tests, clinicians Freetown Declaration, called on national govern- each laboratory should help laboratory programs determine test ments to prioritize laboratory system develop- tier should be offerings for each laboratory tier based on the ment and emphasized the need to foster national based on the established national health care package, clinical ownership; more importantly, participants at these established importance, cost, suitability to the environment, two meetings urged donors and partners to com- national health and level of expertise of the service provider and mit to working in close coordination to support care package, end users. efforts to strengthen sustainable public health clinical laboratory systems. importance, cost, Although the list of tiered test offerings and Limitations diagnostic instruments included in the Maputo suitability, and We recognize several limitations to the analysis Declaration11 is now dated, the founding prin- level of provider described here. Our current analysis targets HIV- ciples and primary objectives are still relevant expertise. related diagnostics only. This choice was due to today, as illustrated by the 2015 Freetown Decla- the high quality of HIV laboratory data that was ration. New technologies are emerging to address Our analysis available. It should be noted that all harmoniza- diagnostic and patient-monitoring challenges, targets HIV- tion and standardization workshops included all along with additional levels of complexity in related diagnostic services and were not limited to just laboratory networks due to the introduction of diagnostics only, HIV diagnostics. POC technology and associated decentralization due to the high The primary measure of harmonization and of laboratory services. A strategic harmoniza- quality of standardization in this article was limited to tion and standardization framework is critical to available data. instrument brand diversity, as a way to measure ensure a coordinated and sustainable laboratory compliance to procurement from a national development agenda. standardized instrument list. Other evaluations Overall, harmonization and standardization have sought to focus on appropriateness of tests efforts have been implemented with mixed success,

Global Health: Science and Practice 2016 | Volume 4 | Number 3 478 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

with some countries only recently implementing 90-90-90 effort among HIV practitioners, there is measures to introduce harmonization policies. Effec- potential for less support and focus on chemistry tively implementing a harmonization and standar- and hematology instrumentation, as well as on dization policy and demonstrating alignment to a other diagnostic specialties. Support for HIV standardized instrument list can take many years. programs has historically included procurement For example, the rate of replacing existing equipment of chemistry- and hematology-related commod- with approved instruments depends on instrument ities and instruments, with the assumption that life span. Using harmonization and standardization local governments will assume responsibility for principles as programs scale up, with new instru- these lower-cost tests moving forward. It is ments purchased for replacements and network therefore critical to improve efforts to ensure that expansion at new sites, will limit growth in instru- national ministries of health can expand and ment and brand diversity. Leveraging scorecards, as sustain not only their existing HIV-related labo- recommended by the Freetown Declaration, may ratory services but also the general public health be another way to advance these efforts. laboratory services, to serve broader health and Since the inception of the global response to surveillance needs. It is important for program the HIV epidemic, procurements of HIV-related managers, supply chain activity managers, and diagnostic instruments have increased markedly. others to work with their colleagues to understand A data-driven laboratory harmonization and the capacity, use, and placement of instrument- standardization approach is one way to ensure based diagnostics. There is still a need to reduce optimal use of laboratory-based instrument diag- excess numbers of products, improve use, decrease nostics and to create efficiencies in product costs, and increase efficiency. procurement, placement, training, and use. This Important gains have already been achieved will be increasingly important to efforts to within national laboratory networks, but there is achieve UNAIDS’ 90-90-90 targets.1 still a need to ensure that countries are able to However, less has been done to regulate pro- provide continued quality laboratory services in curement and reduce instrument diversity within an efficient and sustainable manner. The Maputo chemistry and hematology, where critical safety and Freetown Declarations provide a detailed monitoring tests for antiretroviral therapy are strategic approach that is critical to ensure a performed, or to provide highly needed general coordinated and sustainable laboratory develop- health and screening diagnostics. As a result, ment agenda to address the broader health chemistry and hematology now represent the security agenda in Africa. highest levels of instrument diversity within laboratory networks. Acknowledgments: The content in this manuscript are those of the A high level of instrument diversity has a authors and do not necessarily reflect the views of the United States Agency for International Development or the United States great impact on laboratory commodity forecast- Government. ing, supply chain systems, equipment maintenance, and quality laboratory service delivery. Competing Interests: None declared. Although many years have passed since the Maputo meeting in 2008, the 2015 Freetown REFERENCES Declaration illustrates the continued need to improve adherence to harmonization and stan- 1. Joint United Nations Programme on HIV/AIDS (UNAIDS). 90-90-90: an ambitious treatment target to help end dardization practices. Improved coordination is the AIDS epidemic. Geneva: UNAIDS; 2014. Available from: required, as well as the development, implemen- http://www.unaids.org/en/resources/documents/2014/ tation, and—more importantly—monitoring and 90-90-90 updating of laboratory harmonization and stan- 2. World Health Organization | Regional Office for Africa (WHO | dardization policies over time. Developing a tiered AFRO). The Maputo declaration on strengthening of laboratory systems. Maputo (Mozambique): WHO | AFRO; 2008. laboratory network, procuring from standardized Available from: http://www.who.int/diagnostics_laboratory/ instrument lists, as well as ensuring the align- Maputo-Declaration_2008.pdf ment of laboratory policies, minimum packages 3. Nkengasong J. Strengthening laboratory services and systems of care, and national care and treatment guide- in resource-poor countries. Am J Clin Pathol. 2009;131(6): lines, are all critical to achieving the benefits of 774–774. CrossRef. Medline harmonization and standardization. 4. Justman J, Koblavi-Deme S, Tanuri A, Goldberg A, Gonzalez LF, Gwynn CR. Developing laboratory systems and infrastructure for HIV As donor contributions and priorities shift scale-up: a tool for health systems strengthening in resource-limited in response to viral load scale-up as part of the settings. J Acquir Immune Defic Syndr. 2009;52 Suppl 1:S30–S33.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 479 Harmonized HIV Laboratory Systems: Challenges and Opportunities www.ghspjournal.org

CrossRef. Medline. Free full-text available from: http://icap.columbia. and affected by HIV/AIDS, TB and malaria. Maputo edu/files/uploads/Justman_Developing_a_laboratory_system_and_ (Mozambique): WHO | AFRO; 2008. Available from: http:// infrastructure_for_HIV_scale-up_JAIDs_2009.pdf www.who.int/diagnostics_laboratory/3by5/ 5. African Society for Laboratory Medicine (ASLM). The Freetown Maputo_Meeting_Report_7_7_08.pdf declaration on developing resilient laboratory networks for the 12. Putoto G, Cortese A, Pecorari I, Musi R, Nunziata E. Harmoniza- global health security agenda in Africa. Freetown (Sierra Leone): tion of clinical laboratories in Africa: a multidisciplinary approach ASLM; 2015. Available from: http://aslm.org/what-we-do/ to identify innovative and sustainable technical solutions. global-health-security/freetown-declaration/ Diagnosis. 2015;2(2):129–135. CrossRef 6. Merkel M, Best M. A revitalized laboratory declaration to forge 13. Olmsted SS, Moore M, Meili RC, Duber HC, Wasserman J, a path for action. Lab Culture, The ASLM Newsletter. 2015;14:7. Sama P, et al. Strengthening laboratory systems in resource- Available from: http://aslm.org/stay-informed/press-room/ limited settings. Am J Clin Pathol. 2010;134(3):374–380. news-articles/revitalised-laboratory-declaration-to-forge-a-path- CrossRef. Medline for-action/ 14. Joint United Nations Programme on HIV/AIDS (UNAIDS). HIV/AIDS 7. Partnership for Supply Chain Management (PFSCM), Supply diagnostics technology landscape. 4th ed. Geneva: UNAIDS; 2014. Chain Management System (SCMS). 2008–2015 procurement Available from: http://www.unitaid.eu/images/marketdynamics/ data. Arlington (VA): SCMS; 2016. publications/UNITAID-HIV_Diagnostic_Landscape-4th_edition.pdf 8. Glencross DK, Coetzee LM, Cassim N. An integrated tiered service 15. World Health Organization (WHO) [Internet]. Geneva: WHO; delivery model (ITSDM) based on local CD4 testing demands c2016. Prequalification of in vitro diagnostics. http://www.who. can improve turn-around times and save costs whilst ensuring int/diagnostics_laboratory/evaluations/en/ accessible and scalable CD4 services across a national 16. USAID | DELIVER PROJECT, Task Order 1. Laboratory programme. PLoS One. 2014;9(12):e114727. CrossRef. Medline standardization: lessons learned and practical approaches. 9. Peter TF, Shimada Y, Freeman RR, Ncube BN, Khine AA, Murtagh Arlington (VA): USAID | DELIVER PROJECT, Task Order 1; 2010. MM. The need for standardization in laboratory networks. Am J Available from: http://deliver.jsi.com/dlvr_content/resources/ Clin Pathol. 2009;131(6):867–874. CrossRef allpubs/guidelines/LabStand.pdf 10. Edgil D, Williams J, Smith P, Kuritsky J. Optimising the laboratory 17. USAID | DELIVER PROJECT, Task Order 1. Case study: harmoniza- supply chain: the key to effective laboratory services. Afr J Lab tion of laboratory items in Zambia. Arlington (VA): USAID | DELIVER Med. 2014;3(1). CrossRef PROJECT, Task Order 1; 2010. Available from: http://www.who.int/ 11. World Health Organization | Regional Office for Africa hiv/amds/amds_usaid_zambia_laboratory.pdf (WHO | AFRO). Consultation on technical and operational 18. Massamba C, Mwangi C. The Tanzania experience: clinical recommendations for clinical laboratory testing harmonization laboratory testing harmonization and equipment standardization and standardization: helping to expand sustainable quality at different levels of a tiered health laboratory system. Am J Clin testing to improve the care and treatment of people infected with Pathol. 2009;131(6):861–866. CrossRef. Medline

Peer Reviewed

Received: 2016 Jan 5; Accepted: 2016 Aug 23

Cite this article as: Williams J, Umaru F, Edgil D, Kuritsky J. Progress in harmonizing tiered HIV laboratory systems: challenges and opportunities in 8 African countries. Glob Health Sci Pract. 2016;4(3):467-480. http://dx.doi.org/10.9745/GHSP-D-16-00004.

& Williams et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi.org/ 10.9745/GHSP-D-16-00004.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 480 REVIEW

Postabortion Care: 20 Years of Strong Evidence on Emergency Treatment, Family Planning, and Other Programming Components

Douglas Huber,a Carolyn Curtis,b Laili Irani,c Sara Pappa,d Lauren Arringtone

Twenty years of postabortion care (PAC) studies yield strong evidence that: Misoprostol and vacuum aspiration are comparable in safety and effectiveness for treating incomplete abortion. Misoprostol, which can be provided by trained nurses and midwives, shows substantial promise for extending PAC services to secondary hospitals and primary health posts. Postabortion family planning uptake generally increases rapidly–and unintended pregnancies and repeat abortions can decline as a result–when a range of free contraceptives, including long-acting methods, are offered at the point of treatment; male involvement in counseling–always with the woman's concurrence– can increase family planning uptake and support.

ABSTRACT Worldwide 75 million women need postabortion care (PAC) services each year following safe or unsafe induced abortions and miscarriages. We reviewed more than 550 studies on PAC published between 1994 and 2013 in the peer-reviewed and gray literature, covering emergency treatment, postabortion family planning, organization of services, and related topics that impact practices and health outcomes, particularly in the Global South. In this article, we present ﬁndings from studies with strong evidence that have major implications for programs and practice. For example, vacuum aspiration reduced morbidity, costs, and time in comparison to sharp curettage. Misoprostol 400 mcg sublingually or 600 mcg orally achieved 89% to 99% complete evacuation rates within 2 weeks in multiple studies and was comparable in effectiveness, safety, and acceptability to manual vacuum aspiration. Misoprostol was safely introduced in several PAC programs through mid-level providers, extending services to secondary hospitals and primary health centers. In multiple studies, postabortion family planning uptake before discharge increased by 30–70 percentage points within 1–3 years of strengthening postabortion family planning services; in some cases, increases up to 60 percentage points in 4 months were achieved. Immediate postabortion contraceptive acceptance increased on average from 32% before the interventions to 69% post-intervention. Several studies found that women receiving immediate postabortion intrauterine devices and implants had fewer unintended pregnancies and repeat abortions than those who were offered delayed insertions. Postabortion family planning is endorsed by the professional organizations of obstetricians/gynecologists, midwives, and nurses as a standard of practice; major donors agree, and governments should be encouraged to provide universal access to postabortion family planning. Important program recommendations include offering all postabortion women family planning counseling and services before leaving the facility, especially because fertility returns rapidly (within 2 to 3 weeks); postabortion family planning services can be quickly replicated to multiple sites with high acceptance rates. Voluntary family planninguptakebymethodshouldalwaysbe monitored to document program and provider performance. In addition, vacuum aspiration and misoprostol should replace sharp curettage to treat incomplete abortion for women who meet eligibility criteria. a Innovative Development Expertise & Advisory Services, Inc. (IDEAS), Boxford, MA, USA. INTRODUCTION b United States Agency for International Development, Washington, DC, USA. orldwide, 210 million women become pregnant c Population Reference Bureau, Health Policy Project, Washington, DC, USA. W d Palladium, Health Policy Project, Washington, DC, USA. annually; 135 million will have a live birth, and e University of Maryland, St. Joseph Medical Center, Towson, MD, USA. 75 million, or one-third, will have a spontaneous or Correspondence to Douglas Huber ([email protected]). induced abortion and need postabortion care (PAC).

Global Health: Science and Practice 2016 | Volume 4 | Number 3 481 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

Of the 75 million abortions, 31 million are We searched Scopus, MEDLINE, and POPLINE spontaneous (miscarriages) and 44 million are for relevant PAC interventions that had been induced; half of the induced abortions are unsafe, evaluated, using several search words relevant to performed by persons lacking the necessary skills PAC, such as postabortion care, postabortion con- or in an environment not in conformity with traception, incomplete abortion, abortion compli- minimal medical standards.1 cations, dilation and curettage, misoprostol, and Since 1994, the United States Agency for manual vacuum aspiration. If the article title International Development (USAID) has sup- indicated there might be an intervention that ported implementation of PAC programs in more could be replicated, we reviewed the full-text than 40 countries to address complications relat- article to determine if there was sufficient infor- ed to miscarriage and incomplete abortion.2 PAC mation to be included in the ‘‘What Works’’ com- may be a unique service delivery model that is pendium and that the intervention took place in both curative and preventative—curative in treat- low- or middle-income countries in Asia, Africa, ing incomplete abortion and the symptoms of Latin America, or the post-Soviet states.4,5 hemorrhage and sepsis; preventative in providing Key interventions that had taken place only in family planning services to address unmet need high-income countries (namely Australia, Japan, for contraception and reduce unintended preg- Europe, or the United States) were included only if nancies and repeat abortions. we determined that the interventions could be USAID’sPAC The 3 components of USAID’s PAC model relevant for low-resource country contexts but had model contains are2: not yet been initiated in developing countries. We also reviewed unpublished reports (gray 3 components: 1. Emergency treatment emergency literature) from key websites to supplement the 2. Family planning counseling and service deliv- treatment, family studies published in peer-reviewed publications ery, and where financial and human resources planning, and because of the limited amount of published exist, evaluation and treatment of sexually community literature on PAC. The websites reviewed comprised transmitted infections (STIs) as well as HIV empowerment. United Nations (UN) agencies, the Joint United counseling and/or referral for testing Nations Programme on HIV/AIDS (UNAIDS), the 3. Community empowerment through commu- World Health Organization (WHO), the Cochrane nity awareness and mobilization Collaboration, the Open Source Initiative (OSI), the International Center for Research on Women PAC is an integral part of maternity care, and all (ICRW), Futures Group, Population Services Inter- components of PAC services are essential, as stated national (PSI), the Population Council, the Interna- at the 1994 International Conference on Population tional Council of Women (ICW), the World Bank, and Development. All 3 components have been Family Health International (FHI) (renamed FHI included in USAID programs since 1994.2,3 360 in 2011), Gynuity Health Projects, Ipas, and the In 2007, USAID published the first global PAC Guttmacher Institute. Biomedical information was research compendium, ‘‘What Works, A Policy included insofar as it was relevant to programmatic and Program Guide to the Evidence on Postabor- considerations. The articles and reports included in tion Care,’’ which reviewed the PAC literature our review covered the 3 components of the USAID from 1994 through 2003.4 This article sum- PAC model as well as a fourth component that marizes the major findings on PAC interventions addressed policy, programs, and health systems in with strong evidence, described in the forth- postabortion care. We adapted the Gray scale coming second edition of USAID’s global PAC (Table 1) with its 5 levels to assess the strength of research compendium. The second edition builds research evidence for each article or report cited in on the first edition by including more detailed the compendium.4,7 Strong evidence was drawn findings that address the cycle of repeated primarily from the first 4 levels that included unintended pregnancy and abortion.4-6 randomized controlled trials, well-designed trials with or without randomization, and some well- designed nonexperimental studies from more than METHODS AND SCOPE one center or research group. Evidence was con- ThefirstandsecondeditionsoftheUSAIDPAC sidered ‘‘strong’’ if it had support of at least 2 Gray I, research compendium reviewed in detail more than II, or IIIa studies and/or 5 Gray IIIb, IV, or V studies.7 550 articles and reports published between 1994 Studies with strong evidence for postabortion and 2013, representing 20 years of PAC research. care are highlighted in this article and critical areas

Global Health: Science and Practice 2016 | Volume 4 | Number 3 482 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

TABLE 1. Gray Scale of Strength of Evidence

Strength of Evidence Description

I Strong evidence from at least one systematic review of multiple well-designed, randomized controlled trials. II Strong evidence from at least one properly designed, randomized controlled trial of appropriate size. IIIa Evidence from well-designed trials/studies without randomization that include a control group (e.g., quasi- experimental, matched case-control studies, pre-post with control group). IIIb Evidence from well-designed trials/studies without randomization that do not include a control group (e.g., single group pre-post, cohort, time series/interrupted time series). IV Evidence from well-designed, nonexperimental studies from more than one center or research group. V Opinions of respected authorities, based on clinical evidence, descriptive studies, or reports of expert committees.

Note: Gray includes 5 levels of evidence. For the ‘‘What Works’’ compendiums, level III was subdivided to differentiate between studies and evaluations whose design included control groups (IIIa) and those that did not (IIIb).101 Qualitative studies can be classiﬁed as either level IV or V, depending on number of study participants and other factors. For more detail about these types of studies and their strengths and weaknesses, see Gray (2009).7

for future research are identified. Preference was counseling, including STI/HIV evaluation, (3) com- given to studies from the Global South, but where munity empowerment, and (4) health systems evidence was scant from the South, studies from and policies. the Global North were included. Both editions of the compendium include a full description of the evidence statements along with complete refer- Emergency Treatment of Incomplete ences. In this article, we summarize the strong Abortion evidence we consider most applicable for extending Surgical Treatment PAC services, increasing access to family planning, Vacuum aspiration, either electric vacuum aspira- Electric and improving cost-efficiency, and enhancing client tion (EVA) or manual vacuum aspiration (MVA), manual vacuum satisfaction. is safer and less painful than sharp curettage, aspiration are which is usually performed with general anes- safe, effective, thesia in an operating theater.4 Many of the FINDINGS and preferable to resources we reviewed focused on MVA. Main sharp curettage Recent PAC studies (2004–2013) reinforce most findings included: for uterine of the strong evidence statements from the 2007 Women treated by vacuum aspiration (electric evacuation. PAC research compendium; specifically, those or manual) for incomplete abortion had shorter related to access to services, quality of care, com- procedures and significantly less blood loss munity awareness of PAC, and postabortion than those treated by sharp curettage. Pain is family planning. Recent research also highlights generally less than with sharp curettage.4 the continuing challenges of providing complete PAC services, including family planning, counsel- The use of general anesthesia with sharp ing, and client information. Treatment options for curettage is associated with increased risk of blood loss, cervical injury, uterine perforation, incomplete abortion, task shifting, and decentra- 4 lizing services are rapidly evolving and well docu- and subsequent abdominal hemorrhage. mented. Verbal support (‘‘verbacaine’’) to provide reas- In this section, we summarize key findings surance and diversion during the MVA proce- on the 4 main components covered in the scope of dure and/or paracervical block were generally our literature review: (1) emergency treatment of inadequate for pain management with MVA.8,9 incomplete abortion, (2) postabortion family planning Sedation with analgesics plus emotional

Global Health: Science and Practice 2016 | Volume 4 | Number 3 483 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

support during the procedure gave better Table 2 summarizes findings from 8 high- results.8,10 quality studies conducted in the Global South between 2005 and 2012 in tertiary and district Paracervical block with local anesthesia for 14,17-23 incomplete abortion with an open cervix hospitals. Successful evacuation rates Successful provided no discernible benefit over placebo ranged between 89% and 99% at 1 to 2 weeks evacuation rates in studies of MVA procedures.8,9 post-treatment without recourse to surgical intervention. A follow-up was usually scheduled at with misoprostol Costs can be substantially reduced by introdu- 1 week. If evacuation was not complete, the ranged 89%–99% cing MVA to replace sharp curettage when woman was offered the option of waiting a at 1–2 weeks accompanied by changes in protocols for second week or having an immediate surgical post-treatment. admission, place of procedure (often a simple evacuation, usually by MVA. If not complete at procedure room), and an improved service 2 weeks, a surgical evacuation was performed. delivery model that gives priority to PAC clients. Success rates were lower in studies with shor- Direct costs are reduced by avoiding use of the ter time frames for judging completion and more operating theater, general anesthesia, blood 11-13 frequent follow-up visits. For example, one study transfusions, and prolonged hospital stays. scheduled the first follow-up visit at day 2; if Prophylactic antibiotics to prevent postopera- evacuation was not complete, women could opt to tive infection have not shown effectiveness in return at day 7, but not later. Success rates were the relatively underpowered studies from the lower and also different between the 2 study Global South.14,15 One large study of induced sites, 44% vs. 85%, suggesting major provider and abortion in the United States showed a institutional variations. The authors of the study reduction in postoperative infections for acknowledged that providers’ lack of confidence vacuum aspiration and medical induced abor- in using misoprostol may partly explain the large tion using doxycycline 100 mg twice daily for differences between sites.24 7 days.16 However, infections were rare with Surgical evacuation is still needed for the or without prophylactic antibiotics. 2% to 8% of women with incomplete evacuation following misoprostol; women presenting with complications (e.g., sepsis, heavy bleeding); Medical Management womenwithadvancedgestationalage;and The past decade has seen major advances in women who prefer surgical management.25-27 medical evacuation using misoprostol for treat- Most studies specified that participants live Misoprostol is safe ment of incomplete abortion. Misoprostol is within the facility’s geographic catchment area and effective for rapidly gaining acceptance as a safe, effective or within one hour of travel time, and that treatment of treatment for incomplete abortion in primary and ultrasound be available by referral. Ultrasound incomplete secondary outpatient settings and can be pro- was seldom needed; current misoprostol training abortion and can vided by trained nurses and midwives. It should curricula note that ultrasound is not required to be provided by be an available option for women needing post- diagnose complete evacuation, although in selec- trained nurses and abortion care. ted circumstances it may assist in the evalua- 27 midwives. Evidence from at least 16 studies of mis- tion. None of the 8 misoprostol studies in oprostol points to an optimal dose and route Table 2 documented postabortion contraception of administration of 400 mcg sublingually or uptake, a critical component of postabortion care. 600 mcg orally; both are equally effective for treat- Recent programmatic expansions in Senegal ing uncomplicated first-trimester incomplete abor- document favorable client and clinical outcomes tion, either spontaneous or induced. No advantage when misoprostol is given at lower-level health was seen with higher or multiple doses, or when facilities farther from hospitals.25 Thirteen studies given by other routes of administration.5 in 9 countries included nurse-midwives as provi- Study criteria for misoprostol treatment typi- ders.14,17,18,21,23,25,28-34 cally included first-trimester incomplete abortion Major findings with misoprostol for PAC are without complications and an open cervix. Other as follows: selection criteria in clinical trials usually included women residing within one hour’s travel distance Success rates were comparable with MVA in of a hospital. Back-up surgical evacuation and most studies using doses of 400 mcg sublin- ultrasound were available on site or by ready gually or 600 mcg orally with optional follow- referral to another regional or tertiary hospital. up extended to 2 weeks (Table 2). Surgical

TABLE 2. Effectiveness and Satisfaction With Treatment for Incomplete Abortion, Misoprostol Compared With Surgical Evacuation, 10 Countries, 2005–2012

Effectiveness: Country % With % Client (Sample Study Design:Misoprostol/ Complete Satisfaction Article Size) Surgical ComparisonGroup Evacuation With Procedure Comments

Blandine Burkina Faso 400 mcg misoprostol M: 98% M: 99% PAC with misoprostol introduced to 2 district hospitals with no previous PAC 201217 (N = 99) sublingually/ referral for service. All eligible women chose misoprostol over optional referral for MVA. surgical Dao 200718 Burkina Faso 600 mcg misoprostol orally/ M: 94% M: 97% 2 teaching hospitals. (N = 447) MVA S: 99% S: 98% Weeks Uganda 600 mcg misoprostol orally/ M: 96% M: 94% Misoprostol was associated with less pain and fewer complications but 200514 (N = 317) MVA S: 92% S: 95% increased bleeding. All received antibiotics after treatment.

Taylor Ghana 600 mcg misoprostol orally/ M: 98% M: 94% 44% were very satisfied with misoprostol vs. 8% with MVA; 95% of those 201119 (N = 230) MVA S: 99% S: 89% treated with misoprostol would choose it again vs. 36% treated with MVA. Shwekerela Tanzania 600 mcg misoprostol orally/ M: 99% M: 99% 75% were very satisfied with misoprostol vs.55% with MVA; more side effects 200720 (N = 150) MVA S: 100% S: 100% were associated with misoprostol; greater pain with MVA. Bique Mozambique 600 mcg misoprostol orally/ M: 91% M: 96% 87% were very satisfied with misoprostol vs. 37% with MVA; trained midwife 200721 (N = 270) MVA S: 100% S: 100% provided MVA with only verbal anesthesia; tertiary hospital site.

Montesinos Ecuador 600 mcg misoprostol orally/ M: 94% M: 96% 47% were very satisﬁed with misoprostol vs. 40% with MVA; ultrasound use 201122 (N = 242) MVA S: 100% S: 97% decreased threefold for misoprostol and MVA in 1 year. Shochet Senegal 400 mcg misoprostol Senegal Overall in the Antibiotics given with the surgical option; success rates much higher with 201223 (N = 199) sublingually/ standard surgical M: 93% 5 countries misoprostol after ﬁrst month from introduction. Ultrasound not needed on site. Niger care (MVA or D&C) S: 100% M: 99% Nurses and midwives had prominent roles in care in Burkina Faso, Niger, and (N = 152) Niger S:98% Senegal. Mauritania M: 89% (N = 119) S: 100% Nigeria Mauritania (N = 51) M: 91% Burkina Faso S: 100% (N = 318) Nigeria M: 96% S: 100% Burkina Faso M: 98% S: 100% Abbreviations: D&C, dilation and curettage; M, misoprostol; MVA, manual vacuum aspiration; PAC, postabortion care; S, surgical. 485 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

evacuation (usually MVA) was available on component to reduce preventable child and mater- site or by referral if the woman requested and nal deaths in both the postpartum and postabortion for those who did not meet the criteria for periods is a growing focus for maternal, neonatal, misoprostol. and child health programs.1,38 Many PAC clients Complications were low and comparable to were not using a modern contraceptive method at MVA. Most studies reported less pain with the time of conception. When providing family misoprostol than with MVA.14,20 Bleeding was planning counseling and services, it is necessary to slightly increased, compared with MVA, understand the reproductive intentions of the though not of clinical significance.14 Client woman and her partner. Expanding the method acceptability was high, with substantially mix, including long-acting reversible contraceptives more women reporting they were ‘‘very (LARCs) and permanent methods, allows the satisfied’’ compared with MVA. Misoprostol woman to choose what she wants and what works recipients were also more likely to recommend best for her. Women need adequate counseling and it to a friend (Table 2). choice of methods to make informed voluntary decisions about family planning. High client satisfaction and complete eva- The Family Planning High Impact Practices cuation rates were also achieved when brief on postabortion care summarized the results misoprostol was expanded to secondary and of 15 studies to strengthen postabortion family Postabortion district hospitals and health posts that had planning in 14 countries.39 Postabortion family family planning no previous capability to treat incomplete 17,23,25,28,29 planning uptake before discharge rose by uptake before abortion. 30–70 percentage points within 1 to 3 years in discharge Pain management with ibuprofen, 800 mg to several studies, sometimes increasing by as much increased by 1000 mg, was generally adequate and clearly as 60 percentage points in 4 months.4,39 The inter- 30–70 percentage superior to paracetamol (acetaminophen).35-37 ventions providing postabortion family planning points in 1–3 years Routine ultrasound was not necessary to counseling and services at the same time and when family diagnose incomplete abortion or confirm location as treatment rapidly increased immedi- planning services complete uterine evacuation when misopros- ate postabortion contraceptive acceptance. They included a wide tol was used at less than 12 weeks gestation; found an average 37 percentage point increase of range of methods clinical judgment was adequate for the large women receiving a method before discharge, provided at the majority of cases.14,17,20 from 32% before the intervention (range, 1% to 65%) to 69% post-intervention (range, 25% to same time and Two studies noted that ultrasound available 98%).39 Other post-intervention studies reported location as through referral was helpful for establishing even greater family planning uptake.40 Counsel- treatment. ectopic pregnancy, determining gestational ing and family planning uptake continued to age, and confirming complete evacuation for increase in one Peru study over 3 years after a few cases.17 Typically Z90% of women in support for the intervention stopped, a demon- Burkina Faso and Senegal were managed stration of sustainability.41 One study in Zimbabwe entirely at the primary point of care without comparing a postabortion family planning inter- referral for ultrasound.14,17,20 However, these vention at one hospital with a control hospital studies contained few sites. with no special intervention reported significantly Mid-level clinicians can correctly diagnose greater family planning uptake in the inter- and treat incomplete abortion with misoprostol vention hospital, as well as significantly fewer and accurately determine complete evacua- unintended pregnancies and repeat abortions; the 23,25,28 tion. Expanding the scope of practice for study enrolled 982 women in total, 527 of whom mid-level providers for counseling and provi- were followed for 12 months.42 sion of misoprostol was found to be safe, Multiple studies (published between 1998 17,18,21 effective, and acceptable. and 2010) from 9 countries show that training providers in family planning counseling, service delivery, and MVA increases family planning Postabortion Family Planning Counseling uptake.39,41,43-46 Other factors contributing to and Services success were continuing education in family Although emergency treatment for incomplete planning counseling skills, provider job aids, abortion often receives greater attention than and client education materials.47,48 Placing com- postabortion family planning, the family planning modities at the point of treatment such as in a

Global Health: Science and Practice 2016 | Volume 4 | Number 3 486 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

wall cabinet in the treatment room also increased 706 women as part of PAC services resulted in performance, as did supportive supervision.47,49 58% of the women accepting HIV testing; 14% of After women gave permission for including these women had HIV. HIV testing in conjunction their husbands in counseling, family planning use with PAC was found to be acceptable and relevant Couples increased. Counseling covered follow-up care, rapid in this high-risk setting.66 counseling, with return to fertility, and family planning methods. We did not find documentation that PAC approval from the Men also provided physical, emotional, and mate- services routinely included counseling on STI and woman, improved rial support for PAC clients during recovery. Hus- HIV risk factors and provided condoms for family planning bands who were counseled were 60% more likely to prevention. One study identified this as an unmet uptake and 67 provide a high level of support for family planning need. No studies were found that documented support. use;30%werealsomorelikelytoprovideahigh both diagnosis and treatment for STIs and HIV in level of emotional support.50-53 relation to PAC. Providing all contraceptive methods free to PAC Providing all clients greatly contributed to the increased uptake Other Counseling Concerns contraceptive of postabortion family planning prior to discharge Women who have undergone induced or sponta- methods free 42,46,48,49,54 from the facility. This is a key considera- neous abortion may require special counseling greatly increased tion for governments, ministries of health, and considerations. Increased risk of spontaneous uptake of policy makers. Many women are not able to pay for abortion is associated with several common postabortion contraceptives at the time of treatment. conditions, including malaria, HIV/AIDS, STIs, family planning Delaying the next pregnancy for at least gender-based violence, smoking, excess alcohol because many 6 months after miscarriage or abortion reduces use, exposure to pesticides, and previous sponta- women cannot the incidence of preterm delivery, low birth neous abortion.68-75 Approximately one-third of pay at the time of weight, premature rupture of membranes, and women seeking abortion have been victims of treatment. maternal anemia.55 Therefore, it is particularly abuse sometime during their lifetime.73,76 Sig- important to stress the need for spacing after nificant anxiety is experienced by 40% to 45% of spontaneous abortion when women may desire women before an induced abortion. Distress is About one-third of another pregnancy soon. Advising every woman reduced after the procedure, although about one- women seeking to delay her next pregnancy also removes the fourth may report anxiety and depression from abortion have potential stigma that accepting family planning 1 month to 2 years after the event. The long-term been victims of implies her abortion was induced. psychological outcomes are similar between abuse sometime Insertion of an intrauterine device (IUD) women who have had an induced abortion and during their 5,77,78 immediately after a first-trimester abortion yield- those who have given birth. The short-term lifetime. ed few expulsions and carried no increased risk emotional reactions to miscarriage appear to be as of perforation or infection. There was only a large or larger than those to induced abortion.5,6 minimal increase in expulsion rates over delayed Most psychological outcome studies, however, insertion.56,57 With misoprostol treatment, the have been conducted in developed countries, so it is IUD is usually inserted at the 1-week follow-up difficult to apply the findings to low-resource visit provided that uterine evacuation is com- settings. Some of women’s counseling needs may plete.57 Delayed insertion may increase the like- be amenable to community education and inter- lihood of failing to return for the planned IUD vention as part of community health worker (CHW) insertion, thus placing women at higher risk of activities. However, studies are not yet available to unintended pregnancy.58 document interventions and effectiveness. Several studies found that women receiving A substantial percentage (40%) of PAC clients immediate postabortion LARCs had fewer sub- are under the age of 25, yet they account for sequent unintended pregnancies and repeat almost half of the deaths from unsafe abortion.79 abortions than those who were offered delayed Factors contributing to maternal deaths in young insertions (3 to 6 weeks post-procedure).56,59,60 PAC clients include not seeking PAC services soon enough due to late recognition of complications, HIV Testing and STI Prevention delay due to stigma in the community, lack of Women with HIV who are symptomatic are at an funds for transport and services, and delay due to increased risk for spontaneous abortion.61-65 Few provider bias and attitude to young clients.80,81 studies have been done in this area of PAC Skilled counseling is especially important for and HIV, although some show promising results. younger women whose first interaction with the For example, in Tanzania offering HIV testing to health system may be for postabortion care.1

Global Health: Science and Practice 2016 | Volume 4 | Number 3 487 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

Community Awareness, Mobilization, and In Nepal, technical support service centers Empowerment were crucial for institutionalizing comprehensive The Community Action Cycle—a participatory PAC services. Resistance by doctors and hospital problem-solving approach involving community management to nurses providing MVA was partly diagnosis, planning, implementation, participa- overcome through formal and informal orienta- tory evaluation, and scale-up—can increase tions for doctors. Structured on-the-job training awareness of PAC services, especially through was cost-effective for maintaining PAC skills and training of CHWs to provide postabortion family strengthening services. PAC program data were planning counseling and services, including refer- difficult to collect as they were not included in the ral to facilities.82 In Kenya, CHWs were instru- Ministry of Health and Population’s regular mental in rapidly increasing access to essential health management information system. Chal- PAC services by providing contraceptives to 63% lenges included the frequent transfer of trained of users and referring 90% of PAC clients to PAC service providers; unavailability of family health facilities.83,84 The Community Action Cycle planning services to women having sharp also helped educate communities on recognition curettage procedures; insufficient caseload at of danger signs during pregnancy, such as some sites to support group-based training; and bleeding, to encourage health-seeking behavior irregular supply and replacement of MVA equip- for PAC.85 However, major gaps remain in knowl- ment.86 edge of linkages between community mobiliza- Guatemala expanded its PAC program to 22 of tion and PAC services. the 33 public district hospitals. The 22 district hospitals increased postabortion family planning counseling from 31% to 78% over 18 months; Policy, Programs, and Systems postabortion family planning acceptance before Wide Replication of PAC Including Essential discharge rose from 20% to 49%; and vacuum Organization of Services aspiration increased from 38% to 68%.49 In several countrywide PAC programs, postabor- Tanzania achieved high performance when tion family planning is emerging as a strong initiating PAC with MVA, expanding from 11 to program component. 229 sites between 2005 and 2010. In 2010, the Nepal was one of the first countries to widely PAC program served 9,563 PAC clients, with 84% replicate PAC programming and document valu- of women being counseled on family planning able lessons learned during the expansion phase. and 82% accepting a method.2 Senegal also The lessons are especially relevant as the chal- rapidly expanded services in 23 of 56 districts, lenges are similar throughout most PAC pro- covering 60% of the population by 2006. Formal grams. In 1995, Nepal first initiated PAC services PAC training was given to 523 providers from at the Maternity Hospital in Kathmandu. It then 323 health facilities. Health centers with ade- expanded services to 78 sites in 50 districts quate space, materials, and equipment increased mainly in public-sector health facilities.85 The from 29% to 95% between 2003 and 2006, program focused on (1) replacing sharp curettage and facilities with reorganized services capable with MVA by establishing PAC training centers at of providing 24/7 PAC services rose from 55% the district level using group-based and on-the- to 84%.87 job training, and (2) training staff nurses and In replicating PAC services, Rwanda engaged senior auxiliary nurse-midwives as primary pro- lower-level health facilities to provide PAC, viders, thereby expanding access to primary including misoprostol, and increased use of health centers and linking PAC with family plan- family planning services. Before the pilot pro- ning services. Key achievements included train- gram, only 2 of 50 health centers were providing ing more than 170 providers in PAC and starting PAC services; all other health centers referred 46 new PAC service sites in primary health PAC clients to hospitals. Within a year, 91% of centers with task shifting for most of PAC services PAC clients were treated at health centers and from doctors to nurses. This resulted in the only 9% were referred to hospitals. Overall, increase of MVA use from 47% to 74% and family misoprostol was used to treat 83% of PAC clients, planning uptake of 81% (a high percentage and 59% of women were discharged with a compared with most programs although baseline contraceptive method.29 A system of data collec- data were not available), with injectables being tion and monitoring was implemented to inform the most popular method.86 the scale-up, providing a good example of the

Global Health: Science and Practice 2016 | Volume 4 | Number 3 488 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

inclusion of family planning uptake in the health lacking; this is a major area for further research management information system.29 Wide replica- and documentation. tion of PAC in multiple sites is a strong step toward sustainable and scaled-up services. These are not as well documented in the literature. PAC PROGRAM AND PRACTICE RECOMMENDATIONS Task Shifting The implications for practice flow from PAC An expanded scope of practice for mid-level research documenting practices that impact providers was assessed in a number of studies health outcomes and that are practical and cost- providing PAC service delivery. These studies efficient to implement. These include emergency confirm the safety, effectiveness, and acceptability treatment of incomplete abortion and postabor- of trained nurses and midwives as providers of tion family planning. MVA, affirming their ability to accurately deter- 23,28,88,89 mine complete evacuation. Emergency Treatment of Incomplete Abortion Costs Medical (misoprostol) and surgical management Twelve studies compared costs for PAC services (electric or manual vacuum aspiration) have been using MVA versus sharp curettage. There was proven to offer safer options over sharp curettage Using MVA instead strong evidence that using MVA for PAC instead for treatment of incomplete abortion. In countries of sharp curettage of sharp curettage along with changes in proto- where both misoprostol and vacuum aspiration can signiﬁcantly cols and an improved service delivery model can are authorized, women should be informed of reduce costs of 4,5 significantly reduce costs of care in most cases. appropriate treatment options. Programs should care in most cases. In-patient hospital and personnel costs were the ensure that both are available, staff are trained, main drives of cost for PAC service delivery. and communities are informed, including CHWs. Interventions to reduce costs included4,5: New protocols and training curricula for vacuum aspiration and medical evacuation, for example, Changes in protocol to the standard use of from USAID and Ipas, incorporate the findings of MVA with local anesthesia if the uterus was recent research and will facilitate revised prac- smaller than 12 weeks’ size tices.26,27,97,98 These are important instruments Providing PAC as an ambulatory outpatient for ensuring quality and competence of providers procedure and programs. Provider training on information and counseling (on health status, uterine evacuation Postabortion Family Planning procedure, postabortion contraception, and The voluntary informed choice of whether to use care after leaving the hospital) contraception must always be solely that of the Refresher training and supportive supervision woman. Asking the woman about her reproduc- A number of cost studies document the high tive intentions and providing counseling for all cost of treating abortion complications, the value methods (short-acting methods, LARCs, and of expanding PAC services with a robust family permanent methods) will promote informed and planning component, and the substantial savings voluntary choice for her; her health and that of that could be achieved by preventing unintended her next pregnancy will benefit. Several studies have found an urgent need to improve content pregnancies and abortion through improved All postabortion 90-95 and quality of client counseling. The most impor- contraceptive services. Immediate postabor- women should tion insertion of an IUD after surgical evacuation tant recommendations emerging from PAC studies receive family offers special promise of cost-efficient, high are to offer all women family planning counseling planning contraceptive continuation with a very effective and services before they leave the facility regard- counseling and method.96 less of the method of uterine evacuation, and to complement counseling with written instructions services before One study also made the case that greater use on: they leave the of misoprostol for PAC could free surgical facility. resources to be used for more complicated cases.34 1. Her chosen contraceptive method—including However, good costing studies on use of mis- how to use the method, what to expect, and oprostol compared with surgical evacuation are how to resupply

Global Health: Science and Practice 2016 | Volume 4 | Number 3 489 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

2. The rapid return of fertility after the abortion organizations, major donors, and maternal health (within 2 to 3 weeks) networks.1 In parallel, this alliance strongly advocates universal access to postpartum family 3. The signs and symptoms of postabortion com- 38 plications (e.g., heavy bleeding, fever, increasing planning. Together these advocacy and policy pain) that would need medical attention positions create a foundation for universal access to family planning in the postpartum and post- These actions are endorsed by a joint con- abortion periods, with major implications for sensus statement by the professional organiza- policy, practice, and training, both in-service tions of obstetricians/gynecologists, midwives, and preservice. and nurses along with major donors.1 The International Confederation of Midwives Acknowledgments: We gratefully acknowledge support and (ICM) recently included PAC and postabortion encouragement from the Postabortion Family Planning Project, an ‘‘ Annual Program Statement awarded to EngenderHealth Inc., by the family planning as basic skills in their Essential United States Agency for International Development. Competencies for Basic Midwifery Practices’’.99 Postabortion family planning should likewise Competing Interests: None declared. be incorporated into emergency obstetrical care (EmOC) indicators for managing incomplete abortion.100 REFERENCES Health management information systems 1. International Federation of Gynecology and Obstetrics; need to universally report postabortion con- International Confederation of Midwives; International Council of Nurses; United States Agency for International Development; traceptive services as an essential measure of White Ribbon Alliance; Department for International program and provider performance. Documenta- Development; et al. Post abortion family planning: a key tion should include the percentage of postabor- component of post abortion care. Washington, DC ; 2013. tion clients receiving contraceptives by method Available from: https://www.glowm.com/pdf/PAC-FP-Joint- Statement-November2013-final_printquality.pdf before leaving the facility and at return visits. 2. Fullerton JT, Ottolenghi E. Global postabortion care desk review. Documentation of method-specific uptake in Washington (DC): The Global Health Technical Assistance the postabortion period needs a special focus Project; 2012. when using misoprostol. Many of the misoprostol 3. United Nations (UN). Report of the International Conference on studies failed to report on this key component of Population and Development, Cairo, 5-13 September 1994. PAC. LARCs deserve special emphasis, given that New York: UN; 1995 Available from: http://www.unfpa.org/ sites/default/files/event-pdf/icpd_eng_2.pdf postabortion IUD insertion must be delayed until 4. United States Agency for International Development (USAID), complete evacuation after misoprostol. Postabortion Care Working Group. What works: a policy and program guide to the evidence on postabortion care. Washington (DC): USAID; 2007. Available from: http://www. POLICY, ADVOCACY, AND FUTURE postabortioncare.org/sites/pac/files/Compendium.pdf RESEARCH 5. United States Agency for International Development, Postabortion Care Working Group. What works: a policy Among the many topics for future research, two and program guide to the evidence on postabortion care. especially stand out. Cost studies are needed to 2nd edition. Washington (DC): USAID. Forthcoming 2016. better assess the potential benefits of misoprostol in 6. Curtis C, Huber D, Moss-Knight T. Postabortion family planning: addressing the cycle of repeat unintended pregnancy and comparison with both vacuum aspiration and sharp abortion. Int Perspect Sex Reprod Health. 2010;36(1):44–48. curettage, especially given that sharp curettage is CrossRef. Medline still widely used. Better evaluation is needed on the 7. Gray J. Evidence-based health care and public health: how to effects of misoprostol on expanding the scope of make decisions about health services and public health. practice for nurses and midwives, frequency of Edinburgh (Scotland): Churchill Livingston Elsevier; 2009. surgical procedures, use of lower-level facilities, and 8. Renner RM, Jensen JT, Nichols MD, Edelman AB. Pain control in first-trimester surgical abortion: a systematic review of community engagement. randomized controlled trials. Contraception. 2010;81(5): For women wanting to use the IUD postabor- 372–388. CrossRef. Medline tion, there may be benefits to offering immediate 9. Go´mez PI, Gaitań H, Nova C, Paradas A. Paracervical block in postabortion insertion with vacuum aspiration as incomplete abortion using manual vacuum aspiration: an alternative to delayed insertion with miso- randomized clinical trial. Obstet Gynecol. 2004;103 (5 Pt 1):943–951. CrossRef. Medline prostol and the lost-to-follow-up that inherently occurs—an opportunity for research. 10. Lo´pez JC, Vigil-De Gracia P, Vega-Malek JC, Ruiz E, Vergara V. A randomized comparison of different methods of analgesia in Advocacy for universal access to postabortion abortion using manual vacuum aspiration. Int J Gynaecol family planning is supported by health professional Obstet. 2007;99(2):91–94. CrossRef. Medline

Global Health: Science and Practice 2016 | Volume 4 | Number 3 490 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

11. Shearer JC, Walker DG, Vlassoff M. Costs of post-abortion care 27. Gynuity Health Projects; Ipas. Misoprostol for treatment of in low- and middle-income countries. Int J Gynaecol Obstet. incomplete abortion: training guide. New York: Gynuity; 2012. 2010;108(2):165–169. CrossRef. Medline Available from: http://gynuity.org/resources/info/ 12. Langer A, Heimburger A, Garcıá-Barrios C, Winikoff B. Improving misoprostol-for-treatment-of-incomplete-abortion-a-training- postabortion care in a public hospital in Mexico. In : Haberland N, guide/ Measham M, editors. Responding to Cairo: case studies of 28. Fawole AO, Diop A, Adeyanju AO, Aremu OT, Winikoff B. changing practice in reproductive health and family planning. Misoprostol as first-line treatment for incomplete abortion at a New York: Population Council; 2002. p. 236–256. secondary-level health facility in Nigeria. Int J Gynaecol Obstet. 13. Kizza AP, Rogo KO. Assessment of the manual vacuum 2012;119(2):170–173. CrossRef. Medline aspiration (MVA) equipment in the management of incomplete 29. Republic of Rwanda, Ministry of Health (MOH); Venture abortion. East Afr Med J. 1990;67(11):812–822. Medline Strategies Innovations. Expanding access to postabortion care 14. Weeks A, Alia G, Blum J, Winikoff B, Ekwaru P, Durocher J, Services in Rwanda: final report. Kigali (Rwanda): MOH; 2013. et al. A randomized trial of misoprostol compared with manual Available from: http://bixby.berkeley.edu/wp-content/ vacuum aspiration for incomplete abortion. Obstet Gynecol. uploads/2015/03/VSI_Rwanda-MOH-PAC-Report-2013-02- 2005;106(3):540–547. CrossRef. Medline 20F.pdf 15. May W, Gu¨lmezoglu AM, Ba-Thike K. Antibiotics for incomplete 30. Zimbabwe Ministry of Health and Child Care; Venture abortion. Cochrane Database Syst Rev. 2007;(4):CD001779. Strategies Innovations. Expanding access to postabortion care CrossRef. Medline in Zimbabwe through the integration of misoprostol: final 16. Trussell J, Nucatola D, Fjerstad M, Lichtenberg ES. Reduction in report. Harare (Zimbabwe): The Ministry; 2013. Available infection-related mortality since modifications in the regimen of from: http://bixby.berkeley.edu/wp-content/uploads/2015/ medical abortion. Contraception. 2014;89(3):193–196. 03/VSI_Zim_MOH_PAC-Report-2013-10-14F.pdf CrossRef. Medline 31. Venture Strategies Innovations (VSI). Expanding access to 17. Blandine T, Ouattara AZ, Coral A, et al. Sublingual [corrected] postabortion care services in Angola with the introduction of misoprostol as first-line care for incomplete abortion in Burkina misoprostol; final report in brief. Anaheim (CA): VSI; 2013. Faso. Int J Gynaecol Obstet. 2012;119(2):166–169. CrossRef. Available from: http://bixby.berkeley.edu/publication/ Medline expanding-access-to-postabortion-care-services-in-angola- with-the-introduction-of-misoprostol/ 18. Dao B, Blum J, Thieba B, Raghavan S, Ouedraego M, Lankoande J, et al. Is misoprostol a safe, effective and acceptable alternative 32. Smith PS. Applying the national norms: integration of to manual vacuum aspiration for postabortion care? Results from misoprostol into postabortion care. The Mexico City experience. a randomised trial in Burkina Faso, West Africa. BJOG. Secretaria de Salud del Distrito Federal (SSDF), Mexico. PAC 2007;114(11):1368–1375. CrossRef. Medline Consortium Webinar; 2013 Dec 3. Available from: http://pac- 19. Taylor J, Diop A, Blum J, Dolo O, Winikoff B. Oral misoprostol consortium.org/downloads/PSanhueza%20webinar% as an alternative to surgical management for incomplete 203dec13.pdf abortion in Ghana. Int J Gynaecol Obstet. 2011;112(1):40–44. 33. Ngoc NT, Shochet T, Blum J, Hai PT, Dung DL, Nhan TT, et al. CrossRef. Medline Results from a study using misoprostol for management of 20. Shwekerela B, Kalumuna R, Kipingili R, Mashaka N, Westheimer incomplete abortion in Vietnamese hospitals: implications for E, Clark W, et al. Misoprostol for treatment of incomplete abortion task shifting. BMC Pregnancy Childbirth. 2013;13:118. at the regional hospital level: results from Tanzania. BJOG. CrossRef. Medline 2007;114(11):1363–1367. CrossRef. Medline 34.DahT,AkiodeA,AwahP,FettersT,OkohM,UjahI,etal. 21. Bique C, Usta´ M, Debora B, Chong E, Westheimer E, Winikoff Introducing misoprostol for the treatment of incomplete B. Comparison of misoprostol and manual vacuum aspiration abortion in Nigeria. Afr J Reprod Health. 2011;15(4):42–50. for the treatment of incomplete abortion. Int J Gynaecol Obstet. Medline 2007;98(3):222–226. CrossRef. Medline 35. Avraham S, Gat I, Duvdevani NR, Haas J, Frenkel Y, Seidman 22. Montesinos R, Durocher J, Leoń W, Arellano M, Penã M, Pinto DS. Pre-emptive effect of ibuprofen versus placebo on pain relief E, et al. Oral misoprostol for the management of incomplete and success rates of medical abortion: a double-blind, abortion in Ecuador. Int J Gynaecol Obstet. 2011;115(2): randomized, controlled study. Fertil Steril. 2012;97(3): 135–139. CrossRef. Medline 612–615. CrossRef. Medline 23. Shochet T, Diop A, Gaye A, Nayama M, Sall AB, Bukola F, et al. 36. Livshits A, Machtinger R, David LB, Spira M, Moshe-Zahav A, Sublingual misoprostol versus standard surgical care for Seidman DS. Ibuprofen and paracetamol for pain relief treatment of incomplete abortion in five sub-Saharan African during medical abortion: a double-blind randomized countries. BMC Pregnancy Childbirth. 2012;12(1):127. controlled study. Fertil Steril. 2009;91(5):1877–1880. CrossRef. Medline CrossRef. Medline 24. Blanchard K, Taneepanichskul S, Kiriwat O, Sirimai K, 37. Jackson E, Kapp N. Pain control in first-trimester and second- Svirirojana N, Mavimbela N, et al. Two regimes of misoprostol trimester medical termination of pregnancy: a systematic review. for treatment of incomplete abortion. Obstet Gynecol. Contraception. 2011;83(2):116–126. CrossRef. Medline 2004;103(5 Pt 1):860–865. CrossRef. Medline 38. World Health Organization (WHO), Human Reproduction 25. Gaye A, Diop A, Shochet T, Winikoff B. Decentralizing Programme. Statement for collective action for postpartum postabortion care in Senegal with misoprostol for incomplete family planning. Geneva: WHO; 2013. Available from: abortion. Int J Gynaecol Obstet. 2014;126(3):223–226. http://www.who.int/reproductivehealth/publications/ CrossRef. Medline family_planning/statmt_ppfp/en/ 26. Postabortion Care Consortium [Internet]. New York: 39. High Impact Practice in Family Planning (HIP). Postabortion EngenderHealth; c2015. Misoprostol; [cited 2015 Aug 9] family planning: strengthening the family planning Available from: http://pac-consortium.org/resources/ component of postabortion care. Washington (DC): United misoprostol/ States Agency for International Development; 2012.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 491 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

Available from: https://www.fphighimpactpractices.org/ 53. Rasch V, Lyaruu MA. Unsafe abortion in Tanzania and the need resources/postabortion-family-planning-strengthening-family- for involving men in postabortion contraceptive counseling. Stud planning-component-postabortion-care Fam Plann. 2005;36(4):301–310. CrossRef. Medline 40. Tripney J, Kwan I, Bird KS. Postabortion family planning 54. Carneiro Gomes Ferreira AL, Impieri Souza A, Evangelista counseling and services for women in low-income countries: Pessoa R, Braga C. The effectiveness of contraceptive counseling a systematic review. Contraception. 2013;87(1):17–25. for women in the postabortion period: an intervention study. CrossRef. Medline Contraception. 2011;84(4):377–383. CrossRef. Medline 41. Benson J, Huapaya V. Sustainability of postabortion care in 55. World Health Organization (WHO). Report of a WHO technical Peru. Washington (DC): Population Council, Frontiers in consultation on birth spacing: Geneva, Switzerland, 13–15 Reproductive Health; 2002. Available from: http://citeseerx.ist. June 2005. Geneva: WHO; 2005. Available from: psu.edu/viewdoc/download?doi=10.1.1.553.3045&rep= http://www.who.int/maternal_child_adolescent/documents/ rep1&type=pdf birth_spacing05/en/ 42. Johnson BR, Ndhlovu S, Farr SL, Chipato T. Reducing unplanned 56. Grimes DA, Lopez LM, Schulz KF, Stanwood NL. Immediate pregnancy and abortion in Zimbabwe through postabortion postabortal insertion of intrauterine devices. Cochrane contraception. Stud Fam Plann. 2002;33(2):195–202. Database Syst Rev. 2010;(6):CD001777. CrossRef. Medline CrossRef. Medline 57. Betstadt SJ, Turok DK, Kapp N, Feng KT, Borgatta L. Intrauterine 43. Dabash R. Taking postabortion care services where they device insertion after medical abortion. Contraception. 2011; are needed: an operations research project testing PAC 83(6):517–521. CrossRef. Medline expansion in rural Senegal. New York: Engenderhealth; 2003. 58. Stanwood NL, Grimes DA, Schulz KF. Insertion of an Available from: http://pdf.usaid.gov/pdf_docs/Pnacu603.pdf intrauterine contraceptive device after induced or spontaneous 44. Solo J, Billings DL, Aloo-Obunga C, Ominde A, Makumi M. abortion: a review of the evidence. BJOG. 2001;108(11): Creating linkages between incomplete abortion treatment and 1168–1173. CrossRef. Medline family planning services in Kenya. Stud Fam Plann. 1999; 59. Ames CM, Norman WV. Preventing repeat abortion in Canada: is 30(1):17–27. CrossRef. Medline the immediate insertion of intrauterine devices postabortion a cost- 45. Ministry of Health. Burkina Faso. Introduction of emergency effective option associated with fewer repeat abortions? medical treatment and family planning services for women with Contraception. 2012;85(1):51–55. CrossRef. Medline complications from abortion in Burkina Faso. Nairobi (Kenya): 60. Cremer M, Bullard KA, Mosley RM, Weiselberg C, Molaei M, Population Council; 1998. Lerner V, Alonzo TA, et al. Immediate vs. delayed post-abortal 46. Zhu JL, Zhang WH, Cheng Y, Xu J, Xu X, Gibson D, et al. Impact copper T 380A IUD insertion in cases over 12 weeks of gestation. of post-abortion family planning services on contraceptive use Contraception. 2011;83(6):522–527. CrossRef. Medline and abortion rate among young women in China: a cluster 61. Feldman R, Maposhere C. Safer sex and reproductive choice: randomised trial. Eur J Contracept Reprod Health Care. findings from ‘‘positive women: voices and choices’’ in 2009;14(1):46–54. CrossRef. Medline Zimbabwe. Reprod Health Matters. 2003;11(22):162–173. 47. Stolarsky G, Peshkatari A, Charurat E. An evaluation of CrossRef. Medline ACCESS-FP’s work in Albania: postpartum and postabortion 62. Miotti PG, Dallabetta G, Ndovi E, Liomba G, Saah AJ, family planning. Baltimore (MD): Jhpiego; 2010. Available Chiphangwi J. HIV-1 and pregnant women: associated factors, from: https://www.k4health.org/sites/default/files/Albania% prevalence, estimate of incidence and role in fetal wastage in 20Evaluation%20Report_6-8-10Final_0.pdf central Africa. AIDS. 1990;4(8):733–736. CrossRef. Medline 48. Savelieva I, Pile J, Sacci I, Loganathan R. Postabortion family 63. Kumar RM, Uduman SA, Khurranna AK. Impact of maternal planning operations research study in Perm, Russia. HIV-1 infection on perinatal outcome. Int J Gynaecol Obstet. Washington (DC): Population Council, Frontiers in Reproductive 1995;49(2):137–143. CrossRef. Medline Health; 2003. Available from: http://pdf.usaid.gov/pdf_docs/ Pnacu602.pdf 64. Berer M. HIV/AIDS, pregnancy and maternal mortality and morbidity: implications for care. In: Berer M, Ravindran TKS, 49. Kestler E, Valencia L, Del Valle V, Silva A. Scaling up post-abortion editors. Safe motherhood initiatives: critical issues. care in Guatemala: initial successes at national level. Reprod Health (Reproductive Health Matters). Oxford: Blackwell Science; Matters. 2006;14(27):138–147. CrossRef. Medline 1999. p. 198–210. 50. Abdel-Tawab N, Huntington D, Osman Hassan E, Youssef H, 65. Brocklehurst P, French R. The association between maternal HIV Nawar L. Effects of husband involvement on postabortion infection and perinatal outcome: a systematic review of the patients’ recovery and use of contraception in Egypt. In: literature and meta-analysis. BJOG. 1998;105(8):836–848. Piet-Pelon NJ,Huntington D, editors. Postabortion care: lessons CrossRef. Medline from operations research. New York: Population Council; 1999. p. 16–37. 66. Rasch V, Yambesi F, Massawe S. Post-abortion care and voluntary HIV counselling and testing--an example of 51. Abdel-Tawab N, Huntington D, Osman Hassan E, Youssef H, integrating HIV prevention into reproductive health Nawar L. Recovery from abortion and miscarriage in Egypt: services. Trop Med Int Health. 2006;11(5):697–704. does counseling husbands help? In : Haberland D, Measham A, CrossRef. Medline editors. Responding to Cairo: case studies of changing practice in reproductive health and family Planning. New York: 67. Dickson-Tetteh K, Billings DL. Abortion care services provided by Population Council; 2002. p. 186–204. registered midwives in South Africa. Int Fam Plan Perspect. 2002;28(3):144–147. CrossRef 52. Centre de Formation et de Recherche en Sante´ de la Reproduction et Clinique Gynećologique et Obste´tricale. 68. McGready R, Lee SJ, Wiladphaingern J, Ashley EA, Rijken MJ, Introduction des soins obste´tricaux d’urgence et de la Boel M, et al. Adverse effects of falciparum and vivax malaria planification familiale pour les patientes pre´sentant des and the safety of antimalarial treatment in early pregnancy: complications lieés a` un avortement incomplet. Dakar (Senegal): a population-based study. Lancet Infect Dis. 2012;12(5): Population Council; 1998. 388–396. CrossRef. Medline

Global Health: Science and Practice 2016 | Volume 4 | Number 3 492 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

69. Taha T el-T, Gray RH. Malaria and perinatal mortality in central 84. Muganda R, Mukenge M, Ochieng J, Magak K, Ombaka C, Sudan. Am J Epidemiol. 1993;138(8):563–568. Medline Oguttu M, et al.. Community-based abortion care: final 70. Bloom MS, Fitzgerald EF, Kim K, Neamtiu I, Gurzau ES. evaluation findings from Western Kenya. Presented at: 132nd Spontaneous pregnancy loss in humans and exposure to arsenic APHA Annual Meeting and Exposition; 2004 Nov 6-10, in drinking water. Int J Hyg Environ Health. 2010;213(6): Washington, DC. Available from: https://apha.confex.com/ 401–413. CrossRef. Medline apha/132am/techprogram/paper_90097.htm 71. Blanco-Munõz J, Torres-Sańchez L, Lo´pez-Carrillo L. Exposure to 85. Wickstrom J, Russel N, Escandon I. Engaging communities as maternal and paternal tobacco consumption and risk of spontaneous partners in postabortion care: a desk review of community abortion. Public Health Rep. 2009;124(2):317–322. Medline postabortion care project in Nakuru, Kenya. New York: EngenderHealth, The ACQUIRE Project; 2008. Available from: 72. Kashanian M, Akbarian AR, Baradaran H, Shabandoust SH. http://www.acquireproject.org/archive/files/ Pregnancy outcome following a previous spontaneous abortion 11.0_research_studies/er_study_11.pdf (miscarriage). Gynecol Obstet Invest. 2006;61(3):167–170. CrossRef. Medline 86. JSI Research & Training Institute, Inc., Nepal Family Health Program II: final project report: December 2007-November 73. Leung TW, Leung WC, Chan PL, Ho PC. A comparison of the 2012. Arlington (VA): JSI Research & Training Institute; 2012. prevalence of domestic violence between patients seeking Available from: http://nfhp.jsi.com/Docs/ termination of pregnancy and other general gynecology patients. NFHPIIFinalProjectReport.pdf Int J Gynaecol Obstet. 2002;77(1):47–54. CrossRef. Medline 87. Thiam FT, Suh S. Scaling-up postabortion care services: results 74. Pallitto CC, Garcıá-Moreno C, Jansen HAFM, Heise L, Ellsberg from 23 health districts in Senegal. Cambridge (MA): ’ M, Watts C; WHO Multi-Country Study on Women s Health and Management Sciences for Health; 2006. Available from: Domestic Violence. Intimate partner violence, abortion, and https://www.msh.org/sites/msh.org/files/scaling-up- unintended pregnancy: results from the WHO multi-country postabortion-care-services-results-from-senegal.pdf study on women’s health and domestic violence. Int J Gynaecol Obstet. 2013;120(1):3–9. CrossRef. Medline 88. Akiode A, Fetters T, Daroda R, Okeke B, Oji E. An evaluation of a national intervention to improve the postabortion care content 75. Sto¨ckl H, Filippi V, Watts C, Mbwambo JKK. Induced abortion, of midwifery education in Nigeria. Int J Gynaecol Obstet. pregnancy loss and intimate partner violence in Tanzania: 2010;110(2):186–190. CrossRef. Medline a population based study. BMC Pregnancy Childbirth. 2012;12:12. CrossRef. Medline 89. Gebreselassie H, Usta´ M, Andersen KL, Mitchell EMH. Clinical diagnosis of completeness of medical abortion by nurses: 76. Kaye DK, Mirembe FM, Bantebya G, Johansson A, Ekstrom a reliability study in Mozambique. Contraception. 2012; AM. Domestic violence as risk factor for unwanted pregnancy 86(1):74–78. CrossRef. Medline and induced abortion in Mulago Hospital, Kampala, Uganda. Trop Med Int Health. 2006;11(1):90–101. CrossRef. Medline 90. Vlassoff M, Mugisha F, Sundaram A, Bankole A, Singh S, Amanya L, et al. The health system cost of postabortion care in 77. Bradshaw Z, Slade P. The effects of induced abortion on Uganda. Health Policy Plan. 2014;29(1):56–66. CrossRef. emotional experiences and relationships: a critical review of the Medline literature. Clin Psychol Rev. 2003;23(7):929–958. CrossRef. Medline 91. Billings DL, Benson J. Postabortion care in Latin America: policy and service recommendations from a decade of operations research. 78. Kishida Y. Anxiety in Japanese women after elective abortion. Health Policy Plan. 2005;20(3):158–166. CrossRef. Medline J Obstet Gynecol Neonatal Nurs. 2001;30(5):490–495. CrossRef. Medline 92. Benson J, Okoh M, KrennHrubec K, Lazzarino MAM, Johnston HB. Public hospital costs of treatment of abortion complications 79. World Health Organization (WHO). Safe abortion: technical in Nigeria. Int J Gynaecol Obstet. 2012;118(Suppl 2):S134– and policy guidance for health systems. Geneva: WHO; 2003. S140. CrossRef. Medline Available from: http://www.who.int/reproductivehealth/ publications/unsafe_abortion/9789241548434/en/ 93. Vlassoff M, Fetters T, Kumbi S, Singh S. The health system cost of postabortion care in Ethiopia. Int J Gynaecol Obstet. 2012; 80. World Health Organization (WHO). Unsafe abortion: global 118 (Suppl 2):S127–S133. CrossRef. Medline and regional estimates of incidence of unsafe abortion and associated mortality in 2008. 6th edition. Geneva: WHO; 94. Vlassoff M, Musange SF, Kalisa IR, Ngabo F, Sayinzoga F, Singh 2011. Available from: http://www.who.int/ S, et al. The health system cost of post-abortion care in Rwanda. reproductivehealth/publications/unsafe_abortion/ Health Policy Plan. 2015;30(2):223–233. CrossRef. Medline 9789241501118/en/ 95. Billings DL, Crane BB, Benson J, Solo J, Fetters T. Scaling-up a 81. Ganges F, Curtis C. Postabortion care curriculum: reference public health innovation: a comparative study of post-abortion manual. Baltimore (MD): Jhpiego; 2010. Available from: care in Bolivia and Mexico. Soc Sci Med. 2007;64(11):2210– http://www.postabortioncare.org/sites/pac/files/ 2222. CrossRef. Medline PAC_Manual.pdf 96. Okusanya BO, Oduwole O, Effa EE. Immediate postabortal 82. Community Mobilization for Postabortion Care (COMMPAC) insertion of intrauterine devices. Cochrane Database Syst Rev. Kenya. A guide to action for community mobilization and 2014;7(7):CD001777. CrossRef. Medline empowerment focused on postabortion complications: 97. United States Agency for International Development (USAID). facilitator’s manual. New York; EngenderHealth, RESPOND Postabortion care (PAC): introduction to PAC training. PAC Project ; 2010. Available from: http://respond-project.org/ Global Resources Guide. Washington (DC): USAID; 2010. pages/files/6_pubs/curricula-manuals/COMMPAC- Available from: http://www.postabortioncare.org/content/ Facilitator-Manual-May2010.pdf introduction-pac-training 83. Magak K, Mukenge M. Testing community-level strategies to 98. Ipas; Venture Strategies Innovations. Misoprostol use in reduce unwanted pregnancy and unsafe abortion in Suba District, postabortion care: a service delivery toolkit. Chapel Hill (NC): Kenya. A summary of mid-term evaluation findings. Los Angeles Ipas; 2011. Available from: http://www.ipas.org/en/ (CA): Pacific Institute for Women’s Health, Center for the Study of Resources/Ipas%20Publications/Misoprostol-use-in- Adolescence and Kisumu Medical and Educational Trust; 2004. postabortion-care-A-service-delivery-toolkit.aspx

Global Health: Science and Practice 2016 | Volume 4 | Number 3 493 Postabortion Care: 20 Years of Evidence www.ghspjournal.org

99. International Confederation of Midwives (ICM). Essential Disability (AMDD). Monitoring emergency obstetric care: competencies for basic midwifery practice: 2010, revised 2013. a handbook. Geneva: WHO; 2009. Available from: http:// The Hague (Netherlands): ICM; 2013. Available from: http:// www.who.int/reproductivehealth/publications/monitoring/ www.internationalmidwives.org/assets/uploads/documents/ 9789241547734/en/ CoreDocuments/ICM%20Essential%20Competencies%20for% 101. Gay J, Croce-Galis M, Hardee K. What works for women 20Basic%20Midwifery%20Practice%202010,%20revised% and girls: evidence for HIV/AIDS interventions. Washington 202013.pdf (DC): Population Council, The Evidence Project and What 100. World Health Organization (WHO); United Nations Population Works Association, Inc.; 2016. (Note: In-text citation not Fund (UNFPA); United Nations Children’s Fund (UNICEF); in sequential order.) Available from: http://www.whatworks Mailman School of Public Health, Averting Maternal Death and forwomen.org/

Peer Reviewed

Received: 2016 Feb 18; Accepted: 2016 May 24; First Published Online: 2016 Aug 25

Cite this article as: Huber D, Curtis C, Irani L, Pappa S, Arrington L. Postabortion care: 20 years of strong evidence on emergency treatment, family planning, and other programming components. Glob Health Sci Pract. 2016;4(3):481-494. http://dx.doi.org/10.9745/GHSP-D-16-00052.

& Huber et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/3.0/. When linking to this article, please use the following permanent link: http://dx.doi.org/ 10.9745/GHSP-D-16-00052.

Global Health: Science and Practice 2016 | Volume 4 | Number 3 494 FIELD ACTION REPORT

Improving the Quality of Postabortion Care Services in Togo Increased Uptake of Contraception

Stembile Mugore,a Ntapi Tchiguiri K Kassouta,b Boniface Sebikali,c Laurel Lundstrom,d Abdulmumin Saade