Dedicated to what works in global health programs
GLOBAL HEALTH: SCIENCE AND PRACTICE
2020 Volume 8 Number 4 www.ghspjournal.org EDITORS
Editor-in-Chief Stephen Hodgins, MD, MSc, DrPH, Associate Professor, Global Health, School of Public Health, University of Alberta
Editor-in-Chief Emeritus: James D. Shelton, MD, MPH, Retired Associate Editors Matthew Barnhart, MD, MPH, Senior Science Advisor, USAID, Bureau for Global Health Cara J. Chrisman, PhD, Biomedical Research Advisor, USAID, Bureau for Global Health Elaine Menotti, MPH, Health Development Officer, USAID, Bureau for Global Health Jim Ricca, MD, MPH, Learning and Implementation Science Team Leader, Maternal and Child Survival Program, Jhpiego Madeleine Short Fabic, MHS, Public Health Advisor, USAID, Bureau for Global Health Saad Abdulmumin, MD, PhD, MPH, USAID, Bureau for Global Health Malaria: Michael Macdonald, ScD, Consultant, World Health Organization, Vector Control Unit, Global Malaria Programme Maternal Health: Marge Koblinsky, PhD, Independent Consultant Nutrition: Bruce Cogill, PhD, MS, Consultant Managing Staff Natalie Culbertson, Johns Hopkins Center for Communication Programs Sonia Abraham, MA, Johns Hopkins Center for Communication Programs EDITORIAL BOARD
Zulfiqar Bhutta, The Hospital for Sick Children, Toronto, Aga Emmanuel (Dipo) Otolorin, Jhpiego, Nigeria Khan University, Pakistan James Phillips, Columbia University, USA Kathryn Church, Marie Stopes International, London School Yogesh Rajkotia, ThinkWell, USA of Hygiene and Tropical Medicine, United Kingdom David Sleet, Bizell Group, LLC, Previously Center for Disease Scott Dowell, The Bill and Melinda Gates Foundation, USA Control and Prevention, USA Marelize Görgens, World Bank, USA John Stanback, FHI 360, USA Lennie Kamwendo, White Ribbon Alliance for Safe Lesley Stone, US Department of State/US Agency for Motherhood, Health Service Commission, Malawi International Development, USA Jemilah Mahmood, International Red Cross and Red Crescent Douglas Storey, Johns Hopkins Center for Communication Societies, Malaysia Programs, USA Vinand Nantulya, Busitema University, Uganda
Global Health: Science and Practice (ISSN: 2169-575X) is a no-fee, open-access, peer-reviewed journal published online at www.ghspjournal.org. It is published quarterly by the Johns Hopkins Center for Communication Programs, 111 Market Place, Suite 310, Baltimore, MD 21202. GHSP is made pos- sible by the support of the American People through the United States Agency for International Development (USAID) under the Knowledge SUCCESS (Strengthening Use, Capacity, Collaboration, Exchange, Synthesis, and Sharing) Project. GHSP is editorially independent and does not necessarily represent the views or positions of USAID, the United States Government, or the Johns Hopkins University. For further information, please contact the editors at [email protected].
Global Health: Science and Practice is distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. Cover caption: Nurse in Nigeria performing pulse oximetry on an infant, demonstrating oxygen saturation level to parents. © 2020 Oxygen for Life Initiative Table of Contents December 2020 | Volume 8 | Number 4
EDITORIALS
Learning From Neighbors
We can learn valuable lessons from program efforts that at first glance may seem to be far removed from our own work.
Stephen Hodgins
Glob Health Sci Pract. 2020;8(4):613 https://doi.org/10.9745/GHSP-D-20-00639
COMMENTARIES
Go Where the Virus Is: An HIV Micro-epidemic Control Approach to Stop HIV Transmission
Essentially all HIV transmission is from people living with HIV who are not virally suppressed. An HIV micro-epidemic control approach that differentiates treatment support and prevention services for people living with HIV and their network members according to viral burden could optimize the impact of epidemic control efforts.
Michael M. Cassell, Rose Wilcher, Reshmie A. Ramautarsing, Nittaya Phanuphak, Timothy D. Mastro
Glob Health Sci Pract. 2020;8(4):614–625 https://doi.org/10.9745/GHSP-D-19-00418
Capturing Acquired Wisdom, Enabling Healthful Aging, and Building Multinational Partnerships Through Senior Global Health Mentorship
The undeniable benefit of mentorship by experience senior mentors can meaningfully increase the breadth of their experience and contributions to society as well as address the dire inequality in global health. This model captures wisdom lost to retirement, enables opportunities for purposeful lifespan, underpins sustainable health care systems, and has the potential for building multinational partnerships.
C. Norman Coleman, John E. Wong, Eugenia Wendling, Mary Gospodarowicz, DonnaO’Brien, Taofeeq Abdallah Ige, Simeon Chinedu Aruah, David A. Pistenmaa, Ugo Amaldi, Onyi-Onyinye Balogun, Harmar D. Brereton, Silvia Formenti, Kristen Schroeder, Nelson Chao, Surbhi Grover, Stephen M. Hahn, James Metz, Lawrence Roth, Manjit Dosanjha
Glob Health Sci Pract. 2020;8(4):626–637 https://doi.org/10.9745/7GHSP-D-20-00108
ORIGINAL ARTICLES
Prevention of COVID-19 in Internally Displaced Persons Camps in War-Torn North Kivu, Democratic Republic of the Congo: A Mixed-Methods Study
Internally displaced persons fleeing violent conflict represent a neglected population with heightened vulnerability to pandemic COVID- 19. We provide a rare snapshot of the overwhelming challenges faced by internally displaced persons in Eastern Democratic Republic of the Congo as they brace for COVID-19.
Kasereka M. Claude, Muyisa Sahika Serge, Kahindo Kahatane Alexis, Michael T. Hawkes
Glob Health Sci Pract. 2020;8(4):638–653 https://doi.org/10.9745/GHSP-D-20-00272
Global Health: Science and Practice 2020 | Volume 8 | Number 4 Table of Contents www.ghspjournal.org
Meeting the Global Target in Reproductive, Maternal, Newborn, and Child Health Care Services in Low- and Middle-Income Countries
What progress has been achieved toward reproductive, maternal, newborn, and child health service related Sustainable Development Goals? Analyzing data to estimate coverage of these indicators, we observed that acceleration is needed in coordinated global efforts and government policies to ensure universal access to RMNCH care services by 2030.
Md. Mehedi Hasan, Ricardo J. Soares Magalhaes, Saifuddin Ahmed, Sayem Ahmed, Tuhin Biswas, Yaqoot Fatima, Md. Saimul Islam, Md. Shahadut Hossain, Abdullah A. Mamun
Glob Health Sci Pract. 2020;8(4):654–665 https://doi.org/10.9745/GHSP-D-20-00097
Contraceptive Method Mix: Updates and Implications
Trends in contraceptive method mix show that dominance of 1 method in the mix remains very common, though countries and regions throughout the world are diverse as to which method is dominant. Our analysis argues for continued concerted efforts of programs to increase contraceptive method choice.
Jane T. Bertrand, John Ross, Tara M. Sullivan, Karen Hardee, James D. Shelton
Glob Health Sci Pract. 2020;8(4):666–679 https://doi.org/10.9745/GHSP-D-20-00229
Health Care Worker Preferences and Perspectives on Doses per Container for 2 Lyophilized Vaccines in Senegal, Vietnam, and Zambia
When providing immunization services, health care workers balance the mandate of achieving high coverage with limiting vaccine wastage. Workers in 3 countries said that containers with fewer vaccine doses for measles and BCG would enable them to immunize all children who present, while reducing concerns about wasting vaccine.
Natasha Kanagat, Kirstin Krudwig, Karen A. Wilkins, Sydney Kaweme, Guissimon Phiri, Frances D.Mwansa, Mercy Mvundura, Joanie Robertson, Debra Kristensen, Abdoulaye Gueye, Sang D. Dao, Pham Q. Thai, Huyen T. Nguyen, Thang C. Tran
Glob Health Sci Pract. 2020;8(4):680–688 https://doi.org/10.9745/GHSP-D-20-00112
Remote Mentorship Using Video Conferencing as an Effective Tool to Strengthen Laboratory Quality Management in Clinical Laboratories: Lessons From Cambodia
This program to strengthen laboratory quality management systems in Cambodia demonstrated significant improvements in conformity to ISO 15189 standards in participating laboratories, correlating with laboratory participation time in video conference training activities led by quality improvement mentors over the program implementation period.
Grant Donovan, Siew Kim Ong, Sophanna Song, Nayah Ndefru, Chhayheng Leang, Sophat Sek, Patricia Sadate-Ngatchou, Lucy A. Perrone
Glob Health Sci Pract. 2020;8(4):689–698 https://doi.org/10.9745/GHSP-D-20-00128
Global Health: Science and Practice 2020 | Volume 8 | Number 4 Table of Contents www.ghspjournal.org
Using Community Health Workers and a Smartphone Application to Improve Diabetes Control in Rural Guatemala
A smartphone application providing algorithmic clinical decision support enabled community health workers to improve diabetes control for a group of patients in rural Guatemala. This approach enables task sharing with physicians and other advanced practitioners for chronic disease care, which is particularly important in low-resource settings.
Sean Duffy, Derek Norton, Mark Kelly, Alejandro Chavez, Rafael Tun, Mariana Niño de Guzmán Ramírez, Guanhua Chen, Paul Wise, Jim Svenson
Glob Health Sci Pract. 2020;8(4):699–720 https://doi.org/10.9745/GHSP-D-20-00076
Behavioral Insights Into Micronutrient Powder Use for Childhood Anemia in Arequipa, Peru
Health care provider-caregiver interactions and caregivers’ shifting emotional states between intention formation and use affected their adherence to a government-provided micronutrient powder (MNP) intervention to prevent childhood anemia. In counseling directed to caregivers, we suggest providers offer clear messaging on MNP impact and planning for challenges during MNP use.
Jessica D. Brewer, Julianna Shinnick, Karina Román, Maria P. Santos, Valerie A. Paz-Soldan, Alison M. Buttenheim
Glob Health Sci Pract. 2020;8(4):721–731 https://doi.org/10.9745/GHSP-D-20-00078
A Cluster-Randomized Trial to Test Sharing Histories as a Training Method for Community Health Workers in Peru
Women naturally communicate using life narratives. Through systematic recall and sharing memories of their own childbearing and child rearing experiences, community health workers (CHWs) become engaged and empowered to change their own and other mothers’ health behaviors. Training CHWs with Sharing Histories can improve their capabilities as change agents for better child health.
Laura C. Altobelli, José Cabrejos-Pita, Mary Penny, Stan Becker
Glob Health Sci Pract. 2020;8(4):732–758 https://doi.org/10.9745/GHSP-D-19-00332
A Rapid Cost Modeling Tool for Evaluating and Improving Public Health Supply Chain Designs
The Rapid Supply Chain Modeling Tool enables health system leaders to quickly estimate and compare the cost impact of potential supply chain design improvements in situations where time and budget do not allow for more in-depth modeling approaches.
Michael Krautmann, Mariam Zameer, Dorothy Thomas, Nora Phillips-White, Ana Costache, Pascale R. Leroueil
Glob Health Sci Pract. 2020;8(4):759–770 https://doi.org/10.9745/GHSP-D-20-00227
Global Health: Science and Practice 2020 | Volume 8 | Number 4 Table of Contents www.ghspjournal.org
Lessons Learned From Implementing Prospective, Multicountry Mixed-Methods Evaluations for Gavi and the Global Fund
Lessons learned from implementing evaluations for Gavi, the Vaccine Alliance and the Global Fund for AIDS, Tuberculosis, and Malaria can help inform the design and implementation of ongoing or future evaluations of complex interventions. We share 5 lessons distilled from over 7 years of experience implementing evaluations in 7 countries.
Emily Carnahan, Nikki Gurley, Gilbert Asiimwe, Baltazar Chilundo, Herbert C. Duber, Adama Faye, Carol Kamya, Godefroid Mpanya, Shakilah Nagasha, David Phillips, Nicole Salisbury, Jessica Shearer, Katharine Shelley, for the Gavi Full Country Evaluations Consortium; and Global Fund Prospective Country Evaluation Consortium
Glob Health Sci Pract. 2020;8(4):771–782 https://doi.org/10.9745/GHSP-D-20-00126
Effects of a Community-Based Program on Voluntary Modern Contraceptive Uptake Among Young First-Time Parents in Cross River State, Nigeria
Among young first-time mothers, participation in a comprehensive, community-based program led to a 3-fold increase in voluntary modern contraceptive use and other positive changes. These results demonstrate the importance of investing in interventions for this youth population that ideally address a range of priorities across the first-time parent lifestage.
Gwendolyn Morgan, Anjala Kanesathasan, Akinsewa Akiode
Glob Health Sci Pract. 2020;8(4):783–798 https://doi.org/10.9745/GHSP-D-20-00111
REVIEWS
Routine Family Planning Data in the Low- and Middle-Income Country Context: A Synthesis of Findings From 17 Small Research Grants
A review of 5 years of small grant-funded research highlighted overarching barriers to and opportunities for using family planning data in routine health information systems in low- and middle-income countries. We report on factors affecting data quality, analysis, and use, and suggest strategies to improve routine family planning data.
Bridgit Adamou, Janine Barden-O’Fallon, Katie Williams, Amani Selim
Glob Health Sci Pract. 2020;8(4):799–812 https://doi.org/10.9745/GHSP-D-20-00122
Effectiveness of mHealth Interventions for Improving Contraceptive Use in Low- and Middle-Income Countries: A Systematic Review
Do mHealth interventions help reduce unmet contraceptive needs in low- and middle-income countries by attempting to increase the uptake of modern contraceptive methods? Which mHealth features and behavior change communication components were used in these mHealth interventions? This review aimed to answer these questions and assess the impact of these interventions on contraceptive uptake outcomes.
Banyar Aung, Jason W. Mitchell, Kathryn L. Braun
Glob Health Sci Pract. 2020;8(4):813–826 https://doi.org/10.9745/GHSP-D-20-00069
Global Health: Science and Practice 2020 | Volume 8 | Number 4 Table of Contents www.ghspjournal.org
METHODOLOGIES
A Practical Guide to Using Time-and-Motion Methods to Monitor Compliance With Hand Hygiene Guidelines: Experience From Tanzanian Labor Wards
Understanding hand hygiene behaviors is critical in hospitals. We developed the HANDS at birth tool—and provide information on its design and implementation–to capture the complex patterns of health care workers’ hand hygiene including hand rubbing/washing, glove use, and recontamination.
Giorgia Gon, Said M. Ali, Robert Aunger, Oona M. Campbell, Mícheál de Barra, Marijn de Bruin, Mohammed Juma, Stephen Nash, Amour Tajo, Johanna Westbrook, Susannah Woodd, Wendy J. Graham
Glob Health Sci Pract. 2020;8(4):827–837 https://doi.org/10.9745/10.9745/GHSP-D-20-00221
FIELD ACTION REPORTS
Implementation of a Pediatric Early Warning Score to Improve Communication and Nursing Empowerment in a Rural District Hospital in Rwanda
Implementation of the Pediatric Early Warning Score for Resource-Limited Settings tool improved nurses’ competency and confidence in their triage capabilities. This tool has the potential to improve patient outcomes. However, staff turnover and limited physician buy-in were barriers to sustainability of the tool in low-resource settings.
Shela Sridhar, Alexis Schmid, Francois Biziyaremye, Samantha Hodge, Ngamika Patient, Kim Wilson
Glob Health Sci Pract. 2020;8(4):838–845 https://doi.org/10.9745/GHSP-D-20-00075
PROGRAM CASE STUDIES
Juntos: A Support Program for Families Impacted by Congenital Zika Syndrome in Brazil
Development of Juntos, a community-based family support program for caregivers of children with congenital Zika syndrome, contained innovative approaches such as using mothers as facilitators, fast-track learning, and de-isolation of families affected by Zika.
Antony Duttine, Tracey Smythe, Miriam Ribeiro Calheiros de Sa, Silvia Ferrite, Maria Elisabeth Moreira, Hannah Kuper
Glob Health Sci Pract. 2020;8(4):846–857 https://doi.org/10.9745/GHSP-D-20-00018
Global Health: Science and Practice 2020 | Volume 8 | Number 4 Table of Contents www.ghspjournal.org
TECHNICAL NOTES
Improving Hospital Oxygen Systems for COVID-19 in Low-Resource Settings: Lessons From the Field
Hospitals in low- and middle-income countries urgently need to improve their oxygen systems for COVID-19 and other health emergencies. We share practical tips to improve pulse oximetry and oxygen use, support biomedical engineers to optimize existing oxygen supplies, and expand existing oxygen systems with robust equipment and smart design.
Hamish R. Graham, Sheillah M. Bagayana, Ayobami A. Bakare, Bernard O. Olayo, Stefan S. Peterson, Trevor Duke, Adegoke G. Falade
Glob Health Sci Pract. 2020;8(4):858–862 https://doi.org/10.9745/GHSP-D-20-00224
CORRECTIONS
Corrigendum: Parmaksiz K et al., What Makes a National Pharmaceutical Track and Trace Succeed? Lessons From Turkey Glob Health Sci Pract. 2020;8(4):863 https://doi.org/10.9745/GHSP-D-20-00587
Global Health: Science and Practice 2020 | Volume 8 | Number 4 EDITORIAL
Learning From Neighbors
Stephen Hodginsa
See related article by Duttine et al. to assess relevance, usefulness, and feasibility. Adaptation and refinement of the program was done not only at the end of each of these rounds; the developers also pursued he article by Duttine and colleagues1 in this issue of an intentional approach of fast-track learning—eliciting TGlobal Health: Science and Practice describes efforts re- feedback on an ongoing basis from participants and facil- cently undertaken in Brazil to develop and test an ap- itators and making real-time changes, as necessary, in proach to supporting parents of young children affected content, approach, and logistical arrangements. by Zika. For the vast majority of GHSP readers, we sus- As they explain, not only was the program itself iter- pect that this topic is not directly relevant to their work. atively adapted and modified, so too was the theory of Nevertheless, we believe we can learn valuable lessons change. In the course of trying out the intervention, the from program efforts that may seem, at first glance, to investigators formed a better sense of how it worked be far removed from our own work. and, in turn, made revisions to their theory of change. The authors describe their process in the develop- The authors modeled learning from neighbors, draw- ment and initial testing of a community-based support ing on an approach developed in another context for a program for parents. Their starting points were: (1) a somewhat different need (Bangladeshi families with clearly defined need, and (2) a potentially relevant model. children affected by cerebral palsy) and adapting it to The need was for effective, formalized support of care- their setting to address their specific problem. We en- givers of community-living children with mild to mod- courage you—our readers—also to learn from neigh- erate Zika-related impairments. The model was a bors, drawing from examples like this of how to address caregiver education and support program first devel- a public health problem by: oped in Bangladesh and subsequently adopted in many other countries: Getting to Know Cerebral Palsy. Listening to stakeholders With this identified gap and a potential solution, the Being flexible and prepared to revisit assumptions next step was a needs assessment, which included a review and early design choices of relevant literature and consultation with stake- Building learning and adaptation into your routine holders, both experts and parents of affected children. ways of doing business On the basis of this input, the developers then crafted an initial version of the program design and materials Competing interests: None declared. (adapted to Zika and to a Brazilian cultural setting) and an associated theory of change (a theory of how they thought the intervention would work). REFERENCE 1. Duttine A, Smythe T, Ribeiro Calheiros de Sa M, Ferrite S, Moreira ME, Kuper The program developers then tested the program in H. Juntos: a support programme for families impacted by congenital Zika syn- 2 diverse sites, going through 2 iterative rounds of piloting drome in Brazil. Glob Health Sci Pract. 2020;8(4). CrossRef
Cite this article as: Hodgins S. Learning from neighbors. Glob Health Sci Pract. 2020;8(4):613. https://doi.org/10.9745/GHSP-D-20-00639
© Hodgins. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unre- stricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http:// creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https://doi.org/10.9745/GHSP-D-20-00639
a Editor-in-Chief, Global Health: Science and Practice Journal, and Associate Professor, School of Public Health, University of Alberta, Edmonton, Alberta, Canada. Correspondence to Stephen Hodgins ([email protected]).
Global Health: Science and Practice 2020 | Volume 8 | Number 4 613 COMMENTARY
Go Where the Virus Is: An HIV Micro-epidemic Control Approach to Stop HIV Transmission
Michael M. Cassell,a Rose Wilcher,b Reshmie A. Ramautarsing,c Nittaya Phanuphak,c,d Timothy D. Mastrob
Key Messages who achieve sustained viral suppression and undetect- able levels of circulating virus through good adherence n Essentially all HIV transmission is from people to ART live long, healthy lives and will not transmit HIV living with HIV (PLHIV) who do not know their through sexual contact.2–4 The evidence that people infection status or have not yet achieved viral who have achieved undetectable viral loads will not trans- suppression, making support for these individuals mit HIV sexually—that “undetectable equals untransmit- and their risk contacts a priority for treatment and table” (U=U)—underscores the prevention benefits of prevention efforts. treatment and the rationale for the global call to achieve n Proven approaches exist to reduce viral burden near-universal access to ART and viral suppression among and interrupt HIV transmission from PLHIV who PLHIV.4–6 are not yet virally suppressed, but these Conversely, HIV viral burden (viremia), generally approaches must be implemented with enhanced measured by plasma viral load (HIV RNA copies/mL) focus and scale to maximize benefit. assays, is the primary predictor of HIV-related disease n Improved diagnostic approaches offer new progression, morbidity, mortality, and ongoing trans- opportunities to increase public health impact by mission.4,7 Essentially all HIV transmission originates prioritizing support for unserved or underserved from a shrinking minority of PLHIV globally (41%) who individuals with the greatest viral burdens and do not know their HIV infection status or have not yet among members of their risk networks. achieved viral suppression,8 making support for these n Policy makers should pursue the implementation individuals and their risk contacts a priority for treat- and evaluation of diagnostic approaches that can ment and prevention efforts. Studies have identified a focus services among individuals and networks dose-response relationship in which each 10-fold in- with the greatest viral burdens. crease in HIV plasma viral load results in an increased n Program managers should treat viral burden as a relative risk of HIV transmission of 2.5 to 2.9 per sexual 9,10 primary consideration in the provision of contact. Emerging evidence suggests that even under differentiated HIV services, applying an HIV conditions of near-universal HIV treatment coverage, micro-epidemic control framework to prioritize high viremia and high levels of risk behavior among and tailor services for PLHIV and their risk unserved or underserved PLHIV can sustain epidemic contacts along a continuum of progression to viral HIV transmission.11,12 In a recent U.S. study of HIV suppression. patients in care with a detectable viral load, only a small proportion of PLHIV reported concurrent sexual trans- mission risk behaviors, but most of the individuals in this group had considerably elevated viral loads, increas- ing the probability of transmission. The study found INTRODUCTION that viral loads were likely to be lower among those lobally, a growing majority (59%) of an estimated with a detectable viral load who reported always using G38 million people living with HIV (PLHIV) know condoms.13 their HIV status and have achieved HIV viral suppression High viral burden associated with acute HIV infec- by adhering to antiretroviral therapy (ART).1 Individuals tion (AHI) is a particular concern. Acute infection is characterized by a 2–4-week period of exceptionally a FHI 360, Hanoi, Vietnam. high viremia as HIV replicates rapidly in the body before b FHI 360, Durham, NC, USA. a person’s immune system mounts a response and c Institute of HIV Research and Innovation, Bangkok, Thailand. reduces the level of circulating virus to much lower— d Center of Excellence in Transgender Health, Chulalongkorn University, Bangkok, Thailand. but typically not undetectable—levels for a period of 14,15 Correspondence to Michael Cassell ([email protected]). months to years. Although only a small proportion
Global Health: Science and Practice 2020 | Volume 8 | Number 4 614 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
Liz Brenda Kandeyi, nurse (left), takes a client through the steps for clinical services at Kikuyu Sasa Center, Nairobi, Kenya. © 2017 Nancy Coste/FHI 360 of all PLHIV will be in this brief AHI phase at any individuals with greater adherence, clinical, social given time, per-sex-act transmission probabilities support, and other needs. are considerably higher during periods of acute as Nevertheless, the resources and technologies Prioritizing viral 9,16–18 compared to chronic HIV infection. needed to activate a more comprehensive differ- load testing and In key populations engaged in frequent behav- entiation of support based on viral burden histori- screening options ioral risks, up to an estimated 50% of all HIV trans- cally have been limited. With the advent of for individuals mission occurs from individuals during AHI when expanded access to viral load testing and options with the greatest viremia is very high prior to the development of an to screen for AHI, opportunities now exist to prior- viral burdens can immune response including anti-HIV antibodies itize support for individuals and in risk networks help interrupt (Ab) that yield reactivity on third-generation Ab with the greatest viral burdens. This prioritization epidemic HIV assays.14,17,19–24 The provision of ART during AHI can help interrupt epidemic HIV transmission as- transmission. and of HIV pre-exposure prophylaxis (PrEP) to the sociated with AHI through early diagnosis, HIV risk-network contacts of acutely infected indivi- treatment, and provision of PrEP and other proven duals could prevent a substantial proportion prevention approaches to risk contacts. Because of ongoing HIV transmission. An analysis in HIV morbidity, mortality, and transmission risk Thailand suggested that early diagnosis and treat- are most closely associated with viral burden, this ment during AHI among men who have sex with enhanced focus can guide the allocation of limited men could avert 89% of all new infections in this resources to maximize the impact of prevention 25 population. and treatment efforts. Approaches that differentiate service delivery to better address the preferences and needs of unserved and underserved individuals have been ENVISIONING A MICRO-EPIDEMIC identified as a priority to close outstanding gaps in CONTROL APPROACH THAT access to HIV prevention and treatment.26 In implementing differentiated services, it is increas- DIFFERENTIATES SUPPORT BASED ON ingly clear that a focus on individuals and net- VIRAL BURDEN works with the greatest viral burdens has We propose an HIV micro-epidemic control frame- strategic benefit. For example, programs typically work to characterize these opportunities to acceler- transition individuals who are receiving HIV ate impact, with a primary focus on addressing the treatment and are identified through routine viral differentiated service preferences and needs of indi- load testing as virally suppressed to options for viduals who are not yet virally suppressed, as well less frequent clinical follow-up and multimonth as the members of their risk networks. This frame- dispensing of their antiretroviral medications. work aims to organize and integrate both new and This differentiation offers additional convenience existing approaches to tailor support for PLHIV and to patients and frees up resources and provider their risk contacts based on progression to sustained time to focus support on virally unsuppressed viral suppression. By profiling the characteristics of
Global Health: Science and Practice 2020 | Volume 8 | Number 4 615 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
those who face challenges in achieving viral sup- which will reduce the need for more frequent clinic pression—such as barriers to diagnosis, to treatment visits, a benefit for both PLHIV and clinic staff. initiation and retention post-diagnosis, and to viral Partner notification services, also known as in- load testing and suppression—programs can intro- dex testing, are recommended by the World duce solutions that both help these individuals and Health Organization as a safe, effective strategy to By profiling the that remove barriers for others. accelerate HIV epidemic control by asking PLHIV characteristics of The approach features variable treatment and to list and refer their sexual and injecting partners those who face prevention services and service intensity grouped and biological children to HIV testing services on a challenges in according to 4 different stages along this continu- voluntary basis.27 Offering index testing at least achieving viral um of progression to viral suppression (Figure 1). once to these individuals can help link members suppression, The model also affords program managers with of their networks who may previously have been programs can opportunities to prioritize program efforts based exposed to HIV to relevant testing, prevention, introduce on regional, national, and subnational variations and treatment services. Uninfected network mem- solutions that both in progress with respect to the expansion of HIV bers will not acquire HIV infection from sexual con- help these prevention, testing, treatment, and viral suppres- tact with PLHIV who have undetectable viral loads, individuals and sion coverage. but those who continue to be at elevated HIV infec- remove barriers tion risk from other contacts can be offered PrEP for others. and other HIV prevention services, including con- 1. PLHIV Who Are on Treatment and Virally dom education and access. Routine viral load Suppressed testing is critical to monitor and sustain viral sup- Although progress varies by region, an estimated pression among PLHIV who have previously 59% of PLHIV globally are already receiving HIV achieved suppression. To facilitate viral load testing treatment and are virally suppressed.8 Importantly, access and the provision of efficient retention and per “U=U,” these individuals will not transmit HIV adherence support to PLHIV with a stable treat- to their sexual partners.4 PLHIV who are found to ment status, programs can implement virtual be virally suppressed through routine viral load test- online- and telephone-based support with patient ing are good candidates for multimonth dispensing, consent and appropriate measures in place to
FIGURE 1. An HIV Micro-epidemic Control Model Aims to Prioritize and Focus Treatment and Prevention Efforts Where They Can Have the Greatest Impacts: Among a Shrinking Proportion of Individuals and Risk Networks With the Greatest Viral Burdens
Abbreviations: AHI, acute HIV infection; ART, antiretroviral therapy; HIVþ, HIV-positive; HIV-, HIV-negative; PLHIV, people living with HIV; PrEP, pre-exposure prophylaxis.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 616 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
ensure the security and confidentiality of patient programs are applying machine learning algo- information. The use of point-of-care viral load rithms to automate this process of preventive pri- testing technologies may also reduce testing turn- oritization to enhance care.29 around times and bring added convenience to patients and providers. 3. Undiagnosed PLHIV PLHIV who have not yet received a diagnosis can 2. PLHIV Who Are Diagnosed but not Virally be similarly offered tailored support to maximize Suppressed individual treatment and population-level pre- A substantial proportion of individuals who have vention benefits. An expanded range of options previously received an HIV diagnosis have either for accessing HIV testing services, including HIV not yet initiated ART or have not achieved viral self-testing options with dispensing through phar- suppression.8 Individuals in this group can be fur- macies and peer networks, can help to close gaps ther divided into 3 categories: (1) those who have in diagnosis among PLHIV who might otherwise never been linked to ART; (2) those who have ini- not otherwise access diagnostic or other ser- 27,30,31 tiated ART but have not yet achieved viral sup- vices. Testing services also can be tailored pression or have been lost to follow-up and to focus on key populations facing the greatest stopped ART; and, (3) those who are sustained on HIV infection risks and to engage the risk network treatment but are showing signs of breakthrough members of PLHIV who are not virally suppressed. viremia or treatment failure. For individuals who Incorporating AHI screening into these targeted have never initiated treatment or been lost to testing approaches can improve the detection of follow-up, programs can initiate outreach cam- AHI among individuals who might otherwise paigns through clinical or community staff to have remained undiagnosed. While offering vol- engage or reengage previously diagnosed indivi- untary index testing and risk contact referral ser- duals. These campaigns can promote “U=U” mes- vices to individuals who are newly diagnosed saging, the benefits of new dolutegravir-based or who have recent HIV infection as identified treatment regimens,28 and convenient and confi- though recency testing, programs can also assess dential options for PLHIV to access same-day HIV the differentiating sociodemographic and risk pro- treatment. For individuals who are receiving files of these individuals. These profiles can be ap- treatment but have not achieved viral suppres- plied to further enhance the focus of HIV testing sion, providers can offer additional personalized efforts by bringing testing to individuals with sim- adherence counseling and support. Immediate ilar profiles and by engaging peer mobilizers with Immediate support should be provided to individuals with a similar characteristics to make testing referrals support should be viral load test indicating an unsuppressed viral and distribute HIV self-testing kits in their provided to PLHIV load to help to identify and address root causes of networks. who are adherence or treatment failure that require regi- diagnosed but not men switching. 4. PLHIV With AHI virally suppressed In the process of providing personalized treat- Expanding screening for AHI among key popula- to help address ment support to PLHIV, a range of voluntary index tions and other individuals facing elevated HIV in- root causes of testing options can be offered to encourage refer- fection risks can help realize the largely untapped adherence or rals of their risk contacts to prevention services treatment and prevention benefits of identifying, treatment failure. such as PrEP and condoms until they have treating, and index testing individuals with AHI. achieved viral suppression, or to HIV treatment To maximize efficiency, AHI screening can be pri- services, as needed.27 It may also be useful for pro- oritized for the risk network members of indivi- grams to routinely monitor the sociodemographic duals identified with AHI, those with recent HIV and risk characteristics of individuals who do not infections, and newly identified PLHIV. Screening initiate treatment, fall out of care, or do not can also be focused on key populations reporting achieve viral suppression, to assess how these recent behavioral risks, as well as among those individuals differ from those who engage in treat- with other sexually transmitted infections. Upon ment and sustain good treatment outcomes. By diagnosis, all PLHIV can be offered an accelerated generating profiles of the characteristics of indivi- path to viral suppression with same-day treatment duals more likely to face treatment challenges, initiation. Programs can assess the differentiated programs can apply these to prevent loss and other characteristics of newly diagnosed, recently in- adverse outcomes, helping to accelerate and fected, and acutely infected PLHIV to further opti- sustain progression to viral suppression. Some mize the relevance and focus of HIV testing efforts.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 617 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
The HIV micro- While the treatment and treatment-as- to develop and implement voluntary, safe, equita- epidemic control prevention benefits of prioritizing support accord- ble, and preferred policy and program solutions. approach aims to ing to viral burden may have evident advantages, enhance the HIV micro-epidemic control approach also THE CHALLENGE OF ACUTE HIV aims to enhance prevention benefit by focusing prevention benefit INFECTION by focusing services in the risk networks in which active HIV transmission is occurring. The majority of risk The proposed micro-epidemic control approach interventions in emphasizes diagnosis of and intervention during the risk networks contacts of PLHIV who are undiagnosed will be uninfected but at high risk of acquiring HIV infec- AHI in light of the substantial role that AHI plays in which active HIV in epidemic transmission of HIV. Most current na- transmission is tion, making linkages of these individuals to preven- tion services a priority. These focused prevention tional HIV testing algorithms rely on antibody- occurring. based serological testing that cannot detect AHI. efforts should employ a combination of evidence- As a result, these approaches misdiagnose poten- based prevention strategies relevant to the specific tial core transmitters as HIV-uninfected and miss preferences and needs of the populations being 32,33 critical opportunities to maximize the prevention served. These strategies include, but are not lim- benefits of HIV treatment. Affordable, accurate, ited to, harm reduction programming for people and scalable solutions to diagnose AHI have who inject drugs, support for correct and consistent remained elusive.17 condom use, and expanded access to PrEP. To max- The brief duration of AHI poses a major chal- imize uptake, services should be implemented in a lenge to diagnosis.16,17 Detection of AHI depends friendly manner that is welcoming and convenient on infected individuals having a blood test during to clients and is responsive to their feedback. The brief duration the short AHI period and then establishing the For all risk contacts of PLHIV who are not yet of acute infection presence of HIV RNA or p24 antigen (part of the virally suppressed, PrEP is a critical, evidence- and cost of testing virus) (Figure 2). Individuals facing high infection based, and likely short-duration priority.34 Making poses barriers to risks would need to seek HIV testing with HIV PrEP—and, as relevant, nonoccupational HIV post- diagnosis. RNA or p24 technologies on a frequent basis to in- exposure prophylaxis—offers routine for the con- crease the likelihood of detecting an infection dur- tacts of PLHIV as part of index testing affords ing the acute period. enormous opportunities to focus PrEP where it Cost is also an issue. Point-of-care platforms can have the greatest prevention impact. In cir- for detection of HIV RNA such as Alere Q (Abbott cumstances where partners and other risk contacts Laboratories) and GeneXpert (Cepheid) are now face no other substantial HIV infection risks, these available but are generally perceived as expensive individuals can safely discontinue PrEP once the (US$17–24). Fourth-generation point-of-care rap- PLHIV index client has achieved viral suppression. id HIV tests, such as the Alere HIV Combo kit In addition, the scale up of PrEP as part of index (Abbott Laboratories), detect both p24 antigen and testing services may serve to normalize PrEP and HIV antibodies within 20 minutes at a lower cost expand availability across a wider array of settings, (US$2–4) and can be substituted as the first, sensitive helping to accelerate historically limited progress screening test in a national HIV testing algorithm for 35 towards the achievement of global PrEP targets diagnosis.17 However, these fourth-generation assays and removing barriers to access among men who have much lower sensitivity to detect AHI than HIV have sex with men and individuals who may prefer RNA assays.39 not to disclose their status as key populations.36 Nevertheless, a clinical trial of PrEP in Uganda, Stigma, discrimination, violence, and other South Africa, and Zimbabwe found that 28% of structural factors impose considerable barriers to infections missed by current third-generation rap- service uptake, particularly among PLHIV and id diagnostic tests would have been identified with key populations living in criminalizing environ- the use of Alere HIV Combo, suggesting some ments.37,38 Therefore, the micro-epidemic control advantages of using a fourth-generation test over approach should be implemented in conjunction standard antibody testing.40 Investigators in San with broader efforts to address these structural fac- Francisco found more promising results, with the tors. By identifying the characteristics of individuals Alere Determine (Abbott Laboratories) point-of- for whom structural factors serve as particular bar- care fourth-generation antigen-antibody combo riers to health and well-being, a micro-epidemic rapid test detecting about 54% of the acute cases control framework may help bring additional focus detected through laboratory RNA testing.41 In to structural interventions and facilitate advocacy Thailand, the Alere HIV Combo kit detected 37 of and partnerships with individuals and communities 50 (74%) individuals with HIV RNA confirmed
Global Health: Science and Practice 2020 | Volume 8 | Number 4 618 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
FIGURE 2. The Natural History of HIV Disease Progression in the Absence of Antiretroviral Therapy, Variable Transmission Probabilities Associated With Viral Burden, and Capacities for Detection of HIV Infection Through Different Diagnostics
Abbreviations: Ag, antigen; Ab, antibody; PCR, polymerase chain reaction. acute HIV infection. These limited results suggest 1 RNA polymerase chain reaction (PCR) on dried promising performance of Alere HIV Combo in a blood spot samples; and (2) expanding the use of facility-based setting but require broader evalua- fourth-generation point-of-care rapid HIV testing, tion in diverse settings and populations.42,43 leveraging recency testing data where possible to AHI is sometimes accompanied by transient help focus AHI screening in networks in which clinical flu-like and other symptoms, including ongoing HIV transmission is occurring. rash, fever, sore throat, fatigue, muscle/joint aches, oral and genital ulcers, diarrhea, and swol- len lymph nodes.44,45 However, these symptoms POOLED PCR TO FACILITATE and signs are neither sensitive nor specific for DETECTION OF AND INTERVENTION AHI. Inquiring about the presence of these symp- DURING AHI toms and recent risk behaviors may suggest op- The “gold standard” for detection of AHI is molec- portunities to screen for AHI with an RNA assay if ular testing, specifically HIV-1 RNA PCR. This available or at least a fourth-generation test.46–48 approach is considered the standard of care to fa- Sensitizing populations at risk to AHI signs and cilitate early infant diagnosis among children symptoms, benefits of early detection and treatment, born to HIV-infected mothers. However, PCR is and potential advantages of fourth-generation diag- 17 nostics may also facilitate improved AHI diagnosis relatively expensive. To extend PCR testing effi- and treatment and mitigate HIV resistance risks asso- ciently to all individuals facing elevated HIV infec- ciated with PrEP continuation among individuals tion risks but who have nonreactive serological who may have received false-negative third- testing results as part of a targeted HIV testing generation HIV testing results. strategy, samples can be “pooled” such that quali- To overcome some of the logistical and tative PCR is run on a batch that combines like resource-related barriers to the expansion of sample types sourced from different individuals. AHI screening, diagnosis, and intervention, we Individual results are confirmed as negative for propose 2 potential solutions to encourage further negative pools. For reactive pools, each sample in consideration of both these and other context- the pool is then tested with quantitative PCR viral relevant approaches: (1) conducting pooled HIV- load independently to identify and rapidly
Global Health: Science and Practice 2020 | Volume 8 | Number 4 619 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
intervene with individuals with reactive results. presents some additional value-added opportuni- The Thai Red Cross AIDS Research Center has ties to improve HIV service delivery that are wor- been applying a pooling approach with plasma thy of consideration. Drug-resistance genotyping samples as a cost-efficient strategy to identify and can be performed on samples from all individuals treat individuals with AHI who might otherwise who are confirmed through either molecular or not receive a diagnosis using serological testing.49 serological testing to have HIV infection. In addi- Nevertheless, separation, storing, and transfer of tion, these samples could be used to conduct phy- plasma can pose logistical challenges in resource- logenetic analyses of potential HIV transmission limited settings and incur additional costs. Pooled clusters to enhance the focus of targeted testing PCR testing may also be possible on point-of-care and index testing implementation.53 viral load platforms as these become more afford- able and widely available, realizing additional benefits in terms of efficiency, convenience, and early detection and intervention. COMBINING AHI SCREENING WITH In resource-limited settings, collecting dried RECENCY TESTING TO FACILITATE blood spot samples for pooled PCR is a promising EARLIER HIV DIAGNOSIS AND approach to circumvent some of the cost barriers INTERVENTION associated with point-of-care platforms and the lo- In resource- Rapid HIV recency assays, such as the Asanté HIV- gistical challenges associated with separation, limited settings, 1 Rapid Recency® Assay (Sedia Biosciences) and storing, and transfer of plasma samples. Whole collecting dried the Maxim Swift HIV Recent Infection Assay blood spots can be collected with a finger prick blood spot (Maxim Biomedical Inc.), were developed to help and can be stored and shipped with relative ease. samples for identify individuals who have become HIV infected Recalibration of the PCR is necessary because of a pooled PCR is a within the past year—on average in the past degradation of viral RNA in dried blood that may promising 6months—to help estimate HIV incidence and im- result in a 2-log reduction in assay sensitivity and approach to because of the potential presence of viral DNA, prove the focus of programming in settings, popu- circumvent some which may partially compensate for this loss in lations, and networks in which incident infections logistical and cost 54 sensitivity.50 At least 1 study has demonstrated continue to occur. Rapid recency point-of-care barriers to the accuracy of doing pooled PCR on dried blood antibody-based assays differentiate between recent detecting acute — spot samples for early infant diagnosis and docu- HIV infection when the antibody response is im- “ ” HIV infection. mented a laboratory cost savings of 65% associated mature, as reflected by low avidity or binding — with pooling.50 Other studies have demonstrated strength of the antibody and long-term infections in which a mature antibody response is measured feasibility to diagnose AHI and to reduce the costs 55,56 of ART monitoring in resource-limited settings by strong antibody avidity. The assays can yield “ ” with pooled PCR on dried blood spot samples.51,52 false-recent results among individuals who natu- An illustrative depiction of this diagnostic algo- rally control HIV well (low virus=low antibody) or rithm incorporating AHI screening based on dried are receiving ART, so a recent infection result is blood spot sample pooled PCR testing is provided usually confirmed using a recent infection testing in Figure 3. algorithm in which a viral load test is conducted with results of ≥ 1,000 copies/mL confirming recent To maximize the benefits of screening for AHI, 54,57,58 the time from sample collection to case identifica- infection. tion and intervention must be minimized. A pool- Rapid recency assays only measure antibody ing approach is more practical in high volume avidity after HIV seroconversion; they do not de- settings in which batches can be run every day. tect HIV RNA or p24 antigen and therefore are un- While the time needed to process samples will able to detect AHI. In typical use, they are only vary according to the proximity and availability of offered to individuals who have been confirmed laboratory infrastructure, we anticipate many pro- to have HIV infection with a standard antibody- grams being able to provide results in 1–2 days giv- based national HIV testing algorithm. Recency en the predominantly urban concentration of assays are also pending review for diagnostic pur- laboratory resources and of HIV key population poses by the World Health Organization and are risks in many country settings. The expansion of currently only approved for research use by the point-of-care viral load testing can further reduce U.S. Food and Drug Administration. The World turnaround times, facilitating earlier action. Health Organization has endorsed the use of re- The routine collection of dried blood spot sam- cency assays for surveillance purposes but has ples to detect and intervene during AHI also not yet made a determination regarding program
Global Health: Science and Practice 2020 | Volume 8 | Number 4 620 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
FIGURE 3. An Algorithm for Routine Screening for Acute HIV Infection in Populations Facing Elevated HIV Infection Risks
Abbreviations: ART, antiretroviral therapy; DBS, dried blood spot; HTS, HIV testing services; PCR, polymerase chain reaction; PrEP, pre-exposure prophylaxis; VL, viral load. a Individuals with nonreactive results on the first or “screening” test would be notified as likely uninfected but as falling within a window period for possible HIV infection pending the outcome of the DBS HIV PCR testing.
or individual-level benefits pending further with at least 1 other PLHIV who was not virally evidence.54 suppressed. Therefore, targeting testing among That said, the rapid recency algorithm has the contacts of recently infected individuals could demonstrated capacity to identify individuals improve the capacity of programs to detect and who became HIV infected within the past year, treat previously undiagnosed individuals while fo- and the U.S. President’s Emergency Plan for AIDS cusing prevention services among individuals Relief has identified the scale up of recency testing facing the greatest infection risks. Moreover, con- as a “minimum standard” for HIV program imple- ducting AHI screening among the network con- Testing the mentation in an expanding set of countries receiv- tacts of recently infected individuals, as well as contacts of ing U.S. government support.59 While there targeting AHI screening among individuals with recently infected currently is no rationale to offer differentiated similar risk and sociodemographic profiles to those individuals could counseling or clinical HIV treatment support to with recent infections, could increase capacity to improve a PLHIV with recent versus long-term HIV infection, detect, treat, and prevent transmission during program’s it is more likely that persons with recent infections AHI. capacity to detect are part of ongoing transmission networks. To leverage recency testing data to help focus and treat Individuals with recent infection were recently AHI screening as part of an HIV micro-epidemic undiagnosed acutely infected and were recently in risk contact control model, programs would need to adopt individuals.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 621 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
strategies to: (1) integrate AHI screening into prac- about the presence of 4 current symptoms (fever, tice; (2) secure client informed consent for recency lymphadenopathy, oral thrush, and weight loss), testing and the confidential use of those results; and 3 risk factors in the past 6 months (receptive (3) support confidential profiling of individuals condomless anal intercourse, more than 5 sexual with recent and acute infections; (4) target testing partners, and gonorrhea).47 with AHI screening in populations, settings, and Prioritizing AHI screening as part of index test- networks aligned to these profiles; and (5) priori- ing for the risk network contacts of recently tize index testing with AHI screening among infected individuals, as well as eventually for the the contacts of recently and acutely infected contacts of individuals with AHI as these are iden- individuals. tified, could then help programs increase the like- An illustrative workflow for AHI screening lihood of detecting people with AHI. Case profiles supplemented by recency testing to improve focus can be developed to describe actionable character- is shown in Figure 4. AHI screening could be con- istics of recently and acutely infected individuals ducted using a pooled PCR approach on dried to guide the prioritization of targeted, differentiat- blood spot samples as previously described. ed HIV testing, AHI screening, and prevention However, in this instance, we outline an approach strategies for these individuals and members of in which members of key and priority populations their risk networks. Individuals who are recently could be offered screening for AHI through com- or acutely infected can also be offered opportuni- bined use of a sensitive fourth-generation rapid di- ties to serve as peer mobilizers and/or to distribute agnostic test like the Alere HIV Combo, as well as a HIV self-test kits to help accelerate linkages of risk- and symptom-based verbal screening tool. A their network contacts to testing, treatment, and potentially useful example of a tool validated us- prevention services, as relevant. ing AHI data from the Amsterdam Cohort Studies As individuals are screened as potentially among men who have sex with men consisted of a having AHI, these individuals can be offered im- self-administered weighted survey inquiring mediate confirmatory HIV RNA testing and HIV
FIGURE 4. An Illustrative Workflow for Acute HIV Infection Screening, Additionally Applying Recency Testing Data to Help Improve Focus
Abbreviations: Abþ, antibody positive; Ab , antibody negative; AHI, acute HIV infection; Agþ, antigen positive; Ag , antigen neg- ative; ART, antiretroviral therapy; HIVþ, HIV-positive; PLHIV, people living with HIV.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 622 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
treatment. On the rare occasion that HIV RNA CONCLUSIONS confirmatory testing is not immediately available, Viral burden is the primary predictor of HIV- presumptive HIV treatment could be provided related morbidity, mortality, and ongoing trans- pending confirmation in a manner analogous to mission. Although a majority of PLHIV globally the provision of HIV postexposure prophylaxis, have achieved viral suppression through sus- which is generally considered safe and effective.60 tained access to HIV treatment, achieving an end An advantage of the proposed pooled PCR ap- to the HIV pandemic is contingent on addressing proach to detect AHI is immediate confirmation the preferences and needs of virally unsuppressed of HIV infection. Once confirmation is obtained, and persistently unserved and underserved PLHIV individuals can be sustained on ART, having and members of their risk networks. Proven solu- gained personal immunological benefits from ear- tions exist to prevent and treat HIV, but the ly treatment and having reduced the likelihood of approaches and technologies needed to differenti- ongoing HIV transmission during AHI. For indivi- ate and focus support based on viral burden histor- duals who were screened as having presumptive ically have been limited. Now, with the expansion AHI but who later are determined through HIV of viral load testing and an expanded set of options RNA testing to be uninfected, treatment can be to screen for and treat AHI, we may be better equipped to improve both the impact and efficien- discontinued with minimal risk of harm or of con- cy of efforts to accelerate epidemic control. tributing to development of drug-resistant HIV, in An HIV micro-epidemic control approach that a fashion similar to the discontinuation of HIV prioritizes personalized treatment support for postexposure prophylaxis. These persons can also PLHIV who are not virally suppressed—and in be assessed for the suitability of PrEP. the process focuses HIV testing and relevant HIV Current concerns about the potential impact of prevention and treatment support among their providing acute or recency test results to clients in- network members—offers a framework to inte- clude increased risk of criminalization of key grate these advances into current practice to populations, as well as criminalization of HIV maximize client benefits and overall impact. In transmission and increased risk of gender-based particular, such an approach offers a path to inte- 61 or intimate partner violence. Furthermore, sub- grate the detection and treatment of AHI into jecting patients to tests like recency assays that do routine programming, potentially curbing a sub- not provide additional clinical benefits raises ethi- stantial proportion of ongoing HIV transmission cal concerns. In principle, patients have a right to that occurs during this period and has historically know any information that is part of their medical continued apace beyond the reach of efforts to le- file, and additional information about the current verage HIV treatment as prevention at scale. state of a person’s infection may help providers However, the ultimate advantages of such an ap- enhance counseling, reinforce a person’s reduc- proach remain largely undocumented. Additional tion in risk behaviors that lead to onward trans- investments in the development, implementation, mission, improve partner elicitation process and evaluation of practical strategies to differenti- within index testing services, and allow providers ate support based on viral burden are needed to to use results to prioritize index cases for partner assess the real-world benefit of the proposed HIV notification services. Adverse events or harm re- micro-epidemic control approach. lated to return of acute or recency results have not been reported so far from early programs Acknowledgments: We would like to thank Steve Wignall, Caroline Francis, Rachel Coley, Janet Robinson, Pat Sadate-Ngatchou, Chris 54,62,63 implementing these services, but few stud- Akolo, Meghan DiCarlo, Matthew Avery, and Hally Mahler for their ies have systematically evaluated outcomes relat- technical contributions to this article. ed to potential harm, client perspectives, or the Competing interests: None declared. perspectives of partners of index clients. Given the potential public health benefit of engaging the REFERENCES risk contacts of recently or acutely infected indivi- 1. UNAIDS. UNAIDS Data 2020. UNAIDS; 2020. Accessed duals, the assessment of these outcomes is imper- September 12, 2020. https://www.unaids.org/en/resources/ ative to provide guidance around the messaging of documents/2020/unaids-data 2. Lundgren JD, Babiker AG, Gordin F, et al. Initiation of antiretroviral results in a manner which minimizes risks and therapy in early asymptomatic HIV infection. N Engl J Med. optimizes potential benefits. 2015;373(9):795–807. CrossRef. Medline
Global Health: Science and Practice 2020 | Volume 8 | Number 4 623 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
3. Cohen MS, Chen YQ, McCauley M, et al. Antiretroviral therapy for 21. Chibo D, Kaye M, Birch C. HIV transmissions during seroconversion the prevention of HIV-1 transmission. N Engl J Med. 2016;375 contribute significantly to new infections in men who have sex with (9):830–839. CrossRef. Medline men in Australia. AIDS Res Hum Retroviruses. 2012;28(5):460–464. 4. Eisinger RW, Dieffenbach CW, Fauci AS. HIV viral load and trans- CrossRef. Medline missibility of HIV infection. JAMA. 2019;321(5):451–452. CrossRef. 22. Phanuphak N, Teeratakulpisarn N, van Griensven F, et al. Medline Anogenital HIV RNA in Thai men who have sex with men in Bangkok 5. Bavinton BR, Pinto AN, Phanuphak N, et al. Viral suppression and during acute HIV infection and after randomization to standard vs. J Int AIDS Soc HIV transmission in serodiscordant male couples: an international, intensified antiretroviral regimens. . 2015;18 prospective, observational, cohort study. Lancet HIV. 2018;5(8): (1):19470. CrossRef. Medline e438–e447. CrossRef. Medline 23. Powers KA, Ghani AC, Miller WC, et al. The role of acute and early 6. Rodger AJ, Cambiano V, Bruun T, et al. Risk of HIV transmission HIV infection in the spread of HIV and implications for transmission Lancet through condomless sex in serodifferent gay couples with the HIV- prevention strategies in Lilongwe, Malawi: a modelling study. . – positive partner taking suppressive antiretroviral therapy (PARTNER): 2011;378(9787):256 268. CrossRef. Medline final results of a multicentre, prospective, observational study. Lancet. 24. Brenner BG, Roger M, Stephens D, et al; Montreal PHI Cohort Study 2019;393(10189):2428–2438. CrossRef. Medline Group. Transmission clustering drives the onward spread of the HIV J Infect Dis 7. Crowell TA, Colby DJ, Pinyakorn S, et al. Acute retroviral syndrome is epidemic among men who have sex with men in Quebec. . – associated with high viral burden, CD4 depletion, and immune acti- 2011;204(7):1115 1119. CrossRef. Medline vation in systemic and tissue compartments. Clin Infect Dis. 2018;66 25. Kroon EDMB, Phanuphak N, Shattock AJ, et al. Acute HIV infection (10):1540–1549. CrossRef. Medline detection and immediate treatment estimated to reduce transmission J Int AIDS 8. UNAIDS. UNAIDS Fact Sheet. UNAIDS; 2020. Accessed September by 89% among men who have sex with men in Bangkok. Soc 10, 2020. https://www.unaids.org/sites/default/files/media_ . 2017;20(1):21708. CrossRef. Medline asset/UNAIDS_FactSheet_en.pdf 26. Grimsrud A, Bygrave H, Doherty M, et al. Reimagining HIV service 9. Quinn TC, Wawer MJ, Sewankambo N, et al. Viral load and het- delivery: the role of differentiated care from prevention to suppres- J Int AIDS Soc erosexual transmission of human immunodeficiency virus type 1. sion. . 2016;19(1):21484. CrossRef. Medline N Engl J Med. 2000;342(13):921–929. CrossRef. Medline 27. World Health Organization (WHO). Guidelines on HIV Self-Testing and Partner Notification: Supplement to Consolidated Guidelines on 10. Hughes JP, Baeten JM, Lingappa JR, et al. Determinants of per- HIV Testing Services coital-act HIV-1 infectivity among African HIV-1-serodiscordant . WHO; 2016. Accessed October 9, 2020. couples. J Infect Dis. 2012;205(3):358–365. CrossRef. Medline https://www.who.int/hiv/pub/vct/hiv-self-testing-guidelines/en/ 11. Baral S, Rao A, Sullivan P, et al. The disconnect between individual- 28. Katbi M, Adeoye O, Adedoyin A, Faturiyele I, Adegboye A, Bello M. level and population-level HIV prevention benefits of antiretroviral Virologic response among key populations living with HIV following treatment. Lancet HIV. 2019;6(9):e632–e638. CrossRef. Medline a switch to dolutegravir-based regimen in Southern Nigeria. September 5, 2020. CrossRef 12. van Griensven F, Guadamuz TE, de Lind van Wijngaarden JW, Phanuphak N, Solomon SS, Lo YR. Challenges and emerging op- 29. Bisaso KR, Karungi SA, Kiragga A, Mukonzo JK, Castelnuovo B. A portunities for the HIV prevention, treatment and care cascade in men comparative study of logistic regression based machine learning who have sex with men in Asia Pacific. Sex Transm Infect. 2017;93 techniques for prediction of early virological suppression in antire- (5):356–362. CrossRef. Medline troviral initiating HIV patients. BMC Med Inform Decis Mak. 2018;18 (1):77. CrossRef. Medline 13. Stirratt MJ, Marks G, O’Daniels C, et al. Characterising HIV trans- mission risk among US patients with HIV in care: a cross-sectional 30. Martínez Pérez G, Cox V, Ellman T, et al. ‘I know that I do have HIV study of sexual risk behaviour among individuals with viral load but nobody saw me’: oral HIV self-testing in an informal settlement in above 1500 copies/mL. Sex Transm Infect. 2018;94(3):206–211. South Africa. PLoS One. 2016;11(4):e0152653. CrossRef. Medline CrossRef. Medline 31. Stevens DR, Vrana CJ, Dlin RE, Korte JE. A global review of HIV self- 14. Miller WC, Rosenberg NE, Rutstein SE, Powers KA. Role of acute and testing: themes and implications. AIDS Behav. 2018;22(2):497– early HIV infection in the sexual transmission of HIV. Curr Opin HIV 512. CrossRef. Medline AIDS – . 2010;5(4):277 282. CrossRef. Medline 32. Kurth AE, Celum C, Baeten JM, Vermund SH, Wasserheit JN. 15. Cohen MS, Shaw GM, McMichael AJ, Haynes BF. Acute HIV-1 in- Combination HIV prevention: significance, challenges, and opportu- fection. N Engl J Med. 2011;364(20):1943–1954. CrossRef. nities. Curr HIV/AIDS Rep. 2011;8(1):62–72. CrossRef. Medline Medline 33. UNAIDS. Fast-Tracking Combination Prevention: Towards Reducing 16. Pilcher CD, Tien HC, Eron JJ Jr, et al. Brief but efficient: acute HIV in- New HIV Infections to Fewer than 500,000 by 2020. UNAIDS; fection and the sexual transmission of HIV. J Infect Dis. 2004;189 2015. Accessed October 9, 2020. https://www.unaids.org/sites/ (10):1785–1792. CrossRef. Medline default/files/media_asset/20151019_JC2766_Fast-tracking_ combination_prevention.pdf 17. Rutstein SE, Ananworanich J, Fidler S, et al. Clinical and public health implications of acute and early HIV detection and treatment: a scoping 34. Baeten JM, Heffron R. Pre-exposure prophylaxis to intensify the fight review. JIntAIDSSoc. 2017;20(1):21579. CrossRef. Medline against HIV. Lancet Infect Dis. 2014;14(6):443–445. CrossRef. Medline 18. Hollingsworth TD, Pilcher CD, Hecht FM, Deeks SG, Fraser C. High transmissibility during early HIV infection among men who have sex 35. UNAIDS. Fast-Track Commitments to End AIDS by 2030. UNAIDS; with men—San Francisco, California. J Infect Dis. 2015;211 2016. Accessed October 9, 2020. https://www.unaids.org/sites/ (11):1757–1760. CrossRef. Medline default/files/media_asset/fast-track-commitments_en.pdf 19. Brenner BG, Roger M, Routy JP, et al. High rates of forward trans- 36. Revill P, Dwyer E. Pre-exposure prophylaxis is cost-effective for HIV in mission events after acute/early HIV-1 infection. J Infect Dis. the UK. Lancet Infect Dis. 2018;18(1):10–11. CrossRef. Medline – 2007;195(7):951 959. CrossRef. Medline 37. UNAIDS. Confronting Discrimination: Overcoming HIV-Related 20. Frange P, Meyer L, Deveau C, et al. Recent HIV-1 infection contri- Stigma and Discrimination in Health-Care Settings and Beyond. butes to the viral diffusion over the French territory with a recent in- UNAIDS; 2017. Accessed October 9, 2020. https://www.unaids. creasing frequency. PLoS One. 2012;7(2):e31695. CrossRef. org/sites/default/files/media_asset/confronting-discrimination_ Medline en.pdf
Global Health: Science and Practice 2020 | Volume 8 | Number 4 624 An HIV Micro-epidemic Control Approach to Stop HIV Transmission www.ghspjournal.org
38. Kalichman SC, Shkembi B, Wanyenze RK, et al. Perceived HIV stig- 51. Dowling W, Veldsman K, Katusiime M-G, et al. HIV-1 RNA testing of ma and HIV testing among men and women in rural Uganda: a pooled dried blood spots is feasible to diagnose acute HIV infection population-based study. Lancet HIV. 2020;S2352-3018(20) in resource limited settings. S Afr J Infect Dis. 2017;50–53. CrossRef – 30198 3. CrossRef. Medline 52. Pannus P, Fajardo E, Metcalf C, et al. Pooled HIV-1 viral load testing 39. Duong YT, Mavengere Y, Patel H, et al. Poor performance of the de- using dried blood spots to reduce the cost of monitoring antiretroviral termine HIV-1/2 Ag/Ab combo fourth-generation rapid test for de- treatment in a resource-limited setting. J Acquir Immune Defic Syndr. tection of acute infections in a National Household Survey in 1999;2013:64. CrossRef. Medline J Clin Microbiol – Swaziland. . 2014;52(10):3743 3748. CrossRef. 53. Fearnhill E, Gourlay A, Malyuta R, et al. A phylogenetic analysis of Medline human immunodeficiency virus type 1 sequences in Kiev: findings 40. Livant E, Heaps A, Kelly C, et al; VOICE Study Team. The fourth gen- among key populations. Clin Infect Dis. 2017;65(7):1127–1135. eration Alere TM HIV Combo rapid test improves detection of acute CrossRef. Medline J Clin Virol – infection in MTN-003 (VOICE) samples. . 2017;94:15 54. World Health Organization (WHO). WHO Working Group on HIV 21. CrossRef. Medline Incidence Measurement and Data Use: 3–4 March 2018, Boston, 41. Pilcher CD, Louie B, Facente S, et al. Performance of rapid point-of- MA, USA: Meeting Report. WHO; 2018. Accessed October 9, care and laboratory tests for acute and established HIV infection in 2020. https://apps.who.int/iris/handle/10665/272940 PLoS One San Francisco. . 2013;8(12):e80629. CrossRef. Medline 55. Duong YT, Dobbs T, Mavengere Y, et al. Field validation of limiting- 42. Alere COMBO and Alere q Detect in an HIV PrEP Program in antigen avidity enzyme immunoassay to estimate HIV-1 incidence in Thailand. ClinicalTrials.gov. Accessed September 21, 2020. cross-sectional survey in Swaziland. AIDS Res Hum Retroviruses. https://clinicaltrials.gov/ct2/show/NCT03194880 2019;35(10):896–905. CrossRef. Medline 43. Thai Red Cross AIDS Research Centre. Performance characteristics of 56. Kim AA, Behel S, Northbrook S, Parekh BS. Tracking with recency Alere COMBO and Alere q Detect in an HIV PrEP program in assays to control the epidemic. AIDS. 2019;33(9):1527–1529. Thailand. Thai Red Cross AIDS Research Centre. Published March 5, CrossRef. Medline 2018. Accessed October 5, 2020. https://www.prevention-trcarc. 57. Rice B, de Wit M, Willis R, et al. The Feasibility and Utility of HIV org/th/implementation-sciences/Performance-Characteristics-of- Recent Infection Testing in a Range of Routine Service-Provision Alere-COMBO-and-Alere-q-Detect-in-an-HIV-PrEP-Program-in- Contexts. MeSH Consortium; 2019. Accessed October 9, 2020. Thailand https://trace-recency.org/wp-content/uploads/2019/08/MeSH- 44. Wood E, Kerr T, Rowell G, et al. Does this adult patient have early report-on-HIV-recency-testing-in-routine-settings.pdf HIV infection? The Rational Clinical Examination systematic review. 58. Shah NS, Duong YT, Le LV, et al. Estimating false-recent classification JAMA – . 2014;312(3):278 285. CrossRef. Medline for the limiting-antigen avidity EIA and BED-capture enzyme immu- 45. Robb ML, Ananworanich J. Lessons from acute HIV infection. Curr noassay in Vietnam: implications for HIV-1 incidence estimates. Opin HIV AIDS. 2016;11(6):555–560. CrossRef. Medline AIDS Res Hum Retroviruses. 2017;33(6):546–554. CrossRef. 46. Rutstein SE, Pettifor AE, Phiri S, et al. Incorporating acute HIV Medline screening into routine HIV testing at sexually transmitted infection 59. U.S. President’s Emergency Plan for AIDS Relief (PEPFAR). PEPFAR clinics, and HIV testing and counseling centers in Lilongwe, Malawi. 2019 Country Operational Plan Guidance for all PEPFAR Countries. J Acquir Immune Defic Syndr. 2016;71(3):272–280. CrossRef. PEPFAR; 2018. Accessed October 9, 2020. https://www.state.gov/ Medline wp-content/uploads/2019/08/PEPFAR-Fiscal-Year-2019- 47. Dijkstra M, de Bree GJ, Stolte IG, et al. Development and validation Country-Operational-Plan-Guidance.pdf of a risk score to assist screening for acute HIV-1 infection among 60. U.S. Centers for Disease Control and Prevention (CDC). Updated men who have sex with men. BMC Infect Dis. 2017;17(1):425. Guidelines for Antiretroviral Postexposure Prophylaxis After Sexual, CrossRef. Medline Injection Drug Use, or Other Nonoccupational Exposure to HIV— United States, 2016 48. Hoenigl M, Weibel N, Mehta SR, et al. Development and validation . CDC; 2016. Accessed September 11, 2020. of the San Diego Early Test Score to predict acute and early HIV in- https://stacks.cdc.gov/view/cdc/38856 fection risk in men who have sex with men. Clin Infect Dis. 2015;61 61. Abdool Karim Q, Macklin R, Gruskin S, et al. HIV recency testing: (3):468–475. CrossRef. Medline should results be disclosed to individuals tested? J Int AIDS Soc. 49. De Souza MS, Phanuphak N, Pinyakorn S, et al; RV254SEARCH 2020;23(8):e25584. CrossRef. Medline 010 Study Group. Impact of nucleic acid testing relative to antigen/ 62. Nyirenda R. Recent infection surveillance among pregnant adoles- antibody combination immunoassay on the detection of acute HIV cent girls and young women in Malawi. Paper presented at: IAS infection. AIDS. 2015;29(7):793–800. CrossRef. Medline Conference on HIV Science; July 23, 2019; Mexico City, Mexico. 50. van Schalkwyk C, Maritz J, van Zyl GU, Preiser W, Welte A. Pooled 63. Ngoc B, Nguyen V. No time to wait: how community-led enhanced PCR testing of dried blood spots for infant HIV diagnosis is cost effi- partner services are reaching key populations in Vietnam. Paper cient and accurate. BMC Infect Dis. 2019;19(1):136. CrossRef. presented at: IAS Conference on HIV Science; July 23, 2019; Mexico Medline City, Mexico.
Peer Reviewed
Received: November 26, 2019; Accepted: September 29, 2020; First published online: November 17, 2020
Cite this article as: Cassell MM, Wilcher R, Ramautarsing RA, Phanuphak N, Mastro TD. Go where the virus is: an HIV micro-epidemic control ap- proach to stop HIV transmission. Glob Health Sci Pract. 2020;8(4):614-625. https://doi.org/10.9745/GHSP-D-19-00418
© Cassell et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-19-00418
Global Health: Science and Practice 2020 | Volume 8 | Number 4 625 COMMENTARY
Capturing Acquired Wisdom, Enabling Healthful Aging, and Building Multinational Partnerships Through Senior Global Health Mentorship
C. Norman Coleman,a John E. Wong,b Eugenia Wendling,a Mary Gospodarowicz,c Donna O’Brien,a Taofeeq Abdallah Ige,d Simeon Chinedu Aruah,d,e David A. Pistenmaa,a,f Ugo Amaldi,g Onyi-Onyinye Balogun,a,h Harmar D. Brereton,a,h Silvia Formenti,a,h Kristen Schroeder,a,i,j Nelson Chao,a,i,j Surbhi Grover,a,k,l Stephen M. Hahn,k James Metz,k Lawrence Roth,a Manjit Dosanjha,m
Key Messages INTRODUCTION n opportunity to have a substantial impact on mul- n Capturing the acquired wisdom and experience of A mentors in global health offers a capstone for their tiple challenging societal problems exists in simul- careers and provides a purposeful healthspan for taneously addressing the following: (a) the urgent need these professionals to continue to be engaged in for sustainable health care; (b) the importance of men- meaningful work while leveraging their expertise to torship in enabling the emergence of new generations solve challenging health care problems. of leaders; (c) the essential need for cross-cultural com- 1 n Senior professionals can mentor early career petency to address global crises through problem solv- leaders to help them balance their professional ing across societal boundaries; and (d) options for commitments, interest in global health, and continued productivity by the increasing number of development of needed skills, such as older people. Sustainable health care needs to build on understanding the nuances of cultural competence and adapting solutions to different environments. cancer care, which requires urgent intervention and encompasses noncommunicable and infectious diseases n Institutional leaders, particularly in academic in low- and middle-income countries (LMICs) and geo- medical centers, recognize the importance of global engagement vis-à-vis their educational graphically isolated populations in high-income coun- mission and for recruiting and retaining faculty tries (HICs). Capacity building to meet the cancer care and can benefit economically and gap, which builds sustainable infrastructure for overall programmatically from supporting experienced health care and economic development, can be done senior faculty or retirees to support these efforts. through twinning programs that engage senior health n Program builders should include the opportunity care professionals in meaningful mentoring roles. As for altruistic human service as an integral part of a the capstone of a career, these professionals thereby cre- career and highlight that they can access senior ate next-generation leaders within LMICs and their own mentors and retirees who provide world-class institutions. This article addresses such opportunities expertise and mentorship at “volunteer prices.” available for individuals in the latter part of their careers including postretirement done either as a continuation of their role as career-long mentors or as a new challenge a International Cancer Expert Corps, Washington, DC, USA. to be met with their lifelong experience. The expanding b National University Health System, Singapore. and branching tree of mentors to mentees enables a ca- c Princess Margaret Cancer Center, University of Toronto, Toronto, Canada. d National Hospital, Abuja, Nigeria. reer path in global health and geometric growth to fill in e Abuja College of Medicine, Nigeria. the current enormous capacity gap. f Radiation Research Program, National Cancer Institute, Washington, DC, USA. g TERA Foundation, Novara, Italy. h Department of Radiation Oncology, Weill-Cornell Medical Center, New York City, NY, USA. PURPOSEFUL AGING i Duke University School of Medicine, Durham, NC, USA. The challenges facing society regarding the aging of the j Bugando Cancer Center, Mwanza, Tanzania. population are complex. Concepts that have emerged k University of Pennsylvania, Philadelphia, PA, USA. l over the past few years to address these challenges in- University of Botswana & Princess Marina Hospital, Gaborone, Botswana. m “ — European Organization for Nuclear Research (CERN), Geneva, Switzerland. clude that of healthspan the period of life spent in Correspondence to C. Norman Coleman ([email protected]). good health, free from the chronic diseases and
Global Health: Science and Practice 2020 | Volume 8 | Number 4 626 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
disabilities of aging”2 and the benefit to purpose in population age distribution offers a golden oppor- How best to life (PIL) for improved health outcomes. Musich et tunity to capture global wisdom, address inequal- remain a useful 3 al. noted: ities, and leverage mentorship and innovative contributor to technology to enable sustainable improvement of ’ PIL is strongly associated with improved mental and one scommunity global health. How best to remain a useful contrib- and society is a physical health outcomes among older adults. Thus, utor to one’s community and society is a predict- predictable interventions to improve and/or maintain higher levels able challenge, especially for professionals who of PIL over time may promote successful aging. have developed the highly sophisticated skill sets challenge, especially for This article describes opportunities for profes- required for health care and desire to continue to use their professional knowledge meaningfully. highly skilled sionals to utilize their time and expertise to ad- These senior members of a profession also have professionals who dress the unacceptable gap in cancer care in perspective on the current economic situation in desire to continue underserved communities in LMICs and in geo- health care, medical and scientific knowledge, and to meaningfully graphically isolated areas in HICs. Regardless of societal trends, as well as broad hands-on patient use their whether this type of activity promotes longer or engagement skills that are particularly relevant in 4 knowledge. healthier lives, it captures expertise that is all too health care in which training and advancement fol- often lost and thereby transfers experience and low a skill-based apprenticeship model. wisdom to younger generations. PERSON-TO-PERSON CONNECTIVITY UNIQUE APPROACH TO THE AS A SOLUTION SET CHALLENGE OF GLOBAL HEALTH The unprecedented scope of the problems facing CARE humanity today, including climate change, wealth Cancer and other noncommunicable diseases (NCDs) disparities, xenophobia and related terrorism, poten- represent an increasing share of the global burden of tial for pandemics, and depletion of natural resources, disease in both resource-rich and -poor countries, pri- among others, absolutely requires problem solving marily due to aging, industrialization, sedentary life- across cultures and boundaries. The necessary style, pollution, diet, and the successful approaches to trusted partnerships/friendships and cultural com- and investment in tackling infectious diseases.5,6 petence can come from career-long diplomats, Indeed, addressing the full spectrum of cancer care— altruists, and science-based collaborations, bringing prevention, screening, diagnosis, treatment, and in opportunities for groups such as Peace Corps long-term follow-up—requires addressing the oth- volunteers,10 professional societies, and non- er major NCDs, such as respiratory, cardiovascular, governmental organizations. Such organizations and metabolic diseases, as well as infectious dis- span generations, from the eager student to the eases involved in cancer etiology and those related individuals with decades of experience. The life- to treatment.7 LMICs lack infrastructure, resources, long acquired wisdom of the latter is often lost to and expertise to address this problem. For example, retirement, but it is necessary for effective transi- the workforce shortfall in LMICs is highlighted by tions and the transmission of knowledge. Helping the Lancet Oncology Commission’sGlobalTask those early in their career to visualize a career path Force on Radiotherapy for Cancer Control of the in altruistic service can be a powerful motivator and Union for International Cancer Control.8 Using reinforce their own career choices. current staffing models, this report estimates that, A novel approach to address the health care by 2035, an additional 30,000 radiation oncologists workforce shortfall is the working mentorship and over 100,000 technical personnel, as well as model of the International Cancer Expert Corps clinical support and research staff, will be needed (ICEC).11 It draws on a wide breadth of partners worldwide. The essential health care system exper- and includes the following: tise and infrastructure needs and the benefits that would be derived from filling these health care A collaborative multi-institutional and multi- gaps make this a formidable and compelling national organization with opportunities for a challenge. broad spectrum of experts, who are needed to Mentorship is recognized as an important ele- build an effective health care enterprise to opti- ment in health care training.9 Leveraging the ex- mize resource utilization and facilitate the transfer of professional and technologic experi- pertise and mentorship of senior experts can 12 alleviate this shortfall. Fortunately, because many ence and expertise people are living well past the historical retirement Assignments in established and emerging twin- age of 60–70 years, the upward shift of the ning partnerships with HIC expert academic
Global Health: Science and Practice 2020 | Volume 8 | Number 4 627 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
centers, professional societies, and private prac- may not have the expertise to fully utilize it. This tices mentoring programs in LMICs, thereby of- problem is being addressed by ICEC and its LMIC fering long-term guided progress as opposed to partners,16,17 Medical Physics for World Benefit,18 episodic visits6 the International Atomic Energy Agency Division of Human Health,19 and academia. Tools and resources to guide mentoring and program-building efforts including standard The Figure illustrates the mentorship model operating procedures, the detailed metrics in for patient-centered cancer care, which encom- passes a broad range of expertise including NCDs the ICEC 5-Step Progression Plan for Cancer and infectious diseases. Mentorship includes the Mentorship Care,13 and formal guidelines for education continuum of mentors, with senior mentors guid- includes the and training programs for global settings14 ing early- and mid-career mentors from well- continuum of Ways to contribute expertise to support volun- resourced programs (hubs) who jointly train and mentors, with teer education programs such as Chartrounds’ educate mentees and staff within LMICs and geo- senior mentors case conferences for LMIC participants15 graphically isolated regions in HICs (centers), guiding early- and Opportunities for mentors to get formal recog- thereby geometrically expanding the system of mid-career nition for their contributions, as part of a shared patient-centered care. Senior expertise, a very ex- “ mentors from mission, while assisting in the development of a pensive component of health care ( Solution shop” of Christensen et al.20), can be made avail- well-resourced career path in global health able much less expensively with this sustainable programs who Expertise can come from both people and volunteer mentorship approach. Sharing knowl- jointly train and technology. For health care in developing coun- edge and broad expertise in this manner enhances educate mentees tries and for developed countries in the future, and expands their value well beyond the one-to- within LMICs. where rising expenses are a major societal issue, one mentor-mentee relationship. This innovative building human and technology expertise togeth- paradigm captures acquired wisdom, which is of- er, using the rapidly growing area of artificial intel- ten lost following retirement, to benefit society. ligence and machine learning, can better utilize human resources. Technology requires appropri- ate training and support services. The teams pro- TWINNING: MENTOR-MENTEE viding care in some poorly resourced countries PARTNERSHIPS may have access to excellent (highly publicized The mentorship model illustrated in the Figure and often very expensive) equipment, but they works primarily through twinning programs that
FIGURE. Basic Mentorship Model of Expansion of Expertise for Mentored Patient-Centered Care
Abbreviation: ICEC, International Cancer Expert Corps.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 628 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
Dr. Norman Coleman and Dr. Surbhi Grover discuss future plans for mentorship and innovative technology in Botswana. ©2019 Manjit Dosanjh are collaborative relationships between HIC Dr. Onyinye Balogun, a radiation oncologist university departments or private practice pro- from Weill Cornell Medical School, has established grams (hubs) and programs/facilities in an LMIC training programs in Armenia that have enabled (centers). The value of mentorship and the expo- the radiation oncologists to jump forward a few nential impact of transferring experience are ap- decades in radiation oncology from 2-dimensional parent in the twinning programs that establish a radiation therapy to 3-dimensional techniques. proper infrastructure for education, training, and Training and ongoing telemedicine case discussions mentoring. This capacity-building strategy facili- enable further advancement in techniques that are tates the creation of a sustainable platform for the less toxic and, by allowing higher doses, more effec- mutual sharing of best practices and learning tive. Her work and that of her mentors led the dean through information and technology transfer. to establish a global oncology initiative at the med- The ultimate aim is for the centers to achieve the ical school. required level of expertise to become hubs for Surbhi Grover, MD, completed an MPH degree their respective regions. A successful international under mentorship advice and with support from pioneering example is the King Hussein Cancer the University of Pennsylvania. She has been Center in Jordan, which is now a regional leader hands-on in Botswana establishing evidence- in cancer care. Mentoring at the trainee level is ex- based cancer care guidelines. This work is a major emplified by the work of the Association of advance in care and has transformed the strategies Residents in Radiation Oncology Global Health to manage stock for chemotherapy as part of com- Initiative.21 For problems as large as the gap in prehensive care plans. As one of the first radiation global health care that may seem “too hard” to ad- oncologists to be on the ground in global health, dress, specific examples can make the solution less daunting and even an exciting personal challenge. her program is a highly sought-after rotation for Because going from concept to operational re- residents interested in pursuing careers in global ality is critical, we include a narrative example of health. successful mentorship from a mentor and a men- Kristin Schroeder is a pediatric hematologist- tee (Box) (additional examples are included in oncologist from Duke University who has helped the Supplement). An important starting point is establish pediatric cancer care in Tanzania. She that even pursuing a sustainable career in global has not only implemented a comprehensive can- oncology had been a challenge, yet these mentor- cer care infrastructure, but also helped establish a mentee teams have opened up this possibility to nongovernmental organization to provide care to an emerging generation committed to global any child with cancer. health. The mentor-mentee model has already Taofeeq Abdallah is a medical physicist in demonstrated success, as shown with examples in Nigeria who has established, under the mentor- Table 1. ship of CERN, education and training networking
Global Health: Science and Practice 2020 | Volume 8 | Number 4 629 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
Manjit Dosanjh with Taofeeq Ige co-chairing a session at a workshop on radiation therapy treatment systems held in Gaborone, Botswana. © 2019 Manjit Dosanjh
BOX. Perspectives From a Mentee and Mentor on Their Mentorship Mentee: Taofeeq Abdallah, IGE, PhD, Chief Consultant Physicist and Head of Medical Physics Department, National Hospital, Abuja, Nigeria
Mentor: Manjit Dosanjh, PhD, Senior Advisor for Medical Applications, European Organization for Nuclear Research (CERN), Geneva, Switzerland
Taofeeq Abdallah: The relationship between Manjit and myself since our first meeting in CERN in 2017 has been more than awesome. She brought a fresh perspectives between a mentor and mentee by trying to identify real-time with the situation in the LMIC’s and this has propelled me and my colleagues on this “side of the divide” to push ahead even more vociferously knowing fully well that we can always rally for support anytime that this is needed and she has never disappointed in all the occasions – always rising up to the challenge and offering advice that are most accurate and incisive.
The tangible benefits that this international mentoring relationship have engendered has been first to our numerous patients who have in one way or the other benefited from very rich advice that Manjit has been able to offer from time to time – raising our spirits even in the face of arduous and unfavorable conditions. Since the relationship impacts our patients, this has equally been of great benefit to me professionally and has had a concomitant net benefits to my hospital and even my interactions with colleagues in the region as the president of our professional association (FAMPO – Federation of African Medical Physics Organizations).
Manjit Dosanjh: I got to work with and to know Taofeeq much more closely when the Science and Technology Facilities Council (STFC) team started to prepare a proposal to conduct an Accelerator Design Study (ADS) for a medical linear accelerator (LINAC) for Overseas Development Agency countries to be submitted to the Global Challenges Research Fund.
At my suggestion, both Taofeeq and Simeon Chinedu Aruah were invited to participate in the preparation of the ADS to advise the STFC team about both clinical and medical physics challenges of LINAC use in Nigeria. During the period of the development of the ADS proposal, I realized that Taofeeq and Simeon were not used to communicating and working closely with each other. This fact provided a great opportunity for me to help bridge that gap and build a closer working relationship between them.
Since then, I have been guiding Taofeeq in how to prepare and submit his own projects; he led the last one with myself as a co-applicant. We are now working on a questionnaire gathering information for optimizing a LINAC prototype for future machines suitable for challenging environments. Also, David Pistenmaa and I accepted Simeon and Taofeeq’s invitation to contribute to peer-reviewed manuscripts that they originated and enjoyed the camara- derie in doing so. What has been most rewarding to us over the last 2 years has been to see not only Taofeeq be- come a more understanding and caring leader but also to see the relationships between him and Simeon and their departments growing. These improving collaborations will continue to enhance the quality of treatment of patients with cancer and the reputation of National Hospital Abuja.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 630 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
TABLE 1. International Cancer Expert Corps Mentoring Relationships
Mentee Mentor Year Examples of Achievement Started National Center of Oncology, Yerevan, Armenia, and Weill Cornell, New York, USA Onyinye Balogun, MD Silvia Formenti, MD 2015 Established a training program to Assistant Professor of Chairman facilitate transitioning from 2-dimensional Radiation Oncology, Weill Department of Radiation Oncology, to 3-dimensional treatment planning for Cornell Weill Cornell treatment of cancer with radiotherapy, Associate Director of the Meyer Cancer with a focus on breast cancer Center and Radiation Oncologist in Established education and ongoing train- Chief,New York Presbyterian Hospital ing program to ensure proper implemen- Harmar Brereton, MD tation of Clinical Professor of Medicine image-guided brachytherapy for cervical Geisinger Commonwealth School of cancer. Training is delivered through Medicine and Clinical Assistant Professor didactic lectures and teleconferences of Radiation Oncology, Weill Cornell offering patient case discussion and peer review Established the global oncology initiative at Weill Cornell Medicine Established one of the first ICEC twinning programs linking an emerging cancer treatment program in an LMIC with an advanced cancer treatment program in an HIC Princess Marina Hospital, Gaborone, Botswana, and University of Pennsylvania, Philadelphia, Pennsylvania, USA Surbhi Grover, MD, MPH Stephen Hahn, MD 2014 Increased evidence-based care Assistant Professor of FDA Commissioner establishing guidelines for the top 10 Radiation Oncology, Former Chair, Department of Radiation cancers in Botswana Perelman School of Medicine, Oncology Perelman School of Medicine, Created an educational exchange University of Pennsylvania University of Pennsylvania program between University of University of Botswana & James Metz, MD Botswana and University of Pennsylvania Princess Marina Hospital, Chair, Department of Radiation Developed research programs between Gaborone, Botswana Oncology, Perelman School of Medicine, the University of Botswana and University of Pennsylvania University of Pennsylvania Radiation Oncology expanding research capacity at University of Botswana and linking young investigators to international mentors to support research Advanced strategies to reduce stock-outs of chemotherapy and to improve systems to reduce delays in pathology diagnosed through an initiative with the American Society of Clinical Pathology Botswana is now a destination for radiation oncology residents pursuing careers in global health, orchestrated by the Association of Residents in Radiation Oncology Continued
Global Health: Science and Practice 2020 | Volume 8 | Number 4 631 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
TABLE 1. Continued
Mentee Mentor Year Examples of Achievement Started Bugando Cancer Center, Mwanza, Tanzania, and Duke Children’s Hospital & Health Center, Durham, NC, USA Kristin Schroeder, MD, MPH Nelson Chao, MD, MBA 2014 Established a patient navigator program Pediatric Hematology- Donald D. and Elizabeth G. Cooke that offers education and caregiver Oncology Specialist, Professor guidance throughout the diagnosis and Pediatric Neuro-oncologist Chief, Division of Hematologic patient treatment Bugando Cancer Center, Malignancies and Cellular Therapy/BMT Developed a pediatric cancer clinical Mwanza, Tanzania Director, Global Cancer, Duke University database to monitor patient outcomes Duke Children's Hospital & School of Medicine Established a hospital-based cancer Health Center, North registry Carolina, USA Fostered a streamlined process to speed cancer diagnosis and access to treatment Implemented standard protocols for care Initiated research programs related to Burkitt lymphoma and retinoblastoma treatment, and impact of psychosocial support Founded the NGO, International Cancer Care and Research Excellence Foundation (iCCARE), a nonprofit whose mission is to give any child diagnosed with cancer the same chance of a cure regardless of where they live Her mentorship of 19 individuals includes 2 Fulbright scholars, 5 masters level students, 1 oncology fellow, 2 nurses, 1 resident, 3 medical students, and 3 undergraduate students National Hospital, Abuja, Nigeria, and the European Organization for Nuclear Research (CERN) Taofeeq Abdallah Ige, PhD Manjit Dosanjh, PhD 2017 Established cross-border networking, Chief Consultant Physicist Senior Advisor for Medical Applications, education, and research projects to and Head of Medical Physics CERN enhance the accessibility, effectiveness, Department, National and safety related to the use of medical Hospital, Abuja, Nigeria physics and technologies improving treatment techniques and patient outcomes Fostered mentoring relationships between individuals in HICs and LMICs providing access to expert knowledge, guidance, advice, and building collegial relationships Established knowledge- and information-sharing programs utilizing various platforms including WebEx and videoconferencing and attendance at global scientific meetings Facilitated engagement in research programs resulting in co-authorship on scholarly articles published in leading academic journal publications Continued
Global Health: Science and Practice 2020 | Volume 8 | Number 4 632 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
TABLE 1. Continued
Mentee Mentor Year Examples of Achievement Started National Hospital, Abuja, Nigeria, and the International Cancer Expert Corps (ICEC) Simeon Chinedu Aruah MD, David A. Pistenmaa, MD, PHD, FACR 2017 Fostered academic growth through co- MPH, FWACS Chief Scientific Program Director authorship on publications in top scien- Consultant Radiation and International Cancer Expert Corps tific oncology journals Clinical Oncology Manjit Dosanjh, PhD Presentation of quality papers in different National Hospital Abuja, Senior Advisor for Medical Applications, fora Nigeria CERN Capitalized on opportunities to travel Lecturer University of Abuja outside Nigeria to attend international College of Medicine, Nigeria workshops, which has widened access to Head of Department world class education and training, Radiation and Clinical resulting in improved delivery of quality Oncology cancer care in Nigeria National Hospital Abuja, Increased global visibility of National Nigeria Hospital Abuja through representing Nigeria in the 63rd International Atomic Energy Agency general assembly in September 2019 in Vienna, Austria, and an invitation to represent Nigeria at the UN Disarmament Conference in New York City in May 2020 (postponed be- cause of COVID-19) Increased respect and enhanced image of the National Hospital Abuja within the scientific community Improved the quality of academic lec- tures to resident doctors and undergrad- uate medical students resulting in the fostering of new mentoring relationships within Nigerian hospitals and academic medical centers
Abbreviations: COVID-19, coronavirus disease 2019; HIC, high-income country; ICEC, International Cancer Expert Corps; LMICs, low- and middle-income countries; NGO, nongovernmental organization.
Dr. Surbhi Grover teaches staff in Botswana on the details of a radiation therapy field. Photo credit: ©2015 Surbhi Grover
Global Health: Science and Practice 2020 | Volume 8 | Number 4 633 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
Taofeeq Ige, a medical physicist, educates staff on the technical specifications for radiation dose measurement in Abuja, Nigeria. Photo credit: ©2017 Taofeeq Ige
to enhance safety and teach a sophisticated tech- Critically, the focus on cancer as part of the nique for radiation therapy. health care system encompasses the other Simeon Aruah, MD, MPH, is a young lecturer NCDs—respiratory, cardiovascular, and meta- in radiation and clinical oncology (includes medi- bolic diseases—and infectious diseases that are cal oncology) in Nigeria, who has had a rapid linked to both cancer etiology and complica- Cancer is a logical growth in his academic career with assistance in tions of treatment. Cancer is a logical focus in focus in that it has conducting studies and preparing manuscripts that it has a sense of urgency, similar to infec- asenseof and presentations from his mentors. His linkage tious diseases, and it can be a focal point for urgency, similar to with world-renowned academic mentors in- community involvement. Thus, an opportunity infectious creased the visibility of his program and cancer exists for volunteers with a wide range of skills diseases, and it care in Nigeria. His talent, enthusiasm, and confi- and expertise, including medicine, a broad can be a focal dence have grown, and he has already repre- range of scientific disciplines, and other profes- sions (“broad support” such as cultural experts, point for sented Nigeria at the International Atomic Energy communications, logistics, finance, and legal), community Agency general assembly and will do so at the UN Disarmament Conference in New York City. to effectively transfer knowledge and wisdom, involvement. while reducing the expense associated with Interest in careers involving global oncology personnel. Thus, global mentorship teams can has surged with these pioneering examples, in- educate one another and provide mentorship cluding program leaders willing to support to the local champions who are building pro- trainees and faculty as part of a career path. The grams in underserved communities, enabling experience of the co-authors of this article can at- the geometric expansion of health care neces- test to the positive impact that the mentor-mentee sary to address the enlarging workforce short- relationship has on stimulating transgenerational fall. Cultural competency1 is essential, and it idea sharing and generating energy and a positive benefits from those with in-country experi- outlook for what can be done, despite challenges ence. Answering the questions of “What can I that appear discouraging. do, and how do I do it?” is facilitated by struc- ture with achievable expectations. For a men- OPPORTUNITY FOR A BROAD tor, the expectation is only a 20% time commitment (8 hours per week on average), RANGE OF EXPERT MENTORS with the vast majority of the mentoring by Improved health care in general, building on the planned teleconferencing (with some bidirec- spectrum of cancer care from prevention through tional travel possible) through protocol- and diagnosis, treatment, and long-term follow-up guideline-based care, rather than individual care, are the deliverables. Enabling this goal case management. As described by Crisp,22 requires contributions including and well beyond knowledge and models for care will also evolve patient care delivery from a broad range of from mentees to mentors through reverse experts, as shown in Table 2. innovation.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 634 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
TABLE 2. Broad Spectrum of Expertise Needed for Complex System Solutions in Cancer Care
Medical Science, Non-MD Support
Radiation, medical, and pediatric Prevention and screening Educational tools oncologists Epidemiologists Finance Palliative care Medical physicists Hospital/medical Surgeons including subspecialists Technologists administration Nurses Basic and translational scientists International policy Pathologists Medical education Patient advocacy Radiologists Treatment guidelines Economists General internists Statisticians Sociologists Primary care Social scientists Social workers Infectious diseases Political scientists Cultural experts Gerontologists Regulatory affairs specialists Diplomats Pharmacologists Pharmacists Communications Psychologists Data management and big data Cancer survivors Public health science Information technology Emergency medicine Legal Development
DISCUSSION model creates long-standing relationships and pro- In our opinion, raising the mandatory retirement grams built around the broad range of skills trans- age or eliminating it altogether—as is happening ferred to the mentored specialists and staff in the in many societies—presents new opportunities local communities (Table 2). for those affected by necessary transitions in lead- Several similar large-scale mentorship model ership in health care organizations, governmental programs have found the keys for successful im- and international diplomatic organizations, and plementation to be convenience, flexibility, and academia that free senior personnel to mentor purposefulness. Such examples include the fol- within step-down roles at work or later in retire- lowing: (1) The AARP Foundation Experience ment. This transition of lives and careers provides Corps’ intergenerational volunteer-based tutoring Retirement exceptional opportunities for the older generation program designed to help elementary school stu- provides to pass on its knowledge and wisdom to the youn- dents improve their reading levels and to help exceptional ger generations through mentorship, while en- older adults enrich their lives through literacy,25 opportunities for hancing the quality of their own lives in their (2) the Returned Peace Corps Volunteers,9 (3) the older individuals later years. This sentiment was recently highlight- Japanese government’s “The Community-based to pass on their ed by Jane E. Brody in The New York Times “Want Integrated Care System” providing comprehensive knowledge and ”23 to leave a legacy? Be a mentor. up-to-the-end-of-life support in every communi- wisdom, while For those interested in a continued purpose in ty,26 (4) Singapore’s Action Plan for Successful enhancing their life related to their profession, addressing both Aging that enables seniors to learn new skills in own quality of life. 4,24 healthspan and lifespan requires opportunities joyful endeavors and to deploy these skills, and to use their skills by volunteering time and exper- (5) the National University Health System and tise. This trend (“purposeful healthspan”) utilizes the National University of Singapore’sworkwith this experience at a low cost for a wide range of multiple government agencies to enable whole organizations and interest groups, such as profes- precincts to exploit the elements of successful ag- sional societies, religious organizations, and speci- ing, thus future-proofing Singapore as a livable fic social causes. The model presented here is based city for people of all ages.27 This “wholeofsocie- upon periodic short visits followed by sustainable ty” and “whole of government” approach will en- commitments and continuous mentoring of those able societal change to take place and remain working on site through teleconferencing. The sustainable.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 635 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
CONCLUSION theinfrastructureofcancercarefacilitiesand The confluence of opportunities for continuity expanding the breadth of expertise available to and the spectrum of expertise from senior mentors them, these facilities can serve as focal points for to those early in their career has not been more the development of sustainable on-the-ground pro- apparent than in the current COVID-19 pandem- grams that can have substantial health and econom- ic. On the one hand, the call for retirees to return ic benefits beyond cancer care. Transformational to health care28 speaks to how senior and world- models, as outlined in this article, offer opportunities renowned experts’ skills are useful for their for visionary investments, altruistic contributions, primary expertise and for their role modeling, and exciting and meaningful action for a purposeful gravitas, and potential direct support. Yet, on the aging and improved healthspan. other hand, greater awareness of such usefulness and the presence of senior experience and wisdom Competing interests: None declared. might have averted the wholesale dismissal of 7,300 Peace Corps volunteers29 and fostered a REFERENCES more appropriate transition. 1. Unite for Sight. Cultural competency. Accessed March 1, 2020. Older individuals have opportunities to serve http://www.uniteforsight.org/cultural-competency/ society and humanity. Such opportunities (1) pro- 2. Kaeberlein M. How healthy is the healthspan concept? Geroscience. vide a career capstone, (2) allow timely transfer of 2018;40(4):361–364. CrossRef. Medline institutional responsibility to next-generation lea- 3. Musich S, Wang SS, Kraemer S, Hawkins K, Wicker E. Purpose in life Popul Health ders, (3) establish mentorship relationships for and positive health outcomes among older adults. Manag. 2017;21(2):139–147. CrossRef. Medline world-renowned experts with dedicated profes- 4. Pizzo PA. Prescription for longevity in the 21st century renewing sionals in underserved and geographically remote purpose, building and sustaining social engagement, and embrac- health care regions, (4) provide expensive exper- ing a positive lifestyle. JAMA. 2020;323(5):415–416. CrossRef. tise at “volunteer prices,”11,20 (5) present a model Medline for geometric expansion of diverse expertise and 5. Bertram MY, Sweeny K, Lauer JA, et al. Investing in non- innovative technology that enables development communicable diseases: an estimation of the return on investment for prevention and treatment services. Lancet. 2018;391(10134):2071– of the capacity to effectively address the burgeon- 2078. CrossRef. Medline ing burden of cancer and other NCDs, (6) establish 6. Partridge L, Deelen J, Slagboom PE. Facing up to the global chal- a mentor-based career path for altruistic human lenges of ageing. Nature. 2018;561(7721):45–56. CrossRef. service that is an endangered species in the current Medline “bottom line” finance-driven health care system, 7. Coleman CN, Wendling EN, Pistenmaa DA. A broad impact for (7) emphasize the importance of cultural compe- global oncology. JAMA Oncol. 2019;5(10):1397. CrossRef. tence and listening, and (8) utilize a systems solu- Medline tion approach to improve health care in LMICs by 8. Atun R, Jaffray DA, Barton MB, et al. Expanding global access to ra- diotherapy. Lancet Oncol. 2015;16(10):1153–1186. CrossRef. developing and sustaining local champions. The Medline presence of a gap that can be filled in a rather short 9. Burgess A, van Diggele C, Mellis C. Mentorship in the health profes- timeline from mentor to mentee to LMIC mentee sions: a review. Clin Teach. 2018;15(3):197–202. CrossRef. speaks to the need and impact. Medline Whether being engaged in purposeful activi- 10. Peace Corps. Returned volunteers. Accessed March 1, 2020. ties, such as those described in this article, will in- https://www.peacecorps.gov/returned-volunteers/ crease the length of one’s lifespan is under study. 11. International Cancer Expert Corps. Accessed March 1, 2020. Such study includes understanding the impact of https://www.iceccancer.org aging on the workplace.30 Interestingly, coincid- 12. Coleman CN, Love RR. Transforming science, service, and society. Sci Transl Med. 2014;6(259):259fs42. CrossRef. Medline ing with this current article, Dzau et al.31 recently “ 13. International Cancer Expert Corps (ICEC). ICEC 5-Step Progression announced The National Academy of Medicine Plan for Global Cancer Care©. Accessed May 12, 2019. https:// Grand Challenge in Healthy Longevity.” What is www.iceccancer.org/twinning-programs-overview/ undeniable is that the benefit of such activities 14. National Comprehensive Cancer Network. Accessed March 1, can meaningfully increase the breadth of one’s 2020. https://www.nccn.org/ experiences and contributions to society. Serving 15. Chartrounds.com. Radiation oncology specialists. Accessed March as a senior mentor to mentees in resource-poor 1, 2020. https://chartrounds.org/radiation_oncology.aspx regions of the world can have a spectacular impact 16. Pistenmaa DA, Dosanjh M, Amaldi U, et al. Changing the global ra- diation therapy paradigm. Radiother Oncol. 2018;128(3):393– on the goal of rectifying the staggering lack of ac- 399. CrossRef. Medline cess to care for patients with cancer and other 17. Dosanjh M, Aggarwal A, Pistenmaa D, et al. Developing innovative, NCDs in those regions. In addition, by improving robust and affordable medical linear accelerators for challenging
Global Health: Science and Practice 2020 | Volume 8 | Number 4 636 Capturing Acquired Wisdom and Building Multinational Partnerships www.ghspjournal.org
environments. Clin Oncol (R Coll Radiol). 2019;31(6):352–355. jo), mutual aid (Go-jo), social solidarity care (Kyo-jo), and govern- CrossRef. Medline mental care (Ko-jo). Biosci Trends. 2018;12(1):7–11. CrossRef. 18. Medical Physics for World Benefit. Accessed March 1, 2020. Medline https://www.mpwb.org/ 27. Singapore Ministry of Health (MOH). I Feel Young in My Singapore! Action Plan for Successful Aging 19. International Atomic Energy Agency (IAEA). Division of Human . MOH; 2016. Accessed March 1, Health. Accessed March 1, 2020. https://www.iaea.org/about/ 2020. https://sustainabledevelopment.un.org/content/ organizational-structure/department-of-nuclear-sciences-and- documents/1525Action_Plan_for_Successful_Aging.pdf applications/division-of-human-health 28. NPR. Coronavirus may call on retired medical workers to come back NPR 20. Christensen CM, Grossman JH, Hwang J. The Innovator’s Prescription: to work. . March 17, 2020. Accessed March 28, 2020. https:// A Disruptive Solution for Health Care. McGraw-Hill; 2017. www.npr.org/2020/03/17/817354091/coronavirus-may-call- on-retired-medical-workers-to-come-back-to-work 21. ASTRO. Affiliate Association of Residents in Radiation Oncology (ARRO). Accessed March 1, 2020. https://www.astro.org/ 29. Davidson J. The Peace Corps isn’t just bringing home 7,300 volun- Affiliate/ARRO teers because of the coronavirus. It’s firing them. Washington Post. March 20, 2020. Accessed March 28, 2020. https://www. 22. Crisp N. Turning the World Upside Down. CRC Press; 2010. washingtonpost.com/politics/the-peace-corps-isnt-just-bringing- New York Times 23. Brody JE. Want to leave a legacy? Be a mentor. . home-7300-volunteers-because-of-the-coronavirus-its-firing-them/ March 4, 2019. Accessed March 1, 2020. https://www.nytimes. 2020/03/20/69aa08f4-6ac0-11ea-b5f1-a5a804158597_story. com/2019/03/04/well/live/want-to-leave-a-legacy-be-a-mentor. html html 30. White MS, Burns C, Conlon HA. The impact of an aging population 24. Crimmins EM. Lifespan and healthspan: past, present, and promise. in the workplace. Workplace Health Saf. 2018;66(10):493–498. Gerontologist – . 2015;55(6):901 911. CrossRef. Medline CrossRef. Medline 25. AARP Foundation. Experience corps. Accessed March 1, 2020. 31. Dzau VJ, Inouye SK, Rowe JW, Finkelman E, Yamada T. Enabling https://www.aarp.org/experience-corps/ healthful aging for all—The National Academy of Medicine Grand 26. Sudo K, Kobayashi J, Noda S, Fukuda Y, Takahashi K. Japan’s Challenge in Healthy Longevity. N Engl J Med. 2019;381 healthcare policy for the elderly through the concepts of self-help (Ji- (18):1699–1701. CrossRef. Medline
Peer Reviewed
Received: March 7, 2020; Accepted: August 24, 2020; First published online: October 15, 2020
Cite this article as: Coleman CN, Wong JE, Wendling E, et al. Capturing acquired wisdom, enabling healthful aging, and building multinational part- nerships through senior global health mentorship. Glob Health Sci Pract. 2020;8(4):626-637. https://doi.org/10.9745/7GHSP-D-20-00108
© Coleman et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00108
Global Health: Science and Practice 2020 | Volume 8 | Number 4 637 ORIGINAL ARTICLE
Prevention of COVID-19 in Internally Displaced Persons Camps in War-Torn North Kivu, Democratic Republic of the Congo: A Mixed-Methods Study
– Kasereka M. Claude,a Muyisa Sahika Serge,a Kahindo Kahatane Alexis,b Michael T. Hawkesc g
Key Findings Résumé en français à la fin de l'article.
n Congolese internally displaced persons (IDPs) had high awareness and fear of COVID-19, but low specific knowledge. ABSTRACT Background: The coronavirus disease (COVID-19) pandemic n IDPs face major barriers to implementing COVID- poses a grave threat to refugees and internally displaced persons 19 prevention measures: crowded shelters, fre- (IDPs). We examined knowledge, attitudes, and practices with re- quent movements in and out of the camp for work, spect to COVID-19 prevention among IDPs in war-torn Eastern and lack of soap for hand hygiene. Democratic Republic of the Congo (DRC). n IDPs’ desire for peace and to return to their native Methods: Mixed-methods study with qualitative (focus group dis- homes, where COVID-19 precautions could be cussions, [FGDs]) and quantitative (52-item survey questionnaire) feasibly implemented, overshadowed their en- data collection and synthesis. thusiasm for other control measures such as a Results: FGDs (N=23) and survey questionnaires (N=164 IDPs; vaccine. N=143 comparison group) were conducted in May 2020. FGD participants provided narratives of violence that they had fled. IDPs were statistically more likely to have larger household size, Key Implications experience more extreme poverty, have lower educational attain- ment, and have less access to information through media and in- n Donors and policy makers should consider ternet versus the comparison group (P<.05 for the comparison providing consumables, such as soap for hand group). IDPs had a high level of awareness (99%) and fear hygiene and face masks, to implement COVID-19 (98%) of COVID-19, but lower specific knowledge (15% sufficient precautions. knowledge versus 30% among the comparison group, P<.0001), a difference which remained significant in a multivariable model n The national government or international aid adjusting for confounding. IDPs faced major barriers to imple- agencies should consider providing individual menting COVID-19 prevention measures. Physical distancing family dwellings (e.g., tarpaulin tents) to allow was impossible for IDPs in crowded shelters, and 70% reported IDPs to practice physical distancing. coming in close contact with someone other than a family mem- n National and international governments should ber within the past 24 hours (versus 56% of the comparison take serious measures to restore peace to the group, P=.014). Frequent movements in and out of the camp for area by controlling armed conflict. A safe return to subsistence left IDPs vulnerable to the introduction of COVID-19: their homes would allow IDPs to practice COVID- 61% left the camp on a daily basis and 65% had received a vis- 19 prevention without external aid. itor in the past month. Despite acceptance of hand hygiene for prevention, 92% lacked soap (versus 65% of the comparison group, P<.0001). IDPs’ desire for peace and to return to their na- tive homes, where COVID-19 precautions could be feasibly implemented, overshadowed their perceived benefits of measures a Department of Medicine, Université Catholique du Graben, Butembo, Democratic Republic of the Congo. such as a COVID-19 vaccine. b Department of Ophthalmology, Université Catholique du Graben, Butembo, Conclusions: These findings provide empiric evidence supporting Democratic Republic of the Congo. the vulnerability of IDPs to COVID-19 and call for action to pro- c Department of Pediatrics, University of Alberta, Edmonton, Canada. tect neglected displaced populations. d Department of Medical Microbiology and Immunology, University of Alberta, Edmonton, Canada. e School of Public Health, University of Alberta, Edmonton, Canada. INTRODUCTION f Stollery Science Lab, University of Alberta, Edmonton, Canada. s of August 25, 2020, there have been more than g Women and Children's Research Institute, University of Alberta, Edmonton, A Canada. 24 million cases of coronavirus disease (COVID- Correspondence to Michael Hawkes ([email protected]). 19) confirmed worldwide and 800,000 deaths, with
Global Health: Science and Practice 2020 | Volume 8 | Number 4 638 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
the United States and Europe experiencing impacts of COVID-19 have included sensitization the highest burden.1 African countries have campaigns on handwashing and distribution of reported 298,000 cases and 8,000 deaths (case soap to more than 100,000 IDPs.13 However, as fatality ratio 2.4%).2 Many low-resource set- noted by previous authors, recommendations for tings lack comprehensive surveillance and hand hygiene and physical distancing may be ex- laboratory testing to monitor the spread of tremely difficult to implement in a refugee or IDP COVID-19.3 The presence of displaced popula- camp. How do you self-isolate in a refugee tions (refugees and internally displaced persons camp?10 Several commentators have forewarned [IDPs]) adds further complexity to the COVID- of an impending crisis if COVID-19 strikes in refu- 19 pandemic and control measures in low-and gee or IDP camps.6,10,12 However, a paucity of em- middle-income countries (LMICs) in conflict pirical data from these areas is available. zones. Our overarching goal was to contribute to the In the Democratic Republic of the Congo improvement of prevention strategies against (DRC), the first case of COVID-19 was detected COVID-9 in IDP camps in the DRC. We aimed to on March 10, 2020, in a traveler returning from describe the knowledge, attitudes, and practices 4 France. Since then, more than 9,800 cases and (KAPs) of IDPs in Eastern DRC with respect to the 251 deaths have been confirmed across the DRC. prevention of COVID-19. Our primary endpoint Most cases have been detected in the capital city, was COVID-19 specific knowledge, which we com- Using qualitative Kinshasa. In the province of North Kivu, there pared between IDPs and individuals from neigh- and quantitative have been 203 cases as of August 25, 2020. The boring villages. Other specific objectives included: methods, we primary mode of transmission is community (1) to describe attitudes of IDPs with respect to aimed to provide based.5 In response to the pandemic, the govern- COVID-19 and its prevention; (2) to describe the rich data on a ment declared a state of public health emergency practices used by IDPs for preventing COVID-19; highly vulnerable on March 24, 2020, with broad closure of busi- and (3) to describe barriers faced by IDPs in imple- 4 and neglected nesses, gatherings, and travel. Since this initial menting recommended COVID-19 prevention group facing the lockdown, the government authorized gradual measures. reopening of businesses and public transportation COVID-19 (July 22); schools and universities (August 3); and pandemic in an churches, interprovincial travel, and international METHODS environment of 4 airports (August 15). Study Design extreme scarcity Refugees and migrants are among the world’s and insecurity. 6 We conducted a mixed-methods study with quali- most vulnerable people. Worldwide, there are tative focus group discussion (FGDs) and quantita- approximately 26 million refugees and 46 million tive (52-item survey questionnaire) data collection. IDPs, displaced due to insecurity and natural disas- 7 Mixed-methods research seeks to triangulate data ters. The DRC has the second highest number of 14 from qualitative and quantitative methods. IDPs of any country in the world (after Syria), esti- Convergence of findings from multiple methods mated at more than 5.5 million.8 Displaced popula- may enhance the validity of results (multiple oper- tions, housed in temporary shelters or camps, ationalism).15 We and others have previously used generally have limited access to quality shelter, this methodology to integrate community atti- sanitation, clean water, stable food supply, and tudes, behaviors, and responses into epidemiologi- health care. Under these conditions, COVID-19 16,17 – cal research. With respect to the survey prevention efforts may be challenging.9 11 questionnaire, the study followed a descriptive Impacts of the COVID-19 pandemic on dis- cross-sectional design. placed populations are predicted to be disastrous. Already, resettlement procedures have been sus- pended by the United Nations, alongside wide- Study Setting spread travel bans. The first case of COVID-19 in The province of North Kivu has a population of the island of Lesvos in March 2020 raised the 6.7 million inhabitants and an estimated 1.7 mil- alarm for the 20,000 residents of the Moria refugee lion IDPs.18 The Eastern provinces of the DRC camp, where distancing is a physical impossibility.9 have been the arena of a complex humanitarian In the world's largest refugee camp in Bangladesh, emergency for several decades. Mortality rates which shelters more than 855,000 Rohingya refu- are 70% above pre-war levels, due largely to pre- gees, preparations for COVID-19 have begun, such ventable and treatable infectious diseases rather as portable handwashing facilities at every commu- than the direct effects of conflict.19 Large-scale pop- nity center.12 In Nigeria, efforts to mitigate the ulation displacement has resulted in numerous IDP
Global Health: Science and Practice 2020 | Volume 8 | Number 4 639 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
camps throughout the area.20 The chronic (KMC) with tacit knowledge of the circumstances, threats to security have long been neglected by culture, and language of the IDPs chose the appro- the national government and the international priate wording of the questions and adapted the community.17 content of the questionnaire to the conditions in We selected 3 IDP camps (Mwangaza, Masosi, the IDP camp. The survey was administered to and Luvangira) located 2 to 5 km from the rural IDPs in the 3 camps as follows. commune of Oïcha, North Kivu. These temporary settlements consisted of groups of IDPs sheltered Sampling in school buildings or mud/thatch dwellings on 11 Statistical Unit and Estimation of Sample Size public grounds. Camp census data indicated the The unit of analysis was the household, defined as following populations: Mwangaza (1064 indivi- a family unit, often consisting of male and female duals, 200 households); Masosi (869 individuals, parents and their children. 176 households); and Luvangira (250 individuals, For our primary analysis, we focused on dif- 75 households). Aid for the camp is coordinated ferences in sufficient knowledge (binary vari- by the nongovernmental organization Charité able) between IDPs and the comparison group. Aide et Développement, Axe Oïcha, with inter- A standard sample size calculation indicated 21 mittent assistance from OXFAM, World Food that 138 households would be needed to detect 22 Programme, and International Committee of a difference of 15%, with 95% confidence and 23 the Red Cross. 80% power, assuming that the proportion of IDPs with sufficient knowledge was 20% or FGDs less, based on our previous study of knowledge Participants of FGDs were purposively selected of Ebola virus disease among IDPs.33 from the 3 IDP camps. Participants included adult women (3 FGDs) and men (2 FGDs) who were Sampling Technique heads of households, and youth (1 FGD). FGDs Geospatial sampling34 was used, as in previous were conducted in Congolese Swahili. Discussions studies of mobile populations. IDP camps were di- were recorded, translated, and transcribed into vided geographically into thirds and 1 area was English for subsequent analysis. FGDs lasted chosen at random. All households living within – 30 45 minutes and included 3 or 4 participants in the selected area were included, and the standard- 24 each group. The FGD topic guide was adaptive, ized questionnaire was administered to 1 adult allowing us to confirm findings and explore emerg- member from each household. Our sampling ing themes from each FGD session. Questions were technique was inspired by the cluster sampling open-ended and elastic, allowing participants to method developed by the WHO for monitoring shape the discussion. FGDs were continued until vaccine coverage.35,36 In this approach, a popula- 25 saturation. Thematic analysis was used to identi- tion is divided into a specified number of geo- 26 fy, analyze, and report themes in the FGDs. Two graphic “clusters” (in our case, camps) of a investigators (KMC and MH) read the transcripts known or estimable population size. Within each several times, noted preliminary ideas, produced cluster, the desired number of households are se- initial codes, then generated and refined themes. lected (in our case, approximately one-third were Representative quotations as well as statements of needed to reach the required sample size).36 particular interest were extracted to support the Several strategies are possible for household selec- themes. tion (e.g., enumeration of all households and simple random sampling from this list, or a “ran- Survey Questionnaire dom walk” sampling contiguous households).36 We developed a 52-item questionnaire based on However, random selection in more densely pop- past COVID-19 questionnaires used in Guyana27 ulated areas (e.g., urban settings or, in our case, an and Uganda.28 The choice of questionnaire items IDP camp) can be more challenging, given the was guided by a need for contextually appropriate more complex household types (e.g., apartment questions for low-income settings. We also drew buildings or, in our case, IDPs sheltering in school on past experience from past surveys conducted classrooms). In such settings, a common approach in IDP camps in the area29–32 and from the recent is to divide the geographic area of interest into Ebola virus disease epidemic16,17,33 to design ques- zones, randomly select a zone, and randomly se- tions that would be relevant and understood by lect a starting point within that zone. To reach the participants. A local Congolese physician our desired sample size, we needed to sample
Global Health: Science and Practice 2020 | Volume 8 | Number 4 640 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
approximately one-third of the camp households. 5-point Likert scale (“strongly agree,”“somewhat Therefore, we chose to divide the camp into thirds, agree,”“neutral,”“somewhat disagree,” and “dis- choose 1 cluster at random, and sample all house- agree,” with a possible “I don't know” response). holds within that cluster. For our comparison group, Affective response was measured using 2 ques- we surveyed the surrounding villages (nondisplaced tions about perceived severity and fear of COVID- population) using a nonprobability, purposive, max- 19. We assessed attitudes toward recommended imum variation sampling technique,37 choosing control measures, including physical distancing participants from all demographic categories (men and staying home without working. Mistrust and and women, full age spectrum, employment catego- rumors contributed to community resistance to ry, education attainment, and marital status). control measures during the recent Ebolavirus ep- Participants were aged 18 years or older. idemic in the DRC.17,41 Therefore, we included measures of institutional trust (2 items) and en- List of Variables dorsement of conspiracies related to the SARS- CoV-2 virus (2 items). The questionnaire consisted of several domains related to participant demographics, knowledge of COVID-19, attitudes, and behaviors for pre- Practices venting COVID-19. Participants indicated whether they had taken any action to prevent COVID-19 (“yes” or “no”). Among those who answered affirmatively, ac- Demographics tion(s) they had taken were chosen from a list of Individual respondent characteristics were collected: possible prevention methods (multiple responses age, sex, educational attainment, and marital status. permitted). With respect to physical distancing, In addition, we collected data on household charac- we inquired whether the participant had come in teristics (number of family members, members aged close contact with someone outside the family 60 years and older) and wealth indicators (owner- (responses: “yes” or “no”) and with how many ship of radio, cellular telephone, and bicycle). people they had shaken hands in the past 24 hours (responses: “none,”“1to5,”“more than 5”). Knowledge of COVID-19 Symptoms Participants selected 1 or more barriers to COVID- Participants were asked to choose from a list of 19 prevention from a list of possible barriers (mul- possible sources they drew upon for information tiple responses permitted, with an option to on COVID-19 (multiple selections possible). respond “I can fully protect myself against Using a list of symptoms, including 2 detractor COVID-19”). (false) symptoms (constipation and bleeding), participants were asked to agree whether COVID- Data Collection Technique 19 was associated with each symptom (“yes” or The standardized questionnaire was administered “no”). Recognition of asymptomatic transmis- as a verbal structured interview, with a study team 38,39 sion was assessed with the question: “A per- member asking questions in the local language son who is not sick and who has no symptoms and recording the participant's answers using a can still spread the virus” (responses: “true,” field-adapted electronic data collection tool, false,” or “I don’t know”). Agreement with com- KoboToolbox.42 Study team members were local mon misconceptions (transmission by mosquitos, Congolese health workers with tacit understand- prevention with spicy food) was assessed (responses: ing of the language and culture, biomedical un- “yes,”“no,” or “I don't know”). derstanding of COVID-19, and past experience Participants were considered to have sufficient administering surveys by verbal interview. knowledge of COVID-19 if they identified at least 1 of the cardinal signs and symptoms of COVID-19 40 Data Processing and Analysis (fever, cough, or difficulty breathing), recog- For descriptive statistics, we used median and nized the potential for asymptomatic transmis- interquartile range for continuous variables, and sion, and rejected misconceptions (bleeding as number (percentage) for proportions. Comparative symptom, transmission by mosquitos). statistics were computed using non-parametric methods: Mann-Whitney U-test for continuous Attitudes variables and Chi squared or Fisher’sexacttest We probed a range of attitudes related to COVID- for dichotomous variables, as appropriate. With re- 19 by assessing agreement with statements on a spect to our primary analysis, we expressed the
Global Health: Science and Practice 2020 | Volume 8 | Number 4 641 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
association between IDP status and knowledge as We saw serious atrocities and these will stay in our odds ratio (OR), the cross-product ratio of the entries memories for a long time. —FGD2, F1 in the 2-by-2 contingency table of 2 binary vari- The insecurity has now become permanently established ables.43 Multivariable logistic regression was used to there. They killed people there, including members of my verify the association between IDP status and family. —FGD1, M2 knowledge, with adjustment for confounding vari- ables. Statistical analysis was performed in the R sta- 44 Me, I don't like to be reminded of this. We suffer a lot. tistical environment. —FGD3, M2 The journey IDPs had followed to reach the Ethics Considerations camp was often challenging and circuitous, pass- Participants provided verbal consent to participate ing through multiple temporary dwelling places in the FGD and the questionnaire. Ethics approval before arriving in their current camp: was obtained from the Comité d’Éthique du Nord Kivu (Université Catholique du Graben, ref 003/ We spent nights outside in the bush during the armed TEN/CENK/2020). Operational approval was attacks by those people. —FGD5, F7 granted by the municipal authority (bourgmestre) and the local refugee coordinator. Participant con- In reality, when these people come to kill, you are just driv- fidentiality was respected during implementation en by a reflex to survive initially. And the next day, you ask and analysis of survey results. Data were collected yourself: now what? What do I do? —FGD1, M1 anonymously, without identifiers, and all results were presented in aggregate so that no individual First, it's panic, you have to flee and you don't know participant can be identified. All names and loca- who is where. You leave the house empty-handed, may- tions were removed from FGD quotations to avoid be with a child, and everybody has to flee. The next day, possible identification of the speaker. it's counting the dead and the damages. Then rapidly finding where to stay for security. —FGD1, M3 RESULTS We passed through several areas, depending on the secu- FGD Themes rity situation. There was a lot of back and forth just We began with a qualitative exploration of attempting to restart a stable life. —FGD1, M4 COVID-19 prevention in the IDP camp. We con- Loss of housing, assets, and livelihood meant ducted 6 FGDs, involving 23 participants (total). that IDPs current condition was precarious: The composition of focus groups is shown in Table 1. FGDs generated rich qualitative data, The war...a very bad thing. They attacked my village from which we derived the following themes: several times and we had to abandon everything, even- (1) displacement narratives; (2) population move- tually arriving here. —FGD3, M5 ments in and out of the camp and risk of introduc- ing COVID-19; (3) high level of awareness and Those fields are our guarantee for life. —FGD1, M1 fear of COVID-19; (4) challenges associated with hand hygiene in the camp; (5) impossibility of ...Our saving for the present and the future. It's our physical distancing in the IDP camp; and (6) re- wealth, what keeps us alive, feeds us, pays for health storing peace and security takes priority over care and school for our children. —FGD1, M4 vaccine. Some FGD participants expressed paralysis, We elaborate on each theme and provide rep- resentative quotations. hopelessness, and a sense of abandonment: On 1 side, the insecurity, and on the other, this corona— Displacement Narratives yes, we are scared. I'm just in shock. I can't say anything Unspeakable terror and killings drove FGD partici- at the moment. And tell that government, there, that we pants from their native homes. are abandoned here. —FGD5, F5 I've been in this camp for 6 years, since the beginning of the massacres in the region. —FGD1, M1 Population Movements In and Out of the Camp/ Risk of Introducing COVID-19 We call them the “ba chinja chinja”[throat-slitters]. Some FGD participants pointed out the insecurity —FGD1, M2 and isolation of the camp that restricted travel:
Global Health: Science and Practice 2020 | Volume 8 | Number 4 642 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
TABLE 1. Composition of Focus Groups From 3 Internally Displaced Persons Camps in North Kivu, Democratic Republic of the Congo
Focus Group IDP Participants Location Participant Unique Identifier
1 4 men Mwangaza M1, M2, M3, M4 2 4 women Masosi F1, F2, F3, F4 3 4 men Mwangaza M5, M6, M7, M8 4 3 youth (male) Masosi Y1, Y2, Y3 5 4 men Mwangaza F5, F6, F7, F8 6 4 women Luvangira F9, F10, F11, F12
Abbreviation: IDP, internally displaced person.
The area is very dangerous, they try to limit the move- We are very afraid because we have seen the fami- ments. —FGD4, Y3 lies that lost their family members who died of Ebola. —FGD4, Y1 It's rare to visit others. We spend most of the time here or in the fields. —FGD4, Y3 Ebola killed people, yes, but the radio talks of fright- ening numbers of deaths due to corona. Really very Others identified sources of visitors from out- many. —FGD6, F9 side the camp and noted that many IDPs move out of the camp for work on a daily basis. Even with Ebola here, we went to church, to the market, They [visitors] come from other camps or people who have but with corona, no. The churches are closed, and that's fields that employ us to work in their fields. —FGD2, F2 where we go for consolation, imploring God to protect us. But corona closed the churches. It's serious. —FGD6, No, in terms of leaving the camp, you can't count the F10 number of times. If you stay here, the children will die of hunger. Many times a day to look for something to eat. To the market, to the fields, anywhere that you can Challenges Associated With Hand Hygiene in the find something. —FGD6, F10 Camp Most FGD participants were aware of the recom- There is a constant coming and going of people from out- mendation for frequent handwashing as a preven- side the camp and vice-versa. —FGD 6, F11 tion measure against COVID-19. However, soap and water were not readily accessible in the camp: You have to wash your hands. That's what they say, but High Level of COVID-19 Awareness and Fear we don't have water here. —FGD4, Y2 There was a high level of awareness and fear of COVID-19, which was known as “corona”: Our only source of water is the rain. We collect water when it rains and we keep it. We drink this water. A new disease and very severe. We are afraid of it and When there is none left, our sisters go to the well to get we pray that it stays away from us. —FGD3, M7 water. —FGD4, Y2 Concerning this corona, we have learned about this from There is a little stream about 100 m away. That's what afar. We have never seen a person sick with corona. But we use for all our needs. —FGD6, F9 we have received teaching on corona. —FGD5, F8 To wash our hands, we have water buckets but no soap We have learned that it kills mostly politicians and and it's not enough because there are only 5 buckets for white people. We hope that this disease stays over there, the whole camp [of approximately 800 people]. — away from us. —FGD3, M6 FGD6, F11 As another severe viral epidemic, COVID-19 invited comparisons to the Ebola virus disease ep- They talk about masks, but if we don't even have soap, idemic that had ravaged the region: how can we ask for more? —FGD2, F2
Global Health: Science and Practice 2020 | Volume 8 | Number 4 643 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
Impossibility of Physical Distancing in the IDP small room. We're squeezed one against the other. It's Camp not possible here. —FGD2, F4 Housing was not conducive to physical distancing for many IDPs. Although many IDPs had individu- Restoring Peace and Security Takes Priority Over al family dwellings, some were housed in local Vaccine school buildings, sleeping in classrooms. Several respondents were willing to accept vacci- nation to prevent COVID-19 if a vaccine becomes The director and state authorities allowed us to stay available: here. More and more people came to stay because there was space. —FGD5, F5 I would receive it. For Ebola, people accepted the vac- cine. —FGD5, F6 — We don't pay anything for rent. It's free. FGD1, M1 Others bristled at the idea of a vaccine when more basic needs remain unmet: In the morning, we move our belongings outside until the end of classes. And at night, we bring back our things into Our concern is safety. Even that vaccine doesn't matter the classrooms we occupy. But since the beginning of coro- to us. Let them keep it over there. Even if they vaccinate na, we've stopped moving things in and out. We keep ev- us, and we continue to live in these conditions, what's erything in the rooms where we sleep. —FGD1, M3 the point? —FGD5, F5
Despite being accommodated by the school, If security returns, we will protect ourselves against coro- tragically, IDP children did not attend classes: na, we will respect all the measures, and it's only at that We stay with them outside, or else, they come with us to time that you can start talking about a vaccine or physi- the fields nearby. —FGD3, M6 cal distancing. But in these conditions, I wouldn't accept this vaccine. —FGD5, F5 Where are we going to find money to pay the school fees? In several FGDs, participants emphasized that It's impossible. We are “wakimbizi” [refugees; those COVID-19 prevention recommendations could who fled], as they call us. —FGD3, M5 best be implemented in a more stable, less crowded environment, such as their own homes. Arepeatedtheme A repeated theme was the inability to practice Reestablishing security in the region would allow physical distancing because of crowded condi- was the inability to IDPs to return where prevention could be prac- tions, particularly sleeping quarters in which mul- practice physical ticed. Other prevention strategies were seen as tiple families occupied a single classroom: distancing context inappropriate or even futile: because of Here, it's not possible “ku achana metre moya moya” [to The government should bring back peace, we will go crowded stay 1 meter apart; to practice physical distancing]. If it back to our homes and we will put into practice all that conditions, comes here, we will all die. You have seen the conditions you have taught us. But it's impossible to prevent corona particularly we live in. Our room measures 6 m by 5 m, and there are here. —FGD1, M3 sleeping quarters. 5 families inside. —FGD1, M4 These are measures that don't apply to us. The only medi- Reestablishing There is no soap, water is a problem, we sleep side by cine for us here or the only solution that can help us to fight security in the side. Everything is stacked against if this corona arrives corona here, is security. Bring back peace, and we'll “ region would here, even if we have, until now, escaped from the ba go back home, where we live in good conditions, and ” — — allow IDPs to chinja chinja [throat-slitters]. FGD1, M1 we can respect these recommendations of 1m. return where FGD5, F7 COVID-19 We are crowded in classrooms like sardines. Isn't that — prevention awful? FGD6, F10 Me, I'll only be able to protect myself and my children recommendations when I'm at home. We have our own houses with plen- One nongovernmental organization came here to edu- ty of space, like 6 rooms, but here it's 1 room. One room could be practiced. cate us about corona. We asked the teacher to give us a with several families. Each has his own activities dur- practical demonstration. He just smiled! It's good to ing the day and you don't know who will bring you the teach us, but going back, you should tell the people disease. —FGD6, F11 who sent you that it's not possible to avoid corona over there. —FGD1, M1 Survey Questionnaire Do you see how we sleep? During the day, maybe, we Surveys were conducted between 25 and 29 May can avoid touching each other, but at night we're in a 2020. One IDP approached declined to participate
Global Health: Science and Practice 2020 | Volume 8 | Number 4 644 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
in the questionnaire interview (165/166 [99%] In addition, 107 (65%) of IDPs had received a visi- participation rate). Two participants (1 IDP, 1 com- tor from outside the camp in the past month. Since parison) had never heard of coronavirus (307/309 the pandemic began, IDPs reported leaving the [99%] awareness) and were excluded from the camp less frequently than before in 84 (52%), subsequent analysis. The final sample consisted of more than before in 33 (20%) and about the same 164 IDPs (66 from Mwangaza; 44 Masosi, and as before in 46 (28%) of cases. 54 Luvangira) and 143 in the comparison group. There were 74 women (45%) among the IDPs sur- DISCUSSION veyed and 57 women (40%) in the comparison Our study is unique among COVID-19 KAP sur- group. veys to date for its focus on a displaced population Thirty-five (21%), 82 (50%), and 47 (29%) of with extreme resource limitations. Other KAP IDPs had lived in the camps for less than 1, 1–2, – surveys included health care workers28,45 47 or and more than 2 years, respectively. Sixty-six residents of high-income countries with markedly (41%) of families were temporarily sheltered in different demographics than our study (e.g., 62% school buildings. Others lived in structures made of U.S.48 and 64% of Chinese participants49 had a from wood, thatch, and mud or brick walls with bachelor’s degree or higher, compared to 47% of an iron sheet roof (Table 2). Demographic fea- IDPs in our study who had no formal education at tures, household (family) size, and asset owner- all). Given the radically different challenges of Given the ship differed significantly between IDPs and the COVID-19 prevention in IDP camps, this study fills radically different comparison group (Table 2). IDPs surveyed were a gap in available data from a neglected and isolat- challenges of older, had lower educational attainment, were ed population. IDPs differed from neighboring COVID-19 more commonly farmers, were more commonly Congolese residents in terms of larger household married, had a higher median household size, prevention in IDP size (including 46% of families with a member had lower household ownership of indicator camps, this study over the age of 60), more extreme poverty, lower assets (radio, cell phone, and bicycle), and had dif- fills a gap in educational attainment, less access to information ferent housing structures than the comparison available data through media and internet, less COVID-19 speci- group (Table 2). from a neglected With respect to knowledge of COVID-19, few- fic knowledge, lower rate of physical distancing, and isolated er IDPs correctly identified signs and symptoms, and reduced access to hand hygiene. These factors, population. and fewer recognized the potential for asymptom- as well as the high mobility of IDPs, leaving and atic transmission (Table 3). Overall, 15% of IDPs reentering the camp daily for subsistence labor, es- had sufficient knowledge, versus 30% of the com- tablish their vulnerability to COVID-19. parison group (OR=0.30; 95% confidence interval [CI]=0.17, 0.53; P<.0001). Other factors associat- COVID-19 Knowledge ed with low COVID-19 knowledge in bivariate IDPs and the comparison group both identified lo- analyses (P<.05) included younger age, larger cal radio as their major source of information on household size, and lack of radio ownership. In a COVID-19 (Table 2). Radio, television, and social multivariable logistic regression model adjusting media were more common sources of information for these possible confounders, IDP status remained among the comparison group, whereas church statistically significantly associated with lower was a more common source among IDPs (Table knowledge (adjusted OR=0.17; 95% CI=0.082, 2). Other studies in LMICs (Pakistan,45 Uganda,28 0.34; P<.0001). and Vietnam50) showed that health care workers Attitudes and practices toward COVID-19 pre- accessed World Health Organization or ministry vention are shown in Tables 4 and 5, respectively. of health websites (83%–88%), social media Despite widespread agreement (89%) that physi- (74%–91%), radio or television (46%–79%) for cal distancing was important to prevent COVID- their COVID-19 information, preferences which 19, a higher proportion of IDPs than individuals reflect major differences in education level, em- in the comparison group reported close contact ployment activities, and access to internet from with someone outside the family in the past 24 the IDPs in our study. hours and a higher proportion had shaken hands Knowledge of COVID-19 was poor in IDPs ver- with at least 1 person (Table 5). sus the comparison group (Table 3). Using a simi- IDP respondents indicated that movements in lar questionnaire item, 98% and 93% of health and out of the camp were frequent. By self-report, care workers in Uganda identified fever and cough 83 (61%), 62 (38%), and 19 (12%) left the camp as symptoms of COVID-19,28 compared to 26% on a daily, weekly, and monthly basis, respectively. and 42% of IDPs, respectively, in our study.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 645 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
TABLE 2. Demographics of Survey Questionnaire Respondents Selected From 3 Internally Displaced Persons Camps in North Kivu, Democratic Republic of the Congo
Overall IDPs Comparison (N=307) (N=164) (N=143) P Value
Demographics Age [yr], median (IQR) 37 (24–55) 43 (28–58) 29 (22–45) <.0001 Sex, No. (%) .42 Male 176 (57.3) 90 (54.9) 86 (60.1) Female 131 (42.6) 74 (45.1) 57 (39.9) Education, No. (%) <.0001 None 111 (36.2) 77 (47.0) 34 (23.8) Primary 113 (36.8) 67 (40.9) 46 (32.2) Secondary or above 83 (27.0) 20 (12.2) 63 (44.1) Employment,a No. (%) <.0001 Farming 166 (54.1) 115 (70.1) 51 (35.7) Commerce/trade 20 (6.5) 1 (0.6) 19 (13.3) Health care worker 14 (4.6) 4 (2.4) 10 (7.0) Unemployed 77 (25.1) 37 (22.6) 40 (28.0) Othera 30 (9.8) 7 (4.3) 23 (16.1) Marital status, No. (%) <.0001 Single 66 (21.4) 6 (3.7) 60 (42.0) Married 182 (59.3) 113 (68.9) 69 (48.3) Married (separated) 33 (10.7) 25 (15.2) 8 (5.6) Widowed 26 (8.5) 20 (12.2) 6 (4.2) Household characteristics Household size, median (IQR) 8 (6–10) 9 (7–11) 8 (6–10) .007 Households with member aged >60 years, No. (%) 132 (43.0) 75 (45.7) 57 (40.0) .33 Household assets, No. (%) Radio 158 (51.5) 52 (31.7) 106 (74.1) <.0001 Cell phone 122 (39.7) 31 (18.9) 91 (63.6) <.0001 Bicycle 50 (16.3) 10 (6.1) 40 (28.0) <.0001 Housing, No. (%) Wood, thatch, mud materials 209 (68.8) 79 (49.1) 130 (90.9) <.0001 Brick or wood walls and iron sheet roof 27 (8.9) 16 (9.9) 13 (9.1) School building 68 (22.4) 66 (41.0) 0
Abbreviations: IDP, internally displaced person; IQR, interquartile range. a Other employment included trades (mechanic, carpenter, shoemaker, tailor, mason, gardener), teacher, police officer, pastor, and taxi driver.
Gastrointestinal symptoms were less frequently both IDPs (54%) and the comparison group identified by both Ugandan health care workers (64%) in our study. (35%)28 and IDPs (11%). Misconceptions around Fear of COVID-19 was expressed by 98% of COVID-19 transmission (incorrectly endorsing survey respondents, similar to previous observa- mosquito-borne transmission) were common in tions of high anxiety scores in another survey
Global Health: Science and Practice 2020 | Volume 8 | Number 4 646 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
TABLE 3. Survey Questionnaire Respondents’ Knowledge on COVID-19 Among Internally Displaced Persons, North Kivu, Democratic Republic of the Congo
Overall IDPs Comparison (N=307) (N=164) (N=143) No. (%) No. (%) No. (%) P Value
Source of information on COVID-19 Local radio 278 (90.6) 140 (85.4) 138 (96.5) .002 International radio 11 (3.6) 2 (1.2) 9 (6.3) .03 Television 12 (3.9) 1 (0.6) 11 (7.7) .002 Social media 28 (9.1) 2 (1.2) 26 (18.2) <.0001 Church 40 (13.0) 28 (17.1) 12 (8.4) .04 Friends 81 (26.4) 50 (30.5) 31 (21.7) .11 No response 4 (1.3) 4 (2.4) 0 (0) .13 Recognition of illness What are the signs and symptoms of COVID-19?a I don’t know 62 (20.2) 52 (31.7) 10 (7.0) <.0001 Feverb 70 (22.8) 43 (26.2) 27 (18.9) .16 Coughb 171 (55.7) 69 (42.1) 102 (71.3) <.0001 Difficulty breathingb 109 (35.5) 46 (28.0) 63 (44.1) .005 Sneezing 78 (25.4) 41 (25.0) 37 (25.9) .96 Nasal congestion 140 (45.6) 59 (36.0) 81 (56.7) .0004 Headache 67 (21.8) 46 (28.0) 21 (14.7) .007 Fatigue 92 (29.9) 38 (23.2) 54 (37.8) .008 Joint pain 80 (26.1) 29 (17.7) 51 (35.7) .0006 Muscle pain 27 (8.8) 16 (9.8) 11 (7.7) .66 Loss of appetite 14 (4.6) 6 (3.7) 8 (5.6) .59 Diarrhea 25 (8.1) 18 (11.0) 7 (4.9) .08 Constipationc 2 (0.7) 2 (1.2) 0 (0) .50 Bleedingb,c 25 (8.1) 14 (8.5) 11 (7.7) .95 Asymptomatic spread COVID-19 can be transmitted by someone with no symptoms.b 146 (47.6) 60 (36.6) 86 (60.1) <.0001 Misconceptions COVID-19 can be transmitted by mosquitos.b,c 125 (40.7) 67 (40.9) 58 (40.6) .19 COIVID-19 can be prevented by eating spicy food.c 21 (6.8) 11 (6.7) 10 (7.0) .90 Sufficient knowledge of COVID-19 Knew key symptoms, did not endorse misconceptions 79 (25.7) 24 (14.6) 55 (38.5) <.0001
Abbreviations: COVID-19, coronavirus disease; IDP, internally displaced person. a If participant answered “I don't know,” no further symptoms were solicited. Otherwise, multiple answers were allowed. b Used to assess sufficient knowledge of COVID-19. c Number (percentage) of participants who erroneously endorsed these incorrect signs, symptoms, or statements.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 647 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
TABLE 4. Survey Questionnaire Respondents’ Attitudesa Toward COVID-19 Among Internally Displaced Persons, North Kivu, Democratic Republic of the Congo
Overall IDPs Comparison (N=307) (N=164) (N=143) No. (%) No. (%) No. (%) P Value
Affective response COVID-19 is a serious illness. 301 (98.0) 163 (99.4) 138 (96.5) .03 I am afraid of COVID-19. 300 (97.8) 161 (98.2) 139 (97.2) .08 Reaction to control measures Physical distancing is important to prevent COVID-19. 278 (90.6) 146 (89.0) 132 (92.3) .42 People should be willing to give up their daily duties to stop 243 (79.2) 120 (73.2) 123 (86.0) .14 the spread of COVID-19. Disinformation It is hard to distinguish which information I hear about 244 (79.5) 126 (76.8) 118 (82.5) .61 COVID-19 is true, false, or just a rumour. Institutional trust I trust the government. 207 (67.4) 117 (71.3) 90 (62.9) .09 There is a lot of corruption in the government. 123 (40.1) 70 (42.7) 53 (37.1) .53 Rumors COVID-19 was created in a Chinese laboratory. 58 (18.9) 31 (18.9) 27 (18.9) .88 COVID-19 is a conspiracy created to vaccinate everybody. 37 (12.1) 13 (7.9) 24 (16.8) .03
Abbreviations: COVID-19, coronavirus disease; IDP, internally displaced person. a Participants were asked to rank agreement with the statements on a 5-point Likert scale, with possible answers “strongly agree,”“agree,”“neu- tral,”“disagree,”“strongly disagree,” or “I don't know.” Numbers are n (%) of participants who agreed or strongly agreed with the statements.
47 Many FGD from Iran. Surprisingly, many FGD participants For example, 98% of Chinese residents at the be- participants considered that COVID-19 was even more severe ginning of the pandemic wore masks when going 49 48 considered that than Ebola virus disease (in fact, the case fatality out compared to 24% of U.S. residents. Mask 51 COVID-19 was rate of Ebola virus disease is more than 60%, use was reported by 3.5% of IDPs and 6% of the 52 comparison group, highlighting the lack of per- even more severe compared to less than 2% for COVID-19 ). sonal protective equipment in this setting. Other than Ebola virus Public health messages about the severity of COVID-19 appear to be widely accepted and be- measures more readily available to IDPs were disease. lieved, with FGD participants citing the high num- handwashing (practiced by 98%), distancing ber of deaths in wealthy “white” countries and from others (48%), and avoiding touching the the closing of churches as evidence of danger. face (28%), which were reported in proportions Although mistrust in the government (39%), belief similar to the comparison group. in corruption (42%), belief in conspiracy theories Movement of populations contributes to the (44% and 22%) were prevalent, endorsement of spread of COVID-19. In a large refugee camp in these views did not appear to be associated with Bangladesh, aid workers who enter and leave the prevention practices. This contrasts with surveys camp daily are expected to be the most likely sources of introduction of COVID-19 into the of attitudes toward Ebola virus disease in the same 12 area, in which mistrust, rumors, and misinforma- camp. In the IDP camps in our study, the con- tion were associated with passive and active resis- spicuous lack of aid workers reflects the isolated 17,41 and hazardous environment, as well as the tance to control measures. neglected status of the IDPs. However, 61% of IDPs left the camp on a daily basis, and 65% had COVID-19 Prevention Efforts received a visitor in the past month. Staying COVID-19 prevention practices vary widely be- home was practiced less often among IDPs than tween geographic areas and demographic groups. among the comparison group (P=.039, Table 5).
Global Health: Science and Practice 2020 | Volume 8 | Number 4 648 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
TABLE 5. Survey Questionnaire Respondents’ Practices With Respect to COVID-19 Prevention Among Internally Displaced Persons, North Kivu, Democratic Republic of the Congo
Overall IDPs Comparison (N=307) (N=164) (N=143) No. (%) No. (%) No. (%) P Value
Prevention practices In the past 2 weeks, have you done anything to protect yourself from COVID-19? No 137 (44.6) 77 (47.0) 60 (42.0) Yes 168 (54.7) 85 (51.8) 83 (58.0) .39 If so, what?a Wash hands 149 (88.6) 78 (91.7) 71 (85.5) .30 Stay >2 m from others 75 (44.6) 35 (41.2) 40 (48.2) .45 Avoid touching face 38 (22.6) 24 (28.2) 14 (16.8) .11 Stay home 31 (18.5) 10 (11.8) 21 (25.3) .04 Use disinfectant 10 (6.0) 6 (7.1) 4 (4.8) .75 Wear mask 8 (4.8) 3 (3.5) 5 (6.0) .49 Take medicines without prescription 2 (1.2) 2 (2.4) 0 (0) .50 Change diet 1 (0.6) 1 (1.2) 0 (0) >.99 Physical distancing Apart from family, have you come in close (<2 m) contact with anyone in the past 24 hours? Yes 195 (63.5) 115 (70.1) 80 (55.9) .01 How many people did you shake hands with in the past 24 hours (not counting family members)? 0 155 (50.5) 77 (47.0) 78 (54.5) 1 to 5 71 (23.1) 31 (18.9) 40 (28.0) .02 >5 81 (26.4) 56 (34.1) 25 (17.5) Barriers to prevention What has prevented you from fully protecting yourself from COVID-19? Lack of soap 243 (79.2) 150 (91.5) 93 (65.0) <.0001 Lack of water 193 (62.9) 110 (67.1) 83 (58.0) .11 Insufficient income 67 (21.8) 32 (19.5) 35 (24.5) .38 Lack of masks 55 (17.9) 25 (15.2) 30 (21.0) .26 Lack of information 51 (16.6) 24 (14.6) 27 (18.9) .41 Lack of disinfectant 46 (15.0) 21 (12.8) 25 (17.5) .34 Lack of availability of these items 31 (10.1) 18 (11.0) 13 (9.1) .71 High prices of these items in the market 46 (15.0) 15 (9.2) 31 (21.7) .004 Lack gloves 18 (5.9) 11 (6.7) 7 (4.9) .66 I can fully protect myself against COVID-19 28 (9.2) 7 (4.3) 21 (14.7) .003
Abbreviations: COVID-19, coronavirus disease; IDP, internally displaced person. a Among respondents who had done something to protect against COVID-19.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 649 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
These frequent movements represent opportuni- health workers.50 In our FGDs, some participants ties to introduce COVID-19 into the camp. FGD were willing to accept vaccination as a control participants explained that daily labor in neigh- strategy, whereas others pointed to futility and in- boring fields or trips to the market were impera- appropriateness of what appeared to them as a tive to provide for family needs. Thus, unless food stopgap solution, when the overwhelming prob- security can be assured by other means, restriction lem was displacement from their homes. of movements to prevent COVID-19 is not viable Expressions of futility or fatalism as expressed in the IDP camps studied. by FGD participants in our study are noteworthy Among IDPs who had taken action to prevent and may reflect learned helplessness or loss of COVID-19, hand hygiene was practiced by 92%. self-efficacy among IDPs under extraordinarily However, the most commonly listed barrier to difficult living conditions. The theory of learned prevention was lack of soap (92% of IDPs, versus helplessness54,55 describes pessimistic beliefs about 65% of the comparison group), followed by lack of the efficacy of one's actions and the likelihood of water (67% of IDPs). Distribution of soap to obtaining future rewards. The theory has explana- households in a refugee camp increased hand- tory power among refugees in other contexts, such washing by more than 30% and reduced diarrheal as risky sexual behavior among victims of sexual or 53 illness in a previous study. In Nigeria, COVID-19 gender-based violence.56 Similarly, the concept of control efforts included sensitization campaigns self-efficacy57 refers to the degree of externality in on handwashing were followed by the distribu- control attribution.58 Low self-efficacy is associated 13 tion of soap to IDPs in Borno State. Inspired by with a fatalistic orientation, as exemplified by a these examples, and responding to the near- FGD participant’s response. These theoretical universal lack of soap identified in our survey, we frameworks may explain, at least in part, initially included soap distribution in our community feed- puzzling findings such as rejection of a hypothetical back efforts. vaccine among some FGD respondents and high Avoiding physical contact with others is em- levels of hand shaking despite awareness and fear phasized as a COVID-19 prevention measure. The of COVID-19. majority (89%) of IDPs agreed or strongly agreed that this was an important control measure (Table 4), but 70% had come in close contact Limitations with someone other than a family member (ver- Our study has several limitations. Our survey tool sus 56% of the comparison group, P=.014, Table was not validated against a gold standard instru- 5). The impossibility of physical distancing in the ment for the measurement of COVID-19-related camp, noted by previous authors,10–12 was repeat- KAP among IDPs. However, we took several steps edly emphasized in FGDs. Sleeping quarters were to optimize the validity of the survey: (1) contex- highly congested, with several families often tually relevant questionnaire items using past sur- sleeping in a single classroom. In high-income veys from other LMICs and from North Kivu; countries, where shelter-at-home recommenda- (2) tacit understanding of the local language and tions are more feasible, adherence to physical dis- culture by our study team; and (3) implementa- tancing recommendations remains variable. In the tion of the questionnaire as a verbal interview by United States, 30% of people reported attending local Congolese health workers to allow explana- gatherings with more than 50 people (contrary to tion of questions. The sampling strategy for IDPs public health advice),48 compared to only 3.6% of and the comparison group was not a fully random Chinese survey respondents.49 In our study, 19% sample due to lack of detailed census information. Given challenges of IDPs had shaken hands with 1–5 people in the Instead, for IDPs we used geospatial sampling34 with hand hygiene past 24 hours, and 34% with more than 5 people, from 3 displacement camps. For the comparison and physical which was statistically higher than the compari- group, we used maximum variation sampling, distancing in the son group (P=.023, Table 5). In contrast, 83% of based on demographic features (age, sex, occupa- camps, we Ugandan health care workers avoided shaking tion, and educational attainment). These non- speculated that hands due to COVID-19.28 Given challenges with probability sampling methods are widely used,37 IDPs may have felt hand hygiene and physical distancing in the but findings may not be representative of the en- disempowered to camps, we speculated that IDPs may have felt dis- tire IDP population. Therefore statistical infer- take small efforts empowered to make even small efforts to reduce ences should be interpreted with caution and to reduce physical physical contact with others. should be confirmed in studies with a fully ran- contact with Acceptance of a hypothetical COVID-19 vac- dom sample of the population of interest (IDPs in others. cine was high (92%) in a study of Vietnamese North Kivu, DRC). For our primary analysis
Global Health: Science and Practice 2020 | Volume 8 | Number 4 650 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
(COVID-19 knowledge among IDPs versus the Acknowledgments: We thank the IDPs who participated in the survey comparison group), we adjusted for differences in and FGDs. We thank the local authorities, including Kambale Kibwana (bourgmestre adjoint) for providing operational approval to access the demographic variables between groups in a multi- IDP camp and conduct the study. variable analysis to mitigate the effect of confound- ing. Similarly, FGDs participants represented a Funding: The Association for Health Innovation in Africa (AFHIA) small number of IDPs in the camp; however, satu- provided funding for this study. ration of themes was quickly achieved, suggesting the breadth and diversity of viewpoints in the Competing interests: None declared. camps was captured. REFERENCES 1. Center for Systems Science and Engineering, John Hopkins CONCLUSION University. COVID-19 Dashboard. Accessed August 25, 2020. In summary, our findings provide a snapshot of https://coronavirus.jhu.edu/map.html IDP camps as they brace for COVID-19. Awareness 2. African Union, Africa Centers for Disease Control and Prevention and fear of COVID-19 was high among IDPs, but (CDC). Coronavirus Disease 2019 (COVID-19). Accessed April 30, 2020. https://africacdc.org/covid-19/ only 15% had comprehensive knowledge of the dis- ease. Significant barriers to implementing COVID- 3. Nkengasong J. Let Africa into the market for COVID-19 diagnostics. Nature. April 28, 2020. Accessed September 21, 2020. https:// 19 prevention measures exist in IDP camps, includ- www.nature.com/articles/d41586-020-01265-0 ing crowded sleeping quarters, frequent close con- 4. Ministry of Health, Democratic Republic of the Congo. Stop tact with non-family members, movement in and Coronavirus COVID-19 DRC. Accessed September 21, 2020. out of the camp for work, and lack of access to https://www.stopcoronavirusrdc.info/ hand hygiene. Poignantly, IDPs spoke of a desire for 5. World Health Organization (WHO). Coronavirus Disease (COVID- peace and a return to their homes, where they could 19). Weekly Epidemiological Update. Accessed September 21, 2020. https://www.who.int/docs/default-source/coronaviruse/ capably prevent COVID-19 themselves. These data situation-reports/20200824-weekly-epi-update.pdf?sfvrsn= from a hard-to-reach population in a zone of insecu- 806986d1_4 rity provide a rare glimpse of the desperate condi- 6. The Lancet. COVID-19 will not leave behind refugees and migrants. tions under which IDPs survive, leaving them Lancet. 2020;395(10230):1090. CrossRef. Medline vulnerable to COVID-19. These results call for an 7. UNHCR. Figures at a glance. Accessed September 21, 2020. ethical, inclusive approach to the global pandemic https://www.unhcr.org/figures-at-a-glance.html that leaves no one behind, just as COVID-19 will 8. Internal Displacement Monitoring Centre. Global Internal not respect borders and will not leave behind refu- Displacement Database: 2019 Internal Displacement Figures by 6 Country. Accessed September 21, 2020. https://www.internal- gees and IDPs. displacement.org/database/displacement-data 9. Iacobucci G. Covid-19: Doctors warn of humanitarian catastrophe at RECOMMENDATIONS Europe’s largest refugee camp. BMJ. 2020;368:m1097. CrossRef. These specific recommendations follow from our Medline ’ findings: 10. Kluge HHP, Jakab Z, Bartovic J, D Anna V, Severoni S. Refugee and migrant health in the COVID-19 response. Lancet. 2020;395 IDPs should be provided with adequate facili- (10232):1237–1239. CrossRef. Medline ties and consumables to implement recom- 11. Raju E, Ayeb-Karlsson S. COVID-19: How do you self-isolate in a Int J Public Health – mended COVID-19 precautions. These include refugee camp? . 2020;65(5):515 517. CrossRef. Medline ample water and soap for hand hygiene and 12. Vince G. The world’s largest refugee camp prepares for covid-19. face masks. BMJ. 2020;368:m1205. CrossRef. Medline Additional space and housing should be made 13. Tijjani SJ, Ma L. Is Nigeria prepared and ready to respond to the available to allow IDPs to practice physical dis- COVID-19 pandemic in its conflict-affected northeastern states? Int J Equity Health tancing, particularly within sleeping quarters. . 2020;19(1):77. CrossRef. Medline Separate dwellings (e.g., tarpaulin tents) for in- 14. Tariq S, Woodman J. Using mixed methods in health research. JRSM Short Rep. 2013;4(6). CrossRef. Medline dividual families should be provided. Multiple families sleeping in a classroom (as currently 15. Östlund U, Kidd L, Wengström Y, Rowa-Dewar N. Combining qual- itative and quantitative research within mixed method research observed) is discouraged. designs: a methodological review. Int J Nurs Stud. 2011;48(3):369– Although challenging, restoration of peace 383. CrossRef. Medline by controlling armed conflict in the area is a 16. Kasereka MC, Hawkes MT. ‘The cat that kills people:’ community chief priority for IDPs and would allow a safe beliefs about Ebola origins and implications for disease control in Eastern Democratic Republic of the Congo. Pathog Glob Health. return to their ancestral homes where they 2019;113(4):149–157. CrossRef. Medline could more adequately practice COVID-19 17. Masumbuko Claude K, Underschultz J, Hawkes MT. Social resistance prevention. drives persistent transmission of Ebola virus disease in Eastern
Global Health: Science and Practice 2020 | Volume 8 | Number 4 651 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
Democratic Republic of Congo: a mixed-methods study. PLoS One. 38. Bai Y, Yao L, Wei T, et al. Presumed asymptomatic carrier transmis- 2019;14(9):e0223104. CrossRef. Medline sion of COVID-19. JAMA. 2020;323(14):1406–1407. CrossRef. 18. UNHCR. United Nations High Commissioner for Refugees, Medline Democratic Republic of the Congo, Emergency update on Ituri, North 39. Hu Z, Song C, Xu C, et al. Clinical characteristics of 24 asymptomatic Kivu and South Kivu (22 June - 6 July 2020). Accessed August 27, infections with COVID-19 screened among close contacts in Nanjing, 2020. https://data2.unhcr.org/en/documents/details/78270/ China. Sci China Life Sci. 2020;63(5):706–711. CrossRef. Medline 19. Coghlan B, Brennan RJ, Ngoy P, et al. Mortality in the Democratic 40. World Health Organization (WHO). Global Surveillance for Republic of Congo: a nationwide survey. Lancet. 2006;367 COVID-19 Caused by Human Infection with COVID-19 Virus: (9504):44–51. CrossRef. Medline Interim Guidance, 20 March 2020. Accessed September 21, 2020. 20. UNHCR. Bi-Weekly Emergency Update: Ituri, North Kivu and South https://apps.who.int/iris/handle/10665/331506 Kivu Provinces, Democratic Republic of the Congo, 22 June–6 July 41. Vinck P, Pham PN, Bindu KK, Bedford J, Nilles EJ. Institutional trust 2020. Accessed October 9, 2020. https://data2.unhcr.org/en/ and misinformation in the response to the 2018–19 Ebola outbreak documents/details/78270/ in North Kivu, DR Congo: a population-based survey. Lancet Infect Dis – 21. OXFAM. Accessed September 21, 2020. https://www.oxfam.org/ . 2019;19(5):529 536. CrossRef. Medline en/what-we-do/countries/democratic-republic-congo 42. KoBoToolbox. Accessed May 19, 2020. https://www.kobotoolbox. 22. PAM. Programme Alimentaire Mondial. Accessed September 21, org/ 2020. https://fr.wfp.org/ 43. Edwards AWF. E A. The measure of association in a 2 x 2 table. J 23. CIRC. Comité International de la Croix Rouge. Accessed September Royal Stat Soc. 1963;126(1):109–114. CrossRef 21, 2020. https://www.icrc.org/fr 44. R. R Core Team. The R Project for statistical computing. Accessed 24. Carlsen B, Glenton C. What about N? A methodological study of May 19, 2020. https://www.R-project.org/ BMC Med Res sample-size reporting in focus group studies. 45. Saqlain M, Munir MM, Rehman SU, et al. Knowledge, attitude, Methodol . 2011;11(1):26. CrossRef. Medline practice and perceived barriers among healthcare workers regard- 25. Kitzinger J. Qualitative research: introducing focus groups. BMJ. ing COVID-19: a cross-sectional survey from Pakistan. J Hosp Infect. 1995;311(7000):299–302. CrossRef. Medline 2020;105(3):419–423. CrossRef. Medline 26. Sandelowski M. What’s in a name? Qualitative description revisited. 46. Kamate S, Sharma S, Thakar S, et al. Assessing knowledge, attitudes Res Nurs Health. 2010;33(1):77–84. CrossRef. Medline and practices of dental practitioners regarding the COVID-19 pan- demic: a multinational study. Dent Med Probl. 2020;57(1):11–17. 27. United Nations Children’s Fund. Ministry of Public Health. KAP Survey on COVID 19-Response. Accessed September 21, 2020. CrossRef. Medline https://www.surveymonkey.com/r/KAP_GY 47. Nemati M, Ebrahimi B, Nemati F. Assessment of Iranian nurses’ 28. Olum R, Chekwech G, Wekha G, Nassozi DR, Bongomin F. knowledge and anxiety toward COVID-19 during the current out- Arch Clin Infect Dis Coronavirus disease-2019: knowledge, attitude, and practices of break in Iran. . Online March 29, 2020. CrossRef health care workers at Makerere University Teaching Hospitals, 48. Clements JM. Knowledge and behaviors toward COVID-19 among Uganda. Front Public Health. 2020;8:181. CrossRef. Medline US residents during the early days of the pandemic: cross-sectional JMIR Public Health Surveill 29. Brooks HM, Jean Paul MK, Claude KM, Mocanu V, Hawkes MT. Use online questionnaire. . 2020;6(2): and disuse of malaria bed nets in an internally displaced persons e19161. CrossRef. Medline camp in the Democratic Republic of the Congo: A mixed-methods 49. Zhong BL, Luo W, Li HM, et al. Knowledge, attitudes, and practices study. PLoS One. 2017;12(9):e0185290. CrossRef. Medline towards COVID-19 among Chinese residents during the rapid rise 30. Brooks HM, Jean Paul MK, Claude KM, Houston S, Hawkes MT. period of the COVID-19 outbreak: a quick online cross-sectional Int J Biol Sci – Malaria in an internally displaced persons camp in the Democratic survey. . 2020;16(10):1745 1752. CrossRef. Medline Republic of the Congo. Clin Infect Dis. 2017;65(3):529–530. 50. Huynh G, Nguyen TNH, Tran VK, Vo KN, Vo VT, Pham LA. CrossRef. Medline Knowledge and attitude toward COVID-19 among healthcare Asian Pac J Trop 31. Hamze H, Charchuk R, Jean Paul MK, Claude KM, Léon M, Hawkes workers at District 2 Hospital, Ho Chi Minh City. Med – MT. Lack of household clustering of malaria in a complex humani- . 2020;13(6):260 265. CrossRef tarian emergency: implications for active case detection. Pathog 51. Aruna A, Mbala P, Minikulu L, et al. Ebola Virus Disease Outbreak — Glob Health. 2016;110(6):223–227. CrossRef. Medline Democratic Republic of the Congo, August 2018–November 2019. MMWR Morb Mortal Wkly Rep – 32. Charchuk R, Paul MKJ, Claude KM, Houston S, Hawkes MT. Burden . 2019;68(50):1162 1165. CrossRef. of malaria is higher among children in an internal displacement Medline camp compared to a neighbouring village in the Democratic 52. Abdollahi E, Champredon D, Langley JM, Galvani AP, Moghadas Malar J Republic of the Congo. . 2016;15(1):431. CrossRef. Medline SM. Temporal estimates of case-fatality rate for COVID-19 outbreaks 33. Claude KM, Underschultz J, Hawkes MT. Ebola virus epidemic in in Canada and the United States. CMAJ. 2020;192(25):E666– war-torn eastern DR Congo. Lancet. 2018;392(10156):1399– E670. CrossRef. Medline 1401. CrossRef. Medline 53. Peterson E, Roberts L, Toole MJ, Peterson DE. The effect of soap dis- 34. Wild H, Glowacki L, Maples S, et al. Making pastoralists count: geos- tribution on diarrhoea: Nyamithuthu Refugee Camp. Int J Epidemiol. patial methods for the health surveillance of nomadic populations. Am 1998;27(3):520–524. CrossRef. Medline JTropMedHyg – . 2019;101(3):661 669. CrossRef. Medline 54. Abramson LY, Seligman ME, Teasdale JD. Learned helplessness in 35. Henderson RH, Sundaresan T. Cluster sampling to assess immuniza- humans: critique and reformulation. J Abnorm Psychol. 1978;87 tion coverage: a review of experience with a simplified sampling (1):49–74. CrossRef. Medline Bull World Health Organ – method. . 1982;60(2):253 260. Medline 55. Miller WR, Seligman ME. Depression and learned helplessness in 36. Bennett S, Woods T, Liyanage WM, Smith DL. A simplified general man. J Abnorm Psychol. 1975;84(3):228–238. CrossRef. Medline method for cluster-sample surveys of health in developing countries. 56. John-Langba J. The relationship of sexual and gender-based vio- World Health Stat Q – . 1991;44(3):98 106. Medline lence to sexual-risk behaviour among refugee women in Sub- 37. Mainous AG III, Hougland JG Jr. Survey sampling issues in primary Saharan Africa. World Health Popul. 2007;9(2):26–37. CrossRef. care research. Fam Med. 1991;23(7):539–543. Medline Medline
Global Health: Science and Practice 2020 | Volume 8 | Number 4 652 Mixed-Methods Study of COVID-19 Prevention in Internally Displaced Persons Camps www.ghspjournal.org
57. Bandura A. Self-efficacy: Toward a unifying theory of behav- 58. Rotter JB. Generalized expectancies for internal versus external con- ioral change. Psychol Rev. 1977;84(2):191–215. CrossRef. trol of reinforcement. Psychol Monogr. 1966;80(1):1–28. CrossRef. Medline Medline
En français
La prévention de COVID-19 dans un camp des déplacés internes dans une zone d’insécurité au Nord Kivu, République Démocratique du Congo: une étude avec méthodes mixtes
Message clé
Les déplacés internes (DI) à cause d’un conflit armé constituent une population négligée, vulnérable à la pandémie de COVID-19. Cette étude avec méthodes mixtes donne une rare perspective sur les défis auxquels font face les DI à l’Est de la République Démocratique du Congo devant la menace de COVID-19.
Résumé Introduction: La pandémie de COVID-19 présente une sérieuse menace aux réfugiés et aux déplacés internes (DI). Nous avons étudié les connais- sances, attitudes, et pratiques vis-à-vis COVID-19 parmi les DI dans une zone d’insécurité à l’Est de la République Démocratique du Congo (RDC).
Méthodes: Étude avec méthodes mixtes pour la collecte et analyse de données qualitatives (discussions en groupe, DG) et quantitatives (sondage avec questionnaire de 52 éléments).
Résultats: Des DG (23 participants au total) et un sondage (164 DI de trois camps de déplacés et 143 témoins d’un village voisin) ont été organisés en mai, 2020. Les DI étaient statistiquement plus susceptibles d'avoir une plus grande taille de ménage, une pauvreté extrême, un niveau d'éducation inférieur et un accès plus faible à l'information via les médias et l’internet (P <0,05 pour toutes les comparaisons). Les PDI avaient un niveau élevé de sensibilisation (99%) et de peur (98%) du COVID-19, mais des connaissances spécifiques plus faibles (15% de connaissances suffisantes contre 30% parmi les témoins, P <0,0001), une différence qui est restée significative dans un modèle multivariable ajusté pour les effets confondants. Les DI avaient plusieurs défis quant à la mise en œuvre des recommandations pour prévenir le COVID-19. La distanciation physique était impossible dans leurs abris coincés et 70% des DI ont répondu qu’ils ont été en contact étroit avec une personne autre qu'un membre de la famille au cours des dernières 24 heures (contre 56% des témoins, P=0,014). Les DI devaient souvent sortir du camp pour subvenir à leurs besoins alimentaires, ce qui pourrait permettre l’in- troduction de COVID-19 dans le camp. 61% des DI sortaient du camp quotidiennement, et 65% avaient eu un visiteur dans le mois précédent. Malgré l'acceptation de l'hygiène des mains pour la prévention, 92% manquaient de savon (contre 65% des témoins, P <0,0001). Les DI cherchaient la paix et un retour au village natal encore plus que d’autres mesures de prévention telles qu’un vaccin contre COVID-19.
Conclusions: Ces résultats fournissent des preuves empiriques soutenant la vulnérabilité des DI au COVID-19 et appellent à l'action pour protéger les populations déplacées négligées.
Peer Reviewed
Received: June 21, 2020; Accepted: September 18, 2020; First published online: December 4, 2020
Cite this article as: Claude MK, Serge MS, Alexis KK, Hawkes MT. Prevention of COVID-19 in internally displaced persons camps in war-torn North Kivu, Democratic Republic of the Congo: a mixed-methods study. Glob Health Sci Pract. 2020;8(4):638-653. https://doi.org/10.9745/GHSP-D-20- 00272
© Claude et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00272
Global Health: Science and Practice 2020 | Volume 8 | Number 4 653 ORIGINAL ARTICLE
Meeting the Global Target in Reproductive, Maternal, Newborn, and Child Health Care Services in Low- and Middle-Income Countries
Md. Mehedi Hasan,a,b Ricardo J. Soares Magalhaes,c,d Saifuddin Ahmed,e,f Sayem Ahmed,g,h,i Tuhin Biswas,a,b Yaqoot Fatima,a,j Md. Saimul Islam,k Md. Shahadut Hossain,l Abdullah A. Mamuna,b
Key Findings ABSTRACT
n Introduction: Improving reproductive, maternal, newborn, and child Progress in reproductive, maternal, newborn, health (RMNCH) care services is imperative for reducing maternal and child health care service coverage is and child mortality. Many low- and middle-income countries (LMICs) increasing but is uneven between countries and are striving to achieve RMNCH-related Sustainable Development across subgroups (in terms of wealth, place of Goals (SDGs). We monitored progress, made projections, and calcu- residence, education, age, and sex) within lated the average annual rate of change needed to achieve universal countries. These coverage gaps are projected to (100%) access of RMNCH service indicators by 2030. continue. Methods: We extracted Demographic and Health Survey (DHS) n By 2030, none of the low- and middle-income data of 75 LMICs to estimate the coverage of RMNCH indicators countries would be able to achieve the target of and composite coverage index (CCI) to measure health system universal coverage for oral rehydration therapy strengths. Bayesian linear regression models were fitted to predict for diarrhea treatment or to seek care for acute the coverage of indicators and the probability of achieving targets. respiratory infections. Only a few countries are Results: The projection analysis included 64 countries with avail- likely to achieve universal coverage for demand able information for at least 2 DHS rounds. No countries are pro- for family planning satisfied with modern jected to reach universal CCI by 2030; only Brazil, Cambodia, contraceptive methods, recommended visits for Colombia, Honduras, Morocco, and Sierra Leone will have more antenatal care, and skilled birth attendant for than 90% CCI. None of the LMICs will achieve universal coverage assistance during birth. of all RMNCH indicators by 2030, although some may achieve universal coverage for specific services. To meet targets for universal Key Implications service access by 2030, most LMICs must attain a 2-fold increase in the coverage of indicators from 2019 to 2030. Coverage of n When designing appropriate interventions for RMNCH indicators, the probability of target attainments, and the re- increasing the coverage of reproductive, quired rate of increase vary significantly across the spectrum of maternal, newborn, and child health care sociodemographic disadvantages. Most countries with poor histori- services, program managers should consider cal and current trends for RMNCH coverage are likely to experience disadvantaged and marginalized populations. a similar scenario in 2030. Countries with lower coverage had n Acceleration is needed in coordinated global higherdisparitiesacrossthesubgroupsofwealth,placeofresi- ’ ’ efforts and government policies focusing on dence, and women s/mother s education and age; these disparities marginalized groups, administering cost- are projected to persist in 2030. effective interventions, and implementing Conclusion: None of the LMICs will meet the SDG RMNCH 2030 proactive follow-up for routinely scheduled targets without scaling up essential RMNCH interventions, reduc- health care services. ing gaps in coverage, and reaching marginalized and disadvan- taged populations.
a Institute for Social Science Research, The University of Queensland, g Health Economics and Policy Research Group, Department of Learning, Indooroopilly, Australia. Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden. b The Australian Research Council Centre of Excellence for Children and Families h Department of Tropical Disease Biology, Liverpool School of Tropical Medicine, over the Life Course, The University of Queensland, Indooroopilly, Australia. Liverpool, United Kingdom. c Spatial Epidemiology Laboratory, School of Veterinary Science, The University i Health Systems and Population Studies Division, International Centre for of Queensland, Gatton, Australia. Diarrhoeal Disease Research, Bangladesh, Dhaka, Bangladesh. d Children’s Health and Environment Program, Child Health Research Centre, The j Centre for Rural and Remote Health, James Cook University, Mount Isa, University of Queensland, South Brisbane, Australia. Australia. e Department of Population, Family and Reproductive Health, Johns Hopkins k Department of Statistics, University of Rajshahi, Rajshahi, Bangladesh. Bloomberg School of Public Health, Baltimore, MD, USA. l Department of Statistics, College of Business & Economics, United Arab Emirates f Bill and Melinda Gates Institute for Population and Reproductive Health, Johns University, United Arab Emirates. Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. Correspondence to Md. Mehedi Hasan ([email protected]).
Global Health: Science and Practice 2020 | Volume 8 | Number 4 654 Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
INTRODUCTION and child mortality, coverage of some RMNCH educing maternal and child morbidity and care services remains poor, including mDFPS, oral Rmortality and improving reproductive, ma- rehydration therapy (ORT), and care seeking for 7 ternal, newborn, and child health (RMNCH) are acute respiratory infections (ARI care). However, top priorities of the global health agenda, particularly no projections were made to identify which countries for low- and middle-income countries (LMICs).1 are unlikely to achieve global RMNCH targets. To During the era of the Millennium Development bridge this evidence gap, the Global Burden of Goals (MDGs) between 1990 and 2015, coverage of Diseases (GBD) collaborators recently examined effective RMNCH interventions to reduce maternal trends and projected target attainments of 41 health- and child morbidity and mortality was scaled up in related SDG indicators in many countries and terri- LMICs.2 This global initiative showed rapid progress tories.8 Again, projections of these indicators across in increasing the coverage of RMNCH care services socioeconomically disadvantaged subgroups are such as accelerated coverage of demand for family still missing in the existing literature. planning satisfied with modern contraceptive meth- Trend analysis helps policy makers and pro- ods (mDFPS), presence of a skilled birth attendant gram managers assess current progress, reformu- (SBA), and radically increased coverage of child vac- late policies, and design necessary interventions. cinations, while other services had modest progress Projections for RMNCH care services across differ- Projections for and a few were far behind in meeting the global tar- ent sociodemographic dimensions are central to RMNCH care 3 gets. Despite significant improvements in health identifying the key priority areas or groups (i.e., services across MDGs globally, the population-level inequality be- identifying the most disadvantaged groups to be different socio- tween the poorest and richest households and be- covered under interventions) to reinforce or refor- demographic tween urban and rural areas did not change in many mulate current policies for achieving country dimensions are 1 LMICs. Most importantly, individual-level dispari- goals. A number of studies, including those con- central to ties in terms of gender, age, education, and geograph- ducted by the Countdown Network and GBD, identifying the key ical location suggested further review of global have evaluated the current status, examined trends, priority areas or agendas for designing and implementing RMNCH and made projections of RMNCH care services and 1 groups. interventions was needed. some composite indices at the global, regional, or In 2015, the United Nations General Assembly country level.8–14 However, none of these studies summit global developmental agenda shifted from captured key interventions for RMNCH separately 4 MDGs to Sustainable Development Goals (SDGs). to make projections across subgroups by sociodemo- The top priority of SDG target 3.8 is to achieve uni- graphic stratifications. 5 versal health coverage (UHC), which means that : In this study, we used the most recent data to all individuals and communities receive the health ser- assess progress, make projections, and calculate vices they need without suffering financial hardship. the probability of target attainment and the re- quired average annual rate of change (AARC) for Forty years after the adoption of the historic achieving targets of RMNCH care services across Declaration of Alma-Ata, the World Health various population subgroups within LMICs. We Organization (WHO) in partnership with the also calculated gaps in coverage of services across United Nations Children’s Fund (UNICEF) and a set of sociodemographic dimensions. We did the Ministry of Health of Kazakhstan hosted the our analyses within and between countries to Global Conference on Primary Health Care in identify the most disadvantaged countries and October 2018 to recommit to primary health care groups within countries with inadequate access as the cornerstone of UHC in the new Declaration to RMNCH care services. of Astana.5 Theaimsofthedeclarationaretore- new political commitment to primary health care from governments, nongovernmental organiza- METHODS tions, professional organizations, academia, and Data Sources global health and development organizations. To calculate the coverage of RMNCH care services, RMNCH care services constitute a significant por- we used macro-level (aggregated) data from large- tion of UHC, and reaching and maintaining high scale, population-based, nationally representative rates of coverage of priority interventions indicate cross-sectional surveys conducted repeatedly be- the strength of health systems of a country.6 The tween 1990 and 2018 under the Demographic results of the Countdown Network suggest that in and Health Surveys (DHS) program15 in LMICs. many LMICs with the highest burden of maternal Established in 1984 by the United States Agency
Global Health: Science and Practice 2020 | Volume 8 | Number 4 655 Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
for International Development, the DHS program number of study countries. In addition, we con- aims to provide decision makers in participating structed a composite coverage index (CCI) by us- countries with improved information and analy- ing the 8 RMNCH care service indicators ses useful for informed policy choices, improve co- according to the formula proposed by Boerma et ordination and partnerships in data collection at al. The CCI is a weighted mean of the 8 RMNCH the international and country levels, develop the care service indicators (Supplement 1 includes skills and resources necessary to conduct high- more details).17 To construct the index, we consid- quality demographic and health surveys, improve ered all DHS surveys that contained information data collection and analysis tools and methodolo- on all RMNCH care services. However, we per- gy, and improve the dissemination and utilization formed trend analysis only for the countries with of data.15 The DHS program provides population- based, repeated cross-sectional data that capture a data available for at least 2 DHS rounds to ascer- wide range of monitoring and impact evaluation tain the trends. The estimates of CCI were not indicators in the areas of population, health, and computed for DHS surveys with missing informa- nutrition. Since the program began, more than tion on any of the RMNCH care services. 300 nationally representative household-based surveys have been completed under the DHS proj- Statistical Analyses ect in more than 90 countries. Many of the coun- We estimated the weighted coverage of RMNCH tries have conducted multiple DHS surveys to care services as proportions along with 95% confi- establish trend data that enable them to gauge dence intervals from the original survey data. We progress in their programs. The samples of DHS calculated the coverage of RMNCH care services surveys are generally representative at the nation- across subgroups in terms of wealth quintiles, al, residence (urban to rural), and regional level place of residence, education of women/mother, (departments, states, or divisions). The collection age of women/mother, and sex of child (for child of the DHS sample is usually based on a stratified health care services). We used the variables that multistage cluster design. The data are made avail- DHS constructed to present the estimates in the able by MEASURE DHS. reports. The socioeconomic status of households DHS obtained data through standardized inter- was determined according to the asset-based views of women of reproductive age (15–49 years) wealth index as a proxy measure of household so- from the countries under their program, which 18 included a list of prioritized countries for the cioeconomic status. The DHS constructed the Countdown cycle.7,16 We downloaded, managed, household wealth index based on household and combined the data from the website to track characteristics and ownership of assets by princi- 19 the progress and make projections about coverage pal component analysis. The households were of RMNCH care services at national and subpopula- ranked based on wealth scores and divided into tion levels. quintiles, from the poorest quintile (lowest 20% of the index) to the richest quintile (highest 20% of the index). The DHS generated variables RMNCH Care Service Indicators on place of residence (rural and urban) based on We selected 8 indicators related to RMNCH care geographical and administrative locations and ed- We selected services from a range of intervention areas to as- ucation (no education, primary, secondary and 8indicators sess health care systems or delivery for mothers higher) based on year of schooling. For this study, related to RMNCH and their children throughout their life stages, care services to across the continuum of care and aligning with we categorized the education variable and classi- assess health care global targets.7 These indicators included mDFPS; fied as less than secondary-level education (no ed- systems or antenatal care visits (ANC); presence of an SBA; ucation and primary level) and secondary-level or child immunizations for measles, BCG, and higher education to stratify the study population. delivery for 10 mothers and their 3 doses of diphtheria-pertussis-tetanus (DPT); See DHS reports for more details. Notably, we children. ORT for diarrhea treatment; and ARI care. Global restricted our analysis at the country level but not standard definitions were used in defining at the regional level for 2 reasons. First, some RMNCH care service indicators (Table). Notably, regions had few numbers of countries and had we considered ANC as receiving service at least heterogeneity between survey years, and second, 4 times from any provider or at least once from a we were interested in assessing progress across in- medically trained provider to ensure that the esti- dividual countries so that country-level programs mates of ANC can be captured from the maximum and policies could be implemented.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 656 Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
TABLE. Reproductive, Maternal, Newborn, and Child Health Care Services Indicators for the Composite Coverage Index
Target Used in This Study for Indicators Definitions SDG Target Calculating Probability
Prepregnancy Demand for family planning satisfied The proportion of married women aged 15–49 Universal accessa ≥99% with a modern method among years who do not want any more children or want married women to wait 2 or more years before having another child and are using modern contraception Pregnancy Antenatal care visits The proportion of women aged 15–49 years in the Universal access ≥99% 3 years preceding the survey who received at least 4 visits from any provider or at least 1 visit from a medically trained provider (i.e., a doctor, nurse, or midwife) during their last pregnancy Birth Skilled attendance at birth The proportion of livebirths assisted by a skilled Universal access ≥99% health provider (i.e., a doctor, nurse, or midwife) in the 3 years preceding the survey Infancy and early childhood BCG immunization The proportion of children aged 12–23 months Universal access ≥99% who received 1 dose of the BCG vaccine DPT immunization The proportion of children aged 12–23 months Universal access ≥99% who received 3 doses of the DPT vaccine Measles immunization The proportion of children aged 12–23 months Universal access ≥99% vaccinated against measles Childhood Oral rehydration therapy The proportion of children aged 5 years or younger Universal access ≥99% with diarrhea who received oral rehydration therapy (i.e., oral rehydration salts, recommended home solution, or increased fluids) in the previous 2 weeks Care seeking for symptoms of acute The proportion of children aged 5 years or younger Universal access ≥99% respiratory infections with symptoms of acute respiratory infections for whom medical treatment was sought from an ap- propriate health provider in the previous 2 weeks
Abbreviations: BCG, bacille Calmette-Guérin; DPT, diphtheria, pertussis, and tetanus; SDG, Sustainable Development Goal. a Universal access is 100%.
To examine trends, Bayesian linear regression achieve universal coverage of these services. We models that used a Markov Chain Monte Carlo also validated our estimates drawn from regression algorithm of multiple imputations for missing models with those drawn from the original micro- data were applied to estimate the coverage of data (Supplement 2 and Supplement 3 Table S12). RMNCH care services and trends from 1990 to We used Stata (version 15.1) and R (version 2018 (Supplement 2). We extended this trend 3.5) statistical software to analyze our data. We extracted data analysis to project the coverage of RMNCH care from 283 surveys services up to 2030 as set for achieving the SGD RESULTS target. We reported credible intervals drawn from from 75 LMICs, of Bayesian regression analysis along with the esti- Sample Characteristics which 64 countries mates. We calculated the probability of achieving We extracted data from a total of 283 surveys from were surveyed at the coverage of RMNCH care services as 99% or 75 LMICs, of which 64 countries (272 surveys) least twice and more by 2030 to understand which countries and were surveyed at least twice and included in included in the populations within each country are on track to the trend analysis. Projections of CCI were made trend analysis.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 657 Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
for 59 countries that had information for all 59 countries) are projected to have poor CCI (less 8 RMNCH care services for at least 2 DHS rounds. than 70%) in 2030, with the lowest CCI in Guinea More than 4.2 million women 15–49 years of age (46.7%), Chad (47.1%), Nigeria (48.2%), Yemen were included for reproductive and maternal (54.6%), and Benin (55.6%) (Figure 2). health care services, and more than 2.5 million Among countries included in the trend analy- children under 5 years of age were included for sis, more than 90% coverage is projected to be newborn and child health care services. A detailed achieved by 14 of 62 countries for mDFPS, 41 of description of the survey year and number of par- 64 countries for ANC, 29 of 63 countries for pres- ticipants are presented in Supplement 3 (Table ence of an SBA, 22 of 61 countries for measles im- S1). All the fitted models for projection analysis munization, 28 of 60 countries for 3 doses of DPT achieved convergence. The potential scale reduc- vaccine, 42 of 61 countries for BCG, 3 of 61 coun- tion factor values are summarized in the tries for ORT, and 3 of 62 countries for ARI care by Supplement 3 (Table S2 to Table S11). 2030. In 2030, the lowest levels of coverage are projected to be in Albania (1.5%) for mDFPS, in Trends and Projections Burundi (0.1%) for ANC, in Angola (8.7%) for Significant gaps From 1990 to 2018, the CCI increased in all LMICs presence of an SBA, in Kazakhstan (2.4%) for exist in the and is projected to continue increasing (Figure 1). BCG immunization, in Gabon (11.2%) for 3 doses coverage of However, the progressions varied between coun- of DPT vaccine, in Nicaragua (7.0%) for measles im- RMNCH care tries. Based on the current trend, 34 of 59 countries munization, in Cameroon (15.3%) for ORT, and in services across (56.7%) are projected to have less than 80% CCI Guinea (14.6%) for ARI care (Supplement 3 Figure S9 population by 2030. The country-specific projections showed to Figure S16). subgroups, and that only Brazil (95.6%), Sierra Leone (93.0%), they are projected Cambodia (93.0%), Honduras (90.7%), Colombia Inequalities to continue into (90.5%), and Morocco (90.3%) are likely to have The intracountry inequalities show that signifi- the future. more than 90% CCI. A number of countries (17 of cant gaps exist in the coverage of RMNCH care
FIGURE 1. Progress and Projections of Composite Coverage Index in Low- and Middle-Income Countries
Abbreviation: CCI, composite coverage index.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 658 Reproductive, Maternal, Newborn, and Child Health Services Achievements and
FIGURE 2. Projected Coverage in Percentages of Composite Coverage Index in 2030 Across Countriesa
services across population subgroups, and these poorest households are projected to be larger, gaps are projected to continue into the future yielding greater CCI among the richest compared (Figure 3 and Supplement 3 Figure S17 to Figure to the poorest, with the largest gap in Nigeria by S56). The gaps for CCI between the richest and 63.4 percentage points and the smallest gap in Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
FIGURE 3. Trends in Predicted Composite Coverage Index Across Countries by Wealth Quintiles
Peru by 0.5 percentage point (Figure 4). In con- richest and poorest at the national level and across trast, the CCI is projected to be greater among the urban-rural residence (Supplement 3 Figure S57 poorest compared to the richest by 23.9 percent- to Figure S88). age points in Liberia. Most of the countries with We also tracked progress in newborn and child the largest richest-poorest gaps are likely to expe- health care services based on sex of the child. By rience larger urban-rural gaps as well in the CCI, 2030, the projected coverage of ORT will be less with the greatest CCI gap in the urban population than 80% in most of the LMICs for both boys and by 25.1 percentage points in Nigeria and the smal- girls (Supplement 3 Figure S92). Similarly, the lest gap in Guatemala by almost nil (Figure 4). In coverage of ARI care for both boys and girls is pro- line with richest-poorest and urban-rural gaps, jected to be less than 80% by 2030 in most of the the coverage gaps between women with less countries (Supplement 3 Figure S93). The current than secondary-level education and women with sex-based gaps in child immunization rates secondary-level or higher education are also are also likely to persist in some countries in expected to remain larger in 2030, with the largest 2030 (Supplement 3 Figure S89 to Figure S91). CCI gap among the women with secondary-level education or higher compared with women with None of the LMICs less than secondary-level education in Nigeria by Probability of Target Attainment are likely to 36.1 percentage points and smallest gap in According to the posterior probability, Brazil achieve universal Indonesia by 0.1 percentage points. The CCI gaps (72%) has the highest probability of achieving CCI by 2030, between adolescent and adult women are also ap- universal CCI, followed by Kazakhstan (40%) and although some parent, but these gaps are considerably narrower Sierra Leone (20%) (Supplement 3 Table S13). Our may achieve than gaps observed across wealth, residence, and results indicate that it is unlikely that any of the universal education (Figure 4). Indicator-specific projec- LMICs will achieve universal CCI by 2030. Some coverage for tions highlight that the gaps in the coverage of all countries are likely to achieve universal coverage certain RMNCH 8 RMNCH care services are expected to be largely for some RMNCH care services, particularly ANC care services. apparent in 2030, predominantly between the visits, presence of an SBA, and BCG immunization
Global Health: Science and Practice 2020 | Volume 8 | Number 4 660 Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
FIGURE 4. Projected Gaps in Composite Coverage Index Across Countries by Wealth Quintiles, Place of Residence, and Women’s/Mother’s Education and Age in 2030
Abbreviations: NPE, no education and primary-level education; SHE, secondary or higher-level education. in Armenia, Brazil, Cambodia, and Jordan. But the increases annually between 2019 and 2030, parti- probability of achieving universal coverage for oth- cularly by 9.5% in Chad, 7.5% in Nigeria, 7.2% in er services is close to zero for the majority of the Guinea, and 6.8% in Yemen (Figure 5). The largest countries. The posterior probability of achieving improvements are required for mDFPS for most of universal coverage of RMNCH care services across the countries, urgently in Albania by 28.2%, subgroups is also zero for most of the countries Maldives by 15.0%, Democratic Republic of the (Supplement 3 Table S15 to Table S22). Additi- Congo by 13.3%, Chad by 13.2%, and Yemen by onally, we calculated the posterior probability of 11.1% (Supplement 3 Table S28). Acceleration in countries achieving at least 75% coverage for improving the coverage of both ORT and ARI mDFPS. The results showed that nearly one-third care needs to be at an annual rate of 3%–10% (19 of 62 countries) of the countries are on track for almost all the countries to achieve the targets to achieve the target of at least 75% mDFPS cover- (Supplement 3 Table S61 and Table S67). age with at least 90% probability of attaining the However, the AARC varied across different goal (Supplement 3 Table S23). sociodemographic dimensions within countries (Supplement 3 Table S24 to Table S72 includes Change Rates details for all RMNCH care services). The progression rates in CCI varied over time; slower rates of progression in CCI are projected in most of the countries during 2019–2030 com- DISCUSSION pared with the progression rate during 1990– This study provides the most up-to-date estimates 2018 (Figure 5). Some countries (e.g., Maldives, on the progress of LMICs toward the key RMNCH À0.2%) had retrogression in CCI during 1990– care services, and it predicts coverage of these ser- 2018 that will continue during 2019–2030. The vices by 2030 to detect whether RMNCH targets calculated AARC shows that achieving the target can be achieved. Based on current trends, we will require ramping up the rate at which CCI demonstrated that none of the LMICs would be
Global Health: Science and Practice 2020 | Volume 8 | Number 4 661 Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
FIGURE 5. Average Annual Rate of Changea in Composite Coverage Index in Low- and Middle-Income Countries
a Annual rate of change is calculated as: ln[(rate in latest year/rate in earliest year)]/(latest year À earliest year), with positive values located on the right side of the diagonal line at 0 (in X-axis) denoting an increasing rate, while negative values located on the left side of the diagonal line at 0 (in X-axis) denoting a decreasing rate.
able to meet the target coverage for either ORT for Specific Services Although the diarrhea treatment or ARI care. Although the cov- Our results indicate that all countries are unlikely coverage of erage of RMNCH care services is increasing, the to achieve universal CCI. Some countries are on coverage gaps across sociodemographic dimensions RMNCH care track to achieve universal coverage for childhood remain and are projected to persist. Substantial var- services is immunization for BCG, DPT, and measles vaccines. increasing, iations exist in the coverage of RMNCH care ser- vices between countries and between subgroup Concurrently, some countries such as Maldives, coverage gaps levels within countries. These results emphasize Nigeria, Tajikistan, Yemen, Chad, and Zimbabwe across socio- the need for effective policies focusing on marginal- are projected to have less than 80% childhood im- demographic ized groups, administering cost-effective interven- munization coverage in 2030. The results of our dimensions tions, and implementing proactive follow-up for study demonstrate that coverage of 2 care-seeking remain and are routinely scheduled health care visits to ensure services for child morbidity, ORT, and ARI care will projected to universal access to RMNCH care services. The be remarkably lower (less than 50% in 25 countries persist. results of this study provide evidence to inform of 61 for ORT and 18 countries of 62 for ARI care) global and country leaders and policy makers about than the target coverage in LMICs. The probability the country-specific situations at national and sub- of achieving universal coverage for these 2 services group levels and highlights key areas of interven- tions (such as improving ORT and ARI care by 2030 is roughly zero for all countries, except services) that need urgent attention for increas- Sierra Leone (57% probability) for ORT and Brazil ing the coverage of these services through allo- (39% probability) for ARI care. cating national funding and resources toward By 2030, universal coverage is expected to achieving the 2030 target for RMNCH care be achieved by Liberia for mDFPS; Maldives, services. Armenia, and Cambodia for ANC; and Armenia,
Global Health: Science and Practice 2020 | Volume 8 | Number 4 662 Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
Honduras, and Jordan for presence of an SBA. may partly be explained by broader baseline gaps However, our results demonstrate that most of in ORT and ARI care among the subgroups in the countries are struggling to achieve universal LMICs. In general, people who were poorest, re- coverage of mDFPS, ANC, and presence of an sided in rural areas, or were adolescent and less SBA. In addition, the target coverage of these educated mothers will remain vulnerable for 3 services will not be achieved by most of the sub- achieving the target coverage by 2030. This find- groups within each of the LMICs. The lower cover- ing suggests that children belonging to either of age of mDFPS, ANC, and presence of an SBA these vulnerable groups should be given special among the poorest populations, those living in ru- consideration in the design of interventions to ral areas, and women with less education will im- scale up RMNCH care services. pede LMICs, particularly countries in South and The gap in CCI must be considered before plan- Southeast Asia and sub-Saharan Africa, in achiev- ning for actions to improve the strengths of health ing the target coverage for these 3 services. systems. As the projected estimates reveal that Although the overall CCI increased, we project none of the LMICs will be able to achieve the CCI that LMICs and all subgroups within LMICs will target by 2030, we postulate that the lower CCI not be able to reach universal CCI by 2030, espe- among the poorest, rural, women/mother with cially due to the lower CCI led by mDFPS, ORT, less than secondary-level education, and adoles- and ARI care among adolescent girls and mothers cent women/mother groups has a substantial con- and among women and mothers who are poor, tribution to the lower CCI. To achieve universal have less education, and live in rural areas. Our coverage, accelerations on improvements are es- findings correspond with those from previous sential in LMICs with nearly 4% improvements in studies, with negligible variations,8 which were annual national coverage and 2%–5% improve- mainly driven by the number of time points with ments in annual coverage at subgroup levels in available data analyzed. LMICs. All countries are projected to fail to achieve the CCI target coverage by 2030 at national and subgroup levels, and only some Latin American Equity and Caribbean countries will have more than Based on our results, large coverage gaps exist in 80% CCI and are on track for achieving the target childhood immunization coverage between poor if effective RMNCH strategies can be implemented. and rich households, rural and urban populations, However, most sub-Saharan African countries will mothers with low and high education levels, and be far behind in reaching the CCI target. Similar to adolescent and adult mothers. To achieve univer- LMICs, the subgroup coverage gaps in RMNCH sal immunization coverage by 2030, most coun- care services will constitute the key driver behind tries need to further ramp up of coverage, this target failure. To accomplish the goals of particularly for the poorest and rural populations achieving universal access to RMNCH care services, and less educated and adolescent mothers in sub-Saharan African countries need to increase the LMICs with low coverage of RMNCH care services. coverage of RMNCH care services by more than To increase the coverage of RMNCH care services, 3 times during 2019–2030 than what was calculat- equitable, appropriate, and focused programs ed during 1990–2018, giving particular attention to need to be implemented, and resources need to the poorest, rural, and less educated and adolescent be allocated to increase availability, accessibility, women/mothers. and use of services, particularly for those groups For the future progress of RMNCH care ser- shown to be the furthest behind in the current vices, it is imperative to understand the reasons study (such as poorest, rural, and less educated for lower coverage or gaps in coverage and the as- populations). These programs may help countries sociated factors for high or low coverage across dif- to reduce coverage gaps within countries toward ferent geographical settings. It is well known that achieving the global target of UHC. between- and within-country inequalities and the Our analysis found considerable disparities in lack of financial resources are major constraints the coverage in ORT and ARI care in terms of for improving RMNCH.17,20 In line with previous wealth, place of residence, education and age of evidence,16 our study also demonstrates that cov- mother, and sex of the child. These gaps may per- erage of health care services that can be scheduled sist until 2030 in some LMICs, predominantly in in advance, such as immunization coverage, were countries in sub-Saharan Africa. In most LMICs, higher and are likely to be achieved by 2030, while the coverage for ORT and ARI care will be less those that require emergency on-demand avail- than 80% across most subgroups. This projection ability of workforce and specialized equipment
Global Health: Science and Practice 2020 | Volume 8 | Number 4 663 Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
(e.g., presence of an SBA) and acute care for child- representative data collected from multiple sources hood illness (e.g., ARI care) had lower coverage may better project the future directions of the and are highly unlikely to reach the target by RMNCH care services with lower uncertainty. most of the countries. To improve emergency on- Moreover, all the estimates drawn from DHS data demand care, acceleration of relevant actions and were mostly based on self-reports of respondents increase of investments are crucial for adequate and hence may have recall bias in reporting. access, human resources, and demand-based sup- However, DHS followed standard methodology plies for the population. and questionnaires for more than 3 decades to pro- vide population-based data that are representative Strengths and Limitations at not only national but also subnational levels. In this study, we used globally recognized nation- ally representative data to calculate the coverage CONCLUSIONS of RMNCH care services that provided reliable Although the coverage of RMNCH care services is estimates of trends along with the AARC during improving in LMICs, the progress is uneven with- different periods. We used a set of globally accept- in and between countries and insufficient to meet ed standard outcome interventions that cover life the health SDGs. Most sub-Saharan African and stages of women during prepregnancy to child- South and Southeast Asian countries are very un- hood of their offspring at the population level and likely to achieve target coverages by 2030 due to across the continuum of care. The use of large low coverage overall and high coverage gaps in RMNCH services between the richest and poorest, samples from population-based household surveys urban and rural, and high and low education sub- enabled us to estimate national and subgroup-level groups. These results reflect the urgent need for trends across countries as well as across subgroups health interventions targeting disadvantaged within countries. The unique survey methodology countries and their subgroups to achieve universal and measurement of the DHS allowed this study to access to health services and to reduce health make cross-country comparison of estimates as inequalities during the SDG era. Increasing fund- well. However, the findings of our study need to ing for RMNCH care through cost-effective inter- be interpreted in light of some limitations. ventions may strengthen health care services and For cross-country comparison, we considered can help interventions reach marginalized and a doctor, nurse, or midwife as skilled personnel disadvantaged people. Country leaders, stake- for assisting birth as recommended.21 This under- holders, and agencies need to undertake multidis- estimates the coverage estimates of skilled birth at- ciplinary collaborative actions by going beyond tendance for some countries that may have other their commitment in allocating resources, imple- skilled service providers, such as paramedics, fam- menting programs, and monitoring the progress ily welfare visitors, and community skilled birth and gaps in RMNCH care services toward achiev- ing SDG target 3.8 by 2030. attendants. Because some countries had too few surveys with available information, we could not Acknowledgments: We thank the DHS program for providing access to make projections of RMNCH care services for DHS datasets. those individual countries. Interventions to im- prove RMNCH care services come in phases and Funding: MMH acknowledges the financial assistance from the may reach some subpopulations before others. University of Queensland and the Commonwealth Government of However, we were unable to examine whether Australia to undertake his PhD. the past changes would proceed uniformly in the Competing interests:: None declared. future within and across countries due to the het- erogeneity in survey years within and across countries. Fewer data points for some RMNCH REFERENCES 1. United Nations. The Millennium Development Goals Report 2015. care services for some countries may have created United Nations; 2015. Accessed September 11, 2020. https:// wider credible intervals for the projected estimates www.un.org/millenniumgoals/2015_MDG_Report/pdf/MDG% of the service coverage (e.g., for CCI in Nicaragua). 202015%20rev%20(July%201).pdf Credible intervals with a wide range are normal 2. Victora CG, Barros AJD, Axelson H, et al. How changes in coverage for projection analysis, but they could be nar- affect equity in maternal and child health interventions in 35 Countdown to 2015 countries: an analysis of national surveys. rowed by having multiple time points available Lancet. 2012;380(9848):1149–1156. CrossRef. Medline (e.g., for CCI in Bangladesh). Calculating more re- 3. World Health Organization (WHO), UNICEF. Countdown to 2015: alistic probability estimates is also possible with Maternal, Newborn & Child Survival. Building a Future for Women wider credible intervals. Estimates drawn from and Children. WHO, UNICEF; 2012.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 664 Reproductive, Maternal, Newborn, and Child Health Services Achievements and Opportunities www.ghspjournal.org
4. United Nations. Transforming Our World: The 2030 Agenda for 13. Rahman MS, Rahman MM, Gilmour S, Swe KT, Krull Abe S, Shibuya Sustainable Development. United Nations; 2015. Accessed K. Trends in, and projections of, indicators of universal health cover- September 11, 2020. https://sustainabledevelopment.un.org/ age in Bangladesh, 1995–2030: a Bayesian analysis of population- content/documents/21252030%20Agenda%20for%20Sustainable based household data. Lancet Glob Health. 2018;6(1):e84–e94. %20Development%20web.pdf CrossRef. Medline 5. World Health Organization (WHO). Universal health coverage 14. Lee HY, Huy NV, Choi S. Determinants of early childhood morbidity (UHC). WHO; 2019. Accessed September 11, 2020. https://www. and proper treatment responses in Vietnam: results from the Multiple who.int/news-room/fact-sheets/detail/universal-health-coverage- Indicator Cluster Surveys, 2000–2011. Glob Health Action. 2016; (uhc) 9(1):29304. CrossRef. Medline Guide to DHS Statistics 6. Bryce J, Terreri N, Victora CG, et al. Countdown to 2015: tracking 15. Rutstein SO, Rojas G. . Demographic and intervention coverage for child survival. Lancet. 2006;368 Health Surveys, ORC Macro; 2006. Accessed September 11, 2020. (9541):1067–1076. CrossRef. Medline https://dhsprogram.com/pubs/pdf/DHSG1/Guide_to_DHS_ Statistics_29Oct2012_DHSG1.pdff 7. Boerma T, Requejo J, Victora CG, et al; Countdown to 2030 Colla- boration. Countdown to 2030: tracking progress towards universal 16. Countdown Coverage Writing Group. Countdown to 2015 for coverage for reproductive, maternal, newborn, and child health. maternal, newborn, and child survival: the 2008 report on tracking coverage. Lancet. 2008;371(9620):1247–1258. CrossRef Lancet. 2018;391(10129):1538–1548. CrossRef. Medline 17. Boerma JT, Bryce J, Kinfu Y, Axelson H, Victora CG; Countdown 8. Lozano R, Fullman N, Abate D, et al; GBD 2017 SDG Collaborators. 2008 Equity Analysis Group. Mind the gap: equity and trends in Measuring progress from 1990 to 2017 and projecting attainment to coverage of maternal, newborn, and child health services in 2030 of the health-related Sustainable Development Goals for 54 Countdown countries. Lancet. 2008;371(9620):1259–1267. 195 countries and territories: a systematic analysis for the Global CrossRef. Medline Burden of Disease Study 2017. Lancet. 2018;392(10159):2091– 2138. CrossRef. Medline 18. Gwatkin DR, Rutstein S, Johnson K, Suliman E, Wagstaff A, Amouzou A. Socio-economic differences in health, nutrition, and 9. Ewerling F, Victora CG, Raj A, Coll CVN, Hellwig F, Barros AJD. population within developing countries: an overview. Niger J Clin Demand for family planning satisfied with modern methods among Pract. 2007;10(4):272–282. Medline sexually active women in low- and middle-income countries: who is DHS Comparative Reports No. 6: The DHS lagging behind? Reprod Health. 2018;15(1):42. CrossRef. Medline 19. Rutstein SO, Johnson K. Wealth Index. ORC Macro; 2004. Accessed September 11, 2020. 10. Demographic and Health Surveys (DHS) Program. DHS final reports. https://dhsprogram.com/pubs/pdf/CR6/CR6.pdf Accessed September 11, 2020. https://dhsprogram.com/ 20. Greco G, Powell-Jackson T, Borghi J, Mills A. Countdown to 2015: publications/publication-search.cfm?type=5 assessment of donor assistance to maternal, newborn, and child 11. UNICEF, World Health Organization (WHO). Immunization health between 2003 and 2006. Lancet. 2008;371(9620):1268– Summary. A Statistical Reference Containing Data Through 2013. 1275. CrossRef. Medline UNICEF, WHO; 2014. Accessed September 11, 2020. https:// 21. World Health Organization (WHO). Making Pregnancy Safer: The www.who.int/immunization/monitoring_surveillance/ Critical Role of the Skilled Attendant. A Joint Statement by WHO, Immunization_Summary_2013.pdf ICM and FIGO. WHO; 2004. Accessed September 11, 2020. 12. World Health Organization. Global vaccine action plan 2011– https://apps.who.int/iris/bitstream/handle/10665/42955/ 2020. Vaccine. 2013;31:B5–B31. CrossRef 9241591692.pdf
Peer Reviewed
Received: March 1, 2020; Accepted: August 26, 2020; First published online: October 8, 2020
Cite this article as: Hasan MM, Soares Magalhaes RJ, Ahmed S, et al. Meeting the global target in reproductive, maternal, newborn, and child health care services in low- and middle-income countries. Glob Health Sci Pract. 2020;8(4):654-665. https://doi.org/10.9745/GHSP-D-20-00097
© Hasan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00097
Global Health: Science and Practice 2020 | Volume 8 | Number 4 665 ORIGINAL ARTICLE
Contraceptive Method Mix: Updates and Implications
Jane T. Bertrand,a John Ross,b Tara M. Sullivan,c Karen Hardee,d James D. Sheltone
Key Messages ABSTRACT Context: Improving contraceptive method choice is a goal of inter- n Contraceptive method mix reflects both supply and national family planning. Method mix—the percentage distribution demand. of total contraceptive use across various methods—reflects both n Recent trends include a progression in hormonal supply (availability of affordable methods) and demand (client pre- methods toward implants in sub-Saharan Africa, and ferences). We analyze changes in method mix, regional contrasts, where HIV is common, more condom use in some and the relationship of the mix to contraceptive prevalence. countries. Methods: We use 789 national surveys from the 1960s through n However, dominance of 1 method in the mix remains 2019, from 113 developing countries with at least 1 million peo- very common, though countries and regions ple and with data on use of 8 contraceptive methods. Two mea- “ ” throughout the world are diverse as to which method sures assess the evenness of the mix: method skew (more than is dominant. 50% use is by 1 method), and the average deviation (AD) of the 8 methods’ shares from their mean value. Population weighted n Our analysis argues for continued concerted efforts and unweighted results are compared because they can differ of programs to increase contraceptive method substantially. choice. Results: Use of traditional methods has declined but still repre- n There is no ideal method mix; client preferences are sents 11% of all use (population weighted) or 17% (unweighted key. country average). Vasectomy’s share was historically low with the exception of a few countries but is now even lower. The pre- vious trend toward greater overall evenness in the mix has slowed recently. Sub-Saharan Africa shows a hormonal method progression from oral contraceptives to injectables to implants in a substantial number of countries. In some countries with high HIV prevalence, the condom share has increased. The leading method’s share differs by region: female sterilization in Asia (39%) and in Latin America (31%), the pill in the Middle East/ North Africa (32%), and the injectable in sub-Saharan Africa (36%). Method skew persists in 30% of countries. “Evenness” of mix is not related to contraceptive prevalence. Conclusion: The marked diversity in predominant methods under- scores the conclusion that no single method mix is ideal or appro- priate everywhere. But that diversity across countries, coupled with the persisting high degree of extreme skewness in many of them, argues for continued concerted efforts for programs to in- crease method choice.
INTRODUCTION key principle in both quality of care and the Abroader rights-based approach to family planning is method choice. As defined by the U.S. Agency for International Development, method choice exists a Tulane University School of Public Health and Tropical Medicine, New Orleans, when1: LA, USA. b Independent consultant, New Paltz, NY, USA. client-centered information, counseling, and services enable c Knowledge Management Programs, Johns Hopkins Center for Communication women, youth, men, and couples to decide and freely choose a Programs, Baltimore, MD, USA. d What Works Association, Arlington, VA, USA. contraceptive method that best meets their reproductive desires e Independent consultant, Boyds, MD, USA. and lifestyle, while balancing other considerations important Correspondence to Jane T. Bertrand ([email protected]). to safety, correct use, or switching methods.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 666 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
Method choice is a guide for optimal delivery rise in use of the injectable method starting in the of family planning services. To help ensure that 1990–1995 period. Several analyses were con- clients’ needs are met across time and changing ducted to trace these changes in the context of circumstances, the World Health Organization in their effects upon other methods.14–16 Rossier 2014 recommended that family planning pro- and Corker17 reviewed the use of traditional meth- grams include at least 5 types of modern contra- ods in sub-Saharan Africa. Rossier and colleagues ceptive methods: barrier, short-term reversible, also documented the underreporting of traditional long-term reversible, and permanent, along with methods that can occur in surveys.18 Recently, the emergency contraception.2 Method mix is an indi- United Nations (UN) Population Division published cator that shows the pattern of actual use. It gives a global review of use by method, for all women the percentage distribution of use across all meth- rather than married/in-union women, and with re- ods in a given country, also known as “method gional averages population weighted.19 share.” It can be calculated either in relation to To the extent that method choice (defined women married/in-union or to all women of re- above) is an underlying principle of quality family productive age, using data from a population- planning service delivery in developing countries, based survey. it has important implications for an “ideal” meth- Interest in the method mix of contraceptive od mix. In contrast to earlier attempts to identify use goes back at least to the 1980s and early an “appropriate” method mix for a population, 1990s, focusing not just on the empirical patterns one can argue that the “ideal” method mix occurs but also upon what might constitute an “appropri- when all women in a given country are using their ate” mix.3–5 Choe and Bulatao (1992) compared desired method, consistent with the conditions methods for finding an appropriate mix, based outlined for convenient method choice. However, partly upon the life stage of the woman, whether we are unaware of any research that has attempted before or after marriage, between births, or after to measure method mix from this perspective. the final birth.6 Following that, Galway and Method mix reflects both supply and demand. Stover (1995) published a tool online to help cal- On the supply side, method choice is optimized culate an appropriate mix, based on users’ person- when the full range of contraceptives is available al profiles, the prevailing mix, method preferences with close geographic access, with no stock-outs shown in surveys, and method characteristics, us- or cost barriers, with adequate counseling on the ing Kenya as a case study.7 methods and on the management of side effects, Potter8 (1999) argued that some mixes could and with freedom from any provider bias toward become outmoded as not fitting the emerging or away from particular methods. Method skew needs of the population. That could occur when may signal that potential users have only a limited the early pattern of contraceptive supply and use choice, based on shortcomings in the supply envi- persisted due to being reinforced by feedback ronment. However, the measure of skew, by itself, from users and program managers, as illustrated provides little insight into the reasons for the con- in case studies from Brazil and Mexico. straints on choice.20 Subsequently, Bertrand et al.9 directed atten- Method mix is also influenced by demand, in- tion to method mix in which a single method cluding individual or societal preferences. Clients’ accounted for more than 50% of all use (a attitudes are subject to many influences. They may “skewed” mix) and its relation to the quality of a seek a method because it dominates the environ- national family planning program. Related analy- ment of what is available in the national program, ses with data sets covering most developing coun- as with sterilization in India.21 The introduction of tries followed,10–12 giving attention to changing a new method with low cost may stimulate a de- mix patterns and their relationship to socioeco- mand for it, as with the implant in numerous coun- nomic correlates and to the efforts of family plan- tries in sub-Saharan Africa.22 Demand for a given ning programs. Ross et al.13 developed a different method can be adversely affected by known side approach; rather than looking at the skew due to a effects, health concerns, misconceptions, and rumors. single method, it took account of the distortions in Donors may influence the supply of methods by de- mixes across all methods: the average deviation creasing the cost and supporting training in the provi- (AD) method, which is employed below along sion of the method (e.g., implants). Program directors with measures of skew. and providers may also emphasize certain methods An historic disturbance to the prevailing meth- over others. The private sector can also influence the od mixes occurred especially in countries in east availability of methods. Cultural influences are im- and southern sub-Saharan Africa due to the steep portant. They inhibit sterilization use in the Middle
Global Health: Science and Practice 2020 | Volume 8 | Number 4 667 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
East partly on religious grounds; Islam, as practiced in 1 million, and (3) has the necessary information some countries, equates sterilization with prohibited for contraceptive use of 8 methods: female sterili- mutilation of the body. By contrast, the widespread zation, male sterilization (vasectomy), IUD, im- use of female sterilization in Latin America is accom- plant, pill, injectable, condom, and traditional panied by societal acceptance of the method as a prac- methods; these 8 are the focus of the analysis. tical means of controlling further childbearing among Other methods in the UN series, such as the female women who achieve their desired family size at a condom, Lactational Amenorrhea Method (LAM), young age. Women may especially dislike methods, vaginal barrier, and emergency contraception, ap- suchastheintrauterinedevice(IUD),thatrequire pear infrequently or at zero levels in the UN compi- pelvic examinations. Also, for unmarried young lation of surveys. Moreover, the focus on these women in some societies, confidentiality of contra- 8 methods provides continuity with earlier publica- ceptive adoption, combined with private practice tions.26–28 Although family planning programs and without partner or family interference,23 is important donor agencies promote modern methods of con- to avoid stigma. traception, we have kept traditional methods in Total demand for contraception (influenced by this analysis because its use persists in numerous the desired family size), as well as the method- countries. Also, it allows us to assess the evolution specific demand, interact with and are mediated in method mix from traditional methods to modern by the constraints in the supply environment.24 methods (or vice versa, if that is occurring). Finally, the relative significance of supply and de- Half of the surveys are either Demographic mand factors on method use varies across coun- and Health Surveys (DHS) (34%) or Multiple tries and across subnational entities. All of this Indicator Cluster Surveys (MICS) (16%), and an- reminds us that a perfect method has yet to appear other 27% are listed as “national surveys” done by nor can any 1 method ever be expected to be right various agencies. The rest consist of the Contra- for all clients. ceptive Prevalence Surveys (CPS) or Reproductive This article presents new evidence on patterns Health Surveys (RHS), largely from Latin America; and trends in method mix, overall and by regions, the Pan Arab Project for Child Development Survey We present new as well as in selected countries, for married/in- and Pan Arab Project for Family Health Survey, evidence on union women of reproductive age. Overall, our mainly in the Middle East; and the Performance patterns and aim is to provide the most current picture avail- trends in the Monitoring and Accountability 2020 (PMA2020) able but with some historical information and the * method mix to Surveys from 11 countries. entire time trend for 2 illustrative countries. provide the most By region, 24 countries are in Asia (including The objectives of the article are to: current picture 5 in the Central Asian Republics), 23 in Latin available. (1) Document recent changes in contraceptive America, 21 in the Middle East/North Africa, and method mix in developing countries 45 in sub-Saharan Africa, totaling 113. The num- bers of surveys in these regions, respectively, are (2) Examine the dominance of specific methods 223 from Asia (with 20 in the Central Asian by region and by country Republics), 160 (Latin America), 120 (Middle East/ (3) Test the relationship between evenness of North Africa), and 286 (sub-Saharan Africa), total- method mix and contraceptive prevalence ing 789. (4) Explore the implications of method skew for Regarding timing, the 789 surveys occurred from program applications 1963 to 2018; the median survey date was 2001. By decade, the percentages were 1960s (0.6%), 1970s (7%), 1980s (14%), 1990s (22%), 2000s (31%), DATA AND METHODS and the 2010s (25%). For just the latest surveys in Data for this article come from a large compilation the 113 countries, most occurred in recent years, of national surveys prepared by the UN Population 51 between 2010 and 2014 and 45 between 2015 and Division (UN Department of Economic and Social 2017. Only 17 were conducted before 2010. For Affairs) in its 2019 release.25 The database con- analyses across time, we have annualized the tained 1,202 surveys, from which we retained trend within each country, and in analyses of re- 789, using the following criteria: the country is gional trends we have weighted the data by pop- (1) classified by the UN definition as being in the ulation size. We have not adjusted the regional developing world, (2) has a population exceeding comparisons for calendar time; the dates between
Global Health: Science and Practice 2020 | Volume 8 | Number 4 668 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
the earliest and latest surveys in 1 country are not In general, the closer each method is to the mean necessarily the same as in other countries; more- of 12.5%, the lower the AD value. Over time, if over, the surveys can occur at different periods 1 method’s share moves closer to the mean, either during the development of the national family from above or below it, that reduces the AD value. planning program. Finally, the earliest-latest sur- Depending upon the country, certain methods vey comparisons give the long-term picture of may take zero values in an early survey if they change, and they avoid comparisons between sur- are severely neglected or not yet made available; veys occurring close to each other, which can intro- that makes for a high AD value. On the other duce atypical short-term fluctuations. Correlational hand, the introduction of a new method can in- analyses showed that there is essentially no relation- crease its share of the mix, moving it up from a ship between the size of the gap between the earliest zero share toward the mean of 12.5%. That would and latest surveys and the pace of annual changes. result in a decline in the AD value. In this type of cross-national analysis, one can If all 8 methods had an equal share of the mix, present the data as weighted (based on the popu- at 12.5% each, the AD value would be zero; in lation size of each country) or unweighted (in practice that has never occurred. The actual AD which each country has equal weight). Both have values range from 5 to 19. Perfect evenness does their place. Weighted data—which give every per- not exist in any country, nor would family plan- son equal importance—are useful, for example, in ning experts expect it to. Further, no AD value “ ” calculating the number of modern contraceptive should be considered the ideal ; it simply serves users in the 69 poorest countries in the world as an objective measure that allows one to assign “ ” monitored by FP2020. These estimates appropri- a score of evenness or balance to the method ately reflect the disproportionate contribution of mix of each country. large countries. By contrast, unweighted data— In the following sections, most averages are which give every country equal importance—are population weighted. The levels and changes in useful in assessing progress by country, as in the the mix are first calculated for each country and case of the UN Sustainable Development Goals. then averaged to obtain regional estimates. Rather than choose between weighted or unweighted The analysis includes the following specifics: data, we have opted to present both in this article. For trends, we calculated the change in meth- To assess mix, we employ 2 indicators. The first od mix between the earliest survey and the is “method skew,” which indicates whether any most recent survey conducted in each country single method accounts for more than half of all and then determined the average change for contraceptive use. When that extreme share occurs, each region. the other 7 methods are necessarily relegated to For the latest levels, we determined the con- smaller shares, well below 50%. Other rules could traceptive method mix for each region and be used (e.g., 60% in the FP2020 reports),29 but to for all countries using the most recent survey be consistent with previous articles on method conducted in each country. skew, we have retained the cutoff point at 50. The second measure is the AD, which Ross et We illustrated the long-term dynamics for al.30 (2015) introduced to capture the evenness of changes in method mix for the 2 examples of – – the mix across all methods, thereby augmenting Rwanda (1983 2015) and Ghana (1979 To assess method the information on skew by a single method. 2013). mix, we used Since use of the 8 methods adds to 100%, the av- We identified the 34 countries with a method 2indicators: erage of the 8 shares is always 12.5%, and the skew (>50%) as of the most recent survey method skew and share of each method varies around that average. along with the method causing the skew. average deviation. The AD measure looks at the average of the devia- We obtained the distribution of countries by tions to capture the spread of the shares. A large the AD value and examined its relationship spread usually indicates that just 1 or 2 methods to the maximum share of use by any method, account for most contraceptive use and the others based on the most recent surveys in all rather little. That again suggests a limited choice. countries.
* The full list of surveys used in the UN series included: Caribbean Contraceptive Prevalence Survey; Contraceptive Prevalence Survey; Demographic and Health Survey; Gulf Child Health Survey; Gulf Family Health Survey; Generations and Gender Survey (Rep. of Georgia); Living Standards Measurement Study survey (Pakistan); Multiple Indicator Cluster Survey; National survey that is not part of a multi-country survey program; Pan-Arab Project for ChildDevelopment Survey; Pan-Arab Project for Family Health Survey; Performance Monitoring and Accountability 2020 Survey; Reproductive Health Survey; and World Fertility Survey.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 669 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
We determined the relationship between the North Africa experiencing intermediate degrees AD value and the contraceptive prevalence of change. The most extreme shift was in sub- rate (CPR), based on the most recent surveys Saharan Africa with the injectable replacing tradi- in all countries. tional methods. In the early years, its CPR was of- ten low, so that traditional methods could represent a large percentage of a small pie. RESULTS As explained in the Methods and Data section, we addressed any concern about methodological This analysis captures the dramatic changes in differences across the survey types by rerunning method mix over several decades of international the results just with the DHS and MICS surveys family planning. Among the 113 countries studied, and found essentially no differences in the main 109 had 2 or more surveys, allowing for changes levels and patterns. We therefore decided to use between the earliest and the latest surveys the full set of surveys to enlarge the base by (Supplement). The time periods varied around an regions and to augment the time trends. average interval between surveys of 17 years. Figure 1 summarizes these changes by region and for all countries. The changes are annualized to al- Key Changes in Method Mix in Recent Years low for dissimilar observation periods, and they are From this analysis, we identified 4 key trends. population weighted. The bars above the line de- note gains by a method; those below the line, 1. Traditional Method Use Has Declined Over losses. Changes within each region add to zero. For all countries, traditional methods lost an annu- Time but Remains Substantial al average of 0.42 points of share, or 4.2 points over Traditional method use remains perplexing and 10 years. The pill also lost share, and small losses somewhat controversial among international fami- occurred for male sterilization and the IUD. ly planning experts. Some argue that programs Meanwhile, female sterilization, the implant, the should actively try to move clients from traditional injectable, and the condom gained shares. to modern methods, given the greater effectiveness Among regions, Asia showed the smallest of the latter in preventing pregnancy. Others con- changes while sub-Saharan Africa showed the tend that traditional methods, which are “natural,” most, with Latin America and the Middle East/ serve a valuable purpose; they are noninvasive,
FIGURE 1. Changes in the Method Mix Between Earliest and Latest Surveys, by Method and Region, Change per Year, Weighted by Population
Female Sterilization Male Sterilization IUD Implant Injectable Pill Condom Traditional Methods Middle East/North Sub-Saharan Asia Latin Africa Africa Total America 1.0000
0.5000
0.0000
-0.5000 Change per Year in Share of Mix
-1.0000
-1.5000
Abbreviation: IUD, intrauterine device.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 670 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
free, always available, and have no side effects. Averaging over all countries, the annual rate Some maintain that while family planning pro- of decline for traditional methods has been grams should not necessarily promote traditional 0.42%, or 4.2% over 10 years (Figure 1, total methods, people should know how to use them bar). Regional averages varied considerably, as (particularly withdrawal) in case they are having the above examples suggest. The loss of traditional sex without any other method available. Still share was least in the Middle East/North Africa at others view traditional methods as a bridge to mod- only 0.10% and greatest in sub-Saharan Africa at ern contraceptive use, especially when a woman 1.42%. The loss was quite different between Asia has experienced an unplanned pregnancy while (a low 0.16%) and Latin America (a much higher using a traditional method. 0.62%). Thus, the loss of share for traditional Despite the tremendous strides made in family methods was considerable and quite variable by Despite planning programming worldwide over the past region. The large loss in sub-Saharan Africa prob- tremendous 5 decades, a surprising 11% of all users, or about ably reflects the high initial reliance on traditional strides in 1 in 10, continue to rely on traditional methods. methods, falling to lower levels as modern meth- promoting In each country, trends in the use of each method ods rose. modern are derived from the change between the earliest Two country examples vividly illustrate the contraceptive available survey to the latest one. This approach possible changes in method mix over time. In methods over the provides the experience of the country over the Rwanda, the traditional share fell from 92% in last 50 years, long term, while mitigating short-term fluctua- about1in10users tions and measurement errors. The annual rate of 1983 to only 11% in 2015 (Figure 2), a decline of still rely on change is used to allow for different observation 81%, the largest on record. In Ghana, (Figure 3) periods between the surveys. the traditional share fell from 52% to 18%, a traditional methods.
FIGURE 2. Rwanda: Changes in Method Mix Between 1983 and 2014a
Female Male IUD Implant Injectable Pill Condom Traditional Methods Sterilization Sterilization 100%
90% Traditional Methods
80%
70%
60%
50%
40%
30%
20%
10%
0%
Abbreviation: IUD, intrauterine device. a In the middle of Figure 2, the share due to traditional methods increased and the shares for modern methods fell. The timing corre- sponds to the Rwanda genocide in mid-1994; overall contraceptive use fell from about 20% to about 13% between the surveys of 1992 and 1996 but proportionately less for traditional methods than for resupply methods dependent upon logistics systems.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 671 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
FIGURE 3. Ghana: Changes in Method Mix Between 1979 and 2017
Female Male IUD Implant Injectable Pill Condom Traditional Methods Sterilization Sterilization
100
90
80
70
60
50 Percentage 40
30
20
10
0 2016 1998 2001 2003 1979 1988 1997 2000 2004 2006 2007 1991 2011 2012 2015 1980 1987 1989 1992 1994 1996 1999 2005 2009 2013 2014 1981 1983 1984 1985 1986 1990 1993 1995 2002 2008 2010 2017 1982 Abbreviation: IUD, intrauterine device.
34% decline, less than in Rwanda but down to shares: Nepal 47.1 to 10.5 (the highest current only one-third of the starting level. figure), Thailand 14.2 to 0.5; China 12.1 to 1.7; Myanmar 10.7 to 0.6; India: 8.6 to 0.6; Sri Lanka: 8.2 to 0.0; and Bangladesh 7.1 to 1.9. Most other 2. Vasectomy’s Share of Method Mix Has countries in the data set showed small, non-zero Declined, From Low to Lower percentages for vasectomy, and in no country did Vasectomy has had limited uptake for a combina- vasectomy increase its share over time. tion of reasons related to supply and demand, Regarding national policies, a few countries especially in recent years. In the 789 surveys ex- have promoted the voluntary use of vasectomy amined here, vasectomy equaled or exceeded the with some success (for example reaching 5% of “ ” equal share of 12.5% only in the Republic of the method mix in Colombia by 2016 and in – Korea (all surveys 1985 2006), Nepal (all surveys Brazil by 2013), but the method faces cultural – 1976 2011), and Thailand (14.2% in 1969), and gender barriers, especially in sub-Saharan though close to equality in China (12.1% in Africa, with concerns that men will lose their 1992). Relatively high values elsewhere occurred strength and masculinity if they have the proce- mainly in the early days of family planning pro- dure.31 Vasectomy also faces religious barriers in gramming, from the 1960s through the mid- Muslim countries,32,33 as does female sterilization 1970s, when few other methods were available. in most Muslim countries. However, female steril- As with traditional methods, in early programs, ization accounts for a quarter of all use in Pakistan, the percentages for vasectomy often represented about 7% of the mix in Bangladesh, 13% in Turkey, a large share of quite low prevalence. and 18% in Iran. In any case, few programs have Vasectomy’s share has undergone a drastic de- opted to promote vasectomy in recent years, and in cline in 7 countries where it was important, be- practice, policy makers have shown little political tween the peak year of its use and the year of the will to explicitly promote vasectomy. most recent survey. In each country, its share of Modifications in the mix reflect the relative the method mix has plummeted.** Here are the changes in the prevalence of the methods over declines, in order of the starting levels of the time. If, for example, the use of traditional methods
Global Health: Science and Practice 2020 | Volume 8 | Number 4 672 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
remains about the same while the use of modern The difference reflects the impact of the largest methods increases, leading to a rise in total contra- countries, where fewer rely on traditional meth- ceptive use, that produces a diminishing share of ods. Other methods also reflect the impact of the all use for traditional methods. In India, total prev- largest countries. In Asia, 39% of all users rely on alence of use rose from 40.7% to 53.5% between female sterilization, but a mere 13% do so as the ’ the 1992/93 and the 2015 surveys. Female sterili- average country. The high figure is due to India s zation rose from 27.4% to 36.0%, while male ster- 67% of users on female sterilization, followed by ’ ilization declined from 3.5% to 0.3%. For the mix, China s 34%, which together represent two-thirds ’ that translates to a stable female share of 67.3% in (69%) of the region s population. Table 1 shows that the difference is reversed for the injectable: it both surveys and a decline in the male share from is not important in India and China, but it is very 8.6% to 0.6%. important in Indonesia, the region’s third largest For prevalence, overall sterilization was gain- country. The total rows give the overall contrasts ing. Vasectomy was declining, but female steriliza- for each of the 8 methods, including the large dif- tion was increasing enough to more than ference for female sterilization. compensate, and it was doing so in the context of other method changes (Figure 1). For shares that was the general pattern: in a full set of within- 3. In sub-Saharan Africa, a Hormonal Method country comparisons, the share for female sterili- Progression From Oral Contraceptives to zation rose on average twice as fast as the male Injectables to Implants Is Evident share did. The sub-Saharan Africa region is especially rele- Weighted vs. unweighted results: The mix vant for contraceptive dynamics, as it shows the looks quite different when the results are weight- greatest amount of change as countries move to- ed by the population size of the country versus ward modifications in the method mix. unweighted, when each country has an equal Historically in sub-Saharan Africa, hormonal weight. methods have dominated, apart from traditional Overall, in Table 1, last row, 11% of all users methods. In the 1970s and 1980s, such use con- rely on traditional methods (weighted data), sisted largely of oral contraceptives. But with U.S. whereas the country average is higher at 17%. Food and Drug Administration approval of the
TABLE 1. Method Mix for Latest Surveys to Compare Unweighted and Weighted Results
Female Male Sterilization Sterilization IUD Implant Injectable Pill Condom Traditional Sum
Asia Unweighted 13.4 1.9 24.8 2.5 13.9 17.8 13.2 12.6 100.0 Weighted 38.7 2.6 22.0 1.0 6.6 10.0 10.7 8.4 100.0 Latin America Unweighted 29.1 1.2 8.4 1.9 16.4 19.1 14.3 9.5 100.0 Weighted 30.8 2.6 9.2 2.5 10.7 23.2 13.2 7.8 100.0 Middle East/North Unweighted 7.2 0.3 20.2 0.2 4.6 29.4 8.6 29.5 100.0 Africa Weighted 7.7 0.6 23.4 0.3 5.6 32.0 8.2 22.2 100.0 Sub-Saharan Unweighted 3.9 0.2 3.4 11.3 30.8 21.4 12.0 16.9 100.0 Africa Weighted 3.5 0.1 3.1 13.9 35.7 18.9 7.8 17.0 100.0 Total Unweighted 11.6 0.8 12.0 5.5 19.6 22.0 11.9 16.6 100.0 Weighted 29.1 2.0 17.5 3.3 12.0 14.8 10.2 11.0 100.0
Abbreviation: IUD, intrauterine device.
** The shares were declining even while the prevalence values were rising in Bangladesh and Nepal. The underlying prevalence values (percentages) for the shares listed were: Bangladesh: 0.9 to 1.2; China: 10.2 to 1.4; India: 3.5 to 0.3; Myanmar: 1.8 to 0.3; Sri Lanka: 3.9 to 0.0; Nepal: 1.6 to 5.5; and Thailand: 2.1 to 0.4.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 673 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
injectable DMPA in 1994, injectables progressive- Chad, Ghana, Guinea, Guinea Bissau, Liberia, ly became the predominant method in many Malawi, Mali, Niger, Nigeria, Rwanda, Senegal, countries. Implants first appeared with the ap- Sierra Leone, Timor Leste, Tanzania, Togo, Uganda, proval of 6-capsule Norplant in 1990, followed by and Zambia. more advanced implants. Yet, provision of implants The latest entry in hormonal method choice is remained fairly modest, constrained both by fairly subcutaneous injectable DMPA or DMPA-SC.36 It high cost and a limited service delivery infrastructure provides a lower dose of DMPA in an approach to provide them. However, price/volume guarantees that is more conducive to community service deliv- negotiated between donors and the 2 major im- ery and even to self-injection. DMPA-SC is already plant manufacturers in 2012 and 2013 reduced becoming popular in several African countries due partly to the self-injection option.37 the price dramatically.34 Moreover, improved service delivery mechanisms, notably mobile ser- vice delivery and social franchising, vastly in- 4. Condom Use for Contraception Has Increased creased implant availability. The high and in Some Countries With High HIV Prevalence increasing prevalence of the implant (and its per- Worldwide, HIV prevalence is highest in sub- centage of market share) is due not only to high Saharan African countries.38 Not surprisingly, adoption rates, but to the long continuation of with the advent of HIV, condom use has risen to use that the implant offers. However, after the substantial shares of all contraceptive use in some recommended period of use, removals and rein- of those countries. For Botswana, Lesotho, and sertionsareneeded,soaresultofthegrowing eSwatini (formerly Swaziland), condoms are the numbers of users is that implant removals will ac- first or second most widely used contraceptive celerate, as noted by Christofield and Lacoste.35 method; their shares of the method mix are 69%, A good example of hormonal progression is 37%, and 28%, respectively. Several other coun- Ghana. The leading method in the 1970s and tries have relatively high condom shares: Angola 1980s was the oral contraceptive; it was overtaken (23%), Namibia (22%), and South Africa (16%). by the injectable in the mid-2000s, which in turn In contrast, in other countries, the condom meth- was overtaken by the implant by 2017 (Figure 3). od share is only in the single digits: Zimbabwe Currently, the shares are pill, 16%; injectable, (6%), Mozambique (6%), and Malawi (3%.) It is 26%; and implant, 28%. The general hormonal likely that condom use is higher than these figures progression pattern is evident in at least 21 other indicate, since some women are reluctant to admit countries: Angola, Benin, Burkina Faso, Burundi, condom use; also, when 2 modern methods are
FIGURE 4. Contraceptive Method Mix in Each Region and All Countries, Population Weighted
45 Female Male IUD Implant Injectable Pill Condom Traditional Methods Sterilization Sterilization 40
35
30
25
20 Percentage
15
10
5
0 Asia Latin Middle Sub- Total America East/North Saharan Africa Africa
Abbreviation: IUD, intrauterine device.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 674 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
reported including the condom, the rule is to clas- sify such clients only under the other modern TABLE 2. The 34 Countries That Have a Method method. Skew (>50%) as of the Most Recent Survey and Method Causing the Skew, Based on Women Method Mix and Skew According to Region Married or in Union and Country Method Country Skew (%) 1. The Predominant Method Differs by Region For all countries, as noted above, the most widely Injectable Ethiopia 64.4 used method is female sterilization (29%), fol- Liberia 62.8 lowed much lower by the IUD (18%). The pill (15%), injectable (12%), traditional methods Haiti 61.7 (11%), and condom (10%) follow. The smallest Sierra Leone 54.3 percentages correspond to implants (3%) and va- sectomy (2%) (Figure 4, total bars, and Table 1, Myanmar 52.9 weighted totals). Mozambique 51.9 This overall perspective masks the remarkable Indonesia 51.8 fact that the leading methods differ considerably by region (weighted data) and country: female Madagascar 51.1 sterilization in Asia (39%) and in Latin America Malawi 50.8 (31%), the pill in the Middle East/North Africa Traditional Azerbaijan 76.8 (32%), and the injectable in sub-Saharan Africa (36%). Within individual countries, the shares South Sudan 65.7 vary quite widely. DR Congo 64.8 Why these sharp disparities? The share of each The marked method reflects each region’s own balance of sup- Armenia 51.9 differences in ply and demand influences over time. The sterili- Libya 51.6 method mix reflect zation share builds up gradually from annual Bahrain 51.3 each region’sown adoptions over past years, during which those balance of supply influences would have changed; the same is true Mauritius 50.7 and demand for the other long-acting methods of the IUD and Pill Sudan 77.6 implant. On the other hand, current users of the influences over resupply methods (condoms, pills, injectables) Algeria 77.5 time. come largely from adoptions in the recent past Morocco 74.7 since their average use time is relatively short; Saudi Arabia 62.0 therefore, their use is more sensitive to recent influences, such as supply interruptions and shift- Zimbabwe 61.7 ing method preferences. Disparities in the family Mauritania 59.8 planning environment are large and fundamen- tally different in countries as dissimilar as India Laos 50.6 and Mali, and the result is a blend of cultural back- IUD Turkmenistan 87.5 ground, donor involvement, provider priorities, Uzbekistan 80.0 cost, access, and public response to the methods offered. In general, there is variety in pattern but Tajikistan 64.4 consistency in a region over time. Kyrgyzstan 55.6 Kazakhstan 54.4 2. Method Skew Persists Over Time, but the Egypt 51.5 Evenness of Method Mix Varies Greatly by Country Female Sterilization India 67.7 The number of countries with method skew has Dominican Rep. 58.6 remain unchanged in recent years. Evidence El Salvador 51.7 from the most recent surveys shows that in these 113 countries, 34 countries (or 30%) show a Condom Hong Kong 70.0 skewed method mix, the same as the 30% found Botswana 69.3 26 by Bertrand et al. and slightly lower than the Abbreviation: IUD, intrauterine device. 35% reported by Sullivan et al.27 In short, close to a third of developing countries still have a skewed method mix.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 675 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
In the 34 countries with method skew, the contraceptive prevalence,39 but that can either in- leading method differs considerably. As shown in crease or decrease the evenness of the mix. Based Table 2, the number of countries skewed toward on the 113 most recent surveys, we found no each method is injectable (9), traditional methods statistically significant relationship (R2=0.0065, (7), pill (7), IUD (6), female sterilization (3), and P=.95) between the evenness of method mix as measured by the AD and contraceptive prevalence condom (2). Table 2 also shows the extent of (Figure 5). As the CPR rises, the AD values do not method skew in each country. In no country does systematically change. There is a large variation in male sterilization or the implant have a share the AD values at any level of the CPR. more than 50%, although the share for the im- Several reasons appear to account for this lack plant has reached 46% in Burkina Faso. Also of association. First, some countries, such as China noteworthy, in half (17) of the countries, the or Vietnam, with high CPRs rely on only 1 or method skew exceeds 60%. 2 modern methods, showing a highly skewed Returning to the AD values as a measure of the method mix. Other countries, such as Niger and evenness of the mix, we find that the 113 coun- the Democratic Republic of the Congo, are also tries follow a bell-shaped curve, with a roughly highly skewed, but at low CPR levels. Additional normal distribution. Around the AD median of countries at middle CPRs vary considerably in the 11.8, about half of countries (65) are in a middle spread of their methods, some with narrow range, falling between ADs of 9.9 and 13.7, and spreads and others with wide ones. All this reflects 97 are within the wider range of ADs 8.6 to 15.0. regional disparities in method access and choice as A few are at relatively extreme values; for exam- well as other factors. ple, Nepal in the low range with an AD of only 6.6, and Egypt in the high range with an AD of 14.0. Those in the high range contain the especial- DISCUSSION This analysis shows at least 3 positive trends: a de- ly skewed cases. cline in the shares held by traditional methods in favor of more effective contraceptives, a “hormon- Total Contraceptive Prevalence Is Not Related al progression” in sub-Saharan Africa with coun- to the Evenness of the Mix tries moving from pills to injectables and in many Previous research has indicated that increasing the cases on to widespread implant use, and the in- number of available methods results in higher creased use of condoms in some countries with
FIGURE 5. Relationship Between the Measure of Average Deviation and Contraceptive Prevalence Rate, 113 Latest Surveys
Global Health: Science and Practice 2020 | Volume 8 | Number 4 676 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
high HIV prevalence. Yet, challenges remain. Despite The above analyses allow us to better under- more than 5 decades of international family plan- stand the current status of method mix, its evolu- ning, traditional methods represent an average of tion over time, and its diversity by region and 17% of the method mix in the 113 countries ana- country. Yet, key questions remain. First, to what lyzed, or 11% of all users. And close to a third extent is continuous method skew a problem in (30%) of countries still report method skew, with countries with high CPRs? Numerous countries over half of all use by a single method. have CPRs above 60% and are skewed by the Two unexpected findings are that a more even 50% rule: Dominican Republic, Mexico, and El method mix is not associated with a higher CPR, Salvador for female sterilization, Morocco and and that the leading contraceptive methods differ Zimbabwe for the pill, and near cases for the IUD: considerably more among regions than we would China and Vietnam with 48% of use on the IUD. We are unaware that any of these countries are have anticipated. taking action to improve the evenness of the mix. The current mix is a function of 2 dissimilar dy- Second, is it really a problem if a country namics: use of the long-term methods is an accumu- moves toward greater method skew after the in- lation of adoptions over past years, whereas use of troduction of a new method, if the method short-term methods comes from recent starts, due enlarges choice and helps meet the needs of cli- to their shorter continuation rates. Therefore, the ents? For example, in Burkina Faso 46% of users impact of current program initiatives and other now rely on implants, and other sub-Saharan determinants of use can be considerably greater African countries are moving in this direction. among the short-term methods. Third, in the absence of an “optimal” or “ideal” Some countries have implemented deliberate method mix, are there measures that better cap- measures to diversify method mix. An intensive ture the balance in contraceptive method mix effort in parts of 5 crisis-affected countries (Chad, that some program managers and donors seek the Democratic Republic of the Congo, Djibouti, and that are believed to better meet clients’ diverse Mali, and Pakistan) to widen access to several meth- needs? Bertrand et al.9 proposed using the real-life ods resulted in 61% of clients selecting implants and experience of countries that come closest to hav- IUDs.40 In Indonesia, community-led advocacy ing a fully balanced method mix and also have at efforts implemented in the 6 Improving Contra- least a moderately high CPR, defined as 25%. Yet, ceptive Method Mix project districts yielded in the absence of a widespread initiative to im- increases in uptake of long-acting and permanent prove method mix, any method to improve the contraceptive methods, against a national context measurement of “balance” in method mix seems in which about half of users rely on the injectable.41 to lack programmatic relevance. Yet, elsewhere such initiatives have failed to change Another approach would be to examine possi- the mix, such as efforts in Morocco in the 1990s to ble relationships between family planning pro- 30 increase IUD use in a “pill” country.26 Despite efforts gram effort measures and the characteristics of to encourage the uptake of vasectomy, its use has the mix. If strong programs best service the needs fallen sharply wherever it had claimed a significant of clients, the resulting mix may be closer to a pre- share of use; currently the highest share is 10% in ferred standard. Such work would need to take Nepal, 5% in Brazil and Colombia, and close to zero into consideration the vast divergence among in many developing countries. regions in predominant methods. What explains the persistence of method skew in some countries? The 34 countries we found Limitations with skew are nearly the same as those in the Regarding limitations in this work, one relates to 2006,27 2014,26 and 201530 reviews. Method mix the surveys available. The number of surveys per Method mix is like is like a slow-moving ship: it is possible to change country varied from 1 to 18, which decreased the aslow-moving direction only over time. It is often difficult to dis- sensitivity of the time trends in countries with few ship: it is possible entangle the 2 main categories of factors that in- surveys. Also, the surveys were not conducted in to change fluence skew: limitations on the supply side (lack the same years or at a constant interval, and we in- direction only over of access to a wider range of contraceptives, beset cluded multiple types of surveys (e.g., DHS, MICS, time. by stock-outs, cost barriers, and provider biases) CPS, PMA2020) with their dissimilar methodolo- versus those on the demand side, including in- gies. However, concerns about the latter were grained societal preferences. Is the high level of fe- allayed by the reruns done with only the DHS male sterilization in India or the Dominican and MICS types, which gave very similar results Republic the result of constrained supply of alter- to those produced by the full set. native methods, normatively influenced demand, Our primary focus on method mix resulted in or both? less attention to prevalence. In countries where
Global Health: Science and Practice 2020 | Volume 8 | Number 4 677 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
total prevalence is quite low, the mix among the 2. World Health Organization (WHO). Ensuring Human Rights in the 8 methods is less stable over time, and the share Provision of Contraceptive Information and Services: Guidance and Recommendations estimates are subject to greater sampling error. . WHO; 2014. Accessed September 29, 2020. https://www.who.int/reproductivehealth/publications/family_ Total prevalence has risen in many countries, so planning/human-rights-contraception/en/ that a method can lose share and still keep the 3. Johnson AT. Contraceptive method mix: What determines program same level of prevalence. Historically, countries and individual user perspectives? Presented at: Population Association of have moved for example from a high share of tra- America Annual Meeting; May 3–5, 1984; Minneapolis, MN. ditional methods toward lower shares, even while 4. Snow RC, Chen LC. Towards An Appropriate Contraceptive Method the absolute level of their prevalence may have Mix: Policy Analyses in Three Asian Countries. Working Paper No. changed little. Wanting to focus especially on 5. Harvard University Center for Population and Development trends in the mix both overall and by regions, we Studies; 1991. gave less attention to the complexity of method 5. World Health Organization (WHO). Contraceptive Method Mix: Guidelines for Policy and Service Delivery mix as it occurs in particular countries. Nor did . WHO;1994. we analyze the relationship between method mix 6. Choe MK, Bulatao RA. Defining an appropriate contraceptive meth- and economic status of countries by their GDP per od mix to meet fertility preferences. Presented at: Population Association of America Annual Meeting; 1992; Denver, CO. capita or similar measures. Determining an Appropriate Contraceptive We did not undertake a separate analysis of 7. Galway K, Stover J. Method Mix. Futures Group; 1995. method availability as a determinant of the mix, 8. Potter JE. The persistence of outmoded contraceptive regimes: the as beyond our scope. Measures of availability are cases of Mexico and Brazil. Popul Dev Rev. 1999;25(4):703–739. found in the FP2020 annual report for the 69 poor- CrossRef 42 est countries in the world and in a study of na- 9. Bertrand JT, Rice J, Sullivan TM, Shelton J. Skewed Method Mix: A tional family planning program efforts in more Measure of Quality in Family Planning Programs. MEASURE 43 than 80 developing countries. Any analysis of Evaluation; 2000. Accessed September 30, 2020. https://www. the relationship of availability to other measures measureevaluation.org/resources/publications/wp-00-23 must contend with the problem that data are not 10. Sullivan TM, Bertrand JT, Rice J, Shelton JD. Skewed contraceptive always available for the year corresponding to the method mix: why it happens, why it matters. J Biosoc Sci. 2006;38 latest nationally representative survey; moreover, (4):501–521. CrossRef. Medline “availability” has several dimensions including geo- 11. Seiber EE, Bertrand JT, Sullivan TM. Changes in contraceptive meth- graphic access, cost, and quality of care at the od mix in developing countries. Int Fam Plan Perspect. 2007;33 source of each method. (3):117–123. CrossRef. Medline 12. Bertrand JT, et al. Contraceptive method skew and shifts in method mix in low- and middle-income countries. Int Perspect Sex Reprod CONCLUSION Health, 2014;40(03):144–153. CrossRef. Medline A future step in researching method mix involves 13. Ross J, Keesbury J, Hardee K. Trends in the contraceptive method mix in low- and middle-income countries: analysis using a new “average more in-depth analysis of the methods that pro- deviation” measure. Glob Health Sci Pract. 2015;3(1):34–55. duce the unevenness in method mix in relation CrossRef. Medline to total contraceptive prevalence. Our analyses do 14. Adetunji JA. Rising popularity of injectable contraceptives in sub- not address the complex relationships among Saharan Africa. Afr Popul Stud. 2011;25(2):587–604. CrossRef choice, total prevalence of use, and the various 15. Ross J & Agwanda A. Increased use of injectable contraception in mix patterns. Most use in most countries is Sub-Saharan Africa. Afr J Reprod Health. 2012;16(4):68–80. accounted for by 2 to 3 methods. Limited choices Medline only partly account for that since consumer pre- 16. Sutherland EG, Otterness C, Janowitz B. What happens to contra- ferences enter in, and a full choice of many meth- ceptive use after injectables are introduced? An analysis of 13 coun- ods might not alter the prevailing pattern. tries. Int Perspect Sex Reprod Health. 2011;37(4):202–208. Nevertheless, past experience confirms that the CrossRef. Medline addition of more methods to a narrow mix 17. Rossier C, Corker J. Contemporary use of traditional contraception in increases prevalence, up to some limit. Further sub-Saharan Africa. Popul Dev Rev. 2017;43(Suppl 1):192–215. work into the history of which methods and at CrossRef. Medline what prevalence levels would be of interest. 18. Rossier C, Senderowicz L, Soura A. Do natural methods count? underreporting of natural contraception in urban Burkina Faso. Stud Acknowledgments: The authors wish to thank Rebecca E. Rosenberg for Fam Plann. 2014;45(2):171–182. CrossRef. Medline editorial assistance. John Ross acknowledges support from the Track20 Project of Avenir Health, which is supported by the Bill and Melinda Gates 19. United Nations Department of Economic and Social Affairs, Trends in Contraceptive Use Worldwide Foundation for his work on this article. Population Division (UN). . UN; 2015. Accessed October 9, 2020. https://www.un.org/en/ Competing interests: None declared. development/desa/population/publications/pdf/family/trends ContraceptiveUse2015Report.pdf 20. Bertrand JT, Sullivan TM, Knowles EA, Zeeshan MF, Shelton JD. REFERENCES Contraceptive method skew and shifts in method mix in low- and 1. GH/PRH Priorities for 2014-2020. United States Agency for middle-income countries. Int Perspect Sex Reprod Health. 2014; International Development (USAID). Internal document. USAID. 40(03):144–153. CrossRef. Medline
Global Health: Science and Practice 2020 | Volume 8 | Number 4 678 Contraceptive Method Mix: Updates and Implications www.ghspjournal.org
21. Kuang B, Brodsky I. Global trends in family planning programs, Document Review. FHI 360 and Population Council, The Evidence 1999-2014. Int Perspect Sex Reprod Health. 2016;42(1):33–44. Project; 2016. Accessed September 30, 2020. https://www.fhi360. CrossRef. Medline org/sites/default/files/media/documents/resource-vasectomy-lit- 22. Jacobstein R. Liftoff: the blossoming of contraceptive implant use in review-final.pdf Africa. Glob Health: Sci Pract. 2018;6(1):17–39. CrossRef. Medline 33. Hardee K, Croce-Galis M, Gay J. Are men well served by family Reprod Health 23. Dev R, Kohler P, Feder M, Unger JA, Woods NF, Drake AL. A sys- planning programs? . 2017;14(1):14. CrossRef. tematic review and meta-analysis of postpartum contraceptive use Medline among women in low- and middle-income countries. Reprod Health. 34. Bank D. Guaranteed impact. Stanford Social Innovation Review. 2019;16(1):154. CrossRef. Medline Summer 2016. Accessed September 30, 2020. https://ssir.org/ 24. United Nations Department of Economic and Social Affairs, articles/entry/guaranteed_impact Population Division (UN). Contraceptive Use by Method. UN; 2019. 35. Christofield M, Lacoste M. Accessible contraceptive implant Accessed September 30, 2020. https://www.un.org/en/ removal services: An essential element of quality service delivery and development/desa/population/publications/pdf/family/ scale-up. Glob Health Sci Pract. 2016;4(3):366–372. CrossRef. ContraceptiveUseByMethodDataBooklet2019.pdf Medline 25. United Nations Department of Economic and Social Affairs, 36. Self-injection of DMPA-SC in Ghana. Malawi, DRC, Senegal and Population Division (UN). World Contraceptive Use 2018.UN; Uganda: increasing access, improving continuation, and empower- 2018. Accessed September 30, 2020. https://www.un.org/en/ ing women. PATH webinar. February 13, 2019. Accessed development/desa/population/publications/dataset/ September 30, 2020. https://path.azureedge.net/media/ contraception/wcu2018.asp documents/PATH_DMPA-SC_self-injection_webinar_2019.pdf 26. Bertrand JT, Sullivan TM, Knowles EA, Zeeshan MF, Shelton JD. 37. PATH. The power to prevent pregnancy in women’s hands: DMPA- Contraceptive method skew and shifts in method mix in low- and SC injectable contraception. Published September 12, 2018. middle-income countries. Int Perspect Sex Reprod Health. 2014; Accessed September 30, 2020. https://www.path.org/articles/ 40(03):144–153. CrossRef. Medline dmpa-sc 27. Sullivan TM, Bertrand JT, Rice J, Shelton JD. Skewed contraceptive 38. Kaiser Family Foundation (KFF). The global HIV/AIDS epidemic. method mix: why it happens, why it matters. J Biosoc Sci. 2006; Published July 13, 2020. Accessed September 30, 2020. https:// 38(4):501–521. CrossRef. Medline www.kff.org/global-health-policy/fact-sheet/the-global-hivaids- 28. Ross J, Keesbury J, Hardee K. Trends in the contraceptive method mix epidemic in low- and middle-income countries: analysis using a new “average 39. Ross J, Stover J. Use of modern contraception increases when more deviation” measure. Glob Health Sci Pract. 2015;3(1):34–55. methods become available: analysis of evidence from 1982–2009. CrossRef. Medline Glob Health Sci Pract. 2013;1(2):203–212. CrossRef. Medline 29. FP2020. FP2020: Catalyzing Collaboration 2017-2018. FP2020; 40. Curry DW, Rattan J, Huang S, Noznesky E. Delivering high-quality 2019. Accessed September 30, 2020. http://2017-2018progress. family planning services in crisis-affected settings II: results. Glob familyplanning2020.org Health Sci Pract. 2015;3(1):25–33. CrossRef. Medline 30. Ross J, Keesbury J, Hardee K. Trends in the contraceptive method mix 41. Johns Hopkins Center for Communication Programs (CCP). in low- and middle-income countries: analysis using a new “average Improving Contraceptive Method Mix in Indonesia. CCP; 2016. deviation” measure. Glob Health Sci Pract. 2015;3(1):34–55. Accessed October 9, 2020. https://toolkits.knowledgesuccess.org/ CrossRef. Medline projects/indonesia 31. Shattuck D, Perry B, Packer C, Chin Quee D. A review of 10 years of 42. Avenir Health. StatTrack. Accessed September 30, 2020. http:// vasectomy programming and research in low-resource settings. www.track20.org/StatTrack Glob Health Sci Pract – . 2016;4(4):647 660. CrossRef. Medline 43. Kuang B, Brodsky I. Global trends in family planning programs, 32. Perry B, Packer C, Chin Quee D, et al. Recent Experience and Lessons 1999–2014. Int Perspect Sex Reprod Health. 2016;42(1):33–44. Learned in Vasectomy Programming in Low-Resource Settings: A CrossRef. Medline
Peer Reviewed
Received: May 20, 2020; Accepted: September 22, 2020; First published online: November 6, 2020
Cite this article as: Bertrand JT, Ross J, Sullivan T, Hardee K, Shelton JD. Contraceptive method mix: updates and implications. Glob Health Sci Pract. 2020;8(4):666-679. https://doi.org/10.9745/GHSP-D-20-00229
© Bertrand et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00229
Global Health: Science and Practice 2020 | Volume 8 | Number 4 679 ORIGINAL ARTICLE
Health Care Worker Preferences and Perspectives on Doses per Container for 2 Lyophilized Vaccines in Senegal, Vietnam, and Zambia
Natasha Kanagat,a Kirstin Krudwig,a Karen A. Wilkins,b Sydney Kaweme,b Guissimon Phiri,c Frances D. Mwansa,c Mercy Mvundura,d Joanie Robertson,d Debra Kristensen,e Abdoulaye Gueye,f Sang D. Dao,g Pham Q. Thai,h Huyen T. Nguyen,g Thang C. Trang
Key Findings Résumé en français à la fin de l'article.
n Health care workers (HCWs) in all 3 countries preferred containers with fewer doses for reconstituted vaccines such as BCG and ABSTRACT measles-containing vaccine. Introduction: Limited information exists on health care workers’ (HCWs) perceptions about use of multidose vaccine vials and n HCWs believed that containers with fewer doses their preferences about doses per container (DPC). We present of these vaccines could reduce wastage and findings from qualitative studies conducted in Senegal, Vietnam, missed vaccination opportunities. and Zambia to explore HCWs’ behavior regarding opening vials n HCWs were more willing to open a vial for every and their perceptions and preferences for the number of doses in eligible child when using containers with fewer vials of BCG and measles-containing vaccine (MCV). Zambia doses. and Senegal currently offer MCV in 10-dose vials and BCG in 20-dose vials; 10-dose vials are used for both vaccines in Vietnam. Unused doses in vials of these reconstituted vaccines Key Implications must be discarded within 6 hours. Methods: Key informant interviews (KIIs) were conducted with n Policy makers should consider HCW perspectives frontline HCWs in Senegal, Vietnam, and Zambia. In Senegal when deliberating a change in policy on vial size and Vietnam, the KIIs were conducted as part of broader forma- since HCWs have to balance concerns about tive research; in Zambia, KIIs were conducted in control districts open vial wastage with the guidance to open a using 10-dose MCV vials only and in intervention districts that vial to vaccinate every eligible child. switched from 10- to 5-dose vials during the study. During anal- n Program managers should consider shifting to ysis, themes common to all 3 countries were synthesized. Critical containers with fewer doses for vaccines without themes relevant to country contexts were also examined. preservatives to assuage HCW concerns about Results: HCWs in all 3 countries preferred containers with fewer opening vials for every eligible child while doses for BCG and MCV to reduce wastage and increase the managing wastage. likelihood of vaccinating every eligible child. HCWs in Senegal and HCWs using 10-dose vials in Zambia reported sending unvaccinated children away because not enough children were present to warrant opening a new vial. In Vietnam, where ses- sions are typically held monthly, and in Zambia when the 5-dose vials were used, almost all HCWs reported opening a vial of MCV for even 1 child. Discussion: HCWs prefer vials with fewer DPC. Their concerns about balancing coverage and wastage influence their decisions to vaccinate every eligible child; and their perspectives are cru- a JSI Research & Training Institute, Inc., Arlington, VA, USA. cial to ensuring that all target populations are reached with vac- b Consultant, JSI Research & Training Institute, Inc., Arlington, VA, USA. cines in a timely manner. c Republic of Zambia Ministry of Health, Lusaka, Zambia. d PATH, Seattle, WA, USA. e PATH, Geneva, Switzerland. INTRODUCTION f PATH, Dakar, Senegal. g PATH, Hanoi, Vietnam. any vaccines administered in low- and middle- h M National Institute of Hygiene and Epidemiology, Hanoi, Vietnam. income countries are purchased in multidose vials Correspondence to Natasha Kanagat ([email protected]). (MDVs) and can contain between 2 and 20 doses per vial
Global Health: Science and Practice 2020 | Volume 8 | Number 4 680 Health Care Worker Preferences for Vaccine Doses per Container www.ghspjournal.org
or container.1,2 Several countries buy vaccines in to conduct cost-effectiveness analyses, and MDVs because compared with single-use vials, studied routine administrative data from facili- MDVs sell at a lower price per dose; require lower ties to assess vaccine wastage. The findings cold chain, storage, and transport capacity; and from those studies are forthcoming in other generate less waste.3 Some vaccines contain pre- journals. servatives, whereas others do not. Under the In this article, we report on the BCG and MCV World Health Organization’s (WHO’s) multidose vaccines since they are common to all 3 countries, vial policy,4 remaining doses in open vials of are supplied in MDVs, and must be discarded after vaccines with preservatives can be used for up 6 hours. Zambia and Senegal use a 10-dose to 28 days after opening, as long as storage and measles-rubella (MR) vial and a 20-dose BCG proper handling conditions are met. However, vial. Vietnam uses 10-dose MCV (both measles vaccines without preservatives must be used in a and MR) and BCG vials; measles first dose is given much shorter time frame. Vaccines such as BCG, at 9 months and MR vaccine (second dose) is given measles-containing vaccine (MCV), and yellow at 18 months. fever vaccines do not contain preservatives, and In Senegal, the MR first dose is given at they must be discarded within 6 hours of reconsti- 9 months and is coadministered with yellow fe- tution or at the end of a session, whichever comes ver vaccine. The MR second dose is given at first. Health care workers (HCWs) in low- and 15 months. All routine childhood immunizations middle-income countries who administer these are given during fixed and outreach sessions. vaccines to their target populations are therefore Health facilities conduct fixed sessions that are responsible for deciding when to open a vial, held anywhere from daily to monthly, depending knowing that if not all doses are used within a on the catchment population and size of the facil- short frame of time, they will have to be discarded, ity. Outreach sessions vary in frequency, depend- resulting in open-vial wastage. ing on the number of outreach locations, This article focuses on vaccines without pre- availability of staff to conduct outreach, and oth- servatives. HCWs have to balance the expectation er factors. In Vietnam, fixed sessions are held that they will vaccinate every child with the con- once or twice a month in most of the country, al- cerns about open-vial wastage. Open-vial wast- though in some districts immunization is orga- age tends to increase with vaccines that have nized once weekly. Outreach sessions are not more doses per container (DPC) when the immu- conducted everywhere, and they vary in fre- nization session sizes are small.5 Limited infor- quency where offered. In Zambia, MR is given at mation exists on HCWs’ opinions about the 9 months and 18 months of age. Routine child- desired DPC and how DPC informs their decisions hood immunizations in Zambia are given during on when to vaccinate. Studies by Wallace et al.6 fixed and outreach sessions. Health facilities and Hutchins et al.7 suggest that HCWs’ hesitancy hold sessions anywhere from daily to monthly to open a multidose vaccine vial to avoid vaccine depending on the catchment population, size of wastage contributed to missed opportunities for the facility, availability of staff to conduct out- vaccination (MOVs). HCWs’ behavior regarding reach, and other factors. opening vials is critical to addressing MOVs, In Zambia, the guidance is to open a vial for ev- which emphasizes efforts to reach eligible chil- ery eligible child, and WHO’s multidose vial policy dren at all immunization sessions, including out- is followed: Vaccines with preservatives can be Vaccines with reach, to identify and reduce opportunities kept for up to 28 days, while vaccines without pre- preservatives can missed at the health facility level on a day-to- servatives (i.e., BCG, MCV, and yellow fever) be kept for up to day basis. MOVs can result in inadequate protec- must be discarded 6 hours after reconstitution or 28 days, while tion against disease. at the end of an immunization session, whichever vaccines without The qualitative studies described here (see comes sooner. preservatives Methods section) were part of a larger multi- In Senegal, the national level does not give must be discarded country project to improve the evidence base on guidance to health facility staff on how many eligi- 6hoursafter ’ HCWs decision making relative to DPC. This ar- ble children must be present at a session before reconstitution. ticle focuses on qualitative findings on HCWs’ HCWs can open a vial. As in Zambia, WHO’s mul- perspectives on BCG and MCV, obtained through tidose vial policy is followed. formative research in Senegal and Vietnam and a In Vietnam, sessions are held monthly in most prospective study in Zambia. The project also health facilities; therefore, the policy for all vac- conducted household surveys to examine immu- cines is to discard all remaining doses in opened nization coverage, administered facility surveys vials at the end of each session.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 681 Health Care Worker Preferences for Vaccine Doses per Container www.ghspjournal.org
METHODS researchers focused on the one district that was The qualitative study in Senegal was conducted as conducting weekly immunization, and they in- part of a broader formative research study (Box). cluded 12 communes from that district and there- Sixty health facilities (HFs) were included in the fore oversampled in the district with weekly study and 1 HCW per health facility participated immunization. Thirty HFs were purposively sam- in qualitative interviews. Health facilities were se- pled, taking into account coverage and different lected through stratified random sampling based service delivery models. In the Northern Region, on rural or urban locations, size of birth cohort health facilities were selected either because they served, and distance from the district vaccine had coverage rates below 90% or the lowest cov- store. The study was conducted in 2 regions, erage in the district. In some cases, even if they Louga and Ziguinchor, selected because their vac- had the lowest coverage rates in the district, their cine coverage rates are below national coverage coverage was over 90%. Most HFs in the Northern rates. Region are in rural settings, whereas the facilities The qualitative study in Vietnam was con- in the Central Highlands Region contained more ducted as part of a broader formative research of a mix of rural, peri-urban, and urban sites. In study that included 30 health facilities. Thirty Vietnam, HCWs were asked about measles and HCWs (1 per HF) participated in the qualitative MR separately, because their vaccine schedule interviews. The study was conducted in 4 prov- requires measles for first dose and MR for second inces in 2 regions in Vietnam: Dien Bien, Tuyen dose. Quang, and Yen Bai in Northern Region, and Dak The qualitative study in Zambia was conducted Lak in the Central Highlands Region. Dak Lak as part of a broader implementation research Province was chosen specifically because the im- study. In Zambia, 90 HCW interviews were con- munization session is organized once every week, ducted across 14 districts in 2 provinces. For the whereas in the other sites the immunization ses- implementation research, all districts were paired sions are conducted once or twice a month. The according to average population size per HF and researchers wanted to assess whether HCWs’ per- the number of HFs. From each pair, 1 district was spectives differed across sites with different session randomly assigned to the intervention, while the frequencies. In Northern Region, 2 districts in other district served as the control. During the im- each province and 3 communes in each district plementation period, all HFs in the intervention were selected. In Central Highlands Region, the district received 5-dose vials of MCV, while the
BOX. Introduction to the Dose Per Container Partnership Project Under Which the Studies Presented in the Article Were Conducted The global effort to protect all people from vaccine-preventable diseases has historically leveraged multidose containers in low- and middle-income countries to offer lower prices and reduce the constraints on cold chain space. However, as newer, more expensive vaccines are introduced in multidose formats, the burden of cost efficiency potentially moves from the national-level to the health care worker. To achieve maximum utilization of every dose in a vial and depending on the country’s policies, health care workers need to be strategic about when to open a vial and be diligent about how they care for open vials. In addition, they have to be more active with community outreach and communication to ensure optimal attendance and timely immunization. For these reasons, the number of doses per container (DPC) can hinder a country’s ability to achieve timely and equitable coverage including reaching the urban poor or rural remote. DPC can also influence additional factors like vaccine safe- ty, system costs, supply chain, and wastage. The Dose Per Container Partnership (DPCP) was a multicountry project that aimed to support vaccine product and pro- gram decision making to include considerations of DPC to optimize equitable, timely, safe, and cost-effective coverage. The Partnership implemented country-level research in several countries, including Senegal, Zambia and Vietnam, to generate new evidence on the impact of DPC decisions on an immunization system, to explore current decision making on DPC options, and to inform country and global decisions on vaccine procurement. The Partnership has produced case studies on decision making and multicountry research, as well as videos, resource guides, and briefs on various aspects of DPC. DPCP is funded by the Bill & Melinda Gates Foundation, led by JSI Research & Training Institute, Inc. and jointly imple- mented in partnership with PATH, the Clinton Health Access Initiative, the Highly Extensible Resource for Modeling Event- Driven Supply Chains modeling team, and the International Vaccine Access Center.
Global Health: Science and Practice 2020 | Volume 8 | Number 4 682 Health Care Worker Preferences for Vaccine Doses per Container www.ghspjournal.org
control group continued to use the 10-dose vials. subjects research. Approvals were obtained from The HFs receiving the 5-dose vials were oriented the Ministry of Health/Senegal Ethics Committee to the new vial size, but no other technical support and the Ethics Committee of the Vietnamese was offered over the 1-year implementation peri- National Institute of Hygiene and Epidemiology. od, to minimize influencing the HCWs’ behavior. The implementation study in Zambia received The project wanted to keep the settings as neu- ethical approval from the Biomedical Research tral as possible to allow us to observe differences Ethics Committee of the University of Zambia. in behavior at endline. HCW interviews for the qualitative study were conducted at baseline (32), midline (16), and endline (42). RESULTS Interview teams verified that respondents HCW Perceptions on Reducing Missed from all 3 countries were responsible for providing routine immunizations to ensure they had an un- Opportunities When Using 5-Dose Vials All HCWs were asked (1) whether they vaccinated derstanding of immunization service delivery. HFs All HCWs were every eligible child each time the child was at the were selected to ensure representation across asked whether health facility; (2) if MCV and BCG were offered at large, medium-sized, and small facilities in urban they vaccinated and rural locations within each district. every session; and (3) if they opened a vial of these every eligible At baseline, none of the districts in Zambia had vaccines at a session irrespective of the number of child and if they switched over to the 5-dose vials, so all 32 inter- eligible children present. These questions were opened a vial of views were conducted with HCWs who were asked to assess whether concerns about opening a vaccines without using the 10-dose vials. The midline and end- vial for only 1 child or a few children resulted in preservative HCWs either not opening the vial or waiting for a line interviews, however, were conducted after irrespective of minimum number of children being present to the districts were divided into those receiving how many justify opening it. 10-dosevialsandthosereceiving5-dosevials, children were In Senegal, MCV, BCG, and yellow fever vac- so midline and endline interviews were held present. with HCWs in districts receiving 5-dose vials to cines were not offered at every immunization ses- document their experiences using the new sion. Most respondents said that at least 5 children presentation. had to be present for MCV and yellow fever vac- In all 3 countries, contracted local data collec- cine and 10 children had to be present for BCG be- tors gathered data with oversight and supervision fore they would open a vial. When fewer children by the organization leading the country study. attended a session, HCWs asked them to return on Data analysis for each country was done separate- another date when the next session was sched- ly. For Senegal and Vietnam, responses to the uled. In Senegal, the majority of HCWs recalled qualitative surveys were analyzed in Excel. For turning away a caregiver and child from a vaccina- the questions with predefined response options, tion session at least once in the last 3 months. the responses were counted based on the response Similar findings were also observed for yellow fe- options. For the open-ended questions, the key ver vaccine and MR, which is also in 10-dose vials themes from the responses were also tabulated and is coadministered with MR. and reported. For Zambia, all transcripts were I programmed them for the next session, it’s for tomor- uploaded into NVivo 11, a qualitative data man- row. I have recorded their coordinates [location] and agement software. The qualitative team generated the relays [community health workers] take care to find an initial set of codes derived from the research them, and if they do not come, we call them on the tele- questions to analyze the data. All codes were ac- phone.—Senegal companied by code definitions. The initial set of codes comprised major thematic categories, which In Vietnam, the majority of facilities conduct were refined through analysis, and subcategories sessions once a month, and all vaccines are of- (i.e., subcodes) were developed through iterative fered at each session. Therefore, the majority of analysis. For this article, all the country reports HCWs reported that they opened a vial for and briefs generated from the separate analysis every child and were willing to open a vial were then reviewed to identify major themes during sessions when only 1 child was eligible, common to all 3 countries and summarize find- regardless of potential wastage. Most HCWs in ings. We also highlight country-specific findings. Vietnam did not recall sending children The formative research studies in Senegal and away during an immunization session because Vietnam were determined by PATH’s Research not enough children were present to warrant Determination Committee not to be human opening a vaccine vial. The few HCWs who
Global Health: Science and Practice 2020 | Volume 8 | Number 4 683 Health Care Worker Preferences for Vaccine Doses per Container www.ghspjournal.org
recalled sending children away mentioned that wastage. However, if wastage rates were deemed they advised the caregivers to bring the child higher than expected, HCWs reported that super- back for the next session. visors offered suggestions and strategies to miti- gate wastage. Sending children away without vaccination takes time In Senegal, most HCWs knew the target wast- because the caregivers have to bring their children back age rates for each vaccine. Knowing the target af- for the other immunization session.—Vietnam fected which vaccines were offered at each session. In Zambia, over half of HCWs using 10-dose To minimize wastage rates, HCWs reported not of- vials of MCV reported waiting for a minimum of fering vaccines like BCG and MCV at each session. 5 children before opening a vial, and a minimum When asked how they ensured that a certain num- of 10 children to open a vial of BCG. However, ber of children were present for the BCG and MCV when HCWs using 5-dose vials were asked about sessions, they replied: their practices, they replied that they were less Collect 10 children for BCG, MR [measles rubella], be- concerned about MCV wastage and felt more com- fore opening the bottle [vial], using relays and badjene fortable opening vials to vaccinate children. Most ngokh [community health workers] who will bring the HCWs in Zambia using the 5-dose vial stated that children and remind parents of the RV [vaccination ses- they opened a vial each time an eligible child pre- sion].—Senegal sented during a session and did not wait for a min- Most HCWs in Senegal and the 10-dose dis- imum number of children. tricts of Zambia also reported that MCV and BCG We can open the vial even when we have two children, vaccines were not offered at every session. The we only lose three doses as compared to the time we were reason for not offering these vaccines was to in- using ten-dose vials, this would make us lose eight doses. crease session sizes for these specific vaccines to —Zambia 5-dose vial district reduce wastage. In Vietnam, the session frequency determined In districts that continued to use the 10-dose whether all vaccines were offered at every session. MCV vial in Zambia, the majority of HCWs At sites where immunization was offered once or recalled turning away a child from a vaccination twice a month, all vaccines were offered every session at least once in the past 3 months. In the time. At sites where immunization was offered Zambian districts using 5-dose vials of MCV, very more frequently, not all vaccines were offered ev- few HCWs reported turning children away. ery time, to avoid vaccine wastage. Neither group of respondents had a system to BCG, measles, and Japanese encephalitis (JE) vaccines track whether the children turned away were are injected once every 2 weeks. DPT and MR vaccines brought back to the facilities for vaccinations in are injected once per month. It is because the number of the future. children who need these vaccines is less than that of oth- — Yes, because everyone is concerned on reducing the vac- er vaccines. Vietnam cine wastage. It is a reason why mothers are sent back By contrast, in the Zambian districts using and asked to come a different day when there are 5-dose vials of MCV, the majority of HCWs enough children to open the vial. This is so because ev- reported offering MCV at every fixed session re- eryone wants to reduce the wastage.—Zambia 10-dose gardless of the number of children. The HCWs in vial district the 5-dose districts did not know their wastage rates, but they believed that wastage had dimin- ished with their use of the 5-dose vials, and they Balancing Coverage and Wastage were therefore less concerned about opening the All HCWs were All HCWs were asked whether their supervisors MCV 5-dose vial for fewer children. In all 3 coun- asked whether considered coverage rates or wastage rates more tries, BCG was given on specific days, such as at their supervisors important, since the supervisors’ belief would in- postnatal sessions at health facilities or hospitals, considered fluence what the HCWs placed more emphasis or on a designated day per month, to ensure that coverage rates or on. The belief also could affect HCWs’ behavior if a large number of children would be present and wastage rates they offered certain vaccines at specific times to wastage could be limited. more important. ensure adequate numbers of children to mini- mize wastage. Most HCWs from Senegal, Yes there are days when MCV and BCG is not given dur- Vietnam, and Zambia stated that their supervi- ing outreach, for example you find two babies who have sors deemed coverage more important than been delivered. Are you going to open that vial for BCG
Global Health: Science and Practice 2020 | Volume 8 | Number 4 684 Health Care Worker Preferences for Vaccine Doses per Container www.ghspjournal.org
just for those two? So in such cases we explain to them intermediaries between communities and the It is vital to ’ that we can t offer them BCG or MCV at that particular health system, and they regularly make decisions recognize the role moment. We then advise them to come to the center, es- about the management and delivery of vaccine of vials with fewer — pecially on the last Thursday of the month. Zambia services to achieve recommended coverage levels. DPC for 10-dose district Although many factors contribute to MOVs, it is reconstituted vital to recognize the role of vials with fewer DPC vaccines in for reconstituted vaccines in reducing instances of reducing MOVs. What Are HCWs’ Preferences for DPC? MOVs. As is seen in Vietnam, with the variation of All HCWs expressed a preference for a different timing and vaccines offered during immunization vial size of BCG and MCV with fewer DPC. In sessions, high-level structural decisions are made Senegal, most HCWs preferred fewer DPC for to balance coverage and wastage. This study also All HCWs these vaccines to reduce wastage, and many said highlights the importance of providing HCWs expressed a that this could help to address the challenge of with options that do not require sacrificing vacci- preference for a needing enough eligible children to warrant open- nation coverage or having high wastage. Current different vial size ing a vial, prevent dropouts, and provide services practices create a tension between expectations of BCG and MCV to hard-to-reach children. A couple of HCWs felt and ground realities, obligating HCWs to offer life- with fewer DPC. that fewer DPC might pose challenges for storage saving vaccines infrequently, turn away children, or transportation of vials. In Zambia, HCWs from or risk not meeting expectations on wastage. This the 5-dose districts also expressed a preference for finding supports the conclusions of Wallace et al.9 fewer DPC vials for MCV; none of them wanted to that HCWs either take active measures to reduce return to using 10-dose vials. The majority of wastage or feel some conflict when wastage is HCWs in the 5-dose districts preferred 5-dose high. Separate quantitative analyses from this vials, and the rest preferred fewer than 5 doses. In project confirm HCWs’ perceptions that fewer Vietnam, for BCG and MCV (currently in 10-dose DPC will likely increase coverage children and re- vials), the majority of HCWs expressed a prefer- duce wastage.10 In Zambia, facilities using 5-dose ence for a 1-dose vial, followed by a 5-dose and a vials had 47% lower wastage rates compared 2-dose vial (Table). with those using 10-dose vials. An increase in cov- erage of MR first and second dose respectively by Because of the mobile population here we often lose sight 5% and 3.5% in the districts using 5-dose vials — of children having single dose presentations would was attributable to the intervention (i.e., the use — permit us to vaccinate each child who presents. of the 5-dose vials). Senegal HCWs from all countries also reported turning away children if not enough children were present If vaccine was packed in single dose per vial, we could to warrant opening a vial, and in many cases, no conduct vaccination in more days per month instead of system was in place to ensure that these children doing in 1 day.—Vietnam health facilities conducting would be vaccinated later. This practice goes weekly sessions against WHO recommendations that vaccination programs include daily opportunities for vaccina- It has made things easier for us in that you do not have tion with all vaccines, offering vaccination at to worry about babies not being immunized; it’s rare every contact, including screening at curative that we miss out any child. It has made our work easier; consultations, even if there is only 1 child.11 This our minds are free that we are doing our job [immuniz- behavior also represents a MOV, requiring addi- ing] unlike the BCG.—Zambia 5-dose vial district tional effort by caregivers and HCWs to follow up and increasing the chance that the child will not receive BCG or MCV.12 Similar findings have DISCUSSION been documented in other low- and middle- This multicountry study demonstrates that wait- income countries where BCG and MCV are of- ing for a minimum number of children before fered less frequently than vaccines that do not opening a vial of BCG and MCV could result have to be discarded after 6 hours of reconstitu- in MOVs. Eliminating or greatly reducing MOVs tion, as a way to increase the number of children is critical to achieving the Global Vaccine Ac- present per session before opening a vial.6,13 tion Plan 2011-2020 goal of “90% national cover- Our findings are pertinent to current discus- age and 80% in every district or equivalent sions on session sizes during the coronavirus administrative unit, for all vaccines in the national disease (COVID-19) pandemic. Due to the pandem- immunization schedule.”8 HCWs serve as critical ic, WHO guidance recommends frequent routine
Global Health: Science and Practice 2020 | Volume 8 | Number 4 685 Health Care Worker Preferences for Vaccine Doses per Container www.ghspjournal.org
TABLE. Summary Findings on Vaccine Doses per Container and Health Care Workers’ Perceptions and Practices In 3 Countries
Theme Summary Findings
HCW perceptions on reducing missed Senegal: MCV, BCG, and yellow fever vaccines were not offered every time immunization opportunities when using 5-dose vials sessions were held. During immunization sessions, HCWs reported that they waited for a minimum number of children before opening these vaccines. HCWs recalled turning away a caregiver and child at least once in the past 3 months. Vietnam: Due to Vietnam’s session schedules, which are mostly once a month, most HCWs did not wait for a minimum number of children before opening vials. They did not recall turning away children in the past 3 months. Zambia: In the districts using the 10-dose vials, HCWs waited for a minimum of 5 children to open an MCV vial and 10 children to open a 20-dose BCG vial. In the districts using the 5-dose vials, HCWs opened a vial each time an eligible child presented and did not report turning away a child.
Balancing coverage and wastage In all 3 countries, although coverage was considered more important, HCWs reported that wastage was tracked very closely and they knew they had to minimize wastage as much as possible. In Zambia and Senegal, HCWs did not offer MCV, BCG, or yellow fever vaccines (Senegal only) at every session due to concerns about wastage. The intent was to increase session sizes for these specific vaccines as a way of reducing open vial wastage. In the facilities offering 5-dose vials in Zambia, HCWs believed that their wastage was lower, and they expressed less concern about opening the vial for fewer children compared with the facilities using the 10-dose vials.
HCWs’ preferences for DPC All HCWs expressed a preference for fewer DPC for BCG and MCV (and yellow fever for Senegal) to allow them to vaccinate eligible children, prevent dropouts, and not worry about wastage. No HCWs in the districts in Zambia that used the 5-dose vials during implementation wanted to return to using the 10-dose vials.
Abbreviations: BCG, bacille Calmette-Guérin; DPC, doses per container; MCV, measles-containing vaccine; HCW, health care worker.
immunization sessions of smaller size to reduce the criteria used to select health facilities and key risk of spreading the virus.14 informants. Data analysis was also done by sepa- As more countries As more countries consider changing their rate teams. In 1 country, respondents were consider changing DPC for different vaccines, decisions should take purposefully selected, which may limit the gener- their DPC for HCWs’ perspectives into account. This approach alizability of the results. However, the large sam- different vaccines, is not always the norm. Other DPCP case studies ple used for the qualitative interviews in all decisions should on decision making on DPC in Ghana, Benin, 3 countries ensured that we got an appropriate and adequate number of respondents whose take HCWs’ Côte d’Ivoire, and the Democratic Republic of ’ views likely represent those of the larger popula- perspectives into Congo showed that HCWs perspectives were no- tably absent.15–17 tion of HCWs. In Zambia, we collected data at account. 3 different times to enable us to document behav- We recommend that future research continue ior change in HCWs, especially in the districts that to explore the causal links between HCWs’ prac- switched to using the 5-dose vials. The researchers tices related to vaccine wastage and their impact tried to address these differences by ensuring that on vaccination coverage, MOVs, and cost implica- data on priority themes were collected across all tions. We also recommend additional research on countries, and that research protocols, data collec- HCWs’ preferences in other countries and settings tion tools, and draft reports were shared among to expand the body of evidence regarding HCWs’ teams to establish a level of consistency in the decision making about opening vials. data being collected and analyzed. Another limita- tion is that this study focused on relatively low- performing districts. However, our findings are Limitations likely also applicable to high-performing districts This study had limitations, including differences in given that public-sector resources are always study design between countries and different limited and that striking a balance between
Global Health: Science and Practice 2020 | Volume 8 | Number 4 686 Health Care Worker Preferences for Vaccine Doses per Container www.ghspjournal.org
vaccinating every child and limiting wastage will 3. Drain PK, Nelson CM, Lloyd JS. Single-dose versus multi-dose be a difficult decision for HCWs in both high- and vaccine vials for immunization programmes in developing Bull World Health Organ – low-performing districts. countries. . 2003;81(10):726 731. Medline 4. World Health Organization (WHO). WHO Policy Statement: Multi- CONCLUSION Dose Vial Policy (MDVP): Handling of Multi-Dose Vaccine Vials After Opening This 3-country study contributed evidence on . WHO; 2014. Accessed December 24, 2019. https:// apps.who.int/iris/handle/10665/135972 HCWs’ perceptions and preferences with regard 5. Assi TM, Brown ST, Djibo A, et al. Impact of changing the mea- to various DPC options for reconstituted vaccines. sles vaccine vial size on Niger’s vaccine supply chain: a com- The results suggest that when balancing the man- putational model. BMC Public Health. 2011;11(1):425. CrossRef. Medline date to achieve high coverage and reduce vaccine 6. Wallace AS, Krey K, Hustedt J, et al. Assessment of vaccine wastage wastage, HCWs have to decide when to open a vial rates, missed opportunities, and related knowledge, attitudes and with more DPC. In all 3 countries, high coverage practices during introduction of a second dose of measles-containing rates were considered more important than not vaccine into Cambodia’s national immunization program. Vaccine. 2018;36(30):4517–4524. CrossRef. Medline exceeding wastage targets. However, the desire to 7. Hutchins SS, Jansen HA, Robertson SE, Evans P, Kim-Farley RJ. control or reduce wastage rates, although second- Studies of missed opportunities for immunization in developing and ary, was considered important and did influence industrialized countries. Bull World Health Organ. 1993;71(5):549– HCW behavior. As shown, in the Zambia 5-dose 560. Medline districts, HCWs reported offering MCV at every 8. World Health Organization (WHO). Global vaccine action plan 2011-2020. WHO; 2013. Accessed September 25, 2020. https:// fixed session—a change from when they were us- www.who.int/immunization/global_vaccine_action_plan/GVAP_ ing the 10-dose vials. In Senegal, vaccines eligible doc_2011_2020/en/ for use for 28 days after opening were offered at 9. Wallace AS, Willis F, Nwaze E, et al. Vaccine wastage in Nigeria: an assessment of wastage rates and related vaccinator knowledge, every session, unlike vaccines that had to be dis- attitudes and practices. Vaccine. 2017;35(48 Pt B):6751–6758. carded within 6 hours of reconstitution. HCWs in CrossRef. Medline districts that received the 5-dose vials of MCV 10. Krudwig K, Knittel B, Karim A, et al. The effects of switching from 10 reported that they were more likely to open a vial to 5-dose vials of MR vaccine on vaccination coverage and wastage: a mixed-method study in Zambia. Vaccine. 2020;38(37):5905– for 1 child than they had been when they had 5913. CrossRef. Medline 10-dose vials, representing a possible solution to 11. World Health Organization (WHO). Increasing RI coverage by minimizing MOVs. This change in behavior was reducing missed opportunities for vaccination. Accessed December influenced by their reduced fear of wastage when 24, 2019. https://www.who.int/immunization/programmes_ systems/policies_strategies/MOV_Two-pager.pdf opening a vial with fewer DPC. 12. Smith PJ, Humiston SG, Parnell T, Vannice KS, Salmon DA. The as- sociation between intentional delay of vaccine administration and timely childhood vaccination coverage. Public Health Rep. Acknowledgments: We wish to acknowledge the Ministries of Health of 2010;125(4):534–541. CrossRef. Medline Senegal, Vietnam, and Zambia, particularly the Expanded Programme on Immunization staff and health care workers who supported and 13. Kristensen DD, Bartholomew K, Villadiego S, Lorenson K. What vac- participated in this research. We also thank the data collection teams cine product attributes do immunization program stakeholders val- who conducted interviews in each country. We thank the Bill & Melinda ue? Results from interviews in six low- and middle-income countries. Gates Foundation for funding this important work and to the DPCP Vaccine. 2016;34(50):6236–6242. CrossRef. Medline partner organizations and Technical Advisory Group for reviewing Maintaining Essential Health protocols and supporting the synthesis of results. 14. World Health Organization (WHO). Services: Operational Guidance for the COVID-19 Context.WHO; 2020. Accessed August 4, 2020. https://www.who.int/ Competing interests: None declared. publications/i/item/covid-19-operational-guidance-for- maintaining-essential-health-services-during-an-outbreak 15. Burgess C, Krudwig K, Masket D, Prosser W, Steinglass R. Dose Per REFERENCES Container Partnership: initial insights. Poster presented at: TechNet – 1. World Health Organization (WHO). Trends in use of multi-dose Conference 1017; October 16 20, 2017; Cascais, Portugal. vaccine vials in UNICEF procuring countries: WHO informal Accessed October 5, 2020. https://www.technet-21.org/images/ consultation to develop further guidance on vaccines for the tc2017/Posters/DPCP.pdf UNEP-convened Intergovernmental Negotiating Committee 16. JSI. Dose Per Container Partnership (DPCP) Snapshot. Ghana: Meeting 4 (INC4). April 4, 2012; Geneva, Switzerland. understanding changes in vaccine presentation. Accessed December Accessed December 24, 2019. https://www.who.int/ 24, 2019. https://www.jsi.com/JSIInternet/Inc/Common/_ immunization/sage/meetings/2012/april/consultation_ download_pub.cfm?id=19419&lid=3 INC4_MDVuse_JLiu_20120401.pdf 17. JSI. Dose Per Container Partnership (DPCP) Snapshot. Information 2. Heaton A, Krudwig K, Lorenson T, Burgess C, Cunningham A, gaps in decision-making on vaccine presentation in 3 Francophone Steinglass R. Doses per vaccine vial container: an understated and African countries. Accessed December 24, 2019. https://www.jsi. underestimated driver of performance that needs more evidence. com/JSIInternet/Inc/Common/_download_pub.cfm?id=19418& Vaccine. 2017;35(17):2272–2278. CrossRef. Medline lid=3
Global Health: Science and Practice 2020 | Volume 8 | Number 4 687 Health Care Worker Preferences for Vaccine Doses per Container www.ghspjournal.org
En français
Préférences des professionnels de la santé et perspectives sur les doses par flacon de deux vaccins lyophilisés au Sénégal, au Vietnam et en Zambie
Messages clés
Lorsqu'ils fournissent des services de vaccination, les agents de santé trouvent un équilibre entre la nécessité d'atteindre une couverture élevée et celle de limiter le gaspillage de vaccins. Les travailleurs de 3 pays ont déclaré que des récipients contenant moins de doses de vaccin contre la rougeole et le BCG leur permettraient de vacciner tous les enfants qui se présentent, tout en réduisant les inquiétudes quant au gaspillage de vaccin.
Résumé Introduction: Il existe peu d'informations sur les perceptions des agents de santé (AS) concernant l'utilisation des flacons de vaccins multidoses et leurs préférences en matière de doses par flacon (DPF). Nous présentons les résultats d'études qualitatives menées au Sénégal, au Vietnam et en Zambie pour étudier le comportement des agents de santé concernant l'ouverture des flacons et leurs perceptions et préférences quant au nombre de doses dans les flacons de BCG et de vaccin contre la rougeole. La Zambie et le Sénégal utilisent actuellement le vaccin contre la rougeole en flacons de 10 doses et le BCG en flacons de 20 doses; des flacons de 10 doses sont utilisés pour les deux vaccins au Vietnam. Les doses inutilisées des flacons de ces vaccins reconstitués doivent être jetées dans les 6 heures.
Méthodes: Des entretiens avec des informateurs clés ont été menés avec des agents de santé de première ligne au Sénégal, au Vietnam et en Zambie. Au Sénégal et au Vietnam, les entretiens ont été menées dans le cadre d'une recherche formative plus large; en Zambie, les entretiens ont été menées dans les districts de contrôle en utilisant uniquement des flacons de 10 doses de vaccin contre la rougeole et dans les districts d'intervention qui sont passés de flacons de 10 à 5 doses au cours de l'étude. Au cours de l'analyse, les sujets communs aux trois pays ont été synthétisés. Les sujets critiques pertinents pour les contextes nationaux ont également été examinés.
Résultats: Les travailleurs de la santé des trois pays ont préféré des récipients contenant moins de doses de BCG et de vaccin contre la rougeole afin de réduire le gaspillage et d'augmenter la probabilité de vacciner chaque enfant éligible. Les agents de santé au Sénégal et les agents de santé utilisant des flacons de 10 doses en Zambie ont déclaré avoir renvoyé des enfants non vaccinés parce qu'il n'y avait pas assez d'enfants présents pour justifier l'ouverture d'un nouveau flacon. Au Vietnam, où les séances ont généralement lieu tous les mois, et en Zambie, où les flacons de 5 doses ont été utilisés, presque tous les travailleurs de la santé ont déclaré avoir ouvert un flacon de Rougeole, même pour un seul enfant.
Discussion: Les agents de santé préfèrent les flacons contenant moins de dose par flacon. Leur souci d'équilibrer la couverture et le gaspillage influence leurs décisions de vacciner chaque enfant éligible; et leur point de vue est crucial pour garantir que toutes les populations cibles soient vaccinées en temps voulu.
Peer Reviewed
Received: March 14, 2020; Accepted: August 26, 2020; First published online: December 4, 2020
Cite this article as: Kanagat N, Krudwig K, Wilkins KA, et al. Health care worker preferences and perspectives on doses per container for 2 lyophilized vaccines in Senegal, Vietnam, and Zambia. Glob Health Sci Pract. 2020;8(4):680-688. https://doi.org/10.9745/GHSP-D-20-00112
© Kanagat et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00112
Global Health: Science and Practice 2020 | Volume 8 | Number 4 688 ORIGINAL ARTICLE
Remote Mentorship Using Video Conferencing as an Effective Tool to Strengthen Laboratory Quality Management in Clinical Laboratories: Lessons From Cambodia
Grant Donovan,a Siew Kim Ong,b Sophanna Song,b Nayah Ndefru,c Chhayheng Leang,b Sophat Sek,b Patricia Sadate-Ngatchou,a Lucy A. Perronec
Key Findings ABSTRACT Background: Providing professional development opportunities to n Utilization of a mixed-methods intervention design staff working in clinical laboratories undergoing quality improve- combining remote and in-person training, ment programs can be challenged by limited funding, particularly accompanied by close mentorship, contributed to in resource-limited countries such as Cambodia. Using innovative successful implementation of quality management approaches such as video conferencing can connect mentors with systems in participating laboratories. practitioners regardless of location. This study describes and n Laboratory participation time in video conference evaluates the methods, outputs, and outcomes of a quality im- training activities correlated with better quality provement program implemented in 12 public hospital laborato- management system management and improved ries in Cambodia between January 2018 and April 2019. The conformity to the ISO 15189 standard for medical program used mixed intervention methods including both in- laboratories. person and remote-access training and mentorship. Methods: Training outputs were quantified from the activity Key Implications reports of program trainers and mentors. Program outcomes were measured by pre- and postimplementation audits of labora- n When identifying budget and policy priorities for tory quality management system conformity to international stan- health, policy makers should consider the beneficial dards. Variations in improved outcomes were assessed in relation impact of a sustained human resources training and to the time spent by laboratory personnel in video conference mentorship program on laboratory quality training and mentoring activity. An additional cross-sectional improvement and service delivery efforts. comparison described the difference in final audit scores between n Policy makers should particularly consider the participating and nonparticipating laboratories. potential efficiency and effectiveness of remote- Results: Laboratories significantly improved their audit scores access telementoring and teleconferencing to over the project period, showing significant improvement in all sections of the ISO 15189 standard. Pre- and postaudit score dif- support online communities of practice for laboratory ferences and laboratory personnel participation time in remote professionals because improved connectivity and mentoring activities showed a strong monotonic relationship. knowledge sharing between professionals are Average input per laboratory was 6,02762,454 minutes of par- essential for quality service delivery in a laboratory ticipation in video conference activities with mentors. Audit scores system. of participating laboratories were significantly higher than those of laboratories with no quality improvement program. Conclusion: Laboratories improved significantly in ISO 15189 con- formity following structured laboratory quality management systems training supported by remote and on-site mentoring. The correlation of laboratory participation in video conference activities highlights the utility of remote video conferencing technology to strengthen laborato- ries in resource-limited settings and to build communities of practice to address quality improvement issues in health care. These findings a Department of Global Health, Schools of Public Health and Medicine, are particularly relevant in light of the COVID-19 pandemic. University of Washington, Seattle, WA, USA. b International Training and Education Center for Health, Cambodia, Phnom Penh, Cambodia. INTRODUCTION c International Training and Education Center for Health, Department of Global evelopment of strong laboratory quality manage- Health, Schools of Public Health and Medicine, University of Washington, D Seattle, WA, USA. ment systems (LQMS) is a key component of Correspondence to Lucy A. Perrone, MSPH, PhD ([email protected]). strengthening health systems for improved health
Global Health: Science and Practice 2020 | Volume 8 | Number 4 689 Remote Mentoring Using Video Conferencing for Laboratory Strengthening www.ghspjournal.org
outcomes and disease surveillance in resource- programs in Cambodia and globally has shown limited countries, and it requires standardization the impact that structured and mentored LQMS – and strategic planning.1 3 ISO 15189 accredita- programs can have in resource-limited health tion, which is the international standard for medi- care systems, holding promise for other such pro- cal laboratory quality, provides standardization of grams in Cambodia in the future.8,9 Delivery of LQMS requirements with a strong technical foun- However, delivery of professional training and professional dation for health, safety, and conformity.4,5 These close mentorship for laboratories undergoing QI training and close standards are stringent, however, and have re- programs remains challenged by geographical mentorship for quired a variety of approaches for laboratories and economic constraints. These challenges have laboratories with different resource availability and levels of prompted the use of modern video conferencing 5,6 undergoing QI development to achieve them. In Cambodia, technologies to expand access to consultation programs is a national effort to meet International Health from quality management professionals to distal challenging due to Regulations and improve health services has cul- facilities. Studies have shown that the use of these geographical and minated in an expansive national laboratory sys- technologies, collectively known as telementoring, economic tem to meet the diagnostic and surveillance needs is an efficient and cost-effective tool to provide sea- of the country at both the national and provincial soned or specialized expertise to health professionals constraints. 7 levels. The country has adopted national stan- in remote or resource-limited medical facilities and dards for medical laboratories, integrated a system has an impact on professional behavior and knowl- of external quality assessments through private edge, as well as health outcomes.10–12 One recent and public partnerships, and developed a national program in Southeast Europe used monthly men- laboratory information system to improve surveil- torship through telecommunication to improve lab- lance and care, but structured quality improve- oratory quality in 5 countries and demonstrated ment (QI) programs are limited to only a subset measurable progress within the 6 laboratories sup- of laboratories. Expansion of these quality man- ported.13 One of the most successful models of tele- agement training programs to meet international mentoring, Project ECHO (Extension for Community standards of quality was recommended in a series Healthcare Outcomes), has recently been expanded to of laboratory assessments carried out between laboratory strengthening, and the institute is now part- 7,8 2013 and 2016. One of these assessments mea- nering with at least 5 major professional laboratory sured 11 indicators of laboratory capacity, identify- institutions to provide laboratory training and mentor- ing a low average score of 36% in 22 laboratories, ship communities of practice globally.14 Research with indicators of LQMS averaging only 47% due demonstrating the effectiveness of this model of re- to a lack of quality management systems, trained mote training and mentoring for laboratory strength- quality assurance managers, or continuous im- 8 ening is limited, however, prompting a need for provement practices. quantitative research. The implementation of structured, stepwise programs to improve quality management sys- tems in national and provincial laboratories has Program Description been integral to improving laboratory quality and During the initial phase (Phase 1) of this project in capacity in Cambodia.8,9 In 2001, the U.S. Centers Cambodia in 2014, a group of 12 participating labora- for Disease Control and Prevention partnered with tories received training and mentored technical the Cambodia Ministry of Health (MOH) to imple- support to implement an LQMS according to the ment the Strengthening Laboratory Management newly published World Health Organization (WHO) Toward Accreditation program using the Stepwise Laboratory Quality Stepwise Implementation (LQSI) Laboratory Improvement Process Towards Accred- tool.15 Evaluation of that phase of the program indi- itation (SLIPTA) audit tool, supported by quality cated that consistent on-site mentoring in the local management training and mentorship by trained language with a stepwise action plan enhanced staff QI professionals (BMLS Cambodia, unpublished knowledge of LQMS implementation towards meet- presentation, 2018).9 International Training and ing the ISO 15189 standard, without interrupting Education Center for Health (I-TECH’s) QI program regular laboratory services.8 The successful results of began in 2014 and was intended to expand access to this training and mentoring approach led to an initia- LQMS training and implementation coaching tive by MOH to expand laboratory access to LQMS nationally, delivered through a complementary training and mentorship that prioritized implementa- package of training, mentorship, and technical tion of national standards of quality nationwide. assistance to MOH for national laboratory policy These priorities triggered the need for additional and guideline development. Success of these innovative approaches in 2017 for the second phase
Global Health: Science and Practice 2020 | Volume 8 | Number 4 690 Remote Mentoring Using Video Conferencing for Laboratory Strengthening www.ghspjournal.org
(Phase 2) of the QI program in Cambodia. At the practical proficiency in technical skills. Although time, Cambodia did not have a national standard of subject matter content focused primarily on quality quality, a standard tool for laboratory assessment, or assurance and operational management, laborato- a law to enforce laboratory quality in the public or ry managers and QAOs were also provided with or- private sectors. To address this issue, the Cambodia ganizational leadership skill-building activities. Laboratory Quality Management System Checklist Phase 2 of the QI program also included close for Accreditation (CamLQMS) was developed and mentorship of laboratory staff by trained laboratory adoptedbytheMOHBureauofMedicalLaboratory quality mentors. As described previously,8 mentors Services (BMLS) as a tool for national auditors to as- were technically experienced laboratory profes- sess laboratory quality during on-site performance sionals, trained in QI, proficient in both English audits. The CamLQMS tool was modeled on the and Khmer, and employed in the project full time. WHO-AFRO SLIPTA tool, which is aligned with the Mentors periodically visited laboratories to deliver ISO 15189 standard.7,8,16 Between July 2017 and individualized training and coaching to each of the April 2019, the Phase 2 QI program used the 12 laboratories and to address the gaps identified in CamLQMS tool to track laboratory progress toward the baseline audits; however, for Phase 2 of the meeting ISO 15189 accreditation standards, while project, the majority of mentorship and supportive using a combination of training, on-site and remote coaching reached laboratories remotely using tech- mentorship, and advocacy. nologies such as Zoom, WhatsApp, and Facebook Phase 2 of the QI program directed a set of Messenger. technical approaches and interventions at both Modeled on the ECHO project11 but adapted the national and regional levels to strengthen the independently by the project team for laboratory interconnectivity and collaboration between labo- mentoring, Zoom video conferencing technology ratories in the tiered laboratory system for im- was used to connect with the cohort of laborato- proved public health and clinical functions. At ries weekly (though often 2 or 3 times per week, the national level, the program worked to address on-demand) in a community of practice environ- gaps in the legal and regulatory framework and ment. Weekly training sessions followed a struc- documentation concerning the establishment of tured training schedule designed over a period of national quality and safety standards. At the facil- 16 months. This schedule was organized into ity level, primary activities encompassed the de- weekly topics and followed a format of teacher sign and delivery of job-specific, competency- presentation, laboratory presentation, question based education and training to quality assurance and answer sessions, and action items for the fol- officers (QAOs) and laboratory managers selected lowing week. Time for peer networking was also from the Phase 1 cohort of the 12 national and re- provided, and conversations on Zoom often car- gional clinical laboratories. Facility-based staff ried over into other platforms such as Messenger were trained in operational quality management and WhatsApp in the days following each session. and provided regular mentoring support through Remote training and mentoring sessions were Remote training on-site technical assistance and telementoring con- designed to reach more geographically dispersed and mentoring sultations. In January of 2018, I-TECH Cambodia laboratory professionals without the limitations sessions were partnered with MOH-BMLS to conduct a baseline of resource-intensive travel, thus improving the designed to CamLQMS assessment of participating laborato- cost-effectiveness of activities. Through the use of reach more ries, followed by a national dissemination meeting Zoom Pro accounts, project staff were able to geographically to discuss findings and develop recommendations. schedule meetings for up to 100 participants for dispersed These recommendations included a series of 11 train- up to 2 hours, providing visual presentations and laboratory ing workshops to improve the LQMS operational video demonstrations, with the added benefit professionals practicesofQAOsandlaboratorymanagersandto that each session was recorded and available for 17 without the eliminate deficiencies identified during the audits. later review by participants. These trainings were limitations of LQMS trainings were designed using adult learning designed primarily for QAOs and laboratory man- resource- principles and accepted pedagogy to improve learner agers; however, all laboratory staff were welcome intensive travel. comprehension and competency through a combina- and many additional staff also attended the weekly tion of theoretical and practical learning methods ori- sessions, with each session recording up to 28 parti- ented toward health professionals.18 These trainings cipants from the combined group of laboratories. consisted of large-group formal instruction inter- Importantly, the program enjoyed strong engage- spersed with several focused and interactive sessions ment from MOH-BMLS, which was involved in all over 2–5 days, as well as smaller laboratory-based project planning, implementation, and monitoring training workshops held regionally to emphasize including all formal training sessions, workshops,
Global Health: Science and Practice 2020 | Volume 8 | Number 4 691 Remote Mentoring Using Video Conferencing for Laboratory Strengthening www.ghspjournal.org
and audits. This involvement was essential, ensur- hypothesis testing were performed using STATA ing the continuity and sustainability necessary for 14 statistical software. the program to be replicated within other laborato- ries once current funding had ended. Following Quantifying Mentoring and Training Activity this 16-month period of training and mentoring, a Outputs second CamLQMS audit was conducted and these For the description and enumeration of activity data, along with an assessment of activity outputs, outputs, this study used monitoring and evaluation are the foundation of this study and program records collected by the project team. Outputs of in- evaluation. terest, as listed in the logic model in Table 1, includ- ed (1) the number of trainings attended by personnel of the intended audience per laboratory, (2) the num- METHODS ber of days of on-site mentorship provided to each Research Methodology laboratory, and (3) the amount of video conference This study evaluated the outputs and outcomes of training and mentoring time attributed to individual Phase 2 activities in the 12 participating laboratories laboratory personnel during the evaluation period. during the evaluation period of January 2018 to Data sources included (1) attendance records for the April 2019 between program audits. Our evaluation 11 completed trainings, (2) project team member used an uncontrolled longitudinal study to assess reports, (3) Zoom meeting records extracted from project team member’s Zoom accounts (reviewed to changes in LQMS compliance to international stan- match laboratory and position details of meeting dards between baseline and endpoint measurements. participants to their Zoom user names), and (4) sup- A cross-sectional analysis was then used to compare plementary records of remote mentoring sessions postimplementation LQMS performance and confor- conducted via Messenger and WhatsApp from men- mity of intervention laboratories to a select group tors. Datasets from each of these data sources were of nonintervention laboratories. Data management organized into separate spreadsheets for review and basic descriptive statistics for all evaluation meth- and descriptive analysis. Attendance records for ods were performed using Microsoft Excel for Office all 11 training events were organized by meeting 365. All complex calculations of statistics and date, and participant data were analyzed for each
TABLE 1. Calculations of the 3 Primary Activity Outputs and the Cambodia Laboratory Quality Management System Audit Score Achieved Within the Evaluation Period
No. Completed Video Trainings of Intended Mentor Time on Conference Participants (Total No. Site per Laboratory, Participation Time, Audit Score Participants) Days Minutes Difference, %
Lab A 19 (25) 9 3,766 9 Lab B 21 (26) 10 5,855 13 Lab C 24 (25) 10 2,742 15 Lab D 23 (25) 10 6,320 32 Lab E 25 (29) 13 9,302 37 Lab F 24 (37) 13 9,664 31 Lab G 23 (24) 9 6,800 26 Lab H 21 (27) 13 5,290 28 Lab I 28 (36) 13 7,210 17 Lab J 22 (24) 8 4,434 28 Lab K 22 (26) 10 8,675 15 Lab L 22 (26) 12 2,263 7 Group mean 6 SD 2362 (2864) 1162 6,02762,454 21610
Global Health: Science and Practice 2020 | Volume 8 | Number 4 692 Remote Mentoring Using Video Conferencing for Laboratory Strengthening www.ghspjournal.org
training event, which generally included laboratory for CamLQMS outcome evaluation. Plots were visu- managers and QAOs but at times included directors ally inspected for a linear or monotonic relationship or administrators, biosafety officers, equipment offi- between the 2 variables and then tested for the cers, or stock officers. Counts were calculated by lab- strength of that relationship by using Spearman’s oratory and event; the sum and average were rank order correlation coefficient. Spearman’srank calculated for the group. Records of on-site mentor- was selected as a nonparametric test due to the small ing were similarly analyzed using mentoring reports sample size of the intervention group (n=12), which as primary data sources. Both the number of partici- was expected to increase the test sensitivity to mod- pants per training and the number of in-person visits erate outliers in a Pearson’s test for correlation. were planned and expected to be approximately Because formal trainings and site visits were restrict- equal between laboratories. Laboratories were allot- ed from random variation, our study was unable to ted an equal number of days of on-site mentorship, provide similar correlation assessments between although some training content varied based on lab- these activity outputs and direct program outcomes. oratory need. Meanwhile, scheduled video confer- ence training and mentoring were more client driven and scheduled meetings were provided on Quantifying External Audit Outcomes demand. We used the CamLQMS checklist for accreditation Because program mentors used a Zoom Pro ac- as the primary outcome measure to determine the ’ count for most remote mentoring and training, performance of each laboratory s quality manage- meeting and participant data were automatically ment system before and after the training and men- recorded through the report feature of the Zoom toring program interventions. The CamLQMS software and available for extraction and analysis. checklist was divided into 12 sections of laboratory These reports were then compiled into a dataset quality with a total of 117 questions regarding including a join time, leave time, and a duration whether a particular standard was met, and each of participation (based entirely on duration of at- question was assigned a numerical value that con- tendance) for each user identification (ID) during tributed to the audit score within each section and each meeting as a representative sample of remote in the whole (Table 2). Mentored laboratories com- mentoring activities. Within this dataset, atten- pleted baseline CamLQMS audits in January of dance logs were tracked using participant IDs and 2018 and outcome assessments in April of 2019. crossmatched with participant work site/laborato- Additionally, a control group of representative ry and job title, using mentor reports as supple- public laboratories that did not receive LQMS train- mental records to match and attribute 98% of ing or mentoring (nonmentored/nonintervention) participation time to participating laboratory per- was selected for a cross-sectional comparison. sonnel, to project staff or mentors, or to other par- Control laboratories were selected by MOH-BMLS ticipating stakeholders. Due to the use of multiple as the nearest in comparable capacity in terms of devices by some participants during meetings, a the complementary package of activities and ser- dynamic Gantt chart was employed to visually vices, although these facilities differed significantly and systematically identify duplicate, overlapping from mentored laboratories in terms of baseline usernames. The duplicates were then recategor- level of training and number of staff. Laboratory ized as “device only” regarding position and labo- audits of mentored and nonmentored facilities ratory to exclude them from analysis. All user were conducted by 3 teams of auditors who were logins that indicated multiple participants associated trained by the project team to assess facility confor- with a user ID were duplicated to reflect attendance mity and nonconformity to the CamLQMS check- of those participants. Minutes of participation time list. Each 4-person auditing team was led by a lead were grouped by laboratory and summarized for to- auditor and included at least 1 MOH-BMLS repre- tal participation time of unique attendees from each sentative. During the audit process, the lead auditor laboratory within the sample over the evaluation asked each of the 117 questions of the laboratory in period. Records were then reviewed for additional series, and the team reviewed responses at the end remote training or mentoring events held outside of each audit to determine whether the require- of tracked video conferences to determine how rep- ments of each question were met partially or in full, resentative the sample was out of the total estimate indicated by “yes,”“no,” or “partial.” Questions that of events. Total video conference participation time were not applicable to a laboratory due to individual per laboratory was then plotted in a scatter diagram requirements or organizational complexities of the against the percent differences in pre- and postinter- facility served were answered with “NA.” After com- vention audit scores, described under the methods pletion of all audits, the 3 team leads reviewed all
Global Health: Science and Practice 2020 | Volume 8 | Number 4 693 Remote Mentoring Using Video Conferencing for Laboratory Strengthening www.ghspjournal.org
TABLE 2. Cambodia Laboratory Quality Management System Checklist for Accreditation Score Sheet
Audit Score Sheet Section Total Points
Section 1: Documents and Records 28 Section 2: Management Reviews 14 Section 3: Organization and Personnel 22 Section 4: Client Management and Customer Service 10 Section 5: Equipment 35 Section 6: Evaluation and Audits 15 Section 7: Purchasing and Inventory 24 Section 8: Process Control 32 Section 9: Information Management 21 Section 10: Identification of Nonconformities, Corrective, and Preventive Actions 19 Section 11: Occurrence/Incident Management and Process Improvement 12 Section 12: Facilities and Biosafety 43 Total score 275 Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 (0–150 pts) (151–177 pts) (178–205 pts) (206–232 pts) (233–260 pts) (261–275 pts) <55% 55%–64% 65%–74% 75%–84% 85%–94% ≥95%
audit data together to identify any recording error, audit scores of mentored LQMS laboratories and bias or inconsistencies in scoring methodologies be- nonmentored, non-LQMS comparison laboratories tween teams, and moderated audit point allocations was performed using the Wilcoxon rank-sum test accordingly. Audit scores were calculated as a per- for 2 independent samples. Comparisons were centage of the total value of checklist items for each made for overall audit scores and scores for individ- section and overall for each laboratory. ual audit sections, and all sections with statistically A Wilcoxon signed-rank test for nonparametric significant differences between groups were again comparison for paired samples was performed to documented. determine the strength of the difference between 2018 and 2019 audit scores of mentored labora- tories for each section and summary overall. RESULTS Nonparametric statistics were selected to main- Table 1 shows the outputs for each measured pro- tain consistent assumptions of normality be- gram activity and the corresponding increase in tween the groups of small sample size. Mean CamLQMS audit score as the direct program outcome audit scores and standard deviations were calcu- and reveals an output of 274 (mean=2362) target per- lated in each of the 12 sections for visual compar- sonnel trainings, 72,321 (mean=6,02762,454) min- isons between laboratory groups, and all sections utes of video conference training, and 130 (mean= with statistically significant differences in scores 1162) visits to laboratories, resulting in an average between years were documented with the level positive percent difference of 21610% between of significance. The percent change in overall au- the 2018 and 2019 overall audit scores. Video con- ditscoresineachsectionwascalculatedtopre- ference participation time was calculated from a sent the magnitude of change visually, and these sample size of 153 Zoom meetings with traceable percent differences were used as the primary usage reports out of a total of 261 meetings identi- variables for a Spearman’s rank correlation as- fied from supplemental mentor reports and program sessment of the relationship strength between activity calendar entries. In terms of staff inputs, formal audit score improvement and laboratory person- training and video conference activities included 2 pri- nel participation time in Zoom activities. An as- mary mentors, 2 mentor trainees, the country project sessment of the statistical difference between coordinator, and 3 laboratory systems technical and
Global Health: Science and Practice 2020 | Volume 8 | Number 4 694 Remote Mentoring Using Video Conferencing for Laboratory Strengthening www.ghspjournal.org
senior technical specialists. Additionally, several MOH being the exception, which was significantly different officials from BMLS and the National Institute of at the P<.01 level. In terms of percent difference in Public Health participated in formal trainings and in mean section scores between groups, “client manage- numerous video conference activities. ment and customer service” as well as “occurrence A Wilcoxon signed-rank test indicated that management and process improvement” demonstrat- overall audit scores for mentored laboratories in ed the largest differences of 57% and 52% between 2019 were significantly higher (median score=57%) groups. “Information management” again showed than overall audit scores for the same laboratories in the smallest percent difference (27%) between groups. 2018 (median score=40%, z=3.06, P=.002). In a In an assessment of the relationship between mean comparison of scores for individual audit sections be- audit score differences from 2018 to 2019 and the tween years, the Wilcoxon signed-rank test indicated amount of participation time by individual laboratories that mean 2019 scores for 11 out of 12 audit sections in Zoom video conference training, a Spearman’srank improved significantly (P<.01), with “information correlation showed a strong monotonic relationship management” being the exception, which had been between the 2 variables (rs=0.59, P=0.04) with signifi- maintained but not significantly improved from an cant certainty. already high performance level at baseline (Figure 1). A cross-sectional comparison of the 2019 audit DISCUSSION performance of mentored laboratories with the sam- The quality audit scores of laboratories participat- The quality audit ple of nonmentored laboratories showed a large con- ing in this program improved significantly follow- scores of trast in scores between groups (Figure 1)andby ing implementation of the training and mentoring laboratories section (Figure 2). activities, demonstrating that the QI program improved A 2-sample Wilcoxon ranked-sum (Mann- achieved its intended effect. Laboratory perfor- significantly, Whitney) test indicated that overall audit scores mance from mentored sites was significantly demonstrating for mentored laboratories in 2019 (median=57%) higher in all measured categories of quality man- that the QI were significantly higher than overall audit scores agement than in laboratories with no training or program achieved for nonmentored laboratories (median=23%) in mentoring support, and this study clearly showed its intended effect. the same year (z=3.96, P=.0001). Mann-Whitney a positive correlation between laboratory QI and tests comparing individual audit sections similarly participant contact time with trainers and mentors revealed significant differences in 11 of the 12 sections via remote mentoring. The strong correlation be- (P<.001) between intervention and nonintervention tween remote mentoring through video confer- laboratories, with “information management” again ence calling and improved audit scores indicates
FIGURE 1. Overall 2018–2019 Cambodia Laboratory Quality Management System Checklist for Accreditation Audit Scores for Mentored Public Hospital Laboratoriesa
a The dashed line represents the average audit score for nonmentored laboratories (2018 audit data not available).
Global Health: Science and Practice 2020 | Volume 8 | Number 4 695 Remote Mentoring Using Video Conferencing for Laboratory Strengthening www.ghspjournal.org
FIGURE 2. Mean Audit Scores of Mentored Public Hospital Laboratories Compared With Nonmentored Laboratoriesa
Abbreviations: EQA, external quality audit; IQA, internal quality audit. a Error bars represent the absolute standard deviations from the mean score of each section.
remote mentoring is an effective QI tool and also education and consultation shows promise as a tool presents a cost-effective alternative to on-site to create communities of practice between labora- mentoring, which requires frequent travel to re- torians and other health practitioners in the future, mote, hard-to-reach laboratories. A 1-week site apracticethatwillproveallthemorevaluabledur- visit from Phnom Penh to Kratie, for example, ing the COVID-19 pandemic, given that online plat- costs approximately US$398 for travel, lodging, forms have become the primary means of accessing and per diem for local mentors, but a Zoom professional training and consultation for many Pro account can cost as low as US$45 annually. medical professionals.20–22 Although further studies are needed to evaluate Notably, although attendance in formal, in- the cost-effectiveness of remote versus on-site person training and the number of on-site visits telementoring and other inputs such as on- were semi-controlled for variation and therefore site training or mentorship, our results suggest a could not be tested for a relationship to LQMS im- notable cost-benefit of telementoring for LQMS im- provement, the relationship is expected. In partic- provement compared with on-site training. Remote ular, the content of the program’s formal training mentoring has the further benefit of providing on- curriculum is reflected in several individual audit demand professional support and networking. A sections that demonstrated major improvements. qualitative study of the remote mentoring program Topics such as “documents and records,”“man- in Cambodia identified a number of recurring agement review,”“occurrence management,” themes of benefits identified by participants, includ- and “process control” received particular focus in ing that additional remote training reinforced con- formal trainings, and thus coincided with superior cepts and provided peer learning opportunities and program outcomes. Of note, this program chose to on-demand guidance; however, laboratories strong- deprioritize the topics of “information manage- ly preferred a more structured training format in the ment” and “facilities and safety” due to topical local language if online training was used.19 The overlap with other ongoing national training pro- use of video conferencing technologies for medical grams. During site visits, mentors worked closely
Global Health: Science and Practice 2020 | Volume 8 | Number 4 696 Remote Mentoring Using Video Conferencing for Laboratory Strengthening www.ghspjournal.org
with laboratories on specific technical needs such need for frequent activities that are high-cost ele- as improved use of quality indicators (“occurrence ments such as on-site workshops and coaching. In management”), quality control testing (“process addition, they have the added benefit of reaching a control”), “equipment” verification, and “correc- larger audience than would otherwise be possible tive action.” In later stages of the implementation due to cost. These findings contribute to the limit- period, mentors coached personnel on internal ed body of qualitative research on remote mentor- auditing in preparation for the second round of ing as a practice, which describes success in QI CamLQMS audits. This close mentoring approach outcomes due to improved accountability, collab- was predicted to have contributed to the program orative problem solving, and increased awareness of the importance of laboratory quality.13 This outcomes; however, further research is needed to evaluation strongly suggests that tiered laboratory isolate the impact of our program’s site visit and systems in resource-limited countries such as formal training models from that of remote Cambodia would benefit from national expansion mentoring. of LQMS training and mentorship programs of a Notable limitations and recommendations for similar design, at scale, utilizing a structured cur- future research are as follows. First, the CamLQMS riculum and particularly remote training and checklist for accreditation is designed to assess gaps mentoring methodologies. in conformity within individual laboratories to drive improvement in each specific section of LQMS. Because individual audit sections have different CONCLUSION maximum possible scores, and because some ques- This program used a combination of training, mentoring, and advocacy to achieve rapid and sig- tions are inapplicable to certain laboratories, overall nificant outcomes in quality management system audit comparisons between laboratories and sections development. Participating laboratories performed should be interpreted with caution. Nonetheless, significantly better in audits of performance and thesecomparisonsserveasausefulestimateofpro- conformity than nonintervention laboratories, sug- gram activity efficacy. Second, because our cross- gesting that an expansion of this methodology in sectional comparison of final CamLQMS results in Cambodia may benefit currently nonmentored lab- program laboratories with nonprogram laboratories oratories significantly toward meeting national does not compare rates of change between groups, standards of quality. Although our findings indicate further prospective studies are needed to compare that significant progress has been made in meeting the rate of improvement directly through a pre-post international standards of quality in laboratory design with a larger sample of facilities. The compari- practice, laboratories in the public sector and labo- son group is also limited in its usefulness because of ratories in Cambodia should continue to imple- critical differences in staff size and training input at ment stepwise QI programs with an emphasis on baseline. Control laboratories had 4–8 employees per improved connectivity of laboratories to profes- facility compared with approximately 9–33 employ- sional training and mentorship for effective QI. ees in the participant group and did not benefit from Phase 1 inputs, which resulted in better audit scores Acknowledgments: We acknowledge the efforts of laboratory staff from all 12 participating laboratories in this project and give special thanks to at baseline for mentored laboratories and may have Dr. Michael Noble for adapting the University of British Columbia’s provided a learning advantage over nonintervention quality management course for in-person delivery in Cambodia. We acknowledge and thank Cat Koehn, Joanna Mendelsohn, and Malin laboratories. Finally, because monitoring of Zoom Chao for their project management support. We thank our colleagues session reports was incorporated late into the pro- from the Cambodia Ministry of Health, Bureau of Medical Laboratory gram evaluation, 2% of participation time in video Services, National Institute of Public Health, the U.S. Centers for Disease Control and Prevention in Cambodia, and World Health Organization conferencing could not be associated with or disas- Cambodia for their support of this project. We thank collaborators from sociated from individual laboratories, leaving the the Diagnostic Microbiology Development Program, Fondation Merieux, and Westline Education Group in Phnom Penh. This work was funded by potential of misclassification bias against certain the U.S. Department of Defense, Defense Threat Reduction Agency. laboratories prone to using unidentified devices. Built-in user report tools such as within Zoom serve Funding: This work was funded by the U.S. Department of Defense, as an easy-to-use mechanism for monitoring and Defense Threat Reduction Agency and the views of the authors do not represent those of the funder. evaluation of remote training and mentoring pro- grams; however, some effort is needed to ensure Competing interests: None declared. data quality as it is collected. Conventional in-training programs are re- source intensive; however, as we have described REFERENCES 1. Olmsted SS, Moore M, Meili RC, et al. Strengthening laboratory sys- here, programs that use remote mentoring and tems in resource-limited settings. Am J Clin Pathol. 2010;134(3): training tools such as Zoom can circumvent the 374–380. CrossRef. Medline
Global Health: Science and Practice 2020 | Volume 8 | Number 4 697 Remote Mentoring Using Video Conferencing for Laboratory Strengthening www.ghspjournal.org
2. Carter JY, Lema OE, Wangai MW, Munafu CG, Rees PH, pain for people with advanced dementia nearing the end of life. BMC Nyamongo JA. Laboratory testing improves diagnosis and treatment Health Serv Res. 2018;18(1):228. CrossRef. Medline Afr J Lab Med outcomes in primary health care facilities. .2012; 13. Polansky L, Chester S, Warren M, et al. Can mentorship improve 1(1):8. CrossRef. Medline laboratory quality? A case study from influenza diagnostic laborato- 3. Najjar-Pellet J, Machuron J-L, Bougoudogo F, et al. Clinical labora- ries in Southeast Europe. BMC Health Serv Res. 2019;19(1):49. tory networks contribute to strengthening disease surveillance: the CrossRef. Medline Emerg Health Threats J RESAOLAB Project in West Africa. . 2013;6. 14. University of New Mexico ECHO Institute. Lab strengthening ECHOs CrossRef. Medline globally. Accessed November 12, 2020. https://echo.unm.edu/ 4. Schneider F, Maurer C, Friedberg RC. International Organization for doc/2019/01/lab%20ECHO%20global%202-pager.pdf Ann Lab Med – Standardization (ISO) 15189. . 2017;37(5):365 370. 15. World Health Organization (WHO). Laboratory quality stepwise CrossRef. Medline implementation tool. WHO; 2014. Accessed November 12, 2020. 5. World Health Organization (WHO) Regional Office of South East https://extranet.who.int/lqsi/ Laboratory Quality Standards and Their Implementation Asia. . 16. Nguyen TT, McKinney B, Pierson A, et al. SLIPTA e-Tool improves WHO; 2011. Accessed November 12, 2020. https://www.who. laboratory audit process in Vietnam and Cambodia. Afr J Lab Med. int/medical_devices/publications/lab_quality_standards/en/ 2014;3(2):219. CrossRef. Medline 6. Gachuki T, Sewe R, Mwangi J, et al. Attaining ISO 15189 accredi- 17. Clinical and Laboratory Standards Institute (NCCLS). Application of tation through SLMTA: a journey by Kenya’s National HIV Reference a Quality Management System Model for Laboratory Services, Afr J Lab Med Laboratory. . 2014;3(2):216. CrossRef. Medline Approved Guideline GP26-A3. 3rd ed. NCCLS; 2004. Accessed 7. World Health Organization (WHO). Joint External Evaluation of IHR November 30, 2020. http://demo.nextlab.ir/getattachment/ Core Capacities of the Kingdom of Cambodia: Mission Report, 26 cacaf98e-9473-4727-b9ca-ffdeba3bebc6/CLSI-GP26-A3.aspx August–2 September 2016 . WHO; 2016. Accessed November 12, 18. Marinucci F, Medina-Moreno S, Wattleworth M, Damiano A, 2020. https://apps.who.int/iris/handle/10665/254705 Redfield R. New approach to in-service training of laboratory pro- 8. Perrone LA, Voeurng V, Sek S, et al. Implementation research: a fessionals in sub-Saharan Africa. Int J Biomed Lab Sci. 2011;1:1–6. mentoring programme to improve laboratory quality in Cambodia. Accessed November 12, 2020. https://www.ijbls.org/images/ Bull World Health Organ. 2016;94(10):743–751. CrossRef. stories/201195103723.pdf Medline 19. Ong SK, Donovan G, Ndefru N, et al. Strengthening the clinical lab- 9. Yao K, Maruta T, Luman ET, Nkengasong JN. The SLMTA pro- oratory workforce in Cambodia; a case study of a mixed-method in- gramme: transforming the laboratory landscape in developing service training program to improve laboratory quality management countries. Afr J Lab Med. 2014;3(3). CrossRef. Medline system oversight. Hum Resour Health. 2020;18(1):84. CrossRef 10. Arora S, Thornton K, Murata G, et al. Outcomes of treatment for 20. Kaup S, Kaup S, Jain R, Shivalli S, Pandey S. Sustaining academics hepatitis C virus infection by primary care providers. N Engl J Med. during COVID-19 pandemic: the role of online teaching-learning. 2011;364(23):2199–2207. CrossRef. Medline Indian J Ophthalmol. 2020;68(6):1220–1221. CrossRef. Medline 11. Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The im- 21. Steeves-Reece AL, Elder NC, Graham TA, et al. Rapid deployment of pact of project ECHO on participant and patient outcomes: a sys- a statewide COVID-19 ECHO program for frontline clinicians: early tematic review. Acad Med. 2016;91(10):1439–1461. CrossRef. results and lessons learned. J Rural Health. 2020; CrossRef. Medline Medline 22. Price DW, Campbell CM. Rapid retooling, acquiring new skills, and 12. De Witt Jansen B, Brazil K, Passmore P, et al. Evaluation of the impact competencies in the pandemic era: implications and expectations for of telementoring using ECHO© technology on healthcare profes- physician continuing professional development. J Contin Educ Health sionals’ knowledge and self-efficacy in assessing and managing Prof. 2020;40(2):74–75. CrossRef. Medline
Peer Reviewed
Received: March 21, 2020; Accepted: November 3, 2020; First published online: December 11, 2020
Cite this article as: Donovan G, Ong SK, Song S, et al. Remote mentorship using video conferencing as an effective tool to strengthen laboratory quality management in clinical laboratories: Lessons from Cambodia. Glob Health Sci Pract. 2020;8(4):689-698. https://doi.org/10.9745/GHSP-D-20- 00128
© Donovan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00128
Global Health: Science and Practice 2020 | Volume 8 | Number 4 698 ORIGINAL ARTICLE
Using Community Health Workers and a Smartphone Application to Improve Diabetes Control in Rural Guatemala
Sean Duffy,a Derek Norton,b Mark Kelly,c Alejandro Chavez,d Rafael Tun,e Mariana Niño de Guzmán Ramírez,a Guanhua Chen,b Paul Wise,d Jim Svensonf
Key Findings Resumen en español al final del artículo.
n A smartphone application providing algorithmic clinical decision support enabled community ABSTRACT health workers to improve diabetes control for a Background: The global prevalence of diabetes has nearly dou- group of patients in rural Guatemala. bled since 1980. Seventy-five percent of patients with diabetes live in low- and middle-income countries, such as Guatemala, Key Implications where health care systems are often poorly equipped for chronic disease management. Community health workers (CHWs) and n Program managers should consider equipping mobile health technology have increasingly been applied to the community health workers with clinical decision diabetes epidemic in these settings, although mostly in supportive support applications to enable task sharing for rather than primary roles in diabetes management. We sought to chronic disease management. improve diabetes care in rural Guatemala through the develop- ment of a CHW-led diabetes program and a smartphone appli- n Researchers should examine the efficacy of this cation to provide CHWs with clinical decision support. approach for chronic diseases other than Methods: We worked with our local partners to develop a pro- diabetes and compared to traditional models of gram model and the smartphone application (using the care. CommCare platform) and to train CHWs. We recruited patients with type 2 diabetes living in rural communities. Program evalua- tion used a single-group, pre-post design. Primary outcomes were hemoglobin A1c and the percentage of patients meeting A1c goals compared with baseline. We also followed a variety of process metrics, including application reliability. Results: Eighty-nine patients enrolled during the study period. The hemoglobin A1c percentage decreased significantly at 3 months (-1.0; 95% CI=-1.7, -0.6), 6 months (-1.5; 95% CI=-2.2, -0.8), 9 months (-1.3; 95% CI=-2.0, -0.6), and 12 months (-1.0; 95% CI=-1.7, -0.4). The percentage of patients with A1c 8% increased significantly at 3 months (23.6% to 44.4%, P=.007), 6 months (22.0% to 44.0%, P=.015), and 9 months (23.9% to 45.7%, P=.03). CHWs and supervising physicians agreed with application medication recommendations >90% of the time. Conclusion: Our results suggest that CHWs can safely and effec- tively manage diabetes with the assistance of a smartphone ap- plication and remote physician supervision. This model should be evaluated versus other standards of care and could be a University of Wisconsin School of Medicine and Public Health, Department of Family Medicine and Community Health, Madison, WI, USA. adapted to other low-resource settings and chronic diseases. b University of Wisconsin School of Medicine and Public Health, Department of Biostatistics and Medical Informatics, Madison, WI, USA. c University of California-Los Angeles David Geffen School of Medicine, Internal INTRODUCTION Medicine Residency Program, Los Angeles, CA, USA. he global prevalence of diabetes has increased dra- d Stanford University School of Medicine, Stanford, CA, USA. T e matically over the past several decades, nearly dou- Hospital Obras Sociales Monseñor Gregorio Schaffer, San Lucas Tolimán, 1 Guatemala. bling since 1980, from 4.7% to 8.5% of adults. In 2015, f University of Wisconsin School of Medicine and Public Health, Department of an estimated 5 million deaths and US$673 billion in health Emergency Medicine, Madison, WI, USA. expenditures were attributable to diabetes, accounting for Correspondence to Sean Duffy ([email protected]).
Global Health: Science and Practice 2020 | Volume 8 | Number 4 699 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org
12.8% of global all-cause mortality and 12% of glob- and 29.4% in extreme poverty ( Global Health: Science and Practice 2020 | Volume 8 | Number 4 700 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org first developed an overall model for the program, Development of Clinical Protocols and as outlined in Figure 1. In this model, health pro- Procedures moters meet with patients on a monthly basis. The We developed protocols for assessing glycemic The mobile promoters use a clinical decision support (CDS) control, titration of oral hypoglycemics, identifica- application application to guide each visit. Using data entered tion and management of diabetes complications, included protocols by the promoters, including point-of-care glyce- and patient counseling. We based these protocols for assessing mic testing, the application provides recommen- on guidelines published by the American Diabetes glycemic control, 31 32 dations on the titration of oral hypoglycemics, Association (ADA), WHO, the International titration of oral 30 management of diabetes complications, self-care Diabetes Federation (IDF), and Guatemalan hypoglycemics, 33 counseling, and referral to the supervising physi- organizations. SLM medical director Dr. Rafael identification and cian. After each visit, patient data are uploaded Tun was integral to this process and provided fi- management of to a secure server and reviewed by one of the su- nalapprovalforallprotocols. diabetes pervising physicians, who then communicates complications, any changes in the treatment plan or additional Assessment of Glycemic Control and patient recommendations to the promoters. In order to re- We used point-of-care hemoglobin A1c (A1c) results counseling. move cost as a barrier to care, the diabetes pro- as our primary measure of glycemic control based on gram provides services and medications free of recommendations from ADA and IDF.30,31 Studies charge. have demonstrated the potential of this technology We recognized that the services provided by to improve diabetes care in LMICs.33,34 We utilized this program, while intended to be an improve- A1CNowþ (PTS Diagnostics) point-of-care capillary ment on the status quo, were by no means blood analyzers. The A1CNowþ test produces results comprehensive. Guidelines for limited resource in 5 minutes and can be performed with minimal settings also deem insulin, antihypertensives, and training, allowing for assessment of glycemic control other therapies as essential elements of diabetes by the CHWs during diabetes visits. Guidelines rec- care.30 However, resources were not available to ommend checking A1c every 2–6 months depending implement a comprehensive chronic disease sys- on diabetes control and changes in medication.30,31 tem. Rather, glycemic control through oral medi- We checked A1c every 3 months for all patients dur- cations and lifestyle counseling was deemed the ing the study period to allow for more uniform eval- immediate focus, with additional components to uation of program efficacy. follow with enhanced resources and a successful We also employed monthly BG testing to ti- mobile platform proof-of-concept. trate medications between A1c measurements, FIGURE 1. Overall Model for Sustainable Rural Diabetes Care Program Led by Community Health Workers, Guatemala Global Health: Science and Practice 2020 | Volume 8 | Number 4 701 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org assess for hypo- and hyperglycemia, provide a sec- Identification and Management of Diabetes ondary marker of glycemic control when A1c test- Complications ing was not available or malfunctioned, and as a We developed protocols for common and impor- confirmation of A1c values when checked concur- tant diabetes complications and comorbidities, in- rently. We used the Contour (Bayer) capillary cluding hyper- and hypoglycemia, hypertension, blood testing system for BG testing. We estab- coronary artery disease, chronic kidney disease, lished glycemic targets of A1c 7%, fasting BG diabetic foot ulcers, and diabetic eye disease 80–130 mg/dL, and postprandial BG <180 mg/dL (Table 1). These protocols include recommenda- for most patients, with less stringent targets for tions for referral to the supervising physician and, patients ≥65 years old or those with multiple in some cases (e.g., hypoglycemia), initial treat- comorbidities, or per physician discretion. These ment delivered by CHWs. targets are broadly consistent with ADA and IDF guidelines.30,31 Application Development and Description In addition to We integrated the diabetes protocols into a smart- providing Medication Titration phone application to provide algorithmic decision algorithmic We selected metformin and glyburide (glibencla- support to the CHWs. The application also served decision support mide) as the oral medications in our medication ti- as a data collection tool and medical record. We to the CHWs, the tration protocol because of their long track records designed the application in Spanish for smart- application also in diabetes care, availability in Guatemala, and af- phones and tablets running the Android operating served as a data fordability. Metformin is the first-line medication system (Google LLC), the most common mobile 36 37 collection tool and for all patients, consistent with established guide- operating system in Guatemala and globally. 30,31,35 medical record. lines, with glyburide added as a second We used devices with quad core processors and agent when glycemic targets are not met. For 1 GB of RAM. While most patient visits were con- patients with an initial A1c of ≥9%, the algorithm ducted at least partly in Kaqchikel, we did not calls for dual therapy (metformin and glyburide), translate the application to Kaqchikel based on as recommended by ADA and American Association feedback from the bilingual CHWs because of Clinical Endocrinologists/American College of Kaqchikel is primarily a spoken language and Endocrinology guidelines.31,35 The titration algo- most CHWs are literate only in Spanish. rithm accounts for 4 factors in making medication Earlier versions of the application used Enketo recommendations: glycemic control, current medica- (Enketo LLC) web forms for the user interface tion dose(s), adherence, and side effects. and Ona (Ona Systems) for data storage and TABLE 1. Referral Protocols for Diabetes Complications and Comorbidities for Smartphone Application for Diabetes Care Program, Guatemala Routine Referrals (Within 1–2 Weeks) Urgent Referrals (Within 1–2 Days) Emergency Referrals (Same Day) Stage I hypertension (BP 140-160/90–100 Stage II hypertension (BP 160–200/ Severe hypertension (BP ≥ 200/120 mm mm Hg) 100–120 mm Hg) Hg) Noninfected diabetic ulcer Possibly infected diabetic ulcer, no signs of Fasting blood glucose undetectable high Need for renal function testing systemic infection Postprandial/random blood glucose A1c ≥ 9% despite maximal doses of metfor- Worsening vision undetectable high with mental status min and glyburide for ≥3 months FBG detectable, but ≥400 mg/dL changes A1c ≥ 9% for 3 consecutive checks A1c ≥ 14% Hypoglycemia associated with altered < mental status A1c above glycemic target, but 9% for 4 Patient cannot tolerate minimum doses of consecutive checks metformin and/or glyburide Persistent hypoglycemia despite treatment in the field Recent chest pain, moderate risk of CAD Current chest pain, moderate risk of CAD Current chest pain, high risk of CAD Blood in stool or possible melena Possibly infected diabetic ulcer with signs Patient has other symptoms not addressed of systemic infection by the program protocols Abbreviations: A1c, hemoglobin A1c; BP, blood pressure; CAD, coronary artery disease; FBG, fasting blood glucose. Global Health: Science and Practice 2020 | Volume 8 | Number 4 702 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org management, which was then transitioned to the training materials regarding diabetes self-care de- CommCare platform (Dimagi, Inc.), the most veloped by 2 other Guatemalan organizations widely used mobile platform among frontline that work with indigenous populations, Wuqu’ health workers in LMICs.38 While both platforms Kawoq39 and Hospitalito Atitlán.40 To learn how allow for offline data collection and have branching to conduct finger-stick testing using glucometers logic capabilities, permitting the delivery of algo- and the A1cNowþ device, measure blood pressure rithmic clinical decision support, we transitioned using automatic cuffs, and accurately measure to CommCare because it has more advanced capabil- height, weight, and waist circumference, CHWs ities for storing and modifying longitudinal data, first viewed a demonstration of these skills and includes robust database functions, and allows for ap- then practiced in small groups. Application train- plication updates to be pushed to end-user devices. ing consisted of one-on-one practice with a facili- To maintain data security, we encrypted and pass- tator to simulate a patient visit. word protected all the smartphones running the ap- Total length of training was approximately plication. CommCare is also password protected and 15 hours spread over several sessions. Dr. Duffy uses AES 256-Bit Symmetric Encryption, a HIPAA- conducted the training sessions for the first several compliant encryption standard. groups of CHWs. The coordinators of the CHW Prior to deployment in the field, we tested ap- program led subsequent sessions. After receiving plication language, workflow, and user interface this initial structured training, CHWs were paired with the CHWs and reviewed the embedded clinical with one of the coordinators for patient care to algorithms to ensure that the application provided continue supervised practice until they were able appropriate recommendations. We continued to to complete a visit with minimal direction, a pro- elicit feedback from the CHWs and update the appli- cess which generally took 15 patient visits (ap- cation throughout the study. proximately 9 hours). CHW Training Program Evaluation CHWs were recruited for participation from the Study Design CHWs were general rural health promoter program. We We used a single group, pre- and posttest design. trained in basic trained these CHWs in basic diabetes care (includ- Inclusion criteria for the program were established diabetes care, ing medication management, diabetes self-care type 2 diabetes and age greater than 18 years. protection of and lifestyle counseling, and the recognition and Exclusion criteria were insulin therapy, pregnancy, human subjects, management of complications), protection of hu- renal insufficiency (defined as estimated glomerular and use of testing man subjects, and use of testing equipment (e.g., filtration rate [GFR] <30 mL/min/1.73 m2), and equipment and glucometers) and the application. We adapted physician discretion. the application. Community health workers in Guatemala practice using a smartphone application for diabetes care. Credit: © 2018 José Vicente Macario/San Lucas Mission Global Health: Science and Practice 2020 | Volume 8 | Number 4 703 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org Community health workers in Guatemala practice point-of-care hemoglobin A1c testing. Credit: © 2017 Sean Duffy/University of Wisconsin Clinical Outcome Measures agreed with medication recommendations provid- Primary clinical outcomes were A1c and the per- ed by the application. We also tracked instances in centage of patients with A1c 8% and meeting in- which the application provided erroneous recom- dividual A1c goals compared with baseline. When mendations (as determined by the physician A1c was higher than the detectable range of the reviewing visit data). We administered a Spanish A1CNowþ analyzer (displayed as “>13.0%”), we translation of the System Usability Scale, the most imputed these values conservatively as 13.1%. widely used standardized usability questionnaire, Secondary outcomes included BG, blood pressure, to all CHWs who had used the application. This scale results in a usability score from 0 to 100. We weight, body mass index (BMI), and waist circum- used a grading schema proposed by Bangor et al.,44 ference. When BG was higher than the detectable which rates usability scores less than 50 as “not ac- range of the Contour glucometers (>600 mg/dL, ceptable,” those between 50 and 70 as “marginally displayed as “HI”), we also imputed these values acceptable,” and scores above 70 as “acceptable.” conservatively as 601 mg/dL. This usability survey also solicited written feedback We tracked the prevalence of medication side about the application. Finally, we maintained de- effects, change in medication dose, complications tailed records of program costs in order to estimate of diabetes and related referrals, and adverse the average cost per patient. events, with a focus on hypoglycemia (defined as < BG 70 mg/dL) and hypoglycemia symptoms. Patient Recruitment Based on cases known to the CHWs, we estimated Behavioral Outcome Measures the number of patients with diagnosed diabetes in We administered validated Spanish versions of the rural communities of interest to be approxi- 2 standardized questionnaires—the Diabetes Knowl- mately 150. The CHWs recruited these patients edge Questionnaire (DKQ)41 and the Summary of for the program and we set an enrollment target Diabetes Self-Care Activities (SDSCA)42,43—in June of 100 patients, which reflected the resources and 2018 to 2 subgroups of patients: patients enrolled in CHW capacity available for the program. For each visit, we the past 3 months and those who had been participat- tracked whether ing for 6 months or more. We repeated question- Statistical Analysis the CHWs and the naires for patients in the newly enrolled group in We used R (The R Foundation) for analysis of pro- supervising gram outcomes. We analyzed differences in con- January 2019. physician agreed tinuous variables (e.g., A1c) using generalized with medication additive mixed effects models (GAMMs) with the recommendations Application-Specific and Process Outcomes nonlinear smoothing function on time since pro- provided by the For each visit, we tracked whether the CHWs and gram enrollment. For all health outcomes, base- application. the supervising physician reviewing visit data line covariates of age, sex, and years since the Global Health: Science and Practice 2020 | Volume 8 | Number 4 704 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org participant’s diabetes diagnosis were included as in the patients’ preferred language (Spanish or standard main effects; the penalized regression Kaqchikel). A seed grant from the University splines were used on the longitudinal covariate of Wisconsin Global Health Institute provided of time since enrollment. Models also included funding. subject-specific random intercepts and time-since- enrollment slopes. For the glucose outcome, RESULTS whether the participant had been fasting at the time of measurement was also included in the Enrollment and Retention models as a longitudinal main effect. Eighty-nine patients enrolled during the study For the outcomes of A1c and glucose, values period (February 2017 to June 2019), and that were at the limit of detection were treated as 67 remained in the program at the end of this pe- a typical value in these GAMMs. In order to test if riod (retention rate of 75.3%). Of patients who significant change in health outcomes from base- completed at least one follow-up visit, median line occurred at 3, 6, 9, and 12 months after enroll- follow-up time was 12.1 months (range 1.1–28.2, ment, bootstrapped confidence intervals were IQR 9.8). One patient died while participating in employed. Due to the censored nature of some of the program, 2 withdrew, and 11 were lost to the A1c and glucose values, a sensitivity analysis follow-up. Eight patients were excluded after en- of these outcomes was conducted using a Cox pro- rollment, 4 because of renal failure, 1 because of portional hazard mixed effects modeling struc- recurrent hypoglycemia while taking metformin ture. Model diagnostics revealed a concern for alone, 1 because of hyperglycemia requiring insu- heteroscedasticity in the glucose model. Refitting lin therapy, and 2 because of terminal illness. the model on the natural-log of glucose alleviated Patients completed 920 visits (enrollment and the issue, thus all reported glucose modeling monthly), 80.8% occurring at the designated cen- results are from a model fitted to the natural-log tral location and 19.2% in patient homes. Patients of glucose. who remained in active follow-up completed The same GAMM structure already described 93.8% of possible visits, with all patients (includ- was used to analyze A1c control ( 8%) and A1c ing those who were excluded or lost to follow- goal attainment separately, with the appropriate up) completing 80.7% of possible visits. model setup changes for the outcome being binary instead of continuous. Additionally, for A1c con- Cohort Profile trol/goal attainment, a pre-post study design was Table 2 summarizes the baseline characteristics of mimicked within the data by selecting each parti- enrolled patients, including place of diagnosis (a cipant’s baseline value and their value closest to proxy for prior source of care) and medication the 3, 6, 9, and 12 month follow-up period (within use. Of note, a large majority (82%) of enrolled 645 days, otherwise the observation at follow-up patients were women. Baseline glycemic control was considered missing). These pairs were then was poor, with a mean A1c (standard deviation used to perform a McNemar test on the change in [SD]) of 10.0% (2.5) and only 20% of patients Baseline glycemic A1c control/goal attainment at these 4 follow-up meeting A1c treatment goals. control was poor, times. with a mean A1c of For DKQ and SDCA scores and medication Clinical Outcomes 10.0% and only doses, we used the Shapiro-Wilk test to determine GAMM regression results are displayed in the 20% of patients normality. We then used 2-tailed t tests to assess Supplement. Age at baseline was significantly asso- meeting A1c differences in normally distributed variables and ciated with A1c (b = 0.046, P=0.002), natural-log treatment goals. the Wilcoxon test for nonnormally distributed glucose (b = 0.008, P= 0.003), systolic blood pres- variables. We used a significance threshold (a)of sure (b =0.569, P<0.001), A1c control (OR=1.05, 0.05 for all analyses. P=0.005), and A1c goal attainment (OR=1.08, P<0.001) but not associated with diastolic blood Ethical Oversight and Funding pressure, weight, waist circumference, or BMI. The program was reviewed and approved by the Baseline years since diabetes diagnosis was signifi- University of Wisconsin and Stanford University cantly associated with A1c (b =0.073, P=0.021), institutional review boards, as well as the SLM natural-log glucose (b =0.018, P<0.001), A1c con- Health Program. All patients provided written in- trol (OR=0.90, P=0.005), and A1c goal attainment formed consent after a bilingual CHW explained (OR=0.89, P=0.025), but no other health out- the study and risks and benefits of participation comes. Fasting status was only in the glucose model Global Health: Science and Practice 2020 | Volume 8 | Number 4 705 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org A community health worker in Guatemala conducts a home visit with a diabetes patient using a smartphone application for clinical decision support. Credit: © 2017 Sean Duffy/University of Wisconsin and was significantly associated with natural-log 9 and 12 months was nonsignificant. Weight dis- glucose (b = 0.357, P<0.001). Sex was not associ- played a significant mean gain over baseline at ated with any of the health outcomes examined. 3 months (1.86 lb estimated gain), but no signifi- Time since program enrollment was significant- cant change from baseline at the other time points. ly associated with the outcomes of A1c, natural-log BMI displayed a similar trend to weight. glucose, weight, and BMI (all P<0.001), with non- GAMM results for A1c control/goal attain- linear behavior between times since enrollment ment (Figures 5 and 6) showed an initial trend to- and these outcomes. Figures 2 to 4 show the esti- wards increased attainment until approximately mated behavior over time for A1c, glucose, and 6 months, followed by a trend back towards base- weight; BMI and weight results were very similar line. No significant association was found between to one another, as expected, and only the weight time since enrollment and probability of A1c con- figure is shown. Both A1c and glucose were esti- trol/goal attainment. However, when mimicking a mated to decrease up to around 6 months, and pre/post design and analysis for examining these then slowly rise back towards baseline values after- outcomes at 3, 6, 9, and 12 months from baseline, wards. However, the sparsity of observations after time periods closer to baseline were associated 1 year resulted in increased uncertainty in the esti- with significant increases in the proportion with mated trend after this point. Both weight and BMI A1c control/goal attainment (Table 4), similar to were estimated to increase up to about 6 months af- the A1c continuous analyses above. For A1c con- ter baseline, then to slowly decrease afterwards. trol, significant proportion increases from baseline After 1 year, the uncertainty in the estimation of were detected at 3, 6, and 9 months after baseline the trend increased greatly. (P-values<0.034), but not at 12 months (P= Based on the bootstrapped results for changes 0.121). For A1c goal attainment, significant pro- from baseline at 3, 6, 9, and 12 months (Table 3), portion increases from baseline were detected at A1c displayed significant reductions from baseline 3 and 6 months (P-values<0.020), but not at at all 4 intervals, with the greatest estimated reduc- 9 and 12 months (P-values>0.114). However, for tion at 6 months (1.45 A1c % mean reduction), but both outcomes and at all 4 follow-up periods, still a 1-point reduction estimated at 1 year after en- the raw proportion increased, ranging from a rollment. Natural-log glucose displayed significant 17.1% to 22.0% increase in A1c control, and from reductions from baseline at 3 and 6 months, with 7.3% to 20.0% increase in A1c goal attainment. the greatest estimated reduction at 6 months Sensitivity analyses using Cox mixed effects (0.135 natural-log mL/dL mean reduction; 22.4 mL/ models to handle the true censored nature of the dL reverse transformed for a typical subject in the A1c and glucose values had numerous issues with data; see Table 3 footnote), but the reduction at the assumptions of proportionality. The results of Global Health: Science and Practice 2020 | Volume 8 | Number 4 706 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org the bootstrapped results did not show any signifi- TABLE 2. Baseline Characteristics of Patients cant difference in these variables at the designated Enrolled in a Rural Diabetes Care Program, analysis time points. We also ran unadjusted anal- Guatemala yses for all the models described above, which were consistent with the adjusted results in terms Characteristic (N=89) Value of statistically significant associations and the non- Demographics linear forms between time since intervention and outcomes. Mean age (SD), years 53.5 (13.3) Sex, % female 82 Medication Titration, Side Effects, and Years since diabetes diagnosis, median 4 (6) Adverse Events (IQR) Median daily doses of metformin and glyburide Place of diagnosis, % increased significantly (all P 0.02) from pre- San Lucas Mission rural clinic 40 enrollment to first recommendation at enrollment Private clinic 20 visit (500 to 1,700 mg and 0 to 2.5 mg) and from enrollment visit to last visit (1,700 to 2,550 mg Nongovernmental organization hospital 16 and 2.5 to 5 mg). Patients taking metformin Guatemalan Social Security clinic 12 reported typical gastrointestinal side effects during Government clinic 6 6.7% of visits. Side effects were significant enough to warrant a dosage reduction per the titration Other 6 protocol during 3.9% of metformin-exposed visits Medication use, % (29.9% of metformin-exposed patients, 0.5 events Taking any diabetes medicationa 82 per patient-year of therapy). There were 11 epi- < Metformin 71 sodes of documented hypoglycemia (BG 70 mg/ dL). Glyburide dosage was reduced due to hypo- Glyburide 30 glycemia symptoms or documented hypoglyce- Glimepiride 3 mia for 7.8% of glyburide-exposed visits (36.1% of Natural remedies 18 glyburide-exposed patients, 0.9 events per patient- Clinical measures year of therapy). Nine of the 11 hypoglycemic episodes were Mean hemoglobin A1c (SD), % 10.0 (2.5) mild and resolved with treatment by CHWs or at Proportion with A1c at goal, % 20 home. Two hypoglycemic episodes required hos- Mean body mass index (SD), kg/m2 26.7 (4.6) pitalization for management. Both episodes oc- curred in the same patient, who was taking Mean blood glucose (SD), mg/dL 237 (126) metformin alone and also had concomitant severe Abbreviations: IQR, interquartile range; SD, standard deviation. acute illnesses at the time of the episodes. a Does not include natural remedies. Complications of Diabetes Forty-four patients (49.1%) were identified as the A1c model confirmed the GAMM A1c results, having increased risk of chronic kidney disease. with increased time since enrollment associated Of these patients, 35 (80%) underwent renal “ with decreased A1c values (a significant in- function testing. Mean (SD) GFR was 77.1 (34.7) ” creased hazard of observing A1c at lower values). mL/min/1.73 m2. Twenty-six (74.3%) patients The glucose model did not display a significant as- in this group had normal GFR (>60 mL/min/ sociation between glucose and time since enroll- 1.73 m2), 5 (14.3%) had GFR 30–60 mL/min/ ment. However, from the GAMM results, there 1.73 m2, and 4 (11.4%) had significantly reduced appeared to be nonlinear behavior between glu- renal function with GFR <30 mL/min/1.73 m2. cose and time. The Cox model did not properly ac- A total of 279 referrals were recommended by count for this nonlinearity, and the “fall then rise” the application for one or more potential compli- nature of the trend paired with the assumption cations of diabetes, representing 30.3% of visits. violations could obscure a true association. Of these, patients accepted 134 (48.0%) referrals. The GAMM results did not show any signifi- Based on available referrals tracking data, we esti- cant relationship between time since enrollment mate that patients completed 50.0% of accepted and blood pressure or waist circumference, and referrals, representing 24.0% of all recommended Global Health: Science and Practice 2020 | Volume 8 | Number 4 707 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org FIGURE 2. Estimated Change in Hemoglobin A1c Over Time After Enrollment in Rural Diabetes Care Program, Guatemala FIGURE 3. Estimated Change in Glucose Over Time After Enrollment in Rural Diabetes Care Program, Guatemala referrals. Figure 7 lists referrals by indication. infarction. This patient had well-controlled diabe- Renal function testing was by far the most com- tes on metformin alone and had not reported mon indication for referral (50.5% of all referrals). symptoms of myocardial ischemia or other compli- Our clinical algorithms call for repeat referrals for cations prior to their death. renal function testing until completed for patients for whom it is indicated, contributing to the high Behavioral Outcomes number of referrals for this indication. Thirteen patients who had been in the program for One patient died while participating in the pro- 6 months or more and 11 patients who had en- gram. The probable cause of death was myocardial rolled in the past 3 months completed the DKQ Global Health: Science and Practice 2020 | Volume 8 | Number 4 708 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org FIGURE 4. Estimated Change in Weight Over Time After Enrollment in Rural Diabetes Care Program, Guatemala and SDSCA. DKQ scores did not vary between the and future errors were prevented through applica- 2 groups, with a mean score of 13 for both (P=1). tion updates. Of the newly enrolled patients, we were able to re- Twenty-one CHWs completed the System peat the DKQ 6 months later for 5 patients. There Usability Scale survey in January 2019. The mean was no significant difference between baseline score for fully completed surveys was 61.3 (range and follow-up scores (mean 13 vs. 12.8, P=1). 27.5–87.5) and the mean composite score (ac- Patients who had been enrolled for at least counting for responses from partially completed 6 months had higher average SDSCA scores surveys) was 62.1. Subgroup analysis of scores (scored 0 to 7, with 7 being optimal) in several above and below the predefined “acceptable” self-care categories compared with newly enrolled threshold of 70 showed that CHWs who rated ap- patients (see Table 5). However, only differences plication usability above 70 (n=4) were younger in foot care and dedicated exercise were statistical- (mean age 32.0 vs. 42.2 years), more educated ly significant, with dedicated exercise scores being (mean 10.2 vs. 5.8 years of education), used better in the newly enrolled group. We obtained smartphones more often (median use daily vs. follow-up SDSCA scores for 5 patients in the re- once weekly), and had greater experience with cently enrolled group 6 months after the initial the diabetes application (median use 11–15 times questionnaire, which did not show any statistical- vs. less than 5 times) on average than those with ly significant improvements. scores 70 or below (n=12). Fourteen CHWs pro- vided written subjective feedback on how the application could be improved. Common recom- Application-Specific and Process Outcomes mendations for improvement were to make the CHWs and the reviewing physician agreed application faster and more responsive, reduce Our results with medication recommendations given by the the number of questions and simplify language, suggest that CHWs application for 90.9% of visits. During 53 visits and increase the amount of practice that CHWs enabled by CDS (5.8%), medication recommendations were altered had with the application. by the CHWs after remote consultation with a phy- We estimated a program start-up cost of US technology can sician. The reviewing physician changed medication $3,940 for 100 patients, with continuing costs of safely and recommendations based on data review after 30 vis- US$118 per patient, per year (Table 6). effectively its (3.3%). There were 4 cases in which the appli- manage diabetes cation made inappropriate recommendations or in rural malfunctioned. In each of these cases, patient treat- DISCUSSION Guatemala with ment was corrected through direct communication Our results from the development and implemen- remote physician between the supervising physician and the CHWs tation of this program suggest that CHWs enabled supervision. Global Health: Science and Practice 2020 | Volume 8 | Number 4 709 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org TABLE 3. Bootstrapped Results for Change Since Baseline in Outcomes Among Patients Enrolled in Rural Diabetes Care Program, Guatemala Time Since Estimated 95% CI 95% CI Outcome Baseline Change Lower Bound Upper Bound A1C, % 3 months –1.04a –1.68 –0.559 6 months –1.45a –2.19 –0.813 9 months –1.32a –2.01 –0.636 12 months –1.03a –1.73 –0.385 Glucose, 3 months –0.104a –0.199 –0.0244 natural-log – a – – mL/dL 6 months 0.135 0.232 0.0366 9 months –0.0909 –0.163 0.00368 12 months –0.0677 –0.166 0.0175 Glucose,b 3 months –17.5a –36.3 –5.00 mL/dL 6 months –22.4a –39.2 –5.20 9 months –15.4 –27.3 1.70 12 months –11.6 –30.7 3.28 Systolic BP, 3 months 0.375 –5.02 1.08 mm Hg 6 months 0.75 –3.62 2.48 9 months 1.13 –2.84 3.87 12 months 1.5 –2.82 3.83 Diastolic BP, 3 months –0.0678 –0.812 1.1 mm Hg 6 months –0.189 –1.22 1.77 9 months –0.467 –1.83 1.74 12 months –0.877 –2.59 1.02 Weight, lb 3 months 1.86a 0.355 3.29 6 months 2.84 –0.0432 5.17 9 months 2.44 –1.67 4.88 12 months 1.67 –3.89 4.77 Waist-circumference, 3 months 0.269 –0.474 1.28 cm 6 months 0.51 –0.677 1.86 9 months 0.718 –0.632 1.99 12 months 0.896 –0.253 2.19 BMI, kg/m2 3 months 0.372a 0.0856 0.681 6 months 0.616a 0.0477 1.1 9 months 0.639 –0.229 1.14 12 months 0.538 –0.674 1.1 Continued Global Health: Science and Practice 2020 | Volume 8 | Number 4 710 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org TABLE 3. Continued Outcome Time Since Est. Baseline Difference in Probability 95% CI of Control/ 95% CI Upper Goal Lower Bound Bound A1C control, 3 months 0.127 –0.0238 0.276 A1C 8% 6 months 0.203 –0.0472 0.454 9 months 0.205 –0.0702 0.391 12 months 0.166 –0.0927 0.341 A1C goal, 3 months 0.0588 –0.0096 0.22 A1C subject goal 6 months 0.0999 –0.0188 0.394 9 months 0.106 –0.0276 0.263 12 months 0.0929 –0.0359 0.195 Abbreviations: A1C, hemoglobin A1C; BMI, body mass index; BP, blood pressure; CI, confidence interval. a P<.05. b As these numbers are no longer on the scale of the regression, these values are specific to the type of subject the predictions were performed on (i.e., the values the other covariates are set at for prediction affect these numbers, unlike on the regression scale), which was the most common subject sex (female) and fasting value (true), median baseline age (54 years), and median years since diabetes diagnosis at baseline (4 years) for the subjects in analyses. by CDS technology can safely and effectively significant improvements in the adjusted binary manage diabetes in rural Guatemala with remote A1c outcomes could have been a function of inad- physician supervision. Longitudinal analysis dem- equate power. onstrated significant improvements in the primary The improvements in glycemic control associ- outcome of A1c, including at the predefined time ated with this program are similar to those points of 3, 6, 9, and 12 months after program en- reported for other CHW-led diabetes interven- 21,47,48 rollment. Statistically significant improvements tions in LMICs. A key difference from prior in A1c ranged from 1.0% to 1.4%. These A1c published interventions using CHWs in diabetes improvements also meet the commonly used care is that rather than providing ancillary ser- threshold of 0.5% for a clinically significant vices, such as patient education, in support of tra- change in A1c.45,46 ditional medical care, CHWs in our program were The proportion of patients with A1c 8% and directly providing care: they assessed glycemic control, directed medication therapy, and identi- meeting individualized treatment goals increased fied potential complications with the assistance of at each of these time points as well, with statisti- mobile CDS technology. This approach is relevant cally significant increases at 3, 6, and 9 months The application for similar LMIC settings around the world, where and 3 and 6 months, respectively. However, it provided reliable health systems are faced with a rising tide of dia- recommendations, should be noted that significant covariates of age betes and other chronic diseases in the context with CHWs and and years since diabetes diagnosis were not of dire shortages of physicians, nurses, and other the reviewing accounted for in these results. The GAMM analy- highly trained health workers.7,49 physician ses, which included these covariates, showed a agreeing with the trend in A1c control/goal attainment similar to Decision Support application- that in the continuous A1c analysis, but the con- In general, the application provided reliable recommended trol/goal attainment trend did not meet statistical recommendations, with CHWs and the reviewing medication significance. Given that the continuous GAMM physician agreeing with the application- dosing greater models showed significant improvements in A1c recommended medication dosing greater than than 90% of the over time, the failure to detect statistically 90% of the time. There were only 4 instances time. Global Health: Science and Practice 2020 | Volume 8 | Number 4 711 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org FIGURE 5. Estimated Probability of Hemoglobin A1c Control ( 8%) Over Time After Enrollment in Rural Diabetes Care Program, Guatemala FIGURE 6. Estimated Probability of Meeting A1c Goal Over Time After Enrollment in Rural Diabetes Care Program, Guatemala in which the application provided incorrect experience with the application found the ap- recommendations compared with the estab- plication easier to use. While we elicited feed- lished protocols. System Usability Scale surveys back from CHWs at all points of application of the CHWs suggested marginally acceptable development and deployment, this feedback usability (mean score of 62.1).44 Subgroup was dominated by the coordinators of the analysis suggested that CHWs who had at least CHW program, who were generally better edu- some high school level education, who used cated and had more experience in conducting smartphones regularly, and who had more diabetes visits. Thus, increasing “rank and file” Global Health: Science and Practice 2020 | Volume 8 | Number 4 712 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org TABLE 4. Change in Proportion of Patients Meeting A1c Targets Among Those Enrolled in Rural Diabetes Care Program, Guatemala Outcome Time Since Baseline N Pre Control, % Post Control, % Proportion Change, % McNemar P Value A1c control 3 months 72 23.6 44.4 20.8 .007 6 months 50 22 44 22 .015 9 months 46 23.9 45.7 21.8 .034 12 months 41 26.8 43.9 17.1 .121 Outcome Time Since Baseline N Pre at Goal, % Post at Goal, % Proportion Change, % McNemar P Value A1c at goal 3 months 72 16.7 31.9 15.2 .015 6 months 50 14 34 20 .016 9 months 46 15.2 28.3 13.1 .114 12 months 41 17.1 24.4 7.3 .55 Abbreviation: A1c, hemoglobin A1c. A community health worker in Guatemala checks the blood glucose of a diabetes patient during a home visit. Credit: © 2018 Cesia Castro Chutá/San Lucas Mission CHW involvement in application development WHO and other global health policy leaders is one potential strategy to improve usability. have recognized the potential of mobile CDS tools CHWs also noted the tendency of the applica- to mitigate a lack of highly trained health care tion to lag, negatively impacting usability. Our workers and supportive infrastructure and to im- use of low-end Android devices likely accounts prove the quality of care through the use of algo- for this because we have found the application rithmic protocols.38,51 These organizations have to work much faster on higher-performance called for more rigorous evaluation of such devices. Fortunately, continued progress in mHealth interventions.51,52 Our experience in ru- smartphone development has meant that bud- ral Guatemala adds to the evidence base support- get devices manufactured today are equivalent ing the use of mobile CDS to assist CHWs with to flagship devices 2–3 years ago.50 chronic disease management and could be Global Health: Science and Practice 2020 | Volume 8 | Number 4 713 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org FIGURE 7. Referrals Recommended by Smartphone Application to Supervising Physician by Indication, Guatemalaa a Sum of indications is greater than total number of individual referrals (279) as some referrals had multiple indications. TABLE 5. Comparison of Summary of Diabetes Self-Care Activities (SDSCA) Between New and Established Patientsa Patients Enrolled Patients Enrolled Measure ≥6 Months (n=13) <3 Months (n=11) P Value Healthy diet in the past week 7.0 [1.0] 6.0 [3.0] .294 Healthy diet in general 7.0 [1.0] 6.0 [2.5] .310 Eating fruits and vegetables 4.2 (2.1) 3.2 (2.5) .319 Avoidance of high-fat foods 7.0 [1.0] 6.0 [0.5] .088 Even distribution of carbohydrates 7.0 [0.0] 7.0 [0.0] .755 Specific diet score 5.3 (1.1) 4.5 (1.1) .088 General diet score 7.0 [1.0] 6.0 [2.8] .336 Physical activity 7.0 [0.0] 7.0 [2.0] .414 Dedicated exercise 0.0 [0.0] 0.0 [1.0] .020 Exercise subscore 3.5 [0.0] 3.5 [1.5] .424 Foot care 7.0 [1.0] 4.0 [5.5] .047 Medication adherence 7.0 [0.0] 7.0 [0.0] .849 a Values with parentheses represent mean (SD) and those with brackets represent median [IQR]. adapted for diabetes management in similar LMIC applications.51,53,54 While our program does not settings. This approach could also be applied to directly interface with the government health sys- other chronic diseases amenable to algorithmic tem in Guatemala at this time, such regional or na- care, such as hypertension. We will freely share tional partnerships would be essential for effective the application to allow others to build upon our scale-up. TulaSalud, a nongovernmental organi- work. zation working in the northern highlands of Integration with the greater health system is Guatemala, provides a model for effective scale- integral to the success of mobile health up in collaboration with the Ministry of Health Global Health: Science and Practice 2020 | Volume 8 | Number 4 714 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org improvements in A1c resulting from reconstitu- TABLE 6. Estimated Program Start-Up and tion of medication therapy, particularly when Maintenance Costs baseline A1c is high.56,57 However, 2 factors sup- port the importance of the titration algorithm in Start-Up Costsa our program. First, most patients (82%) reported Expenditure Total cost that they were taking medications at the time of enrollment. Thus, subsequent improvements in Glucometers and lancing devices $140 glycemic control suggest that medication optimi- Smartphones $600 zation and not merely initiation played a role for Automatic blood pressure cuffs $200 most patients. Secondly, median doses of metfor- CommCare feesb $3,000 min and glyburide increased significantly during the follow-up period. Total $3,940 Our data suggest possible attenuation of pro- Continuing per Capita Costs gram effects on glycemic control over time. Expenditure Cost per patient, Although the reduction in A1c remained signifi- per year cant at 12 months after enrollment, A1c reduction Medications (metformin, glyburide, $32 peaked at 6 months and trended back towards aspirin) baseline after this point. So-called “secondary failure” of hypoglycemic medications—a reduc- Hemoglobin A1c tests $38 tion in efficacy over time, particularly for Other testing supplies (e.g., glucose $9 glyburide and in patients with prior long-term, strips, lancets) high-dose treatment58—could also contribute to CHW labor costs $16 long-term attenuation of improvements in glyce- CHW coordinator labor costs $13 mic control. Other costs (e.g., equipment replace- $10 Based on our data, the safety of the interven- ment, data planc) tion was comparable to routine diabetes care delivered in other contexts. Patients experienced Total $118 metformin side effects requiring dosage reduc- Abbreviation: CHW, community health worker. tion at 3.9% of metformin-exposed visits. This a Reflects start-up costs for an anticipated patient population of outcome is comparable to clinical trials of metfor- 100 patients. b min, which generally report a 5% prevalence of This reflects current CommCare fees, which are $250/organi- 59 zation/month for a basic plan. CommCare fees are not reflected metformin intolerance. Thirty-six percent of in continuing per capita costs because they are not dependent glyburide-exposed patients experienced probable on caseload, and in our case, they support other health pro- hypoglycemia symptoms or documented hypogly- grams with thousands of total patients. cemia, with a mean of 0.9 events per patient-year c CommCare projects generally use 100 MB or less of mobile of therapy. None of these episodes were severe. data per month. Published estimates of the frequency of hypogly- cemia attributable to glyburide and other sulfony- lureas vary widely based on event definitions.60–64 and other health care organizations.55 Using the A prospective study of 383 patients that used a CommCare platform, they have developed and definition of hypoglycemia similar to ours (patient deployed mobile applications to assist CHWs in report of hypoglycemia symptoms or documented maternal and child health initiatives, and enable glucose measurement in the hypoglycemic range) care coordination with the Ministry of Health, found a similar prevalence (39%) and incidence across a service area of 3.4 million people. (1.92 events per person-year) of hypoglycemia in patients taking sulfonylureas.65 Medication Titration, Attenuation of Diabetes Based on our Control, and Medication Side Effects Diabetes Self-Care Counseling data, the safety of It is possible that simply establishing consistent While patients enrolled for at least 6 months had the intervention medication therapy through free provision of higher SDSCA scores than newly enrolled patients was comparable medications and regular follow-up, regardless of in several self-care categories, these differences to routine dose titration, accounted for improvements in were only statistically significant for foot care and diabetes care glycemic control. Other studies of diabetes man- dedicated exercise (with exercise scores actually delivered in other agement in LMICs have shown marked better in the newly enrolled group). Additionally, contexts. Global Health: Science and Practice 2020 | Volume 8 | Number 4 715 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org when we repeated this questionnaire with these Another issue inherent in A1c measure- newly enrolled patients after 6 months, there ment is the effect of anemia, hemoglobinopa- were no statistically significant improvements. thies, and other metabolic abnormalities.68 Sample size was very small, including only While hemoglobinopathies are rare in indige- 5 patients for repeated SDSCA questionnaires, so nous populations of the Americas,69 anemia it is difficult to reach any conclusions on the effec- (primarily iron-deficiency anemia) affects more tiveness of CHW-delivered lifestyle counseling. than 20% of women of childbearing age in However, the lack of significant change suggests Guatemala.70 We did not screen subjects for ane- that counseling may need to be intensified and op- mia in this study, so we are unable to assess the timized. Two other interventions in which CHWs potential effect of anemia on our results. and diabetes educators provided self-care counsel- However, the primary outcomes in this study ing to indigenous Guatemalans with diabetes have were longitudinal with each subject acting as reported significant improvements in glycemic their own control, mitigating the potential effect control.57,66 Of note, both of these interventions of skewed A1c results due to anemia in our were relatively intensive, with weekly visits in analysis. one intervention66 and mean counseling time of Our study population was mostly women 10 hours over a 9-month period in the other.57 In (82%). The “men’s health gap”—reduced health contrast, visits in our program occur monthly and care utilization and poorer health outcomes typically include approximately 10 minutes of dia- among men compared to women—is an impor- 71 betes self-care counseling. tant global phenomenon. Other diabetes inter- ventions in rural Guatemala have also struggled to recruit and retain men.33,66 The low participa- Program Costs tion levels of men are likely multifactorial,71 but The estimated cost of this program is less than that in our experience the predominantly agricultural reported for a nurse-led diabetes program in nature of men’s work in these communities, Guatemala: US$118 versus US$220 per patient, entailing long hours and lengthy travel to the per year.33 However, this program provided more fields, is a key factor. Despite offering home visits comprehensive services, including insulin and hy- on weekends, we were unable to overcome these pertension treatment. The cost of our program is barriers. Further research is needed on how to im- also comparable to data from a recent systematic prove outreach to men in rural Guatemala and review of the cost of diabetes treatment in similar contexts. LMICs, which reported average annual treat- Due to a low referral completion rate, relative- ment costs ranging from US$29.91 to US$237.38 per ly few referrals for certain complications of diabe- person.67 tes (such as chest pain and vision problems), and lack of advanced diagnostic testing capabilities at the referral hospital, it is difficult to assess the ac- Limitations curacy and efficacy of our protocols for detection, The primary limitation of this study is the lack of a management, and referral of potential diabetes control group. A future study comparing CHW-led complications. Although we did not have renal care with physician, midlevel provider, or nurse- function testing available for our entire patient led care is necessary to determine the efficacy of population to validate our algorithm for identify- our approach versus standard practice. Another ing patients at higher risk of renal impairment, limitation of our analysis was the substitution of 25.7% of patients who completed renal function inferred values for A1c and glucose when mea- testing had at least some degree of renal function surements fell outside the range of the measure- impairment (GFR <60 mL/min/1.73 m2) and ment devices. This injects a degree of uncertainty 11.4% had significant renal impairment (GFR into the calculated changes in mean A1c and glu- <30 mL/min/1.73 m2). This is similar to the prev- cose throughout the study. However, sensitivity alence of decreased GFR in type 2 diabetics (22%) analysis showed that changes in A1c were robust estimated from a large global study completed in to this limitation in measurement. In addition, 2006.72 Thus, even though we have testing data improvements in the proportion of patients meet- available for renal function, it is difficult to assess ing A1c goals were not affected by this measure- the effectiveness of our algorithm in identifying ment uncertainty, and this outcome supports the high-risk patients. efficacy of the program in improving glycemic Finally, we designed this program and the CDS control. application to fit our specific context of rural Global Health: Science and Practice 2020 | Volume 8 | Number 4 716 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org Guatemala and the specific resources and capacity 5. World Health Organization (WHO). Task Shifting to Tackle Health of our local partner, which may make our findings Worker Shortages. WHO; 2007. Accessed September 25, 2020. http://www.who.int/healthsystems/task_shifting_booklet.pdf less generalizable to other settings. While we are 6. World Health Organization (WHO), PEPFAR, UNAIDS. Task hopeful that others will be able to learn from our Shifting: Rational Redistribution of Tasks among Health Workforce experience and to use the application, significant Teams: Global Recommendations and Guidelines. WHO; 2008. modifications may be required for our model to Accessed September 25, 2020. http://www.who.int/health be used elsewhere. systems/TTR-TaskShifting.pdf 7. Fleck F, Narayan KMV. The mysteries of type 2 diabetes in develop- ing countries. Bull World Health Organ. 2016;94(4):241–242. CrossRef. Medline CONCLUSIONS 8. World Health Organization (WHO). Global Action Plan for the A novel CHW-led diabetes program enabled by Prevention and Control of Noncommunicable Diseases 2013–2020. mobile CDS technology led to improvements in WHO; 2013. Accessed September 25, 2020. http://apps.who.int/ iris/bitstream/10665/94384/1/9789241506236_eng.pdf diabetes control for a rural Guatemalan popula- tion. A task-sharing model using nonphysician 9. Anand TN, Joseph LM, Geetha AV, Prabhakaran D, Jeemon P. Task sharing with non-physician health-care workers for management of health care workers assisted by mHealth applica- blood pressure in low-income and middle-income countries: a sys- tions holds promise for improving the care of dia- tematic review and meta-analysis. Lancet Glob Health. 2019;7(6): betes and other noncommunicable diseases in e761–e771. CrossRef. Medline LMICs, which represent a crucial health challenge 10. Bergström S. Training non-physician mid-level providers of care (as- sociate clinicians) to perform caesarean sections in low-income of the 21st century. Further work is needed to de- countries. Best Pract Res Clin Obstet Gynaecol. 2015;29(8):1092– termine the efficacy of this approach compared 1101. CrossRef. Medline with standard care, to enhance the application to 11. Joshi R, Alim M, Kengne AP, et al. Task shifting for non- allow for the delivery of more comprehensive dia- communicable disease management in low and middle income betes management, and to better support lifestyle countries—a systematic review. PLoS One. 2014;9(8):e103754. CrossRef. Medline changes through enhanced counseling and inter- ventions to improve the nutritional environment. 12. Abegunde D, Shengelia B, Luyten A, et al. Can non-physician health- care workers assess and manage cardiovascular risk in primary care? Bull World Health Organ. 2007;85(6):432–440. CrossRef. Medline Acknowledgments: We would like to especially acknowledge our 13. Ogedegbe G, Gyamfi J, Plange-Rhule J, et al. Task shifting interven- partners in Guatemala—including San Lucas Mission Medical Director tions for cardiovascular risk reduction in low-income and middle- Dr. Rafael Tun and the community health workers, particularly José Vicente Macario, Cesia Castro Chutá, Olga Marina Ajcalon Cuc, and income countries: a systematic review of randomised controlled BMJ Open Dominga Pic Salazar—who have been integral to this project. Dr. David trials. . 2014;4(10):e005983. CrossRef. Medline Rabago was invaluable as a faculty mentor for Dr. Duffy, the primary 14. Vedanthan R, Bernabe-Ortiz A, Herasme OI, et al. Innovative author. Dr. Mindy Smith helped to edit and frame a first draft of the approaches to hypertension control in low- and middle-income manuscript. countries. Cardiol Clin. 2017;35(1):99–115. CrossRef. Medline 15. Mishra SR, Neupane D, Preen D, Kallestrup P, Perry HB. Mitigation of Funding: The University of Wisconsin Global Health Institute provided non-communicable diseases in developing countries with community seed grant funding for this project. Dr. Duffy was also supported by the health workers. Global Health. 2015;11(1):43. CrossRef. Medline National Research Service Award Primary Care Postdoctoral Research Traineeship awarded by the National Institutes of Health, Health 16. Schneider H, Okello D, Lehmann U. The global pendulum swing to- Resources Services Administration (HRSA grant T32 HP10010) during a wards community health workers in low- and middle-income coun- portion of the time spent working on this project. tries: a scoping review of trends, geographical distribution and programmatic orientations, 2005 to 2014. Hum Resour Health. Competing interests: None declared. 2016;14(1):65. CrossRef. Medline 17. World Health Organization (WHO). Community Health Workers: What Do We Know About Them? The State of the Evidence on REFERENCES Programmes, Activities, Costs and Impact on Health Outcomes of Global Report on Diabetes 1. World Health Organization (WHO). . Using Community Health Workers. WHO; 2007. Accessed WHO; 2016. Accessed September 25, 2020. http://apps.who.int/ September 25, 2020. http://www.who.int/hrh/documents/ iris/bitstream/10665/204871/1/9789241565257_eng.pdf community_health_workers_brief.pdf 2. Ogurtsova K, da Rocha Fernandes JD, Huang Y, et al. IDF Diabetes 18. Qi L, Liu Q, Qi X, Wu N, Tang W, Xiong H. Effectiveness of peer Atlas: global estimates for the prevalence of diabetes for 2015 and support for improving glycaemic control in patients with type 2 dia- 2040. Diabetes Res Clin Pract. 2017;128:40–50. CrossRef. Medline betes: a meta-analysis of randomized controlled trials. BMC Public Health 3. Chow CK, Ramasundarahettige C, Hu W, et al.; PURE investigators. . 2015;15(1):471. CrossRef. Medline Availability and affordability of essential medicines for diabetes 19. Palmas W, March D, Darakjy S, et al. Community health worker across high-income, middle-income, and low-income countries: a interventions to improve glycemic control in people with diabetes: a prospective epidemiological study. Lancet Diabetes Endocrinol. systematic review and meta-analysis. J Gen Intern Med. 2015;30 2018;6(10):798–808. CrossRef. Medline (7):1004–1012. CrossRef. Medline 4. World Health Organization (WHO). Innovative Care for Chronic 20. Kim K, Choi JS, Choi E, et al. Effects of community-based health Conditions—Building Blocks for Action. WHO; 2002. Accessed worker interventions to improve chronic disease management and September 25, 2020. http://www.who.int/chp/knowledge/ care among vulnerable populations: a systematic review. Am J Public publications/icccglobalreport.pdf Health. 2016;106(4):e3–e28. CrossRef. Medline Global Health: Science and Practice 2020 | Volume 8 | Number 4 717 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org 21. Alaofè H, Asaolu I, Ehiri J, et al. Community health workers in dia- 37. StatCounter Global Stats. Mobile operating system market share betes prevention and management in developing countries. Ann worldwide. StatCounter; 2019. Accessed February 9, 2020. http:// Glob Health. 2017;83(3–4):661–675. CrossRef. Medline gs.statcounter.com/os-market-share/mobile/worldwide 22. Community Preventive Services Task Force (CPSTF). Interventions 38. Agarwal S, Rosenblum L, Goldschmidt T, Carras M, Labrique AB. Engaging Community Health Workers. The Guide to Community Mobile Technology in Support of Frontline Health Workers. A Preventive Services (The Community Guide). CPSTF; 2017. Accessed Comprehensive Overview of the Landscape Knowledge Gaps and September 25, 2020. https://www.thecommunityguide.org/ Future Directions. Johns Hopkins University Global mHealth Initiative; findings/diabetes-management-interventions-engaging- 2016. https://www.researchgate.net/publication/305676655_ community-health-workers Mobile_Technology_in_Support_of_Frontline_Health_Workers_A_ 23. The World Bank. Guatemala—Data. The World Bank; 2018. comprehensive_overview_of_the_landscape_knowledge_gaps_ Accessed October 1, 2020. https://data.worldbank.org/country/ and_future_directions guatemala 39. Wuqu’ Kawoq. Salud para tu corazón - raxnaqil ri k’uxaj.Wuqu’ 24. Instituto Nacional de Estadística de Guatemala. Mapas de pobreza Kawoq; 2011. rural en guatemala 2011 . Instituto Nacional de Estadística de 40. Chuc Ajanel JM, Ramírez Ramírez V, Dickey AL, Broder M. >Cómo Guatemala; 2013. Accessed September 25, 2020. https://www. Prevenir Y Vivir Sano Con La Diabetes? - Manejando La Diabetes En ine.gob.gt/sistema/uploads/2015/09/28/V3KUhMhfg El Departamento de Sololá. Hospitalito Atitlán; 2015. https:// LJ81djtDdf6H2d7eNm0sWDD.pdf hospitalitoatitlan.org/wp-content/uploads/2018/04/Diabetes- 25. Bream KDW, Breyre A, Garcia K, Calgua E, Chuc JM, Taylor L. Hospitalito-2015-Quiche-30-07-2015.pdf Diabetes prevalence in rural Indigenous Guatemala: a geographic- 41. Garcia AA, Villagomez ET, Brown SA, Kouzekanani K, Hanis CL. randomized cross-sectional analysis of risk. PLoS One. 2018;13(8): The Starr County Diabetes Education Study: development of the e0200434. CrossRef. Medline Spanish-language diabetes knowledge questionnaire. Diabetes 26. Instituto Nacional de Estadística. Guatemala: Estimaciones de la Care. 2001;24(1):16–21. CrossRef. Medline Población total por municipio. Período 2008-2020. Instituto 42. Toobert DJ, Hampson SE, Glasgow RE. The summary of diabetes self- Nacional de Estadística; 2012. Accessed September 25, 2020. care activities measure: results from 7 studies and a revised scale. http://www.oj.gob.gt/estadisticaj/reportes/poblacion-total-por- Diabetes Care. 2000;23(7):943–950. CrossRef. Medline municipio(1).pdf 27. Chary A, Greiner M, Bowers C, Rohloff P. Determining adult type 2 43. Vincent D, McEwen MM, Pasvogel A. The validity and reliability of a diabetes-related health care needs in an indigenous population from Spanish version of the summary of diabetes self-care activities ques- Nurs Res – rural Guatemala: a mixed-methods preliminary study. BMC Health tionnaire. . 2008;57(2):101 106. CrossRef. Medline Serv Res. 2012;12(1):476. CrossRef. Medline 44. Bangor A, Kortum P, Miller J. Determining what individual SUS J Usability Stud 28. International Diabetes Federation. Spotlight on Guatemala. scores mean: adding an adjective rating scale. . – International Diabetes Federation; 2012. Accessed April 16, 2017. 2009;4(3):114 123. Accessed September 25, 2020. https:// https://web.archive.org/web/20170424123524/https://www. uxpajournal.org/determining-what-individual-sus-scores-mean- idf.org/spotlight-guatemala adding-an-adjective-rating-scale/ 29. Feldman N. Stanford physicians work to protect children’s health in 45. Little RR, Rohlfing CL, Sacks DB; National Glycohemoglobin Guatemala. Stanford News. June 17, 2016. Accessed November 1, Standardization Program (NGSP) Steering Committee. Status of he- 2017. https://news.stanford.edu/2016/06/17/stanford- moglobin A1c measurement and goals for improvement: from chaos physicians-use-innovation-protect-childrens-health-guatemala/ to order for improving diabetes care. Clin Chem. 2011;57(2):205– 214. CrossRef. Medline 30. International Diabetes Federation Guideline Development Group. Global guideline for type 2 diabetes. Diabetes Res Clin Pract. 46. Little RR, Rohlfing CL. The long and winding road to optimal HbA1c 2014;104(1):1–52. CrossRef. Medline measurement. Clin Chim Acta. 2013;418:63–71. CrossRef. Medline 31. American Diabetes Association. Standards of Medical Care in 47. Khetan AK, Purushothaman R, Chami T, et al. The effectiveness of Diabetes—2017. Diabetes Care. 2017;40(Supplement 1). http:// community health workers for CVD prevention in LMIC. Glob Heart. care.diabetesjournals.org/content/diacare/suppl/2016/12/15/ 2017;12(3):233–243.e6. CrossRef. Medline 40.Supplement_1.DC1/DC_40_S1_final.pdf 48. Ruddock JS, Poindexter M, Gary-Webb TL, Walker EA, Davis NJ. 32. World Health Organization (WHO). WHO PEN Protocol 1: Innovative strategies to improve diabetes outcomes in disadvantaged prevention of heart attacks, strokes and kidney disease through populations. Diabet Med. 2016;33(6):723–733. CrossRef. Medline integrated management of diabetes and hypertension. WHO; 2013. 49. Hunter DJ, Reddy KS. Noncommunicable diseases. N Engl J Med. Accessed September 25, 2020. http://www.who.int/ncds/ 2013;369(14):1336–1343. CrossRef. Medline management/Protocol1_HeartAttack_strokes_kidneyDisease.pdf 33. Flood D, Mux S, Martinez B, et al. Implementation and outcomes of a 50. Triggs R. Need for speed: how much faster are modern phone pro- comprehensive type 2 diabetes program in rural Guatemala. PLoS cessors. Published June 15, 2019. Accessed August 13, 2020. One. 2016;11(9):e0161152. CrossRef. Medline https://www.androidauthority.com/smartphone-benchmarks-by- age-995777/ 34. Motta LA, Shephard MDS, Brink J, Lawson S, Rheeder P. Point-of- care testing improves diabetes management in a primary care clinic 51. Orton M, Agarwal S, Muhoza P, Vasudevan L, Vu A. Strengthening Glob Health Sci in South Africa. Prim Care Diabetes. 2017;11(3):248–253. delivery of health services using digital devices. Pract – CrossRef. Medline . 2018;6(Suppl 1):S61 S71. CrossRef. Medline 35. Garber AJ, Abrahamson MJ, Barzilay JI, et al. Consensus statement 52. Agarwal S, LeFevre AE, Lee J, et al. WHO mHealth Technical by the American Association of Clinical Endocrinologists and Evidence Review Group. Guidelines for reporting of health interven- American College of Endocrinology on the comprehensive type 2 tions using mobile phones: mobile health (mHealth) evidence report- diabetes management algorithm—2017 executive summary. Endocr ing and assessment (mERA) checklist. BMJ. 2016;352:i1174. Pract. 2017;23(2):207–238. CrossRef. Medline CrossRef. Medline 36. StatCounter Global Stats. Mobile operating system market share 53. World Health Organization (WHO). WHO Guideline: Guatemala. StatCounter; 2019. Accessed February 9, 2020. http:// Recommendations on Digital Interventions for Health System gs.statcounter.com/os-market-share/mobile/guatemala Strengthening. WHO; 2019. Accessed September 25, 2020. Global Health: Science and Practice 2020 | Volume 8 | Number 4 718 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org https://apps.who.int/iris/bitstream/handle/10665/311941/ in patients with type 2 diabetes mellitus treated with sulfonylureas: a 9789241550505-eng.pdf systematic review and meta-analysis. Diabetes Metab Res Rev. – 54. Frost MJ, Tran JB, Khatun F, Friberg IK, Rodríguez DC. What does it 2014;30(1):11 22. CrossRef. Medline take to be an effective national steward of digital health integration for 64. Douros A, Yin H, Yu OHY, Filion KB, Azoulay L, Suissa S. health systems strengthening in low- and middle-income countries? Pharmacologic differences of sulfonylureas and the risk of adverse Glob Health Sci Pract. 2018;6(Suppl 1):S18–S28. CrossRef. Medline cardiovascular and hypoglycemic events. Diabetes Care. 2017;40 – 55. Dimagi Inc. TulaSalud: CommCare for Improving and Monitoring (11):1506 1513. CrossRef. Medline Community Health. Dimagi Inc. https://www.dimagi.com/case- 65. UK Hypoglycaemia Study Group. Risk of hypoglycaemia in types 1 studies/tulasalud-mnch/ and 2 diabetes: effects of treatment modalities and their duration. Diabetologia – 56. Browning C, Chapman A, Yang H, et al. Management of type 2 di- . 2007;50(6):1140 1147. CrossRef. Medline abetes in China: the Happy Life Club, a pragmatic cluster rando- 66. Micikas M, Foster J, Weis A, et al. A community health worker inter- mised controlled trial using health coaches. BMJ Open. 2016;6(3): vention for diabetes self-management among the Tz’utujil Maya of e009319. CrossRef. Medline Guatemala. Health Promot Pract. 2015;16(4):601–608. CrossRef. 57. Flood D, Hawkins J, Rohloff P. A home-based type 2 diabetes self- Medline management intervention in rural Guatemala. Prev Chronic Dis. 67. Afroz A, Alramadan MJ, Hossain MN, et al. Cost-of-illness of type 2 2017;14:170052. CrossRef. Medline diabetes mellitus in low and lower-middle income countries: a sys- BMC Health Serv Res 58. Sola D, Rossi L, Schianca GPC, et al. State of the art paper sulfony- tematic review. . 2018;18(1):972. CrossRef. lureas and their use in clinical practice. Arch Med Sci. 2015;4 Medline – (4):840 848. CrossRef. Medline 68. Radin MS. Pitfalls in hemoglobin A1c measurement: when results 59. McCreight LJ, Bailey CJ, Pearson ER. Metformin and the gastrointes- may be misleading. J Gen Intern Med. 2014;29(2):388–394. tinal tract. Diabetologia. 2016;59(3):426–435. CrossRef. Medline CrossRef. Medline 60. Clemens KK, McArthur E, Dixon SN, Fleet JL, Hramiak I, Garg AX. 69. Williams TN, Weatherall DJ. World distribution, population genet- The hypoglycemic risk of glyburide (glibenclamide) compared with ics, and health burden of the hemoglobinopathies. Cold Spring Harb modified-release gliclazide. Can J Diabetes. 2015;39(4):308–316. Perspect Med. 2012;2(9):a011692. CrossRef. Medline CrossRef. Medline 70. Mujica-Coopman MF, Brito A, López de Romaña D, Ríos-Castillo I, 61. van Dalem J, Brouwers MCGJ, Stehouwer CDA, et al. Risk of hypogly- Cori H, Olivares M. Prevalence of anemia in Latin America and the caemia in users of sulphonylureas compared with metformin in relation Caribbean. Food Nutr Bull. 2015;36(2_suppl)(Suppl):S119–S128. to renal function and sulphonylurea metabolite group: population CrossRef. Medline BMJ based cohort study. . 2016;354:i3625. CrossRef. Medline 71. Baker P, Dworkin SL, Tong S, Banks I, Shand T, Yamey G. The men’s 62. Leonard CE, Han X, Brensinger CM, et al. Comparative risk of serious health gap: men must be included in the global health equity agenda. hypoglycemia with oral antidiabetic monotherapy: a retrospective Bull World Health Organ. 2014;92(8):618–620. CrossRef. Medline Pharmacoepidemiol Drug Saf – cohort study. . 2018;27(1):9 18. 72. Parving HH, Lewis JB, Ravid M, Remuzzi G, Hunsicker LG; DEMAND CrossRef. Medline investigators. Prevalence and risk factors for microalbuminuria in a 63. Schopman JE, Simon ACR, Hoefnagel SJM, Hoekstra JBL, Scholten referred cohort of type II diabetic patients: a global perspective. RJPM, Holleman F. The incidence of mild and severe hypoglycaemia Kidney Int. 2006;69(11):2057–2063. CrossRef. Medline En español Usando Trabajadores Comunitarios de la Salud y una Aplicación de Smartphone para Mejorar el Control de la Diabetes en una Zona Rural de Guatemala Hallazgos claves Una aplicación para smartphones que brinda apoyo algorítmico a las decisiones clínicas permitió a los trabajadores comunitarios de salud mejorar el control de la diabetes en un grupo de pacientes en una zona rural de Guatemala. Implicaciones claves Los administradores de programas deben considerar equipar a los trabajadores comunitarios de salud con aplicaciones de apoyo a la toma de decisiones clínicas para permitir el compartir de tareas para el manejo de enfermedades crónicas. Los investigadores deben examinar la eficacia de esta estrategia para enfermedades crónicas distintas de la diabetes y compararlo con los modelos tradicionales de cuidado médico. Resumen Antecedentes: La prevalencia mundial de diabetes casi se ha duplicado desde 1980. Setenta y cinco por ciento de los pacientes con diabetes viven en países de ingresos bajos y medianos, como Guatemala, donde los sistemas de atención médica a menudo están mal equipados para el manejo de enfermedades crónicas. Los trabajadores comunitarios de la salud (TCS) y la tecnología de salud móvil se han aplicado cada vez más a la epidemia de diabetes en estos entornos, aunque principalmente en funciones de apoyo más que en el manejo directo de la diabetes. Buscamos mejorar la atención de la diabetes en las zonas rurales de Guatemala mediante el desarrollo de un programa de diabetes dirigido por los TCS y una aplicación para smartphones para brindarles apoyo en la toma de decisiones clínicas. Métodos: Trabajamos con nuestros socios locales para desarrollar un modelo de programa y la aplicación para smartphones (utilizando la plataforma CommCare) y para capacitar a los TCS. Reclutamos pacientes con diabetes tipo 2 que vivían en comunidades rurales. La evaluación del programa utilizó un diseño pre-post de un solo grupo. Los resultados primarios fueron la hemoglobina A1c y el porcentaje de pacientes que alcanzaron los objetivos de A1c en comparación con el valor inicial. También seguimos una variedad de métricas de procesos, incluyendo la confiabilidad de la aplicación. Global Health: Science and Practice 2020 | Volume 8 | Number 4 719 Task Sharing Using mHealth to Improve Diabetes Control in Rural Guatemala www.ghspjournal.org Resultados: Ochenta y nueve pacientes fueron inscritos durante el período de estudio. El porcentaje de hemoglobina A1c disminuyó significativamente a los 3 meses ( 1,0; IC del 95%: 1,7 a 0,6), 6 meses ( 1,5; IC del 95%: 2,2 a 0,8), 9 meses ( 1,3; IC del 95%: 2,0 a 0,6) y 12 meses ( 1,0; IC del 95%: 1,7 a 0,4). El porcentaje de pacientes con A1c 8% aumentó significativamente a los 3 meses (23,6% a 44,4%, P= 0,007), 6 meses (22,0% a 44,0%, P= 0,015) y 9 meses (23,9% a 45,7%, P= 0,03). Los TCS y los médicos supervisores estuvieron de acuerdo con las recomen- daciones de la aplicación para el uso de medicamentos más que el 90% del tiempo. Conclusión: Nuestros resultados sugieren que los TCS pueden manejar la diabetes de forma segura y eficaz con la ayuda de una aplicación para smartphones y la supervisión médica remota. Este modelo debe evaluarse frente a otros estándares de cuidado médico y podría adaptarse a otros entornos de escasos recursos otras enfermedades crónicas. Peer Reviewed Received: February 13, 2020; Accepted: September 1, 2020; First published online: December 4, 2020 Cite this article as: Duffy S, Norton D, Kelly M, et al. Using community health workers and a smartphone application to improve diabetes control in rural Guatemala. Glob Health Sci Pract. 2020;8(4):699-720. https://doi.org/10.9745/GHSP-D-20-00076 © Duffy et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https://doi.org/ 10.9745/GHSP-D-20-00076 Global Health: Science and Practice 2020 | Volume 8 | Number 4 720 ORIGINAL ARTICLE Behavioral Insights Into Micronutrient Powder Use for Childhood Anemia in Arequipa, Peru Jessica D. Brewer,a Julianna Shinnick,b Karina Román,c,d Maria P. Santos,a Valerie A. Paz-Soldan,a,d,e Alison M. Buttenheimb,f Key Findings Resumen en español al final del artículo. n Negative experiences with health care providers or inconvenience at the time that micronutrient ABSTRACT powder (MNP) use is initiated may discourage Childhood anemia remains a significant driver of morbidity in future MNP use. low- and middle-income countries, including Peru. To identify be- havioral challenges to using micronutrient powder (MNP) that is n Mental models about nutrition can shape given to supplement children’s diets and prevent anemia, we ap- intentions to use MNP, and having too many plied a behavioral design approach to interviews and focus choices can confuse caregivers. groups with 129 caregivers in Arequipa, Peru. We examined n A single negative experience with MNP can form 3 key points in the decision-making process: accessing MNP strong memories and discourage caregivers from through the health system; forming intentions to use MNP; and giving MNP. MNP use at the time of child feeding. Using the NUDGE (Narrow, Understand, Discover, Generate, Evaluate) approach, Key Implications we identified the following behavioral barriers and facilitators: (1) caregivers’ experiences with health care providers shaped n Training for health care providers should their motivation to access MNP; (2) caregivers felt accessing encourage positive interpersonal interaction with MNP at clinics was inconvenient and created hassle factors; caregivers during initiation of MNP because these (3) caregivers’ mental models about anemia prevention shaped interactions can have a lasting impact on MNP MNP intentions and use; (4) caregivers’ salient negative experiences use. couldhavecausedthemtostopgivingMNP;(5)caregiversforgotto n Education for caregivers should include give MNP if they did not have cues to remind them but could be counseling about potentially challenging side prompted with salient cues; and (6) caregivers were affected by effects so that caregivers are prepared to work emotional, cognitive, and attentional factors during feeding that through them. were difficult to anticipate. Our results, based on a behavioral de- n In future programming, public health practitioners sign approach, suggest opportunities to adapt current messaging, should consider encouraging caretakers to utilize counseling, and education around MNP use. Adaptations include well-timed cues to administer MNP. providing culturally relevant messages, leveraging caregivers’ emo- tional and cognitive states, and encouraging small but impactful changestofeedingroutinestoaddressbarrierstoMNPuse. INTRODUCTION a Department of Global Community Health and Behavioral Sciences, Tulane nemia in children can impair cognitive and motor University School of Public Health and Tropical Medicine, New Orleans, LA, Afunction and cause fatigue and poor school perfor- USA. 1–2 b Department of Family and Community Health, University of Pennsylvania mance. It is a significant public health issue, particularly 3–4 School of Nursing, Philadelphia, PA, USA. affecting low- and middle-income countries (LMICs). c Department of Health Management, Universidad Peruana Cayetano Heredia InPeru,prevalenceofanemiaamongchildrenaged Facultad de Salud Pública y Administración Carlos Vidal Layseca, Lima, Peru. 6 months to 3 years was 43.6% in 2017.5 To combat d Asociación Benéfica PRISMA, Lima, Peru. e Zoonotic Disease Research Lab, Universidad Peruana Cayetano Heredia this high prevalence, in 2014, the Peruvian Ministry of Facultad de Salud Pública y Administración Carlos Vidal Layseca, Arequipa, Health began distributing free micronutrient powders Peru. (MNPs) (or “chispitas”) to children aged 6 months to f Center for Health Incentives and Behavioral Economics, University of Pennsylvania, Philadelphia, PA, USA. 3 years at public health care facilities during well child Correspondence to Jessica D. Brewer ([email protected]). checkups, where caregivers receive guidance from health Global Health: Science and Practice 2020 | Volume 8 | Number 4 721 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org care providers on how to apply MNP to children’s literature are that humans heavily rely on “rules meals.6–7 MNPs are single-dose sachets formulated of thumb” and mental shortcuts to make deci- typically with iron, zinc, folic acid, vitamin A, and vi- sions, are given imperfect information, have tamin C that are consumed by mixing with semisol- time-inconsistent preferences, and have atten- id food.8–9 They are proven to be an efficacious tional and cognition constraints. These decisions intervention for early childhood anemia, and the are often not in their best long-term interest but World Health Organization recommends their use satisfy their immediate needs and desires. in populations where the prevalence of early child- Bringing a behavioral economics perspective hood anemia is 20% or higher.8,10 According to to the analysis of uptake of public health programs Peru’s national guidelines, caregivers are instructed can help identify specific barriers to and facilitators to use MNP daily for 1 year after beginning the first of target behaviors that are not captured by other dose, ideally starting at age 6 months with the intro- approaches. We define behavioral barriers as duction of complementary foods.11–12 those factors arising from cognitive or psychologi- Despite Peru Despite these efforts, childhood anemia preva- cal processes that reduce the likelihood of a target 5 having national lence in Peru has remained high, and an early behavior being carried out; behavioral facilitators 32–34 guidelines on evaluation of national rollout of the MNP program similarly increase that likelihood. Behavioral 7 MNP use, showed low adherence. Previous research on barriers and facilitators often operate separately from conscious cognition or awareness; one impli- adherence is low MNP use in Peru has found that confusion about ’ cation of this is that people’s statements about and child anemia MNP administration, MNP s unpleasant taste, their intentions, motivations, and decisions around prevalence side effects, lack of familial and peer support, and abehavior—particularly a complex or habitual remains high. negative interactions with those who distribute MNP were barriers to adherence.7,13–15 Alterna- behavior—may paint an incomplete picture of that tively, key facilitators for MNP use were interper- behavior’s context. In recent years, innovative sonal support, concern about the long-term effects methods have emerged that map contextual data of anemia, and tailored counseling.7,13–15 Studies about a behavior (including field observations; on MNP use in other countries confirm that these interviews and focus groups with participants, factors affect MNP program effectiveness.16–23 stakeholders, and experts; and existing quantitative Previous research on MNP interventions have led evidence and prior literature) to specific behavioral to programmatic changes in other countries such economicsprinciplestouncovernovelinsights as health care providers including warnings about about barriers and facilitators that can inform inter- possible side effects in their counseling to care- vention design. These methods and approaches have been widely used in a variety of global health givers, recommendations to administer MNP on a 32,33,35,36 flexible instead of fixed schedule, and promotion settings and programmatic domains and in previous work on food choices and human through educational campaigns with community 37–40 health providers, among others.24–28 nutrition. Because the provision of MNP is an active in- tervention that requires multiple steps, sustained Behavioral Economics and Intervention action over time, and the translation of intentions Behavioral Design into behavior, behavioral economics may offer economics Several studies have identified social, psychologi- novel insights into low adherence to MNP use de- analysis may offer cal, and environmental factors that inhibit or en- spite its availability and promotion. Interventions novel insights into able MNP use; we extend that research here with informed by behavioral economics have been low adherence to an applied behavioral design approach, informed used successfully in prior studies to improve ma- by behavioral economics, to understand the be- MNP use despite ternal and child nutrition, from simple changes to havioral processes at play in MNP use. Behavioral its availability and the layout of school cafeterias and providing economics—a field that sits at the intersection of promotion. verbal cues for healthier choices in the United economics and psychology—seeks to understand States,41–42 to incentives and reminders to buy how common mental biases, heuristic thinking, healthy foods in Madagascar that are designed and social forces shape decision making and be- to address specific behavioral barriers (for exam- – havior.29 31 A rich theoretical and empirical col- ple, incentives to address procrastination and lection of literature from behavioral economics present-orientation, and stickers that deliver sa- and related disciplines describes and characterizes lient reminders at the point of purchase and how decision making often deviates from what ra- consumption.).43 tional actor or expected utility models would pre- In this article, we advance our understanding of dict. Consistent findings in this interdisciplinary behavioral barriers to and facilitators of consistent Global Health: Science and Practice 2020 | Volume 8 | Number 4 722 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org MNP use for anemia prevention in Peru. Using for the present analysis. More details about the previously collected contextual inquiry data, we data from the parent study are available from applied a behavioral design approach to uncover Brewer et al.15 novel insights about caregiver choices and actions related to giving MNP. These insights can inform Analytical Approach counseling techniques used in MNP programs. We used the NUDGE (Narrow, Understand, Discover, Generate, Evaluate) approach to analyze behavioral METHODS barriers to and facilitators of MNP use among care- givers. NUDGE was developed to support the sys- Setting tematic and rigorous application of behavioral We conducted our study in Arequipa, the second economics insights to intervention design,45 and is largest city in Peru, which has particularly high one of several published design approaches in- 5 rates of childhood anemia. Despite ongoing formed by behavioral economics and design think- efforts to address anemia, 44.5% of children aged ing.33,46 The use of the term “nudge” is intentional; 6 months to 3 years in Arequipa were diagnosed the approach generates intervention designs that 44 with anemia in 2016. The study was con- are consistent with Sunstein and Thaler’sdefinition ducted in 8 of 29 districts in Arequipa, which of nudges31: accounted for more than half of the cases of ear- ly childhood anemia in the province according any aspect of the choice architecture that alters people's to unpublished sources from the local branch of behavior in a predictable way without forbidding the Ministry of Health. any options or significantly changing their economic incentives. Data NUDGE includes 5 stages: In 2017, we conducted 24 interviews and 12 focus 1. Narrow the focus of the analysis to a specific, groups with caregivers of children aged 6 months relevant behavioral target to 3 years. Caregivers were defined as adults who self-reported spending at least 5 days a week pro- 2. Understand the context of the behavior viding care for the child, whether they were the through inquiry into the decision-making child’s biological parent or otherwise. This inclu- process and related actions sion criterion was established as the only one for 3. Discover insights about barriers to and facilita- recruiting caregivers as we believe the primary tors of the target behavior through structured caregiver is typically in charge of child feeding matching of elements from contextual under- and thus the administration, or lack thereof, of standing developed in Stage 2 to core princi- MNP and so that we could obtain a range of care- ples (cognitive biases and heuristic thinking) giver experiences related to gender, caregiver age, from behavioral economics child age, child history of anemia, caregiver-child 4. Generate intervention strategies and designs relationship, and other factors. Caregivers were to address identified barriers selected for interviews via convenience sampling 5. Evaluate those designs through iterative pro- in and around local health establishments and se- totyping and trialing lected for focus groups through door-to-door re- cruitment in the neighborhoods surrounding the In this article, we report the results from the health establishment. The interview and focus Narrow, Understand, and Discover stages (Figure). group guides (Supplement 1) were developed to Building on a previous analysis utilizing the probe for caregiver experiences in obtaining MNP social-ecological model to identify factors that from health care providers and applying MNP to inhibited and enabled MNP use from Brewer children’s meals, as well as other beliefs about et al.,15 we narrowed our point of inquiry to a de- anemia treatment and prevention. Interviews and fined behavioral target: the regular use of MNP focus groups were conducted by authors JDB, KR, during child feeding. We developed a rich under- and MPS in Spanish, audio recorded, and tran- standing of the context around MNP use through scribed in Spanish. In total, we conducted individu- repeated reading of focus group and interview al interviews with 24 caregivers and 12 focus data. To discover relevant behavioral insights, we groups with 4 to 13 caregivers each, resulting in a first identified the key decisions and actions under- total of 129 caregiver participants. The study team lying the target behavior. Next, using a set of analyzed a subset of data from this parent study prompts about the cues, meanings, and alternatives Global Health: Science and Practice 2020 | Volume 8 | Number 4 723 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org FIGURE. NUDGE Approach as Applied to Analyzing Behavioral Barriers to and Facilitators of Micronutrient Powder Use Among Caregivers Abbreviation: NUDGE, narrow, understand, discover, generate, evaluate. related to decision and action steps, we brain- insight about barriers and facilitators for the target stormed barriers to or facilitators of each step. behavior. Three examples of how a barrier or facil- Each barrier linked the contextual understanding itator is discovered from contextual inquiry and be- developed in the previous stage with 1 or more spe- havioral constructs are shown in the Table. cific behavioral economics constructs (e.g., avail- This process yielded 121 barriers and facilitators ability heuristic or present bias) to discover an related to MNP use, which were de-duplicated to TABLE. Summary of Discovery of Behavioral Barriers/Facilitators to Micronutrient Powder Use Among Caregivers in Peru Prompt (Cue, Action, or Behavioral Decision-making Step Meaning) Contextual Factor Construct Barrier or Facilitator Accessing MNP Cue: The perspectives of Caregivers are more likely to Authority bias Negative interactions with medical people with authority on make decisions about their professionals can cause caregivers MNP hold greater weight children’s health when an to not give MNP. expert gives them the information. Using MNP at the moment Meaning: Is the action Children often react negatively Negativity bias If children refuse food supplemented of child feeding uncomfortable or painful to taste of MNP, making with MNP, caregivers may stop such that it is avoided? feeding difficult or unpleasant giving it. for caregiver. Using MNP at the moment Action: Caregivers Caregivers abruptly decide to Hot-to-cold Visceral reactions when a child is of child feeding abruptly discontinue MNP stop giving MNP when their empathy gap sick can lead to a rapid choice to child has diarrhea or another discontinue MNP. side effect. Abbreviation: MNP, micronutrient powder. Global Health: Science and Practice 2020 | Volume 8 | Number 4 724 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org 68, synthesized to 21, and validated in an iterative avoided asking health care providers questions to process to ensure rigor and completeness. The vali- resolve these doubts about MNP due to social de- dation process included in-depth discussions with sirability bias or wanting to appear competent in authors KR and MPS, who are public health front of health care providers, but this was not researchers involved in original data collection but supported during the validation step based on who had not previously participated in this analytic observations made during data collection. process. This validation process resulted in the ex- In other circumstances, interactions with au- clusion of 4 barriers (and related behavioral con- thority figures could have also acted as facilitators structs) that did not align with the researchers’ for MNP use. For example, sometimes it was easier experiences in the field. The validated list of 17 bar- for caregivers to give MNP if they felt like someone riers and facilitators were further reduced to a set of else (health care providers, family members) told 6 coherent, validated barriers to and facilitators of them they had to give MNP, essentially making the target behavior that can inform intervention the decision for them. However, it should be noted design. Importantly, the results of this analytic pro- that a few caregivers were uncomfortable with cess are at the level of the barrier to or facilitator of health care providers presenting MNP as a re- the focal behavior; while each barrier/facilitator quirement, especially if they already doubted its is informed by 1 or more behavioral constructs, quality or felt like they had not been given expla- the behavioral constructs themselves are not the nations for the reasons to use it. Additionally, results of the analysis. caregivers who felt they had received good infor- mation from authority figures felt more confident RESULTS about using MNP. For example, caregivers expressed We identified 6 behavioral barriers to and facilita- satisfaction when health care providers took the time tors of MNP use that operate at 3 key points along to explain MNP to them in depth and address their the intention-to-behavior continuum: (1) acces- doubts. Finally, framing effects (how health care pro- sing MNP through the health system, (2) forming viders framed anemia to caregivers) affected their an intention to use MNP, and (3) using MNP at the likelihood of administering MNP. For example, care- Caregivers were moment of child feeding. Examples of qualitative givers reported they were more likely to give MNP if more likely to give ’ data illustrative of and supporting the behavioral they were told how anemia could affect their child s MNP if they were constructs underlying each barrier/facilitator are brain and development, whereas being told their told how anemia “ ” available in Supplement 2. child had low hemoglobin was confusing and did could affect their not instill a sense of urgency. child’sbrainand Accessing MNP Through the Health System development. 1. Caregivers’ Experiences With Health Care 2. Caregivers Felt Accessing MNP at Clinics Was Providers Shaped Their Motivation to Access MNP Inconvenient and Created Hassle Factors Caregivers accessed MNP and received counseling Even the smallest amount of friction or hassle re- about its use at health clinics during well child duced the probability that caregivers would access checkups. Because health care providers were MNP or seek information on its use. If caregivers seen as authority figures, the emotional valence were required to attend informational sessions at of these interactions was influential. Some care- inconvenient times or knew they would have to givers reported that negative interactions with wait in long lines for a well child checkup (where health care providers (e.g., feeling dismissed, con- they received MNP and counseling), they may descended to, rushed, shamed) made them reluc- have been reluctant to make the visit. We original- tant to return to health establishments. Negative ly also posited that once caregivers already had interactions may have also dissuaded caregivers MNP at home, they were more likely to give it to from taking the advice about MNP given by that their child, an example of endowment effect, a professional at the visit. Unclear or contradictory psychological phenomenon where people are Unclear or information from health care providers about more likely to keep something they already have contradictory MNP and its use could have led to ambiguity aver- than make the effort to obtain it. However, this ef- information from sion. Some caregivers avoided using MNP if they fect was not validated by other researchers given providers about felt they lacked sufficient information about it, es- that caregivers were given an exact amount of MNP could have pecially if they were unsure about its potential MNP to last them between checkups (approxi- led to ambiguity harms. They expressed uncertainty about both mately 90 sachets) and thus would not have had aversion. effectiveness and administration of MNP. We orig- any extra that would allow for prolonged use. inally posited that caregivers might also have Losing MNP sachets or other problems related to Global Health: Science and Practice 2020 | Volume 8 | Number 4 725 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org the number of sachets received was not a salient continue to use MNP. Negative side effects like di- theme in our analysis. arrhea were immediately evident to caregivers and remained in their memories, compared to Forming an Intention to Use MNP positive effects like higher hemoglobin levels 3. Caregivers’ Mental Models About Anemia which were invisible to the caregiver and had ’ Prevention Shaped MNP Intentions and Use more gradual, long-term effects on the child s Caregivers’ mental models about nutrition, how health. Anecdotal fallacy and base rate neglect MNP works, and what could serve as a substitute also occurred when caregivers gave higher weight for MNP directly shaped intentions to use MNP. to a few salient stories from their peers about trou- Caregivers who perceived multiple ways to treat blesome MNP side effects, as opposed to following ’ Caregivers or prevent anemia, such as a variety of over-the- medical advice from health care professionals. preferences to counter medications, may have experienced choice This over-anchoring on negative effects reduced treat anemia overload and may have looked for simplifying intentions to use MNP going forward. During the through diet heuristics to choose among known alternatives, validation phase, other researchers agreed that reflected their such as salience, familiarity, or ease. Some care- caregivers disproportionately focused on the neg- preference for a givers expressed the belief that over-the-counter ative effects of MNP, especially related to side “natural” solution medications were of higher quality than MNP, giv- effects and taste. over a en that they were distributed by pharmacies (in- pharmacological stead of the public health system) and had a one. monetary cost (as opposed to free distribution). Using MNP at the Moment of Child Feeding Caregivers’ preferences to treat anemia through 5. Caregivers Forgot to Give MNP if They Did Not diet reflected the mental model (sustained primari- Have Cues to Remind Them but Could Be ly by peer or family advice but also recommended Prompted With Salient Cues by health care providers) that a “natural” solution Even when individuals intend to do something was preferable to a pharmacological one. Ad- and have the resources to do it, they often require If the social, ditionally, caregivers often defaulted to the use of a specific prompt from the environment to over- physical, or media food, a traditional and automatic response to treat- come inertia. If the social, physical, or media envi- environment ing illness, prompted by its presence in the home. A ronment failed to cue MNP use at the right time failed to cue MNP few caregivers expressed distrust of MNP because and in the right way, caregivers may have of its manufacture in India, which we hypothesized use at the right defaulted to nonuse. For example, some care- may have reflected a possible “not invented here” time and in the givers did not feel prompted to use MNP if their bias that may have limited regular use. During the right way, child appeared to be healthy, even if the child had validation process, other researchers concluded caregivers may been diagnosed with anemia. It may have been that this barrier may not have been as salient as “ ” have defaulted to possible that they experienced ostrich effect, or other heuristics given its less frequent occurrence nonuse. unwillingness to accept this diagnosis for fear of in the data. However, other mental models that dealing with the repercussions. The authors in- drew analogies between MNP and something volved in data analysis originally posited that if more familiar could have facilitated MNP use. caregivers did not see others in their communities Caregivers who described MNP as being “like vita- using MNP, they may have assumed that others ” mins (versus medication) appeared more likely to did not approve of MNP or simply that not seeing have favorable views of MNP and to feel comfort- peers use MNP could have failed to cue MNP use. able using it. This was not supported during validation. Another early proposed barrier was that working or busy 4. Caregivers’ Salient Negative Experiences caregivers who bought prepared food instead of Could Have Caused Them to Stop Giving MNP cooking homemade meals were less likely to add Caregivers accumulated positive and negative MNP to the purchased food because they did not experiences, both personal and secondhand, go through the process of preparing and serving about MNP. However, negativity bias led them to the food themselves (cue-dependent forgetting); pay attention to and remember the negative however, this was eliminated during validation experiences more. For example, caregivers who given that the relevant data referred to general nu- experienced frustrations with MNP use in the trition practices, not MNP use. Alternatively, sa- past, due to the child experiencing side effects like lient, well-timed cues from the environment could diarrhea or refusing to eat foods with MNP be- have promoted MNP use. Caregivers strongly sug- cause of its taste, may have lost their intention to gested that additional information about MNP on Global Health: Science and Practice 2020 | Volume 8 | Number 4 726 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org mass media, particularly television and radio, could giving MNP; (5) caregivers forgot to give MNP if they have been an effective cue. did not have cues to remind them, but could be prompted with salient cues; and (6) caregivers were 6. Caregivers Were Affected by Emotional, affected by emotional, cognitive, and attentional fac- Cognitive, and Attentional Factors During tors during feeding that were difficult to anticipate. Feeding That Were Difficult to Anticipate Our results support and extend previous findings in The challenges of using MNP in the moment dur- the literature and provide opportunities for designing ing child feeding could have led to procrastination and revising program interventions that incorporate and avoidance, exacerbated by the hot-cold empa- the behavioral constructs underlying the barriers. thy gap. When caregivers were at the clinic and First, as in other studies, we found that author- decided to use MNP, they were in a deliberative ity figures including health care providers have an “ ” opportunity to influence uptake and ongoing use and rational or cold state. This made it difficult 7,13,17,19 to envision what it would have been like to apply of MNP. Our results place unique empha- MNP at mealtime, when caregivers were in an ag- sis on the emotional context of these interactions. itated, cognitively taxed “hot state” due to child Caregivers had different perspectives on what experiences with side effects or dislike of taste. constituted a positive or negative interaction with The counseling that caregivers received at well health care providers; some preferred an authori- tative approach and others preferred collaborative child checkups did not acknowledge this gap or decision making. Although research in Peru and help caregivers plan for it. In the validation pro- Giving caregivers many other countries promotes the use of cultur- cess, other researchers confirmed that aversion to confusing ally appropriate counseling techniques,13,17,26 re- MNP at the moment of use led to its avoidance, es- information search on collaborative decision making between pecially if caregivers did not feel prepared to ad- health care providers and patients regarding nutri- during visits could dress any complications that may arise. tion is limited to the United States.47 Future re- have led to them Some caregivers also over-focused on specific search could examine whether authoritative or feeling they details of MNP administration, known as focusing collaborative counseling styles would be most ef- lacked sufficient effect, which made MNP easy to abandon if they fective in motivating caregivers in Peru. In addi- information on felt they could not perfectly follow the instruc- tion, as in other studies, we found that confusing MNP—reinforcing tions. Because skipping or incorrectly implement- information during consultations could have led the need to ing a step may have resulted in worse taste or side to caregivers feeling they lacked sufficient informa- simplify and tailor effects (e.g., leaving it in food for an extended pe- tion on MNP; this reinforces the need to simplify educational riod of time increased the metallic taste due to cap- and tailor the educational campaigns recom- campaigns. sule breakdown), this focus was understandable, mended by other studies.13,19,26 Our results also but it may have led to abandoning MNP adminis- highlight that framing anemia as “low hemoglobin” tration in the moment after a minor deviation reduced both salience and urgency for caregivers; a from the protocol. Alternatively, when caregivers higher salience framing in Ministry of Health cam- were removed from the process of administering paign and health care provider training materials MNP to the child, they expressed greater satisfac- could emphasize children’s growth and brain tion with MNP and its effect on their child’s development. health. This was the case for caregivers who used Second, our study highlights hassle factors as Cuna Mas, a public daycare that required them to another major barrier when accessing MNP at bring in MNP with their child so the staff could ad- clinics. Although prior studies have identified bar- minister it to the child during the day. riers to accessing health services, few connect those barriers to MNP adherence. Prior behavior DISCUSSION science research has demonstrated that even mini- We identified 6 behavioral barriers to and facilitators mal friction in a health or benefits program reduced of using MNP for anemia prevention among care- take-up.46 In the context of MNP use, hassle factors givers of young children in Arequipa, Peru. These reduced caregivers’ likelihood of accessing MNP are: (1) caregivers’ experiences with health care and receiving information on its use. Therefore, providers shaped their motivation to access MNP; our results point to the need for structural changes (2) caregivers felt accessing MNP at clinics was incon- within the health system (more staff available for venient and created hassle factors; (3) caregivers’ appointments, creating appointment and informa- mental models about anemia prevention shaped tional session schedules outside of caregivers’ work MNP intentions and use; (4) caregivers’ salient nega- hours, and increasing access to MNP sachet “refills” tive experiences could have caused them to stop in community settings). Cuna Más, a public daycare Global Health: Science and Practice 2020 | Volume 8 | Number 4 727 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org system in Peru, and community health promoters empathy gap is particularly relevant for understand- are trusted sources that could expand access to MNP. ing what happened when caregivers intended or Our third and fourth results concerned form- planned to use MNP during child feeding but aban- ing an intention to use MNP that we demonstrated doned that plan in the moment when children was shaped by mental models about nutrition and resisted, refused, reacted strongly to taste, or ex- by negative experiences with MNP. Caregivers perienced side effects such as diarrhea, all of may have had a preference for addressing anemia which are commonly cited barriers in the litera- through diet, a more familiar and less “medical- ture.7,13,15,18,19,55–59 Awareness of the hot-cold ized” approach than MNP. This default preference empathy gap can provide a channel for improved for dietary approaches may have led caregivers to program development. Given that people gener- attend to and implement dietary suggestions rath- ally don’t understand their actions as “state- er than use MNP even when health care providers dependent,” a possible intervention involves made both recommendations. Our results also preparing caregivers for side effects that might revealed that a way to frame MNP to align with cause anxiety. Health care professionals could the caregiver mental models about nutrition was encourage caregivers to think through how they as a vitamin supplement rather than as a medica- would react to a stressful experience while in a tion. Negative experiences with MNP, such as side “cold state” at their medical appointment. This effects or bad taste, and the effect of negative com- strategy is based on evidence from the side effect ments from family and peers, have all been noted reduction literature, often focused on cancer 7,13,15,17–19,22 in previous studies. Although previ- patients, that suggests that preparation for side ous work has typically interpreted negative com- effects can reduce anticipatory symptoms and ments as lack of social support for MNP use and stress and improve coping skills.60–61 This ap- therefore proposed increased informational out- proach would represent a departure from current 17 reach to family members and peers as a solution, Peruvian Ministry of Health trainings that teach our results highlight the importance of including health care providers to counsel caregivers that specific behavioral guidance to family and peers MNP has no side effects.13 to not to overemphasize prior complications and negative experiences. Limitations For any behavior that is new, challenging, and There are several limitations to this study. Brewer not yet habitual, it’s easy to procrastinate. In our et al. describes the limitations in study design and fifth result, we confirmed a common finding from data collection, such as recruitment of participants previous studies that caregivers needed external in and around health centers, the reliance on self- cues to overcome procrastination around MNP reported (rather than observed) barriers to MNP use at mealtime. General prompts in the form use, and the lack of systematic collection of socio- of television and radio spots were useful, and demographic and behavioral characteristics of the they have been used in Peru by Ministry of caregivers and their children (such as time using Health and other organizations to effectively pro- MNP, birth order, etc.).15 Adopting a behavioral mote anemia-specific48 and other positive health – economics perspective limits the identified bar- behaviors.49 50 Our approach also uncovered the riers and facilitators to those with a specific behav- importance of specific, timely, unavoidable cues ioral (as opposed to structural) underpinning. at the moment of child feeding, which may drive The hot-cold Additionally, the behavioral barriers to optimal behavior change more than a TV or radio spot empathy gap MNP use identified in this analysis require further heard earlier in the day. Possible interventions in- helps us confirmation through empirical testing of inter- formed by this insight include encouraging care- understand what ventions designed to address them. Although givers to store MNP sachets with other items that NUDGE is similar to several other approaches us- happened when will be used during mealtime (i.e., with the child’s ing behavioral economics and human-centered caregivers dish or utensils) or sending an SMS message re- design, it is still evolving as an analytic tool and fu- planned to use minder at common mealtimes. ture iterations may further refine the approach. MNP but Finally, we identified “hot-cold empathy gap” abandoned that as a barrier to consistent MNP use. Hot-cold empa- plan when they thy gap has been observed in other health beha- CONCLUSION experienced viors, in which people consistently fail to imagine This study uses behavioral economics and a be- barrierssuchasa and account for what a future “hot” affective or havioral design approach to understand MNP ad- child’s negative cognitive state will be when a plan to act is made ministration and childhood anemia prevention – reaction. ahead of time in a “cold” state.51 54 Hot-cold generally. This approach to analyzing cognitive Global Health: Science and Practice 2020 | Volume 8 | Number 4 728 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org biases and heuristics can generate insights into be- Ministerio de Salud. Accessed October 19, 2020. http://bvs.minsa. havioral influences on adherence that comple- gob.pe/local/MINSA/3933.pdf ment existing approaches to identifying barriers 7. Munares-García O, Gómez-Guizado G. Adherencia a multimicro- nutrientes y factores asociados en niños de 6 a 35 meses de sitios to take-up of evidence-based practices. Our results centinela, Ministerio de Salud, Perú. Rev Bras Epidemiol. 2016;19 led us to focus on various underlying heuristics (3):539–553. CrossRef. Medline that influenced MNP adherence, such as authority 8. De-Regil LM, Suchdev PS, Vist GE, Walleser S, Peña-Rosas JP. Home bias and framing effects, hassle factors, the sa- fortification of foods with multiple micronutrient powders for health lience of negative experiences and certain mental and nutrition in children under two years of age. Cochrane Database Syst Rev. 2011;(9):CD008959. CrossRef. Medline models, appropriately timed cues, and the hot- 9. Zlotkin SH, Schauer C, Christofides A, Sharieff W, Tondeur MC, cold empathy gap. Consideration of these beha- Hyder SMZ. Micronutrient sprinkles to control childhood anaemia. viors and underlying biases may inform aspects of PLoS Med. 2005;2(1):e1. CrossRef. Medline programmatic intervention, such as counseling 10. World Health Organization (WHO). WHO Guideline: Use of practices to promote MNP use, to improve adher- Multiple Micronutrient Powders for Point-of-Use Fortification of Foods Consumed by Infants and Young Children Aged 6–23 ence in an area of Peru that has a disproportion- Months and Children Aged 2–12 Years. WHO; 2016. Accessed ately high burden of anemia. October 19, 2020. https://www.who.int/publications/i/item/ 9789241549943 Acknowledgments: We would like to thank Dr. Ricardo Castillo-Neyra, 11. Nyhus Dhillon C, Sarkar D, Klemm RDW, et al. Executive summary Dr. Michael Z. Levy, Katty Borrini Mayorí, Lina M. Mollesaca Riveros, for the micronutrient powders consultation: Lessons learned for op- Amparo M. Toledo Vizcarra, and the entire field team at the Zoonotic erational guidance. Matern Child Nutr. 2017;13(Suppl 1):e12493. Disease Research Lab in Arequipa, Peru, for their role in project CrossRef. Medline implementation and data collection. We would also like to acknowledge Dr. Richard A. Oberhelman for his role in securing funding in the initial 12. Home Fortification Technical Advisory Group (HF-TAG). data collection for this project and providing research guidance. Programmatic Guidance Brief on Use of Micronutrient Powders (MNP) for Home Fortification. HF-TAG; 2011. October 19, 2020. https://www.unicef.org/nutrition/files/HFTAG_Micronutrient_ Funding: This work was supported by the National Institute on Minority Health and Health Disparities, “Tulane-Xavier Minority Training in Powder_Program_Guidance_Brief.pdf International Health” through the Minority Health International Research 13. Creed-Kanashiro H, Bartolini R, Abad M, Arevalo V. Promoting Training Program [grant number T37 MD001424]; the Stone Center for multi-micronutrient powders (MNP) in Peru: acceptance by care- Latin American Studies at Tulane University with the Tinker Foundation; givers and role of health personnel. Matern Child Nutr. 2016;12 and the Fulbright U.S. Student Program. The conclusions expressed in this (1):152–163. CrossRef. Medline article are entirely those of the listed authors and are not endorsed by the Fulbright Program, the U.S. Department of State, or any of its partner 14. Huamán-Espino L, Aparco JP, Nuñez-Robles E, Gonzáles E, Pillaca organizations. None of the funders had any role in study design, data J, Mayta-Tristán P. Consumo de suplementos con multimicronu- collection and analysis, decision to publish, or manuscript preparation. trientes Chispitas® y anemia en niños de 6 a 35 meses: estudio transversal en el contexto de una intervención poblacional en Rev Peru Med Exp Salud Publica – Competing interests: None declared. Apurímac, Perú. . 2012;29(3):314 323. CrossRef. Medline Author contributions: KR and VPS were involved in study design and 15. Brewer JD, Santos MP, Román K, Riley-Powell AR, Oberhelman RA, administration. Data was collected by JDB, KR, and MPS. Analysis was Paz-Soldan VA. Micronutrient powder use in Arequipa, Peru: bar- conducted by JDB, JS, AB, KR, and MPS. JDB, JS, and AB prepared the riers and enablers across multiple levels. Matern Child Nutr. manuscript, and KR, MPS, and VPS provided input and edited various 2020;16(2):e12915. CrossRef. Medline drafts. 16. Osei A, Septiari A, Suryantan J, et al. Using formative research to inform the design of a home fortification with micronutrient powders REFERENCES (MNP) program in Aileu District, Timor-Leste. Food Nutr Bull. 2014;35(1):68–82. CrossRef. Medline 1. Kassebaum NJ. The global burden of anemia. Hematol Oncol Clin North Am. 2016;30(2):247–308. CrossRef. Medline 17. Sarma H, Uddin MF, Harbour C, Ahmed T. Factors influencing child feeding practices related to home fortification with micronutrient 2. Powers JM, Buchanan GR. Diagnosis and management of iron defi- powder among caregivers of under-5 children in Bangladesh. Food Hematol Oncol Clin North Am – ciency anemia. . 2014;28(4):729 Nutr Bull. 2016;37(3):340–352. CrossRef. Medline 745. CrossRef. Medline 18. Jefferds ME, Ogange L, Owuor M, et al. Formative research explor- 3. Shaw JG, Friedman JF. Iron deficiency anemia: focus on infectious ing acceptability, utilization, and promotion in order to develop a Anemia – diseases in lesser developed countries. . 2011;2011:1 10. micronutrient powder (Sprinkles) intervention among Luo families in CrossRef. Medline Western Kenya. Food Nutr Bull. 2010;31(2 Suppl):S179–S185. 4. Tolentino K, Friedman JF. An update on anemia in less developed CrossRef. Medline Am J Trop Med Hyg – countries. . 2007;77(1):44 51. CrossRef. 19. Kodish S, Rah JH, Kraemer K, de Pee S, Gittelsohn J. Understanding Medline low usage of micronutrient powder in the Kakuma Refugee Camp, Food Nutr Bull 5. Instituto Nacional de Estadística e Informática. Desnutrición crónica Kenya: findings from a qualitative study. . 2011;32 – afectó al 12,2% de la población menor de cinco años de edad en el (3):292 303. CrossRef. Medline año 2018. Accessed October 19, 2020. http://m.inei.gob.pe/ 20. Mirkovic KR, Perrine CG, Subedi GR, et al. Predictors of micronutri- prensa/noticias/desnutricion-cronica-afecto-al-122-de-la- ent powder intake adherence in a pilot programme in Nepal. Public poblacion-menor-de-cinco-anos-de-edad-en-el-ano-2018-11370 Health Nutr. 2016;19(10):1768–1776. CrossRef. Medline 6. Ministerio de Salud. Directiva Sanitaria Que Establece La 21. Reerink I, Namaste SML, Poonawala A, et al. Experiences and les- Suplementación Con Multimicronutrientes Y Hierro Para La sons learned for delivery of micronutrient powders interventions. Prevención De Anemia En Niñas Y Niños Menores De 36 Meses. Matern Child Nutr. 2017;13(Suppl 1):e12495. CrossRef. Medline Global Health: Science and Practice 2020 | Volume 8 | Number 4 729 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org 22. Sutrisna A, Vossenaar M, Izwardy D, Tumilowicz A. Sensory evalu- 208. https://ifama.org/resources/Documents/v17ia/Thapa- ation of foods with added micronutrient powder (MNP) “Taburia” to Lyford.pdf – assess acceptability among children aged 6 24 months and their 40. Liu PJ, Wisdom J, Roberto CA, Liu LJ, Ubel PA. Using behavioral Nutrients caregivers in Indonesia. . 2017;9(9):979. CrossRef. economics to design more effective food policies to address obesity. Medline Appl Econ Perspect Policy. 2014;36(1):6–24. CrossRef 23. Tripp K, Perrine CG, de Campos P, et al. Formative research for the 41. Schwartz MB. The influence of a verbal prompt on school lunch fruit development of a market-based home fortification programme for consumption: a pilot study. Int J Behav Nutr Phys Act. 2007;4(1):6. Matern Child Nutr young children in Niger. . 2011;7 Suppl 3(Suppl CrossRef. Medline 3):82–95. CrossRef. Medline 42. Meyers AW, Stunkard AJ, Coll M. Food accessibility and food 24. Dongre AR, Deshmukh PR, Garg BS. Community-led initiative for choice. A test of Schachter’s externality hypothesis. Arch Gen control of anemia among children 6 to 35 months of age and un- Psychiatry. 1980;37(10):1133–1135. CrossRef. Medline married adolescent girls in rural Wardha, India. Food Nutr Bull. ‘ ’ 2011;32(4):315–323. CrossRef. Medline 43. Rutter T. How the World Bank is nudging attitudes to health and hygiene. The Guardian. March 4, 2016. Accessed October 21, 25. Gross U, Diaz MM, Valle C. Effectiveness of the communication pro- 2020. https://www.theguardian.com/global-development- gram on compliance in a weekly multimicronutrient supplementation professionals-network/2016/mar/04/world-bank-nudging- Food Nutr Bull program in Chiclayo, Peru. . 2006;27(4 Suppl Peru), attitudes-health-hygiene S130–S142. CrossRef. Medline 44. Ministerio de Salud del Perú. Plan Nacional para la Reducción y 26. Pelletier D, DePee S. Micronutrient powder programs: new findings Control de la Anemia Materna Infantil y la Desnutrición Crónica Matern Child Nutr and future directions for implementation science. . Infantil en el Perú: 2017–2021. Ministerio de Salud del Perú; 2016. 2019;15(S5):e12802. CrossRef. Medline Accessed October 21, 2020. http://bvs.minsa.gob.pe/local/ 27. Shankar AV, Asrilla Z, Kadha JK, et al. Programmatic effects of a MINSA/4189.pdf large-scale multiple-micronutrient supplementation trial in Indonesia: 45. Stewart RE, Beidas RS, Last BS, et al. Applying NUDGE to inform using community facilitators as intermediaries for behavior change. design of EBP implementation strategies in community mental health Food Nutr Bull – . 2009;30(2 Suppl):S207 S214. CrossRef. Medline settings. Adm Policy Ment Health. 2020. CrossRef. Medline 28. Tumilowicz A, Schnefke CH, Neufeld LM, Pelto GH. Toward a better 46. Service O, Hallsworth M, Halpern D, et al. EAST: Four Simple Ways understanding of adherence to micronutrient powders: generating to Apply Behavioural Insights. Behavioural Insights; 2014. Accessed theories to guide program design and evaluation based on a review October 21, 2020. https://www.bi.team/wp-content/uploads/ Curr Dev Nutr of published results. . 2017;1(6):e001123. CrossRef. 2015/07/BIT-Publication-EAST_FA_WEB.pdf Medline 47. Dibben MR, Lean M. Achieving compliance in chronic illness man- Behavioral Economics and Healthy 29. Hanoch Y, Barnes A, Rice T. agement: illustrations of trust relationships between physicians and Behaviors: Key Concepts and Current Research . Routledge; 2017. nutrition clinic patients. Health Risk Soc. 2003;5(3):241–258. 30. Roberto CA, Kawachi I. Behavioral Economics and Public Health. CrossRef Oxford University Press; 2016. 48. Marini A, Rokx C, Gallagher P. Standing Tall: Peru’s Success in 31. Thaler RH, Sunstein CR. Nudge: Improving Decisions About Health, Overcoming its Stunting Crisis. World Bank Group; 2017. Wealth, and Happiness. Yale University Press; 2008. 49. Sypher BD, McKinley M, Ventsam S, Valdeavellano EE. Fostering — 32. Buttenheim AM, Levy MZ, Castillo-Neyra R, et al. A behavioral de- reproductive health through entertainment education in the sign approach to improving a Chagas disease vector control cam- Peruvian Amazon: the social construction of Bienvenida Salud. Commun Theory – paign in Peru. BMC Public Health. 2019;19(1):1272. CrossRef. . 2002;12(2):192 205. CrossRef Medline 50. Young SD, Cumberland WG, Nianogo R, Menacho LA, Galea JT, 33. Datta S, Mullainathan S. Behavioral design: a new approach to de- Coates T. The HOPE social media intervention for global HIV pre- Lancet HIV velopment policy. Rev Income Wealth. 2014;60(1):7–35. CrossRef. vention in Peru: a cluster randomised controlled trial. . 2015;2(1):e27–e32. CrossRef. Medline 34. Spring H, Datta S, Sapkota S. Using behavioral science to design a peer comparison intervention for postabortion family planning in 51. Ariely D, Loewenstein G. The heat of the moment: the effect of sexual J Behav Decis Making Nepal. Front Public Health. 2016;4:123. CrossRef. Medline arousal on sexual decision making. . 2006;19 (2):87–98. CrossRef 35. Castillo-Neyra R, Buttenheim AM, Brown J, et al. Behavioral and structural barriers to human post-exposure prophylaxis and other 52. Loewenstein G. Hot-cold empathy gaps and medical decision mak- Health Psychol – preventive practices in Arequipa, Peru, during a canine rabies epi- ing. . 2005;24(4S):S49 S56. CrossRef. Medline demic. PLoS Negl Trop Dis. 2020;14(7):e0008478. CrossRef. 53. Sayette MA, Loewenstein G, Griffin KM, Black JJ. Exploring the cold- Medline to-hot empathy gap in smokers. Psychol Sci. 2008;19(9):926–932. 36. Cohen N, Mendy FT, Wesson J, et al. Behavioral barriers to the use of CrossRef. Medline modern methods of contraception among unmarried youth and 54. Yang H, Carmon Z, Kahn B, et al. The hot–cold decision triangle: a adolescents in eastern Senegal: a qualitative study. BMC Public framework for healthier choices. Mark Lett. 2012;23(2):457–472. Health. 2020;20(1):1025. CrossRef. Medline CrossRef 37. Cravener TL, Schlechter H, Loeb KL, et al. Feeding strategies derived 55. Gera T, Sachdev HPS. Effect of iron supplementation on incidence of from behavioral economics and psychology can increase vegetable infectious illness in children: systematic review. BMJ. 2002;325 intake in children as part of a home-based intervention: results of a (7373):1142. CrossRef. Medline J Acad Nutr Diet – pilot study. . 2015;115(11):1798 1807. CrossRef. 56. Osei A, Septiari A, Suryantan J, et al. Using formative research to Medline inform the design of a home fortification with micronutrient powders 38. List JA, Samek AS. The behavioralist as nutritionist: leveraging be- (MNP) program in Aileu District, Timor-Leste. Food Nutr Bull. havioral economics to improve child food choice and consumption. J 2014;35(1):68–82. CrossRef. Medline Health Econ – . 2015;39:135 146. CrossRef. Medline 57. Sarma H, Uddin MF, Harbour C, Ahmed T. Factors influencing child 39. Thapa JR, Lyford CP. Behavioral economics in the school lunchroom: feeding practices related to home fortification with micronutrient can it affect food supplier decisions? A systematic review. Int Food powder among caregivers of under-5 children in Bangladesh. Food Agribusiness Management Rev. 2014;17(Special Issue A):187– Nutr Bull. 2016;37(3):340–352. CrossRef. Medline Global Health: Science and Practice 2020 | Volume 8 | Number 4 730 Behavioral Insights Into Micronutrient Powder Use in Peru www.ghspjournal.org 58. Soofi S, Cousens S, Iqbal SP, et al. Effect of provision of daily zinc 60. Burish TG, Snyder SL, Jenkins RA. Preparing patients for cancer che- and iron with several micronutrients on growth and morbidity among motherapy: effect of coping preparation and relaxation interven- young children in Pakistan: a cluster-randomised trial. Lancet. tions. J Consult Clin Psychol. 59(4):518. CrossRef. Medline – 2013;382(9886):29 40. CrossRef. Medline 61. Schofield P, Jefford M, Carey M, et al. Preparing patients for threat- 59. Salam RA, MacPhail C, Das JK, Bhutta ZA. Effectiveness of micronu- ening medical treatments: effects of a chemotherapy educational trient powders (MNP) in women and children. BMC Public Health. DVD on anxiety, unmet needs, and self-efficacy. Support Care 2013;13 Suppl 3(Suppl 3):S22. CrossRef. Medline Cancer. 2008;16(1):37–45. CrossRef. Medline En español Perspectivas Conductuales Sobre el Uso de Micronutrientes en Polvo Para la Anemia Infantil en Arequipa, Perú Mensaje clave: Las interacciones entre el personal de salud y los cuidadores de niños y sus estados emocionales cambiantes, desde que forman la intención para usar los micronutrientes en polvo (MNP) hasta que los usan afectaron su adherencia a los MNP, proporcionados por el gobierno para prevenir la anemia infantil. Durante la consejería para cuidadores, sugerimos que el personal de salud proporcione mensajes claros sobre el impacto de los MNP y una estrategia para enfrentar los retos que podrían encontrar en el uso de los MNP. Hallazgos claves: Experiencias negativas con el personal de salud o inconveniencias al momento que se inicia el uso de los MNP pueden desalentar el uso futuro de los MNP. Modelos mentales sobre la nutrición pueden influir en las intenciones para usar los MNP y, a la vez, el hechos de tener demasiadas opciones puede confundir a los cuidadores. Una sola experiencia negativa con los MNP puede formar memorias impactantes y desalentar a los cuidadores a dar los MNP. Implicaciones claves: La capacitación para el personal de salud debería fomentar una interacción interpersonal positiva con los cuidadores durante la iniciación de los MNP porque estas interacciones pueden tener un impacto duradero en el uso de los MNP. La consejería para cuidadores debería incluir información sobre efectos secundarios que podrían tener los niños para estar preparados para manejarlos. En programación futura, los profesionales de la salud pública deben considerar alentar a los cuidadores a utilizar señales oportunas para adminis- trar MNP. Resumen La anemia infantil sigue siendo un factor importante de morbilidad en los países de ingresos bajos y medios, incluyendo al Perú. Para identificar los retos conductuales para el uso de los micronutrientes en polvo (MNP), administrados con el fin de complementar la alimentación de los niños y prevenir la anemia, aplicamos un enfoque del diseño del comportamiento (behavioral design) en entrevistas y grupos focales realizadas con 129 cuidadores en Arequipa, Perú. Examinamos 3 puntos claves en el proceso de toma de decisiones: acceso a los MNP a través del sistema de salud; formación de la intención para usar los MNP; y el uso de los MNP al momento de alimentar al niño. Utilizando la estrategia de NUDGE (de las siglas en inglés para Reducir, Entender, Descubrir, Generar y Evaluar), identificamos las siguientes barreras y facilitadoras de comportamiento: (1)Las experiencias de los cuidadores con el personal de salud determinaron su motivación para acceder a los MNP; (2) Los cuidadores sintieron que acceder a los MNP en las clínicas era inconveniente y creaba molestias (hassle factors); (3) Los modelos mentales de los cuidadores sobre la prevención de anemia influyeron en la intención y el uso de los MNP; (4) Una experiencia negativa marcada de los cuidadores pudo haber hecho que dejaran de dar los MNP; (5) Los cuidadores se olvidaban de dar los MNP si no tenían señales (cues) para recordárselos, pero podían recibir señales oportunas para su administración; y (6) Los cuidadores fueron afectados durante la alimentación por factores emocionales, cognitivos y de atención, que fueron difíciles de anticipar. Nuestros resultados, basados en un enfoque del diseño del comportamiento, sugieren oportunidades de adaptar la comunicación, orientación y educación actual sobre el uso de los MNP. Las modificaciones incluyen proporcionar una comunicación culturalmente relevante, aprovechando los estados emocionales y cognitivos de los cuidadores, y alentar cambios pequeños pero impactantes en la rutina de alimentación para combatir las barreras al uso de los MNP. Peer Reviewed Received: February 19, 2020; Accepted: October 6, 2020; First published online: December 3, 2020 Cite this article as: Brewer JD, Shinnick J, Román K, Santos MP, Paz-Soldan VA, Buttenheim AM. Behavioral insights into micronutrient powder use for childhood anemia in Arequipa, Peru. Glob Health Sci Pract. 2020;8(4):721-731. https://doi.org/10.9745/GHSP-D-20-00078 © Brewer et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00078 Global Health: Science and Practice 2020 | Volume 8 | Number 4 731 ORIGINAL ARTICLE A Cluster-Randomized Trial to Test Sharing Histories as a Training Method for Community Health Workers in Peru Laura C. Altobelli,a,b,c José Cabrejos-Pita,b,d Mary Penny,e Stan Beckerf Key Findings Resumen en español al final del artículo. n Sharing Histories method for training community health workers (CHWs) was associated with ABSTRACT reduced child stunting compared with a standard Background: Community health workers (CHWs) are increasingly CHW training method. deployed to support mothers’ adoption of healthy home practices n The training method’s impact on child stunting was in low- and middle-income countries. However, little is known re- not present when mothers were illiterate. garding how best to train them for the capabilities and cultural competencies needed to support maternal health behavior change. We tested a CHW training method, Sharing Histories Key Implications (SH), in which CHWs recount their own childbearing and child- rearing experiences on which to build new learning. n CHWs can learn better by using Sharing Histories as Methods: We conducted an embedded cluster-randomized trial in the basis for their training; then, they can use the rural Peru in 18 matched clusters. Each cluster was a primary health same method to better help mothers change facility catchment area. Government health staff trained female behaviors. CHWs using SH (experimental clusters) or standard training n Program managers can use this training methods (control clusters). All other training and system- methodology to strengthen capacities of CHW strengthening interventions were equal between study arms. All trainers to improve the cultural literacy of trainees. CHWs conducted home visits with pregnant women and chil- dren aged 0–23 months to teach, monitor health practices and n Primary health care personnel who train CHWs can danger signs, and refer. The primary outcome was height-for- identify local cultural and social norms when CHWs age (HAZ)< 2 Z-scores (stunting) in children aged 0–23 months. share their experiences of childbearing and Household surveys were conducted at baseline (606 cases) and childrearing. 4-year follow-up (606 cases). n Primary health care personnel can easily apply this Results: Maternal and child characteristics were similar in both CHW training method to develop social bonds with study arms at baseline and follow-up. Difference-in-differences CHWs, address CHW cultural competencies, and analysis showed mean HAZ changes were not significantly differ- provide CHWs with a strategy to effectively discuss ent in experimental versus control clusters from baseline to end- sensitive culturally determined behaviors with line (P=.469). However, in the subgroup of literate mothers, mothers. mean HAZ improved by 1.03 on the Z-score scale in experimen- tal clusters compared to control clusters from baseline to endline (P=.059). Using generalized estimating equations, we demon- strated that stunting in children of mothers who were literate was significantly reduced (Beta=0.77; 95% confidence interval=0.23, 1.31; P<.01), adjusting for covariates. Conclusion: Compared with standard training methods, SH may have improved the effectiveness of CHWs as change agents among literate mothers to reduce child stunting. Stunting experi- enced by the children of illiterate mothers may have involved unaddressed determinants of stunting. a Future Generations University, Franklin, WV, USA. b Future Generations, Lima, Peru. c INTRODUCTION Universidad Peruana Cayetano Heredia, Lima, Peru. upporting mothers to adopt healthy home practices d Superintendencia Nacional de Salud, Lima, Peru. S e Nutrition Research Institute, Lima, Peru. could be one of the keys to improving child health. f Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. Health behavior change strategies are frequently used Correspondence to Laura Altobelli ([email protected]). in global health, but the evidence of their effect on health Global Health: Science and Practice 2020 | Volume 8 | Number 4 732 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org outcomes remains unclear due to numerous pitfalls change models for LMICs have been developed us- in their design and evaluation.1 A central challenge ing a mix of theories and strategies.25,26 Even for these strategies is how to effectively help though qualitative research can identify the specific mothers gain knowledge and change behaviors in beliefs that impede the home practice of key health communities with strong traditional beliefs and behaviors, defining the “black box” of how and poor access to health information. A worldwide why mothers, families, and communities hold onto priority for child health and development is the re- cultural beliefs is a major challenge when working to duction of chronic child malnutrition (poor linear empower change agents who can convince mothers growth or stunting), which arises from a broad to change cultural practices.27 range of causes related to home practices for mater- An important part of empowering CHWs is nal nutrition, breastfeeding and weaning, water, helping them build their own self-confidence and sanitation, hygiene, and infection prevention. Peru agency so they can thus empower mothers. These is a low- to middle-income country (LMIC) that has are key dimensions of maternal capabilities need- seen a major overall reduction in stunting since ed to implement new knowledge of proper child 2008, but high rates persist in mountain and jungle care.28,29 We assume that CHW efforts empower CHWs themselves regions where numerous cultural practices nega- communities,9 but it is less commonly recognized need to become tively influence maternal and child health.2,3 that CHWs themselves need to become empow- empowered to be Community health workers (CHWs) are a global ered to be the change agents needed to support the change agents priority to help reach impact and equity goals mothers and families for active self-care. Acco- needed to support through universal health coverage and Sustainable rding to Kane and colleagues, “...to be able to em- mothers and 4–8 Development Goals. CHWs are the lowest level of power the communities they serve, we argue, it is families for active frontline health workers and are frequently volun- essential that CHWs themselves be, and feel, self-care. teers, delivering a wide range of services in homes empowered....”30 A review of randomized con- and communities including health education and trolled trials with a “realist” approach concluded support on nutrition, malaria, tuberculosis, HIV/ that interventions by CHWs worked if there was a AIDS, sexually transmitted infections and noncom- “... sense of relatedness with beneficiaries and municable disease, preventive maternal and repro- public services; increase in self-esteem; sense of ductive health services in the home, management of self-efficacy....”31 The same author concluded uncomplicated childhood illnesses, and access to ser- that if these factors were absent, CHW perfor- vices, among others.7,9,10 CHWs can be critical actors mance would be negatively affected even with for reporting maternal and perinatal deaths occurring the same interventions.31 Some researchers have in the community.11 We know some of what works, suggested the existence of a “secret sauce” that but a large gap remains between that knowledge and would help to empower women with knowledge, “how to make it work.” An estimated 5 million motivation, and increased self-efficacy even when CHWs are deployed worldwide,5 but their effective- scaling up community strategies into government ness and linkage to health subsystems within their programs. This “sauce” could be the next break- communities vary.12 The World Health Organization through to sustainably improve maternal, newborn, (WHO)identifiesCHWsasimportanttotheirGlobal and child health behaviors.32 Strategy for Human Resources,13 but implementa- How to maintain fidelity of empowerment and tion research on CHW programs is needed.14,15 behavior change approaches in the scale-up of inter- To be effective, CHW programs should have ventions with CHWs remains a key challenge.33 As a detailed plans for governance/management, se- part of being and feeling empowered, the cultural lection, training, supervision, engagement with competency of health workers and CHWs is an es- communities, relationship with the health system, sential skill to reach patients of diverse cultures to 16–20 34 scaling up, and monitoring and evaluation. In improve their health literacy. Scaling up CHW We highlight the this study, we highlight the need for the identifica- programs in government systems likely relies in need for the tion and testing of the best methods to train part on how well health providers in primary health identification and 7,21–23 CHWs. We have not found other reports with care (PHC) services can serve as trainers to facilitate testing of the best results on comparative studies. We submit that spe- CHW learning. They are generally not educators, methods to train cial training methods are needed to adequately pre- and they often rely on medical terminology and CHWs. pare CHWs to support home behavior change. heuristic methods to train CHWs. Methods for Behavior change theories developed for industrial- teaching CHWs should be adapted to their educa- ized countries often cannot be applied in areas of tional level, which is very often the same as that of LMICs with embedded cultural beliefs, attitudes, the mothers with whom they will work. The train- and practices.24 More recently, integrated behavior the-trainer model, CHW training materials, Global Health: Science and Practice 2020 | Volume 8 | Number 4 733 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org incentives, and supportive supervision are factors SHARING HISTORIES TRAINING that have yet to be refined in government systems METHOD as well as nongovernmental organization and other The SH teaching method is a key component of an private sector efforts to support CHWs. integrated teaching strategy we developed to im- This study provides a new focus in the devel- prove maternal, neonatal, and child health. It was opment of empowerment and cultural competen- – piloted in Afghanistan, India, and Peru.35 38 cy by testing a training method that builds on Research on autobiographical memory explains cultural beliefs and practices of CHWs so they in- how the Sharing History method helps CHWs learn crease their self-esteem and have a greater sense to effectively convince other women to change beha- of interpersonal relatedness with households and viors.38,39 The utility of memory fits into 3 categories: the health system. From this foundation, they can self function (continuity of the self), social function become more effective behavior change agents (developing and maintaining social bonds), and di- within traditional populations. rective function (guiding present and future beha- We conducted a cluster-randomized controlled viors).40,41 The latter 2 functions are relevant to SH. trial of our new teaching strategy called Sharing The social function creates bonding and trust among Histories (SH) in rural Peru to test the impact on CHWs who share their childbirth and childrearing child stunting when female CHWs are trained memories with each other, and these memories are with this method. We hypothesized that mothers then used by trainers as the basis for learning. The di- would be more likely to change health knowledge rective function has been found to influence future 40 and behaviors, and their children would conse- behavior. Neuroscience research suggests that quently have better growth, if the mothers received physical consolidation of neurons occurs when mem- 42 health information from CHWs trained with the SH ory is stimulated and new information is added. The Sharing method, compared with the situation in which A manual provides details of the SH training method and the community health model (Box).43 Histories training health education interventions were received from CHW trainees can take ownership of and learn method is CHWs trained with standard methods. from their own cultural beliefs and practices, which oriented toward The method is used to train CHWs and com- ’ empowers them to change their practices and helps enhancing CHWs munity supervisors (CSs), as well as to provide di- them become more capable and convincing pro- empowerment, as rect education and counseling of mothers, and it well as their moters of the same key behavioral changes with can be implemented at low cost through the gov- mothers in their communities. The process of shar- knowledge and ernment PHC system. The training method is ori- ing personal experiences could have several bene- skills, as ented toward enhancing CHWs’ empowerment, fits including providing CHWs, who may be timid community as well as their knowledge and skills, as communi- about speaking in a group, with opportunities to change agents. ty change agents. practice verbal expression and revealing cultural BOX. Teaching Strategy of Sharing Histories The teaching strategy of SH builds on CHWs sharing their personal experiences and actions regarding their pregnancies, births, postpartum periods, care and feeding of newborns and infants, and events surrounding any sickness or death. Monthly full-day workshops are held in the primary health care facility. Each of the 7 training module topics has a series of class sessions with 6 steps: 1. If pregnancy is the day’s topic, each CHW shares her pregnancy experiences while the trainers or assistants take notes on the history format. 2. Trainers list key actions mentioned by CHWs, then lead CHWs through a guided discussion of each action, using colors to indicate whether the action is beneficial, neutral, or potentially harmful and then discussing why. 3. Then, picture cards on key practices are used to teach each best practice in a class session, referring to the CHWs’s shared experiences and further asking about and analyzing local customs related to each best practice. 4. Participatory methods are then used to practice what is learned in each class session, such as sociodramas of home visits to teach mothers using the same methods and materials, monitor practices and danger signs, and make referrals. 5. Each class session ends with participatory evaluation of learning, using games and exercises. Between monthly class sessions, each CS meets with 5 or 6 of her assigned CHWs to review the monthly topic, and CHWs practice using the respective flip charts and checklists for monitoring key maternal health practices and danger signs. Global Health: Science and Practice 2020 | Volume 8 | Number 4 734 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org practices that can be used to build new learning that health services, and local health administration would otherwise only be identified through expen- committees45 could better support CHWs. The sive and time-consuming qualitative research. methodology of CHW training was the only factor that varied between the 2 study arms (Table 1). As part of the larger project, selection criteria, METHODS roles, and tasks were clearly defined for each type of human resource for community health: CHWs, Project Intervention CSs, and PHC facility staff who train CHWs and This study was an embedded component of a larg- CSs (Table 2). er integrated project intervention called Health in Key project messages for training CHWs and the Hands of Women, which was aimed at reduc- CSs and for teaching mothers were identified ing chronic child malnutrition by linking strength- from best-practice literature on reducing child ened PHC services and district government to a sustainable community-oriented system to support stunting and from our baseline qualitative studies maternal behavior change for improved maternal, on local practices of the target population. Key neonatal, and child health.44 The area where this messages were delineated in a series of 7 flip charts study was conducted included 3 rural districts focused on 7 areas of maternal knowledge and prac- with 82,000 inhabitants in the area of the upper tice during the first 1,000 days from conception. Huallaga River on the eastern slope of the Andes These 7 flip charts covered the topics: pregnancy, mountains in the Huánuco Region of Peru (Figure birth and postpartum, newborns, breastfeeding, 1). In both experimental and control study arms, child growth and nutrition, infant diarrhea, and in- we implemented interventions to strengthen capa- fant pneumonia. Each flip chart emphasized home bilities and processes so that district government, practices, preventive care services, and recognition FIGURE 1. Location of Rural Districts, Huánuco Region, Peru, Where Cluster-Randomized Controlled Trial of Community Health Worker Training Methodology Was Conducted Global Health: Science and Practice 2020 | Volume 8 | Number 4 735 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org TABLE 1. Comparison of CHW Training Methodology Interventions Used in a Cluster-Randomized Controlled Trial, Huánuco Region, Peru Interventions Experimental Clusters Control Clusters Basic strengthening of primary health care services: orientation to Yes Yes community health strategies, interdisciplinary team building for health staff, self-assessment, and planning for community health actions Basic strengthening of local government to support community Yes Yes MNCH, to gain their commitment to provide financial and incentive support to CHWs and CSs Training of facilitators (health personnel trainers) on adult Yes Yes education methods Provision of a complete set of 7 flip charts to each CHW and CS Yes Yes Training of facilitators on use of facilitator manuals based on Yes No Sharing Histories as the CHW training method Training of facilitators on use of facilitator manuals based on a No Yes standard CHW training method Continuous monthly training of CHWs and CSs using Sharing Yes No Histories as the training method Continuous monthly training of CHWs and CSs using a standard No Yes CHW training method Monthly home visits conducted by CHWs, supported by CSs and Yes Yes health staff to educate mothers, monitor MNCH behaviors, identify danger signs, and refer to the health facility Monthly supervision of CHWs by CSs Yes Yes Abbreviations: CHW, community health workers; CS, community supervisors; MNCH, maternal, neonatal, and child health. of danger signs for which medical care should be in both study arms, These included individual picto- sought. Key messages on water, sanitation, and hy- rial checklists developed by the project team for giene (WASH) practices were included throughout home-monitoring of pregnant/postpartum mothers 46 the 7 flip charts. The flip chart messages and artwork and infants aged 0–23 months and for newborns 48 were previously developed and validated by the re- (checklist adapted from the SEARCH Program ). search team in another rural area of Peru (Cusco)46 Other tools previously developed or adapted and with artwork adapted to reflect local clothing and validated were for community referral, supervision and reporting, mothers’ birthing plan to keep at hair styles of Huánuco. The breastfeeding flip chart 46 49 was adapted from one previously developed in home, and community development planning. 47 Trainers, CSs, and CHWs were selected and Lima. All trainers, CSs, and CHWs received the flip trained between 2010 and 2014 in either the ex- charts, which were identical for both study arms. perimental or control teaching method based on For the trainers, 2 sets of step-by-step training their corresponding PHC facility cluster. Twenty- manuals were developed so they could apply dif- three selected heath staff from 11 experimental ferent methods for training CHWs on the flip chart PHC facilities were trained as trainers in the SH messages: (1) a set that incorporated the SH teach- teaching methodology, and 23 staff from 11 con- ing method and (2) a set that used standard partic- trol PHC facilities were trained separately in stan- ipatory CHW training methods. CHWs in both dard CHW training methods. Trainers received study groups received and studied the same flip 14 nonconsecutive days of training: 6 days in adult charts and used them in home visits to teach learning methodologies and 8 days in use of the mothers. corresponding set of 8 training manuals by type A set of 12 checklists and reporting formats were of teaching methodology (1 for each of the 7 topics other key tools taught to and used by CHWs equally plus 1 introductory manual). Global Health: Science and Practice 2020 | Volume 8 | Number 4 736 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org TABLE 2. Human Resources With Roles and Tasks for Community Promotion of Maternal, Neonatal, and Child Health, Huánuco Region, Peru CHW CS Facilitators (Trainers) Selection process Selected preferably by other women in the Selected by a panel of judges from the local Self-selected with the approval of their community PHC facility and municipal government superior Selection criteria Respected older woman with grown children Female, literate, at least 5 years of prior Health professional; preferably woman CHW experience or auxiliary nurse training; who has a long-term contract in PHC ability to work half-time and travel between facility communities Workload 1 CHW for every 30 families (on average, 2 or 3 1 CS supports 10–15 female CHWs 1 trainer per group of 10–25 CHWs and pregnant women and 2 or 3 children aged 0–23 their respective CS months) Key roles Attend monthly 1-day trainings at nearest PHC Ensure that her assigned CHWs attend Organize and hold 1 monthly facility month training sessions in the PHC workshop for CHWs in their own Meet monthly in small groups with her CS for facility PHC facility reinforcement of training and practice with flip Meet with her CHWs in small groups of Train CHWs and CSs on how to charts and monitoring formats 5–7 CHWs once or twice a month to re- educate and monitor mothers in the Create a map of her 30 households, view the training from the latest home using the flip chart series and identifying pregnant women and children aged workshop in the PHC facility and to monitoring tools, following a facilitator 0–23 months practice using flip charts to teach manual corresponding to each flip chart – mothers Receive training in how to utilize the Visit each pregnant woman and child aged 0 23 months on a monthly basis Accompany CHWs on home visits until facilitator manual that accompanies each the CHW feels comfortable visiting alone of 7 flip charts Fill out simple monitoring checklists, referral slips, and monthly activity report checklists Attend monthly training workshops Use the Sharing Histories teaching along with CHWs in the PHC facility methodology as incorporated into each facilitator manual Key tasks during monthly home visits to preg- nant women and children aged 0–23 months Share histories and teach mothers using flip charts by stage of pregnancy or child age Monitor health practices and record on pictorial checklists by stage of pregnancy or child age Observe for danger signs using pictorial check- lists by stage of pregnancy or child age Make referrals using pictorial referral slips for maternal-child preventive care and when dan- ger signs are detected Incentives In-kind from the health system: certificate of Monthly stipend equivalent to about one- Training and recognition recognition, training, supervision visits by CSs third the salary of an auxiliary nurse (To In-kind from district government: clothing items ensure accountability, monthly payment identifying her as a CHW or CS with name of from district government was based on the district, a food basket and party for annual demonstrated completion of the 4 key Health Promoters’ Day and Christmas roles) In-kind incentives from both the health sector and municipality, the same as for female CHWs Abbreviations: CHW, community health workers; CS, community supervisors; PHC, primary health care. Global Health: Science and Practice 2020 | Volume 8 | Number 4 737 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org Community health worker sharing histories with mother and reviewing flip chart messages. Credit: ©2014 Lurdes Cabello/Future Generations Monthly 1-day training workshops for about groups. This trial is registered with ClinicalTrials. 500 CHWs and 46 CSs were organized and held gov, NCT02903602. by trainers in PHC facilities, unless roads were im- All PHC facilities and their catchment areas in passable during the January to March rainy season. 3 municipal districts were included. Clusters were Each of the 7 topics was taught in 1 to 3 full-day matched in pairs on 2 criteria: category of PHC fa- workshops. Each time the 7 topics were completed, cility resolutive capacity and distance from district another round of monthly workshops was initiated. capital. Matched pairs were then randomly allo- cated to either study arm by the principal investi- Study Objective gator. Interventions were applied at the cluster The study tested The study objective was to test the attributable ef- level for health personnel trainers, CHWs, and the attributable fect of SH on child stunting: height-for-age less CSs. Type of CHW training method depended on the cluster to which they had been randomized. effect of Sharing than -2 Z-scores below the median according to At endline, 2 small communities did not fall Histories on child the WHO growth standard.50 The 2 study groups into the systematic random sampling of sampling stunting. were defined as (1) mothers and children living clusters (as distinguished from the randomized in- in PHC facility catchment areas where CHWs tervention clusters), due to the small size of those were trained with the SH teaching method (exper- communities and the low number of children aged imental clusters); and (2) mothers and children – living in catchment areas where CHWs were 0 23 months. Thus, it was necessary to exclude trained using standard methods (control clusters). each of their respectively matched communities Both study groups received home visits by experi- with which they had been matched before the mentally trained or standard method trained baseline survey. As a result, 18 clusters were in- CHWs, respectively. cluded in final data analyses. Outcomes were mea- sured in 2 independent samples of households at baseline and endline. The allocation of study clus- Study Design ters is illustrated in Figure 2. We conducted a cluster-randomized controlled trial (cRCT) in 22 clusters, with each cluster com- prising a PHC facility and its catchment area popu- Blinding lation. A cRCT design was appropriate in this study The study was triple-blinded (participants, pro- based on criteria to select best methods to evaluate gram implementers, and outcomes assessor). behavior change techniques.51 The cRCT design Ministry of Health and municipal officials were enabled overcoming the difficulties of using distinct unaware of the 2 different teaching methods being training methods for individually randomized used. PHC staff who served as CHW trainers were CHWs and potential contamination between study trained in separately programmed workshops and Global Health: Science and Practice 2020 | Volume 8 | Number 4 738 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org FIGURE 2. Allocation of Clusters and Study Population, Huánuco Region, Peru were unaware of differences in the training meth- to evaluate the impact of the experimental train- ods being used. Experimentally trained CHWs and ing intervention. CSs had no contact with peers from control clus- ters. Household survey teams and data managers were blinded to the study groups. Methods for Household Interview Surveys Cross-sectional household surveys were con- ducted at baseline and at 4-year follow-up on 2 in- Project Evaluation dependent samples of mothers with children aged Several data collection methods were used to as- 0–23 months in the 2 study areas. The surveys sess baseline, process, and impact. Baseline quali- were conducted, under guidance of the study tative studies were conducted on key home team, by an independent research institution practices for maternal and child health and nutri- (Instituto de Investigación Nutricional) that was tion; CHW efforts in communities; views of com- blinded to study groups. munity and municipal leaders of health actions; and assessments of PHC facilities in the project Study Variables area regarding their level of organization for qual- The main outcome variable was stunting, height- ity services and for work in communities. Project 50 for-age less than 2 Z-scores ,asaproxymeasure monitoring provided data on CHW training atten- of health status in children aged 0–23 months. dance, pre- and posttests of CHW learning in train- Independent variables previously shown to be as- ing workshops, and completion of tasks by CSs. At sociated with child stunting were measured, con- ’ the project s end, a sample of CHWs and all CSs sidering the following constructs: (a) birth weight, were interviewed with close-ended questions. An (b) breastfeeding practices, (c) child morbidity, (d) in-depth qualitative study was conducted at end- early home treatment of child illness, (e) use of line with a sample of CHWs, CSs, and trainers. health services for prevention and illness, (f) com- Most importantly, repeat household interview plementary child feeding practices, (g) micronutri- surveys were conducted at baseline and endline ent consumption in pregnancy and infancy, (h) Global Health: Science and Practice 2020 | Volume 8 | Number 4 739 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org WASH practices, and (i) mothers’ knowledge of Survey Instrument danger signs during the first 1,000 days. We also The survey questionnaire was adapted from the asked mothers about their receipt of benefits from Demographic and Health Survey instrument to al- selected government health and social services and low direct comparison of results with national and any CHW home visits during and after their most international survey data.3 Questions were added recent birth. to measure exposure to CHW visits and teaching materials, as well as empowerment and WASH indicators. Survey Sampling Frame We obtained a list of children in each community Survey Team from registers at each PHC facility. The study area Experienced local field interviewers and anthropo- had 22 health facility clusters with approximately metrists were competitively selected and trained for 1500 children aged 0–23 months, yielding a sam- baseline and final surveys. One-third of each team ple that represented about 40% of these children. had full or working knowledge of Quechua, the in- digenous language. Anthropometrists (nurses) re- ceived standardized training for child length and Survey Sample Size weight measurement and were assisted in the field Given a baseline value of 35% stunting in children by the interviewer. Training of survey teams lasted aged 0–23 months, we expected a reduction from 1 week and focused on the consent process and use 35% to 22% in the experimental study arm and of the instrument, with supervised practice with from 35% to 30% in the control arm. With the mothers and their young children from outside the study area. number of children of eligible age in the study area limited to 1500, we applied a finite popula- tion correction to our sample size calculation.52 Anthropometry Measures With a 95% confidence level (CI) and 80% power, Digital platform scales were used to measure child each study group was estimated at 283 cases. An weights, and their precision and accuracy were assumed 7% nonresponse rate gave a total of checked to 100 g before use. The mother was 303 cases per study group. weighed alone first and then with the child wearing only light clothing. Both weights were recorded for later consistency checking. Length was measured using a lightweight folding durable plastic infant- Survey Sample Selection o-meter accurate to 1 mm,53 with a minor modifi- Two-stage sampling of households was conducted cation to prevent movement of the foot board if the independently for experimental and control study child pushed against it. arms. In the first stage for each study group, a list of communities and children aged 0–23 months in each randomized study cluster (PHC facility catch- Survey Field Supervision ment area) was obtained from PHC facility regis- Supervisors accompanied each fieldworker to check tries. From this list, 38 sampling clusters (as survey forms in the field for completeness, and to distinguished from study cluster) were randomly conduct periodic repeat surveys with 10 questions selected by a systematic process with a random after the main interview by the fieldworker. start. The number of sampling clusters in a com- munity was proportional to its size, and each clus- Survey Data Entry and Analysis ter had 8 children. Double data entry was done using Visual Fox Pro. The second stage of sampling for household Consistency and range checks were also done, and surveys was conducted upon arrival in a selected data were checked against the original forms as community. Permission to interview was obtained needed. Breastfeeding and infant feeding practices from village leaders who also helped to identify based on 24-hour dietary recall were evaluated children born in the previous 4 months, who using standard WHO indicators.54 Child anthro- were then added to the list from the respective health facility registry. Children’s names were al- pometry measures were converted to Z-scores of height-for-age, weight-for-age, and weight-for- phabetically listed and systematically selected 50 with a random start. This method avoided select- height, using the 2006 WHO growth standard. ing 2 children from the same family. If a commu- We conducted an intention-to-treat analysis nity had fewer than the required number of considering all sampled subjects in the study areas. eligible children, the team moved to the next clos- Univariate assessment and bivariate tests of associ- est community to reach the needed number. ation were conducted on a wide range of Global Health: Science and Practice 2020 | Volume 8 | Number 4 740 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org independent variables to establish comparability All CSs (46) were interviewed following the of study groups and to identify potential predictors intervention period for the same reasons as listed of child stunting. Bivariate regressions of the out- for the CHW interviews. In addition, the CS inter- come stunting on independent variables were views aimed to identify their perceptions of collabora- assessed with generalized estimating equations tion with health personnel, community leaders, and local government. We adapted a closed-ended ques- (GEE) to adjust for clustering. tionnaire developed for Care Group programs.55 We conducted 2 stages of a difference-in- differences (DID) analysis to test the differences from baseline to endline of the experimental in- Methods for In-Depth Qualitative Study tervention on children’s mean height-for-age Z- Following the intervention period, to triangulate score (HAZ). We first conducted a standard DID quantitative findings, individual in-depth inter- analysis for all children, then by subgroups strati- views were conducted with trainers, CSs, and fied by low and high maternal literacy. The DID CHWs from the experimental group on their experi- analyses compared mean HAZ by 9 pairs of ences and opinions regarding SH as a teaching/ matched clusters, comparing baseline to endline. learning methodology. Control group participants Next, we compared 2 levels of maternal literacy were interviewed on their training and learning within each study group to quantify the effect of experiences. Informed consent was obtained for all the observed interaction of the experimental in- interviews. Using a unique interview guide for each tervention by stratified levels of high and low ma- type of respondent, interviews were recorded, trans- ternal literacy on stunting. lated from Quechua to Spanish as needed, and tran- Finally, we built a GEE model on endline data scribed into Microsoft Word. Analysis was done to determine the effect of the experimental inter- by a trained medical anthropologist with Atlas.ti 56 vention in interaction with maternal literacy ver- software. sus the effect of the control intervention on the outcome stunting. Covariates with a P-value of RESULTS .20 or less in the bivariate regressions on stunting were tested for inclusion in the multivariate mod- Results of Intervention Monitoring el. Covariates that were colinear with the outcome Attendance by CHWs and CSs was 82% or better variable were not included. Data were analyzed for 5 of 6 workshop topics in the monthly trainings using SPSS version 17 for the baseline and version offered at the 22 PHC facilities. Additional small- group training sessions were run by CSs for their 20 for the final survey. respective CHWs once or twice a month within communities for reinforcement of learning. CHWs Ethics and Informed Consent covered missed workshops during these small The Institutional Research Board of the Instituto de group sessions or through a CS visiting them at Investigación Nutricional approved the household home to provide personalized training. survey proposal and consent process. Consent was Verbally applied knowledge tests were given to verbal using an approved standardized protocol. CHWs before and after completing each training module topic. Pretest scores of CHWs averaged 40% and improved substantially to about 80% Methods for Close-Ended Interviews With on posttests. Experimentally trained CHW had CHWs and CSs posttest scores much higher than controls on the Fifty CHWs from each study group (n=100) were growth and nutrition and the diarrhea modules, interviewed after the intervention period to iden- which were 90% and 96% for the experimental tify their sociodemographic characteristics and group, respectively; the CHW control group perceptions of the training received, their roles in scored 82% and 74%, respectively. CHW work- teaching mothers, their participation in CHW shop attendance and pre-post test scores are reported in the Supplement. groups with their CS, and changes in themselves as a result of the training. CHWs were randomly selected from a list of active CHWs in each study Baseline and Follow-Up Household Survey cluster, proportional to the number of CHWs in Results each cluster. Interviews were conducted by nurs- Comparability of Study Groups on Demographic ing students from the local university following Characteristics training and practice in interviewing techniques Mothers were comparable between study groups from expert interviewers. We used a closed- by age, parity, and education. More control group ended questionnaire adapted from Care Group mothers worked for cash or barter in the follow- program materials.55 up survey compared with experimental group Global Health: Science and Practice 2020 | Volume 8 | Number 4 741 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org Among those mothers. Children were comparable between study home visits from a CHW during pregnancy or after visited, mothers in groups in both surveys by age, sex, mean birth- birth was 63.1% in experimental clusters and both study arms weight, and proportion with low birth weight 60.5% in control clusters, with a similar distribu- received an (Table 3). tion of number of visits by study group. Mothers average of in the experimental group were more likely to re- 5.3 visits from a Reported Household Visits by Community Health ceive 1 or more CHW visits compared with those CHW before and/ Workers in the control group among those with any prima- or after their most As reported in the endline household survey, the ry school education (72.0% experimental versus recent pregnancy. proportion of mothers who received 1 or more 66.4% control, P<.01). Among those visited, TABLE 3. Demographic Characteristics of Mothers and Children, by Study Group and Survey, Cluster-Randomized Controlled Trial on Sharing Histories CHW Training Methodology, Huánuco Region, Peru Baseline Survey 2010 Final Survey 2014 Study Group Study Group Demographic Experimental Control Experimental Control Characteristics (n=308) (N=298) Significance (n=290) (n=263) Significance Mothers Age, years, mean (SD) 26.9 (7.8) 27.2 (9.2) .62 27.1 (7.8) 26.1 (6.8) .11 Number of children, mean (SD) 2.6 (1.9) 2.8 (2.2) .18 2.7 (1.8) 2.6 (1.6) .25 Distribution of number of children, % .14 .07 1 37.5 33.0 35.9 32.5 2 22.6 27.9 22.8 28.5 3–4 26.2 21.4 22.8 27.0 5–12 13.6 17.7 18.6 12.2 Total 100.0 100.0 100.0 100.0 Education, years, mean (SD) 4.4 (3.7) 4.7 (3.6) .35 5.4 (3.9) 5.8 (3.5) .21 Distribution of maternal educational level, % (n) .60 .37 No education or cannot read (illiterate) 31.5 (97) 32.3 (96) 24.8 (72) 20.2 (53) Any primary education (literate) 46.2 (142) 42.5 (127) 40.7 (118) 45.2 (119) Any secondary or more (literate) 22.3 (69) 25.2 (75) 34.5 (100) 34.6 (91) Total 100 (308) 100 (298) 100 (290) 100 (263) Works for cash or barter, % 12.8 17.3 .25 34.1 41.8 .04 Children Age in months, mean (SD) 11.4 (6.7) 10.9 (6.8) .39 11.0 (7.0) 11.3 (6.6) .58 Sex of child, female, % 47.7 47.3 .49 50.7 47.9 .29 Birth weight (g), mean (SD) 3,045 (471) 3,042 (285) .92 3,051 (444) 3025 (481) .52 For illiterate mothers 3,063 (454) 2,953 (494) .13 2,980 (448) 3076 (475) .26 For literate mothers 3,042 (479) 3,080 (436) .41 3,074 (442) 3013 (482) .17 Low birth weight (<2,500 g) 10.2 7.2 .30 9.8 10.0 .92 For illiterate mothers 10.3 10.2 .98 9.9 8.2 .75 For literate mothers 10.2 6.7 .20 9.8 10.5 .81 Abbreviation: CHW, community health worker; SD, standard deviation. Global Health: Science and Practice 2020 | Volume 8 | Number 4 742 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org mothers in both study groups received an average Micronutrient Consumption. Study mothers of 5.3 visits from a CHW before and/or after their increased their consumption of iron tablets for the most recent pregnancy, with a range of standard minimum of 3 or more months during 1 to 27 visits. (Table 4). pregnancy at an approximate rate of 50% increase in both study groups. Study children consumed mi- Changes in Knowledge and Practices of Study cronutrient powder (Sprinkles) added to food in Mothers the previous 24 hours in approximately 60% of Knowledge of Key Danger Signs. Maternal both study groups at endline. No micronutrients spontaneous knowledge of 2 or more danger signs were taken at baseline by either group (Table 5). on each of 4 topics (pregnancy, birth, postpartum, Key Breastfeeding Practices. Nearly 90% of and newborns) increased significantly in both study all study children were breastfeeding at the time of groups from baseline to endline. Improvements both surveys. At baseline, three-fourths of study cases from baseline to endline were greater in the experi- had initiated breastfeeding within 1 hour of birth, mental group than in the control group regarding but this declined to two-thirds in both groups at end- danger signs in pregnancy and postpartum. The con- line. Exclusive breastfeeding in children 0–5months trol group of mothers had greater improvement in of age increased significantly from baseline to end- knowledge of danger signs in newborns (Table 5). line by 14.9 points in the experimental group TABLE 4. Home Visits From CHWs Received by Mothers at Endline Survey, 2014, by Study Group, Huánuco Region, Peru Experimental Control Receipt by Mothers of CHW Home Visits (n=290) (n=263) Significance Mothers who received 1 or more CHW home visits, % 63.1 60.5 .29 Distribution of mothers who received 1 or more home visits by educational level, % .59 No education or cannot read (illiterate) 23.5 18.9 Any primary education and can read (literate) 46.4 49.7 Any secondary or more education (literate) 30.1 31.4 Total 100.0 100.0 Mothers who received 1 or more CHW visits received within each educational level, % No education or cannot read (illiterate) 59.7 56.6 .43 Any primary education and can read (literate) 72.0 66.4 .21 Any secondary or more education (literate) 55.0 54.9 .55 Number of CHW home visits received (range 1–27), mean (SD) 5.28 (4.6) (N=180) 5.27 (4.3) (N=159) .99 Number of CHW home visits received within each educational level (range 0–27), 3.28 (4.5) 3.19 (4.2) .81 mean (SD) No education or cannot read (illiterate) (range 0–27) 2.65 (3.9) 2.98 (4.1) .65 Any primary education and can read (literate) (range 0–27) 3.51 (4.0) 3.46 (4.3) .93 Any secondary or more education (literate) (range 0–27) 3.45 (5.3) 2.94 (4.2) .47 Distribution of number of CHW visits received by mothers, % .88 0 37.9 39.2 1or2 19.3 19.0 3–5 22.4 20.2 6–10 13.1 15.2 11–27 7.2 6.1 Total 100.0 100.0 Abbreviations: CHW, community health worker, SD, standard deviation. Global Health: Science and Practice 2020 | Volume 8 | Number 4 743 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org TABLE 5. Changes in Maternal Knowledge and Practice, by Study Group, Huánuco Region, Peru Baseline Survey 2010 Endline Survey 2014 Baseline to Endline Differences Experimental Control Experimental Control N=308 N=298 Significance N=290 N=263 Significance Experimental Control Proportion of mothers with spontaneous correct report of at least 2 danger signs, % Danger signs in pregnancy 39.3 42.6 .28 73.8 74.5 .46 þ34.5 þ27.9 Danger signs during birth 11.0 7.7 .10 27.6 24.7 .25 þ16.6 þ17.0 Danger signs in postpartum 20.5 23.2 .24 38.3 35.7 .30 þ17.8 þ12.5 Danger signs in newborns 23.1 18.1 .08 61.0 61.6 .48 þ37.9 þ43.5 Micronutrient consumption, % Mothers consuming iron tabs 3þ months last pregnancy 51.0 46.6 .16 77.2 74.1 .41 þ26.2 þ27.5 Children with micronutrients added to food in past 24 0.0 0.3 .49 57.9 61.2 .24 þ57.9 þ60.9 hours Children with vitamin A supplement in past 6 months 47.1 46.3 .77 21.3 19.2 .28 25.8 27.1 Proportion of children with nutritional pattern, % Currently breastfeeding 89.6 87.2 .36 88.3 90.1 .29 1.3 þ2.9 Early breastfeeding within 1 hour of birth 77.6 72.8 .11 69.0 66.5 .30 8.6 6.3 Exclusive breastfeeding, 0–5 months 71.8 83.5 .06 86.6 90.2 .35 þ14.8 þ6.7 (N=71) (N=79 ) (N=82) (N=61) Food consumption in past 24 hours for children aged (N=237) (N=219) (N=208) (N=202) 6–23 months, % Iron-rich foods 53.2 53.9 .48 93.8 93.1 .47 þ40.6 þ39.2 Animal protein 32.5 37.4 .16 47.1 49.0 .39 þ14.6 þ11.6 Minimum meal frequency 69.6 68.0 .40 94.2 89.6 .06 þ24.6 þ21.6 Minimum food diversity 56.1 58.4 .34 79.3 75.7 .23 þ19.6 þ17.3 Household water, sanitation, and hygiene practices by maternal literacy, % No animals (except pets) live inside house Illiterate 45.0 50.0 .31 52.8 64.2 .14 þ7.8 þ14.2 Literate 50.6 56.9 .14 65.6 58.6 .08 þ15.0 þ1.7 Uses correct treatment for drinking water Illiterate 80.2 89.5 .17 97.2 79.2 .00 þ17.0 10.3 Literate 86.1 85.9 .29 92.2 96.2 .06 þ6.1 þ10.3 Mother washes hands after defecating Illiterate 28.1 29.5 .48 23.6 34.0 .14 4.5 þ4.5 Literate 50.7 43.2 .08 50.5 39.0 .01 0.2 4.2 Soap is available for hand washinga Illiterate na na — 48.6 37.7 .15 —— Literate na na — 61.5 61.0 .50 —— Uses safe water source Illiterate 49.0 43.2 .26 66.7 63.5 .43 þ17.7 þ20.3 Literate 64.1 59.8 .21 73.0 73.1 .54 þ8.9 þ13.3 Improved cook stove installed in past 4 years Illiterate 51.0 60.0 .14 40.3 22.6 .03 10.7 37.4 Literate 50.2 43.7 .11 40.4 32.9 .07 9.8 10.8 Does not use wood or dried dung as cook fuel Illiterate 6.3 2.1 .14 11.1 7.5 .36 þ4.8 þ5.4 Literate 18.7 15.1 .20 30.7 26.2 .18 þ12.0 þ11.1 Receipt of government health and social services, % Infant food supplementation program 93.8 92.3 .28 0.7 1.1 .45 93.1 91.2 Conditional cash transfer program (Juntos) 52.6 53.0 .49 53.8 52.9 .45 þ1.2 0.1 Continued Global Health: Science and Practice 2020 | Volume 8 | Number 4 744 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org TABLE 5. Continued Baseline Survey 2010 Endline Survey 2014 Baseline to Endline Differences Experimental Control Experimental Control N=308 N=298 Significance N=290 N=263 Significance Experimental Control Municipal Glass of Milk program 76.0 76.2 .52 68.6 68.1 .48 7.4 8.1 Child antiparasite treatment in past 6 months 12.1 15.4 .13 9.0 6.1 .13 3.1 9.3 Participation of mothers in women’s groups with discussion 46.4 44.3 .60 50.0 49.4 .48 þ3.6 þ5.1 of child health and nutritiona a Not assessed at baseline. (71.8% to 86.7%) (x 2(1,N=602)= 20.4; P<.01), child growth. Receipt of a government food sup- butbyonly6.7pointsinthecontrolgroup plement (instant fortified weaning food) by chil- (83.5% to 90.2%) (x 2(1,N=598)=5.6; P<.05) dren aged 6–23 months was nearly universal at (Table 5). baseline but null at endline because the food pro- Key Child Feeding Practices. Children’s gram was discontinued in 2012. One-half of all consumption of iron-rich foods notably increased study mothers received $30 per month as a condi- in both study groups from 53% at baseline to tional cash transfer (Juntos Program). The munic- 93% at endline. Animal protein consumption by ipal Glass of Milk program provided a daily milk children also increased significantly in both study ration to about two-thirds of all mothers with a groups but reached less than 50% at endline child 0–5 months of age and to children aged (Table 5). Minimum meal frequency, established 6–23 months. Government distribution of antipar- as 3 meals per day on average according to age asitic medication and vitamin A supplements de- and breastfeeding status,54 was2.2mealsper clined during the project period. One-half of all day for infants 6–11 months at baseline increasing to ’ 3.5 per day at endline. The experimental group had a mothers participated in a women s group in which significantly higher percentage of minimum meal fre- health and nutrition topics were discussed (Table 5). Stunting was quency than controls at endline (94.2% experimen- reduced in tal versus 89.6% control) (x 2(1, N=584)=4.1; P<.05) Baseline to Endline Changes in Child Stunting experimental (Table 5). Minimum dietary diversity by breastfeed- Changes in Stunting by Demographic clusters by ing status54 showed significant increases in both study Characteristics 4.1% from groups but differences between study groups were The baseline prevalence of child stunting was baseline to nonsignificant at endline (Table 5). 34%–35% in both study groups, unadjusted for endline, while Key Water, Sanitation, and Hygiene clustering. Stunting was reduced in experimental stunting in control Practices. WASH practices at baseline were gen- clusters by 4.1% from baseline (34.4%) to endline clusters plateaued erally much better among literate mothers than (30.3%), while stunting in control clusters plateaued from baseline to nonliterate mothers in both study groups. At end- from baseline (35.3%) to endline (35.0%) (Table 6). endline. line, several key WASH practices differed signifi- The difference at endline is not significant. cantly within strata of maternal literacy. Among At both baseline and endline, stunting was literate mothers, no animals living inside the much lower in children of literate mothers com- home (except pets) and mother washing hands af- pared with children of illiterate mothers, in chil- ter defecation were significantly more frequent in – dren aged 0 11 months compared with those Among literate the experimental group compared to the control – aged 12 23 months, in girls compared with boys, mothers, group. Among illiterate mothers, correct treat- and in those with normal birthweight (2,500 g or handwashing ment of drinking water and installation of an im- more) compared with those with low birth weight after defecation proved cook stove in the past 4 years were ( 2,500 g) (Table 6). and not raising significantly more frequent in the experimental At endline, stunting prevalence among chil- versus control group (Table 5). dren of literate mothers with any primary or sec- animals inside the Government Health and Social Services ondary education was significantly lower in the home were more Received by Mothers and Children. Study experimental group at 24.8% versus 33.8% among frequent in the groups were comparable on receipt of government controls. Among children with normal birth weight experimental services for mothers and children that might affect >2,500 g, stunting was lower in the experimental group at endline. Global Health: Science and Practice 2020 | Volume 8 | Number 4 745 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org TABLE 6. Changes in Prevalence of Growth Stunting in Children Aged 0–23 Months by Demographic Characteristics, Study Group, and Survey, Huánuco Region, Peru Baseline Survey 2010 Endline Survey 2014 Experimental Control Experimental Control (N=305) (N=295) Significance (N=290) (N=263) Significance All study children 34.4 35.3 .45 30.3 35.0 .14 Mother’s educational level No education or cannot read 45.3 43.6 .47 47.2 39.6 .25 Any primary or secondary 29.0 32.0 .29 24.8 33.8 .03 Child’s age 0–11 months 22.9 22.7 .53 19.1 25.9 .10 12–23 months 46.6 48.3 .43 43.6 45.2 .45 Child’s sex Female 21.9 28.6 .12 22.4 29.4 .12 Male 45.9 41.3 .24 38.5 30.1 .43 Child birth weight 2,500 g 44.8 61.9 .18 50.0 46.2 .49 >2,500 g 32.6 33.0 .50 27.9 33.5 .11 group (27.9%) than in the control group (33.5%) Figure 3). Literate mothers were 78% of the study (Table 6). population at endline. In accordance with the findings of significantly greater reduction in stunting in children of literate Predictors of Child Stunting The experimental mothers in the experimental group, we found that Bivariate regressions of potential predictors of CHW learning these mothers, compared with control group peers, child stunting, including type of CHW learning method and had reported the following more frequently: no ani- method (SH versus control) and potential covari- maternal literacy mals kept within the home (65.5% experimental ates, are shown in Table 8, adjusted for clustering. were found to versus 58.6% control, P<.05); handwashing after The experimental CHW learning method and interact in relation defecation (50.5% experimental versus 39.0% con- maternal literacy were found to interact in rela- to stunting. trol, P<.05); and provision of a minimum number of tion to stunting. That is, learning method was associated with a decrease in stunting among chil- feeds per day to their child (94.2% experimental dren of literate mothers, but not among children versus 89.6% control, P<.05) (Table 5). Changes in Stunting in the per Protocol of illiterate mothers. The interaction term had a significant independent association with child Subgroup. Mothers who received 1 or more stunting (Beta=0.75; 95% confidence inter- CHW visits were significantly less educated than val=0.20, 1.30; P<.00) (Table 8). Covariates that mothers who reported no visits. Nevertheless, the had bivariate association with stunting significant prevalence of child stunting did not differ between at P<.20 included child age in months, sex of the visited and non-visited groups (Table 7). child, low birth weight (<2,500 g), birth weight In the per protocol analysis of mothers who in grams, child consumed the minimum diversity had been visited by CHWs, stunting was present of food in the previous 24 hours, number of in 29.0% of children of mothers visited by an exper- months that mother took iron tablets during preg- imentally trained CHW compared with 38.4% in nancy, child consumed multi-micronutrient þ children of mothers served by control CHWs iron supplement in the past 24 hours, household (P=.04). Among literate mothers, these percentages has an improved cookstove installed in the previ- were 22.9% and 36.4%, respectively, for experi- ous 4 years, and child had parasite treatment in mental and control groups (P=.01) (Table 7 and the past 6 months. Variables that were colinear Global Health: Science and Practice 2020 | Volume 8 | Number 4 746 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org TABLE 7. Changes in Prevalence of Growth Stunting in Children Aged 0–23 Months by Receipt of 1 or More CHW Visits, by Maternal Literacy and Study Group at Endline, Huánuco Region, Peru Both Study Groups With CHW Visit Without CHW Visit With CHW Without Visit CHW VisitSignificance Experimental Control Significance Experimental Control Significance All mothers Child stunting, % 33.3 31.3 .34 29.0 38.4 .04 37.2 29.8 .38 Maternal education 5.3 (3.5) 6.0 (3.9) .02 5.1 (3.6) 5.6 (3.4) .20 5.9 (4.2) 6.1 (3.7) .71 in years, mean (SD) N 342 211 183 159 107 104 Illiterate Child stunting, % 47.9 38.5 .19 48.8 46.7 .52 44.8 30.4 .22 Maternal education 0.7 (1.3) 1.2 (1.4) .08 0.7 (1.3) 0.8 (1.2) .67 1.0 (1.4) 1.4 (1.4) .38 in years, mean (SD) N 7352 4330 2923 Literate Child stunting, % 29.4 28.9 .51 22.9 36.4 .01 28.2 29.6 .49 Maternal education 6.5 (2.9) 7.6 (3.1) .00 6.4 (3.0) 6.7 (2.7) .49 7.8 (3.3) 7.5 (2.9) .58 in years, mean (SD) N 269 159 140 129 78 81 Abbreviations: CHW, community health worker, SD, standard deviation. FIGURE 3. ChangesinStuntinginChildrenAged0–23 Months Before and After a Community Health Worker Training Intervention Comparing Experimental and Control Groups by Maternal Literacy, Huánuco Region, Peru Global Health: Science and Practice 2020 | Volume 8 | Number 4 747 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org TABLE 8. Generalized Estimating Equationsa Bivariate Associations With Child Stunting for 553 Children Aged 0–23 Months, Huánuco Region, Peru Predictors of Child Stuntingb Estimate Beta 95% Confidence Interval Significance Experimental group 0.21 0.31, 0.74 .43 Mother is literate 0.64 0.30, 0.98 <.00 Interaction: experimental group maternal literacy 0.75 0.20, 1.30 <.00 Maternal characteristics No. of children born in mother’s lifetime (1–12) 0.12 0.24, 0.01 .07 Mother has remunerated work (0, 1) 0.00 0.34, 0.34 .99 Child characteristics Age (0–23 months) 0.08 0.10, 0.05 <.00 Female child (0, 1) 0.63 0.26, 0.99 <.00 Low birth weight (<2,500 g) 0.75 1.32, 0.72 .01 Birth weight (g) .00 .00, .00 <.00 Child feeding variables Receives breastmilk within 1 hour of birth (0, 1) 0.19 0.62, 0.24 .39 Meets minimum food diversity (0, 1) 0.57 1.18, .054 .07 Meets minimum meal frequency (0, 1) 0.17 0.86, 0.53 .64 Consumes iron-rich foods in past 24 hours (0, 1) 0.35 1.47, 0.78 .55 Consumes animal food source past 24 hours (0, 1) 0.02 0.55, 0.60 .94 Micronutrient consumption Mother took iron during last pregnancy (0–9 months) 0.09 0.02, 0.17 .06 Child consumed Sprinklesc in past 24 hours (0, 1) 0.45 0.86, 0.04 .03 Vitamin A capsule taken by child in past 6 months (0, 1) 0.04 0.56, 0.49 .89 Water, sanitation, and hygiene practices Mother washes hands after defecation (0, 1) 0.10 0.17, 0.36 .46 Soap, ash, or detergent used to wash hands (0, 1) 0.03 0.19, 0.24 .80 Household has safe water source (0, 1) 0.35 0.85, 0.15 .17 Drinking water is treated correctly (0, 1) 0.20 0.33, 0.73 .45 Improved cook stove installed in past 4 years (0, 1) 0.44 0.85, 0.03 .03 Non-pet animals do not live in the home (0, 1) 0.14 0.26, 0.54 .50 Government health and social services Parasite treatment for child in past 6 months (0, 1) 0.90 1.43, 0.37 <.01 Mother in participatory women’s group (0, 1) 0.08 0.47, 0.31 .68 Glass of Milk daily ration for child (0, 1) 0.08 0.48, 0.33 .72 Juntos cash transfer received by mother (0, 1) 0.10 0.46, 0.27 .59 Abbreviation: CHW, community health worker. a Adjusted for clustering. b Outcome variable: stunted=1, not stunted=0. c Multi-micronutrients with iron. Global Health: Science and Practice 2020 | Volume 8 | Number 4 748 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org with the outcome (birth weight in grams, child age group, maternal literacy, and the interaction term in months) were not included in the multivariate (experimental group high maternal literacy) model. plus covariates. Multivariate results showed that the interaction term had a significant association Effect of the Experimental Training with stunting (B=.77; 95% CI=0.23, 1.31; P< Intervention on Stunting .00). In other words, SH was significantly associated DID Analysis with reduced stunting among literate mothers but We conducted a DID analysis for all children, and not among illiterate mothers. This effect was adjust- for stratified subgroups of maternal literacy. ed by significant covariates associated with stunting: Results in Table 9 show that for all children, the child parasite treatment in the past 6 months, child mean HAZ change was not significantly different consumed multi-micronutrients with iron in the past between experimental and control clusters (P= 24hours,andimprovedcookstoveinstalledinthe .469). However, in the subgroup of literate past 4 years. Safe water source had a borderline associ- mothers, the mean HAZ improved by an average ation with stunting. of 1.03 points on the Z-scale in experimental clus- ters as compared to control clusters between base- line and endline, with a significance level of Results of Close-Ended Interviews With P=.059. For the subgroup of illiterate mothers, CHWs and CSs the changes between baseline and endline for ex- Results of interviews with 50 randomly selected perimental versus control clusters were not signif- CHWs from each study group (n=100) and all 27 CSs icantly different. at project end showed that experimentally trained We then used DID to test whether the inter- CHWs and CSs felt more positive about their training vention’s effect differed by maternal literacy as an and learning than those trained with the control interaction effect. We found that the difference of the mean HAZ from baseline to endline in each of method. Interviews the matched clusters between children of high- Experimental CHWs felt more capable of iden- revealed that and low-literacy mothers in the experimental tifying danger signs in mothers and children during experimentally the first 1,000 days than control CHWs (38% ex- versus the control group was significant at trained CHWs and perimental versus 20% control, P<.05). P=.003, using a paired t-test. This finding demon- CSs felt more strated a significant interaction effect of the exper- When CHWs were asked about activities they positive about imental training method on HAZ by level of liked best or found most interesting in the workshops, their training and maternal literacy. experimental CHWs liked all the training activities learning than more frequently than control CHWs. Training ac- those trained with GEE Analysis tivities liked best were the use of the SH method Table 10 shows the main variables submitted to (86% experimental versus 60% control, P<.01), the the control the multivariate model using GEE with adjust- pretest (74% experimental versus 52% control, method. ment for clustering: inclusion in the experimental P<.05), and the posttest (78% experimental versus TABLE 9. DID Analysis of Mean HAZ in 533 Children Aged 0–23 Months, Baseline to Endline, for All Children and for Subgroups of Children Stratified by Maternal Literacy, Huánuco Region, Peru Control Experimental Paired t-test Mean Mean Mean Mean HAZ HAZ Diff. of HAZ HAZ Diff. of Baseline, Endline, Means, Baseline, Endline, Means, Paired DID, T-Statistic Group Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) (df) P Value All 1.55 (.46) 1.55 (.66) 0.0002 (0.36) 1.62 (0.47) 1.47 (0.31) 0.15 (0.46) 0.15 (0.60) 0.760 (8) .469 children Stratified by literacy of mother Illiterate 1.79 (0.51) 1.57 (0.93) 0.22 (0.59) 1.77 (0.44) 1.92 (0.44) 0.16 (0.57) 0.38 (0.72) 1.561 (8) .157 Literate 0.80 (1.13) 1.53 (0.59) 0.74 (1.01) 1.60 (0.54) 1.31 (0.28) 0.29 (0.50) 1.03 (1.40) 2.202 (8) .059 Abbreviations: df, degrees of freedom; DID, difference-in-differences; HAZ, height-for-age-Z-scores, SD, standard deviation. Global Health: Science and Practice 2020 | Volume 8 | Number 4 749 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org Community supervisor meeting with small group of community health workers to reinforce training received from health personnel trainer that month. Credit: ©2013 Elie Gardner TABLE 10. Generalized Estimating Equationsa Multivariate Model for Predictors of Child Stunting With 553 Children Aged 0–23 Months, Huánuco Region, Peru Estimates 95% Wald Confidence Interval Hypothesis Test Wald Predictors of Child Stuntingb Beta SE Lower Upper Chi-square df Significance Community health worker training intervention Experimental group (0, 1) 0.27 0.36 0.97 0.43 .58 1 .45 Mother is literate (0, 1) 0.22 0.11 0.00 0.44 3.69 1 .06 Interaction: experimental group maternal literacy 0.77 0.27 0.23 1.31 7.91 1 <.00 Child nutrition Child consumed Sprinklesc past 24 hours (0, 1) 0.41 0.21 0.81 .00 3.85 1 .05 Water, sanitation, and hygiene Safe water source (0, 1) 0.43 0.25 0.92 0.07 2.87 1 .09 Improved cook stove installed past 4 years (0, 1) 0.49 0.21 0.91 0.07 5.26 1 .02 Government health and social services Parasite treatment for child in past 6 months (0, 1) .85 .34 1.52 0.18 6.23 1 .01 Intercept 1.22 0.35 0.54 1.91 12.19 1 <.00 Goodness of fitd with corrected quasi-likelihood under 672.97 independence model criterion (QICC) Abbreviations: df, degrees of freedom; SE, standard error. a Adjusted for clustering. b Outcome variable: stunted=1, not stunted=0. c Multi-micronutrients with iron. d Information criteria are in smaller-is-better form. Global Health: Science and Practice 2020 | Volume 8 | Number 4 750 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org 48% control, P<.01). Compared with control CHWs, the healthy behavior should be instead, with pic- experimental CHWs more frequently liked or found ture cards (flip chart) later serving to reinforce interesting the explanation of the flip charts (92% ex- what the new behaviors should be. (Names of per- perimental versus 82% control) and use of participa- sons cited have been changed to protect privacy.) tory dynamics for evaluation of learning at the end of each learning session (86% experimental versus They had us remember how we used to do things and — 72% control). then they taught us. CHW Jenny When asked what they had found least entertain- CHWs’ memories of past experiences were fre- ing or interesting in the training workshops, control quently similar because of shared cultural prac- CHWs were more likely to mention disliking all the tices. Certain practices that could be harmful, activities than the experimental CHWs: Sharing such as home birthing, were often collective prac- Histories (0% experimental versus 8% control, tice. Once the practice was mentioned, the group < P .05); pretest (14% experimental versus 38% con- could discuss it openly without recrimination, trol, P<.01); posttest (6% experimental versus with a shared acceptance of new learning by 16% control, P=.11); explanation of the flip chart CHWs. Hearing about their peers’ differences in (2% experimental versus 12% control, P<.05); and practice also provided CHWs with motivation to participatory dynamics for evaluation (12% experi- learn. mental versus 26% control, P=.07). Among 27 CSs interviewed, decentralized group One [CHW] knows...and takes the mother to the hospi- meetings held by experimentally trained CSs with tal, and others [CHWs] don’t know. I felt a greater urge their assigned CHWs were more likely to last over to want to learn.— CHW Maria 2 hours, compared with meetings held by control Some memories were emotionally charged, CSs (64% experimental versus 31% control, P<.05) In addition, more experimentally trained CSs such as pregnancy or birth complications or the reported that community leaders supported the death of a child. When strong emotions and tears work of CHWs in communities than did control were expressed, empathetic responses from fellow CSs, in the following ways: leaders reached out to learners worked to strengthen interpersonal the municipality to implement actions for women relationships. and children (79% experimental versus 69% con- ...Some were born badly, sick.... I thought, I thank Community trol); leaders prepared community work plans God because my daughter was born well. On the other leaders in with activities to protect women and children hand, there are mothers who suffered. I thought, how experimental (71% experimental versus 58% control); leaders is that possible. Sometimes you don’t know. —CHW clusters were more gave orders or made community resolutions to en- Diana courage families to adopt health practices (36% ex- likely to be perimental versus 7% control); and leaders helped By sharing histories, CHWs seemed to become reported as women to seek care in primary care facilities or self-motivated to avoid repetition of their own pri- supportive of CHW hospitals (29% experimental versus 15% control). or erroneous behavior. CHWs said they felt more work than were confident in their ability to explain new practices those of to other mothers through the lens of local belief Results of Qualitative In-Depth Interviews communities in systems; they were able to share newfound control clusters. Postintervention qualitative interviews of CHWs, knowledge with greater force of conviction. CSs, and trainers regarding use of SH suggested mechanisms through which the method could It is a good lesson. Because sharing histories you can find have promoted health behavior change first in fe- out...how that person is living. And you can help them. male CHWs and then in mothers taught by trained ...You give them confidence and they tell you and a so- CHWs. lution can be found. Because by not telling your pro- To begin with, SH seemed to echo traditional blems, you don’t find out anything about anybody. But conceptions in this population that knowledge is if they tell you their histories, their problems, yes. And based on experience. you can help them. —CHW Jenny CHWs noted that after listening to each other’s histories during a training session, seeing a belief CHWs spontaneously shared their own experi- or practice listed on paper marked in red for “pos- ences with the mothers under their care, and in ” turn asked them to share their experiences, there- sibly dangerous effect on health motivated them by promoting empathic connections to strengthen to learn why it could be dangerous. After discuss- the social relationship, bonding, and trust needed ing the traditional practice, they discussed what for influencing traditional knowledge. Global Health: Science and Practice 2020 | Volume 8 | Number 4 751 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org I feel better because now I know...what it is to teach with communication strategies that effectively mothers. Before I didn’t know. So, on that side I feel change health knowledge and practices that affect good and I have learned to teach. —CHW Diana maternal and child health. In the current study, we tested the effect of Sharing Histories (SH), an innova- Health personnel who learned to train CHWs tive CHW teaching methodology that is intended to with the SH method initially struggled with the improve CHWs’ cultural competencies to support im- new way of training, but soon learned to appreci- proved maternal health behaviors.38 ate the method. Culturally traditional women, even those with When CHWs take primary or secondary education, generally have ownership of their At first it was difficult but soon it seemed natural. By special needs for effective learning. Through shar- past experiences sharing histories, they take interest in the topic and the new knowledge sticks with them. For us it is easier to ing their own experiences and hearing those of and improve their teach them like that. —Trainer María other women, CHWs-in-training become quickly abilities to express attuned to a topic and become interested in hear- themselves, their Some examples of cultural beliefs that were ing about practical solutions to use in specific self-esteem, self- expressed by CHWs during the history-sharing circumstances, so they know what to suggest to confidence, and sessions are provided in Table 11. prevent or solve problems. When CHWs take empowerment ownership of their past experiences and improve increase. DISCUSSION their abilities to express themselves, their self- esteem, self-confidence, and empowerment in- Peru is facing one of the fastest-growing equity crease, making them more convincing and gaps among LMICs in the distribution of the bene- effective change agents for health behaviors fits of development. The isolated and mostly rural and for mobilization of appropriate demand for Huánuco Region on the eastern slope of the Andes health services. mountains has one of the highest rates of child Many countries have attempted to develop stunting (low height-for-age) in the country. links between PHC services, community-based Stunting reflects a child’s overall well-being. It ac- health care resources, and households through curately indicates social inequalities, which are the cumulative result of poor fetal growth, inadequate work with CHWs. These programs provide a chan- nutrition, and infectious disease in the first 2 years nel for reaching families with information, of life, associated with deficient home practices for resources, and referrals. CHWs are particularly maternal nutrition, breastfeeding, complementary well positioned to address health behaviors in a feeding and micronutrient consumption, poor ac- culturally appropriate way, but a challenge is cess to health services, and possibly also WASH how to help CHWs develop the cultural and per- practices.57 sonal competencies to support change in other Strikingly absent in Peru and elsewhere is a women’s health belief systems. Government PHC system to reach vulnerable lesser-educated mothers staff who are CHW trainers and who use the SH Community health worker asks a mother and grandmother what they see in the flip chart image. Credit: ©2013 Elie Gardner Global Health: Science and Practice 2020 | Volume 8 | Number 4 752 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org TABLE 11. Examples of Local Beliefs Expressed by CHWs Through Sharing Histories, Huánuco Region, Peru Local Culturally Determined Knowledge, Beliefs, and/or Messages Given by PHC Staff Who Are Solutions Identified From CHWs Standard Messages Taught to CHWs by Trained to Facilitate Sharing Histories Topic Through Sharing Histories PHC Staff Without Sharing Histories and Learning With Picture Cards Pregnancy Danger signs in pregnancy were Danger signs not generally taught in a Picture cards are discussed with moti- not recognized as such. For ex- way to ensure understanding. vational stories of pregnant women with ample, female CHWs did not danger signs and how they can end in know that a mother could die if death, or how they can end well if care she is bleeding during pregnancy is sought. Mothers should spare their and may consider such bleeding energy by working less and eating more as “normal.”Did not identify pain so the infant can have more energy. and frequency of urination as a Picture cards with various images of problem. Heavy work or lifting is danger signs are discussed with indica- continued as normal. Pregnant tions to seek care. women eat less to have smaller baby and easier birth. Many foods are specifically avoided during pregnancy, such as fish which may “impede healing.” Birth Institutional birth is not considered Home births are illegal. Institutional CHWs need to help mothers seek insti- desirable due to fears of male births are obligatory. tutional birth with support from family health providers and horizontal and community members for transport. birth. Women are terrorized by the idea of an episiotomy or ce- sarean delivery that requires transfer to a hospital distant from home and family. Care by a tra- ditional midwife in the presence of family members is valued. Distance is a major barrier at night and holidays when no means of transport are available. Newborns Newborn danger signs that were Information on birth and newborns is not Need for immediate drying and wrap- recognized as potentially fatal discussed with CHW or mothers: CHWs ping of newborn and placement with were infant not wanting to eat and and mothers “don’t need to know”. Only mother for warmth and immediate infant being flaccid or agitated. professional birth and checkups are suckling at the breast. No bath the first Danger signs that were not recog- allowed. day to stay warm. Picture card images nized as such included an odor- of danger signs are provided and dis- ous umbilical stump. Newborn is cussed with indications to seek care. placed to one side to first attend the mother immediately after a home birth, sometimes uncovered due to simple negligence. Breastfeeding Insufficient breastmilk is a family Generally, PHC staff are not trained in All mothers can breastfeed if measures trait, so a mother will expect it and local breastfeeding beliefs and practice are taken to stimulate milk supply. nothing can be done if female or in correct breastfeeding techniques. Herbal tea should not be given to family members had little milk. Infants are taken away for immediate infants, rather the mother should drink Breastmilk is withheld to avoid newborn care and not returned quickly the tea. Trainers detect local myths harming the infant if mother is to the mother. Free formula samples are through CHW shared histories and use angry, ill, or is pregnant again. handed out. those to discuss how to avoid insufficient Herbal tea is given frequently (for breastmilk and maintain exclusive colic and infant thirst). Dozens of breastfeeding for 6 months. myths surrounding breastfeeding practice are expressed. Continued Global Health: Science and Practice 2020 | Volume 8 | Number 4 753 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org TABLE 11. Continued Local Culturally Determined Knowledge, Beliefs, and/or Messages Given by PHC Staff Who Are Solutions Identified From CHWs Standard Messages Taught to CHWs by Trained to Facilitate Sharing Histories Topic Through Sharing Histories PHC Staff Without Sharing Histories and Learning With Picture Cards Complementary feeding When a child aged over 6 months PHC staff recommend taking child off the Continue breastfeeding and try giving does not want to eat, mothers give breast and only give solid food. If breast small amounts of mashed food more only breast milk. Mothers value milk then dries up, give milk formula or frequently during the day. Soups are giving liquid soups and semiliquid cow’s milk. mostly water, which fills the infant’s sto- foods to infants (over semisolids). mach and does not allow space for the Animal-source foods are accept- food they need to grow. Add citrus juice able to give but are not available. to legumes to make them more nutritious (increase iron bioavailability) but animal-source foods should be given as much as possible. Diarrhea Diarrhea occurs when someone PHC staff promote use of oral rehydra- Dirt on hands or on prepared food can looks at the child with an “evil tion fluid and care-seeking for diarrhea. cause diarrhea in some cases, aside eye.” When diapers are damp Use hygiene for prevention (without dis- from other believed causes. Thus, it is from being hung out to dry over- cussion of local beliefs on causation). best to use hygiene practices to avoid night, the dampness in the diaper such cases (i.e., handwashing, keep can “enter” the child and cause animals out of the home, keep the child diarrhea. Traditional healers off the ground or dirty floor, use correct “pass a cuy (guinea pig)” or “pass treatment for drinking water, others). an egg” over the child’s body to draw out bad energy. Dirt or lack of hygiene is not associated with diarrhea. Abbreviations: CHW, community health worker; PHC, primary health care. method can learn firsthand the cultural practices Women’s personal experiences with childbearing of an area to inform their training, without having and child rearing are generally not valued, but SH to rely on costly or nonexistent in-depth research specifically recognizes and builds on them. Rural on local practices. female CHWs are often shy and feel inhibited The CHW program described here can be speaking in front of people they do not know implemented and fully managed at a cost of about well. The SH method provides a platform for US$1.80 per capita per year that can be divided CHWs to speak aloud in a group and practice pub- between the health budget (US$1.20) and local lic speaking. In standard CHW training, classes fre- government (US$0.60) (see Supplement for cost quently begin with trainers asking questions to data). test CHWs’ knowledge, which can be difficult for We hypothesized that the experimental CHW CHWs who fear being wrong. Sharing memories, training method could lead to maternal behavior on the other hand, allows a CHW to talk about change and subsequent improvement in child her own experiences, which are neither right nor growth. This hypothesis was supported by prior wrong and do not require recall of something pre- research showing that recall of autobiographical viously learned. In addition, the safe workshop memory changes future behavior.38,41,58 Prior environment provides each CHW with the oppor- ’ Women spersonal supportive research also showed that narrative tunity to express herself. In this process, trust and experiences with communication using firsthand and secondhand empathic bonding develop among CHWs-in- childbearing and stories is an effective health communication strat- training and trainers, especially when memories child rearing are egy for health behavior change that subsequently are emotionally charged. This circumstance increases generally not improves health status.59 the likelihood that they will collectively adjust to a valued, but SH Qualitative findings helped to explain how new way of thinking and doing things, and it specifically change occurred. The acceptance by CHWs of the promotes mutual social support to sustain new recognizes and SH training method could be related to the fact behaviors. Furthermore, a person’s self-esteem is builds on them. that experience is the traditional basis of learning. strengthened when they understand and take Global Health: Science and Practice 2020 | Volume 8 | Number 4 754 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org ownership of their cultural beliefs. Research on au- (Tables 4 and 7), is consistent with a recent systematic tobiographical memory suggests that a person’s review of factors contributing to equity of CHW ser- negative or “low closure” experiences have more vices that suggests CHWs tend to visit more vulnera- effect on future behavior than positive or “high clo- ble mothers.63 sure” experiences because negative memories maintain a stronger emotional charge and their Limitations 39 details are more present in memory. Second- The main limitation of our study design was the hand memories, heard from others in a group, can small number of study clusters, which was con- affect future behavior similar to one’s own memo- strained by the number of available clusters in the ries. Through first sharing histories in a group and study area, defined as PHC facility jurisdictions. then learning new ways to do things from a trainer, Matching and randomization of study clusters, CHWs seemed to gain enthusiasm about sharing triple-blinding, use of DID and GEE for data analy- new knowledge in a similar way to convince sis, and adjustment for covariates were design or mothers. analysis factors that strengthened power and reli- A potential effect modifier of our results on ability of the study findings.64 Our study did not stunting was the discontinuation of the National measure all potential predictive factors for stunting. Feeding Program government food supplementa- tion program for children ages 6 months to 3 years, CONCLUSIONS that occurred half-way through the 4-year study. Our study suggests that the SH method for train- We postulate that it could have contributed to the ing community health workers (CHWs) was asso- plateauing of stunting prevalence from baseline to ciated with reduced child stunting when mothers endline in the control group (Figure 3). The reduc- were literate. Regardless of educational level, tion of stunting in both study groups could argu- many women who live in traditional societies ably have been greater without this effect. with culturally rooted beliefs may have special We speculate that some of the significant im- needs for effective learning for maternal and child provement in knowledge of danger signs and health and nutrition. These women could be reported child feeding practices in both study reached with behavior change strategies through groups was due to the use of the same training CHWs trained with methods such as SH. Efforts materials (flip charts) for all CHWs. Particularly to prevent stunting with such a behavior change considering the illiterate mothers in our study, strategy could be prioritized in the “low-hanging their significant knowledge and reported behavior fruit” of children of women with at least some ed- improvements did not carry through to improved ucation to quickly reduce global stunting rates. growth of their children. High prevalence of stunt- The poorest children have many other determi- ing at 40% or above at both baseline and endline nants of stunting that are more challenging but among children of illiterate mothers in both study important to address. groups may be partially explained by so-called en- 60 This study extends the research on CHW pro- vironmental enteric dysfunction. We surmise gram implementation. To our best knowledge, that stunting in this population could not be over- this randomized trial is the first to test the applica- come by the adoption of the specific behaviors that tion of autobiographical memories to help women were promoted and measured. This explanation is be empowered with capabilities to serve as change supported by 2 recent major trials. A cRCT in rural agents with other women in their communities. Zimbabwe tested interventions to reduce stunting Our study may also represent an application of by addressing environmental contamination ei- neurological findings on the physical consolida- ther alone or in combination with infant and tion of neurons by building on memory recall to young child nutritional improvements. No reduc- strengthen educational interventions. tion in stunting was found except when a food More effective community health promotion supplement was provided along with nutritional is needed to attain better health outcomes in 61 counselling. Another cRCT in Bangladesh using LMICs. We suggest that an integrated system direct counseling combined with a mass media should focus on strengthened government services campaign found improved feeding practices with that support and sustain careful CHW selection by no improvement in growth.62 women in communities, methodologically sound Finally, our finding that our experimentally CHW training based on autobiographical memories trained CHWs tended to visit less-educated mothers used in conjunction with visual teaching material, more, especially those with only primary education consistent community-based supportive supervision Global Health: Science and Practice 2020 | Volume 8 | Number 4 755 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org of female CHWs, and effective monitoring and eval- low- and middle-income countries: a systematic review. BMC Public uation. Further implementation research on best Health. 2013;13:847. CrossRef. Medline ways to strengthen these conditions for the deploy- 7. Jeet G, Thakur J, Prinja S, Singh M. Community health workers for ment of CHWs can contribute to meeting the basic non-communicable diseases prevention and control in developing PLoS One health rights of mothers and children in LMICs and countries: evidence and implications. . 2017;12(7): e0180640. CrossRef. Medline to reaching the Sustainable Development Goals. 8. Scott K, Beckham S, Gross M, et al. What do we know about community-based health worker programs? A systematic review of Acknowledgments: We acknowledge the Regional Health Directorate of existing reviews on community health workers. Hum Resour Health. the Huánuco Regional Government; the Huánuco health service network 2018;16(1):39. CrossRef. Medline management center and primary health care staff; the local government mayors and staff, and local health administration committees (CLAS) of 9. Global Health Workforce Alliance, World Health Organization Chinchao, Churubamba, Santa Maria del Valle, and Umari Districts; (WHO). Global Experience of Community Health Workers for community health workers; and community supervisors and other Delivery of Health Related Millennium Development Goals: a community partners for their close collaboration on the project Health in Systematic Review, Country Studies, and Recommendations for the Hands of Women: A Test of Teaching Methods (En Manos de Mujeres) Integration into National Health Systems. WHO; 2010. Accessed in Huánuco, Peru. We extend special thanks to the Huánuco Regional November 10, 2020. https://www.who.int/workforcealliance/ Health Director and staff, especially Guillermo Rengifo, María Díaz, María Moreno, Marina Santamaría, and Rosario Peña. We knowledge/resources/chwreport/en/ acknowledge the important contributions of project staff in this and 10. Perry HB, Rassekh B, Gupta S, Freeman P. A comprehensive review previous projects to develop the community health model and CHW of the evidence regarding the effectiveness of community-based pri- training program: Luis Espejo, Vilma Suárez, Alejandro Vargas, Carmen mary health care in improving maternal, neonatal and child health: Phuyo, Dennys Talenas, Lurdes Cabello, Natalie García, Jonathan Bazán, Kary Terán, Eva Hachapuma, Janeth Carrillo, Irene Quispe, 7. Programs with evidence of long-term impact on mortality in chil- J Global Health Enrique Núñez, and Marisol Borda, Yesenia Calderón, and Telésforo dren younger than five years. . 2017;7(1):010907. Félix. We also acknowledge contributions by Hilary Creed, Rocío CrossRef. Medline Valverde, Miguel Campos, César Cárcamo, and others not named here. 11. World Health Organization (WHO), UNICEF, UNFPA, World Bank Special appreciation is due to Daniel C. Taylor for his support Group, the United Nations Population Division. Trends in Maternal throughout. This work is dedicated to Carl E. Taylor, who first identified Mortality 2000–2017 that rural women become eager to learn when asked to tell their . WHO; 2019. Accessed November 12, pregnancy histories. 2020. https://www.who.int/reproductivehealth/publications/ maternal-mortality-2017/en/ How Social Capital in Community Systems Strengthens Funding: This field project with embedded cluster randomized controlled 12. LeBan K. trial research was made possible by a competitive grant to Future Health Systems: People, Structures, Processes. United States Agency Generations from the United States Agency for International Development – for International Development, CORE Group, Maternal and Child Office of Global Health (Child Survival and Health Grants Program, Health Integrated Program; 2011. Accessed November 11, 2020. Cooperative Agreement No. AID-OAA-10-00048) for the project Health in https://coregroup.org/wp-content/uploads/2018/12/ the Hands of Women: A Test of Teaching Methods in addition to support Components_of_a_Community_Health_System_final10-12-2011. from the Carl E. Taylor Endowment for Equity and Empowerment, Future pdf Generations University, and local government partners in Peru. The Maternal Health Task Force of EngenderHealth through a subgrant from the 13. World Health Organization (WHO). Global Strategy on Human Bill and Melinda Gates Foundation (GMH-107-1) provided support to an Resources for Health: Workforce 2030. WHO; 2016. Accessed earlier project in indigenous Andean villages of Cusco to adapt the sharing November 10, 2020. https://www.who.int/hrh/resources/pub_ histories method to Peru. The views expressed in this article are the authors’ globstrathrh-2030/en/ and are not an official position of the funder. 14. Maher D, Cometto G. Research on community-based health workers is needed to achieve the sustainable development goals. Bull World Competing interests: None declared. Health Organ. 2016;94:786. CrossRef 15. Sanders D, Haines A. Implementation research is needed to achieve REFERENCES international health goals. PLoS Med. 2006;3(6):e186. CrossRef. 1. Abad F, Singla D. Challenges to changing health behaviours in de- Medline Soc Sci Med veloping countries: a critical overview. . 2012;75(4): 16. Pallas S, Minhas D, Perez-Escamilla R, Taylor L, Curry L, Bradley E. – 589 594. CrossRef. Medline Community health workers in low- and middle-income countries: 2. Huicho L, Segura E, Huayanay-Espinoza C, et al. Child health and what do we know about scaling up and sustainability? Am J Public nutrition in Peru within an antipoverty political agenda: a Countdown Health. 2013;103(7):74–82. CrossRef. Medline Lancet Global Health – to 2015 country case study. . 2016:4(6):E414 17. Gergen J, Crigler L, Perry HB. National planning for community E426. CrossRef. Medline health worker programs. In: Perry H, Crigler L, eds. Developing & 3. The Demographic and Health Surveys (DHS) Program. Accessed Strengthening Community Health Worker Programs at Scale: A November 12, 2020. https://dhsprogram.com/ Reference Guide and Case Studies for Program Managers and Policy Makers. Jhpiego; 2014. Accessed November 10, 2020. https:// 4. Black R, Taylor CE, Arole S, et al. Comprehensive review of the evi- www.mchip.net/sites/default/files/MCHIP_CHW%20Ref% dence regarding the effectiveness of community-based primary 20Guide.pdf health care in improving maternal, neonatal and child health: 8. summary and recommendations of the Expert Panel. J Global Health. 18. Singh P, Sachs J. 1 million community health workers in sub-Saharan 2017;7(1). CrossRef. Medline Africa by 2015. Lancet. 2013;382:363–365. CrossRef. Medline 5. Perry H, Zulliger R, Rogers M. Community health workers in low-, 19. The Earth Institute. One Million Community Health Workers— middle-, and high-income countries: an overview of their history, re- Technical Task Force Report. 2013. Accessed November 12, 2020. cent evolution, and current effectiveness. Annu Rev Public Health. http://1millionhealthworkers.org/files/2013/01/1mCHW_ 2014;35:399–421. CrossRef. Medline TechnicalTaskForceReport.pdf 6. Gilmore B, McAuliffe E. Effectiveness of community health workers 20. Zulu J, Kinsman J, Michelo C, Hurtig AK. Integrating national community- delivering preventive interventions for maternal and child health in based health worker programmes into health systems: a systematic review Global Health: Science and Practice 2020 | Volume 8 | Number 4 756 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org identifying lessons learned from low- and middle-income countries. BMC and Neonatal Health. 2010–2011. Future Generations; 2012. Public Health. 2014;14:987. CrossRef. Medline Accessed November 10, 2020. https://www.future.edu/wp- 21. Aitken I. Training community health workers for large-scale content/uploads/2020/10/Final_Report-Sharing_Histories- community-based health programs. In: Perry H, Crigler L, eds. Qualitative_study-Cusco-FG_Peru_2012.pdf Developing and Strengthening Community Health Worker Programs 38. Altobelli LC. Sharing Histories—a transformative learning/teaching at Scale: A Reference Guide and Case Studies for Program method to empower community health workers to support health be- Managers and Policy Makers. Jhpiego; 2014. Accessed November havior change of mothers. Hum Resour Health. 2017;15(1):54. 10, 2020. https://www.mchip.net/sites/default/files/MCHIP_ CrossRef. Medline CHW%20Ref%20Guide.pdf 39. Pillemer DB. Momentous Events, Vivid Memories. Harvard University 22. Agarwal S, Kirk K, Sripad P, Bellows B, Abuya T, Warrren C. Setting Press; 1998. the global research agenda for community health systems: literature 40. Pillemer DB. Directive functions of autobiographical memory: the and consultative review. Hum Resour Health. 2019;17:22. CrossRef. guiding power of the specific episode. Memory. 2003;11:193–202. Medline CrossRef. Medline 23. Glenton C, Colvin C, Carlsen B, et al. Barriers and facilitators to the im- 41. Bluck S, Alea N, Habermas T, Rubin DC. A tale of three functions: the plementation of lay health worker programmes to improve access to self-reported uses of autobiographical memory. Social Cognition. maternal and child health: qualitative evidence synthesis. Cochrane 2005;21(1):91–117. CrossRef Database Syst Rev. 2013;10:CD01414. CrossRef. Medline 42. van Kesteren MTR, Brown TI, Wagner AD. Interactions between 24. Suruchi S, Shefner-Rogers C, Skinner J. Health communication cam- memory and new learning: insights from fMRI multivoxel pattern paigns in developing countries. J Creative Commun. 2014;9(1):67– analysis. Front Syst Neurosci. 2016;10:46. CrossRef. Medline 84. CrossRef 43. Altobelli LC, Cabrejos-Pita J, Suarez-Giga V, Taylor DC. “How-To” 25. Briscoe C, Aboud F. Behaviour change communication targeting four Guide for the Learning/Teaching Method “Sharing Histories” and health behaviors in developing countries: a review of change techni- the Modular Training Program in Maternal, Neonatal, Child, and ques. Soc Sci Med. 2012;75(4):612–621. CrossRef. Medline Adolescent Health for Community Health Workers. Future Think BIG Behavioral Integration Guidance 26. Manoff Group. . Manoff Generations University; 2018. Accessed November 13, 2020. Group; 2020. Accessed November 10, 2020. https://thinkbig https://www.future.edu/2020/10/sharing-histories-how-to- online.org/index guide-fg-peru-2018/ 27. Kumar V, Kumar A, Ghosh A, et al. Enculturating science: 44. Altobelli LC, Cabrejos J, Wilcox S. Final Evaluation Report on Health community-centric design of behavior change interactions for accel- in the Hands of Women: A Test of Teaching Methods” Project in Peru: Semin Perinatol – erating health impact. . 2015;39(5):393 415. 2010–2014. Future Generations; 2014. Accessed November 10, CrossRef. Medline 2020. https://www.future.edu/wp-content/uploads/2020/10/ 28. Teti D, Gelfand D. Behavioral competence among mothers of infants Final_Report-Sharing_Histories-Health_in_Hands_of_Women_ in the first year: the mediational role of maternal self-efficacy. Child Project-FG_Peru_2014.pdf Dev – . 1991;62:918 929. CrossRef. Medline 45. Altobelli LC, Acosta-Saal C. Local Health Administration Committees 29. Matare RC, Mbuya MN, Pelto G, Dickin KL, Stoltzfus RJ. Assessing (CLAS): opportunity and empowerment for equity in health in Peru. maternal capabilities in the SHINE Trial: highlighting a hidden link in In: Blas E, Sommerfeld J, Sivasankara Kurup A, eds. Social the causal pathway to child health. Clin Infect Dis. 2015;61(Suppl 7): Determinants Approaches to Public Health: From Concept to S745–S751. CrossRef. Medline Practice. World Health Organization; 2011: 129–146. Accessed 30. Kane S, Kok M, Ormel H, et al. Limits and opportunities to community November 10, 2020. https://apps.who.int/iris/bitstream/handle/ health worker empowerment: a multi-country comparative study. Soc 10665/44492/9789241564137_eng.pdf;jsessionid= Sci Med. 2019;164:27–34. CrossRef. Medline CA0CF6A02DDAAB50584FD6F489FEF07C?sequence=1 Guía 31. Kane S, Gerretson B, Scherpbier R, Dal Poz M, Dieleman M. A realist 46. Altobelli LC, Cabrejos J, Espejo L, Vargas A, Talenas D. Metodológica de la Estrategia de Sectorización para la Promoción synthesis of randomized control trials involving use of community de la Salud en Cogestión con la Comunidad: Para fortalecer la efec- health workers for delivering child health interventions in low and tividad de los servicios de salud en el primer nivel de atención con middle income countries. BMC Health Serv Res. 2010;10:286. enfoque en la salud y nutrición materna, neonatal e infantil en la CrossRef. Medline comunidad. Future Generations; 2012. Accessed November 10, 32. Women’s groups to improve maternal and child health outcomes: 2020. https://www.future.edu/wp-content/uploads/2020/10/ evidence paradigms toward impact at scale. Glob Health Sci Pract. Guide_to_Sectorization_for_Community_Health-FG_Peru_2012.pdf 2015;1(3):323–326. CrossRef. Medline 47. Altobelli LC. Development of complementary training and educa- 33. Castro FG, Barrera M Jr, Martinez CR Jr. The cultural adaptation of tional materials on exclusive breast feeding in Peru. In: prevention interventions: resolving tensions between fidelity and fit. Communication Strategies for Infant and Child Feeding. Cornell Prev Sci. 2001;5(1):41–45. CrossRef. Medline Univ. International Nutrition Monograph Series; 1992. 34. Jongen C, McCalman J, Bainbridge R. Health workforce cultural 48. Bang A. Training Materials for Community Newborn Surveillance. competency interventions: a systematic scoping review. BMC Heath Gadchiroli, India: SEARCH Program; 2000. Serv Res. 2018;18(1):232. CrossRef. Medline 49. Taylor DC, Taylor CE. Just and Lasting Change: When Communities 35. Taylor CE, Aitken I. Community Health Worker Training for Own Their Futures. 2nd ed. The Johns Hopkins University Press; 2016. Women’s Empowerment in Afghanistan—Summary Report. Future Generations; 2006. https://www.future.edu/2020/10/ 50. The WHO child growth standards. Accessed November 13, 2020. community-health-worker-training-afghanistan/2006/ https://www.who.int/childgrowth/en/ 36. Taylor B. Identifying Best Practices Program. Future Generations 51. Michie S, West R, Sheals K, Godinho CA. Evaluating the effective- Arunachal (FGA). Future Generations; 2009. https://www.future. ness of behavior change techniques in health-related behavior: a edu/2020/10/identifying-best-practices-womens-only-workshops- scoping review of methods used. Transl Behav Med. 2018;8(2):212– arunachal/2009/ 224. CrossRef. Medline 37. Altobelli LC, Tito R. Project “Between Us (Women)”: Sharing 52. Valliant R, Dever J, Kreuter F. PracTools: computations for design of Pregnancy Histories as Part of Community Education for Maternal finite population samples. R Journal. 2015;7(2):163–176. Accessed Global Health: Science and Practice 2020 | Volume 8 | Number 4 757 Cluster-Randomized Trial on Training Method for Community Health Workers www.ghspjournal.org November 13, 2020. https://journal.r-project.org/archive/2015/ overview. Health Educ Behav. 2007;34(5):777–792. CrossRef. RJ-2015-028/RJ-2015-028.pdf Medline 53. Perspective Enterprises. Infant length measuring equipment. 60. MAL-ED Network Investigators. The MAL-ED study: a multinational Accessed November 12, 2020. http://www.perspectiveent.com/ and multidisciplinary approach to understand the relationship be- infantometer.htm tween enteric pathogens, malnutrition, gut physiology, physical growth, cognitive development, and immune response in children in 54. World Health Organization (WHO). Indicators for Assessing Infant resource-poor environments. Clin Infect Dis. 2014;59(Suppl 4): and Young Children Feeding Practices, Part I: Definition. WHO; S193–S206. CrossRef. Medline 2008. Accessed November 12, 2020. http://whqlibdoc.who.int/ publications/2008/9789241596664_eng.pdf 61. Humphrey JH, Mbuya MNN, Ntozini R, et al. Independent and combined effects of improved water, sanitation, and hygiene, and 55. Food Security and Nutrition Network Social and Behavioral improved complementary feeding, on child stunting and anaemia in Change Task Force. Care Groups: A Training Manual for Program rural Zimbabwe: a cluster-randomised trial. Lancet Glob Health. Design and Implementation. Technical and Operational 2019;7(1):e132–e147. CrossRef. Medline Performance Support Program; 2014. Accessed November 10, 2020. https://coregroup.org/wp-content/uploads/media- 62. Menon P, Nguyen PH, Saha KK, et al. Combining intensive counsel- backup/documents/Resources/Tools/tops_care_group_ ing by frontline workers with a nationwide mass media campaign training_manual_2014.pdf has large differential impacts on complementary feeding practices but not on child growth: results of a cluster randomized program 56. Atlas.ti Qualitative Data Analysis. Accessed November 10, 2020. evaluation in Bangladesh-3. JNutr. 2016;146(10):2075–2084. https://atlasti.com CrossRef. Medline 57. de Onis M, Branca F. Childhood stunting: a global perspective. 63. McCollum R, Gomez W, Theobald S, Taegtmeyer M. How equitable Matern Child Nutr. 2016;12(Suppl 1):12–26. CrossRef. Medline are community health worker programmes and which programme 58. Biondolillo MJ, Pillemer DB. Using memories to motivate future be- features influence equity of community health worker services? A havior: An experimental exercise intervention. Memory. 2015;23 systematic review. BMC Public Health. 2016;16:416. CrossRef. (3):390–402. CrossRef. Medline Medline 59. Hinyard LJ, Kreuter MW. Using narrative communication as a tool 64. Hayes RJ, Moulton LH. Cluster Randomised Trials. Chapman and for health behavior change: a conceptual, theoretical, and empirical Hall/CRC; 2009. En español Ensayo aleatorizado por clústeres para probar Compartiendo Historias como un método de capacitación para agentes comunitarias de salud en el Perú RESUMEN Antecedentes: Las agentes comunitarias de salud (ACS) se están desplegando cada vez más para apoyar la adopción por parte de las madres de prácticas saludables en el hogar. Sin embargo, se sabe poco en cuanto a la mejor manera de capacitarlas para que tengan las capacidades y competencias culturales necesarias para apoyar el cambio de comportamiento en salud de las madres. Probamos nuestro nuevo método de entrenamiento de ACS, Compartiendo Historias, en el que las ACS relatan sus experiencias de gestación, nacimiento y crianza de sus propios hijos, sobre las que se construye un nuevo aprendizaje. Métodos: Realizamos un ensayo aleatorizado en clústeres en el Perú rural en 18 clústeres apareados. Cada clúster era el ámbito de un establecimiento de salud del primer nivel de atención. El personal de salud capacitó a las ACS con Compartiendo Historias (clústeres experimentales) o un método de capacitación estándar (clústeres de control). Todas las demás intervenciones de capacitación y fortalecimiento del sistema fueron iguales entre los dos brazos del estudio. Todas las ACS llevaron a cabo visitas domiciliarias con mujeres embarazadas y niños de 0 a 23 meses para enseñar, monitorear las prácticas de salud y los signos de peligro, y referirlas. El resultado principal fue baja talla-por-edad menos de -2 Z scores en niños de 0 a 23 meses. Encuestas de hogares repetidas se llevaron a cabo al inicio del estudio como línea de base (606 casos) y en la evaluación final después de 4 años (606 casos). Resultados: Las características de las madres y sus hijos fueron similares tanto en la línea de base como en la evaluación final. El análisis de diferencias-en- diferencias (DiD) mostró que los cambios promedios de talla-por-edad no fueron significativamente diferentes en los clústeres experimentales frente a los de control desde la línea de base hasta la línea final (P=.469). Sin embargo, en el subgrupo de madres que pueden leer (alfabetas), la talla-por-edad mejoró en 1.03 en la escala de Z-score en los clústeres experimentales en comparación con los clústeres de control desde la línea de base hasta la línea final (P=.059). Asimismo, usando ecuaciones de estimación generalizadas (GEE), demostramos que la desnutrición crónica en los hijos de las madres alfabetas se redujó sig- nificativamente (Beta=0,77; intervalo de confianza del 95% 0,23, 1,31; P<.01), controlando adjustando por covariables. Conclusión: En comparación con los métodos de entrenamiento estándar para ACS, Compartiendo Historias puede haber mejorado la eficacia de las ACS como agentes de cambio entre las madres alfabetas para reducir la desnutrición crónica infantil. La baja talla-por-edad experimentada por los hijos de madres analfabetas puede implicar determinantes del retraso en el crecimiento que no fueron abordados en este estudio. Peer Reviewed Received: September 24, 2019; Accepted: October 29, 2020; First published online: December 11, 2020 Cite this article as: Altobelli LC, Cabrejos-Pita J, Penny M, Becker S. A cluster-randomized trial to test Sharing Histories as a training method for com- munity health workers in Peru. Glob Health Sci Pract. 2020;8(4):732-758. https://doi.org/10.9745/GHSP-D-19-00332 © Altobelli et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-19-00332 Global Health: Science and Practice 2020 | Volume 8 | Number 4 758 ORIGINAL ARTICLE A Rapid Cost Modeling Tool for Evaluating and Improving Public Health Supply Chain Designs Michael Krautmann,a Mariam Zameer,b Dorothy Thomas,b Nora Phillips-White,b Ana Costache,b Pascale R. Leroueila Key Findings ABSTRACT Effective and efficient health supply chains play a vital role in n The Rapid Supply Chain Modeling (RSCM) Tool achieving health outcomes by ensuring supplies are available addresses a need for more rapid and flexible ways to ’ for people to access quality health services. However, supplying model the cost impact of changes to a country s health commodities to service delivery points is complex and cost- supply chain design or context. ly in many low- and middle-income countries. Thus, governments n We compared the RSCM Tool against existing cost and partner organizations are often interested in understanding modeling tools and found it capable of producing how to design their health supply chains more cost efficiently. similar results across a wide range of countries and Several modeling tools exist in the public and private market that supply chain designs. can help assess supply chain efficiency and identify supply chain n The ideal user for the RSCM Tool is a technical design improvements. These tools are generally capable of pro- officer familiar with Excel and supply chain concepts; viding users with very precise cost estimates, but they often use the outputs can inform both technical discussions proprietary software and require detailed data inputs. This can and high-level policy decisions. result in a somewhat lengthy and expensive analysis process, which may be prohibitive for many decision makers, especially Key Implications in the early stages of a supply chain design process. For many Health system leaders and their technical use cases, such as advocacy, informing workshop and technical teams should consider using the RSCM Tool meetings, and narrowing down initial design options, decision to streamline the beginning stages of a supply makers may often be willing to trade some detail and accuracy chain design initiative, particularly in the following in exchange for quicker and lower-cost analysis results. To our use scenarios: knowledge, there are no publicly available tools focused on gen- n Generating estimates of high-level impact to inform erating quick, high-level estimates of the cost and efficiency of initial advocacy efforts different supply chain designs. n Sustaining momentum from initial workshops by To address this gap, we designed and tested an Excel-based quickly addressing supply chain questions Rapid Supply Chain Modeling (RSCM) Tool. Our assessment indi- n Narrowing down a wide range of initial supply chain cated that, despite requiring significantly less data, the RSCM design possibilities to help policy makers more Tool can generate cost estimates that are similar to other common quickly focus on the highest-impact design changes analysis and modeling methods. Furthermore, to better under- stand how the RSCM Tool aligns with real-world processes and decision-making timelines, we used it to inform an ongoing im- munization supply chain redesign in Angola. For the use cases described above we believe that the RSCM Tool addresses an im- portant need for quicker and less expensive ways to identify more cost-efficient supply chain designs. BACKGROUND upply chains are a key component of any well- Sfunctioning health system.1 For vaccines, medicines, and other health products to be effective at preventing and treating disease, they must be accessible to the peo- a William Davidson Institute, University of Michigan, Ann Arbor, MI, USA. ple who need them, when and where they are needed. b VillageReach, Seattle, WA, USA. Health supply chains that can reliably deliver these pro- Correspondence to Michael Krautmann ([email protected]). ducts to the point of care are vital to ensuring access to Global Health: Science and Practice 2020 | Volume 8 | Number 4 759 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org quality health care and to achieving positive particularly at the final stages of a supply chain de- health outcomes.2–4 sign process when the focus is on fine-tuning and However, in many low- and middle-income implementing a specific plan. However, there are a countries (LMICs), health supply chains fail to broad range of other instances where the required ensure consistent availability of critical health time and cost are prohibitive. This could include products.5 According to the most recent World situations like conducting initial advocacy for sup- Medicines Report, a typical public clinic in sub- ply chain improvements, informing workshops Saharan Africa averaged only 57% availability and meeting discussions in real time, or narrowing of its required essential medicines, and nearly down a wide range of improvement options in the 25% of all LMIC patients were regularly unable early stages of a supply chain design process. In to access medicines needed for treatment.6 In ad- such cases, leaders from ministries of health and dition, global vaccination coverage has plateaued partner organization would likely trade some level at 80%–85% since 2010, and supply chain ineffi- of detail and accuracy in exchange for reducing ciencies are considered a significant driver.7,8 These the time and cost of analysis. To our knowledge, same supply chains are also expensive to operate, re- there are no publicly available modeling tools quiring millions of dollars of annual funding to sup- that are flexible enough to help decision makers ply thousands of public health facilities throughout evaluate the cost and efficiency of different supply a given country.9,10 For these reasons, improving chain designs, while also minimizing the need for health supply chain efficiency and effectiveness is a data collection and specialized software skills. key objective for donor agencies, governments, and In this article, we present the design and test- – other health care stakeholders.11 13 ing of a Rapid Supply Chain Modeling (RSCM) One important pathway to achieving this ob- Tool aimed at addressing this gap. We describe jective is restructuring and improving a supply key attributes of the RSCM Tool, validate its chain’s design (i.e., the overarching strategy for results against existing supply chain analyses, and organizing a supply chain network and its human explore how it can help inform a country’s supply resources, technologies, and processes). Recent chain redesign process. studies have demonstrated that improving a sup- ’ ply chain s design can lead to more cost-efficient METHODOLOGY: DESIGNING A delivery and better product availability in health To our knowledge, facilities.14–18 Thus, donor agencies like Gavi, the TOOL FOR RAPID ANALYSIS USE there are no Vaccine Alliance, have explicitly incorporated CASES publicly available supply chain design into their supply chain strate- The RSCM Tool is a quantitative, Excel-based modeling tools gies,19 and country governments are prioritizing model designed to quickly estimate costs and basic that are flexible supply chain redesign and strategy development efficiency metrics for multiple supply chain design 20,21 enough to help activities in their national health plans. scenarios. It requires users to input information decision makers The task of analyzing and identifying an im- about the design and general characteristics of a ’ evaluate the cost proved supply chain design can often be challeng- country s current supply chain, including: supply and efficiency of ing for a couple of reasons. First, detailed supply chain network information, such as land area and different supply chain data (e.g., operating costs, product demand, number of facilities; cost parameters, such as the chain designs facility locations) are often unavailable or time in- cost of labor, fuel, or vehicles; and storage and dis- while also tensive to collect. Second, many existing tools tribution guidelines, such as inventory levels or used to collect and analyze such data are intended frequency of delivery. minimizing the to provide a snapshot of the current system,22 Using these inputs, the RSCM Tool models key need for data whereas a design analysis requires flexible models operational supply chain activities and calculates collection and that can predict the impact of large-scale changes several resulting output metrics, including: annu- specialized to the supply chain. Although more sophisticated al operating cost, disaggregated by tier and supply software skills. supply chain modeling and optimization tools do chain function (i.e., storage, transportation, and exist for this purpose, they typically require pro- management); expected utilization of resources prietary software and specialized modeling skills like vehicle and warehouse capacity; and opera- and/or consultants. In total, a supply chain review tional statistics like kilometers traveled or volume and redesign process using these current methods delivered per facility. can require at least 3–6 months and US$250,000– Within the tool, those inputs and outputs US$500,000, according to recent estimates.23 worksheets can be replicated to create multiple The detailed and precise outputs from such supply chain scenarios, which can be compared tools are necessary in some circumstances, under a main dashboard. Global Health: Science and Practice 2020 | Volume 8 | Number 4 760 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org To address the identified need for quicker and tools. We test this hypothesis and quantify the ef- To address the more cost-effective decision-making tools, we fect of the tool’s assumptions in the validation sec- need for quicker made several design decisions that help the RSCM tion below. and more cost- Tool maintain flexibility, quick setup time, and effective decision- minimal data collection needs (Table 1). Standardized “Menu” of Design Levers making tools, we Most global health supply chains can be broken designed the down into a relatively small set of “building block” RSCM Tool to Simplifying Modeling Assumptions design decisions in key functional areas like stor- maintain Like most modeling tools, the RSCM Tool uses age, transportation, and management. By incor- flexibility, quick assumptions to strike a desired balance between porating these design levers along with common setup time, and simplicity and accuracy. Since our goal is to provide pre-set choices/values, a user can easily create minimal data faster results by reducing overall data requirements, and toggle between different distribution strate- collection needs. we ask the user to define a typical facility at each gies for their supply chain network. supply chain tier, rather than requiring detailed de- mand, location, and cost data for every facility. While the results do not provide detailed out- Proxy Data and Worksheets to Address Data puts for individual facilities, we hypothesize that Gaps for high-level, system-wide design analyses, the Since supply chain and cost data are often scarce, RSCM Tool’s outputs will be reasonably similar to we incorporated several supporting worksheets those of other common supply chain modeling and proxy datasets to help users quickly estimate TABLE 1. Key Modeling Tool Design Decisions for Facilitating Rapid Supply Chain Analyses Simplifying modeling assumptions: Reducing data Assumptions: requirements and enable real-time calculation All facilities at a given tier have the same demand quantity per order period Demand is the same for every order period and does not vary over time Facilities within a tier are evenly distributed throughout a given region and, thus, are the same average distance to their nearest re-supply point Standardizing design levers: Providing flexibility Storage: At which levels do you hold and manage to model diverse global health distribution inventory? How much safety stock does each level hold, strategies and how frequently is it replenished? Transportation: What types of vehicles are used to transport replenishment shipments? What type of distribution model is followed at each level (e.g., hub and spoke or multi-stop distribution loops), and are there any travel constraints (e.g., administrative boundaries)? Management: Who is responsible for performing key ordering, transport, and storage functions? What types of technology supports people at each level? Proxying data and worksheets to fill gaps: Supporting worksheets and datasets: Enabling quick estimation of missing data points A model for estimating immunization and/or reproductive health demand volumes and product value, by combining available demand planning methodologies with publicly available demographic and product data A general model for converting the number of units of a health product into a cubic-meter volume using historical product unit volume data Common commercial heuristics for estimating storage capacity of a warehouse based on its overall dimensions A database of typical costs for assets like vehicles, warehousing space, and cold chain equipment Using Excel-based platform for broad accessibility Global Health: Science and Practice 2020 | Volume 8 | Number 4 761 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org values for common data gaps, minimizing time re- RSCM Tool’s deviation from that value as an abso- quired for data collection.24–29 Additionally, we lute percent error. compiled a set of complete input data templates, We performed this calculation at 3 levels of which are proxy data from existing cost analyses cost aggregation: (1) individual cost line items that are formatted to match the RSCM Tool’s (e.g., fuel costs for transportation at health facili- structure. Instead of entering each input value in- ties); (2) total cost for each supply chain tier and dividually, a user can “load” a preset template as a cost category; and (3) total annual cost for the en- starting point for analysis, selectively overriding tire system. For each level of aggregation, we aver- the proxy dataset where better data exists. aged individual error calculations together to obtain a mean absolute percent error (MAPE). A Excel-Based Platform lower MAPE value implies a smaller difference be- Finally, we developed the RSCM tool in Microsoft tween the reference and RSCM tool results. Excel because it is the most widespread software We were able to obtain reference datasets from that can meet the tool’s technical requirements 6 recent cost and modeling analyses, covering and run easily on most computers. Government 7 supply chain design scenarios (Table 2). We staff and implementing partners are often familiar chose these reference analyses in part based on with and comfortable using Excel in their daily our ability to access underlying data since replicat- work, such as for demand planning. For users ing the analyses as closely as possible required a who already have Excel, there is no additional more detailed breakdown of inputs and results cost to accessing the RSCM Tool. Additionally, (e.g., worker salary assumptions, specific vehicle the tool is functional offline, which is essential for types) than what is typically available in public areas with unreliable internet connectivity. This reports. Additionally, we sought out analyses that enables the tool to be easily and widely accessible were produced and vetted by country govern- to multiple stakeholders throughout a country, ments and partners and actually used to inform which would be less likely if it required proprie- key stakeholder decisions. Even though these tary software or license fees. analyses also represent estimates of true supply chain costs, they are the best-established estimates available, and thus, serve as ideal reference values VALIDATING THE RSCM TOOL’S when validating the RSCM Tool. METHODOLOGY Collectively, these analyses encompass a di- To test the validity of the methodology and verse set of current public health supply chains. assumptions described, we conducted an assess- They incorporate several health program areas ment to determine whether the outputs generated and span a range of geographies across Africa and by the RSCM Tool were consistent with other Latin America. They also cover several common established methods for measuring or estimating supply chain designs, including ad hoc facility the costs of different supply chain designs. collection, “level-skipping” or “direct delivery” designs that bypass an administrative tier and a “ ” Validation Approach mobile warehouse design where facility inven- tory is periodically topped-up by visiting resupply Our general approach was to compile detailed vehicles.30 datasets from recent supply chain costing and modeling analyses and replicate each analysis us- ing the RSCM Tool. First, we built complete sets Adjusting Data to Ensure Equivalent of data inputs for the RSCM Tool, compiling them Comparisons from a variety of sources and vetting assumptions These 6 reference analyses were conducted by dif- externally wherever possible. (The Discussion ferent organizations for different purposes; hence, details the main challenges we faced in building they differ in methodological details like the scope these input datasets and how we addressed of costs included, how costs are classified, and them.) Then, we compared the RSCM Tool’s cost analysis method (e.g., simulation modeling vs. di- estimates to the results of the original analyses. rect cost measurement at a sample of facilities). With identical data inputs, we would expect None of these methods is inherently better than any discrepancy in results to be driven by differ- another; each uses a set of data and assumptions ences in the modeling approach and assumptions. that are tailored to its own unique context. For each comparison, we treated the existing anal- However, due to these differences, we often need- ysis as a “reference” value and measured the ed to transform certain data inputs and outputs to Global Health: Science and Practice 2020 | Volume 8 | Number 4 762 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org TABLE 2. Reference Datasets From Cost and Modeling Analyses Used to Validate the Rapid Supply Chain Modeling Tool Study Location Description Bolivia and Guatemala, 2018 Three supply chain costing studies led by ForoLAC (Foro Latinoamericano y del Caribe para el Aseguramiento de Insumos de Salud Reproductiva) that included all major health commodities, including vaccines: Tarija Department, Bolivia Quiché Department, Guatemala Alta Verapaz Department, Guatemala Mozambique, 2015 Modeling analysis conducted by VillageReach for the national and provincial ministries of health, using the HERMES software platform (Highly Extensible Resource for Modeling Event-Driven Supply Chains) to assess 2 immunization supply chain design options for Manica Province in Mozambique (the baseline 4-tier design, and a direct delivery design that skipped 1 of the tiers). Senegal, 2017 Modeling analysis in Senegal estimating the nationwide costs of operating the Informed Push Model strategy for delivering family planning and maternal-child health products. Zimbabwe, 2015 Evaluation of the Zimbabwe Assisted Pull System strategy in Manicaland Province in Zimbabwe, which integrated commodity distribution for most health program areas (except vaccines). ensure an equivalent comparison with the RSCM Identifying Proxies for Missing Data Inputs: Tool. Some RSCM inputs simply were not available Many of the input parameters required by the in a reference dataset, often because of a differ- RSCM Tool lacked a directly comparable value in ence in methodology. For example, the RSCM the reference analyses, requiring us to make sev- Tool uses a road network circuity factor to help eral types of estimates and adjustments, including: estimate distances between facilities. We often had to use Google Maps to develop a rough Aligning Level of Detail: Many RSCM inputs proxy for this parameter because many of the were available in the reference datasets but reference analyses measured actual distances were scoped or grouped differently. For example, between sample facilities. the RSCM Tool handles vehicle costs like fuel, maintenance, and insurance individually, but Similarly, when comparing final outputs, in some datasets use only an aggregate “total oper- the following examples, we often had to adjust ating cost” rate, requiring us to estimate the for differences among the reference analyses in breakdown of that rate into its subcomponents. how specific cost line items were calculated. For example: Inferring Input Values From Results: With some datasets (especially ones that only had Costing Unutilized Assets: Some reference results available), we lacked explicit assump- analyses and the RSCM Tool track all assets tions for required inputs like vehicle mainte- that are owned by a supply chain (e.g., vehicles nance cost rates. However, in many cases, we or storage space), while other analyses track were able to infer a value from data contained only the fraction of those assets that are actively in the results, such as overall maintenance costs used. Both approaches are valid but result in and distances traveled. While we would ordi- different answers unless the assets are fully uti- narily avoid using the detailed reference data- lized. Thus, when comparing against this alter- sets as sources for RSCM Tool inputs, we were nate approach, we scaled down the quantity of comfortable doing so in situations where the vehicles and storage in the RSCM Tool to elimi- input parameter: (1) was an objective, numeric nate any expected idle capacity. value, and; (2) would likely be found elsewhere Assigning “Ownership” of Costs: The RSCM in ministry or partner financial records that Tool assigns the cost of a supply chain activity to would be accessible in a country-level applica- the location where that activity occurred. tion of the tool. Certain analyses, however, assign costs to Global Health: Science and Practice 2020 | Volume 8 | Number 4 763 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org wherever the budget line for those costs is locat- We also saw a wide range of MAPE values ed. Those are not always the same locations (e. across the different reference analyses. Some, g., if health facility vehicle maintenance is such as the 2 Mozambique scenarios, saw rela- funded out of a district budget). tively high MAPE values across all 3 levels of Scoping Specific Cost Categories: Reference comparison. Others, such as Zimbabwe and the analyses differed in the scope of costs that they ForoLAC (Foro Latinoamericano y del Caribe were willing to consider. For example, several para el Aseguramiento de Insumos de Salud analyses omitted depreciation costs for health Reproductiva) studies, were relatively low across facility storage space, since the buildings are of- the board. These differences appear to roughly ten owned by the government and require no correlate with the level of detail available in the rent or mortgage payment. Others chose to in- underlying reference datasets. We discuss further clude a nominal storage space cost, since even- implications of these results in the Discussion tually that health facility building would need section. to be replaced. TESTING THE TOOL IN AN Validation Results Figure 1 shows the difference, measured in MAPE, IMMUNIZATION SUPPLY CHAIN between the RSCM Tool’s cost estimates and those CONTEXT of the 7 reference supply chain scenarios. MAPE We also wanted to test the usability of the tool to values were lowest when comparing total supply understand how it aligned with real-world pro- chain operating costs between the RSCM esti- cesses and timelines for conducting supply chain mates and the reference analyses. At this level, design analyses. In January 2019, United Nations the only comparison to exceed a 4% MAPE value Children’s Fund (UNICEF), with the support of was the Mozambique baseline (6.7%). This implies technical partners, worked with Angola’sExpanded that the modeling assumptions and simplifications Program on Immunization (EPI) to explore a review described above generally have the smallest impact and redesign of its immunization supply chain (iSC). on high-level cost estimates. As the comparisons This engagement provided us with an opportunity became more granular (e.g., comparing costs for to evaluate whether the RSCM Tool could be used an individual tier or cost line item), the differences quickly and easily to produce high-level estimates became somewhat more pronounced. during a real-time supply chain redesign. FIGURE 1. Comparison of Cost Estimates Between the Rapid Supply Chain Modeling Tool and Existing Reference Supply Chain Analyses Bolivia-Tarija Guatemala-Alta Verapaz Guatemala-Quiche Mozambique-Baseline Mozambique-Direct Delivery Senegal-Informed Push Zimbabwe-Assisted Pull 25.0% 22.9% 20.0% 18.7% 16.1% 16.0% MAPE 15.0% 10.0% 9.5% 10.0% 8.0% 6.7% 5.5% 5.3% 5.6% 4.4% 5.0% 3.9% 3.1% 2.1% 2.1% 0.9% 0.9% 1.8% 0.5% 0.9% 0.0% Total Tier-level Line Item Cost Comparisons Cost Comparisons Cost Comparisons Abbreviation: MAPE, mean absolute percent error. Global Health: Science and Practice 2020 | Volume 8 | Number 4 764 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org Angola Immunization Supply Chain Context 2. An “ideal baseline” scenario, which main- Angola’s immunization supply chain consists of tained the current supply chain design but in- 4 tiers (national, province, municipality, and creased cold storage capacity to account for health facility) that align with the Ministry of current shortages, represented the “true” cost Health’s administrative structure. Vaccines travel to run the current design with added cold through national, provincial, and municipality storage. stores on their way to 1,321 health facilities. By 3. A reduced resupply frequency scenario low- conducting a supply chain design review, UNICEF ered transport costs by switching from monthly andEPIaimedtoimprovetheefficiencyofthisiSC to 2-month resupply cycles at the municipality structure and understand how those improvements level. would impact deployment of resources like 4. A level-skipping scenario bypassed the province ’ vehicles and cold chain equipment. UNICEF s level, with the national warehouse delivering System Design Approach (Figure 2)providedan supplies monthly to municipalities, resulting in overarching framework for this design review a 3-tiered distribution structure. process.31 We incorporated the RSCM Tool into the data The process for collecting data and analyzing collection, validation, and modeling stage of this these scenarios involved several key steps: (1) inter- process after an initial stakeholder workshop in viewing national-level EPI officials about key supply May 2019 and conducted a preliminary modeling chain policies, (2) compiling and entering available analysis while awaiting development of a more data into an RSCM Tool data template, (3) identify- detailed optimization model. By providing quick ing proxy data sources to fill any remaining data interim results, our goals were to sustain stake- gaps, and (4) identifying how to model each design holder interest and momentum post-workshop, scenario in the RSCM Tool. In total, this process re- get government buy-in for the scenarios defined quired approximately 3 weeks’ worth of personnel at the workshop, and streamline subsequent time, divided across 3 people. However, that time re- modeling analyses to focus on the most promising quirement could have been reduced significantly areas of improvement. (30%–50% in our estimation) in a scenario where everyone working on the analysis was located to- gether in Angola and fluent in Portuguese, the offi- cial language in Angola. Analysis of 4 Design Scenarios Using the RSCM Tool We used the RSCM Tool to analyze the following RSCM Modeling Results Total cost estimates from our analysis of these sce- 4 supply chain design scenarios that EPI represen- narios are shown in Figure 3. tatives and partners identified at the initial stake- For the analysis, we referenced the “ideal base- holder workshop. line” as 100% as it represents the “true” cost of 1. A baseline scenario represented the current- running the supply chain; this reflects the added state supply chain design, including the 4-tier cost Angola would need to invest in the system re- structure, transportation strategy, and monthly gardless of any design changes. Based on the resupply frequency. results, the level-skipping approach appears to be FIGURE 2. United Nation’s Children’s Fund System Design Approach Used to Review Angola’s Expanded Program on Immunization Supply Chain Global Health: Science and Practice 2020 | Volume 8 | Number 4 765 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org FIGURE 3. Change in Total Annual Operating Cost Estimates by Design Scenario for Angola’s Immunization Supply Chain, With Costs Shown as a Percentage of “Ideal Baseline” Costs Storage Costs Transportation Costs 120% 105% 100% 100% 96% 89% 80% 60% 40% 20% 0% Total Costs (Percentage "Ideal Costs(Percentage of Baseline") Total Baseline Scenario "Ideal Baseline" Reduced Resupply Level-Skipping Scenario Frequency Scenario Scenario the most cost-efficient option, reducing annual costs operating cost numbers from the reference analy- The RSCM Tool by 7% over the current baseline and 11% over the ses. Even at the tier- and category-specific level, adequately ideal baseline. This difference translates to potential estimates were often within 65% of the reference replicated total savings of several hundred thousand US dollars analysis results. operating cost per year. Apart from the obvious reduction in It is not surprising that the RSCM Tool and the storage costs, bypassing the provincial level also numbers from the reference analyses differ somewhat in their results, lowered overall transport costs by enabling more given the differences in their underlying modeling reference efficient transportation routes from national lev- assumptions. For example, we assume identical de- analyses. el to municipalities. We presented these initial results to stake- mand and travel distances for all facilities within a holders from UNICEF’s Supply Division and the given tier, but there is often variation among real- Angola country office in August 2019, and they world facilities (e.g., large, accessible urban health will be using the results to get government buy-in facilities vs. small, remote rural health facilities). for subsequent in-depth modeling. Testing the Those differences tend to average out over a large RSCM in Angola provided an opportunity to as- sample of facilities, but even at a national level, sess the RSCM Tool’s ability to quickly estimate this assumption likely contributes to the MAPE results for stakeholders and provide guidance on values in Figure 1. The data adjustments we de- which options are worth exploring in more depth. scribed in the Methodology section also likely con- tribute to these differences. DISCUSSION Our key question, then, is determining what We tested the RSCM Tool in 2 ways: (1) by validat- constitutes an acceptable MAPE level for the com- ing it against existing supply chain costing analy- parisons shown in Figure 1. This is challenging be- ses, and (2) by using it to help inform an ongoing cause the definition of “acceptable” varies with the supply chain design review in Angola. We discuss urgency and importance of the use case. Users the outcomes of those testing processes and what who need answers very quickly or cheaply are they mean for the usability and limitations of the likely to accept larger discrepancies than those RSCM Tool. who have more resources or a larger decision at stake. For this question, the literature on forecast- Interpretation of the Validation Results ing accuracy (where MAPE is an important metric) The validation comparisons shown in Figure 1 may provide the best guidance regarding what is generally align with our expectations of what a generally considered acceptable. Landscape reviews high-level, rapid tool should be able to achieve. of published forecasts provide numerous examples The RSCM Tool was very good at replicating total of both public and private organizations willing Global Health: Science and Practice 2020 | Volume 8 | Number 4 766 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org to make strategic decisions with error rates of fine-tuning the regions where this solution should 10%–20% or more.32–34 Thus, we assume that sim- apply. ilar rates for the validation study (<10% MAPE at The RSCM Tool generally fared well with the the total cost level, and <20% MAPE for individual data and modeling challenges we encountered line items), would be generally acceptable given the during this analysis in Angola. Data visibility was “rapid” use cases we are targeting. We presented this an issue at municipality and health facility levels, proposed threshold at several global health stake- but national-level government staff were able to holder consultations, workshops, and conferences, describe the “typical” municipality and health fa- and received general approval from participants.35,36 cility easily. We were able to use proxy data and However, as we suggest earlier, the user must ulti- supporting worksheets to estimate other missing mately define what level of accuracy is warranted. values (e.g., typical demand per facility, govern- Furthermore, like the RSCM Tool, the refer- ment salary rates, and average distances between ence analyses from our validation are also estimates storage facilities at each tier). We also developed of true supply chain costs, based on other modeling an estimation method to capture the effect of sup- tools and data analyses. Thus, they are also subject plemental immunization activities, which occur in to similar accuracy challenges, including sample Angola every 6 months and create a temporary and observation bias, unreliable informants, and 30% increase in throughput during those months. assumptions about scaling and interpreting data. The RSCM Tool does not inherently consider We used these existing analyses as reference values month-to-month differences in demand volume, because they represent the best data currently but we were able to use the tool’s storage/trans- available, but they are likely not a perfect represen- port utilization estimates to add buffer capacity to tation of true supply chain costs, and the MAPE handle those temporary demand spikes. In these values shown in Figure 1 should be interpreted ways, we were able to address substantial data vis- with that in mind. Rather than being a true “accu- ibility and modeling challenges and produce racy” measure, the MAPE values represent evi- results in a relatively quick timeframe. dence that, given similar inputs, the RSCM Tool Most importantly, the process of working with can generate supply chain cost estimates that are external stakeholders to conduct this analysis comparable to other established methods. helped clarify 2 key lessons for enabling long- term external use of the RSCM Tool. Lessons From an Immunization Supply Chain 1. Users Must Be Familiar With Data Analysis and Design Context Supply Chain Concepts Although Angola’s iSC system redesign is still in The tool, while designed for ease of use, is not a process, the RSCM Tool was able to generate help- substitute for data analysis and supply chain ful information for stakeholders: knowledge. We have included various features to maximize usability of the tool (e.g., user guides, Overall estimated operating cost of the current detailed interpretation notes, formatting to high- supply chain light errors and omissions), but we cannot predict The rough magnitude of cost savings that could all the possible analysis situations and challenges result from implementing an improved iSC that future users might encounter. The user must design be capable of making informed judgments about how to utilize imperfect data sources and align The estimated cost of addressing current cold them with the RSCM tool’s structure, and how to chain equipment shortages handle novel modeling situations. Thus, a techni- Evidence for the types of improvements (i.e., cal or logistics officer or someone familiar with level skipping) that will likely have the largest assessing supply chain operating costs is likely to financial impact be the ideal long-term user of the RSCM Tool Having these data points early in the design pro- within a government or partner organization. cess can help the EPI better prioritize the effort it puts into iSC design improvements (and compare against 2. Adopting the Tool May Require Sustained other supply chain or health system improvements). Engagement It can also inform specific requests for subsequent Transferring long-term use of the RSCM Tool to an deep-dive analyses, such as identifying optimal external organization may still require sustained modes of transport for a level-skipping solution or engagement, at least while the tool is relatively Global Health: Science and Practice 2020 | Volume 8 | Number 4 767 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org new and unknown. Our Angola analysis suc- small fix (e.g., manually adjusting a few parameters ceeded in generating interest in the tool, but it or creating a few scenarios), then the RSCM Tool was a proof-of-concept for many stakeholders. may still be suitable. They needed to see that initial analysis conducted before they would consider using the tool them- Broadening Access to Supply Chain Design selves the next time around. Thus, while we Analysis designed the tool so that anyone with Excel can Given the RSCM Tool’s characteristics and limita- download it and understand how to use it, we rec- tions, we can envision it being useful in a wide We can envision ognize that in many cases effective dissemination range of situations where health system leaders the RSCM Tool will involve direct engagement with country and need quick, high-level supply chain cost and being useful in partner stakeholders over the course of multiple design insights but lack the resources or time situations where use cases and analyses. for a comprehensive data collection and in- health system depth modeling analysis. leaders need Limitations of the RSCM Tool Methodology quick, high-level Although the RSCM Tool can cover a wide range Advocating for Supply Chain Initiatives supply chain cost of supply chain designs and contexts, in the fol- Building initial political support for supply chain and design lowing situations, the tool’s structure and assump- improvements often requires advocates to dem- insights but lack tions are likely to create bias or error in the results. onstrate the potential cost and impact of those the resources or In some supply chains a single “typical” facility improvements but until that support exists, resources time for a may be difficult to define. This can occur for regions are often unavailable to conduct in-depth and costly comprehensive with unusual geographies, or with facilities highly analysis. Being able to quickly generate high-level, data collection concentrated in 1 area. It could also include supply country-specific data can help advocates more effec- and in-depth chains with extremely variable demand over time tively build initial political support. modeling or across facilities. These supply chains likely re- analysis. quire more detailed data to model in the RSCM Prioritizing Health Investment Decisions Tool, since built-in worksheets and proxy data Health system leaders must allocate funding may be less representative, and assumptions may across numerous initiatives to maximize health change significantly between different scenarios. impact but cannot intensively analyze all potential Some supply chains utilize different strategies options. High-level supply chain cost data can help for different subregions or program areas. If supply leaders better compare supply chain improve- chain designs are not consistent across products or ments with other health investments. facilities (e.g., a region that uses a different distribu- tion strategy than the rest of the country), the RSCM Tool requires the user to create separate Streamlining Traditional Supply Chain Redesign models for each design and then aggregate the Processes results. As our engagement in Angola demonstrated, the For small supply chain networks, a single out- RSCM Tool can serve as a preliminary filter for lier facility can greatly influence results. While the more in-depth modeling analysis, quickly narrow- RSCM Tool can model very small regions (e.g., a ing down a wide range of initial design possibili- district with 20 health facilities), there is a much ties. In this way the RSCM Tool can enable a greater risk that a single outlier facility could more targeted use of complex, expensive model- skew the results. Those outliers will tend to average ing software by allowing to tools to focus only on out over a large enough sample, so our analyses the options with the most potential. Sequencing thus far have typically modeled at least an entire both tools in this fashion can facilitate a quicker province/region (generally more than 100 health and lower-cost supply chain redesign process that facilities). still yields sufficient detail where needed (e.g., for These limitations are an important consider- final budgeting and implementation planning). ation when deciding whether to use the RSCM Tool for a given analysis. If addressing the limita- Informing Real-Time Workshop/Meeting tion is serious enough that it requires a large quan- Discussions tity of individual health facility data to resolve, then When starting from a pre-existing data template, users should also consider other modeling tools that the RSCM Tool can generate scenario analyses in can utilize detailed facility-level data. However, if a matter of minutes or hours. This opens the possi- the limitation can be addressed with a relatively bility of conducting supply chain analyses in Global Health: Science and Practice 2020 | Volume 8 | Number 4 768 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org everyday government/donor meetings or addres- Acknowledgments: The authors would like to thank participants at the sing workshop questions in real-time. 2018 Global Health Supply Chain Summit and 2019 Reproductive Health Supplies Coalition, as well as colleagues from the United States Agency for International Development, John Snow, Inc., Global Health Supply Chain Program, United Nations Children’s Fund (UNICEF) Supply Tailoring Funding Allocations Division, UNICEF Angola, Pharmaceutical Systems Africa, and Merck for Many donor and government organizations allo- Mothers for their valuable contributions in shaping the scope and direction of this project. This work would not have been possible without cate supply chain funding as a percentage of com- financial support from the Bill & Melinda Gates Foundation, Gavi, the modity value,22 which is a quick method but often Vaccine Alliance, and the Reproductive Health Supplies Coalition. uncorrelated with logistics costs. The RSCM Tool can provide quick estimates based on a more pre- Competing interests: None declared. dictive attribute (volume) and tailored to a specific country context. REFERENCES 1. Management Sciences for Health (MSH). MDS-3: Managing Access to Medicines and Health Technologies. MSH; 2012. Accessed Validating Logistics Company Bids October 29, 2020. https://www.msh.org/sites/default/files/ Outsourcing supply chain activities is another sit- mds3-jan2014.pdf uation where cost information is valuable but does 2. Seidman G, Atun R. Do changes to supply chains and procurement not warrant its own extensive study. Having a processes yield cost savings and improve availability of pharmaceu- ticals, vaccines or health products? A systematic review of evidence readily available approximation of supply chain from low-income and middle-income countries. BMJ Glob Health. costs can help donors and governments more ef- 2017;2(2):e000243. CrossRef. Medline fectively negotiate with private logistics compa- 3. World Health Organization (WHO); Bigdeli M, Peters DH, Wagner AK. nies when they are bidding for services. Medicines in Health Systems: Advancing Access, Affordability and Appropriate Use The use scenarios described in this article are . WHO; 2014. Accessed October 29, 2020. https://apps.who.int/iris/bitstream/handle/10665/179197/ not well-served by the existing landscape of sup- 9789241507622_eng.pdf ply chain tools, as they are typically diffuse, 4. Donato S, Parry J, Roth S. ADB Brief: Strong Supply Chains short-lived, and often too small or early stage to Transform Public Health. Asian Development Bank; 2016. Accessed warrant a significant resource investment for October 29, 2020. https://www.adb.org/sites/default/files/ analysis on their own. However, they are still im- publication/214036/strong-supply-chains.pdf portant in building towards and sustaining supply 5. Yadav P. Health product supply chains in developing countries: di- agnosis of the root causes of underperformance and an agenda for chain outcomes. By providing stakeholders with reform. Health Syst Reform. 2015;1(2):142–154. CrossRef. Medline rapid, data-driven insights in these types of diffuse By providing 6. WagnerAK,GravesAJ,ReissSK,LeCatesR,ZhangF,Ross-DegnanD. stakeholders with situations, we can better initiate and sustain policy Access to care and medicines, burden of health care expenditures, and discussions about supply chain, more efficiently risk protection: results from the World Health Survey. Health Policy. rapid, data-driven build consensus around the right types of solutions, 2011;100(2-3):151–158. CrossRef. Medline insights, we can and more effectively generate political will for 7. Glassman A, Oroxom R, Silverman R, Madan Keller J, Kenney C, better initiate Global Immunization and Gavi: Five Priorities for the larger-scale supply chain analyses and initiatives. Schnabel L. policy discussions Next Five Years. Center for Global Development; 2019. Accessed October 29, 2020. https://www.cgdev.org/sites/default/files/ about supply CONCLUSION global-immunization-and-gavi-five-priorities-next-five-years.pdf chain, more 8. Mihigo RM, Okeibunor JC, O’Malley H, Masresha B, Mkanda P, efficiently build Identifying, costing, analyzing supply chain design Zawaira F. Investing in life saving vaccines to guarantee life of future consensus around improvements has traditionally been a highly generations in Africa. Vaccine. 2016;34(48):5827–5832. CrossRef. solutions, and time- and resource-intensive process. The RSCM Medline more effectively Tool reduces these barriers, foregoing some degree 9. Rosen JE, Bancroft E, Hasselback L, Levin C, Mvundura M, Tien M. Last Mile Costs of Public Health Supply Chains in Developing generate political of detail and accuracy to minimize data collection Countries: Recommendations for Inclusion in the United Nations will for supply and enable quick turnaround of results. We tested OneHealth Model . United States Agency for International chain analyses and validated the RSCM Tool and found it capable Development, DELIVER PROJECT; 2012. https://www.psmtoolbox. of replicating high-level results of more traditional org/wp-content/uploads/2017/11/LastMileCost.pdf and initiatives. costing and modeling approaches. It also adapted 10. Shretta R, Johnson B, Smith L, et al. Costing the supply chain for de- livery of ACT and RDTs in the public sector in Benin and Kenya. well to existing country-level supply chain rede- Malar J. 2015;14(1):57. CrossRef. Medline sign processes, helping generate quick preliminary 11. United States Agency for International Development (USAID). USAID results that guide more in-depth modeling and de- Vision for Health Systems Strengthening 2015–2019. USAID; 2015. cision making. We believe the RSCM Tool can Accessed October 28, 2020. https://www.usaid.gov/sites/ help health system leaders make more timely and default/files/documents/1864/HSS-Vision.pdf. informed supply chain decisions, helping ensure 12. The Global Fund. Message from the Executive Director-supply chain processes. Published April 28, 2017. Accessed February 6, 2020. efficient and reliable access to health products https://www.theglobalfund.org/en/oig/updates/2017-04-28- that are critical to improving health outcomes. message-from-the-executive-director-supply-chain-processes/ Global Health: Science and Practice 2020 | Volume 8 | Number 4 769 Rapid Modeling Tool for Evaluating Health Supply Chain Designs www.ghspjournal.org 13. Bill and Melinda Gates Foundation. Health systems strengthening: 25. United Nations Population Division. World population prospects ensuring effective health supply chains. Published March 7, 2017. 2019. Published 2019. Accessed October 28, 2020. https:// Accessed October 28, 2020. https://gcgh.grandchallenges.org/ population.un.org/wpp/Download/Standard/Population/ challenge/health-systems-strengthening-ensuring-effective-health- 26. United Nations Population Fund (UNFPA). Contraceptive price supply-chains-round-19 indicators for the year 2017. Published 2018. Accessed October 28, 14. Vledder M, Friedman J, Sjöblom M, Brown T, Yadav P. Improving 2020. https://www.unfpa.org/resources/contraceptive-price- supply chain for essential drugs in low-income countries: results from indicator-year-2017 a large scale randomized experiment in Zambia. Health Syst Reform. 27. United States Agency for International Development (USAID). 2019;5(2):158–177. CrossRef. Medline Couple years of protection (CYP) conversion factors. Updated June 2, Evidence Brief: System Design 15. World Health Organization (WHO). 2019. Accessed October 28, 2020. https://www.usaid.gov/ Approach to Improve the Immunization Supply Chain . WHO; 2018. global-health/health-areas/family-planning/couple-years- Accessed October 28, 2020. https://apps.who.int/iris/bitstream/ protection-cyp handle/10665/272853/WHO-IVB-18.01-eng.pdf 28. The Demographic and Health Survey Program STATcompiler. 16. Lee BY, Haidari LA, Prosser W, et al. Re-designing the Mozambique Accessed October 28, 2020. http://www.statcompiler.com vaccine supply chain to improve access to vaccines. Vaccine. 2016;34(41):4998–5004. CrossRef. Medline 29. United States Agency for International Development, DELIVER PROJECT. Master Listing of Product Volumes and Weights. 2008. 17. Brown ST, Schreiber B, Cakouros BE, et al. The benefits of redesign- ing Benin’s vaccine supply chain. Vaccine. 2014;32(32):4097– 30. Daff BM, Seck C, Belkhayat H, Sutton P. Informed push distribution of 4103. CrossRef. Medline contraceptives in Senegal reduces stockouts and improves quality of family planning services. Glob Health Sci Pract. 2014;2(2):245– 18. Sarley D, Mahmud M, Idris J, et al. Transforming vaccines supply 252. CrossRef. Medline chains in Nigeria. Vaccine. 2017;35(17):2167–2174. CrossRef. Medline 31. United Nations Children’s Fund (UNICEF). System design approach. Accessed October 28, 2020. https://www.technet-21.org/ 19. Gavi The Vaccine Alliance. Strengthening the Immunisation Supply iscstrengthening/en/system-design-approach Chain. Gavi; 2016. Accessed October 29, 2020. https:// peoplethatdeliver.org/ptd/download/file/fid/424 32. E2Open. 2018 Forecasting and Inventory Benchmark Study. E2Open; 2018. 20. United States Agency for International Development (USAID), DELIVER PROJECT. USAID j DELIVER PROJECT Final Country Report: 33. Nicolaisen MS, Driscoll PA. Ex-post evaluations of demand forecast Tanzania. USAID, DELIVER PROJECT; 2016. https://deliver.jsi. accuracy: a literature review. Transp Rev. 2014;34(4):540–557. com/wp-content/uploads/2016/12/FinaCounRepo_TZ.pdf CrossRef 21. Republic of Ghana Ministry of Health (MOH). Health Commodity 34. Basar MS, Küçükönder H. Measuring the correlation between Supply Chain Master Plan. MOH; 2012. Accessed October 29, commercial and economic states of countries (B2G relations) 2020. http://iaphl.org/wp-content/uploads/2016/05/ and the E-Government Readiness Index by using neural GhanaSCM-2013.pdf networks. Open J Business Management. 2014;02(02):110–115. 22. McCord J, Tien M, Sarley D. Guide to Public Health Supply Chain CrossRef Costing: A Basic Methodology . United States Agency for 35. Krautmann M, Thomas D. Improving health supply chain design effi- International Development, DELIVER PROJECT; 2013. Accessed ciency through rapid and flexible cost modeling. Paper presented at: October 29, 2020. https://publications.jsi.com/JSIInternet/Inc/ Global Health Supply Chain Summit; November 28, 2018; Lusaka, Common/_download_pub.cfm?id=18156&lid=3 Zambia. http://ghscs.com/wp-content/uploads/2018/12/ 23. United Nations Children’s Fund (UNICEF). System Design Summit 1.3Improving-health-SC-through-rapid-cost-modeling82.pdf Final Report . UNICEF; 2017. 36. Krautmann M, Thomas D. A rapid modeling tool to improve repro- 24. World Health Organization (WHO). Effective Vaccine Management ductive health supply chain efficiency. Paper presented at: (EVM) Assessment Tool. Accessed October 29, 2020. https://www. Reproductive Health Supplies Coalition- General Membership who.int/immunization/programmes_systems/supply_chain/evm/ Meeting; March 27, 2019; Kathmandu, Nepal. https://www. en/index3.html rhsupplies.org/gmm2019/schedule.php#expo Peer Reviewed Received: May 10, 2020; Accepted: October 22, 2020; First published online: November 20, 2020 Cite this article as: Krautmann M, Zameer M, Thomas D, Phillips-White N, Costache A, Leroueil P. A rapid cost modeling tool for evaluating and im- proving public health supply chain designs. Glob Health Sci Pract. 2020;8(4):759-770. https://doi.org/10.9745/GHSP-D-20-00227 © Krautmann et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00227 Global Health: Science and Practice 2020 | Volume 8 | Number 4 770 ORIGINAL ARTICLE Lessons Learned From Implementing Prospective, Multicountry Mixed-Methods Evaluations for Gavi and the Global Fund Emily Carnahan,a Nikki Gurley,a Gilbert Asiimwe,b Baltazar Chilundo,c Herbert C. Duber,d,e Adama Faye,f Carol Kamya,b Godefroid Mpanya,g Shakilah Nagasha,b David Phillips,d Nicole Salisbury,a Jessica Shearer,a Katharine Shelley,a for the Gavi Full Country Evaluations Consortium; and Global Fund Prospective Country Evaluation Consortium Key Findings ABSTRACT Introduction: As global health programs have become increas- n We present 5 key lessons distilled from 7 years ingly complex, corresponding evaluations must be designed to of experience implementing evaluations in assess the full complexity of these programs. Gavi and the 7 countries, which include the importance of: Global Fund have commissioned 2 such evaluations to assess the full spectrum of their investments using a prospective mixed- 1. Including an inception phase to engage stakeholders methods approach. We aim to describe lessons learned from and inform a relevant, useful evaluation design implementing these evaluations. 2. Aligning on the degree to which the evaluation is Methods: This article presents a synthesis of lessons learned embedded in the program implementation based on the Gavi and Global Fund prospective mixed-methods 3. Monitoring programmatic, organizational, or con- evaluations, with each evaluation considered a case study. The textual changes and adapting the evaluation lessons are based on the evaluation team’s experience from over accordingly 7 years (2013–2020) implementing these evaluations. The Centers 4. Hiring evaluators with mixed-methods expertise and us- for Disease Control and Prevention Framework for Evaluation in ing tools and approaches that facilitate mixing methods Public Health was used to ground the identification of lessons learned. 5. Contextualizing recommendations and clearly com- Results: We identified 5 lessons learned that build on existing municating their underlying strength of evidence evaluation best practices and include a mix of practical and con- Key Implications ceptual considerations. The lessons cover the importance of (1) including an inception phase to engage stakeholders and in- form a relevant, useful evaluation design; (2) aligning on the de- n Global health initiatives, particularly those funding or implementing complex interventions, should consider gree to which the evaluation is embedded in the program embedding evaluations to understand how and why implementation; (3) monitoring programmatic, organizational, the programs are working to adapt as necessary and or contextual changes and adapting the evaluation accordingly; maximize impact. (4) hiring evaluators with mixed-methods expertise and using tools and approaches that facilitate mixing methods; and (5) con- n Evaluators of complex interventions should continue to textualizing recommendations and clearly communicating their share lessons learned related to balancing stakeholder underlying strength of evidence. “ ” “ ” priorities, aligning on breadth versus depth of the Conclusion: Global health initiatives, particularly those leverag- evaluation scope and ensuring use of the evaluation ing complex interventions, should consider embedding evalua- findings. tions to understand how and why the programs are working. These initiatives can learn from the lessons presented here to in- form the design and implementation of such evaluations. a PATH, Seattle, WA, USA. INTRODUCTION b Infectious Diseases Research Collaboration, Kampala, Uganda. omplex interventions—those composed of several c University of Eduardo Mondlane, Maputo, Mozambique. C d interacting components, sometimes with nonlinear Institute for Health Metrics and Evaluation, University of Washington, Seattle, — WA, USA. causal pathways are widely used to tackle complex 1,2 e Department of Emergency Medicine, University of Washington, Seattle, WA, global health challenges. As programs and interven- USA. tions have become increasingly multidimensional, corre- f Institut de Santé et Développement/University Cheikh Anta Diop, Dakar, Senegal. sponding evaluations must be designed to assess the full g PATH, Kinshasa, Democratic Republic of the Congo. complexity of these programs. Consequently, evaluations Correspondence to Emily Carnahan ([email protected]). may need to consider not only programmatic outcomes, Global Health: Science and Practice 2020 | Volume 8 | Number 4 771 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org but also other outputs and outcomes across the sys- Fund Prospective Country Evaluation (PCE) from tem to understand how to improve programs to 2017 to 2021 and provides oversight through the achieve impact. The goal of these evaluations is to TERG Secretariat. Like the FCE, the PCE aims to gen- understand not only what happened as a result of erate evidence on how Global Fund processes and the program, but crucially why the change oc- policies are enacted in real time in countries to curred. This need has resulted in an increased use achieve Global Fund objectives.10 of mixed methods, emergence of prospective PATH and the Institute for Health Metrics and approaches, and increased emphasis on process Evaluation (IHME) at the University of Washington – evaluation.3 5 have served as the global evaluation partners (GEPs) Gavi, the Vaccine Alliance (Gavi) and the leading a consortium of country evaluation partners Global Fund to Fight AIDS, Tuberculosis, and (CEPs) for the FCE and PCE. These evaluations cover Malaria (the Global Fund) are large multilateral the full spectrum of Gavi/Global Fund support, in- organizations funding country governments and cluding linkages between inputs, activities, outputs, partners to implement necessarily complex inter- outcomes, and impact. A variety of data sources and ventions to improve public health. In 2018, the methods are used to triangulate evidence including funding disbursement of both organizations to- resource tracking, process evaluation (document re- 6,7 taled nearly $USD 4.5 billion, which is being view, meeting observation, and key informant inter- used for packages of programs including vaccine views), root cause analysis, social network analysis, purchasing, cold chain improvements, malaria secondary data analysis, geospatial analyses, value- prevention programs, HIV treatment programs, for-money assessments, and impact modeling tuberculosis control programs, and general health (complete methods available elsewhere11–13). The systems support. Each organization has commis- evaluations aimed to understand how Gavi/Global sioned prospective mixed-methods evaluations to Fund policies and processes translate into country- examine the implementation, outcomes, and im- level implementation to provide actionable, relevant pact of these complex interventions. We define a insights to improve program implementation. Both prospective evaluation as an approach for examin- evaluations were conducted in multiple countries to ing implementation processes and interventions produce country-specific and cross-country synthe- forward in time, which has several advantages sis findings to meet the needs of country and global over retrospective evaluation, including deeper stakeholders. The findings have successfully influ- exploration of local context and implementation enced Gavi/Global Fund policies and processes, barriers and facilitators, ability to monitor phases and it has been suggested that these types of eva- of intervention implementation, and flexibility luations can be used for other global financing built into the design to incorporate emerging eval- mechanisms or initiatives.14 uation questions. Mixed-methods approaches are Our approach has shifted over time to reflect increasingly recognized as critical for health sys- learnings gained through implementing these tems research in low- and middle-income country evaluations since 2013.11 This article adds to the contexts,8 but definitions are numerous and var- 8 existing evaluation literature, and it expands on a ied. We draw from Ozawa and Pongpirul, who complementary article from the Zambia FCE “ define these approaches as evaluations that in- team’s perspective15 by taking a broader cross- tentionally integrate or combine quantitative and country view of lessons learned from 2 prospec- qualitative data to maximize the strengths of each, tive mixed-methods evaluations. We present les- to answer questions that are inadequately an- sons learned across the evaluation life cycle to swered by one approach.” inform the implementation of ongoing or future Gavi’s Evaluation Advisory Committee, a sub- complex evaluations. committee of the Gavi Board composed of indepen- dent evaluation advisors, commissioned the Gavi Full Country Evaluations (FCE) from 2013 to METHODS 2018. The FCE was funded by Gavi and managed by To generate lessons learned, we utilized our expe- the Monitoring and Evaluation (M&E) team within rience conducting prospective mixed-methods the Gavi Secretariat. The FCE aimed to identify dri- evaluations as part of the Gavi FCE and the vers of vaccine coverage and equity, with an empha- Global Fund PCE, considering each evaluation as sis on Gavi’s support of national immunization a case study. Insights came primarily from indivi- programs.9 TheGlobalFundTechnicalEvaluation duals who were involved in the implementation Reference Group (TERG), an independent advisory of the evaluation, both GEPs (PATH and IHME) group of the Global Fund, commissioned the Global that oversaw the evaluations and conducted Global Health: Science and Practice 2020 | Volume 8 | Number 4 772 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org cross-country synthesis, and CEPs that were pri- omission does not mean they are not important marily responsible for data collection, analysis, in evaluation practice. and reporting in their country. The CEPs included research organizations, academic institutions, and nonprofit organizations based in the focus countries RESULTS for each evaluation (FCE: Bangladesh, Mozambique, Uganda, Zambia; PCE: Democratic Republic of the Lesson 1: Include an Inception Phase to Congo, Guatemala, Senegal, Uganda). Engage Stakeholders Throughout the evaluations, GEPs and CEPs For multistakeholder evaluations of complex generated insights through periodic internal interventions, the evaluation team and donors after-action reviews and systematic reflection ses- should include an inception phase to focus on sions for adaptive management.16 The GEPs cate- stakeholder engagement and evaluation design. Support by high-ranking government officials gorized insights according to the Framework for Inception phases Evaluation in Public Health (Figure)17 and com- and donor organizations during the inception were crucial to pared our experience-based insights with existing phase can facilitate early stakeholder engagement. have dedicated best practices within the framework to elucidate The FCE and PCE were each designed with an critical differences. This framework was chosen inception phase of 4 and 6 months, respectively. time for a for its straightforward, comprehensive summary Given the complex nature of the evaluations, the consultative and of the evaluation life cycle and its widespread use. inception phases were crucial to have dedicated collaborative Its key steps included engaging stakeholders; de- time for a consultative and collaborative approach approach to scribing the program; focusing evaluation design; to engage stakeholders in developing a compre- engage gathering credible evidence; justifying conclu- hensive understanding of the programs to be eval- stakeholders in sions; and ensuring use and sharing lessons. uated and refining the evaluation priorities and understanding the We report on the lessons learned that add new approach. Engaging stakeholders can improve programs to be insights to existing best practices and are likely to evaluation design and relevance, facilitate data evaluated and be the most relevant to other teams undertaking collection, and increase the likelihood that evalu- 18–20 refine the complex prospective evaluations (Box). Other les- ation findings are used. priorities and sons learned that reinforce existing practices were In the inception phases, we first relied on approach. omitted in the interest of space, although such CEPs’ knowledge of the local context, reviews of FIGURE. Centers for Disease Control and Prevention Framework for Evaluation in Public Health17 Global Health: Science and Practice 2020 | Volume 8 | Number 4 773 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org BOX. Summary of Lessons Learned from Implementing Prospective, Mixed-Methods Evaluations Lesson 1: For multistakeholder evaluations of complex interventions, the evaluation team and donors should include an inception phase to focus on stakeholder engagement and evaluation design. Support by high-ranking government offi- cials and donor organizations during the inception phase can facilitate early stakeholder engagement. Lesson 2: In a prospective process evaluation, the donor and evaluation team should align on the degree to which the evaluation is embedded in the program implementation; a quasi-embedded approach can balance objectivity and learn- ing. Expectations for program stakeholders’ engagement in the evaluation should be clearly communicated by the eval- uation team. Lesson 3: In evaluations of complex interventions in which the programs, organizations, or contexts are constantly evolving, the evaluation team needs to continuously monitor changes and adapt the evaluation. The evaluation plan should be designed with enough flexibility to adjust evaluation questions and approaches to respond to changes; to sup- port this, buy-in from the donor organization is essential. Lesson 4: To successfully mix methods in a complex evaluation, evaluation teams should ideally include individuals with experience across methods or at minimum, co-locate individuals with various methods backgrounds. Tools and approaches—such as collaborative data review meetings, root cause analyses, and Tableau dashboards—can help to bridge any divide between quantitative and qualitative methods expertise. Lesson 5: In evaluating complex adaptive interventions, the heightened need for attention to feasibility and context of recommendations means evaluators should clearly communicate the strength of evidence underlying each finding and should consider engaging stakeholders in the process of refining findings and recommendations. relevant technical documents, and the organiza- the same level of engagement from the Global tional structure of key institutions to identify rele- Fund Secretariat in the inception phase, in part be- vant stakeholders. We held face-to-face meetings cause the PCE was commissioned independently with individuals or small groups of stakeholders by the TERG. The limited early engagement from to introduce the evaluation, which was essential the Global Fund Secretariat resulted in early chal- to get buy-in from key government officials. lenges for stakeholder buy-in, with downstream These meetings were followed by half- or full-day consequences in terms of accessing information, kick-off meetings in each country with a wide aligning the evaluation findings with decision range of stakeholders representing ministries of making timelines at the Secretariat level, and en- suring widespread dissemination and use of syn- health and finance, implementing partners, tech- thesis findings. nical partners, civil society, and Gavi/Global Fund. In many cases, the kick-off meetings were attended or endorsed by high-ranking stake- Lesson 2: Align on the Approach to holders such as senior government officials who Embedding the Evaluation in the Program encouraged support for the evaluation. For exam- Implementation In a prospective process evaluation, the donor and ple, the Permanent Secretaries of the Ministry of evaluation team should align on the degree to Health opened the FCE inception phase stake- which the evaluation is embedded in the program holder meetings in Uganda and Zambia and the implementation; a quasi-embedded approach can Minister of Health in Senegal presided over the balance objectivity and learning. Expectations for opening ceremony of the PCE. Support from program stakeholders’ engagement in the evalua- high-ranking officials paved the way for easier ac- tion should be clearly communicated by the eval- cess to other government officials and partners for uation team. evaluation data collection and contributed to the The continuum of potential evaluation designs sense of legitimacy of the evaluation, thereby im- ranges from a purely external evaluation that is proving the likelihood that findings would be entirely independent of the program implementa- used. tion to a fully embedded evaluation that is internal Support from funders also helped facilitate to the implementation team.21 Purely external support for the evaluation. At the outset, a formal approaches may be more objective, but they have letter from the Gavi CEO was shared with country limited ability to understand changing program governments to endorse the FCE. During the in- implementation, thereby potentially limiting the ception phase, the Gavi M&E team and the evalu- usefulness of the evaluation. A more embedded ation team jointly met with key Ministry of Health approach allows for more collaboration and feed- personnel and other stakeholders, signaling Gavi’s back loops between the evaluation and program support for the evaluation. The PCE did not have teams to adapt the evaluation to shifts in context, Global Health: Science and Practice 2020 | Volume 8 | Number 4 774 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org programs, or priorities.21 In evaluations of complex Over time, the CEPs have become increasingly em- adaptive program implementation approaches, bedded in country programs, for example, being some degree of embeddedness to understand these added to standing program meeting invites, which shifts is appropriate. Process evaluation in particu- also meant they were on the email distribution list lar requires collaborative and trusting relationships to receive meeting minutes and other key docu- with stakeholders involved in the program imple- ments. While this involvement has enabled CEPs to mentation to facilitate access to information.4 And, track the unfolding processes in real time, gain ac- as noted in lesson 1, engaging stakeholders in the cess to essential documents and data sources, and evaluation can encourage uptake of evaluation share back emerging findings to improve program – findings.18 20 The FCE and PCE took a quasi- implementation, it has also made it challenging for We used a quasi- embedded approach to preserve evaluation objec- CEPs to maintain evaluation independence (or a embedded tivity while collaborating closely with stakeholders perception of independence). In some contexts, approach to to support evaluation relevance, data access, and CEPs joining meetings solely as observers was not preserve — use of findings. acceptable they were expected to contribute if objectivity while — The quasi-embedded approach can encourage they wanted to keep being invited thus, they collaborating with 25 timely learning through feedback loops between shifted into participant observers. The Zambia stakeholders to — the evaluation teams and programs and messag- FCE team highlighted their approach to provide support ing the evaluation in this way, as a “learning plat- meeting notes as a way of adding value,15 and across evaluation form,” helped increase stakeholder buy-in. During all CEPs, evaluator reflexivity was used to balance – relevance, data the initiation of the FCE and PCE, there were con- independence and embeddedness.11,26 28 Over access, and use of cerns that country stakeholders who were the time, the CEPs and stakeholders established shared findings. most familiar with independent outcome evalua- expectations for engagement. This establishment of tions would be resistant to the evaluation if they shared expectations—between evaluators and stake- felt like they were being audited. Gavi anticipated holders, as well as evaluators and donors—should be this concern and emphasized to stakeholders that discussed at the outset and revisited throughout the the FCE was not an evaluation of country pro- evaluation life cycle. grams per se, but of Gavi’s policies and processes, Although the FCE and PCE have used a quasi- which helped to increase stakeholder buy-in. For embedded approach to balance objectivity and the PCE, we shifted our framing to explain the learning, there has been an ongoing tension in evaluation as a learning platform that could pro- how to strike this balance in ensuring use of find- vide support to stakeholders, help answer their ings. Evidence uptake and knowledge translation priority evaluation questions, and provide evi- rarely occur spontaneously and must be supported dence or recommendations to improve their pro- through a combination of “push,”“pull,” and “ex- gram implementation. Many PCE stakeholders change” activities.29 As a result of the messaging of were initially unfamiliar with or had never en- the evaluation as a learning platform and the gaged with a prospective evaluation in practice, embeddedness of evaluators, the evaluation team so the concept of a learning platform was more in- was perceived as being well positioned to engage tuitive. This framing also facilitated buy-in by stakeholders in these knowledge translation activ- differentiating the PCE from past Global Fund eva- ities; however, encouraging the uptake of recom- luations that some country stakeholders perceived mendations also risked compromising evaluation as top-down audits rather than learning opportu- independence. To create accountability with stake- nities. The learning platform positioned CEPs and holders for acting on evaluation findings, while country stakeholders as partners in learning and preserving evaluator independence, the Gavi opened the door for a collaborative relationship. Alliance provided an annual “management re- This approach is in line with calls for more partici- sponse” to the FCE findings and recommenda- patory and collaborative models for learning tions. The management response reported how The initial and evaluation in the international development Gavi had used each finding/recommendation.30 messaging of the field.22–24 This approach could potentially be expanded to FCEandPCEledto This initial messaging of the FCE and PCE as PCE country stakeholders or the Global Fund amore partnerships focused on learning set the stage for Secretariat to create more accountability for the collaborative a more collaborative relationship between evalua- use of findings, while maintaining independence relationship tors and program implementers. The CEPs estab- of the evaluators. Ultimately, it is critical for between lished close relationships with country stakeholders the evaluation team and donor to align on the na- evaluators and who were able to share documents and data, extend ture of collaboration and the role of the evalua- program meeting invitations, and serve as key informants. tion team at the outset of the evaluation because implementers. Global Health: Science and Practice 2020 | Volume 8 | Number 4 775 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org it has implications for how evaluators are re- M&E team and Global Fund TERG at the global ceived and how data are collected, as well as the level. Table 1 summarizes the approaches taken adoption of findings. by the FCE/PCE teams to maintain direct access to stakeholders who could provide insights on the Lesson 3: Continuously Monitor Changes and changes to Gavi/Global Fund policies and process- Adapt the Evaluation es. Weekly calls with the Gavi M&E team and In evaluations of complex interventions in which TERG Secretariat were helpful to regularly solicit the programs, organizations, or contexts are con- updates related to policies, processes, or strategies, stantly evolving, the evaluation team needs to and buy-in from the donor organization is critical continuously monitor changes and adapt the eval- in supporting the evaluation team to fully engage with its staff (e.g., the Gavi Secretariat and Global An evaluation uation. The evaluation plan should be designed with enough flexibility to adjust evaluation ques- Fund Secretariat). plan should be tions and approaches to respond to changes; to As context and priorities shifted throughout designed with support this, buy-in from the donor organization the course of the evaluation, the evaluation ques- enough flexibility is essential. tions had to be updated to reflect these changes, to adjust questions During the inception phases, the evaluation identify emerging questions the evaluation could and approaches teams directly engaged the intended users of the help to address, and ensure evaluation questions to respond to evaluation to inform the evaluation focus, priori- are useful. Buy-in from and engagement of the changes. ties, and evaluation questions (consistent with donor organization and other Secretariat staff evaluation best practices20,21). The initial terms of was essential to ensure relevance of the updated reference for both the FCE and PCE provided over- evaluation questions, and the approaches summa- arching strategic evaluation questions, which the rized in Table 1 served as an opportunity to vali- evaluation teams translated into a list of country- date revised evaluation questions. This ongoing specific and cross-country evaluation questions re- monitoring of the program context and discussion sponsive to the organizational context during the of priorities resulted in the adaptation and revision inception phase. However, as learning institutions, of evaluation questions, and ultimately a more Gavi and the Global Fund frequently update pro- flexible evaluation design. Two examples of how cesses and policies, which may affect evaluation the PCE adapted evaluation questions based on context, objectives, and priorities throughout the shifts at the country level and global level are in- course of the evaluation. (For example, Global Fund’s Operational Policy Manual31 undergoes cluded in Table 2. Although it was necessary to de- numerous revisions throughout the year.) Thus, sign the evaluation to respond to the changing understanding the program and designing a re- program context and priorities, we experienced sponsive evaluation was not limited to the incep- pros and cons associated with designing the evalu- tion phase but required an ongoing assessment as ation to encourage flexibility and adaptation over to how the program was evolving, more consistent time (Table 3). with developmental evaluation approaches.32,33 As we have strengthened relationships with The quasi-embedded approach of the CEPs fa- stakeholders, and stakeholders have a better un- cilitated program monitoring at the country level, derstanding of the scope of the evaluations, this as did collaborative relationships with the Gavi process of adaptation has become more organic, TABLE 1. Approaches Taken by the Evaluation Teams to Engage With the Donor Organizations to Monitor Program Developments Gavi Full Country Evaluation Global Fund Prospective Country Evaluation Weekly calls with the Gavi M&E team (GEP, CEP) Weekly calls with the TERG Secretariat (GEP) KIIs with Secretariat staff throughout the year, with Engagement with Secretariat staff at tri-annual a concentration of KIIs during an annual in-person TERG meetings (GEP, CEP) visit to Geneva (GEP) One-off phone calls with rotating Secretariat teams Semi-annual touchpoints with Gavi Senior scheduled by the TERG Secretariat (GEP, CEP) Country Managers (GEP, CEP) Semi-annual touchpoints with Global Fund Country Teams (GEP, CEP) Abbreviations: CEP, country evaluation partner; GEP, global evaluation partner; KII, key informant interview; M&E, monitoring and evaluation; TERG, technical evaluation reference group. Global Health: Science and Practice 2020 | Volume 8 | Number 4 776 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org TABLE 2. Examples of Changing Prospective Country Evaluation Questions Due to Shifts at the Country and Global Levels Responsive to Country-Level Shift Responsive to Global-Level Shift In Uganda, there was an unanticipated upsurge in In 2020, the Grant Portfolio Solutions team at the malaria cases in 2019, so the Prospective Country Global Fund requested inputs about challenges related Evaluation team added an evaluation question on to Global Fund monitoring and reporting processes whether and how Global Fund policies and structures and opportunities for improvement. enabled the country to respond. The Prospective Country Evaluation was able to quickly Findings indicated that several flexible aspects of the incorporate new evaluation questions into the evaluation Global Fund business model, including modifications scope and shared cross-country findings to inform the to procurement and supply chain plans, facilitated the Secretariat’s revised reporting guidance. country’s response to the malaria upsurge. TABLE 3. Pros and Cons of a Flexible Evaluation Design Pros Cons Is responsive to changing stakeholder needs, Can take months to get stakeholder consensus on thereby increasing stakeholder buy-in and the like- priorities. lihood findings will be used. Requires carefully balancing those stakeholder Has the ability to adjust to unanticipated inputs while remaining objective. implementation delays to refocus on the most time- May mean that evaluation teams are developing ly, relevant evaluation questions. evaluation tools in parallel to prospectively tracking a process that has already started. This may under- mine the planning required for intentional mixed methods approaches. with stakeholder inputs on evaluation questions to team composition and processes, we found shared more proactively and ad hoc. The GEPs/ that mixing of methods and paradigms was diffi- CEPs have also become more adept at identifying cult to achieve. evaluation priorities through ongoing process Across the consortia, a range of staffing models tracking, including areas of cross-country synthe- were represented. Some CEPs had separate quan- sis that are most relevant in informing changes to titative modeling and process evaluation teams, Gavi/Global Fund policies or processes. while others had integrated multidisciplinary teams. Aiming for a multidisciplinary team, pref- Lesson 4: Include People With Mixed- erably with multiple transdisciplinary staff that Methods Expertise on the Evaluation Team had “crossover” between methods expertise proved To successfully mix methods in a complex evalua- most successful. In cases in which CEP teams were tion, evaluation teams should ideally include indi- divided methodologically, co-locating team mem- viduals with experience across methods, or at bers helped to ensure more regular full team meet- minimum co-locate individuals with various ings to review and triangulate emerging evidence, methods backgrounds. Tools and approaches, if not full mixing of methods. such as collaborative data review meetings, root To achieve true mixing of methods and para- cause analyses, and Tableau dashboards, can help digms, it is necessary to have both a well- to bridge any divide between quantitative and integrated team with diverse expertise, as well as Without conscious qualitative methods expertise. established procedures and processes for dialogue attention to team To foster mixed-methods analysis, we learned and analysis. While conducting a mixed-methods composition and that our teams (GEP and CEP) worked best when evaluation has been an ongoing challenge for processes, we team members encompassed various disciplinary some teams, the evaluations have adopted tools found that mixing and methods backgrounds, were co-located, and and approaches to help bridge the gap between of methods and used collaborative approaches to data interpreta- quantitative and qualitative approaches. Using paradigms was tion and synthesis. Without conscious attention collaborative and interactive processes is valuable difficult to achieve. Global Health: Science and Practice 2020 | Volume 8 | Number 4 777 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org Evaluators should in facilitating mixed-methods analysis of the data context of recommendations means evaluators clearly and interpretation of findings. The PCE held joint should clearly communicate the strength of evi- communicate the CEP-GEP data review conference calls (approxi- dence underlying each finding and should consid- strength of mately bimonthly) to share updated quantitative er engaging stakeholders in the process of refining analyses, discuss data quality issues and resolu- evidence for each findings and recommendations. tions, and identify opportunities for further trian- Existing best practices focus on enhancing finding and gulation with process evaluation evidence or the credibility of conclusions by ensuring data are an- consider engaging need for additional data collection. To further fa- alyzed and systematically interpreted, findings are stakeholders in cilitate collective analysis, GEP and CEP held joint refining findings in-person analysis and report writing workshops directly linked to evidence and informed by stake- holder standards, and resulting recommendations and 2 or 3 times per year, in addition to cross-country are contextualized and actionable.17 While the recommendations. synthesis workshops at least once per year. The limitation in a more collaborative analysis process evaluator’s role is to justify the evaluation conclu- is the time and cost of engaging all evaluation sions, engaging stakeholders in the process presents partners—it is a dynamic and (potentially) non- a potential opportunity to further contextualize the linear process that is best served by face-to-face in- findings and facilitate evidence use.18 The FCE/PCE teraction and may take substantive time. teams shared preliminary findings with stake- In terms of tools, root cause analysis was a par- holders for review to ensure we were reporting ticularly effective analytic tool as it encouraged full and accurate information. Occasionally, these — participants to incorporate all the available data reviews would motivate stakeholders to share addi- — qualitative and quantitative and iteratively ex- tional evidence to be incorporated. In determining plore hypotheses collaboratively. (Example FCE when to share emerging findings, the evaluation root cause analyses have been shared else- team must balance the opportunity to gather addi- where.11,15) Similarly, Tableau dashboards were a useful tool to support interpretation of quantita- tional insight from stakeholder reviews with the tive data among team members with a range of potential risk of sharing early findings with insuffi- quantitative data skillsets; all team members had cientevidencethatcouldundermineevaluators’ access to the dashboards and would look at the credibility. quantitative results to generate questions for qual- Additionally, it is important to convey the itative follow-up. For example, the PCE visualized strength of evidence underlying evaluation con- quantitative data from Global Fund grant revi- clusions so stakeholders trust the findings and as- sions to understand budgetary shifts, and then sociated recommendations. This is particularly generated key informant interview questions to true in a mixed-methods evaluation in which understand why the shifts occurred and how they each finding relies on multiple data sources with were affecting implementation activities. varying quality. Moreover, in some settings stake- Finally, we also learned that mixed-methods holders perceived findings based solely on qualita- approaches can be more intentionally incorporat- tive evidence to be less rigorous than quantitative ed by starting from the evaluation question phras- evidence. To clearly signal the strength of evidence, ing. Over time, we shifted to evaluation questions we developed a rubric informed by GRADE and that encouraged mixed-methods data collection other evidence rating systems35 that rated the evi- “ and analysis, such as whether, why, and how dence along a 4-point scale.15 However, while the ” does X outcome occur. For example, the Uganda GRADE rubric considers study design and rates “ FCE team asked the question: What is the effec- randomized trials highly, our scale was limited to tiveness, efficiency, and country ownership of the types of evidence used in the FCE/PCE, so ran- national immunization partnerships and their domized trials were not feasible or fit-for-purpose. ” contribution to program performance? This en- Our rubric considered the extent of triangulation couraged a mixed-methods approach that includ- between data sources and the quality of the ed social network mapping, document review, sources. Table 4 shows the strength of evidence rat- and qualitative interviews to understand the ing used in the PCE, and Simuyemba et al.15 shared structure and added value of the partnership 34 the rubric used in the FCE. Each finding was pub- working on the Gavi HPV vaccine application. lished with a rating to communicate our confidence in the conclusion, accounting for data quality and Lesson 5: Contextualize Recommendations triangulation. and Clearly Communicate Strength of During the FCE, the evaluation team indepen- Evidence dently generated recommendations that were In evaluating complex adaptive interventions, the shared with global and country stakeholders. The heightened need for attention to feasibility and PCE has taken the same approach, but in some Global Health: Science and Practice 2020 | Volume 8 | Number 4 778 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org TABLE 4. Global Fund Prospective Country Evaluation Strength of Evidence Rating Rank Rationale 1 The finding is supported by multiple data sources (good triangulation) that are generally of strong quality. 2 The finding is supported by multiple data sources (moderate triangulation) of lesser quality, or the finding is supported by fewer data sources of higher quality. 3 The finding is supported by few data sources (limited triangulation) of lesser quality. 4 The finding is supported by very limited evidence (single source) or by incomplete or unreliable evidence. In the context of this prospective evaluation, findings with this ranking may be preliminary or emerging, with active and ongoing data collection to follow up. PCE countries we have used annual dissemination monitoring and evaluation adaptation (lesson meetings as an opportunity to iteratively refine 3), facilitated data access to inform a mixed- the recommendations with stakeholders. This ap- methods approach (lesson 4), and led to contextu- proach has been well received and may prove a alized findings and recommendations (lesson 5). promising practice to generate buy-in for acting A second cross-cutting theme is the balance of on the recommendations. objectivity and learning. In recent years, the eval- uation discipline has come to embrace its role in DISCUSSION adaptation and learning, and this has extended greater latitude for how evaluator reflexivity can This article presents 5 lessons distilled from over allow independence coupled with learning.26,36,37 7 years of experience (2013–2020) implementing prospective mixed-methods evaluations of Gavi A spectrum of evaluation models are available, and the Global Fund in 7 countries. While country depending on the nature of interactions between program implementers and evaluators and the de- settings were highly variable, our experiences had 21 some consistency, resulting in a mix of operational gree of embeddedness desired. In the FCE/PCE, and practical “how to” considerations, alongside the quasi-embedded evaluation approach (lesson broader considerations that are sometimes more 2) allowed for timely monitoring of the program “art than science.” context to understand and respond to changing pro- The Framework for Evaluation in Public gram needs (lesson 3). This quasi-embeddedness Health was a useful tool to ground the identifica- also allowed evaluators to communicate the strength ’ tion of lessons learned. However, while the frame- of findings to inform stakeholders action (lesson 5). work suggests a distinct, linear process for Ultimately, stakeholders should consider the level of evaluation, feedback loops existed between steps objectivity and collaboration that would make an in practice, and some steps (e.g., stakeholder en- evaluation fit-for-purpose, and let that inform the ap- gagement) were a focus throughout the duration propriate degree of embeddedness in the evaluation of the evaluations. Our lessons spanned steps in design; there is no one-size-fits-all model for evalua- the evaluation life cycle—and are often interrelat- tion of complex interventions. ed and mutually reinforcing—and therefore we Another key theme across many of the lessons decided against presenting lessons learned aligned relates to the design and focus of the evaluations. to specific steps in the Framework, instead empha- Complex interventions and evaluations of those sizing their cross-cutting nature. interventions often include multiple stakeholder Stakeholder engagement is a key theme that audiences with different evaluation priorities or Stakeholder weaves many lessons together. In the FCE/PCE, goals. The inception phase (lesson 1) should help engagement is a the inception phase was the initial touchpoint to define the scope of the evaluation and bring clarity key theme that engage stakeholders (lesson 1), but strengthening to stakeholders on what the evaluation will—and, weaves many relationships between evaluators and other importantly, will not—address. However, we also lessons together. stakeholders was an ongoing effort. The quasi- advocate for flexibility in the evaluation design embedded approach (lesson 2) facilitated these (lesson 3) to adjust evaluation questions based on relationships, particularly at the country level. shifting context, priorities, or implementation And strong relationships—based on shared approaches. A flexible evaluation design has pros trust, collaboration, and learning—between the and cons (as noted in Table 3), and this is an area evaluators and stakeholders enabled program of continued learning for the PCE, as is discussed Global Health: Science and Practice 2020 | Volume 8 | Number 4 779 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org further in the Implications section. A flexible eval- and grant activities. Over time, both the FCE and uation design requires an ongoing process of align- PCE shifted toward less breadth and more depth, ing and realigning on the evaluation questions with more focused evaluation questions and ana- and scope across multiple stakeholder audiences. lytical approaches. On reflection, it was important Continuously Overall, it has been important to continuously en- for the evaluation teams to start with a broad engaging with gage with stakeholders so they know which ques- scope to understand all the interrelated compo- stakeholders has tions have been prioritized and what types of nents of the complex interventions; with this un- been important so findings to anticipate. derstanding in place, it was possible to narrow the they know which evaluation focus to go further in depth without questions have Implications and Future Research losing the wider context. Finally, we continue to test and refine our ap- been prioritized Our evaluation approach has shifted over more proach to ensuring use of the evaluation findings and what types of than 7 years of implementation.11 As we have re- among target audiences. Lessons 1 and 2 highlight findings to fined our approach, areas still remain in which we our approach to engaging with stakeholders, anticipate. continue to learn and further refinement is re- which engenders buy-in to the evaluation and up- quired. These include balancing stakeholder prior- take of findings. Best practices emphasize tailoring ities, aligning on “breadth” versus “depth” of the dissemination strategies to stakeholders and pro- evaluation scope, and identifying approaches to viding knowledge translation support18,38; how- ensure use of the evaluation findings. ever, the FCE and PCE teams have had limited In terms of balancing stakeholder priorities, resources and capacity to support this effort. Our these multilevel, multistakeholder evaluations more formalized dissemination approaches have were designed to meet the needs of a range of focused primarily on annual written reports and country and global stakeholders. It has proven annual country-based dissemination meetings. challenging to design an evaluation that balances Annual dissemination meetings have worked well the diverse needs of distinct groups of primary to bring together a diverse set of stakeholders to dis- users with differential interests and power. CEPs cuss evaluation findings and recommendations and have been more likely to prioritize evaluation provide input on future evaluation priorities. questions identified by country stakeholders to be However, the timing of annual meetings and responsive to country needs. Conversely, our reports may not align with program implementa- oversight points of contact at Gavi and Global tion timelines or decision-making windows. Thus, Fund have been more likely to prioritize cross- it is important to have multiple modes of dissemi- country evaluation questions that can directly in- We recommend nating findings. We recommend that future eva- form policies or strategies or are responsive to that future luations are resourced to support knowledge their funders and board members. With limited evaluations are translation and more timely sharing of emerging resources, if tradeoffs needed to be made between resourced to findings (e.g., through shorter policy briefs, evalua- being responsive to global versus country priori- support tion team engagement in program meetings) to ful- ties, it was not clear which to prioritize. Striking a ly take advantage of the learning platform. knowledge balance between stakeholder priorities has been translation and an ongoing challenge. more timely A second area of continued learning is how to Limitations sharing of align stakeholders on the tradeoffs between cover- The content for this article draws solely from the emerging ing a wide breadth of topics versus going in depth experiences of the FCE and PCE evaluation teams, findings. on fewer topics. In setting the evaluation ques- meaning the lessons do not directly incorporate tions, the FCE and PCE teams have continuously the perspectives of other key stakeholders (e.g., navigated the tradeoffs between depth versus Gavi, Global Fund, Ministries of Health) on what breadth of the evaluation scope. Process tracking aspects of the evaluation worked well and added (through document review, meeting observation, value versus those needing further refinement. and key informant interviews) was intended to Furthermore, the lessons presented are not ex- understand the breadth of activities, and based on haustive; the authors’ judgment was used to de- stakeholder priorities and emerging findings, eval- termine which lessons were most novel and uation questions could be identified for further in- important to highlight. Another potential limita- depth analysis. However, in practice, it has been tion is that lessons are drawn only from the FCE challenging for CEPs to track all the processes and PCE cases, which are unique evaluations in unfolding—particularly for the PCE since it covers scale and scope and not necessarily generalizable. 3 large disease programs, with many stakeholders However, the case uniqueness also suggests Global Health: Science and Practice 2020 | Volume 8 | Number 4 780 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org lessons may be particularly relevant to other large 2. Petticrew M. When are complex interventions ‘complex’? When are global health initiatives with interest in establish- simple interventions ‘simple’? Eur J Public Health. 2011;21(4):397– 398. CrossRef. Medline ing similar multiyear, independent prospective 3. De Allegri M, Sieleunou I, Abiiro GA, Ridde V. How far is mixed evaluations of their investments, policies, and methods research in the field of health policy and systems in Africa? processes. A scoping review. Health Policy Plan. 2018;33(3):445–455. CrossRef. Medline 4. Moore GF, Audrey S, Barker M, et al. Process evaluation of complex CONCLUSION interventions: Medical Research Council guidance. BMJ. 2015;350: A key benefit of prospective mixed-methods eva- h1258. CrossRef. Medline luations is the opportunity for dynamic and con- 5. Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating tinuous learning because data are collected while complex interventions: the new Medical Research Council guidance. implementation unfolds. This means that evalua- BMJ. 2008;337:a1655. CrossRef. Medline tors can identify what is working or not working 6. Gavi, the Vaccine Alliance. Gavi, the Vaccine Alliance 2018 Annual and explore why. Although this type of evaluation Financial Report. Gavi; 2019. Accessed October 16, 2020. https:// has added value to Gavi’s and Global Fund’s un- www.gavi.org/sites/default/files/document/2019/GAVI% derstanding of their programs, this approach is a 20Alliance%202018%20Annual%20Financial%20Report.pdf new way of working for many evaluators, donors, 7. The Global Fund. Financials. Accessed October 16, 2020. https:// www.theglobalfund.org/en/financials/ and other stakeholders, meaning it can take time ... to understand and engage with. Therefore, this ar- 8. Ozawa S, Pongpirul K. 10 best resources on mixed methods re- search in health systems. Health Policy Plan. 2014;29(3):323–327. ticle presents 5 lessons distilled from over 7 years CrossRef. Medline of experience (2013–2020) implementing pro- 9. GAVI, the Vaccine Alliance. Full country evaluations. Accessed spective, mixed-methods evaluations of Gavi and October 15, 2020. https://www.gavi.org/our-impact/evaluation- the Global Fund in 7 countries. Our aim in writing studies/full-country-evaluations this article was to reflect on and share key lessons 10. The Global Fund, Technical Evaluation Reference Group. Prospective that we hope can inform the design and implemen- country evaluations. Accessed October 15, 2020. https://www. tation of future prospective evaluations of large- theglobalfund.org/en/technical-evaluation-reference-group/ scale, complex global health initiatives. Such global prospective-country-evaluations/ health initiatives, particularly those leveraging 11. Soi C, Shearer JC, Budden A, et al. How to evaluate the implemen- complex interventions, should consider embedding tation of complex health programs in low-income settings: the ap- proach of the Gavi Full Country Evaluations. Health Policy Plan. evaluations to understand how and why the pro- Forthcoming 2020. grams are working to adapt as necessary and maxi- 12. PATH. Overview of Gavi full country evaluations findings. Accessed mize impact. September 22, 2020. https://www.path.org/resources/overview- of-gavi-full-country-evaluations-findings/ Acknowledgments: The authors acknowledge the important 13. PATH. Global Fund Prospective Country Evaluation. Accessed contributions of all evaluation team members who designed and September 22, 2020. https://www.path.org/programs/health- implemented the Gavi Full Country Evaluations and the Global Fund systems-innovation-and-delivery/global-fund-prospective-country- Prospective Country Evaluations from a coalition of organizations: the Institute for Health Metrics and Evaluation (IHME), University of evaluation/ Washington (USA); PATH (USA, DRC, and Senegal); the International 14. Salisbury NA, Asiimwe G, Waiswa P, Latimer A. Operationalising Centre for Diarrhoeal Disease Research (icddr, b) (Bangladesh); the Global Financing Facility (GFF) model: the devil is in the detail. Universidade Eduardo Mondlane, Health Alliance International, BMJ Glob Health. 2019;4(2):e001369. CrossRef. Medline Manhiça Health Research Centre, and National Institute of Health (Mozambique); Infectious Diseases Research Collaboration (IDRC) 15. Simuyemba MC, Ndlovu O, Moyo F, et al. Real-time evaluation pros (Uganda); University of Zambia (Zambia); Centro de Investigación and cons: lessons from the Gavi Full Country Evaluation in Zambia. Epidemiológica en Salud Sexual y Reproductiva (CIESAR) (Guatemala); Evaluation. 2020;26(3):367–379. CrossRef and Institut de Santé et Développement/Université Cheikh-Anta-Diop 16. Ellis S, Carette B, Anseel F, Lievens F. Systematic reflection: implica- (ISED/UCAD) (Senegal). The authors also acknowledge Gavi, the Curr Dir Psychol Sci Vaccine Alliance and the Global Fund for AIDS, Tuberculosis, and tions for learning from failures and successes. . – Malaria for supporting the design and implementation of these 2014;23(1):67 72. CrossRef evaluations. (In doing so, we do not imply they endorse this document.) 17. Centers for Disease Control and Prevention (CDC). Framework for We particularly thank the Gavi Monitoring and Evaluation team and the program evaluation in public health. MMWR Recomm Rep. 1999; Global Fund Technical Evaluation Reference Group (TERG) and TERG 48(RR-11):1–40. Medline Secretariat for their partnership. Finally, we extend our thanks to all stakeholders in the evaluation countries and Geneva who participated in 18. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic the evaluations. review of barriers to and facilitators of the use of evidence by policy- makers. BMC Health Serv Res. 2014;14(1):2. CrossRef. Medline Competing interests: The authors received financial support from Gavi, 19. Preskill H, Jones N. A Practical Guide for Engaging Stakeholders the Vaccine Alliance and the Global Fund for AIDS, Tuberculosis, and in Developing Evaluation Questions. Robert Wood Johnson Malaria to conduct the evaluations. Foundation; 2009. Accessed March 18, 2020. http://hdl.handle. net/10244/683 REFERENCES 20. Patton MQ. Utilization-Focused Evaluation. SAGE Publications; 1. Medical Research Council. Developing and Evaluating Complex 2008. Interventions: New Guidance. Accessed October 16, 2020. https:// 21. Barry D, Kimble LE, Nambiar B, et al. A framework for learning mrc.ukri.org/documents/pdf/complex-interventions-guidance/ about improvement: embedded implementation and evaluation Global Health: Science and Practice 2020 | Volume 8 | Number 4 781 Lessons Learned From Mixed-Methods Evaluations www.ghspjournal.org design to optimize learning. Int J Qual Health Care. 2018;30 30. Gavi, the Vaccine Alliance. 2016 Full Country Evaluations—Alliance (Suppl_1):10–14. CrossRef Management Response. Gavi; 2016. Accessed October 15, 2020. https://www.gavi.org/sites/default/files/document/fourth- 22. Anderson MB, Brown D, Jean I. Time to Listen: Hearing People on the annual-fce-report-%282016%29—gavi-responsepdf.pdf Receiving End of International Aid. 1st ed. CDA Collaborative Learning Projects; 2012. Accessed October 15, 2020. https://www. 31. The Global Fund. Operational Policy Manual. The Global Fund; cdacollaborative.org/publication/time-to-listen-hearing-people-on- 2020. Accessed October 16, 2020. https://www.theglobalfund. the-receiving-end-of-international-aid/ org/media/3266/core_operationalpolicy_manual_en.pdf Eval Pract 23. Archibald T, Sharrock G, Buckley J, Young S. Every practitioner a 32. Quinn Patton M. Developmental evaluation. . 1994; – “knowledge worker”: promoting evaluative thinking to enhance 15(3):311 319. CrossRef learning and adaptive management in international development. In: 33. Gamble JAA. A Developmental Evaluation Primer. The J.W. Vo AT, Archibald T, eds. Evaluative Thinking. New Directions for McConnell Family Foundation; 2008. Accessed October 15, 2020. Evaluation. Wiley Periodicals, Inc., and the American Evaluation https://mcconnellfoundation.ca/wp-content/uploads/2017/07/ Association; 2018:73–91. Accessed October 15, 2020. https:// A-Developmental-Evaluation-Primer-EN.pdf www.crs.org/sites/default/files/tools-research/4_archibald_ 34. Kamya C, Shearer J, Asiimwe G, et al. Evaluating global health sharrock_et_al-2018-new_directions_for_evaluation.pdf partnerships: a case study of a Gavi HPV vaccine application process 24. United States Agency for International Development (USAID). in Uganda. Int J Health Policy Manag. 2017;6(6):327–338. Collaborating, Learning, and Adapting (CLA) Toolkit: Understanding CrossRef. Medline CLA . USAID; 2017. Accessed March 18, 2020. https:// 35. Atkins D, Best D, Briss PA, et al; GRADE Working Group. Grading usaidlearninglab.org/qrg/understanding-cla-0 quality of evidence and strength of recommendations. BMJ. 25. Jorgensen DL. Participant Observation: A Methodology for Human 2004;328(7454):1490. CrossRef. Medline Studies. Sage; 1989. 36. Jacobson C, Hughey KFD, Allen WJ, Rixecker S, Carter RW. Toward Soc Nat Resour 26. Koch T, Harrington A. Reconceptualizing rigour: the case for reflex- more reflexive use of adaptive management. . – ivity. J Adv Nurs. 1998;28(4):882–890. CrossRef. Medline 2009;22(5):484 495. CrossRef 37. Keith RE, Crosson JC, O’Malley AS, Cromp D, Taylor EF. Using the 27. Barry CA, Britten N, Barber N, Bradley C, Stevenson F. Using reflex- Consolidated Framework for Implementation Research (CFIR) to pro- ivity to optimize teamwork in qualitative research. Qual Health Res. duce actionable findings: a rapid-cycle evaluation approach to im- 1999;9(1):26–44. CrossRef. Medline proving implementation. Implement Sci. 2017;12(1):15. CrossRef. 28. Malterud K. Qualitative research: standards, challenges, and guide- Medline lines. Lancet. 2001;358(9280):483–488. CrossRef. Medline 38. Langer L, Tripney J, Gough D. The Science of Using Science: 29. Lavis J, Lomas J, Hamid M, Sewankambo NK. Assessing country- Researching the Use of Research Evidence in Decision-Making. EPPI- level efforts to link research to action. Bull World Health Organ. Centre, Social Science Research Unit, UCL Institute of Education, 2006;84(8):620–628. CrossRef. Medline University College London; 2016. Peer Reviewed Received: March 30, 2020; Accepted: September 30, 2020; First published online: November 11, 2020 Cite this article as: Carnahan E, Gurley N, Asiimwe G, et al. Lessons learned from implementing prospective, multicountry mixed methods evaluations for Gavi and the Global Fund. Glob Health Sci Pract. 2020;8(4):771-782. https://doi.org/10.9745/GHSP-D-20-00126 © Carnahan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00126 Global Health: Science and Practice 2020 | Volume 8 | Number 4 782 ORIGINAL ARTICLE Effects of a Community-Based Program on Voluntary Modern Contraceptive Uptake Among Young First-Time Parents in Cross River State, Nigeria Gwendolyn Morgan,a Anjala Kanesathasan,b Akinsewa Akiodec Key Findings ABSTRACT We implemented and evaluated a program to improve Background: Reproductive health programs for youth have large- — child spacing, modern contraceptive use, and related ly overlooked first-time parents (FTPs) defined as young women gender outcomes among first-time parents in Cross younger than 25 years old who are pregnant or already have 1 child, and their partners. To address this gap, we implemented River State, Nigeria. and evaluated a program to improve child spacing, modern con- n Contraceptive awareness, attitudes, and couples’ traceptive use, and related gender outcomes among FTPs in communication increased significantly from baseline Cross River State (CRS), Nigeria. This paper examines the effec- to endline. tiveness of FTP interventions in improving voluntary uptake of contraception. n After controlling for significant factors related to family Methods: We conducted small group sessions and home visits planning use, first-time mothers were 3.3 times more with FTPs from May to August 2018 in 2 local government areas likely and male partners 3.7 times more likely to be of CRS. A pretest–posttest study examined the effectiveness of using a modern contraceptive method at endline. these interventions regarding healthy timing and spacing of Most first-time mothers and their partners preferred pregnancy/family planning knowledge, attitudes, intentions, com- the contraceptive implant, and a smaller percentage munication, decision making, and contraceptive use. We per- chose the injectable. formed a bivariate analysis and logistic binomial regression to confirm change over time in the primary study outcome, current Key Implications use of a modern method of contraception. We also performed This experience suggests that local and state analysis of demographic characteristics and secondary outcomes governments can adapt and scale up 3 essential program (e.g., birth spacing intentions and couple communication). Results: We interviewed 338 participating first-time mothers elements: (FTMs) and 224 participating partners at baseline and endline. n Ensure the availability of modern contraceptive Important indicators of contraceptive awareness, attitudes, and methods through local health facilities couples’ communication increased significantly from baseline to n Use community-based health workers to endline. Voluntary current modern contraceptive use increased P< conduct home visits with first-time parents to provide from 26% to 79% among nonpregnant FTMs ( .000), and from 44% to 81% among partners (P<.000). After controlling tailored health information and referrals, as well as for significant factors related to family planning use, FTMs were build linkages with the formal health sector 3.3 times more likely (P<.001) and partners 3.7 times more likely n Include activities that address gender norms (P<.000) to be using a modern contraceptive method at endline. and couple dynamics to foster better alignment, Conclusion: Program participation was associated with signifi- communication, and joint action on reproductive cant improvements in voluntary uptake of modern contraceptive issues methods and multiple secondary outcomes. Even within a short timeframe, this intensive, multi-intervention effort achieved signif- icant advancements across healthy timing and spacing of preg- nancy and family planning outcomes for this vulnerable youth population. INTRODUCTION onger birth intervals, facilitated by modern contra- a International Business & Technical Consultants, Inc., Vienna, VA, USA. L b Independent consultant, Washington, DC, USA. ceptive use, are associated with reductions in mater- c Research & Communication Services, Abuja, Nigeria. nal and neonatal mortality and morbidity in low- and 1,2 Correspondence to Gwendolyn Morgan ([email protected]). middle-income countries. Adolescent mothers around Global Health: Science and Practice 2020 | Volume 8 | Number 4 783 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org the world disproportionately experience pregnancy- about family planning and use of contraception, as related death and disease when they start childbear- well as receive social support for using contraceptive ing early and have rapid repeat pregnancies. Global methods and services, are more likely to use contra- studies show that adolescents aged 15–19 years have ception.14–16 Menwhoapproveoftheirfemalepart- less access to voluntary modern contraception, use ners’ contraceptive use, provide support to obtain modern contraceptives less frequently, and have a transport to reach a facility, and provide funds and higher unmet need for modern contraception than permission to access services are critical to facilitating older women.3–5 These factors place adolescent and women’s contraceptive use in many country con- young mothers at risk of negative health outcomes6 texts, including Nigeria.17,18 Yet male partners, par- and highlight the particular vulnerabilities that young ents, and even in-laws may implicitly or explicitly women and mothers face, including those going discourage use of contraception due to concerns through pregnancy, childbirth, and childrearing for about perceived and actual side effects. They may the first time. also simply fail to give permission to visit a health fa- This global pattern is reflected in Cross River cility or give financial support for these services to State (CRS), Nigeria. As of May 2017, available new young mothers, especially those who are not national- and state-level data showed that sexual yet empowered to initiate conversations about fam- activity and motherhood began early. In CRS, ily planning. 18% of adolescent girls aged 15–19 years have In addition, adolescents and young women started childbearing.7 In addition, adolescent and themselves have their own biases and misinfor- young mothers often do not use modern contra- mation about the risks and potential side effects ception to space their second child or subsequent of contraception, such as infertility, permanent children. As a result, rapid repeat pregnancies are damage to reproductive organs, infections, or can- common, with nearly one-quarter of all children cer.19,20 Methods that interrupt the perceived nat- born less than 2 years after a sibling.7 Only 27% of ural pattern of menstruation are largely deemed sexually active adolescent girls (15–19 years) in unacceptable. Adolescents and young women of- CRS reported using a modern contraceptive meth- ten perceive that they will be more likely to expe- od (both married and unmarried).7 rience these side effects if they use long-acting As noted in a 2007 literature review by the contraception, such as an intrauterine device or World Health Organization8, the social and eco- implant. Improving the attitudes of adolescent nomic consequences of adolescent sexuality and and young mothers and their partners about the pregnancy greatly depend on an adolescent’s par- healthy timing and spacing of pregnancies (HTSP) ticular cultural, family, and community setting. In and supporting their informed knowledge and CRS (as in other parts of Nigeria and in the global voluntary use of modern contraception are partic- context), where fertility is highly valued within ular priorities for health programs, given the the institution of marriage, unmarried young higher risks of morbidity and mortality for both mothers often endure additional stigma and dis- the mother and the child.21 First-time parents crimination within their communities and fami- First-time parents (FTPs)—defined as young have largely been lies due to an early, unplanned, and unwanted women under age 25 who are pregnant or already – overlooked in pregnancy.9 11 As a result, they may experience have 1 child, and their partners—have largely reproductive adverse social consequences such as curtailment been overlooked in reproductive health programs health programs of their education, decreased mobility, financial for youth. A 2014 review of global data showed 9,12 for youth. deprivation, and increased social isolation. that many first-time mothers (FTMs) are at in- Young men who unexpectantly become fathers creased risk of poor pregnancy, delivery, and child for the first time may also face numerous chal- health outcomes, a situation compounded by lenges in continuing their education and provid- multiple factors that limit their access to timely ing adequate financial support for themselves and health information and services.22 The needs of their new family.9,13 FTPs extend beyond the scope of many adolescent Adolescent and young mothers also face criti- and youth programs, which often cater to unmar- cal barriers (financial, physical access, family per- ried clients and focus on the prevention of preg- mission, etc.) in seeking health care and services.8 nancy. Issues faced by young parents, such as Key influencers, such as parents, in-laws, hus- infant care and feeding or couple communication bands or male partners, and perhaps older co- and decision making are also not typically includ- wives, typically drive household decision making ed in family planning and pregnancy prevention as well as health care spending. Evidence shows programs aimed at women of reproductive age or that couples who discuss and jointly make decisions even married youth. Global Health: Science and Practice 2020 | Volume 8 | Number 4 784 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org To address this gap in CRS, the Evidence to through the SMGL Initiative. Using a systems ap- Action (E2A) Project, a global family planning proj- proach, SMGL strengthened state, facility, and ect funded by United States Agency for International community networks to address the 3 delays that Development, and Pathfinder International/Nigeria contribute to maternal mortality (delays in decid- launched a program to improve child spacing, vol- ing to seek appropriate services; reaching those untary contraceptive use, and related gender out- services; and receiving timely, quality care once comes among FTMs and their male partners. the service site is accessed) and increase access to Implemented through the Saving Mothers, Giving comprehensive family planning services, including Life (SMGL) Initiative, and in partnership with the long-acting reversible contraceptives, in 108 facili- CRS Ministry of Health, the program focused on in- ties in all 18 LGAs of CRS. Although the SMGL The SMGL creasing access to HTSP and family planning infor- Initiative achieved reductions in facility neonatal Initiative noted a mation and services, as well as addressing the mortality rate and facility maternal mortality ratio, persistent gap in underlying social and gender factors that influence the project team noted a persistent gap in reach- reaching young family planning communication, decision making, ing young women and mothers with family plan- women and and action for FTMs and their male partners. The ning services at the community level, including mothers, including program applied both a life course and a socioecolo- thoseatriskofearlychildbearingandrapidre- 24 those at risk of gical lens to determine the appropriate content and peat pregnancies. early childbearing Informed by evidence from formative research structure of interventions with young FTMs, their and rapid repeat conducted with FTPs in 2017 (Box 1),9 E2A husbands/partners, other key influencers, and the pregnancies. broader community.23 We also built on existing designed a program to improve voluntary modern facility- and community-based family planning contraceptive use and related gender outcomes services, strengthened under the ongoing SMGL among FTMs and their male partners. The FTP Initiative, to provide targeted family planning interventions built on existing SMGL service de- counseling and referral linkages for FTMs and their livery and community platforms in 2 LGAs (Ikom male partners. The package of FTP interventions and Obubra), selected on the basis of local capacity was implemented in 2 local government areas to implement community-based activities and to (LGAs) of CRS, Ikom and Obubra, from May engage FTMs, their husbands/partners, other key through August 2018. influencers, and the broader community to im- This article examines the effectiveness of prove contraceptive access and use. These LGAs community-based FTP interventions in improving also had sufficient numbers of adolescent and FTPs’ demand for HTSP and their voluntary up- young women who were potentially FTMs. While take of contraception through analysis of key indi- specific data on exact numbers of FTMs in these cators obtained from the baseline and endline 2 LGAs are not available, the 2015 census projec- survey results among program participants. These tions estimated that the total population of these indicators, which were part of the initial concep- 2 LGAs was 433,363, and approximately 9% (or – tual model and reflect program content on HTSP 39,002) were females aged 15 24 years. While and family planning, include intentions to space the provision of modern contraceptive methods the next birth by at least 3 years; awareness of was largely done at health facilities, the FTP inter- 3 or more modern contraceptive methods; belief ventions expanded community-based activities to deliver HTSP and family planning information, as that contraception will “spoil” or harm one’s re- well as counseling and referral services, and to ad- productive organs, perceived approval from a dress underlying social and gender factors that in- male partner for a female partner to use contra- fluence family planning–related communication, ception, recent partner communication about decision making, and action. Staff from SMGL and contraceptive use, perceived joint decision making a local community-based organization (CBO) part- about using a method of contraception, and final- ner, the Greater Hands Foundation, implemented ly, current voluntary use of a modern method of activities in 16 health facilities (a subset of public contraception. sector and faith-based facilities working with SMGL in Ikom and Obubra LGAs) and 37 commu- PROGRAM DESCRIPTION nities served by these facilities. Preparations for FTP From 2014 to 2019, E2A and Pathfinder Inter- activities began in early 2018, with the main period national worked closely with the CRS Ministry of of implementation occurring from May through Health and other partners to decrease the mater- August 2018. nal mortality ratio and neonatal mortality rate FTP interventions included peer group ses- and increase contraceptive use across the state sions with FTMs; small group sessions with the Global Health: Science and Practice 2020 | Volume 8 | Number 4 785 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org BOX 1. Evidence to Action Project Formative Research Findings in Cross River State Nigeria Evidence to Action conducted formative research with first-time mothers (FTMs), male partners, mothers of FTMs, and other respondents in Cross River State, Nigeria, in May 2017. The following key findings informed the design of the new first-time parent (FTP) component9: Nearly all FTMs and male partners agreed that birth spacing is beneficial for the mother, infant, and family, and could name at least one benefit of child spacing. Most FTMs and male partners could name or describe at least one modern family planning method, but did not know how to use any of the methods. Some FTPs were not sure whether family planning was safe and were concerned that its use might “spoil the womb,” thereby negatively affecting a woman’s future fertility. Married FTMs generally thought family planning was safe and beneficial, while unmarried FTMs (especially those that had never used contraception) were less likely to believe that family planning is safe for young mothers to space their children. Several men mentioned that they prefer “the local method” of spacing (extended postpartum abstinence), and a few men mentioned that family planning is only appropriate for women who have finished childbearing or for women in school so that they can “concentrate on their studies.” Despite this apprehension about the safety of family planning, most reported that they would approve of their wives/partners using family planning if they wanted to do so to space their births. The formative research findings pointed to limited awareness and use of family planning services and a need to increase awareness across study sites. Recommendations also included provision of accurate and comprehensive information on family planning methods, providing effective counseling on family planning methods and services, and encouraging spousal communication to improve family planning decision making and uptake. The findings also noted young women’s/mothers’ limited use of health facilities, highlighting the need for community-based approaches that reach young people and link them to the larger health system. husbands/partners of peer group members; small linkages with facilities, monitoring reports). CVs group sessions with older women, typically the participated in implementing all elements of the mothers or mothers-in-law of peer group mem- FTP component and were the linchpin between dif- bers; home visits by Greater Hands Foundation ferent activities, especially in connecting FTPs and community volunteers (CVs); community sensiti- communities with health facilities. Field activities zation; and ongoing family planning service de- were closely monitored by project and CBO staff at- livery at facilities and through mobile outreach. tending project activities to observe progress, pro- While the Greater Hands Foundation had already vide supportive supervision, and assist with any Among FTP been active in these communities through SMGL, troubleshooting. Three FTP interventions—FTM interventions, FTM new FTP activities required increased CV capacity peer groups, small groups with husbands/partners, peer groups, small and engagement. E2A worked with the founda- and home visits by CVs—were particularly impor- groups with tion to recruit 25 certified, but not yet employed, tant for improving family planning–related knowl- husbands/ community health extension workers (CHEWs), edge, attitudes, gender dynamics, and actions and 9 partners, and a health worker cadre with 2–3 years of training for increasing access to family planning services. home visits by CVs and typically based in peripheral health facilities. were particularly Recruited CHEWs lived in FTP intervention com- FTM Peer Groups important for munities and were awaiting their official Ministry The core FTP intervention was a small group activ- family planning. of Health posting. These CHEWs (21 women and ity with FTMs, grounded in the concept of creating 4 men) served as CVs dedicated to reaching FTPs safe spaces, peer networks, and role models for and received a monthly stipend (20,000 naira,or young women going through similar life experi- approximately US$55 at the time of the interven- ences.25 Fifty groups were established in May tion) and transport allowance (20,000 naira,or 2018, each led by a young FTM peer leader and approximately US$55) to conduct activities. CVs composed of 12–15 members. Groups met weekly were trained by project and CBO staff on priority in their communities for 14 sessions over the health issues, including danger signs and 3 delays 4-month intervention period. At each 1-hour ses- during pregnancy, HTSP and family planning, exclu- sion, the peer leader used an activity card to guide sive breastfeeding, positive parenting, and gender discussions on a specific health or gender topic, norms and barriers to accessing services. Trainings such as HTSP, a modern contraceptive method, or also stressed communication and facilitation skills, problem solving within relationships (Box 2). CVs as well as project roles and responsibilities (e.g., generally attended all sessions to support peer Global Health: Science and Practice 2020 | Volume 8 | Number 4 786 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org BOX 2. Topics Addressed in First-Time Mother (FTM) Peer Groups, Cross River State Nigeria Evidence to Action (E2A) adapted 12 activity cards from a toolkit developed by the Gender Roles, Equality, and Transformations (GREAT) project, led by the Institute for Reproductive Health of Georgetown University and implemented by Pathfinder International and Save the Children in Northern Uganda.26 E2A developed 2 additional cards, one on exclusive breastfeeding and the other on positive parenting. Topics included the following: Healthy timing and spacing of pregnancy Problem solving in intimate relationships Life aspirations Contraceptive methods: implants, injectables, oral contraceptive pills, condoms, emergency contraception Gender norms Communication and decision making among couples Desired family size Gender-based and intimate partner violence Exclusive breastfeeding Positive parenting leaders, answer questions, and schedule home vis- postpartum to support maternal and infant health its. In total, 599 out of 607 peer group members outcomes. The FTP component supported home vis- attended at least 12 of the 14 sessions. its further into the extended postpartum period for FTM peer group members. CV visits focused primar- Small Group Sessions With Male Partners ily on HTSP/family planning information, counsel- The FTP program prioritized a structured inter- ing, and referral services, but also addressed other vention with the male partners of FTM peer group pre- or postnatal issues as relevant. CVs conducted members, given their influence over health deci- 4–6 home visits with each peer group member sions, including family planning use, and their from May to August 2018, often at the request of own needs. By design, the male partner interven- the FTM or male partner, or in follow-up to an earli- tion began after the FTM peer groups, giving FTMs er conversation or referral. As much as possible, CVs time to determine if they wanted to include their made an effort to engage male partners, older wom- husband/partners. Once identified by the FTMs, en, and other household members, and often helped CVs and “male motivators” (the partners of FTM to address different or conflicting perspectives on peer group leaders) invited husbands/male part- possible health actions. Home visits accounted for ners to the small groups. This peer-to-peer ap- the majority of family planning referrals given and proach worked well, as men were comfortable completed. The multiple points of contact over the discussing the proposed activity with other men 4-month intervention period proved instrumental and also appreciated knowing someone who in building FTPs’ trust and confidence in CVs and, would be in the group. In total, 20 male partner importantly, creating linkages with the broader groups formed in July 2018, engaging 241 men, health system. against a target of 200, in 6 weekly sessions. These The evaluation sessions explored similar health- and gender- METHODOLOGY focused on related topics as the FTM peer groups, including This evaluation employed a quantitative pretest- knowledge, HTSP, modern contraceptive methods, gender posttest design with program participants to eval- attitudes, and norms/roles, fatherhood, and healthy relation- uate outcomes related to knowledge, attitudes, behaviors on ships. Both male and female CVs led these sessions, and behaviors on family planning and HTSP, exclu- family planning as their status as community resource persons sive breastfeeding, child development and parent- and HTSP, helped overcome any inhibitions felt by male parti- ing, and gender-equitable relationships between exclusive cipants. Almost all men (231 of 241) attended the FTMs and their male partners. All data collection breastfeeding, full set of sessions. tools were piloted for suitability, reliability, coher- child development ence, and clarity; corrections were made as needed. and parenting, Household Visits by CVs Baseline and endline structured interviews were and gender- Under SMGL, CVs conducted home visits during carried out using precoded questionnaires adminis- equitable pregnancy, immediately after delivery, and 6 weeks tered to FTMs and their male partners who were relationships. Global Health: Science and Practice 2020 | Volume 8 | Number 4 787 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org members/participants of intervention groups (small Ethical Review group sessions), before and after participation in The study protocol and other required documents these groups. were submitted to the Government of CRS of Nigeria Health Research Ethics Committee (CRS- Sample Size HREC) in Calabar, Nigeria, and to PATH’s research Using a sample size calculation, the program team determination committee (RDC) in the United determined that a sample of 300 FTM peer group States in late 2017. E2A and Pathfinder International members and 200 male partner group members received approval to proceed with the research from would be sufficient to detect a 10-percentage- the CRS-HREC on March 2, 2018. On February 26, point increase in current use of family planning 2018, PATH’s RDC approved the application and de- “ ” (a key program outcome indicator) from an as- termined it to be not research, therefore obviating sumed baseline value of or near zero. This would the need for any additional U.S.-based institutional yield a sample detecting a significant difference review board review, including PATH/US institution- from baseline to endline at the P<.05 level of sig- al review board. nificance with a design effect of 2.0. A 2-stage (peer groups and individual members) cluster Data Collection – sampling scheme was used to sample FTM respon- Baseline data collection took place May 9 18, – dents. The research team proportionately allocat- 2018, for FTMs and July 9 15, 2018, for male part- ners, and endline data collection for both FTMs ed the FTM sample (N=300) among each of the and male partners took place from August 20 to 2 LGAs based on the total number of participants September 2, 2018. At baseline, interviews took and peer groups in each LGA. The study team ran- place during the initial group activities; a trained domly sampled respondents at both baseline and research team conducted private, one-on-one again at endline from the same 32 FTM peer interviews with recruited FTMs and male part- groups. Due to the smaller size of the male partner ners/fathers who agreed to enroll in their respec- program, a research team interviewed all male tive group-based activities and consented to partners participating in the program from each participate in the study. At endline, participants of the 20 male partner groups at baseline and end- were recruited for private one-on-one interviews line. The final achieved sample size was 338 FTMs at the conclusion of the final group session. The re- at baseline, 339 FTMs at endline, 245 male part- search team of field-based staff conducted face-to- ners at baseline, and 225 male partners at endline face structured interviews using standardized, (Table 1). precoded questionnaires at both baseline and TABLE 1. Criteria for Selection of Respondents and Achieved Sample Size Among Young First-Time Parents, Cross River State, Nigeria Baseline Endline Selected Participants Ikom Obubra Ikom Obubra First-time mothers 15 of 24 peer groups 17 of 26 peer groups The same 15 peer groups The same 17 peer groups At least 10 FTM members randomly selected at randomly selected at baseline; selected at baseline were selected at baseline were were randomly sampled baseline; 150 FTMs 188 FTMs randomly selected interviewed at endline; interviewed at endline; at both baseline and randomly selected from from each of the 17 groups 149 FTMs randomly 190 FTMs randomly endline from each of the each of the 15 groups selected from each of the selected from each of the selected peer groups in 15 groups 17 groups both LGAs Male partners of FTMs All (census) 10 peer groups All (census) 10 peer groups All (census) 10 peer groups All (census) 10 peer All male partners selected at baseline; all selected at baseline; all selected at baseline; all groups selected at participating in peer members of each group members of each group members of each group baseline; all members of groups in both LGAs were interviewed at baseline; interviewed at baseline; interviewed at endline; each group interviewed at selected for the study and 123 male partners 122 male partners interviewed 114 male partners endline; 111 male interviewed interviewed interviewed partners interviewed Abbreviations: FTM, first-time mother; LGA, local government area. Global Health: Science and Practice 2020 | Volume 8 | Number 4 788 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org endline. For all interviews, participants received a contraception after controlling for key demo- summary of the study and were requested to sign graphic variables, partner characteristics, couple a consent form (with provisions for thumbprint communication, and attitudes. signatures). Signed consent was obtained and a A different analytical approach was used for copy given to participants. Interviews were con- the male partner sample, as it was a census of all ducted in either English or Pidgin language. program participants (and thus the samples were not independent). The unmatched sample was dropped (n=21), and the McNemar’s test and Data Management and Analysis paired t-test were used to present statistical differ- The research team collected data using Android- ences at baseline and endline using the matched based mobile phones with the Open Data Kit ap- sample (n=224). In addition, all statistical compar- plication. The mobile phone data entry application isons of the male partner data presented in this pa- included built-in consistency checks and skips. per were analyzed using these tests with and The research team uploaded the dataset to a plat- without complex sampling (based on 2-stage clus- form storage server, where it was monitored cen- ter sampling). Significance levels of these findings trally during the period of field data collection. The did not change based on adjusting for 2-stage clus- team then downloaded the dataset to Excel and ter sampling, remaining highly significant. cleaned, labeled, and checked it for inconsistencies. SPSS Version 22 was used to perform a de- scriptive data analysis, using simple frequencies RESULTS and bivariate analyses. Based on the sampling Sociodemographic Characteristics scheme, the baseline and endline FTM samples The baseline and endline survey data provided (which were randomly generated at both times) useful information about the characteristics of were treated as independent samples, and the FTMs and a subset of their male partners who male partner samples were treated as noninde- joined and stayed engaged in interventions. pendent repeated measures. Although FTMs While some recruitment inclusion criteria were were randomly recruited at baseline and endline, set for FTM peer group members (under 25 years, the peer groups to which they belonged were the pregnant or with first child) and their male part- same at baseline and endline. Therefore, the ners (identified and nominated by interested/will- authors of this study conducted a post hoc analysis ing FTM participants), activities were otherwise of independence between the 2 FTM samples and open to FTMs and male partners who wanted to determined that about 75% of the sample at base- participate. Table 2 presents select background line was included again at endline. A sensitivity characteristics of FTMs and a nominated subset of test was therefore conducted with only the repeat- male partners engaged in the FTP interventions. ed measures subset to confirm the robustness of Almost all FTMs were within the required age lim- the analysis and the statistical significance of the it at baseline, with roughly 63% aged 20–24 years FTM findings. All statistical comparisons of the and 29% aged 15–19 years. Participating male FTM data presented in this paper were reanalyzed partners were most likely to be older; 30% of as repeated measures using McNemar’s test of sig- male partners were aged 30 years or older at base- nificance for categorical data and the paired t-test line and endline. for continuous data, both with and without com- At baseline, most FTMs reported that they plex sampling (based on 2-stage cluster sampling), were not married/living with their partner (63%, using the subsample of participants measured at N=338), and 68% of a subset of nominated male both baseline and endline. Significance levels of partners (N=245) reported that they were either these findings did not vary from the analysis of in- married or living with their partner. The majority dependent samples. Therefore, 1-tailed Pearson of FTM participants (86%) had 1 child with a chi-square tests for categorical data and analysis mean age of 6.9 months at baseline, with another of variance F-tests for continuous data were used 14% pregnant with their first child. The data also to present the statistical significance of differences show that most (90%) male partners enrolled in between baseline and endline and other variables the program were also first-time fathers. A major- of interest among FTMs. In addition, a logistic re- ity of both FTMs and male partners who partici- gression analysis of current use of modern contra- pated in the FTM program reported completing a ceptive methods by FTMs was also performed to secondary or higher level of education. Although determine if a significant change took place over most (85%) male partners reported being current- the course of the interventions in the uptake of ly employed at baseline, only about one-third of Global Health: Science and Practice 2020 | Volume 8 | Number 4 789 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org TABLE 2. Percentage Distribution of Age, Marital Status, Local Government Area, and Education Level by Participant Group and Baseline/Endline Among Young First-Time Parents, Cross River State, Nigeria First-Time Mothers Male Partners Baseline Endline Baseline Endline (n=338) (n=339) (n=224) (n=224) Age, % 15–19 years 28.7 28.3a 1.3 0.9 20–24 years 62.7 67.8a 28.6 30.4 25–29 years 1.2 2.7a 40.2 38.8 30 years plus 0.0 0.3a 29.9 29.9 Don’t know/missing 7.4 0.9a 0.0 0.0 Mean age, years 20.6 21.1b 27.5 27.4 Local government area, % Ikom 44.4 44.0 50.9 50.9 Obubra 55.6 56.0 49.1 49.1 Marital status, % Never married 62.7 53.1b 30.4 31.7 Living with partner/married 37.3 45.4b 69.6 68.3 Divorced/separated/widowed 0.0 1.5b 0.0 0.0 No. of living children, % 0 14.5 7.7b 9.8 4.5 1 85.5 92.0b 85.3 90.2 2 0.0 0.3b 4.9 5.4 Age of youngest child (among participants with at least 1 child) Mean age of youngest child (months) 6.9 months 8.7a 8.8 months 10.9 months (n=289) (n=312) (n=199) (n=214) Residential arrangement, % Currently lives with partner 45.0 43.4 75.4 72.8 Education level, % Primary 13.9 10.9 3.6 4.0 Junior Secondary (completed) 35.2 36.6 9.8 10.3 Secondary (completed) 47.6 45.4 67.9 67.0 Polytechnic 1.8 2.9 4.0 3.1 University 1.5 4.1 14.7 15.6 Works to earn money, % Yes 36.1 56.9a 84.8 86.6 a Chi-square P<.000. b Chi-square P<.05. FTMs (36%) reported working at baseline, likely FTMs at endline were slightly older (21.1 years of due to their recent pregnancy and delivery. age) than at baseline (20.6 years of age), as were Between baseline and endline, FTMs had a few their babies, largely due to the 4-month interval significant differences in some of these variables. between data collection efforts for FTMs. (Only Global Health: Science and Practice 2020 | Volume 8 | Number 4 790 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org 2 months elapsed between data collection efforts the interventions, increasing significantly among for male partners.) In addition, by endline, FTMs FTMs from 50% at baseline to 94% at endline were more likely to be in union or married and (P<.000, Pearson chi-square), and among male were more likely to be working to earn money partners from 38% at baseline to 75% at endline than at baseline. As expected, no significant differ- (P<.000, McNemar’s test). ences were noted among male partners from baseline to endline with respect to these key de- Myths and Misperceptions of Using Modern mographic variables. Contraception A key finding from formative research conducted Birth Spacing Intentions prior to the FTP interventions was that many One of the key messages of the FTP interventions FTMs and their male partners believed that using One of the key was to encourage a spacing gap of 3 years or more contraception can damage a woman’s reproduc- messages of the between births. Figure 1 shows that at baseline, tive organs and create difficulties in conceiving or FTP interventions only 17% of FTMs and 40% of male partners at can even cause permanent sterility after discontin- was to encourage baseline preferred no more children or wished to uation. Thus, most believed that it is best for a a spacing gap of wait 3 years or longer to have another child. At woman to use contraception for limiting fertility 3yearsormore endline, 81% of FTMs (P<.000, Pearson chi only after achieving one’s desired family size. between births. square) and 88% of male partners preferred no Correcting this misconception was an area of focus more children or to wait 3 years or longer throughout the interventions. Figure 3 presents < ’ (P .000, McNemar s test). Importantly, an align- the percentage of interviewed participants who ment in birth spacing intentions generally oc- held this belief at baseline and endline. At base- curred for both FTMs and male partners. line, 55% of FTMs and 29% of male partners agreed that using contraception could negatively Awareness of Modern Contraceptive affect a woman’s ability to have children in the fu- Methods ture. At endline, only 1% of FTMs (P<.000, The FTM and male partner interventions empha- Pearson chi-square) and 7% of male partners sized knowledge and use of postpartum contra- held this belief (P<.000, McNemar’s test). ception. Knowledge of modern contraceptive methods increased over the course of the inter- ventions, with the percentage of FTMs and male Spousal/Partner Approval for Using Modern partners who could spontaneously recall at least Contraception 3 modern methods increasing significantly (see Husband’s or partner’s approval for using a meth- Figure 2). The percentage of FTMs and male part- od of contraception (or perceived approval by ners who could spontaneously recall at least FTMs) may be a critical factor in facilitating an 3 modern methods nearly doubled over the life of FTM who is married or in union to accept and use FIGURE 1. Percentage of First-Time Parents Who Do Not Want Another Child or Who Wish to Wait 3 Years or Longer to Have Their Next Child, Cross River State, Nigeria 100.0% 90.0% *P<.000 *P<.000 80.0% 70.0% 60.0% 50.0% 88.4%* 40.0% 81.4%* 30.0% 20.0% 39.7% 10.0% 16.6% 0.0% Baseline (N=338) Endline (N=339) Baseline (N=224) Endline (N=224) First-Time Mothers Male Partners Global Health: Science and Practice 2020 | Volume 8 | Number 4 791 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org FIGURE 2. Percentage of First-Time Parents Who Can Name at Least 3 Modern Contraceptive Methods, Cross River State, Nigeria FIGURE 3. Percentage of First-Time Parents Who Agree That Using Contraceptives May Negatively Affect One’s Ability to Have Children in the Future, Cross River State, Nigeria contraception. FTMs were asked if they thought seek family planning services. At baseline, only that their husband/partner would approve if they about two-thirds (67%) of the FTM participants wanted to use a method of contraception to space thought that their partner would approve of their their next child, and male partners were asked if use of family planning to space their next child, they themselves would approve of their wife/part- which increased significantly to 80% (P<.000, ner using a method of family planning, as well as Pearson chi-square test) at endline (see Table 3). whether or not they would give her money to Male partners, however, were much more likely Global Health: Science and Practice 2020 | Volume 8 | Number 4 792 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org TABLE 3. Percentage Distribution of Partner Support for Family Planning by Participant Group and Baseline/ Endline First-Time Mothers Male Partners Baseline, % Endline, % Baseline, % Endline, % Variable (n=338) (n=339) (n=224) (n=224) Agrees that husband/partner would approve of using family 66.9 79.6a 90.6 93.8 planning to space next child Would give wife/partner money to seek services if she 87.9 93.8b wanted to use family planning to space her next birth a Chi-square P<.000. b McNemar’s test P<.05. to approve at baseline (91%), and this did not both FTMs and male partners. Reported discus- Having change significantly at endline (94%). Nearly all sions about family planning among FTMs (regard- discussions with (94%) of male partners agreed that they would less of marital/union status) doubled from one’s partner or be willing to support their female partner/wife baseline (41%) to endline (80%, P<.000, Pearson other influential with money to seek family planning services at chi square) and increased significantly among endline, which significantly increased from base- people is often male partners from 69% to 91% (P<.000, line (88%, P<.05, McNemar’s test). associated with McNemar’s test). Discussions among FTMs and interest in and male partners with other influential people also voluntary use of increased from baseline to endline (data not Couple Communication on Family Planning family planning. Having discussions with one’s partner or other in- shown), from 28% to 55% for FTMs (P<.000) fluential people is often associated with interest in and from 17% to 42% for male partners (P<.000, and voluntary use of family planning. The FTP McNemar’s test). When asked with whom they interventions included activities and discussion discussed family planning in the past 3 months, around partner communication on family plan- FTMs were most likely to report discussing family ning and birth spacing. Figure 4 presents baseline planning with a mother (43%), sister (34%), or and endline data on discussions about family plan- friend (51%) at endline (n=187, data not shown); ning with partners and other influencers among male partners were most likely to discuss family FIGURE 4. Percentage of First-Time Parents Who Have Discussed Family Planning With Their Partner as a Way to Space Children in Past 3 Months, Cross River State, Nigeria Global Health: Science and Practice 2020 | Volume 8 | Number 4 793 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org planning with a friend (73%) or a brother (21%) FTMs (P<.000, Pearson chi-square), and from at endline (n=95, data not shown). 44% to 81% (n=200) among male partners (P<.000, McNemar’s test). Importantly, other pos- itive changes in family planning knowledge, atti- Couple Decision Making About Family tudes, communication, and decision making all Planning support the overall increase in informed, voluntary Both FTMs and male partners were asked at base- contraceptive use by FTP participants. line and endline about decision-making responsi- A logistic regression analysis was also per- bility for using family planning (data not shown). formed to confirm the bivariate findings above, The percentage of FTMs and male partners who predicting current use of any modern family plan- reported that they should decide together to use ning method (implants, intrauterine devices, family planning was high even at baseline: injectables, oral pills, male or female condoms, 82% (N=338) of FTMs and 88% (n=224) of male emergency contraception, or standard days meth- partners agreed that using family planning should od) among both FTMs and male partners (in sepa- be a joint decision before the intervention. rate models, data not shown). All relevant However, this percentage significantly increased for both participant groups by endline; by the end demographic variables were included in the mod- of the intervention, 96% (N=339; P<.000, el, as well as attitudes toward family planning, ’ The key objective Pearson chi-square) of FTMs and 99% (N=224; couples discussions about family planning, per- of the FTP P<.000, McNemar’s test) of male partners agreed ceived partner approval and joint decision making interventions was that using family planning should be a joint deci- for family planning, as well as a variable repre- to increase current sion. Perhaps even more important, relatively senting the survey wave (baseline/endline, with voluntary use of a few FTMs reported that husbands/partners were baseline as the reference category). This analysis modern the primary decision maker about family plan- revealed that for both FTMs and male partners, contraceptive ning, suggesting that contraceptive use was largely survey wave was highly significant (P<.001) with method. voluntary for these young women. adjusted odds ratios of 3.3 for FTMs and 3.7 for male partners. This means that modern contracep- Current Voluntary Use of Modern tive uptake significantly increased from baseline Contraception to endline for both groups of participants. In other The key objective of the FTP interventions was to words, after controlling for hypothesized predic- increase current voluntary use of a modern con- tors of family planning use, including demograph- traceptive method. Figure 5 shows that current ic factors (age, marital status, education level, age use of a contraceptive method among both FTMs of youngest child) and all attitudes related to fam- and male partners significantly increased from ily planning use presented in this report (including baseline to endline. Current use increased from perceived safety of contraceptive methods, part- 26% (n=288) to 79% (n=316) among nonpregnant ner approval, and decision making related to FIGURE 5. Current Use of a Modern Contraceptive Method (Among First-Time Parents Not Currently Pregnant), Cross River State, Nigeria Global Health: Science and Practice 2020 | Volume 8 | Number 4 794 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org FIGURE 6. Current Modern Contraception Method Used Among First-Time Mothers Not Currently Pregnant (Multiple Responses Possible), Cross River State, Nigeria * P<.000. ** P<.01. family planning use), FTMs were approximately partners in using family planning to space their 3 times more likely and male partners nearly second and subsequent children. Implemented by 4 times more likely to be using a modern family local organizations and resource persons, the planning method at endline, compared with interventions aimed to increase HTSP awareness baseline. and intentions, build awareness of modern con- traceptive methods, dispel key myths and misper- Method of Contraception Used ceptions about family planning, address gender norms and barriers, increase social and partner Figure 6 shows the type of contraceptive method support for family planning use, and provide used among FTM respondents not pregnant at referrals and facility linkages for obtaining specific the time of data collection (multiple responses contraceptive methods. We examined key HTSP were possible). The graph reveals that use of and family planning indicators among participat- implants and injectables increased significantly ing FTMs and their male partners to determine if The interventions from baseline to endline for nonpregnant FTMs, they successfully changed attitudes and behaviors were successful in with implants being the most commonly used related to birth spacing and voluntary contracep- improving birth method among all respondents (men’s reported tive use over the course of the program. spacing intentions use of implants also increased significantly from This intervention evaluation included a and current use of baseline to endline, but since nearly all male part- coordinated baseline and endline questionnaire contraceptive ners had an FTM partner in the program, this among a scientific and robust sample of FTM parti- methods from information is presented for FTMs only). At base- cipants and a census of male participants using a baseline to line, only 17% of FTMs (n=287) reported using an trained team of interviewers and digital mobile endline. implant, whereas 65% of FTMs (n=316) were us- data collection tools. The results show that the ing implants at endline (P<.000, Pearson chi- interventions attracted and retained a diverse square test). Importantly, use of implants aligns range of FTMs and male partners (in terms of key with the overall spacing intentions (majority sociodemographic variables such as marital status reported 3 or more years) indicated by both FTMs and education) and was successful in improving and male partners at endline. birth spacing intentions and current use of contra- ceptive methods from baseline to endline, even DISCUSSION after controlling for key sociodemographic and This article has highlighted selected results of a attitudinal variables. Important indicators of con- program designed to address some of the critical traceptive awareness, attitudes, and couples’ com- barriers faced by young FTMs and their male munication increased significantly from baseline Global Health: Science and Practice 2020 | Volume 8 | Number 4 795 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org to endline, and also significantly predicted current family planning outcomes for FTMs and their male use of a modern contraceptive method over the partners emerging from this programming effort. course of the intervention. Along with overall in- Importantly, endline results show that FTMs and creased contraceptive uptake, FTMs and their their partners were generally aligned on key family partners chose to use more effective and/or long- planning attitudes and birth spacing intentions, acting methods (injectables and implants), per- which may have facilitated increased contraceptive haps reflecting, in part, their longer birth spacing use and method choice. These results suggest that preferences, interest in a newly introduced meth- couple-oriented interventions or joint activities — od (implant), as well as the recent increased avail- can work well even in a context in which many ability of these contraceptives locally. FTPs are not in formal unions or necessarily living These positive findings should be interpreted in the same household. with several limitations in mind. This study relied Our experience in CRS suggests that FTMs and on self-reported information gathered during male partners may be particularly open to HTSP face-to-face interviews, subject to both courtesy and family planning use because they face the and recall bias. This bias was minimized through practical and financial realities of raising a child. training of interviewers and design of the ques- Community-based resources like CBOs, CVs (or tionnaire. Another limitation is related to the pos- similar community health workers), peer leaders, sibility that other family planning activities took and others provided FTPs with tailored and timely place in CRS, Nigeria, in the same or nearby geo- access to information and services, as well as lin- graphic areas, which also may have resulted in kages with health facilities critical to ensuring ac- increased health knowledge and behaviors in- cess to a full range of needed services. Many cluding a high level of family planning uptake FTMs do not routinely access health facilities or by program participants; however, Pathfinder may not be ready to consider family planning International’s CRS program staff reported that options when they do. Therefore, such approaches no other family planning-related partner activi- may work better than only integrating family ties had taken place concurrently in Ikom or planning into clinic-based services (e.g., postpar- Obubra LGAs. In addition, participants might tum family planning, postabortion care services, have changed between the pretest and the post- or even family planning integration into antenatal test regardless of the interventions because they care), especially where there are inequities in ac- are maturing and learning, especially as parents cess to and use of health care by young FTMs. of young infants. This limitation was minimized While our project included multiple interven- by ensuring that baseline and endline data col- tions focused on FTPs, all activities were imple- lection coincided tightly with program imple- mented over a 4-month period through existing mentation. A final limitation is related to the health facilities and a local CBO, using trained self-selection of program participants and pro- and certified (but not yet employed) CHEWs. pensity of more empowered individuals (as op- Several elements, such as home visits, community posed to those in the general population) to outreach, and the provision of modern contracep- participate. While it is highly likely that partici- tive methods, were already included within the pants had some propensity toward the informa- general mandate of the primary health care system. tion and messages received during the program, Building on existing community- and facility-based baselineattitudesandknowledgewereconsis- resources to identify and reach FTPs with tailored tent with findings from the formative research activities generated compelling results and provid- (described previously). ed a model that can be adapted based on available While previous E2A FTP projects primarily fo- resources and scaled-up across the state. cused on activities with FTMs related to HTSP/ family planning, the interventions in CRS engaged CONCLUSION male partners more systematically. We included The E2A experience in CRS shows that tailored group activities with a subset of nominated male interventions with FTPs can achieve important partners of FTM peer group members to address HTSP and family planning results within a rela- gender norms, increase male engagement in HTSP tively short time frame. FTMs and partners are and family planning, and promote couple commu- coping with multiple challenges as new parents nication and joint decision making. Specific activi- and are receptive to information and options that ties were included to generate evidence on both allow them to delay subsequent births. The emerg- the implementation experience and on health and ing high demand for family planning across diverse Global Health: Science and Practice 2020 | Volume 8 | Number 4 796 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org FTMs and partners—especially for more effective 3. Darroch J, Woog V, Bankole A, Ashford LS. Adding It Up: Costs and and longer-acting contraceptive methods—under- Benefits of Meeting the Contraceptive Needs of Adolescents. Guttmacher Institute; 2016. Accessed October 12, 2020. https:// scores the importance of engaging FTPs during this www.guttmacher.org/report/adding-it-meeting-contraceptive- critical moment in their reproductive lives. In needs-of-adolescents particular, the CRS experience suggests 3 essen- 4. UNFPA. Girlhood, Not Motherhood: Preventing Adolescent tial program elements: (1) ensuring the availabil- Pregnancy. UNFPA; 2015. Accessed February 9, 2020. https:// ity of modern contraceptive methods (especially www.unfpa.org/sites/default/files/pub-pdf/Girlhood_not_ implants) through local health facilities; (2) using motherhood_final_web.pdf locally based resource persons or community- 5. Chandra-Mouli V, McCarraher DR, Phillips SJ, Williamson NE, Hainsworth G. Contraception for adolescents in low and middle in- based health workers to conduct home visits come countries: needs, barriers, and access. Reprod Health. with FTPs to provide tailored health information 2014;11(1):1. CrossRef and referrals, as well as build linkages with the 6. Nove A, Matthews Z, Neal S, Camacho AV. Maternal mortality in formal health sector; and (3) using activities that adolescents compared with women of other ages: evidence from Lancet Glob Health – address gender norms and couple dynamics to 144 countries. . 2014;2(3):e155 e164. CrossRef. Medline foster 7. National Population Commission (NPC) Nigeria and ICF better alignment, communication, and joint ac- International. Nigeria Demographic and Health Survey 2013.NPC tion on reproductive issues. All activities can be and ICF International; 2014. Accessed October 12, 2020. https:// implemented through locally based resource per- dhsprogram.com/pubs/pdf/FR293/FR293.pdf sons, who are best positioned to identify and 8. World Health Organization (WHO). Adolescent Pregnancy: Unmet Needs and Undone Deeds: A Review of the Literature and reach young FTPs of different characteristics and Programmes. Issues in Adolescent Health and Development.WHO; situations. The results that can be achieved, along 2007. Accessed October 12, 2020. https://apps.who.int/iris/ with high levels of engagement from FTPs, dem- bitstream/handle/10665/43702/9789241595650_eng.pdf onstrate the importance of investing in these 9. Kanesathasan A, Morgan G. Improving Health and Gender types of interventions, ideally addressing all pri- Outcomes for First-Time Parents in Cross River State, Nigeria. Evidence to Action Project; 2019. Accessed October 12, 2020. orities for family planning, reproductive health, https://www.e2aproject.org/publication/crs-ftp-report/ and maternal, neonatal, and child health across 10. Agunbiade OM. Sexual exploitations, concealment and adolescent the FTP lifestage, from pregnancy through the mothers’ agency in a semiurban community in southwest Nigeria. J postpartum period. Appl Soc Sci. 2014;8(1):24–40. CrossRef 11. Mgbokwere DO, Esienumoh EE, Uyana DA. Perception and attitudes Acknowledgments: The authors wish to thank the Cross River State of parents towards teenage pregnancy in a rural community of Cross Nigeria. Glob J Pure Appl Sci – Government, Nigeria, for its support of this work and the project team River State, . 2015;21(2):181 172. from Pathfinder International, Nigeria, including Farouk Jega, Yemisi CrossRef Femi-Pius, Nenka Alobi, and Benjamin Asemota. We also thank the 12. Lewis CM, Scarborough M, Rose C, Quirin KB. Fighting stigma: an Greater Hands Foundation executive director, Arit Williams, and her Affilia – team. Researchers from Research and Communications Services Limited adolescent mother takes action. . 2007;22(3):302 306. conducted baseline and endline surveys with project participants, under CrossRef the direction of Akinsewa Akiode, principal consultant. We also 13. Weed K, Nicholson JS. Differential social evaluation of pregnant acknowledge Connie Lee, Laurel Lundstrom, Eric Ramirez-Ferrero, teens, teen mothers and teen fathers by university students. Int J Caitlin Thistle, and Cory Wornell for their critical and valuable review of Adolesc Youth – this manuscript and Marylene Wamukoya and Maurice Mutisya for their . 2015;20(1):1 16. CrossRef. Medline helpful advice on statistical analysis. Finally, and most importantly, we 14. Bankole A. Desired fertility and fertility behaviour among the Yoruba thank the hundreds of young first-time mothers and their partners in Cross of Nigeria: a study of couple preferences and subsequent fertility. River State. Population Studies. 1995;49(2):317–328. CrossRef 15. Bawah AA. Spousal communication and family planning behavior in Funding: This publication was made possible through support provided Navrongo: a longitudinal assessment. Stud Fam Plann. 2002;33 by the Office of Population and Reproductive Health, Bureau for Global – Health, U.S. Agency for International Development, under the terms of (2):185 194. CrossRef. Medline Award No. AID-OAA-A-11-00024. The opinions expressed herein are 16. Becker S. Couples and reproductive health: a review of couple stud- those of the author(s) and do not necessarily reflect the views of the U.S. ies. Stud Fam Plann. 1996;27(6):291–306. CrossRef. Medline Agency for International Development. 17. Stephenson R, Baschieri A, Clements S, Hennink M, Madise N. Contextual influences on modern contraceptive use in sub-Saharan Competing interests: None declared. Africa. Am J Public Health. 2007;97(7):1233–1240. CrossRef. Medline REFERENCES 18. Hardee K, Croce-Galis M, Gay J. Men as Contraceptive Users: 1. Stover J, Ross J. Increased contraceptive use has reduced maternal Programs, Outcomes and Recommendations. Working Paper. mortality. Matern Child Health J. 2014;18(1):333. CrossRef. Washington, DC: Population Council, The Evidence Project; 2016. Medline Accessed October 12, 2020. http://evidenceproject.popcouncil. org/wp-content/uploads/2016/09/Men-as-FP-Users_September- 2. Brown W, Ahmed S, Roche N, Sonneveldt E, Darmstadt GL. Impact 2016.pdf of family planning programs in reducing high-risk births due to younger and older maternal age, short birth intervals, and high par- 19. Williamson LM, Parkes A, Wight D, Petticrew M, Hart GJ. Limits to ity. Semin Perinatol. 2015;39(5):338–344. CrossRef. Medline modern contraceptive use among young women in developing Global Health: Science and Practice 2020 | Volume 8 | Number 4 797 Modern Contraceptive Uptake Among First-Time Parents in Cross River State www.ghspjournal.org countries: a systematic review of qualitative research. Reprod Health. 23. Kanesathasan A. Introducing E2A’s First-Time Parent Framework. 2009;6(1):3. CrossRef. Medline Evidence to Action Project; 2019. Accessed October 12, 2020. 20. Abiodun OM, Balogun OR. Sexual activity and contraceptive use https://www.e2aproject.org/wp-content/uploads/E2A-FTP- among young female students of tertiary educational institutions in Framework.pdf Ilorin, Nigeria. Contraception. 2009;79(2):146–149. CrossRef. 24. Evidence to Action Project. Saving Mothers Giving Life (SMGL) Medline Initiation, End of Project Report. Pathfinder/Nigeria; 2019. 21. Loto OM, Ezechi OC, Kalu BKE, Loto AB, Ezechi LO, Ogunniyi SO. 25. Austrian K, Ghati D. Girl-Centered Program Design: A Toolkit to Poor obstetric performance of teenagers: is it age- or quality of Develop, Strengthen & Expand Adolescent Girls Programs.Population care-related? J Obstet Gynaecol. 2004;24(4):395–398. CrossRef. Council; 2010. Accessed October 12, 2020. https://www.popcouncil. Medline org/uploads/pdfs/2010PGY_AdolGirlToolkitComplete.pdf 22. Greene ME, Gay J, Morgan G, et al. Literature Review: Reaching 26. Gender Roles, Equality and Transformations (GREAT) Project. Young First-Time Parents for the Healthy Spacing of Second and GREAT Project How-To Guide: GREAT’s Approach to Improving Subsequent Pregnancies. Evidence to Action Project; 2014. Adolescent Sexual and Reproductive Health and Reducing Gender- Accessed October 12, 2020. https://www.e2aproject.org/wp- Based Violence. Institute for Reproductive Health, Georgetown content/uploads/reaching-first-time-parents-for-pregnancy- University; 2016. Accessed February 10, 2020. http://irh.org/wp- spacing.pdf content/uploads/2016/06/GREAT_Project_How-to-Guide.pdf Peer Reviewed Received: March 13, 2020; Accepted: October 6, 2020; First published online: November 19, 2020 Cite this article as: Morgan G, Kanesathasan A, Akiode A. Effects of a community-based program on voluntary modern contraceptive uptake among young first-time parents in Cross River State, Nigeria. Glob Health Sci Pract. 2020;8(4):783-798. https://doi.org/10.9745/GHSP-D-20-00111 © Morgan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00111 Global Health: Science and Practice 2020 | Volume 8 | Number 4 798 REVIEW Routine Family Planning Data in the Low- and Middle-Income Country Context: A Synthesis of Findings From 17 Small Research Grants Bridgit Adamou,a Janine Barden-O’Fallon,a Katie Williams,a Amani Selimb Key Findings ABSTRACT Health information systems rely on high-quality data to measure, n We found 4 major themes affecting family track, and inform decision making. Currently, the quality, uptake, planning data quality, analysis, and use: and use of family planning data in routine health information sys- tems is limited, presenting an opportunity for improvement on * The enabling environment for managing and many levels. The current synthesis assessed findings from 17 small using family planning information grants that MEASURE Evaluation issued to low- and middle- * Barriers to integrating family planning in routine income country research teams between 2015 and 2019. Main health information systems findings from that research were collaboratively categorized in 4 * Gaps in analyzing, interpreting, and using major themes: (1) the enabling environment for managing and us- routine family planning data ing family planning information; (2) barriers to integration of fam- * Family planning data use in decision making ily planning in routine health information systems; (3) gaps in the analysis, interpretation, and use of routine family planning data; n Systematic, organizational, cultural, and technical and (4) family planning data use in management, programmatic, barriers affect data quality and limit subsequent and budgetary decisions. Data quality at the systemic, organiza- analysis, interpretation, and use of information. tional, technical, and output levels was a crosscutting theme. Collectively, the findings outline barriers to and opportunities for Key Implications improved integration of family planning data and subsequent strengthening of routine health information systems. n Program implementers should consider: * Standardizing family planning indicators across BACKGROUND sectors (public and private) and data collection tools The provision of health care services and information * Conducting regular staff trainings and capacity about their quality and quantity are critical compo- building to improve data literacy, collection, and nents of a health system. These components must func- reporting tion together to strengthen service delivery programs * Investing in the human and technological and improve population health. Countries use health in- resources needed for effective data collection, formation systems (HIS) to measure and track health analysis, and use. services, allowing them to plan, evaluate, and imple- 1 n ment health strategies. An efficient HIS draws from Policy makers should: multiple levels of the health system, using clearly de- * Enact and commit to continuous financial support fined indicators, up-to-date standards and guidelines, * Emphasize well-defined data collection and accessible data collection and analysis tools, and stake- reporting processes, including clearly defined holder collaboration and support to enable evidence- indicators and harmonized data collection tools informed decision making.2 A key component of an HIS * Provide well-supported technical infrastructure. is a routine health information system (RHIS), funda- mentally composed of indicators to track management information needs and data collection, transmission, processing, and analysis, which should all lead to infor- mation use.3 Data from RHISs include service statistics, a University of North Carolina at Chapel Hill, Chapel Hill, NC, USA. management and logistics data, and financial data, and b United States Agency of International Development, Washington, DC, USA. provide information on client health status, facility Correspondence to Bridgit Adamou ([email protected]). and budgetary capacity, and services and resources Global Health: Science and Practice 2020 | Volume 8 | Number 4 799 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org administered or available.4 These RHIS data con- MEASURE Evaluation project, funded by the U.S. stitute the main pillar for monitoring service deliv- Agency for International Development (USAID), ery programs at the national level in low- and provided technical and financial support for middle-income countries (LMICs).5 Despite a researchers in LMICs to investigate issues related sound framework for an effective HIS, earlier re- to the collection, aggregation, and use of routine This synthesis of search found underperforming RHISs due to sev- family planning data. This article synthesizes the family planning– eral factors, such as poor data quality; indicators family planning-specific research results from specific research lacking standardization, clear definitions, and ac- 17 small grant-funded projects, organized by com- results from curate calculations; inadequate electronic data mon themes, to shed light on the status of family 17 small grant– capture and reporting; incomplete data analysis; planning in RHISs. poor management support; and weak use of infor- funded projects 4 sheds light on the mation for planning and decision making. A strong RHIS that supports data-informed decisions METHODS status of family In 2014, MEASURE Evaluation implemented a requires 4 key actions: regularly assessing the or- planning in RHISs. program funded by the USAID Office of Population ganizational, technical, and behavioral factors and Reproductive Health that provided small grants that affect decision making to improve data de- mand and use; engaging data producers (those for research related to the collection, analysis, and who design and manage research and information use of routine family planning data in 24 priority systems) and data users (those who use data in countries. The overarching goal of the program was program improvement and development) in the to produce evidence that could help improve decision-making cycle; improving data quality; RHISs and advance family planning outcomes. and improving data availability, defined as data The MEASURE Evaluation small grants program synthesis, data communication, and access to aimed to (1) address research gaps in routine health data.6 information for family planning/reproductive For many LMICs, accurate collection, report- health (RH) to inform policy and programmatic ing, analysis, and use of routine data from an HIS decision making, (2) strengthen research capaci- are challenging tasks that span health areas, from ty among local agencies, and (3) increase use of maternal and child health to infectious and chron- research findings by providing an opportunity ic diseases.7 It is also a challenge for LMICs to en- for the data to be disseminated to and used by lo- sure that routine family planning data in their cal stakeholders to inform decision making. The HISs are accurate and complete. The family plan- program supported both primary and secondary ning community has paid relatively little attention data collection and analysis. Grant recipients to strengthening RHISs, causing the field to fall be- were required to secure appropriate ethical re- hind other health areas.8 Recent efforts to collect view and approval prior to research implementa- data for the FP2020 global initiative have brought tion. Five rounds of awards were implemented increased attention to family planning service sta- over a 5-year period (2015–2019), generating tistics, data quality, and reporting mechanisms.8 360 applications and resulting in 19 funded Despite the recent attention focused on family research projects in 11 countries (Table). Recipi- planning in RHISs, the production of high-quality ents represented a mix of university, quasi- information sufficient for program planning, mon- governmental, nonprofit, and private research itoring, advocacy, and other decision-making organizations. The grant amounts ranged from US needs has proven difficult. Health care providers $10,000 to US$24,000 in direct funds, with an av- that do collect routine family planning data often erage award of US$14,400. We required recipients find that the larger HIS into which these data feed to complete a technical working paper of their re- lacks the appropriate reporting or synthesis mechan- search results and to conduct at least 1 data use ac- isms; in other cases, the family planning data are of tivity with stakeholders (such as the presentation of poor quality or are not collected consistently.9 findings at technical working group meetings, Knowledge gaps related to routine family planning workshops, or conferences). We provided technical data include how to improve the quality of family assistance as needed throughout the application, planning data, how to address barriers to integrating implementation, writing, and dissemination stages family planning data in RHIS, and how to encourage of the research projects. Details about the program analysis and use of the data to improve family plan- were previously described by Adamou.10 ning outcomes. To synthesize the results of these research pro- To better understand the dynamics of family jects, we reviewed the 19 small grants working planning data collection, integration, and use, the papers, excluding 2 from the synthesis because Global Health: Science and Practice 2020 | Volume 8 | Number 4 800 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org TABLE. Seventeen Recipients of Small Grants Funded by MEASURE Evaluation Phase IV, 2015–2019 Research Organization Research Title Study Objective(s) Geographic Coverage Data Sources Integrated Health Integrating Family Planning Find approaches to improve 2 districts in each of the Desk reviews of all national Initiative Data from Public and Private the national Health 3 regions in Malawi policy documents guiding Health Facilities in Malawi: Information System by family planning data and How Current Approaches integrating family planning data collection; field Align with FP2020 Goals12 data from private-sector observations; 71 KIIs with service delivery points and staff from national-level government facilities institutions of the MOH, zonal offices of the 5 quality control divisions (i.e., zones) in Malawi, and family planning service providers, HMIS officers, health surveillance assistants, family planning coordinators, and data clerks Rivers State of Nigeria Use of Technology to Manage Explore the experiences Rivers State, Nigeria 21 IDIs with state- and LGA- Primary Health Care Health Data in Rivers State, and perceptions of family level HMIS officers, desk Management Board Nigeria: A Qualitative Study planning providers and officers, monitoring and on Family Planning and health information officers evaluation officers, and Routine Health Information on implementing reproductive health Systems13 technology for district coordinators; 2 FGDs with health data collection and 35 facility health information identify factors that affect officers and family planning the sustainability of using providers technology for data management in Rivers State, Nigeria Africa Field Family Planning Indicators Estimate family planning 2 LGAs in each of the Administration of a Epidemiologic Network Assessment and Data Quality indicator performance at following 6 states in questionnaire via interviews Audit in Selected Health the health facility level from Nigeria: Bauchi, Delta, with 114 family planning/ Facilities Across Nigeria14 the HMIS not reflected in Enugu, Kano, Osun, reproductive health focal DHIS2 to determine the and Nasarawa people in selected facilities; quality of family planning 42 KIIs with family planning data at the facility level and stakeholders and key identify challenges to family decision makers in the family planning program planning/reproductive health implementation in sampled units at the LGA and state health facilities in Nigeria levels in the selected states; 6 FGDs with health workers/ service providers The Rescue Initiative- Analyzing, Interpreting, and Explore how effectively 17 counties in 2 states Direct observation at service South Sudan Communicating Routine family planning data in the in South Sudan: Central delivery points, individual Family Planning Data in South RHIS are analyzed, Equatoria and Western questionnaires administered Sudan15 interpreted, and Equatoria to health facility staff, and KIIs communicated, and discuss with a total of 180 study barriers to RHIS data use participants and ownership in 2 states in South Sudan University of the Punjab, The Routine Health Information Review the RHIS in Punjab Punjab province, Document review and 16 KIIs Institute of Social and Systems in Punjab Province, province of Pakistan and Pakistan with lady health workers, the Cultural Studies Pakistan: Exploring the explore the potential for Population Welfare Dept., Potential for Integrating Health integrating community-level Rahnuma–Family Planning Information Systems for Family data into the national HMIS, Association of Pakistan, DHIS Planning Data16 particularly family planning office, United Nations data, collected by public or Population Fund, and United private, for-profit, and not- Nations Children’s Fund for-profit organizations Continued Global Health: Science and Practice 2020 | Volume 8 | Number 4 801 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org TABLE. Continued Research Organization Research Title Study Objective(s) Geographic Coverage Data Sources Department of Integrating Family Planning Investigate the facilitators, Kampala, Jinja, and 16 KIIs with MOH officers, Population Studies, Data in Uganda’s Health best practices, and barriers Hoima districts, HMIS focal persons at non- Makerere University Management Information of integrating family Uganda governmental organizations, System17 planning data into the HMIS focal persons who were district and national HMIS district biostatisticians or in Uganda medical records officers, and providers who were medical records officers at public and private health facilities; a multi-stakeholder dialogue workshop comprised of 11 participants; and a system- atic review of the HMIS in sub- Saharan African countries that are United States Agency for International Development fam- ily planning priorities International Centre for Using DHIS 2 Software to Explore the perceptions and Khulna and Chittagong Document review; 23 IDIs Diarrhoeal Disease Collect Health Data in experiences with using districts in Bangladesh with community health care Research, Bangladesh Bangladesh18 DHIS2 to collect and providers, nurses, health analyze reproductive, new- inspectors, and upazila born, maternal, and child statisticians; 2 FGDs with health data in Bangladesh district statisticians; and and to identify facilitators 11 KIIs with health managers, and barriers to using these HMIS experts, and key deci- data at different levels of the sion makers health care system Research and Experiences and Perceptions Explore and document the 4 administrative KIIs with 160 frontline health Development Division, of Health Staff on Applying experiences and districts in Ghana’s staff (midwives, community Ghana Health Service Information Technology for perspectives of health staff Central Region health nurses, health District Health Data and managers in the 4 information officers, general Management in Ghana19 districts on use of mobile nurses, and physician technology to collect and assistants) at both the district manage health data in and subdistrict levels and district health systems 14 district and regional health managers and policy makers Centre of Population, Improving Family Planning Map out the distribution of Upper East Region, Records review and data Health and Nutrition Service Delivery in all family planning service Ghana extraction from DHIS2; Services Ghana20 providers in the region and survey of all types of service document how the providers in the region’s community-based family 13 districts by interviewing planning information the family planning providers system is linked to the present (435) using a national system to structured interview recommend strategies for questionnaire; 2 FGDs with supporting program the district health planning and management team, staff from implementation and different subdistrict health improving family planning teams, and community health services officers Continued Global Health: Science and Practice 2020 | Volume 8 | Number 4 802 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org TABLE. Continued Research Organization Research Title Study Objective(s) Geographic Coverage Data Sources Governance Links Strengthening Tanzania’s Investigating the benefit of Administrative district Literature review; Tanzania Routine Health Information incorporating indicators of Magu, Mwanza questionnaire-facilitated System: Incorporating Family related to family planning Region, in the Lake individual interviews with Planning Quality Assessment quality assessment in a Victoria zone of 50 health service providers Indicators21 decentralized RHIS in rural Tanzania and community health work- farming districts around ers; 12 KIIs with health ser- Lake Victoria vice providers, pharmacy staff, civil society organiza- tion staff, council health management team members, and district health information officers; 2 FGDs with 40 health service providers and community health workers Matibabu Foundation Integrating Family Planning Investigate integration of Siaya and Nairobi Eight KIIs with MOH officers Data in Kenya’s DHIS 222 family planning data in counties, with a pretest from Siaya and Kisumu DHIS2, the factors related to conducted in Kisumu counties and a representative lack of integration, and county in Kenya from the Division of Health ways to remedy the lack of Information Systems, at the integration national level. Four FGDs were conducted with clinicians, nurses, health records officers, and information officers from both public and private health facilities at all levels, from the primary level to county referral hospitals. Equitable Health Access The Strongest Motivators for Bridge the knowledge gap 3 LGAs of Lagos state, 12 KIIs and 425 question- Initiative Using Routine Health concerning the motivators Nigeria naires with men and women Information in Family behind using routine health working in the health sector Planning: A Prospective Study information in family in Lagos, Nigeria23 planning to improve the use of family planning services Afya Research Africa Family Planning Services in Estimate the general Kenya National family planning– Kenya During a Transition: prevalence of family related DHIS2 data and Utilization Trends Across planning use among Kenya Demographic and Counties24 women of childbearing age Health Survey 2014 data and the prevalence of family planning use by county; analyze the trends in family planning utilization over the period of transition, from 2012 to 2015; and estimate the extent to which counties had integrated reporting of family planning services in Kenya’s DHIS2 Continued Global Health: Science and Practice 2020 | Volume 8 | Number 4 803 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org TABLE. Continued Research Organization Research Title Study Objective(s) Geographic Coverage Data Sources Mzumbe University, Creating a Culture of Data Use Understand health 2 LGAs within each of 13 IDIs with facility in- School of Public in Tanzania: Assessing Health providers’ capacity to the following regions in charges, reproductive and Administration and Providers’ Capacity to analyze collected family Tanzania: Lindia, child health in-charges, data Management Analyze and Use Family planning data and to Geita, and Arusha clerks, and family planning Planning Data25 document available facility-based providers; evidence of health service 2 FGDs with 24 health providers using the providers; and non- collected data in their participant observation in planning processes 12 health facilities Health Promotion Enhancing Use of Routine Explore the type of family Kilimanjaro and Mara 31 KIIs with health officers in Tanzania Health Information for Family planning information regions of Tanzania charge, points of contact for Planning to Influence Decision collected, how the data are family planning or Making in Tanzania26 analyzed, and how the reproductive health and child information informs health, district medical planning and budgeting. It officers, health governance examined ways data are committee, HMIS focal handled across all 5 levels people, and health of the health system (i.e., secretaries from a regional national, regional, district, hospital, district hospital, ward, and village) and health center (ward level), when and how the data are and dispensary (village level) utilized. Association for Use of Routine Health To understand the budget Calabar Municipal Desk review of existing family Reproductive and Information to Inform process within the state LGA in Cross River planning data in Cross River Family Health Budgetary Allocation for MOH and in the health state, Nigeria State and Calabar Municipal Reproductive Health in Cross department of the Calabar LGA, KIIs with staff from River State, Nigeria27 municipal local government relevant ministries, and council; examine the use of questionnaires administered routine health information to middle- and junior-level as evidence for budgetary officers at the state and LGA allocation for reproductive levels health and family planning; identify barriers and constraints to routine data use; explore possible solutions; and dialogue with the stakeholders on how routine health data can be used in the budget process Access Global Ltd. Uganda’s Resources to Understand the extent to Uganda Literature review; retail audits Finance Family Planning which in-country resources in 16 pharmacies in Mukono Commodities: Implications for can mitigate financing district; and 6 researcher- a Total Market Approach29 shortages for family administered questionnaires planning commodities in with family planning Uganda, and the program managers implications of a total market approach Abbreviations: DHIS, District Health Information Software; FGD, focus group discussion; HMIS, health management information system; IDI, in-depth interview; KII, key informant interview; LGA, local government area; MOH, ministry of health; RHIS, routine health information system. the research topics were not specifically related to organized, the findings within each theme were an RHIS. The main findings of the 17 remaining compared and contrasted. We then summarized papers were extracted, reviewed, and organized the results to present main findings for each theme by key concepts through an iterative process and to contribute to an overall understanding of in which all co-authors participated. Themes current strengths, issues, and gaps in family plan- were developed around the key concepts. Once ning data and RHISs in LMICs. Global Health: Science and Practice 2020 | Volume 8 | Number 4 804 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org RESULTS sent them any data, no. —Private health service The synthesis of results yielded the following main provider themes: (1) the enabling environment for manag- In Pakistan, several private facilities are not ing and using family planning information; (2) bar- legally registered, so it is difficult to collect routine riers to integration of family planning in RHISs; health information from them.16 However, re- (3) gaps in the analysis, interpretation, and use of searchers in Uganda found that because the routine family planning data; and (4) use of family Ugandan Ministry of Health mandates regular planning data in management, programmatic, and submission of HIS reports to health districts as a re- budgetary decisions. All papers discussed the issue quirement for private facilities’ renewal of licen- of data quality—the systematic, organizational, sure, private and nongovernmental organization cultural, and technical barriers that contributed to health facilities have greater participation in the data quality problems and the effects of poor data HIS.17 Furthermore, private nonprofit health fa- quality on analysis, interpretation, and use of cilities (such as faith-based health centers) per- information. For this reason, data quality was con- formed better than public facilities with respect to sidered to be a crosscutting theme, and we incorpo- submission of data because of strict rules enforced rated it, as appropriate, in each of the 4 thematic by their governing institutions.11 areas. Appropriate Resources Theme 1: The Enabling Environment for Researchers in Bangladesh identified a shortage of Managing and Using Family Planning human resources, frequent version changes in Information the District Health Information Software, version The first theme identified in the review of the 2 (DHIS2) platform, negative attitudes about elec- small grant-funded research papers was related to tronic data capture systems from some staff, and the enabling environment for the management reliance on donor support as structural barriers to and use of family planning information. We used the success of the HIS.18 Consequently, users of the following definition for enabling environment: the system suggested strong government commit- strong HIS governance and leadership; policy and ment, deployment of data-quality checks, and acces- framework compliance; appropriate resources, sible technology, along with extensive, sustained such as staffing, technology, and tools; and cross- financial support, to make the nationwide imple- sector engagement of actors, including private and mentation of the electronic system successful.18 11 – public entities. The small grant funded reports il- The review also found that a consistent factor lustrated how challenges in the enabling environ- in managing an RHIS and the subsequent enabling ment affected data collection, assessment, and use environment for family planning information was at all levels. the use of new HIS technologies as an important resource for data capture and reporting. Although HIS Governance and Leadership for Compliance the reports mentioned several types of systems, The review indicated that the strength of system many of the national HIS included a web-based The review governance can be gauged by a country’s ability application for electronic data management that showed that the to enforce its reporting policies and guidelines. was accessible through electronic devices with strength of system Study findings from Malawi, Nigeria, and South browser and Internet access. Typically, this appli- governance can Sudan revealed noncompliance and inconsistent cation was DHIS2. Research in Uganda found be gauged by a submission of family planning data to the national that DHIS2 was considered appropriate and user country’s ability to 12–15 HIS. Weak governance structures were reflected friendly, and the web-based reporting eased the shar- enforce its 17 by countries’ inability to enforce guidelines. For ex- ing of health data with stakeholders. Researchers in reporting policies ample, despite the protocol in Malawi that private Ghana found that mobile tools enhanced job perfor- and guidelines. franchises must submit their monthly data summary mance, the quality of data collection, and the efficien- reports to the district health office, private providers cy of data management.19 A study participant shared feltnoobligationtodoso.12 One study participant the following: shared: I can now sit in my office and monitor activities at the When we have compiled the data each month we have a peripheries and even at hard-to-reach areas, which ac- summary, and that summary is sent to our headquar- tivity would otherwise have cost transport, fuel, and ters. Yeah, that’s all, it’s sent to our headquarters. The much time. Now, I can go on [the mobile technology] government has never asked me; of course, I have never and check ...everywhere a health facility is located, or Global Health: Science and Practice 2020 | Volume 8 | Number 4 805 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org a health staff may work with ease using technology. Cross-Sector Engagement —District-level health manager The often-dissonant relationship between public and private health care sectors played a large role The implement- Nevertheless, the implementation of new in stratifying data collection and limiting informa- ation of new technology hindered progress when necessary tion sharing. Even public and private service pro- technology resources and infrastructures were inadequate. viders who operated in the same data catchment hindered progress For example, one-third of the 435 family planning space often used separate protocols, separate plan- when necessary service delivery points surveyed in the Upper East ning procedures, and data collection mechanisms resources and region of Ghana did not have electricity, making 9,16,20 20 that were not standardized. The differing electronic data very challenging. Research from infrastructures approaches to family planning data collection and Rivers State, Nigeria found the new technology were inadequate. reporting weakened data sharing in the absence of led to parallel systems. Health facilities reported collaborative networks. Study respondents in family planning data into DHIS2, but system users Malawi estimated that less than half of the data continued to use paper-based data collection tools generated in the private health facilities were at the health facilities because of logistical chal- 12 lenges with the electronic infrastructure including reported. Although a system existed to flow frequent power outages, hardware problems, bro- data from the facility level to the national HIS, ma- ken mobile devices, and lack of Internet connectiv- jor issues with private-sector actors (e.g., noncom- ity.13 Nearly all (96.6%) of the study participants in pliance, inconsistent data submission, poor-quality data, and reporting delays) prevented interpreta- the Central region of Ghana concurrently used 12 paper-based data collection and reporting tools tion of these data. The study in Pakistan reflected a similar culture of noncompliance and noncooper- and mobile technology for collecting and transferring 16 health data.20 The research teams in Bangladesh and ation. In contrast to these findings, research in Tanzania found similar barriers.18,21 Additionally, the Uganda found that collaborative networks existed researchers in Rivers State, Nigeria reported faulty between donor-funded implementing partners computer equipment, inadequate training on use of and local organizations, enabling training, finan- data tools, and low levels of information and commu- cial support, and technical assistance in designing 13 data collection tools essential for better HIS per- nication technology skills. Study participants com- 17 plained of substandard government-issued mobile formance and sustainability. This was seen as – devices and difficulties using mobile phones for data an opportunity to improve public private facility collection13: interaction by strengthening and standardizing reporting requirements.17 Some of us are not so perfect with the phone, because, eh, at our local government area, we find it difficult to send Theme 2: Barriers to Integration of Family the message on the phone. But when you get to where Planning in RHISs Studies revealed you can connect to the Internet, they say “no service.” The second theme that emerged from the review limitations with You will continue waiting, waiting, waiting until you centered on barriers to the integration or inclusion integrating family are fed up. At the end of the day, the phone itself, which of family planning as a health area in RHISs. planning in RHIS: we are given to serve at the health facility, remains Generally, the studies revealed poor data flow poor data flow faulty. So, it wasn’t so adequate with us. —Health in- from the service delivery points to the district and from service formation officer at public primary health center national HISs; challenges with implementing data collection tools; lack of clear, standardized family delivery points to Another example of inadequate resources to planning indicators; and disjunctive networks of district and support an enabling environment was insufficient collaboration as limitations to the full integration national HISs and funding to support district health offices. This of family planning in RHIS. Many of the studies challenges with translated into scarce resources needed for a fully revealed incomplete integration of family plan- implementing functioning HIS, such as data collection guide- ning data along the designated data-flow chains, data collection lines; computers and mobile devices; paper record and discrepancies existed between mechanisms tools. books and forms; and HIS staff available for data for data collection and management at the national, consolidation, verification, analysis, and support- community, and facility levels.17,20,22 For exam- ive supervision.17,20,22 A district-level study par- ple, research in Kenya revealed that the paper- ticipant in Ghana said20: based national data summary tool, known as the I am one person in this office who enters reports from all MOH 711, which is used as a template to transfer those facilities into the system, who does data assess- data to DHIS2, includes family planning methods ment, who analyzes, validates, and everything. that are not recorded in either family planning Global Health: Science and Practice 2020 | Volume 8 | Number 4 806 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org registers or DHIS2. A health official in Kenya reflected this prioritization problem; although an remarked, “I know there is no specific one [tool] RHIS existed for various health care entities, pub- for family planning that is really standard for lic departments and nongovernmental organiza- all.”22 This lack of data harmonization creates am- tions did not regularly report family planning biguities in the system, compromises data quality, data into it.16 and makes the family planning situation incom- Insufficient human resources for both provi- plete.23,24 Multiple studies found discrepancies in sion of services (and therefore data capture) and the ability to collect and record family planning supportive supervision and feedback, too few data specifically at the facility or community data collection tools (i.e., computers, tablets, level.15,20,22 In Ghana, there were no required forms, and family planning record books), incor- reporting mechanisms for certain community- rect data entry, and lack of harmonization of data collection tools also affected the inclusion of fami- level family planning service providers, such as ly planning data into the RHIS. Problems with data pharmacies and licensed chemical sellers.19 collection tools included electronic and paper- Similarly, the HIS in Pakistan does not have a based forms without family planning indicators, mechanism to record both community- and improper report consolidation, and unavailable facility-based family planning services for each collection mechanisms.16,17 Additionally, many 16 client. Because the country’s management in- health facilities involved in these studies operated formation systems (the DHIS and commodity lo- both with paper-based patient registers and elec- gistics management information systems) are tronic systems, and these disjointed methods led managed by different departments, integrating to missing or incomplete data entry—a problem the systems will require high-level organiza- that was compounded by a lack of training for tional restructuring. data collectors and a lack of supportive supervi- 17,20,22 As suggested in Theme 1, issues with technical sion. For example, when forms are revised, infrastructure, such as mobile and web-reporting not all family planning providers are trained on challenges, and restricted access to computer- the changes, which exacerbates the problem of low data literacy and results in family planning based systems negatively affected data integration 12,17,20 data being excluded from the HIS. A district-level and flow. For example, in Kenya, data entry 17 health officer in Uganda revealed : and editing rights are restricted to the subcounty health records and information officers. This re- I have never heard of nurses and midwives going for re- striction hinders service providers’ ability to effi- fresher training on family planning data in the HMIS ciently and effectively record family planning [health management information system]. data, which ultimately affects what is captured in 22 Poor integration of family planning data into DHIS2. A study respondent explained the the RHIS also stemmed from the limited pool of problem: standardized family planning indicators both in health facility registers and the national HIS.19,23 The task sometimes overwhelms the staffs, who would In Kenya, researchers found that weak indicators end up with forgetfulness. The notion of I’ll tally tomor- at the facility level affected summary data com- row, and again, tomorrow comes—I’ll tally the next piled at the intermediary ministerial level, in turn limiting tertiary indicators in the national HIS.22 day. So, it is continuous. When you come back tallying Without well-defined, standardized indicators at the end of the month, you end up tallying wrong in- harmonized across the HIS, the data collection formation. Your addition might not be right, so you find tools fell short in recording family planning prac- discrepancies in data. DHIS2 is not the same as data in tices and services. The study in Pakistan found the facility. This has happened several times. We even that this data shortcoming spurred provider dissat- have this report last week, during review meeting, and isfaction with the existing family planning indica- 15 underreporting—to mean what we have on the ground tors. Data collection forms did not provide ’ indicator definitions or a place to record changes is not what we have at DHIS2. It s either due to shortage 15 of staffs, or somebody is not able to fill in data at the right in family planning choice by individuals. Indicator limitations led to such data-quality issues time. The ideal is, one should give the service and then as inaccuracy, overreporting, and missing family tally real time, then give the document by the end of the planning measurements.15 day tally. —Facility in-charge at public health facility Theme 3: Gaps in Analysis of Routine Family Organizational factors, such as a failure to pri- Planning Data oritize family planning data, also influenced inte- The third major theme of the review related to gration into the RHISs. Research from Pakistan gaps in analysis of routine family planning data. Global Health: Science and Practice 2020 | Volume 8 | Number 4 807 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org All the studies All the research papers underscored that pro- Theme 4: Family Planning Data Use in underscored that blems, or the perception of problems, with data Management, Programmatic, and problems or quality and reliability resulted in limited analysis Budgetary Decisions perception of and use of routine family planning data. For ex- The final theme identified in this review was family problems with ample, Tanzanian researchers found that more planning data use in management, programmatic, data quality and than 90% of their study respondents agreed that and budgetary decisions. Despite issues with data reliability resulted a big limitation in assessing routine family plan- quality and reliability, routine family planning data in limited analysis ning data was poor-quality data (another being were sometimes key for programmatic decision and use of routine the lack of financial resources to support the col- making.26,27 For example, in northern Tanzania, family planning lection of high-quality routine data).23 The limited RHIS data were perceived to be an effective and im- data. analysis of routine data was also mentioned as a portant resource in decision making for improving 26 result of a lack of training on electronic data cap- family planning services. Amemberofacouncil 21 ture tools, a lack of data literacy among system health management team said : users, poor data analysis skills, overburdened hu- RHIS is a very important tool to us in [council health man resources, and an absence of leadership or management team]. We depend on it to make important – guidance for family planning data analysis.21,25 27 decisions to improve health services in terms of under- The researchers found that there was often an aware- standing demand and resource allocation. ness, but not a full understanding, of family planning Findings revealed However, the findings revealed that many indicators and their ability to accurately capture that many management, programmatic, and budgetary deci- intended information, hampering the appropriate management, sions were not informed by evidence. For exam- analyses.25–27 For example, when researchers in programmatic, ple, researchers in Nigeria found that despite the Tanzania asked study participants (e.g., family plan- and budgetary high unmet need for family planning (30.8%) in decisions were not ning service providers, HMIS officers, district medical Cross River State, only 0.1% of the state’s health informed by officers, facility in-charges) to identify the source of budget was earmarked for RH and family planning 27 – evidence. family planning indicator data, nearly 20% did not in 2014. (For comparison, in 2009 2010, RH acknowledge men to be a source of family planning represented 13.9% of total health expenditures in information, and one-third did not think any family Kenya.28) In one case, the necessary data were not planning data were obtained from youth.27 available; in Uganda, the National Medical Stores, Many of the studies outlined mechanisms development partners, and implementing part- through which family planning data-capture tools ners were unable to access data on the quantity of might be used to improve data quality and thereby family planning commodities imported and the improve data analysis. Researchers in Tanzania cost price because the National Drug Authority recognized that incorporating explicit quality as- did not have the data in retrievable form, even though organizations required this information sessment indicators (such as quality of care or atti- 29 tudes toward family planning) for family planning for calculating budgets and funding needs. Use of the data for decision making often did not occur data into routine data collection could strengthen at lower levels of the system either.15,25 the usefulness of facility-level data when qualita- Several factors limited capacity of information tive and quantitative indicators are analyzed to- system users to analyze and use data in planning. gether.21 The study authors added that an In addition to issues discussed previously—such as additional pathway for improved data quality and the lack of training on the collection, analysis, and reliability was to explore and invest in technology presentation of data or the lack of appropriate options for data capture and transmission that equipment to support data analysis—guidelines were appropriate and cost-effective for rural set- or systems were lacking on how to use routine 21 tings and facilitated easier data analysis. In data for decision making.21,25,27 In Tanzania, data Nigeria, it was suggested that integrating family use at the facility level was rare owing to a lack of planning services in other health areas, such as perceived data ownership. Health providers HIV, immunizations, delivery, and postabortion expressed the belief that data could not be used at care, could improve family planning data quality the point of creation and that they should only and reliability, and therefore analysis and inter- concern themselves with data collection.25 This pretation, by creating a more complete picture of finding was also seen in South Sudan and Nigeria, which family planning services are provided where data appeared to be used only to fulfill where and to whom.13 reporting requirements, not for analysis or Global Health: Science and Practice 2020 | Volume 8 | Number 4 808 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org decision making. To encourage data ownership mobile devices compromises the functionality of and use at the facility level, one study recom- RHISs and the success of electronic HISs, including mended that supervisors at the district level pro- DHIS2. Such difficulties are not specific to family vide regular feedback to facilities on their data, planning; they affect routine health information help facilities analyze the data for their needs, across all health areas.32 Government investments and give providers the opportunity to explain the in these areas will improve the quality and utility data at meetings.25 of data infrastructure to strengthen the capacity of Poor data quality was a barrier to data use for data management systems at health facilities. planning and budgeting in multiple studies.18,25,26 Because many countries’ HISs have been Because many Tanzanian researchers found that data quality as- strengthened to capture data on infectious dis- countries’ HISs surance, particularly accuracy, was a major chal- eases such as HIV, malaria, and tuberculosis, fam- have been lenge in the health facilities visited.25 In an in- ily planning appears to be an afterthought, with strengthened to depth interview, a service provider in Tanzania less attention and strategic planning for routine capture data on 22 explained the consequences of poor data quality family planning data collection and use. The infectious on decision making as follows25: successful integration of family planning data in diseases, family RHISs must accommodate data from disparate planning appears In fact, the work plan is not realistic, there is a big differ- sources, ideally through standardized indicators to be an ence between the work plan and budget. As you can see, and appropriate use of existing data collection afterthought. this center is in the central part of the town. We serve tools along consistent operational guidelines. more people than anticipated. For example, the budget These tools include patient registries and reporting has been prepared for 3,880 clients, but we serve forms at the clinic, subnational, and national 10,000 clients. We normally claim for the same, but levels, among others. When data are not fully cap- they ignore us because we don’t have data. That’s why tured and aggregated from all family planning I say that there is a big difference between work plan service delivery points and levels in the data man- and budget; the main reason for this is lack of correct agement system—as the findings discussed here data. (Service provider at public health facility) revealed—they provide an incomplete picture of In South Sudan, researchers found that only the status of family planning service delivery and one health facility included in the study made use in a given country. This situation in turn action-oriented decisions to mobilize or shift makes evidence-informed decision making diffi- resources based on a comparison of services, and cult. The findings from the research projects point- only one health facility made evidence-based deci- ed to several challenges with data collection tools sions to advocate for more resources by showing (e.g., missing forms, incorrect versions, broken gaps in its ability to meet monthly or annual tar- mobile devices, lack of guidelines for data collec- 15 tion), human resources (e.g., staff shortages, lack gets. Several studies recommended in-service of data management training for personnel, ab- training to improve providers’ appreciation of how sence of supportive supervision), and governance data could inform decisions and build capacity to – (e.g., lack of policies and guidelines for submission analyze and use data.14,15,25 27 of data into the national HIS and lack of account- ability mechanisms), which also affect data inte- DISCUSSION gration and compromise data quality. The findings from the small grant–funded research Data quality, as defined by data accuracy, rele- reports provide an opportunity to identify specific vance, reliability, and timeliness, was found to be examples of how information system challenges problematic in most of the small grant–funded re- and shortfalls affect data quality and use. Similar search. Yet each of these characteristics is neces- Data are integral to what has been reported in other countries, sev- sary to ensure integrity of data for policy and eral small grant–funded studies revealed ongoing programmatic decision making.1,8 A common to HIS perfor- challenges with the technology and infrastructure theme in the research studies was a lack of data mance and necessary for electronic data collection and report- training or solid understanding of the HIS and its improved health ing.30,31 Although health service providers in mul- potential for family planning data analysis and systems and tiple study countries expressed overall positive use. This translates into a lack of appreciation for outcomes, and attitudes toward electronic data management and complete, high-quality health data for decision fundamental DHIS2, the lack of such basic inputs as providers making. changes in data trained in electronic data capture, a consistent Data as a driver for decision making are inte- culture are power supply, reliable Internet connectivity, and gral to HIS performance and the improvement of needed at all a sufficient number of operative computers and health systems and outcomes; data use informs levels. Global Health: Science and Practice 2020 | Volume 8 | Number 4 809 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org funding, policy, and national health goals.1 But if HIS. A common theme among the study findings technical, management, organizational, financial, was poor data quality resulting from incomplete and political barriers to analyzing and using family or missing data from private and nongovernmen- planning data for planning purposes are present, tal organization facilities, insufficient or outdated as was demonstrated across several research stud- data collection tools and forms, missing data col- ies, initiatives to improve the quality of family lection guidelines, poorly defined indicators, and planning data will fail to achieve their potential. shortages of well-trained data-oriented service Fundamental changes in data culture will require providers. Poor-quality data and a lack of data strategies to motivate, mentor, and supervise staff ownership, analysis skills, analysis tools, and a mandate and instruction from higher levels have at all levels, and staff must be included in pro- prevented service providers from learning from grammatic reviews and decisions. their family planning data and making action- oriented decisions. The issues that contribute to Strengths and Limitations poor data quality and its consequences are circu- This synthesis presents the key findings from a lar, self-reinforcing, and systemic. Addressing body of research produced by local researchers in them requires long-term, multipronged inter- LMICs supported through MEASURE Evaluation’s ventions to improve family planning data man- 5-year small grants program. The synthesis pro- agement for well-informed decision making. vides access to research not available through peer-reviewed journals, highlighting context- Acknowledgments: We wish to express gratitude to the finance and ’ specific findings from local researchers with specific administration staff at Palladium (one of MEASURE Evaluation s partners) who executed the subgrantee subagreements; the MEASURE insight on routine family planning data issues. The Evaluation finance and administration staff at the University of North Carolina at Chapel Hill who provided supporting documentation for the research findings have a unique focus on family subgranting process; MEASURE Evaluation’s knowledge management planning in RHISs, and together provide informa- team, particularly William Frazier, who expertly edited the subgrantee research manuscripts and served as a writing mentor for the subgrantees; tion about RHISs that is relevant across systems and most importantly, to our 19 subgrantees, who worked to complete and health areas and specific to the field of family successful research projects and add to our knowledge on family planning. With a focus on routine data (i.e., service planning and routine health information systems. statistics), this synthesis identifies several areas for Funding: This study was made possible by the support of the American action and intervention to improve the functioning People through the United States Agency for International Development of RHISs and production of reliable, usable family (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L-14-00004. General support was provided by the Carolina planning information. The synthesis does not, Population Center. The views expressed in this article do not necessarily however, attempt to present a comprehensive re- reflect the views of USAID or the United States Government. view of literature on RHISs or family planning in- Competing interests: None declared. formation. Furthermore, the identification of key findings and the development of themes are based REFERENCES on the coauthors’ understanding and interpreta- 1. MEASURE Evaluation. What Are the Characteristics of a Strong tion of the research. The authors acknowledge Health Information System? MEASURE Evaluation; 2018. Accessed that the interconnected nature of routine data cap- September 23, 2020. https://www.measureevaluation.org/ ture and production, reporting, analysis, and use resources/publications/fs-18-294 make hard boundaries between themes difficult to 2. MEASURE Evaluation. What Are the Stages of Progression to a Strong HIS and How Are They Measured? MEASURE Evaluation; define. The small grant-funded papers present ad- 2018. Accessed September 23, 2020. https://www. ditional detailed, context-specific research results. measureevaluation.org/resources/publications/fs-18-309 3. Aqil A, Lippeveld T, Hozumi D. PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health informa- CONCLUSION tion systems. Health Policy Plan. 2009;24(3):217–228. CrossRef. The breadth of the small grant-funded research Medline papers revealed several opportunities and barriers 4. MEASURE Evaluation. Routine health information systems. related to the integration of family planning data MEASURE Evaluation; 2016. Accessed September 23, 2020. https://www.measureevaluation.org/our-work/routine-health- in RHISs in LMICs and the countries’ ability to an- information-systems alyze and use the data to make programmatic and 5. Slattery S. What data do national health information systems in- policy decisions. Lack of functioning electronic clude? Routine Health Information Network. Published August 17, 2018. Accessed September 23, 2020. https://www.rhinonet.org/ tools and resources in many contexts prevents what-data-do-national-health-management-information-systems- providers from fully transitioning to an electronic include/ Global Health: Science and Practice 2020 | Volume 8 | Number 4 810 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org 6. Nutley T, Reynolds H. Improving the use of health data for health Accessed September 23, 2020. https://www.measureevaluation. system strengthening. Glob Health Action. 2013;6(1):20001. org/resources/publications/wp-18-224 CrossRef. Medline 20. Agongo EA, Issah K, Williams JE, Ayaba F, Kunfah B, Ofosu 7. Braa J, Hanseth O, Heywood A, Mohammed W, Shaw V. WK. Improving Family Planning Service Delivery in Ghana. Developing health information systems in developing countries: the MEASURE Evaluation; 2018. Accessed September 23, 2020. flexible standards strategy. Manage Inf Syst Q. 2007;31(2):381– https://www.measureevaluation.org/resources/publications/wp- 402. CrossRef 18-215 8. Family Planning 2020 (FP2020). FP2020: The Way Ahead 2016– 21. Kasongi D, Malimusi H, Kwilasa G. Strengthening Tanzania’s 2017. FP2020; 2017. Accessed September 23, 2020. http://www. Routine Health Information System: Incorporating Family Planning track20.org/download/pdf/FP2020%20Progress%20Report% Quality Assessment Indicators. MEASURE Evaluation; 2018. 202016-2017.pdf Accessed September 23, 2020. https://www.measureevaluation. 9. Maternal and Child Survival Program (MCSP). What Data on org/resources/publications/wp-18-223 Maternal and Newborn Health Do National Health Management 22. Mitoko A, Okango F, Onyango-Abuje L, Obango P, Oule L, Information Systems Include? A Review of Data Elements for 24 Low- Abunga D. Integrating Family Planning Data in Kenya’s DHIS 2. and Lower Middle-Income Countries . MCSP; 2018. Accessed MEASURE Evaluation; 2017. Accessed September 23, 2020. September 23, 2020. https://www.mcsprogram.org/resource/ https://www.measureevaluation.org/resources/publications/wp- what-data-on-maternal-and-newborn-health-do-national-health- 17-184 management-information-systems-include/ 23. Afe AJ, Olatoun A, Akinmurele T, Abimdola O, Agboola G. 10. Adamou B. MEASURE Evaluation’s Small Grants for Family The Strongest Motivators for Using Routine Health Information in Planning: Strengthening Research Capacity Around the World . Family Planning: A Prospective Study in Lagos, Nigeria. MEASURE MEASURE Evaluation; 2019. Accessed September 23, 2020. Evaluation; 2017. Accessed September 23, 2020. https://www. https://www.measureevaluation.org/resources/publications/tr- measureevaluation.org/resources/publications/wp-17-200 19-391 24. Ndiritu M, Anyango R, Ombech E, Mwita CC, Gwer S. Family 11. MEASURE Evaluation. What Are the Factors and Conditions of HIS Planning Services in Kenya During Transition: Utilization Trends Performance Progress ? MEASURE Evaluation; 2018. Accessed Across Countries. MEASURE Evaluation; 2018. Accessed September September 23, 2020. https://www.measureevaluation.org/ 23, 2020. https://www.measureevaluation.org/resources/ resources/publications/fs-18-318 publications/wp-17-178 Integrating Family Planning 12. Mwaungulu EE, Dembo Z, Mtema P. Creating a Culture of Data Use in Data from Public and Private Health Facilities in Malawi: How 25. Anasel MG, Swai IL, Masue OS. Tanzania: Assessing Health Providers’ Capacity to Analyze and Use Current Approaches Align with FP2020 Goals. MEASURE Family Planning Data. MEASURE Evaluation; 2019. Accessed Evaluation; 2018. Accessed September 23, 2020. https://www. September 23, 2020. https://www.measureevaluation.org/ measureevaluation.org/resources/publications/wp-18-209 resources/publications/wp-19-232 Use of Technology 13. Edet C, Whyte M, Ogu R, Obomanu G, Harry A. Enhancing Use of Routine Health Information for Family to Manage Health Data in Rivers State, Nigeria: A Qualitative Study 26. Bujari P. Planning to Influence Decision Making in Tanzania. MEASURE on Family Planning and Routine Health Information Systems. Evaluation; 2017. Accessed September 23, 2020. https://www. MEASURE Evaluation; 2017. Accessed September 23, 2020. measureevaluation.org/resources/publications/wp-17-195_en https://www.measureevaluation.org/resources/publications/wp- 17-189_en 27. Hassan A. Use of Routine Health Information to Inform Budgetary Allocations for Reproductive Health in Cross River State, Nigeria. 14. Olugbade O, Ishola O, Ricketts O, Balogun S, Kamateeka M. Family Planning Indicators Assessment and Data Quality Audit in Selected MEASURE Evaluation; 2016. Accessed September 23, 2020. Health Facilities Across Nigeria. MEASURE Evaluation; 2019. https://www.measureevaluation.org/resources/publications/wp- Accessed September 23, 2020. https://www.measureevaluation. 16-168 org/resources/publications/wp-19-237 28. World Health Organization (WHO). Analysis of Reproductive Health and Child Health Subaccounts Results Produced by Countries 15. Moses T, Kaunda Z, Ezeron W. Analyzing, Interpreting, and . Communicating Routine Family Planning Data in South Sudan. WHO; 2012. Accessed September 23, 2020. https://www.who. MEASURE Evaluation; 2019. Accessed September 23, 2020. int/health-accounts/documentation/rmnchmanjiri.pdf https://www.measureevaluation.org/resources/publications/wp- 29. Kalangwa A, Chelimo VM, Nakandh R. Uganda’s Resources to 19-231 Finance Family Planning Commodities: Implications for a Total Market Approach 16. Mustafa M. The Routine Health Information System in Punjab . MEASURE Evaluation; 2018. Accessed Province, Pakistan: Exploring the Potential for Integrating Health September 23, 2020. https://www.measureevaluation.org/ Information Systems for Family Planning Data. MEASURE resources/publications/wp-18-207 Evaluation; 2018. Accessed September 23, 2020. https://www. 30. Garrib A, Stoops N, McKenzie A, et al. An evaluation of the district measureevaluation.org/resources/publications/wp-18-210 health information system in rural South Africa. S Afr Med J. 17. Wandera SO, Kwagala B, Nankinga O, Ndugga P, Kabagenyi A. 2008;98(7):549–552. Medline Integrating Family Planning Data in Uganda’s Health Management 31. Dickey C, O’Connell T, Bedford J, Thiede M. Integrating an Information System. MEASURE Evaluation; 2018. Accessed Approach to Assess UHC Access Barriers into District Health Systems September 23, 2020. https://www.measureevaluation.org/ Strengthening in Uganda, Ghana and Rwanda. United Nations resources/publications/wp-18-202 Children’s Fund; 2014. Accessed September 23, 2020. https:// 18. Begum T, Khan SK, Ferdous J, et al. Using DHIS 2 Software to Collect www.researchgate.net/publication/299754850_Integrating_an_ Health Data in Bangladesh. MEASURE Evaluation; 2019. Accessed approach_to_assess_UHC_access_barriers_into_district_health_ September 23, 2020. https://www.measureevaluation.org/ systems_strengthening_in_Uganda_Ghana_and_Rwanda resources/publications/wp-19-226 32. Nabyonga-Orem J. Monitoring Sustainable Development Goal 3: 19. Edum-Fotwe E, Abbey M, Osei I, Hodgson A. Experiences and how ready are the health information systems in low-income and Perceptions of Health Staff on Applying Information Technology of middle-income countries? BMJ Glob Health. 2017;2(4):e000433. Health Data Management in Ghana. MEASURE Evaluation; 2019. CrossRef. Medline Global Health: Science and Practice 2020 | Volume 8 | Number 4 811 Findings From Small Research Grants on Routine Family Planning Data www.ghspjournal.org Peer Reviewed Received: March 17, 2020; Accepted: August 26, 2020; First published online: October 27, 2020 Cite this article as: Adamou B, Barden-O’Fallon J, Williams, K, Selim A. Routine family planning data in the low- and middle-income country context: a synthesis of findings from 17 small research grants. Glob Health Sci Pract. 2020;8(4):799-812. https://doi.org/10.9745/GHSP-D-20-00122 © Adamou et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00122 Global Health: Science and Practice 2020 | Volume 8 | Number 4 812 REVIEW Effectiveness of mHealth Interventions for Improving Contraceptive Use in Low- and Middle-Income Countries: A Systematic Review Banyar Aung,a,b Jason W. Mitchell,c Kathryn L. Brauna Key Findings ABSTRACT Background: mHealth interventions are being tested to improve n Of the 8 mHealth family planning interventions that contraceptive uptake in low- and middle-income countries (LMICs); met inclusion criteria, 3 studies improved family however, the effectiveness of these interventions has not been sys- planning outcomes and 4 studies experienced tematically reviewed. implementation issues. Objectives: The primary objective of this systematic review was to n Further research is needed to encourage robust assess the effectiveness of mHealth interventions to improve con- program fidelity of mHealth family planning traceptive uptake and adherence in LMICs. A second objective interventions, along with a more thorough was to identify mHealth features and behavior change communi- understanding of what mHealth and behavior cation components used in these mHealth interventions. change components are needed to improve family Methods: A systematic search was conducted of online databases planning outcomes in low- and middle-income for peer-reviewed articles that reported on intervention studies countries. with men and women from LMICs and measured mHealth inter- vention impact on contraceptive uptake and/or adherence. Key Key Implications search terms included “mHealth” or “mobile health,”“contracep- tion” or “family planning,” and “low- and middle-income coun- n A “push” approach, interactive communication, tries.” PRISMA guidelines were followed for reporting review information tailored to participants, motivational methods and findings. The Cochrane risk-of-bias 2 tool for ran- messaging, and male partner involvement appear to domized trials was used to assess the risk of bias of the included be tied to better family planning outcomes. studies. The GRADE approach was used to determine the quality n Program managers and researchers should consider of evidence. improvements in protocols and fidelity that are Results: Eight randomized controlled trial studies met the inclu- needed to more accurately assess how well mHealth sion criteria. Four studies experienced implementation challenges (e.g., intervention components were not utilized fully by partici- family planning interventions impact outcomes in pants, intervention participants did not receive the full interven- low- and middle-income countries. tion content, contamination, low response rate, and/or missing data). Only 3 interventions were found to be effective, and these included a “push” approach, interactive communication, informa- tion tailored to participants, motivational messaging, and male partner involvement. Conclusion: To date, the delivery of mHealth interventions for im- proving family planning in LMICs has met with implementation challenges that have reduced the researcher’s ability to test inter- vention effectiveness. Although 3 of 8 studies found improved contraceptive use in the intervention group, the review cannot draw concrete conclusions on the overall effectiveness of mHealth interventions to increase contraceptive use in LMICs. Further research with robust program fidelity is recommended. INTRODUCTION a Office of Public Health Studies, University of Hawai‘iatManoa, Honolulu, HI, y the end of the Millennium Development Goals in USA. B b Access to Health Fund, United Nations Office for Project Services, Myanmar. 2015, the maternal mortality ratio had declined 1 c Florida International University. by 45% from 1990. Despite this progress, every day, Correspondence to Banyar Aung ([email protected]). 810 mothers—94% from low- and middle-income countries Global Health: Science and Practice 2020 | Volume 8 | Number 4 813 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org (LMICs)—continue to die from preventable causes as- to assess whether mHealth technologies could be sociated with pregnancy and childbirth.2 To help re- used to help reduce unmet contraceptive needs in duce maternal deaths in LMICs, the Safe Motherhood LMICs by attempting to increase the uptake of mod- Initiative outlined family planning as 1 of 6 “pillars” of ern contraceptive methods.18–25 safe motherhood.3 Oneviablesolutiontoreducema- Three published reviews26–28 explored the ef- ternal mortality in LMICs is to increase the uptake of fectiveness of mHealth interventions for different contraceptives, which in turn will reduce the number contraceptive outcomes. Smith et al.28 assessed of unwanted pregnancies.4 Fulfilling the current the effect of interventions delivered via mobile unmet need for contraceptives can help reduce mater- phone for improving contraceptive use in 5 ran- nal mortality by preventing 104,000 maternal deaths domized controlled trials (RCTs) conducted in the annually,5 while also helping to improve child survival United States, Cambodia, and Israel. Only one of rates by promoting the healthy timing and spacing of the studies occurred in an LMIC. The review con- pregnancy.6,7 cluded that interactive voice messages and commu- Unmet needs for family planning are attribut- nication with a counselor improved postabortion ed to insufficient knowledge and access to family contraception, and the combination of unidirec- planning services.8 Many women who want to tional (i.e., one-way messages) and interactive dai- avoid pregnancy do not use modern contraceptive ly educational text messages (i.e., back-and-forth) methods due to limited or inaccurate knowledge improved adherence in using oral contraceptives. about side effects of contraceptives or the misper- In another review, L'Engle et al.26 examined ception that conception is not possible while 35 studies that used mobile phones to improve ad- breastfeeding or during certain times of the men- olescent sexual and reproductive health, inclusive strual cycle.8 In 2017, the United Nations reported of contraceptives. Only 3 of the 35 studies were that at least 1 in 10 married or in-union women from LMICs. The authors found evidence that in- had unmet family planning needs globally.9 cluding text messages in interventions may improve Fulfilling women’s unmet contraceptive needs adolescent sexual health, but the information pro- is an important global public health goal.10 vided in the studies was insufficient for understand- Mobile phone ownership in LMICs has prolif- ing, replicating, or scaling up mHealth interventions. erated,11 providing new technologies to deliver Rousseau et al.27 conducted a systematic review educational and access-related information about with 22 studies to explore the general impact of reproductive health and family planning to hard- smartphone applications on contraceptive decision to-reach populations.12 The use of mobile technol- making and knowledge. Fifteen of the 22 studies ogy in health care (i.e., mobile health or mHealth) were based in the United States, 3 were conducted has gained popularity globally and has been found in an LMIC, and the locations of the 4 remaining to reduce health care costs, improve the quality of studies were not specified. The reviewers found health care, and encourage prevention-related that apps may be useful as aids to improve contra- behaviors.13 In 2018, world governments unani- ceptive use and prescription of contraception, but mously adopted a World Health Assembly resolu- they were not reliable sources of information. The tion calling on the World Health Organization authors noted that the quality of the studies was (WHO) to develop a global digital health strategy heterogeneous, adding to the difficulty in drawing to support countries’ efforts toward universal health conclusions about the impact of mHealth apps on coverage.14 Subsequently, WHO released the guide- contraceptive knowledge and usage. line Recommendations on Digital Interventions for Health Although previous systematic reviews assessed System Strengthening, which endorses the use of mo- the effectiveness of mHealth interventions for family bile technology for targeted client messaging of planning, only the review by Smith and colleagues28 health services in LMICs.15 focused exclusively on contraceptive uptake, while Most mHealth As a platform, mHealth has been used to offer other 2 systematic reviews involved other outcomes research on family educational information about sexual and reproduc- (e.g., contraceptive knowledge). Furthermore, only planning and tive health, as well as the locations of family planning 7 studies included in these 3 reviews were based in uptake of modern service providers.16 Additionally, mHealth affords an LMIC, and only 1 measured contraceptive use. contraceptives has individuals with fewer logistical barriers because In sum, the bulk of research involving mHealth on occurred in they can quickly, conveniently, and confidentially family planning and uptake of modern contracep- higher-income seek information about family planning and related tives has occurred in higher-income countries, with countries, with few resources instead of having to go to a clinic or see a few trials and studies having occurred in LMICs. trials and studies health care provider to obtain this same informa- Given the disparities of maternal mortality and in LMICs. tion.17 Several interventions have been implemented unmet family planning needs in LMICs, a more Global Health: Science and Practice 2020 | Volume 8 | Number 4 814 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org thorough examination of the role of mHealth in im- any modern contraceptives34 including permanent proving the uptake of modern contraceptives in methods (female sterilization and vasectomy); long- LMICs is needed. acting reversible contraceptives (implants and intra- The primary objective of the present systemat- uterine devices); and shorter-acting contraceptives ic review was to assess the effectiveness of mHealth (injectables, pills, male and female condoms, dia- interventions in improving contraceptive uptake in phragms, spermicides, and cervical caps). We ac- LMICs. The secondary objective of the systematic knowledge that other nonbiomedical methods such review was to identify which mHealth features as fertility awareness methods and withdrawal meth- and behavior change communication (BCC) com- ods exist, but these were not included in our defini- ponents were used in the mHealth interventions tion. We accepted whichever method by which the that occurred in LMICs. outcome was assessed in the included mHealth in- tervention trials/studies, including by self-report METHODS through surveys. Interventions were included even if the uptake and/or adherence to contra- Review findings are reported based on the Preferred ception was not the primary outcome measured Reporting Items for Systematic Reviews and Meta- 29 or was measured in conjunction with other con- Analyses-Protocols (PRISMA) guidelines. The re- traceptive outcomes such as knowledge of view protocol was preregistered in the PROSPERO contraception. database (CRD42020153409). Inclusion Criteria Search Strategy The search was conducted by BA in July 2019. A Type of Studies filter was set to include articles from 2005, since Experimental studies that evaluated the interven- mobile subscriptions reached 23% of populations tion effectiveness through RCTs and nonran- in LMICs in 2005 (compared with only 4% in domized interventional studies were considered 2000).35 PubMed, Web of Science, EBSCOhost, for the review. CINAHL, and The Cochrane Library were searched. Key search terms used were “intervention*”; “pro- Type of Participants gram*”; “mHealth”; “mobile health”; “telemedicine”; Women and men from LMICs, as classified by the “cell phone*”; “SMS”; “apps”; “contraception”; “con- 30 World Bank, were included. The WHO defini- traceptive*”; “family planning*”; “birth spacing”; “de- – We focused on tion of women of reproductive age (15 49 years veloping countr*”;and“low and middle income 31 interventions that old) was not used because more than 1 in countr*”. LMICs were further searched by detailing sought to improve 3 (about 250 million) girls were married or in regions such as Africa, Asia, Pacific Islands, South contraceptive union before age 15, with the highest rates found America, Central America, Latin America, Eastern 32 uptake and/or in LMICs in South Asia and sub-Saharan Africa. Europe, and Central Asia. Reference lists of identified adherence Postpartum and postabortion women were also articles were searched. We retrieved study protocols included. of included studies and assessed method details. We compared with contacted authors of included studies if the study pro- standard care or Type of Interventions tocol was not published and when additional infor- another We included studies in which the intervention mation was needed. Only articles in English were intervention. was delivered using any form of mHealth such as included due to the reviewers’ language limitation. mobile apps, messaging platforms or short messag- ing system (SMS), telephone calls, or geolocational Data Collection and Analysis features (e.g., GPS or Global Positioning System). The search was completed by BA and KLB inde- We included the interventions that sought to im- pendently. Duplicates were removed and titles prove contraceptive uptake and/or adherence and abstracts were assessed applying the inclusion compared with standard care or another interven- criteria. Screened articles were read in full, again, tion. mHealth interventions were identified based by applying the inclusion criteria. Discrepancies on the definition of the WHO Global Observatory were resolved by discussion. 33 for eHealth. BA and KLB extracted information from the studies, including the country in which the study Type of Outcome Measures was conducted, intervention details (e.g., mHealth For the purposes of this review, we included the features, mode of delivery, BCC components, fre- outcome measurement of uptake or adherence to quency, duration), participant characteristics (age, Global Health: Science and Practice 2020 | Volume 8 | Number 4 815 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org gender, postabortion, postpartum, etc.), sample measures that were used in the included studies. size, study design, and outcome(s) relative to mod- However, clinical heterogeneity (i.e., variability ern contraceptive use. Microsoft Excel was used to in participants, interventions, outcomes studied) store and organize the extracted data from included and methodological heterogeneity (i.e., variability studies. in study design and risk of bias) of the included mHealth features and BCC components of studies were characterized. interventions were extracted and categorized by BA and JWM into telephone-based, text/SMS, and Assessment of Publication Bias apps; communication pathway (unidirectional or We were unable to execute a funnel plot to identi- interactive); how family planning information was fy the publication bias due to the diversity of inter- “ ” “ ” delivered ( push telephone service, push mes- vention components and outcome measures that “ ” saging service, or pull messaging service); and ad- were used in the included studies. ditional intervention components (motivational message, tailored information, partner counseling, searching for the nearest service provider, “role Data Synthesis model” stories, and intervention delivered via We conducted the analysis according to the guide- health workers). Push approaches referred to the lines specified in the Cochrane Handbook for Systematic 37 delivery of the intervention (family planning infor- Reviews of Interventions. Qualityofevidenceforin- mation) at predefined intervals or frequencies, cluded studies was assessed using the Grading of while pull approaches relied on the consumer Recommendations Assessment, Development and 38 searching for information without being prompted. Evaluation Working Group (GRADE) approach. mHealth features and BCC components were ana- RCTs were considered high quality and were down- “ ” “ lyzed against contraceptive use or adherence graded by 1 level for serious (or 2 levels for very ” outcome. serious ) risk of bias; unexplained heterogeneity; in- directness of evidence; imprecision of effect estimates; or publication bias. Assessment of Study Quality and Risk of Bias Quality assessment of the included studies was done according to the revised Cochrane risk-of- RESULTS bias tool for randomized trials (RoB 2), as only Figure 1 shows the PRISMA flow diagram for the RCTs that met the inclusion criteria were included systematic review. Among the 123 publications in the review.36 We examined 5 bias domains of identified in the database search, 43 duplicates RoB 2: randomization process; deviations from were removed and 80 studies were assessed; all intended intervention; missing outcome data; 80 studies were published in English. After titles measurement of the outcome; and selection of and abstracts were screened, 73 studies were ex- the reported results. The risk-of-bias judgments cluded for not meeting inclusion criteria and for each domain were “low” or “high” risk of bias 7 articles were further assessed. One additional ar- or “some concerns.” Risk of bias was assessed ticle was identified through reference tracing. based on the effect of assignment to intervention, Eight studies met the inclusion criteria for this sys- the “intention-to-treat” effect, for the included tematic review. studies. BA and JWM individually and separately assessed risk of bias for quality before comparing Study Characteristics notes for each included study. Of the 8 studies included in this review, 3 were conducted in Kenya,18,23,25 1 in Cambodia,22 1in Measures of Treatment Effect Ecuador,19 1 in Tajikistan,20 1 in Palestine,21 and All 8 studies were We planned to determine risk ratios, as measures 1 in Bangladesh24 (Table 1). All 8 studies were parallel-group of treatment effect, for dichotomous outcomes, parallel-group RCTs with 1:1 allocation, including RCTs with and mean differences for continuous outcomes, a feasibility study with a small sample size.24 Study 1:1 allocation, but with 95% confidence intervals. However, we settings varied. Some were conducted in urban18 they varied in were unable to obtain adequate data from includ- or peri-urban and rural areas,22 while others were 19,23–25 terms of setting, ed studies to determine effect sizes. conducted in a hospital or clinic setting ;set- 20,21 participants, tings for 2 studies were not specified. The stud- outcomes, and Assessment of Heterogeneity ies also varied by types of participants: postpartum 19,23,25 22,24 theory. We did not conduct a meta-analysis due to the di- mothers, postabortion women, young versity of intervention components and outcome people,20,21 and general public.18 Outcomes for 6 of Global Health: Science and Practice 2020 | Volume 8 | Number 4 816 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org FIGURE 1. PRISMA Flow Diagram for the Systematic Review of Experimental Studies Evaluating the Effectiveness of mHealth Interventions on Contraceptive Uptake in Low- and Middle-Income Countries Database search total n=123 Duplicates removed n=43 Articles excluded n=75 After removing duplicates n=80 Reasons • No contraceptive use outcome (29) • Did not use mHealth (21) • Not in low- or middle-income country (5) • Did not meet study design criteria (13) Articles screened n=7 • Protocols (5) Articles identified through reference tracking n=1 Included in review n=8 All randomized controlled trials (protocols, statistical analyses, and correction articles assessed when available) Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Protocols. the 8 studies were about contraceptive use and content was delivered via one-way text messages – knowledge18,20 22,24,25; the other 2 studies measured (without an app) to participants in the study con- the same outcomes and other maternal and child ducted in Palestine by McCarthy et al.21 Using a health indicators (e.g., exclusive breastfeeding and text messaging platform named m4RH, Johnson immunization coverage).19,23 Threeofthe8studies and colleagues delivered information about family provided a description about the use of behavior planning, a searchable database of clinics providing change theory.20,21,25 family planning services, and an optional role mod- el stories feature.18 mHealth Features Mobile SMS delivery platform Mobile WACh mHealth features used in the 8 included studies and its variant Mobile WACh XY (with male part- varied (Table 2). Two studies used telephone calls. ner involvement) were used by Unger et al.23 and Smith et al.22 delivered interactive voice messages Harrington et al.,25 respectively, to provide inter- and provided counselor support via telephone active intervention contents tailored to participant ’ 24 calls upon participants request through the mes- needs. The intervention tested by Biswas et al., sages. Counselor phone support involved tailored in Bangladesh, was a feasibility study conducted information a range of contraceptive methods and with a small sample size and it found no effect. motivation about using contraception, as well as Method-specific text message reminders were sent helping participants in their search for family planning clinics. In contrast, the telephone call to participants about their select methods. It only in- was made by a nurse to deliver health education volved unidirectional SMS reminders without other about family planning in the study by Maslowsky BCC components. However, the study found an The 3 studies that et al.19 mHealth contraceptive intervention was feasible, reported Six studies used text messages as their prima- citing positive user engagement and participant improving ry mHealth feature to deliver health education acceptability. contraceptive and motivational messages about family plan- Interactive communication was used in uptake used a – ning.18,20,21,23 25 McCarthy et al.20 included an 4studies.19,22,23,25 Seven studies used a push ap- “push” approach app in their intervention in Tajikistan to mainly de- proach whereas only 1 study used a pull ap- to deliver liver one-way text messages about contraception, proach18 to deliver intervention content to information and common beliefs on family planning, and encourage- participants. Of the 3 studies that reported im- an interactive type ment to use family planning. A similar intervention proving contraceptive uptake,22,23,25 all used a communication. Global Health: Science and Practice 2020 | Volume 8 | Number 4 817 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org TABLE 1. Summary of Studies Included in Systematic Review of mHealth Interventions Assessing Contraceptive Uptake in Low- and Middle-Income Countries, N=8 mHealth Delivery Target Sample Sizea Frequency and Posttest and Authors Country Mode Population Study Design (Intervention/Control) Duration Follow-Up Johnson Kenya HE via text messag- General RCT (probably 13,629 (6,817/6,812) Over 3 months 24 hours, et al.18 ing, “role model” public unblinded)b 6 days, stories, clinic 3 months database postenrollment Maslowsky Ecuador Telephone-delivered Postpartum Unblinded 178 (102/76) Within 48 hours of 3 months after et al.19 HE and telephone women RCT hospital discharge. delivery access to a nurse Access to a nurse on-call during the first 30 days of the newborn’s life McCarthy Tajikistan HE via app instant Young people Single-blinded 543 (275/298) 0–3 messages per 4 months after et al.20 messaging (16–24), both RCT day over 4 months baseline genders McCarthy Palestine HE via text Young women Single-blinded 578 (289/289) 0–3 messages per 4 months after et al.21 messaging (18–24) RCT day over 4 months baseline Smith Cambodia Voice messages and Women, Single-blinded 300 (249/251) 6 automated voice 4 and et al.22 phone calls postabortion RCT messages 6 tele- 12 months phone counseling postabortion within 3-month period Unger Kenya HE via text Postpartum 3-arm, un- 300 (100/100/100) Weekly until From antenatal et al.23 messaging women blinded RCT 12 weeks care attendance postpartum and followed through 10, 16, 24 weeks postpartum Biswas Bangladesh HE via text Women, RCT (probably 120 (60 /60) Method-specific 4 months et al.24 messaging postabortion unblinded)b reminders/inter- postabortion vals (daily/weekly) Harrington Kenya HE via text Postpartum Unblinded 254 (125/129) Weekly from enroll- 6 months et al.25 messaging women RCT ment to 6 months postpartum postpartum Abbreviations: HE, health education (contraceptive information); RCT, randomized control trial; app, mobile application. a Data from participants who were analyzed. b Authors did not mention about blinding. This information was deduced from reading the studies. push approach to deliver information and an in- voluntary male partner.22,25 The study by Smith teractive type communication. et al.22 involved counselor phone support that was tailored to the participant’s need and provided motivation to use postabortion contraception and BCC Components information on nearest service providers. Interventions utilized different intervention com- The intervention by Unger et al.23 provided ponents to facilitate behavior change (Table 2), ranging from motivation to use family plan- weekly unidirectional (partial intervention) and ning,20–23 tailoring of information,19–23 partner interactive (full intervention) family planning re- involvement,22,25 service provider search fea- lated educational and motivational SMS tailored tures,18,22 and role model stories.18 Two of the to the recipient, and found that both full and par- 3 interventions that reported improved contracep- tial interventions improved early postpartum con- tive uptake included the involvement of a traceptive use over the control condition. The Global Health: Science and Practice 2020 | Volume 8 | Number 4 818 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org TABLE 2. mHealth Features and Behavior Change Communication Intervention Components Used in Studies Reviewed to Assess Effectiveness of Interventions on Contraceptive Uptake, N=8 Communication Pathway Family Planning Information Delivery Additional Intervention Components Searching for Evidence of “Push” “Push” Nearest Role Theory Effect (Improved via via “Pull”via Motivational Tailored Partner Service Model Framework Contraceptive Authors Unidirectional Interactive Telephone Messaging Messaging Message Information Involvement Provider Stories Used Frequency and Duration Use) Johnson Once Noa et al.18 Maslowsky Once Nob et al.19 McCarthy 0–3 messages per day Noc et al.20 for 4 months McCarthy 0–3 messages per day Nod et al.21 for 4 months Smith 2 times per month for Yese et al.22 3 months Unger Weekly Yesf et al.23 Biswas * Method-specific Nog,h et al.24 reminders (daily/weekly) Harrington Weekly Yesi et al.25 a Application was installed, and consumer received surveys for outcome measurement but had to search app for intervention materials. Recipients of full app showed increased knowledge over recipients of the limited app but no difference in contraceptive use. b Participants received 1 phone-based educational session and were invited to call back for more education and counseling, but only 3 did. Intervention partici- pants reported higher rates of breastfeeding and use of implants, but no differences were seen in overall contraceptive use. c No statistically significant difference in contraceptive use and acceptability between intervention and control. Serious contamination occurred, and both the in- tervention and control participants received intervention messages. d No statistically significant difference between the intervention and control groups in the use of effective contraception at 4 months. Intervention participants were more likely to find at least 1 method of effective contraception acceptable and had a higher mean knowledge score. e Participants received 6 automated, interactive voice messages with counselor phone support, if they opted, and outcome was measured at 4 months and 12 months post- abortion. Intervention group showed higher contraceptive use than the control group at both 4 months and 12 months, but the difference was only significantat4months. f Both unidirectional and interactive short message service (SMS) interventions improved early postpartum contraceptive use over the control condition. g Simple SMS reminder intervention did not improve contraceptive use at 4 months postabortion. h Method-specific text message reminders to use method selected by participants, in their preferred language for the messages, including Bangla (Unicode), English, or phonetic Bangla in English fonts. i The primary outcome of highly effective contractive (with less than 10% of failure rate) use at 6 months postpartum was significantly higher among women in the 2-way SMS group (69.9%) than in the control group (57.4%). Automated SMS text contained health education message and ended with actionable advice or a question to promote engagement. 25 study by Harrington et al. used a variant of the Study Quality and Risk of Bias: Cochrane’s 23 intervention used by Unger et al. (Mobile RoB 2 Tool WACh), but with a voluntary male partner in- Randomization Process volvement (Mobile WACh XY). However, male Cochrane’sRoB2tool(Figure 2) classified 5 studies involvement did not have a significant effect on as low risk and 3 studies with some concerns for risk contraceptive use outcomes compared with hav- of bias in the randomization process domain.18,24,25 ing women as the only participants. In terms of Johnson and colleagues18 used the alternation frequency of intervention delivery, findings from method of allocating participants to intervention these studies suggest that improved contraceptive and control groups, instead of true randomization. use was associated with weekly23 or biweekly22 The trials by Harrington et al.25 and Biswas et al.24 messaging rather than daily or a one-time had important baseline differences between their delivery. control and intervention groups. Global Health: Science and Practice 2020 | Volume 8 | Number 4 819 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org FIGURE 2. Risk of Bias in Studies of mHealth Interventions to Increase Contraceptive Uptake in Low- and Middle-Income Countries, N=8 Deviation From Intended Interventions Measurement of Outcome Five studies were conducted as intended and were All 8 studies had high risk of bias for measurement thereby classified as having a low risk of bias of the outcome (Figure 2). All the studies relied on (Figure 2).18,22–25 However, 3 studies deviated self-reported outcomes obtained from final assess- ments; thus, the assessors were the participants from their original protocol, resulting in a high 19–21 and the outcome measurement may have been risk of bias for this domain. The intervention subjected to social desirability bias. The collection by Maslowsky et al.19 had 2 parts, yet only 1 was of outcome data was not blinded. delivered. It also had substantial contamination, with controls also receiving the intervention, as Selection of the Reported Result 20 occurred in the intervention of McCarthy et al. As shown in Figure 2, half of the studies were at In another study, participants did not receive the low risk for selective outcome reporting since all complete intervention.21 outcomes were reported in their results.20–23 The other half of studies had some concerns for selec- Missing Outcome Data tion of the reported result because the protocols Seven studies had low risk of bias for missing out- containing details about their prespecified analytic come data (Figure 2). The study by Johnson et plan were not published.18,19,24,25 al.18 had some concerns for risk of bias due to hav- ing low retention rates: 20.9% of intervention and Overall Risk of Bias 21.3% of control participants were lost to follow- Cochrane’s RoB 2 tool classifies the overall risk of up for surveys that measured contraceptive up- bias to be considered high risk if any of individual take. To overcome this problem, researchers used domains (e.g., randomization process, missing multiple imputation methods for both groups but outcome data) assessed were deemed high risk.36 some concerns remain for risk of bias in this As a result, all 8 studies were labeled as having an domain.18 overall high risk of bias. Figure 3 provides a Global Health: Science and Practice 2020 | Volume 8 | Number 4 820 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org FIGURE 3. Summary of Risk of Bias of Studies of mHealth Interventions to Increase Contraceptive Uptake in Low- and Middle-Income Countries, N=8 summary of the risk-of-bias assessment for the Given that only 3 of the 8 studies found 8 included studies. improvements in family planning outcomes, the full extent that mHealth contributed to improve- ments in the use of modern contraceptives among participants cannot be determined. It is possible DISCUSSION that certain types of mHealth features may be more advantageous to effect change in the use of Summary of Findings modern contraceptives. For example, interactive To our knowledge, this systematic review is the communication and the use of a push approach first to assess the effectiveness of mHealth inter- to deliver intervention content entails engage- ventions toward increasing contraceptive use in ment with participants. The frequency that the in- LMICs. Other systematic reviews have examined tervention information is delivered in studies that mHealth in family planning interventions, but used the push approach may also have an impact few included studies from LMICs. Additionally, on participants’ use of modern contraceptives. 7 of the 8 studies in the present review were not Some studies in this review delivered intervention The current assessed in previous systematic reviews. Findings information once,18,19 daily,20,21 weekly,23,25 or systematic review from the current systematic review reveal new in- biweekly.22 Positive changes in outcomes were formation about the role that mHealth and BCC found in studies that delivered the intervention reveals new components have in improving contraceptive use information on a weekly or biweekly basis, sug- information about in LMICs. gesting too frequent delivery may not resonate the role of Of the 8 included studies, 3 reported improve- with participants with respect to their family plan- mHealth and BCC ments in family planning outcomes among people ning needs. components in who received the intervention compared with Analysis of the BCC components used among improving 22,23,25 controls. With respect to mHealth, 2 of the the 8 included studies suggests tailoring informa- contraceptive use 23,25 19–23,25 3 studies used text messages, while the other tion to the participant and potentially in LMICs. study used voice messages and telephone counsel- the use of motivational messages20–23 and/or the ing, which included information about the nearest involvement of a male partner22,25 may play a family planning service provider.22 Two common role in improving contraceptive use. Among the traits that the 3 studies shared were the use of in- 3 studies that showed significant improvements teractive communication and a push approach to in outcomes (intervention vs. control), all tailored deliver tailored intervention content to partici- the information delivered, whereas 2 of the stud- pants. Other commonalities were the use of moti- ies used motivational messages22,23 and 2 involved vational messages22,23 and the involvement of a the male partners of participants.22,25 However, male partner in the intervention.22,25 Harrington et al.25 conducted a subgroup analysis Global Health: Science and Practice 2020 | Volume 8 | Number 4 821 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org and found no significant differences in contracep- important determinant being motivation or inten- tive use between participants who had their male tion as the interventions targeted. A systematic re- partner enrolled versus those who did not. view by Cho et al.46 examined the use of theories in mHealth behavior change interventions con- Comparison With Existing Literature ducted in the LMICs and also found that about one-third (5 of 14) of their included studies were Interventions Our review found that interventions that showed based on a behavioral change theory. Well-tested improving significant improvement in contraceptive uptake behavioral change theories are useful to help contraceptive used a combination of unidirectional and interac- guide the design and implementation of family uptake combined tive communication styles and involved multiple planning interventions and programs.46–49 As the unidirectional and BCC components. Notably, simple unidirectional effectiveness of mHealth in family planning inter- interactive text message reminders had no effect on improv- ventions in LMICs remains inconclusive, future communication ing contraceptive uptake. Such findings are con- research that uses behavioral change theory for styles and used sistent with the evidence from the systematic contraception uptake is warranted and needed to multiple BCC review that assessed the effect of mHealth inter- help identify which intervention components components. ventions to improve contraceptive uptake, with (mHealth and behavior change) work best for 80% of studies involved having been conducted family planning and why. in developed countries.28 Systematic reviews on behavior change inter- The International Conference on Population ventions of other health topics that used mHealth and Development set the involvement of men in recommended the inclusion of certain components family planning as a priority area.39 Smith et al.22 to increase the effectiveness of the intervention. provided male partner telephone counseling by a For example, a systematic review on technological- nurse, upon request of the participant, and this ly driven weight-loss interventions by Khaylis et component may have been a contributing factor al.50 identified the following components as essen- in improving contraceptive uptake. Findings from tial for improving outcomes: use of behavior prior studies support this possible explana- – change theory, self-monitoring, counselor feed- tion.40 44 For example, a case study spanning back and communication, social support (motiva- 5 generations of a family in an LMIC setting found tion), and tailoring information. A meta-analysis that male involvement in family planning was as- by Webb et al.49 recommended that technology- sociated with fertility decline in the family (due to based interventions make extensive use of theory, increased use of contraception) and resulted in incorporate more BCC techniques, and use SMS long-term benefits for women.43 In another study, or text messages to effectively promote behavior Tao et al.44 found that involvement of the male change. These reviews, along with the present Behavioral partner in family planning decision making im- one, suggest that the use of behavioral change the- change theories proved family planning knowledge and contra- ories is important to improve targeted behaviors, are important in ceptive continuation. Moreover, a systematic while also recognizing that further investigation is improving review that examined different BCC techniques warranted to decipher which mHealth and BCC targeted used to improve contraceptive use in LMICs found components and in what combinations lead to bet- behaviors, and that the most effective interventions were those ter family planning outcomes. further that involved male partners.45 Prior research sug- investigation will gests the involvement of male partners is advanta- identify which geous for family planning and the uptake of Considerations of Intervention Fidelity, mHealth and BCC contraceptive methods. However, future research Missing Data, and Limited Use components lead is warranted to assess whether the type of male As noted in the risk-of-bias assessments, some to better partners differs (e.g., sexual/romantic relation- studies included in this review reported issues outcomes. ship, family, friend), as well as the amount and with intervention fidelity or missing data. Findings frequency of their involvement toward achieving from this review found important shortcomings in these outcomes. the included interventions that may have affected Only 3 studies included in this review reported the study’s findings. Four out of 5 studies that did using a behavioral change theory.20,21,25 Two of not find any significant changes in outcomes be- them were conducted by the same researchers, tween trial arms had poor implementation or re- who used the Integrated Behavioral Model,20,21 tention issues.18–21 and the other study used the Theory of Planned Regarding fidelity, the study conducted in Behavior.25 They are similar derivative theories Tajikistan by McCarthy et al.,20 found contamina- of general behavioral prediction, with the most tion between trial arms (i.e., some controls received Global Health: Science and Practice 2020 | Volume 8 | Number 4 822 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org a portion of the intervention content) because of a role model stories of current users. The study had misunderstanding between research partners. As low response rates to its 3 assessments (range: such, the trial was assessed as the full intervention 51.8% to 13.5%), and the proportion of partici- versus the partial intervention, instead of what was pants who responded to more than 1 assessment originally planned (i.e., comparing between inter- was low. This large number of missing longitudi- vention and control). Another study by McCarthy nal data affected the statistical power for the study and colleagues21 conducted in Palestine had tech- analyses, which may have influenced their nical problems with the messaging platform used, findings.18 which resulted in 60% of the intervention partici- Not all participants will use all parts of an inter- pants not receiving the full intervention. The out- vention. For example, Maslowsky et al.19 designed comes measured were only based on the effect of a 2-part intervention, with part 1 consisting of a partial receipt of the intervention versus the con- one-time telephone-delivered health education trol. Further, contamination may have also oc- session and part 2 consisted of having access to an curred: 17% (39/235) of the control participants on-call nurse for personalized advice (via tele- reported reading messages for someone else in the phone). Of the 178 study participants, only 3 par- study and 17% (40/229) of intervention partici- ticipants used part 2; participants had to take the pants said that someone else in the study read their initiative to use part 2. Access to the on-call nurse messages. for personalized advice included motivational With respect to missing data, the intervention support and tailored information, including where tested by Johnson et al.18 offered new users the to receive contraceptive services. Numerous rea- m4RH app with text-message-based family plan- sons may exist for why part 2 of the intervention ning information as well as a searchable database was not used by the study participants and how of service providers, with an option to receive its use and nonuse may have impacted the study’s TABLE 3. Quality of Evidence of the Contraceptive Uptake Outcome Using the GRADE Approach in Studies Included in the Review, N=8 Limitations of Detailed Unexplained Design and Heterogeneity or Execution Inconsistency of Indirectness Imprecisions Publication Quality of Study (Risk of Bias) Results of Evidence of Results Bias Evidence Smith À1 ÈÈÈ€ et al.22 Moderate Maslowsky À2 À1 È€€€ et al.19 Very low McCarthy À2 ÈÈ€€ et al.20 Low Johnson À1 ÈÈÈ€ et al.18 Moderate Unger À1 ÈÈÈ€ et al.23 Moderate McCarthy À2 ÈÈ€€ et al.21 Low Biswas À1 À1 ÈÈ€€ et al.24 Low Harrington À1 ÈÈÈ€ et al.25 Moderate Randomized controlled trials were considered to be high quality, but were downgraded by 1 level (serious) or 2 levels (very serious) for each of the following: limitations of detailed design and execution (risk of bias) (e.g., limitations in randomization, deviations from intended interventions), unexplained heterogeneity or inconsistency of results, indirectness of evidence, imprecision of results, and pres- ence of publication bias. Global Health: Science and Practice 2020 | Volume 8 | Number 4 823 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org findings. Future mHealth interventions for family with evaluation (e.g., one-group). Another con- planning ought to integrate monitoring of inter- sideration pertains to whether evidence on vention delivery and other process evaluation mHealth interventions conducted in LMICs is dis- techniques, as well as brief qualitative exit inter- seminated in peer-reviewed outlets (e.g., jour- views or quantitative measures (e.g., Health- nals), as noted by Gurman et al.54 in their ITUES),51 to better understand the reasons why systematic review. participants use and do not use certain parts of an intervention and how their usage affects the study’s findings. CONCLUSION AND Successful intervention outcomes necessitate RECOMMENDATIONS well-implemented programs, and implementation The use of mobile phones and smartphones in fidelity is crucial for the intervention effective- LMICs has proliferated, suggesting mHealth might ness.52 Half of the studies included in this review reported poor implementation or retention issues, be a viable tool for delivering interventions aimed which limits the ability to fully evaluate the inter- at improving family planning outcomes. However, vention and assess its impact on contraceptive up- there is insufficient evidence to conclude whether take outcomes. Future mHealth family planning mHealth interventions improve contraceptive up- trials ought to implement steps to help ensure take in LMICs based on the findings from this re- the fidelity to the protocol and design of the view and other systematic reviews.26,28 Although intervention. 3 of 8 studies in this review showed significant im- provement in contraceptive outcomes, their effec- Quality of the Evidence tiveness cannot be linked to specific mHealth features or BCC components. Quality of the evidence was assessed using the GRADE approach38 (Table 3). Five trials were Moreover, the quality of evidence suggests downgraded by 1 level under the domain of limita- that improvements in the implementation fidelity tions in design and execution because they both and use of behavior change theories are needed had a high risk of bias in the measurement of the for future mHealth family planning interventions outcome.22,23 Under the same domain, 3 trials19–21 in LMICs. Further investigation is warranted to as- were downgraded by 2 levels due to high risk of sess and identify which mHealth features, BCC bias from deviations from intended intervention, components, and theories, as well as in what spe- in addition to high risk of bias in measurement of cific combinations, will lead to better family plan- outcome. Two trials were downgraded by 1 level ning outcomes and for which specific groups and under the imprecision of results domain due to LMIC locations. small sample sizes.19,24 Overall, the quality of evi- dence was graded as moderate in 4 trials, low in Competing interests: None declared. 3trials,andverylowin1trial. Self-reported outcomes are the standard in REFERENCES contraceptive research, but they are subject to social The Millennium Development Goals Report 53 1. United Nations (UN). desirability bias. Additionally, intervention and 2015. UN; 2015. Accessed September 20, 2020. https://www. control participants recruited from the same hospital undp.org/content/undp/en/home/librarypage/mdg/the- or clinics might have shared intervention contents millennium-development-goals-report-2015.html with each other, resulting in contamination. 2. World Health Organization (WHO), UNICEF, UNFPA, World Bank Group, The United Nations Population Division. Trends in Maternal Mortality: 2000 to 2017. Estimates by WHO, UNICEF, UNFPA, Limitations World Bank Group and the United Nations Population Division. It is important to acknowledge that this review WHO; 2019. Accessed September 20, 2020. https://www.who. int/reproductivehealth/publications/maternal-mortality-2000- only included RCTs and nonrandomized studies 2017/en/ to evaluate the effectiveness of mHealth interven- 3. World Health Organization (WHO). Mother–Baby Package: tions. Other types of evidence may exist and ought Implementing Safe Motherhood in Countries. WHO; 1996. to be considered when evaluating the effective- Accessed September 20, 2020. http://apps.who.int/iris/ ness of mHealth interventions for family planning. bitstream/10665/63268/1/WHO_FHE_MSM_94.11_Rev.1.pdf For example, policy makers and other key stake- 4. Cleland J, Conde-Agudelo A, Peterson H, Ross J, Tsui A. Contraception and health. Lancet. 2012;380(9837):149–156. holders may find equal value from assessing how CrossRef. Medline well an mHealth-mediated family planning pro- 5. Ahmed S, Li Q, Liu L, Tsui AO. Maternal deaths averted by contra- gram has achieved its goals and outcomes through ceptive use: an analysis of 172 countries. Lancet. 2012;380 other types of study designs that blend research (9837):111–125. CrossRef. Medline Global Health: Science and Practice 2020 | Volume 8 | Number 4 824 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org 6. Lloyd CB, Ivanov S. The effects of improved child survival on family 21. McCarthy O, Zghayyer H, Stavridis A, et al. A randomized con- planning practice and fertility. Stud Fam Plann. 1988;19(3):141– trolled trial of an intervention delivered by mobile phone text mes- 161. CrossRef. Medline sage to increase the acceptability of effective contraception among young women in Palestine. Trials. 2019;20(1):228. CrossRef. 7. Potts M. Family planning is crucial to child survival. Netw Res Medline Triangle Park N C. 1990;11(4):2. Medline 22. Smith C, Ngo TD, Gold J, et al. Effect of a mobile phone-based inter- 8. Sedgh G, Ashford LS, Hussain R. Unmet Need for Contraception in Developing Countries: Examining Women’s Reasons for Not vention on post-abortion contraception: a randomized controlled tri- al in Cambodia. Bull World Health Organ. 2015;93(12):842– Using a Method. Guttmacher Institute; 2016. Accessed September 9, 850A. CrossRef. Medline 2020. https://www.guttmacher.org/sites/default/files/report_ pdf/unmet-need-for-contraception-in-developing-countries-report. 23. Unger JA, Ronen K, Perrier T, et al. Short message service pdf communication improves exclusive breastfeeding and early postpar- tum contraception in a low- to middle-income country setting: a ran- 9. United Nations (UN), Department of Economic and Social Affairs, domised trial. BJOG. 2018;125(12):1620–1629. CrossRef. Population Division. World Family Planning 2017. Highlights. UN; Medline 2017. Accessed April 6, 2019. http://www.un.org/en/ development/desa/population/publications/pdf/family/ 24. Biswas KK, Hossain A, Chowdhury R, et al. Using mHealth to WFP2017_Highlights.pdf support postabortion contraceptive use: results from a feasibility study in urban Bangladesh. JMIR Form Res. 2017;1(1):e4. CrossRef. 10. Alkema L, Kantorova V, Menozzi C, Biddlecom A. National, region- Medline al, and global rates and trends in contraceptive prevalence and unmet need for family planning between 1990 and 2015: a system- 25. Harrington EK, Drake AL, Matemo D, et al. An mHealth SMS inter- atic and comprehensive analysis. Lancet. 2013;381(9878):1642– vention on postpartum contraceptive use among women and couples Am J Public Health 1652. CrossRef. Medline in Kenya: a randomized controlled trial. . 2019;109(6):934–941. CrossRef. Medline 11. World Bank. World Development Report 2016: Digital Dividends. ’ World Bank; 2016. Accessed September 20, 2020. https://www. 26. L Engle KL, Mangone ER, Parcesepe AM, Agarwal S, Ippoliti NB. worldbank.org/en/publication/wdr2016 Mobile phone interventions for adolescent sexual and reproductive health: a systematic review. Pediatrics. 2016;138(3):e20160884. 12. Mangone ER, Lebrun V, Muessig KE. Mobile phone apps for the CrossRef. Medline prevention of unintended pregnancy: a systematic review and con- 27. Rousseau F, Da Silva Godineau SM, De Casabianca C, Begue C, tent analysis. JMIR Mhealth Uhealth. 2016;4(1):e6. CrossRef. Tessier-Cazeneuve C, Legendre G. State of knowledge on smart- Medline phone applications concerning contraception: a systematic review. J 13. Qiang C, Hausman V, Altman D. Mobile Applications for the Health Gynecol Obstet Hum Reprod. 2019;48(2):83–89. CrossRef Sector. World Bank; 2012. Accessed October 5, 2020. http:// 28. Smith C, Gold J, Ngo TD, Sumpter C, Free C. Mobile phone-based documents1.worldbank.org/curated/en/751411468157784302/ interventions for improving contraception use. Cochrane Database pdf/726040WP0Box370th0report00Apr020120.pdf Syst Rev. 2015(6):CD011159. CrossRef. Medline 14. World Health Organization (WHO). mHealth: Use of Appropriate Digital Technologies for Public Health. Report by the Director- 29. Moher D, Shamseer L, Clarke M, et al.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols General. Seventy-First World Health Assembly. WHO; 2018. (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1. CrossRef. Accessed October 5, 2020. https://apps.who.int/gb/ebwha/pdf_ Medline files/WHA71/A71_20-en.pdf 30. World Bank. World Bank country and lending groups. Accessed WHO Guideline: 15. World Health Organization (WHO). August 19, 2020. https://datahelpdesk.worldbank.org/ Recommendations on Digital Interventions for Health System knowledgebase/articles/906519-world-bank-country-and- Strengthening . WHO; 2019. Accessed July 20, 2020. https://www. lending-groups who.int/reproductivehealth/publications/digital-interventions- Reproductive Health Indicators: health-system-strengthening/en/ 31. World Health Organization (WHO). Guidelines for Their Generation, Interpretation and Analysis for 16. Guisado Lopez R, Ramirez Polo I, Arjona Berral JE, Guisado Global Monitoring. WHO; 2006. Accessed October 1, 2020. Fernandez J, Castelo-Branco C. iContraception®: a software tool to https://apps.who.int/iris/handle/10665/43185 assist professionals in choosing contraceptive methods according to ’ Ending Child Marriage: WHO medical eligibility criteria. J Fam Plann Reprod Health Care. 32. United Nations Children s Fund (UNICEF). Progress and Prospects 2015;41(2):142–145. CrossRef. Medline . UNICEF; 2014. Accessed October 5, 2020. https://data.unicef.org/wp-content/uploads/2015/12/Child- 17. Sridhar A, Chen A, Forbes ER, Glik D. Mobile application for infor- Marriage-Brochure-HR_164.pdf mation on reversible contraception: a randomized controlled trial. mHealth: New Horizons for Am J Obstet Gynecol 33. World Health Organization (WHO). . 2015;212(6):774.e1-774.e7. CrossRef. Health Through Mobile Technologies: Second Global Survey on Medline eHealth. Global Observatory for eHealth Series. WHO; 2011. 18. Johnson D, Juras R, Riley P, et al. A randomized controlled trial of the Accessed September 27, 2020. https://www.who.int/goe/ impact of a family planning mHealth service on knowledge and use publications/goe_mhealth_web.pdf of contraception. Contraception. 2017;95(1):90–97. CrossRef. 34. World Health Organization (WHO) Department of Reproductive Medline Health and Research (WHO/RHR); Johns Hopkins Bloomberg 19. Maslowsky J, Frost S, Hendrick CE, Trujillo Cruz FO, Merajver SD. School of Public Health/Center for Communication Programs Effects of postpartum mobile phone-based education on maternal (CCP). Family Planning: A Global Handbook for Providers. 2018 and infant health in Ecuador. Int J Gynaecol Obstet. 2016;134 Edition. CCP and WHO; 2018. Accessed March 4, 2020. https:// (1):93–98. CrossRef. Medline www.who.int/reproductivehealth/publications/fp-global- 20. McCarthy O, Ahamed I, Kulaeva F, et al. A randomized controlled handbook/en/ trial of an intervention delivered by mobile phone app instant mes- 35. International Telecommunications Union (ITU). World saging to increase the acceptability of effective contraception among telecommunication/ICT indicators database. Accessed October 2, young people in Tajikistan. Reprod Health. 2018;15(1):28. 2020. https://www.itu.int/en/ITU-D/Statistics/Pages/ CrossRef. Medline publications/wtid.aspx Global Health: Science and Practice 2020 | Volume 8 | Number 4 825 Review of Effectiveness of mHealth Interventions for Improving Contraceptive Use www.ghspjournal.org 36. Sterne JAC, Savovic J, Page MJ, et al. RoB 2: a revised tool for 45. Phiri M, King R, Newell JN. Behaviour change techniques and con- assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. traceptive use in low and middle income countries: a review. Reprod CrossRef. Medline Health. 2015;12(1):100. CrossRef. Medline 37. Higgins J, Thomas J, Chandler J, et al. Cochrane Handbook for 46. Cho YM, Lee S, Islam SMS, Kim SY. Theories applied to mHealth Systematic Reviews of Interventions. Version 6.1. Cochrane; 2020. interventions for behavior change in low- and middle-income coun- Accessed October 5, 2020. https://training.cochrane.org/ tries: a systematic review. Telemed J E Health. 2018;24(10):727– cochrane-handbook-systematic-reviews-interventions 741. CrossRef. Medline 38. Schünemann H, Broz²ek J, Guyatt G, Oxman A, eds. Handbook for 47. Prestwich A, Sniehotta FF, Whittington C, Dombrowski SU, Rogers L, Grading the Quality of Evidence and the Strength of Michie S. Does theory influence the effectiveness of health behavior Recommendations Using the GRADE Approach. GRADE Working interventions? Meta-analysis. Health Psychol. 2014;33(5):465–474. Group; 2013. Accessed October 1, 2020. https://gdt.gradepro. CrossRef. Medline org/app/handbook/handbook.html 48. Vandelanotte C, Müller AM, Short CE, et al. Past, present, and future 39. United Nations (UN). Report of the International Conference on of eHealth and mHealth research to improve physical activity and Population and Development. Cairo, 5–13 September 1994. UN; dietary behaviors. J Nutr Educ Behav. 2016;48(3):219–228.e1. 1994. Accessed October 5, 2020. https://www.un.org/en/ CrossRef. Medline development/desa/population/events/pdf/expert/27/ 49. Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote SupportingDocuments/A_CONF.171_13_Rev.1.pdf health behavior change: a systematic review and meta-analysis of 40. Berhane A, Biadgilign S, Berhane A, Memiah P. Male involvement in the impact of theoretical basis, use of behavior change techniques, family planning program in Northern Ethiopia: an application of the and mode of delivery on efficacy. J Med Internet Res. 2010;12(1):e4. Transtheoretical model. Patient Educ Couns. 2015;98(4):469–475. CrossRef. Medline CrossRef. Medline 50. Khaylis A, Yiaslas T, Bergstrom J, Gore-Felton C. A review of 41. Bright C, Onwere SN, Onwere AC, Kamanu CI, Ndukwe PE, Chigbu efficacious technology-based weight-loss interventions: five key Telemed J E Health – E. Improving male involvement in family planning in rural components. . 2010;16(9):931 938. CrossRef. Southeastern Nigeria. Obstet Gynecol. 2015;125(Suppl 1):66S– Medline 67S. CrossRef 51. Yen PY, Wantland D, Bakken S. Development of a customizable AMIA Annu Symp Proc 42. Ha BTT, Jayasuriya R, Owen N. Increasing male involvement in fam- Health IT Usability Evaluation Scale. . – ily planning decision making: trial of a social-cognitive intervention 2010;2010:917 921. Medline in rural Vietnam. Health Educ Res. 2005;20(5):548–556. CrossRef. 52. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A con- Medline ceptual framework for implementation fidelity. Implement Sci. 43. Karra MV, Stark NN, Wolf J. Male involvement in family planning: a 2007;2(1):40. CrossRef. Medline case study spanning five generations of a south Indian family. Stud 53. Stuart GS, Grimes DA. Social desirability bias in family planning Fam Plann. 1997;28(1):24–34. CrossRef. Medline studies: a neglected problem. Contraception. 2009;80(2):108–112. 44. Tao AR, Onono M, Baum S, et al. Providers’ perspectives on male CrossRef. Medline involvement in family planning in the context of a cluster-randomized 54. Gurman TA, Rubin SE, Roess AA. Effectiveness of mHealth behavior controlled trial evaluating integrating family planning into HIV care change communication interventions in developing countries: a sys- in Nyanza Province, Kenya. AIDS Care. 2015;27(1):31–37. tematic review of the literature. J Health Commun. 2012;17(Suppl CrossRef. Medline 1):82–104. CrossRef. Medline Peer Reviewed Received: February 14, 2020; Accepted: September 22, 2020; First published online: November 12, 2020 Cite this article as: Aung B, Mitchell JW, Braun KL. Effectiveness of mHealth interventions for improving contraceptive use in low- and middle-income countries: a systematic review. Glob Health Sci Pract. 2020;8(4):813-826. https://doi.org/10.9745/GHSP-D-20-00069 © Aung et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00069 Global Health: Science and Practice 2020 | Volume 8 | Number 4 826 METHODOLOGY A Practical Guide to Using Time-and-Motion Methods to Monitor Compliance With Hand Hygiene Guidelines: Experience From Tanzanian Labor Wards Giorgia Gon,a Said M. Ali,b Robert Aunger,a Oona M. Campbell,a Mícheál de Barra,c Marijn de Bruin,d,e Mohammed Juma,b Stephen Nash,a Amour Tajo,b Johanna Westbrook,f Susannah Woodd,a Wendy J. Grahama Key Messages ABSTRACT n Background: Good-quality evidence on hand hygiene compli- Time-and-motion methods are a good way of ance among birth attendants in low-resource labor wards is lim- capturing hand hygiene compliance; for ited. The World Health Organization Hand Hygiene Observation example, they can limit observer bias. Form is widely used for directly observing behaviors, but it does n We describe how we designed the HANDS at not support capturing complex patterns of behavior. We devel- Birth tool, the tool format and its elements, its oped the HANDS at Birth tool for direct observational studies of implementation components, the tool’s complex patterns of hand rubbing/washing, glove use, recon- performance, and the implications for data tamination, and their determinants among birth attendants. Understanding these behaviors is particularly critical in wards analysis. with variable patient volumes or unpredictable patient complica- n The advantages of using this tool include simpler tions, such as emergency departments, operating wards, or tri- training, less observer judgment in assessing age and isolation wards during epidemics. Here we provide hand hygiene compliance, and improved ability detailed information on the design and implementation of the to monitor multiple behaviors. HANDS at Birth tool, with a particular focus on low-resource set- tings. We developed the HANDS at Birth tool from available guidelines, unstructured observation, and iterative refinement based on consultation with collaborators and pilot results. We designed the tool with WOMBAT software, which supports col- lecting multidimensional time-and-motion data. Our analysis of the tool’s performance centered on interobserver agreement and convergent validity and the implications of the data structure for data analysis. The HANDS at Birth tool encompasses various hand actions and context-relevant information. Hand actions in- clude procedures relevant during labor and delivery; hand hy- giene or glove actions; and other types of touch. During field implementation, we used the tool for continuous observation of the birth attendant. Interobserver agreement was good (kappa range: 0.7–0.9), and the tool showed convergent validity. Using the HANDS at Birth tool is a feasible way to obtain useful informa- tion about compliance with hand hygiene procedures. The tool could be used after simple training and allows for collection of re- liable information about the complex pattern of hygiene behaviors. Future studies should explore using this tool to observe behavior in labor wards in other settings and in other types of wards. a London School of Hygiene and Tropical Medicine, London, United Kingdom. b Public Health Laboratory-Ivo de Carneri, Pemba, Zanzibar, Tanzania. BACKGROUND c Brunel University London, Department of Life Sciences, Uxbridge, United Kingdom. nfection prevention is paramount to limiting the spread d I Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United of epidemics, such as coronavirus disease 2019 (COVID- Kingdom. 19), severe acute respiratory syndrome, and Ebola, and e Department of IQ Healthcare, Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands. hand hygiene (HH) is at the forefront of prevention efforts 1 f Macquarie University, Sydney, Australia. among health care workers. In addition, health care Correspondence to Giorgia Gon ([email protected]). workers’ HH is essential at the time of birth for preventing Global Health: Science and Practice 2020 | Volume 8 | Number 4 827 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org Time-and-motion health care-associated infections that lead to an observation10 and are increasingly used, but sel- methods can enormous burden of illness among mothers and dom in low- and middle-income countries. These 2–4 overcome some of newborns, even in nonepidemic situations. methods enable observers to record all health care ’ the challenges Accurately understanding the specific actions that workers actions without having to decide which ones represent a new HH opportunity. Instead, with direct contribute to the low compliance levels for HH opportunities are defined during data analysis. observation that occurs in many countries, particularly in low- resource settings, is essential for effective behavior TheHANDSstudy(Hand-hygieneofAttendants because they for Newborn Deliveries and Survival) was a mixed- enable observers change; yet current tools fail to account for health care workers’ workflow and the issue of recontami- methods, cross-sectional research study conducted to record all nation and its drivers. in the 10 highest volume maternity wards in health care 8 Multiple methods exist to measure HH compli- Zanzibar between November 2015 and April 2017. workers’ actions The aim of the study was to explore compliance with ance in health care settings, but observation of without having to HH guidelines and identify factors that explain com- behaviors is considered to be the gold standard.5 decide which ones pliance. HH during labor and delivery is a key oppor- Observation can be done by an observer or by vid- represent a new tunity to prevent transmitting infections to mothers eo recording. A recent validation study suggests 3,11 hand hygiene and newborns ; however, good-quality evidence that both approaches capture similar numbers of opportunity. on HH compliance from low-resource labor wards HH opportunities—moments when health care is limited.12–16 Therefore, we developed the HANDS workers ought to practice hand rubbing/washing6; at Birth tool, based on a time-and-motion design, to however, video recording poses substantial ethical observe the complex patterns of birth attendants’ issues, which often makes it difficult to use, particu- HH and glove use at 3 levels: the opportunity, the in- larly in a process such as childbirth when women dividual, and the facility. We designed the tool with- are vulnerable and undressed. The World Health in WOMBAT software, which is intended to support direct observational studies of health care work. The Organization (WHO) HH Observation Form is an 17,18 excellent, widely used tool for direct observation.7 WOMBAT software package allows collecting multidimensional work tasks, including compliance However, due to its aim and scope, it does not allow with specific tasks, and automatically time-stamps capturing more complex patterns of behavior. For data entry. The current investigation was one of the example, it does not distinguish whether the failure few time-and-motion studies of health care workers to comply was because hand rubbing/washing was conducted with software that automatically records not attempted or because hands were recontami- time and carried out in a low-resource setting.19,20 8 nated after initial washing. Avoiding hand/glove Our aim was to provide very practical details recontamination is implicit in the WHO tool’sHH regarding the design and implementation of the definition because touching a surface carries the direct observational tool to measure HH compli- risk of germ transmission and creates a new HH op- ance to inform researchers and practitioners seek- portunity. It also does not aim to capture the use or ing to thoroughly measure the compliance with “misuse” of gloves.9 Finally, it requires the obser- HH guidelines during labor and delivery, particu- vers to judge when a new HH opportunity arises, larly in low-resource settings. In this article, we thereby reducing the consistency of data collection outline: (1) how we designed the data collection by multiple observers. tool, (2) the tool format and its elements, (3) its im- ’ Defining when a new HH opportunity arises is plementation components, (4) the tool s perfor- particularly difficult in labor and delivery, during mance, and (5) the implications for data analysis. which observers must deal with a transition from observing 1 patient (the mother) to 2 (mother METHODS FOR TOOL DEVELOPMENT and newborn). Furthermore, the amount, type, We developed the HANDS at Birth data collection and location of body fluids can rapidly change dur- tool between March and October 2016 using an ing labor and delivery, and in the context of low- existing systematic process for tool development.13 resource settings, a single health care worker may This process included use of available guidelines, attend many mothers simultaneously. With an of- unstructured observation, and iterative refinement ten unpredictable duration of the different stages based on consultation with collaborators and pilot of labor, the time between hand rubbing/washing results. and delivery of the newborn may be lengthy, dur- ing which time the observer needs to pay close at- tention to assess if any actions occur that lead to a Guidelines’ Review and Semistructured new HH opportunity. Time-and-motion methods Observation can overcome some of these challenges. These We consulted WHO publications, including Hand methods are now at the forefront of health care Hygiene Technical Reference Manual,7 Hand Hygiene Global Health: Science and Practice 2020 | Volume 8 | Number 4 828 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org in Outpatient and Home-Based Care and Long-term Ethics Committee at the University of Aberdeen. Care Facilities,21 and Pregnancy, Childbirth, Post- Details of procedures to consent are described partum and Newborn Care: A Guide for Essential below. Practice.22 We also conducted 11 semi-structured observation sessions in 4 labor wards in Zanzibar TOOL FORMAT AND ELEMENTS during which either a delivery or a vaginal exami- 10 Following Lopetegui et al.’s classification, our Our time-and- nation occurred. All birth attendants’ actions were time-and-motion study used continuous observa- recorded, together with the time when they hap- motion study used tion, in which an external observer focuses on pened and their location. Using this information, continuous 1 subject, in our case, the birth attendant. When a we created a list of procedures (what we also call observation, in birth attendant performed an action, the observer “key attendant-patient interactions”) relevant to whichanexternal recorded the action. We chose to use continuous labor and delivery that also included other hand observer focuses observation because the timing of procedures, actions that can occur before and after each of on one subject, particularly delivery itself, was typically unpre- these procedures. such as a birth dictable, and using alternative methods, such as attendant. short observation sessions at fixed or random Iterative Collaborator Consultation intervals, could have missed many HH opportuni- The project was a partnership of the London ties. Hence, observers were asked to remain in the School of Hygiene and Tropical Medicine, the labor room for the entirety of their allocated shift University of Aberdeen, and the Public Health (about 7 hours for morning/afternoon shifts and Laboratory of Pemba; we sought feedback on the 10 hours for night shifts) and to start recording tool from all project members. Additionally, a observations whenever a patient-attendant inter- 3-hour in-depth consultation was conducted action began. with 2 clinically trained members of the team The tool, available in Supplement 1, includes a (1 general practitioner and 1 midwife) who pro- list of hand actions and context-relevant informa- vided additional feedback. tion (Figure). The hand actions listed were ex- haustive (meaning that the list did not leave any Pilot Activities and Training possible actions out) and mutually exclusive We conducted 3 pilot activities in a labor ward on (meaning that no 2 actions could occur simulta- Pemba Island, Zanzibar, Tanzania. Two data col- neously). We did not design a tool that aimed to lectors conducted the first pilot in June 2016 using capture multitasking or interruptions because an early version of the HANDS at Birth tool. One we did not want to add to the burden on the data collector conducted the second pilot in observers. August 2016 using the tool incorporated into Hand actions were either procedures relevant WOMBAT v2 software on a tablet. Finally, 1 data during labor and delivery (e.g., vaginal examina- collector conducted the third pilot in September tion) (Table 1), HH or glove actions, or other types 2016 using the tool with WOMBAT. Feedback of touches (e.g., touching a pen or equipment). was collected and incorporated to improve the Observers recorded when an attendant left the tool at each stage. room where observation was occurring (when ob- Observers were trained to use the tool over servation was suspended) and when the attendant 3 days using role-plays and presentations. Each re-entered. observer also practiced using the tool in the labor The tool also captured information on the con- ward for 3 hours under trainer supervision (GG). text, such as availability of key infrastructure/ The trainer also conducted 2 hours of observation staffing (e.g., water or the presence of the nurse with each observer and provided them with rele- in-charge) and which woman was being attended vant feedback. During training, minor refine- (first, second, third, etc. since the beginning of the ments were made to the tool. observation session). This process allowed us to We used the STROBE checklist for cross- assess whether birth attendants performed HH sectional studies to design and describe this tool between patients. Observers entered this context- here and in other relevant manuscripts including related information at the beginning of the obser- the study results.23 vation session and updated it only if the situation The project was approved by the Zanzibar changed. Medical Research and Ethics Committee, the Many of the recorded actions required further London School of Hygiene and Tropical Medicine details to be entered. For example, when a deliv- Research Ethics Committee, and the Research ery was observed, the observer also recorded Global Health: Science and Practice 2020 | Volume 8 | Number 4 829 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org FIGURE. Screen Showing HANDS at Birth Tool to Collect Multidimensional Time-and-Motion Data on Hand Actions and Context-Relevant Information on Hand Hygiene. (Left) Screen That Appears When User Logs In to Tool. (Right) Screen That Appears When User Scrolls Down. Abbreviations: BA, birth attendant; VE, vaginal examination. whether the delivery occurred rapidly (within Sample Size Calculations 5 minutes of the woman walking into the labor The data collection timeframe was based on the room), whether there were complications, wheth- expected number of deliveries in the targeted fa- er the observer birth attendant had an assistant, cilities. We estimated the latter, using the formula and whether a premade delivery kit was used. for estimating a proportion from a cross-sectional The observers collected contextual information survey with a=0.05 and 80% power. We used a and details of certain actions because we intended design effect of 2 based on a survey by Rowe et to use these data as potential determinants of HH in al.26 To estimate a hand rubbing/washing compli- the analysis. The determinants collected and associ- ance of 10% with an absolute precision of 63%, ated with HH are described in detail in Gon et al.24 we needed 768 HH opportunities. We estimated the length of observation needed to collect this number, and in practice these data were collected TOOL IMPLEMENTATION during 336 observation sessions ranging from This section characterizes how we used the tool to 13 minutes to 6 hours 45 minutes, with a median collect data and provides considerations for using time of 1 hour and 41 minutes.8 As described in it in future studies. We used the guidance provid- Gon et al,8 we collected information on 781 HH op- ed by Zheng et al.25 for reporting time-and-motion portunities before aseptic procedures (before aseptic studies, and include their full STAMP checklist in- procedures is 1 of the 5 types of HH opportunity pre- formation in Supplement 2. scribed by WHO7). Global Health: Science and Practice 2020 | Volume 8 | Number 4 830 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org becomes accustomed to the same person being on TABLE 1. Relevant Hand Actions During Labor and the ward; (2) the initial attendant-observer pairs Delivery Included in the HANDS at Birth Tool for at each facility were assigned at random (unless Observation of Birth Attendants specific concerns were raised; e.g., some flexibility on choice of types of shifts was allowed to accom- Measuring vital signs modate observers’ needs); and (3) observation Wiping the vagina days should ideally be planned during changes in Vaginal examination shift pattern to allow observation of the same attendants working on different types of shifts. Artificial rupture of membranes The need to observe the same attendant across dif- Episiotomy ferent types of shifts using the same observer in- Catching the baby (delivery) creased the fieldwork duration and therefore had Cord cutting and clamping to be counterbalanced by the need to remain with- in our budget. Cord traction Postdelivery examination of the vagina The Observers Wiping the baby clean after birth Observers were all trained nurse-midwives work- Observers were Supporting breastfeeding ing in managerial roles. Two of them worked in all trained nurse- the study facilities but not in the labor wards. The Manual removal of placenta midwives, whose third observer worked in district-level manage- previous Suturing ment. Their previous knowledge and understand- knowledge and ’ Suctioning baby s nose/mouth ing of the labor process were vital to ensuring understanding of Using bag and mask on the baby quality during data collection and ultimately the the labor process project’s success. Catheter insertion or removal were vital to the success of the Insertion or removal of IV lines Study Participants project. Adjusting IV fluids or changing IV bag All birth attendants present during the observa- Abbreviation: IV, intravenous. tion period who were involved in the childbirth procedures outlined in Table 1 were eligible for ob- servation. We observed a total of 104 birth atten- Planning and Logistics of Data Collection dants across the 10 facilities and between 4 and To obtain representative data on deliveries across 15 birth attendants in each facility. Each attendant was observed for 1–9 observation sessions.8 In each all shifts (morning, afternoon, and night); 3 obser- observation session, only 1 attendant was ob- vers, 1 per shift, conducted observations that cov- served, but that attendant could be caring for mul- ered 24 hour a day. They observed for a total of tiple women and carrying out many procedures. 130 hours in the morning, 153 hours in the after- Attendants in our study were all women, 90% noon, and 205 hours in the night. Each observer were professionally trained, and 10% were health had their own tablet for data collection. Each facil- orderlies/nonprofessionals. ity was visited for a mode of 6 consecutive days The attendants’ responsibilities were usually (range: 5–14 days) between September 17 and allocated during the shift itself. We encouraged December 31, 2016. The order in which we visited observers to listen at staff meetings to learn which the facilities was based on logistics. We arranged attendant was most likely to perform the child- for additional days of observation in 1 facility with birth procedures outlined in Table 1 to decide a high volume of staff to allow all staff to be ob- whom to observe. Observers were instructed to served and in 3 facilities with low volume of deliv- observe each allocated birth attendant roughly We were eries to capture a sufficient number of procedures. equally in each facility. specifically We consulted the ward rosters to allocate indi- interested in the vidual attendants to the observers. Each attendant How to Observe attendants’ had a unique identifier that the observer had to We trained the observers to enter only 1 action at a actions, the record in WOMBAT when observing them. time to facilitate the data input process. We were sequence of these Observers were allocated to shifts based on the fol- specifically interested in the attendants’ actions, actions, and the lowing principles: (1) the same observer should the sequence of these actions, and the length of length of time observe the same attendant so the attendant time between them, rather than the duration of between them. Global Health: Science and Practice 2020 | Volume 8 | Number 4 831 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org each action per se. An action was selected and en- compliance with hand rubbing/washing before tered immediately. We do not have details on aseptic procedures. Compliance was 10% higher when the action ended, but since the actions than the second-best facility and 7 times higher were mutually exclusive, it was clear when one than the worst one. For ethical reasons, observers action replaced another. were trained to notify health workers and the field manager if they observed a potentially harmful When to Observe condition or practice. As described above, a relevant patient-attendant interaction (Table 1) triggered the start of data en- Quality of Data Collection try; observers were expected to be continuously To ensure quality of data collection, we held regu- present in the ward due to the unpredictable na- lar meetings with collectors by telephone and ture of birth. Observers were encouraged to take onsite, communicated via a WhatsApp group, breaks when no women were in labor or when held Skype calls at the end of observations in women were in very early stages of labor and to each facility, and monitored the data uploaded remain where they could see if an emergency ad- monthly. These communication channels enabled mission occurred to avoid missing delivery events. rapid feedback, answers to questions, and mainte- We also encouraged breaks if the observer’s con- nance of morale during long periods of observa- centration level was low. tion. Drivers ensured observers arrived at sites on We instructed observers to end a session when time. Finally, we are confident that the data were a major procedure ended and no further patient unlikely to have been manufactured because activities were in sight, when the observer wanted manufacturing time-stamped data would require to take a break, when there was the opportunity to as much time as conducting and recording actual start observing another birth attendant, or when observations. the birth attendants left the room to perform duties elsewhere. Software and Information Technology Costs Where to Observe The cost of the software and hardware also needs to Observers would usually sit in the labor room. If be considered especially for deployment in low- and no deliveries were happening, we asked observers middle-income countries. WOMBAT 3.0 is available to observe vaginal examinations in other rooms, from the Apple Store (https://apps.apple.com/us/ such as the antenatal ward or examination room. app/wombat-3-0/id1445107457). Data hosting is available at a cost of US$2,500 for a 2-year period, Consent and Study Aim Concealment which allows the use of the software for multiple Written consent was gathered from women in the projects and data collectors. Free packages such as antenatal ward before observation; alternatively, Open Data Kit could be used, but Open Data Kit is women were asked for verbal consent once in the less user friendly for time-and-motion studies. In labor ward, and follow-up for written consent oc- addition, we bought 3 tablets for approximately curred in the postnatal ward before discharge or US$500. before delivery in the antenatal ward.8 Women were told that no demographic information was TOOL PERFORMANCE collected on them and recorded observations were exclusively regarding birth attendants’ be- Interobserver (Interrater) Agreement havior. Permission to observe the attendants was To report on interobserver agreement procedures obtained by the Ministry of Health and verbal con- and findings, we followed the recommendations sent was obtained by the observers when they first by Lopetegui et al.27 for time-and-motion studies visited the facility.8 and consulted the WOMBAT guidelines.18,28 Attendants were told the observation was While piloting the tool, the trainer conducted about the quality of care at birth, not on HH speci- 2 hours of simultaneous observation between the fically, to conceal the study’s focus and reduce the trainer (GG) and each of the observers. We then Hawthorne effect. In all but the facility in Zanzibar verified the extent of agreement between GG and where piloting took place, the focus of the study each of the 3 observers on the basis of 28, 29, and (HH practices) was likely to have been well con- 36 opportunities for hand washing/rubbing, cealed from the birth attendants being observed. glove wearing, and touch events, respectively. The pilot facility in Zanzibar had the highest The observations were based on a total of 11 vaginal Global Health: Science and Practice 2020 | Volume 8 | Number 4 832 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org examinations and 5 deliveries. The exercise was also confident that discrepancies between observers used to provide feedback to the observers. was minimal because our results showed that During the first month of data collection, we hand rubbing/washing compliance before aseptic also assessed interobserver agreement, whereby a procedures did not vary substantially by observer, pair of observers was allocated to 2 of the same as described in Gon et al.8 shifts in the busiest facility and asked to observe the same attendants. Observers were asked to per- form this independently, avoiding communica- Convergent Validity tion or looking at each other’s tablet, but we We assessed the degree to which 2 measures of could not ensure they were blinded, which meant constructs that theoretically should be related that they probably knew we were going to check were in fact related (convergent validity) by show- the data and hence some form of communication ing whether hand rubbing/washing before aseptic might still have occurred. Two pairs carried out procedures compliance varied in the expected di- rection by contextual characteristics. Using the this exercise for 1 morning and 1 afternoon shift 24 each, the other pair for 2 night shifts. Two pairs methods described in Gon et al, we descriptively observed 3 birth attendants, and the third pair ob- showed that higher compliance was present when served 4. the necessary equipment (water and soap or gel) We calculated kappa statistics based on either was available, when fewer women were attended 49 or 50 hand rubbing/washing, hand recontami- in the same observation session (i.e., a lower nation, or glove behaviors per pair of observers. workload was expected to be associated with bet- Observations were based on a total of 9 vaginal ter HH), and when attendants had received HH re- examinations and 11 deliveries. Through visual fresher training in the previous year (Table 2). inspection of the data, we ensured that the beha- viors compared were the same between observers IMPLICATIONS FOR DATA ANALYSIS by checking the reported time and sequence of actions. The kappa statistic calculated for pairs of AND INTERPRETATION observers was good for 2 of the 3 pairs at 0.93 and In Supplement 3, we describe data cleaning, anal- 0.90, but it was below the optimal level of 0.85 for ysis, and interpretation issues that needed to be 1 of the pairs at 0.73.18 In addition, we are also considered, noting in particular, that some data TABLE 2. Hand Rubbing/Washing Compliance Before Aseptic Procedures Among Birth Attendants in Health Facilities in Zanzibar, Tanzania Observed Opportunities/Indications Hand Hygiene Compliance for Hand Hygiene, (Hands Rubbed/Washed) per Guidelinesa When Indicated n (%), N=779 n (%), N=190 Necessary hand hygiene equipment (water and soap or gel) No 48 (6.2) 5 (10.4) Yes 704 (90.4) 177 (25.1) Missing 13 (1.7) 3 (23.1) Inconsistent information 14 (1.8) 5 (35.7) Maximum number of women attended in an observation session 1 541 (69.5) 146 (27.0) 2 196 (25.2) 39 (19.9) 3 36 (4.6) 4 (11.1) Missing 6 (0.8) 1 (16.7) Hand hygiene refresher training in the past 12 months No 347 (44.5) 74 (21.3) Yes 432 (55.5) 116 (26.9) a Number of times when hand hygiene was meant to be performed per guidelines. Global Health: Science and Practice 2020 | Volume 8 | Number 4 833 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org items relied on observer subjectivity (e.g., dura- any clean or sterile equipment being used, and the tion of hand washing) and some variables (e.g., newborn as it was caught and wiped. A break in the variables describing the context) required more delivery flow, indicating a new HH opportunity, stringent training than others. arose if an activity occurred that was outside the patient zone, such as inserting an intravenous line, Data Structure touching the patient beyond the zone, or leaving 8 Astrengthof A strength of WOMBAT is that when each action the room. WOMBAT is that is recorded, the time of that action is automatically Details on the definitions used in our study are 8 when each action logged. Our final dataset was a list of 7,893 time- reported in Gon et al. Potentially, a separate soft- is recorded, the ordered entries. These data were coded to derive ware could be programmed to automatically ana- time of that action HH opportunities and to calculate compliance. lyze this type of data in the future, allowing for definitions to be applied from the outset. is automatically First, each HH opportunity needed to be identified logged. within each observation session, which is further explained below. Second, for HH opportunities Context-Specific Adaptations before aseptic procedures or touching the patient, To classify which surfaces we should include in the the sequence of actions preceding the opportunity patient zone, we used previous formative re- needed to be examined for hand rubbing/washing search29 on the microbiological load of the labor actions, glove use and actions that may lead to a surfaces in Zanzibar, as well as unstructured ob- new HH opportunity. Whereas, for HH opportuni- servation of labor wards conducted within the ties after exposure to body fluids or touching the HANDS project. For example, we excluded the de- patient or the patient’s surrounding, the actions livery bed and trolley from the patient zone be- following the opportunity needed to be examined. cause previous work found that these surfaces We used STATA to analyze these data. were often contaminated with potential patho- gens.8 Other important information to consider Time Stamps include the details of the cloth or plastic sheet We used WOMBAT’s time stamp information in used under the woman’s body during birth, the 2 ways. First, we checked the plausibility of certain cleaning routines of the wards, the type of water actions being linked; for example, a hand rubbing/ available, the delivery equipment preparation, washing action could not be linked to a procedure and the local HH guidelines against which to mea- conducted 10 hours before or after it. Second, we sure hand washing/rubbing duration and tech- calculated the length of time between hand rub- nique. It is not clear that all projects will have the bing/washing and the HH opportunity to deter- capacity to gather this level of contextual informa- mine whether time would predict the likelihood tion; however, capturing the real workflows in of hand recontamination occurring. this context was our aim. Ideally, all definitions should be clear at the A Priori Definitions Required start of a project, but during data collection, the project may accrue context-specific information To estimate HH compliance, we operationalized on the surfaces or the attendants’ workflows, definitions for the systematic flow of patient which should be used to update the definitions. contacts allowed within a given HH opportunity To illustrate this, we present the number of HH op- and the patient zone. By a systematic flow, portunities and hand rubbing/washing compliance which we called a “delivery flow,”8 we referred to results for 4 different patient zone definitions the procedures or actions of interest that defined (Supplement 4). the start of a new HH opportunity, as well as the sequence of these procedures, which occurred without a break and were considered as 1 oppor- DISCUSSION tunity for HH.21 For example, in a given delivery We developed the HANDS at Birth tool to capture flow, a vaginal examination could be followed by the complex HH and glove behaviors of birth the delivery of the baby, but not by touching a attendants, based on state-of-the-art methods: a patient’s shoulder. During a delivery flow, a birth time-and-motion study using a computerized sys- attendant could undertake hand actions within tem (WOMBAT). This approach has been rarely the patient zone without the need for a new HH used to measure HH or to conduct research in opportunity to arise. low-resource settings.10,19,20,30,31 Our time-and- In this study, we defined patient zone as motion study enabled us to accomplish the follow- encompassing a woman’s perineal area and thighs, ing, which would have not been possible with the Global Health: Science and Practice 2020 | Volume 8 | Number 4 834 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org WHO HH Observation Form: (1) to look at wheth- serve very different purposes, with the former be- er birth attendants comply with the complete se- ing aimed at research and the latter at infection quence of behaviors prescribed by the WHO prevention practitioners. Another limitation of guidelines,32 (2) to look at each behavior individ- our tool, and how we used it, is that it requires ually, and (3) to look at different behavior data cleaning and data management. For example, sequences.8 Additionally, our method reduced even though misclassification was minimal, some Our method the risk of observer bias because data collection actions were recorded by mistake at the same reduced the risk of time. In addition, a couple of variables relied on was coded as a series of individual actions rather observer bias observer subjectivity—for example whether a de- than relying on observer judgment that a new HH because data livery happened very fast after the woman’sadmis- opportunity had occurred; hence, opportunities sion in the labor room. The structure of the data collection was were identified retrospectively in a standardized coded as 33 implies that data management is needed to create way. Indeed, hand rubbing/washing compliance HH opportunities and HH compliance results. individual actions was similar between observers in our study, as rather than 8 reported in Gon et al. Beyond HH, the HANDS at relying on Birth tool allowed investigation of other behavior CONCLUSION observer sequences and workflows. In conclusion, we report the process of developing judgment that a a research tool to capture the complexity of HH We are aware of 1 other study that used time- new HH and glove behavior during labor and delivery, in- and-motion methods to report HH of health care opportunity had workers in the context of an intensive care unit in cluding the tool elements, field implementation, occurred. the United States.31 That study’s aims differed tool performance, and implications for analysis. from ours including determining the number of We used a computerized system that was feasible contacts between patients and health care work- to use in low-resource facilities. Advantages of ers, as well as how long they take, and estimating this tool include simpler training, less observer HH compliance specifically before entering a room bias in assessing HH compliance (compared with and after exiting a room. That study did not detail the WHO HH Observation Form), and the ability information on the tool format or content. In com- to monitor multiple behaviors. The data it pro- parison, the HANDS at Birth tool allows for a more duced also showed good reliability and convergent exhaustive list of actions to be recorded, including validity. Future studies should explore the use of those beyond patient-attendant interactions; it this research tool in labor wards in other contexts, also allows looking at all HH opportunities, not as well as in other types of wards. just those related to exiting or entering the room. This tool has the potential to be adapted to ex- Acknowledgments: We thank the Ministry of Health of Zanzibar for their amine HH in other types of wards. We think this participation and engagement in the study. A special thanks to Rukaiya M Said, Mwanafatima Ali Mohammed, Bijuma Mkubwa Abdallah, and detailed examination of HH, including recontami- Asya Hati Vuai who collected all the data. We also thank Marina Daniele nation, is particularly important in wards facing for participating in the consultation exercise aimed at refining the unpredictable volumes of patients or unpredict- definition of hand hygiene opportunity. able patient complications. Examples include emergency departments, operating wards, or iso- Funding: The project was funded by the United Kingdom Research & Innovation Medical Research Council (MRC) Public Health Intervention lation wards during epidemics, such as the current Development scheme (award number MR/N015975/1). This award is isolation wards for COVID-19 patients. In particu- jointly funded by the MRC and the United Kingdom Department for lar during the COVID-19 pandemic, this tool could International Development (DFID) under the MRC/DFID Concordat agreement and is also part of the European & Developing Countries lend itself to examining the key relationship be- Clinical Trials Partnership program supported by the European Union. tween hands and surfaces and the fundamental is- The Soapbox Collaborative also contributed by funding staff involved in sue of pathogen cross-contamination between this project. them.34,35 Competing interests: None declared. Limitations Data Availability: The dataset generated during the current study is Because we were interested in individual determi- available at: https://doi.org/10.17037/DATA.00000778. nants of HH behavior, we observed only 1 birth at- tendant at any 1 time; whereas, the WHO HH REFERENCES Observation Form audit tool is designed to observe 1. Bedford J, Enria D, Giesecke J, et al.; WHO Strategic and Technical multiple health care workers simultaneously, Advisory Group for Infectious Hazards. COVID-19: towards con- trolling of a pandemic. Lancet. 2020;395(10229):1015–1018. which allows collection of more HH opportunities CrossRef. Medline in the same observation session. Importantly, the 2. Zaidi AKM, Huskins WC, Thaver D, Bhutta ZA, Abbas Z, Goldmann HANDS at Birth tool is not intended to substitute DA. Hospital-acquired neonatal infections in developing countries. for the WHO HH Observation Form; the 2 tools Lancet. 2005;365(9465):1175–1188. CrossRef. Medline Global Health: Science and Practice 2020 | Volume 8 | Number 4 835 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org 3. Gould IM. Alexander Gordon, puerperal sepsis, and modern theo- from South India: methods. Hum Resour Health. 2018;16(1):17. ries of infection control—Semmelweis in perspective. Lancet Infect CrossRef. Medline Dis. 2010;10(4):275–278. CrossRef. Medline 20. Manzi F, Schellenberg JA, Hutton G, et al. Human resources for 4. Allegranzi B, Nejad SB, Combescure C, et al. Burden of endemic health care delivery in Tanzania: a multifaceted problem. Hum health-care-associated infection in developing countries: systematic Resour Health. 2012;10(1):3. CrossRef. Medline review and meta-analysis. Lancet. 2011;377(9761):228–241. 21. World Health Organization (WHO). Hand Hygiene in Outpatient CrossRef. Medline and Home-Based Care and Long-Term Care Facilities. WHO; 2012. 5. Arai A, Tanabe M, Nakamura A, et al. Utility of electronic hand hy- Accessed September 8, 2020. http://apps.who.int/iris/bitstream/ ’ giene counting devices for measuring physicians hand hygiene ad- handle/10665/78060/9789241503372_eng.pdf herence applied to outpatient settings. Am J Infect Control. 2016; Pregnancy, Childbirth, 44(12):1481–1485. CrossRef. Medline 22. World Health Organization (WHO). Postpartum and Newborn Care: A Guide for Essential Practice. 3rd 6. Diller T, Kelly JW, Blackhurst D, Steed C, Boeker S, McElveen DC. ed. WHO; 2015. Accessed September 8, 2020. http://apps.who. Estimation of hand hygiene opportunities on an adult medical ward int/iris/bitstream/handle/10665/249580/9789241549356- using 24-hour camera surveillance: validation of the HOW2 eng.pdf Benchmark Study. Am J Infect Control. 2014;42(6):602–607. CrossRef. Medline 23. STROBE Initiative Group. STROBE checklist for cohort, case-control, and cross-sectional studies (combined). Published October 2007. 7. World Health Organization (WHO). Hand Hygiene Technical Accessed September 8, 2020. https://www.strobe-statement.org/ Reference Manual. WHO; 2009. Accessed September 8, 2020. index.php http://apps.who.int/iris/bitstream/handle/10665/44196/ 9789241598606_eng.pdf 24. Gon G, Virgo S, de Barra M, et al. Behavioural determinants of hand 8. Gon G, de Bruin M, de Barra M, et al. Hands washing, glove use, washing and glove recontamination before aseptic procedures at and avoiding recontamination before aseptic procedures at birth: a birth: a time-and-motion study and survey in Zanzibar labour wards. Int J Environ Res Public Health multicenter time-and-motion study conducted in Zanzibar. Am J . 2020;17(4):1438. CrossRef. Medline Infect Control. 2019;47(2):149–156. CrossRef. Medline 25. Zheng K, Guo MH, Hanauer DA. Using the time and motion method 9. Wilson J, Prieto J, Singleton J, O’Connor V, Lynam S, Loveday H. The to study clinical work processes and workflow: methodological J Am Med misuse and overuse of non-sterile gloves: application of an audit tool inconsistencies and a call for standardized research. Inform Assoc – to define the problem. J Infect Prev. 2015;16(1):24–31. CrossRef. . 2011;18(5):704 710. CrossRef. Medline Medline 26. Rowe AK, Lama M, Onikpo F, Deming MS. Design effects and intra- 10. Lopetegui M, Yen PY, Lai A, Jeffries J, Embi P, Payne P. Time motion class correlation coefficients from a health facility cluster survey in studies in healthcare: what are we talking about? J Biomed Inform. Benin. Int J Qual Health Care. 2002;14(6):521–523. CrossRef. 2014;49:292–299. CrossRef. Medline Medline 11. Blencowe H, Cousens S, Mullany LC, et al. Clean birth and postnatal 27. Lopetegui MA, Bai S, Yen P-Y, Lai A, Embi P, Payne PRO. Inter-ob- care practices to reduce neonatal deaths from sepsis and tetanus: a server reliability assessments in time motion studies: the foundation systematic review and Delphi estimation of mortality effect. BMC for meaningful clinical workflow analysis. AMIA Annu Symp Proc. Public Health. 2011;11(Suppl 3):S11. CrossRef. Medline 2013;2013:889–896. Medline 12. Shehu NY, Onyedibe K, Okolo M, et al. Assessment of hand hygiene 28. Kottner J, Audigé L, Brorson S, et al. Guidelines for Reporting compliance among health care workers in a Nigerian Tertiary Reliability and Agreement Studies (GRRAS) were proposed. JClin Hospital. Antimicrob Resist Infect Contr. 2017;6(Suppl 3):P101. Epidemiol. 2011;64(1):96–106. CrossRef. Medline Meeting abstracts from the International Conference on Prevention & 29. Gon G, Ali SM, Towriss C, et al. Unpacking the enabling factors for Infection Control (ICPIC 20170). CrossRef hand, cord and birth-surface hygiene in Zanzibar maternity units. 13. Spector JM, Agrawal P, Kodkany B, et al. Improving quality of care Health Policy Plan. 2017;32(8):1220–1228. CrossRef. Medline for maternal and newborn health: prospective pilot study of the WHO safe childbirth checklist program. PLoS One. 2012;7(5): 30. Tipping MD, Forth VE, Magill DB, Englert K, Williams MV. Systematic e35151. CrossRef. Medline review of time studies evaluating physicians in the hospital setting. JHospMed. 2010;5(6):353–359. CrossRef. Medline 14. Yawson AE, Hesse AAJ. Hand hygiene practices and resources in a teaching hospital in Ghana. J Infect Dev Ctries. 2013;7(04):338– 31. Khan BA, Hui KY, Hui SL, et al. Time-motion analysis of health care ’ ’ 347. CrossRef. Medline workers contact with patients and workers hand hygiene: open vs closed units. Am J Crit Care. 2011;20(3):e75–e79. CrossRef. 15. Danda G, Dube K, Dube P, Mudokwenyu-Rawdon C, Bedwell C. An Medline observational study of midwives’ practices to prevent peripartum sepsis in Zimbabwe. Afr J Midwifery Womens Health. 2015; 32. World Health Organization (WHO). WHO Guidelines on Hand 9(1):17–21. CrossRef Hygiene in Health Care: First Global Patient Safety Challenge Clean Care Is Safer Care 16. Simbar M, Ghafari F, Tork Zahrani S, Alavi Majd H. Assessment of . WHO; 2009. Accessed September 8, 2020. quality of midwifery care in labor and delivery wards of selected https://apps.who.int/iris/bitstream/handle/10665/44102/ Kordestan Medical Science University hospitals. Int J Health Care 9789241597906_eng.pdf Qual Assur. 2009;22(3):266–277. CrossRef. Medline 33. Muller MP. Measuring hand hygiene when it matters. Lancet Infect Dis – 17. Westbrook JI, Ampt A. Design, application and testing of the Work . 2016;16(12):1306 1307. CrossRef. Medline Observation Method by Activity Timing (WOMBAT) to measure clin- 34. Gon G, Dancer S, Dreibelbis R, Graham WJ, Kilpatrick C. Reducing ’ Int J Med Inform icians patterns of work and communication. . hand recontamination of healthcare workers during COVID-19. – 2009;78(Suppl 1):S25 S33. CrossRef. Medline Infect Control Hosp Epidemiol. 2020:1–2. CrossRef. Medline Work Observation Method By Activity Timing: A Guide 18. Westbrook J. 35. Otter J. Considering the role of environmental contamination in the to the Installation and Use of WOMBAT V2 . Centre for Health spread of COVID-19. Reflections on Infection Prevention and Control Systems and Safety Research, Macquarie University; 2016. blog. Published March 12, 2020. Accessed September 8, 2020. 19. Singh S, Upadhyaya S, Deshmukh P, et al. Time motion study using https://reflectionsipc.com/2020/03/12/considering-the-role-of- mixed methods to assess service delivery by frontline health workers environmental-contamination-in-the-spread-of-covid-19/ Global Health: Science and Practice 2020 | Volume 8 | Number 4 836 Time-and-Motion Methods to Monitor Compliance With Hand Hygiene www.ghspjournal.org Peer Reviewed Received: May 13, 2020; Accepted: August 19, 2020; First published online: October 15, 2020 Cite this article as: Gon G, Ali SM, Aunger R, et al. A practical guide to using time-and-motion methods to monitor compliance with hand hygiene guidelines: experience from Tanzanian labor wards. Glob Health Sci Pract. 2020;8(4):827-837. https://doi.org/10.9745/GHSP-D-20-00221 © Gon et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https://doi.org/ 10.9745/GHSP-D-20-00221 Global Health: Science and Practice 2020 | Volume 8 | Number 4 837 FIELD ACTION REPORT Implementation of a Pediatric Early Warning Score to Improve Communication and Nursing Empowerment in a Rural District Hospital in Rwanda Shela Sridhar,a Alexis Schmid,a Francois Biziyaremye,b Samantha Hodge,a Ngamika Patient,c Kim Wilsona Key Findings ABSTRACT n Nursing staff felt more empowered to Background: Pediatric early warning (PEW) scores represent a “track-and-trigger system” that identifies clinical deterioration in communicate clinical findings to the physician a patient’s condition in the hours preceding a sentinel event. team following the Pediatric Early Warning Before implementation, nurses reported feeling unprepared to Score for Resource-Limited Settings tool identify and advocate for acutely ill patients owing to a lack of training and implementation process. skills, vocabulary, and agency. We implemented a Pediatric n The process of implementing the tool triggered Early Warning Score for Resource-Limited Settings (PEWS-RL) more calls from nursing staff to the physician with nurses in a rural district hospital in Rwanda. Although teams to initiate early intervention. PEW scores can improve clinical outcomes, empowering nurses in resource-limited settings to discuss patient acuity with physi- Key Implications cians is a critical first step. Our primary aims were to train nurses to obtain more accurate vital signs and assess their im- n The Pediatric Early Warning Score for portance as early warning signs of clinical deterioration and Resource-Limited Settings tool has the usePEWscorestoimprovecommunicationbetweennursesand potential to improve competency and physicians. confidence of nurses in their triage capabilities. Implementation: The PEWS-RL tool implementation began with a Although traditionally implemented in tertiary training program that was created through discussions with care centers, program managers should nurses, physicians, and the medical director of the hospital. The consider implementing this tool at the district program included lectures and application of learned skills hospital (secondary) level as well. through direct clinical mentorship of nurses, as well as training n Physicians and nurses both play crucial roles in of physicians regarding PEWS-RL as a communication tool. triaging systems. Therefore, program Evaluation: The PEWS-RL protocol was evaluated based on pre- and post-tests to assess improvement in nurses’ knowledge and managers should consider engaging both skill, as well as skills assessments of accurate recognition of clini- groups with the tool before and during cal deterioration. All 6 nurses passed skill testing with >80% ac- implementation. curacy. Nurses’ feelings of empowerment to advocate for patients and to escalate care were assessed through pre- and post- training interviews. Nurses described increased confidence in calling for physician support. Discussion: Implementation of PEW scores increased nurses’ tech- nical skills and feelings of confidence and empowerment; howev- er, the low-resource setting presented major challenges. Barriers to sustainable implementation include the rapid ward staff turn- over as well as limited physician buy-in. Nevertheless, the PEWS- RL tool has the potential to empower nurses and improve patient outcomes if fully embraced by staff. BACKGROUND a Responding to Increased Child Mortality Rate Boston Children’s Hospital, Boston, MA, USA. b Inshuti Mu Buzima-Partners in Health Rwanda, Kigali, Rwanda. irehe District Hospital (KDH) is a public hospital in c K Kirehe District Hospital, Kirehe, Rwanda. rural Rwanda, supported by a partnership with Correspondence to Shela Sridhar ([email protected]). the nongovernmental organization (NGO) Inshuti Mu Global Health: Science and Practice 2020 | Volume 8 | Number 4 838 Implementation of a Pediatric Early Warning Score in Rural Rwanda www.ghspjournal.org Buzima, a local organization of Partners in Health. with the 6 pediatric nurses and 4 physicians. After KDH serves a catchment area of approximately observing multiple deaths on the pediatric ward, 340,000 people; or 13% of Rwanda’s population, in- the Partners in Health-affiliated pediatrician hy- cluding a large refugee settlement with 57,000 resi- pothesized that a driver of the mortality rate was dents.1 About 50% of the population in Kirehe is inadequate communication between nurses and under the age of 17. KDH has a busy general pediatric physicians. This was observed at 2 separate Partners ward, with 60–120 pediatric admissions per month in Health supported hospitals. However, because of children aged 1 month to 15 years old. The staff- KDH was the busier facility at the time, it was chosen ing model for the pediatric ward includes 1 or for the intervention. 2 nurses caring for 10 to 30 pediatric patients, sup- We conducted informal interviews with 4 phy- ported by a general physician covering the pediatric sicians to gain a background understanding of ward as well as a 40-bed neonatal ward. The total physicians’ perceptions regarding nursing compe- medical staff comprised 6 pediatric nurses and tencies. The 4 physicians interviewed were those 10 physicians. In addition, a U.S.-trained pediatri- who spent the greatest amount of time on the pe- cian affiliated with Partners in Health works with diatric ward. We asked the 6 pediatric nurses ques- the hospital to conduct on-site clinical capacity tions regarding their comfort level with triaging building for several months each year. and communication processes. The interviews In 2018, hospital staff noted rising mortality also included open-ended questions to draw out rates in the pediatric ward. A chart review be- additional themes. We asked the interview ques- tween May and October 2018 indicated that for tions in English and used an interpreter who spoke some months the child mortality rate was as high Kinyarwanda and English to translate. Interviews as 6% and the average for the 6 months about 3%. with nurses and physicians lasted no more than The majority of deaths were associated with sepsis 30 minutes. Common themes that emerged for or pneumonia resulting in respiratory failure, of- both nurses and physicians included constraints ten as a consequence of inadequate recognition of on time and human resources, which compro- mised the clinicians’ ability to appropriately prior- altered mental status and respiratory fatigue. Death itize patients and complete tasks. Nurses also cited In interviews, occurred an average of 7 days after admission, with gaps in knowledge and skills in identifying and nurses cited that a a minimum of <24 hours and maximum of 27 days subsequently reporting the status of critically ill barrier to timely after admission. Potential etiologies for death after patients as creating a barrier to timely care. care were gaps in 7 days of admission may have been iatrogenic Nursing leadership highlighted that nurses felt dis- knowledge and (management of patients), but we suspected there empowered to advocate for deteriorating patients. skills in identifying was also poor recognition of warning symptoms They reported lacking a common language around and reporting the and clinical progression of disease that may have assessment of critical illness with physicians and status of critically been missed in the days leading to mortality later therefore feeling unprepared to highlight the acu- ill patients. in the hospital course. ity of a patient’s condition and effectively advocate A variety of factors can contribute to clinical for them. Other nurses reported that their con- deterioration or high rates of mortality in children cerns were sometimes dismissed by physicians with sepsis or pneumonia. However, the incidence who would respond by saying: of such events can be significantly lower in similar Pediatric vitals are different. settings where early recognition of deterioration and prompt initiation of treatment or early trans- General physicians cited concerns with the ac- fer to higher levels of care have been initiated.2 curacy of vital signs reported by nurses as a key More than 95% of pneumonia-related deaths barrier in assessing the severity of pediatric illness. occur in low- and middle-income countries. Multiple physicians indicated that: Pediatric early warning scores However, little data exists on quality-of-care indica- If I want to believe the vitals, I take them myself. tors and practices around pneumonia care in these represent a triage 3 contexts. This paucity of data suggests a gap in as- Using Pediatric Early Warning Score for “track-and- sessment of quality of care in pediatric populations. Triage trigger system” Although time constraints and human resource that can Understanding Hospital Care Delivery allocation are subject to financial constraints, clin- accurately identify Factors ical processes such as accurate triage and commu- up to 85% of To better understand hospital-based care delivery nication are modifiable factors with minimal children who will factors that could be contributing to high pediatric financial burdens on low-resource hospital systems. experience clinical mortality, we conducted key informant interviews Pediatric early warning (PEW) scores represent a deterioration. Global Health: Science and Practice 2020 | Volume 8 | Number 4 839 Implementation of a Pediatric Early Warning Score in Rural Rwanda www.ghspjournal.org triage “track-and-trigger system” that can accurately mechanism that indicate clinical concern.5,8 Few identify up to 85% of children who will experience versions have been evaluated in resource-limited clinical deterioration, such as cardiac arrest or severe settings5,7 where staffing ratios and the level of respiratory compromise, sometimes as early as nurses’ training differ.5 After an analysis of PEW 11 hours before the sentinel event.4 The scores are tools and initial conversations with staff, we deter- a mechanism that can be used to modify triage sys- mined that it would be best to focus on objective tems and standardize communication regarding data (basic vital signs alone) without including acutely ill children. PEW scores have primarily clinical assessment. Given that the PEWS-RL met been used in high-income countries, but they were these criteria and had previously been validated recently adapted for use in resource-limited settings in Rwanda in a tertiary hospital setting, we decid- (Pediatric Early Warning Score in Resource-Limited ed to implement the same version in our hospital Settings [PEWS-RL]) and validated in a tertiary care for consistency and potential nationwide scalabili- setting in Rwanda.5 The PEWS-RL uses basic clinical ty in the future. assessments including respiratory rate, respiratory The PEWS-RL tool (Figure) is purposefully distress, heart rate, temperature, blood pressure, ox- composed solely of vital signs that are attainable ygen use, and mental status. It demonstrated a with minimal equipment or assessment ability. 92% sensitivity and an 87% specificity in identifying This approach was taken because clinical assess- children at risk of clinical deterioration.6 ments, such as blood pressure and respiratory ef- To improve early recognition and communica- fort, are often not examined due to a lack of tion of clinical deterioration in pediatric patients trained personnel and availability of pediatric- by nursing staff, we aimed to implement a stan- sized equipment.5 This tool was utilized across dardized triage system including a standardized the pediatric age range (1 month to 15 years). clinical assessment for patients at risk for clinical Our PEW score included respiratory rate, heart deterioration in our inpatient pediatric ward. We rate, temperature, and mental status; each was hypothesized that this intervention would im- scored at 1 point. Physician notification was trig- prove nurses’ ability to accurately identify critical- gered at a score of 3 on admission or an increase ly ill patients, improve communication about of 3 points on subsequent assessments. Blood critical patients by creating a common language pressure was initially included in the assessment with physicians, and prompt a timely physician re- of the PEWS-RL; however, based on discussions sponse to evaluate and initiate the appropriate with the research team at the University Teaching medical management for a child whose condition Hospital of Kigali, the sensitivity and specificity of is deteriorating. In this report, we describe our the tool did not notably change when blood pres- process for implementation of the PEWS-RL at sure was removed. Therefore, we did not include it the district hospital level, including areas of suc- in our score. cess, challenges, and lessons learned. Training Approach IMPLEMENTING PEW SCORES The initial implementation of PEW scores started We reviewed several versions of the PEW triage with a training program for 6 nurses and 10 physi- tools collaboratively with staff and leadership at cians over a 2-week period in November 2018. A KDH, including the medical director, clinical di- visiting U.S.-based pediatric nurse specialist pro- rector, and the primary general practitioner who vided 1–2 hours of on-site didactic training per PEW systems rounded on the pediatric wards. PEW systems in- day to the pediatric ward nursing staff, focusing include clude 2 components: a score calculated using vital on the clinical importance and implications of ab- 2components:a signs at prescribed intervals during a child’s hospi- normal vital sign values. Although many (but not score based on a talization and a response system, which may be as all) physicians and nurses are exposed to emer- child’s vital signs simple as contacting a physician, that is activated if gency triage and assessment training in school, and a response a specific score threshold on the tool has been ongoing mentorship or recertification is uncom- system that is reached.7,8 Early warning scores commonly eval- mon. The remainder of the day focused on applica- activated if a uate and score vital signs as well as clinical exam tion of learned skills and direct clinical mentorship specific score is assessments, such as level of consciousness, capil- through “real-time” patient assessments and active 8 reached. lary refill, or work of breathing. No general con- feedback at the bedside. Nurse training included sensus exists regarding which components are bedside mentoring during morning rounds and di- essential, the frequency with which they should dactic sessions each afternoon with continued bed- be recorded,5 or the thresholds and scoring side teaching through the day. The training was Global Health: Science and Practice 2020 | Volume 8 | Number 4 840 Implementation of a Pediatric Early Warning Score in Rural Rwanda www.ghspjournal.org FIGURE. Pediatric Early Warning Score for Resource-Limited Settings Tool Used at the University Teaching Hospital of Kigali, Rwanda, and Kirehe District Hospital Abbreviation: BP, blood pressure. Global Health: Science and Practice 2020 | Volume 8 | Number 4 841 Implementation of a Pediatric Early Warning Score in Rural Rwanda www.ghspjournal.org incorporated into the nursing work flow to the ex- clinical skills assessments. Skill competency of tent possible to be minimally disruptive in an al- KDH pediatric nursing staff was evaluated by the ready understaffed environment. pediatric nurse specialist using a standardized Along with clinical and didactic training, checklist immediately after the 2-week training nurses were provided stethoscopes so they could program. The objective exam focused on their manually check heart rates and blood pressures ability to obtain manual vital signs and calculate a as a secondary validation of the cardiorespiratory PEW score. The clinical competency form used monitor used on the unit. They were also provided during the assessment is outlined in the individual pulse oximeters to be used as a second- Supplement. A numeric score as well as written ary check on the existing monitor. Equipment was feedback was provided. All 6 pediatric nurses in- distributed at the start of the training and used dependently passed skill testing with >80% accu- throughout the 2-week course. racy. In addition to the clinical skills assessment, a General physicians were given 2 lectures dedi- written exam was given on the first and last day of cated to understanding the PEW score and re- training to assess the clinical knowledge necessary sponse system by the NGO pediatrician. Lectures to adequately utilize the PEWS-RL as intended for for nurses focused on assessment and reporting of screening and response activation. The average the score, and lectures for physicians focused on pretest score for nurses was 66% with a range responses to different scores and critical thinking of 53%–80%. The average post-test score was around common case scenarios. In addition, the 81% with a range of 67%–100%. pediatrician rounded with the clinical team each morning for the 3 months following the training Nursing Communication with Physicians program and provided ongoing low-dose, high- We conducted a qualitative assessment of the im- frequency mentorship for both rounding physi- pact of our training on nursing communication cians and nurses in using and interpreting the with physicians, focusing on the nurses’ level of PEWS-RL. PEW scores for patients were reviewed empowerment in patient advocacy and escalation each morning to assess for completion and to facil- of care around sentinel events. Pre- and post- itate discussion of any challenges encountered training structured interviews of the nursing staff during the implementation. Through this process, were conducted by an NGO nurse mentor individ- PEW score documentation was integrated into ually and confidentially and in their primary lan- ward rounds and the existing work flow for both guage to promote more open communication. In nurses and physicians. addition, nurses completed a written survey of their communication practices and comfort level Assessment of Outcomes in escalating care before and after completing the Nursing and Physician Knowledge, Skills, and 2-week training program. The survey included Clinical Practices Likert-scale questions with declarative statements Our evaluation of We conducted an evaluation of the impact of such as “I feel comfortable asking physicians ques- the impact of PEWS-RL implementation at KDH on nursing tions” as well as free-form answers to questions PEWS-RL and physician knowledge, skills, and clinical prac- around communication such as “How do you feel implementation tices using interview data and process measures. communication between nurses and physicians on nursing and Our measures included changes in nursing knowl- affects patient care?” Before training, 2 of 6 nurses physician edge and skill in accurate recognition of clinical felt as though their clinical assessments were often knowledge, skills, deterioration, changes in nursing physician com- dismissed by physicians. Post-training, nurses stat- and clinical munication before sentinel events, and nursing ed feeling more confident in their ability to advo- practices was confidence levels in the communication of clinical cate for patients. One nurse articulated: findings to physician staff. The PEWS-RL triage based on Now I have a tool to back me up when I call the doctor. interview data system includes both the risk score obtained by and process the nursing staff and the responsiveness of the Nurses described increased confidence calling measures. physician team. Our assessments focused on the for physician support: primary objective of nursing competency and [We] have something to say. communication rather than the response compo- nent of triage systems. Changes in nursing clinical communication Nursing knowledge and skills in accurate rec- practices were also evaluated by retrospective chart ognition of clinical deterioration was evaluated review. In the 6 months before the implementation using pre- and post-training written tests and of our PEWS protocol, only 8 of 30 patients (27%) Global Health: Science and Practice 2020 | Volume 8 | Number 4 842 Implementation of a Pediatric Early Warning Score in Rural Rwanda www.ghspjournal.org who had been transferred or died had a recorded that the physicians were less interested than the call to a physician in the 11 hours preceding the nursing staff in using the PEWS-RL. The reasons for sentinel event (transfer to a tertiary care center, re- this finding are likely multifactorial. The primary fo- suscitation by KDH staff, or death). However, in the cus of our efforts in training was toward the nursing 2 months immediately following the intervention, staff, and we included only 2 training sessions and we found that the physician was called 63% of the no formal mentorship process for physicians who time (7/11 patients) before a sentinel event. In the did not round on the pediatric ward during the im- 6 months before the intervention, there was a plementation phase. Additionally, of the 16 physi- 3% mortality rate and a 3% transfer rate for all cians on staff at the time of the intervention, only patients in the hospital. In the 2 months after our 4wereabletospendtimeonthepediatricwards intervention, the mortality rate was not significant- during the initial implementation of the protocol. ly different, but the transfer rate had increased by Finally, as our focus was on nursing empowerment, 11%. A chart review also demonstrated an increase we failed to involve the physicians in the planning in physician documentation outside traditional process of protocol development and implementa- rounding hours, suggesting an increase in physi- tion, likely leading to inadequate physician under- cian response frequency; however, physician re- standing and involvement in the protocol. The critical lesson sponse times will need to be evaluated over time The critical lesson learned was the importance learned was the in subsequent iterations. Although the data collect- of engaging physicians and nurses together. We importance of ed from this training program are not adequately were able to implement the first steps in a triage engaging 8 powered to make definitive conclusions regarding system, that is, nursing recognition and empower- physicians and the effect of PEW scores on clinical outcomes or ment to communicate clinical findings. However, nurses together. physician response times, the initial inferences are changes to outcomes and mortality will require promising and merit further investigation. true physician engagement and understanding of the PEW score tool, its implications, and how to re- spond to nursing concerns. As noted in the PEW DISCUSSION score literature, to be effective in reducing morbid- Implementation of our PEW score created an ity and mortality, the tool needs to be implemented opportunity for vital nursing education on high- within a system that is able to respond to the needs quality assessment of vital signs and a deeper of the child; specifically, a provider or a team that clinical understanding of underlying pediatric has the ability to not only accurately assess the pa- physiology. The PEWS-RL implementation also tient and recognize anomalies, but to also imple- The PEWS-RL empowered nurses, provided them with a model, ment appropriate clinical interventions.7,9,10 Our implementation and mentored them on the tools to communicate PEWS intervention focused on addressing the first empowered their assessments to physicians. The public hospi- steps of recognition and empowerment. However, nurses, provided tal-NGO partnership provided the opportunity we did not address the subsequent step, which them a model, for nurses and physicians to observe communica- requires “the assistance is readily available and ap- and mentored tion between an external nurse and physician, propriately skilled...”8 Physician engagement can them on the tools to which modeled important skills in open multidis- be further accomplished by ensuring leadership communicate ciplinary communication and physician trust in and ownership of the tool implementation by the their assessments nursing assessment skills. Given the immediate physicians. Additionally, appointing a physician to physicians. increase we saw in recorded calls to physicians leader in pediatrics may be useful to create physi- being documented and postintervention inter- cian buy-in and organizational accountability. views with nursing staff, the PEWS-RL tool may Although we attempted to educate physicians be effective even on a busy ward with limited and nurses side-by-side during medical rounds, staff and resources. we should have placed a greater focus on individual-physician coaching to address the as- Challenges and Lessons Learned pect of skilled assistance in response to recognition Although nursing knowledge and skills demonstra- of illness. Furthermore, before implementation, bly improved and nursing staff reported feeling we did not adjust the physician schedule to ensure empowered, we encountered several challenges that all medical staff rotated through the ward during the implementation of our protocol. Our during the training. This decision was made to most significant challenge was in motivating physi- minimize disruptions to work flow in the hospital cian engagement. We provided lectures for physi- as well as minimize administrative burden on the cians over the course of the training, but our clinical director who managed the schedules. This interactions and reports from nursing staff suggested reasoning also informed why we did not have Global Health: Science and Practice 2020 | Volume 8 | Number 4 843 Implementation of a Pediatric Early Warning Score in Rural Rwanda www.ghspjournal.org additional meetings during the training with the nursing empowerment in escalating concerns for entire staff to review and address problems that timely physician response. Training and implemen- were noted, but rather managed them on an ad tation of the PEWS-RL resulted in demonstrable hoc, daily basis. However, in subsequent pro- improvements of both technical skills and feelings grams, the short-term disruption may be accept- of confidence and empowerment among the nurs- able if longer-term clinical benefits can be derived. ing staff. Challenges and next steps in quality im- Other barriers to sustainable implementation provement and implementation remain, including of this protocol included the rapid turnover of staff the need to address equipment availability and se- on the ward and a loss of equipment following the curity and the implementation of approaches to training. The chief of nursing was transferred dur- improving physician training and buy-in. ing our training and 1 week later another member However, it remains to be seen if the subse- of the nursing staff left and was replaced by a new quent steps of the track-and-trigger system can be staffer. Next steps to mitigate this limitation in- improved with increased physician involvement in clude creating an on-boarding system for pediatric the implementation process. This next step is cru- nurses as well as new physician staff. Additionally, cial given that our triage system would ultimately to minimize the removal of stethoscopes and oxi- be incomplete without an appropriate response meters from the ward, they should be tagged with and intervention system.8,11 The next steps are to large, bulky labels or affixed to a mobile cart, or design an adequately powered study across multi- the work flow should be changed to include re- ple district hospitals to evaluate the feasibility and trieval of locked equipment at the start of each effect of PEW scores in low-resource rural settings. nursing shift. Although we are encouraged by the These studies should start by focusing on delivery – feedback from our nursing staff, another limita- of care at the hospital level including physician tion was our status as an outside organization. nursing communication, response times, and appro- Survey answers may have been biased due to cul- priateness of medical management to PEW triggers. tural tendencies to avoid criticizing the system or a Based on our implementation experience, it will be desire to provide positive answers. essential to conduct intensive nursing and physician training simultaneously with a dedicated review pro- cess. Nevertheless, based on our initial assessment, Limitations the implementation of PEW scores in a rural district In addition to the procedural challenges we experi- hospital in sub-Saharan Africa has the potential to enced, important limitations in the assessment empower nurses and improve patient outcomes in and interpretation of the outcomes should be con- low-resource settings if fully embraced by staff. sidered. Our assessment focused on nursing com- petencies, with minimal evaluation of physician Acknowledgments: Thank you to Catherine Kirk and Inshuti Mu Buzima responses. This approach prevented us from in Rwanda for their continued support and assistance with our training measuring clinical outcomes and timeliness of program. We thank the pediatric staff at Kirehe District Hospital for their willingness to learn and take on additional work despite their responses, which are critical to any effective triage overwhelming existing patient care responsibilities. Thank you to the system. Additionally, the program was conducted Global Health Department at Boston Children’s Hospital for financial and technical support throughout the process. at one hospital with a small number of nurses. To achieve statistical significance for nursing compe- Competing interests: None declared. tencies, this program would need to be conducted across multiple sites or over multiple iterations at REFERENCES KDH. Finally, although the PEW score has been 1. City Population. Kirehe (District, Rwanda)—Population Statistics, validated in tertiary care centers, it has not yet Charts, Map and Location. 2019. Accessed September 14, 2020. been validated in district-level hospitals where few- https://www.citypopulation.de/php/rwanda-sector-admin.php? er interventions are available to inform the medical adm1id=55 response. 2. Agulnik A, Mora Robles LN, Forbes PW, et al. Improved outcomes after successful implementation of a pediatric early warning system (PEWS) in a resource-limited pediatric oncology hospital. Cancer. SUMMARY AND CONCLUSIONS 2017;123(15):2965–2974. CrossRef. Medline Early recognition and response to clinical deterio- 3. Lazzerini M, Seward N, Lufesi N, et al. Mortality and its risk factors in ration are essential to improving outcomes in Malawian children admitted to hospital with clinical pneumonia, 2001–12: a retrospective observational study. Lancet Glob Health. pediatric care in hospitalized patients in resource- 2016;4(1):e57–e68. CrossRef. Medline limited settings but can be challenging. Initial 4. Leonard MM, Kyriacos U. Student nurses’ recognition of early signs barriers can include limitations in nursing knowl- of abnormal vital sign recordings. Nurse Educ Today. 2015;35(9): edge and skills in pediatric triage and lack of e11–e18. CrossRef. Medline Global Health: Science and Practice 2020 | Volume 8 | Number 4 844 Implementation of a Pediatric Early Warning Score in Rural Rwanda www.ghspjournal.org 5. Rosman SL, Karangwa V, Law M, Monuteaux MC, Briscoe CD, 9. Parshuram CS, Dryden-Palmer K, Farrell C, et al; Canadian Critical McCall N. Provisional validation of a pediatric early warning score Care Trials Group and the EPOCH Investigators. Effect of a pediatric for resource-limited settings. Pediatrics. 2019;143(5):e20183657. early warning system on all-cause mortality in hospitalized pediatric CrossRef. Medline patients. JAMA. 2018;319(10):1002–1012. CrossRef. Medline 6. Akre M, Finkelstein M, Erickson M, Liu M, Vanderbilt L, Billman G. 10. Sefton G, McGrath C, Tume L, Lane S, Lisboa PJG, Carrol ED. What Sensitivity of the pediatric early warning score to identify patient de- impact did a Paediatric Early Warning system have on emergency terioration. Pediatrics. 2010;125(4):e763–e769. CrossRef. Medline admissions to the paediatric intensive care unit? An observational 7. Brown SR, Martinez Garcia D, Agulnik A. Scoping review of pediat- cohort study. Intensive Crit Care Nurs. 2015;31(2):91–99. CrossRef. ric early warning systems (PEWS) in resource-limited and humani- Medline Front Pediatr tarian settings. . 2019;6:410. CrossRef. Medline 11. Kyriacos U, Jelsma J, James M, Jordan S. Monitoring vital signs: de- 8. Chapman SM, Wray J, Oulton K, Peters MJ. Systematic review of velopment of a modified early warning scoring (MEWS) system for paediatric track and trigger systems for hospitalised children. general wards in a developing country. PLoS One. 2014;9(1): Resuscitation. 2016;109:87–109. CrossRef. Medline e87073. CrossRef. Medline Peer Reviewed Received: February 13, 2020; Accepted: August 21, 2020; First published online: October 15, 2020 Cite this article as: Sridhar S, Schmid A, Biziyaremye F, et al. Implementation of a pediatric early warning score to improve communication and nursing empowerment in a rural district hospital in Rwanda. Glob Health Sci Pract. 2020;8(4):838-845. https://doi.org/10.9745/GHSP-D-20-00075 © Sridhar et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00075 Global Health: Science and Practice 2020 | Volume 8 | Number 4 845 PROGRAM CASE STUDY Juntos: A Support Program for Families Impacted by Congenital Zika Syndrome in Brazil Antony Duttine,a Tracey Smythe,a Miriam Ribeiro Calheiros de Sa,b Silvia Ferrite,c Maria Elisabeth Moreira,b Hannah Kupera Key Messages Resumo em português no final do artigo. n A community-based family group program for caregivers of children with congenital Zika syndrome ABSTRACT (CZS) was developed based on an existing program Background: The 2015–2016 Zika virus outbreak in Brazil was for families of children with cerebral palsy and pilot unprecedented and resulted in the birth of more than 3,000 chil- tested in Brazil. dren with congenital Zika syndrome (CZS). These children expe- rience multiple complex health conditions and have limited n Program managers developing group initiatives should services to support them and their family’s needs. consider fast-track learning approaches to adjust their Program Development and Piloting: An existing family support intervention to make it more useful to participants. program for children with cerebral palsy (Getting to Know n Clinicians and managers of Zika initiatives should Cerebral Palsy) was adapted to the Zika context in Brazil through consider that caregivers of children with CZS will expert consultation. The program was pilot tested at 2 sites among 6 groups of caregivers (total of 48 families) from August likely benefit from the ability to engage and 2017 to June 2018. Group observation and focus group discus- exchange with caregivers of children with other sions with facilitators and participants at the end of each session neurodevelopmental disabilities. informed fast-track learning, which was used to tailor the pro- gram for future groups. Fast-track learning—adjusting the inter- n Program managers delivering community interventions vention in real time based on gathered feedback—was found to targeting caregivers should consider engaging expert be a helpful process to inform and hone the program from its ini- mothers as group cofacilitators. tial concept. n Integrating emotional support activities into groups Program Description: The intervention, Juntos, is a facilitated that address child development is important, and participatory group program for caregivers of children who adds value. have CZS. The group sessions are cofacilitated by a parent of a child who has CZS and an allied health professional. The group meets for 10 sessions that last 4 hours. Each session includes an icebreaker, activities, and group discussions. Content covers practical information on caring for a child with a developmental disability including that caused by Zika. Psychosocial support forms an important component, and families are guided from the first week to define and develop their own communities of support. Six pilot groups were successfully run in Rio de Janeiro and Greater Salvador, Bahia. The groups gave positive feedback on acceptability and demand. Conclusions: The program has the potential to be an important tool for community health and social support services in South America in response to Zika. The program can also be applied to children with neurodevelopmental disabilities other than those caused by the Zika virus, which could be important in ensuring families of children with CZS are less isolated. a International Centre for Evidence in Disability, London School of Hygiene & BACKGROUND Tropical Medicine, London, United Kingdom. he Zika outbreak of 2015–2016 in South America b Instituto Nacional de Saúde da Mulher, da Criança e do Adolescente Tcaught the international health community un- Fernandes Figueira, Fiocruz, Rio de Janeiro, Brazil. aware. There had previously been no severe health con- c Department of Speech and Hearing Sciences, Federal University of Bahia, Salvador, Brazil. sequences associated with the virus, despite Zika having 1,2 Correspondence to Antony Duttine ([email protected]). been known since the 1940s. Zika has now been Global Health: Science and Practice 2020 | Volume 8 | Number 4 846 Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org proven to cause developmental impairments in needs of caregivers of children with CZS and cere- children3,4 collectively known as congenital Zika bral palsy in middle-income contexts; (3) meeting syndrome (CZS).5 This syndrome includes micro- with caregivers, specialists, and other local stake- cephaly as the most pronounced and documented holders in Brazil to identify key gaps, challenges, symptom, which is linked with severe and multiple and needs; and (4) reviewing emerging data from Families of impairments. Evidence is emerging that Zika also a sister study measuring the social and economic children with CZS causes an array of other cognitive and physical impact of CZS on caregivers. A full description of lacked adequate impairments that may not be immediately appar- the needs analysis is available.8 access to ent at birth. Microcephaly is likely to be the tip of We found that providing some services for specialized health the iceberg in terms of affected children, as more children with complex multiple impairments at and rehabilitative mild or moderate impairments stemming from in the community level could be crucial to address services, informal utero Zika infection appear to be far more fre- the unmet needs experienced by families of chil- support groups, quent.6 Brazil was the most affected country in the dren with CZS in Brazil and may be more afford- and formalized outbreak. As of March 2020, Brazil had 3,559 con- able than centralized services (which may be caregiver support. firmed cases of CZS with an additional 2,871 cases difficult or costly to access). Families of children under investigation (total 6,430 cases).7 with CZS, particularly those children with more se- Although CZS and cerebral palsy are separate vere impairments, did not have enough access to conditions, because they have similarities, pro- specialized health and rehabilitative services and grams designed for caregivers of children with ce- informal support groups, and formalized support for caregivers was also limited. There was some rebral palsy could provide a strong foundation to Because CZS and 13 concern raised by clinicians that children with adapt a program for the Zika context in Brazil. cerebral palsy mild to moderate impairments stemming from One such program, Getting to Know Cerebral have similarities, Palsy (GTKCP), was developed by the London Zika infection were less likely to attend rehabilita- programs School of Hygiene & Tropical Medicine (LSHTM) tion and that these caregivers were an important designed for after a childhood disability survey showed that group to be targeted. Other researchers have also caregivers of caregivers of children with cerebral palsy in reported on the additional services required to fully children with Bangladesh had very little access to information address the care needs of children with CZS and 9–12 cerebral palsy or support regarding the best way to care for their their families. could provide a child and that available services were extremely Given the results of the needs analysis that limited.14 GTKCP is a 10-session parent-support identified the unmet support needs of parents in foundation to program held in the community that aims to im- Brazil and the positive reception of the principle adapt a program prove parents’ knowledge and skills in caring for of GTKCP for Brazil among local stakeholders, for the Zika researchers at the LSHTM who had been involved their child and improve the quality of life of par- context. in GTKCP and EIP felt that adapting GTKCP and ents and children with developmental disabilities. EIP for the Zika context and Brazilian culture It is hard to estimate the exact reach of the pro- could be potentially useful. Partnership for the gram, but an online community of practice estab- project was established between the LSHTM and lished in 2014 to support the rollout of GTKCP has 2 Brazilian institutions: the Instituto Nacional de 412 members across 72 countries who share knowl- Saúde da Mulher, da Criança e do Adolescente edge and experiences.15,16 GTKCP focuses on par- Fernandes Figueira (IFF) in Rio de Janeiro, and ents of children aged 2 years and older; a new the Universidade Federal da Bahia (UFBA) in version, the Early Intervention Program (EIP), was Salvador. developed for parents of children aged younger This article describes the process of developing than 2 years.17 Program material is available from and piloting the intervention in Brazil, as well as www.ubuntu-hub.org. the final program that was developed (Figure 1). We also reflect on lessons learned as key recom- mendations from this innovative program may be Needs Analysis useful for other global health practitioners design- From April to August 2017, we conducted a needs ing community-based family group interventions. analysis to assess the potential value of a community-based program, based on GTKCP, for caregivers of children with CZS in Brazil. The PROGRAM ADAPTATION AND needs assessment involved: (1) tracking and com- DEVELOPMENT paring emerging literature on the clinical presen- After conducting the needs analysis, we developed tation of CZS with existing literature on cerebral and adapted the program through expert consul- palsy; (2) conducting a literature review on the tation, and then piloted the intervention using a Global Health: Science and Practice 2020 | Volume 8 | Number 4 847 Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org FIGURE 1. Timeline of Juntos Program Development for Caregivers of Children with Congenital Zika Syndrome, Brazil The theory of multiphase approach. A protocol was established the theory of change linked outcomes with activi- change linked by the lead project researcher (AD) to measure ties to explain how and why the desired change 13 outcomes with feasibility of the pilot intervention. was expected to occur and was useful in providing activities to a more comprehensive understanding of steps to explain how and Ethics Approval improve services to be more inclusive and support- why the desired Ethical approval was obtained from the Instituto ive of family and community. Throughout the pro- change was de Saúde Coletiva/UFBA Ethics Ref 2.369.348, gram development process, the theory of change expected to occur. IFF/FIOCRUZ RJ/MS Ethics Ref2.183.547, and was refined to reflect ongoing understanding and LSHTM Ethics Ref 13608. Informed consent was research findings (Figure 2). acquired from all participants. Several areas of adaptation were identified through the emerging literature, clinical experi- ences of managing children with CZS, develop- Initial Adaptation of Program ment of the theory of change, and by the GTKCP/ To support the adaptation, advisory groups were EIP teams. These areas included recommenda- established in Brazil and in the United Kingdom tions to further strengthen and develop specific and included a range of specialists, as well as approaches to recognize and address caregivers’ mothers of children with CZS. psychosocial needs and other clinical issues in The GTKCP and EIP curricula were reviewed children with CZS that were not covered within by the lead project researcher (AD) with other GTKCP or EIP (e.g., irritability; challenges with LSHTM colleagues (TS, HK), Brazilian colleagues breastfeeding or weaning; management of gastro- (SF, MS), the GTKCP and EIP teams, and other stomy including feeding, low vision, or blindness). key identified experts (including specialists). The The EIP groups are cofacilitated by an expert project lead is a physiotherapist with 15 years of mother who has experience caring for a child programmatic experience, including in qualitative with cerebral palsy and a rehabilitation profes- and participatory research and community-based sional (e.g., physiotherapist, occupational thera- pist, or speech and language therapist) who is rehabilitation in low- and middle-income coun- experienced in working with children who have tries. During a May 2017 workshop in London, developmental disabilities. This approach had not the experts convened to discuss the preliminary been used in GTKCP. A decision was made to pilot findings of the needs assessment and to develop test group facilitation by an expert mother com- consensus on a first draft outline of the program, bined with a therapist and assess whether this an initial timeline, constituency of the facilitators would be effective. to lead the caregiver group sessions, and partici- There is a wide range in type and severity of pant inclusion criteria. symptoms among children affected by Zika. The project group developed a theory of change It was agreed that program inclusion criteria to describe how the program relates to broader so- would be: cietal participation of children with developmental delays, including CZS, and the pathways that deter- Caregivers of children who have confirmed or mine the extent to which this intervention may be suspected CZS but not other types of neurode- successful. The theory of change describes what velopmental disabilities changes are needed and the assumptions underly- Caregivers of children residing at home and not ing the achievement of these changes.18 Therefore, currently requiring inpatient hospital care Global Health: Science and Practice 2020 | Volume 8 | Number 4 848 Program Development for Families Impacted by Congenital Zika Syndrome FIGURE 2. Theory of Change for the Juntos Program for Caregivers of Children with Congenital Zika Syndrome, Brazil Caregivers willing to attend the whole program sites, the states of Rio de Janeiro and Bahia. The and living within 1 hour of the group meeting site coordinators’ main responsibilities were to man- location age the logistic components of the pilot groups, in- Children of any age (although given the nature cluding identifying an appropriate location for the of the epidemic in Brazil in 2017–2018, they groups, recruiting facilitators, recruiting researchers, were all aged 3 years and younger) identifying participants, and liaising with local health providers. Children who may be receiving rehabilitation Rio de Janeiro and Greater Salvador, Bahia, services to address individual needs were selected as pilot sites because they had a large More than 1 caregiver of a child (e.g., mother population of children impacted by Zika. Recife, and father, mother and grandmother) was allowed which the LSHTM team visited during the initial to attend the group meeting. country visit, was not selected because several From June to August 2017, the project lead re- other intervention projects by other organizations searcher (AD) drafted the initial program, adapt- were already taking place and contamination of ing the GTKCP and EIP materials with input from outcomes was a concern. Three sites within Rio expert committee members, project teams in de Janeiro and 3 municipalities of Greater Salvador Brazil, and other experts (TS, MS, SF, EM, HK). (Simões Filho, Lauro de Freitas, and Camaçari) were selected because of their proximity to families of children with CZS, availability of an appropriately PILOTING sized venue, and willingness of the local relevant au- We piloted the approach during 2 phases with 6 dif- thorities to accommodate a group. ferent groups and used this information to finalize Facilitators were identified by the site coordi- the program. A future analysis will report the feasi- nators and approved by the project team lead. A bility of the program using qualitative and quanti- total of 8 local facilitators were selected (4 thera- tative data analysis. pists with experience in pediatrics and CZS and 4 expert mothers). In August 2017, a week-long Program Establishment in Brazil facilitator training was conducted in Rio de The partners in Brazil (IFF and UFBA) identified a Janeiro and led by a trainer who has taught the site coordinator (MS and SF) for each of the 2 pilot GTKCP program extensively. The trainer was Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org international, and we used a translator for the ses- on the changes to the program content and struc- sions as well as materials in Brazilian Portuguese. ture based on fast-track learning in the first pilot The training involved education on facilitating a phase. group, practice sessions with reflective learning Two additional support groups were estab- and feedback, and opportunities for discussion. The lished in each pilot setting (4 total), with the pri- project leads and site coordinators selected 2 pairs of mary aim of ascertaining the feasibility of the facilitators to lead the first pilot groups based on their intervention. These support groups had identical performance during the training week. procedures for data collection, real-time feedback, – Two researchers were identified by the site and fast-track learning (February June 2018). After coordinators and approved by the project team the delivery of the groups, the intervention was fur- lead. All the researchers had a background in psy- ther updated, improved, and finalized using the chology, but this was not a prerequisite for the same processes as before. The 2 groups in Rio had 7 role. The researchers participated in a 2-day train- and 9 families, respectively, and the 2 groups in ing in July 2017 on the research approaches and Greater Salvador had 10 and 7 families, data collection methods and on the fast-track respectively. learning approach that would be used to update and adjust the program content based on weekly Summary feedback that they collected from the groups. Six groups ran between August 2017 and June 2018 across 2 phases. The children of the caregivers Pilot Phase 1 were 25 males and 23 females with an average age In August 2017, the first 2 pilot support groups— of 23 months (standard deviation=9 months) at 1 in Rio de Janeiro and 1 in Greater Salvador— their first session. Of the families included in all 6 pi- started meeting weekly. The Rio group had 7 fam- lot groups, all (n=48) stated the mother as the pri- ilies, and the Greater Salvador group had 8. There mary caregiver. The ages of the mothers (n=48) were 10 sessions for each group with a different were 15–20 years (3), 21–25 years (17), 26–30 years topic each week. Researchers used 3 techniques (5), 31–40 years (18), and 41–50 years (3). Thirty- to collect data to inform real-time feedback and six mothers reported they were married, 3 divorced, fast-track learning about the content and process- and9reportedtheyweresingle.Only6mothers es of the session. First, researchers directly ob- reported being in work, with the most common rea- served the sessions and noted the session flow, son for not being in work being that they cared for ’ participants responses, and behaviors of partici- their child (n=34). pants and facilitators. Second, researchers con- During the second and third groups in Greater ducted focus group discussions at the end of each Salvador, held between January and June 2018, of the 10 sessions with participants and (separate- several children with non-Zika related develop- ly) with facilitators to obtain immediate reflec- mental disabilities participated in the sessions. tions and feedback on the session content. The This was done for 2 reasons: (1) to increase the researchers recorded detailed observation notes about the session and comprehensive notes about number of children participating because the focus group discussions that they uploaded to a number of children with CZS who met the inclu- password-secured Google Drive document for sion criteria was quite low, and (2) to assess the content developer (TS) to analyze. Third, whether combining caregivers of children with researchers recorded pertinent comments from CZS and those with other neurodevelopmental participants, facilitators, and site coordinators on disabilities would be a positive experience. images, content, activities, practicalities, and logis- We focused primarily on the caregiver and the tics, which were made outside of the sessions. program, with some interaction with the family, Weekly calls within 48 hours of the session oc- community, and services at the activity and out- curred between the researchers and TS, which put levels as informed by our theory of change allowed for further explanation and contextuali- (Figure 2). The proximal outcomes of the program zation. Content issues were recorded and are expected to be (1) increased participant quality reviewed to update the program in real time and of life and confidence in caring for a child with for 4 weeks after the conclusion of phase 1 in CZS, and (2) an intervention that is feasible to November 2017. scale up and replicate in other contexts. Core to the theory of change is empowering the caregiver Pilot Phase 2 to improve care for their child through developing In December 2017, a 3-day training session provid- support networks and increased knowledge and ed facilitators and site coordinators with information awareness of their child’s needs. Global Health: Science and Practice 2020 | Volume 8 | Number 4 850 Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org Fast-track learning meant that the interven- The feasibility assessment is not detailed in this Through fast-track tion was updated and improved as new informa- article and will be described in a future article on learning, we tion was gathered each week about what was the findings. updated and working or not. For example, practical or adminis- improved the trative issues, such as organization of transport for Finalization of the Program intervention as participants, were changed and updated in real Consensus on the final content of the program was new information time each week. reached through 2 workshops (London, United was gathered As a result of rapid participant feedback, we Kingdom, and Rio de Janeiro, Brazil) in May 2018. each week about made several changes to the program. For example, One group in Greater Salvador was still running. what was working we changed the title of session 8 (highlighting ad- However, feedback that had already been collected “ ” or not. vocacy and empowerment) to uniting our voices ; from the groups was deemed sufficient to be able to “ ” the original title raising our voices translated to finalize the content. The workshops included the “shouting out loud.” In a second example, partici- technical advisory committees, study site coordina- pants felt that the images used in the first 2 pilots, tors, and researchers (psychologists). which used images from GTKCP and EIP, did not ad- equately reflect phenotype, family behavior, and en- vironment in Brazil. Therefore, as participants PROGRAM DESCRIPTION requested, we included images that reflected their The final program intervention is called Juntos, lives to create identification and favor more adher- which means together in Portuguese and Spanish, ence. A local artist was engaged to draw more cultur- to emphasize the importance of inclusion and mutual ally appropriate images for the later groups, which support. Intervention materials comprise a facilitator were perceived more positively. More representation manual and participant materials, such as photo- of fathers in caring roles was also incorporated at this graphs, animations, and video footage. An allied stage. health professional and an expert mother cofacilitate New innovations in Juntos, which were not in groups that meet once a week for 10 sessions. GTCKP or EIP, include information on the Zika vi- Support and guidance for facilitators is provided by rus, strengthened participatory approaches to en- gage participants with community inclusion and project coordinators via telephone, email, and/or disability rights, and a concerted effort to improve WhatsApp. male engagement,19 which was successful to a de- Groups are held at local community facilities, gree (though the female engagement was still such as health centers, offices of local organiza- much higher). Additional content includes group tions, or schools, to minimize participants’ travel discussion on gastrostomy (dysphagia was a com- time and to foster relationships between people mon problem), creating trousers stuffed with pad- who lived relatively near to each other. Nine ses- ding to support children in sitting, using an sions are only for the caregivers and their children, elasticated cloth to rock children who are irritable, and 1 session is open for other community mem- and activities to promote understanding of disabil- bers to attend. The children who come are looked ity rights. In addition, each session includes reflec- after in a separate room or space by volunteers, tion and discussion on the session and on the past but they are present for some of the practical week through an emotional support activity at the aspects whenever relevant. Table 1 describes each end of the session. The facilitators work as a pair Juntos module. together throughout the session; however, the The sessions are participatory and use principles emotional support activity is facilitated by the ex- of adult learning theory.20 Participants learn by shar- pert mother. The first 5 sessions include the same ing their own experiences and realities about topics activity with facilitated questions: that are important to them, which promotes peer How did you find talking about today’s subject? support, critical thinking, and mutual problem solv- Did it raise any emotions or feelings that you ing. The groups start with a light-hearted icebreaker did not expect? towelcomeandwarmuptheconversationandto encourage comfortable interaction. Participants are How have you been feeling this week? then guided through a series of activities, open dis- The predictability of the questions helps parti- cussions, pair work, explanations, and demonstra- cipants to become comfortable with sharing. By tions. Tables 2 and 3 provide examples of session week 5, participants have explored much of their content from session 4 and 6, respectively. thoughts on emotions and feelings, and this then Supportive information was developed for the progresses to reflecting on the future. program that includes short videos on the program Global Health: Science and Practice 2020 | Volume 8 | Number 4 851 Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org The Juntos logo, which facilitators, psychologists, and site coordinators created to complement the program name’s meaning. and different aspects of care. The individual mod- The integration of a component of caregiver ules, full manual, and supportive materials are emotional well-being in this group intervention available in English, Portuguese, and Spanish: demonstrates a novel approach to including psy- https://www.ubuntu-hub.org/resources/juntos. chosocial support to better promote emotional well-being as an integral part of health work, rath- er than being seen as a standalone effort. There is no single recognized theory of how participatory LESSONS LEARNED 23 groups achieve their health impacts and few The fast-track Fast-track learning added value to the interven- studies evaluate how and why different support learning process tion development because it allowed inclusion of networks improve caregiver and child outcomes. demonstrated language, logistics, content, and culturally specific Examples in resource-limited settings include how participant changes in real time. Participants’ feedback during self-help groups for people with mental health feedback was the first pilot phase was utilized to revise the con- conditions, which demonstrate positive impacts valued and tent (for example, providing case studies, images, on both the people with mental health conditions reinforced the and videos of fathers undertaking practical tasks), and their caregivers.24 Additionally, women's self- program’sfocus which may have made the overall content more help groups have resulted in improved maternal on caregivers, a useful for the later groups. The later groups were and neonatal survival.25 Our integration of a group that has aware of this process and recognized some of the mental health component in Juntos illustrates been overlooked changes based on early peers’ feedback. In a con- that groups that address child development can in the wider Zika text of relative distrust and research fatigue,21 practically integrate emotional support activities. response. this process helped to demonstrate how partici- Facilitators reported that they valued having a pant feedback was valued and reinforced that the dedicated space each week to raise issues of emo- program was genuinely and specifically intended tional well-being. The practical components of the for caregivers, an area that had been largely over- sessions often raised some emotions for a partici- looked in the wider Zika response.22 This could be pant,buttherewouldbelittletimetoexplorethese, an important point of learning for global health so the final section allowed further exploration and practitioners implementing community-based group discussion between the group. Evaluation of wheth- programming: bringing together participants, imple- er such a strategy can work in other settings is neces- menters, and researchers to adapt interventions sary, and negative and unanticipated consequences rapidly as feedback is received. In our approach, al- warrant further evaluation in future work. Having though not by initial design, the use of psychologists an expert mother facilitate these sessions was parti- as researchers and observers provided a unique op- cularly important and helped form group connec- portunity for nuanced feedback. This was particularly tions that might not have been possible with an useful for developing and crafting the messaging and allied health professional alone. discussions on emotional well-being and psychoso- In understanding pathways to change, the role cial support. of the expert mother appears to offer crucial Global Health: Science and Practice 2020 | Volume 8 | Number 4 852 Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org TABLE 1. Finalized Module Topics Included in Juntos, A Community-Support Group for Caregivers of Children with Congenital Zika Syndrome in Brazil Module Topics 1. Introduction About the program Information about Zika and Congenital Zika Syndrome How to find information Personal stories 2. Our child Introducing your close family and friends Development milestones for young children Determining your child’s progress Managing irritability and crying 3. Positioning and moving How to position children who need assistance How to assist children to learn to move 4. Eating and drinking Feeding challenges Practical skills to address challenges for your child 5. Communication Importance of communication Practical advice to help your child communicate 6. Play and early stimulation Importance of play for children to develop and learn Early stimulation Making simple toys Inclusion of play in the family and broader community 7. Everyday activities How to use everyday activities to help your child develop Managing seizures 8. Uniting our voices Understand the context of disability rights Education Communicating with your health team Advocating 9. Our community Who is in your community? Common barriers to inclusion Addressing negative attitudes and exclusion Social activity 10. Next steps Summing up Planning next steps for yourself and the group The expert mother encouragement to shared learning between care- experiences to the process and an expertise and in- appears to offer givers and contributes to developing an egalitarian sight that the other did not possess. The allied crucial atmosphere, expanding care practices beyond health professionals immediately saw the value in encouragement to traditional rehabilitation models.26 Relating this this, and there was no sense of protectionism or de- shared learning common ground and a sense of belonging through fensiveness that they needed to be the lead or ex- between caregivers a social support network provides an environment pert given their professional training. and contributes to to improve the knowledge and skills of care- Groups were held in the local community so developing an givers.15 It was critically important that the 2 cofa- that caregivers could build strong local networks. egalitarian cilitators were equals, each bringing their own This also increased interest from caregivers of atmosphere. Global Health: Science and Practice 2020 | Volume 8 | Number 4 853 Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org TABLE 2. Example of Content From Facilitated Group Session 4 on Eating and Drinking from Juntos, A Community-Support Group for Caregivers of Children With Congenital Zika Syndrome in Brazil Example Discussion Aim Icebreaker How easy or difficult is it to swallow in To understand a range of issues that In pairs: One person tries to give the other a each position? How does it feel to be fed? your child may experience with eating drink of water in different positions (e.g., head and drinking leaning back, turned to one side, or flopping forwards) Discussion What is a nutritious or “balanced” diet? To know what a balanced diet is and As a large group to share experiences how to maximize your child’s nutritional intake and prevent malnutrition Activity Discuss—Are the items hard or soft? Can To learn ways to feed your child safely Show a banana and a biscuit and other they be made into a smooth puree? common food How? TABLE 3. Example of Content From Facilitated Group Session 6 on Play and Early Stimulation from Juntos, A Community-Support Group for Caregivers of Children With Congenital Zika Syndrome in Brazil Example Discussion Aim Icebreaker What is play? To understand how our imagination works with In groups of 3: each group is given one in- play and how children have an even greater expensive everyday item (e.g., cup, piece imagination than adults of cloth, container, ball) and everyone uses their imagination to transform the object into something else and acts it out Discussion What have you found play helps your child To know that play gives children an opportunity As a large group to share experiences to do? to explore, learn about their environment and to Does your child need to play? use and develop their senses Activity Discuss—How can you involve short periods To learn ways for play to be fun, and to see how Toy making, such as making bells and rings of play in your daily activities? How can you fun can motivate children to move and learn and with ribbons involve other members of your family in how other family members can be included playing with your child? children with developmental disabilities other than feared. This was also seen in the sessions where CZS and highlights the importance of de-isolating non-CZS caregivers engaged and, in fact, there Zika from other causes of neurodevelopmental dis- was a value perceived to understand that the chal- ability when developing community support pro- lenges being faced were not unique to only care- grams. Juntos does not replace health care services givers of CZS. This was also reinforced frequently but rather seeks to complement services by in session 8 of the Rio sessions, where an external empowering other caregivers to optimize their speaker came from a local Down’s Syndrome organi- child’s care and upbringing. zation to discuss their advocacy approaches; the ses- We received positive feedback during the ses- sions were always extremely well received by sions that combined caregivers of children with CZS with caregivers of children with other neuro- participants. Although the challenges facing children developmental disabilities. There was a recurrent with CZS and their caregivers remain unique and, to expression of comfort among the caregivers when a certain degree, still unknown, there may be an im- engaging with other caregivers in similar situa- portant value to ensure that there are also many com- tions and circumstances that they were not as mon issues faced and a shared approach may be both alone, unique, and isolated as they had perhaps efficient and useful. Global Health: Science and Practice 2020 | Volume 8 | Number 4 854 Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org RECOMMENDATIONS Limitations The needs analysis that we undertook at the be- Our study has limitations. We describe the inter- ginning of the project8 as well as more recent liter- vention development, but assessment of feasibility ature27,28 has highlighted overlaps and similarities and evaluation of replication and scale-up in other between cerebral palsy and CZS. We suggest that countries is now needed. More work is needed on children with CZS and their caregivers may bene- forming a comprehensive facilitator training pro- fit by integrating and linking with services and gram, and further development of the interven- programs for children with other neurodevelop- tion to include all children with developmental mental disabilities. Rehabilitation/therapy ser- disabilities is warranted. If Juntos is found to be vices were already doing this to a large extent, feasible, robust studies to evaluate the cost- and there seems a good scope for other health effectiveness of the intervention will be needed. and social service providers to also ensure service integration. Conversely, newly formed services as CONCLUSIONS ’ a result of the attention to CZS shouldn t be exclu- We developed and refined a participatory community- sive to this population group and should seek to based group intervention to meet the needs of care- include all families and children who may benefit. givers of children with CZS. Juntos has the potential By the nature of its design, Juntos can poten- to be an important resource for community practice. tially be implemented by a range of stakeholders, There is scope to expand across Brazil and in other from nongovernmental organizations to public SouthAmericancountriesandtochildrenwithother community services to primary health settings. developmental disabilities. This flexibility may mean that there is a stronger opportunity for Juntos to be scaled up. The uni- Acknowledgments: The authors wish to acknowledge the valuable versal primary health structure in Brazil—the insights and inputs made by many people in the UK, Brazil and elsewhere — in supporting the development of the program including Mel Adams, Sistema Único de Saúde could be an avenue to Julian Eaton, Maria Antônia Goulart, Mila Mendonça, Kate Milner, Cally further explore. We see opportunities for public/ Tann, Liana Ventura, Joerg Weber, Maria Zuurmond. private partnerships also. Cost is clearly a major factor in the potential for scale up. Facilitator Funding: The project was funded by Wellcome Trust and Department for training can be done in larger groups to reduce International Development, Grant Number 206719/Z/17/Z. costs. In addition, if the facilitator therapists un- Competing interests: AD joined the Pan American Health Organization dertake the role as part of their existing work, (PAHO) during the research period. Work on the research study was these costs may be further reduced. However, we undertaken outside and separate to his PAHO duties. do feel that it is important to remunerate parent facilitators for their work and other costs, such as REFERENCES transport and refreshments, to ensure full partici- 1. Kindhauser MK, Allen T, Frank V, Santhana RS, Dye C. Zika: the or- Bull World Health Organ pation of families. igin and spread of a mosquito-borne virus. . 2016;94(9):675–686C. CrossRef. Medline 2. Posen HJ, Keystone JS, Gubbay JB, Morris SK. Epidemiology of Zika Strengths virus, 1947–2007. BMJ Glob Health. 2016;1(2):e000087. Strengths of this pilot include the development CrossRef. Medline process being informed by a theory of change and 3. Cauchemez S, Besnard M, Bompard P, et al. Association between Zika virus and microcephaly in French Polynesia, 2013–15: a retro- reflective practice and robust methodology that spective study. Lancet. 2016;387(10033):2125–2132. CrossRef. allowed integration of rapid feedback. Real-time Medline feedback and adaption enabled the development of a 4. Rasmussen SA, Jamieson DJ, Honein MA, Petersen LR. Zika virus and — N Engl J Med culture-specific and language-specific intervention, birth defects reviewing the evidence for causality. . 2016;374(20):1981–1987. CrossRef. Medline and the program was developed and refined to 5. Moore CA, Staples JE, Dobyns WB, et al. Characterizing the pattern meet the needs of caregivers of children with CZS of anomalies in congenital Zika syndrome for pediatric clinicians. in Brazil. Running the program in 2 sites concur- JAMA Pediatr. 2017;171(3):288–295. CrossRef. Medline rently (Rio de Janeiro and Greater Salvador) was 6. Nielsen-Saines K, Brasil P, Kerin T, et al. Delayed childhood neuro- an important methodological choice for achieving development and neurosensory alterations in the second year of life in a prospective cohort of ZIKV-exposed children. Nat Med. better final version program. Brazil is huge and di- 2019;25(8):1213–1217. CrossRef. Medline verse, and although these 2 sites do not cover the 7. Brazil Ministry of Health. Situação epidemiológica da síndrome breadth of diversity, piloting in more than 1 site congênita associada à infecção pelo vírus Zika em 2020 até a SE 40. [Report in Portuguese]. Boletim Epidemiológico. 2020;51(42). Accessed and acquiring different feedback added to the November 23, 2020. https://www.gov.br/saude/pt-br/media/pdf/ strength of the study. 2020/outubro/23/boletim_epidemiologico_svs_42.pdf Global Health: Science and Practice 2020 | Volume 8 | Number 4 855 Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org 8. Duttine A, Smythe T, Ribiero Calheiro de Sá M, et al. Congenital Zika impairment in Uganda. Paper presented at: Pediatric Academic Syndrome—assessing the need for a family support programme in Societies Annual Meeting. May 6, 2017; San Francisco, CA. Int J Environ Res Public Health Brazil. . 2020;17(10):3559. CrossRef. 18. Breuer E, Lee L, De Silva M, Lund C. Using theory of change to design Medline and evaluate public health interventions: a systematic review. 9. Broussard CS, Shapiro-Mendoza CK, Peacock G, et al. Public health Implement Sci. 2016;11:63. CrossRef. Medline approach to addressing the needs of children affected by congenital 19. Smythe T, Duttine A, Vieira ACD, Castro BSM, Kuper H. Engagement Pediatrics – Zika syndrome. . 2018;141(Supplement 2):S146 S153. of fathers in parent group interventions for children with congenital CrossRef. Medline Zika syndrome: a qualitative study. Int J Environ Res Public Health. 10. Kuper H, Lyra TM, Moreira MEL, et al. Social and economic impacts 2019;16(20):3862. CrossRef. Medline of congenital Zika syndrome in Brazil: study protocol and rationale 20. Pappas C. The adult learning theory - andragogy - of Malcolm for a mixed-methods study. Wellcome Open Res. 2019;3:127. Knowles. May 9, 2013. Accessed November 9, 2020. https:// CrossRef. Medline elearningindustry.com/the-adult-learning-theory-andragogy-of- 11. Kuper H, Smythe T, Duttine A. Reflections on health promotion and malcolm-knowles disability in low and middle-income countries: case study of 21. Diniz D, Ambrogi I. Research ethics and the Zika legacy in Brazil. parent-support programmes for children with congenital Zika syn- Dev World Bioeth. 2017;17(3):142–143. CrossRef. Medline drome. Int J Environ Res Public Health. 2018;15(3):514. CrossRef. 22. Diniz D. Zika virus, women and ethics. Dev World Bioeth. 2016; Medline 16(2):62–63. CrossRef. Medline 12. Bailey DB Jr, Ventura LO. The likely impact of congenital Zika syn- 23. Gram L, Morrison J, Saville N, et al. Do participatory learning and drome on families: considerations for family supports and services. action women’s groups alone or combined with cash or food trans- Pediatrics – . 2018;141(Suppl. 2):S180 S187. CrossRef. Medline fers expand women’s agency in rural Nepal? J Dev Stud. 2018; 13. Duttine A, Smythe T, Calheiro de Sá MR, Ferrite S, Moreira ME, 55(8):1670–1686. CrossRef. Medline Kuper H. Development and assessment of the feasibility of a Zika 24. Cohen A, Raja S, Underhill C, et al. Sitting with others: mental health Wellcome Open Res family support programme: a study protocol. . self-help groups in northern Ghana. Int J Ment Health Syst. 2012; 2019;4:80. CrossRef. Medline 6(1):1. CrossRef. Medline 14. Murthy GVS, Mactaggart I, Mohammad M, et al. Assessing the 25. Prost A, Colbourn T, Seward N, et al. Women’s groups practising prevalence of sensory and motor impairments in childhood in participatory learning and action to improve maternal and newborn Arch Dis Child Bangladesh using key informants. . 2014;99 health in low-resource settings: a systematic review and meta-analy- – (12):1103 1108. CrossRef. Medline sis. Lancet. 2013;381(9879):1736–1746. CrossRef. Medline 15. Zuurmond M, Nyante G, Baltussen M, et al. A support programme 26. Mol A. The Logic of Care: Health and the Problem of Patient Choice. for caregivers of children with disabilities in Ghana: understanding 1st ed. Routledge; 2008. the impact on the wellbeing of caregivers. Child Care Health Dev. 27. Carvalho AL, Ventura P, Taguchi T, Brandi I, Brites C, Lucena R. 2019;45(1):45–53. CrossRef. Medline Cerebral palsy in children with congenital Zika syndrome: a 2-year 16. Zuurmond M, O’Banion D, Gladstone M, et al. Evaluating the impact neurodevelopmental follow-up. J Child Neurol. 2020;35(3):202– of a community-based parent training programme for children with 207. CrossRef. Medline cerebral palsy in Ghana. PLoS One. 2018;13(9):e0202096. 28. Marques FJP, Teixeira MCS, Barra RR, et al. Children born with con- CrossRef. Medline genital Zika syndrome display atypical gross motor development and 17. Martin KN R, Lassman R, Webb E, et al. Developing an early inter- a higher risk for cerebral palsy. J Child Neurol. 2019;34(2):81–85. vention programme for children at high-risk of neurodevelopmental CrossRef. Medline En português Juntos: Um Programa de Apoio às Famílias Afetadas pela Síndrome Congênita do Vírus Zika no Brasil ABSTRATO Histórico: O surto, sem precedentes, do vírus do Zika em 2015–2016 resultou no nascimento de mais de 3.000 crianças com a Síndrome Congênita do Vírus Zika (SCZ). Essas crianças experenciam múltiplas e complexas condiçöes de saúde com limitado acesso a serviços de apoio tanto para elas quanto para as suas famílias. O desenvolvimento de um programa piloto: um programa existente de apoio às crianças com paralisia cerebral (Getting to Know Cerebral Palsy -Conhecendo a Paralisia Cerebral) foi adaptado para o contexto do Zika no Brasil através de uma consultoria especializada. O programa piloto foi testado em dois locais com 6 grupos de cuidadores (um total de 48 famílias) entre agosto de 2017 e junho de 2018. Grupos focais e de observação com facilitadores e participantes receberam avaliações ao final de cada intervenção que foram utilizadas para adequar o programa para grupos futuros, através da metodologia de aprendizagem rápida. Isso permitiu ajustar as intervenções em tempo real, o que provou ser um processo útil para informar e aprimorar o programa desde a sua concepção inicial. Descriçäo do programa: a iniciativa Junto é um programa de facilitaçäo e participaçäo para grupos de cuidadores de crianças com SCZ. São dez encontros com a duraçäo de 4 horas - cada um inclui uma dinâmica inicial de quebra gelo, atividades e discussöes em grupo. O conteúdo cobre informaçöes práticas sobre os cuidados com crianças com problemas de desenvolvimento, incluído aqueles causados pelo Zika. O apoio psicossocial abrange um componente importante no qual as famílias são orientadas desde a primeira semana sobre como definir e desenvolver suporte em suas comunidades. A realizaçäo com seis grupos no Rio de Janeiro e na aérea metropolitana de Salvador ocorreu de forma exitosa e em ambos os locais houve um retorno positivo em termos de aceitaçäo e demanda. Conclusões: o programa tem o potencial de ser uma ferramenta importante para as aéreas de saúde e prestação de serviços sociais na América do Sul em resposta ao vírus do Zika. Além disso, pode ser adaptado para crianças com problemas no neurodesenvolvimento para além daqueles causados pelo Zika, o que por sua vez, pode ser importante para garantir que as famílias de crianças com SCZ sintam-se menos isoladas. Global Health: Science and Practice 2020 | Volume 8 | Number 4 856 Program Development for Families Impacted by Congenital Zika Syndrome www.ghspjournal.org Aspectos Principais Um programa de orientação comunitária para grupos de família desenvolvido para os cuidadores de crianças com a Síndrome Congênita do vírus Zika (SCZ), baseado em programa anterior focado em crianças com paralisia cerebral, foi testado como uma experiência piloto no Brasil Gestores que desenvolvem atividades de grupo devem considerar as abordagens de aprendizagem rápida para adequar as suas intervenções, tornando-as mais úteis para os participantes. Médicos e gestores de iniciativas para o apoio às vítimas do Zika devem atentar para o fato de que cuidadores de crianças com CZS podem se beneficiar da interlocução e troca com cuidadores de crianças com outras problemas de neurodesenvol- vimento. Gestores de programas focados em intervenções comunitárias devem levar em consideração engajar as mães como um grupo de cofacilitadoras. Integrar atividades de apoio emocional em grupos para abordar a questão do desenvolvimento infantil é algo prático importante e agrega valor. Peer Reviewed Received: December 23, 2019; Accepted: November 4, 2020 Cite this article as: Duttine A, Smythe T, Ribeiro Calheiros de Sa M, Ferrite S, Moreira ME, Kuper H. Juntos: a support programme for families impacted by congenital Zika syndrome in Brazil. Glob Health Sci Pract. 2020;8(4):846-857. https://doi.org/10.9745/GHSP-D-20-00018 © Duttine et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00018 Global Health: Science and Practice 2020 | Volume 8 | Number 4 857 TECHNICAL NOTE Improving Hospital Oxygen Systems for COVID-19 in Low-Resource Settings: Lessons From the Field Hamish R. Graham,a,b Sheillah M. Bagayana,c,d Ayobami A. Bakare,e,f Bernard O. Olayo,g Stefan S. Peterson,h,i,j,k Trevor Duke,a,l Adegoke G. Faladeb,f Key Messages ABSTRACT Oxygen therapy is an essential medicine and core component of n The COVID-19 pandemic has highlighted and ex- effective hospital systems. However, many hospitals in low- and acerbated deficiencies in hospital oxygen systems middle-income countries lack reliable oxygen access—a deficien- globally but is also an opportunity to “build back cy highlighted and exacerbated by the coronavirus disease better.” (COVID-19) pandemic. Oxygen access can be challenged by equipment that is low quality and poorly maintained, lack of clin- n Our collated field experience from African and Asia- ical and technical training and protocols, and deficiencies in Pacific contexts reveal practical strategies whereby local infrastructure and policy environment. We share learnings hospitals can rapidly improve their oxygen systems. from 2 decades of oxygen systems work with hospitals in Africa We share guidance documents (all open access) for and the Asia-Pacific regions, highlighting practical actions that local use and adaptation. hospitals can take to immediately expand oxygen access. These include strategies to: (1) improve pulse oximetry and oxygen Key Implications use, (2) support biomedical engineers to optimize existing oxy- gen supplies, and (3) expand on existing oxygen systems with ro- n Using our practical guides, hospital staff can help: bust equipment and smart design. We make all our resources * Improve pulse oximetry and oxygen use freely available for use and local adaptation. * Optimize existing oxygen supplies * Expand existing oxygen systems with robust BACKGROUND equipment and smart design xygen therapy is an essential medicine and core Ocomponent of hospital systems that has been a n Policy makers and program managers can use standard of care for more than 100 years.1 However, ac- our recommendations to ensure that investments cess to oxygen therapy is limited in many low-resource in oxygen systems are deployed and implemented settings, where the majority of hypoxemic patients who more effectively and efficiently. are admitted to the hospital will not receive oxygen, resulting in an increased risk of death.2 The coronavirus diseases (COVID-19) pandemic has revealed the extent of this “oxygen gap” and stimulated long overdue interest in improving oxygen systems. Approximately 20% of patients who have COVID-19 re- a quire hospital admission for oxygen therapy (with or Centre for International Child Health, University of Melbourne, MCRI, Royal 3 Children’s Hospital, Melbourne, Australia. without extra respiratory support). Although much at- b Department of Paediatrics, University College Hospital, Ibadan, Oyo, Nigeria. tention has focused on ventilator and intensive care unit c FREO2 Uganda, FREO2 Foundation, Kampala, Uganda. capacity, improving basic hospital oxygen systems must d Biomedical consultant, Uganda Ministry of Health, Kampala, Uganda. take priority.4 e Department of Community Medicine, University College Hospital, Ibadan, Oyo, Nigeria. f Oxygen for Life Initiative, Oyo, Nigeria. CHALLENGES IN OXYGEN ACCESS g Center for Public Health and Development, Nairobi, Kenya. h To provide oxygen therapy, we need a reliable oxygen sup- Chief of Health, United Nations Children’s Fund, New York, NY, USA. i Department of Women's and Children's Health, Uppsala University, Uppsala, ply, prompt identification of patients requiring oxygen Sweden. therapy, and appropriate administration by skilled health j Department of Global Public Health, Karolinska Institutet, Stockholm, Sweden. care workers.5 Oxygen supply is typically achieved using k School of Public Health, Makerere University, Kampala, Uganda. oxygen cylinders (filled at an oxygen plant), oxygen con- l School of Medicine and Health Sciences, University of Papua New Guinea, National Capital District, Papua New Guinea. centrators (concentrating oxygen from air on-site), oxygen Correspondence to Hamish R. Graham ([email protected]). plants (piped directly or distributed via cylinders), or liquid Global Health: Science and Practice 2020 | Volume 8 | Number 4 858 Improving Hospital Oxygen Systems for COVID-19 in Low-Resource Settings www.ghspjournal.org oxygen (delivered from a specialized gas plant and Challenges to oxygen access exist alongside stored on-site at very high pressure). Oxygen use is broader systems issues such as unreliable power guided by nurses measuring blood oxygen levels us- supply, health care workforce constraints, high ing a pulse oximeter (or relying on clinical signs if no out-of-pocket health care costs, low health litera- oximeter is available) and titrating oxygen flow rate cy, and underfunded public health and preventive to maintain adequate blood oxygen levels. services. However, achieving reliable supply and appro- However, our work has shown that effective Achieving reliable priate use is challenging, with major barriers due and sustainable change is possible. oxygen supply and to equipment that is low quality and poorly main- appropriate use is tained, lack of clinical and technical training and LESSONS LEARNED challenging, with protocols, and deficiencies in local infrastructure Over the past 2 decades, we have supported hospi- barriers due to 6 and sociopolitical context. tals in Africa and Asia-Pacific regions to improve low-quality and For example, surveys in Nigeria have found oxygen systems using low-cost technology such poorly maintained that although half of hospitals had oxygen cylin- as oxygen concentrators, pulse oximeters, and cyl- equipment and ders or concentrators on inpatient wards, the cylin- inder distribution systems. Our work has shown lack of clinical and how to combine quality equipment and training ders and concentrators were frequently empty or technical training 2,7 to achieve context-appropriate and sustainable im- nonfunctional. Detailed testing in a selection of and protocols. these hospitals found that only 5% of concentra- provement in oxygen systems and improve clinical 5,11–13 tors tested were producing medical grade oxygen.2 outcomes. This experience has informed the Almost no hospitals had pulse oximeters available development of clinical and technical guidance by on the wards.2,7 Procurement of oxygen equip- the World Health Organization (WHO) and the ’ 14–16 ment was haphazard, preventive maintenance United Nations Children s Fund (UNICEF) was nonexistent, and hospital technicians were and is now informing oxygen scale-up for COVID- We show how to 19 (Box). untrained and under-supported. Hospital nurses combine quality To complement existing technical guidance on were unfamiliar with pulse oximetry, and the ma- equipment and COVID-19 response from WHO17 and others, we of- jority of hypoxemic patients were not receiving ox- training to achieve ygen.2,7 Hospital directors bemoaned the cost of fer practical suggestions based on our on-the- sustainable oxygen, with one director describing oxygen as his ground experience to help policy makers, adminis- improvement in “biggest headache.” trators, technicians, and health care workers seeking oxygen systems to rapidly improve their hospital oxygen systems. We, and others, have reported similar findings and improve in Kenya, Uganda, Papua New Guinea, and other clinical outcomes. African and Asia-Pacific contexts.7–11 Indeed, 1. Support Health Care Workers to Use Pulse unreliable oxygen supplies and deficiencies in ox- Oximetry and Oxygen Through Training and ygen use are consistent and persisting problems Protocols for many hospitals in these regions, particularly In many low- and middle-income countries in rural and remote settings. (LMICs), oxygen is absent from medical and BOX. Essential Resources for Additional Information on Oxygen and COVID-19 Repository of oxygen resources (curated by the United Nations Children’s Fund [UNICEF]): https://bit.ly/ OxygenResources World Health Organization (WHO) COVID-19 Technical Guidance: Essential Resource Planning: https://www.who. int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/covid-19-critical-items WHO COVID-19 Technical Guidance: Patient Management: https://www.who.int/emergencies/diseases/novel- coronavirus-2019/technical-guidance/patient-management WHO Academy’s COVID-19 mobile learning app for Android and iPhone/iPad, which contains much of the clinical and technical advice in a conveniently accessible format. WHO COVID-19 Partners Platform: https://covid-19-response.org/, which includes a supply portal for requesting and receiving critical supplies. WHO Medical Devices for COVID-19: https://www.who.int/medical_devices/priority/COVID-19_medequipment/ en/ UNICEF supply catalogue: https://supply.unicef.org/. Order via UNICEF Country Office. UNICEF Supply Division COVID-19 response: https://www.unicef.org/supply/coronavirus-disease-covid-19 Global Health: Science and Practice 2020 | Volume 8 | Number 4 859 Improving Hospital Oxygen Systems for COVID-19 in Low-Resource Settings www.ghspjournal.org Nurse in Nigeria performing pulse oximetry on an infant, demonstrating oxygen saturation level to parents. © 2017 Oxygen for Life Initiative Pulse oximetry can nursing training,18 and pulse oximetry is unavail- fundamentally new concept for many health care enable health able.2 Pulse oximetry, with practical task-based workers and will require encouragement and care workers to training and simple guidelines, can enable health support to integrate it into the workflow.19 target oxygen to care workers to target oxygen toward those who Education and support for health care workers those who need it, need it most, dramatically improving patient oxy- to use pulse oximetry and oxygen therapy should dramatically gen access and clinical outcomes.13 In contexts also cover maintenance and functioning of oxy- improving oxygen where pulse oximetry is not a standard of care, gen equipment, recognizing the critical impor- access and clinical COVID-19 offers an opportunity to establish oxy- tance of health care worker and technician outcomes. gen saturation as the “fifth vital sign.” However, teamwork in maintaining a reliable oxygen sup- although pulse oximetry is a simple skill, it is a ply. We have created practical training materials, Technicians in Nigeria learning to perform preventive maintenance and repair an oxygen concentrator. © 2017 Oxygen for Life Initiative Global Health: Science and Practice 2020 | Volume 8 | Number 4 860 Improving Hospital Oxygen Systems for COVID-19 in Low-Resource Settings www.ghspjournal.org clinical algorithms, and troubleshooting guides, oxygen systems using oxygen concentrators and/ based on WHO guidelines, for others to use and or cylinders, flowmeter stands, and simple piping adapt (Supplements 1–3). (Supplement 4). 2. Assist Biomedical Engineers to Optimize 3. Expand on Existing Oxygen Systems Using Existing Oxygen Supplies Through Training, Robust Equipment and Smart Design Protocols, and Logistic Support WHO and UNICEF have released guidance on Oxygen is a medicine that depends on technology oxygen-related equipment14,16 and specific guid- and requires effective teamwork between health ance for COVID-19.17 This guidance includes care workers, technicians, and managers. However, low-cost oxygen concentrator-based systems that biomedical engineers and hospital technicians are use simple plastic piping and flowmeter stands to frequently left out of decision-making processes provide oxygen to multiple patients simulta- and lack maintenance budgets or system support. neously. These systems have been successfully Training, provision of tools and spare parts, and implemented in African and Asia-Pacific contexts stronger maintenance and transport systems can en- and can be established in a relatively short time able repair and optimization of existing oxygen frame (compared to a new oxygen plant). With equipment and supply chains. Installation of simple the support of several other donors, UNICEF has Hospitals can use piping and individual flowmeters can improve safety delivered almost 15,000 oxygen concentrators our practical (allowing individual titration of flow), efficiency and approximately 15,000 pulse oximeters to installation (sharing a single oxygen source between multiple 69 countries (at the time of writing). Many other guidance to create patients) and infection control (allowing oxygen donors have channeled equipment support directly. smart and efficient sources to be kept away from patient areas). However, there is a real risk that these valuable ward oxygen Including technicians alongside health care work- investments will end up in equipment graveyards systems to put this ers in multidisciplinary teams can help transform a with inadequate consideration to how they are problem-driven “oxygen headache” into focused deployed in hospitals. Hospitals can use our practical influx of oxygen solutions. We have created practical re- installation guidance to create smart and efficient equipment to use sources to assist biomedical engineers/technicians ward oxygen systems to put this influx of equipment rapidly and to build and maintain reliable, user-friendly to use rapidly and effectively (Supplement 4). effectively. Technician, doctor, and nurse in Ondo state, Nigeria, with new oxygen installation including oxygen concen- trator with power stabilizer, oximeter, flowmeter stand, distribution tubing, oxygen analyzer, and user guides. © 2016 Oxygen for Life Initiative Global Health: Science and Practice 2020 | Volume 8 | Number 4 861 Improving Hospital Oxygen Systems for COVID-19 in Low-Resource Settings www.ghspjournal.org CONCLUSIONS pneumonia. Pediatr Pulmonol. 2020; 55S Suppl 1(Suppl 1):S61– S64. CrossRef. Medline Improving patient outcomes always hinges on doing 7. Clinton Health Access Initiative (CHAI). CHAI Nigeria Final the basics well. The COVID-19 pandemic offers the Narrative. CHAI; 2016. opportunity to refocus efforts on the basics of acute 8. Otiangala D, Agai NO, Olayo B, et al. Oxygen insecurity and mor- care, knowing that improvements in oxygen (as well tality in resource-constrained healthcare facilities in rural Kenya. as infection control, triage, laboratory testing, etc.) Pediatr Pulmonol. 2020;55(4):1043–1049. CrossRef. Medline will benefit patients both now and in the future. 9. Dauncey JW, Olupot-Olupot P, Maitland K. Healthcare-provider Improving oxygen systems is an achievable perceptions of barriers to oxygen therapy for paediatric patients in priority for hospitals in LMICs. We propose practi- three government-funded eastern Ugandan hospitals; a qualitative cal steps to support effective and sustainable study. BMC Health Serv Res. 2019;19(1):335. CrossRef. Medline improvements in hospital oxygen systems during 10. McCollum ED, Bjornstad E, Preidis GA, Hosseinipour MC, Lufesi N. the COVID-19 pandemic. We share these learn- Multicenter study of hypoxemia prevalence and quality of oxygen Trans R Soc Trop Med ings in the hope that health care workers, techni- treatment for hospitalized Malawian children. Hyg. 2013;107(5):285–292. CrossRef. Medline cians, hospital managers, and policy makers will be able to take immediate actions toward better 11. Gray AZ, Morpeth M, Duke T, et al. Improved oxygen systems in district hospitals in Lao PDR: a prospective field trial of the impact on oxygen access. All the accompanying oxygen outcomes for childhood pneumonia and equipment sustainability. resources are freely available for users to down- BMJ Paediatr Open. 2017;1(1):e000083. CrossRef. Medline load, use, and adapt to local needs (https://bit.ly/ 12. Duke T, Wandi F, Jonathan M, et al. Improved oxygen systems for OxygenResources). We welcome your feedback. childhood pneumonia: a multihospital effectiveness study in Papua New Guinea. Lancet. 2008;372(9646):1328–1333. CrossRef. Medline Acknowledgments: We would like to thanks Bev Bradley, Cindy McWhorter, and the United Nations Children’s Fund supply division technical 13. Graham HR, Bakare AA, Ayede A, et al. Oxygen systems to improve team; Janet Diaz and the team at the World Health Organization; and clinical care and outcomes for children and neonates: a stepped- Olatayo Olatinwo and the Oxygen for Life Initiative team. wedge cluster-randomised trial in Nigeria. PLoS Medicine. 2019; 16(11):e1002951. CrossRef. Medline Competing interests: HG, AAB, TD, AGF received funding from the Bill 14. World Health Organization (WHO), United Nations Children’s Fund and Melinda Gates Foundation outside the submitted work. The funders (UNICEF). WHO-UNICEF Technical Specifications and Guidance for had no role in the writing or submission of this article. All other authors Oxygen Therapy Devices. WHO; 2019. Accessed September 21, declare no competing interests. 2020. https://apps.who.int/iris/bitstream/handle/10665/ 329874/9789241516914-eng.pdf REFERENCES 15. World Health Organization (WHO). Oxygen Therapy for Children: 1. Duke T, Graham SM, Cherian MN, et al. Oxygen is an essential A Manual for Health Workers. WHO; 2016. Accessed September medicine: a call for international action. Int J Tuberc Lung Dis. 21, 2020. https://apps.who.int/iris/bitstream/handle/10665/ 2010;14(11):1362–1368. Medline 204584/9789241549554_eng.pdf 2. Bakare AA, Graham H, Ayede AI, et al. Providing oxygen to children 16. World Health Organization (WHO). Technical Specifications for and newborns: a multi-faceted technical and clinical assessment of Oxygen Concentrators. WHO; 2015. Accessed September 21, oxygen access and oxygen use in secondary-level hospitals in south- 2020. https://apps.who.int/iris/bitstream/handle/10665/ west Nigeria. Int Health. 2020;12(1):60–68. CrossRef. Medline 199326/9789241509886_eng.pdf 3. Wu Z, McGoogan JM. Characteristics of and important lessons from 17. World Health Organization. Coronavirus Disease (COVID-19) the coronavirus disease 2019 (COVID-19) outbreak in China sum- Technical Guidance: Essential Resource Planning. Accessed mary of a report of 72 314 cases from the Chinese Center for Disease September 21, 2020. https://www.who.int/emergencies/ JAMA – Control and Prevention. . 2020;323(13):1239 1242. diseases/novel-coronavirus-2019/technical-guidance/covid-19- CrossRef. Medline critical-items 4. Baker T, Schell CO, Petersen DB, et al. Essential care of critical illness 18. Ginsburg AS, Van Cleve WC, Thompson MIW, English M. Oxygen Lancet must not be forgotten in the COVID-19 pandemic. . 2020; and pulse oximetry in childhood pneumonia: a survey of healthcare – 395(102320:1253 1254. CrossRef. Medline providers in resource-limited settings. J Trop Pediatr. 2012;58:389– 5. Graham H, Tosif S, Gray A, et al. Providing oxygen to children in 393. CrossRef. Medline Bull World Health Organ hospitals: a realist review. . 2017; 19. Graham HR, Bakare AA, Gray A, et al. Adoption of paediatric and – 95(4):288 302. CrossRef. Medline neonatal pulse oximetry by 12 hospitals in Nigeria: a mixed-methods 6. Graham H, Bakare AA, Fashanu C, Wiwa O, Duke T, Falade AG. realist evaluation. BMJ Global Health. 2018;3:e000812. CrossRef. Oxygen therapy for children: a key tool in reducing deaths from Medline Peer Reviewed Received: May 18, 2020; Accepted: September 21, 2020; First published online: September 28, 2020 Cite this article as: Graham HR, Bagayana SM, Bakare AA, et al. Improving hospital oxygen systems for COVID-19 in low-resource settings: lessons from the field. Glob Health Sci Pract. 2020;8(4):858-862. https://doi.org/10.9745/GHSP-D-20-00224 © Graham et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http://creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https:// doi.org/10.9745/GHSP-D-20-00224 Global Health: Science and Practice 2020 | Volume 8 | Number 4 862 CORRECTION Corrigendum: Parmaksiz K et al., What Makes a National Pharmaceutical Track and Trace Succeed? Lessons From Turkey See corrected article. n the article “What Makes a National Pharmaceutical Pharmacopeia Quality Institute as one of the funders of ITrack and Trace Succeed? Lessons From Turkey” by the work. Koray Pamaksiz et al. (Volume 8, Issue 3), the Funding The article has been corrected accordingly. statement on page 10, incorrectly listed the U.S. Cite this article as: Corrigendum: Parmaksiz et al. What makes a national pharmaceutical track and trace succeed? Lessons from Turkey. Glob Health Sci Pract. 2020; 8(4):863. https://doi.org/10.9745/GHSP-D-20-00587 © Global Health: Science and Practice. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visit http:// creativecommons.org/licenses/by/4.0/. When linking to this article, please use the following permanent link: https://doi.org/10.9745/GHSP-D-20-00587 Global Health: Science and Practice 2020 | Volume 8 | Number 4 863 Knowledge SUCCESS Johns Hopkins Bloomberg School of Public Health Center for Communication Programs 111 Market Place, Suite 310 Baltimore, MD 21202 Phone: 410-659-6134 Fax: 410-659-6266 Email: [email protected] Web: www.ghspjournal.org www.ghspjournal.org