Direct Consulting & Logistics

Evaluating SMS messaging for immunization demand generation in : Immunization Reminder & Information SMS System (IRISS)

Project Brief January 2021 Reasons children are not fully vaccinated in Northwest Nigeria, NICS 2016/2017

Despite some progress made in the Nigerian routine 14% thought 15% had no child was fully faith in immunization (RI) program in recent times, about three million immunised immunisation children remain unvaccinated. According to NDHS 2018, 50% 37%A 32% of children aged 12-23 months received all three pentavalent Lack of 13% reported Mistrust or 10% reported awareness immunisation fears caretaker too vaccines nationally. Coverage is highly variable across the site was too far busy country with the northwest and northeast zones having the 22% Lack17% of time or worst performance. In Northwest Nigeria, only Service delivery other family issues issues 11% of children received 2 Penta3, and a mere 6% were fully Note: Respondents could identify more than one reason for incomplete immunisaton. vaccinated by age two. Map of Kebbi State showing the intervention and control LGAs Augie

Immunization investments by the government and partners in the last decade have primarily addressed supply-side Bimin Kebbi problems. Still, the demand for vaccines remains low due to Intervention LGAs Jega Control LGAs 3,4 Suru a lack of information and knowledge about the RI Dandi

Koko Bease program. Shange Wasagu/Danko

Sakaba Between January 2018 and August 2020, the International Yauri Vaccine Access Centre (IVAC) in collaboration with the Direct

Consulting and Logistics (DCL) and the Kebbi State Primary Health Care Development Agency (KSPHCDA) implemented an SMS reminder intervention to inform and nudge community members and caregivers about vaccination. This gatekeepers (District Heads, Village Heads, Mai- approach has proven successful at improving immunization ungwas, Imams) to pass on to their community and uptake in multiple settings. The project was designed as a congregation. two-arm cluster randomized controlled trial to assess the SMS 3. As individualized/personalized reminders to reminder's impact on vaccine coverage. Fourteen (14) caregivers and other individuals who opted-in to receive intervention local government areas (LGAs) received the SMS messages on the child's vaccination schedule. intervention, and seven (7) control LGAs did not receive the SMS intervention. The Approaches, Strategies, and Challenges The project team conducted the following activities for the The Intervention successful implementation of the IRISS project in Kebbi State. IRISS (Tunatar da ni, meaning "Remind me" in Hausa) used SMS to inform and educate the community and 1. Conducted a formative study to understand the caregivers/parents about the importance of immunization; context and inform message development and and remind them of their child's immunization schedules, design of the IRISS intervention: FGDs and KIIs were including health facility vaccination schedules in their locality. conducted among program managers, community The project aimed to raise parental demand for vaccination members, RI providers, KSPHCDA leadership, and and reduce the proportion of children left out (un-vaccinated) partners in Aleiro, Argungu, Fakai, and Shanga LGAs in and drop out of the RI program (under-vaccinated). October 2018. We collected data on social norms about vaccinations, barriers to immunization uptake, mobile IRISS delivered immunization information to parents, phone use, SMS message testing, and willingness to caregivers, and the public in three ways: accept IRISS. 11 FGDs, 12 IDIs, and 13 KIIs were 1. As general broadcasts to the public with informative conducted. The key results were as follows. messages about immunization in general and a prompt to a. We found positive perceptions and beliefs about opt-in to IRISS e-registry for more information. the importance of vaccination for child health 2. As targeted broadcasts with information on the benefits among participants interviewed, even among those of immunization and immunization clinic days for the without children. nearest health centre to Leads (individuals that b. There was a high level of trust in health workers. responded to general broadcast) and community Traditional and religious leaders were strong 1 influencers for promoting vaccinations. 2 World Health Organization. 2019. Progress and Challenges with Achieving Universal Immunization Coverage. Available at: https://www.who.int/immunization/monitoring_surveillance/who-immuniz.pdf? (accessed October 12, 2020) 3 National Population Commission (NPC) [Nigeria] and ICF. 2019. Nigeria Demographic and Health Survey 2018. Abuja, Nigeria, and Rockville, Maryland, USA: NPC and ICF.

4 International Vaccine Access Center. 2017. Nigeria Brief Template North West Zone. Available at: https://www.jhsph.edu/ivac/wp-content/uploads/2018/04/Nigeria-NICS-Brief-North-West-Zone.pdf (accessed October 12, 2020) National Primary Healthcare Development Agency and National Bureau of Statistics. Nigeria, National Immunization Coverage Survey 2016/17, Final Report. Abuja, Nigeria: National Primary Healthcare Development Agency and National Bureau of Statistics. 1 c. Pockets of misconception about vaccine safety, Focal Persons, and community leaders (Mai-unguwas), beliefs that vaccines are associated with fertility for ease of use and registration accuracy. Following the reduction, and personal and religious beliefs remain pre-test, a new simplified opt-in format for caregivers, among some community members leading to gatekeepers, and health workers was deployed in May vaccine hesitancy. 2019. Also, proxy registration opt-in format was d. The most common barriers to vaccine uptake across implemented to account for low phone ownership all LGAs were a lack of knowledge on vaccines' IRISS opt-in forCaregivers mats for carGatekeepersegivers, gatekeepers, and health workers Health workers benefits and adverse effects following vaccination, Direct registration: Village heads: RI providers: Firstname lastname, DOB, Vhead, firstname lastname, Provider, firstname h u s b a n d d i s a p p r o v a l , a n d Hfname, LGA name Hfname, LGA name lastname, Hfname, Proxy registration: Mai-ungwas: designation, LGA negligence/forgetfulness. name Proxy, firstname lastname, Mai-ungwa, firstname e. Low mobile phone ownership was observed, DOB, Hfname LGA name, lastname, Hfname LGA phone number name especially among rural hard to reach areas, and Imams: most phone owners were husbands. Literacy levels Imam, firstname lastname, Hfname LGA name and the ability to read text messages were low, but most participants indicated access to someone who could read the text messages for them. 4. Sensitized over 7,000 stakeholders in the 14 f. IRISS intervention was perceived as timely and intervention LGAs: Due to overlaps between the useful for educating and reminding parents on Community Engagement Strategy (CES) and the IRISS immunization program objectives concerning registering newborns g. The IRISS intervention was culturally and religiously for immunization service delivery, we integrated the acceptable with strong support from traditional and sensitization of both programs. At the National religious leaders, who were receptive to receiving Emergency Routine Immunization Coordinating Centre SMS messages and disseminating the same to their (NERICC) request, we supported CES activities in Kebbi communities, despite concerns about low phone State. We obtained phone numbers at each level of ownership. government to have a community-level subscription for 2. Developed messages for dissemination: We tested a targeted broadcast. Between January and May 2019, total of 33 messages covering reminder, educational, we successfully sensitized over 7,000 stakeholders motivational, informative, and normative contents in 2 on IRISS and CES in the 14 intervention LGAs LGAs among community leaders and parents of children including the traditional leaders to obtain their buy-in under-five in October 2018. Each message was tested for for the project. Additionally, information, education, clarity, comprehensibility, relevance, cultural and communication (IEC) materials (e.g. one-pager, appropriateness, and ability to motivate action. All calendar, and other training materials) depicting the messages were generally found to be relevant, step-by-step process of registering for SMS reminders motivating, and culturally acceptable. were developed and used during the stakeholder 3. Established the IRISS application for message sensitization training. These IEC materials were pre- dissemination: Working with an in-country developer, tested during FGD before dissemination. Participants Interra Networks, the IRISS application was developed in kept the materials for reference.

Q4 2018 after testing and refinement with insights from List of Stakeholders sensitized in the 14 intervention LGAs and the IEC materials usedCategory Stakeholder No. sensitized (%) IEC materials used Emirs 4 (100%) IRISS One-pager the formative study described above. The system was Emir Focal Persons 4 (100%) IRISS One-pager Traditional Leadership District Heads (DHs) 80 (100%) IRISS One-pager deployed in May 2019 to send the targeted and Village Heads (VHs) 742 (100%) IRISS calendar individualized broadcast to registered contacts. The Mai-unguwas 5707 (100%) IRISS calendar SERICC members 24 (100%) IRISS One-pager NPHCDA CES notebook and application is a web-based message scheduling system Health LGA program officers 56 (100%) IRISS training materials system Ward Community NPHCDA CES notebook and containing the following databases: personnel Engagement Focal Persons 151 (100%) IRISS training materials a. Contact information for individuals opting-in for RI providers 405 (95%) IRISS calendar more immunization messages; community Sample IEC and sensitization materials gatekeepers; community volunteers, health workers, and caregivers b. Child's information for those who opt-in to receive individualized reminders c. Health facility information d. Message library A screenshot of the IRISS application platform

IRISS was generally accepted across all levels sensitized. However, community members raised concerns about the low phone ownership, a preference for Robocalls (IVR) rather than SMS, inadequate network coverage, and low literacy rates in many rural settlements in Kebbi. During the training, To arrive at the registration format, different opt-in text VHs and Mai-unguwas had trouble completing the CES message formats were pilot tested with the study team line listing and registration into IRISS because of low literacy, inability to text, and low phone ownership. members, RI providers, Ward Community Engagement 2 60%

50%

40%

30%

20%

10%

0% Rural, Good Rural, Poor Network Urban Good Urban Poor Total (N = 137) Network (N = 45) (N = 21) Network (N = 45) Network (N = 26)

Encouraged registration Called a meeting Disseminated message via town announcers Disseminated to community members and at gatherings Other Nothing

Thus, health workers were entrusted with the 2. Assessed the reach of the SMS vaccination responsibility of showing/assisting parents to register messaging broadcast among 229 RI providers: To into IRISS. evaluate the reach of the IRISS intervention further, we 5. Implemented the IRISS intervention in the 14 randomly selected one (1) RI provider for each ward intervention LGAs: Registration into IRISS commenced (225) plus four (4) large facilities (N=229 total) and on May 20, 2019, but broadcast messages interviewed them in June 2020 on the proxy registration (geographical, targeted, and personalized) began June by health workers, receipt of IRISS messages and actions 1, 2019. Between June 1, 2019, and May 31, 2020, we: taken by providers, caregivers, and gatekeepers, and a. Reached over 190,000 active subscribers in the impact of COVID-19 on vaccine services. Among the RI 14 intervention LGAs with immunization providers interviewed, 76% registered caregivers and information to improve positive norms about 53% registered gatekeepers using proxy registration. vaccination. These general broadcasts were We found that 97% of RI providers have received conducted between August 2019 and April 2020, messages on the importance of immunization, and 99% with 1,234,771 messages sent and 587,004 (48%) always opened and read their text messages. Seventy of these messages delivered. percent(70%) of RI providers received messages on b. Sent 160,428 targeted broadcasts on the the immunization schedule from IRISS, and 95% said benefits of immunization to 1,010 Community the messages were accurate. Thirty-six percent(36%) gatekeepers, 3,924 Leads, 406 Health workers, and reported a drop in vaccination rates since March 2020 22 State Emergency Routine Immunization and 77% said caregiver fear of COVID-19 was the main Coordinating Committee (SERICC) members reason for the drop-in vaccination rates. between August 1, 2019, and May 28, 2020, and Action taken by RI provider Caregivers Gatekeepers 131,711(82%) of these messages were delivered. after receiving reminders (N=174) (N=121) Informed the parent/gatekeeper 136 (73%) 97 (80%) Between February 24, 2020, and May 31, 2020, Did not inform the 8 (5%) 1 (1%) 15,150 targeted broadcasts on health facility parent/gatekeeper schedules were sent to community gatekeepers, Vaccinated the child 5 (3%) 0 (0%) Conducted proxy registration using 4 (2%) 0 (0%) and 11,969(79%) of these messages were family/neighbour phone delivered. No proxy registration or messages 21 (12%) 23 (19%) received c. Sent 36,722 personalized reminders of immunization due dates for 21,906 children (approx. 61% of our target) registered into the 3. Conducted end-line qualitative study in both IRISS. Between June 1, 2019, and May 31, 2020, intervention and control LGAs: In June 2020, we and 27,542(75%) of these messages were conducted 13 IDIs, 20 KIIs, and 14 FGDs among delivered. program managers, community members, RI providers, Among 426 health facilities providing RI services in our KSPHCDA leadership, and partners in Augie, Aliero, intervention LGAs, 359(84%) conducted IRISS Fakai, Argungu, and Shanga LGAs. The interview registrations between May 20, 2019, and May 31, 2020. assessed the community's awareness, receipt, Also, 326,151 active subscribers in both intervention perception, and acceptability of IRISS post-intervention and control LGAs were sent messages on COVID-19 using qualitative interviews. The key results were as between June 5 and July 31, 2020. follows. a. The immunization program in Kebbi State has Project evaluation recorded change in increasing demand for 1. Assessed the reach of the IRISS intervention among immunization and addressing programmatic 308 community members: To assess our intervention's challenges affecting service availability. This is reach, we randomly sampled from the IRISS database due to a renewed commitment from the state 137 community gatekeepers and 171 caregivers and government and partners to provide technical and interviewed them via phone in May 2020. The objective financial support to improve vaccine programs was to evaluate whether they received the IRISS coverage. messages and assess the actions taken after receiving the messages. Over 80% of gatekeepers received the COVID-19 Impact on Vaccine Services

targeted broadcast messages, and most (65%) Fixed and Outreach disseminated the messages in some way. An estimated No Impact, 67% services, 22% 65% of caregivers received personalized reminders. Over 90% took their child for vaccination after receiving reminders, and another 3% wanted to take their child to get vaccinated but were unable to. Fixed Services Only, 7% Gatekeeper Interviews: When you received the message, 30%what did you do? 25% Outreach Services Only, 4% 20% 15% 10% 5%

0%

Disseminated Disseminated Called a Encouraged Other Nothing message via to community meeting registration town announcers members and at gatherings 3 90% Reasons for COVID-19 Impact on Vaccine Services challenges in delivering the intervention. Most 80% community members who own mobile phones use 70% 60% it to receive calls only and ignore text messages. 50% Among those who can read their messages, 40% receiving multiple messages from unsolicited 30% 20% platforms made them ignore or delete their 10% messages. 0% Movement Fear of PPE not Provider didn’t Vaccine stock Other 4. Conducted an end-line household survey to assess restriction contracting available (i.e. know the policy out at the COVID face mask, on stopping or health facility gloves, hand continuing the impact of the IRISS intervention: A baseline and sanitizer etc) services an end-line household survey were conducted in Augie Fixed Services (N=67) Outreach Services (N=60) LGA in January 2019 and August 2020, respectively. This was to evaluate the change in vaccination rates among b. The COVID outbreak obstructed planned activities caregivers with children aged 12-23 months and the and interventions but not the availability of intervention's impact on parental intention to vaccinate vaccination services during fixed and outreach their children. Post-intervention, we found an increase in sessions. The concern about receiving the COVID the proportion of children who received penta3 vaccine prevented some parents from getting their vaccination and were fully vaccinated. For instance, 35% children vaccinated of children were fully vaccinated at the end-line c. Post-intervention, the husband's disapproval, and compared to only 7% at baseline. 70 adverse events following immunization with the Baseline and Endline Vaccination Status pentavalent vaccine were mostly cited as 60

barriers to vaccination uptake and completion in 50

all the LGAs. Also, poor attitudes of health workers 40 affected compliance and uptake of immunization 30 services. d. The perceived benefit of vaccination in reducing 20 childhood diseases, deaths, and deformities; and 10

the perception that vaccinated children are 0 Fully Vaccinated Partially Vaccinated Never Vaccinated Penta 3 Measles healthier, stronger, and protected during Baseline (N=797) Endline (N=790) outbreaks motivated them to vaccinate. Likewise, the increased knowledge about 5. Assessed vaccination rates between the intervention v a c c i n a t i o n , t r u s t i n h e a l t h w o r k e r s , and control LGAs using the quarterly LQAS: A total of recommendations from other community 1,260 children aged 0-11 months are surveyed every members, and community leaders' influences quarter by the government in Kebbi State, 840 in the 14 encouraged vaccination. intervention LGAs and 420 in the seven (7) control LGAs. e. Participants from the intervention LGAs confirmed We used the LQAS data to compare vaccination rates, they had received IRISS messages including those reasons for non-vaccination, and sources of on COVID-19. In some cases, they remembered the immunization information between the intervention and message content and frequency. Health workers control LGAs and to assess implementation fidelity and from the intervention LGAs also reported the trends in outcomes. The proportion of children accuracy of immunization schedule time and appropriately immunized for age increased in the year frequency of reminders disseminated. and a half before the IRISS intervention commenced in f. There was general acceptability of the IRISS Q2 (June 2019) due to intensification of RI activities intervention at all levels due to its perceived conducted in the state and driven by the NERICC. At impact. For community members, the SMS baseline (Q2 2019), about 84% and 85% of children 0-11 reminder and messages served as a reminder, months were age-appropriately vaccinated in especially for those that are busy and had intervention and control LGAs, respectively. Still, the forgotten; encouraged vaccination uptake; coverage declined at the end line (Q2 2020) with 73% increased awareness and knowledge about children appropriately vaccinated in the intervention vaccination and triggered a discussion about LGAs compared to 80% in control LGAs. The vaccination. For health workers and program government conducted no further LQAS after Q3 2019 managers, the IRISS intervention increased and the COVID pandemic and lockdown began in Q1 vaccination uptake, reduced their workload, and 2020 LQAS. served as a performance monitoring tool. Proportion of children appropriately immunized by age in both intervention and control LGAs, LQAS Q4 2017 to 202080% 85% 83% 80% g. Participants perceived IRISS as a useful and 84% 71% 79% important intervention because it reminds and 73% 54% educates parents on the importance of IRISS 41% intervention commenced in immunization. Also, they found the message Q2 2019 27% content satisfactory and suitable for their culture. 19% 27% COVID pandemic 14% 13% and lockdown Despite the perceived benefit and impact of the 14% 11% intervention, we found that the lack of mobile 5% Q4 2017 Q1 2018 Q2 2018 Q3 2018 Q4 2018 Q1 2019 Q2 2019 Q3 2019 Q4 2019 Q1 2020 Q2 2020 phones and the inability to read text messages Intervention LGAs Control LGAs

5 due to the low literacy level were significant We selected one intervention LGA that will serve as the sentinel LGA to conduct a before and after household survey. Augie LGA was selected based on a set of agreed criteria including median RI coverage from the Q3 2018 Lots Quality Assurance Survey (LQAS) in Kebbi State. 4 6. Conducted a costing evaluation: To determine the automatically enrolled in the DND list. Thus the incremental human, material, and financial resources reason about 50% of the general broadcast needed to implement a similar program at the state level, messages did not deliver-adding to the relatively we conducted KIIs with Kebbi SERICC members and high cost of the broadcast strategy. Kebbi State Finance Officer and reviewed project expenditures in September 2020. The cost of sending Considering these challenges, we explored sending messages SMS messages will depend on the message type, using a bulk SMS platform for a fraction of the cost, but a frequency, and price per SMS the state is able to database of phone numbers is required to use bulk SMS. To negotiate with service providers. If the price per SMS is overcome these challenges, the IRISS intervention was similar to the price used in the study (#2.50 kobo for adjusted to collect phone numbers during the CES and IRISS sending messages and #4 per registration), then the sensitization activities after obtaining consent from the estimated annual cost of sending SMS vaccine reminders people. to caregivers and weekly SMS messages to health workers and community gatekeepers ranges from o Designing the IRISS application: The design of the 3,570,000 – 5,513,000 naira (US$ 9,400 – 14,500) application is an iterative process along the project depending on the registration rates among caregivers timeline. Formative study or field visits should be and gatekeepers. Expanding the IRISS intervention state- conducted before developing the design framework to wide will vary depending on the deployment and avoid back and forth with developers. In the ethos of implementation approach. This will require additional agile development, the design framework should also be resources and activities including: 1) additional IT and flexible to accommodate changes during the project. management personnel at the state and LGA levels; 2) Finally, there may be a need to incorporate Robocalls due annual training and sensitization activities for health to the community's low literacy level, and some workers and community gatekeepers; 3) ongoing gatekeepers are not aware when they receive text outreach to collect gatekeeper phone numbers; 4) messages. monthly supportive supervision at participating health o Feasibility of IRISS application: Constant monitoring of facilities; and 5) additional hardware and software to the IRISS application is needed to ensure its smooth store and maintain the IRISS database. operation. Structured SMS data entry used by caregivers and providers to register children into IRIS was chosen as Lessons Learned and Recommendations a low cost (no need for data and smartphones) mobile o Designing the IRISS intervention: We encountered data collection method. But, structured SMS data entry challenges with the technical and regulatory feasibility of can be complicated, tedious and highly error-prone, our original intervention design. We had to iterate to find especially for low literacy users. There is a need to a work-around for each challenge. First, we went into the incorporate data cleaning or auto-correct software to project with the hypothesis that SMS messages could be minimize the human resources required to clean the geographically targeted using the cell-phone towers that registration error. serve a specific catchment area. Mobile operations do not o Collaborating with Mobile Network Operators use towers for geographical targeting; instead, they use (MNOs): We encountered slow feedback from MNOs phone numbers of subscribers known to be registered or thus should be approached earlier than when services are residing in a specific LGA. Several consultations with needed. There may be a need to work with the technical service providers (SPs) and the National Communication team directly to ascertain the feasibility of intervention Commission (NCC) regarding the feasibility of sending design at the early stage. location-based reminder messages to the ward-level o Project implementation: Future innovations/ revealed the following issues: interventions should leverage existing structures to o SPs cannot send location-based broadcast promote IRISS registration. Low phone ownership and messages to the ward-level. The LGA-level is the literacy levels and the poor network may impact smallest geographic area SPs can send broadcast disseminating immunization messages than initially messages. planned. Continuous supportive supervision and o Sending IRISS SMS reminders through SPs was refresher training is needed to encourage participation expensive for both the IRISS study and for scaled-up and enrolment into IRISS. CES's potentials were not implementation. optimized to improve registration due to a lack of data o Following a new NCC regulation, SPs are prohibited tools and funding for reconciliation meetings. from sharing subscriber phone numbers with third parties. This had implications on our intervention as This study was made possible through a grant from the Bill and the IRISS application needed phone numbers for Melinda Gates Foundation (BMGF) to Johns Hopkins targeted broadcast messages. Thus, the integration International Vaccine Access Center (IVAC). IVAC of our intervention with CES enables us to obtain implemented the project in collaboration with Direct phone numbers at each level of government to have Consulting and Logistics Limited. The project team a community-level subscription for targeted acknowledges BMGF for their technical support during the broadcast. study duration. We also acknowledge the Kebbi State Primary o The NCC has issued a Do-Not-Disturb (DND) Health Care Development Agency and the Kebbi State directive that restricts sending unsolicited Emergency Routine Immunization Coordination Center for marketing messages to subscribers on the DND list. their tremendous support during the study data collection. 1 2 1 According to the NCC, most Kebbi's phone Project team: Chizoba Wonodi , Chisom Obi-Jeff , Cristina Garcia , Alain Labrique1, Winnie David2, Tobi Bamiduro2, Funmilayo numbers are on the DND, and new SIM cards are Adewunmi2, Adbudlrasheed Abdulraheem2 Contact: Dr. Chizoba Wonodi, [email protected] 5 1. Johns Hopkins Bloomberg School of Public Health; 2. Direct Consulting and Logistics