Systems Analysis of Vaccination in the United States: Socio-Behavioral Dynamics, Sentiment, Effectiveness and Efficiency

Home , Vaccine hesitancy

Systems analysis of vaccination in the United States: Socio-behavioral dynamics, sentiment, effectiveness and efficiency

Gloria J. Kang

Dissertation submitted to the Faculty of the Virginia Polytechnic Institute and State University in partial fulfillment of the requirements for the degree of

Doctor of Philosophy in Biomedical and Veterinary Sciences

Kaja M. Abbas, Co-chair Stephen G. Eubank, Co-chair Madhav M. Marathe, Co-chair Bryan L. Lewis Marcella J. Kelly Andreas Handel (External Examiner)

July 9, 2018 Blacksburg, Virginia

Keywords: public health, vaccination, influenza, computational epidemiology Copyright 2018, Gloria J. Kang Systems analysis of vaccination in the United States: Socio-behavioral dynamics, sentiment, effectiveness and efficiency

Gloria J. Kang

(ABSTRACT)

This dissertation examines the socio-behavioral determinants of vaccination and their impacts on public health, using a systems approach that emphasizes the interface between population health research, policy, and practice. First, we identify the facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination in the United States. Next, we examine current vaccine sentiment on social media by constructing and analyzing semantic networks of vaccine information online. Finally, we estimate the health benefits, costs, and cost-effectiveness of influenza vaccination strategies in Seattle usinga dynamic agent-based model. The underlying motivation for this research is to better inform public health policy by leveraging the facilitators and addressing potential barriers against vaccination; by understanding vaccine sentiment to improve health science communication; and by assessing potential vaccination strategies that may provide the greatest gains in health for a given cost in health resources. Systems analysis of vaccination in the United States: Socio-behavioral dynamics, sentiment, effectiveness and efficiency

Gloria J. Kang

(GENERAL AUDIENCE ABSTRACT)

Public health decisions are ultimately left to those in policy, however these decisions are often subjective and rarely informed by data. This dissertation comprises three studies that, individually, examine various public health aspects of vaccination, and collectively, aim to help inform decision makers by bridging the gaps that persist between scientific evidence and the implementation of relevant health policy. First, we identify the facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination in the United States. Next, we examine current vaccine sentiment on social media by constructing and analyzing semantic networks of vaccine information online. Finally, we estimate the health benefits, costs, and cost-effectiveness of influenza vaccination strategies in Seattle usinga dynamic agent-based model. The work presented here demonstrates a systems approach to public health by way of computational modeling and interdisciplinary perspectives that describe vaccination behavior at the intersection of public health research, policy, and practice. The motivation for this research is to better inform public health policy: by leveraging the facilitators and addressing potential barriers against vaccination; by understanding vaccine sentiment to improve health science communication; and by assessing potential vaccination strategies that may provide the greatest gains in health for a given cost in health resources. Dedication

For my grandparents I thank you, I love you, and I miss you

iv Acknowledgments

I am forever indebted to some of the greatest human beings I have been so fortunate in knowing, and I can hardly begin to list all the reasons why (without ugly-crying). This dissertation would have been impossible if not for the personal and professional relationships that made my time in Blacksburg something I will cherish forever. I am eternally grateful to so many for all their intellectual, moral, emotional, (and nutritional) support. To my committee, for coming together as the rare group of interdisciplinary mentors I needed all along, and for protecting me from the demons of graduate education that manifest in many forms—be it financial, bureaucratic, (or eerily humanoid). To Dr. Kaja Abbas, for being the greatest boss/ critic/ friend/ bully the world has ever known; for being demanding, persistent, and always earnest, with levity and ridiculous laughter that remains unmatched. To Dr. Stephen Eubank, for being my hero and someone I continue to look up to as an educator, advocator, scientist, and curious human being; for his tranquility and lessons in staying cool, calm, and collected. To Dr. Madhav Marathe, for his generosity in time, energy, and invaluable feedback on my dissertation; and for always making the lab feel like family. To Dr. Marcella Kelly, for helping me find my “niche” as a scientist; and for providing me annual refuge at Mountain Lake Biological Station which was essential in maintaining my sanity. And to Dr. Bryan Lewis, for his relentless encouragement, compassion, and guidance through one of the most intense periods of my life. Thank you for being a constant reminder as to why I pursued graduate education in the first place, something that is so easy to forget. Additionally, I owe much of my personal growth as a researcher to Drs. Achla Marathe and Samarth Swarup, who oversaw many of the projects I worked on since the very beginning, including the manuscripts for Chapters 3 and 4. The sheer intimidation of having brilliant supervisors drove me to doing my best work—always. To James Schlitt, Alex Telionis, and Daniel Chen, for just about everything else in life; from troubleshooting my code to bringing me dinner 5 nights in a row. I couldn’t have asked

v for a more supportive group of co-workers, best friends, and enablers. And to all the truly wonderful faculty, staff, and students at NDSSL, particularly Dennie Munson, who sawme through to my defense; thank you for putting up with all my shenanigans. You guys forever rule. And of course, to my family for being my biggest cheerleaders and sending me countless care packages that single-handedly fed me for months while I was writing. Lastly, I thank you, the reader. You could be doing anything else in the world right now, but you chose to be right here. I hope you enjoy.

Gloria J. Kang July 2018 Blacksburg, Virginia

vi Contents

List of Figures xi

List of Tables xii

List of Abbreviations xiii

Preface xiv

1 Introduction 1 1.1 Infectious Disease Epidemiology ...... 1 1.2 Infectious Disease Modeling ...... 2 1.3 Computational Epidemiology ...... 3 1.4 Dissertation Focus ...... 3

2 Parental attitudes toward school-located influenza vaccination 5 2.1 Abstract ...... 6 2.2 Introduction ...... 6 2.2.1 Study objective ...... 7 2.2.2 Public health significance ...... 7 2.3 Methods ...... 7 2.3.1 Search strategy ...... 7 2.3.2 Data abstraction and synthesis ...... 7 2.3.3 Inclusion and exclusion criteria ...... 8

vii 2.3.4 PRISMA process ...... 8 2.4 Results ...... 8 2.4.1 Study characteristics ...... 8 2.4.2 Facilitators ...... 10 2.4.3 Barriers ...... 11 2.5 Discussion ...... 12 2.5.1 Facilitators ...... 12 2.5.2 Barriers ...... 13 2.5.3 School-located influenza vaccination in school-based clinics versus delivery by external agencies ...... 13 2.5.4 Limitations ...... 14 2.5.5 Public health implications ...... 14 2.5.6 Systems thinking in school-located influenza vaccination ...... 15

3 Semantic network analysis of vaccine sentiment 24 3.1 Abstract ...... 25 3.2 Introduction ...... 25 3.2.1 Vaccine hesitancy ...... 25 3.2.2 Social network analysis and digital epidemiology ...... 26 3.2.3 Semantic networks ...... 26 3.2.4 Study objective ...... 27 3.2.5 Public health significance ...... 27 3.3 Methods ...... 27 3.3.1 Data retrieval and document selection ...... 27 3.3.2 Vaccine sentiment coding ...... 28 3.3.3 Construction of vaccine sentiment networks ...... 28 3.3.4 Semantic network analysis ...... 28 3.4 Results ...... 29 3.4.1 Document characteristics ...... 29

viii 3.4.2 Document text networks ...... 29 3.4.3 Vaccine sentiment networks ...... 29 3.4.4 Central concepts ...... 30 3.4.5 Dynamic visualizations ...... 30 3.5 Discussion ...... 31 3.5.1 Semantic network analysis of vaccine sentiment ...... 31 3.5.2 Message framing ...... 32 3.5.3 Limitations ...... 32 3.5.4 Implications for public health and vaccine communication ...... 33 3.5.5 Conclusion ...... 33

4 Cost-effectiveness of seasonal influenza vaccination 43 4.1 Abstract ...... 44 4.2 Introduction ...... 45 4.2.1 Background ...... 45 4.2.2 Study objectives ...... 46 4.2.3 Public health significance ...... 46 4.3 Methods ...... 46 4.3.1 Simulation model ...... 47 4.3.2 Direct and indirect effects of vaccination ...... 48 4.3.3 Clinical costs and health outcomes ...... 48 4.3.4 Cost-effectiveness ...... 49 4.3.5 Sensitivity analysis ...... 49 4.4 Results ...... 49 4.4.1 No vaccination scenario ...... 49 4.4.2 Current vaccination scenario ...... 50 4.4.3 Healthy People 2020 vaccination scenario ...... 51 4.4.4 Sensitivity Analysis ...... 51 4.5 Discussion ...... 52

ix 4.5.1 Public health implications ...... 52 4.5.2 Limitations ...... 53 4.5.3 Conclusions ...... 54

5 Conclusions 68 5.1 Summary ...... 68 5.2 Moving Forward ...... 69 5.3 Final Thoughts ...... 70

Bibliography 71

Appendices 83

Appendix A Chapter 3 – Supplemental Information 84 A.1 ChatterGrabber ...... 84 A.2 Study methodology ...... 86 A.2.1 Network annotation and construction ...... 87 A.2.2 Definitions of network measures ...... 87 A.3 Network visualizations ...... 89 A.4 Centrality measures ...... 95 A.5 Data files ...... 98

Appendix B Chapter 4 – Supplemental Information 99 B.1 Healthy People 2020 vaccination scenario ...... 99 B.2 Sensitivity Analysis ...... 105

x List of Figures

2.1 PRISMA flowchart ...... 16

3.1 Maximum k-core subgraphs of semantic networks by vaccine sentiment ... 36 3.2 Significant vaccine concepts by centrality (degree, betweenness, and closeness centrality) ...... 39 3.3 Signficant vaccine concepts by eigenvector centrality ...... 40

4.1 Epidemic curves for all vaccination scenarios ...... 61 4.2 Health outcomes in the current vaccination scenario ...... 63 4.3 Incremental cost-effectiveness ratios in the current vaccination scenario ... 65 4.4 Cost-effectiveness planes for the current vaccination scenario ...... 67

A.1 Full semantic network graphs ...... 89 A.2 Greatest component subgraphs ...... 92

B.1 Health outcomes in the Healthy People 2020 vaccination scenario ...... 100 B.2 Incremental cost-effectiveness ratios in the Healthy People 2020 vaccination scenario ...... 102 B.3 Cost-effectiveness planes for the Healthy People 2020 vaccination scenario .. 104 B.4 Sensitivity analysis ...... 105

xi List of Tables

2.1 Characteristics of reviewed studies on school-located influenza vaccination programs ...... 17 2.2 Facilitating factors for parents regarding school-located influenza vaccination 22 2.3 Barriers for parents to school-located influenza vaccination ...... 23

3.1 Summary of vaccine web page documents ...... 35 3.2 Summary of network measures by individual document and by sentiment group 41 3.3 Nodes by eigenvector centrality ...... 42

4.1 Model parameters ...... 55 4.2 Influenza-related outcomes, probability, and clinical costs of illness ...... 57 4.3 Health outcomes in the current vaccination scenario ...... 58 4.4 Budget impact analysis ...... 59 4.5 Incremental cost-effectiveness ratios in the current vaccination scenario ... 60 4.6 Direct and indirect effects of vaccination ...... 60

A.1 Description of ChatterGrabber parameters ...... 84 A.2 ChatterGrabber search terms ...... 85 A.3 Twitter data via ChatterGrabber ...... 85 A.4 Positive sentiment network: Top centrality ...... 95 A.5 Negative sentiment network: Top centrality ...... 96 A.6 Neutral sentiment network: Top centrality ...... 96 A.7 Top ranked nodes by closeness vitality for the three networks ...... 97

xii List of Abbreviations

ACIP Advisory Committee on Immunization Practices

CDC Centers for Disease Control and Prevention

DALY Disability-adjusted life year

ICER Incremental cost-effectiveness ratio

ILI Influenza-like illness

NLP Natural Language Processing

PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analysis

SLIV School-located influenza vaccination

VE Vaccine efficacy

YLD Years of life lost due to disability

YLL Years of life lost

xiii Preface/Attribution

The following chapters are co-authored manuscripts which have been published or are in preparation:

• Chapter 2: Kang, G. J., Culp, R. K., & Abbas, K. M. (2017). Facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination in the United States: Systematic review. Vaccine, 35(16), 1987-1995.

• Chapter 3: Kang, G. J., Ewing-Nelson, S. R., Mackey, L., Schlitt, J. T., Marathe, A., Abbas, K. M., & Swarup, S. (2017). Semantic network analysis of vaccine sentiment in online social media. Vaccine, 35(29), 3621-3638.

• Chapter 4: Kang, G. J., Lewis, B. L., Marathe, A., & Abbas, K. M. Health benefits, costs, and cost-effectiveness of influenza vaccination in Seattle. (In preparation for journal submission).

xiv Chapter 1

Introduction

Public health refers to organized measures designed to prevent disease, improve health, and prolong life for the global population [1, 2]. As human populations are expected to grow and expand into dense urban areas, outbreaks of emerging and re-emerging infectious diseases pose an ever-increasing threat to human and animal health. As such, mathematical models of infectious disease have become a significant and powerful tool for public health in informing health policy. Disease modeling can provide flexible, epidemiological frameworks that allow for a systems approach to better understand the spread and persistence of infectious pathogens affecting populations. From a broader perspective, the term “systems science” not only refers to the complexity of systems under study [3], but additionally describes the approach by which complex problems should be evaluated [4]. The study of public health is inherently interdisciplinary, and the success of its practice relies upon the collective efforts of diverse stake holders. For these reasons, systems modeling frameworks serve as an effective and efficient intermediary between scientific theory and the real world—an iterative feedback loop between empirical evidence and its practical relevance. In this chapter, I briefly introduce the history of mathematical epidemiology, its interface with public health policy, and describe relevant foundations to the current research topics at hand.

1.1 Infectious Disease Epidemiology

Various sub-fields within epidemiology have expanded and continue to evolve, given funda- mental changes to the way we conceptualize and understand causality. Galea et al. have discussed this difficult yet necessary methodological shift for traditional epidemiology to adopt new standards of using complex systems dynamic approaches, such as simulation

1 2 Chapter 1.

modeling, for population health research [5]. While its methods are still useful, addressing the complexity of modern-day health problems will require more than mere extension of traditional analytic approaches (such as using mul- tilevel/hierarchical regression models) due to significant limitations in the ability to explain the dynamics of human behavior. Because the relationship between behavioral dynamics and disease propagation is often described as non-linear and complex, a common approach to studying dynamical systems—and one of the focal points of this dissertation—is agent-based modeling.

1.2 Infectious Disease Modeling

Mathematical models of infectious disease have a rich interdisciplinary history rooted in ecology and have greatly contributed to our current understanding of the dynamics of infectious disease in humans and animals [6]. In regards to public health, the earliest known instance of using a mathematical approach for health policy is attributed to Daniel Bernoulli and his analysis of smallpox vaccination in 1760 [7]. In an effort to influence policy makers to adopt universal inoculation against smallpox, Bernoulli demonstrated the efficacy of smallpox vaccination while discounting for its potential risks, arguing the need for mass inoculation by presenting the overall benefits to population health. Over 150 years later, classical epidemic modeling formally emerged. Early theoretical frameworks describing observed epidemiological patterns were developed by Hamer, Ross, Soper, Kermack and McKendrick throughout the early 1900’s, giving rise to the foundational principle of infectious disease epidemiology, the “mass action principle” [8]. The principle of mass action posits that disease transmission depends on the rate of contact between susceptible and infectious individuals. In its simplest form, an epidemic model assumes a homogeneous population in which individuals have equal probability of transmitting and acquiring disease; this is best conveyed by compartmental models. Compartmental models are built upon differential equations that describe the rates of flow between compartmentalized disease states. The SIR model (Susceptible, Infectious, Recovered) has been commonly used to illustrate simple transmission dynamics for a uniform population. Practical applications for modeling involve the designing and testing of specific intervention strategies used to mitigate the population-level impacts of epidemic outbreaks. As fields of health and medicine continue to progress, so does our understanding of the biological and social variability present among individuals—demonstrating the heterogeneity and dynamics of population health in terms of individual susceptibility, transmissibility, and health-related behavior. Over time, advances in computing and systems modeling would gradually allow for varying levels of complexity, allowing researchers to more accurately represent such systems with detailed relevance. Gloria J. Kang 3

1.3 Computational Epidemiology

Computational epidemiology is largely attributed to the interdisciplinary collaborations between empirical, theoretical, and applied research disciplines. Given recent advances within the social and computational sciences [9], simulation models are rapidly improving in their abilities to address the complexity of population health issues. A key advantage to this approach is the ability to closely examine the interplay between individual-level differences (such as health behavior) and population-level health outcomes. Heterogeneities in individual behavior significantly affect the course of an epidemic. Under- standing such behavioral differences is paramount to understanding the kinds of prevention and control measures that are most appropriate for a given outbreak. In the context of policy modeling, models that do not take behavioral dynamics into account may not just be unreliable, but may also be unable to effectively inform public health policies, which tend to target individual-level behavior—not macro-scale dynamics [10]. Reviews of modeling the influence of human behavior on the spread of infectious diseases have described models under various assumptions, such as homogeneous populations and underlying network structure [11, 12, 13]. In modern times, the advent of the Internet and social media have dramatically altered the ways in which we communicate and share information. As an extension of network modeling, social network analysis examines the connections of social relationships between individuals, elucidating the transmission dynamics of a social—rather than biological—contagion across a non-physical contact network. Understanding the nature of existing contact networks carries significant public health value. Many of the external forces influencing ourhealth behavior have been detailed in sociological theories of epidemiology [14]. A notable example involving network effects on health, which serves as another focal point of this dissertation, is vaccination behavior.

1.4 Dissertation Focus

Public health decisions are ultimately left to those in policy, however these decisions are often subjective and rarely informed by data. The connecting theme of systems emphasizes systems theory in public health as an ongoing, iterative learning process of understanding, analyzing, and improving health systems, including leadership and collaborative efforts that span across disciplines, sectors, and organizations [15]. The overall aim of this research is to examine the public health impact of social and structural factors affecting individual vaccination behavior in the context of health policy and planning. The following chapters represent the components of a conceptual framework for systems modeling in public health research. Each study proposes a different level of examination regarding vaccination and public health outcomes. These include: the recent implementation 4 Chapter 1. of local vaccination programs via systematic review (chapter 2); the current state of vaccine- related beliefs, attitudes, and information via online social media (chapter 3); and the health economics of vaccination programs via simulation modeling (chapter 4). The motivations for public health modeling underlie the contributions of this work—to better inform decision makers in policy by bridging the gaps that persist between scientific evidence and the implementation of relevant health policy. This dissertation aims to provide a local contribution to the global effort of improving the efficacy, efficiency, and equity ofpublic health practices in mitigating infectious diseases for the betterment of all population health. Chapter 2

Facilitators and Barriers of Parental Attitudes and Beliefs Toward School-Located Influenza Vaccination in the United States: Systematic Review

Attribution

The following chapter is based on the published manuscript: Kang, G. J., Culp, R. K., & Abbas, K. M. (2017). Facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination in the United States: Systematic review. Vaccine, 35(16), 1987-1995. https://doi.org/10.1016/j.vaccine.2017.03.014.

5 6 Chapter 2.

2.1 Abstract

The study objective was to identify facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination in the United States. In 2009, the Advisory Committee on Immunization Practices of the Centers for Disease Control and Prevention expanded their recommendations for influenza vaccination to include school-aged children. We conducted a systematic review of studies focused on facilitators and barriers of parental attitudes toward school-located influenza vaccination in the United States from 1990 to 2016. We reviewed 11 articles by use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework. Facilitators were free/low cost vaccination; having belief in vaccine efficacy, influenza severity, and susceptibility; belief that vaccination is beneficial, important, and a social norm; perception of school setting advantages; trust; and parental presence. Barriers were cost; concerns regarding vaccine safety, efficacy, equipment sterility, and adverse effects; perception of school setting barriers; negative physician advice of contraindications; distrust in vaccines and school-located vaccination programs; and health information privacy concerns. We identified the facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination to assist in the evidence-based design and implementation of influenza vaccination programs targeted for children in the United States and to improve influenza vaccination coverage for population-wide health benefits.

2.2 Introduction

School-based health interventions have been implemented throughout the United States, with most school-based health clinics offering vaccination services to the general school community. In contrast, school-located vaccination programs, and specifically school-located influenza vaccination (SLIV), are dedicated programs for targeted vaccination of school- aged children [16, 17]. SLV programs have been adopted worldwide in countries such as Canada [18], the United Kingdom [19], and Australia [20]. While less common in the United States, school-located programs for influenza vaccination have shown success statewide in Hawaii [21] and in pilot studies in Tennessee [22] and Maryland communities [23]. Since the 2009 H1N1 influenza pandemic, SLIV programs have gained significant public health interest [16] for improving adolescent vaccination rates in non-clinical settings [24, 25, 26, 27], potentially reducing emergency care visits for influenza-like illnesses, lower- ing community influenza risk, decreasing laboratory-confirmed cases, and improving school attendance [28, 29]. In a modeling study by Weycker et al., authors found that vaccinating 20% of children in the United States decreased the total number of influenza cases in the total population by 46%, along with similar decreases in influenza-related mortality and economic costs [30]. However, because SLIV participation ultimately depends on parental consent, there is a need for enhanced understanding of parental attitudes and beliefs regarding SLIV in order to improve influenza vaccination rates among school children in the United States. Gloria J. Kang 7

2.2.1 Study objective

Our study objective was to identify the facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination in the United States, thereby assisting in the evidence-based design and implementation of current and future influenza vaccination programs targeted for children, by leveraging facilitators and addressing potential barriers of parental consent.

2.2.2 Public health significance

In 2009, the Advisory Committee on Immunization Practices (ACIP) of the Centers for Disease Control and Prevention expanded recommendations for targeted influenza vaccination by including school-aged children in the United States [31]. While this has improved vaccination coverage among children (6 months–17 years) from 43.7% during the 2009–2010 influenza season to 59.3% during the 2015–2016 season [32], this is below the target of 70% in the Healthy People 2020 initiative [33]. Despite globally recognized benefits of school-located vaccination, the evidence base for SLIV acceptance in the United States is limited [26, 27], with studies focused on clinical aspects of vaccine efficacy [34], program feasibility [35], and population-level benefits [36]. We conducted a systematic review to address this evidence gap to improve influenza vaccination coverage by identifying facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination for children in the United States.

2.3 Methods

2.3.1 Search strategy

We conducted our search using PubMed and Web of Science databases for articles written in the English language, published between 01/01/1990 and 10/01/2016, and contained the following the terms: (influenza) AND (vaccine OR vaccination OR immunization) AND (school OR school-located OR school-based) AND (parent OR parental).

2.3.2 Data abstraction and synthesis

The data abstraction and synthesis process were conducted by two authors (GJK and RKC) independently; we resolved discordant decisions through consensus. Data abstraction and synthesis included the following four steps: identification, screening, eligibility, and inclusion. During the identification step, articles were identified using the aforementioned search 8 Chapter 2. strategy. During screening, duplicate articles were removed, and the titles and abstracts of the remaining articles were screened to determine relevance to our study objectives. During the eligibility step, article full text was analyzed to further determine relevance to our study objectives and to be used for inclusion.

2.3.3 Inclusion and exclusion criteria

We included articles that focused on childhood/adolescent age groups to target school-aged children in grades Pre K-12 which met the following study criteria: (1) conducted qualitative and/or quantitative analysis regarding influenza vaccination for school-aged children in the United States; and (2) assessed parental factors associated with the acceptance, hesitancy, or refusal of utilizing school-located influenza vaccination for children, including parental knowledge, beliefs, and attitudes. We excluded studies that focused on general vaccine delivery (i.e. non-specific to influenza vaccine), studies of non-explicit parent populations (such as school personnel and health care workers who may also be parents), and studies taking place outside the United States.

2.3.4 PRISMA process

Figure 2.1 illustrates the process flow diagram of identification, screening, eligibility, and inclusion of articles for the systematic review, using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework [37]. Eleven articles met our selection criteria for systematic review of facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination in the United States. While we have included quantitative metrics of the clinical effect size of statistical association for each of the 11 studies, we have excluded quantitative synthesis using meta-analysis due to the heterogeneity in study design and population sampling of these 11 studies.

2.4 Results

2.4.1 Study characteristics

We identified 11 articles focused on school-located influenza vaccination (SLIV) for analysis based on the inclusion and exclusion criteria of our systematic review. Table 2.1 illustrates the objectives of the 11 studies, SLIV context (hypothetical or actual program context), school settings, geographic area, type of survey and/or focus group, parental sample sizes, and significant inferences regarding parental attitudes and beliefs of SLIV for school-aged children in the United States. Gloria J. Kang 9

Allison et al. surveyed elementary school parents in Salt Lake City, Utah and found that SLIV programs should address vaccine safety, benefit, cost, and convenience, while promoting vaccination as a social norm [24]. Brown et al. conducted an online survey of a nationally representative sample of parents, whose youngest child was less than 15 years old. While the convenience of SLIV promoted parental acceptance, parents preferred a medical location for proper administration and for care of potential medical needs and side effects. Vaccine safety was a significant barrier to consent [26]. Carpenter et al. briefly surveyed parents of large metropolitan public school system in Knoxville, Tennessee and found that significant barriers to SLIV participation included concerns regarding vaccine adverse effects and vaccine virus transmission to household members with health issues such as asthma [22]. A two-year survey conducted by Cheung et al. in urban elementary schools of Los Angeles County, California found that parents with better understanding of influenza risks and influenza vaccine benefits were more likely to consent to SLIV[38]. Gargano et al. surveyed middle and high school parents in Richmond County, Georgia and found that SLIV acceptance by parents correlated with parental beliefs of influenza vaccination being a social norm and perception of illness severity prevented by vaccination in general [39]. Kelminson et al. conducted a survey of parents in urban/suburban middle schools in Aurora, Colorado and found that belief in vaccine importance was associated with SLIV acceptance; parental absence during vaccination was a major barrier to consent [40]. Kempe et al. conducted a survey of public elementary school parents in a low-income area of Denver, Colorado and found that SLIV was strongly supported by parents due to belief in vaccine efficacy and convenience of a school setting, while the barriers involved concerns regarding vaccine safety and parental absence during vaccination [41]. Focus group discussions of parents and students were conducted by Herbert et al. in a low- income, rural county of Georgia; the barriers of non-participating parents in SLIV involved distrust, suspicions of the vaccination clinic, and the lengthy consent process [42]. Middle- man et al. held focus groups of elementary, middle, and high school parents in a large urban school district of Houston, Texas and found that parental attitudes to SLIV were impacted by safety and trust issues regarding vaccines in general; programs should effectively communicate information regarding competency of health personnel administering the vaccine and equipment sterility [43]. In a related study, Middleman et al. conducted a survey of parent-student dyads in a large urban Houston school district; authors found that parental participation in SLIV was impacted by perceptions of equipment sterility, universal access of vaccines for all students, and cost [27]. Lastly, Won et al. conducted a 2-year survey of middle school parents in a low-income urban school district and found that baseline trust in SLIV programs was moderately high among low-income parents, while higher trust and participation of SLIV may be attained by increasing parental perception of vaccine importance [44]. 10 Chapter 2.

2.4.2 Facilitators

The facilitators of parental attitudes and beliefs toward SLIV in the United States are illustrated in Table 2.2 and described below. Cost: Parents were willing to participate in SLIV if they had no additional out-of-pocket expenses [24]. Free or low cost vaccines were significant facilitators of parental acceptance [43] but were less important when compared to other factors [27]. Vaccine efficacy: Parents with higher belief in vaccine efficacy were inclined to participate in SLIV [41]. Influenza severity: Parents with higher perceived severity of adolescent illness, including influenza, were more likely to accept SLIV[39]. Perceived severity of influenza illness was a predicting factor for parental consent [38]. Influenza illness susceptibility: Parental belief of their child being susceptible to influenza was a predicting factor of SLIV consent38 [ ] and associated with acceptance if vaccines were offered for free [24]. Parents who had worried about the H1N1 virus in 2009 were also more likely to consent to SLIV participation [26]. Vaccine benefits: Parents with higher perceived benefit of influenza vaccine protecting against illness [38], combined with stronger belief in vaccination as a social norm [39] were more inclined to accept SLIV. The belief in vaccine benefit was also associated with acceptance if vaccines were offered free of cost24 [ ]. Vaccine importance: Parental perception of vaccine importance was directly correlated with acceptance and trust in SLIV [40, 44]. Vaccination as a social norm: Social norms were associated with parental acceptance of school-located vaccination in general and for influenza vaccine specifically when compared to other adolescent vaccines [39]. Parental belief in vaccination as a social norm was associated with acceptance of SLIV if the vaccine was offered for free24 [ ]. Influenza vaccine does not cause influenza: Parental belief in influenza vaccine not causing influenza was associated with acceptance of SLIV if the vaccine was offered forfree [24]. Medical setting barriers: Endorsement of medical setting barriers such as inconvenience and time constraints promoted SLIV acceptance [24]. School setting advantages: Parents perceiving school-located vaccinations as convenient also facilitated SLIV acceptance [26, 41, 42, 43]. Parental presence during vaccination: Flexible vaccination scheduling, such as during evenings or weekends, allowing parents to accompany children increased likelihood of SLIV participation [43]. Gloria J. Kang 11

Discussion with health care provider: Positive discussion about influenza vaccination and advice from a health care provider promoted parental consent and participation [24]. Trust in school health personnel: Having knowledge of credentials and having trust in the competency of health personnel administering vaccines improved parental consent [43]. Universal vaccine access in school: Ensuring availability of influenza vaccines for all students was an important factor for parental acceptance—more important than offering free or low cost vaccines [27].

2.4.3 Barriers

The barriers of parental attitudes and beliefs toward SLIV in the United States are illustrated in Table 2.3 and described below. Cost: Parents were less likely to participate in SLIV due to cost [40, 41] especially with multiple children in the household [24], however, it was not a primary concern when compared to other barriers [43]. Vaccine safety: Parental concerns of vaccine safety in general, including influenza vaccine in particular [24, 38, 41] and risks [26] lowered their support to participate in SLIV. Equipment sterility: Negative perceptions regarding sterility of equipment used for vaccine administration in a school setting was a significant factor impacting parental decision to trust and participate in SLIV [27]. Vaccine efficacy: Parents concerned with vaccine efficacy were less willing to participate in SLIV [24]. Influenza non-susceptibility: Parents with belief that their children were not susceptible to influenza were less likely to participate in24 SLIV[ ]. Adverse effects: Parents concerned of vaccine side effects were less likely to consent to SLIV [38, 42], with common concerns involving adverse effects of the live-attenuated influenza vaccine [22]. Influenza illness acquisition from vaccine: Parental concerns regarding influenza illness acquisition from the influenza vaccine was a barrier to SLIV participation22 [ ]. Medical setting advantages: Parents preferred a medical setting for vaccination due to trust and safety issues regarding the child’s well-being [41, 42], potential side effects, and for proper vaccine administration [27, 38, 43]. School setting barriers: Parental consent and acceptance of school vaccine delivery involved concerns regarding competency of person delivering the vaccine [24], the lengthy consent process [42], disorganization of the school [40], and the inability to address potential medical issues [43]. 12 Chapter 2.

Parental absence during vaccination: Parents wanting to be present during the child’s vaccination were less inclined to consent for SLIV in their absence [24, 38, 41]. Parents who felt that their children would want them present during vaccination was also a notable barrier [40]. Discussion with health care provider: Receiving negative physician advice based on incorrect contraindications of the live-attenuated influenza vaccine deterred parental participation in SLIV [22]. Distrust of vaccines and vaccination programs: Parents expressing skepticism of the influenza vaccine and/or the school-located vaccination program opted to either vaccinate their children through primary care physicians and pharmacies, or forgo influenza vaccination entirely. Negative attitudes toward the university-implemented vaccination program and associated misconceptions of research being performed on their children (i.e. to test an experimental vaccine) was a distinct barrier to SLIV participation [42]. Health insurance information: Parents were unwilling to provide health insurance information for billing, acting as a barrier to SLIV participation [41]. Health information privacy: Parents who were uncertain of the use/misuse of health information collected from their children’s medical records were reluctant to consent to SLIV [42]. Pharmaceutical company: Poor communication and lack of knowledge regarding the pharmaceutical company manufacturing the influenza vaccine deterred parent participation in SLIV [43].

2.5 Discussion

2.5.1 Facilitators

Our review found that free or low cost vaccines generally facilitated parental acceptance of school-located influenza vaccination (SLIV) [24, 27, 43]. Parental acceptance is likely to be further facilitated by the Affordable Care Act [45] of 2010 which requires influenza (and other vaccines) to be covered by health insurance without charging a copayment or coinsurance, and the uninsured rate has declined by 43% from 16.0% in 2010 to 9.1% in 2015 [46]. Parents perceiving the convenience of a school setting over medical settings for vaccination were relatively more likely to consent [24, 26, 41, 42, 43]; having a positive discussion with a health care provider [24] and trusting the competency of health personnel administering the vaccine [43] significantly enhanced parental attitudes and acceptance for SLIV programs. Parents also preferred the scheduling of SLIV to take place after school or during weekends to allow parents the ability to accompany children during vaccination [43]. Additionally, the availability of influenza vaccines for all students was an important factor Gloria J. Kang 13 for parents [27, 43]. Studies utilizing the Health Belief Model (HBM) [47] suggested that parents with enhanced perceptions of influenza susceptibility and severity, risks of H1N1 influenza, and benefitsof influenza vaccination (including belief that the influenza vaccine does not cause influenza) were more likely to accept SLIV for their children [24, 38, 39, 41]. Having beliefs in vaccine efficacy [41], vaccine importance [40, 44], and vaccination as a social norm [24, 39] also promoted SLIV acceptance among parents. While most parents accepting of vaccines also consented to SLIV, some parents with no intention of vaccinating for influenza also stated willingness to participate if SLIV became available [24, 39].

2.5.2 Barriers

Significant barriers to SLIV acceptance were often related to the elements of the influenza vaccine, including concerns regarding vaccine safety [24, 26, 27, 38, 41, 43], vaccine efficacy [24], vaccine adverse effects [22, 38, 42], and the risk of influenza acquisition from the vaccine itself [22]. Parental distrust of the school-located vaccination program was a notable barrier to participation, particularly for SLIV implemented by an external entity in a school setting without a health clinic [42]. Vaccine trust issues involved skeptical attitudes toward the vaccine [26, 42, 43], concerns regarding equipment sterility and cleanliness of the school location [27, 43], and lacking knowledge of the pharmaceutical company that manufactured the vaccine [43]. Parents were unwilling to provide health insurance information for billing [41], and due to distrust in the vaccination program, parents felt uncertain regarding the use/misuse of health information collected from medical records of their children [42]. Trust issues, safety concerns, and medical setting advantages presented barriers for vaccination within a school setting [26, 38, 41, 42, 43]. Common concerns involved competency of health personnel administering the vaccine and their ability to address potential medical issues in a school setting [24, 40, 42, 43]; many parents did not want their children to receive vaccination in their absence [24, 38, 40, 41]. Other barriers included parental belief that their children were not susceptible to influenza24 [ ] and having received physician advice that negatively portrayed live-attenuated influenza vaccination due to an incorrect understanding of contraindications [22]. Lastly, vaccine cost was generally perceived as a minor barrier for parents [24, 40, 41, 43].

2.5.3 School-located influenza vaccination in school-based clinics versus delivery by external agencies

The studies included in this systematic review assessed parental attitudes and beliefs in relation to hypothetical SLIV scenarios as well as pilot program contexts. The pilot studies 14 Chapter 2. summarized here utilized external agencies such as health departments [22, 38, 41], university research staff [42], and hospitals [44] to deliver influenza vaccination in schools, as opposed to utilizing a school-based health clinic that is offered year-round; these two scenarios may present different issues of trust and concern among parents. Due to considerable heterogeneity in the format of school-located vaccination programs [40], future SLIV programs should take various scenarios into consideration during planning phases.

2.5.4 Limitations

Studies in this review reported limitations of low response rates [22, 26, 27, 38, 39, 44], limited generalizability [24, 27, 38, 39, 41, 42, 43, 44], and potential selection bias [27, 38, 39, 40, 43]. Some studies were geared toward hypothetical SLIV programs in the future [26, 27, 41], and thereby, the responses of parents were based on potential action rather than actual behavior. Differences in survey development, analysis, and subjective interpretation of qualitative responses of parents by authors limited comparability across studies as well as prioritization of parental barriers and facilitators. However, study findings encompass diversely varied populations and geographic regions within the United States which provides collective insight for potential prioritization within specific communities. While the review of literature in this study is from 1990 to 2016, publication dates of reviewed articles span from 2007 to 2015, with only two studies conducted before the 2009 H1N1 influenza pandemic. Thus, the analysis timeline of this systematic review may bebiased toward studies after the 2009 H1N1 pandemic and possibly reflect elevated awareness of influenza among parental attitudes and beliefs toward SLIV programs. Additionally this may be reflective of the nature of discourse surrounding recent utilization of school-located immunization programs, signifying a young and evolving concept and area which necessitates further study.

2.5.5 Public health implications

Effective from the 2010–2011 influenza season, the ACIP recommends seasonal influenza vaccination annually for individuals aged 6 months and older without contraindications to prevent and control seasonal and pandemic influenza [48]. The Healthy People 2020 initiative includes the target of influenza vaccination coverage of 70% [33]. Yet, influenza vaccination coverage in the general population was below par, ranging from 36.8% in Nevada to 56.6% in South Dakota during the 2015–2016 influenza season, with a national vaccination coverage among children (6 months–17 years) of 59.3% [32]. In this systematic review, we identified the facilitators and barriers of parental attitudes and beliefs toward SLIV for children in the United States that can assist in improving coverage and effectiveness of SLV programs. Specifically, influenza vaccination coverage is improved Gloria J. Kang 15

among children whose parents did not plan to vaccinate in the absence of a school-located program [24, 39]. Further, improving influenza vaccination coverage among school children in general improves herd immunity in the total population. The Affordable Care Act [45] of 2010 lowered the uninsured rate by 43% from 16.0% in 2010 to 9.1% in 2015 [46], and health insurance now covers influenza vaccines without additional out-of-pocket payments. While cost has become a lesser barrier, SLV programs can facilitate improved access to influenza vaccination for school-aged children.

2.5.6 Systems thinking in school-located influenza vaccination

Health program strategies based on systems thinking focus on an ongoing iterative learning of systems understanding, analysis and improvement, and leadership and collaboration across disciplines, sectors, and organizations [15]. School-located influenza vaccinations are collaborative programs between health and education sectors with great potential for improving influenza vaccination coverage among school-aged children. SLIV programs directly benefit vaccinated children who express protective immune response, as well as indirectly benefiting the larger community by reducing transmission pathways. We identified facilitators and barriers of parental attitudes and beliefs toward SLIV from a systems thinking perspective. Through systematic understanding, analysis, and identification of facilitators and barriers, this study provides evidence to improve the design and implementation of current and future SLIV programs by leveraging key promoting factors and addressing potential barriers.

Funding

This study is supported by NIH/NIGMS R01GM109718 and NSF/NRT 1545362; the funding sources had no role in study design; collection, analysis, and interpretation of data; writing of the paper; or the decision to submit it for publication. 16 Chapter 2.

Figure 2.1: PRISMA flowchart. The Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) flow diagram of article identification, screening, eligibility, and inclusion in the systematic review is illustrated. Articles focused on the facilitators and barriers of parental attitudes and belief toward school-located influenza vaccination in the United States were included, while articles focused on non-influenza vaccination, non-parent populations, and regions outside of United States were excluded. Gloria J. Kang 17 Most families were Caucasian or Latino in a single school of a single season, limiting generalizability Limitations SLIV programs should address barriers of cost and inconvenience, promote immunization as a social norm, and address parental concerns regarding vaccine safety and benefit Study findings Contexts, objectives, school settings, 75% of parents, including 59% (76/129) who did not plan to immunize would consent to SLIV if offered for free; facilitators were belief in benefit (AOR: 6.1; 95% CI: 2.7–14.0), endorsement of medical setting barriers (AOR: 3.7; 95% CI: 1.3–10.3), belief that immunization is a social norm (AOR: 3.3; 95% CI: 1.4–7.6), and that the child is susceptible to influenza(AOR: 2.6; 95% CI: 1.2–5.7) Statistical effect size Health Belief Model Health behavior model Cross-sectional survey of 259 parents (out of 397) design 1 public elementary school (K-6) in Salt Lake City, UT School setting Methods; study To identify parental beliefs, barriers, and acceptance of SLIV Objective Characteristics of school-located influenza vaccination studies. SLIV program ] Hypothetical 24 Study Context [ geographic area, survey/focus groupation, type, significant inferences, parent sampleinfluenzaand limitations vaccination sizes, (SLIV) health from studies behaviorCI: for school-aged pertaining model, Confidence childreninterval). effectto sizeparental in theattitudes United of statistical States associ- (AOR: Adjustedandbeliefs of school-located odds ratio, RR: Risk ratio, Table 2.1: 18 Chapter 2. Low response rate limited generalizability; responses to hypothetical program do not reflect likelihood of action Low response rate limited rep- resentativeness Limitations SLIV convenience promoted acceptance, but medical location was preferred for proper administration and potential care of side effects; vaccine safety was a significant barrier Barriers to SLIV participation included concerns regarding vaccine adverse effects, influenza acquisition through vaccine, and concerns of vaccine virus transmission to household members with health issues such as asthma Findings 51% of parents would consent to SLIV; SLIV was more convenient than the regular location (42.1% of consenting parents versus 19.9% of non-consenting parents, P <0.001), however, regular location was preferred over SLIV in case of side effects (46.4% vs. 20.9%, P <0.001) and for proper administration of the vaccine (31.0% vs. 21.0%, P <0.001) Non-participation in the vaccination campaign was reported by 53% (34/64) parents of black students and 36% (494/1339) parents of non- black students (RR: 1.44; 95% CI: 1.13–1.83) Stat. effect Survey based on literature review of vaccine topics, focus groups, and pretest interviews Feedback survey regarding SLIV participation Model Online survey of 1088 parents whose youngest child was <14 years old Brief survey of 1432 parents Design Table 2.1 continued from previous page Nationally representative online sample 76 schools (50 elementary, 14 middle, 12 high schools) from 1 large metropolitan public school system (K-12) in Knox County, TN Setting To assess the feasibility of SLIV To evaluate the feasibility and success of SLIV Objective SLIV program (2 doses LAIV) delivered by county health department ] Hypothetical ] SLIV campaign 26 22 Study Context [ [ Gloria J. Kang 19 Low response rates suggest selection bias; higher consent rates of survey respondents (compared to school enrollees) skews responses; respondents were from low-income, mostly Hispanic (followed by Asian) population, limiting generalizability Low response rate; potential duplication of responses; single county population, predominantly black, low-income; generalizability is limited Limitations Parents with better understanding of influenza risks and influenza vaccine benefits were more likely to consent for SLIV SLIV acceptance by parents correlated with beliefs of influenza vaccination being a social norm and perception of illness severity prevented by vaccination in general Findings During 2009 influenza pandemic, parents concerned about influenza severity were twice as likely to consent for influenza vaccination compared to unconcerned parents (OR: 2.04; 95% CI:1.19–3.51). During year 2, facilitators of parental consent were perception of high susceptibility to influenza (OR: 2.19, 95% CI:1.50–3.19) and high vaccine benefit (OR: 2.23, 95% CI:1.47–3.40) Facilitators of SLIV were higher parental perception of benefits to vaccination (AOR = 1.3; 95% CI 1.1–1.5) and increased social norms (AOR = 1.44; 95% CI 1.12–1.84) Stat. effect Health Belief Model Health Belief Model; Theory of Reasoned Action Model Survey of 1259 parents (Year 1); 1496 parents (Year 2) Telephone & web survey of 686 parents for 3 years Design Table 2.1 continued from previous page 8 urban elementary schools (PreK-6, ages 5-13) in 2 school districts, Los Angeles County, CA 6 middle and 5 high schools; urban, Richmond County, GA Setting To determine predictors of consent based on parental attitudes for SLIV To determine parental attitudes and acceptance of school-located vaccination of middle and high school students for four adolescent recommended vaccines, including influenza vaccine Objective delivered by county public health department in 8 schools in year 1 and continued in 4 schools in year 2 program for adolescent vaccines in general; SLV for influenza had been previously available ] SLIV campaign ] Hypothetical 38 39 Study Context [ [ 20 Chapter 2. Purposive sampling of respondents from rural Georgia, majority black and low-income limits generalizability; focus groups conducted over 2 years after program implementation, suggesting recall bias; no demographic or socioeconomic data collected No collection of income data; possible sampling bias; responses to hypothetical program do not reflect likelihood of action Low-income, urban population of mostly ethnic minorities limits generalizability; survey conducted during same year as 2009 H1N1 pandemic Limitations Barriers of non-participating parents of SLIV involved distrust, suspicions of vaccination clinic, and lengthy consent process Belief in vaccine importance was associated with SLIV acceptance by parents; major barrier was parental absence during vaccination SLIV was strongly supported by parents with belief in vaccine efficacy and convenience of school setting; barriers involved concerns about vaccine safety and parental absence during vaccination Findings Not applicable; no quantitative/ statistical analysis 81% of parents agreed that SLIV would be safe and convenient, however, 47% preferred another vaccination site Facilitators were belief in vaccine efficacy (RR: 1.49; 95% CI: 1.23–1.84) and convenience of school delivery (RR: 2.37; 95% CI: 1.82–3.45). Barriers were safety concerns of influenza vaccine (RR: 0.80; 95% CI: 0.72–0.88) and not wanting their child vaccinated without a parent (RR: 0.74; 95% CI: 0.64–0.83) Stat. effect Focus groups, open discussion on attitudes toward SLIV Survey based on medical literature Health Belief Model Model Focus group discussions of 41 parents (and 44 students) Cross-sectional mailed survey of 500 (out of 806) parents Survey of 699 parents Design Table 2.1 continued from previous page Middle and high school; rural county in Georgia 3 urban/suburban middle schools (grade 6) in Aurora, CO 20 public elementary schools (K-6), low SES; Denver, CO Setting To characterize decision-making process and reasons of parents and students for participation in SLIV To examine parental attitudes toward adolescent vaccination in school settings, including influenza vaccine To assess parental attitudes and supportive factors for SLIV of elementary school students Objective place 2+ years after implementation of a 3-year, multi- component influenza vaccination program delivered by research group SLIV program campaigns per school delivered by city public health department and community health services ] Study takes ] Hypothetical ] 2 SLIV 42 41 40 Study Context [ [ [ Gloria J. Kang 21 Selection bias of highly educated parents limits generalizability 1 large, urban school district limits generalizability; low response rate and potential selection bias; responses to hypothetical program do not reflect likelihood of action Limitations Low response rate and non-response bias; cannot validate responses from self-reported data Parental attitudes to SLIV are impacted by safety and trust issues regarding vaccines in general; programs should effectively communicate information of competency of health personnel administering the vaccine and of equipment sterility SLIV participation by parents is impacted by equipment sterility, universal access of vaccines for all students, and cost Findings Baseline trust in SLV was moderately high among low-income parents; higher trust and participation can be attained by increasing parents’ perception of vaccine importance Not applicable—no quantitative/statistical analysis Not applicable - no quantitative/statistical analysis Stat. effect Annual household income, survey language version, participation in a previous SLIV, child’s health insurance status, and perceived vaccine importance were significantly associated with parental trust in SLIV (multiple linear regression analysis; R2: .06, p <.001) ] 49 Questions were based on medical literature Survey based on focus groups Model Trust survey adapted from Dugan et al. [ ] of 49 37 parents; 5 focus group interviews Survey of 566 parent-student dyads Survey based on trust measures [ Design 1608 parents (year 1); 844 parents (year 2) Table 2.1 continued from previous page 1 elementary, 2 middle, 3 high schools in large urban school district in Houston, TX 3 middle and 3 high schools in large, urban school district; Houston independent school district 8 middle schools in a large, low-income, urban school district in Texas Setting To determine factors influencing parental consent of SLIV To describe parent and student perspectives for participation in SLIV To determine parental trust and effect of trust-building interventions in school-located vaccination, including influenza vaccination Objective SLIV program SLIV program vaccination campaigns for multiple vaccines, including influenza vaccine, and delivered by researchers and local hospital ] Hypothetical ] Hypothetical ] 3 school located 43 27 44 Study Context [ [ [ 22 Chapter 2.

Table 2.2: Facilitators. Facilitators of parental attitudes and beliefs toward school-located influenza vaccination programs

Promoting Factor Description Study Cost Offering free/low cost vaccines [24, 27, 43] Vaccine efficacy Belief in vaccine efficacy [41] Influenza severity Belief in perceived severity of influenza [38, 39] Influenza illness Parental belief in children being susceptible to [24, 26, 38] susceptibility influenza and risk concerns of H1N1 influenza Vaccine benefits Belief in benefit of influenza vaccine to protect [24, 38, 39] against influenza illness Vaccine importance Belief in importance of vaccination in general [41, 44] Vaccination is a social Belief that vaccination is a social norm [24, 39] norm Influenza vaccine does Belief that the influenza vaccine does not cause [24] not cause influenza influenza Medical setting barriers Perception of inconvenience in accessing regular [24] medical settings for vaccination School setting Perception of convenience in accessing school setting [26, 41, 42, 43] advantages for vaccination Parental presence Parents being present during vaccination after school [43] during vaccination or during weekends Discussion with health Positive discussion with health care provider about [24] care provider influenza vaccination Trust in school health Trust in competency of health personnel [43] personnel administering the influenza vaccine Universal vaccine Access and availability of influenza vaccine for all [27, 43] access in school students in school Gloria J. Kang 23

Table 2.3: Barriers. Barriers of parental attitudes and beliefs toward school-located influenza vaccination programs.

Barrier Description Study Cost Concerns of potential billing related to [24, 40, 41, 43] school-located vaccination Vaccine safety Safety concerns of vaccines in general, [24, 26, 27, 38, including the influenza vaccine 41, 43] Equipment sterility Trust concerns of cleanliness and sterility of [27, 43] equipment used for vaccination Vaccine efficacy Concerns of vaccine efficacy [24] Influenza Parental belief that their children are not [24] non-susceptibility susceptible to influenza Adverse effects Concerns of adverse effects from vaccination [22, 38, 42] Influenza illness Concerns of acquisition of influenza illness from [22, 41] acquisition from influenza vaccine vaccine Medical setting Parents preferred vaccination at regular [26, 38, 41, 42, 43] advantages medical settings for trust and safety reasons School setting barriers Concerns regarding competency of person [24, 40, 42, 43] administering the vaccine, school disorganization, and inability to address medical issues Parental absence Parents did not want their children to receive [24, 38, 40, 41] during vaccination vaccinations in their absence Discussion with health Negative physician advice based on incorrect [22] care provider live-attenuated influenza vaccine contraindications and concerns of vaccine virus transmission to household members with health issues such as asthma Distrust of vaccines and Distrust and skepticism about the vaccination [42, 43] vaccination programs program and vaccines in general, including influenza vaccine. Health insurance Unwillingness of parents to provide health [41] information insurance information Health information Privacy concerns of use/misuse of collected [42] privacy medical information and distrust of vaccination program Pharmaceutical Lack of knowledge of pharmaceutical company [43] company manufacturing the influenza vaccine Chapter 3

Semantic Network Analysis of Vaccine Sentiment in Online Social Media

Attribution

The following chapter is based on the published manuscript: Kang, G. J., Ewing-Nelson, S. R., Mackey, L., Schlitt, J. T., Marathe, A., Abbas, K. M., & Swarup, S. (2017). Semantic network analysis of vaccine sentiment in online social media. Vaccine, 35(29), 3621-3638. https://doi.org/10.1016/j.vaccine.2017.05.052.

24 Gloria J. Kang 25

3.1 Abstract

Objective: To examine current vaccine sentiment on social media by constructing and analyzing semantic networks of vaccine information from highly shared websites of Twitter users in the United States; and to assist public health communication of vaccines. Background: Vaccine hesitancy continues to contribute to sub-optimal vaccination coverage in the United States, posing significant risk of disease outbreaks, yet remains poorly understood. Methods: We constructed semantic networks of vaccine information from internet articles shared by Twitter users in the United States. We analyzed resulting network topology, compared semantic differences, and identified the most salient concepts within networks expressing positive, negative, and neutral vaccine sentiment. Results: The semantic network of positive vaccine sentiment demonstrated greater cohesiveness in discourse compared to the larger, less-connected network of negative vaccine sentiment. The positive sentiment network centered around parents and focused on communicating health risks and benefits, highlighting medical concepts such as measles, autism, HPV vaccine, vaccine-autism link, meningococcal disease, and MMR vaccine. In contrast, the negative network centered around children and focused on organizational bodies such as CDC, vaccine industry, doctors, mainstream media, pharmaceutical companies, and United States. The prevalence of negative vaccine sentiment was demonstrated through diverse messaging, framed around skepticism and distrust of government organizations that communicate scientific evidence supporting positive vaccine benefits. Conclusion: Semantic network analysis of vaccine sentiment in online social media can enhance understanding of the scope and variability of current attitudes and beliefs toward vaccines. Our study synthesizes quantitative and qualitative evidence from an interdisciplinary approach to better understand complex drivers of vaccine hesitancy for public health communication, to improve vaccine confidence and vaccination coverage in the United States.

3.2 Introduction

3.2.1 Vaccine hesitancy

Sub-optimal vaccination coverage in the United States continues to pose significant risk of disease outbreaks, in part, due to vaccine hesitancy [50]. Vaccine hesitancy refers to a combination of beliefs, attitudes, and behaviors that influence an individual’s decision to vaccinate despite vaccine availability; these behaviors include refusal, delay, or reluctant acceptance despite having active concerns [51, 52]. Strategies to address vaccine refusal have focused on individual reasons for not vaccinating, however, evidence of successful in- 26 Chapter 3.

terventions remains limited. A review of vaccine hesitancy interventions expressed weak support for current strategies in mitigating vaccine resistance [53]; interventions targeted toward anti-vaccination groups are likely to be ineffective, unsustainable, and potentially more detrimental compared to no intervention at all [53, 54, 55]. Vaccine hesitancy stems from socio-cultural, political, and otherwise non-medical factors that are poorly understood [56]. The underlying causes of vaccine hesitancy should not be attributed to scientific illiteracy alone [57], but rather viewed as a deliberative and structured process that requires contextualized examination at local levels [58, 59]. In the case of our study, we focus on semantic and rhetorical qualities of vaccine communication amongst the general public within contexts of differing vaccine sentiment.

3.2.2 Social network analysis and digital epidemiology

The advent of the Internet and social media has provided new platforms for persuasion and rapid spread of (mis)information, bringing forth new challenges and opportunities to an age- old public health problem. Social Network Analysis (SNA) broadly studies social interactions of contact networks with significant implications for public health [3], such as contributing evidence that belief systems are a primary barrier to vaccination [60]. Novel public health tools such as SNA employ computational frameworks in the context of digital epidemiology [61]. Online social media such as Twitter are novel avenues to acquire real-time data of attitudes, beliefs, and behaviors, particularly for underrepresented demographic groups who disproportionately comprise Twitter users [62]. By leveraging online data, studies can examine the dynamics of massively interacting populations, such as online health sentiment and its potential impact on infectious disease outbreaks [63, 64].

3.2.3 Semantic networks

Semantic networks are graphical representations of knowledge based on meaningful relationships of written text, structured as a network of words cognitively related to one another [65, 66], in this study, vaccine information. Within the semantic network, nodes are words that represent concepts found in text. The connections between nodes are referred to as edges which represent relationships between connected concepts. Semantic networks allow extraction of meaningful ideas by identifying emergent clusters of concepts rather than analyzing frequencies of isolated words [67]; in this way, analyzing online social media can enhance understanding of complex health behavior, particularly for vaccine hesitancy. Similar studies have analyzed websites using search engine results and natural language processing (NLP) [68, 69]. Text network analysis traditionally employs semi-automated techniques in which information is extracted and analyzed using both human and computerized methods, dealing with challenges such as co-reference resolution, synonym resolution, Gloria J. Kang 27

and ambiguity [70]. To limit these issues, we constructed semantic networks manually and then performed network analysis within our study. Both proximate and non-proximate determinants of vaccine hesitancy necessitate an interdisciplinary approach [71, 72]. Our study presents a novel framework that applies methods of network analysis to semantic networks [73] within the context of vaccine sentiment.

3.2.4 Study objective

Our objective was to examine current vaccine sentiment on social media by constructing and analyzing semantic networks of vaccine information from highly shared websites of Twitter users in the United States.

3.2.5 Public health significance

The Strategic Advisory Group of Experts on Immunization Working Group (SAGE) on Vac- cine Hesitancy reported specific research needs to better understand context-specific causes underlying vaccine hesitancy [74]. To help address this gap, we utilized quantitative network methods in analyzing qualitative aspects of vaccine information—an efficient approach to investigating the scope and variability of current attitudes and beliefs toward vaccines. Such findings are pivotal in informing and improving public health communication ofvaccine confidence.

3.3 Methods

3.3.1 Data retrieval and document selection

We used ChatterGrabber [75], a web-scraping tool that randomly samples public tweets of Twitter users in the United States. Details on ChatterGrabber including search term conditions, qualifiers, and exclusions are available in Appendix A.1. Web page links from collected tweets identified current sources of vaccine information based on the frequency of link shares during the time of data collection. Our analysis focuses on the textual content of relevant web page articles (also referred to as documents) and not the tweeted text per se. Document types selected for analysis included blog posts, media stories, informational articles, and news reports. We excluded academic publications, court documents, and media formats such as images, PDF files, and videos. A total of 26,389 tweets were collected between April 16, 2015 and May 29, 2015 from which we obtained 8416 unique web links. To generalize findings from a representative pool of 28 Chapter 3. popular vaccine articles, we screened the top 100 most shared links for relevance from which we randomly sampled 50 for analysis; we excluded articles concerning non-human vaccines.

3.3.2 Vaccine sentiment coding

Articles were read for content and manually coded as having either positive, negative, or neutral sentiment toward vaccines. Coding was determined by whole-text assessment which included examining the title/headline and the source/domain of articles. In general, differences between sentiment were determined based on consistency of statements that clearly identified group affiliation, such as encouraging vaccination and highlighting benefits (positive sentiment) or discouraging vaccination and highlighting risks (negative sentiment). Ar- ticles that were ambiguous or mixed in sentiment were coded as neutral. Three researchers (GJK, SRE, LM) independently coded a subset of 10 articles for sentiment; there was no inter-annotator variability and resulted in consistent sentiment coding.

3.3.3 Construction of vaccine sentiment networks

Methodological details describing the annotation, construction, and analysis of semantic networks can be found in Appendix A.2. Individual document text networks were merged by sentiment group, thereby aggregating similar documents into a single semantic network, one for each vaccine sentiment (positive, negative, and neutral). Node and edge labels were standardized for consistency by resolving lexical differences and grammatical dependencies across disparate sources.

3.3.4 Semantic network analysis

Our analysis of positive, negative, and neutral sentiment networks was focused on the greatest connected component (GC) (or subgraph). We utilized several measures of network analysis to the generated semantic networks in order to limit biased interpretation of individual network metrics [73]; descriptions of network measures are available in Appendix A.2.2. Descriptive statistics included network size, density, and diameter, where network size is the total number of nodes (i.e., vaccine concepts); density measures the interconnectedness of nodes [76]; and diameter characterizes compactness of the network. We evaluated multiple measures of centrality which describes the importance, influence, or significance of concepts within the semantic network in various ways [77]; specific types include degree centrality, betweenness centrality, closeness centrality, and eigenvector centrality [78]. Community detection algorithms [79] describe cohesive groups in the network [80], and clusters of important vaccine concepts were visualized by the network’s maximum k-core Gloria J. Kang 29

(the maximal connected subgraph in which all nodes have degree of at least k) [81]. We assessed differences in emphasis framing, which is the salience of certain story elements over others [82], for central concepts from networks of differing sentiment. Closeness vitality83 [ ] measures how much the distances between all pairs of nodes change when a particular node is removed. This is an indicator of how much each node contributes to the overall structural cohesion of the network. NetworkX [84] and iGraph [85] were used in network construction and network analysis. Graph visualizations were created in Gephi [86].

3.4 Results

3.4.1 Document characteristics

From the sample of webpages (n = 50), we coded 23 documents as having positive vaccine sentiment, 21 documents with negative vaccine sentiment, and 6 documents were classified as neutral. Table 3.1 summarizes document characteristics grouped by vaccine sentiment. Blog posts were the most shared document type overall, followed by news and ”alternative news” for positive and negative sentiment articles respectively. Content of positive sentiment documents focused on specific childhood, adolescent, and adult vaccines, whereas negative sentiment documents focused primarily on childhood vaccines and vaccination in general.

3.4.2 Document text networks

Negative sentiment documents (n = 21) formed the largest semantic networks with a mean network size of 90.9 concepts (nodes) per document, compared to smaller networks of positive sentiment (n = 23) and neutral sentiment documents (n = 6) with a mean of 51.3 and 43.8 concepts per document respectively. A complete listing of measured network properties for vaccine documents are summarized in Table 3.2.

3.4.3 Vaccine sentiment networks

Document text networks were aggregated by vaccine sentiment to form 3 semantic networks representing positive, negative, and neutral sentiment. In regards to the greatest connected component, size indicates the number of concepts in the network, whereas density describes interconnectedness of the concepts. The greatest component of the negative network was largest in size (1140 concepts) but less dense (0.0027) than the positive network (0.0061) which was also much smaller in size (585 concepts) (Table 3.2). Community detection analysis [79] identified 21 distinct communities within the positive network, 31 communities in 30 Chapter 3. the negative, and 10 communities in the neutral network. Compared to the original number of merged documents per sentiment network, the number of cohesive communities exceeded the number of original documents within the negative and neutral networks, whereas the positive network formed fewer communities than the original number of documents used in merging. Community findings and density measures for the positive network suggest a more cohesive and interconnected belief system among positive sentiment concepts compared to the larger, less-connected network of negative sentiment. Correspondingly, the average clustering coefficient (i.e., the tendency of nodes to form groups) and average node centrality for degree, betweenness, closeness, and eigenvector centrality were higher for the positive network compared to the negative. Positive and negative networks exhibited structural sim- ilarities in regards to diameter (12 and 13, respectively) and average path length (4.5 and 4.8, respectively). Graph visualizations of maximum k-core subgraphs in each sentiment network highlight clusters of significant concepts in Figure 3.1.

3.4.4 Central concepts

Excluding expected nodes such as vaccines and vaccination, the most central concepts for the positive network included parents, measles, children, SB 277, autism, community, religious groups, anti-vaccination, vaccine-autism link, HPV vaccine, meningococcal disease, and MMR vaccine. Significant concepts within the negative sentiment network were children, thimerosal, CDC, vaccine industry, mercury, autism, flu shots, mainstream media, doctors, SB 277, vaccine ingredients, mandatory vaccines, and pharmaceutical companies. And the most central concepts of the neutral network were SB 277, anti-vaccination, parents, children, pertussis vaccine, homeschool, education, pertussis, vaccine-autism link, side effects, Dwoskin Family Foundation, whole-cell vaccine, effective, acellular pertussis vaccine, and high-dose flu vaccine. Figure 3.2 plots significant concepts of each sentiment network by centrality measures for degree, betweenness, and closeness centrality; centrality values are available in Appendix A.4. The most central concepts (greater than 2 standard deviations from the mean) ranked by eigenvector centrality are plotted in Figure 3.3 and fully listed in Table 3.3.

3.4.5 Dynamic visualizations

Dynamic, interactive visualizations and network data files from this study are available in Appendix A.5. Gloria J. Kang 31

3.5 Discussion

3.5.1 Semantic network analysis of vaccine sentiment

A long line of research in the psychology of memory and semantic processing has provided evidence for semantic network-like organization of internal representations and spreading activation as a process by which memories are activated and meaning is processed [87, 88, 89, 90]. In this model, when an item in memory is activated, e.g., by a person reading about it or hearing about it, the activation spreads from that node in the person’s internal semantic network to nearby nodes. Spreading activation is also hypothesized as the model for the automatic activation of attitudes [91]. From this perspective, closeness centrality is a useful metric to understand the organization of the vaccination semantic networks (though other centrality measures are quite similar in ranking, as the results show). Closeness centrality is a direct measure of which concepts are likely to be activated repeatedly in each of the semantic networks, even as different concepts are mentioned. Many central concepts of the positive network were present in the negative network, but not vice versa. For example, while positive and neutral sentiment documents explicitly addressed the concept of anti-vaccination, negative sentiment articles did not. In regards to highly central concepts of the negative network, the positive network lacked any reference to the vaccine industry and mainstream media; CDC and doctors also held lesser significance in the context of positive vaccine sentiment. Significant concepts within the positive network were related to health and medicine, such as measles, autism, HPV vaccine, vaccine-autism link, meningococcal disease, and MMR vaccine. In contrast, significant concepts of the negative network referred to organizational bodies such as CDC, vaccine industry, doctors, mainstream media, pharmaceutical companies, and United States. A notable contrast was the emergence of parents as the most central concept in the positive network, versus children, the most central node in the negative network. Documents expressing positive and neutral vaccine sentiment were characterized by dense semantic networks with fewer concepts, compared to the semantic network of negative sentiment which presented a high number of vaccine concepts with low connectivity. Compared to the positive sentiment network, the negative sentiment network has more components, lower edge density, a larger diameter, and larger average path length (Table 3.2). Hence, positive sentiment documents indicated greater cohesiveness in vaccine-positive discourse compared to vaccine-negative documents which addressed a broad range of topics as potential contributors to vaccine hesitancy. 32 Chapter 3.

3.5.2 Message framing

Our study revealed sentiment-specific terminology used in framing positive and negative messages within vaccine communication. This included differences in term valence such as required vaccines versus mandated vaccines and side effects versus adverse effects, the selective targeting of parents versus children, and the overall presentation of evidence-based science versus social commentary related to issues of governance for the positive and negative vaccine sentiment networks, respectively. Overall, the prevalence of negative vaccine sentiment was demonstrated through diverse messaging, framed around institutional distrust and skepticism towards the organizations that deliver scientific evidence of positive vaccine benefits. This is also shown by thelist of top nodes for the closeness vitality measure for each network (Appendix A.7) which is an indicator of the concepts which are responsible for providing structural cohesion to the semantic network [92]. Positive and negative vaccine articles largely differed in the framing of trust. Positive articles emphasized trust in vaccination by relying on scientific evidence as trusted authority. Negative articles framed trust issues not around vaccination science itself, but around the institutions that govern or finance matters of personal health. Neutral vaccine articles exem- plified various sources of news coverage that expressed a mix of both positive and negative attitudes toward vaccines. Top news stories at the time of data collection included a new study debunking the vaccine-autism link and the passing of California Senate Bill 277 [93], which removed exemptions from school vaccination requirements. News coverage generally expressed positive vaccine sentiment, reporting official statements and statistics. In contrast, news coverage by negative vaccine articles additionally introduced a range of tangential topics, often proposing arguments through rhetorical questions and re-framing official statistics.

3.5.3 Limitations

We assumed that popular vaccination information shared on Twitter is representative of prevalent vaccine sentiment, but may not reflect the broad spectrum of vaccine sentiment in the general population. Coding documents for neutral sentiment was difficult since documents presented a mix of both positive and negative attitudes, and not truly vaccine-neutral. Because health behaviors are founded upon a variety of beliefs and attitudes that change over time, vaccine sentiment categories are difficult to delineate since they do not exist as polarized groups. While we attempted to resolve issues of meaning and context by manually transcribing implicit statements into explicit statements, reference resolution grew increasingly difficult across different documents. Consequently, there is potential inconsistency from the manual annotation of document text into network data, particularly when dealing with ambiguous language such as slang, hyperbole, and poetic devices. Despite these limitations, employing Gloria J. Kang 33 human interpretation of text greatly enhances qualitative aspects of data and is arguably more accurate than current NLP methods which lack explicit domain-specific knowledge or situational information [70]. Lastly, our analysis did not assess the qualitative relationships of connected concepts. Future studies incorporating edge data can provide detailed insight into the comparison of belief structures of varying vaccine sentiment. Our study presents only a broad overview of general network measures. Greater depth into specific metrics, such as community detection analysis, can provide useful insight andshould be addressed in future studies.

3.5.4 Implications for public health and vaccine communication

The SAGE Working Group on Vaccine Hesitancy [74] states that communication is a tool to address vaccine sentiment rather than a determinant of hesitancy. However, poor communication can undermine vaccine acceptance in any setting [83]. Our study lends itself to the development of effective communication strategies for target populations. Identifying specific factors that influence vaccine hesitancy should be an integral component ofevery immunization program. Semantic network analysis of vaccine sentiment in online social media can enhance our understanding of the scope and variability of attitudes and beliefs toward vaccination. Our findings emphasize the need to improve the framing and messaging of public healthcom- munication, that not only highlights the vaccine benefits, but also addresses specific issues related to vaccine hesitancy and institutional distrust. Enhancing public trust in relevant scientific institutions and engaging in efficient public health communication is criticalin improving vaccine confidence and vaccination coverage94 [ ].

3.5.5 Conclusion

We discussed findings from a novel framework that uses semantic network analysis asan efficient and effective way to analyze vaccine sentiment. This study adds to a growing body of vaccine hesitancy research by investigating emerging topics and the various discourse surrounding current vaccine perspectives. Findings related to significant concepts, the structure of its relations, and semantic qualities can better inform targeted vaccine communication strategies and enhance effectiveness of public health efforts to increase vaccine confidence.

Funding

This study is supported by NIH/NIGMS R01GM109718, NSF/NRT 1545362, and NSF IBSS Grant SMA-1520359. The funding sources had no role in study design; collection, analysis, 34 Chapter 3. and interpretation of data; writing of the paper; or the decision to submit it for publication. Gloria J. Kang 35

Table 3.1: Summary of sampled vaccine web pages (documents). The table summarizes article characteristics by vaccine sentiment group and describes document type, article source, target vaccine population, vaccine type focus, and specific vaccine topics.

Vaccine documents (Total Positive (n = 23) Negative (n = 21) Neutral (n = 6) n = 50)

Document type Blog: 8 (34.8%) Blog: 15 (71.4%) News: 4 (66.7%) News: 7 (30.4%) Alternative News: 2 (9.5%) Blog: 1 (16.7%) Magazine: 5 (21.7%) Magazine: 2 (9.5%) Magazine: 1 (16.7%) Information: 3 (13.0%) Commercial: 1 (4.8%) News: 1 (4.8%) Article source type Media: 9 (39.1%) Media: 15 (71.4%) Gov’t: 2 (33.3%) Gov’t: 8 (34.8%) Industry: 3 (14.3%) Media: 2 (33.3%) News: 4 (17.4%) Personal: 2 (9.5%) News: 2 (33.3%) Industry: 1 (4.4%) Forum: 1 (4.8%) Resource: 1 (4.4%) Target vaccine population Child: 16 (69.6%) Child: 15 (71.4%) Child: 3 (50.0%) Adolescent: 3 (13.0%) Adolescent: 0 Adolescent: 2 (33.3%) Adult: 1 (4.4%) Adult: 0 Adult: 1 (16.7%) Multiple: 3 (13.0%) Multiple: 6 (28.6%) Multiple: 0 Vaccine type focus General: 8 (34.8%) General: 14 (66.7%) General: 3 (50%) Specific: 15 (65.2%) Specific: 7 (33.3%) Specific: 3 (50%) Specific vaccines Measles/MMR: 9 Shingles: 1 Whooping cough: 2 HPV: 3 Polio: 1 Influenza: 1 Influenza: 1 Gardasil: 1 Meningococcal: 1 Measles: 1 Rubella: 1 Swine flu: 1 Tdap: 1 Hepatitis B: 1 36 Chapter 3.

Figure 3.1: Maximum k-core subgraphs of semantic networks by vaccine sentiment. Visualizations of maximum k-cores (i.e., the maximal connected subgraph in which all nodes have degree of at least k) for networks of (a) positive vaccine sentiment (k = 4), (b) negative vaccine sentiment (k = 4), and (c) neutral vaccine sentiment (k = 2) where increasing node and text size represents increasing betweenness centrality.

(a) Positive sentiment network: Max k-core subgraph (k = 4) Gloria J. Kang 37

(b) Negative sentiment network: Max k-core subgraph (k = 4) 38 Chapter 3.

Figure 3.2: Significant vaccine concepts by measures of degree centrality, betweenness centrality, and closeness centrality. The figure includes centrality measures for significant concepts from positive, negative, and neutral sentiment networks. Degree centrality (point size), betweenness centrality (x-axis), and closeness centrality (y-axis) are plotted. 40 Chapter 3.

Figure 3.3: Significant concepts ranked by eigenvector centrality. The figure plots the most central nodes by eigenvector centrality score for networks of positive, negative, and neutral vaccine sentiment. Gloria J. Kang 41

Table 3.2: Summary of measures for article text networks and sentiment group networks. The table describes network characteristics of extracted web documents; joint semantic networks of positive, negative, and neutral vaccine sentiment; and the correspond- ing greatest connected component. Measures describe network size, density, and average centrality.

Vaccine sentiment Positive Negative Neutral Document text networks Number of documents (Total=50) 23 documents 21 documents 6 documents Average number of nodes (per document) 53.1 nodes 90.9 nodes 43.8 nodes Average number of edges (per document) 49 edges 90.7 edges 39.7 edges Average degree (per document) 1.9 1.98 1.8 Vaccine sentiment networks Average degree 3.356 2.95 2.348 Number of connected components 21 49 12 Greatest component subgraph Nodes / Total network nodes 585/652 nodes 1140/1257 nodes 171/201 nodes Edges / Total network edges 1042/1094 edges 1783/1854 edges 216/236 edges Average degree 3.562 3.128 2.526 Diameter 12 13 17 Density 0.0061 0.0027 0.0149 Number of communities 21 31 10 Average path length 4.492 4.77 6.78 Average degree centrality 0.0061 0.0027 0.0149 Average betweenness centrality 0.006 0.0033 0.0342 Average closeness centrality 0.2292 0.2161 0.1533 Average node connectivity 1.3117 1.1835 1.035 Average clustering coefficient 0.196 0.14 0.131 42 Chapter 3.

Table 3.3: Nodes by eigenvector centrality. The table lists the most central concepts of each sentiment network ranked by eigenvector centrality score (greater than 2 standard deviations from the mean).

Eigenvector centrality Positive sentiment network Negative sentiment network Neutral sentiment network Mean = 0.0626, Std Dev = 0.0936 Mean = 0.0318, Std Dev = 0.06 Mean = 0.0975, Std Dev = 0.11 parents 1 vaccines 1 SB 277 1 vaccines 0.8209 children 0.6188 vaccines 0.4304 measles 0.7458 thimerosal 0.5248 anti-vaccination 0.4177 vaccination 0.6373 CDC 0.5054 parents 0.3863 children 0.5382 vaccine industry 0.4898 children 0.383 SB 277 0.4207 mercury 0.444 pertussis vaccine 0.354 autism 0.4025 autism 0.3894 home-school 0.3209 community 0.3937 flu shots 0.3367 education 0.3206 religious groups 0.3905 mainstream media 0.3342 anti-vaccination 0.3802 doctors 0.2862 vaccine-autism link 0.3608 SB 277 0.2659 herd immunity 0.3058 vaccine ingredients 0.2632 vaccine refusal 0.3024 mandatory vaccines 0.2457 vaccination 0.3013 pharmaceutical 0.24 exemption companies personal belief 0.2909 vaccine-autism link 0.2041 exemption disease 0.2829 toxic chemical 0.1999 ingredients measles vaccine 0.2706 aluminum 0.1889 schools 0.2685 vaccination 0.1853 HPV vaccine 0.2674 monosodium glutamate 0.1811 vaccine delay 0.2603 hepatitis B vaccine 0.1793 meningococcal 0.2551 vaccine-injured 0.1763 disease children vaccine safety 0.1721 evidence 0.1655 informed consent 0.1643 intelligent questions 0.1612 formaldehyde 0.1609 pregnant women 0.1598 pandemic H1N1 swine 0.1595 flu vaccine Big Pharma 0.1591 vaccines are safe 0.1565 quackery 0.1552 vaccine damage 0.1547 SV40 0.1545 science 0.1531 Chapter 4

Health Benefits, Costs, and Cost-Effectiveness of Influenza Vaccination in Seattle

Attribution

This chapter is based on the following manuscript (in preparation for journal submission): Gloria J. Kang, Bryan Lewis, Achla Marathe, and Kaja Abbas. Health benefits, costs, and cost-effectiveness of influenza vaccination in Seattle.

43 44 Chapter 4.

4.1 Abstract

Influenza vaccination coverage for persons 6 months and older in the U.S. has beenaround 41.8%-47.1% during the 2010-2017 influenza seasons, remaining below Healthy People 2020’s target goal of 70% influenza vaccination coverage. The objective of this study was to estimate the health benefits, costs, and cost-effectiveness of influenza vaccination using Seattleasa case study. We simulated socio-behavioral interactions and influenza transmission dynamics using agent- based modeling on a synthetic social contact network of Seattle. We simulated three scenarios representing no vaccination, influenza vaccination rates similar to current coverage, and the influenza vaccination target goal of the Healthy People 2020 initiative. Weesti- mated the health impact of vaccination on influenza cases, deaths, and disability-adjusted life years averted per 100,000 population. We then assessed the costs and cost-effectiveness of vaccination scenarios accounting for both direct and indirect (herd-protection) effects from vaccination. We found that in the current vaccination scenario assuming vaccine efficacy of 40%, vaccination averted an average of 13,096 cases, 30 deaths, and 967 disability-adjusted life years (DALYs) (per 100,000 population) compared to the no vaccination scenario. The clinical cost of influenza illness was $630,203,357 with no vaccination, compared to $286,813,908 with vaccination at current coverage. While the budget impact of influenza vaccination was $39,273,077, vaccination at current coverage was a cost-saving intervention with savings of $304,116,367. Additionally, in the Healthy People 2020 scenario assuming vaccine efficacy of 20%, vaccination averted an average of 4,293 cases, 10 deaths, and 328DALYs (per 100,000 population) compared to the current vaccination scenario. Clinical costs totaled $349,311,511 with a budget impact of $68,253,354, resulting in total cost savings of $82,523,757 in the Healthy People 2020 scenario compared to current vaccination. The indirect benefit gained in health outcomes and cost savings by blocking influenza transmission by vaccinated individuals to susceptible individuals was larger than the direct benefit among vaccinated individuals. Based on our vaccination scenarios, efficacy assumptions, and inclusion of herd-protection effects, we found that influenza vaccination is a cost-saving intervention under current levels of vaccination coverage as well as Healthy People 2020’s target coverage goal when assuming vaccine efficacy of at least 20%. Study findings demonstrate the potential for influenza vaccination programs to be cost-saving strategies that provide good value in improved health and use of public health resources in urban areas such as Seattle. Gloria J. Kang 45

4.2 Introduction

4.2.1 Background

For the 2010-2015 influenza seasons, influenza-associated illnesses ranged from 9.2-35.6 million, with an estimated 4.3-16.7 million outpatient medical visits, 139,000-708,000 hospitalizations, and 12,000-56,000 respiratory and circulatory deaths due to influenza [95]. Based on the 2003 US population, Molinari et al. estimated an annual average of $10.4 billion (95% CI, $4.1-$22.2 billion) in direct medical costs due to seasonal influenza (assuming 10 million outpatient visits, 300,000 hospitalizations, and 41,000 deaths) [96]. More recently, Putri et al. estimated that for the 2015 US population the economic burden to the healthcare system averaged $3.2 billion ($1.5-$11.7 billion), much lower than Molinari et al.’s previous estimate [97]. The incidence of symptomatic influenza among vaccinated and unvaccinated individuals, including both medically attended and unattended infections, is estimated to be around 8% in the U.S. with seasonal variation ranging around 3-11% [98]. In 2009, the Centers for Disease Control and Prevention’s Advisory Committee on Immunization Practices (ACIP) expanded recommendations for seasonal influenza vaccination to include individuals aged 6 months and older [99], and Healthy People 2020’s current target goal for influenza vaccination coverage is 70% [100]. However, overall vaccination coverage in the U.S. has maintained levels around 45% for influenza seasons 2010-11 through 2016-17101 [ ]. Cost-effectiveness studies in public health can help inform decision-makers on the intervention strategies that are expected to provide the greatest gains in population health for a given expenditure in health resources [102]. A recent review found that influenza vaccination of high-risk adults is highly cost-effective, whereas vaccination of healthy adults varies greatly—as cost-effectiveness is sensitive to compliance, vaccine efficacy, and productivity loss, among others [103]. Although seasonal influenza vaccination reduces influenza-related mortality and morbidity through both direct and indirect effects of vaccine protection [104], cost-effectiveness studies often do not account for indirect protection conferred through herd immunity which results in underestimations of cost-effectiveness103 [ ]. Previous studies have examined the cost-effectiveness of influenza vaccination for specific age and risk groups separately, such as in school-aged children [105, 106], pregnant women [107], and the elderly [108, 109], with a majority of studies based on static models [110]. To address this, our study conducted a cost-effectiveness analysis of influenza vaccination, using an agent-based dynamic transmission model for different age and risk groups in Seattle, enabling us to measure both the direct and indirect effects of vaccination. 46 Chapter 4.

4.2.2 Study objectives

This analysis evaluates the clinical costs, health outcomes, and cost-effectiveness of vaccination strategies for mitigating seasonal influenza in Seattle, WA. Specifically, we examine 4 scenarios (2 comparator and 2 intervention scenarios) with differing assumptions of vaccination coverage and efficacy: Scenario 1.) No vaccination (comparator); Scenario 2.) Current vaccination assuming 40% vaccine efficacy—which reflects current practice of influenza vaccination in the U.S. in terms of coverage and average vaccine efficacy; Scenario 3.) Current vaccination assuming 20% vaccine efficacy (comparator)—which represents current coverage levels assuming a low efficacy vaccine; and Scenario 4.) Healthy People 2020 vaccination— which represents Healthy People 2020’s target goal of 70% vaccination coverage for seasonal influenza. We compared Scenario 1 with Scenario 2, and Scenario 3 with Scenario 4by conducting a cost-effectiveness analysis using a highly-detailed agent-based model ofSeat- tle’s population that encompasses influenza disease dynamics, individual-level behaviors, and realistic social contacts.

4.2.3 Public health significance

Current influenza vaccination levels remain below Healthy People 2020’s target goal of70% coverage in children and adults aged 6 months and older (which uses the 2010-11 influenza season as the comparative baseline in which 46.9% of children aged 6 months-17 years and 38.1% of adults aged 18 years and older were vaccinated) [100]. At the time of this study, the 2017-18 influenza season was particularly severe with pending estimations of vaccine efficacy expected to be relatively low[111]. Dynamic modeling of the health and economic impacts of influenza will become increasingly important as population demographics shift over time and space, altering contact network structure, and hence affecting disease transmission dynamics. The outcomes of this study can provide valuable insight to policy-makers and budget managers of health care systems to better understand the intervention costs and health outcomes of implementing large-scale vaccination programs for urban populations such as Seattle.

4.3 Methods

We assessed the impact of age-specific influenza vaccination strategies using a dynamic model to simulate seasonal influenza transmission in Seattle. Outcomes from the epidemic model are expressed as number of infections which are then used as inputs in a decision tree to estimate health outcomes. Outcome based medical costs were multiplied by the number of cases under each health outcome in order to calculate the total costs and the cost-effectiveness of intervention. Data was derived from best available sources when possible and are referenced as noted. Results are expressed in clinical cases averted, deaths averted, disability-adjusted life Gloria J. Kang 47 years (DALYs) averted, cost of illness, cost of vaccination, and incremental cost-effectiveness ratios (ICERs).

4.3.1 Simulation model

We simulated seasonal influenza by using an agent-based model which tracks influenza infections, vaccination, and disease transmission dynamics across a synthetic social contact network of Seattle [112, 113, 114]. Model parameters are detailed in Table 4.1. We used the mean output of 25 simulation runs for a duration of 365 days to obtain age-specific clinical attack rates.

Population

We considered the population of Seattle (3,406,876) divided into four age groups: 0 to 4 years (223,608), 5 to 19 years (639,661), 20 to 64 years (2,235,049), and 65+ years (308,558) [115]. Influenza infections within each age group were further divided into two risk groups: high risk and non-high risk (Table 4.1), where high risk denotes individuals with preexisting medical conditions that increase their susceptibility to influenza-related complications [116].

Disease model

We represent the natural course of disease with 4 commonly used disease states: susceptible, exposed, infectious, recovered. Individuals begin in a susceptible state until they become exposed. The exposed state lasts for the duration of the latent period, at the end of which the exposed individual becomes infectious. During the infectious period, an infectious individual probabilistically infects its susceptible contacts based on the transmission rate, defined as the probability of transmission per minute of contact with a symptomatic infectious individual. After the infectious period, an infectious individual becomes recovered (or removed). We adjusted the transmission probability per unit time to achieve an overall attack rate of (seasonal influenza) under current vaccination scenario conditions. This resulted ina transmission probability of 0.006 per hour of contact time (0.000105 per minute).

Vaccination

We examined 3 different vaccination coverage levels including no vaccination (scenario 1), current vaccination (scenario 2 and 3), and Healthy People 2020 vaccination of 70% coverage (scenario 4). Coverage levels in the current vaccination scenario were based on CDC estimates of seasonal influenza vaccination coverage in the U.S. [101, 117]. We selected the lowest coverage level reported for each age group for seasons 2010-11 through 2016-17 to 48 Chapter 4.

simulate realistic compliance levels reflecting current practice. We assumed 51% vaccine compliance in 0-19 year olds, 33% in 20-64 year olds, and 63% in 65+ year olds. Based on average efficacy estimates for influenza vaccination, vaccine efficacy was set to40%inthe current vaccination scenario [118].

4.3.2 Direct and indirect effects of vaccination

Dynamic models enables us to capture indirect effects of vaccination beyond the index population targeted by intervention, simulating herd protection afforded by the structure of the underlying contact network. The contact chain in the social network allows us to measure the indirect protection provided by vaccination as it blocks transmission to secondary, tertiary and further contacts. Vaccination was modeled assuming all-or-nothing vaccination in which the proportion of the population directly protected by vaccination is equal to vaccine efficacy × vaccine compliance. Using simulation results, we examined the relative contribution of direct and indirect effects from vaccination by comparing the number of cases directly averted by vaccination to the total number of cases averted.

4.3.3 Clinical costs and health outcomes

Our analysis takes a clinical perspective and measures costs related to vaccination and direct medical expenses related to influenza illness. All costs are drawn from the literature and adjusted to US 2018$ values using the medical cost component of the U.S. Consumer Price Index [119]. Cost and health outcome parameters are listed in Table 3.2. We used a decision tree to determine influenza-related health outcomes which include death, hospitalization, outpatient, and ill but not seeking care [116]. Associated costs related to hospital admissions, outpatient visits, prescription medication, and over-the-counter drug costs were adapted from Carias et al. [120]; decision tree probability of health outcomes per age and risk group were adapted from Meltzer et al. [116]. Effectiveness was measured in net costs or savings in health care services, and the number of influenza-related cases, deaths, and DALYs averted per 100,000 population from vaccination. The cost of seasonal influenza vaccination varies widely depending on setting, ranging from $14 in pharmacies to $63.80 in family-size clinics [96]. For this study we used a vaccine cost of $28.62 [121] which includes costs associated with vaccine administration and vaccine adverse effects. With the issue of rising health care costs, health care system authorities have started to require a budget impact analysis when conducting effectiveness and cost- effectiveness studies [122]. A budget impact analysis takes the perspective of the budget manager to provide an understanding of the total budget required to fund the intervention. To determine the budget impact of influenza vaccination, the unit cost of vaccination was multiplied by the number of individuals vaccinated in the population. Gloria J. Kang 49

4.3.4 Cost-effectiveness

When comparing vaccination scenarios, incremental cost-effectiveness ratios (ICERs) capture differences in cost, morbidity, and mortality, and typically demonstrate costs spent orsaved per unit gain in effectiveness. We calculated ICERs per case averted, per death averted, and per DALY averted using the following formula: (clinical costs in base scenario) − (clinical costs in new scenario + vaccination costs) health outcomes averted in which the ICER value indicates the dollar amount per health outcome averted. Disability-adjusted life years (DALYs) measure the years of healthy life lost due to disease where 1 DALY equals 1 healthy life-year lost. DALYs are the sum of years of life lost (YLL) due to premature death from disease and the years of life lost due to disability (YLD) [123], which are calculated using the following equations: YLL = number of deaths * remaining life expectancy per age group [124] YLD = number of cases * average duration of influenza [125] * disability weight [126].

4.3.5 Sensitivity analysis

We performed a univariate sensitivity analysis by varying vaccine efficacy across intervention scenarios for measurable effects on epidemic and economic outcomes. Epidemic curves for all scenarios are provided in Figure 4.1. It is important to note that at efficacies of 50% and higher in the current vaccination scenario, the epidemic was unable to complete within the simulation duration of 1 year and was excluded from this analysis. Similarly in the Healthy People 2020 scenario, vaccinating 70% of the population was only able to sustain epidemics at lower vaccine efficacies of 10% and 20%. At 30% and 40% vaccine efficacy, the epidemic failed to complete within the simulation duration of 1 year, and vaccine efficacy of 50% and 60% produced an R0<1, failing to produce an epidemic entirely. Hence, our analysis reports findings assuming vaccine efficacy of 10%, 20%, 30%, and 40% in thecurrent vaccination scenario and assuming vaccine efficacy of 10% and 20% in the Healthy People 2020 vaccination scenario.

4.4 Results

4.4.1 No vaccination scenario

In the no vaccination scenario, the symptomatic attack rate of influenza was 24.5% with a total of 836,366 influenza cases. Per 100,000 population, no vaccination resulted in24,549 50 Chapter 4. cases, 54 deaths, and 1,793 DALYs from influenza illness. Clinical costs from influenza totaled $630,203,351.

4.4.2 Current vaccination scenario

We describe the comparison between the no vaccination scenario and the current vaccination scenario assuming 40% vaccine efficacy.

Health Outcomes

Current vaccination coverage (assuming 40% vaccine efficacy) reduced the no vaccination scenario’s attack rate from 36.6% to 17.1% (n = 582,374); this includes both symptomatic (67%) and asymptomatic (33%) infections, resulting in a symptomatic attack rate of 11.4% (n = 390,191). The current vaccination scenario averted a total of 446,176 cases. Per 100,000 population, current vaccination averted 13,096 cases, 30 deaths, and 967 DALYs when compared with the no vaccination scenario (Table 4.3; Figure 4.2). The greatest number of cases were averted within the 5-19 age group (23,368 per 100K) and the non-high risk group (13,441 per 100K), while the greatest number of deaths and DALYs were averted within the 65+ age group (137 deaths and 1,968 DALYs averted per 100K) and within the high risk group (167 deaths and 5,215 DALYs averted per 100K). High risk individuals within the 5-19 age group accounted for the most number of DALYs overall with 9,085 DALYs per 100K (averting 10,305 DALYs per 100K compared with no vaccination); this is partly due to the greater number of life-years lost due to premature mortality from influenza in the younger age group compared to older groups.

Costs

The cost of influenza illness totaled $286,813,908 in the current vaccination scenario witha budget impact of $39,273,077 in vaccination costs (Table 4.4), resulting in cost savings of $304,116,367 when compared to the cost of illness from the no vaccination scenario.

Cost-Effectiveness

Incremental cost-effectiveness ratios were calculated per case, death, and DALY averted for each age and risk group (Figure 4.3 and Table 4.5). We found that the current vaccination scenario saves $682 per case averted, $296,085 per death averted, and $9,233 per DALY averted in the whole population compared with the no vaccination scenario. The elderly age group and high risk group were associated the greatest cost savings per case averted; this is partly due to the relatively lower number of cases averted among 65+ year olds and high risk Gloria J. Kang 51 individuals. In contrast, those in the 5-19 age group and those at non-high risk produced the greatest cost savings per death averted; savings are lowest in the 65+ age group and high risk group partly due to the higher number of deaths averted compared to other age and risk groups. In regards to ICER per DALY averted, cost savings were greatest in the 65+ age group and for non-high risk individuals. With respect to the greatest dollar amount saved per unit of health effect (across all age and risk groups), current vaccination saved $5,061 per case averted in high risk elderly; $1,906,242 per death averted among non-high risk 5-19 year olds; and $27,219 per DALY averted among non-high risk 20-64 year olds.

Direct and Indirect Effects of Vaccination

Current vaccination averted about 53% of infections incurred in the no vaccination scenario. Out of all cases averted, more cases were averted due to indirect effects (herd-protection) from vaccination (68.7%) than direct effects (31.32%) (Table 4.6). Compared to other age groups, the cases averted among the 65+ age group were attributed to indirect effects the least (58.5%) and from direct effects (41.5%) the most.

4.4.3 Healthy People 2020 vaccination scenario

We additionally examined the costs and health outcomes of Healthy People 2020’s target goal of 70% vaccination coverage compared with the current vaccination scenario, assuming vaccine efficacy of 20%. The purpose of this secondary analysis was to describe scenarios under potentially more realistic assumptions regarding current vaccination practice as well as lowered expectations for future vaccine efficacy. In summary, the clinical costs from influenza illness in the Healthy People 2020 scenario totaled $349,311,511 with a budget impact of $68,253,354 in vaccination costs (Table 4.4); this resulted in cost savings of $82,523,757 when compared with the current vaccination scenario assuming 20% vaccine efficacy. Per 100,000 population, the Healthy People 2020 scenario averted 4,293 cases, 10 deaths, and 328 DALYs due to influenza, with incremental cost-effectiveness ratios of $564 saved per case averted, $246,524 saved per death averted, and $7,384 saved per DALY averted. Detailed results for the Healthy People 2020 scenario are available in Appendix B.1.

4.4.4 Sensitivity Analysis

We performed a univariate sensitivity analysis varying vaccine efficacy—10%, 20%, 30%, and 40% in the current vaccination scenario and vaccine efficacy of 10% and 20% in the Healthy People 2020 vaccination scenario (Appendix B.2). ICERs appeared to be sensitive to low vaccine efficacy (10%). At vaccine efficacy of 10%, vaccination was no longer cost savingfor 52 Chapter 4.

the 0-4 age group; this may be due to weaker herd protection effects from a lower efficacy vaccine.

4.5 Discussion

The vaccination scenarios considered in this analysis were found to be cost-saving strategies for all age and risk groups when assuming vaccine efficacy of at least 20%. Cost-effectiveness planes depicting incremental costs and outcomes for the current scenario compared with no vaccination are shown in Figure 4.4 (and for Healthy People 2020 scenario compared with current vaccination in Appendix B.3). Incremental health outcomes are plotted along the x-axis as DALYs averted per 100,000 population with incremental costs along the y-axis as dollars per 100,000 population. This illustrates relative differences in costs and outcomes for each age and risk group across scenarios (hence, the slope from the origin = calculated ICER), as well as sensitivity to vaccine efficacy. In the case of cost-saving programs, cost-effectiveness planes may provide useful information to health policy-makers by illustrating the comparative health effects of vaccination for different age and risk groups (as denoted by DALYs averted per 100K) and across scenarios. For example, the cost-effectiveness plane for the current vaccination scenario (Figure 4.4) indicates the high risk, 5-19 age group as incurring the greatest health benefit from vaccination, followed by the high risk, 0-4 age group. In contrast, the ICER plot (Figure 4.4) shows the greatest cost savings are attributed to DALYs averted in the non-high risk, 20-64 age group. Such insight comparing differential gains in health and cost for age and risk groups within a population may help inform targeted vaccination plans based on relevant criteria for prioritizing vaccine distribution.

4.5.1 Public health implications

Given the uncertainty and sensitivity of seasonal influenza severity and vaccine efficacy, optimizing for age-specific vaccination (as opposed to attempting population-wide increases in coverage) may improve the effectiveness of vaccination programs by allocating health resources based on the greatest benefits to population health. Furthermore, the re-evaluation of “high risk” criteria will be critical when considering universal vaccination, particularly for regions with targeted programs [103]. Because potential cost savings afforded by such strategies would allow for budgetary redistribution to other public health needs, a targeted approach may help achieve overall program feasibility when balancing multiple health objectives, such as the case for Healthy People 2020 initiatives. Other studies have examined the prioritization and optimization of vaccine distribution in terms of maximizing health benefits and returns on investment [127, 128]. However, such decisions will ultimately be left up to policy-makers who will inevitably face additional chal- Gloria J. Kang 53 lenges involving the fairness and ethical considerations of programs in question. Simulation and model-based studies can help to inform those decisions in ways that may limit unintended consequences and maximize health impact, particularly in the face of resource limitations and outbreak situations that will require rapid response from decision-makers.

4.5.2 Limitations

The results of cost-effectiveness studies are affected by the perspective, programmatic design, setting, and inclusion of herd immunity [103]. Demographics and contact network structure of a population strongly influences the age-dependent incidence of influenza and vaccination effectiveness [129], thus limiting the generalizability of our results to regions dissimilar to the population of Seattle. Our simulation model did not include environmental drivers (such as temperature, humidity, rainfall and wind speed) which influence influenza seasonality and transmission dynamics. Because of this, epidemic curves were less likely to complete within the simulated duration (of a single season) at higher levels of vaccination coverage and/or efficacy. However, the impact of low-efficacy influenza vaccines on epidemic outcomesmay become more critical and more relevant for economic evaluation, warranting further study [128]. Additionally, we only considered vaccination as the sole intervention in the population and did not account for other preventative behaviors that may provide additional protection against influenza transmission (such as hand washing, social distancing, and antiviral use). The assumed distribution of high and non-high risk within a population affects the total cost of influenza illness. Past risk estimates may reflect relatively lower levels of vaccination coverage at that time and may overestimate the burden of influenza-related complications and associated costs when adjusting to the present study. This may explain our relatively higher estimates of influenza illness costs compared to recently updated literature values [97]. Although our findings may be conservative in regards to estimated costs of illness, our analysis still suggests that influenza vaccination may be a cost-saving intervention when considering regions like Seattle. Despite these limitations, our findings are nevertheless indicative of the utility in dynamic modelling for assessing the costs and health outcomes of seasonal influenza vaccination— highlighting the importance of leveraging our understanding of contact network effects on both the health and financial aspects of infectious disease transmission across large populations. Future work should additionally undertake a societal perspective, as well as including both pharmaceutical and non-pharmaceutical interventions which differ in both cost and efficacy. 54 Chapter 4.

4.5.3 Conclusions

Our study aims to improve understanding of vaccination program costs and health outcomes for seasonal influenza in a large urban population such as Seattle. Findings from our analysis help to illustrate potential budget impacts as well as potential cost savings from influenza vaccination while accounting for both direct and indirect protective effects. Differences in clinical costs, health outcomes, and cost-effectiveness of vaccination across sub-populations may be explained by individual behavioral heterogeneities that arise from differential contact patterns of age groups. Indirect benefit gained in health outcomes and cost savings by blocking influenza transmission by vaccinated individuals to susceptible individuals was larger than the direct benefit among vaccinated individuals. Estimations accounting forin- direct protective effects further demonstrate the significance of network effects on influenza transmission dynamics, providing greater insight on the potential for savings in terms of health and cost for a given population. Understanding both direct and indirect effects of vaccination should encourage sustainable vaccination programs that maintain consistently aggressive efforts needed for herd-immunity. Such findings can help to better understand cost inputs, health outcomes, and reveal potential cost savings that can be put toward other health programs. In this way, we hope to aid policy-makers to utilize more objective approaches in decision-making, in re-examining their understanding of budgetary constraints, and re-appraising them in terms of health potential.

Funding

This study is supported by National Institutes of Health grant no. 1R01GM109718, NIH Models of Infectious Disease Agent Study (MIDAS) grant no. 2U01GM070694-11 and 3U01FM070694-09S1, National Science Foundation grant no. NRT-DESE-154362, and De- fense Threat Reduction Agency grant no. HDTRA1-11-1-0016. The funding sources had no role in study design; collection, analysis, and interpretation of data; writing of the paper; or the decision to submit it for publication. Gloria J. Kang 55

Table 4.1: Model parameters. Simulation model parameters and variables describing population, disease dynamics, and vaccination.

Parameter Value Source Population of Seattle 3,406,876 [115] 0-4 yrs (Pre-school) = 223,608 (6.56%) 5-19 yrs (School-age) = 639,661 (18.7%) Age groups [115] 20-64 yrs (Adults) = 2,235,049 (65.6%) 65 yrs (Seniors) = 308,558 (9.05%) 0-4 yrs = 0.064 5-19 yrs = 0.064 High risk proportion [96, 116] 20-64 yrs = 0.144 65+ yrs = 0.512 No vaccination = 24.5% Illness attack rates by Current scenario (40% VE) = 11.4% Simulation scenario Healthy People scenario (20% VE) = 13.9% calibration Seeding of infected 5 random infections daily Simulation individuals calibration Probability of 0.000105 Simulation transmission per minute of calibration contact with a symptomatic infectious person Latent period 1 days (sd: 0.6325) [130] Infectious period 2 days (sd: 1.0583) [130] Serial interval 2.8 days [131] Asymptomatic 0.33 Assumed probability: Probability of [132, 133, 134] transmission per minute of contact with an asymptomatic infectious person in comparison to a symptomatic infectious person Symptomatic proportion: 0.67 Assumed [135] Proportion of symptomatic infection in influenza infected individuals 56 Chapter 4.

Table 4.1 continued from previous page Parameter Value Source

Influenza-related health Death, hospitalization, outpatient, ill but not [116] outcomes seeking care Current vaccination scenario: 0-4 years = 51% Vaccine compliance 5-19 years = 51% [101] 20-64 years = 33% 65+ years = 63% Healthy People 2020 scenario: 0-4 years = 70% 5-19 years = 70% [100] 20-64 years = 70% 65+ years = 70% Current vaccination scenario: 10, 20, 30, 40% Vaccine efficacy Healthy People 2020 scenario: 10, 20% Assumed [118] (Sensitivity analysis: 10–60%) Cost of influenza vaccine $28.62 [121] Gloria J. Kang 57

Table 4.2: Influenza-related outcomes, probability, and clinical costs of illness. Probability distributions of influenza-related health outcomes including death, hospitalization, and outpatient for each age and risk group (adapted from [116]).

High risk Non-high risk Age Lower Likeliest Upper Lower Likeliest Upper Probability group distribution Death 0-4 0.00036 0.0006 0.01821 0.00004 0.00007 0.0003 Triangular 5-19 0.00036 0.0006 0.01821 0.00004 0.00007 0.0003 Triangular 20-64 0.00083 - 0.02487 0.00021 0.00031 0.00039 Uniform 65+ 0.023 - 0.02963 0.00233 0.00351 0.00284 Uniform Hospital. 0-4 0.006 - 0.02143 0.00057 - 0.0069 Uniform 5-19 0.006 - 0.02143 0.00057 - 0.0069 Uniform 20-64 0.00692 - 0.02235 0.0015 - 0.01196 Uniform 65+ 0.03333 - 0.06842 0.0125 - 0.01579 Uniform Outpatient 0-4 0.82571 - 0.95952 0.47143 - 0.54762 Uniform 5-19 0.82571 - 0.95952 0.47143 - 0.54762 Uniform 20-64 0.58333 - 0.64783 0.33333 - 0.36957 Uniform 65+ 0.65833 - 0.68421 0.375 - 0.38947 Uniform

Clinical costs of influenza-related outcomes for each age and risk group (rounded tothe nearest whole dollar), adapted from [120].

Age group High risk Non-high risk Death 0-4 $62,219 $66,710 5-19 $254,287 $225,913 20-64 $97,114 $89,472 65+ $60,581 $44,633 Hospitalization 0-4 $28,296 $12,696 5-19 $54,502 $26,038 20-64 $39,924 $30,497 65+ $25,823 $17,157 Outpatient 0-4 $815 $458 5-19 $1,140 $375 20-64 $899 $571 65+ $3,505 $1,247 Ill, not medically attended 0-4 $5 $5 5-19 $5 $5 20-64 $5 $5 65+ $5 $5 58 Chapter 4.

Table 4.3: Health outcomes in the current vaccination scenario compared to no vaccination (40% vaccine efficacy). Cases, deaths, and disability-adjusted life years per 100,000 population by age and risk group. Cases averted refer to the original number of cases from the no vaccination scenario saved by current vaccination. All values denote rates per 100,000 population. (H = High risk, NH = Non-high risk).

No Vaccination Scenario Current Vaccination Scenario Cases per 100k Cases per 100k Cases averted per 100k Age group H risk NH risk All H risk NH risk All H risk NH risk All 0-4 22234 22234 22234 8998 8998 8998 13236 13236 13236 5-19 43970 43970 43970 20601 20601 20601 23368 23368 23368 20-64 20557 20558 20558 9854 9854 9854 10703 10703 10703 65+ 14883 14883 14883 5847 5847 5847 9036 9036 9036 All 20718 25263 24549 9471 11822 11453 11248 13441 13096 Deaths per 100k Deaths per 100k Deaths averted per 100k Age group H risk NH risk All H risk NH risk All H risk NH risk All 0-4 144.7 3 12.1 58.5 1.2 4.9 86.1 1.8 7.2 5-19 286.6 6 23.9 134.3 2.8 11.2 152.3 3.2 12.7 20-64 265.5 6.2 43.6 127.3 3 20.9 138.2 3.2 22.7 65+ 391.7 51.4 225.7 153.9 20.2 88.7 237.8 31.2 137 All 301.2 8.3 54.3 133.8 3.7 24.2 167.3 4.6 30.1 DALYs per 100K DALYs per 100k DALYs averted per 100k Age group H risk NH risk All H risk NH risk All H risk NH risk All 0-4 11219 255 957 4540 103 387 6679 152 570 5-19 19391 446 1659 9085 209 777 10305 237 882 20-64 10349 262 1715 4961 126 822 5388 137 893 65+ 5616 750 3242 2207 295 1274 3410 456 1968 All 9667 326 1793 4452 150 826 5215 175 967 Gloria J. Kang 59

Table 4.4: Budget impact analysis. Total vaccination costs of each scenario; vaccination costs in the table reflect differences in population size by age and risk group.

Budget impact of the Healthy People 2020 vaccination scenario.

Age group Pop size High risk prop. Non-high risk prop. Total budget impact 0-4 223,608 $208,885 $3,054,942 $3,263,827 5-19 639,661 $597,544 $8,739,076 $9,336,620 20-64 2,235,049 $3,039,717 $18,069,427 $21,109,144 65+ 308,558 $2,848,505 $2,714,981 $5,563,486 Total 3,406,876 $6,694,650 $32,578,426 $39,273,077

Budget impact of the Healthy People 2020 vaccination scenario.

Age group Pop size High risk prop. Non-high risk prop. Total budget impact 0-4 223,608 $286,705 $4,193,058 $4,479,763 5-19 639,661 $820,158 $11,994,810 $12,814,968 20-64 2,235,049 $6,447,884 $38,329,088 $44,776,972 65+ 308,558 $3,165,005 $3,016,646 $6,181,651 Total 3,406,876 $10,719,752 $57,533,602 $68,253,354 60 Chapter 4.

Table 4.5: Incremental cost-effectiveness ratios for the current vaccination scenario (40% vaccine efficacy) compared to no vaccination. Incremental cost- effectiveness ratios were calculated per case, death, and DALY averted for each ageand risk group. All values represent dollars saved per outcome averted, denoting a cost-saving intervention.

ICER Age group High risk Non-high risk All risk $ per case averted 0-4 years $1,447 $219 $297 5-19 years $2,963 $280 $451 20-64 years $2,336 $376 $658 65 years $5,127 $743 $2,988 All ages $3,070 $343 $711 $ per death averted 0-4 years $222,390 $1,612,828 $547,452 5-19 years $454,575 $2,057,975 $829,100 20-64 years $180,855 $1,242,969 $310,721 65 years $194,790 $215,025 $197,041 All ages $206,340 $1,004,195 $308,703 $ per DALY averted 0-4 years $2,868 $19,063 $6,912 5-19 years $6,720 $27,550 $11,964 20-64 years $4,640 $29,502 $7,894 65 years $13,586 $14,745 $13,717 All ages $6,621 $26,282 $9,627

Table 4.6: Direct and indirect effects of vaccination in the current vaccination scenario (40% vaccine efficacy) compared to no vaccination. Direct effects of vaccination represent averted cases amongst the vaccinated population, calculated as vaccine coverage * efficacy. Indirect effects account for the remaining difference of averted cases in the unvaccinated population. Indirect effects contributed to a larger proportion of cases averted in each age group than direct effects given current vaccination coverage at40% efficacy.

Age Coverage Cases Cases Direct Indirect % Direct Indi- group averted effect effect Averted % rect % 0-4 51% 30,030 44,176 15,138 29,038 59.5% 34.3% 65.7% 5-19 51% 196,686 223,102 85,637 137,466 53.2% 38.4% 61.6% 20-64 33% 328,732 357,043 90,522 266,520 52.1% 25.4% 74.7% 65+ 63% 26,927 41,613 17,272 24,341 60.7% 41.5% 58.5% Total 582,374 665,934 208,569 457,365 53.4% 31.3% 68.7% Gloria J. Kang 61

Figure 4.1: Epidemic curves for all vaccination scenarios. Daily and cumulative incidence of current vaccination and Healthy People vaccination scenarios with varying vaccine efficacy

Current vaccination scenario (a) Daily incidence

(b) Cumulative incidence 62 Chapter 4.

Healthy People 2020 vaccination scenario (c) Daily incidence

(d) Cumulative incidence Gloria J. Kang 63

Figure 4.2: Health outcomes by age and risk group in the current vaccination scenario (assuming 40% VE) compared to no vaccination. Cases, deaths, and disability-adjusted life years averted per 100,000 population.

(a) Cases averted

(b) Deaths averted 64 Chapter 4.

Figure 4.3: Incremental cost-effectiveness ratios for the current vaccination scenario compared to no vaccination. ICER per case, death and disability-adjusted life year averted by age and risk group

(a) ICER per case averted by age and risk group.

(b) ICER per death averted by age and risk group. 66 Chapter 4.

Figure 4.4: Cost-effectiveness planes for current vaccination compared to novac- cination. Incremental costs vs. DALYs averted for varied vaccine efficacy by age and risk group.

(a) Plots per risk group

(b) Plots per age group Chapter 5

Conclusions

5.1 Summary

In chapter 2, we found that facilitators of parental consent to school-located influenza vaccination included free/low cost vaccination; having belief in vaccine efficacy, influenza severity, and susceptibility; belief that vaccination is beneficial, important, and a social norm; perception of school setting advantages; trust; and parental presence. Barriers of parental consent included cost; concerns regarding vaccine safety, efficacy, equipment sterility, and adverse effects; perception of school setting barriers; negative physician advice of contraindications; distrust in vaccines and school-located vaccination programs; and health information privacy concerns. Study findings may assist in the evidence-based design and implementation ofin- fluenza vaccination programs targeted for children in the U.S.—and to improve influenza vaccination coverage for population-wide health benefits. In chapter 3, we examined current vaccine sentiment on social media by constructing and analyzing semantic networks of vaccine information from highly shared websites of Twitter users in the United States—and to assist public health communication of vaccines. We analyzed resulting network topology, compared semantic differences, and identified the most salient concepts within networks expressing positive, negative, and neutral vaccine sentiment. We found that the semantic network of positive vaccine sentiment demonstrated greater cohesiveness in discourse compared to the larger, less-connected network of negative vaccine sentiment. The positive sentiment network centered around parents and focused on communicating health risks and benefits, highlighting medical concepts such as measles, autism, HPV vaccine, vaccine-autism link, meningococcal disease, and MMR vaccine. In contrast, the negative network centered around children and focused on organizational bodies such as CDC, vaccine industry, doctors, mainstream media, pharmaceutical companies, and United States. The prevalence of negative vaccine sentiment was demonstrated through diverse messaging, framed around skepticism and distrust of government organizations that

68 Gloria J. Kang 69 communicate scientific evidence supporting positive vaccine benefits. The determinants of health behavior are largely social and increasingly political such that traditional public health practices may require reevaluation to prevent unintended population- level consequences. Analyzing structural properties of knowledge graphs provide unique insight into contentious topics of public debate, such as the long-standing issue of vaccine hesitancy. Our findings support a novel framework with the potential to uncover meaningful aspects of textual information that enhances our ability to understand vaccine sentiment—by elucidating the variability and scope of beliefs and attitudes. In chapter 4, we estimated the health benefits, costs, and cost-effectiveness of influenza vaccination using Seattle as a case study. We simulated socio-behavioral interactions and influenza transmission dynamics using an agent-based model and synthetic social contact network of Seattle. We simulated three scenarios encompassing no vaccination, influenza vaccination rates similar to current coverage, and the influenza vaccination target goal of the Healthy People 2020 initiative. We estimated the health impact of vaccination on influenza cases, deaths, and disability-adjusted life years averted per 100,000 population. We then assessed the costs and cost-effectiveness of vaccination scenarios accounting for both direct and indirect (herd-protection) effects from vaccination. We found that in thecurrent vaccination scenario assuming vaccine efficacy of 40%, vaccination averted an average of 13,096 cases, 30 deaths, and 967 DALYs (per 100,000 population) compared to the no vaccination scenario. The clinical cost of influenza illness was $630m with no vaccination, compared to $286.8m with vaccination at current coverage. While the budget impact of influenza vaccination was $39.3m, vaccination at current coverage was a cost-saving intervention with savings of $304m. Additionally, in the Healthy People 2020 scenario assuming low vaccine efficacy of 20%, vaccination also resulted in a cost-saving program, with estimated savings of $82.5m in clinical costs. The indirect benefit gained in health outcomes and cost savings by blocking influenza transmission by vaccinated individuals to susceptible individuals was larger than the direct benefit among vaccinated individuals. Based on our vaccination scenarios, efficacy assumptions, and inclusion of herd-protection effects, we found that influenza vaccination is a cost-saving intervention under current levels of vaccination coverage as well as Healthy People 2020’s target coverage goal when assuming vaccine efficacy of at least 20%. Study findings demonstrate the potential for influenza vaccination programs to be cost-saving strategies that provide good value in improved health and use of public health resources in urban areas such as Seattle.

5.2 Moving Forward

Initiatives for disease control will require sustained efforts at political, financial, and individual levels. As public health improves in accessibility and availability of health resources, focus must be placed on the usage and uptake of interventions rather than just its distri- 70 Chapter 5. bution. Surveillance systems must additionally take this into consideration as opposed to studying the dynamics of disease transmission occurring in the absence of context. Understanding the heterogeneity of behavior will become increasingly important for public health and in public health modeling. In order to reduce the potential for unintended health consequences, it will be imperative for researchers to better quantify the landscape of susceptibility, for health programs to limit their use of cursory interventions, and for those involved to develop sustainable frameworks conducive for health policy modeling. Systems theory is not some new concept. For public health however, systems dynamic modeling remains in its infancy as a niche sub-field. As populations urbanize on a global scale, researchers will increasingly rely upon the analysis of large digital data sets. Effective training (of public health students) in the technical and computational skills necessary for data analytics and systems modeling has proven to be a difficult transition [5]; slow adoption risks considerable variation in the skill sets of public health graduates. Akin to the dynamic processes that shape population health, training future cohorts of epi- demiologists and public health scientists will require critical re-evaluations of the current discourse and its conventional practices. As domain experts, it will be paramount to the utility of such models that our understanding contributes to a greater synthesis of interdisciplinary efforts in research and its applications to the real world.

5.3 Final Thoughts

The studies presented in this dissertation describe current understandings of vaccination in the United States as locally implemented health programs, as perceived online by social media, and through computational modeling of health and economic outcomes at the population-level. Mathematical models of disease transmission have a rich interdisciplinary history and potentially an even richer interdisciplinary future. Future advancements and progress in the field of public health will likely depend increasingly on novel methodology and innovative technologies involving Big Data. As computational methods advance, so will our abilities to test previously held assumptions, compare modeling methodologies, and better determine the most appropriate ways to approach, and maybe even solve, complex systems problems in health. Bibliography

[1] World Health Organization. Influenza, 2008.

[2] C J E Metcalf, W J Edmunds, and J Lessler. Six challenges in modelling for public health policy. Epidemics, 10:93–96, 3 2015.

[3] Douglas A Luke and Jenine K Harris. Network Analysis in Public Health: History, Methods, and Applications. Annual Review of Public Health, 28(1):69–93, 4 2007.

[4] Jack B Homer and Gary B Hirsch. System dynamics modeling for public health: background and opportunities. American journal of public health, 96(3):452–458, 2006.

[5] Sandro Galea, Matthew Riddle, and George A. Kaplan. Causal thinking and complex system approaches in epidemiology. International Journal of Epidemiology, 39(1):97– 106, 2010.

[6] Roy M Anderson. Populations and Infectious Diseases: Ecology or Epidemiology? Journal of Animal Ecology, 60(1):1–50, 2 1991.

[7] Daniel Bernoulli and Sally Blower. An attempt at a new analysis of the mortality caused by smallpox and of the advantages of inoculation to prevent it. Rev. Med. Virol., 14(5):275–288, 2004.

[8] Nicholas C Grassly and Christophe Fraser. Mathematical models of infectious disease transmission. Nature Reviews Microbiology, pages 1–11, 5 2008.

[9] Madhav V. Marathe and Naren Ramakrishnan. Recent advances in computational epidemiology. IEEE Intell. Syst., 28(4):96–101, jul 2013.

[10] Sebastian Funk, Shweta Bansal, Chris T. Bauch, Ken T.D. D Eames, W. John Ed- munds, Alison P. Galvani, and Petra Klepac. Nine challenges in incorporating the dynamics of behaviour in infectious diseases models. Epidemics, 10:21–25, 3 2015.

[11] S Funk, M Salathe, and V A A Jansen. Modelling the influence of human behaviour on the spread of infectious diseases: a review. Journal of The Royal Society Interface, 7(50):1247–1256, 7 2010.

71 72 BIBLIOGRAPHY

[12] S. Bansal, B. T Grenfell, and L. A. Meyers. When individual behaviour matters: homogeneous and network models in epidemiology. Journal of The Royal Society Interface, 4(16):879–891, 2007. [13] Matt J. Keeling and Ken T.D. Eames. Networks and epidemic models. Journal of The Royal Society Interface, 2(4):295–307, 2005. [14] C T Bauch and A P Galvani. Social and biological contagions. Science (New York, 1 2013. [15] R. Chad Swanson, Adriano Cattaneo, Elizabeth Bradley, Somsak Chunharas, Rifat Atun, Kaja M. Abbas, Korina Katsaliaki, Navonil Mustafee, Benjamin Mason Meier, and Allan Best. Rethinking health systems strengthening: Key systems thinking tools and strategies for transformational change. Health Policy and Planning, 27(SUPPL. 4):54–61, 2012. [16] Amy Middleman. School-located vaccination for adolescents: Past, present, and future and implications for HPV vaccine delivery. Hum. Vaccin. Immunother., 12(6):1599– 1605, June 2016. [17] Harry F Hull and Christopher S Ambrose. Current experience with school-located influenza vaccination programs in the united states. Hum. Vaccin., 7(2):153–160, 2011. [18] Gina Ogilvie, Maureen Anderson, Fawziah Marra, Shelly McNeil, Karen Pielak, Meena Dawar, Marilyn McIvor, Thomas Ehlen, Simon Dobson, Deborah Money, David M Patrick, and Monika Naus. A population-based evaluation of a publicly funded, school- based HPV vaccine program in british columbia, canada: parental factors associated with HPV vaccine receipt. PLoS Med., 7(5):e1000270, May 2010. [19] Michelle Russell, Vinita Raheja, and Rotimi Jaiyesimi. Human papillomavirus vaccination in adolescence. Perspect. Public Health, 133(6):320–324, November 2013. [20] Julia M L Brotherton, Shelley L Deeks, Sue Campbell-Lloyd, Avner Misrachi, Irene Passaris, Karen Peterson, Helen Pitcher, Megan Scully, Maureen Watson, and Rosalind Webby. Interim estimates of human papillomavirus vaccination coverage in the school- based program in australia. Commun. Dis. Intell. Q. Rep., 32(4):457–461, December 2008. [21] Paul V Effler, Carl Chu, Howard He, Kate Gaynor, Steve Sakamoto, Marcia Nagao, Lisa Mendez, and Sarah Y Park. Statewide school-located influenza vaccination program for children 5-13 years of age, hawaii, USA. Emerg. Infect. Dis., 16(2):244–250, February 2010. [22] L R Carpenter, J Lott, B M Lawson, S Hall, A S Craig, W Schaffner, and T F Jones. Mass Distribution of Free, Intranasally Administered Influenza Vaccine in a Public School System. PEDIATRICS, 120(1):e172–e178, 7 2007. Gloria J. Kang 73

[23] James C King, Jr, Jeffrey J Stoddard, Manjusha J Gaglani, Kristine A Moore, Lau- rence Magder, Elizabeth McClure, Judith D Rubin, Janet A Englund, and Kath- leen Neuzil. Effectiveness of school-based influenza vaccination. N. Engl. J. Med., 355(24):2523–2532, December 2006.

[24] Mandy A Allison, Maria Reyes, Paul Young, Lynne Calame, Xiaoming Sheng, Hsin- yi Cindy Weng, and Carrie L Byington. Parental attitudes about influenza immunization and school-based immunization for school-aged children. The Pediatric infectious disease journal, 29(8):751–755, 8 2010.

[25] Amy B Middleman and Jessica S Tung. Urban middle school parent perspectives: the vaccines they are willing to have their children receive using school-based immunization programs. J. Adolesc. Health, 47(3):249–253, September 2010.

[26] Derek S Brown, Sarah E Arnold, Garrett Asay, Suchita A Lorick, Bo-Hyun Cho, Ri- cardo Basurto-Davila, and Mark L Messonnier. Parent attitudes about school-located influenza vaccination clinics. Vaccine, 32(9):1043–1048, 2 2014.

[27] Amy B Middleman, Mary B Short, and Jean S Doak. School-located influenza immunization programs: Factors important to parents and students. Vaccine, 30(33):4993– 4999, 7 2012.

[28] Cuc H Tran, Jonathan D Sugimoto, Juliet R C Pulliam, Kathleen A Ryan, Paul D Myers, Joan B Castleman, Randell Doty, Jackie Johnson, Jim Stringfellow, Nadia Ko- vacevich, Joe Brew, Lai Ling Cheung, Brad Caron, Gloria Lipori, Christopher A Harle, Charles Alexander, Yang Yang, Ira M Longini, M Elizabeth Halloran, J Glenn Morris, and Parker A Small. School-located influenza vaccination reduces community risk for influenza and influenza-like illness emergency care visits. PLoS ONE, 9(12):e114479, 1 2014.

[29] Pia S Pannaraj, Hai-Lin Wang, Hector Rivas, Hilda Wiryawan, Michael Smit, Nicole Green, Grace M Aldrovandi, Alvin Nelson El Amin, and Laurene Mascola. School- located influenza vaccination decreases laboratory-confirmed influenza and improves school attendance. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America, 59(3):325–332, 8 2014.

[30] Derek Weycker, John Edelsberg, M Elizabeth Halloran, Ira M Longini, Jr, Azhar Nizam, Vincent Ciuryla, and Gerry Oster. Population-wide benefits of routine vaccination of children against influenza. Vaccine, 23(10):1284–1293, January 2005.

[31] Centers for Disease Control and Prevention (CDC). Update on influenza a (H1N1) 2009 monovalent vaccines. MMWR Morb. Mortal. Wkly. Rep., 58(39):1100–1101, October 2009. 74 BIBLIOGRAPHY

[32] Centers for Disease Control and Prevention. Flu vaccination coverage, united states, 2015-16 influenza season | FluVaxView | seasonal influenza (flu) |CDC. https://www. cdc.gov/flu/fluvaxview/coverage-1516estimates.htm. Accessed: 2017-2-21.

[33] Healthy People 2020. Healthy people 2020. https://www.healthypeople.gov/. Ac- cessed: 2017-2-21.

[34] Pedro A Piedra, Manjusha J Gaglani, Claudia A Kozinetz, Gayla Herschler, Mark Riggs, Melissa Griffith, Charles Fewlass, Matt Watts, Colin Hessel, Julie Cordova, and W Paul Glezen. Herd immunity in adults against influenza-related illnesses with use of the trivalent-live attenuated influenza vaccine (CAIV-T) in children. Vaccine, 23(13):1540–1548, February 2005.

[35] Ginny E Cummings, Elizabeth Ruff, Stephen H Guthrie, Margaret A Hoffmaster, Larry L Leitch, and James C King, Jr. Successful use of volunteers to conduct school- located mass influenza vaccination clinics. Pediatrics, 129 Suppl 2:S88–95, March 2012.

[36] James C King, Jr, Richard Lichenstein, Ginny E Cummings, and Laurence S Magder. Impact of influenza vaccination of schoolchildren on medical outcomes among all resi- dents of maryland. Vaccine, 28(49):7737–7742, November 2010.

[37] David Moher, Alessandro Liberati, Jennifer Tetzlaff, Douglas G Altman, and PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med., 6(7):e1000097, July 2009.

[38] Susan Cheung, Hai-Lin Wang, Laurene Mascola, Alvin Nelson El Amin, and Pia S Pannaraj. Parental perceptions and predictors of consent for school-located influenza vaccination in urban elementary school children in the United States. Influenza and other respiratory viruses, 9(5):255–262, 9 2015.

[39] Lisa M Gargano, Paul Weiss, Natasha L Underwood, Katherine Seib, Jessica M Sales, Tara M Vogt, Kimberly Rask, Christopher Morfaw, Dennis L Murray, Ralph J Di- Clemente, and James M Hughes. School-Located vaccination clinics for adolescents: Correlates of acceptance among parents. J. Community Health, 40(4):660–669, De- cember 2014.

[40] Karen Kelminson, Alison Saville, Laura Seewald, Shannon Stokley, L Miriam Dickin- son, Matthew F Daley, Christina Suh, and Allison Kempe. Parental views of school- located delivery of adolescent vaccines. The Journal of adolescent health : official publication of the Society for Adolescent Medicine, 51(2):190–196, 8 2012.

[41] Allison Kempe, Matthew F Daley, Jennifer Pyrzanowski, Tara M Vogt, Elizabeth J Campagna, L Miriam Dickinson, Simon J Hambidge, and Judith C Shlay. School- located influenza vaccination with third-party billing: what do parents think? Aca- demic pediatrics, 14(3):241–248, 5 2014. Gloria J. Kang 75

[42] N L Herbert, L M Gargano, J E Painter, J M Sales, C Morfaw, D Murray, R J DiClemente, and J M Hughes. Understanding reasons for participating in a school- based influenza vaccination program and decision-making dynamics among adolescents and parents. Health Education Research, 28(4):663–672, 7 2013. [43] Amy B Middleman, Mary B Short, and Jean S Doak. Focusing on flu: Parent perspectives on school-located immunization programs for influenza vaccine. Human Vaccines & Immunotherapeutics, 8(10):1395–1400, 10 2012. [44] Tiana L Won, Amy B Middleman M D M S E D M P H, Beth A Auslander Ph D, and Mary B Short Ph D. Trust and a School-Located Immunization Program. Journal of Adolescent Health, 56(S):S33–S39, 5 2015. [45] HHS. About the law. https://www.hhs.gov/healthcare/about-the-law/, June 2013. Accessed: 2017-2-21. [46] Barack Obama. United states health care reform: Progress to date and next steps. JAMA, 316(5):525–532, August 2016. [47] N K Janz, V L Champion, and V J Strecher. The health belief model. health behavior and health education: theory, research and practice. edited by: Glanz k, rimer BK, lewis FM, 2002. [48] Centers for Disease Control and Prevention. Cdc - acip - advisory committee on immunization practices (ACIP) home page - vaccines. http://www.cdc.gov/vaccines/ acip/. Accessed: 2015-7-3. [49] Elizabeth Dugan, Felicia Trachtenberg, and Mark A Hall. Development of abbreviated measures to assess patient trust in a physician, a health insurer, and the medical profession. BMC Health Serv. Res., 5:64, October 2005. [50] Jennifer Zipprich, Kathleen Winter, Jill Hacker, Dongxiang Xia, James Watt, Kath- leen Harriman, and Centers for Disease Control and Prevention (CDC). Measles outbreak-california, december 2014-february 2015. MMWR Morb. Mortal. Wkly. Rep., 64(6):153–154, February 2015. [51] Patrick Peretti-Watel, Jeremy K Ward, William S Schulz, Pierre Verger, and Heidi J Larson. Vaccine Hesitancy: Clarifying a Theoretical Framework for an Ambiguous Notion. PLoS currents, pages 1–12, 1 2015. [52] Gloria J. Kang, Rachel K. Culp, and Kaja M. Abbas. Facilitators and barriers of parental attitudes and beliefs toward school-located influenza vaccination in the United States: Systematic review. Vaccine, 35(16):1987–1995, 4 2017. [53] Eve Dubé, Dominique Gagnon, Noni E MacDonald, and SAGE Working Group on Vac- cine Hesitancy. Strategies intended to address vaccine hesitancy: Review of published reviews. Vaccine, 33(34):4191–4203, 8 2015. 76 BIBLIOGRAPHY

[54] Cornelia Betsch and Robert Böhm. Detrimental effects of introducing partial compul- sory vaccination: experimental evidence. European journal of public health, 8 2015.

[55] C Betsch, R Bohm, and G B Chapman. Using behavioral insights to increase vaccination policy effectiveness. Policy Insights from the Behavioral and Brain Sciences, 2(1):61–73, 2015.

[56] Heidi J Larson, Louis Z Cooper, Juhani Eskola, Samuel L Katz, and Scott Ratzan. Addressing the vaccine confidence gap. Lancet, 378(9790):526–535, August 2011.

[57] Caitlin Jarrett, Rose Wilson, Maureen O’Leary, Elisabeth Eckersberger, and Heidi J Larson. Strategies for addressing vaccine hesitancy – A systematic review. Vaccine, 4 2015.

[58] Bernice Hausman. Vaccination and the Public in the 21st Century, 3 2016.

[59] Heidi Y Lawrence, Bernice L Hausman, and Clare J Dannenberg. Reframing Medicine’s Publics: The Local as a Public of Vaccine Refusal. Journal of Medical Humanities, 35(2):111–129, 1 2014.

[60] M Salathe and S Bonhoeffer. The effect of opinion clustering on disease outbreaks. Journal of The Royal Society Interface, 5(29):1505–1508, 12 2008.

[61] Marcel Salathe, Linus Bengtsson, Todd J Bodnar, Devon D Brewer, John S Brownstein, Caroline Buckee, Ellsworth M Campbell, Ciro Cattuto, Shashank Khandelwal, Patri- cia L Mabry, and Others. Digital epidemiology. PLoS Comput. Biol., 8(7):e1002616, 2012.

[62] Mark Dredze, David A Broniatowski, Michael C Smith, and Karen M Hilyard. Under- standing Vaccine Refusal. American Journal of Preventive Medicine, 50(4):550–552, 4 2016.

[63] Marcel Salathé and Shashank Khandelwal. Assessing vaccination sentiments with online social media: implications for infectious disease dynamics and control. PLoS Computational Biology, 7(10):e1002199, 10 2011.

[64] Marcel Salathé, Duy Q Vu, Shashank Khandelwal, and David R Hunter. The dynamics of health behavior sentiments on a large online social network. EPJ Data Sci., 2(1):4, April 2013.

[65] Peter Gloor and Jana Diesner. Semantic social networks. In Encyclopedia of Social Network Analysis and Mining, pages 1654–1659. Springer, 2014.

[66] P W Eklund and O Haemmerlé. Conceptual structures: Knowledge visualization and reasoning. In 16th International Conference on Conceptual Structures, ICCS, pages 7–11, 2008. Gloria J. Kang 77

[67] M L Doerfel. What constitutes semantic network analysis? a comparison of research and methodologies. Connect., 1998.

[68] J B Ruiz and G A Barnett. Exploring the presentation of HPV information online: A semantic network analysis of websites. Vaccine, 33(29):3354–3359, 1 2015.

[69] Jeanette B Ruiz and Robert A Bell. Understanding vaccination resistance: Vac- cine search term selection bias and the valence of retrieved information. Vaccine, 32(44):5776–5780, 10 2014.

[70] Philipp Drieger. Semantic Network Analysis as a Method for Visual Text Analytics. Procedia - Social and Behavioral Sciences, 79:4–17, 6 2013.

[71] Caroline M Poland and Emily K Brunson. The need for a multi-disciplinary perspective on vaccine hesitancy and acceptance. Vaccine, 33(2):277–279, 1 2015.

[72] Heidi Larson, Julie Leask, Sian Aggett, Nick Sevdalis, and Angus Thomson. A Multi- disciplinary Research Agenda for Understanding Vaccine-Related Decisions. Vaccines, 1(3):293–304, 1 2013.

[73] Christophe Thovex and Francky Trichet. Semantic social networks analysis. Social Network Analysis and Mining, 3(1):35–49, 2 2012.

[74] SAGE Working Group on Vaccine Hesitancy. Revised report of the SAGE working group on vaccine hesitancy. http://www.who.int/immunization/sage/meetings/ 2014/october/SAGE_working_group_revised_report_vaccine_hesitancy.pdf? ua=1, November 2014. Accessed: 2017-4-25.

[75] James T Schlitt, Bryan Lewis, and Stephen Eubank. ChatterGrabber: A Lightweight Easy to Use Social Media Surveillance Toolkit. Online Journal of Public Health Infor- matics, 7(1), 2 2015.

[76] Stanley Wasserman and Katherine Faust. Social Network Analysis: Methods and Applications. Cambridge University Press, November 1994.

[77] M Van Steen. Graph theory and complex networks. di.unipi.it, 2010.

[78] Linton C Freeman. Centrality in social networks conceptual clarification. Soc. Net- works, 1(3):215–239, 1978.

[79] M E J Newman and M Girvan. Finding and evaluating community structure in networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys., 69(2 Pt 2):026113, February 2004.

[80] Santo Fortunato. Community detection in graphs. Phys. Rep., 486(3-5):75–174, 2010. 78 BIBLIOGRAPHY

[81] V Batagelj and M Zaversnik. An o(m) algorithm for cores decomposition of networks. Advances in Data Analysis and Classification, October 2003.

[82] Bjorn Burscher, Rens Vliegenthart, and Claes H de Vreese. Frames beyond words. Soc. Sci. Comput. Rev., 34(5):530–545, 2016.

[83] Susan Goldstein, Noni E MacDonald, Sherine Guirguis, SAGE Working Group on Vac- cine Hesitancy, and the SAGE Working Group on Vaccine Hesitancy2. Health communication and vaccine hesitancy. Vaccine, 33(34):4212–4214, 8 2015.

[84] D A Schult and P Swart. Exploring network structure, dynamics, and function using NetworkX. permalink.lanl.gov, 2008.

[85] Gabor Csardi and Tamas Nepusz. The igraph software package for complex network research. InterJournal, Complex Systems, 1695(5):1–9, 2006.

[86] M Bastian, S Heymann, and M Jacomy. Gephi: an open source software for exploring and manipulating networks, 2009.

[87] Allan M Collins and M Ross Quillian. Retrieval time from semantic memory. Journal of verbal learning and verbal behavior, 8(2):240–247, 1969.

[88] Allan M Collins and Elizabeth F Loftus. A spreading-activation theory of semantic processing. Psychological review, 82(6):407, 1975.

[89] John R Anderson. A spreading activation theory of memory. Journal of verbal learning and verbal behavior, 22(3):261–295, 1983.

[90] Gary S Dell. A spreading-activation theory of retrieval in sentence production. Psy- chological review, 93(3):283, 1986.

[91] Russell H Fazio, David M Sanbonmatsu, Martha C Powell, and Frank R Kardes. On the automatic activation of attitudes. Journal of personality and social psychology, 50(2):229, 1986.

[92] Dirk Koschützki, Katharina Anna Lehmann, Leon Peeters, Stefan Richter, Dagmar Tenfelde-Podehl, and Oliver Zlotowski. Centrality indices. In Network analysis, pages 16–61. Springer, 2005.

[93] Pan and Allen. An act to amend sections 120325, 120335, 120370, and 120375 of, to add section 120338 to, and to repeal section 120365 of, the health and safety code, relating to public health., 2015.

[94] Helena Miton and Hugo Mercier. Cognitive obstacles to Pro-Vaccination beliefs. Trends Cogn. Sci., 19(11):633–636, November 2015. Gloria J. Kang 79

[95] Melissa A. Rolfes, Ivo M. Foppa, Shikha Garg, Brendan Flannery, Lynnette Brammer, James A. Singleton, Erin Burns, Daniel Jernigan, Sonja J. Olsen, Joseph Bresee, and Carrie Reed. Annual estimates of the burden of seasonal influenza in the United States: A tool for strengthening influenza surveillance and preparedness. Influenza Other Respi. Viruses, 12(1):132–137, jan 2018.

[96] Noelle-Angelique M Molinari, Ismael R Ortega-Sanchez, Mark L Messonnier, William W Thompson, Pascale M Wortley, Eric Weintraub, and Carolyn B Bridges. The annual impact of seasonal influenza in the {US}: measuring disease burden and costs. Vaccine, 25(27):5086–5096, jun 2007.

[97] Wayan CWS Putri, David J Muscatello, Melissa S Stockwell, and Anthony T Newall. Economic burden of seasonal influenza in the united states. Vaccine, 36(27):3960–3966, 2018.

[98] Jerome I Tokars, Sonja J Olsen, and Carrie Reed. The seasonal incidence of symptomatic influenza in the United States. Clin Infect Dis, dec 2017.

[99] Anthony E Fiore, David K Shay, Karen Broder, John K Iskander, Timothy M Uyeki, Gina Mootrey, Joseph S Bresee, Nancy J Cox, Centers for Disease Control, and Pre- vention. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices ({ACIP}), 2009. {MMWR} Recomm Rep, 58({RR}-8):1–52, jul 2009.

[100] Healthy People 2020. Immunization and Infectious Diseases \textbar Healthy People 2020.

[101] CDC. 2010-11 through 2016-17 Influenza Seasons Vaccination Coverage Trend Report.

[102] Scott D Grosse, Steven M Teutsch, and Anne C Haddix. Lessons from cost-effectiveness research for United States public health policy. Annu Rev Public Heal., 28:365–391, 2007.

[103] Eon E K Ting, Beate Sander, and Wendy J Ungar. Systematic review of the cost- effectiveness of influenza immunization programs. Vaccine, 35(15):1828–1843, apr 2017.

[104] Martin Eichner, Markus Schwehm, Linda Eichner, and Laetitia Gerlier. Direct and indirect effects of influenza vaccination. {BMC} Infect Dis, 17(1):308, apr 2017.

[105] G M Cohen and M D Nettleman. Economic impact of influenza vaccination in preschool children. Pediatrics, 106(5):973–976, nov 2000.

[106] B R Luce, K M Zangwill, C S Palmer, P M Mendelman, L Yan, M C Wolff, I Cho, SM Marcy, D Iacuzio, and R B Belshe. Cost-effectiveness analysis of an intranasal influenza vaccine for the prevention of influenza in healthy children. Pediatrics, 108(2):E24, aug 2001. 80 BIBLIOGRAPHY

[107] Yao Ding, Kenneth M Zangwill, Joel W Hay, Norma J Allred, and Sylvia H Yeh. Cost-benefit analysis of in-hospital influenza vaccination of postpartum women. Obs. Gynecol, 119(2 Pt 1):306–314, feb 2012.

[108] Kristin L Nichol and Michael Goodman. Cost effectiveness of influenza vaccination for healthy persons between ages 65 and 74 years. Vaccine, 20:S21–S24, may 2002.

[109] Ayman Chit, Julie Roiz, Benjamin Briquet, and David P Greenberg. Expected cost effectiveness of high-dose trivalent influenza vaccine in {US} seniors. Vaccine, 33(5):734– 741, jan 2015.

[110] Anna K Lugnér and Maarten J Postma. Mitigation of pandemic influenza: review of cost-effectiveness studies. Expert Rev Pharmacoecon Outcomes Res, 9(6):547–558, dec 2009.

[111] Brendan Flannery, Jessie R. Chung, Swathi N. Thaker, Arnold S. Monto, Emily T. Martin, Edward A. Belongia, Huong Q. McLean, Manjusha Gaglani, Kempapura Murthy, Richard K. Zimmerman, Mary Patricia Nowalk, Michael L. Jackson, Lisa A. Jackson, Angie Foust, Wendy Sessions, LaShondra Berman, Sarah Spencer, and Ali- cia M. Fry. Interim Estimates of 2016–17 Seasonal Influenza Vaccine Effectiveness — United States, February 2017. MMWR. Morb. Mortal. Wkly. Rep., 66(6):167–171, feb 2017.

[112] Richard J Beckman, Keith A Baggerly, and Michael D McKay. Creating synthetic baseline populations. Transp Res Part A Policy Pr., 30(6 {PART} A):415–429, nov 1996.

[113] Chris Barrett, Keith Bisset, Jonathan Leidig, Achla Marathe, and Madhav Marathe. Economic and social impact of influenza mitigation strategies by demographic class. Epidemics, 3(1):19–31, mar 2011.

[114] Barrett, Christopher L Bisset, Keith R Eubank, Stephen G Feng, Xizhou; Marathe, and Madhav V. {EpiSimdemics}: an efficient algorithm for simulating the spread of infectious disease over large realistic social networks. {IEEE} Press, nov 2008.

[115] K. R. Bisset, S. Deodhar, H. Makkapati, M. V. Marathe, P. Stretz, and C. L. Barrett. Simfrastructure: A Flexible and Adaptable Middleware Platform for Modeling and Analysis of Socially Coupled Systems. In 2013 13th IEEE/ACM Int. Symp. Clust. Cloud, Grid Comput., pages 506–513. IEEE, may 2013.

[116] M I Meltzer, N J Cox, and K Fukuda. The economic impact of pandemic influenza in the United States: priorities for intervention. Emerg. Infect Dis, 5(5):659–671, oct 1999.

[117] CDC. Flu Vaccination Coverage, United States, 2016-17 Influenza Season. Gloria J. Kang 81

[118] CDC. Seasonal influenza vaccine effectiveness, 2005-2016 | health professionals | seasonal influenza (flu). http://www.cdc.gov/flu/professionals/vaccination/ effectiveness-studies.htm. Accessed: 2018-4-22.

[119] Burearu of Labor Statistics. Consumer price index - january 2018. Retrieved June 19, 2018.

[120] Cristina Carias, Carrie Reed, Inkyu K Kim, Ivo M Foppa, Matthew Biggerstaff, Mar- tin I Meltzer, Lyn Finelli, and David L Swerdlow. Net Costs Due to Seasonal Influenza Vaccination-United States, 2005-2009. {PLoS} {ONE}, 10(7):e0132922, jul 2015.

[121] Byung-Kwang Yoo, Peter G Szilagyi, Stanley J Schaffer, Sharon G Humiston, Cyn- thia M Rand, Christina S Albertin, Phyllis Vincelli, Aaron K Blumkin, Laura P Shone, and Margaret S Coleman. Cost of universal influenza vaccination of children in pediatric practices. Pediatrics, 124 Suppl:S499–506, dec 2009.

[122] Sean D Sullivan, Josephine A Mauskopf, Federico Augustovski, J Jaime Caro, Karen M Lee, Mark Minchin, Ewa Orlewska, Pete Penna, Jose-Manuel Rodriguez Barrios, and Wen-Yi Shau. Budget impact analysis-principles of good practice: report of the {IS- POR} 2012 Budget Impact Analysis Good Practice {II} Task Force. Value Heal., 17(1):5–14, feb 2014.

[123] WHO. {WHO} \textbar Metrics: Disability-Adjusted Life Year ({DALY}).

[124] Elizabeth Arias, Melonie Heron, and Jiaquan Xu. United states life tables, 2014. Natl Vital Stat Rep, 66(4):1–64, aug 2017.

[125] Anthony E Fiore, David K Shay, Karen Broder, John K Iskander, Timothy M Uyeki, Gina Mootrey, Joseph S Bresee, Nancy S Cox, Centers for Disease Control, Prevention (CDC), and Advisory Committee on Immunization Practices (ACIP). Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices ({ACIP}), 2008. {MMWR} Recomm Rep, 57({RR}-7):1–60, aug 2008.

[126] GBD. Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet, 388(10053):1545–1602, oct 2016.

[127] Nargesalsadat Dorratoltaj, Achla Marathe, Bryan L Lewis, Samarth Swarup, Stephen G Eubank, and Kaja M Abbas. Epidemiological and economic impact of pandemic influenza in Chicago: Priorities for vaccine interventions. {PLoS} Comput Biol, 13(6):e1005521, jun 2017.

[128] Pratha Sah, Jan Medlock, Meagan C Fitzpatrick, Burton H Singer, and Alison P Galvani. Optimizing the impact of low-efficacy influenza vaccines. 115(20):5151–5156, 2018. 82 BIBLIOGRAPHY

[129] Ruprecht Schmidt-Ott, Markus Schwehm, and Martin Eichner. Influence of social contact patterns and demographic factors on influenza simulation results. {BMC} Infect Dis, 16(1):646, nov 2016.

[130] A. Cori, A. J. Valleron, F. Carrat, G. Scalia Tomba, G. Thomas, and P. Y. Boëlle. Estimating influenza latency and infectious period durations using viral excretion data. Epidemics, 4(3):132–138, aug 2012.

[131] Margaretha Annelie Vink, Martinus Christoffel Jozef Bootsma, and Jacco Wallinga. Serial intervals of respiratory infectious diseases: A systematic review and analysis, nov 2014.

[132] L R Elveback, J P Fox, E Ackerman, A Langworthy, M Boyd, and L Gatewood. An influenza simulation model for immunization studies. Am. J. Epidemiol., 103(2):152– 165, feb 1976.

[133] Joseph T Wu, Steven Riley, Christophe Fraser, and Gabriel M Leung. Reducing the impact of the next influenza pandemic using household-based public health interventions. PLoS Med., 3(9):e361, sep 2006.

[134] Eleni Patrozou and Leonard A Mermel. Does influenza transmission occur from asymptomatic infection or prior to symptom onset? Public Heal. Rep., 124(2):193–196, mar 2009.

[135] Fabrice Carrat, Elisabeta Vergu, Neil M. Ferguson, Magali Lemaitre, Simon Cauchemez, Steve Leach, and Alain Jacques Valleron. Time lines of infection and disease in human influenza: A review of volunteer challenge studies, apr 2008.

[136] John F Sowa. Semantic networks. In Encyclopedia of Cognitive Science. John Wiley & Sons, Ltd, 2006.

[137] Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, pages 601–610, New York, NY, USA, 2014. ACM.

[138] Elie Raad and Richard Chbeir. Socio-Graph representations, concepts, data, and analysis. In Professor Reda Alhajj and Professor Jon Rokne, editors, Encyclopedia of Social Network Analysis and Mining, pages 1936–1946. Springer New York, 2014.

[139] Sébastien Heymann. Gephi. In Professor Reda Alhajj and Professor Jon Rokne, editors, Encyclopedia of Social Network Analysis and Mining, pages 612–625. Springer New York, 2014. Appendices

83 Appendix A

Supplemental Information: Semantic network analysis of vaccine sentiment in online social media

A.1 ChatterGrabber

ChatterGrabber search terms were selected through an iterative process involving manual selection and testing of data retrieval which are detailed elsewhere [75].

Table A.1: Description of ChatterGrabber parameters

Parameter Location United States Tweet data Text, ID, time posted, retweet count, favorite count User data Screen name, language Media data Url, display Url

84 Gloria J. Kang 85

Table A.2: ChatterGrabber search terms

Conditions Qualifiers Exclusions Vaccine Autism Bullshit Vaccinat Autistic Conspiracy Penn & teller Vacine Gave my Penn and teller Vacinate Gave me Enter the kingdom of heaven MMR Oprah Heroin Antivac Aspergers Eye of a needle Poison Thread Jenny mccarthy Molds Kristin cavallari Record Conspiracy Efficacy Mercury Shoot up Aluminum Needle exchange Bravo Morphine Anti Knit Manufacturers Crochet Have known Fracking Vaccine choice Insulin Your child Malware Your right Pincushion Cancer Addict Fertility Fuel Constitution Needlework Risks Felt Dangerous Caffeine Scaling Space

Table A.3: Twitter data via ChatterGrabber

n Total number of collected tweets 26,389 Number of unique urls 8416 Number of unique domains 2372 Number of web articles selected for analysis 50 86 Appendix A.

A.2 Study methodology

Rationale

A semantic network is a structured representation of knowledge based on meaningful relationships of written text. Within the semantic network, nodes are words that represent concepts found in text. The method of viewing natural language text as a network of interconnected concepts was originally used for reasoning and inference [136]. In this way, semantic approach allows for the extraction of meaningful ideas by identifying emergent clusters of concepts rather than analyzing frequencies of isolated words [67]. Text analysis traditionally employs semi-automated techniques in which information is extracted and analyzed using both human and computerized methods. Other studies have analyzed websites using search engine results and natural language processing (NLP) [68, 69]. NLP generally relies on measuring the frequency or co-occurrence of words as the basis of significance using a predetermined set of relational concepts. The limitations in this approach includes potentially ignoring grammatical dependencies that limit accurate interpretation and significant challenges in co-reference resolution, synonyms, and ambiguity70 [ ]. For these reasons, we compare the topological properties of connectivity in the semantic network instead of focusing on frequency-based ranking mechanisms. We primarily focus on centrality measures which is a common method for social network analysis but is rarely used in semantic networks. Network-based measures allow us to identify nodes that are important in connecting the network, even if they lack prominent frequency. Semantic network analysis of vaccine sentiment enables interdisciplinary methods of computational and social sciences by which public health understanding can be enhanced to improve vaccine communication. Our study presents a novel framework that applies methods of network analysis to semantic networks [73] within the context of vaccine sentiment.

Limitations

Our study has several limitations. First, we assume that widely-shared information is representative of major belief systems. Secondly, creating networks from real-world data present several challenges inherent to the domain of information science itself; broadly these challenges include information extraction, knowledge representation, and contextual interpretation. A drawback to traditional semantic analysis is the inability to infer implicit ideas and their relationships within natural language text. While we attempted to resolve issues of meaning and context by translating implicit statements into explicit statements, this made co-reference resolution increasingly difficult within individual networks and more so across them. Subsequently, this results in potential inconsistency while manually annotating network metadata, particularly when dealing with Gloria J. Kang 87 ambiguous language such as slang, metaphors, and other literary devices. Despite these limitations, human interpretation of text is arguably more accurate than natural language processing methods. Next, identifying “neutral” sentiment was difficult as documents presented a mix of both positive and negative attitudes—and not truly vaccine-neutral. Relatively small sample size of neutral articles may have limited meaningful comparison as well. Sentiment categories are difficult to delineate because they do not exist as polarized groups but rather on aspectrum; health behaviors such as vaccine hesitancy are founded upon a variety of combined beliefs and attitudes that change over time. Nevertheless, our sample of extracted web documents using Twitter data may accurately reflect the current distribution of vaccine sentiment. Our analysis did not assess the qualitative data of edges, although present within network metadata. Accounting for various edge types in future studies can further establish semantic relationships of important network concepts. This is an exploratory study; future studies employing this framework can adequately apply natural language processing and machine learning techniques for more efficient analysis of text data.

A.2.1 Network annotation and construction

To create document networks, article text was manually transcribed into structured belief statements, or relevant information extracted from natural language text. Similar to methods of information extraction used by the Knowledge Vault project [137], document text was formatted as triples, in which (subject, predicate, object) correspond to (node, edge, node) in the network. For example, the sentence “Vaccines prevent communicable diseases” is represented by (vaccines, prevent, communicable diseases). Three researchers initially an- notated a subset of 10 documents to gauge inter-annotator variability in transcribing article documents into network data sets. All co-references were resolved and the original text was adhered to as much as possible. Discordant results were resolved through consensus in order to maintain standard formatting of network data. Final network data sets were synthesized by standardizing terminology, resolving grammatical dependencies and lexical differences in the semantic network. The resulting standards for network vocabulary were based on term frequency. For example, synonymous nodes labeled “communicable diseases”, “infectious diseases”, and “contagious diseases”, we applied the most commonly used term across same-sentiment documents (in this case “infectious diseases”) to replace labels of all semantically equivalent nodes.

A.2.2 Definitions of network measures

Network size is the total number of nodes or vaccine-related concepts. Density measures the interconnectedness of nodes, calculated as the proportion of existing edges (or relations 88 Appendix A. between concepts) over all possible edges in the network [76]. Diameter characterizes the compactness of the network, measured as the longest path of all shortest paths across all node pairs. Degree centrality characterizes how connected a node is to other nodes in the network, measured by its number of connections (and normalized by the total number of network connections) [78]. Betweenness centrality measures the frequency of a given node on the shortest paths to all other pairs of connected nodes, representing the probability of a concept to be involved in connecting two other concepts in the semantic network [76, 78]. Closeness centrality measures closeness, calculating the sum of the shortest paths between a node to all other nodes in the network [78]. Nodes with smaller path lengths have higher closeness centrality and are interpreted to be more important concepts than nodes with longer paths [138]. Lastly, eigenvector centrality provides a more complex measure of node influence by assigning relative scores to all concepts in the network, based on the number and quality of its relationships; a concept is significant to the extent that it is connected to other significant concepts [139]. Community detection using the Newman-Girvan algorithm detects communities by consec- utively removing each edge with the highest edge betweenness from the graph [79]. Edge- betweenness refers to the number of shortest paths from one node to another that traverse through that edge. Cohesive groups in the network are measured by modularity, in which a good partition has more intra-community edges than expected at random; modularity values other than zero represent deviations from randomness [80]. Gloria J. Kang 89

A.3 Network visualizations

Figure A.1: Full semantic network graphs by vaccine sentiment

(a) Full positive semantic network 90 Appendix A.

(b) Full negative semantic network Gloria J. Kang 91

Figure A.2: Greatest component subgraph by vaccine sentiment. Increasing node size represents greater betweenness centrality.

(a) Positive semantic network: Greatest component Gloria J. Kang 93

(b) Negative semantic network: Greatest component 94 Appendix A.

A.4 Centrality measures

Centrality characterizes the importance, influence, or power of vaccine-related concepts in the semantic network. The table lists measures for the most central concepts (greater than 2 standard deviations from the network mean) by degree centrality, betweenness centrality, closeness centrality, and eigenvector centrality for positive, negative, and neutral sentiment networks.

Table A.4: Positive sentiment network: Top centrality

Degree Betweenness Closeness Eigenvector centrality centrality centrality centrality Mean = 0.0061 Mean = 0.006 Mean = 0.2292 Mean = 0.0626 SD = 0.0107 SD = 0.0203 SD = 0.038 SD = 0.0936 vaccines 0.1079 parents 0.2718 parents 0.3687 parents 1 parents 0.0993 vaccines 0.2176 vaccines 0.3482 vaccines 0.8209 measles 0.0993 measles 0.1546 children 0.3415 measles 0.7458 vaccination 0.0856 anti- 0.1261 measles 0.3382 vaccination 0.6373 vaccination autism 0.0616 religious groups 0.1018 community 0.3227 children 0.5382 HPV vaccine 0.0565 vaccine-autism 0.0917 religious groups 0.3219 SB 277 0.4207 link vaccine-autism 0.0531 meningococcal 0.0905 autism 0.3188 autism 0.4025 link disease meningococcal 0.0531 children 0.0825 SB 277 0.3158 community 0.3937 disease anti- 0.0479 autism 0.0799 vaccine-autism 0.3148 religious groups 0.3905 vaccination link children 0.0445 HPV vaccine 0.0732 anti- 0.3121 anti- 0.3802 vaccination vaccination MMR vaccine 0.0411 community 0.0574 vaccination 0.31 vaccine-autism 0.3608 link religious groups 0.0394 SB 277 0.0571 herd immunity 0.3058 measles vaccine 0.0377 measles vaccine 0.0523 vaccine refusal 0.3024 SB 277 0.0342 side effects 0.051 vaccination 0.3013 exemption disease 0.0308 Gardasil 0.0496 personal belief 0.2909 exemption vaccination 0.0291 disease 0.2829 exemption autism risk 0.0291 measles vaccine 0.2706 schools 0.2685 HPV vaccine 0.2674 vaccine delay 0.2603 meningococcal 0.2551 disease 96 Appendix A.

Table A.5: Negative sentiment network: Top centrality

Degree Betweenness Closeness Eigenvector centrality centrality centrality centrality Mean = 0.0149 Mean = 0.0342 Mean = 0.1533 Mean = 0.0975 Std Dev = Std Dev = Std Dev = Std Dev = 0.11 0.0204 0.0839 0.0296 SB 277 0.1824 vaccines 0.5749 vaccines 0.2335 SB 277 1 vaccines 0.1118 Dwoskin 0.4092 side effects 0.2208 vaccines 0.4304 Family Foundation anti- 0.1059 pertussis 0.3947 pertussis 0.2199 anti- 0.4177 vaccination vaccine vaccine vaccination pertussis 0.0824 vaccine-autism 0.362 whole-cell 0.2133 parents 0.3863 vaccine link vaccine pertussis 0.0824 SB 277 0.3294 effective 0.2133 children 0.383 high-dose flu 0.0647 children 0.2643 pertussis 0.354 vaccine vaccine anti- 0.2554 home-school 0.3209 vaccination side effects 0.2347 education 0.3206 acellular 0.2077 pertussis vaccine

Table A.6: Neutral sentiment network: Top centrality

Degree Betweenness Closeness Eigenvector centrality centrality centrality centrality Mean = 0.0149 Mean = 0.0342 Mean = 0.1533 Mean = 0.0975 SD = 0.0204 SD = 0.0839 SD = 0.0296 SD = 0.11 SB 277 0.1824 vaccines 0.5749 vaccines 0.2335 SB 277 1 vaccines 0.1118 Dwoskin 0.4092 side effects 0.2208 vaccines 0.4304 Family Foundation anti- 0.1059 pertussis 0.3947 pertussis 0.2199 anti- 0.4177 vaccination vaccine vaccine vaccination pertussis 0.0824 vaccine-autism 0.362 whole-cell 0.2133 parents 0.3863 vaccine link vaccine pertussis 0.0824 SB 277 0.3294 effective 0.2133 children 0.383 high-dose flu 0.0647 children 0.2643 pertussis 0.354 vaccine vaccine anti- 0.2554 home-school 0.3209 vaccination side effects 0.2347 education 0.3206 acellular 0.2077 pertussis vaccine Gloria J. Kang 97

Table A.7: Top ranked nodes by closeness vitality for the three networks

Negative sentiment Neutral sentiment Positive sentiment network network network Mean = 19148.407 Mean = 7029.871 Mean = 8449.754 Std Dev = 24052.786 Std Dev = 16597.291 Std Dev = 9778.734 Thimerosal 239154 Vaccines 127564 Meningococcal 79948 disease MTHFR C677T 222220 Dwoskin family 109972 Vaccination 77396 defect foundation Millions of dollars 179468 Vaccine-autism link 100468 Polio vaccine 74438 opposition Children with autism 201122 SB 277 49768 Wakefield study 64018 Measles mortality 179468 Acellular pertussis 48048 HPV vaccine 63748 vaccine Vaccine court 172456 Artificial vaccine 43430 Vaccines 61934 National vaccine 168948 Anti-vaccination 41638 Autism 61016 injurty compensation program Anti-vaccination 145200 Generation Rescue 37594 Orthodox Hasidic 55846 Jews Measles 141736 immune response 34424 Measles 47038 Adverse effects 141140 Focus for Health 32640 Hepatitis A vaccine 44804 98 Appendix A.

A.5 Data files

Data files and dynamic web-based interactive visualizations of semantic networks canbe accessed online at: http://dx.doi.org/10.1016/j.vaccine.2017.05.052. Appendix B

Supplemental Information: Cost-effectiveness of seasonal influenza vaccination

B.1 Healthy People 2020 vaccination scenario

99 100 Appendix B.

Figure B.1: Health outcomes (per 100,000 population) in the Healthy People 2020 vaccination scenario compared to prevalent vaccination (20% vaccine efficacy). Cases, deaths, and disability-adjusted life years averted by age and risk group. Gloria J. Kang 101 102 Appendix B.

Figure B.2: Incremental cost-effectiveness ratios in the Healthy People 2020 vaccination scenario compared to prevalent vaccination (20% vaccine efficacy). ICER per case, death and disability-adjusted life year averted by age and risk group Gloria J. Kang 103 104 Appendix B.

Figure B.3: Cost-effectiveness planes for Healthy People vaccination compared to current vaccination. Cost saved vs. DALYs averted for the Healthy People vaccination scenario compared to current vaccination (20% VE).

(a) Plots by risk group

(b) Plots by age group Gloria J. Kang 105

B.2 Sensitivity Analysis

Figure B.4: Univariate sensitivity analysis in the current vaccination scenario: Incremental cost-effectiveness ratios varying vaccine efficacy. ICERs appeared to be sensitive tolow vaccine efficacy (10%). At vaccine efficacy of 10%, vaccination was no longer cost savingfor the 0-4 age group; this may be due to weaker herd protection effects from a lower efficacy vaccine. 106 Appendix B.