Behavioral Inoculation

BEHAVIORAL INOCULATION

An effective communication strategy to induce resistance

towardsAn effective negative communicat internet messagesion strategy about to the induce HPV resistance-vaccination? towards negative internet messages about the HPV-vaccination?

Written by: Marloes Beerling (890725-044-040)

Supervisors: Dr. L. Mollema Dr. E.W.M.L. de Vet

MSc programme: Applied Communication Science

Specialization: Strategic Communication in Innovation

Wageningen University Department of Social Sciences

MSc Thesis Chair Group: Knowledge, Information and Technology

Abstract

Background In 2009, the vaccination against the Human Papillomavirus (HPV), the virus that can cause cervical cancer, was included in the National Immunization Programme for 12 and 13 year old girls. The uptake of the HPV- vaccination was lower than expected (50%). It is believed that one of the main reasons for the low vaccination rate was the negative and incorrect media coverage about the vaccination, specifically on the internet. Therefore, communication tools are needed to make parents of 12 and 13 year old girls more resilient towards these media messages. The inoculation strategy (McGuire, 1961) is such a communication tool and posits that individuals can be made resistant to persuasive attacks by exposing them to weak arguments against their current attitude, including a refutation of these arguments (McGuire, 1961). The purpose of this study is to assess whether McGuire’s inoculation theory is an effective strategy to strengthen the attitudes of parents towards the HPV-vaccination.

Methods An online two-phase experiment with three conditions was carried out among 390 parents and guardians of 12 and 13 year old girls. Phase 1 consisted of a baseline measurement. In addition, during phase 1 participants in condition 1 were asked to read a message written according to the principles of behavioral inoculation and participants in condition 2 were asked to read the mini-magazine that is currently used by the RIVM to inform parents about the HPV-vaccination. Condition 3 was the control condition. Seven days after the participants completed phase 1, participants were invited to take part in phase 2 of the experiment. The participants read a persuasive attack in the form of a vaccine-critical internet message and filled out the follow-up measurement. Resilience towards persuasive messages was assessed by the outcome variables attitude extremity (how positive or negative parents are towards HPV-vaccination) and attitude certainty (how certain parents are about their attitude towards HPV-vaccination).

Results Participants in the inoculation condition (condition 1) did not have a greater feeling of threat than participants in the mini-magazine and the control condition, indicating that the experimental inoculation manipulation was not successful. Overall, the participants who received the mini-magazine scored higher at follow-up on attitude extremity and attitude certainty scores than the participants in the inoculation and control condition. All participants got more negative at follow-up measurement on attitude extremity scores, especially the prior positive and high involved participants; this negative change was significant for the inoculation and control condition.

Conclusions Behavioural inoculation is not an effective strategy to induce resistance to persuasion on the topic of HPV- vaccination. The behavioural inoculation treatment was less effective than the currently used HPV mini- magazine and only slightly more effective than not giving any additional information to the parents. Therefore, implementation of texts written according to the principles of behavioral inoculation is not recommended. In addition, the results suggest that the persuasive attack was responsible for the negative attitude change among parents. This finding indicates that more attention needs to be paid to strategic communication interventions that induce resistance towards negative media messages on the topic of HPV-vaccination.

Keywords Inoculation theory, behavioral inoculation, HPV-vaccination, public communication

Table of Contents 1. General Background ______4 1.1 Introduction ______4 1.2 Problem Statement ______5 1.3 Research Objective ______5 1.4 Overview of this report ______6 2. Theoretical Framework ______7 2.1 Inoculation Theory ______7 2.1.1 The inoculation mechanism ______7 2.1.1.1 Threat ______7 2.1.1.2 Refutational preemption ______8 2.1.2 Additional factors that influence behavioral inoculation ______8 2.1.2.1 Inoculation time ______8 2.1.2.2 Issue involvement ______9 2.1.2.3 Valence of prior attitude ______9 2.2 Operationalizing resistance to persuasion ______9 2.3 Empirical review______10 2.3.1 Empirical support for behavioral inoculation ______10 2.3.2 Empirical review of behavioural inoculation in health promotion ______10 3. Sub questions & hypotheses ______12 4. Research Method ______14 4.1 Target population ______14 4.1.1 Characteristics of dropouts ______14 4.1.2 Deletions ______15 4.2 Research design and procedure ______15 4.2.1 Procedure ______15 4.3 Information interventions ______16 4.3.1 Inoculation message (condition 1) ______16 4.3.2 Information brochure HPV-vaccination {condition 2) ______17 4.4 Variables ______17 4.4.1 Background variables ______17 4.4.2 Main variables ______18 4.4.3 Manipulation check ______19 4.4.3 Covariates ______20 4.4.4 Evaluation measures ______21 4.5 Statistical analysis ______21 5. Results ______22

5.1 Description of participants ______22 5.1.2 Comparison study population with Dutch population ______22 5.2 Distribution of variables ______23 5.3 Randomization check ______24 5.4 Manipulation check ______24 5.5 Correlations between covariates and outcome variables ______26 5.6 Hypothesis testing ______26 6. Discussion and Conclusion ______32 6.1 Main results ______32 6.1.1 Finding 1 ______32 6.1.2 Finding 2 ______33 6.2 Limitations ______33 6.3 Practical implications ______34 6.4 Recommendations for future research ______34 6.5 Final conclusions ______35 Acknowledgements ______35 References ______36 Appendices______40

1. General Background 1.1 Introduction Cervical cancer is the second most prevalent type of cancer among women in the Netherlands (RIVM, 2013). Every year 600 women are diagnosed with cervical cancer and on a yearly basis 200 - 250 women die because of this disease in the Netherlands. Cervical cancer is caused by the Human Papillomavirus (HPV), a virus transmitted through sexual intercourse. In few cases this virus alters body cells, leading to cervical cancer. Women aged between 30 - 60 years are checked for precancerous tissue changes once every five years by a routine screening (Hofman, 2014). In addition to the screening, a bivalent vaccination against the two most common variants of the Human Papillomavirus (HPV16 and HPV18) became available in 2007. Together, these two variants are responsible for 70% of the cervical cancer cases. Expected is that, on the long term, the vaccination will prevent 50% to 70% of the cervical cancer cases, provided that the vaccination is taken by the majority of the target group (Schiffman & Castle, 2003). To date, follow-up data of girls vaccinated against HPV confirm that the HPV-vaccination is effective in preventing an HPV-infection for at least 9.4 years (Naud et al., 2014). The HPV-vaccination is most effective when it is taken prior to the first sexual intercourse, as the effectiveness of the vaccination decreases after the first infection with HPV (Naud et al., 2014).

In 2009, the Dutch government included the bivalent HPV-vaccination, consisting of three shots1, in the National Immunization Program (NIP), which is voluntary and free of charge. The RIVM (the Dutch National Institute for Public Health and Environment) is responsible for the management and evaluation of the NIP. In 2009, girls in the age category between 12 to 16 years were invited to participate in the so-called catch-up campaign. From 2010 onwards, girls are invited to vaccinate against HPV in the year they turn 13 years old. While the Dutch law assigns full responsibility to the girls in deciding whether they want to get vaccinated against HPV, in practice the attitude of the parents - and especially the attitude of the mothers – plays a key role in the decision making process of their daughters (Van Keulen et al., 2010).

The introduction of the HPV-vaccination was accompanied by an elaborate educational campaign executed by the RIVM, as the acceptation of the vaccine was not self-evident (Conyn-van Spaendonck, 2010). Many aspects of the HPV-vaccination were different from the other vaccinations included in the NIP: the HPV-vaccination focused on a new target group; the goal of the vaccination is to prevent cancer instead of a childhood disease; the long-term effectiveness and possible health-risks associated with the vaccine were (and still are) unknown; the vaccine targets a disease on the long-term and a preventative measure to detect cervical cancer (the routine screening) is already present (Conyn-Van Speandonck, 2010; Hofman, 2014). Nevertheless, the Dutch government and the RIVM expected that at least 70% of the girls would decide to vaccinate against HPV.

However, the vaccination rate was much lower than expected: In May 2009 only 50% of the girls had decided to get all three shots (Van Lier et al., 2009). Currently, the HPV-vaccination coverage is 59% (2014), while the average vaccination coverage of childhood diseases in the NIP is 95% (Van Lier et al., 2014). It is believed that one of the main reasons for the low vaccination rate was the negative media coverage about the HPV- vaccination. While the majority of the traditional media communicated both the vaccination advice of the RIVM and the criticism on this advice, the internet was, to a larger extend than traditional media, negative about the HPV-vaccination (Van Keulen et al., 2010). Especially on anti-vaccination websites many tall stories about the HPV-vaccination circulated (van Keulen et al., 2010). Some websites claimed that HPV-vaccination could lead to serious health problems, such as infertility and paralysis. Others claimed that there was no proof that the vaccination would do more good than harm and wrote that “it is one big experiment!” (Conyn-van Spaendonck, 2010). Other arguments used were that the components of the vaccine were not safe and that HPV-vaccination was ineffective in preventing cervical cancer (Kata, 2010). These arguments were

1 From January 2014 onwards the HPV-vaccination has been reduced to two shots, as research of the manufacturer of the HPV-vaccine in the Netherlands (GlaxoSmithKline) found that two vaccinations are as effective as three vaccinations (RIVM, 2015).

accompanied by photographs, pictures and personal stories in order to appeal to the emotions of their readers (Wolfe, Sharp & Lipsky, 2002). Research confirms that a majority of the anti-vaccination websites (68%) contain personal and emotional content, such as pictures and stories of children who supposedly got harmed by vaccinations (Betsch, 2011; Zillien et al., 2008). In comparison: only 19% of these websites referred to scientific information (Betsch, 2011).

Despite the contested content of anti-vaccination websites, many people believe and act upon the information given on anti-vaccination blogs and websites. Research of Betsch et al. (2010) shows that browsing an anti- vaccination website for 5 to 10 minutes significantly increases the perception of threat related to vaccination and significantly decreases the vaccination intention of the participants. Van Keulen et al. (2010) confirm this finding in the context of HPV-vaccination: they found that media sources that brought critical news about the HPV-vaccination had a negative influence on the HPV-vaccination intention. In addition, Van Keulen et al. (2010) found that significantly more mothers who use the internet as an information source decided not to (fully) vaccinate their daughter than those who did not use the internet to read information about the HPV- vaccination. Considering that 46% of the parents look for vaccination information online and that social media are the main information source for girls about the HPV-vaccination (Van Keulen, 2010), it is believed the negative internet coverage was an important factor in the low vaccination rate (Rondy et al., 2010).

1.2 Problem Statement The RIVM, the National Institute for Public Health and Environment, is the official provider of information about the HPV-vaccination. For the RIVM it is problematic that daughters and their parents are influenced by the information that is offered on anti-vaccination websites. The goal of the RIVM concerning the HPV- vaccination, and the National Immunization Program in general, is to provide their citizens with the information and knowledge necessary to make a well-informed choice about their health (Conyn-van Spaendonck, 2012). At the same time, the goal of the RIVM is to reach a vaccination rate that is as high as possible, as this leads to a reduction in the number of cervical cancer cases. Therefore, the RIVM is looking for ways to make daughters and their parents more resilient towards the negative and incorrect messages offered online. The main communication means the RIVM uses to inform parents and daughters about the HPV-vaccination is the brochure ‘Information about the HPV-vaccination’, also referred to as the HPV mini-magazine. In this brochure the advocated opinion is that the HPV-vaccination should be taken, as it promotes public health by lowering the number of cervical cancer cases (Van Poppel, 2011). For 55% of the parents and 78% of the daughters the information brochure provided by the RIVM had an influence on their HPV-vaccination decision (TNO Communication Report, 2013). Therefore, the information brochure is a powerful tool for the RIVM to prepare parents and daughters against the negative messages provided on the internet.

1.3 Research Objective The goal of this study is to design and evaluate a communication strategy that can strengthen the resilience of attitudes towards HPV-vaccination of parents. To gain insights into communication strategies that are intended to shape, reinforce or change the responses to a message (Miller, 1980), we turn to the field of persuasive communication. A specific string of research in the field of persuasive communication that concerns itself with resistance towards persuasion is the field of ‘behavioral inoculation’, introduced by McGuire in 1961. McGuire proposes that behavioral inoculation resembles the process of biological immunization. When a person is exposed to weak arguments against his or her current attitude, including a refutation of these arguments, this increases resistance when the individual is exposed to stronger counterarguments later on (McGuire, 1961). Pfau et al. (2003) claim that “there is no question that inoculation works” in both laboratory and applied settings (p. 39). A recent meta-analysis of Bains and Ranas (2010) confirms once more that behavioral inoculation is an effective strategy to increase resistance towards persuasion on a variety of topics. Also in the field of health-related behaviors inoculation theory has been applied few times, specifically on the topics of smoke and alcohol prevention, for example by Pfau, Van Bockern & Kang (1992) and Pfau & Van Bockern (1994). In the limited research conducted so far, the results of behavioral inoculation in the field of health-

related behaviors are promising. Therefore, several researchers have pointed out the need for application of inoculation theory in a wider variety of health settings (Compton & Pfau, 2009; Godbold & Pfau, 2000). Other researchers have suggested that behavioral inoculation might be valuable for communication about vaccinations (Paulussen et al., 2006; Harmsen, 2014) and for communication about the HPV-vaccination (Klop, 2014). With this study, we set out to see if the inoculation theory is indeed a successful communication strategy to strengthen attitudes and build resistance towards persuasion on the topic of HPV-vaccination among parents of 12 and 13 year old girls. Therefore, the main research question is formulated as follows:

Is ‘behavioral inoculation’ an effective strategy to induce resistance towards negative internet messages about the HPV-vaccination among parents of 12 and 13 year old girls?

The aim of this study is twofold. The first aim of this study is to contribute to the limited scientific body on behavioral inoculation research conducted in the health setting so far, specifically on the topic of (HPV-) vaccination. The second aim of this study is related to the practical implications of this research. As this study is commissioned by the RIVM, the goal is to implement texts written according to inoculation theory in the communication means of the RIVM about the HPV-vaccination, if behavioral inoculation is indeed effective in creating more resilient attitudes towards negative and incorrect media messages about the HPV-vaccination.

1.4 Overview of this report After this first introductory chapter, the second chapter of this thesis covers the theoretical concepts underlying this research. In chapter three the secondary research questions and the related hypotheses are posed. In chapter four the methodology used for this study is described; starting with a description of the target population, the study design and the research procedure and followed by an explanation of the (experimental) interventions materials used for this study and an overview of the items included in the questionnaires. In chapter five the results of this study are presented. In the following sixth chapter conclusions are drawn from the results described in chapter five. In addition, the limitations of this study are discussed, followed by the practical implications of this study and the recommendations for further research.

2. Theoretical Framework In this chapter, the theory of behavioral inoculation is further explained. In paragraph 2.1 the origin and mechanisms of inoculation theory are described. In paragraph 2.2 the concept of attitude strength is explained. Subsequently, in paragraph 2.3 an empirical review is given of the inoculation research conducted so far.

2.1 Inoculation Theory Inoculation research examines the process by which people can be made resistant to persuasion (Wood, 2007). The goal of inoculation theory is to strengthen currently held attitudes, thereby building resistance to future persuasive attacks (McGuire, 1961a; Gass & Seiter, 2003). The theory of attitude inoculation is based on early research about one-sided and two-sided messages (Lumsdaine & Janis, 1953). One-sided messages are messages that only consider one side of the argument (foe example: alcohol is bad for you), while two-sided arguments look at both sides of an argument (for example: drinking alcohol is bad for you, but it also helps you having a good time) (Lumsdaine & Janis, 1953). While one- sided and two-sided messages had the same short-term effect, the effect of two-sided messages was more long lasting, especially when the counterarguments were refuted in the message. This led to the idea of “attitude inoculation” (McGuire, 1961a; McGuire & Papageorgis, 1961). Inoculation theory claims that attitude inoculation resembles the process of medical immunization (or “inoculation”). With medical immunization a person is exposed to a weak version of a virus, which causes the immune system to start producing antibodies and thereby builds resistance to the virus. When the body is exposed to a non-weakened virus later on, the body is prepared to protect itself against the virus. Thus, the body will not get ill. McGuire claimed that the principle of immunization can also be applied to beliefs and attitudes. When a person is exposed to weak arguments countering their current attitude, including a refutation of these arguments, this increases resistance when the individual is exposed to stronger counterarguments later on (McGuire, 1961a). It is key that the threat is strong enough to trigger the individual’s defense, yet it should not be so strong that it changes the current attitude unfavorably (Pfau & Szabo, 2006). Initially, McGuire narrowed inoculation theory to cultural truisms, described by McGuire as beliefs that are commonly held in society (for example: brushing your teeth is good). McGuire assumed that, as with medical immunization, beliefs that were never challenged were most vulnerable to attacks, as the receiver is not trained or motivated to defend these attitudes. Later, other researchers extended the application of inoculation theory to other, more controversial beliefs and attitudes (among others: Adams & Beatty, 1977). Recent examples include the application of inoculation theory to genetically modified food (Wood, 2007), animal testing (Nabi, 2003) and legalizing drugs (Pfau et al., 1997).

2.1.1 The inoculation mechanism An inoculation message consists of two elements: a threat and a refutational preemption of that threat. The element of threat makes the participants aware of the vulnerability of their attitudes; the process of refutational preemption gives the participants the information to strengthen their current attitudes (Compton & Pfau, 2007). In the following paragraphs, both concepts are explained more elaborately.

2.1.1.1 Threat The inoculation message starts with posing a threat to the currently held attitudes of the receiver, by claiming that the currently held attitude is under attack. This threat serves as a motivational trigger: the individual realizes that his or her current attitude is vulnerable, making the individual motivated to strengthen his attitude. This will motivate the individual to consciously read and process the inoculation message, setting in motion the internal process of resistance (Compton & Pfau, 2007; Pfau & Szabo, 2006). Researchers, under which Compton and Pfau (2005) and McGuire (1964), claim that especially the element of threat is key to inoculation theory, stating that “inoculation is impossible without threat” (2005, p. 100) and that “threat is the most distinguishing feature of inoculation” (Pfau, 1997, p. 137). The threat component broadens the utility of inoculation theory, as by starting the process of internal resistance, behavioral inoculation does not only

protect against the counterarguments refuted in the refutational preemption, but also against arguments not refuted in the inoculation message. Thus, inoculation theory creates a “broad blanket of protection against specific counterarguments raised in refutational preemption and against those counterarguments not raised” (Pfau, 1997, p. 137). It is important to note that by the use of inoculation theory individuals are stimulated to process information consciously. This relates to the Elaboration Likelihood Model (ELM), a framework that provides insights in the processes underlying persuasive communication messages (Petty & Cacioppe, 1986). The ELM is based on the assumption that people are not able or motivated to evaluate every message they receive. Therefore, some messages are thoroughly processed via the central route, while other messages are only processed in a more shallow, unconscious way, via the so-called peripheral route. The ELM claims that attitudes that are constructed through the central route are more stable and less easily influenced. The goal of inoculation theory is to create stable and strong attitudes and to increase attitude strength; the threat element plays an important role in stimulating individuals to consciously process the inoculation message via the central route.

2.1.1.2 Refutational preemption After the individual is made aware of the vulnerability of his attitude, the process of refutational preemption takes place, referred to by Pfau & Szabo (2006) as “replying to counterarguments before they occur” (p. 235). The inoculation message raises opposing arguments and then counters them. The refutational preemption provides specific information that the individuals can use later on to protect and defend their attitudes. In the time between the inoculation treatment and the ‘real-life’ attack, the individual is motivated to keep on generating counterarguments and refutations (McGuire, 1964). The two components, threat and refutational preemption, work together; first threat, then refutational preemption (Pfau, 1997, p. 137). Threat provides motivation, while refutational preemption provides scripts (McGuire, 1964). In inoculation research, two variations of the refutational preemption exist, namely passive and active refutations, referring to the extent to which the message calls upon active participation of its readers. In the original research of McGuire (1961a) participants were given arguments against their current attitude plus a request to write refutations to these arguments. McGuire referred to this approach as ‘active refutation’, as the participants had to actively come up with and write content themselves. In later research by McGuire & Papageorgis, 1961) the inoculation message already contained the refutations of the counterarguments. The participants were only asked to read the refutations, McGuire & Papageorgis (1961) refer to this procedure as ‘passive refutation’. In contrast with the expectations of McGuire and Papageorgis, passive refutation was superior in building resistance. They hypothesized that passive refutations gave the participants the specific content to defend their attitude, without placing too much of a cognitive burden on the participants. Since then, inoculation research based on passive refutations has been the core of inoculation research, thereby placing more emphasis on the design of effective inoculation messages (Banas & Rains, 2010).

2.1.2 Additional factors that influence behavioral inoculation Next to the elements of threat and refutational preemption, research has identified several other elements that influence the process and /or outcomes of behavioral inoculation, namely: (1) the amount of time between the inoculation message and the attack message, (2) the involvement of the receiver (on the inoculation topic) and (3) the valence of the prior attitude of the receiver (Banas & Rains, 2010; Compton & Pfau, 2004). In the following paragraphs, these moderating elements will be further described.

2.1.2.1 Inoculation time An element that can influence the inoculation process is the time between the inoculation message and the simulated persuasive attack, the so-called “inoculation time” or “inoculation delay” (McGuire, 1961b). The literature shows mixed results when it comes to the optimal delay between the inoculation message and the persuasive attack. Godbold & Pfau (2000) found better results for an immediate attack (1 day) than for an attack two weeks later. Others (among others Wood et al., 2007) found better results with a two-week interval.

Also the meta-analysis of Banas & Rains (2010) does not provide any clear-cut results on the most efficient interval between the inoculation and the attack message, as they found that inoculation is as effective after two days as after thirteen days. However, their results do show that the interval should not exceed 13 days.

2.1.2.2 Issue involvement Another element that plays a role in the inoculation process is the extent to which a person is concerned with a topic, referred to as ‘issue involvement’ (Petty & Cacioppo, 1979). Different studies have investigated the optimum level of issue involvement for behavioral inoculation to work best. These studies do not provide univocal results. For example, Pfau et al. (2010) found that if a person is highly involved in the topic of the inoculation message, the person will be more motivated to actively process the inoculation message and thus the behavioral inoculation is more effective. In contrast, another study by Pfau et al. (2003) found that individuals who are highly involved in an issue are less sensitive for an inoculation treatment: their high level of involvement ensures that the person is already alert and prepared for counterarguments, which leads to the threat having no effect. Another study found that Individuals who have low involvement in a topic might not experience the threat as such and thus: individuals are not motivated to start the internal process of resistance (Compton & Pfau, 2005). Because of the many contradictory findings, Banas & Rains (2010) recently conducted a meta-analysis in order to gain insights in the level of issue involvement on which behavioral inoculation is most effective. They found that there is no significant relation between level of involvement and the effectiveness of inoculation. Yet, it is clear that issue involvement has some role in the process of behavioral inoculation (Banas & Rains, 2010).

2.1.2.3 Valence of prior attitude The valence of the prior attitude towards the topic also has an influence on the effectiveness of behavioral inoculation. In the majority of the existing inoculation research, the inoculation message is tailored towards the currently held attitude. Thus, if a person is negative towards a topic, the individual will be asked to read an inoculation message that reinforces this negative attitude and vice versa. Although this approach confirms the behavioral inoculation mechanism in theoretical terms, it leaves out the real world context, in which it might not be possible to tailor messages towards the prior attitude of the individual (Wood et al., 2007; Littlejohn, 1978). Therefore, there has been recent interest in the influence of behavioral inoculation on individuals with an opposing position towards the opinion advocated in the inoculation message. A study by Wood et al. (2007) entailed a three-phase-experiment on the topic of agricultural biotechnology. Her results show that all participants, also those with initially neutral and negative standpoints towards the topic, got slightly more positive towards the topic of agricultural biotechnology after inoculation. Therefore, the research of Wood confirmed that the previously held attitudes influence the outcomes of behavioral inoculation.

2.2 Operationalizing resistance to persuasion In order to assess whether an inoculation message is effective in inducing resistance to persuasion, the study design of inoculation research includes a control group and a simulated persuasive attack (McGuire, 1961a). By pre- and post-measuring the attitude of the inoculated group and the control group towards the topic inoculated for, it can be assessed whether the inoculation message was effective in increasing resistance. The construct that is widely used in behavioral inoculation studies to assess whether an attitude became more resilient towards persuasive attacks, is the concept of ‘attitude strength’. Attitude strength is defined by Krosnick & Smith (1999) as the degree to which an attitude is resistant to change and influences cognition and behavior. Krosnick & Smith (1999) claim that an attitude is strong when it has the following four characteristics: (1) a strong attitude is resilient to persuasive attacks, (2) a strong attitude is stable over time, (3) a strong attitude induces the motivation to process information thoroughly and (4) a strong attitude determines behavioral intention and behavior to a higher extent than a weak attitude (Petty & Krosnick, 1994). Several researchers have studied what distinguishes strong attitudes from weak attitudes and thus: what causes an attitude to be strong (Petty & Krosnick, 1994). Different attributes that influence attitude

strength have been identified. In this study we will follow the division of Pomerantz et al. (1995). Pomerantz identified five attributes of attitude strength: (1) personal importance, (2) ego involvement, (3) knowledgeability, (4) attitude-certainty and (5) attitude-extremity. The first attribute of attitude strength, personal importance, describes the degree to which an individual feels committed to an object and the extent to which one finds an object or attitude important. If a person finds an object important, it is assumed that the attitude strength is higher. The second attribute, ego involvement, describes the relationship between an individual’s values and the attitude towards the object or issue (Sherif & Cantril, 1947). Ego involvement refers to the belief that one’s attitude reflects an inner self- value (Pomerantz et al., 1993). When the ego involvement is high, the attitude towards the object is likely to be stronger. The third attribute of attitude strength, knowledgeability, refers to the amount of information an individual has about an object that accompanies his or her attitude (Petty & Krosnick, 1994). Related to this dimension is the ease with which an individual can retrieve this information. If a person is knowledgeable about an object related to an attitude, the attitude is likely to be stronger (Pomerantz et al., 1995). The attribute attitude extremity describes the degree to which the individual is positive or negative towards an object. An attitude is seen as more extreme when it deviates more from the neutral midpoint (Pomerantz et al., 1995). The third dimension, attitude certainty describes the extent to which the receiver is certain that his or her attitude towards an object is correct (Pomerantz et al., 1995). The assumption is that the more certain an individual is about their currently held attitude towards an object, the stronger their attitude becomes. Several researchers (among others Krosnick et al. 1993 & Pomerantz et al., 1995) have claimed that especially the combination of the attributes attitude extremity and attitude certainty form a strong indicator of resistance to persuasion.

2.3 Empirical review In order to gain insights in the effectiveness and outcomes of behavioral inoculation as a frame of reference for this study, in the following paragraphs several inoculation studies will be discussed. We start by discussing the empirical support of inoculation theory. Subsequently, applications of behavioral inoculation in the health setting are discussed.

2.3.1 Empirical support for behavioral inoculation Behavorial inoculation has been successfully applied to many different topics, such as political campaigning (Pfau & Burgeoon, 1988), advertising (Pfau et al., 1992), credit card usage (Compton & Pfau, 2004), health promotion (Van Bockern & Kang, 1992) and crisis communication (Wan & Pfau, 2004).The meta-analysis of Banas & Rains (2010) confirmed once more that, based on 52 articles, behavioral inoculation is effective in fostering resistance to persuasion. However, it is important to note that some individual inoculation studies have found no support for inoculation theory. For example, Benoit (1991) conducted two behavioral inoculation studies; one on the topic of abortion and one on the topic of the justice system. Both studies did not support the notion that behavioral inoculation increases resistance and leads to increased counter arguing. In addition, Benoit (1991) found that a supportive treatment was more effective in increasing resistance to persuasion. He concluded that inoculation might only be effective on topics that are not controversial. However, as previously mentioned, later behavioral inoculation studies on controversial topics such as animal testing (Nabi, 2003) and legalizing drugs (Pfau et al., 2003) have found positive results. In addition, the meta-analysis of Banas & Rains (2010) found that overall inoculation was more effective than both the supportive treatment and the ‘no information treatment’ in creating resilient attitudes, contrasting the individual findings of the study of Benoit (1991).

2.3.2 Empirical review of behavioural inoculation in health promotion Although behavioral inoculation has been investigated in a variety of domains, relatively few of these studies focused on the health domain. The few studies that were performed mainly focused on alcohol and smoking prevention among youngsters. These studies have consistently found support for inoculation theory (Van Bockern & Kang (1992); Pfau & Van Bockern (1994); Hansen & Graham, 1991; Duryea, 1984). The study on

smoke prevention by Bockern & Kang (1992) and Pfau & Van Bockern (1994) even found that eighty-four weeks after the initial inoculation intervention, the inoculation message still had a positive effect. To our knowledge, one behavioral inoculation study has been performed on the topic of HPV- vaccination by Goodings (2012). He found that, when participants were highly involved and when they had a prior positive attitude towards HPV-vaccination, participants got more resilient towards negative messages about the HPV-vaccination. In addition, he found that the inoculation message had no influence on participants with a prior negative or neutral attitude. Therefore, he advised to implement texts written according to the principles of behavioural inoculation in the communication about the HPV-vaccination. However, it should be noted that Goodings only found minor, but significant, differences between the participants exposed to the inoculation message and the participants in the control condition.

3. Sub questions & hypotheses Based on the introduction and the theoretical framework, four secondary research questions are posed. The goal of these four research questions is to compare the effects of the inoculation message, the currently used mini-magazine, and ‘no additional information’ with each other, in order to gain insights in which strategy is most effective in inducing resistance to persuasion. In this study, following Krosnick et al. (1993) resistance to persuasion is operationalized with two outcome variables, namely attitude extremity and attitude certainty. It is important to note that for the outcome variable attitude extremity the goal is to reach more positive attitudes, not to reach more extreme attitudes.

The first secondary research question aims to answer whether behavioral inoculation is more effective in increasing the resilience of parents and legal guardians of 12 and 13-year old girls towards the HPV-vaccination than the currently used mini-magazine and no additional information. Therefore, the first secondary question is formulated as follows:

Sub question 1 Is behavioral inoculation more effective than the HPV mini-magazine and ‘no additional information’ in increasing the extremity and certainty with which parents hold their attitude towards HPV-vaccination?

In order to answer this research question, the following two hypotheses are posed:

H1: At follow-up measurement, participants who received the inoculation message have more positive attitudes towards HPV-vaccination than the participants who received the HPV mini-magazine or no additional information.

H2: At follow-up measurement, participants who have received the inoculation message are more certain about their attitudes than the participants who received the HPV mini-magazine or no additional information.

The second secondary research question is related to the concept of issue involvement and is formulated as follows:

Sub question 2 What is the effect of issue involvement on the ability of behavioral inoculation in increasing the extremity and certainty with which parents hold their attitude in comparison to the HPV mini-magazine and ‘no additional information’?

Although in the literature there is no consensus on which level of issue involvement work best, in this study we base our hypotheses on the similar study of Goodings (2012), who found that behavioral inoculation is most effective in inducing resistance to persuasion when individuals are high involved. Therefore, the following hypotheses are posed:

H3.1: At follow-up measurement, high involved participants who have received the inoculation message remain more positive and certain about their attitude than the high involved participants who received the HPV mini-magazine or no additional information.

H3.2: For the low involved participants, there are no differences between the participants that received the inoculation message, the HPV mini-magazine or ‘no additional information’ on the extremity and certainty with which parents hold their attitude towards HPV-vaccination.

The third secondary research question aims to answer what influence behavioral inoculation has on parents and guardians with either a supportive or an opposing attitude towards the opinion advocated in the

inoculation message. As noted in paragraph 2.1.3.3, little is known about the effect of behavioral inoculation on individuals with an attitude opposing the opinion advocated in the inoculation message. As a big minority of the parents and girls are negative towards HPV-vaccination, it is very relevant to gain insights in these effects. Therefore, the following secondary research question is posed;

Sub question 3 What is the effect of attitude valence (either a positive or a negative attitude towards HPV- vaccination) on the ability of behavioral inoculation in increasing the extremity and certainty with which parents hold their attitude in comparison to the HPV mini-magazine and ‘no additional information’?

In order to answer this research question, the following two hypotheses are posed, based on the findings of Wood et al. (2007) and Goodings (2012):

H4.1: At follow-up measurement, participants with a prior positive attitude who have received the inoculation message remain more positive and certain about their attitude than participants with a prior positive attitude who received the HPV mini-magazine or no additional information.

H4.2: For the participants with a prior negative attitude, there are no differences between the participants that received the inoculation message, the HPV mini-magazine or ‘no additional information’ on the extremity and certainty with which parents hold their attitude.

Finally, in order to gain insights into the (sub) groups for which behavioral inoculation works best, we are interested in the effect of the level of issue involvement combined with either a positive or a negative attitude. Therefore, the following fourth research question is posed:

Sub question 4 What is the effect of attitude valence (either a positive or a negative attitude towards HPV- vaccination) combined with the level of issue involvement on the ability of behavioral inoculation in increasing the extremity and certainty with which parents hold their attitude in comparison to the HPV mini-magazine and ‘no additional information’?

In order to answer this research question, the following four hypotheses (based on Goodings, 2012) are tested in this research:

H5.1: At follow-up measurement, high involved participants with a prior positive attitude that have received the inoculation message remain more positive and certain about their attitude than the participants who received the HPV mini-magazine or no additional information.

H5.2: At follow-up measurement, high involved participants with a prior negative attitude that have received the inoculation message remain more positive and certain about their attitude than the participants who received the HPV mini-magazine or no additional information.

H5.3: For the participants with a prior negative attitude and a low issue involvement, there are no differences between the participants that received the inoculation message, the HPV mini-magazine or ‘no additional information’ on the extremity and certainty with which parents hold their attitude.

H5.4: For the participants with a prior positive attitude and a low issue involvement, there are no differences between the participants that received the inoculation message, the HPV mini-magazine or ‘no additional information’ on the extremity and certainty with which parents hold their attitude.

4. Research Method In this chapter the research method used to perform this study is described. First, the target population is described in paragraph 4.1. Then, the research design and the research procedure are explained in paragraph 4.2. In paragraph 4.3 the information interventions are described, followed by an elaborate description of the constructs and items used for the questionnaire in paragraph 4.4.

4.1 Target population The target group of this study are parents and legal guardians of girls who are invited to vaccinate against HPV in 2015. In the Netherlands, girls are invited to vaccinate against HPV in the year they turn 13 years old. A sample of the girls invited for vaccination in 2015 was drawn from the vaccination information database Praeventis of the RIVM; their parents or guardians were approached to participate in this study. The goal was to draw a random sample. Unfortunately, because of a sampling error the participants within the three conditions were from the same regional areas, therefore the randomization failed. The regional distributions of the participations over the three conditions are included in Appendix 6. Research of Van Keulen et al. (2010) shows that especially mothers play an important role in the decision of their daughters on HPV-vaccination. Therefore, in the invitation letter it was explained that the research mainly focused on the opinion of mothers towards HPV-vaccination. However, fathers and male guardians were also welcome to participate if they were closely involved with the vaccination decision of their daughter.

For this study the goal was to reach a number of 80 participants per research condition (n = 240). Based on a dropout follow-up rate of 30%, a minimum number of 312 participants needed to fill out the first questionnaire. Based on the experience of the RIVM, a response-rate of 7,5% was taken into account. Therefore, 4158 parents and/ or guardians were approached to participate in this study. In the end, a total of 999 (24.0%) participants completed the first questionnaire; a number of 534 participants completed both questionnaires, of which 496 (11.9%) questionnaires were eligible for measurement. Figure 1 displays the number of parents and guardians that participated in the baseline and follow-up questionnaires.

4.1.1 Characteristics of dropouts The dropout percentage was approximately the same among the three conditions: for the inoculation condition 347 participants filled out the baseline questionnaire and 166 (47.4%) of them completed both questionnaires. For the mini-magazine condition 331 participants filled out the baseline measurement; of them 180 (54.4%) completed both questionnaires. For the control condition 324 participants filled out the baseline questionnaire, of them 150 (46.3%) filled out both questionnaires. A comparison of the demographic characteristics of the participants who only completed the baseline questionnaire and the participants who completed both questionnaires shows that – on average – slightly less participants with a low educational level participated in both questionnaires (16.8% at baseline versus 11.5% at follow-up). In addition, less participants with an Islamic, Jewish, Buddhist or Hindu background participated in the follow-up questionnaire (5.7% versus 3.4%).

Figure 1: Flow diagram of questionnaires at 14 baseline and follow-up.

4.1.2 Deletions Because of a technical error, a number of 106 participants in the mini-magazine condition were not able to open the mini-magazine. As the goal of this study was to gain insights in the effectiveness of the communication interventions ‘inoculation’ and ‘mini-magazine’, it was decided to only analyze those participants that were exposed to the intervention as intended. Therefore, 106 participants in the mini- magazine condition were deleted from further analysis. All the analyses that follow in the next paragraphs, are performed on the mini-magazine condition with a number of 74 participants.

4.2 Research design and procedure The study has an experimental design (see Figure 2), with three conditions. Participants were randomly assigned to one of the three conditions. The experiment consisted of two phases. Phase 1 consisted of a baseline measurement. In addition, participants in condition 1 and 2 received an information intervention: the participants in condition 1 were asked to read the inoculation message and participants in condition 2 were asked to read the information brochure (‘mini-magazine’) currently used to inform parents and daughters. Condition 3 did not receive any additional information. Phase 2, for which parents were invited seven days after they participated in phase 1, consisted of a ‘persuasive attack’ and a follow-up measurement for all participants.

Figure 2: Research Design.

4.2.1 Procedure Parents were invited to participate in this study by a letter. The letter contained information about the research, contact information of the researchers and a personal code. In addition, the letter contained a link to the questionnaire. Randomization was done beforehand, meaning that the invited participants got a website- link letter to either the questionnaire for the inoculation condition, the mini-magazine condition or the control condition. The invitation letter is included in Appendix 2. A week after the initial invitation, a reminder was send by post to the parents who did not yet fill out the questionnaires. The reminder letter is included in Appendix 3. From January 7 to February 13 2015, the online experimental questionnaire was conducted. The online platform ‘Questback’ (www.questback.nl) was used to distribute and conduct the questionnaires. Before the participants started with the questionnaire, a short introduction to the research and the questionnaire was given. In addition, the participants were asked to read some general information about HPV and the HPV- vaccination, in order to give the parents a minimum amount of information before filling out the questionnaire. Then, all participants were asked to fill out a pre-measurement questionnaire about (1) their demographics, (2) their intention towards HPV-vaccination, (3) their current attitude towards HPV-vaccination, (4) the strength of their current attitude, (5) their beliefs about the vaccination, (6) their risk perceptions and outcome

expectations, (7) their issue involvement, (8) their trust in the institutions involved with the HPV-vaccination and (9) the extent to which they thought critically about information about the HPV-vaccination. Subsequently, the participants were asked to perform different tasks depending on the assigned research condition. The participants in condition 1 were asked to read the inoculation message. In paragraph 4.3.1 the content of the inoculation message is further explained. Afterwards, participants in this condition were asked whether (and if yes, how intensively) they read the inoculation message. The participants in condition 2 were asked to read the brochure that is currently used to inform parents and daughters about the HPV-vaccination. In paragraph 4.3.2 the content of this brochure, in the form of a mini-magazine, is further explained. Participants were given the option to open the mini-magazine in a PDF-file, giving them the choice whether they wanted to read the mini-magazine or not, thereby resembling reality. Afterwards the parents in this condition were asked if they had opened the mini-magazine and how intensively they read it. The participants in the control condition did not read any additional information about the HPV-vaccination. Lastly, all participants were asked to rate the vulnerability of their attitude. The goal of this measurement was to assess whether inoculation manipulation was effective.

After seven days, all participants were contacted via email with the request to participate in the second questionnaire of the study. A reminder was send after seven days to the parents who did not yet fill out the second questionnaire. The invitation (and reminder) e-mail contained a link to the second online questionnaire. After a short introduction, all three groups were asked to the read the ‘persuasive attack’. For the persuasive attack an article was used that appeared on a website of a Dutch newspaper (De Telegraaf) on March 20, 2012. The article tells the story of the 13 year old Sarina, who claims to have experienced severe and long-term side effects after her HPV-vaccination. The article also contains a black and white picture of Sarina and her mother (for the article, see Appendix 4). The choice for this article as the persuasive attack was based on the following considerations: first, the article is grammatically well written in comparison to other articles that appear on anti-vaccination websites. In addition, from the perspective of the RIVM, the article poses health risks associated with HPV-vaccination, yet it also provides other alternative, explanations for Sarina’s health problems. After reading the persuasive attack, the respondents were asked to fill out a follow-up questionnaire on the following topics: (1) their behavioral intention towards HPV-vaccination, (2) their attitude towards HPV- vaccination, (3) the strength of their attitude, (4) their beliefs about the vaccination, (5) their involvement towards the HPV-vaccination, (6) trust in the institutions involved with the HPV-vaccination and (7) the extent to which they thought critically about information about the HPV-vaccination. Afterwards, participants in all conditions were asked to evaluate the persuasive attack. In addition, the participants in the inoculation condition were asked to rate the inoculation message they read in the first questionnaire on readability, clearness and trustworthiness.

This study was approved by the Social Sciences Ethical Committee of Wageningen University (see Appendix 1 for the letter of approval).

4.3 Information interventions 4.3.1 Inoculation message (condition 1) The inoculation message consisted of (1) an introduction and (2) the raising and refutation of anti-vaccination arguments. In the introduction of the inoculation message it was claimed that some of the arguments used on anti-vaccination websites are incorrect. Then, it was stated that this could lead to a vaccination decision based on incorrect information. The goal of the introduction was to make parents aware that their current attitude towards the HPV-vaccination was vulnerable, thus functioning as a motivational trigger to process the second part of the inoculation message, the refutational preemption, elaborately. Subsequently, the anti-vaccination arguments were raised. To assess which arguments are commonly used on anti-vaccination websites, an assessment was made of three popular anti-vaccination websites in the Netherlands, namely:

www.vaccinatieraad.nl, www.nvkp.nl and www.vaccinvrij.nl. Most arguments against HPV-vaccination focused on the vaccine being unsafe when administered and the negative health effects of the vaccine on the long term. Based on this information, the following three anti-vaccination arguments were selected to refute in the inoculation message:

[1] HPV-vaccination has severe long-term side effects. [2] The components of the HPV-vaccine are unsafe. [2a] The HPV-vaccine contains insect cells. [2b] The HPV-vaccine contains rat poison.

Refutations – The anti-vaccination arguments were refuted based on the information on the RIVM website. In addition, several scientific articles were consulted for more information about why the anti-vaccination arguments were not true. The anti-vaccination arguments were refuted according to the guidelines of Betsch (2014) and Lewandowsky et al. (2012) about debunking myths about vaccinations. They propose that before an anti-vaccination argument is raised it is important that the reader is warned that an anti-vaccination argument will be mentioned. In addition, Lewandowsky et al. (2012) claim that retractions can cause confusion in the mind of the readers. By only claiming that the information was incorrect, a coherence gap can arise (“If it wasn’t this, what else could it be?”). The resulting gap may motivate reliance on the previously held misinformation. Therefore, it is important that the source of the misinformation are explained and an alternative explanation for why the myth is incorrect is given. The final inoculation message is included in Appendix 4.

4.3.2 Information brochure HPV-vaccination {condition 2) Participants in condition 2 were asked to read the mini-magazine that is currently used by the RIVM to inform parents and daughters about the HPV-vaccination. On the first pages of the brochure information is given about the Human Papillomavirus, cervical cancer and the HPV-vaccination. On the following pages, quotes are given of mothers, daughters and health care workers on their opinion about the HPV-vaccination. Subsequently, an interview with a mother and a daughter about their decision making process concerning the HPV-vaccination is included. On the next two pages, ten frequently asked questions about HPV-vaccination are raised and answered. Then, some facts about the HPV-vaccination are given. Finally, it is described what happens when an HPV-infection enters the body either when a person is vaccinated or when a person is not vaccinated against HPV. The information brochure is included in Appendix 5.

4.4 Variables In this paragraph, the constructs and items of the questionnaires are described. The baseline and follow-up measurements were conducted in Dutch. For purposes of this paragraph, the questions are translated to English. The original questions are included in Appendix 4. If possible, the multiple items that measured the same construct were averaged to one concept. If a construct consisted of two items, a Pearson’s correlation was calculated to assess whether the internal consistency was sufficient to combine the items. A significance level of beneath 0.05 was used as a criterion for sufficient correlation. If a construct consisted of more than two items, a Cronbach’s alpha was calculated. If the value of the Cronbach’s alpha was higher than 0.60, a sum score was calculated.

4.4.1 Background variables Parents were asked to indicate their gender, date of birth and the first four numbers of their zip code. In addition, the number of other children was asked, together with the request to indicate the age and gender of every child. Educational level was measured by asking parents about their highest educational achievement. Based on their answers, participants were divided in three educational categories: low (no or primary education; lbo, lts, or similar; vmbo, mavo, mulo, or similar), middle (mbo, mts, or similar; havo, vwo, gymnasium, or similar) and high (hbo, hts, or similar; university education). Country of birth was measured by

asking in which country both parents and their daughter were born. Among the Dutch population, the most prevalent countries of birth are The Netherlands, Surinam, (former) Netherlands Antilles or Aruba, Turkey and Morocco. Therefore, these countries were response options, together with the option “other”. For the analysis the birth countries ‘Surinam, former Netherlands Antilles and Aruba’ and ‘Turkey and Morocco’ were merged. Religion was assessed by asking the participants whether they were Protestant Christian, Roman Catholic, Islamic, Jewish, Buddhist, Hindu, atheist (no religion) or ‘different religion’. For the analysis, the religious backgrounds ‘Buddhist, Hindu, Islamic and Jewish’ were combined. The extent to which parents believed in the anthroposophic or homeopathic way of thinking was assessed by asking parents whether they completely disagreed (score = 1) or completely agreed (score = 5) with the question “To what extent do you agree with the way of thinking in the anthroposophy/homeopathy?”. A score of 0 was assigned to the answer category “unfamiliar to me”.

4.4.2 Main variables To test the main hypotheses of this study, the constructs ‘attitude strength’, ‘issue involvement’, ‘general attitude towards HPV-vaccination’ and ‘intention to vaccinate’ were measured. The variables ‘general attitude towards HPV-vaccination’ and ‘intention to vaccinate’ were measured in order to assess the valence of the prior attitude of the parents. In the end, for the analysis we choose to primarily use the variable ‘general attitude towards HPV-vaccination’, as we thought this was a better indicator of attitude valence than the concept of intention. In addition, it was measured to what extent participants thought critically about information about the HPV-vaccination. Initially, this concept of ‘critical thinking’ was meant as an additional outcome measure. However, for the final analyses we decided not to use this outcome measure. For transparency purposes, both concepts and their measures are described in the next section.

Attitude strength – The measurement of attitude strength is based on Pomerantz’s (1995) definition of attitude strength, consisting of five different dimensions. Participants were asked to indicate the extent to which they agreed or disagreed on a 7-point scale with eight statements (two statements per dimension). A score of 1 corresponded with the answer option “completely disagree” and a score of 7 corresponded with the answer option “completely agree”. The fifth dimension of attitude strength, ‘atitude extremity’, was measured with one item on a 17-point-scale. Attitude certainty was assessed by the items: “It is not likely that I change my mind about the HPV- vaccination of my daughter” and “I am certain that my opinion about the HPV-vaccination of my daughter is correct”. Both items are derived from Bloemer and De Ruyter (2004). Because the correlation of the two items was sufficient (r = 0,66 for baseline measurement and r = 0.71 for follow-up measurement, p < 0.01), a scale score was calculated. Personal involvement was assessed by the items: “The HPV-vaccination of my daugther is a very important topic to me personally” and “I see myself as a person who is involved with the HPV-vaccination of my daugther”. These items are also derived from Bloemer and De Ruyter (2004). Because the correlation between the two items was sufficient (r = 0.59 for baseline measurement and r = 0.54 for follow-up measurement, p < 0.01), a scale score was calculated. Ego involvement was measured by the statements: “The HPV-vaccination fits to the way I look at life” and “The decision on whether to vaccinate my daughter says something about who I am”. The first item is based on the research of Pomerantz et al. (1995); the second item is inspired on an item of the research of Betty, Homer & Kahle (1988). Because the correlation of the two items was sufficient (r = 0.36 for baseline measurement and r = 0.37 for follow-up measurement, p < 0.01), a scale score was calculated. Subjective knowledge was measured by asking participants to indicate their self-assessed knowledgibility on two items on a 7-point scale: “I am very aware of the pro’s and con’s of the HPV-vaccination of my daugther” and “I have much knowledge on HPV-vaccination”. Both items are based on research by Bloemer and De Ruyter (2004). Because the correlation of the two items was sufficient (r = 0.70 for baseline measurement and r = 0.72 for follow-up measurement, p < 0.01), a scale score was calculated.

The last dimension af attitude strength, attitude extremity, was assessed by asking participants to rate their attitude valence on a scale from 0 (very negative) to 17 (very positive) (Pfau et al., 2003). In this study, we are mainly interested in creating certain and stable attitudes. Therefore, following Krosnick et al. (1993) the variables attitude certainty and attitude extremity were selected as two main outcome variables.

Issue involvement – Issue involvement was measured by the revised Personal Involvement Inventory of Zaichkowsky (1994). This scale is often used in inoculation research to assess involvement (for example: Pfau et al., 2003; Godbold & Pfau, 1998). The participants were asked to rate their attitude towards HPV-vaccination on a dual 7-point scale on the following items: important/ unimportant, relevant/ irrelevant, worthless/ valuable, means nothing/ means a lot to me. The internal consistency of the four items was high with α = 0.93 for baseline measurement and α = 0.94 for follow-up measurement. Therefore, a scale score was calculated.

General attitude towards HPV-vaccination – General attitude towards HPV-vaccination was measured by asking the participants for their opinion about their daugther’s HPV-vaccination on a dual 7-point scale (1 = completely disagree, 7 = completely agree). The participants were asked to rate their attitude on the following seven items: bad/ very good, very unnecessary/ very necessary, very undesirable/ very desirable, very unpleasant/ very pleasant, very harmful/ very unharmful, very disturbing/ very comforting. The items are derived from Harmsen et al. (2013) and are consistent with items used in other HPV-vaccination research (Van Keulen et al., 2010) and other inoculation research, for example by Goodbold & Pfau (2000). Because the internal consistency of the items was high (α = 0.95 for baseline measurement and α = 0.94 for follow-up measurement), a scale score for intention was calculated.

Intention - The intention to vaccinate against HPV was assessed by asking the participants to indicate on a 7- point scale (1 = completely disagree, 7 = completely agree) the extent to which they agreed with the following three statements: (1) “I plan to vaccinate my daughter against HPV”, (2) “It is very likely that I will vaccinate my daughter against HPV”, (3) “I expect that I will vaccinate my daugther against HPV”. All three items were derived from the research of Harmsen (2013) and are consistent with the items that measure intention derived from the Theory of Planned Behaviour by Ajzen (1985). Because the internal consistency of the three items was sufficient (α = 0.95 for baseline measurement and α = 0.97 for follow-up measurement), a scale score for intention was calculated.

Critical thinking – To gain insights in the extent to which parents critically look at the information about the HPV-vaccination, the following three items were added to the questionnaire: (1)“I can make a distinction between correct and incorrect information about the HPV-vaccination”, (2) “I am critical towards the information about the HPV-vaccination that I receive from different sources” and (3) “I consider the trustworthiness of a source when I read information about the HPV-vaccination”. Parents were asked to indicate on a 7-point scale whether they completely disagreed (score = 1) or completely agreed (score = 7) with the items. Because the internal consistency of the three items was sufficient (α = 0.64 for baseline measurement and α = 0.70 for follow-up measurement), a scale score was calculated.

4.4.3 Manipulation check To assess whether the experimental inoculation was effective in eliciting a feeling of vulnerability among the parents about their attitude towards the HPV-vaccination, participants were asked to imagine that they visited a website on the internet that adviced them not to vaccinate their daughter. Parents were asked to indicate on a 7-point scale if, to them, this was a safe or dangerous idea, non-scary or scary idea, a non-threatening or a threatening idea, a non-risky or a risky idea, a harmless or a harmful idea. Participants in all three conditions were asked to answer this question. Participants were asked to rate these item at the end of the baseline measurement. Because the internal consistency of the items was sufficient {α = 0.91) a scale score was calculated.

4.4.3 Covariates Following Van Keulen et al. (2010), the constructs knowledge, positive and negative outcome expectations, risk perceptions, trust in institions and beliefs about the HPV-vaccination can have an influence on the attitude towards HPV-vaccination. In order to be able to assess whether these constructs had an influence on the outcome variables, these constructs were also measured in the questionnaire.

Knowledge - Actual knowledge was also measured by asking parents to indicate whether four different statements were correct, incorrect or not known by the participant. The following statements (by van Keulen et al., 2010) were used to assess actual knowledge: (1) HPV is a virus, (2) the HPV-vaccination protects fully against cervical cancer, (3) HPV is sexually transmittable and (4) women who are vaccinated against HPV are advised to continue their participation in the cervical cancer screening. A score of 1 point was assigned when the participants answered a statement correctly, a score of -1 was assigned when the participants answered a statement incorrectly. The answer option ‘not known’ was assigned a score of 0.

Trust in institutions – Trust in institutions was measured by asking the participants to rate on a 5-point scale how much the participants trusted science, health care and the Ministry of Public Health in the context of HPV- vaccination. The items were based on Van Keulen et al. (2010). A score of 1 corresponded with the answer option “completely disagree”, a score of 5 corresponded with the answer option “completely agree”. Because the internal consistency of the items was high (α = 0.85 for baseline measurement and α = 0.87 for follow-up measurement), a scale score for ‘trust in institutions’ was calculated.

Positive outcome expectations – Participants were asked to indicate their positive outcome expectations on a 7-point scale (1 = completely disagree, 7 = completely agree). Positive outcome expectations were measured with the following two items: (1) “if my daughter is vaccinated against HPV, she will not get cervical cancer”, (2) “If my daughter gets vaccinated against HPV, she will not get infected with HPV”. Because the correlation of the two items was sufficient (r = 0.64 for baseline measurement) a sum score was calculated.

Negative outcome expectations - Negative outcome expectations were measured with the items: (1) “My daughter will experience troublesome side effects, such as a headache”, (2) “If my daughter is vaccinated against HPV, she will have unsafe sex in the future”. Both items were based on the research of Van Keulen et al. (2010). There was no significant correlation between these two items (r = 0.33, p = 0.21). Therefore, no scale score could be generated. These variables were not taking into account as possible covariates.

Risk perceptions - The following two items on a seven-point scale assessed the risk perceptions of parents: (1) “If my daughter does not get vaccinated against HPV, the chance that she will get cervical cancer is low” and (2) “If my daughter does not get vaccinated against HPV, the chance that she will get cervical cancer is high” (Van Keulen et al., 2010). A score of 1 corresponded with the answer option “completely disagree”, a score of 7 corresponded with the answer option “completely agree”. Because the correlation between the items was sufficient (r = -0.29, p < 0.01) for baseline measurement, a scale score was calculated.

Beliefs – Beliefs about the HPV-vaccination were measured with seven items. Parents could indicate on a 7- point scale if they agreed or disagreed with the following statements: (1) “I think the HPV-vaccine is effective in preventing cervical cancer”, (2) “I think the HPV-vaccine can cause long-term health problems”, (3) “The HPV- vaccination has been introduced so that the drug manufacturer can make a lot of money”, (4) “The government shows its responsibility for the health of the Dutch population by offering the HPV-vaccination”, (5) “It is not clear if the HPV-vaccination protects well against cervical cancer”, (6) “There is too little known about the adverse side-effects of the HPV-vaccination on the long-term” and (7) “I think that there are unsafe substances in the HPV-vaccine”. The items are derived from McRee et al. (2010) and Van Keulen et al. (2010). Because the internal consistency of the items was high (α = 0.85 for baseline measurement), a scale score was calculated.

4.4.4 Evaluation measures Evaluation inoculation message – At the end of questionnaire 2, the participants in the inoculation condition were asked to evaluate the inoculation message. First, the participants were asked to rate the likelihood of the anti-vaccination arguments that were cited in the inoculation message. The internal consistency of these four items was sufficient (α = 0.86). Therefore, a scale score was calculated. Subsequently, the participants were asked to rate the reliability, credibility and the understandability of the refutations of the anti-vaccination arguments on a 5-point scale (1 = completely disagree, 5 = completely agree). As the internal consistency of the three items was sufficient with a Cronbach’s α of 0.88, a scale score was calculated.

Evaluation “persuasive attack” – At the end of questionnaire 2, the participants in all conditions were asked to evaluate the persuasive attack. The participants were asked to rate the persuasive attack on a 5-point dual scale on the items: non-credible/ credible, not likely/ likely, untrustworthy/ trustworthy, not touching/ touching, non-shocking/ shocking. Because the internal consistency of the items was high with a Cronbach’s α = 0.88 a scale score for intention was calculated.

4.5 Statistical analysis The data were analyzed using IBM SPSS 22. All analyses used a significance criterion of p < 0.05. As the number of participants in the mini-magazine condition was already low, missing values were deleted list-wise to limit the loss of data. This had as a result that slightly different study populations were used for the different analyses. The main hypotheses were tested with two different statistical tests. We first looked at the differences between baseline and follow-up scores within the conditions with a paired sample t-test. This in order to gain insights into what effects the individual information interventions had on attitude extremity and attitude certainty. Second, we looked at the differences between the conditions to see which information intervention was most effective. The literature shows two different options to adjust for baseline measurement when comparing follow-up measurements between groups. The first option is to analyze the absolute change. The second option is to adjust for baseline using an ANCOVA test. Twisk and Proper (2004) suggest that when conducting a randomized study, an ANCOVA is the best option. For every hypothesis two ANCOVA’s were run: first, an ANCOVA was performed that was only adjusted for baseline measurement; second an ANCOVA was performed adjusted for baseline and relevant covariates. Bonferonni post-hoc tests with pairwise comparisons were run to assess which conditions differed significantly from one another.

5. Results In this chapter the results of this study are presented. First, the background characteristics of the participants are described in paragraph 5.1. in paragraph 5.2 the normality of the distribution of the different outcome and moderating variables are explored. In paragraph 5.3 the randomization of the respondents over the three conditions is considered. Subsequently, In paragraph 5.4 the inoculation manipulation is considered and in paragraph 5.5 the results of evaluation of the persuasive attack, the anti-vaccination arguments and the refutations are described. Next, in paragraph 5.6 the hypotheses of this research are tested.

5.1 Description of participants In total 390 parents completed both questionnaires and were exposed to the intervention as intended. The mean age of the participants was 44.2 years (SD = 4.50) with a range from 33 to 68 years. A percentage of 93.8% of the participants was female, 6.2% of the participants was male. The majority of the participants (90.5%) indicated that they were from the Netherlands, 1.8% of the participants came from Surinam, the (former) Netherlands Antilles or Aruba, a percentage of 0.8% came from Turkey and 6,9% of the participants were from a self-indicated ‘other’ origin. The country of birth of the other parent or guardian seemed to follow this division broadly. From the participants, 11.5% indicated that they had a low educational level, 45.1% indicated that they were middle educated and 43.3% had a high educational level. Of the participants, 51.0% claimed that they were not religious, 25.0% indicated that they were Catholic, 20.6% was Protestant-Christian and 3,4% indicated that they were either Islamic, Jewish, Buddhist or Hindu. With respect to a homeopathic philosophy of life, 14.1% of the participants disagreed or slightly disagreed with a homeopathic philosophy of life, 21.8% indicated to have a neutral stance towards homeopathy and 56.5% claimed to be slightly positive or positive towards homeopathy. Of the participants 7.7% indicated that they were not familiar with homeopathy. On agreement with an anthroposophic philosophy of life, 17.4% indicated that they disagreed or slightly disagreed; 24.9% indicated to be neutral and 21.0% agreed or slightly agreed with an anthropomorphic way of life. Of all participants, 36.7% were unfamiliar with the term. The numbers (n) and percentages (%) and, if applicable, the means and standard deviations of the distributions of the demographic variables per condition are added in Table 1.

5.1.2 Comparison study population with Dutch population To assess whether the study population was representative for the Dutch population, a comparison was made between level of education, country of birth and religious background between the study sample and the Dutch population from the data of the ‘Centraal Bureau voor Statistiek’ (CBS). As a reference group for the study population men and women with an age between 35-65 year were selected from the CBS data. In this study parents with a Dutch country of birth were overrepresented (90.5% versus 81.5% in the Dutch population). As for religious background, in the study population more parents had a Protestant- Christian background than in the Dutch population (20.6% versus 16.0%), less parents had an Islamic, Jewish, Buddhist, or Hindu background (3.4% versus 5.0% in the whole population) and less parents had no religious background (51.0% for the study population versus 47.0% in the Dutch population). As for the level of education, in this study 11.5% of the parents had a low educational level versus 14.0% in the Dutch population; 45.1% of the parents in this study had a ‘middle’ education level versus 40.0% in the Dutch population and 43.3% had a high educational level versus 42.0% in in the Dutch population.

Table 1: Background information of participants for conditions 1, 2, 3 and total (n = 390). C1 C2 [Mini- C3 [Control] Total Conditions [Inoculation] magazine] comparable? M (SD) M (SD) M (SD) M (SD) Age (years) 43.80 44.74 44.32 44.18 F(2, 387) = 0.96, (4.18) (4.54) (4.69) (4.50) p = 0.38 n = 159 n = 73 n = 145 n = 377 n % n % n % n % Gender Total = Total = Total = Total = 166 74 150 390 Female 159 95.8% 68 91.9% 139 92.7% 366 93.8% Male 7 4.2% 6 8.1% 11 7.3% 24 6.2% Country of birth Total = Total = Total = Total = X2 (6) = 11.01, p 166 73 150 390 = 0.09 Netherlands 156 94.0% 69 93.2% 128 85.3% 353 90.5% Surinam and (former) 1 0.6% 0 0.0% 6 4,0% 7 1.8% Netherlands Antilles or Aruba Turkey & Morocco 1 0.6% 0 0.0% 2 1.3% 3 0.8% Other 8 4.8% 5 6.8% 14 9.3% 37 6.9% Country of birth of other parent Total = Total = Total = Total = X2 (2) = 8.26, p = 165 73 148 386 0.08 Netherlands 154 93,3% 63 86,3% 124 83,8% 341 88,3% Surinam and (former) 2 1.2% 0 0.0% 2 1.4% 4 1.0% Netherlands Antilles or Aruba Turkey & Morocco 2 1.2% 1 1.4% 3 2.0% 6 1.6% Other 7 4.2% 9 12.3% 18 12.2% 34 8.8% Not applicable 0 0.0% 0 0.0% 1 0.7% 1 0.3% Educational level Total = Total = Total = Total = X2 (4) = 2,40, 166 74 150 390 p = 0.66 Low 23 13.9% 6 8.1% 16 10.7% 45 11.5% Intermediate 75 45.2% 36 48.6% 65 43.3% 176 45.1% High 68 41.0% 32 43.2% 69 46.0% 169 43.3% Religion Total = Total = Total = Total = X2 (6) = 18.12, 165 74 149 388 p = 0.01 Protestant-Christian 43 26.1% 9 12.2% 28 18.8% 80 20.6% Catholic 34 20.6% 29 39.2% 34 22.8% 97 25.0% Islam, Judaism, Buddhism, 3 1.8% 1 1.4% 9 6.0% 13 3.4% Hinduism No religion 85 51.5% 35 47.3% 78 52.3% 198 51.0% Homeopathy Total = Total = Total = Total = X2 (6) = 3.69, 166 73 151 390 p = 0,72 Disagree/ slightly disagree 19 11.4% 12 16.2% 24 15.9% 55 14.1% Neutral 41 24.7% 16 21.6% 28 18.5% 85 21.8% Sightly agree/ Agree 95 57.2% 40 54.1% 85 56.3% 220 56.4% Unfamiliar with the term 11 6.6% 5 6.8% 14 9.3% 30 7.7% Anthroposophy Total = Total = Total = Total X2 (6) = 5.31, 166 73 151 = 390 p = 0.51 Disagree/ slightly disagree 27 16.3% 14 18.9% 27 17.9% 68 17.4% Neutral 39 23.5% 16 21.6% 42 27.8% 97 24.9% Sightly agree/ Agree 33 19.9% 21 28.4% 28 18.5% 82 21.0% Unfamiliar with the term 67 40.4% 22 29.7% 54 35.8% 143 36.7% Searched for information? Total = Total = Total = Total = X2 (2)= 0.26, p = 166 73 149 388 0.88 Yes 25 15.1% 12 16.4% 23 15.2% 60 15.5% No 141 84.9% 61 83.6% 126 84.3% 328 84.5%

5.2 Distribution of variables All variables, except for ‘attitude extremity’ and ‘objective knowledge’ were approximately normally distributed, as established by a visual inspection of a QQ-plot. The variables ‘attitude extremity’ and ‘objective knowledge’ slightly deviated from a normal distribution. However, literature shows that AN(C)OVA’s are fairly robust to non-normality (Norman, 2010). Therefore, all the following tests are ran based on the assumption that the variables were normally distributed.

5.3 Randomization check To ensure that the participants in the different conditions did not differ from each other at baseline measurement on background characteristics, chi-squares (for categorical variables) and ANOVA’s (for continuous variables) were executed. Based on these analyses, no significant differences were found between the three conditions on age, gender, country of birth of both parents and educational level (see table 1, last row). However, there was a significant difference in distribution of religious backgrounds between the three conditions, X2 (6) = 18.21, p = 0.01. Therefore, ‘religious background’ has been taken into account as a covariate for testing the hypotheses. Subsequently, it was assessed whether there were differences at baseline measurement between the three conditions on all measured moderating and outcome variables. The results of the ANOVA’s are added in the last row of Table 2. Based on these analyses, no statistically significant differences were found between the three conditions at baseline measurement, except for the variables ‘objective knowledge’ and ‘critical thinking’. However, as explained in paragraph 4.4, the concept of ‘critical thinking’ is not taken into account in this study as either a covariate or outcome variable. Therefore, only the concept ‘objective knowledge’ has been taken into account as a covariate for testing the hypotheses.

5.4 Manipulation check In order to test whether the experimental inoculation manipulation was effective, a one-way ANOVA was executed. The ANOVA showed a statistically significant difference between the three conditions, F (2, 382) = 3.99, p = 0.02. However, a Bonferonni post-hoc test with pairwise comparisons only showed a statistical difference between the mini-magazine condition (M = 4.45; SD = 1.24) and the control condition (M = 3.98; SD = 1.24) with a mean difference of 0.47, p = 0.03. Thus, the experimental inoculation manipulation was not successful. The HPV information brochure manipulation was effective, but only in comparison to control condition.

5.5 Evaluation of persuasive attack, anti-vaccination arguments and refutations Participants evaluated the persuasive attack with a mean score of 3.61 (SD = 0.74) on a scale from 1 to 5, where score 5 corresponded with an evaluation of the persuasive attack as being credible, likely, trustworthy, touching and shocking. This indicates that the participants experienced the persuasive attack as severe. The participants in the inoculation condition evaluated the anti-vaccination arguments that were brought up in the inoculation message with a mean score of 2.31 (SD = 0.84), where a score 1 corresponded with the arguments being unlikely and a score of 5 corresponded with the arguments being likely. This indicates that the participants in the inoculation condition found the anti-vaccination arguments neutral, but on average slightly more unlikely than likely. The refutations of the anti-vaccination arguments were assessed with a mean score of 3.83 (SD = 0.92), where a score of 5 corresponded with the refutations being reliable, credible and understandable, showing that the participants evaluated the inoculation condition evaluated the refutations as positive.

Table 2: Means (SD’s) of measured variables at baseline and follow-up per condition. C1 [Inoculation] C2 [mini-magazine] C3 [Control] Comparison

Baseline Follow-up Baseline Follow-up Baseline Follow-up Conditions Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) comparable at baseline? Intention 5.18 (1.81) 4.94 (1.72) 5.54 (1.51) 5.65 (1.55) 5.27 (1.73) 5.00 (1.68) F(2, 388) = 1.75, n = 160 n = 160 n = 71 n = 71 n = 144 n = 144 p = 0.18

General attitude towards 4.76 (1.42) 4.35 (1.27) 4.94 (1.22) 4.73 (1.04) 4.77 (1.32) 4.33 (1.22) F(2, 385) = 0.43, HPV-vaccination n = 160 n = 160 n = 72 n = 72 n = 150 n = 150 p = 0.65

Attitude strength Attitude certainty 5.55 (1.31) 5.39 (1.27) 5.64 (1.29) 5.73 (0.98) 5.55 (1.29) 5.37 (1.23) F(2, 386) = 0.09, n = 165 n = 165 n = 73 n = 73 n = 147 n = 147 p = 0.91

Personal importance 5.53 (1.19) 5.45 (1.12) 5.77 (0.89) 5.63 (0.78) 5.59 (1.17) 5.44 (1.06) F(2, 385) = 0.82, n = 166 n = 166 n = 73 n = 73 n =147 n = 147 p = 0.43

Ego involvement 4.84 (1.27) 4.75 (1.30) 4.97 (1.49) 5.10 (1.24) 4.66 (1.31) 4.69 (1.13) F(2, 385) = 1.50, n = 164 n = 164 n = 73 n = 73 n = 148 n = 148 p = 0.22

Subjective Knowledge 4.44 (1.49) 4.65 (1.35) 4.43 (1.40) 5.06 (0.97) 4.25 (1.53) 4.54 (1.32) F(2, 384) = 0.62, n = 165 n = 165 n = 72 n = 72 n = 146 n = 146 p = 0.54

Attitude extremity 11.66 (4,62) 10.73 (4.22) 12.70 (4.09) 12.12 (3.62) 11.65 (4.33) 10.54 (4.09) F(2, 388) = 1.75, n = 165 n = 165 n = 74 n = 74 n = 151 n = 151 p = 0.18

Beliefs about HPV- 4.34 (1.06) n/a 4.62 (1.02) n/a 4.34 (0.97) n/a F(2, 388) = 2.30, vaccination n = 166 n = 74 n = 151 p = 0,10

Trust in institutions 3.99 (1.19) 3.86 (1.27) 4.29 (1.17) 4.27 (1.14) 3.98 (1.14) 3.90 (1.18) F(2, 378) = 1.85, n = 159 n = 159 n = 69 n = 69 n = 149 n = 150 p = 0.16

Issue involvement 5.14 (1.41) 4.86 (1.26) 5.54 (1.15) 5.33 (0.96) 5.17 (1.34) 4.86 (1.23) F (2, 388) = N = 163 n = 163 n = 72 n = 72 n = 151 n = 151 2.19, p = 0.11

Positive outcome 3.00 (1.44) n/a 2.91(1.45) n/a 3.07 (1.36) n/a F(2, 386) = 0.33, expectations n = 165 n = 74 n = 150) p = 0.72

Risk perceptions 3.56 (1.01) n/a 3.57 (0.87) n/a 3.66 (0.89) n/a F(2, 385) = 0.46, n = 164 n = 74 n = 150 p = 0.63

Critical thinking 5.21 (0.84) 5.35 (0.84) 5.49 (0.93) 5.71 (0.80) 5.20 (0.86) 5.34 (0.80) F(2, 386) = 3.32, n = 166 n = 166 n = 72 n = 72 n = 150 n = 150 p = 0.04

Evaluation of attitude 4.29 (1.31) n/a 4.45 (1.24) n/a 3.98 (1.24) n/a n/a vulnerability n = 165 n = 72 n = 148

Objective knowledge 3.07 (1.41) 2.78 (1.51) 3.30 (1.07) 3.47 (1.02) 2.70 (1.39) 2. (1,51) F(2, 382) = 3.99, n = 166 n = 166 n = 74 n = 74 n = 151 n = 151 p = 0.02

Evaluation of refutations n/a 3.83 (0.92) n/a n/a n/a n/a n/a (only inoculation condition) n = 138

Evaluation persuasive n/a 3.59 (0.79) n/a . 3.47 (0.67) n/a 3.70 (0.72) F(2, 381) = 2.40, attack n = 161 n = 74 n = 149 p = 0.09

Evaluation of anti- n/a 2.31 (0.84) n/a n/a n/a n/a n/a vaccination arguments n = 138 (only inoculation condition)

5.5 Correlations between covariates and outcome variables In order to test which covariates influenced the outcome Table 3: Pearson’s and Spearman’s rho correlations variables and therefore have to be taken into between covariates and outcome variables. consideration when testing the hypotheses, correlations between the measured possible covariates and the two Attitude Attitude extremity certainty outcome variables, attitude extremity and attitude Age (years) (n = 377) 0.08 0.10 certainty, were calculated. For the variables gender, age, Gender (n = 390) 0.09 0.00 intention, general attitude towards HPV-vaccination, Educational level (n = 390) 0.05 0.14** beliefs, trust in institutions, risk perceptions and objective Homeopathy (n = 390) -0.02 0.06 knowledge a Pearson’s correlation was calculated. For the variables anthroposophy, homeopathy and educational Antroposofie (n = 390) -0.01 -0.04 level a Spearman’s rho was calculated. The correlations Beliefs about HPV-vaccination 0.68** 0.33** (n = 391) between the covariates and the outcome measures are Positive outcome expectations 0.27** -0.02 added in table 3. For the outcome variable ‘attitude Trust in institutions 0.53** 0.26** extremity’ the variables positive outcome expectations, (n = 381) beliefs, trust in institutions and knowledge correlated on p Issue involvement 0.67** 0.18** (n = 391) = 0.01 level. Therefore, these variables were added as Risk perceptions 0.02 -0.07 covariates when testing the hypotheses. For the outcome (n = 388) Objective knowledge -0.05 0.02 variable ‘attitude certainty’ the variables educational level, (n = 391) beliefs, trust in institutions and knowledge correlated on p Note: ** is significant at p < 0.01 level. = 0.01 level and were added as covariates.

5.6 Hypothesis testing This study measured the influence of the experimental and control conditions on the outcome variables attitude extremity and attitude certainty. As noted in paragraph 4.5, for every hypothesis we first looked at the differences within the conditions. In table 4, 5 and 6 the number of participants, means, standard deviations and outcomes of these paired sample t-test are displayed, separately for every condition. Second, we looked at the differences between the conditions. In table 7 and 8 the input and outcomes of these between-condition

ANCOVA tests are displayed, separately for the two outcome measures. In Appendix 7, the unadjusted Mbaseline and Mfollow-up are graphically displayed, in order to give a visual representation of the changes within and between conditions. Following the results shown in paragraph 4.3 and 4.5, the covariates taken into account for the outcome measure attitude extremity are: (1) religious background, (2) positive outcome expectations, (3) beliefs about the HPV-vaccination, (4) trust in institutions and (5) knowledge about the HPV-vaccination at baseline measurement. The covariates taken into account for the outcome measure attitude certainty are: (1) religious background, (2) educational level, (3) beliefs about the HPV-vaccination, (4) trust in institutions and (5) knowledge about the HPV-vaccination at baseline measurement.

Table 4: Input and outcomes of the paired sample t-tests for the inoculation condition. . Extremity Certainty N Means (SD) T, p N Means (SD) T, p All participants 165 -0.93 (3.04) T = 3.96, 165 -0.17 (1.35) T = 1.59, p = 0.00 p = 0.11 Issue involvement Low involved 91 -0.15 (2.44) T = 0.60, 90 0.14 (1.21) T = -1.14, p = 0.55 p = 0.26 High involved 74 -1.89 (3.41) T = 4.77, 75 -0.54 (1.42) T = 3.29, p = 0,00 p = 0.00 Valence Negative 86 -0.24 (2.51) T = 0.91, 86 0.12 (1.19) T = -0.96, p = 0.37 p = 0.34

Positive 77 -1.79 (3.33) T = 4.77, 77 -0.56 (1.29) T = 3.84, p = 0.00 p = 0.00 Issue involvement x valence Negative and low 78 -0.15 (2.49) T = 0,55, 77 0.08 (1.05) T = -0.65, p = 0.59 p = 0.52 Negative and high 8 -1.13 (2.70) T = 1.18, 9 0.72 (1.12) T = 1.93 p = 0.28 p = 0.09 Positive and low 11 -0.64 (1.75) T = 1.21, 11 -0.14 (1.27) T = -0.36, p = 0.26 p = 0.73 Positive and high 66 -1.99 (3.50) T = 4.61, 66 -0.68 (1.27) T = 4.37, p = 0.00 p = 0.00

Table 5: Input and outcomes of the paired sample t-tests for the mini-magazine condition. Extremity Certainty N Means (SD) T, p N Means (SD) T, p All participants 74 -0.58 (2.84) T = 1.76, 73 0.08 (0.94) T = 1.59, p = 0.08 p = 0.11 Issue involvement Low involved 40 -0.03 (3.14) T = 0.05, 39 -0.04 (1.14) T = -0.30, p = 0.96 p = 0.77 High involved 34 -1.24 (2.32) T = 3.10, 68 -0.43 (1.17) T = 3.06, p = 0.00 p = 0.01 Valence Negative 38 0.29 (2.99) T = -0.60, 76 0.05 (1.16) T = -0.35, p = 0.55 p = 0.73 Positive 38 -1.50 (2.40) T = 3.76, 76 -0.42 (1.15) T = 3.11, p = 0.00 p = 0.03 Issue involvement x valence Negative and low 32 0.53 (3.12) T = -0.96, 67 0.15 (1.13) T= 1.08, p = 0.34 p = 0.29 Negative and high 6 -1.00 (1.79) T = 1.37, 9 0.72 (1.12) T = 1.93, p = 0.23 p = 0.09 Positive and low 8 -2.25 (2.19) T = 2.91, 12 -0.58 (0.97) T= 2.08, p = 0.02 p = 0.06 Positive and high 28 -1.29 (2.45) T = 2.78, 59 -0.39 (1.18) T = 2.53, p = 0.01 p = 0.01

Table 6: Input and outcomes of the paired sample t-test for the control condition Extremity Certainty N Means (SD) T, p N Means (SD) T, p All participants 151 -1.11 (3.01) T = 4.54, 147 -0.18 (1.17) T = 1.86, p = 0.00 p = 0.06 Issue involvement Low involved 81 -0.58 (3.34) T = 1.56, 79 -0.04 (1.14) T = -0.30, p = 0.12 p = 0.77 High involved 90 -1.73 (2.46) T = 5.88, 68 0.43 (1.17) T = 3.06, p = 0.00 p = 0.00 Valence Negative 77 -0.96 (3.34) T = 2.71, 76 -0.05 (1.16) T = -0.35, p = 0.01 p = 0.73 Positive 73 -1.40 (2.75) T = 4.33, 76 0.42 (1,15) T = 3.11, p = 0.00 p = 0.03 Issue involvement x valence Negative and low 68 -0.66 (2.92) T = 1.87, 67 0.15 (1.13) T = 1.08, p = 0.07 p = 0.29 Negative and high 9 -3.22 (3.56) T = 2.71, 9 -0.72 (1.12) T = 1.93, p = 0.03 p = 0.09 Positive and low 12 -0.83 (4.75) T = 0.61, 12 -0.58 (0.97) T = 2.08, p = 0.56 p = 0.06 Positive and high 61 -1.51 (2.21) T = 5.33, 59 -0.39 (0.18) T = 2.53, p =0.10 p = 0.01

Table 7: Input and outcomes of the ANCOVA’s on attitude extremity scores. ANCOVA adjusted for baseline ANCOVA adjusted for baseline and covariates

N C1 C2 C3 F, p N C1 C2 C3 F, p Ma *(SE) Ma *(SE) Ma* (SE) Ma**(SE) Ma**(SE) Ma**(SE) All participants 390 10.86 11.53 10.68 F = 2.57, 366 10.89 11.43 10.73 F = 1.89, (0.21) (0.31) (0.22) p = 0.08 (0.20) (0.30) (0.21) p = 0.15 Issue involvement Low involved 212 8.99 9.54 8.70 F = 1.35, 199 10.96 9.53 8.64 F = 1.51, (0.28) (0.42) (0.30) p = 0.26 (0.27) (0.41) (0.29) p = 0.22 High involved 178 13.13 14.03 12.94 F = 1.90, 166 13.5 13.86 13.08 F = 1.04, (0.31) (0.47) (0.33) p = 0.15 (0,.30) (0.44) (0.31) p = 0.36 Valence Negative 201 8.75 9,67 8.10 F = 4.72, 193 8.76 9.60 8.10 F = 4.26, (0.28) (0.42) (0.29) p = 0.10 (0.27) (0.42) (0.29) p = 0.02 Positive 186 13.26 13.81 13.27 F = 0.64, 181 13.23 13.76 13.35 F = 0.56, (0.29) (0.43) (0.30) p = 0.53 (0.28) (0.42) (0.29) p = 0.57 Issue involvement x valence Negative and low 178 8.36 9.32 7.93 F = 3.21, 171 8.42 9.26 7.95 F = 2.88, (0.29) (0.45) (0.31) p = 0.04 (0.28) (0.44) (0.31) p = 0.06 Negative and high 23 11.82 12.41 9.31 F = 2.24, 18 11.70 11.92 9.51 F = 0.90, (1.01) (1.23) (0.99) p = 0.13 (1.39) (1.79) (1.24) p = 0.43 Positive and low 30 12.83 11.72 12.60 F = 0.56, 27 12.82 12.42 12.14 F = 0.27, (0.70) (0.83) (0.67) p = 0.58 (0.66) (0.82) (0.63) p = 0.76 Positive and high 155 13.34 14.37 13.42 F = 1.69, 150 13.37 14.11 13.52 F = 0.80, (0.32) (0.50) (0.34) p = 0.19 (0.31) (0.49) (0.33) p = 0.45 * adjusted mean attitude extremity corrected for baseline ** adjusted mean attitude extremity corrected for baseline and covariates

Table 8: Input and outcomes of the ANCOVA’s on attitude certainty scores.

ANCOVA corrected for baseline ANCOVA corrected for baseline and covariates *adjusted mean attitude extremity corrected for baseline N C1 C2 C3 F, p N C1 C2 C3 F, p Ma*(SE) Ma*(SE) Ma*(SE) Ma**(SE) Ma**(SE) Ma**(SE) All** participants adjusted mean attitude385 extremity5.40 corrected5.70 for5.38 baseline andF = 2.70, covariates 362 5.41 5.67 5.38 F = 2.04, (0.09) (0.12) (0.08) p = 0.07 (0.08) (0.12) (0.09) p = 0.13 Issue involvement Low involved 208 5.20 5.26 5.12 F = 0.31, 195 5.16 5.21 5.15 F = 0.05, (0.10) (0.16) (0.11) p = 0.73 (0.11) (0.17) (0.12) p = 0.96 High involved 177 5.63 5.22 5.68 F = 4.00, 166 5.66 5.15 5.71 F = 2.77, (0.12) (0.18) (0.13) p = 0.02 (0.12) (0.18) (0.13) p = 0.07 Valence Negative 199 5.07 5.25 5.00 F = 0.82, 195 5.01 5.18 5.02 F = 0.34, (0.11) (0.16) (0.16) p = 0.44 (0.12) (0.18) (0.12) p = 0.72 Positive 184 5.74 6.20 5.79 F = 2.77, 174 5.77 6.22 5.80 F = 2.64, (0.12) (0.17) (0.12) p = 0.07 (0.11) (0.17) (0.12) p = 0.08 Issue involvement x valence Negative and low 175 5.03 5.17 5.09 F= 0,24, 163 4.96 5.09 5.15 F = 0.61, (0.11) (0.17) (0.12) p = 0,79 (0.12) (0.18) (0.13) p = 0.55 Negative and high 24 5,31 5.73 4.37 F = 3.47, 17 5.08 5.93 4.23 F = 3.51, (0.35) (0.43) (0.35) p = 0.05 (0.37) (0.45) (0.38) p = 0.06 Positive and low 31 6.13 5.77 5.37 F = 2.44, 25 6.02 6.28 5.23 F = 5.19, (0.25) (0.29) (0.24) p = 0.11 (0.22) (0.29) (0.21) p = 0.02 Positive and high 153 5.68 5.31 5.88 F = 3.58, 144 5.72 6.25 5.90 F = 2,58, (0.13) (0.20) (0.14) p = 0.03 (0.13) (0.20) (0.14) p = 0.08 * adjusted mean attitude certainty corrected for baseline ** adjusted mean attitude certainty corrected for baseline and covariates

Hypothesis 1: Attitude extremity The first hypothesis predicted that participants in the inoculation condition would have more positive attitudes at follow-up measurement than the participants in the mini-magazine and control condition. On attitude extremity scores, significant decreases were found between baseline and follow-up measurement within the inoculation condition and the control condition. Focusing on the differences between the three conditions, no statistical differences were found both on the test adjusted for baseline and on the test adjusted for baseline and covariates. Therefore, the hypothesis that the participants in the inoculation condition would have more positive attitudes at follow-up measurement than the mini-magazine and control condition, could not be confirmed.

Hypothesis 2: Attitude certainty The second hypothesis predicted that the participants in the inoculation condition would be more certain about their attitude than the participants that received the mini-magazine or no additional information. Focusing on the changes within the conditions between baseline and follow-up measurement, no significant changes within the three conditions were found. Comparing the conditions with each other, both the test corrected for baseline measurement and the test corrected for baseline and covariates did not show a significant difference between the three conditions at follow-up attitude certainty scores. Therefore, the hypothesis that the participants in the inoculation condition would be more certain about their attitude than the participants in the other two conditions, could not be confirmed.

Hypothesis 3: Issue involvement The third hypothesis predicted that high involved participants who received the inoculation message would remain more positive and certain about their attitude than the high involved participants in the other two conditions (hypothesis 3.1). For the low involved participants no differences were expected between the conditions (hypothesis 3.2). By the use of a median split, the participants were divided into two groups: participants with a score lower than the median (value of median = 5.5) were characterized as low involved, participants with a score higher than the median were characterized as high involved.

Attitude extremity: On attitude extremity scores, significant decreases were found within all three conditions for the high involved participants. For the low involved participants, no significant changes were found within the conditions. Comparing both the low involved participants between the conditions and the high-involved participants between the conditions, no statistical differences were found for both the test adjusted for baseline and the test adjusted for baseline and covariates.

Attitude certainty: On attitude certainty scores, for the high involved participants a significant decrease was found within the inoculation and the control condition. For the low involved participants, no significant differences were found within the conditions. Looking at the differences between the three conditions, for the low involved participants no significant difference was found both on the test adjusted for baseline and the test adjusted for baseline and covariates. For the high involved participants a significant difference on attitude certainty scores was found on the test adjusted for baseline between the mini-magazine condition and the control condition (mean difference = 0.55, p = 0.05) and the mini-magazine and inoculation condition (mean difference = 0.59, p = 0.02). Based on these results, hypothesis 3.1 could not be confirmed, as the mini-magazine was more effective than the inoculation and control condition under the high involved participants in establishing certain attitudes. Hypothesis 3.2 was confirmed, as no statistically differences were found between the three conditions on attitude extremity and attitude certainty scores.

Hypothesis 4: Attitude valence The fourth hypothesis predicted that at follow-up measurement, participants with a prior positive attitude who had received the inoculation message would remain more positive and certain about their attitude than participants in the other two conditions (hypothesis 4.1). For the participants with a prior negative attitude, no statistical differences were expected between the conditions (hypothesis 4.2). By the use of a median split, the participants in the three conditions were divided in two groups: participants with a score lower than the median (in this case the median score is 5.0, based on the general attitude at baseline measurement) were characterized as ‘negative’, participants with a score higher than the median, were characterized as ‘positive’.

Attitude extremity: Within all three conditions, participants with a prior positive attitude towards HPV- vaccination got significantly more negative at follow up. For the participants with a prior negative attitude, there was only a significant decrease within the control condition. Comparing the participants with a prior negative attitude between the condition, there was a significant difference both corrected for baseline measurement and corrected for baseline and covariates between the mini-magazine and control condition (mean difference = 1.57, p = 0.01). For the participants with a prior positive attitude towards HPV-vaccination no statistically significant difference between the three conditions was found.

Attitude certainty: Within the three condition no significant differences between baseline and follow-up measurement for the participants with a prior negative attitude towards HPV-vaccination were found. For participants with a prior positive attitude, a statistical decrease was found within the inoculation and the control condition. Comparing the participants with a prior negative attitude between the conditions and participants with a prior positive attitude between the conditions, no statistically significant differences were found for both the test corrected for baseline and the test corrected for baseline and covariates.

Based on these results, hypothesis 4.1 could not be confirmed, as no statistical differences were found between the conditions for the high involved participants. Hypothesis 4.2 could also not be confirmed, as for the prior negative participants the mini-magazine was more effective in establishing positive attitudes (in comparison to the control condition).

Hypothesis 5: Attitude valence x issue involvement The fifth hypothesis proposed an interaction between attitude valence and issue involvement on the outcomes of behavioral inoculation. Hypotheses 5.1 and 5.2 proposed that for high involved and prior positive participants (H5.1) and for high involved and prior negative participants (H5.2), the inoculation message would be more effective in inducing resistance to persuasion than the mini-magazine and control condition. Hypotheses 5.3 and 5.4 proposed that for the participants with a prior negative attitude and low issue involvement (H5.3) and for participants with a prior positive attitude and low issue involvement (H5.4), no differences between the conditions would be found. To test these hypotheses, the participants were divided according to both their negative or positive orientation and their high or low issue involvement, thereby creating four groups in every condition, namely: negative and low involved, negative and high involved, positive and low involved and positive and high involved. Almost all high involved participants were positive towards HPV-vaccination and almost all low involved participants were negative towards vaccination. Therefore, for the groups ‘positive and low involved’ and ‘negative and high involved’, the number of participants was very small (n between 7-31).

Attitude extremity: Looking at the changes within the conditions, it was shown that the positive and high involved participants got significantly more negative towards HPV-vaccination within all three conditions. For the negative and low involved participants no significant changes were found. For the positive and low involved

participants there was a significant decrease within the mini-magazine condition between baseline and follow- up. For the negative and high involved participants there was a significant decrease in the control condition. When comparing the four subgroups between the three conditions, no significant differences between the positive and high involved, the positive and low involved and the negative and high involved groups were found on both the test adjusted for baseline and the test adjusted for baseline and covariates. For the negative and low involved participants, a significant difference was found between the mini-magazine and the control condition on the test adjusted for baseline, where the participants who received the mini-magazine remained more positive towards HPV-vaccination than the participants in the control condition (mean difference = 1.40, p = 0.04). This significance disappeared when adjusted both for baseline and covariates.

Attitude certainty: Focusing on the change of attitude certainty scores within the conditions, it was shown that the prior positive and high involved participants in the inoculation and the control condition decreased significantly. For the negative and low involved, the positive and low involved and the negative and high- involved participants no significant differences were found. When comparing the four groups with each other between the three conditions, no significant differences between the negative and low involved participants groups and the negative and high involved participants groups were found, both on the test corrected for baseline and the test corrected for baseline and covariates. For the positive and high involved participant groups a significant difference was found between the mini-magazine and inoculation condition (mean difference = 0.63, p = 0.03). Between the positive and low involved participants a significant difference was found between the inoculation and control condition (mean difference = 0.79, p = 0.05) and between the mini-magazine and control condition (mean difference = 1.05, p = 0.03), but only on the test adjusted for baseline and covariates.

Based on these results, hypotheses 5.1 and 5.2 could not be confirmed. For high involved and prior positive participants no differences between the conditions on attitude extremity were found; on attitude certainty the mini-magazine was more effective in creating certain attitudes. Hypotheses 5.3 and 5.4 were also not confirmed, as in contrast to the hypotheses significant differences were found between the conditions. For low involved and prior negative participants a significant difference was found on attitude certainty scores between the mini-magazine and control condition. For the low involved and prior positive participants the mini- magazine and inoculation message were more effective than the control condition in creating certain attitudes.

6. Discussion and Conclusion The present study assessed whether behavioral inoculation is an effective communication strategy to induce resistance towards negative media messages on the topic of HPV-vaccination. This study resulted in two main findings. First, behavioural inoculation was not effective in inducing resistance to persuasion: the inoculation message was less effective in maintaining and establishing resilient attitudes than the HPV mini-magazine and only slightly more effective in inducing resistance to persuasion than receiving no additional information. The mini-magazine also proved to be more effective in creating strong attitudes when the participants were divided based on high and low issue involvement and on prior positive and prior negative attitudes. The second main finding is that all participants got more negative at the follow-up measurement on attitude extremity scores. In the following paragraph these main findings are embedded in the existing literature. Subsequently, the limitations of this study will be discussed and recommendations are given for practice and further research.

6.1 Main results 6.1.1 Finding 1: Behavioral inoculation was not effective in inducing resistance to persuasion The finding that behavioral inoculation was not effective in inducing resistance to persuasion is in contradiction with the existing literature. Although there are few other studies that also did not find support for inoculation theory, in general the theory has proven to be effective in creating stronger attitudes (Banas & Rains, 2010), also regarding HPV-vaccination (Goodings, 2012). The literature provides several indications that might explain why the inoculation intervention was not effective in this study. The first possible explanation is that the threat manipulation was not successful. Compton & Pfau (2005) claim that the threat manipulation is key to inoculation theory, as it sets in motion the internal process of resistance, thereby also increasing resistance to arguments that were not raised in the inoculation message. Thus, if the threat manipulation failed, possibly the internal process of resistance was not put into motion. This could account for the ineffective behavioral inoculation intervention. Another possible explanation comes from the study of Pashupati, Arpan & Nikolaev (2002), one of the few experiments in which the inoculation message also did not lead to more resilient attitudes. They found that the initial attitudes were the strongest predictor of the attitudes at follow-up measurement, not the information intervention to which the participants were exposed. Pashupati et al. suggested that the inoculation intervention was not effective because they focused on long-held and stable attitudes: a one-time (behavioral inoculation) intervention might not be enough stimulus to change such attitudes. As, in the current study, the certainty with which parents held their attitude was quite high (5.60 on a scale from 1 to 7) this might be a possible explanation for the results. However, as all participants got more negative towards HPV- vaccination at follow-up measurement, it seems that in contrast to the information interventions, the negative media message did have the power to change attitudes significantly with just one message. In paragraph 6.1.2 a possible explanation for these differences in effects is put forward. In addition, the current study found that the HPV mini-magazine, which can be seen as supportive information, was more effective in inducing resistance to persuasion than the inoculation message. This finding is not in line with the results of the meta-analysis of Banas & Rains (2010), who found that inoculation materials were better in creating resistance to persuasion than supportive materials. However, few individual studies have found similar results. For example, Benoit (1991) found that for controversial issues, supportive materials were more effective than inoculation messages in inducing resistance to persuasion. Benoit concluded that only for cultural truisms it can be said with certainty that behavioral inoculation is more effective than supportive materials. Szybillo & Heslin (1973) found a similar result and claimed that behavioral inoculation only works for controversial issues under certain conditions (e.g. the attitudes should not be held too strongly and the topic should be relatively new, so that the individuals have not yet actively been in contact with counterarguments). Thus, because of its controversial nature and because of the certainty with which the attitudes towards HPV-vaccination are held, HPV-vaccination might not be a suitable topic for behavioral inoculation. Interestingly, a previous study by Goodings (2012) did find support for the effectiveness of behavioral inoculation on the topic of HPV-vaccination. He found that participants who were highly involved in the topic

and participants who had a prior positive attitude towards HPV-vaccination, got more positive and certain about their attitude after inoculation. These findings are in contrast with the results of this study, as we found a decrease on attitude extremity scores for all participants. This decrease was greatest among the high-involved and prior positive participants who received the inoculation message or no additional information. In addition, Goodings (2012) concluded that for the low involved and prior negative participants the inoculation message had no significant effect. For the low involved participant group, our findings are in line with the study of Goodings. However, for the prior negative participant group and for the low involved and prior negative participant group, the mini-magazine was more effective in inducing resistance to persuasion. Looking at differences in the research design between the study of Goodings and the current study, there are several issues that could account for the different findings. One specific difference that could account for the different outcomes between the study of Goodings and the current study is the content of the persuasive attack. The study of Goodings made use of a short, news-like item which claimed that there was ‘great anxiety’ about the safety of the vaccine. In this study, participants were asked to read a critical message on the HPV-vaccination from the internet that contained personal story. Therefore, the difference in content of the persuasive attack in the present study compared to the study of Goodings can be a reason for the lack of support of inoculation theory in this study. In paragraph 6.1.2 we will elaborate on this further.

6.1.2 Finding 2: Participants got more negative towards HPV-vaccination The finding that on average all participants got more negative on attitude extremity towards HPV-vaccination at the follow-up measurement suggests that possibly the persuasive attack was the cause of the negative attitude change, not the information interventions. It seems that the persuasive attack had a stronger influence than the information interventions, leading to an overall decrease on attitude extremity scores. The literature supports the suggestion that negative and incorrect media messages can have a negative influence on parents’ attitude towards vaccination and can decrease the vaccination intention (Betsch et al., 2010). Research also confirms that narratives, such as the story used for the persuasive attack, are more persuasive than informational and educational materials, because they are easier to understand and more relatable, credible and concrete (Betsch et al., 2012). In addition, previous research found that narratives speak to the emotions of their readers, which can have a distorting negative influence on the risk perceptions that individuals hold (Bean, 2011). Therefore, based on the literature, it is plausible that the persuasive attack caused the negative attitude change, rather than the information interventions.

6.2 Limitations This study has several limitations. First, the experimental inoculation manipulation performed in this study had several shortcomings. As discussed in paragraph 5.4, the participants in the inoculation condition did not experience a greater feeling of elicited threat than the participants in the mini-magazine and control condition. These results indicate that the inoculation manipulation was not successful, which could account for the ineffectiveness of the behavioral inoculation treatment. In addition, the participants in the inoculation condition evaluated the anti-vaccination arguments in the inoculation message as slightly more unlikely than likely, which is also an indication that the inoculation message was not successful in given parents a feeling of threat. Another limitation of this study is the failed randomization of the participants over the three conditions. The participants within the conditions were from the same regional areas. This resulted in a significant difference between the conditions on religious background. Although the analyses were corrected for religious background, it is possible that the participants were also not equal on other variables that were not measured in the questionnaire.

Two other limitations of this study relate to the power of the research results. The first limitation is that of the 180 participants in the mini-magazine condition, 106 participants indicated that they did not read the mini- magazine. Comments at the end of the questionnaire suggested that many participants were not able to open the mini-magazine because of a technical error. As this study measured the effectiveness of the communication

interventions in creating resilient attitudes, these participants were removed from the analysis. Therefore, the mini-magazine consisted of only 76 participants in comparison to 166 participants in the inoculation condition and 150 participants in the control condition. Thus, the results for the mini-magazine condition have less power2. The second power limitation relates to the creation of four subgroups for testing hypothesis 5. The results show that the majority of the participants were either high involved and positive towards HPV- vaccination or low involved and negative towards HPV-vaccination. Only few participants were characterized as high involved and negative towards HPV-vaccination and low involved and positive towards HPV-vaccination, with small numbers of 17 to 31 participants per subgroup. Thus, the results for these two subgroups have little power and should be interpreted very carefully.

Another limitation of this study is that the research population was not fully comparable with the general Dutch public according to the data of the CBS. In the study population more participants were from Dutch descent, more participants had a high educational level and more participants had a Protestant-Christian background. Lastly, a critical remark about the analysis of the results has to be made. For the analysis we choose to divide the participants into different participant (sub)groups by the use of a median split. We decided to use this technique because it is often used in behavioral inoculation research (for example: Pfau et al., 2007) and therefore it allowed us to draw a comparison between the results of this research and the results of other behavioral inoculation research. However, statistical literature (among others Field, 2010 and McCallum et al., 2002) questions the validity of a median split, as information gets lost when dichotomizing a continuous variable, which could distort the results. This should be taken into consideration when interpreting the results.

6.3 Practical implications Based on the results of this study, implementation of information written according to the principles of behavioral inoculation in the current communication means of RIVM (the HPV mini-magazine) is not recommended. In addition, on the RIVM website several myths about the HPV-vaccination are debunked according to the principles of behavioral inoculation. The results of this study suggest that this is not the most effective strategy to create resistant attitudes. Merely informing parents in a supportive manner might be more effective. However, additional research is needed to back up this conclusion. The results of this study indicate that negative media messages have a negative impact on the attitude towards HPV-vaccination. Although the HPV mini-magazine is more effective in creating resilient attitudes than behavioral inoculation and giving no additional information to parents, also the parents who were exposed to the HPV mini-magazine, got more negative towards HPV-vaccination. Thus, it seems that the current information materials also do not manage to give parents the information or tools to make them resistant towards incorrect media messages. Therefore, this study shows the need for more attention for communication strategies (or other interventions) that induce resistance towards negative media messages.

6.4 Recommendations for future research Based on the outcomes of this study, three recommendations for future research can be given. The first recommendation is to conduct another behavioral inoculation study on the topic of HPV-vaccination. As discussed in paragraph 6.2, this study has several limitations and shortcomings. Especially the inability of the inoculation message to elicit a feeling of threat might be a reason for the ineffectiveness of the inoculation manipulation. Therefore, based on the results of this study, it cannot be concluded with certainty that behavioral inoculation is not effective in the context of HPV-vaccination. An additional inoculation study could provide more conclusive results. For this additional study, the inoculation message should be quantitatively

2 In order to gain insights in the possible differences in results for the complete dataset (n = 496) and the clean dataset (n = 390), the main hypotheses were also tested for the complete dataset. The same conditions differed significantly from each based on analysis of the complete dataset.

pre-tested on a small group of parents to ensure that a sufficient feeling of threat is elicited. In addition, a fourth condition should be added, in which the parents are not exposed to a persuasive attack, in order to confirm that the persuasive attack was responsible for the decrease on attitude extremity scores. The second recommendation relates to the finding of this study that possibly not the communication interventions, but the persuasive attack was responsible for the overall negative attitude change among the participants. Although it is generally assumed that negative internet messages about the HPV vaccination are an important reason for the low vaccination rate (Van Keulen et al., 2010, Conyn-Van Speandonck, 2010), to our knowledge no experimental studies have been conducted to establish this causal relationship. A first effort has been made by Betsch et al. (2010), who tested whether vaccination critical websites led to higher perceptions of risk associated with vaccinations. We recommend to also execute such an experiment on the topic of HPV-vaccination behavior, to gain insights in the effects of negative internet messages. The third recommendation relates to the notion in the literature that narratives and emotional stories are more persuasive than informational and educational materials, as they appeal to the feelings of the receiver. Following Bean (2011), we suggest to conduct a communication experiment which tests if HPV promotion materials that make use of emotionally compelling information are effective in increasing resistance to persuasion.

6.5 Final conclusions This study shows that behavioural inoculation is not an effective communication strategy to induce resistance to persuasion against negative media messages on the topic of HPV-vaccination. The currently used HPV mini- magazine was more effective in establishing positive and certain attitudes than the inoculation message. Therefore, implementation of texts written according to the principles of behavioral inoculation in the communication means of the RIVM about the HPV-vaccination is not recommended. In addition, this study suggests that the persuasive attack was responsible for the negative attitude change among parents. This once again confirms that negative and incorrect media messages have an influence on the attitude of parents towards (HPV-)vaccination. This finding indicates that more attention needs to be paid to strategic communication interventions that induce resistance towards negative media messages on the topic of HPV-vaccination.

Acknowledgements I would like to thank several people for their help with this research project. First of all, I would like to thank my RIVM supervisor Liesbeth Mollema for her guidance, her support and the nice collaboration; I very much appreciated your enthusiasm for and commitment to this project. I would also like to thank my supervisor Emely de Vet for her guidance, interest and especially for her practical advices on how to improve and continue with this project. Third, I would like to thank my fellow student Rebekka Mejda for test reading this thesis.

In addition, I would like to thank several RIVM employees for their contributions to this project. I am grateful to Sylvia Leeman, Julia Blijman, Hans van Vliet and Marina Conyn-Van Spaendonck for taking the time to review the draft versions of the questionnaires. Last but not least, I would also like to thank Hester de Melker for her input and advice on both the questionnaires and results section.

References

Adams, W.C., & Beatty, M.J. (1977). Dogmatism, need for social approval, and resistance to persuasion. Communication Monographs, 44, 321 - 325.

Banas, J.A. & Rains, S.A. (2010). A Meta-Analysis of Research on Inoculation Theory. Communication Monographs, 77 (3), 281 – 311.

Bean, S.J. (2011). Emerging and continuing trend in vaccine opposition website content. Vaccine, 29 (10), 1874 – 1880. doi: 10.1016/j.vaccine.2011.01.003.

Benoit, W. L. (1991). Two tests of the mechanism of inoculation theory. The Southern Communication Journal, 56, 219 - 229. Retrieved from: http://www.communicationcache.com/uploads/1/0/8/8/10887248/two_tests_of_the_mechanism_of_inoculat ion_theory.pdf

Betsch, C. (2011). Innovations in communication: the internet and the psychology of vaccination decisions. Euro surveillance : bulletin Europeen sur les maladies transmissible. European communicable disease bulletin, 16 (17), 1-6.

Betsch, C. (2014). ECDC technical report: Measles and rubella elimination: communicating the Importance of vaccination. Retrieved from: http://ecdc.europa.eu/en/publications/Publications/Measles-rubella-elimination- communicating-importance-vaccination.pdf

Betsch, C., Renkewitz, F., Betsch, T., & Ulshöfer, C. (2010). The influence of vaccine-critical websites on perceiving vaccination risks. Journal of health psychology, 15 (3), 446-455. doi: 10.1177/1359105309353647

Bloemer, J., & De Ruyter, K. (2002). The impact of attitude strength on the acceptance of green services. Journal of Retailing and Consumer Services 9 (1), 45-52.

Centraal Bureau voor Statistiek (2014). De religieuze kaart van Nederland [Web document]. Retrieved from: http://www.cbs.nl/NR/rdonlyres/20EC6E0B-B87A-4CFE-818B-579FB779009F/0/20140209b15art.pdf

Centraal Bureau voor Statistiek; Statline (2014). Bevolking op 1 januari: Leeftijd, geboorteland en regio [Web document]. Retrieved from: http://statline.cbs.nl/Statweb/selection/?VW=T&DM=SLNL&PA=70648ned&D1=0&D2=3-4&D3=0-2%2c4- 8&D4=0-4&D5=l&HDR=G3&STB=G1%2cG2%2cG4%2cT

Compton, J.A., & Pfau, M. (2004). Use of Inoculation to Foster Resistance to Credit Card Marketing Targeting College Students. Journal of Applied Communication Research 32 (4), 343-364.

Compton, J. & Pfau, M. (2009). Spreading Inoculation: Inoculation, Resistance to Influence, and Word-of-Mouth Communication. Communication Theory, 19, 9 – 28.

Conyn-van Spaendonck, M. (2010). Harde lessen uit introductie van HPV-vaccinatie. Spectrum, 1, 5 – 6.

Conyn-van Spaendonck, M. (2012). Het Rijksvaccinatieprogramma: vrijwillig maar niet vrijblijvend [PowerPoint Slides]. Retrieved from: http://itswww.uvt.nl/lis/es/apa/apa-handleiding.pdf

Davies, P., Chapman, S., & Leask, J. (2002). Antivaccination activists on the world wide web. Archives of Disease in Childhood, 87, 22-25.

Downs, J. S., de Bruin, W. B., & Fischhoff, B. (2008). Parents’ vaccination comprehension and decisions. Vaccine, 2 (12), 1595-1607.

Duryea, E.J. (1984). Application of Inoculation Theory to Preventive Alcohol Education. Health Education, 15 (1), 4 – 7. doi: 10.1080/00970050.1984.10615709

Field, A. (2009). Discovering Statistics using SPSS. London, England: Sage

Gass, R., & Seiter, J. (2003). Perspectives on Persuasion, Social Influence, and Compliance Gaining. Boston, MA: Allyn & Bacon.

Godbold, L.C. & Pfau, M. (2000). Conferring Resistance to Peer Pressure Among Adolescents: Using Inoculation Theory to Discourage Alcohol Use. Communication Research, 27 (4), 411 – 437.

Goodings, U. (2012). De HPV-vaccinatiecampagne in Nederland: Het resistenter maken van de attitudes van de doelgroep door inoculatie [Master’s Thesis, Universiteit van Amsterdam, Amsterdam, The Netherlands].

Hansen, W. B. & Graham, J. W. (1991). Preventing alcohol, marijuana, and cigarette use among adolescents: Peer pressure resistance training versus establishing conservative norms. Preventive Medicine, 20 (3), 414 – 430.

Harmsen, I.A., Mollema, L., Ruiter, R. A. C., Paulussen, T. G. W., de Melker, H. E., & Kok, G. (2013). Why parents refuse childhood vaccination: a qualitative study using online focus groups. BMC Public Health, 13, 1183 – 1191.

Harmsen, I.A. (2014). Vaccinating: Self-evident or not? Development of a monitoring system to evaluate acceptance of the National Immunization Program (Doctoral dissertation).

Hofman, R., Van Empelen, P., Vogel, I., Raat, H., van Ballegooijen, M., Korfage, I.J. (2013). Parental decisional strategies regarding HPV vaccination prior to media debates: a focus-group study. Journal of Health Communication, 18, 866-880.

Klop, H. (2014). Prikkelende Informatie: Onzekere en stellige communicatie bij voorlichting over HPV-vaccinatie en het effect op intentie tot vaccineren, vertrouwen in deskundigheid en geloofwaardigheid van de communicatie [Master’s Thesis, University of Utrecht, Utrecht, the Netherlands]. Retrieved from: http://dspace.library.uu.nl/handle/1874/294500.

Krosnick, J.A. & Smith, W.R. (1994). Attitude Strength. Encyclopedia of Human Behavior, 1, 279-289.

Lewandowsky, S. Ecker, U.K.H., Seifert, C.M., Schwarz, N. & Cook, J. (2012). Misinformation and Its Correction: Continued Influence and Successful Debiasing. Psychological Science in the Public Interest December, 13 (3), 106-131. doi: 10.1177/1529100612451018

Littlejohn, S. W. (1978). Theories of human communication. Columbus, OH: Merrill.

Lumsdaine, A. A. & Janis, I. L. (1953). Resistance to "counterpropaganda" produced by one-sided and two-sided "propaganda" presentations. Public Opinion Quarterly, 17, 311-318.

MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7 (1), 19–40.

McGuire, W. J. (1961a). The effectiveness of supportive and refutational defenses in immunizing defenses. Sociometry, 24, 184-197.

McGuire, W.J. (1961b). Inducing resistance to persuasion: Some contemporary approaches. In L. Berkowitz (Ed.), Advances in experimental social psychology (pp. 191-229). New York, NY: Academic Press.

McGuire, W. J., & Papageorgis, D. (1961). The relative efficacy of various types of prior belief-defense in producing immunity against persuasion. Journal of Abnormal Social Psychology, 62, 327 – 337.

McRee, A., Brewer, N.T., Reiter, P.L., Gottlieb, S.L., & Smith, J.S. (2010). The Carolina HPV immunization attitudes and beliefs scale (CHIAS): Scale development and associations with intentions to vaccinate. Sexually Transmitted Diseases 37 (4), 234-239.

Ministerie van Onderwijs, Cultuur en Wetenschap (2013). Opleidingsniveau van de Nederlandse bevolking [Web page]. Retrieved from: http://www.trendsinbeeld.minocw.nl/grafieken/3_1_2_31.php

Miller, G. R. (1980). On being persuaded: Some basic distinctions. In M. Roloff, & G. R. Miller (Eds.), Persuasion: New directions in theory and research (pp 11–28). Beverly Hills, CA: Sage.

Nabi, R. L. (2003)."Feeling" resistance: Exploring the role of emotionally evocative visuals in inducing inoculation. Media Psychology, 5, 199 - 223.

Naud, P. S., Roteli-Martins, C. M., De Carvalho, N. S., Teixeira, J. C., de Borba, P. C., Sanchez, N., Zahaf, T., Catteau, G., Geeraerts, B. & Descamps, D., (2014). Sustained efficacy, immunogenicity, and safety of the HPV- 16/18 AS04-adjuvanted vaccine: Final analysis of a long-term follow-up study up to 9.4 years post-vaccination. Human vaccines & immunotherapeutics, 10 (8), 2147 – 2163.

Norman, G. (2010). Likert scales, levels of measurement and the ‘‘laws’’ of statistics. Advances in Health Science Education, 15 (5), 625-632.

Pashupati, K., Arpan, L. & Nikolaev , A. (2002). Corporate Advertising as Inoculation against Negative News: An Experimental Investigation of Efficacy and Presentation Order Effects. Journal of Current Issues& Research in Advertising, 24 (2), 1 -5.

Paulussen, T.G., Hoekstra, F., Lanting, C.I., Buijs, G.B., Hirasing, R.A. (2005). Determinants of Dutch parents' decisions to vaccinate their child. Vaccine, 24 (5), 644 – 651.

Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion. Advances in Experimental Social Psychology, 19, 123 – 131. https://www.unimuenster.de/imperia/md/content/psyifp/aeechterhoff/wintersemester2011- 12/vorlesungkommperskonflikt/petty_cacioppo_elm_advaexpsocpsy_buchkapitel1986.pdf

Petty, R. E., & Cacioppo, J. T. (2009). The elaboration likelihood model of persuastion. In R. J. DiClemente, R. A. Crosby & M. Kegler (Eds.), Merging theories in health promotion practice and research (2nd ed., pp. 185-241). San Francisco: Jossey-Bass.

Petty, R.E. & Krosnick, J.A. (1994). Attitude Strength: Antecedents & Consequences. New York, NY: Psychology Press.

Pfau, M. (1997). The inoculation model of resistance to influence. In F.J. Boster & G. Barnett (Eds.), Progress in communication sciences, 13, 133–171. Norwood, NJ:Ablex.

Pfau, M. (1995). Designing messages for behavioral inoculation. In M. E. & R. L. Parrot (Eds.), Designing health messages: Approaches from communication theory and public health practice, 99-113. Thousand Oaks, CA: Sage.

Pfau, M., Tusing, K.J., Koerner, A.F., Lee, W., Godbold, L.C., Penaloza, L.J., Hong, Y., & Yang, V.H. (1997). Nuances in inoculation: The role of inoculation approach, ego involvement, and mesage processing disposition in resistance. Communication Quarterly 45 (4), 461-481

Pfau, M. & Szabo, E.A. (2006). Incoluation and Persistance to Persuasion. In: J. Seiter & R. Gass (Eds.). Perspectives on persuasion, social influence and compliance gaining (pp. 265 – 286). Boston: Allyn and Bacon.

Pfau, M., Van Bockern, S. & Kang, J.G. (1992). Use of inoculation to promote resistance to smoking initiation among adolescents. Communication Monographs, 59, 213 – 230.

Pfau, M., & Bockern, S. van. (1994). The Persistence of Inoculation in Conferring Resistance to Smoking Initiation Among Adolescents: the second year. Human Communication Research 20 (3), 413-430.

Pfau, M., & Burgoon, M. (1988). Inoculation in political campaign communication. Human Communication Research, 15, 91 – 111.

Pfau, M., Roskos-Ewoldsen, D., Wood, M., Yin, S., Cho, J., Lu, K. & Shen, L. (2003). Communication Monographs, 70 (1), 39-51. doi: 10.1080/0363775032000104577.

Pomerantz, E.M., Chaiken, S. & Tordesillas, R.S. (1995). Attitude Strength & Resistance Processess. Journal of Personality and Social Psychology, 69 (3), 408 – 419.

Poppel, L. (2011). Solving potential disputes in health brochures with pragmatic argumentation.

Schiffman, M., Castle, P. E., Jeronimo, J., Rodriguez, A. C. & Wacholder, S., (2007). Human papillomavirus and cervical cancer. Lancet, 370, 890-907.

Sherif, M. & Cantril, H. (1947). The psychology of ego-involvements: Social attitudes and identifications. Hoboken, NJ, US: John Wiley & Sons Inc.

Twisk, J. & Proper, K. (2004). Evaluation of the results of a randomized controlled trial: how to define changes between baseline and follow-up. Journal of Clinical Epidemiology, 57 (3), 223 – 228.

Rijksinstituut voor Volksgezondheid en Milieu (2015). Baarmoederhalskanker [Web page]. Retrieved from: http://www.rivm.nl/Onderwerpen/B/Bevolkingsonderzoek_baarmoederhalskanker/Veelgestelde_vragen/Baar moederhalskanker

Rondy, M., Van Lier, A., Van De Kassteele, J., Rust, L., & de Melker, H. (2010). Determinants for HPV vaccine uptake in the Netherlands: a multilevel study. Vaccine, 28, 2070-2075.

Van Keulen, H. M., Otten, W., van der Pal, S., Kocken, P., & Ruiter, R. A. C. (2010). Onderzoek naar de HPV vaccinatiebereidheid bij moeders en dochters naar aanleiding van de inhaalcampagne in Nederland [A study into willingness of mothers and daughters to be vaccinated against HPV within the framework of the catch-up campaign in the Netherlands]. Leiden: TNO Quality of Life.

Van Lier, A., Oomen, P., Conyn-van Spaenendonck, M. A. E., Drijfhout, I. H., de Hoogh, P. A. A. M., & de Melker, H. E. (2013). Vaccination coverage of the National Immunisation Programme in the Netherlands; year 2013 [In Dutch]. Bilthoven: National Institute for Public Health and the Environment (RIVM).

Van Lier, E.A., Oomen, P.J., Oostenburg, M.W.M., Zwakhals, S.L.N., Drijfhout, I.H., de Hoogh, P.A.A.M. (2009). Vaccinatiegraad Rijksvaccinatieprogramma Nederland, verslagjaar 2009. Bilthoven: RIVM.

Wan, H.H., & Pfau, M. (2004). The relative effectiveness of inoculation, bolstering, and combined approaches in crisis communication .Journal of Public Relations Research, 16, 301 - 328.

Wolfe, R.M., Sharp , L.K., Lipsky, M.S. (2002). Content and design attributes of anti-vaccination web sites. Journal of American Medical Association, 26, 287 - 324). doi: 10.1001/jama.287.24.3245.jbr20117

Wood, L.M. (2007). Rethinking the Inoculation Analogy: Effects on Subjects With Differing Preexisting Attitudes. Human Communication Research 33(3), 357-378.

Zaichkowsky, J.L. (1985). Measuring the Involvement Construct. Journal of Consumer Research 12(3), 341-352.

ZIllien, N. & Hargittai, E. (2009). Digital distinction: Status-specific. Internet Uses Social Science Quarterly, 90, (2), 274–291

Appendix 1: Letter of Approval Social Sciences Ethical Committee

Appendix 2: Invitation letter

Appendix 3: Reminder letter

Appendix 4: Questionnaires Onderzoek informatievoorziening HPV vaccinatie (1)

Geachte ouder/verzorger,

Het Rijksinstituut voor Volksgezondheid en Milieu (RIVM) en Wageningen Universiteit doen samen onderzoek naar de informatievoorziening over de HPV-vaccinatie. Met de resultaten van dit onderzoek kan het RIVM de informatie voor meisjes en hun ouders/verzorgers verbeteren.

Het onderzoek bestaat uit twee vragenlijsten. Vandaag vult u de eerste vragenlijst in. Over 7 dagen sturen wij u een email met daarin een link voor het invullen van de tweede vragenlijst. U kunt altijd besluiten om uw deelname aan dit onderzoek te beëindigen. Dit doet u door de internetpagina af te sluiten.

Wij willen nogmaals benadrukken dat alle antwoorden vertrouwelijk en anoniem worden verwerkt. Gedurende het onderzoek hebben we een aantal keer uw emailadres nodig. Uw emailadres zal direct na het onderzoek worden verwijderd.

Tijdsduur Het invullen van de vragenlijst kost ongeveer 15 tot 20 minuten.

Inhoud van de eerste vragenlijst De vragenlijst start met een aantal vragen over uw persoonlijke situatie. Daarna volgen een aantal vragen over uw houding ten aanzien van de HPV-vaccinatie. Ook vragen wij u informatie te lezen over de HPV- vaccinatie. In deze informatie worden twee onjuiste argumenten tegen HPV-vaccinatie, afkomstig van anti-vaccinatie websites, besproken en weerlegd.

Vooruitblik naar inhoud van de tweede vragenlijst De tweede vragenlijst, die u over 7 dagen ontvangt, start met een bericht van het internet over de HPV- vaccinatie. Daarna stellen wij u een aantal vragen over uw houding ten opzichte van de vaccinatie.

Sommige stellingen lijken op elkaar, dit is nodig om uw mening zo goed mogelijk in kaart te kunnen brengen.

Heeft u vragen of opmerkingen over deelname aan dit onderzoek? Neem dan contact op met de Voordatonderzoekers u start op met emailadres: het invullen [email protected] van de vragenlijst, willen wij u kortof bel informeren naar 030-2743813 over het (Marloes HPV en de HPVBeerling).-vaccinatie .

HPVVeel ensucces baarmoederhalskanker met het invullen van de vragenlijst! HPV is de afkorting voor ‘humaan papillomavirus’, het virus dat baarmoederhalskanker kan veroorzaken. Uw dochter kan besmet raken met dit virus door seksueel contact. De kans dat uw dochter het virus een keer oploopt is groot. Meestal ruimt het lichaam dit virus vanzelf op. Maar soms ontstaat een voorstadium

van baarmoederhalskanker en kan er jaren later baarmoederhalskanker ontstaan. Baarmoederhalskanker

wordt jaarlijks bij zo’n 600 vrouwen ontdekt. Elk jaar gaan er ongeveer 200 vrouwen in Nederland aan

dood.

HPV-vaccinatie Meisjes krijgen in het jaar dat ze 13 jaar worden een uitnodiging voor twee HPV-prikken. De vaccinaties zijn gratis en niet verplicht. Binnen het Rijksvaccinatieprogramma wordt het vaccin Cervarix gebruikt. Cervarix beschermt tegen twee HPV varianten die zo’n 70% van alle gevallen van baarmoederhalskanker veroorzaken. De vaccinatie wordt gegeven bij een JGZ organisatie. De prik wordt gezet in de bovenarm.

Naderhand kunnen de meisjes daar spierpijn krijgen, die soms een week kan duren. Sommige meisjes 44 krijgen tijdelijk last van buikpijn, misselijkheid, moeheid, hoofdpijn of koorts. De meeste klachten zijn mild en gaan vanzelf weer over.

In dit gedeelte van de vragenlijst worden een aantal vragen gesteld over uw persoonlijke situatie.

1. Wat is uw persoonlijke code? Uw persoonlijke code vindt u op de achterzijde van de uitnodigingsbrief.

2. Wat is uw geslacht? o man o vrouw

3. Wat is uw geboortedatum? Dag-Maand-Jaar

4. Wat zijn de vier cijfers van uw postcode? 1234

5. Wat is uw emailadres?

6. Wat is uw geboorteland? o Nederland o Suriname o (voormalig) Nederlandse Antillen of Aruba o Turkije o Marokko o anders, namelijk: …

7. Wat is het geboorteland van de andere verzorger van uw dochter geboren in 2002? o Nederland o Suriname o (voormalig) Nederlandse Antillen of Aruba o Turkije o Marokko o anders, namelijk: … o niet van toepassing

8. Wat is uw hoogst voltooide opleiding? (m.u.v. basisonderwijs wil dit zeggen dat u een diploma voor deze opleiding behaald heeft) o geen of basisonderwijs (lagere school) o vmbo, mavo, mulo, of gelijkwaardig o havo, vwo, gymnasium, of gelijkwaardig o lbo, lts, of gelijkwaardig o mbo, mts, of gelijkwaardig o hbo, hts, of gelijkwaardig o wetenschappelijk onderwijs (universiteit)

9. Kunt u van al uw kinderen aangeven wat hun leeftijd en geslacht is? kind 1: Leeftijd m/v kind 2: Leeftijd m/v kind 3: Leeftijd m/v kind 4: Leeftijd m/v kind 5: Leeftijd m/v kind 6: Leeftijd m/v kind 7: Leeftijd m/v etc.

10. Tot welke geloofsovertuiging rekent u zichzelf? o Protestants christelijk o Rooms katholiek o Islam o Jodendom o Boeddhisme o Hindoeïsme o Andere geloofsovertuiging, namelijk ……….. o Geen geloof

11. In hoeverre bent u het eens met de denkwijze in de antroposofie? Mee oneens Beetje mee Niet mee Beetje mee Mee eens Ik ben hier niet oneens oneens, niet eens bekend mee. mee eens o o o o o 

12. In hoeverre bent u het eens met de denkwijze in de homeopathie? Mee oneens Beetje mee Niet mee Beetje mee Mee eens Ik ben hier niet oneens oneens, niet eens bekend mee. mee eens o o o o o 

13. Heeft u pas geleden op internet gezocht naar informatie over de HPV-vaccinatie? Ja/ Nee

In dit gedeelte van de vragenlijst worden een aantal vragen gesteld over uw houding ten aanzien van de HPV-vaccinatie van uw dochter.

14. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

eens

mee mee mee mee

beetje beetje beetje

oneens / / oneens

niet mee niet mee niet

oneens oneens oneens

helemaal helemaal helemaal

mee eens mee eens mee eens mee Ik ben van plan mijn dochter te laten o o o o o o o vaccineren tegen HPV. Het is zeer waarschijnlijk dat ik mijn o o o o o o o dochter laat vaccineren tegen HPV. Ik verwacht dat ik mijn dochter laat o o o o o o o vaccineren tegen HPV.

15. Ik vind het vaccineren van mijn dochter tegen HPV: heel erg slecht o o o o o o o heel erg goed heel erg onnodig o o o o o o o heel erg nodig

heel erg onwenselijk o o o o o o o heel erg wenselijk heel erg onplezierig o o o o o o o heel erg plezierig heel erg schadelijk o o o o o o o heel erg onschadelijk heel erg verontrustend o o o o o o o heel erg geruststellend

16. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

eens

mee mee mee mee

beetje beetje beetje

oneens / / oneens

niet mee niet mee niet

oneens oneens oneens

helemaal helemaal helemaal

mee eens mee eens mee eens mee Het is niet aannemelijk dat ik van gedachten verander over de HPV-vaccinatie o o o o o o o van mijn dochter. Ik sta achter mijn mening over de HPV- o o o o o o o vaccinatie van mijn dochter. De HPV-vaccinatie van mijn dochter is voor mij persoonlijk een erg belangrijk o o o o o o o onderwerp. Ik zie mijzelf als een persoon die betrokken o o o o o o o is bij de HPV-vaccinatie van mijn dochter. De HPV-vaccinatie past goed bij hoe ik in o o o o o o o het leven sta. De beslissing om mijn dochter wel of niet te o o o o o o o vaccineren zegt iets over wie ik ben. Ik ben mij bewust van de voor-en nadelen o o o o o o o van HPV-vaccinatie van mijn dochter. Ik heb veel kennis over de HPV-vaccinatie. o o o o o o o

17. Ik vind het vaccineren van mijn dochter tegen HPV: zeer negatief __-__-__-__-__-__-__-__-__-__-__-__-__-__-__-__-__ zeer positief

18. Geef bij elk van de volgende stellingen aan of deze volgens u waar of niet waar zijn: Waar Niet waar Weet ik niet HPV is een virus. o o o De HPV-vaccinatie beschermt volledig tegen o o o baarmoederhalskanker. HPV is seksueel overdraagbaar. o o o Vrouwen die tegen HPV zijn gevaccineerd worden in Nederland aangeraden toch mee te blijven doen aan het o o o bevolkingsonderzoek baarmoederhalskanker (= het uitstrijkje).

19. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

eens

mee mee mee mee

beetje beetje beetje

oneens / / oneens

niet mee niet mee niet

oneens oneens oneens

helemaal helemaal helemaal

mee eens mee een mee eens mee Ik denk dat het HPV-vaccin effectief is in het o o o o o o o voorkomen van baarmoederhalskanker. Ik denk dat het HPV-vaccin langdurige o o o o o o o

gezondheidsproblemen kan veroorzaken. De HPV-vaccinatie is ingevoerd omdat de medicijnfabrikant daar veel geld aan kan o o o o o o o verdienen. De overheid toont haar verantwoordelijkheid voor de gezondheid o o o o o o o van de Nederlandse bevolking door het aanbieden van de HPV-vaccinatie. Het is onduidelijk of de HPV-vaccinatie goed beschermt tegen o o o o o o o baarmoederhalskanker. Er is te weinig bekend over de nadelige bijwerkingen van de HPV-vaccinatie op de o o o o o o o lange termijn. Ik denk dat er onveilige stoffen in het HPV- o o o o o o o vaccin zitten.

20. Als het gaat om de HPV-vaccinatie, hoe groot is dan uw vertrouwen in de volgende instanties? Zeer klein Beetje Neutraal Beetje Zeer groot klein groot De wetenschap o o o o o De gezondheidszorg o o o o o Ministerie van Volksgezondheid o o o o o

21. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

eens

mee mee mee mee

beetje beetje beetje

oneens / / oneens

niet mee niet mee niet

oneens oneens oneens

helemaal helemaal helemaal

mee eens mee eens mee eens mee Als mijn dochter zich laat vaccineren tegen HPV, dan zal zij geen o o o o o o o baarmoederhalskanker krijgen. Als mijn dochter zich laat vaccineren tegen HPV, dan zal zij niet besmet worden met o o o o o o o HPV. Mijn dochter zal na HPV-vaccinatie last krijgen van vervelende bijwerkingen, zoals o o o o o o o hoofdpijn. Als mijn dochter zich laat vaccineren tegen HPV, dan zal zij in de toekomst onveilig o o o o o o o vrijen. Het is onduidelijk of de HPV-vaccinatie goed beschermt tegen o o o o o o o baarmoederhalskanker. Als ik mijn dochter niet laat vaccineren tegen HPV, dan is de kans dat zij o o o o o o o baarmoederhalskanker krijgt klein. Als ik mijn dochter niet laat vaccineren tegen HPV, dan is de kans dat zij baarmoederhalskanker krijgt groot.

22. De HPV-vaccinatie van mijn dochter is voor mij: onbelangrijk __-__-__-__-__-__-__ belangrijk Irrelevant __-__-__-__-__-__-__ relevant waardeloos __-__-__-__-__-__-__ waardevol betekent niets voor me __-__-__-__-__-__-__ betekent veel voor me

23. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

eens

mee mee mee mee

beetje beetje beetje

oneens / / oneens

niet mee niet mee niet

oneens oneens oneens

helemaal helemaal helemaal

mee eens mee eens mee eens mee Ik kan juiste en onjuiste informatie over de HPV-vaccinatie goed van elkaar o o o o o o o onderscheiden. Ik kijk kritisch naar de informatie die ik van verschillende bronnen ontvang over o o o o o o o de HPV-vaccinatie. Ik neem de betrouwbaarheid van de bron in overweging als ik informatie over de o o o o o o o HPV-vaccinatie lees.

24. Wij zijn geïnteresseerd in hoeverre uw houding ten aanzien van de HPV-vaccinatie het vaccinatiegedrag van uw dochter voorspelt. Wij willen daarom medio 2015 graag nagaan of uw dochter uiteindelijk heeft deelgenomen aan de HPV-vaccinatie. Deze informatie zal uitsluitend voor doeleinden gerelateerd aan dit onderzoek gebruikt worden. Geeft u hiervoor toestemming?

 Ik geef toestemming aan de onderzoekers om medio 2015 de HPV-vaccinatiestatus van mijn dochter op te vragen.

Groep 2: Uw dochter wordt in 2015 uitgenodigd voor de HPV-vaccinatie. Uitgebreide informatie over de vaccinatie vindt u in de onderstaande brochure (zie het PDF bestand). Deze brochure is afkomstig van het RIVM.

Heeft u de brochure geopend? Ja/ Nee

Zo Ja: Hoe uitgebreid heeft u de brochure gelezen?

IkGroep heb de1: brochure oppervlakkig gelezen __-__-__-__-__ Ik heb de brochure intensief gelezen

Wij vragen u om onderstaande tekst te lezen.

In 2009 is de HPV-vaccinatie geïntroduceerd. In de media was er destijds veel aandacht voor argumenten tégen deze vaccinatie. Nog steeds verschijnt er regelmatig negatieve informatie op internet over de HPV-vaccinatie. Vooral op anti-vaccinatie websites. Een aantal argumenten die op het internet genoemd worden, zijn feitelijk onjuist. Uit onderzoek blijkt dat deze argumenten echter voor veel ouders aannemelijk zijn. Dit kan invloed hebben op uw houding ten aanzien van de HPV-vaccinatie [1]. Dit kan betekenen dat u de beslissing over het vaccineren van uw dochter op onjuiste informatie baseert. Hieronder worden twee onjuiste argumenten, afkomstig van anti-vaccinatie websites, besproken en weerlegd.

Onjuist argument 1: De HPV-vaccinatie heeft ernstige lange termijn bijwerkingen.

Lange termijn bijwerkingen HPV-vaccinatie Een veelgebruikt argument tegen HPV-vaccinatie is dat de vaccinatie ernstige lange termijn bijwerkingen heeft. Er wordt nu wereldwijd ongeveer 20 jaar onderzoek gedaan naar de vaccins tegen baarmoederhalskanker. In die 20 jaar zijn er geen lange termijn bijwerkingen van de HPV-vaccinatie vastgesteld. Ook de ervaring van vijf jaar HPV-vaccinatie in Nederland bevestigt dat de vaccinatie veilig is. Het klopt dat de HPV-vaccinatie een relatief nieuwe vaccinatie is. Met andere vaccins in het Rijksvaccinatieprogramma is soms al 50 jaar ervaring. Maar een vaccin wordt net als alle geneesmiddelen zeer goed getest volgens strenge internationale eisen voordat het beschikbaar komt [2]. Nadat een vaccin is ingevoerd, wordt deze continu gemonitord door het onafhankelijke instituut Lareb (Nederlands Bijwerkingen Centrum). Lareb bevestigt dat er tot op heden geen lange termijn bijwerkingen van de vaccinatie bekend zijn.

Chronische vermoeidheid en HPV-vaccinatie Ook chronische vermoeidheid wordt in relatie gebracht met de HPV-vaccinatie. In 2013 heeft Lareb hier onderzoek naar gedaan. In de periode tussen 2009 en 2013 kreeg Lareb 31 meldingen van langdurige vermoeidheid na HPV-vaccinatie. Op basis van de analyse van die meldingen heeft Lareb niet kunnen vaststellen dat er een relatie bestaat tussen chronische vermoeidheid en de HPV-vaccinatie [3]. Vooral omdat langdurige vermoeidheid een klacht is die bij meisjes op deze leeftijd ook los van vaccinatie voorkomt. Twee gebeurtenissen, zoals HPV-vaccinatie en chronische vermoeidheid, kunnen in dezelfde periode voorkomen, maar dit hoeft niet te betekenen dat er een relatie bestaat. Internationaal onderzoek bevestigt dat er sinds introductie van het HPV-vaccin geen toename is in chronische vermoeidheid bij meisjes van 12 en 13 jaar [4].

Onjuist argument 2: Bestandsdelen van het HPV-vaccin.

Insectencellen Op anti-vaccinatie websites wordt gezegd dat het HPV-vaccin cellen van insecten bevat. In het HPV-vaccin zitten de eiwitten HPV16 en HPV18. Dit zijn de stoffen die ervoor zorgen dat er na de vaccinatie een afweerreactie in het lichaam ontstaat. Deze eiwitten zijn gemaakt met behulp van gekweekte insectencellen. Zodra de eiwitten gemaakt zijn, worden de eiwitten gezuiverd en gescheiden van de insectencellen. De gezuiverde eiwitten worden toegevoegd aan het vaccin; de insectencellen niet. Er zitten dus geen insectencellen in het HPV-vaccin.

Rattengif Ook wordt op anti-vaccinatiewebsites gezegd dat het HPV-vaccin rattengif bevat. In sommige bestrijdingsmiddelen zit de stof ‘natriumboraat’, dit is de stof waarna wordt verwezen op de anti-vaccinatie websites. Natriumboraat wordt gebruikt in veel verschillende producten (waaronder cosmetica en tandpasta), maar niet in Cervarix, het HPV-vaccin dat in het Rijksvaccinatieprogramma zit. Het HPV-vaccin dat in Amerika wordt gebruikt, genaamd Gardasil, bevat wel natriumboraat. De hoeveelheid natriumboraat in één dosis van het Gardasil vaccin is 0,000035 gram. Dit is zo’n kleine hoeveelheid, dat het niet schadelijk is voor het lichaam.

Bronnen: [1] Betsch, C., Renkewitz, F., Betsch, T., & Ulshofer, C. (2010). The influence of vaccine-critical websites on perceiving vaccination risks. Journal of health psychology, 15(3), 446-455. [2] De HPV-vaccinatie die in het Rijksvaccinatieprogramma is opgenomen is getest door de Amerikaanse Food and Drug Administration (FDA) en de European Medicines Agency (EMA). [3] LAREB Rapportage 2012 ‘Bijwerkingen van het Rijksvaccinatieprogramma’. Beschikbaar via de link: http://www.lareb.nl/Nieuws/2013/Rapportage-bijwerkingen-Rijksvaccinatieprogramma-2 [4] Donegan, K., Beau-Lejdstrom, R., King, B., Seabroke, S., Thomson, A. & Bryan, P. (2013). Bivalent human papillomavirus vaccine and the risk of fatigue syndromes in girls in the UK. Vigilance and Risk Management of Medicines. Gepubliceerd in: Vaccine.

Einde bericht

Alle groepen: 20. Stel, u bezoekt een website op het internet die u adviseert om uw dochter niet te vaccineren. Wat vindt u van dit idee? Ik vind dit: een veilig idee __-__-__-__-__-__-__ een gevaarlijk idee geen eng idee __-__-__-__-__-__-__ een eng idee een niet bedreigend idee __-__-__-__-__-__-__ een bedreigend idee geen riskant idee __-__-__-__-__-__-__ een riskant idee een onschadelijk idee __-__-__-__-__-__-__ een schadelijk idee

Afsluitend bericht vragenlijst 1 Hartelijk bedankt voor het invullen van de eerste vragenlijst! Over zeven dagen benaderen wij u via de e- mail met het verzoek om de tweede vragenlijst in te vullen. Voor de resultaten van dit onderzoek is het belangrijk dat u ook deze vragenlijst invult.

Onderzoek informatievoorziening HPV vaccinatie (2)

Geachte ouder/ verzorger,

Vandaag vult u de tweede vragenlijst in van het onderzoek over de informatievoorziening rondom de HPV-vaccinatie.

Wij willen nogmaals benadrukken dat alle antwoorden anoniem en vertrouwelijk worden verwerkt. Gedurende het onderzoek hebben we een aantal keer uw emailadres nodig. Uw emailadres zal direct na het onderzoek worden verwijderd. U kunt altijd besluiten om uw deelname aan dit onderzoek te beëindigen.

Tijdsduur Het invullen van de tweede vragenlijst kost ongeveer 15 tot 20 minuten.

Inhoud van de tweede vragenlijst Eerst vragen wij u om een bericht van het internet te lezen over de HPV-vaccinatie. Daarna stellen wij u een aantal vragen over uw houding ten opzichte van de HPV-vaccinatie. Sommige vragen lijken op elkaar, dit is nodig om uw mening zo goed mogelijk in kaart te brengen.

Heeft u vragen of opmerkingen over deelname aan dit onderzoek? Neem dan contact op met de onderzoekers op emailadres: [email protected] of bel naar 030-2743813 (Marloes Beerling).

Veel succes met het invullen van deze vragenlijst!

1. Wat is uw persoonlijke code? Uw persoonlijke code vindt u op de achterzijde van de uitnodigingsbrief. Klik op “Volgende” om de vragenlijst te starten.

Wij vragen u om het onderstaande bericht te lezen. Dit bericht verscheen op het internet op 20 maart 2012.

Vragenlijst 2

Hieronder stellen we u een aantal vragen over uw houding ten aanzien van de HPV-vaccinatie van uw dochter.

2. Heeft u in de periode tussen de eerste vragenlijst en dit moment aanvullende informatie op het internet gezocht over de HPV-vaccinatie van uw dochter? Ja/ Nee

3. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

eens

mee mee mee mee

beetje beetje beetje

oneens / / oneens

niet mee niet mee niet

oneens oneens oneens

helemaal helemaal helemaal

4. Ik vind het vaccineren van mijn dochter tegen HPV: heel erg slecht o o o o o o o heel erg goed heel erg onnodig o o o o o o o heel erg nodig heel erg onwenselijk o o o o o o o heel erg wenselijk heel erg onplezierig o o o o o o o heel erg plezierig heel erg schadelijk o o o o o o o heel erg onschadelijk heel erg verontrustend o o o o o o o heel erg geruststellend

5. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

eens

mee mee mee mee

beetje beetje beetje

oneens / / oneens

niet mee niet mee niet

oneens oneens oneens

helemaal helemaal helemaal

6. Ik vind het vaccineren van mijn dochter tegen HPV: zeer negatief __-__-__-__-__-__-__-__-__-__-__-__-__-__-__-__-__ zeer positief

7. Geef bij elk van de volgende stellingen aan of deze volgens u waar of niet waar zijn: Waar Niet waar Weet ik niet HPV is een virus. o o o De HPV-vaccinatie beschermt volledig tegen o o o baarmoederhalskanker. HPV is seksueel overdraagbaar. o o o Vrouwen die tegen HPV zijn gevaccineerd worden in Nederland aangeraden toch mee te blijven doen aan het o o o bevolkingsonderzoek baarmoederhalskanker (= het uitstrijkje).

8. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

eens

mee mee mee mee

beetje beetje beetje

oneens / / oneens

niet mee niet mee niet

oneens oneens oneens

helemaal helemaal helemaal

mee eens mee eens mee eens mee Ik denk dat het HPV-vaccin effectief is in het o o o o o o o voorkomen van baarmoederhalskanker. Ik denk dat het HPV-vaccin langdurige o o o o o o o gezondheidsproblemen kan veroorzaken. De HPV-vaccinatie is ingevoerd omdat de medicijnfabrikant daar veel geld aan kan o o o o o o o verdienen. De overheid toont haar verantwoordelijkheid voor de gezondheid o o o o o o o van de Nederlandse bevolking door het aanbieden van de HPV-vaccinatie. Het is onduidelijk of de HPV-vaccinatie goed beschermt tegen o o o o o o o baarmoederhalskanker. Er is te weinig bekend over de nadelige bijwerkingen van de HPV-vaccinatie op de o o o o o o o lange termijn. Ik denk dat er onveilige stoffen in het HPV- o o o o o o o vaccin zitten.

9. De HPV-inenting van mijn dochter is voor mij: onbelangrijk __-__-__-__-__-__-__ belangrijk Irrelevant __-__-__-__-__-__-__ relevant waardeloos __-__-__-__-__-__-__ waardevol betekent niets voor me __-__-__-__-__-__-__ betekent veel voor me

10. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

t mee t

eens

mee mee mee mee

beetje beetje beetje

oneens / / oneens

nie mee niet

oneens oneens oneens

helemaal helemaal helemaal

11. Deze vraag gaat over de tekst die u net gelezen heeft met de titel ‘Lijdensweg na meidenprik’. Geef aan in hoeverre u het eens of oneens bent met de volgende stellingen:

Ik vind het verhaal van Sarina: ongeloofwaardig o o o o o geloofwaardig niet aannemelijk o o o o o aannemelijk onbetrouwbaar o o o o o betrouwbaar niet aangrijpend o o o o o aangrijpend niet schokkend o o o o o schokkend

Alleen groep 1: In de eerste vragenlijst van dit onderzoek heeft u informatie gelezen over de HPV-vaccinatie. In deze tekst werden drie veelgehoorde argumenten tegen HPV-vaccinatie weerlegd. Deze informatie is hieronder nogmaals weergegeven.

[kopie informatie over HPV-vaccinatie uit vragenlijst 1]

12. We zijn geïnteresseerd in uw mening over de anti HPV-vaccinatie argumenten die in bovenstaande tekst worden genoemd. Geef bij elk van de volgende stellingen aan in hoeverre u de stelling waarschijnlijk vindt. Beetje Heel Beetje Heel onwaar Neutraal onwaarschijnlijk waarschijnlijk waarschijnlijk schijnlijk De HPV-vaccinatie heeft ernstige o o o o o lange termijn bijwerkingen. HPV-vaccinatie kan chronische o o o o o vermoeidheid veroorzaken. Het HPV-vaccin bevat cellen van o o o o o insecten. Het HPV-vaccin bevat rattengif. o o o o o

13. We zijn geïnteresseerd in uw mening over de weerleggingen van de anti-vaccinatie argumenten in het bovenstaande bericht. Geef bij elk van de volgende stellingen aan in hoeverre u het hier mee eens of oneens

bent.

mee mee mee

Beetje Beetje Beetje

oneens oneens

neutraal

mee eens mee eens mee

Ik vind de weerleggingen van de anti- vaccinatie argumenten door het RIVM o o o o o betrouwbaar. Ik vind de weerleggingen van de anti- vaccinatie argumenten door het RIVM o o o o o geloofwaardig. Ik vind de weerleggingen van de anti- vaccinatie argumenten door het RIVM o o o o o begrijpelijk.

14. Wij zijn geïnteresseerd in hoe u uw deelname aan dit onderzoek ervaren heeft. Daarom willen wij u graag een aantal vragen stellen, telefonisch of via e-mail. Deelname kost ongeveer 5 – 10 minuten. Wilt u hieraan meewerken?

 Ja, ik geef toestemming aan de onderzoekers om telefonisch contact met mij op te nemen. Ik ben bereikbaar op telefoonnummer:

 Ja, ik geef toestemming aan de onderzoekers om mij via e-mail te benaderen. Mijn emailadres is:

15. U kunt ook hieronder aangeven hoe u uw deelname aan dit onderzoek ervaren heeft:

16. Wij willen u later (in 2015) nog eens benaderen om te meten of uw houding ten aanzien van uw dochters HPV-vaccinatie met de tijd is veranderd. Geeft u hiervoor toestemming?

 Ik geef toestemming aan de onderzoekers om mij via email te benaderen voor deelname aan een korte vervolg vragenlijst

17. Wilt u op de hoogte worden gesteld van de onderzoeksresultaten van dit onderzoek? Zo ja: Vul dan hieronder uw emailadres in.

Appendix 5: Mini-magazine

Appendix 6: Regional areas per condition

Condition 1 Condition 2 Condition 3 (inoculation) (mini-mag) (control)

Appendix 7: Graphs Mbaseline and Mfollow-up per hypothesis Hypothesis 1 Hypothesis 1: Hypothesis 2:

Hypothesis 3 (extremity) Hypothesis 3 (certainty)

Hypothesis 4 (extremity) Hypothesis 4 (certainty)

Hypothesis 5 (extremity) Hypothesis 5 (certainty)