1 Inoculating the Public against Misinformation about Climate Change: A Replication Study Christina M. C. Bonda Matt N. Williamsab aSchool of Psychology, Massey University, New Zealand bTo whom correspondence should be addressed. Address: School of Psychology, Massey University, Private Bag 102904, North Shore, Auckland 0745, New Zealand. Email [email protected] Keywords: Climate change, conservation psychology, misinformation, replication, open science, inoculation. 2 Abstract The Earth’s climate is changing due to anthropogenic greenhouse gas emissions. Conservation psychology has the capacity to produce research that can inform efforts to modify human behavior to mitigate climate change. However, psychology has recently been facing a replication crisis: Several recent studies have found that the findings of many published psychological studies cannot be reproduced in independent replications. In response to this crisis, psychologists have begun to pursue practices that can improve the replicability and credibility of findings—for example, preregistering data collection and analysis plans before collecting data, and openly sharing data for re-analysis. However, open science practices such as these are not yet widely employed in conservation psychology. We argue that replicability is especially important in conservation psychology given the field’s focus on high stakes applied research. We provide an example of a preregistered replication (of van der Linden et al. 2017). van der Linden et al. reported that they were able to successfully “inoculate” participants against politically motivated misinformation about climate change by pre-emptively warning them of this misinformation. In our replication study, we preregistered hypotheses based on van der Linden et al’s study, along with a detailed data collection and analysis plan (available at https://osf.io/8ymj6/). Our replication study used a mixed between-within design, with data collected via Mechanical Turk (N = 792). We were able to replicate some (but not all) of van der Linden et al’s findings. Specifically, we found that providing information about the scientific consensus on climate change increased perceptions of scientific consensus, as did an inoculation intervention provided prior to provision of misinformation. However, we were unable to replicate their finding that an inoculation intervention counteracted the effect of misinformation to a greater extent than simply providing information about scientific consensus. 3 Article Impact Statement “Inoculation” can combat misinformation about climate change but may not be more effective than a simple message about scientific consensus. General Introduction The Earth is warming, and human activities are primarily to blame (IPCC 2014). This conclusion is the subject of a remarkably strong scientific consensus: Approximately 97% of actively publishing climate scientists agree that the Earth is warming and human activities are responsible for most of this warming (Cook et al. 2016). Conservation biologists have a crucial role to play in understanding how climate change will impact global biodiversity, habitats, and ecosystems—and how some of these impacts might be alleviated. Yet, at its core, anthropogenic climate change is, by definition, a problem caused by human behavior. As such, researchers in the field of conservation psychology have an especially important part to play in helping to address the problem of climate change. Ultimately, if the extent of climate change is to be mitigated, human behavior will have to change (Reddy et al. 2017). In recent years, however, key problems with the replicability of findings in psychology have been identified—a phenomenon sometimes referred to as the replication1 crisis (see Earp & Trafimow 2015). Recent large-scale multi-lab replication efforts have ascertained that the key findings (e.g., the presence of a statistically significant effect in a particular direction) cannot be replicated for roughly a third of published studies in psychology (e.g., Open Science Collaboration 2015). Problems with replicability are not restricted to psychology, however; concerns about replicability have also been noted in fields 1 Broadly speaking, to replicate a study it is to repeat it with new data; in contrast, to reproduce it is to use the original data and computer code and thereby produce the same findings (although different definitions are in use, and the terms are sometimes applied interchangeably; see Barba 2018). 4 including ecology and evolution (Kelly 2019), environmental epidemiology (Bartell 2019), and cancer biology (Baker & Dolgin 2017). The publication of unreplicable—and potentially untrustworthy—findings can obviously have negative consequences. The stakes in conservation psychology and conservation biology are particularly high: Conservation biology exists, after all, as a response to a biodiversity extinction crisis (Teel et al. 2018). The publication of unreplicable and untrustworthy research in conservation biology and conservation psychology could have serious consequences. For example, publishing findings that particular behavior change programmes are effective ways to reduce greenhouse gas emissions when they are in fact ineffective could result in the waste of scarce time and resources. Researchers within and outside psychology have responded to the apparent replication crisis by applying a variety of new research strategies. One strategy is openly sharing the (deidentified) raw data underlying findings online, such that readers and reviewers can reproduce the authors’ analyses and attempt different ones (albeit that de-identifying data may not always be possible; see Meyer 2018 for a more detailed discussion). This can potentially identify problems with the reproducibility of reported results (e.g., Hardwicke et al. 2018), but it can also help the research community identify cases where specific findings are not robust to slightly different decisions during data collection and analysis. A related strategy is openly sharing the materials underlying findings (e.g., computer code, measurement instruments, etc.; Munafò et al. 2017). An even more distinctive change in the way that research is performed has been the practice of preregistration (Nosek et al. 2018). In a preregistered study, the researchers specify their hypotheses and plan for data collection and analysis before analyzing data (ideally before even collecting it). They lodge this information in an online repository, where readers and reviewers will eventually be able to access it. Preregistration can help to combat 5 the problem of “researcher degrees of freedom” (Simmons et al. 2011 p. 1359) by reducing the risk that decisions about which data analyses are reported are made on the basis of whether they produce desired statistically significant findings (or at least to make it clearer to readers if this has been the case; see Parker et al. 2019). Along with the increasing application of open data, open materials, and preregistration, an important way to address apparent replication problems in psychology and elsewhere is simply for more replication studies to be conducted and published (Koole & Lakens 2012). Introduction to Empirical Study In the remainder of this paper, we report a study demonstrating open and reproducible practices in conservation psychology. Specifically, we report a replication of a study by van der Linden et al. (2017) who reported that an “inoculation” technique could be useful for reducing the effect of misinformation on participants’ beliefs about the extent of scientific consensus about climate change. Despite the overwhelming consensus amongst scientists (see Cook et al. 2016), the public are divided in their beliefs about the causes (and existence) of climate change (for international studies see Pelham 2009; Capstick et al. 2015). Climate change denial amongst some members of the public represents a real barrier to effective conservation action on climate change (Gifford 2011; Bohr 2016). Members of the public also tend to underestimate the degree of scientific consensus about climate change (e.g., Hamilton 2016). One plausible reason why some members of the public tend to underestimate the degree of scientific consensus on climate change is the sharing of inaccurate information about climate change in online and mainstream media (Oreskes & Conway 2010; Farrell 2016). A notable example of misinformation about climate change is the “Oregon petition”—a petition arguing that there 6 is no evidence that greenhouse gas emissions could cause disruption to the Earth’s climate (“Global warming petition project” n.d.). The petition claims to have over 31,000 signatories—although fewer than 1% of signatories have any expertise in climate science (Lewandowsky et al. 2017). One potentially useful strategy for combating misinformation is that of attitudinal inoculation, an idea studied in social psychology for over 50 years (Anderson & McGuire 1965). In this context, inoculation means preparing a person for the possibility that they may be exposed to misinformation by presenting them with a weak example of the challenging information along with a refutation (“refutational pre-empting”). This is believed to increase resistance to the challenging information, much in the same way that a vaccine confers resistance to a virus by exposing the body to a weakened version of that virus. A large body of research has applied and tested inoculation interventions in contexts including health,