sign in sign up
<<
Home , Area 51, Eurabia, Group polarization, Locus of control

“The truth is in doubt here!”: Critically Assessing Sources and in a Conspiratorial World

A quantitative study of mentality and endorsement in relation to news media skepticism and -literacy, source preference, critical thinking dispositions and -skills.

Ronan Eric Reilly

Master’s Thesis in Journalism

UNIVERSITY OF OSLO Institute of Media and Communication 15. December 2020

Ronan Eric Reilly

II “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

“The truth is in doubt here!”: Critically Assessing Sources and Information in a Conspiratorial World

A quantitative study of conspiracy mentality and conspiracy theory endorsement in relation to news media skepticism and -literacy, source preference, critical thinking dispositions and -skills.

“History is much more the product of chaos

“… if you assume a big than of conspiracy.”

enough conspiracy, ― Zbigniew Brzezinski you can explain anything, including the cosmos itself.”

― Fritz Leiber, The Big Time “Feedback loops, echo chambers,

circular reinforcement. All could play a part in escalating the utterly imaginary to the level of , sometimes with fatal consequences.”

― Jasper Fforde, Early Riser

“It's all one big conspiracy. Just there's nobody pulling all the strings.” “It’s who’s controlling ― James Oswald the that I’m more concerned about.”

― Angela Mullins, Working for Uncle Henry

III Ronan Eric Reilly

© Ronan Eric Reilly 2020 “The truth is in doubt here!”: Critically Assessing Sources and Information in a Conspiratorial World Ronan Eric Reilly http://www.duo.uio.no/ Trykk: Reprosentralen, Universitetet i Oslo

IV “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

V Ronan Eric Reilly

Summary

Conspiracy theories thrive in today’s overly saturated media landscape, where media consumers constantly are bombarded with an abundance of information from both mainstream and alternative outlets. Facing this scenario, audiences have little other choice than to somehow decide who and what to believe, and to invoke some level of effort and skill in interpreting information. This thesis seeks to explore both ‘conspiracy mentality’ (an underlying disposition toward viewing events as the product of conspiracies) and the actual endorsement of certain specific conspiracy theories in relation to several variables relevant to media consumption. Two main areas of focus are explored: a) News media trust, source preference, and literacy; and b) Critical thinking dispositions and skills. By identifying, presenting, and drawing on relevant literature, as well as evaluating former endeavors regarding measurement of specific variables, the primary aim is to quantitatively explore these relationships through answering eight research questions. A quantitative, fixed-response cross-sectional questionnaire containing primarily a combination of previously validated-, existing but altered-, as well as self-developed multi-item Likert measuring scales forms the basis of findings. Data is collected from a demographically diverse non-probability convenience sample selection of Norwegians (N=125) of all ages, socio-cultural backgrounds, and with varying political views – as well as media preferences and habits. Initial observations reveal a moderately high level of general conspiracy mentality within the sample – as well as a prevailing presence of respondents who endorse various conspiracy theories. A series of bivariate correlation analyses and one-way analyses of variance (ANOVA’s) are run based on the gathered data in order to answer all research questions. Findings suggest that high skepticism towards mainstream media, as well as a positive towards – and frequent consumption of – ‘alternative’ news media are all significantly related to both conspiracy mentality and conspiracy theory endorsement. Findings further suggest that susceptibility to informal fallacious arguments, in addition to an inability to identify nonsensical, but seemingly meaningful ideas (‘pseudo-profound bullshit’), are also correlated with conspiracy theory endorsement and -mentality. Partial, albeit weak, relationships to some aspects of news media literacy – such as understanding how filtering of information in the media can affect perceptions of reality – are also discovered in relation to conspiracy theory endorsement. Surprisingly, no significant findings are uncovered in relation to critical thinking disposition or source criticism abilities; although it is hypothesized that methodological flaws may be at fault for some of these results. Some possible implications of findings for journalists and media workers are discussed where applicable throughout, and upon conclusion.

VI “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Foreword

Returning to university to finally achieve a master’s degree having passed my mid-thirties ended up being a mixed-emotion endeavor. In many ways it has been challenging – as work, life, children, and family obligations etc. inevitably have consumed most available time. On the other hand, it has been highly liberating and rewarding – not only to sporadically re-walk in the social footsteps of a mid-twenties version of myself, but also to be able to approach academic work with an extended amount of life experience and perspective that was unavailable to me ten-fifteen years ago.

During my time here at UiO I have made many new acquaintances – several from various and countries across the world. A big thanks to all of you for enriching my ‘second run’ here. The list of names is long, but you all hopefully know who you are!

To all the 125 people who have participated in the survey – I realize it was very comprehensive, and your valuable time is much appreciated. I cherish all the feedback I have received – especially the positive remarks from everyone who found the questions interesting. For very different reasons I was also intrigued by the (luckily very few) posts in response to my survey request which referred to me as e.g. “a brainwashed and corrupt mass media liar”; and the ‘likes’ these posts generated. Whether you enjoy a good conspiracy theory or not, you have all played an important part in the story of completing this thesis.

A large thank you naturally also goes out to my supervisor at IMK, Dr. Kjetil Rødje (S18/F18/S19/F19/S20/F20). Your advice and insight – as well as your patience with regard to continuous delays and shifting aims – has been pivotal to the eventual completion of this thesis.

Thank you to my lovely children, Stella Edine and Edwin Olaver. Although you have absolutely no conceptual idea what a master’s thesis is yet, or why daddy has been so busy; you both keep me alert and on my feet, while putting immensely into perspective what is most important in life.

Thanks to their mother – my dear girlfriend, Stine – for all your endless support; and for both pulling most of the weight at home and having to spend most nights alone during these final weeks, while additionally having to put up with my many annoyances. I couldn’t have done it without you.

Thank you to my father, Brendan Reilly, who’s generally inquisitive mind and unorthodox approaches to conventional norms has been an inspiration. Although I do not always agree with

VII Ronan Eric Reilly your alternative views or means of analysis, I have always considered our discussions to be positively challenging and very interesting.

Last, but not least, a very special thank you is reserved for my beloved mother, Edith Reilly, who tragically passed away in 2007; the same year as research into conspiracy theories began emerging in a serious way. Your memory will forever be cherished, and your many extensively detailed opinions about the world – many of which tended to carry a largely conspiratorial central theme – have acted as in choosing my topic of interest. May you for all eternity rest in peace.

And to all of you who have not been mentioned, but who feel you should have been mentioned… Consider yourself mentioned!

All in all, completion of this thesis has been an immensely long-winded process (not helped in any way at all by the global COVID-19 pandemic that went on for the final months of it, or the concussion I contracted which postponed completion by an additional semester). Although I am more than happy to disembark after a long journey, I am also truly grateful for the fact that my circumstances granted me the possibility to endure this voyage; both when enjoying the calm academic waters and when navigating the high methodological seas.

Drammen, Norway December 15, 2020

VIII “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Table of Contents

1 Introduction ...... 1 1.1 Purpose of thesis ...... 3 1.2 Focus areas and research questions ...... 5 1.3 Structure and organization...... 8 2 Literary review ...... 10 2.1 Conspiracy theories ...... 10 Definitions and terminological explanations ...... 10 Conspiracy theory characteristics and traits ...... 12 The distribution of conspiracy theories ...... 16 Psychological conspiracy theory research: Who , and why? ...... 18 Measuring conspiracy theory endorsement and mentality ...... 22 2.2 A:News media trust, source preference, and literacy ...... 22 News media trust, credibility, and skepticism (RQ1) ...... 23 ‘Mainstream’ vs ‘alternative’ news media (RQ2 + RQ3) ...... 24 News media literacy (RQ4)...... 27 2.3 B:Critical thinking dispositions and skills...... 32 Dispositions towards critical thinking (RQ5) ...... 33 Skills: Detecting logically fallacious argumentation (RQ6) ...... 35 Skills: Detecting ‘pseudo-profound ’bullshit’ (RQ7) ...... 38 Skills: Information evaluation / source criticism ability (RQ8) ...... 41 3 ...... 43 3.1 Choice of method ...... 43 3.2 Ethical considerations and practicalities ...... 43 3.3 Sampling, validity, and reliability ...... 45 3.4 Design, structure, and measurement...... 46 3.5 Choosing and operationalizing variables ...... 50 Demographic data ...... 50 Alternative media usage and news media habits ...... 50 Dependent variables: Conspiracy mentality / -theory endorsement ...... 51 Independent variables (A): News media ‘skepticism’ and ‘literacy’ ...... 52 Independent variables (B): Critical thinking dispositions and skills ...... 55

IX Ronan Eric Reilly

3.6 Distribution and collection of data ...... 57 3.7 Data analysis preparations and considerations ...... 59 4 Analyses and Discussions ...... 63 4.1 Overview of sample data ...... 63 Univariate analysis of demography...... 63 Univariate analysis of media views, habits, and preferences ...... 65 Initial exploration of data from Likert scales ...... 67 4.2 Conspiracy theory endorsement and -mentality ...... 69 4.3 A:News media trust, source preference, and literacy ...... 72 RQ1: Mainstream media skepticism ...... 72 RQ2: Frequency of reported alternative media usage ...... 74 RQ3: Attitudes towards alternative media ...... 75 RQ4: News media literacy ...... 78 4.4 B: Critical thinking dispositions and skills ...... 81 RQ5: Critical thinking disposition ...... 81 RQ6: Susceptibility to informal logical fallacies ...... 84 RQ7: Pseudo-profound bullshit detection ...... 86 RQ8: Source criticism ability ...... 89 5 Summary, conclusions, and thoughts ...... 92 5.1 Recap and summary ...... 92 5.2 Limitations and weaknesses ...... 95 5.3 Conclusive discussion ...... 99 5.4 Final thoughts ...... 105 References: ...... 109 Appendecies: ...... 120

X “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

List of Figures and Tables

Figure 1.2.1 – Flowchart of main thesis interest areas, variables, and research questions...... 6 Figure 2.1.1 – Conventional vs conspiratorial thinking (Lewandowsky & Cook, 2020, p. 3) ...... 13 Figure 2.1.2 – Seven traits of conspiratorial (Lewandowsky & Cook, 2020, p. 6) ...... 15 Figure 2.2.1 – News Media Literacy Model (Craft, Maksl, & Ashley, 2013, p. 6) ...... 29 Table 4.1.1 – Basic descriptive statistics of demographic data ...... 63 Figure 4.1.2 – Charts and diagrams of sample demography ...... 64 Figure 4.1.3 – Primary platform of news consumption (percent) ...... 65 Figure 4.1.4 – Frequency of ‘common and familiar’ news source consumption (percent) ...... 65 Figure 4.1.5 – Frequency of ‘unfamiliar recommended’ news source consumption (percent) ...... 66 Figure 4.1.6 – Assessment of ‘alternative news’ meaning (percent) ...... 66 Figure 4.1.7 – Frequency of ‘alternative news’ consumption (percent) ...... 66 Figure 4.1.8 – Preferred news outlet (percent) ...... 67 Table 4.1.9 – Descriptive and reliability statistics (α) for all Likert scales in the study ...... 68 Figure 4.2.1 – CMQ distribution (N=125) ...... 69 Table 4.2.2 – Endorsement of various conspiracy theories (N=125) ...... 70 Table 4.2.3 – Correlations: CMQ vs VCTE / TVC2 variants (N=125) ...... 71 Figure 4.2.4 – Scatterplot: CMQ vs VCTE7 (N=125) ...... 72 Figure 4.4.1 – Ratio of responses to CTDS, CO and RS scales (N=125) ...... 83 Table 4.4.2 – Overview over how respondents assessed arguments (N=125) ...... 85 Table 4.4.3 – Overview of responses to bullshit- and profound statements (N=125) ...... 89 Figure 4.4.4 – Average rate of responses to CRAAP-scale items (N=125) ...... 90

XI

“The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

1 Introduction

A media saturated world of conspiracy theories In a world where we all are inevitability bombarded with media messages from an abundance of content distributors it can be challenging to choose who one should listen to and what to believe. Fake news stories and misinformation flourish in a metaphorical jungle of news outlets and channels, but arguably one of the most fascinating phenomena within the sphere of information distribution are ‘conspiracy theories’. We have all heard of them – these ‘truths’ and secretly planned 'cover ups’ that allegedly are carried out by those who ‘really are in control’, while the naive public are being spoon-fed ‘official stories’ by the mainstream media. 9/11 was an inside job; the Moon landings never happened; vaccines are designed to keep people sick; the COVID19- virus was deliberately unleashed upon the world; an alien – part reptilian – race is controlling worldwide governments; and, yes – the earth is indeed flat! Such theories have been around for a long time, and almost everyone has some form of relationship to them. They represent a cultural segment of our society, and they can manifest in relation to almost any world event – major or minor – presenting with varying degrees of plausibility. Hardly any single event of significance seems to go by without generating a conspiracy-buzz; be it economic crises, deaths of public figures, terrorist attacks, natural disasters, major accidents, military conflicts, or – as currently highly relevant – flu pandemics and election results. Byford (2011, pp. 2-6) describes societal conspiracy tendencies dating back more than 250 years, and for decades it has been commonly claimed by lay people, journalists and academics that we live in an “age of conspiracism” (e.g. Byford, 2011, p. 3; van Prooijen & Douglas, 2017, p. 324). The extremeness of such theories, and the nature of where, how, and how quickly they spread seems ever increasing. Undoubtedly, an escalating presence of both mainstream and alternative media outlets, social media channels, and countless other sources of information propel this trend as we move deeper into the ‘web 2.0’-era.

Commonly endorsed Many may assume that those who believe in conspiracy theories are quirky and irrational, but irrespective of background and intelligence-level it appears that all sorts of people can end up subscribing to seemingly far-fetched explanations of various events. One of many to point this out was Kelly Garrett (2016), who stated that: “Research into misperception, rumor, and conspiracy theory suggests that even reasonable individuals can reach conclusions that don’t align with the best available evidence”. It is also paradoxical that it generally seems like many people can tend to be highly skeptical, while at the same time often indiscriminately believe many highly unlikely things. Byford (2011, p. 6) demonstrated that according to opinion polls around the world a substantial

1 Ronan Eric Reilly portion of the population readily admits to subscribing to some form of conspiracy theory. In fact, we live in a world where the have elected a president who openly condones, spreads, and even creates such theories. Many, e.g. Oliver and Wood (2014a, p. 952), have found that most Americans endorse at least one conspiracy theory. According to a poll by Jensen (2013) almost a third of Americans believed that a secretive authoritarian power elite with a globalist agenda is conspiring to eventually rule the world, and more than half believed that did not act alone in the of John F. Kennedy. Furthermore, Barkun (2003, pp. 81-83) elaborated around findings showing that between half and 2/3 of Americans believed in advanced government cover-ups of . Conspiracies are also largely believed to be behind several dramatic events, including the 1996 explosion of flight TWA 800 over (Miller, 2002, p. 40), the Apollo Moon landing (Swami et al., 2013, p. 72), and the deliberate creation and spread of AIDS (Goertzel, 1994, p. 733). Several polls have also suggested that up to 40 percent of respondents believed that the official account of the 9/11 attacks were a cover-up, or that the US government were somehow involved in the collapse of the Twin Towers (Swami & Coles, 2010, p. 560). Between a fifth and a third of Germans and Canadians, and close to 40 % of Britons believed this too (Byford, 2011, pp. 6-7), and as many as 3/4 of the sample taken from five Muslim countries alleged a US/Israeli conspiracy which did not involve Al-Qaida (Gentzkow & Shapiro, 2004, p. 120). Also, van Prooijen and Douglas (2018, p. 900) have pointed to several worldwide studies from many corners of the world (e.g. Poland, Malaysia, Indonesia, and rural Africa) suggesting widespread general conspiracy belief.

Serious consequences Although several conspiracy theories are nothing more than amusing anecdotes that spread through social media or lurk in the vicinity of reliable news, they can often lead to serious consequences. Although the potential level of harm they can cause greatly differs, outcomes can end up being of global importance. While it may seem relatively harmless that some people chose to condone and share ‘evidence’ that the earth is flat, several grotesque terrorist attacks have had conspiratorial origins. Conspiracy beliefs are associated empirically with populism (Castanho Silva, Vegetti, & Littvay, 2017, p. 423) and political extremism (van Prooijen, Krouwel, & Pollet, 2015, p. 570). Extremist movements such as Neo-Nazis and religious fundamentalists are characterized by excessive conspiracy beliefs, and conspiracy theories can causally contribute to violent tendencies and radicalization (Bartlett & Miller, 2010). Timothy McVeigh e.g. committed the 1996 Oklahoma City bombing as a reaction to a perceived belief that America was run as a dictatorship by Jews and Blacks (Byford, 2011, p. 15). Similarly, the 22. July 2011 attacks carried out by on Norwegian government buildings and Utøya were rooted in the extreme right-wing

2 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Islamophobic “” conspiracy theory (van Buuren, 2013, p. 206). Another typical danger is avoidance of necessary medical treatment (e.g. Oliver & Wood, 2014b, pp. 817-818), as well as refusal to get vaccinated (Jolley & Douglas, 2014a, p. 1), due to medical- and anti-vaxxer conspiracy theories. Similarly, in relation to climate change, van der Linden (2015, p. 171) found that seeing a conspiracy video significantly reduced both belief in widespread scientific agreement on human-caused climate change and the will to engage in pro- climate activities. Considering how easily participants’ opinions were swayed in such experiments, the degree to which audiences can be affected by similar media messages is a major concern. There are also more indirect threats which are of pivotal importance to democracies. For example, some of the most alarming developments in recent world politics are closely interwoven with conspiracy theories (Uscinski, 2019a), such as the election of as president (p. 3) and Brexit (p. 9). Misunderstandings, misleading argumentation, and skewed information spread via alternative media sources are likely key reasons for such occurrences. On an overall basis it is ironic and problematic how increased amounts of available information in many ways can lead to a generally less knowledgeable public. Clickbait is often more alluring than real news, emotions can take precedence over facts, and social media has increasingly facilitated the emergence of ‘filter bubbles’ (Sunstein, 2009), and ‘echo chambers’ (Pariser, 2011); arenas where audiences either block out or repeat and confirm amongst each other exactly what they want to hear. Well- functioning democratic societies are dependent on informed decisions, and the constant spread of misinformation in the media – and endorsement of conspiracy theories in particular – aligns poorly with these prerogatives (Craft, Ashley, & Maksl, 2017, p. 389).

1.1 Purpose of thesis This thesis acknowledges the degree to which conspiracy theories are spreading, and the serious consequences they can cause. It wishes to establish and investigate possible connections between belief in such theories and a lacking trust and/or preference towards mainstream news media, as well as limited knowledge regarding how the news media functions. Additionally, it seeks to explore possible connections to limited critical thinking dispositions and/or skills displayed by audiences. Journalists need to uphold democratic principles by obliging to be dedicated in keeping the public well informed, and they have a ‘social responsibility’ to fulfil these principles by providing an arena for debate, conveying information, monitoring power/government, and delivering content anchored in principles regarding independence, diversity and objectivity (Olsen, 2013, p. 52). It is therefore important for them to be knowledgeable regarding the attitudes and impressionability of audiences. Hence, this thesis will focus on media consumers, and on

3 Ronan Eric Reilly investigating how their assessments and understanding of various news media, and how their critical thinking abilities, can relate to both endorsement of various conspiracy theories and having a ‘conspiracy mentality’; a general propensity towards conspirational thinking (Imhoff & Bruder, 2014, p. 25). The reviewed literature and research conducted in this thesis can hopefully contribute in a small way to understanding some of the challenges journalism and media workers currently face in our information-saturated society.

News media: Trust, choice, and understanding An abundancy of information inevitably spawns a need to make a choice regarding who to trust – especially when faced with so many contradicting viewpoints. Both rational and irrational reasons can dictate what sources one chooses to trust, but as conspiracy theories run contrary to established beliefs it is logical to assume that they will be represented more frequently by those who do not trust mainstream news. Choosing to distrust such outlets inevitably must lead to placing trust elsewhere, and presumably media consumers broadly agree that a distinction can be drawn between ‘mainstream’ and ‘alternative’ news outlets. While many may advocate loudly that ‘alternative media’ are responsible for spreading fake news and conspiracy theories, others may disagree and even claim the opposite; while some will perhaps not emphasize such dichotomous labelling. This thesis proposes that both frequently consuming ‘alternative’ news and assessing it in a positive manner will be related to conspiracy theory endorsement and -mentality. Moreover, it presumes that a certain amount of fundamental knowledge about the news media is required in order to make sensible choices regarding who to trust and what information is credible. Not all people are aware of how e.g. agenda-setting works or the powerful effects of framing. Craft et al. (2017) points to an increasing emergence of research suggesting that highly ‘news literate’ people are more critical and engaged news navigators and consumers (p. 389). Conspiracy theories are rooted in misinformation, and as most information about the world somehow stems from interpretations of media messages it is presumable that low levels of news media literacy will be associated with such beliefs.

Interpreting and processing media messages While research into news media literacy underscores the importance of critical thinking in general, this thesis wishes to pursue this concept in greater detail. In addition to media consumers’ individual understanding of media mechanisms, and regardless of what sources they place their trust in, various aspects regarding how media messages are interpreted will be called into question. Logic and reason may not always steer interpretations and decisions, which is understandable as there is inevitably not enough motivation or time in the world to critically process everything. It is hypothesized that people are equipped with a degree of general disposition towards critical thinking which they use as a basis for decision making and general evaluation when faced

4 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world with e.g. news stories or alleged claims (e.g. Facione, 2011; Sosu, 2013). An attempt will be made to identify a connection between conspiracy theory belief and low levels of such disposition. It is further important to invoke specific critical thinking skills when interpreting information. Media consumers can find themselves vulnerable to rhetorical techniques, poor argumentation, and fallacious reasoning, which can purposely be used to exploit or convince. This is especially evident in areas like marketing and politics, and frequent persuasion attempts are also continuously attempted by both ordinary ‘mainstream’ and ‘alternative’ news media outlets – including those who chose to harbor and convey conspiracy theories. In addition, many media consumers are potentially influenced by a tendency to search for causal relationships where none necessarily exist. Things can simply sound like they make a lot of sense sometimes, and this may too easily lead to conviction under certain circumstances. Practically oriented critical thinking skills – such as detecting informal logical fallacies in arguments and identifying statements that are phrased to sound profound but mean nothing (referred to as ‘pseudo-profound bullshit’) – will therefore be explored. Additionally, being skillful at various aspects of source criticism is very important. A lack of ability to apply any of these critical thinking skills will presumably be related to both heightened conspiracy theory endorsement and -mentality.

1.2 Focus areas and research questions

In sum, and rooted primarily in existing research, this thesis seeks to examine measured conspiracy mentality and conspiracy theory endorsement in relation to the broader overall two focus-areas of:

a) News media trust, source preference, and literacy; and b) Critical thinking dispositions and skills.

Main research questions More specifically, based on these areas of interest, a total of eight research questions will be formulated and investigated. Relating to media trust, alternative media preference, and news media literacy, the following four research questions will be asked:

RQ1: What relationship exists between skepticism towards mainstream media and conspiracy theory endorsement / conspiracy mentality? RQ2: What relationship exists between frequency levels of alternative media consumption and conspiracy theory endorsement / conspiracy mentality? RQ3: What relationship exists between different attitudes towards alternative media sources and conspiracy theory endorsement / conspiracy mentality? RQ4: What relationship exists between news media literacy levels and endorsement of conspiracy theories / conspiracy mentality?

5 Ronan Eric Reilly

Then, four additional questions regarding critical thinking dispositions and skills are treated:

RQ5: Is there a relationship between individual levels of critical thinking disposition and endorsement of conspiracy theories / conspiracy mentality? RQ6: Is there a relationship between susceptibility to arguments based on informal logical fallacies and endorsement of conspiracy theories / conspiracy mentality? RQ7: Is there a relationship between failing to detect ‘pseudo-profound bullshit’ and endorsement of conspiracy theories / conspiracy mentality? RQ8: It there a relationship between level of source criticism ability and endorsement of conspiracy theories / conspiracy mentality?

Figure 1.2.1 – Flowchart of Conspiracy theory endorsement main thesis interest areas, variables, and research questions. Conspiracy mentality

A: News media trust, source preference, and literacy B: Critical thinking dispositions and skills

RQ1: RQ3: RQ7: RQ5: Mainstream media Alternative media Pseudo-profound bullshit Critical thinking disposition skepticism attitudes detection ability

RQ2: RQ6: RQ4: RQ8: Frequency of alternative Susceptibility to informal News media literacy Source criticism ability media consumption logical fallacies

Figure 1.2.1 illustrates graphically the makeup of the thesis’ dependent variables, interest areas, and corresponding research questions. In the analysis phase these research questions will be broken into hypotheses which will assist in providing answers. Due to the nature of the gathered data – as will be elaborated upon – all questions are asked with a “within a non-probability general population sample of N=125 Norwegian people”-premise. This prerequisite will not be added to every question; however, it is important to point out that it is implied. All performed statistical tests will also measure responses to a ‘true verifiable conspiracies’-scale which will be viewed as a comparison variable, but no additional hypotheses will be formulated for tests performed on it.

Method This is a quantitative study, and all data necessary for answering these research questions are gathered using a cross-sectional survey questionnaire. Rooted in the scientific literature that will be treated, a combination of pre-existing and validated Likert-scales measuring specific constructs will be adapted and implemented. Additionally, some constructs are self-defined and measured by newly created scales which are prototypical in nature. The literary review and methodology sections will

6 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

provide an in-depth elaboration over all choices made, and backgrounds for these decisions. Through analyzing the gathered data collected from a variety of Norwegian respondents (N=125; non-probability, general public sample), an effort will be made to re-affirm findings from previous research – as well as to hopefully discover new relationships which are statistically significant in relation to conspiracy theory endorsement and -mentality. Interpretations will rely on statistical SPSS output from bivariate correlations and ANOVA’s (analyses of variance).

Expectations It is expected that the answers given by the surveyed aggregate of respondents will reveal that both conspiracy theory endorsement and -mentality will be positively correlated with mainstream media skepticism, and negatively with all aspects of news media literacy, to a statistically significant degree. Additionally, it is expected that mean endorsement- and mentality-levels will significantly vary in relation to differences in ‘frequency of alternative media consumption’-groups. The means are also expected to be significantly different in relation to those who positively evaluate alternative sources as oppose to those who view them pejoratively. Similarly, different levels of critical thinking disposition will expectedly reveal statistically significant contrasting mentality- and endorsement mean values. Furthermore, statistically significant positive correlations between conspiracy theory endorsement/ -mentality and both informal logical fallacy susceptibility and ‘pseudo-profound bullshit receptivity’ are expected. Conversely, statistically significant negative correlations are anticipated for source criticism ability.

Scope Many of the addressed phenomena, concepts and constructs in this thesis are potentially limitless in their exploration possibilities, and as conspiracy theory research material is also growing substantially it would be impossible to cover but a fraction of it in a limited master’s thesis. Focused-upon notions such as e.g. ‘critical thinking’ and ‘news media literacy’ do not only encompass many sub-dimensions, but there are also no universally agreed upon specific definitions. Even the ‘conspiracy theory’-term itself is open to interpretation. Considerable effort will be devoted to explaining, defining, and concretizing all applicable terms, but some concepts will have to be treated very broadly, such as e.g. ‘alternative media’. This concept will be elaborated upon, but as a construct it will only be researched based on respondents’ own interpretations of ‘non- mainstream’ media; without distinguishing between main themes, distribution methods or topics of focus. Research will naturally also be limited in the sense that it will pertain only to a small group of non-probability sampled Norwegians, but in addition to the planned statistical operations some descriptive statistics of findings – such as which conspiracy theories are most endorsed within the sample – will also be reviewed. The various Likert-scales that are used all have their limitations,

7 Ronan Eric Reilly and many findings will e.g. generalize entire constructs based on only a few questionnaire-items. Most weaknesses will however be thoroughly pointed out where applicable throughout the thesis, and extensively towards the end. It is not a core aim to actively seek to debunk any conspiracy theories, but rather to explore the mechanisms behind endorsement and tendency to subscribe to them in light of media message evaluation and interpretation. Also, it is not the intent of the thesis to purposely plot against alternative media with the aim of proving them inferior to mainstream; nor is it the intention to do the opposite. Moreover, it is also unfortunately beyond the scope of this thesis to assess all the covariate interactions between independent variables in the individual research questions. The aim is simply to identify and discuss relationships addressed in the eight research questions, all the while hoping that findings possibly can provide some insight for journalists and media workers regarding audience preferences, and how they assess information. Some secondary sub-themes will however also be investigated – such as comparisons between conspiracies and conspiracy theories, and the relationship between conspiracy mentality and conspiracy theory endorsement.

Scientific positioning Conspiracy theories have been studied by many scientific fields, but the lion’s share of research is rooted in schools of and media studies. Although several fields inevitably will be heavily involved, the primary focus will be centered around media studies given the journalism- and media-oriented nature of the thesis. There are however blurry lines between media studies and many humanistic- and social sciences such as psychology, , and communication- and rhetorical studies. An influence of elements from such areas will be largely revealed throughout.

1.3 Structure and organization

The thesis will begin with a literary review, which at first – in section 2.1 – will address the main dependent variable, conspiracy theories. Following a brief historical overview of research, it will initially attempt to present relevant working definitions of central terms, before elaborating on the vastness of research that has been done with regard to their traits and characteristics, mechanisms leading to their spread, underlying motivational reasons for subscribing to them, and research regarding measurement possibilities. Chapter 2.2 is devoted to the treatment of the thesis’ first area of interest – news media trust, source preference, and literacy. Accordingly, it will act as a basis for answering RQ1 through RQ4 by exploring relevant current literature and introducing various means of measuring specific constructs. It will contain working definitions of – and explanations regarding – what is meant by terms such as ‘media trust, -skepticism, and -credibility’, ‘mainstream media’ and ‘alternative

8 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world media’, as well as elaborating extensively on what ‘news media literacy’ encompasses. Next, chapter 2.3 will present research regarding critical thinking dispositions and skills – as per the second area of interest to this thesis. As a means toward answering RQ5 through RQ8, an overview of literature defining critical thinking and its components will be presented. Firstly, an attempt will be made to dissect, present, and narrow down this broad term, before focusing specifically and elaborating on research pertaining to critical thinking disposition, informal logical fallacies, ‘pseudo-profound bullshit’, and source criticism. In section 3: ‘Methodology’, an attempt will be made to present the choice of method for this study, as well as provide in-depth explanations and deliberations pertaining to the process from groundwork and construction to distribution, collection, and preparation of data for analysis. Important sections include 3.5 – where specific variables are determined and operationalized – as well as 3.3 and 3.4, which respectively review sampling, validity, and reliability considerations, and explain how the questionnaire was designed and structured. The ‘Analyses and discussions’-section, 4, begins by performing an initial investigation of the sample data through univariate analyses and exploration; first of demography and answers given regarding media views, habits and preferences – and then of data from all Likert-scales. It proceeds to investigate the main dependent variables – conspiracy mentality and conspiracy theory endorsement – in conjunction with each other, and how common conspiracy mentality and endorsement are within the sample. The rest of the section treats every research question chronologically; providing findings followed by discussions relating to these. The fifth and final section offers summaries and conclusions, where findings are recapped and summarized in context with the thesis purpose. Limitations and weaknesses are contemplated before a conclusive discussion is provided involving e.g. practical implications of findings for journalism and media workers. In closing, some final thoughts are offered.

9 Ronan Eric Reilly

2 Literary review

2.1 Conspiracy theories Brief historical overview of research and media interest Conspiracy theories have gained increasing interest among journalists and media scholars in the recent years. Although the historian Richard Hofstadter (1966) began some academic investigations in the mid twentieth century limited scholarly involvement would be devoted to this phenomenon until the 1990’s, and it was not until 2007 that a real interdisciplinary research agenda emerged (Uscinski, 2019a, p. 2). Psychologists led the way – investigating the mechanisms behind such beliefs, and common personality traits associated with them. Rapidly, their endeavors would prove to engage political scientists, and soon after an ensemble of various other social scientists within e.g. sociology, social anthropology, and would join (Uscinski, 2019a, p. 2). This interest was likely transposed onto the public as journalists turned their attention to the topic shortly thereafter, and from treating conspiracy theories as a fringe phenomenon, many media companies began taking it upon themselves to write vividly and frequently about them. Uscinski (2019a) noted the fact that in November 2017 the term occurred within at least one article almost every day in – contrasting immensely with the zero occurrences found 40 years earlier (p. 3). Byford (2011) pointed out that there has been a lack of cooperation regarding conspiracy theory research across individual disciplines or professions between historians, political scientists, psychologists, and journalists (p. 3), but an ever-increasing interest in the topic has caused many scholars (e.g. Uscinski, 2019a) to urge all fields to come together, share findings, and propose future research. For now, this literary review will only be able to lightly scratch the surface of the plentiful existing work, but it will strive towards identifying what is most relevant to the current research as seen from a media studies perspective; although inevitably involving several related fields. From a journalistic perspective it is paradoxical that while many media channels recognize the importance of reporting on emerging conspiracy theory research – and increasingly emphasize the possible dangers of their endorsement – others remain persistent in continuing to spread them at a radical rate.

Definitions and terminological explanations The term ‘conspiracy theory’ did not become associated with the concept as we think of it today until the emergence of the aforementioned twentieth century research (Uscinski, 2019a, p. 2). Throughout the ages many media outlets have claimed that something is not a conspiracy theory if it actually happened, and to this day arguments will potentially arise among audiences over the

10 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world semantic definitions of the separate words ‘conspiracy’ and ‘theory’ joined together, and the ‘conspiracy theory’ as a phenomenon. As with any term subject to scientific research it is therefore particularly important to begin with concrete definitions to avoid ambiguity. This thesis will deal primarily with the terms ‘conspiracy’, ‘conspiracy theory’, ‘conspiracy theory endorsement’ and ‘conspiracy mentality’. ‘Conspiracy’, as defined by Oxford Learners Dictionary, is “a secret plan by a group of people to do something harmful or illegal” (Conspiracy, 2020). For the purpose of using the term within scientific discussion, Uscinski (2019b) extended this simplistic definition into: “a secret arrangement between two or more actors to usurp political or economic power, violate established rights, hoard vital secrets, or unlawfully alter government institutions to benefit themselves at the expense of the common good” (p. 48). He contrast ‘conspiracies’ with ‘conspiracy theories’, dichotomously differentiating between them as ‘probably true/proven’ and ‘probably false/unproven’ (pp. 48-49). This thesis will adapt this distinction, but to avoid confusion the term ‘conspiracy’ may be specified as ‘true’, ‘verifiable’, or ‘actual’ where applicable. The term ‘conspiracy theory’ does not have a single agreed upon definition. In its most basic form, a dictionary describes it as “the belief that a secret but powerful organization is responsible for an event” (Conspiracy theory, 2020). In the eyes of many scholars however, the term carries a multitude of pejorative connotations with it, and this simplistic definition does not do justice to its cultural and ideological baggage. This is pointed out e.g. by Butter and Knight (2016, p. 7), who also highlight that the lack of agreement on a definition causes confusion in research literature. Bost (2019, p. 271) acknowledges these scholarly issues, and also calls attention to the inconsistent usage of the term in public discourse and the media. This thesis will align with Uscinski (2019b), who sees conspiracy theories as an “accusatory perception which may or may not be true, and which usually conflicts with the appropriate authorities” (p. 48). They are “…explanations of past, ongoing or future events or circumstances that act as a main causal factor for a small group of powerful persons, the conspirators, acting in secret for their own benefit and against the common good” (Uscinski, 2019b, p. 48). Throughout this thesis the term will be treated primarily as the irrational and opposite counterpart of the ‘conspiracy’, but e.g. Uscinski (2019b) does acknowledge that what today is considered a conspiracy theory can become a conspiracy if somehow proper epistemic authorities can turn accusations into verified information and accepted knowledge (p. 49). ‘Conspiracy theory endorsement’ is a term widely used in this thesis as it is a main dependent variable. It simply addresses the act of believing in or subscribing to one or more specific conspiracy theories (Uscinski, 2019b, p. 50). The term itself does not imply any stable or specific underlying reasons for doing so, but it is hypothesized that the other widely used dependent

11 Ronan Eric Reilly variable, ‘conspiracy mentality’, may often be a major cause for endorsement. Equivalent terms used in the literature are ‘conspirational predisposition’, ‘conspiracist ideation’, ‘conspiracy ’, ‘conspiracy ’, ‘conspirational ’ or ‘conspiricism’. They all describe the concept of holding the same underlying worldview or disposition toward seeing events and circumstances as the product of conspiracies (Uscinski, 2019b, p. 50). Specifically, Bruder et al. (2013) defines conspiracy mentality as “…the general propensity to subscribe to theories blaming a conspiracy of ill-intending individuals or groups for important societal phenomena” (p. 2); which by Imhoff and Bruder (2014) later has been more simply viewed as a “general propensity towards conspirational thinking” (p. 25), and seen as a mentality that can be studied and understood much like a general political attitude. This thesis differentiates the conspiracy theory endorsement term from conspiracy mentality in that the former refers to the ‘act’ of actual subscription, while the latter presumes the existence of a predispositional general tendency to be acceptant of conspiratorial claims. It is important from a media perspective to separate these constructs; as while e.g. journalists practically may be able to affect endorsement in some cases, altering someone’s mentality potentially poses very different challenges. It will therefore be very interesting to determine possible differences between various studied variables and their potentially differing relationship to endorsement and mentality, respectively. Further elaborations regarding both these terms and their differences will be revisited later in this chapter.

Conspiracy theory characteristics and traits

Conspiracies vs conspiracy theories As per definitions, a distinction will be drawn between verifiable conspiracies and conspiracy theories. This is important, as it is an aim to distinguish between audiences who critically assess information with the intention of seeking the truth – or who are able to better evaluate what information is true – and those who possibly are accepting of conspiracy claims; either because of underlying predispositions or an inability to assess received information well. The dependent endorsement variable in the current study will base its measures on a scale which includes both conspiracies and conspiracy theory items, and this will be elaborated upon in chapter 3.5. Real conspiracies undoubtably occur all the time, and it is not by default irrational to be suspicious or critical of what the media chooses to report. Healthy skepticism is also as important to media workers as it is to audiences. Investigative journalists would not have exposed the Watergate scandal without asking questions and uncovering actual evidence; a story which incidentally turned out to be a true conspiracy (Gray & Burroughs, 1987). Another example of an actual conspiracy is the CIA’s ‘MK Ultra’ operations – where the innocent public were used as guinea pigs in mind- control experiments (Valentine, 2016). Volkswagen also did conspire to cheat emissions tests, the

12 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world tobacco industry did deceive the public regarding harmful health effects of smoking (Lewandowsky & Cook, 2020, p. 3), and the Phoebus Cartel – consisting of major producers of lightbulbs – did conspire to purposely reduce the life expectancy of bulbs in order to increase profits (MacKinnon, 2016). These conspiracies have all been thoroughly investigated and exposed through e.g. industry documents, government inquiry, or whistleblower action etc. Conversely, conspiracy theories seem to persist without verifiable evidence ever surfacing. Most are what Barkun (2003) describes as ‘event conspiracy theories’ (p. 6), and range from ‘classics’ such as e.g. CIA involvement in the JFK-assassination, faking the Moon landing, , and staging 9/11 – to the more recent ‘Pizzagate’, or various deliberate COVID-19 plots. Barkun also suggests that some theories are ‘systemic’ (p. 6), involving notions of ongoing governmental efforts at covering up the existence of UFO’s, or hidden conspiratorial groups who wish to take control of the world – such as the Illuminati. He further describes intricate ‘superconspiracies’ (p. 6), which link together multiple events and groups in elaborate ways; an example being the worldwide infiltration of alien ‘lizard people’ who are in control of world governments, royal families, and banks etc. (Icke, 1999). Keeley (1999, pp. 122-123) pointed out that the more complicated and bigger a conspiracy theory becomes, the less likely it is to be true. Grimes (2016) estimated e.g. that faking the Moon landing would require 411,000 people to be involved, and it would be exposed within 3.68 years (pp. 8, 11). Regardless of their level of intricacy or likelihood, conspiracy theories widely share the same common traits, and are all based on flawed patterns of thinking (Byford, 2011, pp. 71-94; Keeley, 1999, pp. 118-119; Lewandowsky & Cook, 2020, p. 6), which deem them more a matter of faith than something worth proving or disproving (Barkun, 2003, pp. 3-4). The sheer unlikelihood and absurdness of some theories alone should often be sufficient to reject them downright, yet many media consumers increasingly seem to fail in doing so. Figure 2.1.1 provides a highly simplified

Figure 2.1.1 – Conventional vs conspiratorial thinking comparison between conventional and conspiratorial (Lewandowsky & Cook, 2020, p. 3) thinking patterns. A more in-depth review of the components of conspiracy theories, their use of rhetoric, and their common traits will follow as this is seen as being very important in order to gain a proper understanding of how they are constructed, and how they can entice audiences.

Conspiracy theory anatomy, rhetoric, and traits In addition to offering explanations for incidents or circumstances, conspiracy theories are often also thrilling and fascinating stories for consumers to enjoy, and many media outlets are well aware of how to utilize this. At the core of these stories are evil antagonists, elaborate arguments, and a

13 Ronan Eric Reilly distinct triad of narrative elements consisting of conspirators, a plan and an endeavor to maintain (Byford, 2011, p. 71). Byford (2011) describes nine characteristically typical elements. There is usually a conspiratorial group (p. 71), where classic examples are the Illuminati or the Freemasons, who have certain important characteristics (p. 75); e.g. cruel and powerful or thriving on human fear. A conspiratorial plan (p. 76) – often believed to be revealed or concealed in some secret document – must be in place in order to exert manipulation of many by few (p. 79). A Manichaean view of politics (p. 82) – a dichotomous ‘good vs bad’ portrayal of major issues – often suggests that conspirators’ evil aspirations are on the verge of being fulfilled. However, the naïve (p. 83) of conspiracy theories is the premise that if good people can communicate these sinister plots well enough for the public to understand the ‘real truth’, then the conspiratorial groups will become powerless. Furthermore, the importance of foreknowledge (p. 86) – planning or knowing about incidents in advance – is key. For instance, if influential people chose to cancel a flight or did not show up for work on 9/11, this is often regarded as ‘evidence’ that they were involved. Many alternative news sources may tend to feast on mainstream-skeptical audiences and can easily create appealing articles by pointing fingers and over-simplifying premises and plots in accordance with these narrative elements. Byford (2011) points to two aspects of rhetorical trickery that are important. Firstly, a rhetoric of scientific inquiry (p. 88) is widely embraced, consisting of e.g. an elaborate with demonstrations and presentation of so called ‘proof’. Conspiracy theorists see themselves as investigators and researchers, and often regard each other to be ‘experts’; resulting in a quasi- academic style and a rooted in irrefutable logic. This stands in stark contrast to the notion of falsifiability, which is a foundational pillar of science. It is therefore an ironic observation that science skepticism or denialism towards various peer reviewed scientific research is widely endured by many conspiracy theorists, yet they still seek to emulate mainstream scholarship and enquiry striving towards being judged by those very standards (p. 89). Another widely applied tool a conspiracy theorist will seek to use, and a ‘trick’ which very often can be seen throughout all media outlets, is the rhetoric of just asking questions (p. 90). This can manifest as a form of agenda setting, where the audience’s attention is diverted away from clear inconsistencies in a theory by focusing only on what is unanswered. Also, instead of attempting to explain a complicated matter one can simply infer that absence of clear evidence, or missing answers to irrelevant questions, are in themselves an attempt at concealing the truth. Informal logical fallacies – which will be revisited in section 2.3 – are also interwoven with such potential trickery, and many conspiracy theories suggested by countless news sources are often intentionally or unintentionally riddled with flawed reasoning. High levels of critical thinking disposition and source criticism ability, as well as e.g.

14 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world skillfully detecting fallacies should therefore reduce the likelihood of endorsing such theories. In compliment to these proposed elements, Lewandowsky and Cook (2020, p. 6) – based on Lewandowsky et al. (2015, pp. 148- 149) – have postulated that there are seven traits which characterize Figure 2.1.2 – Seven traits of conspiratorial belief (Lewandowsky & Cook, 2020, p. 6) conspiracy theories (see figure 2.1.2 for an overview). Firstly, they are contradictory – meaning they are likely to entertain mutually exclusive claims. Wood, Douglas, and Sutton (2012, p. 767) e.g. found that many would accept both the theory that Princess Diana was murdered, while also believing that she faked her own death; or found it more likely that Osama bin Laden was already dead when the US raided his compound if they simultaneously believed he was still alive. An incoherent belief system also seems to be secondary to disbelief in ‘official’ accounts, and hence this overrides suspicion. As mainstream media reports often constitute the very notion of ‘officialness’, it is highly likely that conspiracy theorists will report being skeptical towards mainstream news. Extreme degrees of suspicion can prevent belief in anything that does not fit into the conspiracy theory (Keeley, 1999, p. 116). behind any presumed conspiracy are invariably assumed to have nefarious intent, (Keeley, 1999, p. 117), and the official account must be based on ; i.e. something must be wrong (Lewandowsky, Cook, & Lloyd, 2016, p. 175; Wood et al., 2012, p. 767). Conspiratorial thinking also involves a self-perception of simultaneously being a persecuted victim, and a hero taking on the villainous conspirators (Lewandowsky et al., 2015, p. 148). As such, related mainstream media skepticism is likely to spawn an inquiry into ‘alternative’ media counterparts in search of real ‘truths’. Furthermore, conspiracy theories are self-sealing and immune to evidence because of their inherent dependency on circular reasoning (Lewandowsky & Cook, 2020, p. 7; Sunstein & Vermeule, 2009, p. 204; Wood et al., 2012, p. 767). The stronger the evidence against a conspiracy, the more the conspirators must want people to believe their version of events. Finally, randomness is re-interpreted and woven into a broader interconnected pattern due to a belief that nothing occurs by accident (Swami, Chamorro‐Premuzic, & Furnham, 2010, p. 758). Therefore, an article reporting on e.g. small insignificant details – such as intact windows at the Pentagon after the hijacked 9/11 aircraft impacted – can be viewed in isolation by audiences and taken to be ‘evidence’ that there never was a crash. Based on these traits, one can easily imagine how a devious creator of media content could invoke a conspiracy within any event by stirring up stories involving arbitrary surrounding circumstances. Therefore, both considering sources and critically evaluating information is of pivotal importance.

15 Ronan Eric Reilly

The distribution of conspiracy theories

Popularity and use in the media The widespread popularity of conspiracy theories is reflected in vast amounts of literature, movies, series’, and documentaries etc. centralizing on conspiracy-themes (Byford, 2011, p. 7), and both mainstream and alternative outlets constantly harvest the ‘market’ generated by the appeal these theories offer. Several entertainment shows within many genres are based on conspiracy related themes, such as the popular 90’s show “The X-files”; or more recently the Norwegian podcast “Konspirasjonspodden”, which assesses conspiracies and attempts to debunk them. Other prime examples are Hollywood blockbusters such as “JFK” or “The Da Vinci Code”. Arguably a major appeal of such ‘good’ stories are their ability to intertwine fascinating conspiracies and cover-ups with real history. Network television has also for years boosted conspiracy interest through the airing of documentaries investigating various conspiracy theories (Byford, 2011, pp. 8-9). ‘Speculative history’ e.g. is a genre mastered by such channels as the History Channel which feasts directly of conspiracy rhetoric and pseudo-science in order to entertain; with shows like “Ancient Aliens” or “Ghost Hunters” being examples. ‘Experts’ who share viewpoints are often equally accredited regardless of whether they are actual scholars, trained historians, amateur enthusiasts, ‘ancient astronaut theorists’ or conspiracy buffs. D. Thompson (2008) points out that this tendency inevitably dilutes the weight of genuine scholarship. This is worrying, as the mimicking of ‘real’ scientific inquiry easily can fuel an entire crowd of curious audiences. Again, to avoid being persuaded by misinformation it is imperative to critically evaluate sources and information well, while also being able to analyze and interpret media messages. This is especially important because many alternative media sources – some of which masquerade as serious news – continuously hash out conspiracy theories and fake news. A notorious example of such a source is e.g. ’ ‘Infowars’. A motivator for spread can simply be to generate revenue due to views or click rates, but because of their appeal conspiracy theories are often intentionally constructed or amplified for strategic or political reasons. Incoherent conspiracy rhetoric can be applied to escape inconvenient conclusions and/or undermine the strength of scientific evidence. E.g. ‘climate denial’ is filled with claims that global temperatures have declined, while simultaneously proposing that temperature cannot be measured accurately (Lewandowsky et al., 2016, p. 175). Other recent examples of such tactical misuse is the Russian government’s attempts at purposely spreading various political conspiracy theories in the West (e.g. Avramov, 2018), and the claims currently being made by the Trump-campaign that Democrats must have ‘somehow’ cheated with the mail-in ballots in an attempt to avoid conceding the presidency in the US 2020 election. Many outlets can also unintentionally end up spreading untrue or unsubstantiated stories simply due to poor research or

16 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world biased partisan opinions within the editorial environment. There are vast amounts of different outlets ranging in seriousness, competency, agenda and focus, and ‘alternative’ media sources are not suggested to be the exclusive providers of false news. Far from all ‘mainstream’ news media are free from fault when it comes to supporting and strengthening conspiracy beliefs (Byford, 2011, p. 9). To combat misinformation, the recent years have seen an emergence of fact-checking companies such as ‘Politifact.com’ in the USA or ‘Faktisk.no’ in Norway. Although this is an important step in the right direction, many audiences still have a long way to go in successfully evaluating sources and information.

Mechanisms of spread Regardless of whether conspiracy theories are motivated by political tactics or by generating traffic, they do spread like wildfire. One could simply blame ‘the internet’ and non-gatekept alternative media in a general way for this, however there appear to be mechanisms at play with regard to endorsement of conspiracy theories which are non-specific to their means of distribution. This can be challenging from a media perspective, e.g. as journalists largely relinquish control of a message once information (or misinformation) has entered into circulation. Sunstein (2014) has investigated some possible mechanisms attempting to explain why conspiracy theories potentially can snowball. He suggests that crippled epistemologies – simply a general lack of comprehensive and accurate knowledge – is one fundamental reason (pp. 10, 12). Many extremists and conspiracy theorist audiences unfortunately fall into this category, where a lack of relevant information supports their ‘crippled’ views. Another fundamental reason for spread is rumors and speculation (p. 12), whether they intentionally are in motion by sources who stand to profit, or unintentionally come about through economic distress, perceived injustice, or finding certain events disturbing. Sunstein presents four conspiracy cascades (pp. 15-19), which essentially are simplified scenarios of the very complicated processes in which information passes, or ‘cascades’, through a chain of people. As a conspiracy theory is ‘handed down’, thresholds for acceptance of what one is being told – combined with prior knowledge on the matter, motivational aspects, and perceived ethos attributed to the source of the information – are some key components which determine what one will entertain; and accordingly continue to spread. Possible motivational factors will be treated in the next section. Links are further drawn between cascades and (p. 19) – a well- established phenomenon by which members of a group typically end up with more extreme views due to a collective mindset. Once cascades arise or polarization occurs – and an increasingly extreme view develops – selection effects (p. 21) are likely to materialize; where sceptics may begin leaving the group, while those who remain will be fewer in number but more extreme in their views. This process is closely related to the introductorily presented concept of ‘echo chambers’. It

17 Ronan Eric Reilly is highly important to consider the media’s role in relation to these mechanisms.

Psychological conspiracy theory research: Who believes, and why?

Predisposition or circumstance? The previous sections have dealt with the characteristics of conspiracy theories and how they spread. The following will review some research addressing individual reasons why audiences may choose to endorse them. This is important to the thesis for many reasons. Firstly, one of the main grounds for differentiating between two dependent variables – ‘conspiracy mentality’ and ‘conspiracy theory endorsement’ – is diverging scholarly views regarding the psychological roots of conspiracy belief. Additionally, any discoveries suggesting underlying dispositions taking precedence over circumstantial reasons are important to media workers because it likely decreases the possibility of conscientious journalists succeeding in influencing audiences and debunking such theories. Conversely, it increases the likelihood of devious outlets exerting minimal effort to generate stories that resonate with conspiracy claims. Although conspiracy theories differ widely in their content, findings suggest that the best predictor of belief in one conspiracy theory is belief in a different one. It is therefore claimed that conspiracy theory endorsement presumably is based on a general ‘monological’ worldview (e.g. Goertzel, 1994; Sutton & Douglas, 2014; Swami et al., 2011), and hence some scholars argue that people can possess a relatively stable conspiracy mindset or mentality (e.g. Imhoff & Bruder, 2014); as per the measured dependent variable in this thesis. Butter and Knight (2016) point out that while many studies strive to find such stable personality-specific traits of conspiracy belief, others have chosen to focus more on specific reasons; e.g. faulty reasoning, and susceptibility towards and etc. (p. 3). Many other studies have suggested that circumstantial factors or certain topics are needed to trigger personality traits associated with conspiracy endorsement (e.g. Douglas & Sutton, 2015, p. 102). These findings are more practically valuable to journalism, as they to a larger degree could allow for affecting audiences through narratives. It is acknowledged that a complex interplay of many factors is inevitably responsible for suspecting conspiracies, but measuring separate dependent variables will hopefully help to uncover whether there is a major difference between relationships to the independent variables when considering both the tendency to endorse and the actual endorsement of some conspiracy theories.

Motivational factors As this thesis is concerned with audiences’ motivations for news consumption and reasons for choosing where to place trust, it is inevitably fruitful to review psychological research pertaining to motivational factors associated with conspiracy belief. Such factors are also intrinsic to the

18 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world mentioned conspiracy cascading process. According to e.g. Douglas, Sutton, and Cichocka (2017), vast amounts of ad-hoc psychological conspiracy theory research loosely culminates into reasonings that are socially motivated; maintaining a positive image of the self or the social group, existentially motivated; feeling safe and in control of one’s environment, or epistemically motivated; based on understanding one’s environment (p. 538). To shed light on the complex interaction between the media and media consumers’ conspiracy theory endorsement this will be elaborated on.

Social motivations Belief in conspiracy theories are found to be highly sensitive to social context (van Prooijen & Douglas, 2018, p. 898). Such beliefs can be particularly appealing to people who find the positive image of themselves or someone in their social group to be threatened, and they are associated with collective narcissism (Cichocka, Marchlewska, de Zavala, et al., 2016, p. 572). Conspiracy belief is however also associated with narcistic individuals who have low self-esteem and require a lot of external validation (Cichocka, Marchlewska, & de Zavala, 2016, p. 157). Studies also suggest a link between conspiracy theories and a desire to be unique (Lantian et al., 2017, p. 160), or simply experiencing boredom (Brotherton & Eser, 2015, p. 1). Other studies have shown that feeling socially excluded can lead people to believe in both superstitions and such theories (Graeupner & Coman, 2017, p. 218). Moreover, a connection between conspiracy theories and disenchantment with politicians and scientists has been found (e.g. Jolley & Douglas, 2014b, p. 35; Lewandowsky, Oberauer, & Gignac, 2013, p. 622). Several of these social aspects can be seen in relation to the spread of conspiracy theories in various media. E.g. sharing alternative articles or posts via social media may make some people feel unique, and responses generated can fulfil external validation for narcissistic needs. Also, those who are bored may read a fascinating story with a lowered guard due to a lack of better things to do. Additionally, disenchantment with politicians and scientists is likely to be associated with non-mainstream media – which can fuel an urge for alternative views.

Existential motivations Hofstadter (1966), theorizing around what he called the ‘paranoid style’, was introductorily accredited for pioneering the initial conspiracy theory research. He argued that this ‘style’ was a result of ‘uncommonly angry ’ with ‘distorted’ judgement (Hofstadter, 1971, pp. 2-3). Some scholars have viewed conspiracy theories to be a result of psychopathology, such as extreme or delusional ideation (e.g. Groh, 1987; Plomin & Post, 1997), but e.g. Sunstein and Vermeule (2009) claim such explanations are incomplete because conspiracy theories are so widespread. Some of Hofstadter’s views have remained influential, such as those suggesting that they are more likely to be endorsed by those who feel powerless, disadvantaged, or voiceless – especially when facing a catastrophe (Swami & Coles, 2010, p. 561). Several findings have since

19 Ronan Eric Reilly supported this; e.g. that people develop beliefs about the world partly to obtain a sense of safety and security (Hart, 2014, p. 19; Hart & Graether, 2018, p. 229). Moreover, studies have linked conspiracy belief to anxiousness (Grzesiak-Feldman, 2013, p. 100), political cynicism (Swami et al., 2010) and displays of distrust in authority (e.g. Wood et al., 2012, p. 767); which align with aforementioned social reasons and clearly can be seen as grounds for mainstream media skepticism. Douglas and Sutton (2008, p. 210) showed that people can be effectively persuaded by pro- conspiracy material without being aware of it. This is particularly interesting from a media influence perspective, as readers of certain news may not be readily aware of the impact their consumption can have on them. Generally, today’s media landscape is a risky ‘battleground’ for people vulnerable to many of the existential motives addressed here, and one can easily envisage the path from a need to calm powerlessness or anxiety – while already distrusting authorities – to a pursuit of certain shady genres of media content.

Epistemic motivations People will always try to make sense of the world around them, and a large portion of information about what goes on in that world is provided by the news media. When satisfactory information is unavailable uncertainty may be quenched by far-fetched explanations, and conspiracy theories can provide internally consistent explanations when e.g. official accounts for major events are lacking (Douglas et al., 2017, p. 539; Marchlewska, Cichocka, & Kossowska, 2018, p. 109). There seems to be an irrational ‘need’ for attributing great causes to proportionately large and important events (e.g. McCauley & Jacques, 1979, p. 637). It is e.g. simply difficult to accept that one man, with no grand cunning plan or backing from major contenders, could be the reason a president or a crowd of people were killed. Such motivations could potentially override logic and reason. Of direct relevance to this thesis, conspiracy belief is found to be correlated with lower levels of critical, rational, and analytic thinking (Swami et al., 2014, p. 572). This is likely due to belief in conspiracy theories being strongly rooted in negative emotions and intuitive automatic processes, and therefore non-emotional thinking appears to decrease these beliefs (van Prooijen & Douglas, 2018, p. 902). Accordingly, the combination of motivation to be rational and thinking analytically predicts conspiracy theory skepticism (Ståhl & Van Prooijen, 2018, p. 155). This thesis seeks to find correlations between conspiracy theories and several aspects of critical thinking. Given the abundance of related research this could be considered a re-affirmation more than an investigation, but – as will be revisited in chapter 2.3 and 3.5 – exact scales and constructs used in this research will differ from previous measures; hopefully adding to existing findings. Another aspect of focus in this thesis, which will also be revisited in 2.3, is the possible correlation between conspiracy theories and a tendency to perceive profoundness in nonsensical,

20 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world but superficially meaningful ideas (‘pseudo-profound bullshit’). Pennycook et al. (2015) found that those who do are indeed more likely to engage in conspiratorial ideation. This tendency might be related to the vast amount of research pointed out by van Prooijen and Douglas (2018) suggesting that perceiving patterns in randomness, as well as perceiving agency and intentionality where none exists, predicts belief in conspiracy theories (p. 901). Such findings help explain why conspiracy theorists often will see images in random noise or perceive patterns in arbitrary stock market information (Jennifer A. Whitson & Galinsky, 2008, p. 115), as well as attribute ‘suspicious and revealing’ connections between e.g. 9/11-conspiracies and folding a 20 dollar bill a certain way to unveil images of burning twin-towers. Furthermore, conspiracy theories have also been associated with belief in paranormal phenomena, as well as other possibly ‘flawed’ ways of viewing the world, such as thinking (e.g. Butter & Knight, 2016, p. 3; Jennifer A Whitson, Galinsky, & Kay, 2015, p. 89). Pennycook et al. (2015) also drew connections to paranormal beliefs in the same ‘pseudo-profound bullshit’-research that found conspiracy mentality connections. Such findings may indicate a link between endorsement of conspiracy theories and certain types of ‘far out’ alternative media content – e.g. the aforementioned ‘speculative history’ genre. Other studies have focused on relationships between conspiracy mentality and various cognitive biases. Connections have been found e.g. in relation to mistaken causal and overreliance on intentionality; the ‘fundamental attribution error’, faulty estimations of probability; the ‘conjunction fallacy’, and an unwillingness to reject beliefs in the face of contrary evidence; ‘confirmation ’ (Brotherton & French, 2014, 2015). Moulding et al. (2016, p. 345) established a connection with a general reliance on heuristics. All these findings align with and add to the assumption that conspiracy beliefs do not appear grounded in controlled, analytic mental processes. This thesis hypothesizes that similar mechanisms also will come into play in relation to e.g. informal logical fallacies in argumentation. It is further considered likely that associations between susceptibility to biases or heuristics and conspiracy mentality are reinforced by critical thinking dispositions, which shape the way media consumers search for, interpret, and question evidence (e.g. Swami et al., 2014, p. 573). Such dispositions are central to the investigation this thesis pursues in relation to RQ5. R. F. West, Toplak, and Stanovich (2008) found modest relationships between heuristics, biases and such dispositions (p. 930), but Swami et al. (2014) pointed out that the relationship to conspiracist ideation has received limited attention (p. 573); which is why it is of interest to this thesis. Banas and Miller (2013) have however shown that analytic, cognitively- focused messages reduce the effectiveness of conspiracy theories (p. 184). Pennycook et al. (2015, p. 560) are among several to distinguish between ‘reflexive’ and ‘reflective’ open-mindedness - the former of which is an intuitive mindset that accepts information without much processing, while the

21 Ronan Eric Reilly latter seeks to critically analyze information. Reflexive thinking is likely to be linked with e.g. susceptibility to both informal logical fallacies and pseudo-profound bullshit – which in turn presumably correlate with conspiracy theory endorsement and -mentality. Dispositions and critical thinking skills will be revisited and reviewed further in chapter 2.3.

Measuring conspiracy theory endorsement and mentality Methodological approaches used to measure conspiracy belief usually consist of questionnaires that rank respondents on a scale and pair scores with e.g. psychological and sociological variables (Butter & Knight, 2016, p. 3). This thesis will follow in these footsteps and make use of two conspiracy theory related scales. To measure conspiracy mentality, Bruder et al. (2013)’s ‘Conspiracy Mentality Questionnaire’ will be adapted. It measures five items pertaining to conspiracy mentality irrespective of specific theories. Having examined its factorial structure, reliability, and validity through several studies the authors claim it to efficiently assess differences in general conspiracist ideation tendencies across countries irrespective of cultural differences, and they also found it to predict specific conspiracy theory belief better than other existing measures (p. 1). As this thesis wishes to separate such a ‘mentality’ from actual ‘endorsement’ an additional scale for the latter construct has been created, consisting of several specific theories. Several variants will be compounded, some including both true and completely concocted conspiracies. It is fruitful to additionally review some other proposed scales used for similar purposes, and to point out that no single universal measure has been agreed upon. One example of another scale is the ‘Belief in Conspiracy Theories Inventory’ (Swami et al., 2010, p. 753), which asks respondents to rate belief in specific conspiracy claims; much like the scale created for the current research. Moreover, the ‘Generic Conspiracist Beliefs Scale’ (Brotherton, French, & Pickering, 2013, p. 1) does the same, but uses more generalized conspiracy claims. Lantian et al. (2016, p. 1) developed a single item measure which asks for a rating on one elaborate multi-part question. Most surveys measure belief in well-known conspiracy theories, but some researchers include imaginary scenarios (e.g. Leman & Cinnirella, 2007, p. 7). The self-created scale in this thesis was partially inspired by this, as illustrated by the adding of a concocted item. Operationalizations and detailed explanations of the scales applicable to this research will be presented in chapter 3.5, along with all other scales used in this study. An overview of all scales can also be found in appendix B.

2.2 A:News media trust, source preference, and literacy The following section will treat literature pertaining to the first area of interest to this thesis – media trust, source preference, and literacy; hereunder RQ1 through RQ4.

22 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

News media trust, credibility, and skepticism (RQ1) Rooted in the numerous references throughout chapter 2.1 connecting conspiracy theories with e.g. suspicions and disbelief in official accounts and authorities, RQ1 suggests that it stands to reason that those who engage with such theories will be generally skeptical towards mainstream news.

Definitions and clarifications In order to investigate these relationships, it is necessary to specify clearly what is meant by ‘skepticism’; hereunder to understand the fluctuating nature of the term in relation to ‘trust’ and ‘credibility’, and how they correspond. According to (Tsfati, 2003, p. 67) social sciences utilize the ‘trust’ concept when describing lasting relations between ‘trustors’ and ‘trustees’; those who place trust, and those in which trust is placed. A trustor will expect to somehow gain by interacting with a trustee, but an assessment of their credibility may call this trust into question. Rotter (1967) defines ‘trust’ to be an expectancy “…that the word, promise, verbal or written statement of another individual can be relied upon” (p. 651). With this as a foundation, Tsfati (2003) defines ‘media skepticism’ as “a subjective feeling of alienation and mistrust toward the mainstream news media” (p. 67). This can be taken to mean that that they are perceived as being neither credible nor reliable, and e.g. that journalists do not live by their professional standards to be fair and objective, that they do not always tell the whole story, and that they would sacrifice accuracy and precision for personal and commercial gains (Tsfati & Cappella, 2003, p. 506). In sum, this thesis will treat media ‘skepticism’ synonymously with ‘mistrust’, and as a polar opposite to media ‘trust’. In line with Tsfati (2003, pp. 67-68), the concept will be applied to the way audiences perceive the function of mainstream media as a whole; e.g. generally feeling that one cannot trust news one gets from newspapers, TV etc. A differentiation can further be drawn between ‘media skepticism’ and e.g. source- or media ‘credibility’ as defined in many studies. The former goes beyond objective evaluations, as it also considers the subjective emotions of audiences and potential anger toward societal media functions etc. (Tsfati, 2003, p. 68). As belief in conspiracy theories is shown to often be based on such emotions too, a connection is highly plausible.

Public levels of trust In the United States, where a large portion of conspiracy theory research comes from, surveys have revealed a long-lasting lowering of trust in mainstream news. An already low 30% rate of Americans reporting a ‘great deal’ of confidence in them back in the early 1970’s had shrunk to a staggering 12% in 2000 (Cook, Gronke, & Rattliff, 2000, p. 29; Tsfati & Peri, 2006, p. 167). A more recent 2019-poll found that Americans remained largely mistrustful, as in the period between 2017 and 2019 only 41%-45% of respondents answered that they had a ‘fair amount’ or more of trust in newspapers, television and radio to report the news “fully, accurately and fairly” (Brenan,

23 Ronan Eric Reilly

2019). Scholars, media pundits, and journalists have expressed a very real concern that general news consumption could drop because of this confidence-crisis (Tsfati & Peri, 2006, pp. 166-167). This could in turn lead to a less knowledgeable public, and accordingly facilitate the spread of conspiracy theories and misinformation. It is striking that as skepticism toward mainstream news has increased over the decades, conspiracy theories have increasingly flourished. This thesis is therefore interested in correlations between media skepticism and such theories. As a counter to the US-trend, a poll conducted in Norway by Medietilsynet (2019) showed that eight out of ten people answered that Norwegian media are generally trustworthy. It also revealed that the nation’s primary broadcaster, NRK, had an 82% answer-rate for ‘fairly high’ amount of trust or more. Conversely, only 5% gave this rating to the common alternative news media website Document.no. These rather large contrasts to the USA are interesting to observe, and suggest much less of a need to address media skepticism issues in Norway. Individual connections between low media trust and high conspiracy endorsement and -mentality is however expected to remain static regardless of overall response rates in polls.

Measuring news media skepticism Most measures of news media trust in communication research have been based on the ‘media credibility’ concept (Kohring & Matthes, 2007, p. 232); not ‘media skepticism’. Still, some of these measures seem to have been interchangeably used to measure ‘skepticism’ with success (e.g. Tsfati & Peri, 2006, p. 175), and therefore this thesis will accept doing so also. Tsfati and Peri utilized a widely accredited measurement tool, the ‘news credibility scale’, which was developed by Gaziano and McGrath (1986), further adapted from a twelve- into a more efficient five-item scale by Meyer (1988) – and additionally cross-validated by M. D. West (1994). This scale demonstrated that various components of media trust – such as accuracy, fairness, unbiasedness, trustworthiness, and telling the whole story – load together very well (Tsfati & Cappella, 2003, pp. 506-507). Although more advanced multidimensional scales have been suggested to measure similar constructs – e.g. Kohring and Matthes (2007, p. 244), who proposed a scale based on selectivity of topics and facts, accuracy of depictions, and journalistic assessment – this thesis has chosen to stick with the Gaziano/Meyer scale due to its simplicity and largely proven functionality. Chapter 3.5, in addition to appendix B, will briefly elaborate on the operationalization of this scale.

‘Mainstream’ vs ‘alternative’ news media (RQ2 + RQ3) The distinction between ‘mainstream’ and ‘alternative’ media has been addressed on numerous occasions throughout this thesis. Separating between these terms is central to both the second and third research questions, which hypothesize that since both alternative media and conspiracy

24 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world theories often originate from non-mainstream ideas, both frequent consumption of such media (RQ2) – and a sympathetic assessment of them (RQ3) – will yield statistically significant relationships with such beliefs. Furthermore, it stands to reason that a mistrust in mainstream media will spawn an urge to seek more perceivably accurate information elsewhere. Unsurprisingly, when Tsfati and Cappella (2003) examined this relationship, they found that people who trust the mainstream media consume more mainstream news, and that media skeptics have less mainstream and more non-mainstream news exposure (p. 504). Therefore, it also makes sense that if conspiracy theory endorsement and -mentality are correlated with mainstream skepticism then a similar relationship should exist with regard to inclinations towards the alternative.

Academic elaborations and differences Although the dichotomous differentiation between ‘mainstream’ and ‘alternative’ is widely agreed to exist both in scholarly work and in public discourse, it is necessary to elaborate on what separates one from the other; and how this thesis chooses to view these terms. Firstly, for clarification, the widely used concept ‘mass media’ is sometimes erroneously used as a synonym to ‘mainstream media’, which can cause confusion. The ‘mass media’-term applies to a wide array of media technologies, and does not speak to how visible in society sources are – or the size of their audiences or budgets – but simply their availability (J. B. Thompson, 1995, pp. 26-28). As such, most ‘alternative media’ are regarded to be just as much a part of ‘mass media’ as the ‘mainstream’. According to e.g. Robinson (2019), ‘mainstream media’ refer to all large mass news media which collectively shape and reflect prevailing thought currents (p. 89). They are in many ways the ‘obvious’ sources to trust e.g. due to the widespread intersubjective agreement between them in most cases, and because of the likelihood that their visibility will inevitably lead to cross-validation of information by proper epistemic authorities if content is poorly researched or untrue. In Norway, news outlets with responsible editors can be held accountable by e.g. government appointed Medietilsynet; whose purpose is to promote democracy and freedom of speech (Medietilsynet, 2020), and PFU (Pressens Faglige Utvalg); an impartial commission self-instated by the press industry (PFU, 2020). Such outlets can face sanctions for misconduct and violations of ownership laws and press ethics etc. (Medieklagenemnda, 2020; PFU, 2020). While this is true in Norway, other traditions may apply in e.g. , Nigeria, or even USA, which can leave mainstream media more vulnerable and make skepticism towards them less irrational. As the research in this thesis is aimed at Norwegians it is important to keep this in mind, and one may even go as far as to assume that mainstream media skepticism in Norway is less justifiable and more irrational than in many other countries. Still, many previous non-Norwegian findings suggest that consumers of mainstream media are less likely to believe conspiracies (e.g. Stempel, Hargrove, & Stempel III, 2007, p. 353).

25 Ronan Eric Reilly

Some degree of mainstream skepticism should nevertheless not be considered irrational or unjustified, as many such sources often are news conglomerates who are politically motivated or possibly well-funded by powerful institutions. Ownership-issues and concerns regarding viewpoint homogenization are therefore highly rational. In fact, e.g. Craft et al. (2013, p. 2) found that higher levels of news media literacy – a concept that will be revisited in the next section – increased news media skepticism. This was however only to a slight degree, and with regard to all news media in a general way. The importance of a healthy amount of skepticism is not to be underestimated, but – more so than not – displaying high general mainstream media skepticism on a continuous basis is assumed to be largely a consequence of one or several of the presented emotions which also are associated with conspiracy mentality. ‘Alternative media’ can engage in any number of topics from e.g. those offering mildly differing or partisan interpretations of daily mainstream news – or even sporting events – to those who set their own agendas by e.g. focusing on extremist views or paranormal phenomenon. This thesis will treat them all very narrowly as an umbrella-term, as scope unfortunately disallows for more. Tsfati and Peri (2006, pp. 167-168) address the way media scholars differ in how they conceptualize differences between mainstream or alternative media. While some simply differentiate based on the size of the audience (Tsfati & Peri, 2006, p. 168; Turow, 1997), Downing (2001) claims them to represent different visions of hegemonic policies, priorities, and perspectives than those of mainstream sources, and that they widely rely on different production-, or distribution methods. Their aim is often to advocate for the interests of those excluded from the mainstream – such as the poor, or political and ethnic minorities – and to offer non-commercial views to those of government and corporate interest (Atton, 2002). In line with this, Fuchs (2010) also suggests that alternative media are “mass media that challenge the dominant capitalist forms of media production, media structures, content, distribution, and reception” (p. 178). Clearly, a wide array of outlets fall within these academic boundaries. Although there can be clear value in the existence of many alternative media outlets, their counter-hegemonic and innate anti-elitist obsession with opposition resonate well with the hallmarks of conspiracy theories, and the underlying psychological factors associated with them; making a connection highly plausible. In fact, many outlets even center outright around systemic or ‘superconspiracy’ material, as described in chapter 2.1.

Subjective preference as a research construct Media consumers’ opinions and assessments of what these terms constitute will likely differ based on their feelings towards media providers, interests they have in certain topics, and their world views. While acknowledging attempts at scholarly definitions, it is these subjective assessments that are of primary interest to RQ3 as a research construct. It is highly unlikely that respondents will be

26 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world up to speed on current academic definitions, but this thesis will have to assume that everyone can agree that there is a dichotomous division between ‘mainstream’ and ‘alternative’ perspectives without specifying the exact difference. Much like the term ‘conspiracy theory’, many may perceive ‘alternative media’ to be a term viewed derogatively by those who favor ‘mainstream’ views. Conversely, the ‘mainstream’ terminology can be widely used in an oppositional or dismissive manner among certain social groups; while a few will possibly not have a strong opinion – or perhaps hold a more pragmatic or neutral view. This thesis will propose that some media consumers may conceivably take pride in ‘choosing sides’ and seeking out alternative news sources; possibly due to a perceived sense of being more critical and inquisitive than the general public – and viewing both themselves and their preferred sources to be brave ‘truth-seekers’ who have the courage and drive to investigate what is ‘really going on’. It will further suggest that many will likely consider ‘alternative media’ to be more or less synonymous with low quality journalism and maliciously motivated creators and spreaders of biased views, fake news, and conspiracy theories. The span between these two extremes is naturally quite large, and it is presumed that viewing ‘alternative’ media favorably will be associated with the perceived ‘truth seeking’ displayed by those who endorse conspiracy theories. The same presumption is made for frequent consumption of such material. An attempt will be made to distinguish between respondents’ attitudes by inferring them from selections made to opinionated descriptions of alternative media in the distributed survey. The specific operationalization of variables to measure attitudes towards – and frequency of – alternative media will be elaborated upon in chapter 3.5.

News media literacy (RQ4) Both emotional and logical reasons may be at the root of choosing which sources to trust, but placing trust without understanding media mechanisms or interpreting news messages correctly can have unfortunate outcomes. Potter (2010b, pp. 680-681) emphasizes that media literacy can help people protect themselves from potentially harmful media influence, and that the broad aim of ‘news media literacy’ is to empower news consumers to seek useful and accurate information so that they may make informed decisions (Craft et al., 2017, p. 389). Another poll conducted by Medietilsynet (2019) showed that 43% of the Norwegian population had a high level of media literacy, while only 9% scored low; leaving approximately half of the respondents with average scores. The fourth research question bases itself on the assumption that both understanding and critical evaluation of media messages is key to reducing endorsement of conspiracy theories.

Definitions and elaborations Exactly what the ‘media literacy’ concept encompasses is widely debated (Potter, 2010b, p. 675).

27 Ronan Eric Reilly

For the purpose of this thesis it is relevant to first present some common definitions, and to make a distinction between the concept of ‘media literacy’ and it’s sub-concept of ‘news media literacy’ (Ashley, Maksl, & Craft, 2013, p. 8). In their dictionary of Media Literacy, Silverblatt and Eliceiri (1997) define media literacy as ‘‘a critical-thinking skill that enables audiences to decipher the information that they receive through the channels of mass communications and empowers them to develop independent judgments about media content’’ (p. 48). Hobbs and Frost (2003), elaborating on Aufderheide (1993, p. 6), offer a more compressed definition: “… the ability to access, analyze, evaluate and communicate messages in a variety of forms’’ (p. 334). Similarly, news media literacy refers to an ability to access, analyze, and interpret news messages (e.g. Craft et al., 2017, p. 392); i.e. it is the knowledge and motivations needed to identify and engage with journalism. It addresses how and why audiences interact with news media, how they make sense of what they consume, and how they are affected by their own news consumption (Maksl, Ashley, & Craft, 2015, p. 29). Potter (2010b, p. 680) postulates that the most frequently mentioned skill when addressing media literacy is critical thinking, but that this term seems to broadly and non-specifically refer to several mental processes in related scholarly work. This thesis will further investigate critical thinking in chapter 2.3, but in conjunction with media literacy research will be rooted in the work of (Potter, 2004, 2010a), who provides a theoretical scheme based on a set of skills pertaining to five knowledge structures; media effects, media content, media industries, real world, and the self. He does not see media literacy as an innate trait, but claims there to be a broad agreement that it is a multi-dimensional skillset that must be developed (Potter, 2010b, p. 681). In accordance with Potter’s views, a news media literate person interprets news messages based on their knowledge of the news media industry according to their personal needs and motivations. The fostering of critical attitudes toward news content, production and distribution is key (Craft et al., 2017, p. 393); as is the notion that a gap exists between reality and media message representations (Ashley et al., 2013, p. 7). The more complete the understanding an audience has of the conditions in which news is produced, the better it is equipped to access, evaluate, analyze, and communicate news media products (Craft et al., 2013, p. 4). As traditional news outlets shrink or get replaced by social media algorithms and other alternative arenas, the importance of literacy increases, and the likelihood of both accepting and spreading unsubstantiated conspiracy theories should decrease when literacy levels are high. Ashley et al. (2013) points out that news media literacy fosters increased news consumption (p. 8) – which in turn spawns civic engagement and democratic participation.

A model of news media literacy The five aforementioned knowledge structures theorized by (Potter, 2004, 2010a) provide a useful framework for further investigation. Potter (2004) culminated these structures into a model

28 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world emphasizing ‘conscious processing of information’, ‘preparation for exposure to messages’ and ‘knowledge about media systems’ (p. 68). Building on this model, Craft et al. (2013) adapted the essence specifically with news media literacy in mind, and Figure 2.2.1 – News Media Literacy Model (Craft, Maksl, & created the ‘News Media Literacy Model’ (p. 6) (see figure Ashley, 2013, p. 6) 2.2.1). The model encompasses Potter’s knowledge structures (media content, media industries, media effects, the real world and the self) which interact with a ‘personal locus’ (a combination of drives, needs and intellectual abilities) that govern information processing and construct meaning. The more robust the knowledge structures and personal locus are the higher the level of media literacy is expected to be (p. 6). Specific to news media literacy is knowledge about content, industries, and effects of news media; such as values that underlie news, how it is constructed, impacts of news media economics, ownership and control over news content, and consequences of news media exposure (p. 6). Potter (2004) points out the difference in understanding the distinction between the ‘real world’ and how news media can depict reality(p. 92). Unawareness of such distortions could increase acceptance of conspiracy messages. Knowledge of ‘the self’ pertains to awareness of own motivations for seeking news content. This self-awareness requires knowledge of own cognitive, emotional, and moral development, as well as both conscious and unconscious personal goals for obtaining information (Craft et al., 2013, p. 6). Higher levels of news media literacy will result from greater knowledge about news topics, the conditions under which news is produced, and the effects news can have on people; as well as extent of control and consciousness in relation to news consumption (Craft et al., 2013, p. 6). The news media literacy model aids in examination of the knowledge structure-personal locus interplay; e.g. a person who is driven to consume news but lacks knowledge about it will not be able to make much sense of what is consumed. Conversely, a demotivated reader with solid knowledge structures will also display low literacy (Craft et al., 2013, p. 6). In relation to the current study, it is plausible to stipulate that e.g. many consumers of alternative news sources who lack knowledge structures will be more likely to endorse conspiracy theories. This has previously also been pointed out and investigated, e.g. by Craft et al. (2017).

Conceptualizing and measuring news media literacy Measurement of media literacy has been attempted in several ways. Arke and Primack (2009, p. 55), in an attempt to create a measurement tool of their own, described several studies which focused on analysis of responses to various media messages. However, the first to use a subjective Likert-type scale – similar to what this thesis will use – was Primack et al. (2006), who developed a

29 Ronan Eric Reilly scale to assess media literacy specifically with regard to pro-smoking media messages. Concerning news media literacy as an isolated construct, two separate specific attempts have been identified upon reviewing the literature; both of which are interestingly developed by the same group of researchers. Craft et al. (2013) developed a scale based on the aforementioned ‘news media literacy model’ (figure 2.2.1), seeking to investigate the three components as separate constructs; mindful versus automatic thought-processing of news, perceived self-control versus news media being in control, and knowledge regarding news production and their effects. The first construct was measured with an adapted five-item ‘need for cognition’ scale from Epstein et al. (1996, p. 399) which assessed the degree of engagement in activities requiring deep and effortful thinking. They further adapted and converted a ‘health ’ scale by Wallston, Strudler Wallston, and DeVellis (1978, p. 164) to create a ‘media locus of control’ scale (Craft et al., 2013, pp. 26-27) that measured individual perception of control over media influences. Finally, they developed a measure for ‘news media knowledge structures’ (NMKS) concerning knowledge about the news media system, typical content frames in which news is produced, and possible effects news media exposure can cause (Craft et al., 2013, pp. 8-9). They found that their complete scale successfully distinguished high from low news media literacy in participating teenagers, and this was further validated in various studies involving other high school students, college students, and adults (Craft et al., 2017, p. 392). Research has demonstrated that all three measures appear to successfully measure news media literacy, however the ‘knowledge’– and ‘need for cognition’-aspects have proven to do so strongest and most consistently (Craft et al., 2017, p. 392). The other news media literacy scale by Ashley et al. (2013, p. 13) was successfully adapted from the aforementioned ‘smoking’-framework used by Primack et al. (2006, p. 371). It includes three domains; ‘authors and audiences’ (AA), ‘messages and meanings’ (MM), and ‘representation and reality’ (RR). AA addresses how authors target specific audiences; often motivated by the desire for profit. MM pertains to the fact that messages can be interpreted differently by different viewers, and that production techniques can be used to affect audience attitudes and behaviors. RR focuses on how media can affect perceptions of reality by filtering and/or omitting information (Ashley et al., 2013, p. 11). Initial development of this scale produced fifteen-items that were internally consistent and reliable across samples. Additionally, content validity was established through use of theory, focus groups and expert consultation (Ashley et al., 2013, p. 14). Vraga et al. (2015) questioned the Ashley et al. (2013)-scale, and suggested a more multi-dimensional approach by adding the dimensions of ‘self-perceived media literacy’ and perceptions of ‘media literacy value’ (p. 41). According to Vraga et al. (2015, p. 43) the former was meant to fulfil a similar measurement construct as the Craft et al. (2013) ‘media locus of control’ derived from (Potter,

30 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

2004). The latter was suggested to capture the value that people place on news media in relation to society, but this dimension is deemed irrelevant to the current study. Vraga et al. (2015) further suggested some minor changes in relation to specific items in the Ashley et al. (2013)-scale, and this thesis also proposes a few. For instance, it is felt that the RR-dimension fails to capture awareness of what is believed to be a very important mechanism within reality perception; the influence of agenda setting theory. This theory is centered around the notion that the press and the media do not reflect reality, but filter and shape it, and that frequent and prominent media focus leads the public to perceive certain issues as more important than others (Rogers & Dearing, 1988). Furthermore, the Vraga et al. (2015)-study suggests individually measuring the dimensions of AA, MM, and RR instead of joining them into a single news media literacy construct (p. 47). This thesis agrees with this view, and chapter 3.5 will elaborate on how this research will measure news media literacy as separate constructs rooted in the material reviewed here.

News media literacy and conspiracy theories: Previous connections Links have previously been found between media literacy training and perceptions of increased credibility, trust, and less hostile interpretations of news media content (Vraga et al., 2012, p. 942), as well as reduced perceptions of media bias (Vraga, Tully, & Rojas, 2009, p. 68). Measurements with the aforementioned scales have also revealed that high levels of literacy are positively correlated with knowledge of current events and intrinsic motivation for news consumption; but interestingly also news skepticism (e.g. Craft et al., 2013, p. 2), as previously pointed out. According to Craft et al. (2017, p. 392), such connections could illustrate how knowledge and motivations for news consumption can shape understanding of news, and that this also could influence how conspiracy theories are interpreted or endorsed. Subsequent investigations found support for this (p. 388), as greater measured knowledge about the news media actually did predict a lower likelihood of conspiracy theory endorsement. This thesis hopes to reproduce similar findings by showing that the more news media knowledge one has, and the more understanding of agendas, commercial contexts, and possible effects on public opinion, the less likely one is to believe in such theories (p. 396). Unexpectedly – even though they point to other studies with similar results – neither a high need for cognition nor a perceived media locus of control seemed to account for their findings (p. 396). One would have expected both the dimensions of willfully invoking cognitive efforts and the perception of control over media influence to play a vital role in countering conspiracy theories, and therefore this thesis will retest for these connections. The aforementioned Arke and Primack (2009) scale successfully measured parallels between media literacy and critical thinking – a link previously also drawn by Feuerstein (1999), who found that as pupils increased their experience with their media literacy program, they showed greater

31 Ronan Eric Reilly media analysis abilities and critical thinking skills (p. 43). Although only indirectly relevant to this thesis, studies connecting media literacy with critical thinking add to the overall interconnections between the many independent variables, and act as a natural segue into chapter 2.3 and part B.

2.3 B:Critical thinking dispositions and skills It has been extensively pointed out in chapters 2.1 and 2.2 that critical thinking is a key component not only of media literacy, but that plentiful research relates it directly to conspiracy theory endorsement and -mentality. With regard to the second interest area in this thesis, critical thinking dispositions and skills – and accordingly RQ5 through RQ8 – the focus will be shifted primarily towards how audiences understand and process the content of media messages.

Defining and explaining critical thinking The intellectual roots of critical thinking are as ancient as its etymology, and they are traceable to the ancient teaching practices of Socrates. He highlighted the fact that inadequate evidence, confused meanings and self-contradictions were often covered by largely empty rhetoric, and he pointed to a need for clarity and logical consistency by relying on evidence and sound rationale (Paul, Elder, & Bartell, 1997, p. 8). It has been a topic of interest ever since, and today it is considered a complex term that has been defined in many ways, and which consists of a multitude of constructs. One prevalent and commonly referred to definition identified in the literature is: “… the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.” (e.g. Clarke, 2019, p. 5; Scriven & Paul, 1987). Fasko (2003) summarized elements from several definitions, proposing critical thinking to be “the propensity and skills to engage in activity and ‘mental activity’ with reflective skepticism focused on deciding what to believe or do” (p. 8), and Ennis (2015) simplified most other definitions by stating that “critical thinking is reasonable reflective thinking focused on deciding what to believe or do” (p. 32). Most definitions draw an important distinction between ’reflexive’ (referred to as ‘system 1’ or ‘automatic’ in some literature) and ’reflective’ (‘system 2’/’mindful’) processing of information (e.g. Facione, 2011, p. 16; Pennycook et al., 2015, p. 560). Reflexive thinking is signified by a heavy reliance on e.g. heuristics, readily associated ideas, and vivid memories (Facione, 2011, p. 16). Audiences facing an abundancy of media information will not be able – nor will they have the motivation – to evaluate everything to their maximum cognitive ability. As reviewed in chapter 2.1, reflexive thinking was a trait strongly associated with endorsement of conspiracy theories. Conversely, reflective thinking is critical and requires more cognitive effort in order to process

32 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world abstract concepts, use reason, and analyze e.g. media messages; which generally leads to better assessments based on more fully developed evidence (p. 16). A healthy balance of both systems of thinking are vital for survival, but trusting media messages with conspiratorial claims without critically reflecting over their content will naturally much more likely lead to their endorsement.

Dispositions vs skills Another important distinction acknowledged by most definitions is that critical thinking has both a dispositional/propitious and cognitive/skillful dimension (e.g. Facione et al., 1995; Fasko, 2003; Ku, 2009; Sosu, 2013, p. 107). This thesis assumes both dimensions to be important for identifying false claims, such as conspiracy theories, when interpreting information and argumentation from media channels. Dispositions examine the “…tendency to do something, given certain conditions” (Ennis, 1996, p. 166), or how individuals approach a task (Ku & Ho, 2010, p. 54). More specifically they have been defined as a constellation of attitudes, intellectual virtues or habits of mind which describe the way one reasons, argues and makes decisions (e.g. Facione et al., 1995, p. 1; Sosu, 2013, p. 108). Cognitive skills, as described by (Facione, 2011) involve actual ability to interpret, analyze, evaluate, infer, explain, and self-regulate when facing concrete tasks (p. 5). The importance of the argumentation aspect has been highlighted e.g. by Watson and Glaser (1980) and Halpern (1998), and the analysis of arguments in relation to critical thinking is concerned with logically evaluating their correctness (Salmon, 2013, p. 12). While expecting that both dispositional and skillful dimensions are related to the subscription of conspiracies theories, this thesis will attempt to assess them as separate constructs. This is useful to do because if it turns out that relationships exist between conspiracy mentality or endorsement and dispositions, but not skills – or vice versa – the challenges journalism and media workers face can potentially be very different. After investigating dispositions (RQ5), three specific skills will be addressed consecutively; the ability to evaluate logical fallacies in arguments (RQ6); the ability to differentiate between profoundness and seemingly meaningful nonsense (RQ7), and the ability to be source critical. These are all seen as pivotal skills audiences need to analyze and assess media messages critically.

Dispositions towards critical thinking (RQ5) As conspiracy theories in many ways are the product of flawed reasoning and argumentation – and they are steered largely by attitudes – RQ5 suggests that the level of their endorsement will be related to individual levels of critical thinking disposition; as defined in the previous section. Especially, it is stands to reason that high conspiracy mentality – which also is a disposition – will correspond well with a generally low underlying tendency-level to think critically.

33 Ronan Eric Reilly

Concretizing dispositions To study possible relationships, it is necessary to review in more detail what the extents of such dispositions are as a specific construct. The first systematic empirical approach to categorization of thinking dispositions was known as the ‘Delphi Report’ (Facione, 1990), and proposed as many as nineteen broad dispositions which critical thinkers were expected to have (p. 13). Drawing on this research, Facione and Facione (1992) reduced these to seven. Several other theorists have also identified such characteristics, however (Perkins, Jay, & Tishman, 1993) were the first to combine all proposed dispositions into a comprehensive model and theory, stipulating that there are seven ‘master dispositions’ (p. 6). Many notions, such as open-mindedness, intellectual curiosity, and reflective thinking, are addressed in various dispositional taxonomies, but there is no clear agreement about the number there should be (Sosu, 2013, p. 108). This thesis choses to rely on Sosu (2013), who upon review of relevant literature suggested that critical thinking dispositions can fruitfully be captured through a combination of ‘reflective skepticism’; the tendency to learn from past experiences and to question evidence, and ‘critical openness’; the tendency to actively be open to new ideas, critical in evaluation of these ideas, and adjust thinking tendencies to match with convincing evidence (p. 115). These categories align with the aims of RQ5, and while conspiracy theorists hardly can be accused of a reluctancy to accept ‘new’ ideas, conspiracy thinking is shown to largely disregard evidence in lieu of emotion and faith.

Measuring critical thinking disposition There are several tests developed to measure various aspects of critical thinking. Although scholars widely agree that measures should capture dispositional components (e.g. Ennis, 2003; Halpern, 2003; Ku, 2009), Sosu (2013, p. 109) points out that most tests pertain largely to skills. In his endeavor to contribute to this shortcoming, he highlighted that the only existing instrument at the time that was developed to measure thinking disposition was the ‘California Critical Thinking Dispositional Inventory’ (Facione & Facione, 1992). Although a widely used measure, many issues regarding e.g. validity, reliability, and length have been pointed out (e.g. Sosu, 2013, p. 109; Walsh, Seldomridge, & Badros, 2007). Other non-specific scales have attempted to capture the construct, including various ‘need for cognition’ scales – which incidentally also will be utilized in this thesis as an aspect of ‘media literacy’. Sosu (2013) recognized a weakness in the of disposition- measures, and created the 11-item ‘critical thinking disposition scale’ based on his two-factor taxonomy of ‘reflective skepticism’ and ‘critical openness’ (p. 113). This thesis will make use of this scale as it serves as a highly useful framework for measurement, and it has been tested for validity, reliability, and stability of factor structure through both exploratory and multigroup confirmatory factor analyses (pp. 115-116). Chapter 3.5 and appendix B will offer elaborations.

34 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Skills: Detecting logically fallacious argumentation (RQ6) As presented, interpretation of argumentation is an important critical thinking skill. All media providers continuously use or convey arguments in order to convince and persuade, and those who have conspiracist agendas are in no way any exception. RQ6 postulates that there are clear grounds to believe that a connection between acceptance of poor argumentation and endorsement of conspiracy theories exists, as chapter 2.1 showed that they follow a very distinct set of flawed rhetorical traits. It is therefore highly conceivable that skillfully detecting fallacious argumentation could reduce the likelihood of subscribing to such theories. This section will explore rhetoric and arguments; specifically focusing on the ability to detect ‘informal logical fallacies’.

Modes of persuasion and arguments Aristotle provided the early teachings about rhetoric and introduced the common modes of persuasion; appeal to ethos, pathos, and logos. These are still inevitable components of the argumentation process (e.g. Mshvenieradze, 2013, p. 1939). Ethos relates to the credibility one attributes to the character of a source (p. 1940). As presented in chapter 2.1, conspiracy theorists may e.g. avail of scientific inquiry rhetoric to increase their credibility, or in other ways utilize the faith audiences instill in their media source of choice; as per chapter 2.2. The Pathos relates to the emotional aspects of convincement (p. 1940), and in turn can be seen in relation to much of the psychological research done on conspiracy belief – such as helping to reduce uncertainty through extreme explanations or providing an outlet for paranoid notions. As this section pertains to critical thinking it is primarily concerned with the Logos; dealing with logic and reason, analytical skills, and the argumentation itself (p. 1939). It is not always clear upon interpreting information whether actual arguments are even being made, and reasons or conclusions can often simply be implied. In chapter 2.1, one of the presented persuasion strategies deployed by conspiracy theorists was e.g. the rhetoric of ‘just asking questions’; simply leading an audience to infer the presence of devious intentions or events. Unlike subjective claims, beliefs, or opinions – e.g. “conspiracy theories often turn out to be true” or “Donald Trump is an idiot” – arguments are made up of premises (reasons or supporting claims) and a conclusion (Bennett, 2018, p. 2). To appeal to audiences, media messages will often substitute structured arguments with colorfully framed representations which often can mislead or camouflage information. A distinction must be made between formal and informal logical arguments. A formal logical argument is based on deductive reasoning and relates to its logical form, so if the premises are true it follows logically that the conclusion must also be true (p. 3). Informal arguments are based on inductive reasoning (probability rather than necessity), and general conclusions are induced from specific instances (p. 3); which can lead to a multitude of fallacious conclusions that both purposely and accidentally can influence audiences.

35 Ronan Eric Reilly

Logical fallacies The term ‘logical fallacy’ is often abbreviated to pertain simply to errors in reasoning. They have however been defined more specifically, e.g. by Jevons (1872) as “the modes in which, by neglecting the rules of logic, we often fall into erroneous reasoning” (p. 169), or more recently by Bennett (2018) as “a concept within argumentation that commonly leads to an error in reasoning due to the deceptive nature of its presentation” (p. 9). Media outlets can often resort to ‘fallacious tactics’, where fallacies are purposely deployed as a rhetorical trick to provoke erroneous reasoning (p. 7). Effective persuasion techniques are commonly attempted by media providers with an agenda – such as e.g. pushing conspiracy material, but fallacies can be committed both by arguers and audiences (p. 7). Bennett (2018, p. 9) flags three criteria for something to be considered a logical fallacy; that there must be an error in reasoning – not a factual error; that the possibility to commonly apply it to an argument either in the form or interpretation is present; and that it must be deceptive, and often capable of fooling the average adult. This thesis is mainly concerned with informal fallacies, which pertain to inductive or deductive defects found in the content of an argument, but just as there are informal and formal arguments, there are also formal fallacies based on errors in logical form. All fallacies do however require the presence of an actual argument.

Practical examples To measure respondents’ susceptibility towards such fallacious argumentation, this thesis will fashion a Likert-scale consisting of several self-authored arguments based on various fallacies. Chapter 3.5, accompanied by appendix B, will provide a detailed description of how this scale will be created, but it is fruitful to review some common informal logical fallacies; as they provide a foundation for understanding the inspiration behind the scale items. Although several common and named fallacies are suggested – with specific descriptions and characteristics – they are often identified and categorized by non-academic resources such as websites etc. Accordingly, there is not an agreed upon finite number of ‘accepted’ fallacies or official classification taxonomies. One reason for this is their ambiguous nature, as many of them can fit within several different potential categories at once (Bennett, 2018, p. 10). Examples of some commonly postulated informal fallacies are the argumentum ad hominem; whereby one attacks the person making the argument rather than the argument itself (Bennett, 2018, p. 20), the strawman fallacy; where one substitutes the oppositional argument with a distorted or exaggerated version of the arguments position (p. 299), the fallacy of anecdotal evidence; which often ‘cherry picks’ evidence to suit a claim by pointing to individual cases that confirm one position while ignoring contradictory cases (pp. 106,128). Hasty generalizations – making claims based on general or insufficient observations – are also common and can overlap

36 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world with many other fallacies. Further examples include slippery slope argumentation; where a relatively insignificant event has an ultimate outcome based on a highly improbable sequence of subsequent events (Bennett, 2018, p. 289), the false dichotomy fallacy; where a choice between two options is presented as the only possibility when endless other options exist (p. 173), and the burden of proof fallacy; which posits that something must be true if the opponent cannot disprove it (p. 286). Faulty comparisons are also common, where one compares unrelated matters to make something look e.g. more desirable or more justified than it really is (p. 179). There is ironically even a fallacy fallacy (argumentum ad logicam), which dismisses an entire line of arguments based on the presence of a possibly insignificant fallacy (p. 103). Conclusions often do not follow from premises, and non-sequitur fallacies can occur either formally; due to the argument-structure (as explained earlier), or informally, by presenting irrelevant reasons for a conclusion (p. 245). Conclusions are also often attributed to reasonings that are not necessarily relevant through the use of various appeals; be they e.g. to false authorities (such as non-expert claims), nature, common belief (ad populum), popularity, emotion, or age etc. (pp. 64,73,48,79,60,102). Just because something e.g. has been practiced or claimed for years, is made from ‘natural’ ingredients, or gets worn often by many people, it does not necessarily make it better or more correct than something else. Countless other fallacies could just as well have been exemplified in this section, but descriptions have been limited primarily to those which will be used in the measurement scale.

Logical fallacies vs heuristics and other cognitive biases While heuristics and other cognitive biases are largely unconscious processes that bypass reason – e.g. being fooled by availability or representativeness – logical fallacies are tied solely to the evaluation of arguments (Bennett, 2018, p. 6). A link between cognitive biases and the endorsement of conspiracy theories has previously been pointed out, and this thesis expects logical fallacies to also have such a connection. This is because they are both potentially a result of reflexive thinking, and because they are interlinked e.g.in the sense that it is possible for logical fallacies to result from cognitive biases (p. 7). The ‘bandwagon effect’ is an example of a that causes people to adapt popular behavior. This can in turn lead to the aforementioned ad populum fallacy where a conclusion is reached that something must be good based on the premise that most people would do it. Bennett (2018, p. 6) argues that although crossovers can occur, cognitive biases and heuristics do not necessarily imply logical fallacies or cause audiences to fall for them. Invoking conscious evaluation of an argument can in fact often help to counteract a cognitive bias. Evaluating arguments and spotting conspiracy claims in media messages is naturally more complicated than just identifying some logical fallacies in statements out of context. RQ6 will however isolate focus on susceptibility towards informal logical fallacies in relation to conspiracy

37 Ronan Eric Reilly theory endorsement and mentality without considering or controlling for other factors. No previous specific scale or measure for such a construct has been identified in the literature, and as mentioned, chapter 3.5 (and appendix B) will elaborate upon how this thesis has solved this.

Skills: Detecting ‘pseudo-profound ’bullshit’ (RQ7) What popularly may be referred to as ‘bullshit’ is probably more pervasive in the media now than ever before. A great deal of content – such as inspirational quotes etc. – can present eloquently without really making any sense. This thesis sees it as a critical thinking skill to be able to identify such attempts. RQ7 hypothesizes that a belief in conspiracy theories can originate from a tendency to be accepting of manipulative nonsense; hence an inability to detect such attempts will likely be correlated with their endorsement.

‘Bullshit’ – not simply nonsense If one were to go by its dictionary definition, “bullshit” would quite simply be synonymous with “nonsense” (Bullshit, 2020). Pioneering academic research of ‘bullshit’, Pennycook et al. (2015) found this comparison to not really capture the essence of the term, and they argued that it is rather constituted by “something that implies but does not contain adequate meaning or truth” (p. 549). They made this contrast to mere ‘nonsense’ clear by comparing a syntactically correct collection of random buzzwords attempting to convey profound meaning; e.g. “Hidden meaning transforms unparalleled abstract beauty”, with actual nonsense in the form of non-syntactical random words; e.g. “Unparalleled transforms meaning beauty hidden abstract” (p. 549). Quotes used in their research were inspired by or borrowed from various real sources; e.g. alternative medicine ‘guru’ Deepak Chopra – who seems to portray ‘meaning’ to audiences, but upon closer review apparently only attempts to convey a sense of profundity at the expense of a clear meaning or truth (p. 550).

Pseudo-profound bullshit Being well aware that ‘bullshit’ can exist in many forms or be defined in many ways, Pennycook et al. (2015) chose to focus specifically on what they labelled ‘pseudo-profound bullshit’ (p. 550). They did recognize, however, that such ‘bullshit’ clearly contrasts with e.g. ‘an exaggerated story told over drinks’ (p. 550). This thesis would have preferred to address a form of ‘bullshit’ that was more practically comparable to conspiracy rhetoric, but due to a lack of more befitting existing research constructs and measurement proposals it will – like Pennycook and his colleagues – focus on ‘pseudo-profound bullshit’. Although not quite the same, many aspects of conspiracy claims often share a lot of similarities with meaningless impressive assertions. Treated literature also has revealed that it is common to see interconnections between audiences who believe in conspiracy theories and those who pursue alternative media genres of a paranormal, pseudoscientific or

38 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world spiritual nature – such as one may categorize said Deepak Chopra and similar ‘wisdom-peddlers’ to be. If one makes the assumption that certain audiences have a common inclination towards an aggregation of such media sources, then pseudo-profound bullshit quotes should resonate well with them; which in turn may suggest that they also have an affiliation with conspiracy material. In researching bullshit, Pennycook et al. (2015) found that some people are indeed more receptive to it, and suggested that those who accept bullshit content may simply fail to grasp that interpretation requires special consideration. An ambiguous audience might assume that a profound and deep meaning is masked by vagueness in a media message, and that a meaning surely must be implied (pp. 550-551). It is possible to construe that someone who would assume meaning in a vague claim would also potentially disregard missing information and more easily entertain a conspiracy theory. In relation to critical thinking, and as addressed on numerous occasions, reflexive processing will often supersede reflective analysis. Pennycook et al. (2015) made the assumption that those who are more analytic should be more likely to detect the need for deeper analysis when exposed to pseudo-profound bullshit (p. 551), and found that an excessively ‘open’ mind inflated judgements regardless of the content; causing a general bias toward accepting bullshit statements as true (p. 551). This relates to previously addressed discussions regarding the role of e.g. cognitive biases and seeing patterns in random stimuli. There are however many factors which are likely to lead someone to successfully detect the need for reflective skepticism, depending on the type of bullshit encountered and the context in which it is presented (p. 551).

Pseudo-profound bullshit and relationships to conspiracy- and other beliefs Drawing on their own assumptions, Pennycook et al. (2015) tested how receptivity towards ‘pseudo-profound bullshit’-claims related to a collection of variables. Findings included that those more receptive to bullshit were less reflective and lower in cognitive abilities such as verbal and fluid intelligence and numeracy (p. 559). Also, as previously addressed in chapter 2.1, they were more prone to ontological confusions and epistemically suspect claims such as religious and paranormal beliefs, and were more likely to endorse complementary and alternative medicine; as well as – most importantly in relation to this study – conspiratorial claims. Studies by Hart and Graether (2018) revealed similar findings utilizing similar measurement tools; as both independently and together with e.g. dangerous-world beliefs, bullshit receptivity predicted conspiracy beliefs (p. 229). This relationship will expectedly also resurface upon analysis of RQ7. Building on this research, a recent study by Pennycook and Rand (2019) found that pseudo‐ profound bullshit receptivity correlated positively with perceptions of fake news accuracy, and negatively with media truth discernment; i.e. the ability to separate fake news from real news (p. 1). Even though fake news is not synonymous with conspiracy theories per se, these findings are still

39 Ronan Eric Reilly interesting and relevant – especially from a media studies perspective. They interpreted their overall findings to suggest that belief in fake news to some extent may be driven by a general tendency to be overly acceptant of weak claims (Pennycook & Rand, 2019, p. 1). Yet again, reflexive open- mindedness is attributed partial responsibility for epistemically suspect beliefs (p. 1); which conspiracy theories also are.

Measuring pseudo-profound bullshit and profoundness The only means of measuring pseudo-profound bullshit that can be identified in the literature are rooted in the described work by Pennycook et al. (2015). One of their scales was designed to measure ‘bullshit receptivity’; i.e. the tendency to accept bullshit claims as true profoundness. This measure was achieved through presenting participants with ten bullshit statements consisting of randomly organized buzzwords (as previously described), and asking them to rate their perceived level of profoundness on a 1-5 Likert scale (p. 552). While it is possible that ‘bullshit-receptivity’ could both reflect a general tendency to perceive any sentence as meaningful or that this tendency could be specific to bullshit sentences, they further devised another scale by adding a selection of actual profound motivational statements. A measure for ‘bullshit sensitivity’ was then created based on the difference between profundity ratings between truly meaningful quotes and bullshit quotes (p. 556); thus measuring respondents’ ability to correctly assess and decipher between bullshit and true profoundness. Both ‘receptivity’– and ‘sensitivity’-scales were used to measure the previously described findings with regard to e.g. conspiracy belief, but the former yielded more accurate and consistent results (p. 559). In later investigations into behavior predictions, Erlandsson et al. (2018) criticized the ‘bullshit-sensitivity’ scale for its inherent scaling-problems when measuring difference-scores, and suggested it to be more fruitful to perform separate measures and infer bullshit-sensitivity indirectly (p. 2). They therefore introduced a third scale measuring ‘profoundness-receptivity’; based simply on perceived meaningfulness of genuinely profound sentences independently from bullshit. This scale has never been tested in relation to conspiracy mentality or endorsement. This thesis is primarily interested in the relationship between conspiracy belief and ‘bullshit receptivity’ and does not see much value in measuring ‘bullshit sensitivity’. In addition to the problems described by Erlandsson et al., it is assessed that such a scale would not fully capture the distinction between understanding what is profound and what is not. It would be strange if there was a pattern to actively dismissing similarly phrased profound statements, as – regardless of whether they are bullshit or not – they all should sound alike if reflexively interpreted; deeming it unclear what to make of any possible findings. As a secondary investigation, however, it is plausible that an ability to detect the presence of underlying profoundness in statements could have

40 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world an opposite relationship to conspiracy belief; all the while this may indicate actively engaging in cognitive effort. Among other reasons, this study will therefore also include a measure for ‘profoundness-receptivity’. The research in this thesis in relation to pseudo-profound bullshit will be largely inspired by the presented work, and a further explanation of applicable bullshit- and profoundness-receptivity measures will be found in chapter 3.5 as well as in appendix B.

Skills: Information evaluation / source criticism ability (RQ8) The eighth and final research question addresses the likelihood of avoiding acceptance of unsubstantiated conspiracy theories by skillfully maneuvering through credible source material, and evaluating information and the makeup of content. This thesis sees it as a vital critical thinking skill to exert the necessary effort and understanding needed to avoid malicious sources and poor information; a process often referred to as ‘source criticism’. Essentially, the ability to identify what to be aware of could pertain to any source – be it a research paper, a document, a person, a photo, an observation, or anything else used to obtain knowledge. Within the context of this thesis it is fitting to ascertain that news articles and various media content are the ‘sources’ in question.

Currency, relevance, authority, accuracy, and purpose In an attempt to find a way to easily make students remember important aspects of source criticism, the librarian and teacher Sarah Blakeslee (2004) fashioned a source criticism ‘guide’ where the main aspects very fittingly culminated into the acronym CRAAP – encompassing currency, relevance, authority, accuracy, and purpose (p. 6). For the complete list, including its sub-items, please refer to appendix C. Originally this ‘checklist’, as it effectively has become, was intended to be an aid in assessing work across academic disciplines; and e.g. several universities have incorporated it as a part of their education (Berg, 2017). However, since the emergence of the CRAAP test its usefulness has extended to many areas, and several institutions have recommended it as an effective tool for evaluating news sources (e.g. WKU, 2020). The CRAAP test suggests the following dimensions of investigation (Blakeslee, 2004, p. 6): Currency pertains to the timeliness of the information and calls for checking when the information was published or written, and if potential updates may have been done. In addition, if the source is web-based, it is encouraged that one validates and examines the functionality of links. Relevance addresses the importance of the information within the context of what is in question and implores that one assesses e.g. the authors’ intended audience. Authority applies to research regarding the source of the information, and calls into question the authors qualifications, credentials, and affiliations, as well as presence of contact information. Accuracy questions the reliability, truthfulness, and correctness of the content, and posits

41 Ronan Eric Reilly that one should identify supporting evidence and whether other sources can verify this. It further calls for analyzing the use of emotional and possibly biased language, and identifying possible spelling- and/or grammatical errors etc. Finally, the purpose speaks to consciously evaluating why the information exists. Does it intend to teach, sell, entertain, or persuade, and is this intention blatantly evident? Furthermore, does it convey objective and impartial information, or is it possibly attempting to push political, ideological, cultural, religious, or institutional opinions or propaganda? Utilizing these aspects in the process of critically evaluating news sources should provide a decent ‘workflow’ which could expectedly reduce the risk of falling for fake news or buying into unsubstantiated conspiracy theories.

Measuring source criticism ability Assuming one possesses the disposition and ability to think critically, it is likely that one would exert skills pertaining to source criticism when presented with any news story. When faced with an article, though, it is one thing to be aware of own capabilities and know what one should do – it is however a whole other story to both motivate oneself and remember to actually do it. This thesis finds it fruitful to attempt to isolate the skill of practically applying source criticism as a specific measurable scale-construct. As searching the literature has not resulted in the successful acquisition of any Likert-scale for the purpose doing so, one will be created by conceptualizing that source criticism ability is constituted by drawing on the checkpoints from the CRAAP-test and skillfully applying them. The measurement of this variable is a little bit different from previously attempted skill-measures, as it aims to query respondents directly regarding what they specifically believe their own source criticism abilities to be. Although the CRAAP test is in no way is the only method developed for aiding effective information evaluation, it is assessed to be a workable foundation upon which to build a measurement scale. As usual, chapter 3.5 – in conjunction with appendix B – will provide an explanation for the specifics of how this scale will be fashioned.

42 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

3 Methodology

3.1 Choice of method Early in the planning process it became clear that the primary aim with this research would be to identify general tendencies and relationships between personality traits, beliefs, thoughts, opinions, and actions. Considering the fact that Fink (2013) describes surveys as “information collection methods used to describe, compare or explain individual and societal knowledge, feelings, values, preferences and behavior” (p. 2), and that most previous research exploring similar variables has been conducted using surveys, it seemed obvious to pursue this methodological path. A decision to forego other methods, such as in-depth interviews or focus groups, meant sacrificing the possibility of gathering detailed qualitative data, but the quantitative nature of this study dictated that it would be most suitable to create a multi-variable questionnaire. A cross-sectional design based primarily on Likert-scale questions was fashioned, which also collected some nominal and ordinal data pertaining to demography and media habits and views. Questions were collected as closed responses for the purpose of simplifying data processing. As a self-administered questionnaire it was primarily distributed online via several social media channels, debate forums and commentary sections. For pragmatic reasons non-probability sampling methods where used, but an attempt was made to distribute it in as large a scale as possible when taking the available resources into consideration. Using a survey for this research was advantageous because it facilitated the obtainment of many answers from a variety of sources relatively easily, and the testing of a wide number of constructs on many people in a limited amount of time (Wright, 2017). Surveys are also financially lucrative, and generate data that can be easily analyzed if well-planned (Wright, 2017). They are however prone to various biases which threaten the validity of results (Assael & Keon, 1982). The following chapter will provide in-depth explanations and considerations regarding the methodological process of this study, the creation of the survey and operationalization of specific variable measures, the means to which distribution and gathering transpired, and finally how the data was prepared for analysis. The full version of the survey as distributed in its entirety is available in appendix A; in Norwegian, as it was administered in the local language of the sample respondents. An English run-through of all Likert-scale items is included in appendix B, along with an overview of their operationalization and origin.

3.2 Ethical considerations and practicalities According to the Norwegian National Research Ethics Committee, research ethics are “a codification of scientific morality in practice”. They refer to the many values, norms, and

43 Ronan Eric Reilly institutional arrangements that help constitute and regulate scientific activities (NESH, 2016, p. 5), and such concerns were as much a primary consideration in the execution of this study as they would be in any other research. The Personal Data Act pertains to protecting personal integrity and privacy (p. 7), and states that there is a general obligation to provide participants with adequate information about the field of research, the purpose of the research, who has funded the project, who will receive access to the information, the intended use of the results, and the consequences of participation (p. 13). These obligations are highly important in order for participants to be able to give informed (p. 15). A frontpage disclaimer, available in its entirety (in Norwegian) in appendix A, was authored to provide participants with information deemed necessary to do so. It clearly explained the purpose and aim of the research, central themes, estimated completion time, and informed about the promise to treat all responses anonymously. It was also open regarding its affiliation with the University of Oslo, and provided the name and contact information of the responsible researcher; allowing for the possibility to have any potential concerns answered. As this research consisted of anonymous surveying, the necessary level of caution did not need to be as rigorous as if one were e.g. to conduct personal in-depth interviews with named subjects. Any potential abuse of information, irresponsible behavior, or possible harm one could end up inflicting upon subjects was by default kept to a minimum as the origin of the collected material would be indeterminable. If the research had planned for gathering personal data a rigorous process of national registration would have been triggered, and secure data-handling procedures would have needed to be implemented. Although the plan always was to keep the survey completely anonymous – and accordingly avoid processing any personal data – a natural first step was still to contact NSD (The Norwegian Centre for Research Data), and precautionarily verify that registration would be unnecessary. The only resulting requirement was that the chosen software solution for data collection would have the option to not gather respondents’ IP-addresses; as this would technically be considered personal data. An extensive research procedure was carried out to determine which software/service provider would be most suitable, and a series of different solutions, including Nettskjema, SurveyMonkey and EasyQuest were considered. Google Forms was ultimately chosen due to a combination of customizability, ease of use, cost-free accessibility, and – most importantly – their claim to meet the ethical IP- anonymity demands set by NSD. Regardless of making the assessment that the collected data was not going to be sensitive in nature or be individually identifiable, a choice was still made to store the gathered information responsibly in accordance with the Personal Data Act (NESH, 2016, p. 18). Therefore, all respondent data was stored only locally on a personal password protected computer. Ethical concerns related to anonymous data handling during the data collection and analysis phases – including issues that

44 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world arose while conducting the study – will be revisited in chapter 3.6.

3.3 Sampling, validity, and reliability Sampling The target aim of this study would ideally have been to collect a large-scale probability sample suited to generalize tendencies to the entire Norwegian population. As this is merely a master’s thesis – with the entailed limitations of resources, time, and opportunity – such endeavors were deemed impossible. A prerequisite for generalizing sample data to even a small population would have been to assess the variability among its members, specify a precision level, and use data gathering techniques suitable for probability sampling with randomly selected respondents (Foster, Jefferies, & Diamond, 2015, p. 129). The pragmatic and realistic alternative was to obtain as many respondents as possible using non-probability methods; primarily utilizing convenience sampling, but snowball- and purposive sampling were also partially attempted. According to Foster et al. (2015, p. 128) sampling by convenience means just that; availability being the only factor dictating who responds. Snowball sampling is a variant of this approach, where respondents are encouraged to share the questionnaire with acquaintances. Purposive sampling was also attempted by seeking out a portion of respondents from various ‘alternative’ channels focusing on extreme political views or conspiracy theories. Chapter 3.6 will elaborate on how and where distribution occurred. Although data obtained using these methods will never technically allow findings to be analyzed in a generalizable manner, humble attempts were made at reaching many audiences and gathering a sample consisting of as diverse a group of respondents as possible.

General assessment of possible errors, reliability, and validity As this is a non-probability sample there is no way to know the degree to which results could differ from any imaginable population. Sampling errors are possible issues of validity that may occur in relation to choosing the right sample-collection strategy (Foster et al., 2015, pp. 125-126). Estimating sampling variability or identifying bias in the form of e.g. possible sampling errors is however only applicable when considering probability samples. Reliability is the ability of a measure to produce the same results under the same conditions (Field, 2009, p. 12), but as the same response conditions can never be recreated in a non-probability sample any attempt at evaluating this is rendered mute. The fact that answers also were gathered anonymously makes it impossible to even hypothetically consider gathering the same group of respondents to retake the survey, although in theory one could compare some of the survey results with information from other populations. It is however also possible to view reliability in terms of claiming two respondents to be equal in terms of a measured construct, and that they therefore

45 Ronan Eric Reilly should score similarly. In practice, split-half reliability can be used to do this (Field, 2009, p. 674). With a basis in Likert-scale data, an important maneuver to perform is therefore the assessment of Cronbach’s alpha (α), which utilizes this to measure reliability – or internal consistency – of items. Therefore, each Likert scale in the study is assessed by its Cronbach’s alpha value in chapter 4.1. Whereas sampling errors are of no consequence to this study, non-sampling errors – essentially all other possible errors than those associated with sampling (Lavrakas, 2008, p. 539) – are highly relevant. These are errors which are linked to e.g. poor execution or vague questions, bad responses, or defective methods of data collection or processing etc. (Foster et al., 2015, pp. 125- 126; Lavrakas, 2008, p. 539), and they are all threats to validity; the degree to which an instrument measures what it was designed to measure (Field, 2009, p. 11). In relation to this study there are a number of errors that are highly likely to pose such a threat – especially with regard to Likert scale responses. For instance, it is common for respondents to overreport desirable behavior and/or underreport undesirable behavior so that others will view them favorably; a phenomenon known as social desirability bias (Krumpal, 2013, p. 2028). In the context of this study this tendency may become assertive, as many of the questions pertain to skillsets with potentially obvious desirable options; e.g. with regard to not affirming conspiracy theory endorsement, or overestimating the degree to which one is source critical. In addition, acquiescence bias can often occur. Particularly when in doubt about an answer, it is suggested that respondents have a tendency to agree with all the questions or to indicate a positive connotation (Kuru & Pasek, 2016, p. 82). Furthermore, central tendency bias can occur as a result of unwillingness to display an opinion when in doubt (Douven, 2018, p. 1203). Moreover, respondents do not always understand or have the motivation to read or think through questions. Careless responses can therefore easily occur, where answers are given randomly or inconsistently (Weijters, Baumgartner, & Schillewaert, 2013, p. 321). This is a major consideration due to the relatively extensive length of the questionnaire. It is also essential to evaluate all the possible validity issues that may have arisen due to poor or ambiguous phrasing of the questions, potentially forgetting to include all possible answer options, or respondents simply misunderstanding or interpretating questions differently. More in-depth assessments of validity and reliability will also be carried out as a natural part of discussions which will follow from performed analyses in sections 4 and 5.

3.4 Design, structure, and measurement Basic design As no data captured beyond a single point in time was needed, it was clear that the applicable design of the survey should be cross-sectional (Fink, 2013, p. 101). To ease the analysis, it was also considered unnecessary to include anything other than closed questions with fixed responses. All

46 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world questions were therefore configured to require an answer, although when asking for conceivably sensitive demographic information a ‘do not wish to answer’-option was also made available. Although the questionnaire did not allow for any questions to be skipped, Likert-scales and questions about media habits were not set up to force non-neutral choices. This would likely have facilitated for multiple careless responses by undecided respondents, or if questions were misunderstood, and thereby could threaten validity and/or scale reliability. More importantly, ‘no opinion’ or ’don’t know’ answers were also widely considered not to be ‘missing data’, but rather to be valuable and necessary response-options in most cases.

Pilot study As recommended, and by some deemed an absolute necessity (e.g. Fink, 2013, p. 7), a pilot study (N=9) was conducted as a part of the design process. The objective was to determine several factors, such as average duration for completion, understandability of questions, and followability. General feedback on anything from typographical errors to overcomplicated phrasings and clarity of language in general was also appreciated. Feedback further aided in assessing what items or scales possibly would require modification, and importing test-data into SPSS helped understand how analyses may be performed in practice. Differences between the original design and the specific changes made based on the pilot study will be pointed out throughout this section where applicable.

Likert scales Most variables in this study were measured using psychometric Likert scales. Named after their inventor, psychologist Rensis Likert (1932), they have become widely used as a means of scaling responses in survey research. In response to a Likert item, respondents indicate their level of agreement or disagreement on a symmetric scale, attempting to capture the intensity of their feelings (Joshi et al., 2015, p. 397). A Likert-item is often formulated as a statement that the respondent is asked to evaluate quantitatively, and levels of agreement/disagreement or preference/non-preference are commonly used as response-dimensions; e.g. a possible range from 1 (strongly disagree) to 5 (strongly agree), with a neutral ‘option 3’, is provided (Joshi et al., 2015, p. 397). Although sometimes one can choose to analyze Likert items individually, a true Likert scale is made up of the sum of responses to several such items (Joshi et al., 2015, p. 398). This thesis operates with such summative scales for groups of items, and it achieved this by adding together all item scores in any given scale and dividing the sum by the total number of items to compute an arithmetic mean average score. It is important to note that there are many divergent opinions on several issues regarding how to treat the resulting Likert data when performing analyses, and some of these will be addressed in chapter 3.7.

47 Ronan Eric Reilly

Limitations to self-constructed scale development Several Likert scale measurement tools were adapted for use in this study. Some have been modified from their original versions, some were left unchanged, and some were developed from scratch. Any scale being developed should ideally be put through a rigorous process of repeated pilot testing and various factor- or component analyses in order to deem it adequate as a reliable measurement tool for specific constructs. Such endeavors were outside of the feasible scope of this thesis, and a decision was made not to engage in this level of complexity for pragmatic reasons. Appropriate variations of these procedures have already been carried out for the adapted scales, but repercussions limiting the study are likely to have occurred due to e.g. replacing, rephrasing, or tweaking some items. It is important not to dismiss this when interpreting results.

Language translation The survey was distributed only in Norway, and to Norwegian speaking respondents. Original English scales were all translated into Norwegian, as the survey in its entirety was administered in the country’s native language. It is relevant to point out that translation-related issues may have occurred due this. Fink (2013, p. 75) generally recommends that translations undergo a comprehensive procedure involving several fluent speakers and multiple backtranslations and revisions. This is another process which circumstance did not allow to transpire. Hopefully, the essence and nuances were captured to an acceptable degree, as the author of this thesis – who is a fluent native speaker of both English and Norwegian – has undertaken the translation. Items were revisited on several occasions and pilot-respondents were queried about possible confusions.

Length and completion time The initial version of the survey was much more comprehensive than the final one. Several scales initially contained more items, and additional scales and variables were involved – such as separate informal and formal ‘logical fallacies’-scales, a ‘susceptibility to persuasion’-scale, and two extra dimensions of the NML-scale, etc. This version took up to 35 minutes to complete. Common sense led to a suspicion that this would be way too long, and the pilot study revealed that when the survey exceeded 15-20 minutes in duration motivation took a serious hit due to complexity and annoyance. As a result, many heavy modifications and tweaks were implemented. Several scales were condensed into fewer items, and the aforementioned scales were dropped completely; resulting in an estimated completion time of approximately 15 minutes.

Reverse coding and rephrasing Another lesson learned from the pilot study was that the answers to some of the questions were simply too ‘obvious’; which was an already held suspicion. To remedy this, a rephrasing of some items was performed. Many of the original adapted scales also contained several items which were

48 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world all coded in the same direction. A choice was made to randomly reverse code some of these in an attempt to minimize effects of the previously mentioned social desirability- and acquiescence biases, as well as reduce careless responses. This was done in spite of some research suggesting that even though reverse-coding items is methodologically sound it can often hinder model fit (e.g. Weijters et al., 2013, p. 320).

Concealing the aim In addition to not wanting questions to call for obvious answers, it was also not desirable to have it be overly apparent to respondents what the study was attempting to measure. Therefore, it was an aim to downplay the fact that conspiracy theory endorsement and conspiracy mentality were the most important targets of investigation. This was assumed to help achieve more honest responses and a reduction of social desirability bias. As pointed out e.g. by Butter and Knight (2016, p. 9), the ‘conspiracy theory’ label is not simply a neutral description of oppositional belief. Because the term effectively can be an insult, many may not willingly admit to being a conspiracy theorist. Feedback from the pilot study suggested that respondents felt this was a good idea, and the manner in which the conspiracy theories were presented did not attract an overly large amount of attention. In keeping with the aim, when queried on what they thought the survey was about, the consensus among pilot respondents was that it related to media habits, source criticism, and general assessments, views and thoughts.

Order of questions, coherence, and flow The order of the questions was also quite different in the earlier versions. Feedback from the pilot study formed the basis for a decision to make rearrangements to the entire structure. It was considered advantageous to not use up respondents’ patience on easy questions when their motivation was the highest, and important questions requiring the most cognitive effort were therefore repositioned closer to the beginning. Conversely, demographic questions were moved to the end. A sensible balance needed to be found, though, between placing difficult questions early on and avoiding immediate demotivation from respondents by overwhelming them before they were invested. Assessments were also made regarding which variables to sensibly group together in sections, and how these groups should naturally succeed each other. It was an overall aim to achieve a generally tidy impression and recognizable patterns for respondents to follow. To preserve the same logic, feel and flow throughout, a choice was also made to standardize the formatting of all Likert-scale questions. In addition to hopefully decreasing the rate of careless responses, this was done to simplify resulting response data for simpler analysis, as well as facilitate for statistical accuracy. Accordingly, all Likert-scales ended up with the same number of rating options (5), and the positive/negative direction of ratings was made consistent (1=affirmative/positive thru

49 Ronan Eric Reilly

5=negative/unfavorable). Original 7- or 11-point scales were converted, and some scales were reversed to accommodate for this. A pre-analysis re-reversal exercise was therefore necessary.

3.5 Choosing and operationalizing variables While their scientific underpinnings have already been introduced in the literary review, this section will systematically go through each variable in detail; explaining the process of adaptation from pre-existing scales, and elaborating on modifications that have been made where applicable. It will also describe the creation process involved for previously non-existing scales. For a complete overview of all scales, including their origin and item operationalization, please refer to appendix B.

Demographic data A section of the survey was dedicated to gathering information about some basic demographics. Gathered nominal and ordinal data consisted of gender, age, income, education level, daily primary activity, residency, political preference, and religious affiliation. Although it was ensured that all responses would be collected anonymously, many may not wish to share personal or sensitive information. Respondents were therefore given the courtesy option to ‘not wish to answer’ some of these items. The primary reason for gathering this information was to assess the diversity of the sample, which in turn could help say something about the general representativeness of the results.

Alternative media usage and news media habits A series of nominal and ordinal data were gathered to gain an understanding of respondents’ basic news media habits, such as their primary platform of news consumption, frequency of ‘common and familiar’ news source consumption, frequency of ‘unfamiliar recommended’ news source consumption, assessment of ‘alternative news’ meaning, frequency of ‘alternative news’ consumption (based on individual evaluation of the meaning of this term), and preferred news outlet. Reasons for gathering this information was twofold. As with the demographic data, the first reason was for basic descriptive purposes as a means towards assessing the diversity of the sample. Secondly, concealing the main purpose of the research was an underlying ulterior motive to avoid e.g. social desirability bias. While giving the impression that all these questions were interesting to the study, the thesis was primarily concerned with measuring two variables: “frequency of ‘alternative news’ consumption” and “assessment of ‘alternative news’ meaning”. The former intended to help answer RQ2, and simply measured frequency of alternative media consumption based on self-reported habits. A choice was made to collect categorized ordinal data matching the format asked by most of the questions in the ‘media usage and habits’-section. The question phrased was “How often do you read news from a media source you consider to be alternative?”, and eight

50 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world alternative answers ranging from ‘several times a day’ to ‘never’ were presented; including a ‘don’t know’ option. In the analysis these would be recoded into fewer alternatives to prepare for one-way ANOVA’s. As presented in chapter 2.2, the latter variable would act as a basis for answering RQ3, and responses given were presumed to infer differing attitudes regarding the agenda and purpose of alternative media. As previously explained, the actual academic definition of ‘alternative media’ is not as interesting to this study as the individual perception of what they attempt to achieve and how. Based on the survey-question “Some news sources are often described (or describe themselves) as being ‘alternative’. What does this term mean to you?”, six options were given – including an ‘other’ and a ‘don’t know/no opinion’ possibility. Respondents who answered “ambitious and brave news sources who dare to tell the truth” were considered highly positive/sympathetic towards alternative news sources. A neutral view was assumed when answering “niche news without any particular agenda”, and a negative/dismissive attitude was taken to be held when the derogatory answer was “politically biased news with questionable journalism”. The last possible answer was “news sources who have lost faith in mainstream media, and feel a need to separately investigate matters”, which presumably reflected a mildly sympathetic view.

Dependent variables: Conspiracy mentality / -theory endorsement

Conspiracy mentality questionnaire (CMQ) As explained in chapter 2.1, a decision was made to measure conspiracy mentality using the 5-item ‘Conspiracy Mentality Questionnaire’ (CMQ) by Bruder et al. (2013), which includes items such as: “I think that many very important things happen in the world, which the public is never informed about.”, and “I think that politicians usually do not tell us the true motives for their decisions.” Primary reasons for choosing this scale were the limited amount of items it used to measure the construct, the fact that it had been tested and validated through a series of different studies, as well as its aforementioned claim to be able to efficiently assess the general tendency to believe in conspiracy theories in a cross-cultural context (Bruder et al., 2013, pp. 1, 12). The original scale scores items on an unusual 11-point Likert scale where low scores are dismissive, but as explained, a 5-point score system conversion was made to keep the survey measures consistent.

Various conspiracy theory endorsement (VCTE10 / VCTE8 / VCTE7 / TVC2) To determine endorsement of some actual and specific conspiracy theories a “Various Conspiracy Theory Endorsement”-scale (VCTE) was created. This was done by collecting and briefly formulating several such theories and asking respondents to rate the likelihood of them being true on a 1-5 Likert scale. Most of the presented items were phrasings of event conspiracy claims, e.g. those pertaining to ‘faked Moon landing’, ‘’ or ‘9/11 being an ‘inside job’’. Others – e.g.

51 Ronan Eric Reilly suggesting that ‘a handful of people control the world’ – could be considered systemic, and that ‘climate issues are greatly exaggerated’ is commonly viewed as a tactical conspiracy. The complete list can be found in appendix B. One aim with this measure was to descriptively get an overview over the level of endorsement of these theories within the sample, but it was primarily fashioned to be a Likert-scale measure for the construct of ‘conspiracy theory endorsement’ to act alongside the CMQ as a dependent variable. This scale was also intended to help re-assess how well conspiracy mentality was correlated with endorsement of these actual conspiracy theories within the sample. A balance had to be found regarding which theories to include, and how extreme or unlikely options should be. Although it would have been interesting to include them, a choice was made to stay clear of the most extreme theories in fear of social desirability bias occurring. Respondents could potentially be scared to admit their beliefs, or prompted to dissociate with other much less irrational ‘9/11-coverups’ or ‘Diana assassination’ suspicions in fear of becoming associated with ‘lizard-people’-believers or ‘flat earthers’. Partially with the intent to counter such an effect, two verifiably true ‘conspiracy’-items were also included (‘MK Ultra mind control experiments’ and ‘Phoebus Light bulb cartel’). On their own, however, these two items also ended up constituting an additional separate scale of measurement – the TVC2 (‘true verifiable conspiracies’). It was used for reference purposes throughout the study to identify possible diverging relationships between the independent variables and conspiracy theory endorsement, as opposed to possible knowledge about more rational actual conspiracies. In addition, one item (suggesting that the Russians systematically surveil people’s homes through wire-tapped radios) was entirely concocted with the purpose of investigating whether it would affect assessment of likelihood to be previously familiar with a conspiracy theory, or if a previously unknown theory with the ‘hallmarks’ of a ‘conspiracy theory’ – as described in chapter 2.1 – would be assessed equally. Based on the included examples, the VCTE scales ended up consisting of a full ten-item VCTE10 scale, a seven-item ‘false conspiracy theories’-scale (VCTE7), an eight-item ‘false and fake’-scale (VCTE8), a two-item ‘true verifiable conspiracies’-scale (TVC2), and a single non-scale fake item (VCTE1). The relationships between the variations of these scales were assessed, and the most fruitful constellations for further use were identified. The VCTE variants evolved from an original much more extensive scale through pilot testing and repeated re-tweaking. Ultimately, it was expected that the resulting scales would be able to capture and reveal a wide variety of conspiracy endorsement in respondents.

Independent variables (A): News media ‘skepticism’ and ‘literacy’ This section will present the variables used to measure relevant constructs for interest area A of this thesis. Measurements related to RQ2 and RQ3 have already been treated in the ‘alternative media usage and news media habits’-section, and the following will deal with RQ1 and RQ4.

52 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

News media credibility scale (NCRED) – RQ1 As presented in 2.2, to measure mainstream news skepticism this thesis utilized a simple but effective scale (Gaziano & McGrath, 1986; Meyer, 1988; M. D. West, 1994) which has been proven trustworthy in measuring both news credibility and skepticism in the past. Respondents were asked to rate on a 1-5 Likert-scale their assessment of news providers’ accuracy, fairness, unbiasedness, trustworthiness, and tendency to tell the ‘whole story’.

News media literacy scale(s) (NML/NFC2/MLOC3) – RQ4 As explained in chapter 2.2, there are not many scales developed to measure news media literacy. This thesis has chosen to see it as a multidimensional construct, and based on an overall assessment of successful past measurements a choice was made to avail of constructs from Craft et al. (2017) – which in turn were based on Craft et al. (2013)’s news media literacy model. The aim of this model was to capture the three aspects of mindful thought processing, perceived control over news media influence, and knowledge about the news media system. The variables were however chosen to be assembled differently in this study; e.g. by excluding the use of the NMKS – despite it yielding the strongest and most consistent results in relation to conspiracy theory endorsement (Craft et al., 2017, p. 396). The reason for this was partially that the relationship to conspiracy theories had already been established with this measure, but also the fact that the NMKS was measured through multiple-choice tasks which made it hard to integrate into the Likert based framework of this study. Furthermore, all the questions in the original NMKS pertain to media systems in the United States, and they would therefore require heavy modifications to accommodate to the Norwegian media landscape. This was attempted at first, but for various reasons – including feedback from the pilot study deeming the questions ‘too easy’ – this endeavor was abandoned. Instead, a new ‘hybrid’ News Media Literacy scale was created – retaining the MLOC and NFC components (in reduced versions), and substituting the NMKS with the alternative NML scale by (Ashley et al., 2013).

News media literacy scale (NML) The alternative NML scale consisted of the full scale (13 items), and its individual 3 sub-scales: Authors and Audiences (AA), Messages and Meanings (MM), and Representations and Reality (RR). This scale produced data that was more compatible with the survey and appeared to also measure knowledge about various news media structures even if it was done differently than the NMKS. As with all other scales, these were also adapted into 5-point scores. As treated in chapter 2.2, Vraga et al. (2015) decided to make some adjustments to Ashley et al. (2013)’s original scale, e.g. by dropping a poorly loaded item (p. 45). As also mentioned, it was deemed fruitful to make some further modifications. Changes included reverse coding some items, as well as merging similar items, removing seemingly irrelevant ones, and adding new ones.

53 Ronan Eric Reilly

Authors and Audiences (AA): Three items, including statements such as “News companies choose stories based on what will attract the biggest audience”, and “Individuals largely chose news sources irrespective of their own political values” (reverse coded), were presented to measure respondents’ basic understanding of how news providers and consumers interact. Messages and Meanings (MM): Six items (original scale has seven) were presented in order to capture knowledge about how meanings are created based on conveyed information. Items included such statements as: “News is not created to gain attention from the public, but exists to inform” (reverse coded from original “News is designed to attract an audience’s attention”), and “Various production techniques (camera angle, lighting, etc.) can affect viewers’ perception of a story” (merged original two items referring to lighting and production techniques separately). Representations and Reality (RR): Respondents were asked to assess four items (originally three), pertaining to the filtering of information in the media and how this affects perceptions of reality. Examples of items included “A news story about is more likely to be featured prominently” and “News try to avoid overdramatizing reality” (reverse coded from original “News makes things more dramatic than they really are”). Furthermore, based on the previously mentioned inability to find an item that measures awareness of agenda setting theory in the NML scale, it was decided to add an extra item (which also was reverse coded): “The importance of a news story will be primarily judged by the seriousness of its nature – not by the frequency with which it is presented and emphasized.” As far as this research is concerned, this is a very important aspect of understanding in relation to media representations of reality, and accordingly needed to be added.

Need for cognition (NFC2) The NFC2 scale was based on assessment of several pre-existing scales where the essence of the items was adapted and reduced. The main reason for item-reduction was feedback from the pilot study that the questionnaire was too long. Previous measures of this construct have included a varying number of items. Since Cacioppo and Petty (1982) developed a 34-item scale, this number has been attempted reduced; e.g. by Lins de Holanda Coelho, Hanel, and Wolf (2018)’s six-item scale. Craft et al. (2017), Craft et al. (2013), and Maksl et al. (2015) all used a five-item scale. To create a significantly reduced two-item scale, the two items with the highest factor loadings found in both the Holanda et al. and Craft/Maksl-versions were chosen. Resulting items measuring NFC were: “I prefer complex to simple problems” and “I would rather do something that requires little thought than something that is sure to challenge my thinking abilities” (reverse coded).

Media locus of control (MLOC3) In a similar way to the NFC2 scale above, this variable was created and modified from the more extensive scale that e.g. Craft et al. (2013) and Maksl et al. (2015) adapted from Wallston et al.

54 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

(1978)’s six-item health-locus scale. This scale was reduced to three items by pragmatically judging the essence of the construct and omitting or merging similar items. The resulting items were presented to respondents: “It is my own responsibility to control how I get my information, and where I get it from”, “It is my own fault if I am misinformed by the news media”, and “It is up to me to evaluate news messages I receive and, if needed, seek out correct information”.

Independent variables (B): Critical thinking dispositions and skills The following will elaborate on interest area B, and variable measures related to RQ5 through RQ8.

Critical thinking disposition scale (CTDS) – RQ5 Critical thinking disposition was measured using the scale developed by Sosu (2013), which was presented in chapter 2.3. Examples of this scale having been used in conjunction with similar research before have not been uncovered in the literature. The scale consists of the two sub domains: ‘critical openness’ (CO) and ‘reflective skepticism’ (RS). Three scores were calculated – the two sub-domains individually and the complete scale total. Sosu (2013) provided keys for grouping scores along with his research. Accordingly, although this scale appears to be measured like most others in the study it was not based on a computed arithmetic mean score. Instead, it summarized the scores of all answers and categorized them into ‘low’, ‘moderate’ and ‘high’ levels of disposition, suitable for ANOVA analysis. For score-intervals, please confer with appendix B. Critical openness (CO) was assessed from seven items, such as “I usually try to think about the bigger picture during a discussion” and “I don’t generally feel a need to justify my choices to other people” (reverse coded from original “it is important to justify the choices I make”). Reflective skepticism (RS) consisted of four items, including “I often re-evaluate my experiences so that I can learn from them” and “As long as my methods work, I rarely see any good reason why I should change the way I do things” (reverse coded from original “I often think about my actions to see whether I could improve them”).

Informal logical fallacy argumentation scale (ILFARG) – RQ6 The ILFARG-scale was not based on any previous measure and was composed entirely from self- created arguments inspired by and derived from common informal logical fallacies as described in chapter 2.3. A total of eleven fallacious arguments were presented, concocted out of various fallacies such as anecdotal evidence, faulty comparison, ad populum/appeal to popularity, strawman, false dichotomy, burden of proof fallacy, non sequitur, ad-hominem, and appeals to nature, age, and false authority. Items included: “Nobody can disprove that we all have a destiny, and therefore it is likely that all major events in life are predetermined, and “If a skincare product contains only ecological and natural ingredients, then it must be better for the skin than a product

55 Ronan Eric Reilly that is full of artificial substances”. Respondents were prompted to assess whether these were strong or poor arguments on a 1-5 scale (3=neutral), being instructed specifically to try avoiding having own opinions factor into this judgment. The idea was to present arguments that were as varied as possible, and preferably in a large enough quantity to be able to establish a pattern of interpretation skills. Originally this scale presented close to thirty fallacious arguments, but the pilot study revealed a need to shorten the number of items drastically. The aim of the scale was to capture respondents’ overall ability to detect fallacious arguments separate from own opinion, and the final scale consisted of a total of fifteen items, whereof four were non-fallacious items such as “Science does not have all the answers, but it is likely that a cancer researcher knows more than a chiropractor about what can cure cancer”. These four items were not included in the measurement as it was not of interest to evaluate whether respondents incorrectly assessed reasonable arguments or non-illogical statements. They were created and randomly placed between the fallacious arguments because it was deemed necessary to not only provide respondents with fallacies. Also, careless responding would possibly have occurred without these items because it is unlikely anyone would expect all arguments to be poor, or that all answers were ‘meant to be’ scored equally.

Pseudo-profound bullshit/profoundness scales (BSREC4/PREC4) – RQ7 Inspired by Pennycook et al. (2015) and Erlandsson et al. (2018)’s research reviewed in chapter 2.3, ‘pseudo-profound bullshit receptivity’ and ‘profoundness receptivity’ was measured. Firstly, it was deemed necessary to include profound items in the scale in order to reduce bias in the ‘receptivity’– measure. Similarly to the assumptions made regarding the ILFARG-scale, if one were to provide respondents with only bullshit statements they would most likely not answer all options equally due to an assumption that there would be an intended difference between them. Secondly, differing results between the two scales could possibly also indicate less random – i.e. more valid and reliable – ‘receptivity’-measurement. As original scales did not translate well into Norwegian and were quite long, a decision was made to reduce and combine some of the original statements with similarly sounding self-authored ones. Examples of resulting (translated) profound bullshit statements included in the final scale were: “Hidden meaning transforms unparalleled abstract beauty” (similar to original Pennycook et al. scale) and “Grace is a vision of wisdom and strength, but not necessarily of primary vitality” (self-created). True profoundness statements attempted to portray an actual meaning or underlying ‘wisdom’ within their phrasing. They were inspired by sayings and words of wisdom found in various online locations, and included statements such as: “Even he who shall move an entire mountain must begin by carrying small rocks” and “Living is like drawing on a blank sheet of paper without having an eraser”. Respondents were asked to rate on a scale from 1-5 (3=neutral) how meaningful they found the statements to be. The question

56 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world elaborated on this by instructing to make an overall assessment based on coherence/meaningfulness and whether a deeper profound meaning could be found. Resulting scores were calculated into two scales – BSREC4; four items measuring detection of pseudo-profound bullshit, and PREC4; four items measuring detection of profoundness. They were joined and presented to respondents as a single 8-item scale in randomized sequence.

Source criticism ability scale (CRAAP) A CRAAP-scale was created by itemizing the checkpoints from Blakeslee (2004)’s CRAAP test presented in chapter 2.3; which also can be found in their entirety in appendix C. This was achieved by converting all relevant checkpoints into a ten-item scale with a series of items phrased as specific source criticism-‘actions’. These were all scored on a 1-5 scale ranging from ‘often’ to ‘very rarely’ (3=’no opinion’) when asked: “You are faced with, or are presented with, a news story. How often or rarely would you say you do the following?”. Examples of ratable items derived from the scale were: “I assess how long ago the story was published or updated last”, “I check to see if the information can be verified by other sources”, and “I assess if the language used is emotionally charged or neutral”. A certain amount of rephrasing and merging of points was deemed necessary in order to create this scale. As with most of the other scales, an arithmetic mean was computed based on the responses, and a total ‘source criticism ability level’ could be quantified.

3.6 Distribution and collection of data The survey was conducted during the months of September and October 2019. It was widely distributed across multiple arenas, and two versions were created based on the channels they were distributed in. Only the introductory presentation text with contact information and incentive etc. was altered, while questions, their order, and the layout remained unchanged. The first version was distributed in the form of a link provided by a public post on the authors’ personal profile, and this was considered the primary channel of distribution. The post was followed up and administrated continuously with status updates, messages of gratitude and encouragements to like, share and comment etc. It included a twofold incentive – described in detail both in the post itself, but also in the disclaimer-text in the actual questionnaire. The first incentive was a raffle where a 500 NOK cash payment to a random person who liked and shared the post would be drawn upon completion, provided a total of at least 100 responses was reached. Actual participation in the study was encouraged, but not a requirement, as this would prove difficult to manage due to the protection of participant anonymity. The second incentive promised 5 NOK per response to ‘Barnekreftforeningen’ (The Norwegian Children’s Cancer Union), with the added bonus of 500 NOK upon achievement of 100 responses. The aim was to

57 Ronan Eric Reilly achieve a total donation of at least 1000 NOK to this charitable cause. The full text with terms (in Norwegian) can be found on the first page of the complete survey in Appendix A. The second version did not include any competitions or incentives and was more neutral in its disclaimer text. For personal protection purposes it was not desirable to attract attention or link the questionnaire to a personal Facebook profile. This version was created for distribution in Facebook-groups, but also various hobby- and political activist forums, commentary sections of both alternative and mainstream newspapers and various affiliated social media – as well as social media groups purposely focusing on sharing and discussing conspiracy theories and extreme views etc. It was an extensive endeavor to map out all feasible platforms worth pursuing which also would allow for the distribution of such a questionnaire. Many forums were discovered which were obvious platforms, but with regard to the purposive sampling endeavor it was difficult to find many very highly active Norway-based forums discussing conspiracy theories exclusively. After identifying several channels, the commencement of a lengthy process of enrollment began. Authoring various versions of enticing posts presenting the survey, and identifying appropriate threads within forums to post in, was the next step. This included tweaking text to suit the different arenas, as content needed to resonate with the expected audience within the given channels; e.g. different phrasing was used in conspiracy theory groups than mainstream newspaper forums. Continuous upkeep of all these posts and communicating with administrators etc. was also necessary. To join Facebook-groups and certain forums, a choice was made to create a new profile for the purpose of personal discretion. This led to some ethical issues, as it would not be research- ethically viable to encourage data collection under a pseudonym or false identity. Therefore, a profile was registered with a neutral profile picture and a reshuffled partial real name. This profile could later be removed without having to sacrifice one’s own original profile. In keeping with proper research practice full and valid contact information remained available in the disclaimer of the questionnaire itself. Among the many arenas selected for distribution were online debate forums such as ’VG Debatt’ (vgd.no), ‘Norsk Freakforum’ (freak.no), ‘Diskusjon.no’, ‘Babyverden’ (forum.babyverden.no), ‘Steigan.no’ (commentary section), ‘Reddit’ (reddit.com/r/norge) etc., and both external media affiliated and general non-affiliated Facebook-groups such as ‘Document.no’, ‘Resetts tilhengere’, ‘Konspirasjonsteorier’, ‘Konspirasjonsland’, ‘Debatt Norge’, ‘Ytringsfrihet er absolutt’, ‘Nyhetsspeilet.no’, ‘Rasjonalitet’, ‘Dagsavisen debatt’, ‘Et bedre Norge’, ‘Nordnorsk debatt’, ‘Nettavisen - friske meninger’, and ‘Konspirasjonspodden’, among others. Additionally, in keeping with snowball sampling, everyone was encouraged to pass along the questionnaire to acquaintances. Due to the anonymous nature of the data gathering, though, there is no way to know

58 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world which arenas led to replies or what relationships exist between answers and where they originated. During the distribution and collection phase, many respondents chose very politely to provide feedback appraising how interesting they found the survey to be. As anticipated, though, certain members of some of the Facebook-groups did react with perceivably semi-threatening comments in response to the questionnaire-request; e.g. claiming that media education was “brainwashing”, or not wishing to interact with a “group of corrupt mass media liars”. These reactions are consistent with much of the research covered – as is the fact that such comments rapidly gained ‘likes’ among likeminded members of the same groups.

Ethical concerns with continuous data handling As explained in chapter 3.2, this research was conducted anonymously, and accordingly necessary ethical concerns were kept to a minimum. Both during and after the data collection period some anonymity considerations were however inescapable. It was e.g. of pivotal importance not to look at any data before a considerable amount of responses were gathered – and certainly never open survey-responses immediately after completion. Specific responses could e.g. potentially align with personal Facebook-post comments stating that someone had just answered. For the same reason, upon exporting any collected data all generated timestamps were deleted, and chronological order of reception was randomized; allowing there to be no trace between time of answers and possible comments. Another issue arose shortly before distribution. A carefully planned incentive-system had originally been developed to allow for a personal cash prize raffle, and a separate link to an independent anonymous form obtaining personal contact information was optionally available at the end of the questionnaire. This endeavor was however deemed unsuitable, as it was assessed that respondents would become skeptical if they were asked to enter contact information after entering an anonymous study. Despite it being highly likely that personally targeted incentives may have increased response rates, it was ultimately regarded as too ethically ‘risky’ to be responsible for personal contact information; even though the system theoretically allowed for full anonymity by separating contact information from specific responses.

3.7 Data analysis preparations and considerations At the point of termination, a total of N=125 responses was reached. It would have been beneficial to the study if this number had been significantly higher, but ultimately – given the complexity and length of the survey – triple digit responses were considered to be a satisfactory achievement. N=100 was in advance also deemed an ‘acceptable level’ by the thesis supervisor. All incentive targets were achieved, and promised payments were made to a randomly selected participant, as well as to ‘Barnekreftforeningen’.

59 Ronan Eric Reilly

Transferring and preparing raw data The next step was to export all raw data from Google Forms to SPSS. To achieve this, a series of operations were carried out. As Google Forms is not directly compatible with SPSS, a conversion and systemizing process was first done in Microsoft Excel, where e.g. response data was converted from text to numerical 1-5 values. Furthermore, each variable identifier (title) needed to be shortened from a cell with an entire question or option to an understandable, yet concise label. During this process, the timestamps were removed from all responses to prevent any connection to respondents and ensure anonymity in further data handling. Upon completion of the Excel-file, it was imported into SPSS (Version 26) where the raw data needed some further preparations. Firstly, nominal, ordinal and scale (interval or ratio) data types were configured. Secondly, other properties, such as number of decimals, missing values and value labels were defined. Then, a lengthy procedure of separating, computing, recalculating, joining and defining items based on which multi- item Likert scale they belonged to was carried out – all the while remembering to flip the values of reverse coded items. Also, certain variables needed to be recoded into fewer categories, and all labels and non-numeric values were translated into English.

Parametric vs non-parametric statistical assumptions, and levels of measurement An important consideration when performing statistical operations is the choice between running parametric or non-parametric tests. This decision dictates how data will be prepared and treated. This thesis has primarily planned for data analysis using parametric methods, as they are often more powerful and generally preferred to non-parametric equivalents (e.g.Wadgave & Khairnar, 2019, p. 7469). Achieving the most accurate results from such tests, however, often hinges on fulfilment of four primary assumptions: a normally distributed sampling distribution, homogeneity of variance, interval or ratio data, and independence (Field, 2009, p. 133). The latter assumption was not regarded a problem to meet in this study, as it was highly unlikely that participants had affected each other’s responses. It is however necessary to elaborate on the former three. The first two assumptions need to be assessed for each individual statistical operation, as they are based on the relationship between compared variables. One can assess normal distribution both graphically and numerically, but ordinarily it is regarded that as a sample size increases beyond N=30 (which is the case in most circumstances throughout this thesis), then the Central Limit Theorem invokes that the sampling distribution is likely to be normally distributed (Field, 2009, p. 42). This would have been more likely to be the case if the gathered responses had been from a probability sample, though, which sadly they are not. Therefore, a Shapiro-Wilks test for normality is run where applicable, and results are reported as part of the analysis. Homogeneity of variance, an assumption which in correlational designs mean that the variance of one variable should be stable at all levels of the

60 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world other variable (Field, 2009, p. 133), is evaluated using the Lavene’s test where applicable. Both the Shapiro-Wilks- and Lavene’s tests should yield a ρ-value greater than 0,05 (statistically non- significant) in order for the data to be considered normally distributed and homogeneous, respectively (Field, 2009, pp. 147, 150). The final, and perhaps most important, assumption pertains to the measurement level of the gathered data, and questions can be raised as to what data the many Likert scales produce. As mentioned in chapter 3.4, a long-standing discussion regarding how to treat Likert-scale data exists within the scientific community (e.g. Carifio & Perla, 2007; Jamieson, 2004; Murray, 2013). Many argue that such data are ordinal at best because one cannot claim there to be an equal distance between all the options, in line with the requirements for interval data (e.g. van Alphen et al., 1994, p. 200); e.g. ‘strongly agree’ and ‘somewhat agree’ cannot be considered to be spaced similarly as ‘no opinion’ and ‘somewhat disagree’. Also, Likert-scales are arbitrary in nature, with no objective numerical value; holding values which are merely determined by the survey-designing researcher and the desired level of detail he or she chooses (e.g. Munshi, 2014, p. 1). It is pointed out e.g. by Jamieson (2004, p. 1217), though, that despite this it has become relatively common to both assume that interval-level measurement is produced by Likert scales, as well as to apply this in practice. This trend is evident also in most of the presented literature in this thesis, and many of the adopted scales have previously treated data as interval. Carifio and Perla (2007, p. 106) have in fact shown that many studies indeed do support Likert-scales producing interval-like data suitable for parametric tests, such as the ANOVA’s, as long as scales consisting of multiple averaged items are used as oppose to single Likert items. This is in keeping with the intents of this thesis, and despite the heated ongoing debates regarding the legitimacy and validity of doing so, this study ultimately considers it defendable to primarily treat Likert data as interval-level when performing analyses.

Statistical tests When examining relationships between variables where both are interval, bivariate correlation analyses are run to assess the correlation coefficients and significance levels. The parametric Pearson correlation coefficient values are used as a basis for analysis. In order to cross-examine that potentially drastically inaccurate parametric relationships do not occur, the non-parametric Spearman’s rho values are calculated in addition. As long as the Pearson and Spearman coefficients both indicate significant relationships and are relatively close in value, this will be regarded as support that findings can be trusted irrespective of potentially violated assumptions; and normality- and homogeneity-tests are therefore not be run for correlation analyses. Also, unless noteworthy discrepancies occur, only the Pearson coefficients will be reported. A commonly made mistake when interpreting correlated data is assuming that correlation implies causality. Sometimes, though,

61 Ronan Eric Reilly variables can have close relationships without one necessarily explaining the other, or there could be additional underlying variables causing both to spuriously correlate (Foster et al., 2015, p. 182). Relationships can sometimes also occur simply due to complete or partial coincidence. Regardless of significant findings, this will be considered during interpretation. When testing nominal or ordinal categorical groups against interval data, one-way independent sample ANOVA’s (analysis of variance) are run. An ANOVA is a statistical test that compares differences in mean scores between groups. It is suitable, and preferable to e.g. multiple t- tests, as this research compares more than two conditions (Field, 2009, p. 348). When significant results are discovered, Tukey HSD post-hoc tests are carried out in order to determine which specific groups have significantly different mean values. Ordinarily, many researchers may choose to discard the ANOVA entirely if assumptions fail – especially if both homogeneity of variance and normality are violated. Should tests reveal that variables do not fulfil the assumption criteria, traditionally a choice could be made to either conduct a logarithmic transformation, or to utilize non-parametric counterpart tests such as the Kruskal-Wallis H test (Field, 2009, p. 345). As doing so falls outside of a scope of complication this thesis wishes to remain within, it chooses to merely point out these weaknesses if they occur.

62 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

4 Analyses and Discussions

The analysis section seeks to investigate and discuss gathered and prepared findings based on the reviewed literature. Initially, an effort is made to describe nominal and ordinal information, such as demographic data and media habits and views. As explained in chapter 3.5, this is done primarily to make assessments regarding the sample diversity, which is particularly important in such a small, non-probability sample. Additional explorations of the data are made, before the two main interest areas and their respective four research questions are treated chronologically. Relevant hypotheses are formulated and tested based on these, and interpretations of results and discussions follow.

4.1 Overview of sample data

Univariate analysis of demography In reviewing the demographic data, a lot can be understood about the diversity of participating respondents in the study, and accordingly how well represented overall traits are in the further analysis. Some potentially sensitive information was optional to give, but from table 4.1.1 we can read that most respondents happily have chosen not to opt out of sharing it, as there is little missing data. We can further visualize by assessing the mode scores that the most common response in every category looks to loosely resemble that of a hypothetical ‘average Norwegian’; e.g. most people earn 450-600.000 kr. and have a ‘medium’ amount of education. This is valuable, as it can suggest that a common public opinion is more highly represented than that of peripheral groups. Figure 4.1.2 provides a more nuanced graphic representation of all responses and how they are distributed. Most ordinally measured data – such as age, education level and income – appear to be distributed evenly with ascending values before and descending values after the very apparent modes, which are centered on middle responses. With the possible exception of overrepresented young adults and middle-aged people, this does suggest a sensible proportion of sample responses in relation to society in general. It does however also mean that for analysis purposes there is an uneven amount of data from different groups if one wanted to compare them directly. The fact that all groups are represented at all, though – e.g. from under 16’s to over 75’s, and from under 100.000 kr. earned per year to over 1 million – is considered very valuable to the study. An overweight of male respondents are represented, however not to the extent that it would be considered problematic to worry about significant gender bias. Although a clear non-religious tendency is present, there are

Gender Age Income Edu. level Daily activity Residency Pol. pref. Valid 125 125 110 115 121 125 116 118 N Missing 0 0 15 10 4 0 9 7 Mode Male 35-44 450’K–600’K Uni’ 3+ F.T work (70%+) Large city (100 K+) Varies / No pref. Not religious Table 4.1.1 – Basic descriptive statistics of demographic data

63 Ronan Eric Reilly

Gender Religion Don't know / Have Christian not decided 17 % Female Male 11 % 37 % 63 % Other Not religious religion 71 % 1 % Income Age 35,5 % 37,6 % 21,8 % 24,8 % 10,9 % 11,8 % 11,8 % 12,8 % 5,5 % 8,0 % 8,0 % 2,7 % 0,8 % 4,8 % 3,2 %

0-100.000 Kr 300.001-450.000 600.001-750.000 Over 1.000.000 Kr 16 or 17-24 25-34 35-44 45-54 55-64 65-74 75 or Kr Kr younger older Education level Residency 35,2 % 45,2 % 29,6 % 22,4 % 23,5 % 20,0 % 5,6 % 5,6 % 4,3 % 7,0 % 1,6 %

Secondary High school (or Univerity / Univerity / More than 5 Large city Medium size Small city / Town (2000 Small town Rural (no school / Basic equivalent) College (e.g. 3 College (e.g. years of higher (100.000+) city (25.000 Town (5000 - 5000) (less than town mandatory year Bachelors year Masters education - 100.000) - 25.000) 2000) centre) schooling degree) degree)

61,2 % Daily activity

8,3 % 9,1 % 3,3 % 0,8 % 0,8 % 5,0 % 7,4 % 2,5 % 1,7 %

Full time work Part time work Self employed Student Home with Unemployed Long term Retired Combination of Other (over 70%) (70% or less) children illness / 2 or more disability Political party preference 21,6 % 15,5 % 12,1 % 8,6 % 10,3 % 10,3 % 6,9 % 4,3 % 4,3 % 5,2 % 0,9 %

This varies / AP H FrP KrF V Sp SV R MdG Others No clear preference Figure 4.1.2 – Charts and diagrams of sample demography enough religious respondents (18%) in the sample to have their views be well represented (albeit they are almost exclusively Christian). Political preferences are remarkably diverse, however based on actual election polls the assessment of these responses is that they are much more evenly distributed than e.g. a probability sample of all Norwegians would have been. The ‘residency’ variable indicates that there are representatives from all sorts of geographical districts, although less than 15% are from a town or rural area with below 5000 people; which means that over 80% live in cities (whereby over a third in a large 100.000 + city). Again, this makes sense in relation to an imagined larger population. The ‘daily activity’ variable was originally included to portray respondents’ tendency to be occupied by conventional productive daily activities, such as work or study. Given that this is a thesis for a university degree, responses from students are strangely scarce at 9,1%, although it was purposely attempted to limit recruitment from fellow students in order to avoid bias in terms of an overweight of media knowledge etc. More surprising is the small

64 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world number of respondents who report a lack of ‘productive activities’ in the form of unemployment (0,8%) – or even part-time work (3,3%). All in all, the overall responses consist of close to 85% indicating daily purposeful activities. As most of the remaining percentage (7,4%) are retired, this leaves a bit of a bias in the sample population – whereby particularly those who are unemployed seem underrepresented. This is important, as a weight of research pertaining to conspiracy theories often is associated with the emotional states that often tend to accompany such circumstances. The sum of these observations leaves the overall impression that this is a quite diverse sample; perhaps even more so than anticipated. It is likely that the study captures views, attitudes, and opinions of people of all ages from vast backgrounds and geographical areas, with different degrees of income and education, scattered political preferences, and differing religious affiliations.

Univariate analysis of media views, habits, and preferences Media views, habits and preferences were measured through six questions, the results of which are all presented here. As explained in 3.5, two of these will be revisited when analyzing RQ2 and RQ3. However, in this section all questions are treated as a means of assessing the sample diversity. Observations are briefly commented, and graphical representations presented.

Primary platform of news consumption

Figure 4.1.3 – Primary platform of news consumption (percent) Other No clear preference Others share with me what I need to know Radio Paper format newspapers TV Social Media (Facebook, etc.) Internet (online news etc - NOT Social Media) 0 10 20 30 40 50 60 70 80

The sample has provided data with a very clear mode, as 71,2% report to find their news via various Internet sources. The runner-up category, Social Media, is also online based, and scores 12%. These two categories alone make up almost 85% of the responses. This is highly expected, as all respondents found the survey via online channels. It does however mean that the study lacks opinions and views of most ‘analogue’ news consumers.

Frequency of ‘common and familiar’ news source consumption

Figure 4.1.4 – Frequency of ‘common and familiar’ news source consumption (percent) Rarer than monthly Monthly Weekly 4-6 days a week Daily Several times a day 0 10 20 30 40 50 60 70 It seems clear that almost every respondent is a highly active consumer of the news sources they regard as common and familiar, with more than 85% reporting to do so at a minimum of once a day.

65 Ronan Eric Reilly

Only less than 5% report to consume such news monthly or rarer. This question does not say anything about what sources these respondents prefer, but it does address their own sense of staying up to date.

Frequency of ‘unfamiliar recommended’ news source consumption

Figure 4.1.5 – Frequency of ‘unfamiliar recommended’ news source consumption (percent) Never Rarer than monthly Monthly Weekly 4-6 days a week Daily Several times a day 0 5 10 15 20 25 30 35

This question attempts to capture how often respondents admit to consuming news where the sources are unfamiliar; e.g. when one has stumbled across an article or it has been shared via social media etc. Answers are interestingly diverse, and they suggest a sample where almost 70% read recommended news weekly or more often; whereby over half of them do so on a daily basis. At the same time, there are in total almost as many (over 30%) who do this monthly or rarer, while approximately 40% do it several times a week. This is interesting to the study because it allows for analysis of a sample containing both careful and ‘stubborn’ news consumers, as well as those who are potentially very accepting of new sources.

Assessment of ‘alternative news’ meaning Figure 4.1.6 – Assessment of ‘alternative news’ meaning (percent) News sources who have lost faith in mass media, and feel a need to… Politically biased news with questionable journalism Ambitious and brave news sources, who dare to tell the truth Niche news without any particular agenda 0 5 10 15 20 25 30 35 40 45 It is a valuable observation that respondents have differing assessments regarding how they view alternative news. Again, this points to a diverse sample, where ‘only’ 42,5% appear to regard alternative media as particularly biased and questionable. As this is the question which will help answer RQ3 later on it is important to observe that all the different views are represented, although ideally it would have been advantageous to have more than 13,3% responses regarding alternative news to be “ambitious and brave…”.

Frequency of ‘alternative news’ consumption

Figure 4.1.7 – Frequency of ‘alternative news’ consumption (percent) Never Rarer than monthly Monthly Weekly 4-6 days a week Daily Several times a day 0 5 10 15 20 25 30 Yet again, these responses show highly diverse and evenly distributed habits within the sample

66 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world regarding consumption of alternative news. This is very beneficial to the study, as it provides a solid foundation upon which to investigate RQ2 later on. Surprisingly, slightly north of half the respondents report consumption of such news on at least a weekly basis, which conversely means that less than half answered monthly or rarer. These responses do not say anything about why respondents consume alternative news, or if they rate such sources highly or not, but they nevertheless indicate that what respondents themselves regard to be ‘alternative news’ does get consumed a lot within the sample.

Preferred news outlet Figure 4.1.8 – Preferred news outlet (percent) Other Other alternative international news sources One of the major international news mass media (BBC, CNN, Sky, Al Jazeera etc.) Important news is shared with me via social media etc. Other Norwegian alternative news sources Other Norwegian known and established news sources One of the major Norwegian mass media (VG, Dagbladet, Aftenposten, NRK, TV2,… 0 10 20 30 40 50 60 70 80 Regardless of all other habits, preferences and views, this question reveals that close to 70% of respondents prefer common Norwegian mainstream news outlets in competition with any other outlet. Even though every possible answer is represented, there is a clear mode and a detectable tendency. This does provide a certain degree of perspective to the actual everyday practice of respondents, and it coincides well with general prior assumptions held regarding media habits among the public. However, there is in fact close to 15% of respondents who have indicated a preference towards either Norwegian or international alternative news sources, and therefore this group is also relatively well represented. The overall assessment of diversity within the sample is quite satisfactory in terms of both appearing to represent most segments of people within a general population, as well as with regard to the number of differing responses to the questions that are important later in the study.

Initial exploration of data from Likert scales This section explores descriptive statistics for all Likert scales in the study, and table 4.1.9 summarizes SPSS output for essential statistical values. There are no missing values, hence all scales contain complete data from the entire sample (N=125). As mentioned, appendix B provides an overview of the operationalization of all scales, as this table only refers to their abbreviations.

General observations regarding central tendencies and deviations The significance of individual statistics will be commented on where applicable in the following sections, but as a general observation both mean and median measures of central tendency appear to be quite close to each other in value on all scales. This suggests that they all have a relatively even distribution. Knowing that all scales are scored between values of 1,00 and 5,00, leaving a range of

67 Ronan Eric Reilly

4,00 – and observing that no standard deviations are outside values of σ = .42 and σ = 1.09 – further suggests that the variance is generally relatively low. Overall, there is no reason to believe that it should be unfeasible to perform fruitful analyses based on these data.

Reliability / internal consistency of scales As previously presented, a widely accepted way of assessing the reliability, or internal consistency, of a multi-item scale is by calculating and assessing the ‘Cronbach’s Alpha’ (α) coefficient. According to Field (2009, p. 675), many scholars Table 4.1.9 – Descriptive and reliability statistics (α) for all Likert scales in the study N of Cronbach's Mean Std. Dev. Scale Median Valid N argue that this value should be as high as possible Items α (x̄) (σ) in order for a scale to be reliable. Preferably it CMQ 5 ,835 2,2832 ,92464 2,0000 125 VCTE10 10 ,799 3,5176 ,76307 3,5000 125 should exceed α = 0,7, although values as low as VCTE8 8 ,817 3,5869 ,82023 3,6429 125 VCTE7 7 ,819 3,8057 ,91030 3,8571 125 α = 0,5 are often deemed ‘acceptable’; although TVC2 2 ,502 2,5840 1,01349 2,5000 125 NCRED 5 ,925 3,1104 1,08671 3,2000 125 often advised against. Others are not as strict in NML_AA 3 ,283 1,8187 ,61420 1,6667 125 their evaluation and argue that one must always NML_MM 6 ,239 2,1040 ,46982 2,0000 125 NML_RR* 3* ,464 2,1307 ,76070 2,0000 125 assess scales individually regardless of α-value, NML_ALL* 12* ,574 2,0393 ,43166 2,0000 125 NFC2 2 ,644 2,2520 ,94105 2,0000 125 and that e.g. a lower number of scale-items allows MLOC3 3 ,544 2,0053 ,72461 2,0000 125 for a much lower value (Field, 2009, p. 675). CTDS_CO 7 ,392 3,6514 ,44084 3,5714 125 CTDS_RS 4 ,339 3,5960 ,64663 3,5000 125 Using SPSS, the α-values have been calculated CTDS_ALL 11 ,507 3,6313 ,41665 3,6364 125 ILFARG 11 ,672 3,6538 ,60509 3,7273 125 for every scale (See table 4.1.9), including those BSREC4 4 ,784 3,5300 ,89780 3,5000 125 PREC4 4 ,477 2,1580 ,71669 2,0000 125 who are adapted and previously validated in other CRAAP 10 ,833 1,9464 ,68821 1,8000 125 studies. Some very unexpected findings are * One item was omitted to increase α revealed when assessing these values. While some of the self-constructed scales – such as most VCTE-variants, BSREC4, CRAAP, and to a certain degree also ILFARG – surprisingly seem to withstand the reliability test very well, at least three of the adapted pre-validated scales display surprisingly low values. Most CTDS variants, but particularly the subscales of NML, are rejectable by most standards. Also, the self-created PREC4 (α = .477) is highly questionable. This is problematic to the study and must be taken into careful consideration when interpreting results. Cronbach’s alpha scores are calculated by performing a ‘scale reliability analysis’ in SPSS, which also can provide an overview of individual item-statistics; including potential α-values for the scale if an item is deleted. Field (2009, pp. 678-681) describes it as common research practice to remove such items if they drastically reduce reliability. Using this procedure, it was possible to ‘salvage’ to some extent the NML_RR scale by dropping one item, NML_RR01 (see Appendix B); thus increasing α-value from α = .371 to α = .464. This also had a positive impact on NML_ALL (α = .574). Luckily, there appears to be very decent inter-scale reliability scores for both dependent variables; CMQ (α = .835) and VCTE7 (α = .819). It is worth noting that a low alpha value is not

68 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world always synonymous with poor reliability, though, and there can be many reasons for α-values (Field, 2009, p. 675). As mentioned, having few items in a scale can be a reason for low scores without it necessarily meaning that a scale is unreliable. Possibly this could explain scores such as TVC2 (α = .502), NFC2 (α = .644) and MLOC3 (α = .544). Where applicable, scale reliability will be included in analyses and discussions.

4.2 Conspiracy theory endorsement and -mentality Level of conspiracy mentality within sample In order to understand the general level of conspiracy mentality within the sample and whether Figure 4.2.1 – CMQ distribution (N=125) there is a good balance of respondents who think differently, it is necessary to interpret some of the descriptive statistics for the CMQ which were presented in table 4.1.9. Using SPSS, a visual representation of the distribution in the form of a histogram is also generated (figure 4.2.1). Basic descriptive

CMQ statistics indicate that within the sample the average mean score is x̄ = 2,28 and the median = 2, indicating a slight positive skew. A visual inspection of figure 4.2.1 corroborates this. On the CMQ-scale a score of 3 is equivalent to a neutral response, whereas a lower score indicates higher levels of conspiracy mentality. The standard deviation is σ = .92, which suggests that few answers are concentrated much higher on the scale than slightly above a neutral 3, and that several could be close to 1. When interpreted along with a visual inspection, this shows that those low in conspiracy mentality (high scores) are more unevenly spread out, and many have even provided the lowest possible score. The mean is as therefore as high as it is due to a few very high values. Practically, the conclusion to these findings is that they suggest a moderate to high tendency towards conspiracy mentality within the sample.

Conspiracy theory endorsement within sample Conspiracy theory endorsement – as opposed to conspiracy mentality – is a very practical measure, and it is interesting to descriptively display how sample respondents have judged the likelihood of all the theories they were presented with. A table (4.2.2) was generated in SPSS, and it shows the sums and percentages of belief in every evaluated theory – as well as values for those who have no opinion or do not believe them to be likely. This was achieved by recalculating new variables based on the existing 1-5 Likert scale scores. Scores indicating high or moderate plausibility (1-2) were recoded into ‘believe’, and moderate or high non-plausibility scores (4-5) were recoded into ‘do not believe’. Neutral scores (3) remained ‘no opinion’-answers. It is important to note that the nuances in responses with regard to assessed degree of likelihood is neutralized by performing such a

69 Ronan Eric Reilly

recoding, but that as a Likert-scale variable used in Believe No opinion Do not believe further analyses all these values will remain. N Percent N Percent N Percent CIA MK Ultra (True) 66 52,8% 37 29,6% 22 17,6% The table makes for a lot of interesting reading. It Lightbulb cartel (True) 59 47,2% 38 30,4% 28 22,4% Elite few rule the world 49 39,2% 11 8,8% 65 52,0% is immediately encouraging to observe that the two Climate hoax 47 37,6% 8 6,4% 70 56,0% Vaccination cover-ups 32 25,6% 10 8,0% 83 66,4% ‘true’ conspiracies undoubtably are the most Russian surveillance (Fake) 29 23,2% 45 36,0% 51 40,8% endorsed; one just shy of 50% endorsement, and Area 51 25 20,0% 29 23,2% 71 56,8% 9/11 - Inside job 15 12,0% 23 18,4% 87 69,6% the other slightly over. In fact, these are also the Diana assassination 14 11,2% 30 24,0% 81 64,8% Fake Moon landing 8 6,4% 10 8,0% 107 85,6% only two theories that have more believers than Table 4.2.2 – Endorsement of various conspiracy theories (N=125) non-believers – and by far so. As an alarming contrast to this finding, though, close to 60 % of respondents do not blatantly reject the entirely fabricated ‘Russian radio surveillance’-conspiracy. In fact, almost a quarter of respondents (23,2%) actually find it likely, meaning it is more commonly endorsed than traditional classics such as ‘Area 51’, ‘9/11 inside job’ and ‘faked Moon landing’ – and only slightly less believed than ‘vaccination cover-ups’. Another important observation is how high the scores are for ‘climate change hoax’ and ‘elite few rule the world’ – where almost 40% of respondents assess these to be somewhat or very likely. Two thirds (66,4 %) of respondents answer that they think it is somewhat or very unlikely that vaccines do not contain harmful substances or can cause autism, and that this information is being withheld from the public. This means that almost one third are open to this being true, and a quarter of everyone actually find this to be likely. The only theory that is generally assessed to be unlikely by a major percentage of respondents is the ‘fake Moon landing’, but even this notion is considered debatable to almost 15%; nearly half of which find it to be likely. The general take-away point here is the overall large number of respondents who widely endorse one or more of the presented conspiracy theories. The premise set in the introduction of this thesis regarding the general commonness of endorsement therefore seems to also highly apply to the respondents in this sample. When additionally taking into consideration the high percentages who believe or have no opinion – meaning they do not necessarily deem them unlikely – it is apparent that potential societal challenges would be prevalent if this sample were considered generalizable.

Conspiracy mentality (CMQ) vs Conspiracy Theory Endorsement (VCTE) Chapter 2.1 elaborated on how a lot of research has suggested that a monological worldview and underlying dispositions can predict a stable tendency to endorse conspiracy theories; i.e. a conspiracy mentality. Some have however demonstrated that this is not necessarily always the case, and that circumstantial factors can be important. This possibility and lack of agreement is one of the reasons why this thesis has chosen to measure both conspiracy mentality and the actual endorsement of some conspiracies as separate constructs, and it is of major interest to assess the

70 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world relationship between these dependent variables. Although the CMQ-scale naturally has already been tested in relation to the endorsement of many conspiracy theories before – as this is how it was originally developed – the various VCTE-scales used here are not identical to the items used in those studies, and obviously neither are the respondents. It is still highly expected that there will be a statistically significant relationship between the CMQ and various variants of the VCTE-scale. In fact, it would potentially be considered problematic to the validity of this study if no significant relationships happened to be found. It is fruitful and interesting to compare all variants of the VCTE (the full 10 items, the 7 false items, the single fake item, and the 8 items including both the false and the fake; as well as the 2 true TVC2-items). Comparing all scale variants will possibly help determine whether there are differences in endorsement-levels based on having such a mentality no matter whether the conspiracy theory is false, true, or completely fabricated and previously unknown to respondents. Based on some of the literature presented in chapter 2.1 it is plausible that this will be the case. If so, it will be considered as support that one indeed can be predisposed to an acceptance of conspiracy claims in general. In order to determine the relationship between these variables the following hypothesis is tested:

Ho: There will not be a significant positive correlation between CMQ-scores and scores on any variants of the VCTE/TVC2 scales (ρ > .05). HA: There will be a significant positive correlation between CMQ-scores and scores on at least one variant of VCTE/TVC2 scales (ρ ≤ .05). Using SPSS to run a bivariate analysis of the Pearson correlations between the CMQ and the VCTE-variants does indeed reveal a fit for every variant – all highly significant at the 99% level (ρ < .01). See table 4.2.3 for an overview. The CMQ corresponds moderately high with VCTE10 (r = .651 ρ = .000) and VCTE7 (false) (r = .628 ρ = .000), as well as moderately with VCTE8 (false+fake) (r = .585 ρ = .000). As we can see there is also a highly significant, although quite a bit lower, correlation between CMQ and both the VCTE1 (fake) (r = .357 ρ = .000) and TVC2 (true) (r = .287 ρ = .001). This is interesting, as it can suggest that having a conspiracy mentality can indeed mean that one may tend to endorse conspiracy theories regardless of the circumstances and their possible level of verifiability and likelihood. The connection is however strongest in relation to endorsement of those which are false. The lowest correlation occurs with regard to the ‘true’ conspiracies; possibly suggesting that many without a high conspiracy mentality have critically evaluated the truthfulness or likelihood of these conspiracies over other conspiracy theories - i.e. those who either know about (or find plausible) that true conspiracies are likely will not necessarily

Table 4.2.3 – Correlations: CMQ vs VCTE / TVC2 variants (N=125) endorse all the other theories. It VCTE8 VCTE7 VCTE1 TVC2 VCTE10 (false+fake) (false) (fake) (true) also seems plausible that Pearson Correlation ,651** ,585** ,628** ,357** ,287** Sig. (2-tailed) ,000 ,000 ,000 ,000 ,001 substantially fewer than all of CMQ Spearman’s rho ,696** ,588** ,689** ,349** ,277** those who generally subscribe to Sig. (2-tailed) ,000 ,000 ,000 ,000 ,002 **. Correlation is significant at the 0.01 level (2-tailed). common conspiracy theories

71 Ronan Eric Reilly will find a new entirely random theory plausible just because they have a high conspiracy mentality. It may however appear that many actually do, based on findings in the previous section. Even though higher r values were anticipated, particularly for the VCTE7 (false), it is fair to claim that the null-hypothesis can be rejected, and that the CMQ and every variant of the VCTE variables, including TVC2, are significantly correlated. It is important, however, to note that the single fake item has methodological flaws as it in fact is a single Likert item – not a scale. Still, if one were to treat it as ordinal data, the non-parametric Spearman’s rho still yields equivalent values (rs = .349 ρ = .001). Comparing these results with findings in the previous section – which show a very different character to endorsement levels of the true conspiracies as oppose to the conspiracy theories – lead to an assumption that those who endorse conspiracies can do so for many other reasons than having a conspiracy mentality. As a consequence of all these findings, a choice is made to use the VCTE7 (only the false 7) as the dependent variable alongside the CMQ in further studies. Figure 4.2.4 illustrates visually how well they correspond with each other. This decision is made primarily to avoid ambiguity and confusion, as it seems plausible that additional or different Figure 4.2.4 – Scatterplot: CMQ vs VCTE7 (N=125) mechanisms could also lay beneath endorsement of both the 6,00 true conspiracies and the fake item. However, the TVC2 (true 5,00 verifiable conspiracies) will for comparison purposes be 4,00

separately included in further analysis and discussions – 3,00 VCTE7 although without interfering directly with the main hypotheses. 2,00 1,00 This will be done because it is of great value to the study to 0,00 differentiate between those who irrationally believe in 1,00 2,00 3,00 4,00 5,00 conspiracy theories, and those who possibly are just familiar CMQ with true conspiratory events.

4.3 A:News media trust, source preference, and literacy

The following sections provide analyses of RQ1 through RQ4 and discussions pertaining to the first main area of interest to this thesis: News media trust, source preference, and literacy.

RQ1: Mainstream media skepticism Firstly, the relationship between conspiracy theory endorsement, conspiracy mentality and individual assessment of mainstream media skepticism is investigated:

RQ1: What relationship exists between skepticism towards mainstream media and conspiracy theory endorsement / conspiracy mentality?

72 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Expectancies and hypotheses Based on the literature presented in chapter 2.2, it is expected that both having a high conspiracy mentality (low CMQ-score) and a high level of conspiracy endorsement (low VCTE7-score) will correlate with mainstream media skepticism. The higher an individual’s NCRED-score is the more skepticism towards the mainstream media they express. As high levels of conspiracy mentality and conspiracy theory endorsement are based on low CMQ and VCTE7 scores, a negative correlation is expected. Accordingly, the following hypothesis is formulated:

Ho: There will not be a significant negative correlation between NCRED-scores and CMQ- and/or VCTE7-scores (ρ > .05). HA: There will be a significant negative correlation between NCRED-scores and CMQ- and/or VCTE7-scores (ρ ≤ .05). Please refer to Appendix D, as well as table 4.1.9, for relevant statistical operations performed as a basis for analysis, as well as visual representations in the form of scatterplots. Analysis Firstly, an assessment of the central tendency for this dataset reveals a highly centralized mean response (x̄ = 3.110), however, with one of the largest standard deviations out of any of the scales (σ = 1.087). This can be taken to indicate that respondents do not largely agree on their evaluation of media credibility – not that most respondents’ assessment is neutral. Rather, it is likely that the questionnaire has captured an even number of respondents who have opposing views. An analysis of bivariate correlation in SPSS generates output that suggests rejection of the null hypothesis, as a very significant – yet moderate and similar – negative relationship exists between both CMQ and NCRED (r = -.496 ρ = .001), as well as VCTE7 and NCRED (r = -.512 ρ = .001). Even TVC2 (r = -.207 ρ = .021) is significantly correlated, however this relationship is quite low and less significant. Visual inspections of scatterplots correspond with these findings.

Discussion The descriptive data gathered by the N=125 sample seems to coincide quite poorly with the 2019 poll by ‘Medietilsynet’ presented in 2.2, where eight out of ten respondents felt that Norwegian media are generally trustworthy. As such, the sample would most likely not be a good representation of the general public opinion in Norway. Irrespective of this, the difference in answers provide a decent foundation upon which to evaluate the main hypothesis. The analysis shows, as expected, that there are highly significant relationships between being skeptical towards mainstream news and both endorsement of conspiracy theories and having a conspiracy mentality in general; although the strength of the relationships (approx. r = -.5) are only considered moderate. In chapter 2.2, media skepticism was defined as a subjective feeling of alienation and mistrust toward the mainstream news media. It is not unlikely that e.g. distrust in authorities and feelings of powerlessness – as shown in chapter 2.1 to be associated with conspiracy belief – potentially could account for such skepticism. These findings cannot conclusively prove this to be the reason, but

73 Ronan Eric Reilly the investigation can conclude that there seems to be a connection between subjective feelings of alienation and mistrust toward the mainstream news media and both the tendency to view events and circumstances as the product of conspiracies – in addition to endorsing actual conspiracy theories. Comparing these finding to those of much lower strength and significance found in relation to the ‘true verifiable’ conspiracies is also interesting. It could seem that less rational beliefs (i.e. conspiracist beliefs) are more associated with news skepticism than more rational ones (i.e. knowledge about actual conspiracies). It is only natural that those who are critical enough to be aware of MK Ultra and Phoebus Cartel conspiracies would also display some healthy skepticism towards mainstream media, and it is not necessarily ‘wrong’ or ‘irrational’ to be a little bit skeptical of mainstream news when considering the possibility of e.g. viewpoint homogenization, ownership- issues or commercial gains aspects, as discussed in chapter 2.2. News skepticism could up to a point be considered a strength in terms of critically assessing information. High levels of skepticism are not necessarily rational, though, and neither is the endorsement of conspiracy theories.

RQ2: Frequency of reported alternative media usage Being skeptical towards mainstream media often may imply placing trust elsewhere, e.g. in various alternative sources. The first of two research questions pertaining to such sources is:

RQ2: What relationship exists between frequency levels of alternative media consumption and conspiracy theory endorsement / conspiracy mentality?

Expectancies and hypotheses Based on reasons viewed in chapter 2.2, it is expected that those who report reading alternative media often will be more likely to have a conspiracy mentality and endorse conspiracy theories than those who read alternative media seldom or never. Therefore, the following is hypothesized:

Ho: There will not be a significant difference in the relationship between mean scores of CMQ /VCTE7 and reported frequency in which alternative media sources are consumed (ρ > .05). HA: There will be a significant difference in the relationship between mean scores of CMQ /VCTE7 and reported frequency in which alternative media sources are consumed (ρ ≤ .05). The statistical material and necessary calculations performed in SPSS to answer this are available in Appendix E, as well as in table 4.1.9.

Analysis As presented in 3.5, the variable asking “How often do you read news from a media source you consider to be alternative?”, with eight frequency options, was recoded into three alternatives in order to more purposefully perform the analysis. Resulting categories were: ‘at least once a day’, ‘a few days a week’ and ‘monthly or rarer’. ‘Don’t know’-answers were disregarded. Based on these three alternative media consumption frequency conditions, a one-way between answers ANOVA is

74 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world conducted to compare their effect on conspiracy theory endorsement and -mentality. There is a significant effect of reported alternative media consumption frequency at the ρ < .0 05 level on both CMQ [F(2, 118) = 3.74, ρ = 0.027] and VCTE7 [F(2, 118) = 7.96, ρ = 0.001] for the three conditions. TVC2 does not produce a significant effect [F(2, 118) = 0.564, ρ = 0.570]. Post hoc comparisons using the Tukey HSD test indicate, in relation to CMQ, that the mean score for the ‘at least once a day’ condition (x̄ = 1.93, σ = 0.79) is significantly different than the ‘monthly or rarer’ condition (x̄ = 2.49, σ = 0.93). The ‘a few days a week’ condition (x̄ = 2.26, σ = 0.97) does not significantly differ from the ‘at least once a day’ and ‘monthly or rarer’ conditions. Similarly, in relation to VCTE7, the mean score for the ‘at least once a day’ condition (x̄ = 3.34, σ = 0.97) is significantly different than the ‘monthly or rarer’ condition (x̄ = 4.12, σ = 0.75). As with the CMQ, however, the ‘a few days a week’ condition (x̄ = 3.73, σ = 0.96) does not significantly differ from the ‘at least once a day’ and ‘monthly or rarer’ conditions. It is necessary to note that all the assumptions required to conduct parametric tests are not technically met for all variables. Shapiro-Wilks tests and boxplot inspections (see appendix E) reveal non-normal distributions for most conditions. All compared groups are N > 30, but as it is a non-probability sample the normality assumption cannot be assured. Lavene’s statistics do suggest equal variance between all tested groups, and the homogeneity of variance assumption is met. In sum, the assessment is that enough information is present to allow rejection of the null hypothesis.

Discussion Taken together, these results suggest that those who report consuming alternative media at least once a day have a higher likelihood of both having a conspiracy mentality and endorsing common conspiracy theories than those who consume it less than monthly or never. However, it should be noted that consumption must be very frequent in order to see an effect. Consuming alternative media a few days a week does not appear to have any significant relationship with neither endorsement nor mentality. Therefore, reading the occasional alternative article does not appear to play an important role in peoples’ endorsement of conspiracy theories. However, those who have a clear alternative preference, and hence possibly avoid a lot of mainstream media, seemingly do. Interestingly, no significant effects were found at all when comparing to TVC2, suggesting that no frequency of alternative media consumption is indicative of endorsing ‘true’ conspiracies, but only false ones. These findings are in line with expectancies.

RQ3: Attitudes towards alternative media The other variable pertaining to alternative media deals with how respondents view such sources, irrespective of any academic definitions:

75 Ronan Eric Reilly

RQ3: What relationship exists between different attitudes towards alternative media sources and conspiracy theory endorsement / conspiracy mentality?

Expectancies and hypotheses As explained in 3.5, this study has chosen to infer attitudes regarding alternative media based on survey respondents’ answers to the question inquiring about what ‘alternative media’ means to them. Based on answers, a categorization was made between those who are considered highly positive/sympathetic, those who reflect a mildly sympathetic view, those who suggest having a negative/dismissive attitude, and those who seem to be neutral/indifferent. It is expected that there will be a statistically significant difference in relation to both conspiracy mentality and conspiracy theory endorsement between those who hold negative attitudes regarding alternative media in comparison with those who are positive. High levels of conspiracy mentality (low CMQ-score) and a high level of endorsement (low VCTE7-score) are both expected to be found in those who are positive – and vice versa. The following is hypothesized:

Ho: There will not be a significant difference in mean scores of CMQ or VCTE7 depending on answers suggesting positive vs negative attitudes towards alternative media (ρ > .05). HA: There will be a significant difference in mean scores of CMQ or VCTE7 depending on answers suggesting positive vs negative attitudes towards alternative media (ρ ≤ .05). Please refer to Appendix F, as well as chart 4.1.6 and table 4.1.9, for relevant statistics. Analysis Another one-way between answers ANOVA is conducted to compare the effect of alternative media attitudes on conspiracy mentality and conspiracy theory endorsement based on the four differing attitudes. Again, there is a significant effect of these given opinions/attitudes towards alternative media sources at the ρ < .0 05 level on both CMQ [F(3, 109) = 4.42, ρ = 0.006] and VCTE7 [F(3, 109) = 8.70, ρ = 0.000] for the four conditions. TVC2 does not produce a significant effect [F(3, 109) = 0.77, ρ = 0.515]. Post hoc comparisons using the Tukey HSD test indicate, in relation to CMQ, that the mean score for the negative “Politically biased news with questionable journalism” condition (x̄ = 2.67, σ = 0.91) is significantly different from both the positive “Ambitious and brave news sources, who dare to tell the truth” condition (x̄ = 1.96, σ = 0.95), and the mildly sympathetic “News sources who have lost faith in mass media, and feel a need to separately investigate matters” condition (x̄ = 2.08, σ = 0.81). However, the neutral “Niche news without any particular agenda” condition (x̄ = 2.07, σ = 1.06) does not significantly differ from any of the other conditions. Also, in relation to VCTE7, the mean score for the ‘negative’ condition (x̄ = 4.29, σ = 0.67) is significantly different from both the ‘positive’ condition (x̄ = 3.27, σ = 1.28), and the ‘mildly sympathetic’ condition (x̄ = 3.60, σ = 0.80). However, as with the CMQ, the ‘neutral’ condition (x̄

76 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

= 3.69, σ = 0.57) does not significantly differ from any of the other conditions. Boxplot inspections and uneven Shapiro-Wilks-values in relation to many group pairings suggest a violation of the normality assumption for several conditions. Due to there both being fewer than 30 cases in some of these pairings and the non-probability nature of the sample, this could potentially be problematic. For the CMQ the Levene’s test suggests fulfillment of the criteria for homogeneity of variance (ρ = .538), however this assumption fails for both VCTE7 (ρ = .002) and TVC2 (ρ = .035). The findings allow for the null hypotheses to be rejected and the alternative kept, although this is only barring acceptance of these potential assumption-issues.

Discussion Viewing this all together, the results suggest that those who are both moderately and highly positive/sympathetic towards alternative media have a higher likelihood of both having a conspiracy mentality and endorsing common conspiracy theories than those who are negative/dismissive. Viewing alternative media neutrally does not appear to significantly affect either variable. Furthermore, the fact that the same findings are significant for both CMQ and VCTE7 further strengthens the connection between conspiracy mentality and endorsement. In addition, yet again, it is interesting to observe that the ‘true verifiable’ (TVC2) conspiracies are not significantly related to alternative media attitudes either. Broadly, findings indicate that a significant relationship exists both between a moderately and highly positive reported attitude regarding alternative media. When viewing this together with findings from RQ2 regarding frequent consumption, all findings align with literature presented in both chapters 2.1 and 2.2 and are expected. It is however worth calling attention to a weakness that is discovered and acknowledged in hindsight regarding the answer which was meant to reflect a mildly sympathetic view: “News sources who have lost faith in mainstream media, and feel a need to separately investigate matters”. A realization is made that this answer is not formulated in a way that can ensure isolation of a specific attitude, as – even if it is unlikely – it is entirely possible that it could be given by someone who understands alternative media’s motivation, yet still does not approve of their agenda. It is however quite unlikely that e.g. an answer like “Ambitious and brave news sources, who dare to tell the truth” reflects anything other than a highly positive/sympathetic view; although it is important to point out that this is only an assumption. A potential weakness worth making a note of regarding this option, though, is the fact that there are only 11 responses upon which to draw conclusions. If one chooses to accept the findings, they do however align very well with the perceived ‘truth-seeking’-tendencies associated with conspiracy mentality. Another observation worth pointing out, is that even the respondents who have a negative view of alternative media still have an average mean conspiracy mentality that is higher than neutral

77 Ronan Eric Reilly

(x̄ = 2.67, σ = 0.91). In relation to the samples’ overall positive skew this is not very surprising, but it would have been less surprising if the mean averaged at least a little higher than a neutral 3. The difference in relation to those who are positive to alternative media is nevertheless still substantial, which a mean below 2 (x̄ = 1.96, σ = 0.95) indeed indicates. Those who do not rate alternative media highly display a clearly lower conspiracy theory endorsement rate too (x̄ = 4.29, σ = 0.67) in relation to those who do (x̄ = 3.27, σ = 1.28). The mean score of the latter is not alarmingly high, although the high standard deviation reveals that those who endorse very many conspiracy theories are seemingly much more represented in the ‘positive’ group. It would however not appear that just because one rates alternative media positively it means that one necessarily will endorse the most conspiracy theories, as a mean above a neutral 3 suggests. It is however hard to determine a concrete pattern, as the high standard deviation indicates that many scores differ quite a bit from the mean. It seems, though, that those who the ‘positive’ group can be both highly dismissive and highly accepting of conspiracy theories while the ‘negative’ group are stably dismissive.

RQ4: News media literacy

Being highly news media literate can be important when choosing sources, and it can aid in the avoidance of misinformation. As extensively explored in chapter 2.2, the concept of ‘news media literacy’ is treated as a complicated multidimensional construct. The fourth research question attempts to break it down, originating with a simple question:

RQ4: What relationship exists between news media literacy levels and endorsement of conspiracy theories / conspiracy mentality?

Expectancies and hypotheses To approach this question three sub-hypotheses are necessary. The first involves the complete ‘news media literacy’ (NML) scale and its three sub-scales measuring knowledge about ‘authors and audiences’ (AA), ‘messages and meanings’ (MM), and ‘representations of reality’ (RR). The second and third hypotheses address the ‘need for cognition’ (NFC2) and ‘media locus of control’ (MLOC3) scales, respectively. In each scale a low score indicates a high level; e.g. a low score on the NML-scale signifies a high level of knowledge about the news system. As CMQ- and VCTE7- scales also are scored in this direction, negative correlations are expected, because it is anticipated – based on the three components in Craft et al. (2013)’s ‘news media literacy model’ (figure 2.2.1) – that high levels of media system knowledge, a high need for mindful thought processing, and a high need to be in control of news media’s influence will all be associated with low levels of conspiracy theory endorsement and -mentality. The first hypothesis is comprised of four different components, and seeks to investigate the ‘knowledge about the news media system’-aspect of the NML model:

78 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

H1o: Scores on the complete NML scale, or any of its subsections (AA, RR, MM), will not be significantly negatively correlated with CMQ- or VCTE7-scores (ρ > .05). H1A: Scores on the complete NML scale, or any of its subsections (AA, RR, MM), will be significantly negatively correlated with CMQ- or VCTE7- scores (ρ ≤ .05). The next hypothesis pertains to the ‘mindful thought processing’ aspect of the NML model:

H2o: Scores on the NFC2 scale will not be significantly negatively correlated with CMQ- or VCTE7-scores (ρ > .05). H2A: Scores on the NFC2 scale will be significantly negatively correlated with CMQ- or VCTE7-scores (ρ ≤ .05). The last aspect of the ‘news media literacy’-model – the degree of being ‘in control of news media’s influence’ – is measured by the ‘media locus of control’-scale:

H3o: Scores on the MLOC3 scale will not be significantly negatively correlated with CMQ- or VCTE7-scores (ρ > .05). H3A: Scores on the MLOC3 scale will be significantly negatively correlated with CMQ- or VCTE7-scores (ρ ≤ .05). As usual, an Appendix (G) is generated containing the necessary statistical material and visual aids upon which to perform the analysis. In addition, table 4.1.9 provides relevant scale data.

Analysis H1: Calculated bivariate correlations between NML-variants and both the CMQ and VCTE7 show surprisingly weak and insignificant relationships. The complete NML scale, as well as the RR subscale, do however reveal weak negative significant findings in relation to VCTE7, but not CMQ. In relation to the CMQ, NML_ALL (r = .043 ρ < .632), AA (r = .114 ρ = .206), MM (r = -.061 ρ = .500), and RR (r = -.115 ρ = .202). For the VCTE7, similar findings are revealed for AA (r = -.073 ρ = .418) and MM (r = -.142 ρ = .113). Weak significant findings are however found for NML_ALL (r = -.203 ρ = .023) and RR (r = -.226 ρ = .011). An additional finding that is worth making a note of, however weak it also may be, is the significant positive correlation between TVC2 and both NML_ALL (r = .205 ρ = .022) and RR (r = .209 ρ = .019). In sum, only weak partial support for the alternative hypothesis (H1A) is found, but it is still sufficient to be able to reject the null hypothesis (H1o). H2: There does not appear to be any significant correlation between the NFC2 and either the CMQ (r = .087 ρ = .335) or the VCTE7 (r = -.015 ρ = .868). In fact, an additional visual assessment of scatterplots suggests an almost complete lack of relationships. The null hypothesis (H2o) is therefore retained. Additionally, the NFC2 seems to be unrelated to TVC2 (r = .109 ρ = .228). H3: Again, there seems to be no correlation of significance between the MLOC3 and either the CMQ (r = -.138 ρ = .125) or the VCTE7 (r = -.132 ρ = .141). It is worth making a note of the fact that the non-parametric Spearman’s rho does suggest a slight significant relationship with the

VCTE7 (rs = -,204 ρ = .022), and the contours of a visually detected scatterplot-pattern do seem to allow for this. The null hypothesis (H3o) is nevertheless retained also in this circumstance, as this is not deemed an adequately strong finding to adapt the alternative. The MLOC3 seems to be unrelated to TVC2 (r = .049 ρ = .589) also.

79 Ronan Eric Reilly

Discussion Initially, it is worth pointing out that mean scores for all five applicable scales are close to 2, suggesting that there is an overweight of respondents within the sample that display high levels of news media literacy. In addition, these scores suggest that more respondents claim to have a tendency towards engaging in activities that require cognitive effort, and that the majority feel in control of their media influence. Based on the results from testing these three hypotheses, there does not appear to be a particularly interesting relationship to be found between almost any aspect of news media literacy and neither conspiracy mentality nor conspiracy theory endorsement. With slight exceptions, most levels of understanding of how the news media works – as well as the need for mindful thought processing, or any perceived level of being in control over influence from the media – seem to play a significant role within the sample in relation to who will tend to generally believe or endorse such theories. These findings are extremely surprising when considering the presented previous research in chapter 2.2, suggesting a connection between news media literacy and conspiracy theory belief. To be fair, it was pointed out that Craft et al. (2017) also failed to make connections between conspiracy theory endorsement and both the ’media locus of control’ and ‘need for cognition’ aspects. As such, findings are in line with some of the previous research. The fact that neither a high need for mindful thought processing nor a high need to be in control of news media’s influence correlates with a tendency to endorse conspiracy theories remains a surprise irrespectively. It was pointed out in chapter 2.1 that many conspiracy theorists see themselves as brave and cognitively active ‘truth seekers’ who are in control of the media information they wish to get. They may therefore not passively be influenced into believing in such theories. This interpretation could have been feasible, but one would possibly then have expected significant findings of an opposite nature. In relation to the NML scale (‘knowledge about the news media’), e.g. Craft et al. (2017) previously did find that it was less likely that one would believe in conspiracy theories if one knew a lot about the news media itself, as well as the kinds of news that are commonly covered, commercial contexts, and the effects news can have on public opinion. These results do not seem to replicate in this study as clearly. Importantly, though – as previously pointed out – a different means of measurement was used here. The Authors and Audiences (AA) subscale measured respondents’ basic understanding of how news providers and consumers interact, and the Messages and Meanings (MM) subscale attempted to capture how meanings are created based on conveyed information. Knowledge about either of these mechanisms are seemingly not related to conspiracy mentality or conspiracy theory endorsement, though. Representations and Reality (RR) – pertaining to the filtering of information in the media and how this affects perceptions of reality – did to a

80 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world minor degree, however, appear to significantly correlate with VCTE7. It is nevertheless important to interpret this result with extreme caution, as not only is the relationship quite weak (r = -.226), but the reliability of the RR scale is at best questionable (α = .464); a value which was achieved only after removing an item to boost its score from the mid-300’s. In addition, this scale was modified heavily from the original NML_RR scale, so there is a high probability of validity issues. If one were to accept these methodological weaknesses, one could conclude that there seems to be somewhat of a connection between endorsement of conspiracy theories (although not having a conspiracy mentality) and a lack of understanding of how the media can affect perceptions of reality through filtering or omitting information; e.g. overexposing conflicts or setting agendas which can cause overestimating the importance of particular stories in relation to others. The fact that only the RR subscale produces significant results highlights the value of separating out specific dimensions of news media literacy. This is particularly evident when considering that the NML-scale as a whole – if not separated into subscales – seemingly also produces a significant relationship with the VCTE7. It is, however, presumable that only the RR items are responsible for this, and that not splitting up the scale would have misleadingly suggested that AA and MM items also were of significant relevance to conspiracy theory endorsement. Another interesting take-away point, which stands in contrast to this negative relationship with conspiracy theory endorsement, is the positive correlation between ‘true verifiable conspiracies’ (TVC2) and the same RR-subscale (as well as the complete NML-scale as a whole also). This could suggest – at least to a very mild extent – that there conversely is a vague relationship between understanding some of the same aspects regarding how the news media can influence the public, and the ability to identify verifiable conspiracies over conspiracy theories.

4.4 B: Critical thinking dispositions and skills The previous section explored the first part of this thesis’ main two areas of investigation, dealing with media trust, preference, and news media literacy; the latter of which yielded some strange results. The concept of ‘critical thinking’ has been theorized to be highly connected to such literacy, although the literature has only described this somewhat generally. The following section deals with some specific critical thinking constructs; as per the second area of interest: critical thinking dispositions and skills. A closer look will be taken at variables pertaining to critical thinking disposition (RQ5), susceptibility towards informal logical fallacies (RQ6), ability to detect pseudo- profound bullshit (RQ7), and source criticism ability (RQ8).

RQ5: Critical thinking disposition

The first of these questions seeks to investigate general disposition, and asks the following:

81 Ronan Eric Reilly

RQ5: Is there a relationship between individual levels of critical thinking disposition and endorsement of conspiracy theories / conspiracy mentality?

Expectancies and hypotheses It is expected that those who have high levels of critical thinking disposition endorse fewer conspiracy theories and have a lower conspiracy mentality than those with low levels. Sosu (2013)’s ‘critical thinking disposition scale’ (CTDS) – including its two subcomponents ‘critical openness’ (CO) and ‘reflective skepticism’ (RS) – provide the basis for measuring this construct. As explained, Sosu (2013) developed value keys for his scale, and these have been computed (refer to Appendix B for specific values). The CTDS and its subsections are therefore grouped into ordinal ‘low’, ‘moderate’, or ‘high’ categories, as opposed to comparing arithmetic means as in most of the Likert-scales. The hypothesis formulated is therefore:

Ho: There will not be a significant difference in mean scores of CMQ or VCTE7 regardless of scores levels on the complete CTDS scale, or any of its subsections (CO, RS) (ρ > .05). HA: There will be a significant difference in mean scores of CMQ or VCTE7 depending on score levels on the complete CTDS scale, or any of its subsections (CO, RS) (ρ ≤ .05). Appendix H, as well as table 4.1.9 contains relevant statistics.

Analysis Because the resulting data is grouped, it is again appropriate to use one-way ANOVA’s to compare the effect of critical thinking disposition levels on conspiracy theory endorsement and -mentality for all three conditions (low, moderate, and high) in all three scales. For the complete CTDS-scale no significant effect is found on either CMQ [F(2, 122) = 1.65, ρ = .197] or VCTE7 [F(2, 122) = 1.81, ρ = .169] for the three conditions. TVC2 produces very different values, however none of which are significant either [F(2, 122) = 0.23, ρ = .798]. For the ‘critical openness’ subscale of the CTDS no significant effect is found on either CMQ [F(2, 122) = 0.12, ρ = .884] or VCTE7 [F(2, 122) = 2.27, ρ = .108] for the three conditions. Again, TVC2 also produces highly insignificant values [F(2, 122) = 0.28, ρ = .760]. Finally, the ‘reflective skepticism’ subscale does not differ in its results from the two previous ones either, as again both CMQ [F(2, 122) = 1.55, ρ = .216] and VCTE7 [F(2, 122) = 0.31, ρ = .732] yields no significant effect for the three conditions. TVC2 produces no significant values this time either [F(2, 122) = 2.53, ρ = .084]. Again, assumptions of normality and homogeneity of variance are not met for all conditions in any of the scales. However, it is unlikely that unmet assumptions are causing a lack of significant findings. In conclusion, the null hypothesis is supported on all counts.

Discussion Findings based on Sosu’s scale(s) suggest that no levels of critical thinking disposition, critical

82 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world openness, or reflective skepticism seem to have any significant relationship with levels of conspiracy theory endorsement or -mentality. This means that e.g. generally considering the wider implications of actions, critically evaluating new ideas, or adjusting thinking tendencies in the face of convincing evidence theoretically should have no bearing on levels of conspiracy belief. None of these qualities appear to be linked with knowledge about true conspiracies either. As with many of the news media literacy measures, these finding are extremely surprising because conspiracy theories are in many ways thought to be the product of flawed reasoning. The literature has not revealed the use of this particular scale for comparing similar constructs in the past. Regardless, the weight of related research covered extensively in chapters 2.1 and 2.3 of the literary review – suggesting a connection with at least conspiracy theory endorsement – does not fit well with these results, and neither does this thesis’ general expectancies. Looking behind the results by observing mean scores, standard deviations, and frequencies of responses per category may be fruitful when facing such unexpected findings. Mean scores are approximately 3,6 for all scales, and standard deviations lie between σ = 0,42 and σ = 0,65. This suggests that very many responses are close to the neutral option or slightly above. The ANOVA measures the means of grouped values, and the frequencies of responses within each group reflect these statistics (as depicted graphically in figure 4.4.1). There is a real chance that since a very high number of respondents in the sample are in the ‘moderate’-category, the amount of ‘high’ and ‘low’ respondents versing each other may not be enough to yield fruitful results. As with the NML, it is also highly important to bear in mind the abysmal Cronbach’s alpha scores (e.g. CTDS_CO α = .392; CTDS_RS α = .339) before reading too much into these findings. For whatever reason, it appears likely that Sosu’s scale is internally inconsistent and unreliable in relation to this sample. Possibly, reverse coding endeavors or language translation could be at fault for this. It also seems quite unlikely that e.g. only 14 out of 125 respondents would fall in the ‘low’-category for the complete CTDS-scale, and this might not have happened if other measures were used to assess this construct. It is also strange that more respondents did not give more socially desirable answers or overrate themselves and end up in the ‘high’ category. Introductorily it was pointed out that ‘anyone’ can end up endorsing conspiracy theories.

These results – providing one relies Figure 4.4.1 – Ratio of responses to CTDS, CO and RS scales (N=125) on their accuracy – certainly suggest that conspiracy theories can be freely endorsed regardless of how much of a disposition towards critical thinking CTDS_ALL CTDS_CO someone has. CTDS_RS

83 Ronan Eric Reilly

RQ6: Susceptibility to informal logical fallacies The next research question attempts to evaluate conspiracy mentality and conspiracy theory endorsement in relation to the first of three more practical critical thinking skills:

RQ6: Is there a relationship between susceptibility to arguments based on informal logical fallacies and endorsement of conspiracy theories / conspiracy mentality?

Expectancies and hypotheses This question involves the self-created ‘informal logical fallacy arguments’-scale (ILFARG). As an arithmetic mean interval scale, the ILFARG is scored in such a way that low values represent the highest degree of incorrect assessment, and as such it is expected to correlate positively with CMQ and VCTE7. This is because having a high susceptibility to informal logical fallacies is likely to coincide with having a high degree of conspiracy theory endorsement; and possibly also conspiracy mentality. The following is hypothesized:

H1o: There will not be a significant positive correlation between ILFARG-scores and CMQ- and/or VCTE7-scores (ρ > .05). H1A: There will be a significant positive correlation between ILFARG-scores and CMQ- and/or VCTE7-scores (ρ ≤ .05). Please refer to Appendix I and table 4.1.9 for statistics and visual representations.

Analysis It can be read from the bivariate analysis that the ILFARG-scale correlates very significantly with both the CMQ (r = .278 ρ = .002) and VCTE7 (r = .571 ρ = .000). Although significant, the CMQ only suggests being mildly correlated, however the VCTE7 is moderately high. This allows for a rejection of the null hypothesis. There is no significant result found between the ILFARG and TVC2 (r = .117 ρ = .193). Visual inspections of scatterplots correspond with these findings.

Discussion These findings suggest, as expected, that there indeed is a connection between susceptibility to informal logically fallacious argumentation and both conspiracy mentality and conspiracy theory endorsement; especially with regard to the latter. As a counter to this, there does not appear to be any obvious connection between endorsement of true conspiracies and such argumentation. These findings seem clear, but it is important to interpret them with caution; primarily because the scale has not been properly verified and tested to determine whether it is validly capturing the intended construct. For a prototypical scale, though, an alpha value of α = .672 is not too bad. Such a value could suggest that it somewhat reliably measures ‘something’. Still, it is possible that underlying attitudes may have created severe bias, as some of the items are heavily opinionated. Although the phrasing of the scale-question encouraged respondents to disregard their own feelings or views when assessing the arguments, some respondents may not have been able to overcome underlying attitudes. This could cause major validity issues, as the scale e.g. might potentially be measuring

84 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world more of an ‘inability to see beyond one’s own opinion’ than the intended construct of susceptibility towards the fallacies themselves. This would be a substantial weakness, as there possibly would be very different results if sentences containing the same fallacies, but with other topics, were presented. Although a selection of very diverse subjects has been attempted, there is a prevalence of arguments influenced by e.g. spiritual, alternative medical or politically radical views. Such views are heavily linked with conspiracy theory belief in the literature, and as such the scale could be accidentally capturing coinciding beliefs. Paradoxically, logical fallacies may be at the core of all these beliefs, and therefore the scale could also – as intended – be capturing just that. There are many fallacious scale-items which are not as typically associated with such beliefs, though, and it does work in the scale’s favor that it has a relatively high alpha value. Another possibility is that the scale is measuring a tendency to ‘not think thoroughly through things’ before making judgements – i.e. reflexively thinking. Nevertheless, the difference in significance seen here between the ILFARG and the VCTE7 and TVC2, respectively, is quite striking. Regardless of what construct the scale is measuring it would seem that those who endorse many conspiracy theories are quite highly prone to it, while there is no clear connection to those who acknowledge verifiable conspiracies. For the purpose of this research, a choice is made to assume that the scale measures what it is supposed to, and therefore the findings are taken to suggest that accepting flawed argumentation - which is at the root of conspiracy theories in general – is unsurprisingly linked to their endorsement. Conspiracy mentality also significantly correlates to a slight degree, but this could potentially be more of a spurious relationship; although it is possible that illogical interpretation of arguments could result in-part from a way of interpreting information based on how one sees the world. While a lot of literature reviewed in chapters 2.1 and 2.3 point to this possibility, not enough information is available to draw this conclusion specifically. Given the fact that higher ILFARG-scores represent a lower susceptibility towards informal logical fallacies, the mean score (x̄ = 3.65) suggests that the average sample respondent tends to detect more fallacies than not. A standard deviation of σ = .61 further reveals that very few score below a neutral 3, and that Correctly assessed No opinion Incorrectly assessed

N Percent N Percent N Percent many score around 4; Anecdotal evidence / Hasty generalization 109 87,2% 5 4,0% 11 8,8% suggesting that respondents Argumentum ad populum 108 86,4% 9 7,2% 8 6,4% False dichotomy 103 82,4% 10 8,0% 12 9,6% largely do not fall for many of Strawman 92 73,6% 15 12,0% 18 14,4% Non sequitur (formal) 84 67,2% 17 13,6% 24 19,2% the fallacies. A further Appeal to nature 81 64,8% 18 14,4% 26 20,8% Ad-Hominem 79 63,2% 16 12,8% 30 24,0% investigation of this data is Appeal to age / Appeal to false authority 71 56,8% 29 23,2% 25 20,0% fruitful, and a descriptive ‘Fallacy’ fallacy / Non sequitur 66 52,8% 9 7,2% 50 40,0% Burden of proof fallacy 31 24,8% 29 23,2% 65 52,0% overview over the entire Faulty comparison 29 23,2% 9 7,2% 87 69,6% Table 4.4.2 – Overview over how respondents assessed arguments (N=125)

85 Ronan Eric Reilly samples’ correctly or incorrectly assessed arguments is presented in table 4.4.2. The table only refers to these arguments by their categorized fallacy. For a complete overview of the full items/arguments, including the four non-fallacious and reasonable ‘decoy’ -items, please refer to Appendix B. This overview was created in SPSS by recoding Likert-ratings indicating that a fallacious argument was ‘poor’ (3 and 4) into ‘correctly assessed’, and that it was ‘strong’ (1 and 2) into ‘incorrectly assessed’; while 3 remained ‘no opinion’. The four non-fallacious ‘decoy’-items were excluded from the analysis. From this table we can read that many of the fallacious arguments presented are in fact assessed incorrectly as being either very or somewhat strong arguments. Certain fallacies stick out as being particularly prone to susceptibility, including the ’false analogy’ (69,6%), ‘burden of proof’ (52 %) and ‘fallacy fallacy’/’non sequitur’(40%), with many more scoring around 15-25 %. Assuming these findings could be accurately replicated, a devious journalist or media worker certainly would be able to draw on this to influence readers. Conversely, a journalist wishing to reduce the spread of misinformation would know to be careful of accidentally committing fallacies him-/herself. Either way, there does appear to be a clear relationship between susceptibility to arguments based on informal logical fallacies and both conspiracy theory endorsement and -mentality; but particularly on endorsement.

RQ7: Pseudo-profound bullshit detection It is further interesting to explore if similar relationships can apply to detection of pseudo-profound bullshit, which is the next skill to be investigated:

RQ7: Is there a relationship between failing to detect ‘pseudo-profound bullshit’ and endorsement of conspiracy theories / conspiracy mentality?

Expectancies and hypotheses It is expected that conspiracy belief is associated with acceptance of pseudo-profound bullshit. The primary aim is to measure ‘bullshit receptivity’ (BSREC4), which is hypothesized to correlate positively with CMQ and VCTE7. This is because low scores on BSREC4 indicate high levels of meaningfulness-assessment of ‘bullshit’ statements, while low CMQ/VCTE7 scores are indicative of a high conspiracy mentality and endorsement-rate. Therefore:

H1o: There will not be a significant positive correlation between BSREC4-scores and CMQ and/or VCTE7-scores (ρ > .05). H1A: There will be a significant positive correlation between BSREC4-scores and CMQ and/or VCTE7-scores (ρ ≤ .05). As elaborated upon, ‘profundity reception’ (PREC4) is also tested as a secondary measure. To help ensure reliability and validity of the BSREC4, findings hope to indicate that PREC4 is either negatively correlated with CMQ / VCTE7, or that correlations are insignificant. This would suggest (if negative) that one either judges profoundness as meaningless if one has conspiratorial beliefs, or (if insignificant) that everyone is just as likely to detect (or fail to detect) profoundness – whether

86 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world they endorse conspiracy theories or not. Such results would possibly indicate that BSREC4-findings are unique to those who endorse conspiracy theories. In addition, the ability to recognize profoundness – or at least some meaning – is expected to be a trait held by those who endorse few conspiracy theories and are less conspiracy-minded. This would be supported if PREC4 happens to be positively correlated with TVC2. The main investigation of interest is however:

H2o: There will be a significant positive correlation between PREC4-scores and CMQ and/or VCTE7-scores (ρ ≤ .05). H2A: There will not be a significant positive correlation between PREC4-scores and CMQ and/or VCTE7-scores (ρ > .05). SPSS provides statistics that can be found in Appendix J and table 4.1.9.

Analysis H1: There appears to be significant, albeit quite moderate, correlations between BSREC4 and both CMQ (r = .362 ρ = .000) and VCTE7 (r = .403 ρ = .000). Therefore, the first null hypothesis (H1o) can be rejected fully. The TVC2 (r = .138 ρ = .125) shows no such relationship. H2: In relation to the PREC4, no significant correlations exist between it and CMQ (r = .136 ρ = .131) or VCTE7 (r = .059 ρ = .514). There is not even an insignificant negative tendency. Accordingly, the second null hypothesis (H2o) is also rejected. In addition, TVC2 and PREC4 significantly correlate mildly positive (r = .324 ρ = .000). Again, scatterplots collaborate all these findings upon visual inspection.

Discussion These findings are consistent with those revealed by Pennycook et al. (2015), and suggest that – within the sample – there is a relationship between having a conspiracy mentality, as well as endorsing several conspiracy theories, and seeing profound meaning in non-meaningful statements. Viewing the secondary ‘profoundness receptivity’ investigation in conjunction with this helps to determine that this tendency is unique to conspiracy-oriented respondents, because it would appear that every sample respondent is just as likely to detect (or fail to detect) true profoundness. It is however not possible to conclusively determine whether those who are prone to conspiracy beliefs would simply perceive any statement as meaningful, or if this tendency is specific to bullshit. Some respondents could have just as easily answered that most statements were meaningful without actually detecting true profoundness. It is however somewhat unlikely that anyone would disagree with the profound statements and agree with the bullshit statements unless their answers or decisions were random, as all the statements are phrased ‘impressively’ in a similar way. Findings have elevated interest when also considering the fact that those who score high on endorsement of the verifiable conspiracies (TVC2) actually appear to have a weak but significantly correlated tendency to detect meaningful profoundness – which highly conspiratorial respondents do not. Obviously, a given respondent does not necessarily either endorse conspiracy theories or

87 Ronan Eric Reilly verifiable conspiracies, and a single respondent can – and probably often does – endorse both. However, it is interesting to observe that there seems to be overall connections between verifiable conspiracies and profoundness detection, while on the other hand a tendency to see meaning where none exists (or possibly be acceptant of any impressive claim) is related to conspiracy theory endorsement. This also resonates well with previously treated psychological research in chapter 2.1 regarding epistemic motivations, such as conspiratorial thinking being linked with finding patterns and meaning in random information. As with the ILFARG, these scales are not without their problems. PREC4 (α = .477) measures poor internal reliability, although the primary scale of interest BSREC4 (α = .784) scores surprisingly high. Moreover, it would be arrogant to assume that full conclusions about respondents’ abilities can be drawn from assessing a few sentences in a questionnaire. Also, this research does make inferences regarding what constitutes ‘meaningfulness’. Validity-wise it is possible that one is measuring more of a tendency to not exert cognitive effort in judging the sentences than an ability/inability to correctly assess bullshit or profundity. If that were the case, though, this would likely be a trait respondents displayed in the ‘real world’ too, and where similar lacking cognitive efforts could be the cause of failing to make correct assessments about real news stories and information. This would be consistent with literature suggesting that a lack of mindful reflective thinking is activated when faced with e.g. impressively phrased information. The possibility of results being due to careless response bias must naturally also be considered, however, this is not necessarily more so an issue for this scale than it is for the others, and the significant results do stand regardless. In many ways, though, such bias would culminate in the same result; that one ultimately fails to make correct judgements about what constitutes profoundness and/or bullshit. However, it would be unclear whether this was a motivational issue or one of ability. As in the logical fallacies section, it is informative to take a descriptive look at how respondents have answered. We can read from table 4.4.3 (created using the same method as table 4.4.2) that there are many mistakes being made within the sample in relation to detection of both bullshit and profoundness. There are somewhat fewer incorrect assessments, and a lot more responses without an opinion, than there were in relation to the fallacious arguments addressed by RQ6. Mean values and standard deviations found in table 4.1.9 suggest, however, that average sample levels of correct assessment are quite similar to those found in the ILFARG. While BSREC4 averages a mean score of x̄ = 3,53 (.53 higher/more correctly assessed than a neutral 3), the PREC4 (x̄ = 2,16) is also .84 more correctly assessed than neutral (as correct answers are scored opposite from BSREC4). It is important to once again note that the table does not take scale nuances into account, as it treats moderate and strong responses equally. Standard deviations and

88 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

sample distribution may Correctly assessed No opinion Incorrectly assessed

N % N % N % therefore appear strange if Profound: Even he who shall move an entire mountain must begin by carrying 115 92,0% 4 3,2% 6 4,8% viewed together with the small rocks. Profound: We should concentrate on the future, as it is where we will spend 101 80,8% 8 6,4% 16 12,8% table. Why some statements the rest of our lives. Profound: Living is like drawing on a are so clearly incorrectly blank sheet of paper without having an 90 72,0% 21 16,8% 14 11,2% eraser. Bullshit: Tranquility contains solely assessed than others would life's parallel thoughts, but primarily in 65 52,0% 45 36,0% 15 12,0% eternity the truth will be released. Profound: Selfishness is not to live as require a whole new study you wish to, but to ask others to live as 56 44,8% 16 12,8% 53 42,4% you wish them to. Bullshit: Hidden meaning transforms to answer. Possibly, there is 56 44,8% 49 39,2% 20 16,0% unparalleled abstract beauty. Bullshit: Grace is a vision of wisdom an element of syntactic and strength, but not necessarily of 54 43,2% 57 45,6% 14 11,2% primary vitality. Bullshit: Eternal is he who does not complexity to blame for 50 40,0% 42 33,6% 33 26,4% turn around when the road branches. this. Extreme values call Table 4.4.3 – Overview of responses to bullshit- and profound statements (N=125) for a possible scale moderation if it were to be used in future studies, and are also a major concern to the validity of the current study. For instance, the intended-to-be-profound statement “Selfishness is not to live as you wish to, but to ask others to live as you wish them to” is for some reason judged to be meaningless by as many as 42,4% of respondents. In contrast, the (also profound) statement “Even he who shall move an entire mountain must begin by carrying small rocks” was hardly misjudged by anyone (92% correct). The bullshit statement “Grace is a vision of wisdom and strength, but not necessarily of primary vitality” was not incorrectly assessed by many (11,2%), but almost half (45,6%) did not convey an opinion. In fact, when studying the table in detail it becomes clear that close to 40% of all respondents have opted out of having an opinion when offered any bullshit-statement. This could suggest that the statements spawned uncertainty, and that central tendency bias could have occurred due to not wanting to answer a complicated statement incorrectly. In a way this amounts to a lot of ‘missing’ information, as well as unbalanced data; both of which could be problematic to the interpretation of findings. The ‘profoundness recognition’ scale does not seem too affected by this, but ‘bullshit receptivity’ is the primary target of investigation, and therefore this is a concern. The chosen conclusion to RQ7 is nevertheless that a relationship ultimately does exist between failing to detect pseudo-profound bullshit and both endorsement of conspiracy theories and conspiracy mentality.

RQ8: Source criticism ability The final variable to be investigated is source criticism ability when evaluating news stories:

RQ8: It there a relationship between level of source criticism ability and endorsement of conspiracy theories / conspiracy mentality?

89 Ronan Eric Reilly

Expectancies and hypotheses This thesis presumes that the more capable a media consumer is of being source critical, the lower their level of endorsement in conspiracy theories will be. This relationship will possibly also apply to conspiracy mentality. The CRAAP-scale is scored in a direction where low scores indicate high source criticism ability. Hence, a positive correlation is hypothesized:

Ho: There will not be a significant positive correlation between CRAAP-scores and CMQ- and/or VCTE7-scores (ρ > .05). HA: There will be a significant positive correlation between CRAAP-scores and CMQ- and/or VCTE7-scores (ρ ≤ .05). In addition to the 4.1.9 table, the final statistics-Appendix (K) provides relevant SPSS output.

Analysis No significant correlations are found between CRAAP and CMQ (r = .104 ρ = .247) or VCTE7 (r = .079 ρ = .382). Accordingly, the null hypothesis is not rejected. No significant relationship is found for the TVC2 either (r = .125 ρ = .165). Collaboratively, visual inspections of scatterplots show no indication of linear relationships.

Discussion These findings suggest that the level of ability to be source critical has seemingly no significant bearing on endorsement of conspiracy theories, and that conspiracy mentality level is also independent of this. This lack of relationship also applies to true conspiracies. Essentially, these results suggest that neither rational nor irrational conspiracist beliefs are due to poor (or even skillful) source criticism. This is not in line with expectancies, and it is extremely surprising. As there is such a strong conviction that lacking the ability to be source critical must be a relevant factor in relation to conspiracy theory endorsement at the very least, a closer investigation into the validity of the measurement scale and potential bias in data collection must be done. The scale- items all seek answers to how often respondents claim to

Negative responses perform ‘actions’ from the CRAAP checklist when faced with 15 % a news story (1=Often, 3=No opinion, 5=Very rarely). An Affirmative responses 80 % initial look at the statistics indicate a low mean score (x̄ = 1,95), meaning the average respondent appears to be highly Neutral responses 5 % source critical. A moderately low standard deviation (σ = .69) further suggests that almost no responses have an average Figure 4.4.4 – Average rate of responses to CRAAP-scale items (N=125) score above a neutral 3. This means that almost every respondent gave answers suggesting that they were quite or very source critical. To additionally put this into perspective, figure 4.4.4 shows a descriptive overview of the answers given by respondents, and it presents the average rate of affirmative vs negative responses. The figure was generated by recoding variables in SPSS into ‘affirmative’ (1-2), ‘neutral’ (3) and ‘negative’ (4-5),

90 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world and then counting cases for each circumstance. The general perception of own ability to be source critical is seemingly very high, as the mean response to 8 out of the 10 scale-items was affirmative (either ‘often’ or ‘some times’), while only 1,5 of the 10 actions were answered with a ‘rarely or never’ response – and 0,5 neutrally. The CRAAP scale used to measure the correlation does not only take affirmative vs negative responses into account, but also considers the graded rating on the whole 5-point Likert-scale. As such, it should give a more nuanced basis of evaluation. It is nevertheless evident by assessing these data that theoretically audiences seem to ‘know what to look for’ in media messages, as findings indicate that the average respondent believes him-/herself to consider 8 out of 10 items on the CRAAP-checklist when faced with a news story. While the scale certainly cannot be criticized for poor internal reliability (α = .833), this does not prove that it necessarily measures the desired construct. The extreme skewness of affirmative vs negative responses within the sample invokes serious doubt regarding the validity of the findings. There is reason to believe that self-reporting these answers allows for major social desirability- and acquiescence biases, and that the scale does not appear to measure the intended construct of ‘actual source criticism ability in practice’. Rather, what has been shown by these results is that very few respondents in the sample assess themselves to be poor at source criticism. The intended construct would likely have been more fruitfully measured using some sort of practical tests or problem- solving tasks. The only conclusion this study choses to draw from RQ8 is that most sample respondents, irrespective of mindset or conspiracy theory endorsement level, believe themselves to be highly skilled at considering the currency, relevance, authority, accuracy, and purpose of news articles they are faced with. Ultimately, one could choose to conclude that no significant relationship seemingly exists between endorsement of conspiracy theories or conspiracy mentality level and self-assessed source criticism ability.

91 Ronan Eric Reilly

5 Summary, conclusions, and thoughts

5.1 Recap and summary This thesis has covered a lot of ground. Many investigations have been conducted, and several concepts have been treated – some very broadly, and some in extensive detail. The final chapter will summarize all findings, assess limitations to the studies, discuss potential implications findings may have, and offer some final thoughts.

Recap of thesis purpose The purpose for writing this thesis was rooted in the current overload of media information available in the world today, and a sense that many audiences have an insubstantial understanding of both who they should trust and how to critically evaluate information. As a possible consequence of this, misinformation and fake news stories flourish, and conspiracy theories – which have been very persistent for decades – are likely more prevalent now than ever. Causing large societal problems ranging from terrorist attacks to major democratic issues, they are also a major challenge to journalists and media workers trying to uphold their duty of keeping the public informed. The aim was to evaluate the endorsement level of some specific conspiracy theories, as well as a general propensity towards conspirational thinking (conspiracy mentality) in relation to two primary focus areas: a) News media trust, source preference, and literacy; and b) Critical thinking dispositions and skills. To facilitate the relatively broad amount of research at hand, eight research questions were fashioned, all of which were assisted by supporting hypotheses. A comprehensive literary review initially treated a wide array of relevant conspiracy theory research, extensively exploring what they are and how as a concept they are distinguishable from true and proven conspiracies. It further dealt with their traits and characteristics, their media presence, mechanisms causing their spread, and what psychological motivators have been found to be related to their endorsement This was followed by a review of literature in two individual sections. Section A treated media trust and alternative vs mainstream media differences, as well as some scholarly work pertaining to news media literacy. Then, in section B, the concept of critical thinking was explored, and a distinction was made between dispositions and skills. Literature involving the former was followed by an introduction to the three specific critical thinking skill areas of interest to this thesis; logically fallacious argumentation, ‘pseudo-profound bullshit’, and source criticism. Results from a diverse non-probability general public sample (N=125) provided the data from the distributed questionnaire, which was fashioned primarily from both self-created and previously validated and adapted Likert-scales. This data was analyzed using bivariate correlation analyses and ANOVA’s.

92 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Summary of main findings The gathered sample data was found to be pleasingly diverse, as answers were captured from a broad array of characteristically differing respondents – both demographically and with regard to the variety of opinions given. A general tendency was found in the sample towards having a conspiracy mentality, and a large number of actual conspiracy theories were quite commonly endorsed. Noteworthy was the fact that the two true and verifiable conspiracies (not actual conspiracy theories) were on overall by far assessed to be the most believable; a positive take-away seen from a practical viewpoint. It was however worrying to discover that almost a quarter of all respondents found a completely fabricated conspiracy theory – which was concocted both for and by this study – to be highly plausible. Given the introductorily presented research, these findings were not too surprising; although there was a certain amount of unsubstantiated anticipation that a Norwegian sample possibly would reveal less conspiracy-mindedness. Another important finding was the fact that the main dependent variables, ‘conspiracy mentality’ and ‘various conspiracy theory endorsement’ significantly correlated quite highly with each other. This relationship was important to assess the validity of both scales, while it also provided support for answering a sub- theme of this thesis; whether or not a conspiracy mentality can be a general predictor of conspiracy endorsement. These findings cannot speak to how important other circumstances may be, but they did re-affirm multiple previous findings that an underlying mentality seemingly is a major factor. The first of the two main areas of interest – A: News media trust, source preference, and literacy’ – posed four research questions. RQ1 asked what relationship exists between skepticism towards mainstream media and conspiracy theory endorsement / conspiracy mentality. As expected, findings suggested that those who were more skeptical towards mainstream media news had a higher conspiracy mentality and were more likely to endorse the most common conspiracy theories. RQ2 and RQ3 dealt with possible connections to alternative media and queried respectively the relationships between conspiracy theory endorsement / conspiracy mentality and frequency levels of alternative media consumption, as well as different attitudes towards alternative media. Discoveries included e.g. that those who consumed alternative media at least once a day had a higher likelihood of both having a conspiracy mentality and endorsing common conspiracy theories than those who consumed it less than monthly or never. Medium amounts of consumption did not seem to have a significant effect. Additionally, no amount of alternative consumption had any significant bearing on evaluations regarding actual conspiracies. Furthermore, results suggested that those who displayed any level of positive assessment of alternative media sources were more likely to both have a higher conspiracy mentality and endorse a larger amount of conspiracy theories than those who viewed them negatively. RQ4 wanted to know what relationship exists

93 Ronan Eric Reilly between news media literacy levels and endorsement of conspiracy theories / conspiracy mentality, which was a more complicated multi-dimensional endeavor. Surprisingly, findings pertaining to news media literacy were not as related to conspiracy theory endorsement and conspiracy mentality as expected. Both a high need for mindful thought processing and a high need to be in control of news media’s influence appeared entirely unrelated. This was true for most knowledge about how news providers and consumers interact, including how well news messages can be interpreted differently because of e.g. production techniques, framing effects, and the fact that people are different. However, a lack of understanding regarding how the media can affect and filter perceptions of reality through e.g. overexposing conflicts or agenda setting did to a slight degree appear to significantly correlate with endorsement of conspiracy theories; but interestingly not conspiracy mentality. The second main area of interest, B: ‘critical thinking dispositions and skills’, concerned itself with four additional research questions. RQ5 asked if there is a relationship between individual levels of critical thinking disposition and endorsement of conspiracy theories / conspiracy mentality. No statistically significant findings connecting differences in disposition towards critical thinking with conspiracies were uncovered. Among possible reasons for this it was however pointed out that an extremely poor alpha-value, and an overly centralized distribution of answers, may have been reasons to suspect that the measures used were potentially unreliable; hence weakening these findings. In relation to RQ6, however – asking if there is a relationship between susceptibility to arguments based on informal logical fallacies and endorsement of conspiracy theories / conspiracy mentality – significant correlations were found; and with regard to conspiracy theory endorsement this relationship was also moderately high. The importance of the fact that the ILFARG-scale is newly developed and previously untested was however pointed out – encouraging caution to be taken when interpreting these results. It was nevertheless particularly interesting to observe that highly insignificant correlations were found in relation to the scale measuring conspiracies; suggesting that a lack of ability or motivation to recognize poor arguments was associated with endorsement of unsubstantiated conspiracy theories, but not knowledge about actual conspiracies. Similarly, with regard to RQ7 – querying the relationship between failing to detect ‘pseudo-profound bullshit’ and endorsement of conspiracy theories / conspiracy mentality – correlations of significance were found for both conspiracy mentality and conspiracy theory endorsement in relation to the tendency to be overly accepting of seemingly profound nonsensical statements. Interestingly the ‘true verifiable conspiracies’-variable behaved oppositely to the other two variables yet again. Whereas conspiracy theory endorsement and -mentality were not significantly correlated with ‘profoundness receptivity’, verifiable conspiracies not only were, but

94 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world they were also not correlated with ‘bullshit receptivity’ like the other two scales. It was concluded that conspiracy belief is related to acceptance of bullshit claims either because of an inability to detect it, or possibly due to a propensity to generally agree with all impressive statements. With regard to finding out whether there is a relationship between level of source criticism ability and endorsement of conspiracy theories / conspiracy mentality – as RQ8 wished to explore – findings were very different. No significant relationship seemed to be present between any variables, and – if one were to take these results at face value – source criticism ability would appear to be an essentially irrelevant trait in relation to the assessment of conspiracy theory likelihood. This was regarded to be very strange; however, it was suggested that the extreme imbalance in the gathered data was largely responsible for what presumably were biased results – as a very evident overweight of affirmative responses likely resulted from answers being self-reported. It is likely that many respondents either overestimated their own abilities, or that both acquiescence- and social desirability biases may have occurred. The only drawable conclusion was that sample respondents generally regarded themselves to be highly source critical, and that there was seemingly no significantly detectable relationship between self-assessed source criticism ability and level of conspiracy theory endorsement and conspiracy mentality.

5.2 Limitations and weaknesses Although the conducted studies have reached several interesting and significant findings, it is important to stress their many limitations – both methodologically and otherwise – as this ultimately is a low-resource master’s thesis. Many problems have already been pointed out, and some have even been partly attributed blame for certain conclusions. This section will summarize many of these issues, while also adding some overall considerations. Firstly, all results are restricted because of the non-probability nature of the sample, and the modest sample size of only N=125. Therefore, although demographically the sample turned out to be surprisingly diverse, findings are unfortunately limited to exploring the opinions of these 125 voluntary respondents. Although the sample data is not generalizable per se, perhaps in a small way – due to the diversity with regard to demographics etc. – it is imaginable that it could represent people from the general public to a degree which allows for somewhat meaningful results. It should at least be able to do so to the same extent as several other studies which make use of student groups or various recruited ad hoc samples. In reviewing a lot of research, it seems to be a recurring observable trend in many social scientific studies to base their findings on small groups without the use of probability samples. And like this thesis, they often attempt to use inferential statistics to generalize findings through extensive use of parametric testing; even when assumptions seemingly

95 Ronan Eric Reilly are not met. Another important consideration is the heated debate regarding Likert data and whether it fruitfully can be treated at an interval level, which the research conducted here has chosen to do. Although several assumptions fail in many of the ANOVA’s, at least all the bivariate correlation coefficients were backup-checked by the non-parametric Spearman’s rho. And although they were never technically reported, most ANOVA’s were also crosschecked with rank based Kruskal Wallis H tests yielding similar significant results. In relation to the use of survey questionnaires as a method, it has already been pointed out that major limitations apply; and accordingly, several things can potentially have gone wrong. As pointed out on numerous occasions, issues with e.g. acquiescence- and social desirability biases are likely to have occurred. People will often answer what they believe to be the ‘most correct’, and do not always practice what they preach. It is therefore interesting to observe that the scales requiring assessment without obvious options, such as the ILFARG and BSREC4, are the ones which largely yield significant results. Furthermore, careless response- and central tendency biases have been addressed as potential sources of non-probability errors several times. They are especially likely to have happened in this research due to the magnitude of the completion time alone. It is plausible that many answers have been given very quickly, as respondents could have become impatient and demotivated. One the one hand this potentially biases findings. On the other hand – especially with regard to assessment-questions such as in ILFARG and BSREC4 – the pace and motivation with which one practically navigates the ‘real world’ media landscape, assesses arguments and judges statements, may actually be quite similar; and hence these scales may possibly yield more representative results due to a ‘mimicking of real life circumstances’. Some questions in this study have also been particularly complicated, though, and accordingly increasingly prone to causing measurement errors. Furthermore, validity issues can potentially also have happened based on poor question phrasing or misunderstood interpretations. In addition, several scale-items which were translated into Norwegian may also have lost nuances of information. Weaknesses in validity may undoubtedly also have occurred due to pragmatic considerations – such as decisions made to shorten some scales, and to add/remove and reverse code/rephrase some items. In addition, in some cases – e.g. with the ‘news media literacy’-scale – conscious choices were made to make several modifications, such as replacing an item measuring one particular aspect with an item that measured something entirely different (awareness regarding agenda setting). Also, both the NFC2 and MLOC3 scales used in this study were drastically stripped for items, which could affect results in a major way. In addition, due to pragmatic considerations, none of the self-created scales have gone through the proper validation processes ensuring their scientific soundness. Also possibly resulting in major weaknesses to many of the findings, is the lack of internal consistency/reliability in several

96 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world of the Likert-scales, as demonstrated through the Cronbach’s alpha values tested in chapter 4.1. Surprisingly, though, these low alpha-scores seem to be more of an issue in the previously validated and adapted scales than the new self-constructed ones. Luckily, it is primarily the scales with the lowest α-values which have been involved in the least significant findings. Then again, it is possible – although quite unlikely – that this could have been the cause of type II-errors/false negatives, whereby null hypotheses may falsely have been retained due to inconsistent measures. Furthermore, it is realized that for ILFARG and BSREC4/PREC4 scales the entire constructs are predicated on the assumption that the interpretation of a few very specific sentences will be sufficient. It is of course incredibly important to remember that the few examples created in these scales (and provided to only 125 people) cannot ever reflect accurately how ‘everyone’ would respond to ‘all’ logical fallacies and bullshit- or profound statements. As previously pointed out also, some of the ILFARG-statements are in addition highly opinionated, allowing many attitude- biasing issues. Regardless of the questions’ incitement to ignore personal feelings or views when assessing the arguments, some respondents may still not have been able to look beyond their own attitudes. Hence, the scale may be invalid and measure e.g. more of an inability to see beyond own opinion than the intended construct of susceptibility to the fallacies. There is, of course, also a huge similar issue in relation to the main dependent variable ‘conspiracy theory endorsement’, as measuring how likely one assumes a certain amount of specific conspiracy theories to be does not exclude or preclude that a list of other theories could not have yielded completely different results. Additionally, there are many reasons why using questionnaires to measure complicated constructs like these are problematic. E.g. Butter and Knight (2016, p. 5) pointed out the problem with relying on surveys for this purpose, because they operate with a very crude notion of what it means to ‘believe’ in such theories. If modelling certain beliefs as a continuum, for instance, it becomes problematic that neutral responses like ‘no opinion’ can mean several different things. Also – upon re-interpreting phrasings of some of these theories – there are particularly two of them that stand out as ambiguous in retrospect. It could potentially weaken the study that both the ‘elite few rule the world’ and ‘climate hoax’ items are vaguely formulated. The latter e.g. could simply express a sense of exaggerated political agendas on climate issues rather than the conspiracy theory that ‘science is lying about it’. Although it is only a secondary investigatory measure, it must also be pointed out that the ‘scale’ for verifiable conspiracies (TVC2) only has two items, which as an interval Likert-scale measure is methodologically unsound. If e.g. a respondent were to endorse one conspiracy but not the other, then measures would make no sense. These items were originally created to crosscheck relationships to the CMQ for any conspiracy related theory, and were never intended to perform the measures that they have. For

97 Ronan Eric Reilly future studies it would be fruitful to plan and create a proper multi-item scale for true conspiracies. As pointed out, the term ‘alternative media’ has been treated in a very general way in this thesis. Broad distinctions have been made, and no differences between e.g. right- or left wing, extremist, paranormal, independent, or impartial etc. have been considered at all. The limited focus has been on simplifying the object of study to a dichotomous differentiation between ‘mainstream’ and ‘alternative’. Had the study expanded to focusing on implementing an increased amount of categorical distinctions, the findings would expectedly have been much more nuanced. With the obvious benefit of hindsight, it has throughout writing this thesis also become clear how several of the ‘media habits’ measurements could have been improved. For instance, with regard to measuring attitudes towards alternative and mainstream media, it would have simplified the analysis greatly – and possibly improved the accuracy of measurement – if one simply had changed out multiple choice questions with another Likert scale. One could e.g. have asked directly “To what degree do you approve of alternative news sources as opposed to mainstream news sources?”, and requested respondents to range news source preference on a 1-5 scale from ‘strictly mainstream sources’ to ‘strictly alternative sources’. In relation to many of the other constructs measured, it is contemplated that future endeavors might benefit from implementing more practical ‘tests’ which presumably would produce more correct assessments. E.g. measuring source criticism abilities would likely have yielded very different results if administered as a series of source- checking assignments, rather than providing ‘obvious’ self-report options. Similarly, the CTDS and NML scales could possibly have benefited from such approaches. Doing so may have been the key to successful media literacy measurement for Craft et al. (2013), when they used the NMKS multiple-choice assignment; an approach the current study did not chose to adapt. Concerning the relationships between the many variables in the study assessed through bivariate correlation analysis, findings do present with some very clear limitations. Strictly speaking there is not sufficient information in this study to help suggest what causes what or which variable effects which. Only the presence or absence of statistically significant relationships are uncovered. Therefore, one cannot determine with certainty e.g. whether having a conspiracy mentality causes higher skepticism towards mainstream media, a more positive attitude regarding alternative media, and/or susceptibility to pseudo-profound bullshit – or vice versa. Neither can one say whether conspiracy theory endorsement is the result of or reason for lacking understanding of certain news media literacy aspects, frequent consumption of alternative media, or poor ability to detect fallacious argumentation. Also, as correlation does not imply causation, it is hypothetically highly likely that many discovered relationships between scales are spurious. Therefore, other than observing that there is a relationship between e.g. conspiracy mentality and endorsement of

98 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world conspiracy theories – or any other variables in the study for that matter – it is not clear whether they are directly related to one another, or if they both could be the result of another completely separate variable within or outside of this study. As treated in the literary review, such factors could e.g. be social; such as people’s desire to be unique, or existential; due to anxiousness, suspicion, or a sense of powerlessness and a need to blame hidden villains. Reasons could also be epistemically rooted – e.g. by needing to explain big and important events with proportionately big and important causes, or the brain needing to make sense of random patterns. Undoubtably, there will always be an extremely complicated interplay between all these factors irrespective of which findings have been, or ever will be, found. As such, possible limitations to this study are inevitably endless.

5.3 Conclusive discussion Although many findings in the conducted studies were not overly surprising and/or they have already been pointed out in prior research, re-affirming the degree to which investigated factors can be central to conspiracy theory belief has still been a fruitful endeavor. In addition to the many discoveries made in this thesis, the concept of ‘conspiracy theories’ has also been thoroughly reviewed, and their endorsement has been attributed to an intricate interplay between knowledge, psychological aspects, and critical thinking mechanisms. Practical examples of their popularity and persistence in the media have been offered, and it is evident that while many media industries benefit greatly from both their entertainment value and/or potential tactical advantages, many journalists aiming to convey accurate representations of events face immense challenges. Devious outlets can easily take advantage of audiences by using fallacious tactics and telling ‘good stories’ involving cruel plans which entice disbelief of official accounts and provide ‘satisfying’ explanations that many people want or ‘need’ to hear. As covered in the summary, support was found for what the initially presented literature had suggested – namely that a surprisingly large number of people seemed to believe at least a few conspiracy theories regardless of what the reasons may be. No Norway-specific research was found investigating this, however – if one were to hypothetically generalize findings from the limited non-probability sample in this study – Norwegians do not appear to be any different. From a media perspective, and from any other perspective for that matter, this is worrying for several reasons. A fundamental premise for addressing this topic in the first place was that democratic societies are dependent on informed decisions, and that the spread of misinformation hinders this aim. The media has the power to tell the stories of society, and therefore influences public thinking, beliefs, and behavior. Given journalists’ role in upholding democratic principles by keeping the public well informed, they should responsibly work towards exposing and pointing out weak arguments and untrue stories;

99 Ronan Eric Reilly however, this can appear to be easier said than done for many reasons.

A: News media trust, source preference, and literacy Relating to the first area of interest, it is from a media studies and journalistic point of view seen as important to be aware of consumers’ trust in the news media, their assessment of various sources, and their news media literacy levels. Luckily, judging by the 2019 poll by Medietilsynet presented in chapter 2.2 – and contrary to results from the gathered sample in this study and many representative foreign polls – the Norwegian public generally seem to have a trusting relationship to the mainstream media. This could be because Norwegian mainstream media possibly are more trustworthy due to e.g. stricter public control. In turn, this could indicate that those who are skeptical to mainstream news in Norway – although fewer in number – are potentially more irrational and/or extreme in their views. Nevertheless, awareness of the finding that there appears to be a relationship between media distrust and conspiracy theory endorsement is valuable to journalists. Perhaps of even more concrete worth is the connection found between both positive attitudes towards alternative media and high frequency of their consumption. Does this mean that everyone who prefers alternative sources or reads a lot of non-mainstream articles should be outed as a conspiracy buff? It would seem not. Although findings suggest higher average conspiracy theory endorsement and -mentality for those who frequently read alternative news, their average endorsement rate is not alarmingly high. Also, the findings are significant only in relation to those who have a very frequent consumption. Irrespective of how often one reads alternative news it appears that the more positively one rates their purpose and agenda, the higher the probability is that one will endorse conspiracy theories. This study has not investigated the direct relationship between those who display mainstream distrust and alternative trust, but literature presented in chapter 2.2, e.g. by Tsfati and Cappella (2003), suggests that those who are mainstream skeptics also do prefer alternative news. Hence, a clear assumption can be made that there is a detectable pattern between both conspiracy theory endorsement and -mentality and preferring alternative sources to conventional ones. A problem with this study, as pointed out, is that it only measures ‘alternative’ as an individually perceived contrast term to ‘mainstream’. Therefore, it perhaps mistakenly addresses them in an unfairly pejorative manner, and only says something about held attitudes towards some sources – failing to practically categorize their individually objective levels of credibility. Conscientious and hardworking journalists employed by serious alternative outlets will hopefully understand that they are not necessarily the primary targets of study here, and that such news sources also can be very important with regard to maintaining power/government monitoring as well as upholding principles of independence, diversity and objectivity. This thesis has also pointed out and discussed how mainstream skepticism is not synonymous with

100 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world irrationality, and how healthy amounts of reflective skepticism on the contrary is a major contributor in both the detection of misinformation and the driving force behind actually uncovering facts – and even exposing true conspiracies; which of course also do occur. While news skepticism is highly correlated with conspiracy theories it is also a trait displayed to a slight degree by those who are highly news media literate, as shown by e.g. Craft et al. (2013, p. 2). In line with this, this thesis also shows a significant but much weaker correlation between mainstream media skepticism and those who report being aware of the true conspiracies than the much higher correlation shown in relation to conspiracy theory endorsement. Based on reviewed research, it was expected that news media literacy would require an entire section at the end where the importance of teaching it to the public would be stressed. However, findings indicate that most people already display a relatively high level of news media literacy. This is of course operating under the premise that findings in this study loosely can be representative of a larger population – which they unfortunately cannot. The representative poll referred to in chapter 2.2 (using different measures) did however suggest that only 9% of Norwegians display low literacy, while close to half scored high. A clear tendency to endorse conspiracy theories was nevertheless expected among audiences within in the sample who displayed lacking knowledge about the way the news media works, but surprisingly findings suggested there is not much reason for concern about such literacy in relation to conspiracy theory belief. Having an underlying conspiracy mentality appeared to be unrelated to understanding the media; although it was concluded that a weak possible link may exist between conspiracy theory endorsement and an unfamiliarity with how the media can provide an unbalanced representation of reality; e.g. through such mechanisms as overexposure and agenda setting. This makes sense because conspiracy theory narratives clearly rely on overly emphasizing certain aspects of events and narrow areas of focus; which devious media workers can utilize. The other key dimensions of literacy, individual perception of control over media influences and engagement in activities requiring effortful thinking, have also been highlighted in discussions. Neither prior research nor the current study have found significant connections to these variables. It remains paradoxical that a feeling of being in control of media influence is not somehow related to conspiracy belief when feelings of powerlessness is such a commonly associated trait. Perhaps actively ‘choosing to control’ media message influence could somehow be a reactionary measure to this powerlessness? Future research could maybe benefit form focusing on this relationship. When considering the insignificant findings in relation to the ‘need for cognition’ aspect along with the also non-significant ‘critical thinking disposition’– and ‘source criticism ability’-findings, this could collectively point to a general tendency for those who affiliate with conspiracist belief to see themselves as ‘truth seeking’ critical thinkers – as much as, or even more so than, other media

101 Ronan Eric Reilly consumers. This would be consistent with previous research, and it would also fit well with their reasoning for being overly skeptical to mainstream news and positive to alternative sources.

B: Critical thinking dispositions and skills Both the underlying critical thinking tendencies audiences have, and their skills when interpreting media messages are important to media studies. In the case of this research it has been hypothesized that regardless of who one trusts or what levels of knowledge one has about the news media, it is likely that such dispositions and abilities will be key in assessing the probability of e.g. conspiracy theories. It is however difficult to have any concrete opinion about the findings produced by measuring critical thinking disposition in this study. The fact that there is no statistically significant relationship at all between endorsement of unsubstantiated conspiracy theories – or the substantiated conspiracies for that matter – and being a critically open or a reflectively skeptic individual is very strange. Methodological flaws have been suspected to be at the root of these results, but it is also possible – as it was pointed out in the last section – that those who endorse conspiracy theories may answer based on a belief that they are highly critical thinkers; whether objectively the case or not. This does however not quite align with the fact that an extreme overweight of all respondents only scored moderately on all the thinking disposition scales. For different reasons it is also hard to have an opinion about the results generated by the relationships to ‘source criticism ability’. It has been discussed that self-assessed ability is all that is believed to have been measured, and that this is what is unrelated to conspiracy theory endorsement and - mentality. Irrespective of conspiracy belief, most respondents seemingly assessed themselves to be highly source critical and confident in their own abilities, and regardless of the current study this is possibly an important observation with regard to media studies in general. Taken together with the average high level of news media literacy found, one could be tricked into thinking that the respondents – although only moderately disposed towards critical thinking – were generally both very knowledgeable about the news media and how to navigate information. Again, this would strictly speaking only apply if one could generalize these non-probability findings. It is nevertheless apparent that this sample has displayed these tendencies, while simultaneously observing that there was a broad acceptance of conspiracy theories within it, and that many of the respondents interpret their daily media messages from a high conspiracy mentality viewpoint. This combination of questionable beliefs and elevated evaluations of own abilities can be important to address. The results from the ‘critical thinking disposition’– and ‘source criticism ability’– relationships align poorly with the significant results found in relation to both the argument evaluation and bullshit-detection skills. If one can trust these measurements, perhaps the most interesting finding in the entire thesis is the relationship between informal logical fallacies and

102 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world conspiracy theories. This is mostly due to it being such a concrete and tangible finding, and also because it is not found in the literature to be a previously directly drawn relationship. Although not surprising per se, given e.g. the rhetoric vices and traits associated with conspiracy theories presented in chapter 2.1, it is compelling to see this correlation occur in practice. From a journalistic perspective this very clearly indicates the importance of considering the use of precise language and argumentation, given the impressionability of audiences who largely affiliate with conspiracy theories. It also highlights how easily certain outlets can purposely deceive media consumers through narration. Although the validity of the scale has been called into question for many different reasons, findings are highly interesting because the measurement is based on an attempt to have respondents assess statements through the active use of critical thinking skills, as oppose to self-assessing own abilities. This is possibly a much more ‘realistic’ approach, and it is likely to paint a truer picture of respondents’ assessments in practice. The significant results from the pseudo-profound bullshit receptivity- (and profoundness receptivity) scale(s) measure a different construct, but they do so largely in the same manner. These findings are also very interesting both because of the same ‘real world’-comparability, but also due to the results themselves – that perceiving intended profoundness where it is implied but non-existent is linked to conspiracy theory endorsement; as previously also found by Pennycook et al. (2015). Although it was not determined whether it was the bullshit claims themselves or a tendency to believe any impressive sounding statement – or any statement for that matter – that were at the core, it still seems that a degree of reflective critical thinking must be absent in practice. Again, both media studies and journalists can benefit from this knowledge.

Other discussion points In chapter 2.1 of the literary review it was explained that many have theorized that belief in conspiracy theories is a stable predispositional tendency, and this was a major reason for measuring conspiracy mentality in this study. The actual endorsement of some theories was also measured, due in part to scholars not agreeing completely on the importance of circumstantial factors. As it turned out, these two variables not only unsurprisingly correlated well with each other, but also seemed to produce significant effects on almost all the same independent variables throughout investigations. Although the correlations were not always equally high for both, this was still a clear indication that conspiracy mentality and actual endorsement are highly connected; hence strengthening e.g. Imhoff and Bruder (2014)’s claims that a predispositional conspiracy mentality seemingly can predict at least a certain amount of actual endorsement. Even without being able to speak to how important circumstantial factors can be, it is potentially bad news for journalists and media workers that audiences can have such a stable predisposition. This is both because it is very difficult to change an

103 Ronan Eric Reilly underlying mentality through methods available to such professions, and – as previously reviewed and findings here have strengthened – because such a mentality is also deeply intertwined with e.g. mainstream media skepticism. This underscores challenges in combating their endorsement, and one may ask how a journalist can enlighten or educate an audience that views them as a potential ‘enemy’. From a practical media production perspective, it would likely have been easier to fight these theories if there were more topical or circumstantial reasons why audiences believed in them. Still, even if conspiracy mentality is not necessarily something one can affect or constructively influence directly through news production, it can be useful to be aware of the fact that many readers can have such an underlying predisposition, and that it accordingly may be less likely for them to accept certain news stories. Conversely, it is a major concern how easily the same readers may find more conspiratorial stories probable. The alarming fact that almost a quarter of respondents in this study found the completely fabricated ‘Russian surveillance’ conspiracy theory to be likely may be related to such a susceptibility. Theoretically, one would think that finding a way to encourage increased media literacy and source criticism could be effective preventative measures in reducing misconceptions, false beliefs, and conspiracy theory endorsement. Unfortunately, if one accepts the findings in this study, such endeavors are effectively rendered mute because they suggest these factors to be unrelated to conspiracy belief. It would seem that a better strategy could involve somehow increasing practical critical skills in interpreting fallacious arguments and nonsensical statements. Also, increasing mainstream media trust and reducing excessive reliance on alternative media content should be helpful, but this is perhaps likely to often be more a consequence of conspiracy mentality than a reason for it. Due to their strong variable- relationship it is in fact entirely possible that conspiracy mentality alone could be responsible for the actual endorsement of conspiracy theories regardless of the influence resulting from other variables, and that it is the cause of spurious relationships. Such a mentality could be hindering the critical assessment of e.g. certain arguments because of lacking motivation to consider alternative views. The research conducted in this thesis is at a much too elementary level to provide conclusions that can suggest or exclude this, and it is therefore limited in its concrete value to media workers and journalist. In retrospect, it would likely have been a fruitful endeavor to run statistical analyses where ‘conspiracy mentality’ was used as a control variable rather than a separate dependent variable. This would possibly have provided a clearer understanding of mechanisms independent of an underlying mentality and could have been of more practical value. On an overall basis, the two-item scale measuring endorsement of substantiated conspiracies has acted as a sort of ‘control variable’ to the dependent conspiracy theory endorsement variable throughout studies. If one were to disregard the fact that this ‘scale’ was pointed out to be

104 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world methodologically unsound, it is interesting to observe that with few exceptions it consistently either produced significantly opposite correlations to CMQ and VCTE7 scales – or only one or the other revealed a significant relationship with measured independent variables. This was interpreted to suggest that different mechanisms – perhaps an added level of critical understanding (or knowledge) – were indeed at play in the assessment of conspiracy theories than when substantiated conspiracies were in question. Ultimately, though, it has been beyond the scope of this thesis to assess variables with regard to the covariate interaction between them. No variables have been directly controlled for, nor have any complicated variables been compounded. Interesting relationships to pursue in order to gain further insight would have been e.g. alternative media consumption frequency’s relationship with conspiracy theory endorsement when controlling for news media literacy, or how alternative media preference and susceptibility to fallacious arguments together could affect conspiracy theory endorsement. Given a larger scope to operate within, purely demographic relationships were also considered as part of the investigation; e.g. the relationship between conspiracy mentality and gender, age, income, education level, political preference etc. Also, although separate from the main aims, it would have been interesting to look at relationships between other variables irrespective of conspiracy theories. For instance, what relationships exist between those who prefer alternative media and being news media literate, or between critical thinking disposition and various skills. To achieve many of these findings it would have been fruitful to run a series of e.g. MANCOVA’s or other more complicated relevant statistical operations. Also, one could have run regression analyses to predict outcomes from increasing effects on one variable due to another. The gathered data material could allow for such investigations; however, these will remain endeavors for another time.

5.4 Final thoughts As the title and introduction of this thesis suggests, we do live in a world filled with misinformation, fake news and conspiracy theories. Simultaneously, there will at any given time be people in positions of power who do conspire with others to obtain their goals; and many will often be in doubt about the truth when faced with diverging media messages in all shapes and forms. This thesis has assessed both emotional and rational aspects regarding how audiences make sense of news stories. It has been thoroughly shown that there are so many factors that work together to form the basis of conspiracy theory endorsement that pinpointing single specific constructs, such as e.g. ‘susceptibility towards informal logical fallacies’ or ‘being a mainstream media sceptic’ does in many ways feel a tad simplistic. A highly complex combination of who someone is and what they know will in the end always dictate their views and beliefs when facing media messages, and what

105 Ronan Eric Reilly ultimately is deemed the ‘truth’ will inevitably be an interpretation of the information they choose to trust and why. A major problem is that many audiences who consume information are seemingly not occupied with attempting to falsify claims and notions. It is interesting to consider that so many people will frequently practice denialism, while simultaneously accepting pseudo-scientific arguments (which they themselves perceive as actual science) to explain events alternatively. Greatly simplifying Sunstein (2014, pp. 15-19)’s concept of ‘conspiracy cascades’ from chapter 2.1, consumed information is often reinforced by clinging to ‘echo chambers’ and like-minded sources, which continuously hash out support for the premises of e.g. conspiracy theories; in turn only resulting in a further re-affirmation and reinforcement of these ‘truths’. Herein lies a great paradox. Somehow the intention of ‘seeking the truth’, although admirable from a deontological perspective, often suffers the ironic practical consequence of achieving the exact opposite outcome; resulting in the spread of misinformation. Of course, people conspire – they just do! Watergate, MK Ultra, and the Phoebus cartel are only a few examples of large-scale conspiracies that actually panned out. However, no evidence that withstands scrutiny seems to suggest a US-staging of the 9/11 attacks or faking the Moon landing. Lewandowsky and Cook (2020, p. 10) point out the importance of identifying the CONSPIR traits of conspiratorial belief presented in 2.1 (figure 2.1.2) when assessing alternative news stories that contradict general representations. Healthy skepticism, pursuit of actual evidence, and consistency are required to stay well informed and ultimately be better equipped to decipher possible true conspiracies from conspiracy theories. A general sense of trust, preferred choice of source – mainstream or alternative – and any kind of fancy spin or trickery should never take precedence over informed rational decisions and proper assessment of information credibility. This thesis in no way proposes that one should disregard alternative sources and refrain from questioning mainstream media; nor does it encourage the automatic dismissal of any conspiracy claim. In many ways it does the opposite, by suggesting that the same caution be taken when considering all media sources or information – whether it comes from a major broadcaster or a randomly shared link via social media. If the real goal is to actually seek the ‘truth’ it is important to distinguish between healthy skepticism and irrational belief. This is true for both highly unlikely scenarios – such as claims that the earth is flat or that all members of the Democratic party abduct and eat children – as well as for possibly more plausible claims; such as lying about the presence of mass destructive weapons to initiate a war. Identifying fallacious arguments and identifying ‘bullshit’-claims are only a few examples of very important critical thinking skills to aid in this endeavor, although many more such skills exist beyond what this thesis could ever capture. Content providers would be advised to exert caution in their narration when reporting on incidents, and audiences may benefit

106 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world from self-examining their abilities to detect fallacies and impressive nonsense; as well as avoiding overestimation of e.g. own source criticism skills. Also, it is entirely possible to entertain a good story or theory all the while understanding that it is unfalsifiable, and that it accordingly might make sense to not read too much into it. An increasing emergence of fact checking websites etc. is a solid effort by media producers to help audiences in their truth seeking, but perhaps it is out of the hands of mainstream journalists to ultimately hinder prevailing conspiracy theories. Distrust against authorities appears to be ingrained in some media consumers; a distrust which seemingly can include and taint all mainstream media. For many audiences it can seem like the root motivators of news consumption are more based on the many underlying social, existential, or epistemic psychological factors presented in chapter 2.1 than on finding out what is actually true. Conspiracy mentality is interconnected with these motivators, and those who have such a mentality may possibly benefit from working on psychological underlying motivational issues before assessment of critical thinking dispositions and skills become applicable at all. A tendency to rely more on ‘faith’ than reason to satisfy personal needs can likely stand in the way of applying such skills even if one is well capable of doing so. It is conceivably quite a tall order for media workers to help anyone with this, and for journalists, democracy, and society in general, conspiracy theories continue to be problematic – with so many existing ones persisting and so many new ones continuously emerging. When also considering the unlikely and extreme level to which respondents regard themselves to be source critical, this is alarming. Especially worrying is the abundance of research suggesting that it is almost impossible to change the minds of people who have adapted belief in a conspiracy theory (e.g. Butter & Knight, 2016, p. 4; Sunstein & Vermeule, 2009, p. 211). Luckily, many have – and continuously do – reflect on possible ‘cures’ for conspiracism. E.g. Lewandowsky and Cook (2020) have accumulated several such ‘debunking’ strategies, many of which are in line with the interest areas covered in this thesis. Logically explaining flawed reasoning, and pointing out source weaknesses, as well as providing accurate facts or links to fact checking sites are seen to be effective strategies which work well on the general public (p. 9). Unfortunately, the self-sealing nature of conspiracy theories means that any evidence disproving a theory may be interpreted as further evidence for the conspiracy by those high in conspiracy mentality. Hence, it would appear that communication efforts may need to differentiate between varying audiences; and that different journalistic strategies are required to break through to notorious conspiracy theory enthusiasts as oppose to those who value evidence (p. 7). Research has also identified some approaches that have shown to be potentially effective for reasoning with those who have a conspiracy mindset. Examples of such approaches are the use of counter-messages created by ‘trusted messengers’ (e.g. former members of relevant extremist communities) (Schmit

107 Ronan Eric Reilly et al., 2018), or developing the conspiracy theorist’s open-mindedness by being empathic and seeking to build a mutual understanding (Ponsot, Autixier, & Madriaza, 2017, p. 16); a strategy which is of course problematic given the nature of one-way flowing information between journalists and audiences. Moreover, according to Hemmingsen and Castro (2017, p. 6), firm attempts at deconstructing, aggressively counterarguing, or ridiculing the premises of a conspiracy theory tends to not be a good idea. Additionally, as conspiracy theorists largely perceive themselves as clever critical thinkers – which is in line with self-assessed findings in this thesis – affirming the value of their critical thinking abilities before somehow redirecting attention towards a critical analysis of the conspiracy theory itself can be an effective method (Voogt, 2017, p. 42). Ideologically, the onus to reduce misinformation by being responsible and carrying out proper research is very much on those who are serious providers of alternative news; as these are seemingly the ‘trusted’ outlets who hold the real power to influence misconceptions and reduce belief in unsubstantiated conspiracy theories among those who believe and spread them the most. Magically having all alternative outlets become credible is however a simplistic utopian view that may be as likely to achieve as asking those who start wars to simply stop doing it. It is however possible to imagine that journalists could use alternative channels as a platform to appeal to the ethos of conspiracy-minded audiences. Although hypothetical and controversial, it would be an interesting experiment to somehow measure changes in followers’ views if serious journalists were to 'borrow' an existing alternative news outlet and alter a biased story narrative. Further pursuing in- depth possible investigations into countering conspiracy theories would be very interesting – but this shall however remain a potentially central aim to another thesis at another time.

Meanwhile, conspiracy theories rest comfortably in an environment that allows them to keep thriving. As this thesis is being finalized the world is facing an extreme global pandemic, and unsurprisingly new conspiracy theories flourish in the media every day: “COVID-19 was purposely created in a lab as a biological weapon”; ”COVID-19 spreads rapidly via the mobile 5G network”; and “COVID-19 does not actually exist at all!” (Lynas, 2020). When the truth is in doubt, who should one trust, and how can one best assess what is most likely to be true? While reviewed literature and findings in this thesis may help answer to a limited degree how knowledge and rational thought can help decide this, there are still countless unanswered questions regarding what causes conspiracy belief, and how to potentially reduce the endorsement and spread of conspiracy theories. Our society is increasingly vulnerable to the consequences they can generate, and media workers’ challenges are unfortunately neither getting smaller nor easier as the “age of conspiracism” progresses into the 2020’s.

108 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

References:

Arke, E. T., & Primack, B. A. (2009). Quantifying media literacy: Development, reliability, and validity of a new measure. Educational media international, 46(1), 53-65.

Ashley, S., Maksl, A., & Craft, S. (2013). Developing a News Media Literacy Scale. Journalism and Mass Communication Educator, 68(1), 7-21.

Assael, H., & Keon, J. (1982). Nonsampling vs. Sampling Errors in Survey Research. Journal of Marketing, 46(2), 114-123.

Atton, C. (2002). Alternative Media. London: SAGE Publications Ltd.

Aufderheide, P. (1993). Media Literacy. A Report of the National Leadership Conference on Media Literacy. Queenstown, MD: The Aspen Institute.

Avramov, K. (2018). By another way of deception: The use of conspiracy theories as a foreign policy tool in the arsenal of the hybrid warfare. Information & Security, 39(1), 151-161.

Banas, J. A., & Miller, G. (2013). Inducing resistance to conspiracy theory propaganda: Testing inoculation and metainoculation strategies. Human Communication Research, 39(2), 184- 207.

Barkun, M. (2003). A of Conspiracy: Apocalyptic Visions in Contemporary America (Vol. 15). Berkeley, CA: University of California Press.

Bartlett, J., & Miller, C. (2010). The power of unreason: Conspiracy theories, extremism and counter-terrorism. London: Demos

Bennett, B. (2018). Logically Fallacious: The ultimate collection of over 300 logical fallacies (Academic Edition). Sudbury, MA: Archieboy Holdings, LLC.

Berg, C. (2017). Teaching Website Evaluation The CRAAP Test and the Evolution of an Approach. Retrieved from http://www.internetatschools.com/Articles/Editorial/Features/Teaching- Website-Evaluation-The-CRAAP-Test-and-the-Evolution-of-an-Approach-116769.aspx

Blakeslee, S. (2004). The CRAAP test. LOEX Quarterly, 31(3), 6-7.

Bost, P. R. (2019). The Truth is Around Here Somewhere. In J. E. Uscinski (Ed.), Conspiracy Theories and the People Who Believe Them (pp. 269-283). New York, NY: Oxford University Press.

Brenan, M. (2019). Americans' Trust in Mass Media Edges Down to 41%. Retrieved from https://news.gallup.com/poll/267047/americans-trust-mass-media-edges-down.aspx

Brotherton, R., & Eser, S. (2015). Bored to fears: Boredom proneness, paranoia, and conspiracy theories. Personality and Individual Differences, 80, 1-5.

Brotherton, R., & French, C. C. (2014). Belief in conspiracy theories and susceptibility to the conjunction fallacy. Applied Cognitive Psychology, 28(2), 238-248.

109 Ronan Eric Reilly

Brotherton, R., & French, C. C. (2015). Intention seekers: Conspiracist ideation and biased attributions of intentionality. PloS one, 10(5), e0124125.

Brotherton, R., French, C. C., & Pickering, A. (2013). Measuring Belief in Conspiracy Theories: The Generic Conspiracist Beliefs Scale. Frontiers in Psychology, 4(279).

Bruder, M., Haffke, P., Neave, N., Nouripanah, N., & Imhoff, R. (2013). Measuring individual differences in generic beliefs in conspiracy theories across cultures: Conspiracy Mentality Questionnaire. Frontiers in Psychology, 4, 1-15.

Bullshit. (2020). In: Oxford Learners Dictionaries. Retrieved from https://www.oxfordlearnersdictionaries.com/definition/english/bullshit_1?q=bullshit.

Butter, M., & Knight, P. (2016). Bridging the Great Divide: Conspiracy Theory Research for the 21st Century. Diogenes, 1-13.

Byford, J. (2011). Conspiracy theories: A critical introduction: Springer.

Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of personality and , 42(1), 116.

Carifio, J., & Perla, R. J. (2007). Ten common misunderstandings, misconceptions, persistent myths and urban legends about Likert scales and Likert response formats and their antidotes. Journal of social sciences, 3(3), 106-116.

Castanho Silva, B., Vegetti, F., & Littvay, L. (2017). The elite is up to something: Exploring the relation between populism and belief in conspiracy theories. Swiss Political Science Review, 23(4), 423-443.

Cichocka, A., Marchlewska, M., & de Zavala, A. G. (2016). Does self-love or self-hate predict conspiracy beliefs? Narcissism, self-esteem, and the endorsement of conspiracy theories. Social Psychological and Personality Science, 7(2), 157-166.

Cichocka, A., Marchlewska, M., de Zavala, A. G., & Olechowski, M. (2016). ‘They will not control us’: Ingroup positivity and belief in intergroup conspiracies. British Journal of Psychology, 107(3), 556-576.

Clarke, J. (2019). Critical Dialogues: Thinking Together in Turbulent Times. Bristol: Policy Press.

Conspiracy. (2020). In: Oxford Learners Dictionaries. Retrieved from https://www.oxfordlearnersdictionaries.com/definition/english/conspiracy?q=conspiracy.

Conspiracy theory. (2020). In: Oxford Learners Dictionaries. Retrieved from https://www.oxfordlearnersdictionaries.com/definition/english/conspiracy- theory?q=conspiracy+theory.

Cook, T. E., Gronke, P., & Rattliff, J. (2000). Disdaining the media: The American public’s changing attitudes toward the news. Paper presented at the annual meeting of the American Political Science Association, Washington, DC.

Craft, S., Ashley, S., & Maksl, A. (2017). News media literacy and conspiracy theory endorsement. Communication and the Public, 2(4), 388-401.

110 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Craft, S., Maksl, A. M., & Ashley, S. D. (2013). Measuring news media literacy: How knowledge and motivations combine to create news-literate teens. Retrieved from https://mospace.umsystem.edu/xmlui/bitstream/handle/10355/34713/MeasuringNewsMedia Literacy.pdf?sequence=1&isAllowed=y

Douglas, K. M., & Sutton, R. M. (2008). The hidden impact of conspiracy theories: Perceived and actual influence of theories surrounding the death of Princess Diana. The Journal of social psychology, 148(2), 210-222.

Douglas, K. M., & Sutton, R. M. (2015). Climate change: Why the conspiracy theories are dangerous. Bulletin of the Atomic Scientists, 71(2), 98-106.

Douglas, K. M., Sutton, R. M., & Cichocka, A. (2017). The Psychology of Conspiracy Theories. Current Directions in Psychological Science, 26(6), 538-542.

Douven, I. (2018). A Bayesian perspective on Likert scales and central tendency. Psychonomic bulletin & review, 25(3), 1203-1211.

Downing, J. D. H. (2001). Radical Media: Rebellious Communication and Social Movements. Thousand Oaks, CA: Sage Publications.

Ennis, R. H. (1996). Critical thinking dispositions: Their nature and assessability. Informal logic, 18(2), 165-182.

Ennis, R. H. (2003). Critical thinking assessment. In D. Fasko (Ed.), Critical thinking and reasoning: Current research, theory and practice (pp. 293–313). Cresskill, NJ: Hampton Press.

Ennis, R. H. (2015). Critical Thinking: A Streamlined Conception. In M. Davies & R. Barnett (Eds.), The Palgrave Handbook of Critical Thinking in Higher Education. New York, NY: Palgrave Macmillan.

Epstein, S., Pacini, R., Denes-Raj, V., & Heier, H. (1996). Individual differences in intuitive– experiential and analytical–rational thinking styles. Journal of personality and social psychology, 71(2), 390.

Erlandsson, A., Nilsson, A., Tinghög, G., & Västfjäll, D. (2018). Bullshit-sensitivity predicts . PloS one, 13(7), e0201474.

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction (“The Delphi Report”). Millbrae, CA: California Academic Press. ERIC Document Reproduction, No. ED 315423.

Facione, P. A. (2011). Critical thinking: What it is and why it counts. Insight assessment, 2007(1), 1-23.

Facione, P. A., & Facione, N. C. (1992). California critical thinking disposition inventory. Millbrae, CA: California Academic Press.

Facione, P. A., Sanchez, C. A., Facione, N. C., & Gainen, J. (1995). The disposition toward critical thinking. The Journal of General Education, 44(1), 1-25.

111 Ronan Eric Reilly

Fasko, D. (2003). Critical thinking: Origins, historical development, future directions. In D. Fasko (Ed.), Critical thinking and reasoning: Current research, theory and practice (pp. 3-20). Cresskill, NJ: Hampton Press.

Feuerstein, M. (1999). Media literacy in support of critical thinking. Journal of Educational Media, 24(1), 43-54.

Field, A. (2009). Discovering Statistics using SPSS. Thousand Oaks, CA: SAGE Publications Inc.

Fink, A. (2013). How to Conduct Surveys: A Step-by-Step Guide (Fifth edition ed.). Los Angeles, CA: SAGE Publications, Inc.

Foster, L., Jefferies, J., & Diamond, I. (2015). Beginning statistics : An introduction for social scientists (2nd ed. ed.). Los Angeles, CA: Sage.

Fuchs, C. (2010). Alternative media as critical media. European journal of social theory, 13(2), 173-192.

Gaziano, C., & McGrath, K. (1986). Measuring the concept of credibility. Journalism Quarterly, 63(3), 451-462.

Gentzkow, M. A., & Shapiro, J. M. (2004). Media, education and anti-Americanism in the Muslim world. Journal of Economic perspectives, 18(3), 117-133.

Goertzel, T. (1994). Belief in conspiracy theories. , 15(4), 731-742.

Graeupner, D., & Coman, A. (2017). The dark side of meaning-making: How social exclusion leads to superstitious thinking. Journal of Experimental Social Psychology, 69, 218-222.

Gray, L., & Burroughs, W. (1987). Constitutional Issues: Watergate and the Constitution. Social Education, 51(2), 88-90.

Grimes, D. R. (2016). On the viability of conspiratorial beliefs. PloS one, 11(1).

Groh, D. (1987). The temptation of conspiracy theory, or: Why do bad things happen to good people? In F. Graumann & S. Moscovici (Eds.), Changing conceptions of conspiracy (pp. 1- 37). New York, NY: Springer.

Grzesiak-Feldman, M. (2013). The effect of high-anxiety situations on conspiracy thinking. Current Psychology, 32(1), 100-118.

Halpern, D. F. (1998). Teaching critical thinking for transfer across domains: Disposition, skills, structure training, and metacognitive monitoring. American psychologist, 53(4), 449.

Halpern, D. F. (2003). The “how” and “why” of critical thinking assessment. In Critical thinking and reasoning: Current research, theory and practice (pp. 355–366). Cresskill, NJ: Hampton Press.

Hart, J. (2014). Toward an integrative theory of psychological defense. Perspectives on psychological science, 9(1), 19-39.

Hart, J., & Graether, M. (2018). Something’s going on here: Psychological predictors of belief in conspiracy theories. Journal of Individual Differences, 39(4), 229-237.

112 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Hemmingsen, A.-S., & Castro, K. I. (2017). The trouble with counter-narratives: DIIS Report.

Hobbs, R., & Frost, R. (2003). Measuring the Acquisition of Media-Literacy Skills. Reading Research Quarterly, 38(8), 330-355.

Hofstadter, R. (1966). The paranoid style in American politics. In H. R. (Ed.), The paranoid style in American politics and other essays (pp. 3-44). New York, NY: Knopf.

Hofstadter, R. (1971). The fear of conspiracy: Images of un-American subversion from the revolution to the present. In D. B. Davis (Ed.), The paranoid style in American politics (Vol. 113, pp. 2-8). Ithaca, NY: Cornell University Press.

Icke, D. (1999). The biggest secret. USA: Bridge of Love Publications.

Imhoff, R., & Bruder, M. (2014). Speaking (un‐) truth to power: Conspiracy mentality as a generalised political attitude. European Journal of Personality, 28(1), 25-43.

Jamieson, S. (2004). Likert scales: how to (ab) use them. Medical education, 38(12), 1217-1218.

Jensen, T. (2013). Democrats and Republicans differ on conspiracy theory beliefs. Retrieved from http://www.publicpolicypolling.com/polls/democrats-and-republicansdiffer-on-conspiracy- theory-beliefs/

Jevons, W. S. (1872). Elementary lessons in logic: Deductive and inductive. London: MacMillan and Co.

Jolley, D., & Douglas, K. M. (2014a). The effects of anti-vaccine conspiracy theories on vaccination intentions. PloS one, 9(2), e89177.

Jolley, D., & Douglas, K. M. (2014b). The social consequences of conspiracism: Exposure to conspiracy theories decreases intentions to engage in politics and to reduce one's carbon footprint. British Journal of Psychology, 105(1), 35-56.

Joshi, A., Kale, S., Chandel, S., & Pal, D. K. (2015). Likert scale: Explored and explained. Current Journal of Applied Science and Technology, 396-403.

Keeley, B. L. (1999). Of conspiracy theories. The Journal of Philosophy, 96(3), 109-126.

Kelly Garrett, R. (2016). Making sense of the Scalia conspiracy theory. Retrieved from https://theconversation.com/making-sense-of-thescalia-conspiracy-theory-55083

Kohring, M., & Matthes, J. (2007). Trust in news media: Development and validation of a multidimensional scale. Communication research, 34(2), 231-252.

Krumpal, I. (2013). Determinants of social desirability bias in sensitive surveys: a literature review. Quality & Quantity, 47(4), 2025-2047.

Ku, K. Y. L. (2009). Assessing students’ critical thinking performance: Urging for measurements using multi-response format. Thinking skills and , 4(1), 70-76.

Ku, K. Y. L., & Ho, I. T. (2010). Dispositional factors predicting Chinese students’ critical thinking performance. Personality and Individual Differences, 48(1), 54-58.

113 Ronan Eric Reilly

Kuru, O., & Pasek, J. (2016). Improving social media measurement in surveys: Avoiding acquiescence bias in Facebook research. Computers in Human Behavior, 57, 82-92.

Lantian, A., Muller, D., Nurra, C., & Douglas, K. M. (2016). Measuring Belief in Conspiracy Theories: Validation of a French and English Single-Item Scale. International Review of Social Psychology, 29(1), 1-14.

Lantian, A., Muller, D., Nurra, C., & Douglas, K. M. (2017). I know things they don’t know! Social psychology, 48(3), 160-173.

Lavrakas, P. J. (2008). Nonsampling Error. In P. J. Lavrakas (Ed.), Encyclopedia of Survey Research Methods (pp. 539-540). Thousand Oaks, CA: Sage Publications, Inc.

Leman, P. J., & Cinnirella, M. (2007). A major event has a major cause: Evidence for the role of heuristics in reasoning about conspiracy theories. Social Psychological Review, 9(2), 18-28.

Lewandowsky, S., & Cook, J. (2020). The Conspiracy Theory Handbook. Retrieved from http://sks.to/conspiracy

Lewandowsky, S., Cook, J., & Lloyd, E. (2016). The ‘Alice in Wonderland’mechanics of the rejection of (climate) science: simulating coherence by conspiracism. Synthese, 195(1), 175- 196.

Lewandowsky, S., Cook, J., Oberauer, K., Brophy, S., Lloyd, E. A., & Marriott, M. (2015). Recurrent fury: Conspiratorial discourse in the blogosphere triggered by research on the role of conspiracist ideation in climate denial. Journal of Social and Political Psychology, 3(1), 142-178.

Lewandowsky, S., Oberauer, K., & Gignac, G. E. (2013). NASA faked the moon landing— therefore,(climate) science is a hoax: An anatomy of the motivated rejection of science. Psychological science, 24(5), 622-633.

Likert, R. (1932). A technique for the measurement of attitudes. Archives of psychology.

Lins de Holanda Coelho, G., Hanel, P. H. P., & Wolf, L. J. (2018). The very efficient assessment of need for cognition: Developing a six-item version. Assessment, 27(8), 1870-1885.

Lynas, M. (2020). COVID: Top 10 current conspiracy theories. Retrieved from https://allianceforscience.cornell.edu/blog/2020/04/covid-top-10-current-conspiracy-theories

MacKinnon, J. B. (2016). The L.E.D. Quandary: Why There's No Such Thing as "Built to Last". The New Yorker. Retrieved from https://www.newyorker.com/business/currency/the-l-e-d- quandary-why-theres-no-such-thing-as-built-to-last

Maksl, A., Ashley, S., & Craft, S. (2015). Measuring News Media Literacy. Journal of Media Literacy Education, 6(3), 29-45.

Marchlewska, M., Cichocka, A., & Kossowska, M. (2018). Addicted to answers: Need for cognitive closure and the endorsement of conspiracy beliefs. European journal of social psychology, 48(2), 109-117.

114 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

McCauley, C., & Jacques, S. (1979). The popularity of conspiracy theories of presidential assassination: A Bayesian analysis. Journal of personality and social psychology, 37(5), 637.

Medieklagenemnda. (2020). Om Medieklagenemnda. Retrieved from https://www.klagenemndssekretariatet.no/medieklagenemnda/om-oss

Medietilsynet. (2019). Kritisk medieforståelse i den norske befolkningen, Delrapport 1. Retrieved from https://medietilsynet.no/globalassets/publikasjoner/2020/kritisk-medieforstaelse- samlerapport-og-delrapporter/delrapport-1-kmf-falske-nyheter.pdf

Medietilsynet. (2020). Våre oppgåver. Retrieved from https://medietilsynet.no/om/vare-oppgaver/

Meyer, P. (1988). Defining and Measuring Credibility of Newspapers: Developing an Index. Journalism Quarterly, 65(3), 567-574.

Miller, S. (2002). Conspiracy theories: public arguments as coded social critiques: a rhetorical analysis of the TWA flight 800 conspiracy theories. Argumentation and Advocacy, 39(1), 40-56.

Moulding, R., Nix-Carnell, S., Schnabel, A., Nedeljkovic, M., Burnside, E. E., Lentini, A. F., & Mehzabin, N. (2016). Better the devil you know than a world you don't? Intolerance of uncertainty and worldview explanations for belief in conspiracy theories. Personality and Individual Differences, 98, 345-354.

Mshvenieradze, T. (2013). Logos ethos and pathos in political discourse. Theory and Practice in Language Studies, 3(11), 1939-1945.

Munshi, J. (2014). A method for constructing Likert scales. SSRN 2419366.

Murray, J. (2013). Likert data: what to use, parametric or non-parametric? International Journal of Business and Social Science, 4(11), 258-264.

NESH. (2016). Guidelines for Research Ethics in the Social Sciences, Humanities, Law and Theology: The National Committee for Research Ethics in the Social Sciences and the Humanities.

Oliver, J. E., & Wood, T. J. (2014a). Conspiracy theories and the paranoid style (s) of mass opinion. American Journal of Political Science, 58(4), 952-966.

Oliver, J. E., & Wood, T. J. (2014b). Medical conspiracy theories and health behaviors in the United States. JAMA internal medicine, 174(5), 817-818.

Olsen, R. K. (2013). Hva er journalistisk kvalitet? In S. E. Omdal (Ed.), Journalistikk og demokrati. Hvor går mediene? Hva kan gjøres? (pp. 51-64). Oslo: Fritt Ord.

Pariser, E. (2011). The : what the Internet is hiding from you. New York: Penguin Press.

Paul, R. W., Elder, L., & Bartell, T. (1997). California Teacher Preparation for Instruction in Critical Thinking: Research Findings and Policy Recommendations. Sacramento, CA: California Commission on Teacher Credentialing. ERIC Document Reproduction, No. ED 437379.

115 Ronan Eric Reilly

Pennycook, G., Cheyne, J. A., Barr, N., Koehler, D. J., & Fugelsang, J. A. (2015). On the reception and detection of pseudo-profound bullshit. Judgment and Decision Making, 10(6), 549-563.

Pennycook, G., & Rand, D. G. (2019). Who falls for fake news? The roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking. Journal of personality, 2019(00), 1– 16.

Perkins, D. N., Jay, E., & Tishman, S. (1993). Beyond abilities: A dispositional theory of thinking. Merrill-Palmer Quarterly (1982-), 1-21.

PFU. (2020). Spørsmål og svar. Retrieved from https://presse.no/pfu/sporsmal-og-svar/

Plomin, R. S., & Post, J. M. (1997). Political paranoia. New Haven, CT: Yale University Press.

Ponsot, A. S., Autixier, C., & Madriaza, P. (2017). Factors facilitating the successful implementation of a prevention of violent radicalization intervention as identified by front- line practitioners. Journal for Deradicalization(16), 1-33.

Potter, W. J. (2004). Theory of media literacy: A cognitive approach. Thousand Oaks, CA: Sage.

Potter, W. J. (2010a). Media literacy, 5th edition. Thousand Oaks, CA: Sage.

Potter, W. J. (2010b). The state of media literacy. Journal of Broadcasting & Electronic Media, 54(4), 675-696.

Primack, B. A., Gold, M. A., Switzer, G. E., Hobbs, R., Land, S. R., & Fine, M. J. (2006). Development and validation of a smoking media literacy scale for adolescents. Archives of pediatrics & adolescent medicine, 160(4), 369-374.

Robinson, G. (2019). Mass Commnunication and Journalism. Waltham Abbey Essex: ED-Tech Press.

Rogers, E. M., & Dearing, J. W. (1988). Agenda-Setting Research: Where Has It Been, Where Is It Going? In J. A. Anderson (Ed.), Communication Yearbook 11 (pp. 555-594). New York, NY: Routledge.

Rotter, J. B. (1967). A new scale for the measurement of interpersonal trust. Journal of personality, 35, 651–665.

Salmon, M. H. (2013). Introduction to Logic and Critical Thinking, Sixth Edition. Boston, MA: Wadsworth Cengage Learning.

Schmit, J. B., Rieger, D., Ernst, J., & Roth, H.-J. (2018). Critical media literacy and Islamist online propaganda: The feasibility, applicability and impact of three learning arrangements. International Journal of Conflict and , 12, 1-19.

Scriven, M., & Paul, R. (1987). Critical thinking. Paper presented at the The 8th Annual International Conference on Critical Thinking and Education Reform, CA.

Silverblatt, A., & Eliceiri, E. M. E. (1997). Dictionary of media literacy. Westport, CT: Greenwood Press.

Sosu, E. M. (2013). The development and psychometric validation of a Critical Thinking Disposition Scale. Thinking skills and creativity, 9, 107-119.

116 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Stempel, C., Hargrove, T., & Stempel III, G. H. (2007). Media use, social structure, and belief in 9/11 conspiracy theories. Journalism & Mass Communication Quarterly, 84(2), 353-372.

Ståhl, T., & Van Prooijen, J.-W. (2018). Epistemic rationality: Skepticism toward unfounded beliefs requires sufficient cognitive ability and motivation to be rational. Personality and Individual Differences, 122, 155-163.

Sunstein, C. R. (2009). Republic.Com 2.0. Princeton, NJ: Princeton University Press.

Sunstein, C. R. (2014). Conspiracy theories & other dangerous ideas. New York: Simon & Schuster.

Sunstein, C. R., & Vermeule, A. (2009). Conspiracy Theories: Causes and Cures. Journal of Political Philosophy, 17(2), 202-227.

Sutton, R. M., & Douglas, K. M. (2014). Examining the monological nature of conspiracy theories. In J.-W. van Prooijen & P. A. M. van Lange (Eds.), Power, politics, and paranoia: Why people are suspicious oftheir leaders (pp. 254– 273). Cambridge, UK: Cambridge University Press.

Swami, V., Chamorro‐Premuzic, T., & Furnham, A. (2010). Unanswered questions: A preliminary investigation of personality and individual difference predictors of 9/11 conspiracist beliefs. Applied Cognitive Psychology, 24(6), 749-761.

Swami, V., & Coles, R. (2010). The truth is out there: Belief in conspiracy theories. The Psychologist, 23(7), 560-563.

Swami, V., Coles, R., Stieger, S., Pietschnig, J., Furnham, A., Rehim, S., & Voracek, M. (2011). Conspiracist ideation in Britain and Austria: Evidence of a monological belief system and associations between individual psychological differences and real‐world and fictitious conspiracy theories. British Journal of Psychology, 102(3), 443-463.

Swami, V., Pietschnig, J., Tran, U. S., Nader, I. W., Stieger, S., & Voracek, M. (2013). Lunar lies: The impact of informational framing and individual differences in shaping conspiracist beliefs about the moon landings. Applied Cognitive Psychology, 27(1), 71-80.

Swami, V., Voracek, M., Stieger, S., Tran, U. S., & Furnham, A. (2014). Analytic thinking reduces belief in conspiracy theories. Cognition, 133(3), 572-585.

Thompson, D. (2008). Counterknowledge: How We Surrendered to Conspiracy Theories, Quack Medicine, Bogus Science and Fake History. London: Atlantic Books.

Thompson, J. B. (1995). The Media and Modernity: A Social Theory of the Media. Stanford, CA: Stanford University Press.

Tsfati, Y. (2003). Media Skepticism and Climate of Opinion Perception. International Journal of Public Opinion Research, 15(1), 65-82.

Tsfati, Y., & Cappella, J. N. (2003). Do people watch what they do not trust? Exploring the association between news media skepticism and exposure. Communication research, 30(5), 504-529.

117 Ronan Eric Reilly

Tsfati, Y., & Peri, Y. (2006). Mainstream media skepticism and exposure to sectorial and extranational news media: The case of . Mass Communication & Society, 9(2), 165- 187.

Turow, J. (1997). Breaking up America: Advertisers and the new media world. Chicago: University of Chicago Press.

Uscinski, J. E. (2019a). Down the Rabbit Hole We Go! In J. E. Uscinski (Ed.), Conspiracy Theories and the People Who Believe Them (pp. 1-32). New York, NY: Oxford University Press.

Uscinski, J. E. (2019b). What is a Conspiracy Theory? In J. E. Uscinski (Ed.), Conspiracy Theories and the People Who Believe Them (pp. 47-52). New York, NY: Oxford University Press.

Valentine, D. (2016). The CIA as Organized Crime: How Illegal Operations Corrupt America and the World. Atlanta, GA: Clarity Press. van Alphen, A., Halfens, R., Hasman, A., & Imbos, T. (1994). Likert or Rasch? Nothing is more applicable than good theory. Journal of advanced nursing, 20(1), 196-201. van Buuren, J. (2013). Spur to violence?: Anders Behring Breivik and the Eurabia conspiracy. Nordic journal of migration research, 3(4), 205-215. van der Linden, S. (2015). The conspiracy-effect: Exposure to conspiracy theories (about global warming) decreases pro-social behavior and science acceptance. Personality and Individual Differences, 87(C), 171-173. van Prooijen, J.-W., & Douglas, K. M. (2017). Conspiracy theories as part of history: The role of societal crisis situations. Memory Studies, 10, 323-333. van Prooijen, J.-W., & Douglas, K. M. (2018). Belief in conspiracy theories: Basic principles of an emerging research domain. European journal of social psychology, 48(7), 897-908. van Prooijen, J.-W., Krouwel, A. P., & Pollet, T. V. (2015). Political extremism predicts belief in conspiracy theories. Social Psychological and Personality Science, 6(5), 570-578.

Voogt, S. (2017). Countering far-right recruitment online: CAPE’s practitioner experience. Journal of policing, intelligence and counter terrorism, 12(1), 34-46.

Vraga, E. K., Tully, M., Akin, H., & Rojas, H. (2012). Modifying perceptions of hostility and credibility of news coverage of an environmental controversy through media literacy. Journalism, 13(7), 942-959.

Vraga, E. K., Tully, M., Kotcher, J. E., Smithson, A.-B., & Broeckelman-Post, M. (2015). A Multi- Dimensional Approach to Measuring News Media Literacy. Journal of Media Literacy Education, 7(3), 41-53.

Vraga, E. K., Tully, M., & Rojas, H. (2009). Media literacy training reduces perception of bias. Newspaper Research Journal, 30(4), 68-81.

Wadgave, U., & Khairnar, M. R. (2019). Parametric test for non-normally distributed continuous data: For and against. Electronic Physician, 11(2), 7468-7470.

118 “The truth is in doubt here!”: Critically assessing sources and information in a conspiratorial world

Wallston, K. A., Strudler Wallston, B., & DeVellis, R. (1978). Development of the multidimensional health locus of control (MHLC) scales. Health education monographs, 6(1), 160-170.

Walsh, C. M., Seldomridge, L. A., & Badros, K. K. (2007). California critical thinking disposition inventory: Further factor analytic examination. Perceptual and motor skills, 104(1), 141- 151.

Watson, G., & Glaser, E. M. (1980). Watson–Glaser critical thinking appraisal. Cleveland, OH: Psychological Corporation.

Weijters, B., Baumgartner, H., & Schillewaert, N. (2013). Reversed item bias: An integrative model. Psychological methods, 18(3), 320.

West, M. D. (1994). Validating a scale for the measurement of credibility: A covariance structure modeling approach. Journalism Quarterly, 71(1), 159-168.

West, R. F., Toplak, M. E., & Stanovich, K. E. (2008). Heuristics and biases as measures of critical thinking: Associations with cognitive ability and thinking dispositions. Journal of , 100(4), 930-941.

Whitson, J. A., & Galinsky, A. D. (2008). Lacking control increases illusory pattern perception. science, 322(5898), 115-117.

Whitson, J. A., Galinsky, A. D., & Kay, A. (2015). The emotional roots of conspiratorial perceptions, system justification, and belief in the paranormal. Journal of Experimental Social Psychology, 56, 89-95.

WKU. (2020). Evaluating Information and Avoiding Fake News. Retrieved from https://libguides.wku.edu/c.php?g=740256&p=5330482

Wood, M. J., Douglas, K. M., & Sutton, R. M. (2012). Dead and alive: Beliefs in contradictory conspiracy theories. Social Psychological and Personality Science, 3(6), 767-773.

Wright, K. B. (2017). Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring Software Packages, and Web Survey Services. Journal of Computer-Mediated Communication, 10(3).

119 Ronan Eric Reilly

Appendecies:

Appendix A: Complete questionnaire (Norwegian). Appendix B: Likert scales: Operationalization and origin. Appendix C: CRAAP-test Appendix D: RQ1 Statistics Appendix E: RQ2 Statistics Appendix F: RQ3 Statistics Appendix G: RQ4 Statistics Appendix H: RQ5 Statistics Appendix I: RQ6 Statistics Appendix J: RQ7 Statistics Appendix K: RQ8 Statistics

120 APPENDIX A: COMPLETE QUESTIONNAIRE (NORWEGIAN)

Nyhetsmedier: Tanker, Tillit, Teorier og Tolkninger *Må fylles ut

Velkommen!

Dette er en undersøkelse om mennesker i møte med omverdenen og nyhetsmediene - og om hvordan vi tolker, vurderer og prosesserer informasjonen vi mottar. Undersøkelsen inngår som en del av forskningsgrunnlaget i en masteroppgave i Journalistikk ved Universitet i Oslo. Det settes enormt stor pris på at du avsetter din verdifulle tid til dette, og svarene du avgir bidrar til medieforskning. Undersøkelsen vil typisk ta ca 10-15 minutter å gjennomføre.

Som et lite insentiv til deltagelse vil det for hver eneste mottatte respons bli overført kr 5,- i støtte til Barnekreftforeningen. Dersom minst 100 responser oppnås, så tillegges totaldonasjonen ytterligere kr 500,-.

Jeg skulle gjerne også premiert deltagere direkte, men for å kunne ivareta personvern og sikre anonymitet har jeg valgt å ikke forsøke å føre lister med kontaktinformasjon til respondenter. Hvis du ikke har åpnet denne undersøkelsen via linken min på Facebook, så gjør jeg allikevel oppmerksom på at du der kan være med i trekningen av kr 500,- dersom du liker og deler innlegget (forutsetter 100+ responser): https://www.facebook.com/ronan.e.reilly/posts/10162122688495648

Denne undersøkelsen er altså helt anonym, og det er dermed ikke mulig å spore hvem som har avgitt hvilke svar.

Ansvarlig for undersøkelsen er: Ronan Eric Reilly

Eventuelle henvendelser eller spørsmål kan rettes til: [email protected]

For å oppnå korrekte resultater er det viktig at du hele tiden forsøker å svare så presist som mulig. Les spørsmålene nøye, og forsøk i størst mulig grad å unngå å svare slik du "antar at det forventes".

Klikk "Neste" for å begynne...

På forhånd tusen takk:)

i

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

Litt om dine nyhetsmedievaner Til å begynne med følger noen spørsmål som kartlegger hvor og hvordan du konsumerer nyheter.

1. Hvilket medie vil du si at du primært benytter til nyheter? * Hvis du bruker flere, velg det alternativet du benytter mest. Markér bare én oval. Internett (nettaviser o.l. - IKKE sosiale medier). Sosiale mediekanaler (Facebook, Twitter o.l). TV. Papiraviser. Magasiner / Tidsskrifter. Radio. Oppsøker lite nyheter selv, men får med meg det jeg trenger å vite gjennom interaksjon med andre. Jeg har ingen klare preferanser. Annet.

2. Hvor ofte vil du si at du oppsøker dine vante nyhetskilder? * Eks: Setter på TV- eller radionyhetene, kjøper aviser/tidsskrifter, besøker nettsteder direkte eller via apper o.l. Markér bare én oval. Flere ganger om dagen. Daglig. 4-6 dager i uken. Ukentlig. Månedlig. Sjeldnere enn månedlig. Aldri. Vet ikke / Har ikke tatt stilling til dette.

3. Hvor ofte vil du si at du finner nye nyhetskilder uten at du aktivt har oppsøkt de selv? * Eks: Åpner forslag/linker til nyhetssaker i sosiale medier, via blogger eller epost, blir gitt brosjyrer/skriv e.l. Markér bare én oval. Flere ganger om dagen. Daglig. 4-6 dager i uken. Ukentlig. Månedlig. Sjeldnere enn månedlig. Aldri. Vet ikke / Har ikke tatt stilling til dette.

4. Enkelte nyhetskilder blir ofte beskrevet som (eller betegner seg selv som) "alternative". Hva betyr dette begrepet for deg? * Velg det som passer best med din oppfatning. Markér bare én oval. Nyhetskilder for spesielt interesserte innenfor visse temaer, uten noen særskilt agenda. Ambisiøse og modige nyhetskilder som virkelig tør å formidle sannheten.

ii

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

Nyhetskilder med politiske vridninger, kritikkverdig journalistikk og lav kvalitet og troverdighet. Nyhetskilder som har mistet tilliten til massemediene, og som derfor føler et behov for å gjøre egne undersøkelser. Annet. Vet ikke / Har ikke tatt stilling til dette.

5. Hvor ofte vil du si at du leser en nyhetssak fra en kilde du selv anser for å være alternativ? * Markér bare én oval. Flere ganger om dagen. Daglig. 4-6 dager i uken. Ukentlig. Månedlig. Sjeldnere enn månedlig. Aldri. Vet ikke / Har ikke tatt stilling til dette.

6. Hvilke av følgende medier beskriver best din foretrukne nyhetskilde? * Markér bare én oval. En av de største norske massemediene (VG, Dagbladet, Aftenposten, NRK, TV2, Nettavisen o.l). Andre norske kjente etablerte nyhetskilder. Andre norske alternative nyhetskilder. Viktige nyheter deles stort sett med meg via sosiale mediekanaler, epost e.l. Store utenlandske nyhetsmassemedier (BBC, CNN, Sky, Al Jazeera o.l). Andre alternative utenlandske nyhetskilder. Annet. Vet ikke / Har ikke tatt stilling til dette.

Tillit til verden og nyhetsmediene Det følger nå noen utsagn om ditt syn på verden og nyhetsmediene, og det er ønskelig at du vurderer hvor enig du er.

7. Jeg tror at... * Vennligst indiker på en skala fra 1-5 hvor enig/uenig du er. 1= Helt enig - 3= Ingen formening - 5= Helt uenig. Markér bare én oval per rad 1 2 3 4 5 Ingen Helt enig Litt enig Litt uenig Helt uenig formening

...det ofte skjer mange viktige ting i verden som befolkningen aldri blir informert om.

...det er svært vanlig at politikere ikke forteller oss de virkelige motivene bak valgene de tar.

...myndighetene ved hjelp av ulike etater følger nøye med på/overvåker innbyggerne sine.

...det er en sammenheng mellom mange hendelser selv om det tilsynelatende ikke virker slik, og at det finnes hemmeligholdte årsaker til dette.

iii

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

...det finnes ukjente organisasjoner som har meget stor innflytelse på politiske avgjørelser.

8. Jeg mener generelt sett at nyhetsformidlende massemedier... * Vennligst indiker på en skala fra 1-5 hvor enig/uenig du er. 1= Helt enig - 3= Ingen formening - 5= Helt uenig. Markér bare én oval per rad 1 2 3 4 5 Ingen Helt enig Litt enig Litt uenig Helt uenig formening

...er til å stole på.

...rapporterer korrekte fakta.

...er rettferdige. ...forteller "hele historien".

...er objektive.

9. Jeg syntes at det er... * Vennligst indiker på en skala fra 1-5 hvor enig/uenig du er. 1= Helt enig - 3= Ingen formening - 5= Helt uenig. Markér bare én oval per rad 1 2 3 4 5 Ingen Helt enig Litt enig Litt uenig Helt uenig formening ...mitt eget ansvar å styre hvor og hvordan jeg oppnår informasjon.

...min egen skyld dersom jeg blir feilinformert av nyhetsmediene.

...opp til meg å evaluere nyhetene jeg mottar, og om nødvendig fremskaffe annen informasjon.

Argumenter og mening Du blir nå presentert for en rekke argumenter og utsagn, og det er ønskelig at du gir din vurdering av disse.

10. Hvor gode mener du argumentene nedenfor er? Forsøk å vurdere objektivt hvor godt påstandene støttes av det som hevdes, og IKKE om du selv er enig eller uenig. * Vennligst indiker på en skala fra 1-5. 1= Veldig godt argument - 3= Ingen formening - 5= Veldig dårlig argument. Markér bare én oval per rad 1 2 3 4 5 Veldig godt Litt godt Ingen Litt dårlig Veldig dårlig argum. argum. formening argum. argum.

Det finnes trolig folk som kan forutse fremtiden, fordi det er utallige eksempler på spådommer som oppfylles.

Selv om ytringsfrihet kan brukes som et påskudd for å si hva som helst til hvem som helst, så betyr ikke det at det alltid er lurt å gjøre det.

Det er altfor lav strafferamme for noe såpass grusomt som voldtekt, i og med at enkelte må sitte mye lengre i fengsel for skatteunndragelse.

iv

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

Politikk er komplisert,og meninger om enkeltsaker bør ikke overskygge helhetlig tankegang.

Millioner av mennesker verden over drikker PepsiMAX, så derfor må det være et veldig bra produkt.

Hvis FrP sier at vi må prioritere midler til nye veier, så mener de at de ønsker økt trafikkvolum og utslippsnivå.

Det er grunn til å tro at dersom man fremstår som vennlig og kunnskapsrik vil man bli bedre likt enn om man er irritert og useriøs.

Ingen kan motbevise at det finnes en skjebne, og derfor er det sannsynlig at alle store hendelser i livet er forutbestemte.

Integreringsproblemet må løses enten med innvandringsstopp, eller ved å få i gang et fungerende system for norsk språk- og kulturopplæring.

Akupunktur har blitt praktisert i tusener av år, og er en fullverdig del av helsevesenet i Øst-Asia. Dette gir et solid grunnlag for å hevde at det fungerer utmerket.

Dersom noen påstår at jorden er flat, og de baserer dette på feilaktige beviser og logiske brister, så betyr det at jorden ikke er flat.

Noen menn er veterinærer, og noen veterinærer er blåøyde. Derfor er noen menn blåøyde.

Støre mener vi må prioritere de vanskeligstilte, men han kan ikke forstå hva de trenger fordi han selv er søkkrik og kommer fra en velstående familie.

Vitenskapen har ikke alle svarene, men det er sannsynlig at en kreftforsker vet mer enn en kiropraktor om hva som kan kurere kreft.

Hvis en hudkrem inneholder kun økologiske og naturlige ingredienser, så må den være bedre for huden enn en krem som er full av kunstige stoffer.

11. Hvor meningsfulle syntes du følgende utsagn er? Gjør en totalvurdering av om både innholdet er forståelig, og om du finner en dypere underliggende betydning. * Vennligst indiker på en skala fra 1-5. 1= Veldig meningsfullt - 3= Ingen formening - 5= Veldig lite meningsfullt. Markér bare én oval per rad 1 2 3 4 5 Veldig Litt Litt lite Veldig lite Ingen menings- menings- menings- meningsfull formening fullt fullt fullt t Egoisme er ikke å leve som man selv ønsker, men å be andre leve som man selv ønsker.

Ynde er en visjon av visdom og styrke, men ikke nødvendigvis av primær vitalitet.

v

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

Evigvarende er den som ikke snur når veien deler seg.

Vi bør konsentrere oss om fremtiden, det er der vi skal være resten av livet.

Selv den som skal flytte et helt fjell, må begynne med å bære vekk små steiner.

Skjult betydning endrer uforlignelig abstrakt skjønnhet.

Stillheten rommer bare livets parallelle tanker, men først i evigheten vil sannheten være fri.

Å leve er som å tegne på blanke ark uten viskelær. Vurderinger og tanker om nyhetsmediene

Vurderinger og tanker om nyhetsmediene Nå følger en rekke spørsmål om nyhetsmediene, og om hvordan du oppfatter at de opererer og fungerer.

12. Hvor enig er du i følgende utsagn om nyhetsmediene?: * Vennligst indiker på en skala fra 1-5 hvor enig du er. 1= Helt enig - 3= Ingen formening - 5= Helt uenig. Markér bare én oval per rad 1 2 3 4 5

Ingen Helt enig Litt enig Litt uenig Helt uenig formening

Eieren av et medieselskap påvirker innholdet som blir produsert.

Nyhetsprodusenter velger nyhetssaker basert på hva som vil tiltrekke seg flest lesere/seere.

Folk velger nyhetskilder uavhengig av egne politiske verdisyn.

Hvis to personer ser akkurat den samme nyhetssaken er det gitt at de begge vil sitte igjen med den samme informasjonen.

Det er ikke alle mennesker som lar seg påvirke av hva de hører om i nyhetene.

Nyhetsdekningen av en politisk kandidat vil påvirke menneskers oppfatning av kandidaten.

Nyheter er ikke konstruert for å få oppmerksomhet fra publikum, men er laget for å informere.

Ulike teknikker under produksjonen (kameravinkling, lyssetting m.m) kan påvirke seeres oppfatning av hva som blir vist.

vi

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

Når bilder tas er det fotografer som avgjør hva det er viktigst å fokusere på.

Nyheter førsøker å unngå overdramatisering av virkeligheten.

En potensiell nyhetssak med godt bildemateriale har større sannsynlighet for å bli vist.

En nyhetssak som omhandler konflikt har større sannsynlighet for å bli fremhevet.

Man bedømmer viktigheten av en nyhetssak mer ut ifra alvoret i saken enn hvor ofte den blir omtalt og fremhevet.

Alternative teorier Det vil nå presenteres noen kontroversielle teorier som er basert på forklaringer eller syn som massemediene ikke nødvendigvis anerkjenner.

13. Hvor sannsynlig tror du det er at følgende teorier stemmer? * Vennligst indiker på en skala fra 1-5. 1= Svært sannsynlig - 3= Ingen formening - 5= Svært usannsynlig. Markér bare én oval per rad 1 2 3 4 5 Litt Svært Litt Ingen Svært usannsynl sannsynlig sannsynlig formening usannsynlig ig Legemiddelindustrien tilbakeholder informasjon om at mange vaksiner inneholder svært skadelige stoffer, og at de kan forårsake alvorlige bivirkninger som bl.a autisme.

Amerikanske myndigheter var selv involverte i terrorangrepene på WTC og Pentagon mm. i 2001.

CIA har utført bevissthets- og adferdseksperimenter på mennesker ved å hypnotisere, samt administrere bl.a LSD til intetanende mennesker.

Klimaproblemer som «global oppvarming» er i stor grad overdrevet, og hauses opp av aktører med politiske agendaer.

Månelandingen i 1969 ble spilt inn i et studio, og fabrikkert av NASA for å vinne «romkappløpet» med Sovjetunionen.

Det finnes et strengt bevoktet område, kalt Area 51, hvor det har pågått forskning på utenomjordiske livsformer og teknologi i mange tiår.

Russiske myndigheter har overvåket private hjem ved å samarbeide med radioprodusenter, som har sørget for systematisk installasjon av avlyttingsutstyr i hjemmeradioer.

vii

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

Prinsesse Diana døde ikke i en bilulykke grunnet ruspåvirket sjåfør i Paris i 1997, men ble utsatt for et attentat.

For å øke profitten ble de største lyspæreprodusentene på 1920-tallet enige om at samtlige med vilje skulle drastisk begrense levetiden på alle produserte lyspærer.

Det finnes en håndfull mennesker som utgjør en maktelite, og disse kontrollerer alle verdens politiske myndigheter og finansinstitusjoner.

Vurderinger og handlinger i møte med nyhetssaker I møte med en nyhetssak, hva slags tanker gjør du deg? Husk at det er viktig å være så ærlig med deg selv som mulig.

14. Du kommer over, eller blir presentert for, en nyhetssak. Hvor ofte eller sjelden vil du si at du gjør følgende? : * Vennligst indiker på en skala fra 1-5. 1= Ofte - 3= Ingen formening - 5= Svært sjelden. Markér bare én oval per rad 1 2 3 4 5 Noen Ingen Svært Ofte Sjelden ganger formening sjelden Jeg vurderer hvor lenge det er siden saken ble publisert og/eller endret sist.

Jeg vurderer hvilket publikum artikkelen er skrevet for.

Jeg vurderer hva som er kilden og/eller forfatterens kvalifikasjoner.

Jeg undersøker om det fremlegges ordentlige beviser for påstandene.

Jeg sjekker om informasjonen kan bekreftes av andre kilder.

Jeg vurderer om språket er følelsesladet eller nøytralt.

Jeg ser etter skrivefeil og/eller dårlig språk.

Jeg vurderer om saken er laget for å informere, selge, underholde eller overbevise.

Jeg vurderer om saken er basert på faktaopplysninger, synspunkter eller propaganda.

Jeg førsøker å avdekke om synspunktene er objektive, eller om de er farget av f.eks ideologiske, kulturelle eller religiøse holdninger.

viii

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

Tanker og kritiske blikk Nå følger en rekke spørsmål om hvordan du tenker og vurderer. Spørsmålene er ment generelt for alle situasjoner i livet, og IKKE kun i møte med mediene.

15. Vurder hvor enig du er i følgende utsagn om deg selv. * Vennligst indiker på en skala fra 1-5. 1= Helt enig - 3= Ingen formening - 5= Helt uenig. MERK: Gjelder ikke bare i forbindelse med mediebruk, men generelt for alle situasjoner i livet. Markér bare én oval per rad 1 2 3 4 5 Ingen Helt enig Litt enig Litt uenig Helt uenig formening

Jeg foretrekker kompliserte problemstillinger fremfor enkle.

Jeg liker aktiviteter som krever lite tenking bedre enn de som utfordrer tenkeferdighetene mine. I diskusjoner forsøker jeg hovedsakelig å vurdere helhetsbildet.

Jeg tyr som regel til kjente rutiner og metoder når jeg skal gjøre ting.

Jeg benytter meg av flere enn én enkelt kilde når jeg skal finne informasjon.

Jeg er til stadighet på utkikk etter nye ideer.

Jeg finner sjelden argumenter som er gode nok til å utfordre meg på ting jeg virkelig tror på. Det er viktig å forstå andres synspunkter når man diskuterer et tema.

Jeg føler stort sett ikke behov for å forsvare valgene jeg tar overfor andre.

Jeg reflekterer ofte over mine erfaringer, slik at jeg kan lære fra dem.

Det hender ganske ofte at jeg bedømmer noe før jeg har undersøkt om informasjonskilden er pålitelig.

Jeg tenker vanligvis over de langsiktige konsekvensene av en avgjørelse før jeg handler.

Så lenge metodene mine fungerer, så ser jeg sjelden god grunn til å forsøke å forbedre måten jeg gjør ting på.

Litt om hvem du er Tusen takk for tålmodigheten så langt... Undersøkelsen er straks ferdig, men helt til slutt er det fint om du kan gi litt grunnleggende opplysninger om deg selv. Minner igjen om at dette samles inn helt anonymt, men enkelte av spørsmålene gir deg selvsagt muligheten til det dersom du allikevel ikke skulle ønske å svare.

16. Er du.. * Markér bare én oval. Mann Kvinne

ix

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

17. Hvor gammel er du? * Markér bare én oval. 16 år eller yngre 17-24 25-34 35-44 45-54 55-64 65-74 75 år eller eldre

18. Oppgi gjerne din omtrentlige årsinntekt (brutto/før skatt) * Markér bare én oval. 0-100.000 Kr 100.001- 300.000 Kr 300.001 - 450.000 Kr 450.001 - 600.000 Kr 600.001 - 750.000 Kr 750.001 - 1.000.000 Kr Over 1.000.000 Kr Ønsker ikke å svare på dette

19. Hva er din høyeste fullførte utdannelse? * Markér bare én oval. Grunnskole / Ungdomsskole Videregående skole (3-årig) Universitet/Høyskole (3-årig Bachelor e.l grad) Universitet/Høyskole (5-årig Master e.l grad) Over 5 år høyere utdanning Ønsker ikke å svare på dette

20. Hva beskriver best ditt daglige hovedvirke de siste 2 årene? * Markér bare én oval. Heltidsarbeid (over 70% stilling) Deltidsarbeid (70% stilling eller mindre) Selvstendig næringsdrivende Student Hjemmeværende med barn Arbeidsledig / Arbeidssøkende Langtidspermittert Langtidssykemeldt / Uføretrygdet Pensjonist Kombinasjon av 2 eller flere Annet Ønsker ikke å svare på dette

x

APPENDIX A COMPLETE QUESTIONNAIRE (NORWEGIAN)

21. Velg alternativet som best beskriver ditt bosted * Markér bare én oval. Stor by (over 100.000 innbyggere) Middels stor by (25.000 - 100.000 innbyggere) Liten by / Tettsted (5000 - 25.000 innbyggere) Tettsted (2000 - 5000 innbyggere) Veldig lite tettsted (under 2000 innbyggere) Landlige omgivelser uten bysentrum

22. Hvem vil du si at du er mest enig med politisk? * Markér bare én oval. Dette varierer / Har ingen klar preferanse Arbeiderpartiet Høyre Fremskrittspartiet Kristelig Folkeparti Venstre Senterpartiet SV Rødt Miljøpartiet De Grønne Andre Ønsker ikke å svare på dette

23. Ser du på deg selv som religiøs? * Markér bare én oval. Ja, jeg er kristen Ja, jeg er muslim Ja, jeg er tilknyttet en annen religion enn kristendom og islam Jeg er ikke religiøs Jeg vet ikke helt / Har ikke bestemt meg Ønsker ikke å svare på dette

xi

APPENDIX B: LIKERT SCALES: OPERATIONALIZATION AND ORIGIN

Conspiracy Mentality Questionnaire (CMQ) Origin: Bruder et al. (2013) Scale: 1 (Strongly agree) - 5 (Strongly disagree); 3 = No opinon

Some statements regarding views about the CMQ_01 I think that many very important things happen in the world, which the public is never informed about. world and the news media will follow. CMQ_02 I think that politicians usually do not tell us the true motives for their decisions. Please rate the level to which you agree. CMQ_03 I think that government agencies closely monitor all citizens. CMQ_04 I think that events which superficially seem to lack a connection are often the result of secret activities. CMQ_05 I think that there are secret organizations that greatly influence political decisions.

News Media Credibility Scale (NCRED) Origin: Meyer (1988) / Gaziano & McGrath (1986) / West (1994) Scale: 1 (Strongly agree) - 5 (Strongly disagree); 3 = No opinon

Some statements regarding views about the NCRED_01 I generally believe that the news media can be trusted. world and the news media will follow. NCRED_02 I generally believe that the news media are accurate. Please rate the level at which you agree. NCRED_03 I generally believe that the news media are fair. NCRED_04 I generally believe that the news media tell the whole story. NCRED_05 I generally believe that the news media are unbiased.

Media Locus of Control Scale (MLOC3) Origin: Self composed/reduced - derived from Craft et al (2013) / Maksl et al (2015) / Wallston & Strudler Wallston (1978) Scale: 1 (Strongly agree) - 5 (Strongly disagree); 3 = No opinon

Some statements regarding views about the MLOC3_01 I believe that it is my own responsibility to control how I get my information, and where I get it from. world and the news media will follow. MLOC3_02 I believe that it is my own fault if I am misinformed by the news media. Please rate the level at which you agree. MLOC3_03 I believe that it is up to me to evaluate news messages I receive and, if needed, seek out other information.

Informal Logically Fallacious Arguments Scale (ILFARG) Origin: Self composed (see remarks for categorization) Scale: 1 (Very strong) - 5 (Very poor); 3 = No opinon

How would you rate the quality of the ILFARG_01 It is likely that some people can foresee the future, as there are countless examples of predictions that come true. Fallacy of 'anecdotal evidence' / 'hasty generalization' arguments below? Try to objectively assess ILFARG_02 Although 'the right to freedom of speech' can be used as an excuse to say anything to anyone, it does not mean that that it always is a good idea to do so. Reasonable argument / statement; Not included in analysis how well the conclusions are supportet by the claims, and NOT your level of ILFARG_03 The sentencing for the terrible act of rape is to low, given the fact that some people are incercerated much longer for tax evasion. Fallacy of 'faulty comparison' agreement with the contents. ILFARG_04 Politics are complicated, and therefore opinions about single matters ought to not overshadow holistic views. Reasonable argument / statement; Not included in analysis ILFARG_05 Millions of people all over the world drink PepsiMAX, and therefore it must be a very good product. Fallacy of 'argumentum ad populum' ILFARG_06 If FrP say that we must prioritze funds for new roads, they are really saying hat they wish to increase traffic volume and emissions. 'Strawman' fallacy ILFARG_07 There is reason to believe that someone who appears to be friendy and knowledgabe will be better liked than if they seem annoyed and unserious. Reasonable argument / statement; Not included in analysis ILFARG_08 Nobody can disprove that we all have a destiny, and therefore it is likely that all major events in life are predetermined. 'Burden of proof' fallacy ILFARG_09 The problem of integrating immigrants must be solved either by stopping immigration or by implementing a better system for teacing Norwegian language and culture. Fallacy of 'false dichotomy' ILFARG_10 Acupuncture has been practiced for thousands of years and is a fully accepted part of East-Asian healthcare. This provides a solid foundation to claim that it works well. Fallacy of 'appeal to age' / 'appeal to false authority' ILFARG_11 If someone claims that the earth is flat, and they base this on incorrect evidence and logical flaws, it means that the earth is not flat. 'fallacy fallacy' / 'non-sequitur'' ILFARG_12 Some men are veterinarians, and some veterinarians have blue eyes. Therefore, some men have blue eyes. Formal 'non-sequitur' fallacy ILFARG_13 Støre says we must prioritize the disadvantaged, but he cannot possibly understand what they need given his large personal wealth and affluent upbringing. 'Ad Hominem' fallacy ILFARG_14 Science does not have all the answers, but it is nevertheless more likely that a cancer researcher knows more about curing cancer than a chiropractor. Reasonable argument / statement; Not included in analysis ILFARG_15 If a skincare product contains only ecological and natural ingredients, then it must be better for the skin than a product that is full of artificial substances. Fallacy of 'appeal to nature'

Pseudo-profound Bullshit Receptivity- / Profundity Receptivity Scale(s) (BSREC4 / PREC4) Origin: Self composed - inspired by Pennycook et al. (2015) / Erlandsson et al. (2018) (see remarks for categorization) Scale: 1 (Very meaningful) - 5 (Very meaningless); 3 = No opinon

How meaningful do you find the following PREC4_01 Selfishness is not to live as you wish to, but to ask others to live as you wish them to. statements to be? Make an overall BSREC4_01 Grace is a vision of wisdom and strength, but not necessarily of primary vitality. assessment of both the understandability of the contents, and on whether a deeper BSREC4_02 Eternal is he who does not turn around when the road branches. profound meaning can be found. PREC4_02 We should concentrate on the future, as it is where we will spend the rest of our lives. PREC4_03 Even he who shall move an entire mountain must begin by carrying small rocks. BSREC4_03 Hidden meaning transforms unparalleled abstract beauty. BSREC4_04 Tranquility contains solely life's parallel thoughts, but primarily in eternity the truth will be released. PREC4_04 Living is like drawing on a blank sheet of paper without having an eraser.

xii

APPENDIX B LIKERT SCALES OPERATIONALIZATION / ORIGIN

News Media Literacy Scale (NML) Origin: Ashley et al. (2013) / Vraga et al. (2015) - some alterations made Scale: 1 (Strongly agree) - 5 (Strongly disagree); 3 = No opinon

To what degree do you agree with the NML_AA_01 The owner of a media company influences the content that is produced. following claims about the news media? NML_AA_02 News companies choose stories based on what will attract the biggest audience. NML_AA_03 Individuals largely chose news sources irrespective of their own political values. Item altered from original scale; Reverse Coded NML_MM_01 If two people see the same news story it is given that they will both be left with the same information. Item altered from original scale; Reverse Coded NML_MM_02 All people do not let themselves get affected by what they hear about in the news. Item altered from original scale; Reverse Coded NML_MM_03 News coverage of a political candidate will influence people’s opinions. NML_MM_04 News is not created to gain attention from the public, but exists to inform. Item altered from original scale; Reverse Coded NML_MM_05 Various production techniques (camera angle, lighting, etc.) can affect viewers’ perception of a story. Item from original scale joined together NML_MM_06 When taking pictures, photographers decide what is most important. NML_RR_01 News media attempt to avoid overdramatizing reality. Item altered from original scale; Reverse Coded; Omitted from analysis NML_RR_02 A news story that has good pictures is more likely to show up in the news. NML_RR_03 A news story about conflict is more likely to be featured prominently. NML_RR_04 The importance of a news story will be primarily judged by the seriousness of its nature - not by the frequency with which it is presented and emphasized. New self-composed item added to original scale; Revese Coded

Various Conspiracy Theory Endorsement Scale(s) (VCTE10 / VCTE8 / VCTE7 / TVC2) Origin: Self composed (selection of varous conspiracy theories, true conspiracies and self-concocted conspiracy theories) Scale: 1 (Very likely) - 5 (Very unlikely); 3 = No opinon

How likely do you think it is that the VCTE10_01 / VCTE7_01 The medical industry withold information about the content of highly toxic substances in many vaccines, and their possible adverse side-effects - such as autism. Common conspiracy theory following theories are true? VCTE10_02 / VCTE7_02 The American government were themselves involved in the terrorist attacks on the WTC and Pentagon etc. in 2001. Common conspiracy theory VCTE10_03 / TVC2_01 The CIA have conducted conciousness- and bahavioural experiments on unsuspecting people by administering i.a. LSD and hypnotizing them. True/actual documented conspiracy VCTE10_04 / VCTE7_03 Clamate issues such as "global warming" are largely exaggerated by instigators with political agendas. Common conspiracy theory VCTE10_05 / VCTE7_04 The Moon landing in 1969 was recorded in a studio and facbricated by NASA in order to win the "space race" with the Sovjet Union. Common conspiracy theory VCTE10_06 / VCTE7_05 There is a heavily guarded area, called "Area 51", where research on extraterrestrial lifeforms and technology has been going on for years. Common conspiracy theory VCTE10_07 / Fake Consp. Russian authorities have systematically surveilled private homes by cooperating with radio maufacturers who have installed wire tapping devices in home radios. Self-concocted fake/previously unknown conspiracy theory VCTE10_08 / VCTE7_06 Princess Dianawas not killed because of a car acccident due to her driver being intoxicated, but she was deliberately assassinated. Common conspiracy theory VCTE10_09 / TVC2_02 To increase profits, the largest manufacturerers of light bulbs in the 1920's made a collective decision to purposely restrict the glow time of all produced bulbs. True/actual documented conspiracy VCTE10_10 / VCTE7_07 There is a handful of people who constitute a power elite, and they control all the worlds political authorities and financial institutions. Common conspiracy theory

Source Criticism Ability Scale (CRAAP) Origin: Self composed / derived from CRAAP-test (Blakeslee, 2004) Scale: 1 (Often) - 5 (Very rarely); 3 = No opinon

You are faced with, or are being presented CRAAP_01 I assess how long ago the story was published or updated last. with, a news story. How often or rarely CRAAP_02 I assess what audience the article was written for. would you say you do the following? CRAAP_03 I assess the source and/or the authors qualifications. CRAAP_04 I check for evidence that what is claimed is true. CRAAP_05 I check to see if the information can be verified by other sources CRAAP_06 I assess if the language used is emotionally charged or neutral CRAAP_07 I look for misspellings and/or poor language. CRAAP_08 I assess whether the article is created to inform, sell, entertain or convince. CRAAP_09 I assess whether the article is based on facts, personal views or propaganda. CRAAP_10 I atttempt to uncover wheter views are objective, or if they are biased by e.g. ideological, cultural or religious beliefs.

Need For Cognition Scale (NFC2) Origin: Self composed/reduced - derived from Craft et al. (2013;2017) / Maksl et al. (2015) / Holanda & Coelho’s (2018) Scale: 1 (Strongly agree) - 5 (Strongly disagree); 3 = No opinon

Assess how much you agree with the NFC2_01 I prefer complex to simple problems. following statements about yourself. NFC2_02 I would rather do something that requires little thought than something that is sure to challenge my thinking abilities. Reverse Coded

Critical Thinking Disposition Scale (CTDS) Origin: Sosu (2013) - some alterations made Scale: 1 (Strongly agree) - 5 (Strongly disagree); 3 = No opinon

Assess how much you agree with the CTDS_01 / CO_01 I usually try to think about the bigger picture during a discussion. following statements about yourself. CTDS_02 / CO_02 I normally adhere to my habits and routine methods when I do things. Item altered from original scale; Reverse Coded CTDS_03 / CO_03 I use more than one source to find out information for myself. Composite score intervals: CTDS_04 / CO_04 I am often on the lookout for new ideas. CTDS_05 / CO_05 I seldom find arguments that are good enough to challenge my firmly held beliefs. Item altered from original scale; Reverse Coded Complete scale (Range 11-55): CTDS_06 / CO_06 It’s important to understand other people’s viewpoint on an issue. Low (11-34); Moderate (35-44); High (45-55) CTDS_07 / CO_07 I don’t generally feel a need to justify my choices to other people. Item altered from original scale; Reverse Coded Critical openness (CO) (Range 7-35): CTDS_08 / RS_01 I often re-evaluate my experiences so that I can learn from them. Low (7-21); Moderate (22-28); High (29-35) CTDS_09 / RS_02 I often make judgements about something I learn before I have investigated if the source is reliable. Item altered from original scale; Reverse Coded Reflective skepticism (RS) (Range 4-20): CTDS_10 / RS_03 I usually think about the wider implications of a decision before taking action. Low (4-12); Moderate (13-16); High (17-20) CTDS_12 / RS_04 As long as my methods work, I rarely see any good reason why I should change the way I do things. Item altered from original scale; Reverse Coded

xiii

APPENDIX C: CRAAP TEST

Evaluating Information – Applying the CRAAP Test

Meriam Library  California State University, Chico

When you search for information, you're going to find lots of it . . . but is it good information? You will have to determine that for yourself, and the CRAAP Test can help. The CRAAP Test is a list of questions to help you evaluate the information you find. Different criteria will be more or less important depending on your situation or need. Key: ◼ indicates criteria is for Web

Evaluation Criteria Currency: The timeliness of the information. • When was the information published or posted? • Has the information been revised or updated? • Does your topic require current information, or will older sources work as well? ◼Are the links functional?

Relevance: The importance of the information for your needs. • Does the information relate to your topic or answer your question? • Who is the intended audience? • Is the information at an appropriate level (i.e. not too elementary or advanced for your needs)? • Have you looked at a variety of sources before determining this is one you will use? • Would you be comfortable citing this source in your research paper?

Authority: The source of the information. • Who is the author/publisher/source/sponsor? • What are the author's credentials or organizational affiliations? • Is the author qualified to write on the topic? • Is there contact information, such as a publisher or email address? ◼Does the URL reveal anything about the author or source? examples: .com .edu .gov .org .net

Accuracy: The reliability, truthfulness and correctness of the content. • Where does the information come from? • Is the information supported by evidence? • Has the information been reviewed or refereed? • Can you verify any of the information in another source or from personal knowledge? • Does the language or tone seem unbiased and free of emotion? • Are there spelling, grammar or typographical errors?

Purpose: The reason the information exists. • What is the purpose of the information? Is it to inform, teach, sell, entertain or persuade? • Do the authors/sponsors make their intentions or purpose clear? • Is the information fact, opinion or propaganda? • Does the point of view appear objective and impartial? • Are there political, ideological, cultural, religious, institutional or personal biases?

9/17/10

xiv

APPENDIX D: RQ1 STATISTICS

NCRED CORRELATIONS

DESCRIPTIVE STATISTICS

Mean Std. Deviation N NCRED 3,1104 1,08671 125 CMQ 2,2832 ,92464 125 VCTE7 3,8057 ,91030 125 TVC2 2,5840 1,01349 125

CORRELATIONS

CMQ VCTE7 TVC2 Pearson Correlation -,496** -,512** -,207* Sig. (2-tailed) ,000 ,000 ,021 NCRED Spearman's rho -,493** -,498** -,203* Sig. (2-tailed) ,000 ,000 ,023 *. Correlation is significant at the 0.05 level (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).

SCATTERPLOTS

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 NCRED_tot_mean NCRED_tot_mean NCRED_tot_mean

xv

APPENDIX E: RQ2 STATISTICS

ALTERNATIVE MEDIA CONSUMPTION FREQUENCY ANOVA:

DESCRIPTIVE STATISTICS

95% Confidence Std. Std. Interval for Mean N Mean Minimum Maximum Deviation Error Lower Upper Bound Bound At least 30 1,9333 ,78667 ,14363 1,6396 2,2271 1,00 4,40 once a day A few days a 35 2,2571 ,97205 ,16431 1,9232 2,5911 1,00 5,00 CMQ week Monthly or 56 2,4929 ,92557 ,12368 2,2450 2,7407 1,00 4,60 rarer Total 121 2,2860 ,92802 ,08437 2,1189 2,4530 1,00 5,00 At least 30 3,3429 ,97296 ,17764 2,9795 3,7062 1,14 5,00 once a day A few days a 35 3,7306 ,95566 ,16154 3,4023 4,0589 1,29 5,00 VCTE7 week Monthly or 56 4,1173 ,74908 ,10010 3,9167 4,3180 2,14 5,00 rarer Total 121 3,8135 ,91934 ,08358 3,6480 3,9789 1,14 5,00 At least 30 2,6000 1,07799 ,19681 2,1975 3,0025 1,00 5,00 once a day A few days a 35 2,4286 1,00105 ,16921 2,0847 2,7724 1,00 5,00 TVC2 week Monthly or 56 2,6607 1,00502 ,13430 2,3916 2,9299 1,00 5,00 rarer Total 121 2,5785 1,01859 ,09260 2,3952 2,7619 1,00 5,00

ASSUMPTION TEST: HOMOGENEITY OF VARIANCES

Levene Statistic df1 df2 Sig. CMQ Based on Mean 1,455 2 118 ,238 VCTE7 Based on Mean 1,361 2 118 ,260 TVC2 Based on Mean ,353 2 118 ,704

ASSUMPTION TEST: NORMALITY

Kolmogorov-Smirnova Shapiro-Wilk NMH_freq_alt_rec3 Statistic df Sig. Statistic df Sig. At least once a day ,143 30 ,119 ,887 30 ,004 CMQ A few days a week ,134 35 ,117 ,923 35 ,017 Monthly or rarer ,129 56 ,021 ,952 56 ,025 At least once a day ,129 30 ,200* ,931 30 ,051 VCTE7 A few days a week ,124 35 ,191 ,936 35 ,042 Monthly or rarer ,174 56 ,000 ,907 56 ,000 At least once a day ,178 30 ,016 ,938 30 ,080 TVC2 A few days a week ,129 35 ,152 ,943 35 ,071 Monthly or rarer ,118 56 ,051 ,953 56 ,028 *. This is a lower bound of the true significance. a. Lilliefors Significance Correction

xvi

APPENDIX E RQ2 STATISTICS

BOXPLOTS

ONE-WAY ANOVA

Sum of Squares df Mean Square F Sig. Between Groups 6,157 2 3,078 3,737 ,027 CMQ Within Groups 97,190 118 ,824 Total 103,346 120 Between Groups 12,056 2 6,028 7,959 ,001 VCTE7 Within Groups 89,366 118 ,757 Total 101,422 120 Between Groups 1,179 2 ,590 ,564 ,570 TVC2 Within Groups 123,325 118 1,045 Total 124,504 120

TUKEY HSD: POST-HOC MULTIPLE COMPARISONS

Mean 95% Confidence Interval Dependent (I) NMH_freq_alt_rec3 (J) NMH_freq_alt_rec3 Difference (I- Std. Error Sig. Upper Variable Lower Bound J) Bound A few days a week -,32381 ,22580 ,327 -,8598 ,2122 At least once a day Monthly or rarer -,55952* ,20534 ,020 -1,0469 -,0721 At least once a day ,32381 ,22580 ,327 -,2122 ,8598 CMQ A few days a week Monthly or rarer -,23571 ,19555 ,452 -,6999 ,2285 At least once a day ,55952* ,20534 ,020 ,0721 1,0469 Monthly or rarer A few days a week ,23571 ,19555 ,452 -,2285 ,6999 A few days a week -,38776 ,21653 ,177 -,9017 ,1262 At least once a day Monthly or rarer -,77449* ,19690 ,000 -1,2419 -,3071 At least once a day ,38776 ,21653 ,177 -,1262 ,9017 VCTE7 A few days a week Monthly or rarer -,38673 ,18752 ,102 -,8318 ,0584 At least once a day ,77449* ,19690 ,000 ,3071 1,2419 Monthly or rarer A few days a week ,38673 ,18752 ,102 -,0584 ,8318 A few days a week ,17143 ,25436 ,779 -,4323 ,7752 At least once a day Monthly or rarer -,06071 ,23130 ,963 -,6097 ,4883 At least once a day -,17143 ,25436 ,779 -,7752 ,4323 TVC2 A few days a week Monthly or rarer -,23214 ,22028 ,545 -,7550 ,2907 At least once a day ,06071 ,23130 ,963 -,4883 ,6097 Monthly or rarer A few days a week ,23214 ,22028 ,545 -,2907 ,7550 *. The mean difference is significant at the 0.05 level.

xvii

APPENDIX F: RQ3 STATISTICS

ALTERNATIVE MEDIA ATTITUDES ANOVA:

DESCRIPTIVE STATISTICS

95% Confidence Interval Std. for Mean N Mean Std. Error Min Max Dev. Lower Upper Bound Bound Niche news 11 2,0727 1,06310 ,32054 1,3585 2,7869 1,00 4,20 (…) Ambitious and 15 1,9600 ,95379 ,24627 1,4318 2,4882 1,00 4,40 brave (…) Politically biased news 48 2,6708 ,91465 ,13202 2,4052 2,9364 1,20 5,00 CMQ (…) News sources who have lost 39 2,0821 ,80749 ,12930 1,8203 2,3438 1,00 4,60 faith (…) Total 113 2,3150 ,93953 ,08838 2,1399 2,4902 1,00 5,00 Niche news 11 3,6883 ,57434 ,17317 3,3025 4,0742 2,71 4,57 (…) Ambitious and 15 3,2667 1,27646 ,32958 2,5598 3,9735 1,14 5,00 brave (…) Politically biased news 48 4,2887 ,66648 ,09620 4,0952 4,4822 2,71 5,00 VCTE7 (…) News sources who have lost 39 3,5971 ,80270 ,12854 3,3369 3,8573 2,14 4,86 faith (…) Total 113 3,8559 ,88938 ,08367 3,6901 4,0217 1,14 5,00 Niche news 11 2,3636 ,89696 ,27044 1,7610 2,9662 1,00 3,50 (…) Ambitious and 15 2,5333 1,04312 ,26933 1,9557 3,1110 1,00 4,50 brave (…) Politically biased news 48 2,7292 1,16216 ,16774 2,3917 3,0666 1,00 5,00 TVC2 (…) News sources who have lost 39 2,4359 ,84433 ,13520 2,1622 2,7096 1,00 5,00 faith (…) Total 113 2,5664 1,01881 ,09584 2,3765 2,7563 1,00 5,00

ASSUMPTION TEST: HOMOGENEITY OF VARIANCES

Levene Statistic df1 df2 Sig. CMQ Based on Mean ,726 3 109 ,538 VCTE7 Based on Mean 5,146 3 109 ,002 TVC2 Based on Mean 2,969 3 109 ,035

xviii

APPENDIX F RQ3 STATISTICS

ASSUMPTION TEST: NORMALITY

Kolmogorov-Smirnova Shapiro-Wilk NMH_att_alt Statistic df Sig. Statistic df Sig. Niche news (…) ,180 11 ,200* ,892 11 ,147 Ambitious and brave news (…) ,217 15 ,057 ,834 15 ,010 CMQ Politically biased news (…) ,137 48 ,024 ,956 48 ,071 News sources who have lost faith (…) ,149 39 ,028 ,918 39 ,008 Niche news (…) ,161 11 ,200* ,939 11 ,509 Ambitious and brave news (…) ,151 15 ,200* ,916 15 ,165 VCTE7 Politically biased news (…) ,187 48 ,000 ,875 48 ,000 News sources who have lost faith (…) ,125 39 ,131 ,943 39 ,049 Niche news (…) ,197 11 ,200* ,914 11 ,269 Ambitious and brave news (…) ,229 15 ,034 ,933 15 ,305 TVC2 Politically biased news (…) ,134 48 ,031 ,944 48 ,023 News sources who have lost faith (…) ,175 39 ,004 ,906 39 ,003 *. This is a lower bound of the true significance. a. Lilliefors Significance Correction

BOXPLOTS

ONE-WAY ANOVA

Sum of Squares df Mean Square F Sig. Between Groups 10,730 3 3,577 4,423 ,006 CMQ Within Groups 88,134 109 ,809 Total 98,864 112 Between Groups 17,120 3 5,707 8,703 ,000 VCTE7 Within Groups 71,471 109 ,656 Total 88,592 112 Between Groups 2,405 3 ,802 ,767 ,515 TVC2 Within Groups 113,848 109 1,044 Total 116,252 112

xix

APPENDIX F RQ3 STATISTICS

TUKEY HSD: POST-HOC MULTIPLE COMPARISONS

95% Confidence Dependent Mean Std. Interval (I) NMH_att_alt NMH_att_alt Sig. Variable Difference (I-J) Error Lower Upper Bound Bound Ambitious and brave news ,11273 ,35695 ,989 -,8186 1,0440 (…) Politically biased news Niche news (…) -,59811 ,30059 ,198 -1,3824 ,1862 (…) News sources who have -,00932 ,30698 1,000 -,8103 ,7916 lost faith (…)

Niche news (…) -,11273 ,35695 ,989 -1,0440 ,8186

Ambitious and brave news Politically biased news -,71083* ,26599 ,043 -1,4048 -,0168 (…) (…) News sources who have -,12205 ,27320 ,970 -,8349 ,5908 lost faith (…) CMQ Niche news (…) ,59811 ,30059 ,198 -,1862 1,3824

Politically biased news Ambitious and brave news ,71083* ,26599 ,043 ,0168 1,4048 (…) (…) News sources who have ,58878* ,19385 ,016 ,0830 1,0946 lost faith (…)

Niche news (…) ,00932 ,30698 1,000 -,7916 ,8103

News sources who have Ambitious and brave news ,12205 ,27320 ,970 -,5908 ,8349 lost faith (…) (…) Politically biased news -,58878* ,19385 ,016 -1,0946 -,0830 (…) Ambitious and brave news ,42165 ,32144 ,557 -,4170 1,2603 (…) Politically biased news Niche news (…) -,60038 ,27068 ,125 -1,3066 ,1059 (…) News sources who have ,09124 ,27645 ,988 -,6300 ,8125 lost faith (…)

Niche news (…) -,42165 ,32144 ,557 -1,2603 ,4170

Ambitious and brave news Politically biased news -1,02202* ,23953 ,000 -1,6470 -,3971 (…) (…) News sources who have -,33040 ,24602 ,538 -,9723 ,3115 lost faith (…) VCTE7 Niche news (…) ,60038 ,27068 ,125 -,1059 1,3066

Politically biased news Ambitious and brave news 1,02202* ,23953 ,000 ,3971 1,6470 (…) (…) News sources who have ,69162* ,17457 ,001 ,2362 1,1471 lost faith (…)

Niche news (…) -,09124 ,27645 ,988 -,8125 ,6300

News sources who have Ambitious and brave news ,33040 ,24602 ,538 -,3115 ,9723 lost faith (…) (…) Politically biased news -,69162* ,17457 ,001 -1,1471 -,2362 (…) Ambitious and brave news TVC2 Niche news (…) -,16970 ,40569 ,975 -1,2282 ,8888 (…)

xx

APPENDIX F RQ3 STATISTICS

Politically biased news -,36553 ,34163 ,708 -1,2569 ,5258 (…) News sources who have -,07226 ,34890 ,997 -,9826 ,8381 lost faith (…) Niche news (…) ,16970 ,40569 ,975 -,8888 1,2282

Ambitious and brave news Politically biased news -,19583 ,30231 ,916 -,9846 ,5929 (…) (…) News sources who have ,09744 ,31050 ,989 -,7127 ,9076 lost faith (…) Niche news (…) ,36553 ,34163 ,708 -,5258 1,2569

Politically biased news Ambitious and brave news ,19583 ,30231 ,916 -,5929 ,9846 (…) (…) News sources who have ,29327 ,22032 ,545 -,2816 ,8681 lost faith (…) Niche news (…) ,07226 ,34890 ,997 -,8381 ,9826

News sources who have Ambitious and brave news -,09744 ,31050 ,989 -,9076 ,7127 lost faith (…) (…) Politically biased news -,29327 ,22032 ,545 -,8681 ,2816 (…) *. The mean difference is significant at the 0.05 level.

xxi

APPENDIX G: RQ4 STATISTICS

NML CORRELATIONS:

DESCRIPTIVE STATISTICS

Mean Std. Deviation N CMQ 2,2832 ,92464 125 VCTE7 3,8057 ,91030 125 TVC2 2,5840 1,01349 125 NML_ALL 2,0393 ,43166 125 NML_AA 1,8187 ,61420 125 NML_MM 2,1040 ,46982 125 NML_RR 2,1307 ,76070 125

CORRELATIONS

CMQ VCTE7 TVC2 Pearson Correlation -,043 -,203* ,205* Sig. (2-tailed) ,632 ,023 ,022 NML_ALL Spearman's rho -,016 -,222* ,196* Sig. (2-tailed) ,857 ,013 ,029 Pearson Correlation ,114 -,073 ,083 Sig. (2-tailed) ,206 ,418 ,358 NML_AA Spearman's rho ,069 -,133 ,072 Sig. (2-tailed) ,446 ,141 ,428 Pearson Correlation -,061 -,142 ,154 Sig. (2-tailed) ,500 ,113 ,087 NML_MM Spearman's rho -,001 -,142 ,145 Sig. (2-tailed) ,990 ,114 ,106 Pearson Correlation -,115 -,226* ,209* Sig. (2-tailed) ,202 ,011 ,019 NML_RR Spearman's rho -,090 -,221* ,203* Sig. (2-tailed) ,316 ,013 ,023

*. Correlation is significant at the 0.05 level (2-tailed). SCATTERPLOTS

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 1,00 2,00 3,00 4,00 0,00 1,00 2,00 3,00 4,00 0,00 1,00 2,00 3,00 4,00 NML_ALL_tot_mean_dropitemRR0 NML_ALL_tot_mean_dropitemRR0 NML_ALL_tot_mean_dropitemRR0 1rc 1rc 1rc

xxii

APPENDIX G RQ4 STATISTICS

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 1,00 2,00 3,00 4,00 0,00 1,00 2,00 3,00 4,00 0,00 1,00 2,00 3,00 4,00 NML_AA_tot_mean NML_AA_tot_mean NML_AA_tot_mean

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 1,00 2,00 3,00 4,00 0,00 1,00 2,00 3,00 4,00 0,00 1,00 2,00 3,00 4,00 NML_MM_tot_mean NML_MM_tot_mean NML_MM_tot_mean

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 NML_RR_tot_mean_dropitem01rc NML_RR_tot_mean_dropitem01rc NML_RR_tot_mean_dropitem01rc

NFC2 CORRELATIONS: DESCRIPTIVE STATISTICS

Mean Std. Deviation N CMQ 2,2832 ,92464 125 VCTE7 3,8057 ,91030 125 TVC2 2,5840 1,01349 125 NFC2 2,2520 ,94105 125

xxiii

APPENDIX G RQ4 STATISTICS

CORRELATIONS

CMQ VCTE7 TVC2 Pearson Correlation ,087 -,015 ,109 Sig. (2-tailed) ,335 ,868 ,228 NFC2 Spearman's rho ,101 -,036 ,130 Sig. (2-tailed) ,264 ,694 ,147

SCATTERPLOTS

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 NFC2_tot_mean NFC2_tot_mean NFC2_tot_mean

MLOC3 CORRELATIONS:

DESCRIPTIVE STATISTICS

Mean Std. Deviation N CMQ 2,2832 ,92464 125 VCTE7 3,8057 ,91030 125 TVC2 2,5840 1,01349 125 MLOC3 2,0053 ,72461 125

CORRELATIONS

CMQ VCTE7 TVC2 Pearson Correlation -,138 -,132 ,049 Sig. (2-tailed) ,125 ,141 ,589 MLOC3 Spearman's rho -,139 -,204* ,026 Sig. (2-tailed) ,122 ,022 ,777

*. Correlation is significant at the 0.05 level (2-tailed). SCATTERPLOTS

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 MLOC3_tot_mean MLOC3_tot_mean MLOC3_tot_mean

xxiv

APPENDIX H: RQ5 STATISTICS

CTDS_ALL ANOVA:

DESCRIPTIVE STATISTICS

95% Confidence Std. Std. Interval for Mean N Mean Minimum Maximum Deviation Error Lower Upper Bound Bound Low 14 2,5857 ,89258 ,23855 2,0704 3,1011 1,20 4,20 Moderate 91 2,1934 ,84772 ,08886 2,0169 2,3700 1,00 5,00 CMQ High 20 2,4800 1,21811 ,27238 1,9099 3,0501 1,00 4,60 Total 125 2,2832 ,92464 ,08270 2,1195 2,4469 1,00 5,00 Low 14 3,9694 1,00576 ,26880 3,3887 4,5501 1,86 5,00 Moderate 91 3,7143 ,92017 ,09646 3,5227 3,9059 1,14 5,00 VCTE7 High 20 4,1071 ,74140 ,16578 3,7602 4,4541 2,71 5,00 Total 125 3,8057 ,91030 ,08142 3,6446 3,9669 1,14 5,00 Low 14 2,7500 ,93541 ,25000 2,2099 3,2901 1,00 4,00 Moderate 91 2,5714 ,99043 ,10383 2,3652 2,7777 1,00 5,00 TVC2 High 20 2,5250 1,19731 ,26773 1,9646 3,0854 1,00 4,50 Total 125 2,5840 1,01349 ,09065 2,4046 2,7634 1,00 5,00

ASSUMPTION TEST: HOMOGENEITY OF VARIANCES

Levene Statistic df1 df2 Sig. CMQ Based on Mean 4,331 2 122 ,015 VCTE7 Based on Mean ,458 2 122 ,634 TVC2 Based on Mean 2,277 2 122 ,107

ASSUMPTION TEST: NORMALITY

Kolmogorov-Smirnova Shapiro-Wilk CTDS_ALL_displevel Statistic df Sig. Statistic df Sig. Low ,167 14 ,200* ,955 14 ,633 CMQ Moderate ,107 91 ,012 ,945 91 ,001 High ,253 20 ,002 ,871 20 ,012 Low ,283 14 ,003 ,818 14 ,008 VCTE7 Moderate ,093 91 ,048 ,940 91 ,000 High ,184 20 ,073 ,911 20 ,067 Low ,217 14 ,072 ,899 14 ,109 TVC2 Moderate ,157 91 ,000 ,939 91 ,000 High ,204 20 ,028 ,881 20 ,019 *. This is a lower bound of the true significance. a. Lilliefors Significance Correction

xxv

APPENDIX H RQ5 STATISTICS

BOXPLOTS

ONE-WAY ANOVA

Sum of Squares df Mean Square F Sig. Between Groups 2,790 2 1,395 1,648 ,197 CMQ Within Groups 103,225 122 ,846 Total 106,015 124 Between Groups 2,953 2 1,476 1,805 ,169 VCTE7 Within Groups 99,798 122 ,818 Total 102,751 124 Between Groups ,470 2 ,235 ,226 ,798 TVC2 Within Groups 126,898 122 1,040 Total 127,368 124

CTDS_CO ANOVA:

DESCRIPTIVE STATISTICS

95% Confidence Std. Std. Interval for Mean N Mean Minimum Maximum Deviation Error Lower Upper Bound Bound Low 10 2,2600 ,76041 ,24046 1,7160 2,8040 1,00 3,40 Moderate 93 2,2645 ,92038 ,09544 2,0750 2,4541 1,00 5,00 CMQ High 22 2,3727 1,03886 ,22148 1,9121 2,8333 1,00 4,40 Total 125 2,2832 ,92464 ,08270 2,1195 2,4469 1,00 5,00 Low 10 3,7857 1,02851 ,32524 3,0500 4,5215 1,86 4,71 Moderate 93 3,7204 ,91398 ,09478 3,5322 3,9087 1,14 5,00 VCTE7 High 22 4,1753 ,77739 ,16574 3,8306 4,5200 2,57 5,00 Total 125 3,8057 ,91030 ,08142 3,6446 3,9669 1,14 5,00 Low 10 2,8000 ,82327 ,26034 2,2111 3,3889 1,00 3,50 Moderate 93 2,5538 1,01205 ,10495 2,3453 2,7622 1,00 5,00 TVC2 High 22 2,6136 1,12262 ,23934 2,1159 3,1114 1,00 4,50 Total 125 2,5840 1,01349 ,09065 2,4046 2,7634 1,00 5,00

xxvi

APPENDIX H RQ5 STATISTICS

ASSUMPTION TEST: HOMOGENEITY OF VARIANCES

Levene Statistic df1 df2 Sig. CMQ Based on Mean 1,259 2 122 ,288 VCTE7 Based on Mean ,371 2 122 ,691 TVC2 Based on Mean ,786 2 122 ,458

ASSUMPTION TEST: NORMALITY

Kolmogorov-Smirnova Shapiro-Wilk CTDS_CO_displevel Statistic df Sig. Statistic df Sig. Low ,231 10 ,138 ,937 10 ,518 CMQ Moderate ,140 93 ,000 ,930 93 ,000 High ,186 22 ,047 ,922 22 ,083 Low ,236 10 ,122 ,846 10 ,052 VCTE7 Moderate ,097 93 ,031 ,940 93 ,000 High ,210 22 ,012 ,876 22 ,010 Low ,202 10 ,200* ,841 10 ,045 TVC2 Moderate ,136 93 ,000 ,941 93 ,000 High ,180 22 ,062 ,900 22 ,030 *. This is a lower bound of the true significance. a. Lilliefors Significance Correction

BOXPLOTS

ONE-WAY ANOVA

Sum of Squares df Mean Square F Sig. Between Groups ,214 2 ,107 ,123 ,884 CMQ Within Groups 105,801 122 ,867 Total 106,015 124 Between Groups 3,686 2 1,843 2,270 ,108 VCTE7 Within Groups 99,065 122 ,812 Total 102,751 124 Between Groups ,571 2 ,285 ,275 ,760 TVC2 Within Groups 126,797 122 1,039 Total 127,368 124

xxvii

APPENDIX H RQ5 STATISTICS

CTDS_RS ANOVA:

DESCRIPTIVE STATISTICS

95% Confidence Std. Std. Interval for Mean N Mean Minimum Maximum Deviation Error Lower Upper Bound Bound Low 28 2,4143 ,85015 ,16066 2,0846 2,7439 1,00 4,20 Moderate 69 2,1536 ,85828 ,10332 1,9474 2,3598 1,00 5,00 CMQ High 28 2,4714 1,11848 ,21137 2,0377 2,9051 1,00 4,60 Total 125 2,2832 ,92464 ,08270 2,1195 2,4469 1,00 5,00 Low 28 3,8367 ,93614 ,17691 3,4737 4,1997 1,86 5,00 Moderate 69 3,7516 ,86988 ,10472 3,5426 3,9605 1,14 5,00 VCTE7 High 28 3,9082 1,00205 ,18937 3,5196 4,2967 1,29 5,00 Total 125 3,8057 ,91030 ,08142 3,6446 3,9669 1,14 5,00 Low 28 2,5179 ,96688 ,18272 2,1429 2,8928 1,00 4,50 Moderate 69 2,7464 ,99122 ,11933 2,5083 2,9845 1,00 5,00 TVC2 High 28 2,2500 1,05848 ,20003 1,8396 2,6604 1,00 4,50 Total 125 2,5840 1,01349 ,09065 2,4046 2,7634 1,00 5,00

ASSUMPTION TEST: HOMOGENEITY OF VARIANCES

Levene Statistic df1 df2 Sig. CMQ Based on Mean 3,275 2 122 ,041 VCTE7 Based on Mean ,307 2 122 ,736 TVC2 Based on Mean ,810 2 122 ,447

ASSUMPTION TEST: NORMALITY

Kolmogorov-Smirnova Shapiro-Wilk CTDS_RS_displevel Statistic df Sig. Statistic df Sig. Low ,135 28 ,200* ,964 28 ,426 CMQ Moderate ,126 69 ,008 ,921 69 ,000 High ,199 28 ,006 ,914 28 ,025 Low ,200 28 ,006 ,882 28 ,005 VCTE7 Moderate ,101 69 ,080 ,951 69 ,008 High ,138 28 ,185 ,873 28 ,003 Low ,135 28 ,200* ,948 28 ,177 TVC2 Moderate ,167 69 ,000 ,943 69 ,004 High ,189 28 ,012 ,902 28 ,013 *. This is a lower bound of the true significance. a. Lilliefors Significance Correction

xxviii

APPENDIX H RQ5 STATISTICS

BOXPLOTS

ONE-WAY ANOVA

Sum of Squares df Mean Square F Sig. CMQ Between Groups 2,632 2 1,316 1,553 ,216 Within Groups 103,383 122 ,847 Total 106,015 124 VCTE7 Between Groups ,523 2 ,262 ,312 ,732 Within Groups 102,228 122 ,838 Total 102,751 124 TVC2 Between Groups 5,065 2 2,533 2,526 ,084 Within Groups 122,303 122 1,002 Total 127,368 124

xxix

APPENDIX I: RQ6 STATISTICS

ILFARG CORRELATIONS:

DESCRIPTIVE STATISTICS

Mean Std. Deviation N CMQ 2,2832 ,92464 125 VCTE7 3,8057 ,91030 125 TVC2 2,5840 1,01349 125 ILFARG 3,6538 ,60509 125

CORRELATIONS

CMQ VCTE7 TVC2 Pearson Correlation ,278** ,571** ,117 Sig. (2-tailed) ,002 ,000 ,193 ILFARG Spearman's rho ,296** ,539** ,105 Sig. (2-tailed) ,001 ,000 ,245

*. Correlation is significant at the 0.05 level (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).

SCATTERPLOTS

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00

0,00 0,00 0,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 ILFARG ILFARG ILFARG_tot_mean

xxx

APPENDIX J: RQ7 STATISTICS

BSREC4 CORRELATIONS:

DESCRIPTIVE STATISTICS

Mean Std. Deviation N CMQ 2,2832 ,92464 125 VCTE7 3,8057 ,91030 125 TVC2 2,5840 1,01349 125 BSREC4 3,5300 ,89780 125

CORRELATIONS

CMQ VCTE7 TVC2 Pearson Correlation ,362** ,403** ,138 Sig. (2-tailed) ,000 ,000 ,125 BSREC4 Spearman's rho ,338** ,370** ,149 Sig. (2-tailed) ,000 ,000 ,096

*. Correlation is significant at the 0.05 level (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).

SCATTERPLOTS

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 BSREC4_tot_mean BSREC4_tot_mean BSREC4_tot_mean

xxxi

APPENDIX J RQ7 STATISTICS

PREC4 CORRELATIONS:

DESCRIPTIVE STATISTICS

Mean Std. Deviation N CMQ 2,2832 ,92464 125 VCTE7 3,8057 ,91030 125 TVC2 2,5840 1,01349 125 PREC4 2,1580 ,71669 125

CORRELATIONS

CMQ VCTE7 TVC2 Pearson Correlation ,136 ,059 ,324** Sig. (2-tailed) ,131 ,514 ,000 PREC4 Spearman's rho ,110 ,099 ,280** Sig. (2-tailed) ,220 ,270 ,002

**. Correlation is significant at the 0.01 level (2-tailed).

SCATTERPLOTS

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 PREC4_tot_mean PREC4_tot_mean PREC4_tot_mean

xxxii

APPENDIX K: RQ8 STATISTICS

CRAAP CORRELATIONS:

DESCRIPTIVE STATISTICS

Mean Std. Deviation N CMQ 2,2832 ,92464 125 VCTE7 3,8057 ,91030 125 TVC2 2,5840 1,01349 125 CRAAP 1,9464 ,68821 125

CORRELATIONS

CMQ VCTE7 TVC2 Pearson Correlation ,104 ,079 ,125 Sig. (2-tailed) ,247 ,382 ,165 CRAAP Spearman's rho ,138 ,062 ,130 Sig. (2-tailed) ,126 ,489 ,149

SCATTERPLOTS

6,00 6,00 6,00 5,00 5,00 5,00 4,00 4,00 4,00 3,00 3,00 3,00

2,00 2,00 2,00

TVC2_mean VCTE7_mean

CMQ_tot_mean 1,00 1,00 1,00 0,00 0,00 0,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 0,00 2,00 4,00 6,00 CRAAP_tot_mean CRAAP_tot_mean CRAAP_tot_mean

xxxiii