A Digital Media Literacy Intervention Increases Discernment Between
Total Page：16
File Type：pdf, Size：1020Kb
A digital media literacy intervention increases discernment between mainstream and false news in the United States and India Andrew M. Guessa,1,2 , Michael Lernerb,c,1 , Benjamin Lyonsd,1 , Jacob M. Montgomerye,1, Brendan Nyhanf,1, Jason Reiflerg,1, and Neelanjan Sircarh,1 aDepartment of Politics, Princeton University, Princeton, NJ 08544; bDepartment of Political Science, University of Michigan, Ann Arbor, MI 48109-1045; cGerald R. Ford School of Public Policy, University of Michigan, Ann Arbor, MI 48109-1045; dDepartment of Communication, University of Utah, Salt Lake City, UT 84112; eDepartment of Political Science, Washington University in St. Louis, St. Louis, MO 63130-4899; fDepartment of Government, Dartmouth College, Hanover, NH 03755; gDepartment of Politics, University of Exeter, Exeter EX4 4RJ, United Kingdom; and hDepartment of Political Science, Ashoka University, Sonipat, Haryana 131029, India Edited by David G. Rand, Massachusetts Institute of Technology, Cambridge, MA, and accepted by Editorial Board Member Margaret Levi April 28, 2020 (received for review November 20, 2019) Widespread belief in misinformation circulating online is a crit- navigate the digital world. For example, the Pew Research Cen- ical challenge for modern societies. While research to date has ter found as recently as 2017 that only 17% of US adults have focused on psychological and political antecedents to this phe- the skills and confidence to learn new information effectively nomenon, few studies have explored the role of digital media online (8). Nonetheless, people worldwide increasingly obtain literacy shortfalls. Using data from preregistered survey exper- news and information from social media platforms that lack tra- iments conducted around recent elections in the United States ditional editorial controls (9, 10), allowing politicians and other and India, we assess the effectiveness of an intervention modeled actors to widely disseminate misinformation via algorithmic news closely on the world’s largest media literacy campaign, which pro- feeds. Without the necessary digital media literacy skills, people vided “tips” on how to spot false news to people in 14 countries. frequently fall victim to dubious claims they encounter in this Our results indicate that exposure to this intervention reduced context. the perceived accuracy of both mainstream and false news head- These concerns have become especially salient in the United POLITICAL SCIENCES lines, but effects on the latter were significantly larger. As a result, States and India in recent years. In the United States, low- the intervention improved discernment between mainstream and quality online articles were distributed widely on social media in false news headlines among both a nationally representative sam- the months before the 2016 US presidential election (11). This ple in the United States (by 26.5%) and a highly educated online phenomenon created widespread fears that fake news was mis- sample in India (by 17.5%). This increase in discernment remained leading people at a massive scale (12). Smartphone use has also measurable several weeks later in the United States (but not in made India, the world’s largest democracy, a fertile environment India). However, we find no effects among a representative sam- for online rumors and misinformation. Viral misinformation ple of respondents in a largely rural area of northern India, where rates of social media use are far lower. Significance digital literacy j social media j misinformation Few people are prepared to effectively navigate the online information environment. This global deficit in digital media ocial media platforms have proved to be fertile ground for literacy has been identified as a critical factor explain- Sinflammatory political misinformation. People around the ing widespread belief in online misinformation, leading to world increasingly worry that so-called “fake news” and other changes in education policy and the design of technology plat- forms of dubious or false information are misleading voters—a forms. However, little rigorous evidence exists documenting fear that has inspired government actions to address the problem the relationship between digital media literacy and people’s in a number of countries (1, 2). ability to distinguish between low- and high-quality news Research into online misinformation has thus far focused on online. This large-scale study evaluates the effectiveness of political, economic, and psychological factors (3–5). In this arti- a real-world digital media literacy intervention in both the cle, we focus on another human vulnerability to online political United States and India. Our largely encouraging results indi- misinformation: shortfalls in digital media literacy. cate that relatively short, scalable interventions could be While largely overlooked in the emerging empirical litera- effective in fighting misinformation around the world. ture on digital disinformation and fake news, the concept of digital media literacy usefully captures the skills and competen- Author contributions: A.M.G., M.L., B.L., J.M.M., B.N., J.R., and N.S. designed research, performed research, analyzed data, and wrote the paper.y cies needed to successfully navigate a fragmented and complex information ecosystem (6). Even under ideal conditions, most The authors declare no competing interest.y people struggle to reliably evaluate the quality of information This open access article is distributed under Creative Commons Attribution-NonCommercial- NoDerivatives License 4.0 (CC BY-NC-ND).y they encounter online because they lack the skills and contex- This article is a PNAS Direct Submission. D.G.R. is a guest editor invited by the Editorial tual knowledge required to effectively distinguish between high- Board.y and low-quality news content. Data deposition: Data files and scripts necessary to replicate the results in this article The connection between digital media literacy and misinfor- are available at the Dataverse repository at https://doi.org/10.7910/DVN/Q5QINN. The US mation was identified early by theorists. “Misinformation—and study preanalysis plan is available at https://osf.io/u3sgc. The India study preanalysis plan disinformation—breeds as easily as creativity in the fever-swamp is available at https://osf.io/97rnz.y of personal publishing,” according to an influential 1997 intro- 1 A.M.G., M.L., B.L., J.M.M., B.N., J.R., and N.S. contributed equally to this work.y duction to the subject. “It will take all of the critical skills users 2 To whom correspondence may be addressed. Email: [email protected] can muster to separate truth from fiction” (ref. 7, p. xii). This article contains supporting information online at https://www.pnas.org/lookup/suppl/ More than 20 y later, these warnings seem prescient. Sur- doi:10.1073/pnas.1920498117/-/DCSupplemental.y vey research shows that few people are prepared to effectively www.pnas.org/cgi/doi/10.1073/pnas.1920498117 PNAS Latest Articles j 1 of 10 Downloaded by guest on September 27, 2021 spread via WhatsApp in India has reportedly provoked hatred ticism toward catchy headlines). Importantly, the success of this and ethnic violence (13). Moreover, online political misinforma- approach does not require readers to take burdensome steps like tion became a significant concern during the 2019 Indian general conducting research or thinking deeply about each piece of news election as political parties engaged in aggressive digital cam- they encounter (which is typically impossible in practice given paign efforts via short message service (SMS) and messaging the volume of stories that social media users encounter). Instead, applications like WhatsApp (14, 15). For instance, one analy- this intervention aims to provide simple decision rules that help sis found that over 25% of the news shared on Facebook during people distinguish between mainstream and false news, which we the election by the governing Bharatiya Janata Party (BJP) came call “discernment” following ref. 4. from dubious outlets (16). There are important reasons to be skeptical about the effec- Many nonprofits and governments are seeking to counter tiveness of this approach. Prior research has found that media these trends (and the related threat of foreign manipulation literacy interventions like this can help people think critically campaigns) by improving the digital media literacy of news con- about the media content they receive (43). However, prior stud- sumers (17–20). For instance, American universities increasingly ies focus mostly on offline health behavior; the extent to which teach media literacy to undergraduate students (21) and similar these interventions are effective for controversial political claims efforts are also being proposed at the kindergarten to grade 12 or online (mis)information is largely unknown. Moreover, such (22). Similarly, WhatsApp and the National Association of Soft- interventions may struggle to overcome people’s reliance on ware and Service Companies announced plans to train nearly heuristics such as familiarity and congeniality that news con- 100,000 people in India through in-person events and posts on sumers use to evaluate the credibility of online stories (44, 45). social media to spot misinformation (23). Finally, attempting to identify false news through close scrutiny Despite the attention and resources these initiatives have of a headline differs from the typical approach of professional received, however, little large-scale evidence exists on the effec-