THE ROLE OF TECHNOLOGY IN ONLINE MISINFORMATION SARAH KREPS JUNE 2020 EXECUTIVE SUMMARY States have long interfered in the domestic politics of other states. Foreign election interference is nothing new, nor are misinformation campaigns. The new feature of the 2016 election was the role of technology in personalizing and then amplifying the information to maximize the impact. As a 2019 Senate Select Committee on Intelligence report concluded, malicious actors will continue to weaponize information and develop increasingly sophisticated tools for personalizing, targeting, and scaling up the content. This report focuses on those tools. It outlines the logic of digital personalization, which uses big data to analyze individual interests to determine the types of messages most likely to resonate with particular demographics. The report speaks to the role of artificial intelligence, machine learning, and neural networks in creating tools that distinguish quickly between objects, for example a stop sign versus a kite, or in a battlefield context, a combatant versus a civilian. Those same technologies can also operate in the service of misinformation through text prediction tools that receive user inputs and produce new text that is as credible as the original text itself. The report addresses potential policy solutions that can counter digital personalization, closing with a discussion of regulatory or normative tools that are less likely to be effective in countering the adverse effects of digital technology. INTRODUCTION and machine learning about user behavior to manipulate public opinion, allowed social media Meddling in domestic elections is nothing new as bots to target individuals or demographics known a tool of foreign influence. In the first two-party to be susceptible to politically sensitive messaging. election in 1796, France engaged in aggressive propaganda1 to tilt the public opinion scales in favor As the Senate Select Committee on Intelligence of the pro-French candidate, Thomas Jefferson, concluded,3 the Russian Internet Research Agency through a campaign of misinformation and fear. (IRA) that used social media to divide and exercise American voters clearly understood American The innovation of the 2016 presidential psychology and “what buttons to press.” The election, therefore, was not foreign interests or IRA, however, did not fully exploit some of the misinformation, but the technology used to promote technological tools that would have allowed it to those foreign interests and misinformation. achieve greater effect. In particular, the Senate 2 Computational propaganda, the use of big data report notes that the IRA did not use Facebook’s 1 “Custom Audiences” feature that would have PERSONALIZATION AT SCALE enabled more micro-targeting of advertisements on divisive issues. Nonetheless, Senator Mark Warner AND THE ROLE OF AI (D-VA) of the Intelligence Committee foreshadowed In a November 2016 article, McKinsey Digital that the IRA and other malicious actors would published an article5 titled: “Marketing’s Holy Grail: remedy any previous shortcomings: Digital personalization at scale.” The authors note that an era of instant gratification means that There’s no doubt that bad actors will continue customers decide quickly what they like, which to try to weaponize the scale and reach of social means that companies must curate and deliver media platforms to erode public confidence and personalized content. “The tailoring of messages foster chaos. The Russian playbook is out in the or offers based on their actual behavior” is key open for other foreign and domestic adversaries to luring and keeping consumers, the McKinsey to expand upon — and their techniques will only article wrote. Step one in that journey is to get more sophisticated.4 understand consumer behavior with as much data This report outlines the way that advances in digital as possible, which is where technology comes in. technology will increasingly allow adversaries Big data combined with machine learning ensures to expand their techniques in ways that drive that individuals receive “the appropriate trigger misinformation. In particular, it speaks to the message,” in the article’s words. availability of user data and powerful artificial Although pitched to companies, the personalization intelligence, a combination that enables platforms of content is not restricted to marketing. In 2016, to personalize content. While conventional the Russian IRA deployed similar principles in propaganda efforts were pitched to mass audiences the 2016 election. According to the U.S. Senate and limited to manipulation of the median voter, Select Committee on Intelligence report “Russian tailored and personalized messages allow malicious Active Measures Campaigns and Interference actors to psychologically manipulate all corners of in the 2016 Election,” the IRA used targeted the ideological spectrum, thereby achieving a larger advertisements, falsified news articles, and social potential effect. media amplification tools to polarize Americans.6 To make these points, the report first outlines the Far from a strategy oriented toward a mass public, concept of digital personalization, in which users the IRA information operation relied on digital are targeted with content tailored to their interests personalization: determining what types of sites and sensitivities. It then offers a discussion of individuals frequented, correlating between those how artificial intelligence fits into that digital behaviors and demographic information, and personalization picture, integrating insights about finding ways to reach the groups that would be machine learning and neural networks to show how most triggered by racially, ethnically, or religiously- algorithms can learn distinguish between objects charged content. or create synthetic text. The report next shows From there, the IRA could create thousands of how AI can be used maliciously in the service of microsegments, not all of which were created misinformation, focusing on text prediction tools equal. In the election context, as the Senate that receive user inputs and produce styles and Intelligence Committee report notes,7 “no single substance that are as credible as the original text group of Americans was targeted by IRA information itself. It then addresses potential policy solutions operatives more than African-Americans.” Social that can counter personalization via AI, and closes media content with racial undertones — whether with a discussion of regulatory or normative tools advertisements, memes, or tweets — targeted that are less likely to be effective in countering the African Americans, for example, with an eye toward adverse effects of digital technology. 2 generating resentment toward out-groups, co- Even if the differences between human and opting participation in protest behavior, or even machine reasoning remain large, machines can convincing individuals to sit out from elections. One think and indeed are increasingly outperforming advertisement8 sought to stir up nativist sentiment humans on at least certain tasks. In 1997, IBM’s through an image about Islam taking over the world, chess-playing computer called Deep Blue beat the posted by an account called “Heart of Texas.” world chess champion Garry Kasparov. In 2015, AlphaGo, developed by DeepMind Technologies Micro-targeted messaging is not onerous, provided (later acquired by Google), defeated a human that large amounts of user data are available professional player of the game Go, considered to generate profiles of personal likes, dislikes, more difficult for computers to win than chess ideology, and psychology. Generating new content because of the game structure. that targets those micro-segments is, however, more resource-intensive. Computers have gained these skills from advancements in artificial intelligence. Learning Individuals who work at Russia’s IRA work 12-hour algorithms are generated through a process in 9 shifts and are required to meet quotas in terms of which neural networks (a combination of neurons comments, blog posts, or page views. The work is analogous to those in a human brain) make tedious, and writing new content about the same connections between cause and effect, or steps that topics or themes — for example, elevating the are correct and incorrect. In the context of AlphaGo, image of Russia or increasing division or confusion the neural network analyzed millions of moves that in the American political landscape — has its human Go experts had made, then played against challenges. Enter artificial intelligence, which can itself to reinforce what it had learned, fine-tuning help overcome these creativity obstacles. and updating to predict and preempt moves. Beyond the context of gaming, neural networks can ADVANCES IN ARTIFICIAL classify images, video, or text by identifying patterns INTELLIGENCE and shapes, engaging in logical reasoning about the identity of those images, and engaging in feedback Diving deeper into the ways that AI can facilitate corrections that improve the performance of the misinformation requires taking a step back and network. Training autonomous vehicle algorithms examining the technology itself. The term “artificial involves feeding the machine thousands of images intelligence” is one that is used frequently, but and angles of stop signs, for example, so that the rarely uniformly. It refers generally to something car can accurately recognize and heed a stop the mathematician Alan Turing called a “thinking