Downloaded by guest on September 23, 2021 swl sisipiain o uuersac nsca media misinformation social computational on and research future polarization, science. for social political implications campaigns, its influence determine as election— to well presidential 2016 inability as the important influenced our accounts several already study—especially IRA discord whether discussing were our sow by who of to conclude those limitations We failed with polarized. have interacted highly findings mostly might these they trolls Together, because Russian usage. Twitter that of suggest politics, frequency in interest high accounts high ideological IRA and network, Twitter strong with their with interaction within respondents that homophily among over find common behaviors also most and We were attitudes period. political 1-mo impacted of a substantially evi- measures accounts no distinctive IRA find 6 with we interaction models, that tree dence non- regression from with Bayesian (IRA) merged Agency Using Research Twitter. 2017 Internet Russian late the Republican about from data 1,239 public users of Twitter that behaviors Democratic data online and longitudinal and using attitudes known the question of yet describe this behaviors not study and is attitudes We political it Americans. the campaigns, shaped such efforts these of stud- how strategy of number the growing analyze a political Though ies increase States. United to the in designed 2019) divisions have 15, campaigns countries April review other social-media for and (received Russia launched 2019 that 22, concern October widespread approved is and There Norway, Oslo, Oslo, of University Underdal, Arild by Edited Denmark; g Copenhagen, 1353 Copenhagen, 27708; NC Durham, University, Duke a Merhout Friedolin . . fractures, erode . fiction, societal . and . “exploit entities. reality between to lines the designed blur specifically, argued more multifaceted known have were States and, Scholars organization United the (IRA). an against campaigns Agency of these that Research auspices Internet the the under as 2013 ene- political since its Committee, attack mies Intelligence to in platforms Senate social-media users US used has Twitter the Russia to American According of 2017. behaviors late the and shaped attitudes campaign political influence social-media and government’s immigration sian as such issues amplify (6–8). social control and gun divisive vast Americans involve about impersonate reportedly debates that campaigns armies These States. social-media polarization political United cam- fan the coordinated across to countries about in other commonplace alarm and Russia is increasing by is paigns subterfuge there such internet, the Though social create (5). enables to order also discord in incivility—it others impersonate interactions of to consequences users social-media social-media the of reduces only nature not (1–4). anonymous behavior uncivil relatively facilitate The platforms such that concern ing T science social 2017 Bail late A. Christopher in of users behaviors Twitter and American attitudes political the on Agency’s impact Research Internet Russian the Assessing www.pnas.org/cgi/doi/10.1073/pnas.1906420116 oaiainLb ueUiest,Dra,N 27708; NC Durham, University, Duke Lab, Polarization eateto ttsia cec,Dk nvriy uhm C27708 NC Durham, University, Duke Science, Statistical of Department hssuypoie rlmnr seseto o h Rus- the how of assessment preliminary a provides study This odmcaiepbi icus bu oiis hr sgrow- is there politics, about media discourse social public of democratize potential to the celebrated once scholars hough n ndmcayisl”() ail xadn liter- expanding rapidly A (9). itself” democracy in and | oilmedia social a,e a,b,c,1 enFreelon Deen , ra Guay Brian , | oiia polarization political d eateto oiia cec,Dk nvriy uhm C27708; NC Durham, University, Duke Science, Political of Department f colo ei n oraim nvriyo ot aoiaa hplHl,Cae il C27599; NC Hill, Chapel Hill, Chapel at Carolina North of University Journalism, and Media of School f a,d,2 n lxne Volfovsky Alexander and , ml Maloney Emily , | b computational eateto oilg,Dk nvriy uhm C27708; NC Durham, University, Duke Sociology, of Department rs nmedia in trust a,b,2 ia Combs Aidan , doi:10.1073/pnas.1906420116/-/DCSupplemental hsatcecnan uprigifrainoln at online information supporting contains article This Dataverse, Harvard at 2 accessed be can 1 study this for dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/UATZBA.y Data deposition: Data (13). conflict” extended aggression of of era act an singular in a ushered age: that media social the prompted of Harbor effort Pearl Times this American of York scope inspiring New The The in (12). rallies anecdotal succeeded attend also to accounts is activists There IRA (6). that 2017 evidence in num- posts, significantly In the the Twitter increased posts—and campaign. Instagram 57,000 bers of 2,600 than this and because more posts, of Facebook produced perhaps 2,400 IRA sophistication is the apparent alone, This 2016 and public. scale American sheer behav- the and attitudes of political iors the on influence fill profound to exerted attempt initial an is study gap. Our research this 11). the of (10, whether behaviors public examined and attitudes have American the studies impacted no actually knowledge, efforts (6, our these strategies social to social-influence Yet, on Russian 9). activity into 7, IRA insight of gain depth to and media breadth the examines ature oeiaie ies . C BY-NC-ND) (CC 4.0 NoDerivatives License distributed under is article access open This Submission.y Direct PNAS a is article This interest.y wrote competing A.V. no and declare F.M., authors D.S.H., The B.G., C.A.B., E.M., and B.G., data; C.A.B., paper.y analyzed study; the research; A.V. this designed for and F.M., data A.V. collected D.S.H., and A.V. F.M., A.C., and D.F., F.M., D.S.H., A.C., A.C., E.M., B.G., E.M., C.A.B., B.G., C.A.B., contributions: Author a,g ..adEM otiue qal oti work.y this to equally contributed E.M. and [email protected]. y Email: addressed. be may correspondence whom To apino iifrain oiia icus,ad2016 and discourse, dynamics. campaign political election presidential misinformation, on Russia’s of impact campaign leave the about but questions campaigns, not important influence may unanswered online Americans to susceptible suggest easily results be These high politics. and fre- in Twitter chambers,” interest “echo use social-media trolls who strong individuals with have quently, among atti- interactions common political Descriptively, most 6 were behaviors. interacting impacted that and substantially evidence tudes accounts no these find Inter- Russian We with the Agency. by 2017 Research operated late net accounts from Twitter Democrats on and data Republicans with longitudi- 1,239 combining open on an question data remains nal this public investigate the We influ- on question. online impact of their strategy campaigns, the ence analyze studies numerous While Significance oua idmidctsta usassca-ei campaign social-media Russia’s that indicates wisdom Popular a,b .Snhn Hillygus Sunshine D. , odsrb h usa apina “the as campaign Russian the describe to e eateto oilg,Uiest of University Sociology, of Department . y raieCmosAttribution-NonCommercial- Commons Creative c afr colo ulcPolicy, Public of School Sanford . y https://www.pnas.org/lookup/suppl/ a,c,d NSLts Articles Latest PNAS , | https:// f8 of 1

POLITICAL SCIENCES Studies that examine the content of Russia’s social-media stereotypes, and political behaviors of a large group of American campaign reveal that it was primarily designed to hasten polit- social-media users. ical polarization in the United States by focusing on divisive issues such as police brutality (14, 15). According to Stella et Research Design al. (8), such efforts “can deeply influence reality perception, Perhaps the most valid research design for studying the impact affecting millions of people’s voting behavior. Hence, maneu- of Russia’s social-media campaign would be to dispatch troll- vering opinion dynamics by disseminating forged content over style messages as part of a field experiment on a social-media online ecosystems is an effective pathway for social hacking.” platform. Needless to say, such a study would be highly uneth- Howard et al. (6) similarly argue that “the IRA Twitter data ical. Instead, we rely on observational methods that avoid such shows a long and successful campaign that resulted in false pitfalls, but provide a less direct estimate of the causal effect accounts being effectively woven into the fabric of online US of IRA trolls. Our analysis leverages a longitudinal survey of political conversations right up until their suspension. These 1,239 Republicans and Democrats who use Twitter frequently embedded assets each targeted specific audiences they sought to that was fielded in late 2017 by Bail et al. (3). Though this manipulate and radicalize, with some gaining meaningful influ- study was designed for other purposes, unhashed, restricted- ence in online communities after months of behavior designed access data provided to us by Twitter’s Elections Integrity Hub to blend their activities with those of authentic and highly allowed us to identify individuals who interacted with IRA engaged US users.” Such conclusions are largely based upon accounts between survey waves. Thus, we have political attitudes qualitative analyses of the content produced by IRA accounts measured preinteraction and postinteraction, allowing us to esti- and counts of the number of times Twitter users engaged with mate individual-level changes in political attitudes and behaviors IRA messages. over time. That sizable populations interacted with IRA messages on In October 2017, Bail et al. (3) hired the survey firm YouGov Twitter, however, does not necessarily mean that such messages to interview members of its large online panel who 1) iden- influenced public attitudes. Foundational research in political tify as either a Republican or a Democrat; 2) visit Twitter at science, sociology, and social psychology provides ample rea- least 3 times a week; 3) reside within the United States; 4) son to question whether the IRA campaign exerted significant were willing to share their Twitter handle or ID; and 5) did not impact. Studies of political communication and campaigns, for set their account to protected, or a nonpublic setting. Respon- example, have repeatedly demonstrated that it is very difficult to dents were also stratified in order to recruit approximately equal change peoples’ views (16). Political messages tend to have “min- numbers of respondents who identify as “strong” or “weak” imal effects” because the individuals most likely to be exposed partisans. Readers should therefore note that this sample is to persuasive messages are also those who are most entrenched not representative of the general US population and does not in their views (17). In other words, if only Twitter users with include independent voters (see SI Appendix for a full compari- very strong political views are exposed to IRA trolls, it might son of the demographics of our sample compared to the general not make their views any more extreme. An extensive literature population). confirms that such “minimal effects” are the norm in politi- Respondents were paid the equivalent of $11 for completing cal advertising—even when microtargeting of ads to users is this survey via the survey firm’s points system. In November 2017, employed (18). Indeed, a recent meta-analysis concludes that all survey respondents were offered $12 to complete a follow-up the average treatment effect of political campaigning is precisely survey with the same battery of questions fielded in the orig- zero (19). If American political agents struggle to persuade vot- inal survey. The study by Bail et al. (3) also involved a field ers, it seems that foreign agents might struggle even more to experiment. In the period between the 2 surveys just described, influence public opinion—not only because the relatively anony- respondents were randomized into a treatment condition where mous nature of social-media interactions may raise issues of they were offered financial incentives to follow a Twitter bot cre- source credibility, but also because of linguistic and cultural ated by the authors that was designed to expose them to messages barriers that undoubtedly make the development of persuasive from opinion leaders from the opposing political party. Our anal- messaging more challenging. ysis accounts for this design, and we provide a more detailed Still, previous studies of IRA activity indicate that the orga- description of the sampling procedure, survey attrition, and other nization not only aimed to persuade voters to support a par- research design issues in SI Appendix. The bots of Bail et al. (3) ticular candidate or policy, but also impersonated stereotypes did not retweet any messages from IRA troll accounts. about Republicans and Democrats in order to increase affec- Both of the surveys of Bail et al. (3) included a series of tive political polarization (6). Such efforts could have indirect questions about political attitudes that we used to measure 4 effects beyond changing issue attitudes. There is some evi- of the 6 outcomes analyzed below. The survey included 2 mea- dence, for example, that reactions to social-media messages can sures of affective political polarization: a feeling thermometer, have a strong impact on audiences above and beyond the mes- where respondents were asked to rate the opposing political sages themselves (20). Observing a trusted social-media contact party on a scale of 0 to 100; and a social distance scale, where argue with an IRA account that impersonates a member of the respondents were asked whether they would be unhappy if a opposing political party, for example, could thus contribute to member of the opposing political party married someone in stereotypes about the other side. Even if the trolls did not exac- their family or if they had to socialize or work closely with a erbate ideological polarization, that is, they may have abetted member of the opposing party. The surveys also included 2 mea- affective polarization (21). Finally, there is some evidence that sures of ideological polarization: a 7-point ideology scale, where IRA accounts were not only aiming to influence attitudes, but respondents were asked to place themselves on a continuum that also political behaviors. For example, some studies indicate that ranges from “liberal” to “conservative”; and a 10-item index that the IRA attempted to demobilize African-American voters by asked respondents to agree or disagree with a series of liberal- spreading negative messages about Hillary Clinton prior to the or conservative-leaning statements. The ideological polariza- 2016 US presidential election (6). Hence, any analysis of the tion measures were coded such that positive values represent impact of the IRA campaign must not only examine its direct increasing ideological polarization and negative values repre- impact upon Americans’ opinions about policy issues, but also sent decreasing polarization. The full text of these questions is their attitudes toward each other and their overall level of politi- available in SI Appendix. cal engagement. Our study is thus designed to scrutinize whether Because previous studies indicate that interaction with IRA interactions with IRA trolls shaped the issue attitudes, partisan accounts may lead people to detach themselves from politics,

2 of 8 | www.pnas.org/cgi/doi/10.1073/pnas.1906420116 Bail et al. Downloaded by guest on September 23, 2021 Downloaded by guest on September 23, 2021 w ae rfiedt.Teeicue1 iayidctrthat indicator binary a 1) include firm’s These survey data. of the profile observation via panel through provided own made or (3), behavior, al. Twitter et respondents’ Bail of political by study collected influence original either the were might measures These that behaviors. and factors attitudes confounding for accounts account IRA with engagement of Appendix). form (SI common mentions than less retweets far because a part are below—in to findings unlikely substantive via be our would analysis impact measures messages sensitivity missing a these such performed that indicated We to which determine users. Twitter to exposed other able were or not con- trolls were respondents retweeted we our cases, occasionally some In whether sample In themselves. our nontrolls. by messages in such tent trolls deleted IRA fol- Twitter the original were since addition, the who study, those alongside our by in retweets does messages respondents it IRA they by that of is lowed long measure retweets how our include of or limitation not messages further that A such verify them. viewed viewed cannot actually accounts—we people troll these mention our who fol- Though who people accounts. accounts—or troll low IRA follow to who exposure people identifies of measure types all capture not was sample partisans). weak our and that strong however, both include (recall, to study partici- stratified original to interest the invited political were in of partisans pate levels only higher since respondents, reflect among may interaction sample early IRA of our in rate million accounts within elevated The 69 IRA 2%. with its approximately By or interacted of 2018, engagements. had million users those 1.4 active of that monthly reported impact Twitter the respon- comparison, providing estimating time, of first for 3.7% the for waves, leverage account IRA survey an accounts. with our troll interacted between dents with month engaged the directly During 11.3% with interacted and respondents accounts, our of IRA 19.0% that show in data time) informa- Twitter over interactions detailed of (and provide types respondents different We across these of tweet). distribution the their friend about in respondent’s tion a troll when message a troll’s troll mentions a a to not (following exposed tweets was that being troll’s but activities a or viewing troll or to engagements, a lead indirect mentions reasonably and could that troll) retweeting tweet a tweet, a by troll liking produced or a tweet, (liking troll troll a a with engagements direct accounts. Twitter, IRA accounts. from 4,256 of dataset most consists unhashed for currently the which identifiers received numeric and describe requested We available, not currently does is data it these of but the Committee version Intelligence public through A Senate 2018. US early source identified the in and Twitter were Twitter the of accounts work via joint These accounts above. IRA all described with iden- account accounts associated numeric obtained tifiers IRA we constructed First, We follows. with above. as described measure this survey interacted 2nd and respondents 1st the between whether of cator in system scoring ideological this Appendix. of SI description detailed provide more We same followed a described. the just respondent sample using leader identification opinion the party network-based calculated respondent’s accounts ideolog- that the political shared network the that of Twitter of respondent’s percentage measure each the a elected of included by bias followed we ical are Second, who people leaders” (22). “opinion or officials 4,176 officials the of elected sample network- are after a a who and built via that before identified method were followed sampling number accounts user Political the survey. each counted 2-wave accounts we political First, from of observed accounts. outcomes Twitter behavioral 2 respondents’ includes also analysis our ale al. et Bail u oesas nlddasre fcnrlvralsto variables control of series a included also models Our does it that is interaction troll of measure our of limitation One both of inclusive as interactions troll operationalized We indi- binary a is models our in variable independent key The vrl,the Overall, Appendix. SI rntta neetd ol o a o olwwa’ on on affairs going what’s Others public follow not. and you or government say on in you going Would election most interested. an affairs that public there’s aren’t and whether government time, in the on of going news what’s seem overall people follow “Some of to higher question: following measure where the a from usage derived 3) interest Twitter frequency; of greater frequency indicate 2) scores of Republican; measure a as 4-point identifies a respondent the whether describes eepoe aeinrgeso re oetmt average estimate to trees accounts. regression IRA Bayesian with interacted employed 2nd indicating respondent We and our variable survey binary 1st of the a each the as whether measured in between was change treatment variables surveys; modeled behavioral we and measures, attitudinal outcome 6 our views. entrenched sus- more political least their be of may effects—because who persuasion those to were ceptible trolls respondents with significant. the interact to that statistically likely indicates most analysis not initial is our summarize, Though association To usage. this accounts IRA Democrats, Twitter with interact than of to likely polit- frequency more overall appeared by is party). Republicans followed interaction political IRA interest, own of ical respondent’s predictor strongest the 2nd toward The lean they accounts that political of follow echo percentage respondents’ the stan- of as strength (measured presents the chambers an is 1 strongest interaction mentions the IRA Fig. that of that showing predictor respondents). model, tweet this 1,239 from a coefficients of or dardized liking liking, (76 or retweeting, Bail account mentioning, account by IRA by conducted IRA 2017 survey an October 1st following in interacted the (3) to respondent al. prior the indi- et account not binary IRA a or an is whether with model describes this in that outcome cator regres- The binomial accounts. interacted who IRA employed those of with first characteristics the identify we accounts to IRA models variables, sion with outcome interactions our of on impact the analyzing Before Findings (3). respon- al. et the the Bail in of condition whether study treatment original experimental describes the to that geographic below assigned 4 was indicator reported of dent models binary 1 all in a Finally, located States. include is male, United or the is white, in respondent is regions 6) degree, the and college whether income; a respondent’s family indicate has of 4) that measure time]?”; variables continuous the a binary of 5) most birth; or of time, year the of some then, ihitrs npltc,adwoueTitrmr hnoc a were day a once than Survey more accounts. IRA Twitter CIs. with use network, interact 95% who to Twitter likely and and their most politics, 90% in in describe homophily interest ideological high lines strong blue standard- with describe and respondents circles estimates, Purple IRA. point Russian ized the with associated accounts 1. Fig. # Twitter AccountsFollowed % Co−Partisans Followed Frequency of Twitter Use oetmt h mato neatn ihIAacut on accounts IRA with interacting of impact the estimate To Interest inPolitics College Degree Family Income North Central North Year ofBirth ioilrgeso oe rdcigitrcinwt Twitter with interaction predicting model regression Binomial Republican Northeast South White Male 10123 2 1 0 −1 . . . hrl tal nynwand now only all, at [hardly NSLts Articles Latest PNAS | f8 of 3

POLITICAL SCIENCES treatment effects upon treated respondents (or ATTs), as well that were regularly active each day from approximately 300 to as heterogeneous treatment effects by other covariates in the 100. Seventy-five percent of the troll accounts our respondents models. This family of techniques uses nonparametric Bayesian interacted with were created prior to the crackdown—and had regression to reduce dimensionally adaptive random basis ele- large followings, suggesting that they were particularly effective ments. More specifically, we employed the Bayesian causal for- at avoiding detection—but it is important to consider the possi- est (BCF) model (23), which incorporates an estimate of the bility that we would have found a bigger impact if our interactions propensity function within the response model that amounts to went back further in time. A 2nd limitation is that our treat- a covariate-dependent prior on the regression function. This ment measure is binary, but we might expect a greater effect technique is particularly well suited to identify heterogeneous from multiple interactions (a dosage effect). Likewise, we might treatment effects by allowing treatment heterogeneity to be reg- expect variation in the impact based on the nature of the interac- ularized separately by each control variable in the model. We tion; direct engagement with a tweet (e.g., liking, retweeting, or omitted individuals who interacted with trolls prior to the 1st mentioning a tweet) might have more influence than an indirect survey wave, leaving 44 individuals treated and 1,106 in the con- engagement (e.g., following a troll or a friend mentions a troll). trol group for the main models presented below. We conducted To address these concerns, we obtained additional survey data 3 additional sets of analyses that expanded the number of treated collected as part of YouGov profile surveys outside of our 1- cases considerably—a less conservative operationalization of month study window. Unfortunately, the only relevant survey treatment in which pretreated individuals were included (SI question consistently measured in profile surveys across this time Appendix), a synthetic treatment measure in the sensitivity anal- period was a 5-point ideology scale, which we were able to obtain ysis (SI Appendix), and an expanded time frame using additional for all but 2 of our respondents. Of the outcomes considered survey data presented below—with no change in results. in our earlier analysis, self-identified ideology might be some- Fig. 2 reports the ATTs of interacting with IRA accounts on what less susceptible to persuasion, but it still allows for a test of each of the outcomes in our model with intervals derived from ideological polarization across a wider time frame. the 97.5th and 2.5th quantiles, respectively. These effects were Treatment is defined as interacting with a troll between the standardized to have a mean of 0 and SD of 1 to facilitate inter- earliest and latest measure of ideology available for each respon- pretation. As this figure shows, IRA interaction did not have dent in the YouGov profile dataset between February 2016 and a significant association with any of our 6 outcome variables. April 2018. These models include the same specifications as Figs. 3–5 plot the individual treatment effects of troll interaction those in the preceding analysis, including the exclusion of pre- against the variables associated with troll interaction, collapsed treated respondents who interacted with trolls prior to our first into binary categories to facilitate interpretation. Fig. 3 shows no measure of ideology in the profile dataset. We ran separate mod- substantial differences in the effect of troll interactions, accord- els for different levels of dosage—with the treated group in each ing to level of political interest across each of our outcomes. model defined as respondents who interacted with trolls 1 or Figs. 4 and 5 report no substantial differences for frequency of more times (treated = 213, control = 1,017), 2 or more times Twitter usage or party identification, though there is a sugges- (treated = 110, control = 1,120), or 3 or more times (treated = tive difference between Republicans and Democrats in change in 67, control = 1,163). The larger size of the treatment group in the number of political accounts that respondents followed. We this analysis allowed us to explore the effect of different types observed no other heterogeneous treatment effects by all other of interactions. While in the preceding analysis, we operational- covariates in our models either (SI Appendix). ized troll interactions as either direct or indirect engagement, the latter might not be sufficient to change political attitudes. Assessing Dosage Effects over an Extended Time Period Therefore, we also ran separate models in which treatment was Our results show no evidence that interacting with Russian Twit- operationalized only as direct engagement. ter trolls influenced the attitudes or behaviors of Republicans Fig. 6 reports the ATT and associated 95% credible intervals. and Democrats who use Twitter at least 3 times a week, but As in the preceding models, measuring the effect of interacting the analyses are limited in several ways. First, our measure of with trolls on ideological polarization, positive coefficients rep- interactions is restricted to the time period between October resent increasing ideological polarization, while negative coef- and November 2017. Yet, Twitter suspended many of these ficients represent decreasing polarization. The outcome was accounts in early 2017, dropping the number of IRA accounts again standardized to have a mean of 0 and SD of 1 to facili- tate interpretation. Despite increasing the size of the treatment group substantially, we continued to find no significant effects

Change in Opposing Party Feeling Thermometer of interacting with IRA trolls across all models and both types of interactions. The results showed no evidence of ideologi- cal polarization—if anything, though small and not statistically Change in Desired Social Distance from Opposing Party significant, the ATTs were negative.

Change in 7−Point Self−rated Ideology Conclusion Coordinated attempts to create political polarization in the Change in Liberalism/Conservativism Index United States by Russia and other foreign governments have become a focus of public concern in recent years. Yet, to our

Change in # of Political Accounts Followed knowledge, no studies have systematically examined whether such campaigns have actually impacted the political attitudes or behaviors of Americans. Analyzing one of the largest known Change in % Co−Partisans Followed on Twitter efforts to date using a combination of unique datasets, we found

−0.4 −0.2 0.0 0.2 no substantial effects of interacting with Russian IRA accounts on the affective attitudes of Democrats and Republicans who Fig. 2. BCF models describing the effect of interacting with Russian IRA accounts on change in political attitudes and behaviors of Republican and use Twitter frequently toward each other, their opinions about Democratic Twitter users who responded to 2 surveys fielded between substantive political issues, or their engagement with politics on October and November 2017. Purple circles describe the average treat- Twitter in late 2017. ment effects on the treated, and blue lines describe 95% credible intervals. Even though we find no evidence that Russian trolls polar- Interaction with IRA accounts has no significant effect on all 6 outcomes. ized the political attitudes and behaviors of partisan Twitter

4 of 8 | www.pnas.org/cgi/doi/10.1073/pnas.1906420116 Bail et al. Downloaded by guest on September 23, 2021 Downloaded by guest on September 23, 2021 ileeto,wihi ieyrgre saciia ucuefor al. juncture et Bail critical a as regarded widely is which presiden- 2016 election, the trolls tial during behavior IRA time. or whether in attitudes determine point public single systematically influenced a to con- at unable population diminish were 1 We to not because limited media should was social analysis effects on our campaigns null influence foreign these about 2017, cern late in users −0.75 −0.50 −0.25 −0.20 −0.15 −0.10 −0.05 0.00 0.05 0.10 0.00 0.25 0.00 i.3. Fig. niiulefcso neatn ihRsinIAacut npltclattdsadbhvo ylvlo esitrs sn BCFs. using interest news of level by behavior and attitudes political on accounts IRA Russian with interacting of effects Individual Change in Thermometer Rating of Opposing Party Change inDesiredSocialDistancefrom Change OpposingParty Party Change inThermometerRatingofOpposing Hi ihNw neetLwNw neetHg esItrs Low News Interest HighNews Interest Low News Interest High News Interest Low News Interest HighNews Interest Low News Interest High News Interest g esItrs o esItrs Hi Low News Interest h News Interest Change in # Political Accounts Followed Change in % Co−Partisans inTwitter in%Co−Partisans Change Network Change in#Political AccountsFollowed hnei ee−on dooySaeChangeinLiberalism/ConservatismIndex Change inSeven−Point IdeologyScale −0.050 −0.025 0.000 0.025 n s wte eaieyfeunl a es ie week). a times 3 least (at frequently party relatively Republican Twitter or use Democratic the and with identified who people focused. we which impactful upon more Russian period become late-2017 the the to that since evolved possible has also campaign is government’s It campaigns. misinformation −0.2 −0.1 −0.1 0.0 0.1 0.2 0.0 0.1 ute iiaino u nlssi hti a etitdto restricted was it that is analysis our of limitation further A g esItrs Low News Interest h News Interest NSLts Articles Latest PNAS | f8 of 5

POLITICAL SCIENCES Change in Thermometer Rating of Opposing Party Change in Desired Social Distance from Opposing Party

0.00

0.1 −0.05

−0.10

0.0

−0.15

−0.20 −0.1 Frequent Twitter Users Infrequent Twitter Users Frequent Twitter Users Infrequent Twitter Users

Change in Seven−Point Ideology Scale Change in Liberalism/Conservatism Index

0.2 0.25

0.1 0.00

0.0 −0.25

−0.1 −0.50

−0.75 −0.2 Frequent Twitter Users Infrequent Twitter Users Frequent Twitter Users Infrequent Twitter Users

Change in # Political Accounts Followed Change in % Co−Partisans in Twitter Network

0.025

0.10

0.000

0.05

−0.025

0.00

−0.050 Frequent Twitter Users Infrequent Twitter Users Frequent Twitter Users Infrequent Twitter Users

Fig. 4. Individual effects of interacting with Russian IRA accounts on political attitudes and behavior by frequency of Twitter usage using BCFs.

It is possible that trolls have a stronger influence on political ysis only examines Twitter. Though Twitter remains one of the independents or those more detached from politics in general more influential social-media platforms in the United States at (though we did not observe significant effects among those who the time of this writing—and was targeted by the Russian IRA expressed weak attachments to either party). Our study was also far more than other social-media platforms—it has a substan- limited to the United States, whereas reports indicate that the tially smaller user base than Facebook and offers a unique, and IRA is active in many other countries as well. Finally, our anal- highly public, form of social-media engagement to its users. It

6 of 8 | www.pnas.org/cgi/doi/10.1073/pnas.1906420116 Bail et al. Downloaded by guest on September 23, 2021 Downloaded by guest on September 23, 2021 ale al. et media Bail or candidate influenced can- we trolls example, Russian For if outcomes. determine behavioral not and political of set more or been engagement. audiences user of have for types structures might other other with influence platforms other Russian on that pronounced possible thus is nte iiaini htoraayi vlae limited a evaluated analysis our that is limitation Another −0.75 −0.50 −0.25 −0.20 −0.15 −0.10 −0.05 0.00 0.05 0.10 0.00 0.25 0.00 i.5. Fig. niiulefcso neatn ihRsinIAacut npltclattdsadbhvo ypria dnicto sn BCFs. using identification partisan by behavior and attitudes political on accounts IRA Russian with interacting of effects Individual Change in Thermometer Rating of Opposing Party Change inDesiredSocialDistancefrom Change OpposingParty Party Change inThermometerRatingofOpposing Change in # Political Accounts Followed Change in % Co−Partisans inTwitter in%Co−Partisans Change Network Change in#Political AccountsFollowed eort eulcn eort Republicans Democrats Republicans Republicans Democrats Democrats Republicans Republicans Democrats Democrats Republicans Democrats hnei ee−on dooySaeChangeinLiberalism/ConservatismIndex Change inSeven−Point IdeologyScale −0.050 −0.025 0.000 0.025 eal oln u aat oe ls ial,teobserva- the Finally, files. the voter of to identification rigorous data prevents study troll our our not of link nature or tional to whether able of study might salience be not studies future the such could behavior—though voting changing ways, shaped also by interaction other We or in issues. trust opinion societal political public about shaped attitudes they as if or behavior −0.2 −0.1 −0.1 0.0 0.1 0.2 0.0 0.1 NSLts Articles Latest PNAS | f8 of 7

POLITICAL SCIENCES Interaction Type ter engagements may further explain why we observed null All Engagements effects above. Direct Engagements Only While there are myriad reasons to be concerned about the Russian trolling campaign—and future efforts from other for- eign adversaries both online and offline—it is noteworthy that 3+ the people most at risk of interacting with trolls—those with strong partisan beliefs—are also the least likely to change their attitudes. In other words, Russian trolls may not have sig- nificantly polarized the American public because they mostly 2+ interacted with those who were already polarized. We conclude by noting important implications of our study for

Number of Troll Interactions Number of Troll future research on social media, political polarization, and com- putational social science (24). Given the high-profile nature of 1+ the Russian IRA efforts, it is critical to have systematic empir- ical assessment of the impact on the public. While there is still much to be learned, our study offers an important contribution

−0.50 −0.25 0.00 0.25 0.50 to this understudied issue. In addition, our study contributes to the growing field of computational social science and, more Fig. 6. Assessing dosage effects on self-reported ideology over an extended specifically, provides an example of how conventional forms of time period. Circles describe standardized point estimates, and lines describe research such as public opinion surveys can be fruitfully com- 95% credible intervals for different amounts of direct and indirect engage- ment with IRA accounts. Here, again, interactions with troll accounts bined with observational text and network data collected from over an extended period show no significant association with ideological social-media sites in order to address complex phenomena such polarization. as the impact of social-media influence campaigns on politi- cal attitudes and behavior. Though further studies are urgently causal impact of the IRA campaign. Because of these limitations, needed on this issue, we hope our contribution will provide a additional research is needed to validate our findings through model to future researchers who aim to study this complex and studies of other social-media campaigns, other platforms, and multifaceted issue. with different research methods. Despite these issues, our results offer an important reminder Materials and Methods that the American public is not tabula rasa and may not be See SI Appendix for a detailed description of the materials and methods easily manipulated by propaganda. Our results show that even used within this study, additional robustness checks, and links to our though active partisan Twitter users engaged with trolls at a replication materials. This research was approved by the Institutional substantially higher rate than reported by Twitter, the vast Review Boards at Duke University and New York University. All participants majority (80%) did not interact with an IRA account. And provided informed consent. Data for this study can be accessed at for those who did, these interactions represented a minus- https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/ UATZBA (25). cule share of their Twitter activity—on average, just 0.1% of their liking, mentioning, and retweeting on Twitter. That ACKNOWLEDGMENTS. This work was supported by the NSF and Duke interactions with trolls were such a small fraction of all Twit- University. We thank the Twitter Elections Integrity Hub for sharing data.

1. C. R. Sunstein, # Republic: Divided Democracy in the Age of Social Media (Princeton 13. K. Roose, Social media’s forever war. NY Times, 17 December 2018. https://www. University Press, Princeton, NJ, 2018). nytimes.com/2018/12/17/technology/social-media-russia-interference.html. Accessed 2. E. Bakshy, S. Messing, L. A. Adamic, Exposure to ideologically diverse news and 13 November 2019. opinion on Facebook. Science 348, 1130–1132 (2015). 14. D. A. Broniatowski et al., Weaponized health communication: Twitter bots and 3. C. A. Bail et al., Exposure to opposing views on social media can increase political Russian trolls amplify the vaccine debate. Am. J. Public Health 108, 1378–1384 (2018). polarization. Proc. Natl. Acad. Sci. U.S.A. 115, 9216–9221 (2018). 15. L. G. Stewart, A. Arif, K. Starbird, “Examining trolls and polarization with a retweet 4. A. Guess, J. Nagler, J. Tucker, Less than you think: Prevalence and predictors of fake network” in Proceedings of WSDM Workshop on Misinformation and Misbehavior news dissemination on Facebook. Sci. Adv. 5, eaau4586 (2019). Mining on the Web (MIS2) (ACM, New York, 2018). 5. W. Phillips, This Is Why We Can’t Have Nice Things: Mapping the Relationship 16. B. R. Berelson, P. F. Lazarsfeld, W. N. McPhee, W. N. McPhee, Voting: A Study of Between Online Trolling and Mainstream Culture (MIT Press, Cambridge, MA, Opinion Formation in a Presidential Campaign (University of Chicago Press, Chicago, 2015). 1954). 6. P. N. Howard, B. Ganesh, D. Liotsiou, J. Kelly, C. Franc¸ois, “The IRA, Social 17. J. R. Zaller, The Nature and Origins of Mass Opinion (Cambridge University Press, Media and Political Polarization in the United States, 2012-2018” (Working Cambridge, UK, 1992). Paper 2018.2, Project on Computational Propaganda, Oxford, 2018). https:// 18. K. Endres, C. Panagopoulos, Cross-pressure and voting behavior: Results from comprop.oii.ox.ac.uk/wp-content/uploads/sites/93/2018/12/The-IRA-Social-Media- randomized experiments. J. Politics 81, 1090–1095 (2018). and-Political-Polarization.pdf. Accessed 13 November 2019. 19. J. L. Kalla, D. E. Broockman, The minimal persuasive effects of campaign contact in 7. D. L. Linvill, P. L. Warren, “Troll factories: The internet research agency and state- general elections: Evidence from 49 field experiments. Am. Pol. Sci. Rev. 112, 148–166 sponsored agenda building” (Working paper, Clemson University, Clemson, SC, (2018). 2018). http://pwarren.people.clemson.edu/Linvill Warren TrollFactory.pdf. Accessed 20. V. Dounoucos, D. S. Hillygus, C. Carlson, The message and the medium: An experi- 13 November 2019. mental evaluation of the effects of Twitter commentary on campaign messages. J. 8. M. Stella, E. Ferrara, M. De Domenico, Bots increase exposure to negative and inflam- Inf. Technol. Polit. 16, 66–76 (2019). matory content in online social systems. Proc. Natl. Acad. Sci. U.S.A. 115, 12435–12440 21. S. Iyengar, S. J. Westwood, Fear and loathing across party lines: New evidence on (2018). group polarization. Am. J. Pol. Sci. 59, 690–707 (2015). 9. R. DiResta et al., “The tactics & tropes of the Internet Research Agency” 22. P. Barbera, Birds of the same feather tweet together: Bayesian ideal point estimation (White paper, New Knowledge, Austin, TX, 2018; https://disinformationreport.blob. using Twitter data. Political Anal. 23, 76–91 (2015). core.windows.net/disinformation-report/NewKnowledge-Disinformation-Report- 23. P. R. Hahn, J. S. Murray, C. Carvalho, Bayesian regression tree models for causal infer- Whitepaper.pdf). ence: Regularization, confounding, and heterogeneous effects. arXiv:1706.09523 (23 10. J. Tucker et al., “Social media, political polarization, and political disinformation: A May 2019). Review of the scientific literature” (Tech. Rep., Hewlett Foundation, Menlo Park, CA, 24. D. Lazer et al., Life in the network: The coming age of computational social science. 2018). Science 323, 721–723 (2009). 11. S. Aral, D. Eckles, Protecting elections from social media manipulation. Science 365, 25. C. A. Bail et al., Replication data for ”Assessing the Russian Internet Agency’s impact 858–861 (2019). on the political attitudes and behaviors of U.S. Twitter users in late 2017” 12. K. H. Jamieson, Cyberwar: How Russian Hackers and Trolls Helped Elect a President Proceedings of the National Academy of Sciences. https://dataverse.harvard.edu/ What We Don’t, Can’t, and Do Know (Oxford University Press, Oxford, UK, 2018). dataset.xhtml?persistentId=doi:10.7910/DVN/UATZBA. Deposited 7 November 2019.

8 of 8 | www.pnas.org/cgi/doi/10.1073/pnas.1906420116 Bail et al. Downloaded by guest on September 23, 2021