Red Bots Do It Better: Comparative Analysis of Social Bot Partisan Behavior Luca Luceri* Ashok Deb University of Applied Sciences and Arts of Southern USC Information Sciences Institute Switzerland, and University of Bern Marina del Rey, CA Manno, Switzerland [email protected] [email protected] Adam Badawy Emilio Ferrara USC Information Sciences Institute USC Information Sciences Institute Marina del Rey, CA Marina del Rey, CA [email protected] [email protected] ABSTRACT INTRODUCTION Recent research brought awareness of the issue of bots on social During the last decade, social media have become the conventional media and the significant risks of mass manipulation of public opin- communication channel to socialize, share opinions, and access the ion in the context of political discussion. In this work, we leverage news. Accuracy, truthfulness, and authenticity of the shared content Twitter to study the discourse during the 2018 US midterm elections are necessary ingredients to maintain a healthy online discussion. and analyze social bot activity and interactions with humans. We However, in recent times, social media have been dealing with a collected 2.6 million tweets for 42 days around the election day from considerable growth of false content and fake accounts. The re- nearly 1 million users. We use the collected tweets to answer three sulting wave of misinformation (and disinformation) highlights the research questions: ¹iº Do social bots lean and behave according to pitfalls of social media networks and their potential harms to several a political ideology? ¹iiº Can we observe different strategies among constituents of our society, ranging from politics to public health. liberal and conservative bots? ¹iiiº How effective are bot strategies? In fact, social media networks have been used for malicious pur- We show that social bots can be accurately classified according poses to a great extent [11]. Various studies raised awareness about to their political leaning and behave accordingly. Conservative bots the risk of mass manipulation of public opinion, especially in the con- share most of the topics of discussion with their human counterparts, text of political discussion. Disinformation campaigns [2, 5, 12, 14– while liberal bots show less overlap and a more inflammatory attitude. 16, 21, 23, 25, 29] and social bots [3, 4, 20, 22, 24, 28, 30, 31] have We studied bot interactions with humans and observed different been indicated as factors contributing to social media manipulation. strategies. Finally, we measured bots embeddedness in the social The 2016 US Presidential election represents a prime example network and the effectiveness of their activities. Results show that of the significant perils of mass manipulation of political discourse. conservative bots are more deeply embedded in the social network Badawy et al. [1] studied the Russian interference in the election and more effective than liberal bots at exerting influence on humans. and the activity of Russian trolls on Twitter. Im et al. [17] suggested that troll accounts are still active to these days. The presence of KEYWORDS social bots does not show any sign of decline [10, 31] despite the social media, political elections, social bots, political manipulation attempts from social network providers to suspend suspected, ma- licious accounts. Various research efforts have been focusing on ACM Reference Format: the analysis, detection, and countermeasures development against Luca Luceri, Ashok Deb, Adam Badawy, and Emilio Ferrara. 2019. Red Bots Do It Better: Comparative Analysis of Social Bot Partisan Behavior. In social bots. Ferrara et al. [13] highlighted the consequences asso- arXiv:1902.02765v2 [cs.SI] 8 Feb 2019 International Workshop on Misinformation, Computational Fact-Checking ciated with bot activity in social media. The online conversation and Credible Web, May 14, 2019, San Francisco, CA. ACM, New York, NY, related to the 2016 US presidential election was further examined USA, 6 pages. https://doi.org/10.1145/1122445.1122456 [3] to quantify the extent of social bots activity. More recently, Stella et al. [26] discussed bots’ strategy of targeting influential humans to *Also with USC Information Sciences Institute. manipulate online conversation during the Catalan referendum for L. Luceri & A. Deb contributed equally to this work. independence, whereas Shao et al. [24] analyzed the role of social bots in spreading articles from low credibility sources. Deb et al. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed [10] focused on the 2018 US Midterms elections with the objective for profit or commercial advantage and that copies bear this notice and the full citation to find instances of voter suppression. on the first page. Copyrights for components of this work owned by others than the In this work, we investigate social bots behavior by analyzing author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission their activity, strategy, and interactions with humans. We aim to and/or a fee. Request permissions from [email protected]. answer the following research questions (RQs) regarding social bots The Web Conference ’19, May 14, 2019, San Francisco, CA behavior during the 2018 US Midterms election. © 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-9999-9/18/06. $15.00 https://doi.org/10.1145/1122445.1122456 The Web Conference ’19, May 14, 2019, San Francisco, CA L. Luceri, A. Deb, A. Badawy, and E. Ferrara RQ1: Do social bots lean and behave according to a political ide- Table 1: Dataset statistics ology? We investigate whether social bots can be classified based on their political inclination into liberal or conservative Statistic Count leaning. Further, we explore to what extent they act similarly # of Tweets 452,288 to the corresponding human counterparts. # of Retweets 1,869,313 RQ2: Can we observe different strategies among liberal and con- # of Replies 267,973 # of Users 997,406 servative bots? We examine the differences between social bot strategies to mimic humans and infiltrate political discus- METHODOLOGY sion. For this purpose, we measure bot activity in terms of volume and frequency of posts, interactions with humans, and Bot Detection embeddedness in the social network. Nowadays, bot detection is a fundamental asset for understanding RQ3: Are bot strategies effective? We introduce four metrics to social media manipulation and, more specifically, to reveal malicious estimate the effectiveness of bot strategies and to evaluate the accounts. In the last few years, the problem of detecting automated degree of human interplay with social bots. accounts gathered both attention and concern [13], also bringing a We leverage Twitter to capture the political discourse during the wide variety of approaches to the table [7, 8, 19, 27]. While increas- 2018 US midterm elections. We collected 2.6 million tweets for ingly sophisticated techniques keep emerging [19], in this study, we 2 42 days around election day from nearly 1 million users. We then employ the widely used Botometer. explore collected data and attain the following findings: Botometer is a machine learning-based tool developed by Indiana University [9, 28] to detect social bots in Twitter. It is based on an • We show that social bots are embedded in each political side ensemble classifier [6] that aims to provide an indicator, namely bot and behave accordingly. Conservative bots abide by the topic score, used to classify an account either as a bot or as a human. To discussed by the human counterpart more than liberal bots, feed the classifier, the Botometer API extracts about 1,200 features which in turn exhibit a more provocative attitude. related to the Twitter account under analysis. These features fall in • We examined bots’ interactions with humans and observed six broad categories and characterize the account’s profile, friends, different strategies. Conservative bots stand in a more central social network, temporal activity patterns, language, and sentiment. social network position, and divide their interactions between Botometer outputs a bot score: the lower the score, the higher the humans and other conservative bots, whereas liberal bots probability that the user is human. In this study we use version v3 of focused mainly on the interplay with the human counterparts. Botometer, which brings some innovations, as detailed in [31]. Most • We measured the effectiveness of these strategies and recog- importantly, the bot scores are now rescaled (and not centered around nized the strategy of conservative bots as the most effective 0.5 anymore) through a non-linear re-calibration of the model. in terms of influence exerted on human users. In Figure 1, we depict the bot score distribution of the 997,406 DATA distinct users in our datasets. The distribution exhibits a right skew: most of the probability mass is in the range [0, 0.2] and some peaks In this study, we use Twitter to investigate the partisan behavior can be noticed around 0.3. Prior studies used the 0.5 threshold to of malicious accounts during the 2018 US midterm elections. For separate humans from bots. However, according to the re-calibration this purpose, we carried out a data collection from the month prior introduced in Botometer v3 [31], along with the emergence of in- (October 6, 2018) to two weeks after (November 19, 2018) the day creasingly more sophisticated bots, we here lower the bot score of the election. We kept the collection running after the election threshold to 0.3 (i.e., a user is labeled as a bot if the score is above day as several races remained unresolved. We employed the Python 0.3). This threshold corresponds to the same level of sensitivity module Twyton to collect tweets through the Twitter Streaming setting of 0.5 in prior versions of Botometer (cf.