Misleading Repurposing on Twitter Tuğrulcan Elmas Rebekah Overdorf LSIR, EPFL LSIR, EPFL Lausanne, Switzerland Lausanne, Switzerland [email protected] [email protected] Ömer Faruk Akgül Karl Aberer Bilkent University LSIR, EPFL Ankara, Turkey Lausanne, Switzerland [email protected] [email protected] ABSTRACT and even the style or language of the tweets. We refer to this type Twitter allows users to change their screen name and other profile of drastic shift in characteristics and subject as repurposing. attributes, which allows a malicious user to change their account’s While not all instances of account repurposing are misleading or identity or purpose while retaining their followers. We present malicious, this behavior can be indicative of misconduct. Take, for the first large scale and principled study of this phenomenon of example, a malicious user who wants to spread political propaganda misleading account repurposing on Twitter. We analyze two large on Twitter. He creates an account that appeals to certain users, e.g. datasets to understand account repurposing. We find 3,500 repur- a catphishing account, with the goal of increasing the number of posed accounts in the Twitter Elections Integrity Datasets. We also followers. He uses the account only to maximize follows and other find more than 100,000 accounts that have more than 5,000 follow- types of engagements — an activity we refer to as grinding. Once ers and were active in the first six months of 2020 using Twitter’s the account has a sufficient number of followers, the malicious user 1% real-time sample. We analyze a series of common features of changes the characteristics of the profile to be a patriotic retiree repurposed accounts that give us insight into the mechanics of and uses the account to spread political propaganda. repurposing and into how accounts can come to be repurposed. In this example, the account itself never changes hands; the same We also analyze three online markets that facilitate selling and person always holds the account credentials. However, repurposing buying Twitter accounts to better understand the effect that repur- is common when account ownership is transferred. One way an posing has on the Twitter account market. We provide the dataset of account can change hands is by mutual agreement of the previous popular accounts that are flagged as repurposed by our framework. and new owners, which may be the result of commercial activity. We see this most often with accounts that have a high follower KEYWORDS count (i.e. popular accounts). The new owner bought the account for a reason (e.g. political propaganda, spam) so they then change twitter, sockpuppets, disinformation the account to serve the purpose for which they bought it. Such accounts can even be rented out temporarily to different clients at different times, with the account being repurposed with each 1 INTRODUCTION transaction. An account can change hands when it is compromised, "As Gregor Samsa woke one morning from uneasy dreams, he found at which point the new owner can remove all content of the previous himself transformed into some kind of monstrous vermin.” - Franz account owner so that they can use the account for another purpose, Kafka / Metamorphosis [18] again retaining the follower count. Social media platforms allow users to change their profile in- It may seem more complicated to repurpose an account than formation in order to keep up with real world or online identity create a new one, but repurposing is more attractive for certain changes. For example, a user may change their real-world name adversaries. Repurposing helps a malicious user build trust in an arXiv:2010.10600v1 [cs.SI] 20 Oct 2020 and want their online identity to reflect that change, they may want account. Users may trust an older account with many followers to make their profile more anonymous, they may want to make more than a new one with few followers [23]. Similarly, fake account their profile harder to find via search, or they may make acareer detection algorithms are more lenient towards older accounts with change and want to change their description field to reflect it. a history of engagements and followers over new accounts with Not all attribute changes are genuine, however. Sporadically, no or few followers [8, 15]. Repurposing is also more economic, in journalists, bloggers, activists, and everyday Twitter users report some sense, as adversaries can repurpose an account while keeping instances of accounts that were formerly different in every way the associated email address and the phone number the same. changing overnight, like Gregor Samsa in the Kafka novel: the ac- While repurposing can occur on any platform, exact policies for count of a young man tweeting about French politics switches to an changing profile information vary, meaning that some platforms impersonal account promoting Pixar movies, or the account of an are more susceptible to account repurposing. Controversially [16, attractive young American women switches to a patriotic middle- 19, 21], Facebook limits accounts to be only personal profiles that aged man tweeting only in Chinese. In instances such as these, correspond to a real person [4] and disallows owning multiple accounts keep their followers, but transform all of their character- accounts, so anonymity and name changes are uncommon and istics overnight: name, screen name, description, location, website, Tuğrulcan Elmas, Rebekah Overdorf, Ömer Faruk Akgül, and Karl Aberer regulated. Users can create a page to post anonymously to some pro-opposition account, “Oy ve Ötesi” (English: Vote and Beyond). extent, but pages’ features are limited, e.g. they cannot befriend or He found that just prior to being called “Oy ve Hilesi,” the account interact with personal profiles. “had a sexy girl profile picture and was tweeting romantic quotes” as On the other hand, Twitter, the focus of this paper, allows for part of a scheme to artificially gain followers. Once it had acquired personal profiles, anonymous accounts using pseudonyms, and 40,000 followers, the account was sold on a webmaster forum for hobby accounts (e.g. parody, commentary, and fan accounts). These 200 Turkish Lira (∼$70 USD at that time). [31] different types of accounts are functionally the same; unlike Face- In this case, the account was initially created to grind followers book, there is no distinction between personal profiles and pages, and engagements — its only purpose was to become popular. Once so a personal profile can easily be transformed into a hobby orfan it gained 40,000 followers and appeared to be popular and therefore account. Twitter also allows users to manage multiple accounts someone with influence, the owner sold it to someone who wanted and even has an official how-to guide on the subject [2]. Twitter to influence social media users. The new owner purged the old does impose some restrictions: they do not allow impersonation or and irrelevant tweets, changed the name, the description and the platform manipulation (e.g. creating multiple accounts to inflate profile picture and, thus, shifted from a fake personal profile ofa popularity of a user or a narrative) [3]. woman to an anonymous account used to attack the opposition. Many repurposing accounts appear to change because they are Interestingly, the account kept its friends list (those accounts that it sold. Twitter explicitly prohibits account transfers or sales [35]. followed) at the time of the repurposing. However, it only followed However, there is no rule that explicitly regulates repurposing two accounts by the time of this analysis, but managed to retain of one’s own account. On Facebook however, misleading name 36,000 followers. This may be because the account needed to keep changes on pages are not allowed. Facebook states that "Name its followers that follow it back as part of a follow-back scheme. changes and merges must not result in a misleading or unintended In 2018 a popular security blogger and researcher @x0rz found connection and must not substantially change the Page’s subject that a number of fake accounts were attacking far-right French matter" [13]. It also notifies the followers or members of pages politicians in a coordinated manner. All of the accounts had the and groups when a name change occurs. Twitter has neither a same gmail address and stolen photos. Later, the accounts were misleading name change policy nor notification policy. repurposed: they purged all of their tweets, and claimed to be “some This contributions of this paper are as follows: sort of artificial neural network company or laboratory filled with fake content" [39]. It is not clear if the accounts were still employed • We study (§2) and define (§3) the phenomenon of account by the same adversary or changed hands. repurposing Twitter. We also stumbled on a similar instance of repurposing accounts • We establish a ground truth dataset of repurposed accounts “in the wild”, which we detected while following accounts that were using datasets published by Twitter Election Integrity (§4.2). posting content related to the gilet jaune (“yellow vest”) movement • We design and build a framework to detect misleading re- via the Twitter API. One account, which claimed to be a gilet jaune purposings of Twitter accounts in the wild (§4). supporter, was initially spreading polarizing content in support of • We detect and analyze the phenomenon of account repur- the protesters. It was then suddenly repurposed to be an account posing on two datasets: a set of suspended accounts which promoting Toy Story 4. The account has since been suspended. were involved in coordinated inauthentic behavior (§5.1) None of those accounts were popular, and they likely had little and popular accounts found via Internet Archive’s Twitter impact on their own.