Digital Vigilantism, Social Media and Cyber Criminality

This project is funded by the European Union

Issue 20 | December 2020

Digital vigilantism, social media and cyber criminality

Karen Allen and Isel van Zyl

Summary

Social media platforms have become powerful tools to amplify and share narratives. Like many evolving technologies, platforms such as Twitter or Facebook may be used for social good or for ill. Digital vigilantism (whereby social media platforms are used for organised shaming, hounding or doxing of a target deemed by the online community to have transgressed norms) may have real-world consequences. In extreme cases this may result in acts of ‘terrorism’ or other forms of unlawful killing.1 Yet while digital vigilantism may appropriate traditional law enforcement mechanisms, in some cases it may also try to hold law enforcement to account.

Key points

• Digital vigilantism, including using video material, is found in Kenya and South Africa, with doxing and hounding observed in conversations about foreigners in South Africa and police brutality in Kenya. • In Kenya, digital vigilantism seeks to hold traditional law enforcement mechanisms to account through acts of shaming, usually without usurping their powers. • In South Africa, digital vigilantism did at times seems to seek to usurp traditional law enforcement. • In both countries, engagement and promotion of new ideas was minimal. • Key influencers leverage existing networks to amplify their messages, including by using video material – some of it fake – to extend their reach. RESEARCH PAPER RESEARCH ESTIMATED ONLINE POPULATION Recommendations

As a key player in the UN’s Open Ended Working Group and Group of Government Experts on Advancing Responsible Government State 22 MILLION out of 58.9 million Behaviour in Cyberspace, South Africa is in a position to highlight the potential harms associated with the weaponising of social media through acts of vigilantism. 8.8 MILLION out of 51.3 million • Build capacity among policymakers and police to identify the early warning signs of digital vigilante behaviour.

• Raise public awareness of the potential cyber harms associated with digital vigilante campaigns, which may put a target’s personally identifiable information in the public domain. KENYA SOUTH AFRICA

• Social media platforms, government and civil Retaliation is conducted either virtually or through society need to develop local relationships to ‘real-world’ actions and may take the form of public understand the context and touchstone issues in denunciations or naming and shaming. which digital vigilante campaigns may develop and to devise localised tests to determine when A much-cited case from the African continent concerned content should be considered for removal. the naming and shaming of an American dentist who took part in a trophy hunt of a much-loved lion called Cecil in a Zimbabwean national park in 2018. The campaign that followed not only resulted in the online Introduction targeting of Walter Palmer and his family but also had ‘real-world’ consequences. His practice was targeted by With the rapid uptake of the Internet across many parts protesters who prevented him from going to work and of Africa, social media usage is increasing exponentially. Palmer received death threats.5 There are an estimated 8.8 million users in Kenya out of a population of 51.3 million, and 22 million users in South Africa out of a population of 58.9 million.2 The term ‘digital vigilantism’ Opinion is mixed as to whether platforms such as Twitter is contested and multi- should be viewed as publishers or platforms.3 dimensional, and is often Messages on social media sites driven by algorithms can be transmitted with extraordinary speed, scale and used as an umbrella phrase efficiency, shaping narratives and at times triggering real- world consequences. Those consequences may include Another prominent example of digital vigilantism was motivating a user to share an opinion, buy a product, the so-called Pizzagate affair6 in the United States (US), read an article, attend an event or mobilise for action. in which a conspiracy theory propagated by the much- The term ‘digital vigilantism’ is contested and multi- discussed QAnon movement circulated online. The dimensional, and is often used as an umbrella phrase to bizarre narrative appeared to target an aide of former include flagging, investigating, hounding and organised presidential hopeful Hillary Clinton. It resulted in the leaking.4 It is characterised as a form of collective justice arrest of a self-proclaimed ‘investigator’ who had opened against a perceived transgression by an individual fire on a popular restaurant in Washington DC – a or group. location at the centre of the conspiracy theory.

2 Digital vigilantism, social media and cyber criminality The incident has sparked a lively debate on the limits to This paper seeks to map the presence of digital social media freedoms in the era of ‘fake news’. vigilantism and explain the dynamics of influence. It makes a forensic examination of this type of There is much debate about whether digital vigilantism online activity through case studies in Kenya and is a spontaneous or organised phenomenon,7 and where South Africa linked to touchstone issues during the it lies along the continuum of freedom of expression COVID-19 pandemic. versus regulated comment. The purpose of this study is to locate it (in its extreme form) as one of the unintended Whereas other studies have sought to look at who the consequences of emerging digital technology. influencers are, this paper aims to examine how the influencer operates and leverages existing networks in Digital vigilantism falls into the category of information order to amplify a message. operations. Arguably, it should be considered in the framework of a broader investigation into cybercrime and disinformation, and the weaponisation of social Methodology media platforms. Indeed, as two prominent scholars on The research is divided into two sections. The first takes a the subject state, descriptive narrative approach to review current literature When lawmakers and business leaders discuss on digital vigilantism and its many manifestations. As ‘cyber attacks’, they’re generally thinking of network an emerging area of scholarship, much of this work has intrusions and exfiltration of data – for example focused on North America and Europe. password phishing, malware or DDOS attacks … Consequently, this paper relies heavily on primary Information warfare, by contrast, is an attack on sources, in particular elite interviews with scholars, cognitive infrastructure, on people themselves, on members of civil society, journalists and industry actors society, and on systems of information and belief.8 to understand the characteristics of digital vigilantism Put another way, information operations are arguably in Africa. As will be shown, digital vigilantism is highly part of a wider cybersecurity challenge ‘driven by context specific. algorithms and … conducted with unprecedented scale and efficiency’. The second part of this paper presents two detailed case studies that investigate and describe the practice of digital vigilantism in Kenya and South Africa. These At times digital vigilantism countries have been selected for their high levels of social media usage and active civil society. They differ, competes with traditional law however, in the way digital vigilantism (in its broadest enforcement, while in other form) seeks to challenge or augment existing law enforcement structures such as the police. Depending settings it seeks to reinforce it on context, they may appropriate the role of law enforcement or push for greater accountability by law enforcement. Therefore, international, regional and local discussions about developments in cyberspace and the role of In order to acquire empirical data, the ENACT Internet intermediaries who are both powerful ‘arbiters commissioned data analytics firm BrandsEye to work as and gatekeepers of expression’9 need to consider the its partner. The purpose was to measure social media balance between freedom of expression and regulating activity in Kenya and South Africa from January 2020 extreme online harms. to May 2020, focusing on two touchstone issues that have dominated discussions on Twitter during the At times digital vigilantism competes with traditional COVID-19 pandemic. law enforcement, while in other settings it can seek to reinforce it. As a form of parallel justice, it arguably The issues we tracked were xenophobia in South Africa ‘upsets institutions such as journalism, legal systems and and allegations of police brutality in Kenya. The topics policing which are normally entrusted with revealing, received much coverage in traditional media, which in judging and punishing transgressive behaviours in a turn stimulated the conversation online and added to democratic context’.10 the amplification effect.

Research Paper 20 / December 2020 3 In measuring influence and engagement our study Furthermore, the multiplier effect created by the excluded influencers who were politicians or traditional algorithms used on social media platforms enables an media organisations. Instead it focused on ‘ordinary idle or unguarded comment to generate a momentum citizens’ whose engagements or Opportunities to See of its own among a community of shared users. In (OTS) had a potential reach of over 50 000 unique extreme cases this can be transformed into real- Twitter users. world action.

The primary aim of the data analysis was to The methods and motives of digital vigilante behaviour demonstrate social media’s patterns of influence, are influential factors in determining the extent to which networks, sentiment and amplification effect, rather digital vigilantism can be considered a cyber threat. If than conduct a deep dive into the primary authors targets’ personally identifiable information is posted of content. As a result, with the exception of actors online, as a form of doxing, it could expose them to cyber whose identities are already a source of discussion harms, including data theft and fraud.12 in mainstream media, the names of social media participants have been redacted. Social media in effect enables Tweets were captured by data analytics tools and a statistically significant sample was circulated among the digital vigilantes to be judge, ‘crowd’ – i.e. human analysts. They were then classified according to whether they registered a complaint; jury and executioner, with offered support; called for action; sought solutions; few consequences or revealed identifying information (e.g. cell phone numbers, addresses) in posts. In an interview, Prof. Brett van Niekerk, a senior lecturer A detailed analysis was conducted on the transmission in cybercrime at the University of KwaZulu-Natal,13 said mechanism for authored tweets and the potential reach there was a ‘ready market’ on the ‘dark web’ for this of key influencers promoting particular keywords or kind of source material, which can be appropriated and hashtags. The study was limited to tweets posted from weaponised by cyber criminals. inside Kenya and South Africa. These included tweets in both English and vernacular languages. Lovelock argues that social media platforms do not have the norms, regimes or professional standards employed Digital vigilantism: by traditional media and the criminal justice system to state of the current debate discuss law enforcement matters publicly. Put another way, social media removes the institutional constraints, Digital vigilantism is characterised as a spectrum of reporting restrictions and multiple sourcing conventions behaviours, including organised shaming, hounding, that characterise traditional newsrooms. investigating or doxing a target. Whether it is classified Inflammatory discussions on social media platforms as a cyber threat is arguably dependent on context. ‘[involve] eluding many barriers that institutions This includes the issue being discussed and the have set up to avoid harming innocents and also to broader environment or setting in which that issue separate establishment of the facts from any decision is articulated. on the appropriate sanction (if any)’.14 Social media in For example, race remains a sensitive issue in post- effect enables digital vigilantes or so-called ‘keyboard apartheid South Africa and online discussions on this warriors’ to be judge, jury and executioner, with subject are frequently highly polarised.11 So too is the few consequences. issue of foreigners in South Africa, as the case study Online justice seeking involving shaming ‘is a social will illustrate. emotion in the sense that other people and institutions Likewise, in Kenya the theme of police impunity has leverage it to control others’.15 According to Trottier, it is historically been a contentious issue that appears to a form of social surveillance dependent on visibility and be especially prominent in times of crisis (e.g. during executed by what is often a ‘spontaneously arranged elections or when issues of national security arise). network of actors’.16

4 Digital vigilantism, social media and cyber criminality Digital platforms have the potential to amplify that … I don’t get OTPs [one-time passwords] anymore and shame with very little opportunity to revoke it. ‘Through my Facebook account was hacked several times.’ mediated coordination, the perceived transgressor This type of activity thus has both a real-world and a becomes the “transgressor victim” of scrutiny cyber dimension. and denunciation.’17

Some scholars have questioned whether digital While political actors and other public figures can vigilantism, or more specifically cybersecurity vigilantism leverage their following on social media platforms, this (in which actors share a common goal of securing the study focuses on other influencers – i.e. ‘ordinary’ citizens Internet), could be ‘shaping the future of cooperative – and how narratives are networked rather than who is criminal justice’.18 authoring them.

However, this paper confines itself to examining actions For the purposes of this paper the term ‘digital in which the primary purpose is collective denunciation vigilantism’ is value neutral and in no way intends in response to an online comment or real-world actions, to diminish the constitutional principle of freedom rather than network intrusions or regular cyberattacks. of expression. However, it does seek to reveal the mechanics of influence online. It also raises the prospect Digital vigilantism is also considered a form of that this may be appropriated by actors to challenge heightened empowerment, with group identity forming democratic institutions such as elections or law an important part of this, along with the power to share enforcement and criminal justice institutions. personally identifiable information on a target.

In 2017 a study was made of digital vigilantism among State of regulation alt-right activists in Charlottesville, US,19 where racial Social media platforms such as Facebook and Twitter tensions have polarised the community. It argues that largely rely on self-governance. However, moves in the ‘[p]ublic shaming establishes boundaries between “in United Kingdom (UK) to compel the companies behind groups” and “out groups”. It may also be characterized the platforms to remove content that promotes violence, as an act of “retaliation” in response to a shared sense of cyber-bullying, terrorism and child abuse may set the “moral outrage”.’20 tone for tighter controls elsewhere.23

‘I still can’t buy anything online South Africa ... I don’t get OTPs anymore Offences relating to cybercrime are primarily regulated under the Electronic Communications and Transactions and my Facebook account was Act 25 of 2002 (ECT Act). At the time of writing, the hacked several times.’ Cybercrimes Bill 2019 had secured its passage through Parliament but was yet to be signed into law. Once this happens, it will codify and consolidate various South African journalist Karima Brown was a victim laws under the ECT Act and also stipulate a number of of online doxing that became the subject of a highly new offences. publicised court case.21 During an interview for this These offences, known as ‘malicious communications’, paper, she shed light on the real-world consequences of as outlined in Part II of the bill,24 include promoting having her personal details, including cell phone number a data message that advocates, promotes or incites and address, made visible online. hate, discrimination or violence against a person or a After being targeted by the leadership of the opposition group of persons. It also includes the distribution of political party the Economic Freedom Fighters, she intimate content without a person’s consent – so-called received WhatsApp messages calling for her to be ‘raped, ‘revenge porn’. sexually assaulted and have my skin peeled off’.22 Section 14 of the bill aims to make it a criminal act to The online consequences of this experience illustrate the incite violence or damage to property in an electronic cyber legacy of doxing. ‘I still cannot buy anything online data message. Section 15 will criminalise data messages

Research Paper 20 / December 2020 5 that threaten individuals or groups with violence or Moreover, a new piece of draft legislation, the Prevention damage to property. Groups are characterised according and Combatting of Hate Crimes and Hate Speech Bill, is to factors such as race, gender, marital status, gender, currently making its passage through Parliament, further ethnic or social orientation, and belief. complicating the issue. If passed, it will make hate crime a criminal offence with criminal penalties.26 In terms of jurisdiction, an offence will be deemed to have been committed if the person charged is a The debate remains polarised, with some arguing that citizen or resident of South Africa or was arrested in definition creep is leading to an erosion of free speech, South Africa. Jurisdiction also applies if the offence was while others maintain that, given South Africa’s troubled directed at a person in South Africa. past, some protections are needed.

However, in practice, challenges may exist in identifying The Protection of Personal Information Act (POPI or the primary offender. Is it the author of a tweet or POPIA), which will become fully enforceable in 2021, hashtag, or those who propagate it? Given that a piece imposes limits on personal data held by companies, of content can be shared millions of times, are those including social media companies. Therefore, in theory who distribute it also culpable? Can algorithms be held at least, social media platforms could be held liable to account? for any personally identifiable information that is weaponised online. The Film and Publications Amendment Act 2019 amends the 1996 legislation to appoint an enforcement Kenya committee to regulate the distribution of online content, including films and games. The committee has the The Computer and Cybercrimes Act was enacted into power to refer material to the National Director of law in 2018. It is designed to combat online crime and Public Prosecutions. harassment, including cyber-espionage and online scams. Given Kenya’s energetic tech sector and high uptake of mobile money platforms such as M-Pesa, The debate remains polarised, lawmakers considered the legislation an urgent priority. with some arguing that However, ambiguities over definitions and scope have opened it up to considerable criticism.27 definition creep is leading to Some argue that the law’s provisions are too broad and an erosion of free speech that it has the potential to be used as a tool of state surveillance and to stifle free speech. Press reports claim that ‘offenders convicted for sharing “false” or “fictitious” This act is largely focused on preventing revenge information and propagating hate speech will be liable porn and child pornography from being published. to a fine of 5 million shillings ($49,776.01) or sentenced to However, as mentioned above, the status of social media two years in jail, or both’,28 yet what constitutes ‘false’ or platforms as publishers or forums remains an area of ‘fictitious’ is unclear. much contestation. Concerns about the scope of the law need to be South Africa’s Constitution allows for freedom of considered within the context of growing fears about the expression, but has a limited number of exclusions securitisation of the state in Kenya. This has been a trend to prevent hate speech. These include ‘advocacy hastened by the ongoing threat of violent extremist and of hatred that is based on race, ethnicity, gender or terrorist attacks in recent years.29 Civil liberties groups religion, and that constitutes incitement to cause harm’. worry that the legislation represents executive overreach. However, the debate about defining hate speech is an Like South Africa, Kenya has introduced data protection ongoing one.25 legislation – the Data Protection Act 201930 – that The Promotion of Equality and Prevention of Unfair is broadly in line with international data protection Discrimination Act – otherwise known as the Equality Bill principles. As mentioned above, in theory social media – prescribes a broader definition to include behaviours platforms could be held liable for the weaponisation that some would argue amount to hurtful speech rather of personal identifiable information as an act of than hate speech. digital vigilantism.

6 Digital vigilantism, social media and cyber criminality More specifically, a proposed law is being debated in target foreign nationals or attempts to dox or hound Kenya that targets social media. The Kenya Information individuals and thus challenge traditional methods of and Communication Amendment Bill 2019 – dubbed law enforcement. This may be conceived as a classic the Social Media Bill – is proving to be controversial. form of digital vigilantism. It too has elicited criticism from civil rights groups that it In contrast, users in Kenya exhibited digital vigilante could be used as a form of surveillance and control over behaviours that sought to hold traditional law opposition voices.31 enforcement to account through acts of shaming, rather than to usurp the role of law enforcement.

In theory social media platforms A number of search phrases and hashtags were could be held liable for the compiled to capture the data. These are listed in Table 1. The aim was to capture all conversations weaponisation of personal relating to these tags and, in the South African case, to classify a sample (9 681 conversations) as positive, identifiable information negative or neutral.

This classification was undertaken by human analysts The draft legislation potentially requires bloggers to and included words in vernacular languages generated obtain a licence from the communications authority by users in Kenya and South Africa. before being permitted to publish online. Owners of social media platforms – the definition of which is rather South Africa broad – will also be required to obtain a licence. They could find themselves in conflict with data protection During the South African investi- regulations if they are required by the state to hand over gation between January and data on users of the platform. May 2020 a prominent hashtag that emerged on Furthermore, administrators of online groups on Twitter was #PutSouthAfrica- WhatsApp, for example, will have to regulate their First. It was generated in members’ behaviour online or be held liable. mid-April by user @uLeratoPillay, a key influencer in the nationalist/xenophobia debate in South Africa. Digital vigilantism in Users on Twitter can engage with a hashtag and use it Kenya and South Africa: to either spread new ideas or re-share it among their case studies community without introducing additional comment. In tracking the dynamics of network activity on Twitter, In order to understand the networks of influence on our research found that engagement with the hashtag social media platforms and to capture the extent #PutSouthAfricaFirst was low. of digital vigilantism as a practice in Africa, the ISS partnered with BrandsEye. In other words, as Figure 1 shows, the author of the hashtag had high volumes of re-shares, but the level of The team conducted an empirical study between engagement or new ideas (indicated by the line) was January and May 2020 focusing on 315 000 low. Three-quarters of all interactions with this hashtag conversations on Twitter. Researchers monitored yielded no new ideas. Instead, the original hashtag was conversations on two touchstone issues: circulated among networks and had the potential to reach 48 million unique Twitter users. That is equivalent • Attitudes to foreign nationals in South Africa (at times expressed as xenophobia) to addressing the entire population of Uganda at once. • Excessive use of force by the police in Kenya Conversations relating to foreign nationals in South Both issues were highly topical during the height of the Africa were largely negative or neutral, with nearly half COVID-19 pandemic and demonstrated two types of of all exchanges being negative. The narrative around digital vigilantism. Chinese nationals, about whom conversations were particularly prominent, constituted 48.6% of all negative In the South African context, this form of online mentions. In contrast, 13.5% of negative mentions activity was largely characterised by calls to action to occurred in conversations about Nigerian nationals.

Research Paper 20 / December 2020 7 Table 1: Search phrases: South Africa and Kenya The focus of the conversations was largely a perception that foreigners impede employment opportunities for Search phrases South Africans. In the case of Nigerians, much of the South Africa conversation linked this community to the drugs trade • Afrophobia (hatred of cultures and peoples of Africa) in South Africa. Less than 10% of tweets involved positive • Amagrigamba (demeaning term for black foreigner) conversations about foreign nationals. • Amakula (derogatory name for Indians) • Amakwerkwere (derogatory name for Tswana) Figure 2 shows that ‘Chinese’ was the most frequently Amashangaan (tribal authority) • used word in negative conversations by South African • Chinese • Foreigner authors. To understand the context, it is necessary • Grigamba (demeaning term for black foreigner) to known that in April 2020 Chinese authorities in • Kwerekwere (derogatory word for foreigner esp Nigerians) Guangdong Province, which has China’s largest African • Makwerekwere (slang for foreigners) • Malawian community, began a campaign to forcibly test Africans • Mozambican for the coronavirus. This reportedly involved ordering • Nigerian them to self-isolate or quarantine in designated hotels. • Putshouthafricafirst • Shangaan (term used interchangeably with Tsonga) • Somali • Xenophobia 48 million Twitter users Kenya • action against police killings NETWORK NETWORK • #arbitraryarrest NETWORK • askaribrutality NETWORK • #curfewinkenyapolice #curfewkenyapolice NETWORK • HASHTAGNETWORK CIRCULATED AMONG • #endpolicebrutality • extrajudicialkillings VARIOUS NETWORKS NETWORK killpolisi NETWORK • NETWORK • policebrutal NETWORK • #policebrutality

• policekill NETWORK • polisibrutal #PutSouthAfricaFirst Note: Both positive and negative sentiments were tracked.

Figure 1: #PutSouthAfricaFirst: high volumes of activity, low levels of engagement

320 Volume 300 Engagement 280 260 240 220 200 180 160 140 120 100

# Mentions / Engagement 80 60 40 20 0 11 11 13 13 31 17 17 15 15 21 21 19 19 01 23 23 27 27 25 25 29 29 03 03 07 07 05 05 09 09 April May

8 Digital vigilantism, social media and cyber criminality Landlords allegedly evicted African residents, Figure 3: Images linked to fake news drive forcing many to sleep on the street. In response, the anti-Chinese sentiment conversation on Twitter around Chinese nationals living in South Africa grew increasingly xenophobic and assumed a retaliatory character.

Figure 2: ‘Chinese’ a prominent focus for South Africans on Twitter

The ISS–BrandsEye study identified other seemingly influential accounts but has withheld their details in the interests of data privacy. The primary focus was on how influence is networked rather than who the influencers are. Interestingly, the @uLeratoPillay account disappeared (apparently suspended by Twitter) and then re-surfaced in September with 59 million followers, Our research showed how one incident of so-called declaring ‘I’m back’. ‘fake news’, propelled by potent images posted online, was able to drive anti-Chinese sentiment on Twitter. Self-proclaimed ‘public protector’ @AdvoBarryRoux Allegations about a Chinese factory in South Africa (a parody account not linked in any way to the subjecting illegal immigrants to forced labour helped prominent South African advocate) displayed acts of to drive the sinophobia. digital vigilantism throughout the period of our study.

As the images in Figure 3 show, engagements in early January 2020 revolved around Chinese hegemony and alleged corruption, with the hashtag #ChinaMustFall. 49.5% 9.6% In April one user highlighted the original image as being fake and implored Twitter users to ‘stop spreading fake news’. TWITTER SENTIMENT For the purposes of this study, an influencer is defined BREAKDOWN as an account with more than 50 000 followers. The ISS– BrandsEye study isolated these influencers or so-called Almost half of all mentions were xenophobic ‘keyboard warriors’ from mainstream media, political and activist accounts. It found that this group comprised the Positive greatest volume of engagement at just under 60%. Negative In many cases the primary intention of such authors was Neutral to expose or share information about incidents or events while remaining anonymous. Prominent accounts, already widely reported on in the media, include parody 40.9% accounts such as @uLeratoPillay and @AdvoBarryRoux.

Research Paper 20 / December 2020 9 On 3 June the account’s author tried to find a Nigerian spread the coronavirus in the Free State. Local Twitter who was allegedly selling drugs to children. This mention users tried to locate him. was re-shared 492 times and received eight replies. In a call to action, there is an attempt to identify, Owing to the author’s level of influence or following, that locate or crowdsource other details of an individual, post’s OTS was more than 1 million unique users. which could expose them to real-world consequences. As Figure 4 shows, there was an attempt to crowdsource It could also lead to cyber harm, if someone’s personal information on the individual in order to take action information (e.g. ID number or email address) is put against him. This was a clear attempt at doxing. into the public domain. Table 2 gives a breakdown of classification themes. Other posts authored by @AdvoBarryRoux included an embedded video that purported to show a Chinese Figure 5: Digital vigilantism – doxing business owner paying a Zimbabwean national at a building site (presumably designed to fuel the nationalist debate about employment opportunities and foreigners). It was potentially seen by nearly 7 million Twitter users.

Figure 4: Evidence of digital vigilantism 492 8 RETWEETS REPLIES

1 051 303 OTS Almost three-quarters of conversations consisted of xenophobic statements, while blaming foreigners was the second most prominent theme (authors felt supporting foreigners was unpatriotic). Conversations involving a call to action revolved around encouraging South Africans to ‘fight hard for South Africa’ by denying immigrants access. Attempts to find solutions consisted Significantly, when posts were classified, it emerged of anti-xenophobic comments, mirroring a statement by that one out of every 10 posts was a call to action. These Mmusi Maimane. mentions encouraged fellow South Africans to ‘fight hard for South Africa’ by denying immigrants access or by identifying individuals. Just under 4% of engagements Conversations about foreigners sought to neutralise anti-foreigner sentiment.

Our study also found examples of a form of doxing called were characterised by ‘human flesh search engine’ or voluntary crowdsourcing. xenophobic rhetoric with a Its purpose is to identify and expose individuals. Although this occurred in just 1.2% of our sampled nationalist agenda conversation, it is a form of vigilantism that could have real-world consequences. In summary, the South African case study showed that As Figure 5 shows, attempts were made to identify a conversations about foreigners were characterised by Chinese businessman following allegations that he had xenophobic rhetoric with a nationalist agenda. Clear

10 Digital vigilantism, social media and cyber criminality Table 2: One in 10 posts a call to action (%)

Volume Unique authors Engagement Positive Negative

Xenophobia 71.0 68.1 51.6 0.7 97.9

Blames foreigners 13.5 14.3 28.7 0.6 98.3

Call to action 10.5 11.8 10.6 0.0 97.9

Solution seeking 3.8 4.4 8.7 11.8 84.3

Posts identifying information 1.2 1.4 0.5 6.3 50.0

examples of doxing, hounding and voluntary crowd Kenya has a long history of policing with excessive force, sourcing were found. often causing unnecessary deaths. This began under British colonial rule, and has been exacerbated by Influencers were able to dominate conversations poor recruitment policies, corruption and inadequate owing to the low level of engagement (new ideas) from accountability for police actions. other users. Nearly three-quarters (73.9%) of the overall conversation with the hashtag #PutSouthAfricaFirst, In an act of digital vigilantism, Kenyans have thus turned generated by @uLerato Pillay, was a re-share. to Twitter.

In some cases it was embedded in other forms of The most notable theme recently has been the use of content relating to sport, make-up or popular culture, excessive force to contain the spread of COVID-19, with or linked to another prominent user in order to leverage the hashtag #CurfewInKenya or #CurfewKenya. their following and amplify the reach of the original Table 3 shows how Kenyans try to increase police tweet. This shows how influential authors (who are accountability by reporting incidents of alleged brutality usually ordinary people) can drive specific views with on Twitter. The imposition of a curfew drew particular real-world consequences, thus undermining traditional Twitter traffic following a widely reported incident in law enforcement. April in the neighbourhood of Likoni near Mombasa, when police opened fire on ferry commuters, killing Kenya a 13-year-old boy.32 In Kenya our study focused on more than 40 000 tweets from January to May 2020 Table 3: Shaming and police accountability about alleged policy violence associated with attempts to police government Most mentioned words restrictions associated with the COVID-19 #CurfewKenya 9 524 pandemic. It generated over 5 million OTS #CurfewInKenya 7 812 for hashtags and keywords such as #policebrutality, #curfew and #innocent. This demonstrated the kill 7 402 amplification effect of Twitter with respect to key themes kenyans 6 888 of concern. curfew 5 805

However, unlike the South African case study, in the officers 5 563 overwhelming majority of cases Kenyans on Twitter used people 5 454 the digital vigilante technique of collective shaming to seek accountability. brutality 3 976

In other words, Twitter was used as a device to call on brutal 3 830 existing law enforcement mechanisms to take action citizens 3 169 against alleged brutality, rather than for the Twitter teargas 2 884 community to take the law into its own hands.

Research Paper 20 / December 2020 11 The Kenya study revealed instances of hounding or A critical finding of the Kenyan case study concerned doxing of certain police officers whom the Twitter the motivation of Twitter users and the intended targets community blamed for acting violently. Although of their tweets. Our study found that the overwhelming doxing comprised a small portion of the overall majority of conversations were aimed at the police conversation (0.04%), its potency and real-world impact service or police oversight bodies. This is illustrated is potentially large. in Table 4, which shows Twitter users reaching out to official police handles. In most instances the public posted graphic content claiming to show police violence or the victims of such During the period of study (January to May 2020), we violence, without identifying the officers involved. Most found no evidence of official police handles responding of these posts were shared by influential figures such as to complaints raised on Twitter. However, in June 2020 Boniface Mwangi, a former photo journalist and social Hillary N Mutyambai, inspector general of the National activist turned politician. Police Service (tweeting as @IG_NPS), emerged online to ‘encourage all citizens to keep reporting cases of While the police did get some positive press, 93% of police brutality’. conversations were classified as complaints against the police. One prominent figure who sought to calm He told Twitter users to ‘forward videos to us’, suggesting tensions, the Kenyan singer Suzanna Owíyo, was a police tactic to co-opt social media users into the threatened with being cancelled33 – another form of crime fighting project against rogue officers, rather than digital vigilante behaviour – after she posted online. standing in opposition to it.

It also suggests a responsiveness on the part of the police Figure 6: Flagging police behaviour through to being shamed by the public on Twitter, albeit two graphic images months after Kenya’s President Uhuru Kenyatta made a public statement apologising for police brutality.34

76% of all conversations were re-shares of an original tweet As was the case in South Africa, conversations were largely reactive and spread through re-shares, giving a limited pool of authors dominance over the narrative. A total of 76% of all conversations were re-shares of an original tweet.

12 Digital vigilantism, social media and cyber criminality Table 4: Authors reach out to government and official police handles

Top 10 authors Volume mentioned Stakeholder @NPSOfficial_KE 550 Police

@Fredmatiangi 293 Government

@Ipoa_ke 269 Police authority

@Ig_nps 256 Police

@Policeke 201 Police

@Statehousekenya 194 Government

@Dci_kenya 170 Police

@Citizentvkenya 166 Press

@Ntvkenya 128 Press

@Moh_kenya 120 Government

Conclusion Prof Daniel Trottier, a respected scholar in the field of digital vigilantism, offered his assessment of the This empirical analysis of Twitter conversations in Kenya ISS–BrandsEye findings and highlighted that although and South Africa highlights the range of online activities digital vigilantism ‘is more about targeting the individual, that may constitute digital vigilante behaviour. It also inevitably there is always an underlying message about underscores the challenges that social media platforms the collective’.36 may face in understanding the context and nuances of locally specific narratives and calls to action. This is often amplified by the fact that many social media platform The amplification properties controllers are remote, both physically and contextually, from the source of issues on Twitter around which digital of social media platforms vigilantes may coalesce. underscore the potency of Proposed social media legislation in Kenya35 would make it incumbent on platform owners to open an office in digital vigilante activity the territory where the content they host can be viewed. Given the global nature of social media, it is far from Collective punishment for an identified community is clear how this could be achieved in practice. a contentious area of the law. In international settings In both the Kenyan and South African studies, three- it can amount to a war crime under the Geneva 37 quarters of engagements on the two key themes were Conventions. In other settings, e.g. through the use of re-shares. This lack of engagement or new ideas helps radio rather than social media, collective punishment of to amplify the original authored message or hashtag. a particular community has also been identified as a war It also vests considerable power in the original author crime, with Rwanda in 1994 being a case in point.38 to control the narrative. For example, the hashtag While this is beyond the scope of this paper, the #PutSouthAfricaFirst authored by @uLeratoPillay amplification properties of social media platforms such potentially reached 48 million users between April as Twitter underscore the potential potency of digital and May 2020. vigilante activity when directed at a particular target or This positions digital vigilantism as an important area group of targets. This potency has also arguably been for further investigation, given the growing interest in a motivating factor driving governments to consider disinformation campaigns and information operations tighter regulations on social media rather than self- on social media. regulation by the platform owners themselves.

Research Paper 20 / December 2020 13 Trottier also identified from the ISS–BrandsEye study The debate on ‘acceptable norms of behaviour’ on social how ‘expert’ authors who use Twitter to propagate media platforms is a shifting one and is constrained nationalist narratives – such as those behind the by domestic regulatory measures (e.g. hate speech handle @uLeratoPillay – are able to dodge community legislation). However, at the international level, broader guidelines (i.e. the rules governing social media discussions about social media norms and cyber security platforms). They shape their content in such a way that are being held at the UN’s Open Ended Working Group it does not violate the rules, but can still generate its on Developments in the Field of Information and own nationalist momentum. Telecommunications (OEWG), among others.40

He observed that ‘often the person putting forward Members are seeking to develop a normative this call for action doesn’t need to say the loud framework on Internet governance, which is likely to part loudly and can use certain key words so prove challenging given the broad membership of the 41 people know what they are talking about’.39 Code OEWG and the various domestic norms on freedom words are understood by like-minded members of expression versus security. of a community and reinforce a sense of group As in many other spheres of cyber policy, there is belonging and purpose. arguably a strong case for multi-stakeholder engagement in devising the rules governing state behaviour and social While legislation to regulate social media usage is media, given its increasing importance in democratic being discussed in both Kenya and South Africa, processes such as elections, policing, justice. measures requiring active users to register before using social media (in the same way that mobile phone While this study has investigated individual acts of digital users must register their SIM cards) could endanger vigilantism and how conversations are networked, there freedom of expression and undermine fundamental is growing interest globally in the weaponisation of social democratic principles. media platforms by states and the use of proxies, which may include private actors subcontracted by a state.42 It could also enable governments to censure content or place pressure on social media platform controllers Although the ISS–BrandsEye study found no evidence to pre-screen content before it is posted on their sites. of foreign meddling on social media with respect to Social media platform controllers are likely to resist the two touchstone issues studied, the various shades this, as they do not wish to be seen to be policing of digital vigilantism require further investigation, the Internet. In addition, tracing the originators of inlcuding exploring the untested terrain for organised authored accounts can be highly complex. criminal activity.

Notes

1 The Paris beheading case in October 2020 points to a 5 L Bever, Walter Palmer, dentist who hunted and killed Cecil contemporary extreme form of digital vigilantism. S de la the lion, returns to work, The Washington Post, 8 September Hamaide and C Pailliez, Beheaded French teacher was part 2015, https://www.washingtonpost.com/news/morning-mix/ of angry social media campaign, Reuters, 17 October 2020, wp/2015/09/08/walter-palmer-dentist-who-hunted-and- https://www.reuters.com/article/uk-france-security-teacher- killed-cecil-the-lion-returns-to-work/ idUKKBN2720S1 6 E Lipton, Man motivated by ‘Pizzagate’ conspiracy 2 Slideshare, Digital 2019 Kenya, https://www.slideshare.net/ theory arrested in Washington gunfire,New York Times, DataReportal/digital-2019-kenya-january-2019-v01 5 December 2016, https://www.nytimes.com/2016/12/05/ us/pizzagate-comet-ping-pong-edgar-maddison-welch. 3 If they are considered publishers the implication is that html?searchResultPosition=7 they should be rule bound and tightly regulated. See also J Jarvis, Platforms are not publishers, The Atlantic, 10 August 7 See R Gabdulhakov, Citizen-led justice in post-Communist 2018, https://www.theatlantic.com/ideas/archive/2018/08/ Russia: from comrade courts to dotComrade vigilantism, the-messy-democratizing-beauty-of-the-internet/567194/ Erasmus University, Rotterdam. 4 B Lovelock, The many shades of digital vigilantism: 8 J Morgan and R DiResta, Information operations are a a typology of online justice, Global Crime, 2019, cybersecurity problem: toward a new strategic paradigm to https://doi.org/10.1080/17440572.2019.1614444 combat disinformation, Just Security, 10 July 2018.

14 Digital vigilantism, social media and cyber criminality 9 R MacKinnon et al., Fostering freedom online: the role of Conversation, 29 May 2018, https://theconversation.com/ Internet intermediaries, Center for Global Communications kenyas-new-cybercrime-law-opens-the-door-to-privacy- Studies, 2012, https://repository.upenn.edu/cgcs_ violations-censorship-97271 publications/21/ 28 Reuters, Kenya’s president signs cybercrimes law opposed 10 B Lovelock, The many shades of digital vigilantism: by media rights groups, 16 May 2018, https://www.reuters. A typology of online self-justice, Global Crime, 2019, com/article/us-kenya-lawmaking-idUSKCN1IH1KX 11 https://www.news24.com/citypress/News/penny- 29 K Allen, Why is the US ramping up anti-terrorism efforts in sparrow-back-in-court-on-criminal-charges-for-racist- Kenya?, ISS Today, 26 March 2020, https://issafrica.org/iss- comments-20160912 today/why-is-the-us-ramping-up-anti-terrorism-efforts-in- kenya 12 For a discussion on the cyber harms associated with doxing see Center for Internet Security, EI-ISAC cybersecurity 30 Kenya Gazette Supplement, The Data Protection Act spotlight: doxing, https://www.cisecurity.org/spotlight/ei- 2019, 11 November 2019, http://kenyalaw.org/kl/fileadmin/ isac-cybersecurity-spotlight-doxing/ pdfdownloads/Acts/2019/TheDataProtectionAct__ No24of2019.pdf 13 Interview, Dr Brett van Niekerk, Senior Lecturer in Cybersecurity at University of KwaZulu-Natal, August 2020. 31 For a broad discussion see D Mwathe and J Syekei, Highlights on proposed law introducing strict regulation 14 B Lovelock, The many shades of digital vigilantism: of social media, Bowmans, 16 October 2019, https://www. A typology of online self-justice, Global Crime, 2019, 15, bowmanslaw.com/insights/intellectual-property/highlights- https://doi.org/10.1080/17440572.2019.1614444 on-proposed-law-introducing-strict-regulation-of-social- 15 D Monahan, Digital shaming for social justice and social media/ harm, Surveillance and Society, 17 September 2018, , Kenya to probe fatal police shooting of https://medium.com/surveillance-and-society/digital- 32 Al Jazeera teenager, 1 April 2020, https://www.aljazeera.com/ shaming-for-social-justice-and-social-harm-c5b3e6c56572 videos/2020/04/01/kenya-to-probe-fatal-police-shooting-of- 16 D Trottier, Coming to terms with shame: exploring teenager/ mediated visibility against transgressions, Surveillance 33 For a discussion on cancel culture see D Brown, Twitter’s and Society, 2018, https://ojs.library.queensu.ca/index.php/ surveillance-and-society/article/view/6811/7683 cancel culture: a force for good or a digital witch hunt? The answer is complicated, USA Today, 17 July 2020, 17 Ibid. https://www.usatoday.com/story/tech/2020/07/17/has- 18 K Silva, Vigilantism and cooperative criminal justice: is there twitters-cancel-culture-gone-too-far/5445804002/ a place for cybersecurity vigilantes in cybercrime fighting?, 34 Al Jazeera, Kenyan president apologises for police violence International Review of Law, Computers & Technology, 32:1, during curfew, 1 June 2020, https://www.aljazeera.com/ 2018, https://www.tandfonline.com/doi/full/10.1080/1360086 videos/2020/04/01/kenya-to-probe-fatal-police-shooting-of- 9.2018.1418142 teenager/ 19 KV Legocki, L Walker and T Kiesler, Sound and fury: digital 35 KenyaLaw, Kenya Communications and vigilantism as a form of consumer voice, Journal of Public Information (Amendment) Bill 2019, http:// Policy and Marketing, 29, 2020, https://journals.sagepub. kenyalaw.org/kl/fileadmin/pdfdownloads/bills/2019/ com/doi/10.1177/0743915620902403 TheKenyaInformationandCommunication__Amendment_ 20 Ibid. Bill_2019_NA_Bills_No._61.pdf 21 R Davis, EFF court losses mount as Karima Brown wins court 36 Interview, Prof. Daniel Trottier, August 2020. battle but faces criticism of her own, Daily Maverick, 6 June 37 ‘No general penalty, pecuniary or otherwise, shall be 2019, https://www.dailymaverick.co.za/article/2019-06-06- inflicted upon the population on account of the acts of eff-court-losses-mount-as-karima-brown-wins-battle-but- individuals for which they cannot be regarded as jointly faces-criticism-of-her-own/ and severally responsible.’ 22 Interview, Karima Brown, September 2019. 38 For further discussion on this see Genocide Archive of 23 BBC News, Social media: How do other governments Rwanda, Radio Télévision Libre des Mille Collines, regulate it?, 12 February 2020, https://www.bbc.com/news/ https://genocidearchiverwanda.org.rw/index.php/Radio_ technology-47135058 Télévision_Libre_des_Mille_Collines 24 Republic of South Africa, Cybercrimes Bill, https://static. 39 Interview with Prof Daniel Trottier by ISS August 2020 pmg.org.za/NA_bills2017_bill06B-2017.pdf 40 DWO Geneva Internet Platform, Open-Ended Working 25 P de Vos, Act’s hate speech provision hard to understand, Group (OEWG) – first substantive meeting, https://dig. says Supreme Court of Appeal, Daily Maverick, 3 December watch/events/open-ended-working-group-oewg-first- 2019, https://www.dailymaverick.co.za/opinionista/2019-12- substantive-session 03-acts-hate-speech-provision-hard-to-understand-says- 41 United Nations Office for Disarmament Affairs, Open- supreme-court-of-appeal/ Ended Working Group, https://www.un.org/disarmament/ 26 For a discussion on definitions see ForSA, Hate speech: open-ended-working-group/ where do we stand?, 20 April 2020, https://forsa.org.za/hate- 42 T Hatmaker, Russian trolls are outsourcing to Africa to speech-where-do-we-stand/ stoke US racial tensions, TechCrunch, 13 March 2020, 27 For a discussion see M Muendo, Kenya’s new cybercrime https://techcrunch.com/2020/03/12/twitter-facebook- law opens the door to privacy violations, censorship, The disinformation-africa-ghana-nigeria-ira-russia/

Research Paper 20 / December 2020 15 This project is funded by the European Union

About the authors

Karen Allen is a Senior Research Advisor on Emerging Threats in Africa. She holds a Master’s degree in International Relations and Contemporary War from King’s College London and is a Visiting Fellow at the same institution. She was previously a BBC foreign correspondent working across East and Southern Africa and Afghanistan.

Isel van Zyl, Research Officer in the Complex Threats in Africa programme, holds a Master’s degree in Advanced European and International Studies from the Centre international de formation européenne (CIFE) in Nice, France.

About ENACT

ENACT builds knowledge and skills to enhance Africa’s response to transnational organised crime. ENACT analyses how organised crime affects stability, governance, the rule of law and development in Africa, and works to mitigate its impact. ENACT is implemented by the ISS and INTERPOL, in affiliation with the Global Initiative Against Transnational Organized Crime.

Acknowledgements

ENACT is funded by the European Union (EU). This publication has been produced with the assistance of the EU.

Cover image © Adobe Stock – Julien Eichinger. The contents of this publication are the sole responsibility of the author and can in no way be taken to reflect the views or position of the European Union, or the ENACT partnership. Authors contribute to ENACT publications in their personal capacity. © 2020, ENACT. Copyright in the volume as a whole is vested in ENACT, its partners, the EU and the author, and no part may be reproduced in whole or in part without the express permission, in writing, of the author and the ENACT partnership.