Understanding Linkages Between Online Harmful Speech Practices And Offline Violent Action 2020 This work is licensed under a creative commons Attribution 4.0 International License. You can modify and build upon this document non-commercially, as long as you give credit to the original authors and license your new creation under the identical terms. Suggested citation: Content Policy Research Group. (2020). Understanding linkages between online harmful speech practices and offline violent action. Digital Empowerment Foundation: New Delhi. You can read the online copy under Reports Section at https://www.defindia.org/publications/ Digital Empowerment Foundation House no. 44, 2nd and 3rd Floor (next to Naraina IIT Academy) Kalu Sarai (near IIT Flyover) New Delhi – 110016 Tel: 91-11-42233100 / Fax: 91-11-26532787 Email: [email protected] | URL: www.defindia.org Understanding Linkages Between Online Harmful Speech Practices And Offline Violent Action 2 CONTENTS Situating the Research – The Indian Context 04 Regulation of Hate Speech and its Discontents 06 Understanding the Online Practice of Hate Speech 08 Methodology 11 Modalities of Speech Practices 12 Framework for Social Processes of Direct Action 23 Application of Framework to International Incidents 31 Evaluating Facebook’s Content Moderation Policy 50 Recommendations 61 FUNDING DISCLAIMER In 2019, the Digital Empowerment Foundation was one of the recipients of Facebook Content Policy Research Awards to understand the linkages between hate speech and offline violence in India. Understanding Linkages Between Online Harmful Speech Practices And Offline Violent Action 3 SITUATING THE RESEARCH – THE INDIAN CONTEXT In 2020 over 50% of the 1.3 billion Indian population In tandem with the mob violence around child- has access to social media; this is up from 19% in lifting rumours, there was a similar peak of violence 2015, 30% in 2017, and 46% in 20191. This rapid associated with cow-protection vigilantism with social media penetration is said to be the product 2017 being the worst year with highest number of cheap mobile data and increased smartphone of casualties7. The April 2020 lynching of Hindu penetration2. The average Indian social media user priests in Palghar, Maharashtra and Bulandshahr, spends more than 17 hours on the platforms each Uttar Pradesh on rumours of thieving provided week which is more than social media users in China a grim reminder of India’s recent history with and United States3. misinformation and offline violence8. The unfolding of the COVID – 19 pandemic and its impact on However, along with this digital transformation – social and economic life has further served to calcify over the past few years, India has seen the alarming identity-based divisions in the country9. rise in violence that have been linked to social media. Affordances provided by social media in the form of The implication of the private messaging service distributed democratic and civic participation has also towards the perpetration of localized and even large provided the fertile ground for online vitriol, social scale civic violence led to government cognizance of panic, and targeted campaigns of intimidation and its role. In the wake of the incidents of lynching, the harassment. Ministry of Electronics and Information Technology (MEITY) wrote to WhatsApp to “take immediate The years 2017 and 2018 saw an alarming rise in steps to tackle the menace of misuse of their platform mob violence pan-India fueled by rumours circulated wherein inflammatory messages were circulated 4 via social media related to child-lifting, organ- that led to unfortunate incidents”10. In subsequent 5 harvesting, and cow slaughter . With regard to child notices, the government stated the requirements to – lifting rumours, despite the repeating trends across institute traceability on the platform while have been the vast expanse of the country they were primarily the subject of multiple petitions concerning related driven around possible presence of outsiders in the primary and ancillary issues11. community who were touted as child lifters. This is despite there being no instances of kidnappings being reported or suspected in the lynching spots in the previous 3 months6. killings, one year: And a pattern to the lynchings. Retrieved from https://indianexpress.com/article/india/murderous-mob-lynching- incidents-in-india-dhule-whatsapp-rumour-5247741/ [4 May 2020]. 1 Statista. (July 2020). India social network penetration 2015- 7Saldanha, A. (2017). 2017 Deadliest Year For Cow-Related Hate 2025. Retrieved from https://www.statista.com/statistics/240960/ Crime Since 2010, 86% Of Those Killed Muslim. IndiaSpend. share-of-indian-population-using-social-networks/ [25 October https://archive.indiaspend.com/cover-story/2017-deadliest-year- 2020]. for-cow-related-hate-crime-since-2010-86-of-those-killed-mus- 2Venkatramakrishnan, R. (30 September 2018). India wants lim-12662. [4 May 2020] WhatsApp to break encryption and trace inflammatory messag- 8PTI. (2020). Three More Cops Suspended in Connection With es. Should it?. Scroll.in. Retrieved from https://scroll.in/arti- Mob Lynching of Sadhus in Palghar. News18. Retrieved from cle/895645/india-wants-whatsapp-to-break-encryption-and-trace- https://www.news18.com/news/india/three-more-cops-suspended- inflammatory-messages-should-it [25 October 2020]. in-connection-with-mob-lynching-of-sadhus-in-palghar-2598035. 3McKinsey Global Institute. (2019). Digital India: Technology html [4 May 2020]; Sharma, S. (2020). 2 sadhus killed in- to transform a connected nation. Retrieved from https://www. side Bulandshahr temple in UP, accused arrested. IndiaToday. mckinsey.com/~/media/McKinsey/Business%20Functions/McK- Retrieved from https://www.indiatoday.in/crime/story/2-sad- insey%20Digital/Our%20Insights/Digital%20India%20Technolo- hus-killed-inside-bulandshahr-temple-in-up-accused-arrest- gy%20to%20transform%20a%20connected%20nation/MGI-Digi- ed-1671948-2020-04-28 [4 May 2020]. tal-India-Report-April-2019.pdf [25 October 2020]. 9Nizamuddin, F. (2020). The COVID-19 pandemic and the infra- 4See Saldanha, A., Rajput, P., & Hazare, J. (2018). Child-Lifting structure of hate in India. Rosa Luxemburg Stiftung. Retrieved Rumours: 33 Killed In 69 Mob Attacks Since Jan 2017. Before from http://www.irgac.org/2020/07/22/the-covid-19-pandemic-and- That Only 1 Attack In 2012. IndiaSpend. Retrieved from https:// the-infrastructure-of-hate-in-india/ [25 October 2020]. www.indiaspend.com/child-lifting-rumours-33-killed-in-69-mob- 10PIB. (20 July 2018). WhatsApp told to find more effective solu- attacks-since-jan-2017-before-that-only-1-attack-in-2012-2012/ [4 tions. Retrieved from https://pib.gov.in/PressReleaseIframePage. May 2020] aspx?PRID=1539410 [25 October 2020]. 5 Madrigal, A.C. (25 September 2018). India’s lynching epidem- 11Agarwal, A. (31 January 2020). Supreme Court directs Madras ic and the problem with blaming tech. The Atlantic. Retrieved HC to transfer all files in the WhatsApp traceability case. Mediana- from https://www.theatlantic.com/technology/archive/2018/09/ ma. Retrieved from https://www.medianama.com/2020/01/223-su- whatsapp/571276/ [25 October 2020]. preme-court-to-madras-hc-transfer-all-files-in-whatsapp-traceabili- 6IndianExpress. (15 July 2018). Murderous mob — 9 states, 27 ty-case/ [25 October 2020]. Understanding Linkages Between Online Harmful Speech Practices And Offline Violent Action 4 In response to the rising concerns, WhatsApp The Delhi Assembly Committee on Peace and limited the number of forwards and appointed a Harmony issued summons to Facebook over its grievance officer for India after the government and alleged “deliberate and intentional” inaction to the Supreme Court demanded that it improve its contain hateful content during the riots that ravaged approach to safety12. In a wider regulatory move, North-East Delhi in February 2020 and left over 50 MEITY released the draft Intermediary Guidelines people dead18. On 31 August 2020, the Committee (Amendment) Rules, 2018 under Section 79 of the said that it seemed prima facie that Facebook has Information Technology Act, 2000. The Rules were a role in the riots and should be treated as a co- targeted at expanding the liability of social media accused19. intermediaries for content hosted on their platforms13. Facebook India’s Vice-President and Managing The 2018 Rules mandated a 72-hour window Director Ajit Mohan refused to appear before the of traceability on receiving complaint from law Delhi Assembly Committee citing two primary enforcement; disable access within 24-hours to reasons: (i) regulation of intermediaries like Facebook content deemed defamatory, against national falls within the purview of the Central Government; security, or in violation of Article 19(2) of the (ii) the subject of law and order in Delhi also falls Indian Constitution; requirement for all platforms within the purview of the Central Government20. with more than 5 million users to have a registered Facebook further filed a plea before the Supreme office in India14. It also included the deployment of Court in connection to the summons which then automated tools to detect and take down ‘unlawful’ asked the Delhi committee to halt any coercive action content15. In January 2020, it was reported that the against Mohan till 15 October 202021. government plans to change the proposed rules such that monitoring and take down measures apply only to big social media companies16. Despite the centrality of WhatsApp within India’s content policy debate and its implication within adverse social phenomenon – more recently Facebook’s murky history within the Myanmar and Sri Lanka India violence echoed within the context of the North – East Delhi riots. India is the biggest market for Facebook - it has 260 million active users in India which is the highest they have in any country in the world with 52% of Indians using Facebook as a source of news17. 12Venkatramakrishnan, R. (30 September 2018). India wants WhatsApp to break encryption and trace inflammatory messag- es. Should it?. Scroll.in. Retrieved from https://scroll.in/arti- cle/895645/india-wants-whatsapp-to-break-encryption-and-trace- inflammatory-messages-should-it [25 October 2020].