Counter-Radicalization Bot Research: Using Social Bots to Fight Violent
Total Page:16
File Type:pdf, Size:1020Kb
C O R P O R A T I O N WILLIAM MARCELLINO, MADELINE MAGNUSON, ANNE STICKELLS, BENJAMIN BOUDREAUX, TODD C. HELMUS, EDWARD GEIST, ZEV WINKELMAN Counter-Radicalization Bot Research Using Social Bots to Fight Violent Extremism For more information on this publication, visit www.rand.org/t/RR2705 Library of Congress Cataloging-in-Publication Data is available for this publication. ISBN: 978-1-9774-0394-0 Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2020 RAND Corporation R® is a registered trademark. Cover: monsitj/Adobe Stock. Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Preface Given the success violent extremist groups have had online— recruiting, funding, and messaging—the U.S. government (USG) has an interest in effective, agile, and scalable online responses. This report examines the applicability of automated social media (SM) accounts, known as bots, to address this problem. While this report was primarily directed at countering groups like the Islamic State of Iraq and the Levant (ISIL), the findings are also applicable to the growing threat of adver- sary state-sponsored SM information operations. Readers will find an overview of bot technology, a discussion of legal and ethical consider- ations around bot deployment, a framework for assessing risk/reward in bot operations, and recommendations for the USG in developing and deploying such bot programs. The research reported here was completed in August 2018 and underwent security review with the sponsor and the Defense Office of Prepublication and Security Review before public release. This research was sponsored by the U.S. Department of State and the Combating Terrorism Technical Support Office and conducted within the International Security and Defense Policy Center of the RAND National Security Research Division (NSRD), which operates the National Defense Research Institute (NDRI), a federally funded research and development center sponsored by the Office of the Secre- tary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense intelligence enterprise. iii iv Counter-Radicalization Bot Research For more information on the RAND International Security and Defense Policy Center, see www.rand.org/nsrd/ndri/centers/isdp or contact the director (contact information is provided on the webpage). Contents Preface ............................................................................. iii Figures and Tables ...............................................................vii Summary .......................................................................... ix Acknowledgments .............................................................. xix Abbreviations .................................................................... xxi CHAPTER ONE Social Chatbots: An Introduction ............................................. 1 An Introduction to Bots ........................................................... 3 Bot Technology Review ........................................................... 4 Implementing Bots on SM Platforms ............................................ 5 Bot Types ...........................................................................10 Continuing Challenges ...........................................................19 Bot Overview Summary ........................................................ 24 CHAPTER TWO Current Status of Bot Technology ........................................... 27 Case Studies ...................................................................... 27 Maturity Model .................................................................. 50 Conclusions ....................................................................... 54 CHAPTER THREE Potential Legal and Ethical Risks ............................................57 Legal Considerations .............................................................58 Ethical Considerations ...........................................................67 v vi Counter-Radicalization Bot Research CHAPTER FOUR Concepts of Operation and Assessment .....................................73 Assessment Criteria and Levels ..................................................74 Bot Variables .......................................................................75 Categories of Social Bot Operations ............................................91 An Example CONOP Assessment ............................................ 107 CHAPTER FIVE Recommendations ............................................................. 109 Development and Deployment ................................................ 111 Legal and Ethical Issues ........................................................ 114 Conclusions ...................................................................... 117 APPENDIX A. Technology Review: Methods and Goals .............................. 119 References ....................................................................... 121 Figures and Tables Figures 3.1. Transparency Spectrum .............................................70 Tables S.1. Bot Terminology ...................................................... x S.2. Bot Types ............................................................. xi S.3. Concepts of Action: Influence and Inform ...................... xiv S.4. Concepts of Action: Degrade/Disrupt Violent Extremist Networks ............................................................. xiv S.5. Concepts of Action: Collect Intelligence ..........................xv 1.1. Bot Types .............................................................11 1.2. Four Requirements for Adaptive Tactical Countermessaging Bots .............................................21 2.1. Sensing Capabilities .................................................51 2.2. Deciding Capabilities ...............................................51 2.3. Action Capabilities ..................................................53 3.1. Terms of Service for Select Social Media Platforms or Messaging Services ...............................................69 4.1. Assessment Levels ....................................................74 4.2. Variable Assessment: Deployer .....................................76 4.3. Variable Assessment: Audience .....................................78 4.4. Variable Assessment: Platform......................................81 4.5. Variable Assessment: Communication Strategy ..................81 4.6. Variable Assessment: Activation ....................................83 4.7. Variable Assessment: Automation ..................................85 4.8. Variable Assessment: Reliance on Artificial Intelligence and Deep Learning ................................................. 86 vii viii Counter-Radicalization Bot Research 4.9. Variable Assessment: Data Retention ............................. 87 4.10. Variable Assessment: Volume ...................................... 88 4.11. Variable Assessment: “Human” Disguise .........................89 4.12. Variable Assessment: Attribution ................................. 90 4.13. Concepts of Action: Influence and Inform ...................... 92 4.14. Concepts of Action: Degrade/Disrupt Violent Extremist Networks ............................................................. 98 4.15. Concepts of Action: Collect Intelligence ........................ 104 4.16. Bot CONOP: Matchmaker ...................................... 108 Summary The speed and diffusion of online recruitment for violent extremist organizations (VEOs) such as the Islamic State of Iraq and the Levant (ISIL) have challenged existing efforts to effectively intervene and engage in counter-radicalization in the digital space. This problem contributes to global instability and violence. Groups like ISIL identify susceptible individuals through open social media (SM) dialogue and eventually seek private conversations online and offline for recruiting. This shift from open and discoverable online dialogue to private and discreet recruitment can happen quickly and offers a short window for intervention before the conversation and the targeted individuals disappear. The counter-radicalization messaging enterprise of the U.S. gov- ernment (USG) may benefit from a sophisticated capability to rap- idly detect targets of VEO recruitment efforts and deliver counter- radicalization content to them. Our report examines the applicability of promising emerging technology tools, particularly automated SM accounts known as bots, to this problem. While this