The Emerging Risk of Virtual Societal Warfare Social Manipulation in a Changing Information Environment
Total Page:16
File Type:pdf, Size:1020Kb
The Emerging Risk of Virtual Societal Warfare Social Manipulation in a Changing Information Environment Michael J. Mazarr, Ryan Michael Bauer, Abigail Casey, Sarah Anita Heintz, Luke J. Matthews C O R P O R A T I O N For more information on this publication, visit www.rand.org/t/RR2714 Library of Congress Cataloging-in-Publication Data is available for this publication. ISBN: 978-1-9774-0272-1 Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2019 RAND Corporation R® is a registered trademark. Cover: JohnDWilliams/iStock/Getty Images Plus. Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Preface This analysis is part of a larger study on techniques of social manipula- tion and was motivated by recent Russian efforts to manipulate West- ern information environments. This study focuses on the future of social manipulation efforts and involved a survey of multiple, overlap- ping information-related technologies and their potential for manipu- lation. It describes the emerging phenomenon of virtual societal warfare and suggests avenues for Western democracies to respond. The research was sponsored by the Office of Net Assessment, Office of the Secretary of Defense, and conducted within the Inter- national Security and Defense Policy Center of the RAND National Defense Research Institute, a federally funded research and develop- ment center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community. For more information on the RAND International Security and Defense Policy Center, see https://www.rand.org/nsrd/ndri/centers/ isdp.html or contact the director (contact information is provided on the webpage). iii Contents Preface ............................................................................. iii Table ................................................................................ ix Summary .......................................................................... xi Acknowledgments .............................................................. xix Abbreviations ................................................................... xxi CHAPTER ONE A New Form of Conflict ......................................................... 1 Definitions and Concepts ......................................................... 5 Approach and Methodology .....................................................10 CHAPTER TWO The Evolving Infosphere .......................................................13 Broader Social Trends ............................................................14 Trends in the Infosphere: Knowledge and Awareness ........................18 Networked Dynamics and the Viral Spread of Information .................21 Sensationalism .................................................................... 23 Fragmentation of Information Sources .........................................25 Concentration of Information Platforms ...................................... 28 The Role of Echo Chambers .....................................................29 The Role of Influencers ..........................................................35 The Trolling Ethic ............................................................... 36 The Rule of Data ................................................................. 38 Summary: A Changing Information Environment for Advanced Democracies ................................................................. 40 v vi The Emerging Risk of Virtual Societal Warfare CHAPTER THREE Insights from Social Science .................................................. 43 Attitudes and Attitude Change ..................................................45 When Attitudes Change: Criteria for Attitude Shift ........................ 48 Correcting Disinformation: Beyond the Backfire Effect .....................55 Factors Influencing Social Trust ................................................59 The Nature of Attitudes and the Shifting Infosphere ........................62 CHAPTER FOUR Emerging Technologies .........................................................65 Precision Targeting of Influence ................................................67 Artificial Intelligence ............................................................ 68 Algorithmic Decisionmaking ....................................................72 Virtual and Augmented Reality .................................................75 Internet of Things ................................................................ 77 Voice-Enabled Interfaces ........................................................ 80 Blockchain and Distributed Ledger Systems ..................................83 Video and Audio Fakery ........................................................ 84 Surveillance Systems ............................................................. 92 Conclusion: A Rapidly Changing Infosphere ................................. 94 CHAPTER FIVE Future 1: The Death of Reality (2025 Scenario) .......................... 97 A Note About Methodology .................................................... 97 The General Future .............................................................. 99 A Future of Virtual and Invented Reality .................................... 104 The End of a Shared Picture of the World ................................... 111 Information Aggression and Manipulation in the Death of Reality ...... 112 CHAPTER SIX Future 2: Silos of Belief (2024 Scenario) .................................. 117 The General Future ............................................................. 117 A Fragmented Society .......................................................... 119 The Wars of the Silos ........................................................... 122 A Burgeoning Landscape of Cyberharassment .............................. 124 Information-Based Aggression and the Silo Future ......................... 128 Contents vii CHAPTER SEVEN Future 3: The Rise of the Algorithms (2026 Scenario) ................. 131 The General Future ............................................................. 131 The Cloud of Knowing ......................................................... 132 Interactive Data ................................................................. 136 The Cloud Knows Where You Have Been—and What You Think ...... 140 Ruled by Algorithm: Surrendering to the POPE ............................ 142 Vulnerabilities and Risks ....................................................... 147 Information Aggression in an Algorithmic Future .......................... 149 CHAPTER EIGHT The Emerging Risk of Virtual Societal Warfare ......................... 153 A New Form of Conflict: Virtual Societal Warfare ........................ 155 Dealing with the Threat of Virtual Societal Aggression ................... 161 Designing a Response: An Initial Agenda.................................... 163 References ....................................................................... 171 Table 4.1. Emerging Technologies of Social Manipulation and Virtual Societal Aggression ........................................ 66 ix Summary The year 2016 and beyond saw an explosion of interest in issues of disinformation, propaganda, information manipulation and fakery, “fake news,” “Truth Decay,” and related trends—a broad phenomenon that can be termed hostile social manipulation. In this study, we define this concept as the purposeful, systematic generation and dissemination of information to produce harmful social, political, and economic outcomes in a target country by affecting beliefs, attitudes, and behavior. Examples of this rising challenge include Russian efforts to influence elections and sow discord in the West through propaganda and disinformation; the role of social media platforms such as Facebook in spreading such misinformation; and burgeoning Chinese programs to shape regional narratives and gain political leverage in specific countries. U.S. intelli- gence services have concluded that Russia employed such techniques to influence the 2016 election, and Moscow continues to employ them— sometimes brazenly despite U.S. warnings—in the United States and Europe. As significant as these developments have been, they may only represent the beginning of what an aggressive nation can accomplish with techniques and technologies designed to disrupt and shape the information environment of a target country. This report’s primary conclusion