credo of “move fast, break things?” break fast, “move of credo without written consent. SR-2097 consent. written without
reserved. Reproduction is prohibited prohibited is Reproduction reserved.
compete with a business that espouses a a espouses that business a with compete
© 2019 Institute for the future. All rights rights All future. the for Institute 2019 © greater distrust? greater grapple with in our effort to combat disinformation. combat to effort our in with grapple words, how does an ethical business business ethical an does how words,
media tools without engendering engendering without tools media trends driving compromised cognitive immunity, as well as the difficult choices we must must we choices difficult the as well as immunity, cognitive compromised driving trends mitigate adverse consequences? In other other In consequences? adverse mitigate
and technology of workings the to improve our defenses. Then, flip the map over to gain a deeper understanding of the the of understanding deeper a gain to over map the flip Then, defenses. our improve to resources to develop mechanisms that that mechanisms develop to resources
in people educate we do How
bad actors use to exploit our cognitive biases. The map also reveals seven strategies strategies seven reveals also map The biases. cognitive our exploit to use actors bad competitive, while also taking time and and time taking also while competitive,
Media Manipulation Media nd the attack methods methods attack the nd a system immune cognitive the of overview visual a for side map
How can a media business stay stay business media a can How
www.iftf.org
650.854.6322
vs. Literacy Media in the battle between disinformation tactics and healthy immune responses. Start with the the with Start responses. immune healthy and tactics disinformation between battle the in
Innovation vs. Deliberation vs. Innovation
94301 CA Alto, Palo
This map presents the dilemmas, drivers, and future forces that will play important roles roles important play will that forces future and drivers, dilemmas, the presents map This
essential to a healthy society? healthy a to essential Avenue Hamilton 201
HOW TO USE THIS MAP MAP THIS USE TO HOW while maintaining shared worldviews worldviews shared maintaining while magazine? a or newspaper a
How do we maintain healthy diversity diversity healthy maintain we do How like publisher, a it is extent what to and
phone company or an email service, service, email an or company phone Cognitive Diversity Cognitive
a platform a common carrier, like a a like carrier, common a platform a
vs. Cohesion Cognitive
individual and community levels. community and individual
its users publish? To what extent is is extent what To publish? users its
system, suited to today’s realities. We need to uprade our “cognitive immune systems” at at systems” immune “cognitive our uprade to need We realities. today’s to suited system,
media platform bear for the content content the for bear platform media
To preserve democracy, we need to develop new immune mechanisms, a new immune immune new a mechanisms, immune new develop to need we democracy, preserve To advertisers? How much responsibility does a social social a does responsibility much How
put their users above sponsors and and sponsors above users their put
Platforms vs. Publishers vs. Platforms
respond to the rapid changes in our information environment. environment. information our in changes rapid the to respond in the disinformation age disinformation the in investment in media platforms that that platforms media in investment
transformation, with old structures, regulatory systems, and cultural norms straining to to straining norms cultural and systems, regulatory structures, old with transformation,
How do we encourage capital capital encourage we do How
and behaviors of billions of their users. And yet, we still are at the early stages of this this of stages early the at are still we yet, And users. their of billions of behaviors and maximization in our media ecosystem? ecosystem? media our in maximization lines? the draw democracy defending
of dollars. Social media platforms have unprecedented insight into the desires, fears, fears, desires, the into insight unprecedented have platforms media Social dollars. of
expression? Who decides where to to where decides Who expression? How do we move beyond profit profit beyond move we do How
cameras, a staff, transmitting equipment, and an FCC license, all all license, FCC an and equipment, transmitting staff, a cameras, of which totaled millions millions totaled which of
What are the boundaries of free free of boundaries the are What
Social Returns Social
account has a greater potential audience than a 1990s television station with a newsroom, newsroom, a with station television 1990s a than audience potential greater a has account
IMMUNE SYSTEM IMMUNE
Free Speech vs. Censorship vs. Speech Free vs. Returns Economic
An individual with a $500 laptop, a $30 a month Internet service plan, and a YouTube YouTube a and plan, service Internet month a $30 a laptop, $500 a with individual An
power to shape our dominant narratives, and the power to influence influence to power the and narratives, dominant our shape to power our cognition. cognition. our
and learn to manage, in the process balancing difficult choices. difficult balancing process the in manage, to learn and COGNITIVE have altered existing power dynamics by re-defining who has the power of voice, the the voice, of power the has who re-defining by dynamics power existing altered have
these dilemmas. Like parasites in an ecosystem, dilemmas are things we have to live with with live to have we things are dilemmas ecosystem, an in parasites Like dilemmas. these ever before. For better and worse, networked communications tools and technologies technologies and tools communications networked worse, and better For before. ever
organization based in Palo Alto, California. Alto, Palo in based organization
to protect. We must accept that there are challenges we cannot find solutions to. We call call We to. solutions find cannot we challenges are there that accept must We protect. to world, however, attacks against democracy have the potential to be more devastating than than devastating more be to potential the have democracy against attacks however, world,
is a registered 501(c)(3) nonprofit nonprofit 501(c)(3) registered a is Future
over-react, a cognitive immune system can sometimes damage the system it’s designed designed it’s system the damage sometimes can system immune cognitive a over-react,
a democratic society we tolerate and even invite many voices. In voices. many invite even and tolerate we society democratic a today’s highly networked networked highly today’s HEALTHY A BUILDING
Institute for the the for Institute future. sustainable more
out disinformation. In the same way that a biological immune syst immune biological a that way same the In disinformation. out em can backfire or or backfire can em manipulation. Many are attempting to persuade others in many different directions, and in in and directions, different many in others persuade to attempting are Many manipulation.
across all sectors that together support a a support together that sectors all across
It is important to note that a healthy cognitive immune system isn’ system immune cognitive healthy a that note to important is It t a cure-all for wiping wiping for cure-all a t
The cognition of individuals and of the larger body politic has always been a target for for target a been always has politic body larger the of and individuals of cognition The
coherent views of transformative possibilities possibilities transformative of views coherent
IFTF methodologies and toolsets yield yield toolsets and methodologies IFTF
dilemmas is ill suited to ward off these new attackers, which include bots, trolls, and much more. much and trolls, bots, include which attackers, new these off ward to suited ill is
change and develop world-ready strategies. strategies. world-ready develop and change
we’ve built over decades decades over built we’ve defenses the And manipulation. of forms new many and attacks
navigate complex complex navigate to training foresight and
Cognitive immune system system immune Cognitive our individual and collective cognitive immune systems, opening opening systems, immune cognitive collective and individual our us up to disinformation disinformation to up us
forecasts, custom research, research, custom forecasts, global IFTF upon
Dramatic changes in our media and information technology landscapes landscapes technology information and media our in changes Dramatic are weakening weakening are
depended depended have organizations impact social
The metaphor is particularly apt in describing the state of our body body our of state the describing in apt particularly is metaphor The politic today. today. politic
over 50 years, businesses, governments, and and governments, businesses, years, 50 over
organization. For For organization. thinking futures leading
erode, or help sustain, democracy. democracy. sustain, help or erode, mechanisms are no longer effective, allowing parasites to thrive and weaken the host. host. the weaken and thrive to parasites allowing effective, longer no are mechanisms
the world’s world’s the is Future the for Institute
must understand the forces—technological, cultural, economic—on the horizon that could could that horizon the economic—on cultural, forces—technological, the understand must mechanisms for warding off such attackers. But under conditions of change, these these change, of conditions under But attackers. such off warding for mechanisms
Building a cognitive immune system will require a systemic view of our networked society. We We society. networked our of view systemic a require will system immune cognitive a Building ABOUT INSTITUTE FOR THE FUTURE THE FOR INSTITUTE ABOUT healthy functioning of the host. Under static conditions, these syst these conditions, static Under host. the of functioning healthy ems have established established have ems
system to use as entry points to grow and propagate, in the process undermining the the undermining process the in propagate, and grow to points entry as use to system
against malicious or unwanted intrusions that threaten the body politic. body the threaten that intrusions unwanted or malicious against parasites from attack constant under —invaders on the lookout for weaknesses in the the in weaknesses for lookout the on —invaders
combination to form a socio-technical infrastructure that protects our networked society society networked our protects that infrastructure socio-technical a form to combination Evolutionary biologists observe that complex biological and ecological systems are are systems ecological and biological complex that observe biologists Evolutionary
These boosters are tools, laws, regulations, cultural norms, and skills that work in in work that skills and norms, cultural regulations, laws, tools, are boosters These
for healthy cognition. cognition. healthy for Activators Immunity develop must we thrive to democracy For
in the disinformation age disinformation the in that work against flourishing within a functioning democracy. functioning a within flourishing against work that
fairly, making us susceptible to misinformation that raises prejudices, fears, and beliefs beliefs and fears, prejudices, raises that misinformation to susceptible us making fairly,
as small group solidarity, they can also compromise our ability to think rationally or judge judge or rationally think to ability our compromise also can they solidarity, group small as democracy defending
and cataloged over the years. While cognitive biases may convey certain benefits, such such benefits, certain convey may biases cognitive While years. the over cataloged and
identified identified have scientists behavioral and psychologists that humans in Biases Cognitive
map) are designed to exploit network dynamics as well as one or more of the dozens of of dozens the of more or one as well as dynamics network exploit to designed are map)
IMMUNE SYSTEM IMMUNE
they employ (described in detail on the other side of this this of side other the on detail in (described employ they Vectors Attack The viruses.
by groups that excel in the creation and distribution of infectious social-media-ready social-media-ready infectious of distribution and creation the in excel that groups by
COGNITIVE responding to social, environmental, and economic challenges—is being undermined undermined being challenges—is economic and environmental, social, to responding
This is a precarious time in which democracy—our widely shared set of values for for values of set shared widely democracy—our which in time precarious a is This
What is cognitive immunity? cognitive is What HEALTHY A BUILDING
Future forces describe the technological, social, economic, environmental, and political factors likely to have an effect on a given domain. Future forces are important to Future Forces consider in combination because they form complex systems of interconnected drivers of change. Here we have applied a future forces analysis to the domain of cognitive immunity as a first step towards designing the tools we’ll need to create a more resilient and democratic society.
NEW EVIDENCE & NEW AUTHORITIES What’s Driving VULNERABLE DEMOCRATIC STATE A WORLD OF BLACK BOXES
Cognitive Trust in the State is Being Undermined by 21st Century Innovations Decisions About Our Lives are Increasingly Being Made By Opaque Algorithms People are Placing Their Trust in New, Untested Forms of Authority “Twentieth century political structures are drowning in a twenty-first century ocean of deregulated Today, the Web serves content to us via various devices. Tomorrow, it will be an overlay on physical Surveys reveal a widespread decline in trust of institutions of all kinds—governments, media outlets, Vulnerability? finance, autonomous technology, religious militancy and great-power rivalry,” writes Rana Dasgupta, reality. Already, high-resolution data about all aspects of our lives feed myriad machine learning corporations, multilaterals, and philanthropic organizations. Even academic and science experts no author of After Nations. The nation state still has substantial power to shape people’s lives and global algorithms that conduct an orchestra of AI agents to deliver custom content and act on our behalf. As longer enjoy the level of prestige they once had. People instead are turning to new sources of evidence A broad array of technological and social conditions but its powers are increasingly being tested by several forces, key among them free flow of the algorithms deepen in complexity, they’ll increasingly become black boxes—systems in which the and authority. Enabled by massive amounts of data and analytics technologies, we are creating new drivers affect the disinformation tactics capital, global connectivity, and climate disruptions where actions and policy choices outside of the inputs and outputs are understood but the internal workings are a mystery to the users and, frequently, pillars of trust. For instance, digital reputation systems allow us to trust strangers in ways we’ve never done before. With the rise of ride-share services like Uber and Lyft, parents are now comfortable and efforts to inoculate democracies control of individual states and localities have profound national and local consequences. The state still the creators. This lack of algorithmic accountability and transparency will dramatically increase the exercises control over the movements of people, but the costs of maintaining such controls are rising risk of failure and make it easier to obscure deceptive, unethical, and malicious practices behind allowing their children to get into cars driven by people they’ve never met. against them. Although each driver in itself under the strain of climate migration and as disparities in living conditions become more visible. inscrutable code. is important, they are not independent of one another. Their real impact comes from Rise of Global Plutocrats Global Criminal Economy Climate Change Refugees Everything is Media Algorithmic Bias Surveillance as a Feature A Shift Toward Autocracy Replication Crisis Personal Trust Networks and Oligarchies We’re entering a world of mixed Algorithms help humans decide A simple, spoken language their combinatorial efficacy. The international criminal By 2050, according to People who don’t trust The ability to repeat Our personal network According to Oxfam, in 2018, economy and global plutocrats scientists, there will be some reality in which digital content who to hire, who to loan money, request to an AI-powered the globalist, technocratic experiments with consistent moderates our worldview. 26 people owned as much are merging to form a 200 million environmental is extending well beyond and who to send to prison. device triggers a cascade establishment, under whose results is the cornerstone Today, global connectivity wealth as the 3.8 billion people new governance system refugees. Climate change televisions, smartphones, They also determine what of interactions and activities leadership they have been of science. In recent years, enables us to connect with who make up the poorest half that employs a criminal will become the cause of and computer displays to appears at the top of online involving digital and physical disenfranchised, are inclined however, some fundamental millions like us, amplifying of humanity. Their success is services industry of lawyers, most wars and political include our homes, vehicles, search results and the headlines objects. The intelligent, to trust those who appeal to studies in social sciences our beliefs and anxieties. This not connected to the fortunes accountants, and offshore asset upheavals. At the same buildings, and bodies. As the in your news feed. Addressing connected devices around their resentments. The result have failed the replication test, fragmentation will challenge of their fellow national citizens managers and other enablers. time, global environmental always-on interactions become the biases in algorithms is often us are personal panopticons, is a worldwide movement leading to a proliferation of anti- widely shared notions of reality, Reality Mining Decrypting the Brain and their goal is to defund They also use sophisticated youth movements are increasingly algorithmically difficult because of their black always happy to help but toward populist and autocratic science conspiracies. truth, and evidence. public goods, thus undermining surveillance and propaganda likely to put pressure on mediated, reality itself will box nature. potentially serving as portals “authorities.” Every click, purchase, view, and action we At the intersection of neuroscience, the state as they see it as techniques to shape opinions multinational organizations becomes malleable, sparking for undesired influence and make online is a data point that machine behavioral economics, cognitive a threat to their successful and information flows to sway and local governments to act an AI arms race between surveillance. learning algorithms use to gain insight psychology, and big data, researchers operation. popular sentiment. independently of nation state deepfake generation and into who we are, how we live, and what are uncovering the mysteries of how policies. detection technology. motivates us. the mind works, resulting in a real science of persuasion. Internet of Actions CULTURE OF EXTREMES CULT OF INNOVATION REGULATORY PARALYSIS A network of connected smart objects and VUCA World systems will negotiate, initiate, delegate, First introduced by the Army War College, and complete tasks based on our patterns the acronym refers to the volatility, New Media Amplifies Emotional Extremes to Maximize Revenue Shadow Side of the Move Fast, Break Things Ethos Rapid Advances in Technology are Difficult to Understand and Regulate of behavior and implicit wishes. IoT uncertainty, complexity, and ambiguity Machine learning, artificial intelligence, and insights from neuroscience are driving the development of The long-held desire for stable employment has given way to lionization of entrepreneurship. Silicon The global rise of digital disinformation and user data abuse creates an urgent need for regulation. But devices are notoriously insecure and their of the present day, creating a sense of our media infrastructure. These tools present unparalleled opportunities for expression and access to Valley’s singular focus on innovation and easy access to capital has resulted in the rapid growth of policies to address them are lagging. Congress has failed to move on curbing deceptive practices in behavioral data can be hijacked and used disorientation that’s ripe for exploitation. knowledge. They bring together people across geographic and institutional boundaries, but they also tech companies whose ethos is “move fast, break things.” They prioritize innovation, rapid growth, social media ad buying and preventing automated profiles on social media from amplifying particular for political intelligence. increase polarization, empower marginal extremes, and promote narrow-interest filter-bubble groups. and investor returns with little consideration of shadow sides of their products and services. In the individuals and information while simultaneously suppressing others. A few European nations are taking Wealth Inequality Democratic societies that value freedom of choice and expression can become overwhelmed with an process, they are disrupting traditional media and other trusted sources of information before we have steps to curb malicious data practices—but most democratic countries around the world are unable or Coding For Virality Economic inequality undermines overall overabundance of noise that obscures and distracts from quality information. Authoritarian societies a chance to put in place appropriate regulatory mechanisms to mitigate negative impacts. unwilling to identify and block efforts. A deep understanding of how ideas spread levels of trust within the population by can constrain access to information and people’s abilities to coordinate activities.These technologies through a particular medium, not solely due increasing social distance between can surreptitiously provoke extremes of human emotion and implicitly manipulate behavior. Business Model Based on Tech Discovers Ethics Follow the Money Regulatory Capture Technological Ignorance Fear of Being Left Behind to their content, but how their wrappers members of the society, making people Addiction and Data Capture Silicon Valley’s ethos of “build The death of journalism as we According to the New York Without intimate knowledge of AI is the new competitive (e.g. memes, gifs, tweets, celebrity believe that their compatriots are different Fragmented Identities Emotion-Driven Media Transparency Downsides Social media platforms are and ship” has resulted in social once knew it is due in large part Times, “four of the biggest new technologies, it is nearly battleground for companies endorsements) facilitate propagation from them. Familiar institutional markers Platforms use neuroscience and Extreme transparency increases engineered to keep users on media platforms that fail to to Advertising models in social technology companies are impossible for policy makers and governments. Many through the cultural environment, making of identity—work, family, behavioral economics to keep levels of accountability but the site as long as possible, consider how their products are media that deprive professional amassing an army of lobbyists to generate effective regulation believe that whoever builds media easier to weaponize. education, religion, politics, and users engaged for as long as might actually widen the collect data on everything they used and abused. Tech firms media operations of needed as they prepare for what in the digital space. Regulators and deploys most advanced Monopolization citizenship—are splintering. possible. Fact-challenged news Overton Window—the range of do, and sell it to advertisers. are just now reckoning with how funding. While venture could be an epic fight over and legislators are in a position AI will be a global economic Concentration of economic and New information tools enable that stimulates high-arousal ideas tolerated by the public. Treating advertisers as their to address potential misuses of capitalists put their money into their futures.” Armed with of constant catch up—trying to leader. The fear of stifling Digital Noise technological power in the hands of a few the creation of affinities around emotions is shared more than When we are confronted with clients, rather than users, has their products with a number of for-profit social media firms, money and deep knowledge solve problems after the fact innovation is often as the Information overload, perpetual distraction, information and social media platforms narrow sets of values, opening accurate articles that stimulate a steady stream of shocking led to deceptive practices that ethics and technology initiatives there is a dearth of investments of advanced technology, rather than anticipating them. reason for not curbing the and cognitive exhaustion has become our leads to emergence and hardening of opportunities for bad actors to low-arousal emotions. Emotion revelations or pronouncements, exploit tactics of propaganda at and university courses in public interest, non-profit, they are powerful foes to power of largest players. steady state. class divisions. destabilize societies and shift is the main fuel of the new we reset our normative the expense of the public. skyrocketing. and commons-based media any who challenge their established norms. media ecosystem. expectations. platforms and businesses. business models. BUILDING A HEALTHY WHAT IS COGNITIVE IMMUNITY?
This is a precarious time in which democracy-our widely shared set of values for COGNITIVE responding to social, environmental, and economic challenges-is being undermined by groups that excel in the creation and distribution of infectious social-media-ready viruses. The Attack Vectors they employ (described in detail on the other side of this IMMUNE SYSTEM map) are designed to exploit network dynamics as well as one or more of the dozens of Cognitive Biases in humans that psychologists and behavioral scientists have identified and cataloged over the years. While cognitive biases may convey certain benefits, such defending democracy as small group solidarity, they can also compromise our ability to think rationally or judge fairly, making us susceptible to misinformation that raises prejudices, fears, and beliefs in the disinformation age that work against flourishing within a functioning democracy.
For democracy to thrive we must develop Immunity Activators for healthy cognition. These boosters are tools, laws, regulations, cultural norms, and skills that work in Evolutionary biologists observe that complex biological and ecological systems are combination to form a socio-technical infrastructure that protects our networked society under constant attack from parasites-invaders on the lookout for weaknesses in the against malicious or unwanted intrusions that threaten the body politic. system to use as entry points to grow and propagate, in the process undermining the healthy functioning of the host. Under static conditions, these systems have established Building a cognitive immune system will require a systemicview of our networked society. We mechanisms for warding off such attackers. But under conditions of change, these must understand the forces-technological, cultural, economic-on the horizon that could mechanisms are no longer effective, allowing parasites to thrive and weaken the host. erode, or help sustain, democracy.
The metaphor is particularly apt in describing the state of our body politic today. Dramatic changes in our media and information technology landscapes are weakening our individual and collective cognitive immune systems, opening us up to disinformation COGNITIVE IMMUNE SYSTEM attacks and many new forms of manipulation. And the defenses we've built over decades is ill suited to ward off these new attackers, which include bots, trolls, and much more. DILEMMAS
The cognition of individuals and of the larger body politic has always been a target for It is important to note that a healthy cognitive immune system isn't a cure-all for wiping manipulation. Many are attempting to persuade others in many different directions, and in out disinformation. In the same way that a biological immune system can backfire or a democratic society we tolerate and even invite many voices. In today's highly networked over-react, a cognitive immune system can sometimes damage the system it's designed world, however, attacks against democracy have the potential to be more devastating than to protect. We must accept that there are challenges we cannot find solutions to. We call ever before. For better and worse, networked communications tools and technologies these dilemmas. Like parasites in an ecosystem, dilemmas are things we have to live with have altered existing power dynamics by re-defining who has the power of voice, the and learn to manage, in the process balancing difficult choices. power to shape our dominant narratives, and the power to influence our cognition. An individual with a $500 laptop, a $30 a month Internet service plan, and a YouTube Free Speech vs. Censorship Economic Returns vs. account has a greater potential audience than a 1990s television station with a newsroom, Social Returns cameras, a staff, transmitting equipment, and an FCC license, all of which totaled millions What are the boundaries of free of dollars. Social media platforms have unprecedented insight into the desires, fears, expression? Who decides where to How do we move beyond profit and behaviors of billions of their users. And yet, we still are at the early stages of this draw the lines? maximization in our media ecosystem? transformation, with old structures, regulatory systems, and cultural norms straining to How do we encourage capital investment in media platforms that respond to the rapid changes in our information environment. Platformsvs. Publishers put their users above sponsors and How much responsibility does a social To preserve democracy, we need to develop new immune mechanisms, a new immune advertisers? media platform bear for the content system, suited to today's realities. We need to uprade our "cognitive immune systems" at its users publish? To what extent is individual and community levels. a platform a common carrier, like a Cognitive Cohesion vs. phone company or an email service, Cognitive Diversity and to what extent is it a publisher, like How do we maintain healthy diversity HOW TO USE THIS MAP a newspaper or a magazine? while maintaining shared worldviews essential to a healthy society? This map presents the dilemmas, drivers, and future forces that will play important roles Innovation vs. Deliberation in the battle between disinformation tactics and healthy immune responses. Start with the Media Literacy vs. map side for a visual overview of the cognitive immune system and the attack methods How can a media business stay Media Manipulation bad actors use to exploit our cognitive biases. The map also reveals seven strategies competitive, while also taking time and How do we educate people in to improve our defenses. Then, flip the map over to gain a deeper understanding of the resources to develop mechanisms that the workings of technology and trends driving compromised cognitive immunity, as well as the difficult choices we must mitigate adverse consequences? In other media tools without engendering grapple with in our effort to combat disinformation. words, how does an ethical business greater distrust? compete with a business that espousesa credo of "move fast,break things?"
credo of “move fast, break things?” break fast, “move of credo without written consent. SR-2097 consent. written without
reserved. Reproduction is prohibited prohibited is Reproduction reserved.
compete with a business that espouses a a espouses that business a with compete
© 2019 Institute for the future. All rights rights All future. the for Institute 2019 © greater distrust? greater
grapple with in our effort to combat disinformation. combat to effort our in with grapple words, how does an ethical business business ethical an does how words,
media tools without engendering engendering without tools media trends driving compromised cognitive immunity, as well as the difficult choices we must must we choices difficult the as well as immunity, cognitive compromised driving trends mitigate adverse consequences? In other other In consequences? adverse mitigate
and technology of workings the
to improve our defenses. Then, flip the map over to gain a deeper understanding of the the of understanding deeper a gain to over map the flip Then, defenses. our improve to resources to develop mechanisms that that mechanisms develop to resources
in people educate we do How
bad actors use to exploit our cognitive biases. The map also reveals seven strategies strategies seven reveals also map The biases. cognitive our exploit to use actors bad competitive, while also taking time and and time taking also while competitive,
Media Manipulation Media
nd the attack methods methods attack the nd a system immune cognitive the of overview visual a for side map
How can a media business stay stay business media a can How
www.iftf.org
650.854.6322
vs. Literacy Media in the battle between disinformation tactics and healthy immune responses. Start with the the with Start responses. immune healthy and tactics disinformation between battle the in
Innovation vs. Deliberation vs. Innovation
94301 CA Alto, Palo
This map presents the dilemmas, drivers, and future forces that will play important roles roles important play will that forces future and drivers, dilemmas, the presents map This
essential to a healthy society? healthy a to essential Avenue Hamilton 201
HOW TO USE THIS MAP MAP THIS USE TO HOW while maintaining shared worldviews worldviews shared maintaining while magazine? a or newspaper a
How do we maintain healthy diversity diversity healthy maintain we do How like publisher, a it is extent what to and
phone company or an email service, service, email an or company phone Cognitive Diversity Cognitive
a platform a common carrier, like a a like carrier, common a platform a
vs. Cohesion Cognitive
individual and community levels. community and individual
its users publish? To what extent is is extent what To publish? users its
system, suited to today’s realities. We need to uprade our “cognitive immune systems” at at systems” immune “cognitive our uprade to need We realities. today’s to suited system,
media platform bear for the content content the for bear platform media
To preserve democracy, we need to develop new immune mechanisms, a new immune immune new a mechanisms, immune new develop to need we democracy, preserve To advertisers? How much responsibility does a social social a does responsibility much How
put their users above sponsors and and sponsors above users their put
Platforms vs. Publishers vs. Platforms
respond to the rapid changes in our information environment. environment. information our in changes rapid the to respond in the disinformation age disinformation the in investment in media platforms that that platforms media in investment
transformation, with old structures, regulatory systems, and cultural norms straining to to straining norms cultural and systems, regulatory structures, old with transformation,
How do we encourage capital capital encourage we do How
and behaviors of billions of their users. And yet, we still are at the early stages of this this of stages early the at are still we yet, And users. their of billions of behaviors and maximization in our media ecosystem? ecosystem? media our in maximization lines? the draw democracy defending
of dollars. Social media platforms have unprecedented insight into the desires, fears, fears, desires, the into insight unprecedented have platforms media Social dollars. of
How do we move beyond profit profit beyond move we do How to where decides Who expression?
of which totaled millions millions totaled which of all license, FCC an and equipment, transmitting staff, a cameras,
What are the boundaries of free free of boundaries the are What
Social Returns Social
account has a greater potential audience than a 1990s television station with a newsroom, newsroom, a with station television 1990s a than audience potential greater a has account
IMMUNE SYSTEM IMMUNE
vs. Returns Economic Censorship vs. Speech Free
An individual with a $500 laptop, a $30 a month Internet service plan, and a YouTube YouTube a and plan, service Internet month a $30 a laptop, $500 a with individual An
our cognition. cognition. our influence to power the and narratives, dominant our shape to power
and learn to manage, in the process balancing difficult choices. difficult balancing process the in manage, to learn and COGNITIVE have altered existing power dynamics by re-defining who has the power of voice, the the voice, of power the has who re-defining by dynamics power existing altered have
these dilemmas. Like parasites in an ecosystem, dilemmas are things we have to live with with live to have we things are dilemmas ecosystem, an in parasites Like dilemmas. these
ever before. For better and worse, networked communications tools and technologies technologies and tools communications networked worse, and better For before. ever
organization based in Palo Alto, California. Alto, Palo in based organization
to protect. We must accept that there are challenges we cannot find solutions to. We call call We to. solutions find cannot we challenges are there that accept must We protect. to
world, however, attacks against democracy have the potential to be more devastating than than devastating more be to potential the have democracy against attacks however, world,
is a registered 501(c)(3) nonprofit nonprofit 501(c)(3) registered a is Future
over-react, a cognitive immune system can sometimes damage the system it’s designed designed it’s system the damage sometimes can system immune cognitive a over-react,
today’s highly networked networked highly today’s In voices. many invite even and tolerate we society democratic a HEALTHY A BUILDING
Institute for the the for Institute future. sustainable more
em can backfire or or backfire can em syst immune biological a that way same the In disinformation. out
manipulation. Many are attempting to persuade others in many different directions, and in in and directions, different many in others persuade to attempting are Many manipulation.
across all sectors that together support a a support together that sectors all across
t a cure-all for wiping wiping for cure-all a t isn’ system immune cognitive healthy a that note to important is It
The cognition of individuals and of the larger body politic has always been a target for for target a been always has politic body larger the of and individuals of cognition The
coherent views of transformative possibilities possibilities transformative of views coherent
IFTF methodologies and toolsets yield yield toolsets and methodologies IFTF
dilemmas
is ill suited to ward off these new attackers, which include bots, trolls, and much more. much and trolls, bots, include which attackers, new these off ward to suited ill is
change and develop world-ready strategies. strategies. world-ready develop and change
we’ve built over decades decades over built we’ve defenses the And manipulation. of forms new many and attacks
navigate complex complex navigate to training foresight and
Cognitive immune system system immune Cognitive
us up to disinformation disinformation to up us opening systems, immune cognitive collective and individual our
forecasts, custom research, research, custom forecasts, global IFTF upon
are weakening weakening are landscapes technology information and media our in changes Dramatic
depended depended have organizations impact social
politic today. today. politic body our of state the describing in apt particularly is metaphor The
over 50 years, businesses, governments, and and governments, businesses, years, 50 over
organization. For For organization. thinking futures leading
erode, or help sustain, democracy. democracy. sustain, help or erode,
mechanisms are no longer effective, allowing parasites to thrive and weaken the host. host. the weaken and thrive to parasites allowing effective, longer no are mechanisms
the world’s world’s the is Future the for Institute
must understand the forces—technological, cultural, economic—on the horizon that could could that horizon the economic—on cultural, forces—technological, the understand must mechanisms for warding off such attackers. But under conditions of change, these these change, of conditions under But attackers. such off warding for mechanisms
Building a cognitive immune system will require a systemic view of our networked society. We We society. networked our of view systemic a require will system immune cognitive a Building ABOUT INSTITUTE FOR THE FUTURE THE FOR INSTITUTE ABOUT ems have established established have ems syst these conditions, static Under host. the of functioning healthy
system to use as entry points to grow and propagate, in the process undermining the the undermining process the in propagate, and grow to points entry as use to system
against malicious or unwanted intrusions that threaten the body politic. body the threaten that intrusions unwanted or malicious against —invaders on the lookout for weaknesses in the the in weaknesses for lookout the on —invaders parasites from attack constant under
combination to form a socio-technical infrastructure that protects our networked society society networked our protects that infrastructure socio-technical a form to combination Evolutionary biologists observe that complex biological and ecological systems are are systems ecological and biological complex that observe biologists Evolutionary
These boosters are tools, laws, regulations, cultural norms, and skills that work in in work that skills and norms, cultural regulations, laws, tools, are boosters These
for healthy cognition. cognition. healthy for Activators Immunity develop must we thrive to democracy For
in the disinformation age disinformation the in that work against flourishing within a functioning democracy. functioning a within flourishing against work that
fairly, making us susceptible to misinformation that raises prejudices, fears, and beliefs beliefs and fears, prejudices, raises that misinformation to susceptible us making fairly,
as small group solidarity, they can also compromise our ability to think rationally or judge judge or rationally think to ability our compromise also can they solidarity, group small as democracy defending
and cataloged over the years. While cognitive biases may convey certain benefits, such such benefits, certain convey may biases cognitive While years. the over cataloged and
identified identified have scientists behavioral and psychologists that humans in Biases Cognitive
map) are designed to exploit network dynamics as well as one or more of the dozens of of dozens the of more or one as well as dynamics network exploit to designed are map)
IMMUNE SYSTEM IMMUNE
they employ (described in detail on the other side of this this of side other the on detail in (described employ they Vectors Attack The viruses.
by groups that excel in the creation and distribution of infectious social-media-ready social-media-ready infectious of distribution and creation the in excel that groups by
COGNITIVE responding to social, environmental, and economic challenges—is being undermined undermined being challenges—is economic and environmental, social, to responding
This is a precarious time in which democracy—our widely shared set of values for for values of set shared widely democracy—our which in time precarious a is This
What is cognitive immunity? cognitive is What HEALTHY A BUILDING
Future forces describe the technological, social, economic, environmental, and political factors likely to have an effect on a given domain. Future forces are important to Future Forces consider in combination because they form complex systems of interconnected drivers of change. Here we have applied a future forces analysis to the domain of cognitive immunity as a first step towards designing the tools we’ll need to create a more resilient and democratic society.
NEW EVIDENCE & NEW AUTHORITIES What’s Driving VULNERABLE DEMOCRATIC STATE A WORLD OF BLACK BOXES
Cognitive Trust in the State is Being Undermined by 21st Century Innovations Decisions About Our Lives are Increasingly Being Made By Opaque Algorithms People are Placing Their Trust in New, Untested Forms of Authority “Twentieth century political structures are drowning in a twenty-first century ocean of deregulated Today, the Web serves content to us via various devices. Tomorrow, it will be an overlay on physical Surveys reveal a widespread decline in trust of institutions of all kinds—governments, media outlets, Vulnerability? finance, autonomous technology, religious militancy and great-power rivalry,” writes Rana Dasgupta, reality. Already, high-resolution data about all aspects of our lives feed myriad machine learning corporations, multilaterals, and philanthropic organizations. Even academic and science experts no author of After Nations. The nation state still has substantial power to shape people’s lives and global algorithms that conduct an orchestra of AI agents to deliver custom content and act on our behalf. As longer enjoy the level of prestige they once had. People instead are turning to new sources of evidence A broad array of technological and social conditions but its powers are increasingly being tested by several forces, key among them free flow of the algorithms deepen in complexity, they’ll increasingly become black boxes—systems in which the and authority. Enabled by massive amounts of data and analytics technologies, we are creating new drivers affect the disinformation tactics capital, global connectivity, and climate disruptions where actions and policy choices outside of the inputs and outputs are understood but the internal workings are a mystery to the users and, frequently, pillars of trust. For instance, digital reputation systems allow us to trust strangers in ways we’ve never done before. With the rise of ride-share services like Uber and Lyft, parents are now comfortable and efforts to inoculate democracies control of individual states and localities have profound national and local consequences. The state still the creators. This lack of algorithmic accountability and transparency will dramatically increase the exercises control over the movements of people, but the costs of maintaining such controls are rising risk of failure and make it easier to obscure deceptive, unethical, and malicious practices behind allowing their children to get into cars driven by people they’ve never met. against them. Although each driver in itself under the strain of climate migration and as disparities in living conditions become more visible. inscrutable code. is important, they are not independent of one another. Their real impact comes from Rise of Global Plutocrats Global Criminal Economy Climate Change Refugees Everything is Media Algorithmic Bias Surveillance as a Feature A Shift Toward Autocracy Replication Crisis Personal Trust Networks and Oligarchies We’re entering a world of mixed Algorithms help humans decide A simple, spoken language their combinatorial efficacy. The international criminal By 2050, according to People who don’t trust The ability to repeat Our personal network According to Oxfam, in 2018, economy and global plutocrats scientists, there will be some reality in which digital content who to hire, who to loan money, request to an AI-powered the globalist, technocratic experiments with consistent moderates our worldview. 26 people owned as much are merging to form a 200 million environmental is extending well beyond and who to send to prison. device triggers a cascade establishment, under whose results is the cornerstone Today, global connectivity wealth as the 3.8 billion people new governance system refugees. Climate change televisions, smartphones, They also determine what of interactions and activities leadership they have been of science. In recent years, enables us to connect with who make up the poorest half that employs a criminal will become the cause of and computer displays to appears at the top of online involving digital and physical disenfranchised, are inclined however, some fundamental millions like us, amplifying of humanity. Their success is services industry of lawyers, most wars and political include our homes, vehicles, search results and the headlines objects. The intelligent, to trust those who appeal to studies in social sciences our beliefs and anxieties. This not connected to the fortunes accountants, and offshore asset upheavals. At the same buildings, and bodies. As the in your news feed. Addressing connected devices around their resentments. The result have failed the replication test, fragmentation will challenge of their fellow national citizens managers and other enablers. time, global environmental always-on interactions become the biases in algorithms is often us are personal panopticons, is a worldwide movement leading to a proliferation of anti- widely shared notions of reality, Reality Mining Decrypting the Brain and their goal is to defund They also use sophisticated youth movements are increasingly algorithmically difficult because of their black always happy to help but toward populist and autocratic science conspiracies. truth, and evidence. public goods, thus undermining surveillance and propaganda likely to put pressure on mediated, reality itself will box nature. potentially serving as portals “authorities.” Every click, purchase, view, and action we At the intersection of neuroscience, the state as they see it as techniques to shape opinions multinational organizations becomes malleable, sparking for undesired influence and make online is a data point that machine behavioral economics, cognitive a threat to their successful and information flows to sway and local governments to act an AI arms race between surveillance. learning algorithms use to gain insight psychology, and big data, researchers operation. popular sentiment. independently of nation state deepfake generation and into who we are, how we live, and what are uncovering the mysteries of how policies. detection technology. motivates us. the mind works, resulting in a real science of persuasion. Internet of Actions CULTURE OF EXTREMES CULT OF INNOVATION REGULATORY PARALYSIS A network of connected smart objects and VUCA World systems will negotiate, initiate, delegate, First introduced by the Army War College, and complete tasks based on our patterns the acronym refers to the volatility, New Media Amplifies Emotional Extremes to Maximize Revenue Shadow Side of the Move Fast, Break Things Ethos Rapid Advances in Technology are Difficult to Understand and Regulate of behavior and implicit wishes. IoT uncertainty, complexity, and ambiguity Machine learning, artificial intelligence, and insights from neuroscience are driving the development of The long-held desire for stable employment has given way to lionization of entrepreneurship. Silicon The global rise of digital disinformation and user data abuse creates an urgent need for regulation. But devices are notoriously insecure and their of the present day, creating a sense of our media infrastructure. These tools present unparalleled opportunities for expression and access to Valley’s singular focus on innovation and easy access to capital has resulted in the rapid growth of policies to address them are lagging. Congress has failed to move on curbing deceptive practices in behavioral data can be hijacked and used disorientation that’s ripe for exploitation. knowledge. They bring together people across geographic and institutional boundaries, but they also tech companies whose ethos is “move fast, break things.” They prioritize innovation, rapid growth, social media ad buying and preventing automated profiles on social media from amplifying particular for political intelligence. increase polarization, empower marginal extremes, and promote narrow-interest filter-bubble groups. and investor returns with little consideration of shadow sides of their products and services. In the individuals and information while simultaneously suppressing others. A few European nations are taking Wealth Inequality Democratic societies that value freedom of choice and expression can become overwhelmed with an process, they are disrupting traditional media and other trusted sources of information before we have steps to curb malicious data practices—but most democratic countries around the world are unable or Coding For Virality Economic inequality undermines overall overabundance of noise that obscures and distracts from quality information. Authoritarian societies a chance to put in place appropriate regulatory mechanisms to mitigate negative impacts. unwilling to identify and block efforts. A deep understanding of how ideas spread levels of trust within the population by can constrain access to information and people’s abilities to coordinate activities.These technologies through a particular medium, not solely due increasing social distance between can surreptitiously provoke extremes of human emotion and implicitly manipulate behavior. Business Model Based on Tech Discovers Ethics Follow the Money Regulatory Capture Technological Ignorance Fear of Being Left Behind to their content, but how their wrappers members of the society, making people Addiction and Data Capture Silicon Valley’s ethos of “build The death of journalism as we According to the New York Without intimate knowledge of AI is the new competitive (e.g. memes, gifs, tweets, celebrity believe that their compatriots are different Fragmented Identities Emotion-Driven Media Transparency Downsides Social media platforms are and ship” has resulted in social once knew it is due in large part Times, “four of the biggest new technologies, it is nearly battleground for companies endorsements) facilitate propagation from them. Familiar institutional markers Platforms use neuroscience and Extreme transparency increases engineered to keep users on media platforms that fail to to Advertising models in social technology companies are impossible for policy makers and governments. Many through the cultural environment, making of identity—work, family, behavioral economics to keep levels of accountability but the site as long as possible, consider how their products are media that deprive professional amassing an army of lobbyists to generate effective regulation believe that whoever builds media easier to weaponize. education, religion, politics, and users engaged for as long as might actually widen the collect data on everything they used and abused. Tech firms media operations of needed as they prepare for what in the digital space. Regulators and deploys most advanced Monopolization citizenship—are splintering. possible. Fact-challenged news Overton Window—the range of do, and sell it to advertisers. are just now reckoning with how funding. While venture could be an epic fight over and legislators are in a position AI will be a global economic Concentration of economic and New information tools enable that stimulates high-arousal ideas tolerated by the public. Treating advertisers as their to address potential misuses of capitalists put their money into their futures.” Armed with of constant catch up—trying to leader. The fear of stifling Digital Noise technological power in the hands of a few the creation of affinities around emotions is shared more than When we are confronted with clients, rather than users, has their products with a number of for-profit social media firms, money and deep knowledge solve problems after the fact innovation is often as the Information overload, perpetual distraction, information and social media platforms narrow sets of values, opening accurate articles that stimulate a steady stream of shocking led to deceptive practices that ethics and technology initiatives there is a dearth of investments of advanced technology, rather than anticipating them. reason for not curbing the and cognitive exhaustion has become our leads to emergence and hardening of opportunities for bad actors to low-arousal emotions. Emotion revelations or pronouncements, exploit tactics of propaganda at and university courses in public interest, non-profit, they are powerful foes to power of largest players. steady state. class divisions. destabilize societies and shift is the main fuel of the new we reset our normative the expense of the public. skyrocketing. and commons-based media any who challenge their established norms. media ecosystem. expectations. platforms and businesses. business models.
credo of “move fast, break things?” break fast, “move of credo without written consent. SR-2097 consent. written without
reserved. Reproduction is prohibited prohibited is Reproduction reserved.
compete with a business that espouses a a espouses that business a with compete
© 2019 Institute for the future. All rights rights All future. the for Institute 2019 © greater distrust? greater grapple with in our effort to combat disinformation. combat to effort our in with grapple words, how does an ethical business business ethical an does how words,
media tools without engendering engendering without tools media trends driving compromised cognitive immunity, as well as the difficult choices we must must we choices difficult the as well as immunity, cognitive compromised driving trends mitigate adverse consequences? In other other In consequences? adverse mitigate
and technology of workings the to improve our defenses. Then, flip the map over to gain a deeper understanding of the the of understanding deeper a gain to over map the flip Then, defenses. our improve to resources to develop mechanisms that that mechanisms develop to resources
in people educate we do How
bad actors use to exploit our cognitive biases. The map also reveals seven strategies strategies seven reveals also map The biases. cognitive our exploit to use actors bad competitive, while also taking time and and time taking also while competitive,
Media Manipulation Media nd the attack methods methods attack the nd a system immune cognitive the of overview visual a for side map
How can a media business stay stay business media a can How
www.iftf.org
650.854.6322
vs. Literacy Media in the battle between disinformation tactics and healthy immune responses. Start with the the with Start responses. immune healthy and tactics disinformation between battle the in
Innovation vs. Deliberation vs. Innovation
94301 CA Alto, Palo
This map presents the dilemmas, drivers, and future forces that will play important roles roles important play will that forces future and drivers, dilemmas, the presents map This
essential to a healthy society? healthy a to essential Avenue Hamilton 201
HOW TO USE THIS MAP MAP THIS USE TO HOW while maintaining shared worldviews worldviews shared maintaining while magazine? a or newspaper a
How do we maintain healthy diversity diversity healthy maintain we do How like publisher, a it is extent what to and
phone company or an email service, service, email an or company phone Cognitive Diversity Cognitive
a platform a common carrier, like a a like carrier, common a platform a
vs. Cohesion Cognitive
individual and community levels. community and individual
its users publish? To what extent is is extent what To publish? users its
system, suited to today’s realities. We need to uprade our “cognitive immune systems” at at systems” immune “cognitive our uprade to need We realities. today’s to suited system,
media platform bear for the content content the for bear platform media
To preserve democracy, we need to develop new immune mechanisms, a new immune immune new a mechanisms, immune new develop to need we democracy, preserve To advertisers? How much responsibility does a social social a does responsibility much How
put their users above sponsors and and sponsors above users their put
Platforms vs. Publishers vs. Platforms
respond to the rapid changes in our information environment. environment. information our in changes rapid the to respond in the disinformation age disinformation the in investment in media platforms that that platforms media in investment
transformation, with old structures, regulatory systems, and cultural norms straining to to straining norms cultural and systems, regulatory structures, old with transformation,
How do we encourage capital capital encourage we do How
and behaviors of billions of their users. And yet, we still are at the early stages of this this of stages early the at are still we yet, And users. their of billions of behaviors and maximization in our media ecosystem? ecosystem? media our in maximization lines? the draw democracy defending
of dollars. Social media platforms have unprecedented insight into the desires, fears, fears, desires, the into insight unprecedented have platforms media Social dollars. of
How do we move beyond profit profit beyond move we do How to where decides Who expression?
of which totaled millions millions totaled which of all license, FCC an and equipment, transmitting staff, a cameras,
What are the boundaries of free free of boundaries the are What
Social Returns Social
account has a greater potential audience than a 1990s television station with a newsroom, newsroom, a with station television 1990s a than audience potential greater a has account
IMMUNE SYSTEM IMMUNE
vs. Returns Economic Censorship vs. Speech Free
An individual with a $500 laptop, a $30 a month Internet service plan, and a YouTube YouTube a and plan, service Internet month a $30 a laptop, $500 a with individual An
our cognition. cognition. our influence to power the and narratives, dominant our shape to power
and learn to manage, in the process balancing difficult choices. difficult balancing process the in manage, to learn and COGNITIVE have altered existing power dynamics by re-defining who has the power of voice, the the voice, of power the has who re-defining by dynamics power existing altered have
these dilemmas. Like parasites in an ecosystem, dilemmas are things we have to live with with live to have we things are dilemmas ecosystem, an in parasites Like dilemmas. these ever before. For better and worse, networked communications tools and technologies technologies and tools communications networked worse, and better For before. ever
organization based in Palo Alto, California. Alto, Palo in based organization
to protect. We must accept that there are challenges we cannot find solutions to. We call call We to. solutions find cannot we challenges are there that accept must We protect. to world, however, attacks against democracy have the potential to be more devastating than than devastating more be to potential the have democracy against attacks however, world,
is a registered 501(c)(3) nonprofit nonprofit 501(c)(3) registered a is Future
over-react, a cognitive immune system can sometimes damage the system it’s designed designed it’s system the damage sometimes can system immune cognitive a over-react,
today’s highly networked networked highly today’s In voices. many invite even and tolerate we society democratic a HEALTHY A BUILDING
Institute for the the for Institute future. sustainable more
em can backfire or or backfire can em syst immune biological a that way same the In disinformation. out manipulation. Many are attempting to persuade others in many different directions, and in in and directions, different many in others persuade to attempting are Many manipulation.
across all sectors that together support a a support together that sectors all across
t a cure-all for wiping wiping for cure-all a t isn’ system immune cognitive healthy a that note to important is It
The cognition of individuals and of the larger body politic has always been a target for for target a been always has politic body larger the of and individuals of cognition The
coherent views of transformative possibilities possibilities transformative of views coherent
IFTF methodologies and toolsets yield yield toolsets and methodologies IFTF
dilemmas is ill suited to ward off these new attackers, which include bots, trolls, and much more. much and trolls, bots, include which attackers, new these off ward to suited ill is
change and develop world-ready strategies. strategies. world-ready develop and change
we’ve built over decades decades over built we’ve defenses the And manipulation. of forms new many and attacks
navigate complex complex navigate to training foresight and
Cognitive immune system system immune Cognitive us up to disinformation disinformation to up us opening systems, immune cognitive collective and individual our
forecasts, custom research, research, custom forecasts, global IFTF upon
are weakening weakening are landscapes technology information and media our in changes Dramatic
depended depended have organizations impact social
politic today. today. politic body our of state the describing in apt particularly is metaphor The
over 50 years, businesses, governments, and and governments, businesses, years, 50 over
organization. For For organization. thinking futures leading
erode, or help sustain, democracy. democracy. sustain, help or erode, mechanisms are no longer effective, allowing parasites to thrive and weaken the host. host. the weaken and thrive to parasites allowing effective, longer no are mechanisms
the world’s world’s the is Future the for Institute
must understand the forces—technological, cultural, economic—on the horizon that could could that horizon the economic—on cultural, forces—technological, the understand must mechanisms for warding off such attackers. But under conditions of change, these these change, of conditions under But attackers. such off warding for mechanisms
Building a cognitive immune system will require a systemic view of our networked society. We We society. networked our of view systemic a require will system immune cognitive a Building ABOUT INSTITUTE FOR THE FUTURE THE FOR INSTITUTE ABOUT ems have established established have ems syst these conditions, static Under host. the of functioning healthy
system to use as entry points to grow and propagate, in the process undermining the the undermining process the in propagate, and grow to points entry as use to system
against malicious or unwanted intrusions that threaten the body politic. body the threaten that intrusions unwanted or malicious against —invaders on the lookout for weaknesses in the the in weaknesses for lookout the on —invaders parasites from attack constant under
combination to form a socio-technical infrastructure that protects our networked society society networked our protects that infrastructure socio-technical a form to combination Evolutionary biologists observe that complex biological and ecological systems are are systems ecological and biological complex that observe biologists Evolutionary
These boosters are tools, laws, regulations, cultural norms, and skills that work in in work that skills and norms, cultural regulations, laws, tools, are boosters These
for healthy cognition. cognition. healthy for Activators Immunity develop must we thrive to democracy For
in the disinformation age disinformation the in that work against flourishing within a functioning democracy. functioning a within flourishing against work that
fairly, making us susceptible to misinformation that raises prejudices, fears, and beliefs beliefs and fears, prejudices, raises that misinformation to susceptible us making fairly,
as small group solidarity, they can also compromise our ability to think rationally or judge judge or rationally think to ability our compromise also can they solidarity, group small as democracy defending
and cataloged over the years. While cognitive biases may convey certain benefits, such such benefits, certain convey may biases cognitive While years. the over cataloged and
identified identified have scientists behavioral and psychologists that humans in Biases Cognitive
map) are designed to exploit network dynamics as well as one or more of the dozens of of dozens the of more or one as well as dynamics network exploit to designed are map)
IMMUNE SYSTEM IMMUNE
they employ (described in detail on the other side of this this of side other the on detail in (described employ they Vectors Attack The viruses.
by groups that excel in the creation and distribution of infectious social-media-ready social-media-ready infectious of distribution and creation the in excel that groups by
COGNITIVE responding to social, environmental, and economic challenges—is being undermined undermined being challenges—is economic and environmental, social, to responding
This is a precarious time in which democracy—our widely shared set of values for for values of set shared widely democracy—our which in time precarious a is This
What is cognitive immunity? cognitive is What HEALTHY A BUILDING
Future forces describe the technological, social, economic, environmental, and political factors likely to have an effect on a given domain. Future forces are important to Future Forces consider in combination because they form complex systems of interconnected drivers of change. Here we have applied a future forces analysis to the domain of cognitive immunity as a first step towards designing the tools we’ll need to create a more resilient and democratic society.
NEW EVIDENCE & NEW AUTHORITIES What’s Driving VULNERABLE DEMOCRATIC STATE A WORLD OF BLACK BOXES
Cognitive Trust in the State is Being Undermined by 21st Century Innovations Decisions About Our Lives are Increasingly Being Made By Opaque Algorithms People are Placing Their Trust in New, Untested Forms of Authority “Twentieth century political structures are drowning in a twenty-first century ocean of deregulated Today, the Web serves content to us via various devices. Tomorrow, it will be an overlay on physical Surveys reveal a widespread decline in trust of institutions of all kinds—governments, media outlets, Vulnerability? finance, autonomous technology, religious militancy and great-power rivalry,” writes Rana Dasgupta, reality. Already, high-resolution data about all aspects of our lives feed myriad machine learning corporations, multilaterals, and philanthropic organizations. Even academic and science experts no author of After Nations. The nation state still has substantial power to shape people’s lives and global algorithms that conduct an orchestra of AI agents to deliver custom content and act on our behalf. As longer enjoy the level of prestige they once had. People instead are turning to new sources of evidence A broad array of technological and social conditions but its powers are increasingly being tested by several forces, key among them free flow of the algorithms deepen in complexity, they’ll increasingly become black boxes—systems in which the and authority. Enabled by massive amounts of data and analytics technologies, we are creating new drivers affect the disinformation tactics capital, global connectivity, and climate disruptions where actions and policy choices outside of the inputs and outputs are understood but the internal workings are a mystery to the users and, frequently, pillars of trust. For instance, digital reputation systems allow us to trust strangers in ways we’ve never done before. With the rise of ride-share services like Uber and Lyft, parents are now comfortable and efforts to inoculate democracies control of individual states and localities have profound national and local consequences. The state still the creators. This lack of algorithmic accountability and transparency will dramatically increase the exercises control over the movements of people, but the costs of maintaining such controls are rising risk of failure and make it easier to obscure deceptive, unethical, and malicious practices behind allowing their children to get into cars driven by people they’ve never met. against them. Although each driver in itself under the strain of climate migration and as disparities in living conditions become more visible. inscrutable code. is important, they are not independent of one another. Their real impact comes from Rise of Global Plutocrats Global Criminal Economy Climate Change Refugees Everything is Media Algorithmic Bias Surveillance as a Feature A Shift Toward Autocracy Replication Crisis Personal Trust Networks and Oligarchies We’re entering a world of mixed Algorithms help humans decide A simple, spoken language their combinatorial efficacy. The international criminal By 2050, according to People who don’t trust The ability to repeat Our personal network According to Oxfam, in 2018, economy and global plutocrats scientists, there will be some reality in which digital content who to hire, who to loan money, request to an AI-powered the globalist, technocratic experiments with consistent moderates our worldview. 26 people owned as much are merging to form a 200 million environmental is extending well beyond and who to send to prison. device triggers a cascade establishment, under whose results is the cornerstone Today, global connectivity wealth as the 3.8 billion people new governance system refugees. Climate change televisions, smartphones, They also determine what of interactions and activities leadership they have been of science. In recent years, enables us to connect with who make up the poorest half that employs a criminal will become the cause of and computer displays to appears at the top of online involving digital and physical disenfranchised, are inclined however, some fundamental millions like us, amplifying of humanity. Their success is services industry of lawyers, most wars and political include our homes, vehicles, search results and the headlines objects. The intelligent, to trust those who appeal to studies in social sciences our beliefs and anxieties. This not connected to the fortunes accountants, and offshore asset upheavals. At the same buildings, and bodies. As the in your news feed. Addressing connected devices around their resentments. The result have failed the replication test, fragmentation will challenge of their fellow national citizens managers and other enablers. time, global environmental always-on interactions become the biases in algorithms is often us are personal panopticons, is a worldwide movement leading to a proliferation of anti- widely shared notions of reality, Reality Mining Decrypting the Brain and their goal is to defund They also use sophisticated youth movements are increasingly algorithmically difficult because of their black always happy to help but toward populist and autocratic science conspiracies. truth, and evidence. public goods, thus undermining surveillance and propaganda likely to put pressure on mediated, reality itself will box nature. potentially serving as portals “authorities.” Every click, purchase, view, and action we At the intersection of neuroscience, the state as they see it as techniques to shape opinions multinational organizations becomes malleable, sparking for undesired influence and make online is a data point that machine behavioral economics, cognitive a threat to their successful and information flows to sway and local governments to act an AI arms race between surveillance. learning algorithms use to gain insight psychology, and big data, researchers operation. popular sentiment. independently of nation state deepfake generation and into who we are, how we live, and what are uncovering the mysteries of how policies. detection technology. motivates us. the mind works, resulting in a real science of persuasion. Internet of Actions CULTURE OF EXTREMES CULT OF INNOVATION REGULATORY PARALYSIS A network of connected smart objects and VUCA World systems will negotiate, initiate, delegate, First introduced by the Army War College, and complete tasks based on our patterns the acronym refers to the volatility, New Media Amplifies Emotional Extremes to Maximize Revenue Shadow Side of the Move Fast, Break Things Ethos Rapid Advances in Technology are Difficult to Understand and Regulate of behavior and implicit wishes. IoT uncertainty, complexity, and ambiguity Machine learning, artificial intelligence, and insights from neuroscience are driving the development of The long-held desire for stable employment has given way to lionization of entrepreneurship. Silicon The global rise of digital disinformation and user data abuse creates an urgent need for regulation. But devices are notoriously insecure and their of the present day, creating a sense of our media infrastructure. These tools present unparalleled opportunities for expression and access to Valley’s singular focus on innovation and easy access to capital has resulted in the rapid growth of policies to address them are lagging. Congress has failed to move on curbing deceptive practices in behavioral data can be hijacked and used disorientation that’s ripe for exploitation. knowledge. They bring together people across geographic and institutional boundaries, but they also tech companies whose ethos is “move fast, break things.” They prioritize innovation, rapid growth, social media ad buying and preventing automated profiles on social media from amplifying particular for political intelligence. increase polarization, empower marginal extremes, and promote narrow-interest filter-bubble groups. and investor returns with little consideration of shadow sides of their products and services. In the individuals and information while simultaneously suppressing others. A few European nations are taking Wealth Inequality Democratic societies that value freedom of choice and expression can become overwhelmed with an process, they are disrupting traditional media and other trusted sources of information before we have steps to curb malicious data practices—but most democratic countries around the world are unable or Coding For Virality Economic inequality undermines overall overabundance of noise that obscures and distracts from quality information. Authoritarian societies a chance to put in place appropriate regulatory mechanisms to mitigate negative impacts. unwilling to identify and block efforts. A deep understanding of how ideas spread levels of trust within the population by can constrain access to information and people’s abilities to coordinate activities.These technologies through a particular medium, not solely due increasing social distance between can surreptitiously provoke extremes of human emotion and implicitly manipulate behavior. Business Model Based on Tech Discovers Ethics Follow the Money Regulatory Capture Technological Ignorance Fear of Being Left Behind to their content, but how their wrappers members of the society, making people Addiction and Data Capture Silicon Valley’s ethos of “build The death of journalism as we According to the New York Without intimate knowledge of AI is the new competitive (e.g. memes, gifs, tweets, celebrity believe that their compatriots are different Fragmented Identities Emotion-Driven Media Transparency Downsides Social media platforms are and ship” has resulted in social once knew it is due in large part Times, “four of the biggest new technologies, it is nearly battleground for companies endorsements) facilitate propagation from them. Familiar institutional markers Platforms use neuroscience and Extreme transparency increases engineered to keep users on media platforms that fail to to Advertising models in social technology companies are impossible for policy makers and governments. Many through the cultural environment, making of identity—work, family, behavioral economics to keep levels of accountability but the site as long as possible, consider how their products are media that deprive professional amassing an army of lobbyists to generate effective regulation believe that whoever builds media easier to weaponize. education, religion, politics, and users engaged for as long as might actually widen the collect data on everything they used and abused. Tech firms media operations of needed as they prepare for what in the digital space. Regulators and deploys most advanced Monopolization citizenship—are splintering. possible. Fact-challenged news Overton Window—the range of do, and sell it to advertisers. are just now reckoning with how funding. While venture could be an epic fight over and legislators are in a position AI will be a global economic Concentration of economic and New information tools enable that stimulates high-arousal ideas tolerated by the public. Treating advertisers as their to address potential misuses of capitalists put their money into their futures.” Armed with of constant catch up—trying to leader. The fear of stifling Digital Noise technological power in the hands of a few the creation of affinities around emotions is shared more than When we are confronted with clients, rather than users, has their products with a number of for-profit social media firms, money and deep knowledge solve problems after the fact innovation is often as the Information overload, perpetual distraction, information and social media platforms narrow sets of values, opening accurate articles that stimulate a steady stream of shocking led to deceptive practices that ethics and technology initiatives there is a dearth of investments of advanced technology, rather than anticipating them. reason for not curbing the and cognitive exhaustion has become our leads to emergence and hardening of opportunities for bad actors to low-arousal emotions. Emotion revelations or pronouncements, exploit tactics of propaganda at and university courses in public interest, non-profit, they are powerful foes to power of largest players. steady state. class divisions. destabilize societies and shift is the main fuel of the new we reset our normative the expense of the public. skyrocketing. and commons-based media any who challenge their established norms. media ecosystem. expectations. platforms and businesses. business models. Disinformation BUILDING A HEALTHY False information, such as conspiracy theories and fake news, that is purposefully intended to IMMUNITY ACTIVATOR mislead, confuse, and exhaust our Cognitive Immune System PUBLIC MEDIA powers of cognition.
Dark Advertising PLATFORMS Hyper-partisan Media Use of psychometric data to News that is not necessarily false a guide Private social media platforms are built upon similar but is sensationalized through a specifically target individuals core Internet technologies as commons-based ventures Misleading Narratives online with advertisements that partisan lens. like Wikipedia. However, commons-based platforms Stories that appear to be are not shown to all users. These do not have access to the same levels of funding, plausible, but skew facts and advertisements can be used Disinformation attack vectors that and must rely on donations, grants, and other context to distort the reality of to sway individuals’ behaviors philanthropic investments. An alternative is news events to fit a particular during elections or push ATTACK exploit cognitive biases and immunity capital pools that support platforms more agenda or ideology. individuals towards extremism, beneficial to the social fabric. activators for defending democracy Nudging/ amongst other goals. FALSE Recommending Censorship iNFORMATION Reinforcement incentives and The removal of content or users indirect suggestion tactics to promoting content. Content can influence thoughts and behavior. be removed manually through takedowns, content containing AR/VR certain keywords or from certain Multisensory immersive media users can be made unpublishable. ATTACK Astroturfing can be designed to fabricate convincing but false reality. Human users that appear STRUCTURAL to be grassroots activists and organizers but are in MANIPULATIONS fact coordinated by, and Deepfakes Realistic videos and sometimes paid by, political voice recordings, that groups, corporations, and are difficult to ascertain individuals. as fake, created by Search Engine algorithms. IMMUNITY ACTIVATOR Optimization Manipulating search engine DATA results to promote certain content and control the Shadow Banning OWNERSHIP RULES narrative. Can be used A user’s content is downregulated, not shown IMMUNITY ACTIVATOR to spread disinformation in other users’ feeds, and/or does not appear in Tech giants use data generated by people to create and steer people towards searches. Then users are not informed that they REALISTIC multi-billion dollar businesses. What if you could own, extremism. have been shadow banned, and often remain trade, or donate your personal data? What if the data unaware of the action taken. against them. TECH APPROACHES were a public good? Treating data as a personal or public asset significantly changes the economics Algorithm-based solutions designed to identify, filter, and operating principles of current social Confirmation Bias and fight various forms of digital deception, will result in media businesses, removing some of the Tendency to selectively seek an arms race between bad actors and the platform capabilities for media manipulation. and interpret information so as engineers, and add black boxes to the space to confirm existing beliefs, often of public discourse. Any technological tools Mere-Exposure Effect disregarding contradictory data. deployed must involve human-machine The phenomenon by which partnership that combine computational repeated exposure to an idea, power with qualitative insight. person, or thing leads individuals to prefer it. Also known as the IMMUNITY ACTIVATOR familiarity principle. Early Warning Trolling Systems Engaging in many activities Bandwagon Effect ranging from harassment, With a wide knowledge gap between rapidly innovating The desire to adopt the new meme creation and tech companies and government agencies struggling to beliefs and trends of others promotion, and the keep up, it’s clear that policy makers and legislators increases as more people disruption of discussions could benefit by having tools and processes for Cognitive adopt them. with intentionally understanding and identifying implications of inflammatory dialogue. Truth Bias Biases new technologies on the horizon before The assumption that the they diffuse widely, and implementing Leaking information one is receiving early warning systems to signal when Public release of is true. This bias is central to regulatory interventions are needed. confidential information human communication. gathered through Impersonating illegitimate methods. Assuming real individuals’ identities by attackers. Spiral of Silence ATTACK The fear of being excluded from the group leading individuals to DIRTY remain silent on ideas that they Ingroup Bias feel are minority opinions, but are TRICKS The tendency of individuals actually held by the majority. to treat members of their in-group more favorably than Sockpuppets/ members of the out-group. Cyborgs Hacking Human users that seek to Obtaining unauthorized Doxxing manipulate online discourse, IMMUNITY ACTIVATOR access to a computer usually with manufactured Publishing an individual’s or or private network. identities, acting in organization’s private information INDEpeNDENT coordination, and using bot-like (e.g. home address, cell phone methods (automated mass PLATFORM number, and national identity posting, coordinated approval numbers) publicly online. Botnets Review BODIES Phishing Coordinated, large-scale and spamming, etc.). Malicious email, phone network of bots. calls, or texts designed Governments struggle to regulate social media platforms, to give attackers access while the platforms themselves have failed to institute to private data or login effective self-regulation. There is a need for an credentials. IMMUNITY ACTIVATOR independent oversight body to regulate the use of technology for the manipulation of public BEYOND MEDIA Social Bots opinion across platforms and countries. Bots that are intended to Transparency will be a key mechanism LITERACY Sleeper Bots impersonate and/or interact for auditing potential misuses of Automated accounts that engage with humans on social media. social media platforms. Media literacy and critical thinking are commonly in all of the aforementioned ATTACK considered to be the building blocks of a strong cognitive behaviors, but are distinguished immune system. But the most effective strategies to in that they lie dormant for long defend against disinformation are self-awareness stretches of time and are activated BOTS IMMUNITY ACTIVATOR of how our filters affect our interpretation of for mass, coordinated attacks information, promoting social cohesion when needed. NEW SOCIAL instead of technological alienation, and strengthening social and cultural MEDIA NORMS resiliency.
We can use research from psychology, evolutionary Amplifier Bots biology, immunology, network dynamics, political and Roadblock Bots Automated accounts that spread information by other sciences to not only construct media platforms Automated accounts that sharing, retweeting, and republishing content. Some that safeguard culture against negative influences spam hashtags and conversation only interact with other bots in coordinate efforts but that also promote prosocial behaviors online. threads with irrelevant information to manipulate trending algorithms. Often work in After all, information spreads because of some and gibberish in order to disrupt conjunction with approval bots. combination of cognitive biases and network streams of communication and Approval Bot effects. We need to understand both and Automated accounts coordination. apply this understanding to fostering that retweet, “like,” and cognitive and cultural immunity. comment on specific posts and profiles to enhance their credibility.
© 2019 Institute for the Future. All rights reserved. Reproduction is prohibited without written consent. SR-2097
credo of “move fast, break things?” break fast, “move of credo © 2019 Institute for the future. All rights reserved. Reproduction is prohibited without written consent. SR-2097 consent. written without prohibited is Reproduction reserved. rights All future. the for Institute 2019 © compete with a business that espouses a a espouses that business a with compete
greater distrust? greater grapple with in our effort to combat disinformation. combat to effort our in with grapple words, how does an ethical business business ethical an does how words,
media tools without engendering engendering without tools media trends driving compromised cognitive immunity, as well as the difficult choices we must must we choices difficult the as well as immunity, cognitive compromised driving trends mitigate adverse consequences? In other other In consequences? adverse mitigate
and technology of workings the to improve our defenses. Then, flip the map over to gain a deeper understanding of the the of understanding deeper a gain to over map the flip Then, defenses. our improve to resources to develop mechanisms that that mechanisms develop to resources
in people educate we do How
bad actors use to exploit our cognitive biases. The map also reveals seven strategies strategies seven reveals also map The biases. cognitive our exploit to use actors bad competitive, while also taking time and and time taking also while competitive,
Media Manipulation Media nd the attack methods methods attack the nd a system immune cognitive the of overview visual a for side map
How can a media business stay stay business media a can How
www.iftf.org
650.854.6322
vs. Literacy Media in the battle between disinformation tactics and healthy immune responses. Start with the the with Start responses. immune healthy and tactics disinformation between battle the in
Innovation vs. Deliberation vs. Innovation
94301 CA Alto, Palo
This map presents the dilemmas, drivers, and future forces that will play important roles roles important play will that forces future and drivers, dilemmas, the presents map This
essential to a healthy society? healthy a to essential Avenue Hamilton 201
HOW TO USE THIS MAP MAP THIS USE TO HOW while maintaining shared worldviews worldviews shared maintaining while magazine? a or newspaper a
How do we maintain healthy diversity diversity healthy maintain we do How like publisher, a it is extent what to and
phone company or an email service, service, email an or company phone Cognitive Diversity Cognitive
a platform a common carrier, like a a like carrier, common a platform a
vs. Cohesion Cognitive
individual and community levels. community and individual
its users publish? To what extent is is extent what To publish? users its
system, suited to today’s realities. We need to uprade our “cognitive immune systems” at at systems” immune “cognitive our uprade to need We realities. today’s to suited system,
media platform bear for the content content the for bear platform media
To preserve democracy, we need to develop new immune mechanisms, a new immune immune new a mechanisms, immune new develop to need we democracy, preserve To advertisers? How much responsibility does a social social a does responsibility much How
put their users above sponsors and and sponsors above users their put
Platforms vs. Publishers vs. Platforms
respond to the rapid changes in our information environment. environment. information our in changes rapid the to respond in the disinformation age disinformation the in investment in media platforms that that platforms media in investment
transformation, with old structures, regulatory systems, and cultural norms straining to to straining norms cultural and systems, regulatory structures, old with transformation,
How do we encourage capital capital encourage we do How
and behaviors of billions of their users. And yet, we still are at the early stages of this this of stages early the at are still we yet, And users. their of billions of behaviors and maximization in our media ecosystem? ecosystem? media our in maximization lines? the draw democracy defending
of dollars. Social media platforms have unprecedented insight into the desires, fears, fears, desires, the into insight unprecedented have platforms media Social dollars. of
expression? Who decides where to to where decides Who expression? How do we move beyond profit profit beyond move we do How
cameras, a staff, transmitting equipment, and an FCC license, all all license, FCC an and equipment, transmitting staff, a cameras, of which totaled millions millions totaled which of
What are the boundaries of free free of boundaries the are What
Social Returns Social
Becca Lewis Becca
account has a greater potential audience than a 1990s television station with a newsroom, newsroom, a with station television 1990s a than audience potential greater a has account
IMMUNE SYSTEM IMMUNE
Free Speech vs. Censorship vs. Speech Free vs. Returns Economic
Samuel Woolley, Ph.D. Woolley, Samuel Ph.D. Knutson, Brian
An individual with a $500 laptop, a $30 a month Internet service plan, and a YouTube YouTube a and plan, service Internet month a $30 a laptop, $500 a with individual An
Tamsin Woolley-Barker, Ph.D. Woolley-Barker, Tamsin Ph.D. Jeffery, Lyn
power to shape our dominant narratives, and the power to influence influence to power the and narratives, dominant our shape to power our cognition. cognition. our
Lawrence Wilkinson Lawrence J.D. Heller, Brittan
and learn to manage, in the process balancing difficult choices. difficult balancing process the in manage, to learn and COGNITIVE have altered existing power dynamics by re-defining who has the power of voice, the the voice, of power the has who re-defining by dynamics power existing altered have
Drew Sullivan Drew Farano Adriano
these dilemmas. Like parasites in an ecosystem, dilemmas are things we have to live with with live to have we things are dilemmas ecosystem, an in parasites Like dilemmas. these ever before. For better and worse, networked communications tools and technologies technologies and tools communications networked worse, and better For before. ever
Juliana Schroeder Juliana Ph.D. English-Lueck, Jan
to protect. We must accept that there are challenges we cannot find solutions to. We call call We to. solutions find cannot we challenges are there that accept must We protect. to world, however, attacks against democracy have the potential to be more devastating than than devastating more be to potential the have democracy against attacks however, world,
Nick Monaco Nick Bice Ed
over-react, a cognitive immune system can sometimes damage the system it’s designed designed it’s system the damage sometimes can system immune cognitive a over-react, a democratic society we tolerate and even invite many voices. In voices. many invite even and tolerate we society democratic a today’s highly networked networked highly today’s HEALTHY A BUILDING
out disinformation. In the same way that a biological immune syst immune biological a that way same the In disinformation. out em can backfire or or backfire can em manipulation. Many are attempting to persuade others in many different directions, and in in and directions, different many in others persuade to attempting are Many manipulation. and workshop attendees: workshop and
It is important to note that a healthy cognitive immune system isn’ system immune cognitive healthy a that note to important is It t a cure-all for wiping wiping for cure-all a t experts contributing to thanks special With The cognition of individuals and of the larger body politic has always been a target for for target a been always has politic body larger the of and individuals of cognition The
dilemmas is ill suited to ward off these new attackers, which include bots, trolls, and much more. much and trolls, bots, include which attackers, new these off ward to suited ill is
Robin Weiss Robin Bruno Helen we’ve built over decades decades over built we’ve defenses the And manipulation. of forms new many and attacks
Karin Lubeck Karin Bogott Robin Cognitive immune system system immune Cognitive our individual and collective cognitive immune systems, opening opening systems, immune cognitive collective and individual our us up to disinformation disinformation to up us
Trent Kuhn Trent Hagan Jean Dramatic changes in our media and information technology landscapes landscapes technology information and media our in changes Dramatic are weakening weakening are
The metaphor is particularly apt in describing the state of our body body our of state the describing in apt particularly is metaphor The politic today. today. politic DESIGN AND PRODUCTION AND DESIGN
erode, or help sustain, democracy. democracy. sustain, help or erode, mechanisms are no longer effective, allowing parasites to thrive and weaken the host. host. the weaken and thrive to parasites allowing effective, longer no are mechanisms
must understand the forces—technological, cultural, economic—on the horizon that could could that horizon the economic—on cultural, forces—technological, the understand must mechanisms for warding off such attackers. But under conditions of change, these these change, of conditions under But attackers. such off warding for mechanisms David Pescovitz David Frauenfelder Mark
Building a cognitive immune system will require a systemic view of our networked society. We We society. networked our of view systemic a require will system immune cognitive a Building healthy functioning of the host. Under static conditions, these syst these conditions, static Under host. the of functioning healthy ems have established established have ems Katie Joseff Katie Gorbis Marina
system to use as entry points to grow and propagate, in the process undermining the the undermining process the in propagate, and grow to points entry as use to system
AUTHORS
against malicious or unwanted intrusions that threaten the body politic. body the threaten that intrusions unwanted or malicious against parasites from attack constant under —invaders on the lookout for weaknesses in the the in weaknesses for lookout the on —invaders
combination to form a socio-technical infrastructure that protects our networked society society networked our protects that infrastructure socio-technical a form to combination Evolutionary biologists observe that complex biological and ecological systems are are systems ecological and biological complex that observe biologists Evolutionary
These boosters are tools, laws, regulations, cultural norms, and skills that work in in work that skills and norms, cultural regulations, laws, tools, are boosters These
for healthy cognition. cognition. healthy for Activators Immunity develop must we thrive to democracy For
in the disinformation age disinformation the in that work against flourishing within a functioning democracy. functioning a within flourishing against work that
fairly, making us susceptible to misinformation that raises prejudices, fears, and beliefs beliefs and fears, prejudices, raises that misinformation to susceptible us making fairly,
as small group solidarity, they can also compromise our ability to think rationally or judge judge or rationally think to ability our compromise also can they solidarity, group small as democracy defending
and cataloged over the years. While cognitive biases may convey certain benefits, such such benefits, certain convey may biases cognitive While years. the over cataloged and
501(c)(3) nonprofit organization based in Palo Alto, California. Alto, Palo in based organization nonprofit 501(c)(3)
identified identified have scientists behavioral and psychologists that humans in Biases Cognitive
is a registered registered a is Future the for Institute future. sustainable more a support
map) are designed to exploit network dynamics as well as one or more of the dozens of of dozens the of more or one as well as dynamics network exploit to designed are map) IMMUNE SYSTEM IMMUNE
views of transformative possibilities across all sectors that together together that sectors all across possibilities transformative of views
they employ (described in detail on the other side of this this of side other the on detail in (described employ they Vectors Attack The viruses. world-ready strategies. IFTF methodologies and toolsets yield coherent coherent yield toolsets and methodologies IFTF strategies. world-ready
by groups that excel in the creation and distribution of infectious social-media-ready social-media-ready infectious of distribution and creation the in excel that groups by navigate complex change and develop develop and change complex navigate to training foresight and research,
COGNITIVE
responding to social, environmental, and economic challenges—is being undermined undermined being challenges—is economic and environmental, social, to responding forecasts, custom custom forecasts, global IFTF upon depended have organizations
This is a precarious time in which democracy—our widely shared set of values for for values of set shared widely democracy—our which in time precarious a is This For over 50 years, businesses, governments, and social impact impact social and governments, businesses, years, 50 over For
organization. organization. thinking futures leading world’s the is Future the for Institute
What is cognitive immunity? cognitive is What HEALTHY A BUILDING ABOUT INSTITUTE FOR THE FUTURE THE FOR INSTITUTE ABOUT
Future forces describe the technological, social, economic, environmental, and political factors likely to have an effect on a given domain. Future forces are important to Future Forces consider in combination because they form complex systems of interconnected drivers of change. Here we have applied a future forces analysis to the domain of cognitive immunity as a first step towards designing the tools we’ll need to create a more resilient and democratic society.
NEW EVIDENCE & NEW AUTHORITIES What’s Driving VULNERABLE DEMOCRATIC STATE A WORLD OF BLACK BOXES
Cognitive Trust in the State is Being Undermined by 21st Century Innovations Decisions About Our Lives are Increasingly Being Made By Opaque Algorithms People are Placing Their Trust in New, Untested Forms of Authority “Twentieth century political structures are drowning in a twenty-first century ocean of deregulated Today, the Web serves content to us via various devices. Tomorrow, it will be an overlay on physical Surveys reveal a widespread decline in trust of institutions of all kinds—governments, media outlets, Vulnerability? finance, autonomous technology, religious militancy and great-power rivalry,” writes Rana Dasgupta, reality. Already, high-resolution data about all aspects of our lives feed myriad machine learning corporations, multilaterals, and philanthropic organizations. Even academic and science experts no author of After Nations. The nation state still has substantial power to shape people’s lives and global algorithms that conduct an orchestra of AI agents to deliver custom content and act on our behalf. As longer enjoy the level of prestige they once had. People instead are turning to new sources of evidence A broad array of technological and social conditions but its powers are increasingly being tested by several forces, key among them free flow of the algorithms deepen in complexity, they’ll increasingly become black boxes—systems in which the and authority. Enabled by massive amounts of data and analytics technologies, we are creating new drivers affect the disinformation tactics capital, global connectivity, and climate disruptions where actions and policy choices outside of the inputs and outputs are understood but the internal workings are a mystery to the users and, frequently, pillars of trust. For instance, digital reputation systems allow us to trust strangers in ways we’ve never done before. With the rise of ride-share services like Uber and Lyft, parents are now comfortable and efforts to inoculate democracies control of individual states and localities have profound national and local consequences. The state still the creators. This lack of algorithmic accountability and transparency will dramatically increase the exercises control over the movements of people, but the costs of maintaining such controls are rising risk of failure and make it easier to obscure deceptive, unethical, and malicious practices behind allowing their children to get into cars driven by people they’ve never met. against them. Although each driver in itself under the strain of climate migration and as disparities in living conditions become more visible. inscrutable code. is important, they are not independent of one another. Their real impact comes from Rise of Global Plutocrats Global Criminal Economy Climate Change Refugees Everything is Media Algorithmic Bias Surveillance as a Feature A Shift Toward Autocracy Replication Crisis Personal Trust Networks and Oligarchies We’re entering a world of mixed Algorithms help humans decide A simple, spoken language their combinatorial efficacy. The international criminal By 2050, according to People who don’t trust The ability to repeat Our personal network According to Oxfam, in 2018, economy and global plutocrats scientists, there will be some reality in which digital content who to hire, who to loan money, request to an AI-powered the globalist, technocratic experiments with consistent moderates our worldview. 26 people owned as much are merging to form a 200 million environmental is extending well beyond and who to send to prison. device triggers a cascade establishment, under whose results is the cornerstone Today, global connectivity wealth as the 3.8 billion people new governance system refugees. Climate change televisions, smartphones, They also determine what of interactions and activities leadership they have been of science. In recent years, enables us to connect with who make up the poorest half that employs a criminal will become the cause of and computer displays to appears at the top of online involving digital and physical disenfranchised, are inclined however, some fundamental millions like us, amplifying of humanity. Their success is services industry of lawyers, most wars and political include our homes, vehicles, search results and the headlines objects. The intelligent, to trust those who appeal to studies in social sciences our beliefs and anxieties. This not connected to the fortunes accountants, and offshore asset upheavals. At the same buildings, and bodies. As the in your news feed. Addressing connected devices around their resentments. The result have failed the replication test, fragmentation will challenge of their fellow national citizens managers and other enablers. time, global environmental always-on interactions become the biases in algorithms is often us are personal panopticons, is a worldwide movement leading to a proliferation of anti- widely shared notions of reality, Reality Mining Decrypting the Brain and their goal is to defund They also use sophisticated youth movements are increasingly algorithmically difficult because of their black always happy to help but toward populist and autocratic science conspiracies. truth, and evidence. public goods, thus undermining surveillance and propaganda likely to put pressure on mediated, reality itself will box nature. potentially serving as portals “authorities.” Every click, purchase, view, and action we At the intersection of neuroscience, the state as they see it as techniques to shape opinions multinational organizations becomes malleable, sparking for undesired influence and make online is a data point that machine behavioral economics, cognitive a threat to their successful and information flows to sway and local governments to act an AI arms race between surveillance. learning algorithms use to gain insight psychology, and big data, researchers operation. popular sentiment. independently of nation state deepfake generation and into who we are, how we live, and what are uncovering the mysteries of how policies. detection technology. motivates us. the mind works, resulting in a real science of persuasion. Internet of Actions CULTURE OF EXTREMES CULT OF INNOVATION REGULATORY PARALYSIS A network of connected smart objects and VUCA World systems will negotiate, initiate, delegate, First introduced by the Army War College, and complete tasks based on our patterns the acronym refers to the volatility, New Media Amplifies Emotional Extremes to Maximize Revenue Shadow Side of the Move Fast, Break Things Ethos Rapid Advances in Technology are Difficult to Understand and Regulate of behavior and implicit wishes. IoT uncertainty, complexity, and ambiguity Machine learning, artificial intelligence, and insights from neuroscience are driving the development of The long-held desire for stable employment has given way to lionization of entrepreneurship. Silicon The global rise of digital disinformation and user data abuse creates an urgent need for regulation. But devices are notoriously insecure and their of the present day, creating a sense of our media infrastructure. These tools present unparalleled opportunities for expression and access to Valley’s singular focus on innovation and easy access to capital has resulted in the rapid growth of policies to address them are lagging. Congress has failed to move on curbing deceptive practices in behavioral data can be hijacked and used disorientation that’s ripe for exploitation. knowledge. They bring together people across geographic and institutional boundaries, but they also tech companies whose ethos is “move fast, break things.” They prioritize innovation, rapid growth, social media ad buying and preventing automated profiles on social media from amplifying particular for political intelligence. increase polarization, empower marginal extremes, and promote narrow-interest filter-bubble groups. and investor returns with little consideration of shadow sides of their products and services. In the individuals and information while simultaneously suppressing others. A few European nations are taking Wealth Inequality Democratic societies that value freedom of choice and expression can become overwhelmed with an process, they are disrupting traditional media and other trusted sources of information before we have steps to curb malicious data practices—but most democratic countries around the world are unable or Coding For Virality Economic inequality undermines overall overabundance of noise that obscures and distracts from quality information. Authoritarian societies a chance to put in place appropriate regulatory mechanisms to mitigate negative impacts. unwilling to identify and block efforts. A deep understanding of how ideas spread levels of trust within the population by can constrain access to information and people’s abilities to coordinate activities.These technologies through a particular medium, not solely due increasing social distance between can surreptitiously provoke extremes of human emotion and implicitly manipulate behavior. Business Model Based on Tech Discovers Ethics Follow the Money Regulatory Capture Technological Ignorance Fear of Being Left Behind to their content, but how their wrappers members of the society, making people Addiction and Data Capture Silicon Valley’s ethos of “build The death of journalism as we According to the New York Without intimate knowledge of AI is the new competitive (e.g. memes, gifs, tweets, celebrity believe that their compatriots are different Fragmented Identities Emotion-Driven Media Transparency Downsides Social media platforms are and ship” has resulted in social once knew it is due in large part Times, “four of the biggest new technologies, it is nearly battleground for companies endorsements) facilitate propagation from them. Familiar institutional markers Platforms use neuroscience and Extreme transparency increases engineered to keep users on media platforms that fail to to Advertising models in social technology companies are impossible for policy makers and governments. Many through the cultural environment, making of identity—work, family, behavioral economics to keep levels of accountability but the site as long as possible, consider how their products are media that deprive professional amassing an army of lobbyists to generate effective regulation believe that whoever builds media easier to weaponize. education, religion, politics, and users engaged for as long as might actually widen the collect data on everything they used and abused. Tech firms media operations of needed as they prepare for what in the digital space. Regulators and deploys most advanced Monopolization citizenship—are splintering. possible. Fact-challenged news Overton Window—the range of do, and sell it to advertisers. are just now reckoning with how funding. While venture could be an epic fight over and legislators are in a position AI will be a global economic Concentration of economic and New information tools enable that stimulates high-arousal ideas tolerated by the public. Treating advertisers as their to address potential misuses of capitalists put their money into their futures.” Armed with of constant catch up—trying to leader. The fear of stifling Digital Noise technological power in the hands of a few the creation of affinities around emotions is shared more than When we are confronted with clients, rather than users, has their products with a number of for-profit social media firms, money and deep knowledge solve problems after the fact innovation is often as the Information overload, perpetual distraction, information and social media platforms narrow sets of values, opening accurate articles that stimulate a steady stream of shocking led to deceptive practices that ethics and technology initiatives there is a dearth of investments of advanced technology, rather than anticipating them. reason for not curbing the and cognitive exhaustion has become our leads to emergence and hardening of opportunities for bad actors to low-arousal emotions. Emotion revelations or pronouncements, exploit tactics of propaganda at and university courses in public interest, non-profit, they are powerful foes to power of largest players. steady state. class divisions. destabilize societies and shift is the main fuel of the new we reset our normative the expense of the public. skyrocketing. and commons-based media any who challenge their established norms. media ecosystem. expectations. platforms and businesses. business models.