Future Future Now

Reconfiguring Reconfiguring Reality Future Future Now

Future Now: Reconfiguring Reality Issue 3 | 2017 INSTITUTE FOR THE FUTURE a maker’s guide to the Internet of Actions Welcome to 2017 Welcome to 2019 Welcome to 2021 Welcome to 2023 Welcome to 2025 Welcome to 2027

What is technology for, if not reconfiguring real- ant. The technologies we develop and deploy ity? Humans have been using tools to redirect, reflect our choices and values. We have gotten reuse, and recombine materials and processes good at making tools for productivity, efficiency, since day one. It’s what we do. Today, we are and entertainment. But is that all we want? I on the cusp of a technology revolution that will hope not. The Internet of Actions offers us the reconfigure reality at the speed of light with a power to reshape the world—everything from vastly distributed network of machines. transforming the taste and experience of a meal, The Internet of Information has changed the to virtually reliving past memories and strength- way we move in the world; it has enabled us to ening our bonds with loved ones. direct the power of nearly infinite cloud comput- These tools will be in almost everyone’s ing with simple voice commands; giving smaller hands. Everyone, including people you may actors powerful resources that once belonged not like, will be able to enlist bots and various only to large organizations. It has let us expand machines to act in the world. Video filters aimed our world view—and hide in filter bubbles. at “improving” our appearance will become tools Over the next decade, we will increasingly to create lifelike spoofs. We may lose control engage with an Internet of Actions—a distrib- over our own image. Local law is getting embed- uted global network of autonomous robots ded into and enforced by physical infrastruc- and intelligent systems. Such technology will tures. Weapons developers are grappling with allow us to reconfigure reality using increas- questions around if, and how, to embed rules ingly sophisticated strategies. Data science and of warfare into autonomous machines. Even as machine learning will enable us to create new these technologies empower us, they will create pathways to alter human perception. Ubiquitous new questions about how to interact, negotiate, sensing and utility machine intelligence will and navigate these possibilities with each other. create opportunities to encode human activ- Over the course of the year, we have talked to ity into distributed systems. Advances in fields experts in tech and design, explored the founda- from nanoengineering, 3D printing, and robot- tional technologies of an Internet of Actions, and ics will enable us to manipulate matter. As voice forecasted how to combine these technologies and gestural interfaces become pervasive, new into strategies for reconfiguring reality. approaches to designing personality and emo- This magazine is a maker’s guide to the Inter- tional responses will emerge together with ani- net of Actions. Use it with its companion map mate objects and environments. and card game, to anticipate possibilities, create As you grapple with questions and opportu- opportunities, ward off challenges, and begin nities to put these strategies to work, consider: acting to reconfigure reality today. What future do you want to make? Ultimately, these emerging technologies are Rod Falcon, Tech Futures Lab Director subservient to our beliefs about what is import- Institute for the Future Future Reconfiguring Now Reality

Make the Future Now Forecasting 2027

4 Strategies for Reconfiguring Reality 14 Cutting the Akashic Records Reality is becoming indexed, 8 Who’s in Control? stored, and searchable Navigating levels of autonomy David Pescovitz Max Elder 15 Universal Basic Assets 10 What Does Your Spoon Stand For? Rethinking who owns our digital assets Optimizing values with design choices Bradley Kreit and Jessica Cussins Sarah Smith and Jessica Cussins 16 Embodied Learning 12 What Lies Beyond the Virtual Assistant? Immersive media’s educational potential Metaphors to expand possibilities Alexander Goldman and Toshi Hoo Dylan Hendricks 18 Your New Telepresence Superpowers Get ready for your robotic upgrade Carla Sinclair 14 20 Teaching Robots How to Work Together A conversation with swarm robotics pioneer Nora Ayanian Mike Liebhold, Mark Frauenfelder, David Pescovitz

22 The Rise of Computational Propaganda A conversation with IFTF Fellow for Good 24 Sam Woolley Mark Frauenfelder

24 Post Money The basket of desires and how sellers can tap it Chris Kalaboukias

26 Digital Twins What if we could get better answers 26 from what-if questions? Mark Frauenfelder

28 Don’t Fear the Black Box Interpreting the future of AI Matthew Hutson Scenes from 2027

32 Kill Decision 60 Scenes from 2027 Maria Konnikova and Kevin Kelly Rod Falcon, Bradley Kreit, discuss how robot violence could Jamais Cascio, Mark Frauenfelder, make the world a better place Eri Gentry, Ben Hamamoto, Mark Frauenfelder Toshi Hoo, Scott Minneman

36 Billions and Billions Transacted 61 The Pass | I Shouldn’t Have Done It How the Internet of Actions will Smart charter community Cerrito Guapo transform everything to do with food disciplines a rule-breaker Max Elder Jamais Cascio

40 Homegrown Governance 65 The Ruse | The Roshki Ruse When smart homes become citizens I make fake backstories for electronic of common-interest communities gadgets, and I am not ashamed Kathi Vian Mark Frauenfelder

44 A Wearable MRI Machine 69 The Caregiver | The Artificial Caregiver Talking telepathy with Mary Lou Jepsen A lost childhood toy returns Scott Minneman in an unexpected context Ben Hamamoto 48 Make It Just So A look at the coming world 73 The Fleet | Car and Rider of anticipatory retail Behind the “autonomous” fleet Scott Minneman Scott Minneman

52 Simulcra and Simulations When video forgeries of human beings roam free Jamais Cascio 61

56 Wetware-Hardware Hybrids The future of computing and neural interfacing Steve M. Potter

78 78 Fill in the Blanks Train your brain to see more clearly what isn’t there (yet) Jane McGonigal Make the future now Strategies for Reconfiguring Reality

As advances in machine learning, virtual and augmented reality, robotics, and 3D printing combine over the next decade, they’ll create new strategies for enlisting networked machines to foster business, social, and civic value. Stemming from the combination of technical innovations in hardware, computing and processing, and interface development that will enable the Internet of Actions, each strategy contains four approaches for reconfiguring reality.

Altering human perceptions Immerse | Illuminate | Control | Influence

Until recently, you could reasonably expect sensory inputs for people in the same place to be, in large part, available to all. Bar- ring impairment, a person near you could be assumed to see, hear, smell, and touch what you were seeing, hearing, smelling, and touching. The coming decade will be marked by a proliferation of augmentation technologies that will dramatically change that tenet of human experience and alter our personal sensory inputs in increasingly sophisticated ways. Initially, visual and auditory stimuli will be targeted, with better methods for augmenting our visual field with artificial objects and psychoacoustically placed sounds that realistically seem to be in our sensory range. Coupled with intuitive modalities of using our bodies to sense and manipulate artificial constructs, using gaze, gestures, and haptics, new forms of communication and entertainment blending real and virtual worlds will proliferate. People will be empowered by, and possibly bombarded with, immersive experiences that dial in different amounts of real and synthetic content. Hidden layers of information will be illuminated, and we will be able to better control our engagement with the world. But we’ll also be subject to, and have new abilities to influence others through bots and other emerging interfaces. In the next decade, these technologies will deliver more of the cues that humans use for sensing position and motion, and synthetic stimuli will transition from overlaid intruders in the real world to an inextricable and expected part of everyday experience.

4 INSTITUTE FOR THE FUTURE Make the Future Now

Encoding human activity Embed | Execute | Rebalance | Remix

People exhibit complex behaviors and navigate through a vast web of contingent situations, but this ongoing cognitive load can be exhausting and frustrating. Early efforts to support personal choices with wearable sensors and smart spaces required determined, proactive users with a high tolerance for errors. Current offerings are improving on all fronts but still appeal only to niche groups and work best in highly circumscribed conditions. Users essentially train themselves to be good triggers and live their lives to suit available actions. Artificial intelligence—primarily in the form of machine-learning algorithms, affordable and compact accurate biometric instrumentation, incorporation of context and moods, and data fused from myriad environmental sensors—will allow us to codify what people want, and enable a vast assortment of new applications to emerge in coming years. Algorithms, both embedded in specific devices and orchestrating constellations of devices from the cloud, will develop to operate independently—on our behalf and on their own. These systems analyze inputs, detect patterns, find correlations, test hypotheses, draw conclusions, and execute and share results, outstripping humans with the data quantity and rationality incorporated in each choice. Offload- ing some portion of our everyday decision-making will initially be based on rules we approve. In time, we’ll come to trust more autonomous machines to operate using learned behaviors and improvised improvements. Eventually we’ll count on machines to innovate on their own by identifying patterns and new opportunities aligned with the simple value systems we’ve encoded into them. Algorithmic interventions will rebalance our lives by sim- plifying repetitive tasks, improving our health, extending independent living, enhancing learning, and flagging opportunities for social interactions. Other domains will emerge as well, both mundane and liberating, including remixing wetware of the brain with hardware to accelerate improvements in physical machines. Tensions among privacy, utility, and public good will persist, likely creating a divide between those who attempt to fly “under the data radar”—using personal clouds, encryption, jamming, and other means—and those who continue the current trend of surrendering data for perceived benefits.

FUTURE NOW: RECONFIGURING REALITY 5 Manipulating matter Model | Make | Swarm | Route

Our ability to create objects and environments that suit our tastes and fulfill our specific needs is transitioning from selecting among commercial offerings to a mixture of personal authorship, accurately modeled form and function, and automatic adaptation. In the coming decade, broad swaths of our environment will be alterable to suit day-to-day and moment-to-moment whims, as the just-in-time and on-demand pace of fashion, decor, and function all accelerate. Developments in precise multimaterial additive fabrication, novel materials with alterable appearance, and highly customizable modes of fabrication will combine to support fluid expressions of appearance and function, all delivered to the right place at the ideal time. Clothing, many consumer goods, furnish- ings, interiors, and facades will be in a flux of customization and tweaks, authored, specified, and made rather than selected from off-the-shelf products and services. Brands will participate by offering palettes for individual expression, and a new stripe of designer/curator will emerge and attract followings by creating specific items and looks. Robots, drones, and increasingly integrated and intelligent machines, incorporating novel materials with myriad physical properties, at scales from micro to macro, will become widespread. Operating in configurations from singles to collaborating swarms, privately owned and deployed as services, these devices will route matter to enable ongoing transformations in the home, in boutique service bureaus, and in distributed indus- trial settings. From shaping distant landscapes and architecture to jewelry and adornment, these advances will amplify and extend human capabilities to places and scales beyond our current reach.

6 INSTITUTE FOR THE FUTURE Make the Future Now

Animating objects and environments Infuse | Visualize | Anticipate | Orchestrate

Humans are used to explicit control of machines, while our deal- ings with other creatures engage a much broader range of interactions. Upcoming breakthroughs in natural language processing, conversational interfaces, gestural inputs, and affective computing will infuse machines with increasingly lifelike interactive capabilities. In concert with expanding artificial intelligence and decision-making abilities, our objects and environments will attempt to predict our desires, visualize past events, and anticipate actions that we will appreciate and value. Our technological systems will take full advantage of these new modalities—one may act like a kooky pet, another may be a confidante, another still may be an advisor or nurse, while some subset will still be responsive to do our specific bidding. As affective computing systems become the norm, some of our spaces and devices will accommodate our moods and attitudes, while others will try to change, nudge, and shape our behaviors in ways both explicit and implicit. Negotia- tions and arguments with our devices and systems will undoubtedly occur, but so will delightful and surprising experiences of devices and environments orchestrating to tailor themselves to our circumstances. Being greeted with an ideal refreshing bev- erage, treated to entertainment that buoys our mood, or perfectly tutored by process-aware kitchen appliances—“Knead the bread dough. Don’t punch it!”—will outnumber the times we find ourselves asking our spaces and devices to explain why they took a particular action. Interaction with an animated world can tune our desires and expectations and can shape our responses to assumptions and errors. How annoyed can you possibly get at a robotic pet that’s brought you a yoga mat and is doing its best downward dog?

FUTURE NOW: RECONFIGURING REALITY 7 Navigating levels of autonomy Who’s in Control?

by Max Elder

Over the next decade, autonomous bots and machines will migrate out of a few select industries and terrains and become active participants in a wide array of social, business, and civic spheres. The introduction of these new actors will require us to sort through complex decisions regarding human- machine collaboration and oversight. To make sense of the kinds of decisions and trade-offs we’ll face, we can gain inspiration from the world of autonomous vehicles where five levels of autonomy give structure to key questions around the responsibilities of the driver, the machine, and the machine’s designers.

Levels of Autonomy Level 0: No Automation Level 4: High Automation There is no automation whatsoever. Everything is automated within specific Humans perform all the activities. use-cases and can operate seamlessly without humans. Level 1: Machine Assistance There is some action-specific assistance, but Level 5: Full Automation the human operator is still fully in control. Total automation, in every circumstance, for anything a human can do (and beyond). Level 2: Partial Control There is some partial, task-specific control from machines, but humans are still in Each level of automation introduces new trade control of everything. offs and questions, even when the net result is almost certainly positive. In the airline indus- Level 3: Conditional Control try, for instance, the increasing sophistication of Functions are automated, but human autopilot has radically reduced crashes even as it oversight—and the occasional full human has eroded pilot skill. As you begin developing takeover—is necessary. your long-term strategies and designs, consider the following tradeoffs:

8 INSTITUTE FOR THE FUTURE Make the Future Now

Tradeoffs Moving Levels of Autonomy from 0 ➞ 5

Humans have fundamental efficiency limitations: We have limited attention spans, we need to eat and Efficiency sleep, and we get easily distracted. As actions are automated, they can be done more quickly, more accurately, and 24 hours a day.

When a human decides to do something, we can always ask her why she made that decision. Some autonomous software can produce incredibly accu- Understanding rate and efficient decisions but, when asked, fall completely silent. These black boxes can make it impossible for us to understand the autonomous decision procedures.

Human skills can atrophy as machines replace more human actions. Some of these skills may not be important (e.g., the loss of calligraphy), but oth- Skills ers may be (e.g., flying a plane full of people). Auton- omy levels 2-3 are often the trickiest to implement because they require an attentive, but often inactive human operator.

The presence of human judgment decreases as autonomy increases. In some cases, humans are great at incorporating a lot of information quickly Judgment and determining what is significant in order to make a quick decision. That is a good thing. In other cases, humans have cognitive biases and ulterior motives, which are bad.

Autonomous systems engender complacency in humans, especially with regard to decision-mak- Complacency ing. Complacency will be a particularly pernicious problem, since it is easier to be complacent the more autonomous these systems become.

While humans may be the least predictable variable in a situation involving any level of automation, Adaptability they are the most adaptable. When something goes wrong, adaptability is often the deciding factor between difficulty and disaster.

FUTURE NOW: RECONFIGURING REALITY 9 gained prominence in the 1950’s in an ode to a value of a different kind: convenience. However, as concerns about climate change and sustain- ability have grown in recent years, compostable Optimizing values with design choices cutlery has also grown in popularity. Other shifts are more imaginative or subversive. For example, What Does Your it is not uncommon to see people playing spoons as a musical instrument in folk bands. Spoon Stand For? But what a spoon has been capable of to date has been fairly limited. The technologies of the Internet of Actions is transforming even the humble spoon into an object that can be re-imagined and designed to optimize for a wide set of values. The convergence of additional technologies by Sarah Smith and Jessica Cussins possible today however is changing what is possible in every space, and cutlery is no different. So-called smart spoons, embedded with computing power, are now available and add additional layers of ability to these items we use daily. The range of design possibilities is expanding, and the choices we make are a reflection of the values we hold today. The idea of technology as morally neutral is per- Bringing different values to a technology, vasive. We are told that it is only the use of a tech- even when that technology is as simple as a nology that makes it good or bad. But, technology spoon, has a meaningful impact in the world. The is persuasive. It is designed with optimal users idea of design ethics is to think critically about and purposes in mind. It exerts influence over what a design is optimized for, who it is intended our lives and helps create the kind of world we to benefit, and in what way. Many digital tech- inhabit. nologies are optimized to encourage time spent The spoon is one of humankind’s earliest tech- on a company’s service. But this is rarely in the nological inventions, and seems about as neutral interest of the user. What other kinds of values as it gets. Is it really? are being explored within the space of our eating tools? The following signals highlight how pri- From Power to Convenience oritizing different values can alter the impact of Spoons have actually been used to signify wealth, a technology. They also encourage us to test our power, and religious significance for thousands assumptions about the objects we use everyday. of years. Although many early spoons were made from wood, which was cheaper and easier to Dignity and Independence carve, spoons were also made of silver and gold The company Liftware (now called Verily) has to distinguish the wealthy and elite. Spoons designed “smart spoons” that are self-stabilizing, engraved with hieroglyphics were used for reli- which helps people who struggle to hold food gious purposes in Egypt some three thousand steady for a variety of reasons to eat with greater years ago. British kings were given spoons cere- ease. Liftware became part of Google Life Sci- monially, and spoons of varying degrees of pre- ences (now called Verily) in 2014. The company’s ciousness were often given as christening gifts, first product, Liftware Steady, allows someone probably resulting in the phrase, “born with a with a hand tremor, for example from Parkin- silver spoon in his/her mouth.” In other words, son’s disease, to eat more easily. Liftware Level the material, design, and ceremony surrounding helps people with limited hand or arm mobil- the spoon have made it not just a tool, but also a ity, for example from cerebral palsy, spinal cord signifier of class, religion, ability, and social sta- injury, or Huntington’s disease, to eat more eas- tus. But these values are not necessarily constant. ily. The spoon is electronic and uses a microchip Fast forward centuries and the plastic spoon and sensors to respond to the hand’s movements.

10 INSTITUTE FOR THE FUTURE Make the Future Now

While online reviews reveal that the prod- practice in China of reselling so-called “gutter uct does not work for everyone, it does seem to oil” that is toxic and harmful, but can end up improve the experience at least to some degree in your food. These chopsticks, called Kuaisou, for many others. Although using a special set of apparently started off as “something of an April utensils can be stigmatizing in its own way, Lift- Fool’s prank” according to a Baidu representa- ware empowers people to feed themselves with- tive, but were met with real interest. out spilling food. It also enables its users to do It is unclear if this product ever made it to things many of us take for granted, like go out for market, and it is hardly the most straightforward dinner with friends without feeling stigmatized. strategy for shutting down dangerous and illegal It is designed to be discreet and to minimize the activity. Nonetheless, it’s a nice idea that your stress and embarrassment of spilling one’s food. eating utensils can keep an eye out for you, so to This design helps people with reduced motor speak, and warn you if the food you are about to function to regain some of the dignity and inde- eat is unsafe. Baidu representatives also talked pendence that comes from being able to eat on about future versions being able to tell the origin one’s own. of different ingredients and the nutrition they contain. Mindful Health Unlike Liftware, which aims to increase the ease Novelty and Fun of eating, the HAPIfork aims to make it a little Not all ideas for eating utensils have such lofty more challenging. The purpose of this, is to cause goals in mind. The so-called “selfie spoon” you to eat slower, which is beneficial according unveiled by cereal brand Cinnamon Toast to the company because it can improve digestion Crunch is pretty much what it sounds like. It is a and support weight control. Using HAPIfork is free spoon people can order with an attached sel- supposed to help you “lose weight, feel great!” fie stick that allows people to photograph them- The HAPIfork uses lights and vibrations to alert selves during breakfast. Their website states, you when you are eating too quickly. It also “Selfies and cereal together at last!” and assures tracks your eating habits: the amount of time you this is, “Really a Thing!” It’s a brilliant marketing spent eating and how much you ate, which you ploy to get kids to advertise cereal. It’s fun to use can then monitor on its website. the device, which is now sold out. Another company called Spün Utensils, claims to calculate the nutritional information So What? in the food you eat, and offer you “as much (or Each of these individual products is far from as little) data on your eating habits as you’d like.” perfect. A recent study published in the journal It also provides nutritional recommendations. It Appetite actually found that the HAPIfork made works through the use of motion sensors and by people eat slightly slower, but made no difference weighing each bite of food. Similar to the HAPI- on the total amount they ate. Moreover, when fork, it also uses vibrations to alert you if you are people experience a device like this as “food eating too quickly, as well as if you have reached shaming,” they may avoid using it, or it could your calorie goal. even backfire. Any particular technology may be A slight twist on these ideas, an “electric the wrong place for a value to be expressed, and flavoring fork” announced last year by Hiromi good intentions can certainly go awry. Nakamura at the Rekimoto Lab in Japan, tricks Nonetheless, all technologies are value-laden. your taste buds. The fork uses a small amount Uncovering those values, and thinking critically of electricity to stimulate the tongue and gen- about what you actually want from them opens erate the sensation of eating something salty. up new opportunities. As the simple technolo- The product is currently targeted at those who gies around us—from spoons to everything else need a low or zero salt diet, such as hypertensive —become increasingly infused with algorithms patients. and intentions, the question of what values to encode and how to overcome biases will become Safety and Security more difficult to ignore. Over the next decade, In 2014, the Chinese search giant Baidu unveiled we’ll program machines with a much wider set a set of “smart chopsticks” that can check for of goals, priorities, and values, and they will be contaminated cooking oil. There is an illegal much more effective at achieving their ends.

FUTURE NOW: RECONFIGURING REALITY 11 Ray Dalio, the CEO of the world’s largest hedge fund, has tasked his researchers at Bridge- water with creating an intelligent agent that can reliably apply his personal principles to the firm’s Metaphors to expand possibilities decision-making process: hiring and firing workers, pursuing investments, and navigating inter- What Lies nal conflicts. Replika represents another early foray in this direction, crafting your algorithmic Beyond the Virtual doppelgänger by mining text chat conversations. Assistant? Why you may want to work with the Surrogate: What better way to broaden your empire or extend the reach of your brand then employing a small army of digital clones for product pitches, contract negotiations, and press junkets? by Dylan Hendricks Why you may be wary: With the proliferation of personal data online, there’s little to stop third par- ties from creating Surrogates without consent, unleashing an army of impostors to spearhead fraudulent schemes or competitive ventures.

2. The Consultant: The Efficiency Czar That Never Sleeps The age of the intelligent agent is upon us, but the There’s nothing that intelligent agents under- true consequences of their emergence remain stand so well as the goals and functions of logical largely unexplored. Is Siri just another genre of systems. For companies not proactively develop- software interface, or do the technological off- ing their own artificial intelligence (AI) applica- spring of Alexa and Google Now portend much tions, a visit from the Consultant may represent broader transformations to our workforces, the most obvious strategic investment of the next bureaucracies, and social institutions? To fully decade: assessing processes, identifying efficien- grasp the nascent possibilities of our digital cies, and laying the groundwork for the introduc- helpers, we must first expand the metaphors we tion of their algorithmic peers. employ to make sense of them. What lies beyond Urban planners in the cities of Pittsburgh the “virtual assistant,” toiling away at our driving and Hangzhou have already experienced the directions and calendar schedules, and just how benefits of algorithmic consulting in their traf- will tomorrow’s intelligent agents break through fic grids, enjoying 25 percent fewer traffic snarls the silicon ceiling to realize their potential? with intelligently directed traffic lights. Google We’ve identified five algorithmically en- similarly discovered 40 percent reductions in hanced roles that companies and institutions data centers cooling costs by enlisting the help of all sizes should prepare to engage with in the of their own DeepMind program. Meanwhile, JP coming decade. The future for intelligent agents Morgan Chase and Japan’s Fukoku Mutual Life is so bright, Alexa may need to dim the smart corporation have begun outsourcing the vetting lights and order herself a new pair of shades. and auditing of customer contracts to high-speed algorithmic bureaucrats.

1. The Surrogate: Your Digital Clone Army Why you may want to work with the Consultant: As Virtual assistants may help lighten the burden of the landscape of intelligent agents evolves, Con- daily logistics, but offer no reprieve to the fun- sultants won’t offer strategic differentiators so damental constraint on productivity: time in the much as helping maintain minimum viability in day. The surrogate promises to chip away at that an AI-rich world. historic obstacle by formally representing your interests and making decisions on your behalf.

12 INSTITUTE FOR THE FUTURE Make the Future Now

Why you may be wary: As the lore of the intelli- processing to identify works written by the same gent Consultant grows, the algorithmic snake authors based on subtle patterns in their writ- oil salesmen won’t be far behind, offering fan- ing—to the detriment of would-be plagiarists. In tastical outcomes that are often too ambitious the UK, police have embarked on an algorithmic to be realized but too compelling to completely project called VALCRI to analyze crime scene ignored. data and suggest plausible hypotheses.

3. The Ambassador: Brokering Why you may want to work with the Detective: It’s your Relationships We Didn’t Know We Had personal private eye, minus the chain-smoking While humans in the near future may spend more and cynical one-liners. of their time interacting with bots, intelligent agents will also play a growing role in helping Why you may be wary: Machine learning sys- us communicate with each other. For bridging tems often produce correct results that can’t cultural gaps, or aiding large multi-stakeholder be explained in human-understandable terms. efforts, you will find no moderator as patient or Even if you know who committed the crime, objective as the intelligent Ambassador. the inscrutability of the Detective’s algorithmic Beyond the impressive language-translation evidence trail may pose legal dilemmas with no technologies recently developed by Microsoft, easy solutions. Google, and others, Kore is developing bots that can translate the goals and context of large enter- 5. The Oracle: Digital Prophet or Internet- prise systems for regular people. The Swedish AI Connected Ouija Board? firm Gagai AB has taken this one step further by At IFTF we always say, “No one can predict the announcing a plan to create dolphin-to-English future.” But what if intelligent agents can? The translation capabilities by leveraging an exten- Oracle promises to question our innate assump- sive database of recorded dolphin sounds. tions by making bold predictions based on vast data sets and the hidden patterns within them. Why you may want to work with the Ambassador: As We’ll likely never understand the basis for these corporations become more global in their reach, predictions, but that won’t always stop them the importance of nuanced cross-cultural com- from being accurate. munication has never been more important, Researchers at the Illinois Institute of both inside the company and out. Technology have proposed a machine-learning framework for predicting Supreme Court deci- Why you may be wary: To date, subtle social cues sions based on the language used in proceed- have not been intelligent agents’ strong suit. ings, while the UK’s Medical Research Council While this is likely to change over time, intelli- is pursuing an AI framework for predicting gent agents will need a few more years to grow when hearts will fail from ambient data sources. out of their socially awkward phase. Scouring satellite data, Descartes Labs recently announced a project to predict global food 4. The Detective: Connecting the Dots a shortages well before they happen. Million Times a Second The mythos of the private detective has loomed Why you may want to work with the Oracle: In this large in western culture for many decades, inves- hyper-competitive, hyper-global world, every tigating personal infidelities and burning shoe conceivable edge makes a difference. Everyone’s leather on open-ended mysteries. In the near appetite for eccentricity is sure to increase if the future, hiring intelligent Detectives to solve life’s Oracle’s projections for future demand and con- complex riddles may become as second-nature ditions are right even some of the time. as looking up trivia on Google and Wikipedia. A Baidu-produced AI system in Chongqing Why you may be wary: The stock market has recently reunited a family with their long-lost already demonstrated how quickly algorithmic son by algorithmically aging his boyhood face decision-makers can be spooked by erroneous and identifying him as an adult among thou- data. As our faith in the Oracle’s mysterious sands of hours of video surveillance footage. The power grows, the implications of faulty conclu- Emma Identity project uses natural language sions become more dire.

FUTURE NOW: RECONFIGURING REALITY 13 Cutting the Akashic Records Reality is becoming indexed, stored, and searchable

by David Pescovitz

Science fiction author and futurist Arthur C. records that are stored in the digital aether of Clarke famously said that “Any sufficiently the Internet. Pervasive networks of sensors, from advanced technology is indistinguishable from microscale environmental monitors to wearable magic.” Indeed, the smartphone in our pockets computers to ubiquitous surveillance cameras, is a magical device. Entering a high-resolution are recording our world at myriad scales. We virtual world is a magical experience. And the share our life experiences on social media. Every Internet of Action’s power to reconfigure our interaction we have becomes a data point. Mean- perception of reality is a magical process. Con- while, emerging neurotechnologies can detect sidering how these kinds of technologies will our emotions while machine learning systems evolve over the coming decades, or centuries, recognize people and their behaviors in the phys- is nothing short of astonishing. Thinking about ical world and tag them with metadata. Reality is the future through the lens of magic can help us becoming indexed, stored, and searchable. get our heads around the possibilities that were Yet the Akashic records are not just a high- once considered by many to be impossible. definition collective consciousness. They’re also Consider the Akashic records, an idea devel- described as an oracle of sorts. And while nobody oped in occult philosophies during the late 18th can actually predict the future, we can simulate and early 19th centuries. Akasha is a Sanskrit and visualize how it could play out. Enabled by word from traditional Indian cosmology that breakthroughs in complexity science and super- refers to the “aether.” In the late 1800s, writ- computing along with the endless streams of data ers in the occult philosophy movement called populating our nascent Akashic records, we’ll “theosophy” posited the idea of the “Akashic simulate multiple tomorrows at increasingly records,” a compendium imprinted in the aka- higher fidelity. sha of all human experiences from the past, Sure, a real system of Akashic records is still present, and future. According to theosophists far off on the horizon. But the questions that the like Rudolf Steiner and CW Leadbeater, the concept raises around privacy, equitability, con- Akashic records can be accessed through cer- trol, and access are relevant right now. Grappling tain states of altered consciousness. A far out with those issues doesn’t require magical think- idea, for certain. ing. But it does demand an open mind, as does Yet in the last few decades, we’ve begun to looking at tomorrow’s technology through the create a kind of primitive system of Akashic lens of magic.

14 INSTITUTE FOR THE FUTURE Forecasting 2027

Universal Basic Assets Rethinking who owns our digital assets

by Bradley Kreit and Jessica Cussins

when everybody worked in large companies, and all of their benefits were tied to that employment.”

2. These factors are transforming the underlying value of assets by “making it possible to produce goods and services in abundance without IFTF’s Executive Director Marina Gorbis spent employing large numbers of workers. As a result, years exploring and making sense of how tech- the system that worked relatively well under con- nology will transform work and organizations. ditions of scarcity is poorly suited to fulfill the In her 2013 book The Nature of The Future: Dis- needs of many when products and knowledge patches from the Socialstructed World, Gorbis can be produced in abundance by relatively few.” explored how peer-to-peer production is upend- ing many traditional institutional formats. Since 3. These forces are likely to exacerbate current the publication of her book, she has been work- inequalities and destabilize social systems. Even ing with policy makers, civic leaders, and execu- for the wealthy, extreme inequality comes with tives to rethink many of our assumptions about extreme costs. “Historical research shows that the nature of work and organizations. concentration of wealth and power requires In her new work, Gorbis is again pushing investments in vast networks of expensive secu- boundaries, challenging us to reimagine the rity institutions leading to what Dr. Rachel Klein- social safety net to ensure productivity gains feld calls ‘privilege violence.’ Some have to build from emerging technologies minimize damag- protective bunkers on secluded islands as they ing social outcomes through a concept she calls prepare for the inevitable social upheavals.” “universal basic assets.” This extends current discussions around universal basic income to a 4. Technological shifts are fundamentally remak- larger set of assets that everyone should be enti- ing what we can consider to be an asset. “Land tled to as technical advances supercharge pro- was once the primary asset, then it became jobs ductivity. Here are four of Gorbis’ observations with benefits. Now, to obtain economic security about the future of universal basic assets. and prosperity, you also need a whole new class of assets: data, access to AI, and software. Uni- 1. Over the next decade, we’ll start to see the versal basic assets takes into account access to emergence of machines—such as autonomous traditional, physical, and financial assets, as well cars—that do not simply drive themselves, but as the growing pools of digital assets. We also manage and monetize themselves. “The problem need to find new mechanisms for assigning value with that is not so much that it’s a race against the to activities like caregiving, creative work, and machine, but a race against the institution. Our whole civic engagement that maintain our social fabric institutional infrastructure is based on the time but do not currently carry monetary value.”

FUTURE NOW: RECONFIGURING REALITY 15 For centuries, the Western world has conceived of the mind and body as separate. The mind decides, feels, interprets, and thinks. And the body serves as a fleshy transportation device, Embodied Learning sending sensory information to the body and carrying out commands. “Embodied cognition” Immersive media’s educational theory changes all that. It asserts that the mind potential often thinks through the body, using sensing, movement, and physicality as a form of processing. And this way of thinking about thinking suggests that we often learn better by using our whole body to understand, experience, and interact with learning content. Over the next decade, as body tracking and haptic feedback by Alexander Goldman technologies get more precise, immersive media and Toshi Hoo will leverage our embodied cognitive processes in novel ways to supercharge the way we learn everything from playing the piano to coding. Here are some examples of how this approach is already being applied with today’s technologies.

Leveraging Spatial Awareness Altar Virtual is one of a handful of companies aiming to leverage embodied learning by connecting early tools for physicalizing VR to cognitive learning techniques, like the “memory palace” trick. The popular technique for memo- rization uses a form of imagined physicality for learning. In it, you imagine walking room-to- In 1990, Steve Jobs famously promoted the room in your home and putting the information computer as “the equivalent of a bicycle for our you want to remember on bits of the walls and minds.” In some ways, that’s how computers are counters. The very act of imagining walking still understood by most people today: as devices through the space helps you recall that informa- that we use primarily with our brains. But that’s tion with astounding accuracy. Altar and other about to change. Advances in haptics, mixed and companies are creating new virtual environ- virtual reality, and full-body motion capture are ments tailor-made to leverage our natural pro- turning computers into devices for the whole pensity to learn and remember through spatial body. And this has some important implications relationships. for learning and education. Much has already been made about the Tapping Muscle Memory potential for mixed-reality learning. Many have There are several projects that use gesture inter- pointed out that augmented reality (AR) enables faces for teaching to exploit research findings us to overlay helpful educational information that indicate we learn better by engaging more onto places and objects in the physical world, than one sense (i.e., having students physically and virtual reality (VR) could allow students move numbers to solve mathematical equa- to drop into any learning environment on the tions). For example, the Virtual ENVironment planet. But the true potential of this technology Interaction software was designed to teach kids may lie in something else entirely. programming by having them input code by That’s because, at the same time that comput- performing dance moves, instead of typing on ers are shifting from tools for the mind alone to a keyboard. This both creates a more fun and tools for the whole body, we’re experiencing a engaging learning experience and taps into the shift in how we understand the mind itself. link between memory and physical movement.

16 INSTITUTE FOR THE FUTURE Forecasting 2027

Embodying Avatars techniques. For instance, the creators of a spe- Researchers Saadia Khan and John Black have cialized set of eye-tracking glasses conducted a pioneered what they call “surrogate embodied study that mapped where a master pianist looks cognition,” having students use avatars in virtual when they are playing a piano as a way of better worlds to enhance learning. Though the students understanding the unconscious activities that are only manipulating an avatar on a screen, master practitioners engage in. Going forward, because they identify with the avatar, they imag- machines will increasingly track not just our ine themselves physically moving which, neu- movements and conscious expressions, but also roscience shows, activates the same parts of the track things our body is doing that we might not brain as if the students were actually moving in be aware of, uncovering new insights into the real life. Going forward, we could see the “sur- learning process. rogate” removed from the techniques pioneered for “surrogate embodied cognition” as students Rewiring the Brain go from controlling avatars with their hands to Researchers at Duke University conducted an moving through virtual space with their whole experiment using VR to help paraplegics con- bodies. trol robotic exoskeleton legs. After the study ended, researchers were surprised to discover Understanding Unconscious that seven out of eight of their paraplegic sub- Aspects of Mastery jects (who had been using VR mostly to train the Oftentimes, when an experienced performer in brain-computer interface to the robotic legs) had any given craft is trying to describe to a novice actually begun to regain motor control of their the process of executing a task they have mas- real legs. This points to the potential to use VR tery over, they are not consciously aware of some to actually rewire neural pathways in the human of their most important actions. But full-body body, opening opportunities for learning that we capture technology could reveal some of these can scarcely begin to conceive.

FUTURE NOW: RECONFIGURING REALITY 17 Currently, telepresence robots are still in their infancy. Yes, an out-of-town mother can play hide-and-seek with her daughter at home, a physical therapist can show her long-distance dad how to do stretches for his strained back, and Your New a person can save a life by calling 9-1-1 after virtually checking in on an older relative and find- Telepresence ing him unconscious (real examples). But these robots are limited. They have wheels instead of Superpowers legs, so stairs are a problem, and they don’t have arms, so doing chores, pushing an elevator but- Get ready for your robotic upgrade ton, or simply waving hello is out of the question. However, telepresence robots are quickly evolving with more human-like capabilities that will enable a richer experience for everyone involved. by Carla Sinclair Here are three examples of robotic upgrades we can expect to see in the near future:

1. See and Hear Like a Superhero Right now a tutor beaming into a student’s house might strain her eyes to read the algebra work- sheet. And an employee beaming into the office might have trouble hearing what the boss is say- Thuc Vu, CEO and Co-founder of the Silicon Val- ing if the office is competing with outside noise. ley startup OhmniLabs, says his favorite thing to But Suitable Technologies, who makes the telep- do with a telepresence robot is to cook with his resence Beam Robot in Palo Alto, CA, launched grandmother. She is teaching him how to pre- some prototypes this year with improvements to pare Vietnamese dishes he enjoyed as a child. both hearing and vision. He lives in the United States while she lives in “We have a telepresence robot that’ll have Vietnam, but with the Ohmni robot that his super senses,” says Bo Preising, the company’s company makes, “It’s the closest thing to being Chief Strategy and Product Officer. Preising says there in person without physically being there.” these audio and visual “super senses” should be The 35-year-old entrepreneur says that from his on the market in the next year or two. house in California he steers the Segway-shaped One improvement will include their pan-tilt- robot, which sports a large display screen, zoom camera, which allows the robot’s head to around his grandmother’s kitchen. There he move up and down to match the eye level of those can “poke my camera into the different ingredi- around it. “The pan-tilt-zoom is like your eye- ents, pots, and pans and ask her questions.” His ball,” says Preising. “If your head is stationary, grandmother can pour, mix, and add ingredients you can move your eyes around … and then the hands free, without holding a phone or tablet to extra feature, which human vision doesn’t really talk to him, “so it becomes much more of a rich, do so well, is … you can zoom in on something far engaging interaction versus just sitting in front across the room, like 20 feet away, and actually of Skype.” With the growing popularity of telep- be able to read a business card.” Imagine how resence robots, the telecommunication network helpful supervision will be for someone looking is evolving into a “teleaction” network. Meaning at documents at a virtual business meeting, or a long-distance communication is moving beyond long-distance caregiver looking at lesions on a a merely verbal or textual conversation (phone, patient’s skin. email, text) into the broader communicative Robots will also have a better audio system realm of shared experiences. And with lighter with noise cancellation so that people can hear weight, less expensive robots coming onto the with more clarity. “We have an array of micro- market, we can expect to see more telepresence phones that can get rid of all the extra noise in robots not only in corporate, government, and the room,” says Preising. “If you imagine your- academic institutions, but also in private homes. self in a situation where you’ve got a jackhammer

18 INSTITUTE FOR THE FUTURE Forecasting 2027

outside of the house, our microphones can delete 3. Chip in With Housework, Rosie the Robot Style that noise so you’ll be able to hear what’s going There’s an episode ofThe Jetson’s in which Rosie, on in the room better than people who are actu- the robot maid, lights up a cigar for Mr. Spacely ally in the room.” As for people actually in the (George’s boss) with a lighter that is built right room, they will be able to hear the person driving into her hand. That’s the idea behind robots that the robot with more clarity than anything cur- can do chores. OhmniLabs is brainstorming rently on the market. about ways to build a robot that can vacuum, clean the dishes, and do laundry. 2. Pick Up Faraway Objects Thinking outside the box, Vu says that rather Arms for telepresence robots are already a real- than have the robot use its “hand” to pick up a ity, but they’re not available to consumers—yet. tool such as a scrub brush, the robot would have Suitable Technologies has robotic arms at the built-in (or attachable) tools. “Instead of having company that can manipulate objects and pick the robot pick up a vacuum cleaner and move it things up off the floor. But, according to Preising, around the home, we can build the vacuum right “They are still very expensive for most people,” into the robot, in the base.” and the user interface is still a bit challenging. So if your college kid runs out of the house To accelerate the development and affordabil- after leaving a pile of dishes in the sink, you’ll ity of the arm, OhmniLabs has invited develop- be able to tell him to march right over to his lap- ers from around the world to brainstorm with top or smartphone and clean up his mess. And them as part of an open-source community col- maybe you can even hang out with him in the laboration. “The ability to pick up an object is a kitchen while he’s at it. key capability we are working on,” says Vu. “We While Priesing pragmatically says this is are working on a lightweight but also affordable something that might be available in 15-30 years, and highly capable arm to allow the robot to pick Vu says that he could see a housekeeping telep- up and manipulate objects in the real world.” Vu resence robot, like the robotic arm, ready in five says that it’s absolutely possible to achieve this years. in five years.

FUTURE NOW: RECONFIGURING REALITY 19 Your drones fly in beautifully coordinated, synchronized ways. How you are able to achieve this? We were able to get 49 small [drones] flying together, which is, to my knowledge, the first time that’s been done indoors, basically by using an external motion capture system to determine where the robots actually are and then relay that Teaching Robots info back to the individual drones to help them How to Work navigate. A majority of the computation is actually done on board. The drones get information Together about their location from the server that is running the motion capture system, but the updates A conversation with are not frequent enough for stable flight since swarm robotics pioneer there are so many drones and so few radios. So we fuse information from their on-board accel- erometers with the updates from the motion capture system. Nora What about the robots’ behavior? It seems remi- niscent of a school of fish or birds. Ayanian In terms of the large groups of robots moving in synch, that may be. And there are people in robotics that study the movement of birds Interview by and fish. But those animals don’t do the things Mike Liebhold, Mark Frauenfelder, David Pescovitz that I want my systems to be able to do. I work normally on smaller groups of robots where we have specific tasks that we want them to accomplish and they are not all doing the same thing … I was thinking about how these robots could be In a video created by the Automatic Coor- better and asked, “Why is nobody studying how dination of Teams Lab at the University of people interact with each other?” If you’ve ever Southern California, a set of 49 drones line worked in a group with others, we usually allo- the floor of an empty studio. They lift off in cate tasks based on people’s skills. Tactics like synch, never breaking the shape of a perfect this can be used by robots as well. And the nice square, and proceed to soar, dive, and twirl thing about humans is that you can ask them, like synchronized swimmers. Then, suddenly, “How did you determine that you should do they break formation and rearrange into the this and the other person should do that?” And letters U-S-C. This demonstration appears, they’ll actually answer. That’s how I got to one on the surface, similar to the one put on at of the problems that I’m working on now: How this year’s Super Bowl. But the small size of do you learn from human coordination in order the robots, the precision of their coordinated to get robots to coordinate better? For example, movements, and their ability to navigate in- we want to learn how diversity helps humans doors represent substantial breakthroughs. solve complicated problems, so that we can allow And the future applications of this technolo- robots to take advantage of diversity in teams as gy extend far beyond entertaining spectacle. well. We are also interested in how humans solve Multi-robot coordination could revolutionize problems that robots encounter, with only the everything from farming to disaster response. sensor information that robots would have. By IFTF caught up with the Lab’s director, Nora working on these projects, we hope to get teams Ayanian, recently named one of MIT’s top 35 of robots out of the lab and into the real world, visionaries under 35, to get her perspective doing things like environmental monitoring and on where this could all go in the next decade. precision agriculture.

20 INSTITUTE FOR THE FUTURE Forecasting 2027

Are there practical applications of multi-robot tion that’s available online, such as maps and geo- coordination that are mainstream today? physical data, or to exchange information with Absolutely. It’s been used in manufactur- other connected devices, like mobile phones. ing for a long time and in warehousing. That’s how you get your packages in two days or less. What else do you see as changing in meaningful They’ve basically turned the warehousing par- ways? adigm upside down. Instead of people going I think that having a unified way of interacting and picking the packages off of the shelves, the with these devices might be the most interesting shelves actually come to the people—each of the change. Right now, everything has its own spe- shelves is carried over by a robot. Multi-robot cific mode of interaction, communication, and systems are also working in mining, so there’s operation. Over time those will all, potentially, lots of these systems at work today; it’s just that merge into some standard, where we all interact not everybody knows about all of them. with these devices in the same ways. And that will lead to a lot more innovation in what these Where is this going? What does a future of abun- devices can do. We really only touch the tip of the dant computing power mean for your work? iceberg in what multi-robot systems can do today. It means a robot can operate without external influence. It can make decisions, take large What would this world look like? What could you stacks of data and figure out what they mean, and imagine in the next decade? decide what to do. There is certainly a limitation I imagine a world that’s like The Jetsons, where now—anything that requires a large amount of we have robots all around us and they are not data analysis is impossible to do on board for necessarily part of one multi-robot system, but some of the robots, especially the flying ones. instead, there are multiple different systems of The ability to do more on board means that they robots. There are big problems that we can make can be more self-sufficient and rely on the cloud a huge impact on—like saving lives by containing less or use the cloud in a more interesting way— or putting out forest fires or search and rescue. for example, instead of using it to crunch data a But they will also just make our lives a lot easier. robot needs on its own, the cloud can be used to One day having Rosie the Robot will be kind of share information across robots, to get informa- normal.

FUTURE NOW: RECONFIGURING REALITY 21 What is a bot? When I speak about bots in the context of the Computational Propaganda Project, or my other work on disinformation, what I’m usually speaking about is social bots, which are a special type of automated software program that runs a profile on social media or that automates a profile on The Rise of social media. We coined the term “political bots” and we’re interested in social bots that do politi- Computational cal things online.

Propaganda How does a bot actually work? How does it write coherent sentences, and craft messages intend- A conversation ed to promote a specific agenda? with IFTF Fellow for Good There are different ways a social bot can be constructed. One of the things you can do is build a bot that accesses repositories of information, so the bot can be linked to a series of phrases that Sam the programmer has pre-written. The bot can say, “What do you think about this?” And then link to an article. Maybe the article’s from Breitbart, Woolley or from MSNBC, with the goal of sharing particular news. That’s a fairly rudimentary way of programming a bot. The other thing you can do is Interview by Mark Frauenfelder build bots that aren’t meant to communicate on the front-end with people at all. Rather, they’re built to do what we call “passive interaction” with particular profiles. Those types of accounts do On January 17, 2014, Girl 4 Trump USA joined nothing but retweet content on Twitter. They’re Twitter. She was silent for a week, but on Jan- built specifically to be sold to a bidder that then uary 24, she suddenly got busy, posting an uses them to retweet their content. average of 1,289 tweets a day, many of which were in support of U.S. President Donald How effective are bots at fulfilling their purpose? Trump. By the time Twitter figured out that There’s certainly a spectrum of effect. During Girl 4 Trump USA was a bot, “she” had tweet- the U.S. election, for instance, bots were able to ed 34,800 times. Twitter deleted the account, infiltrate the highest levels of social media influ- along with a large number of other Twitter ence by interacting with accounts that we know bots with “MAGA,” “deplorable,” and “trump” to be human accounts, and that we know to be in the handle and avatar images of young retweeted and liked and followed and interacted women in bikinis or halter tops, all posting the with by lots and lots of different people. Those same headlines from sources like the Krem- accounts often interacted with bot content, lin broadcaster RT. But Twitter can’t stop the retweeting bot-related content. The bots were flood of bots on its platform, and the botmak- active members of that person’s social sphere and ers are getting smarter at escaping detection. network. We know that bots can absolutely have What’s going on? That’s what Sam Woolley an effect on the communication processes that is trying to find out. Woolley, who joined Insti- happen during an election. They can be used to tute for the Future as a Research Director, was inject information into the dialogue. the Director of Research at the Computation- Another way that bots can be used is to take al Propaganda Project at Oxford University. up a particular conversation topic or to support a We asked Sam to share highlights of his re- particular person, candidate, or idea, in order to search showing how political botnets—what create something I call “manufactured consen- he calls computational propaganda—are be- sus.” Basically, the bot, or botnet—which is a col- ing used to influence public opinion. lection of social bots in this case—is massively

22 INSTITUTE FOR THE FUTURE Forecasting 2027

boosting a topic. The botnets leverage their com- wrote about his experiment buying a really cheap putational advantage to post thousands of times Twitter following. Over time, the bot network fell faster than a human could, boosting a hashtag off, but lots and lots of humans also followed or a topic or a person to make them look much him, so at the end of the day, even though most of more popular than they are. And oftentimes, the bots got deleted, he had a much larger follow- what we’re seeing is that Facebook’s news feed ing because of the illusion of popularity. will pick up that topic because it thinks it’s real The more expensive networks, which you traffic and shows it to regular people. can purchase on the Dark Web or using other mechanisms, including hiring contractors that There’s an arms race between bots and people offer this as part of their kit, can range in the tens who want to stop them. of thousands of dollars. Each bot persona will Yeah, there is. It’s absolutely an arms race, have its own Twitter profile, Facebook profile, a and to be frank with you, oftentimes the peo- LinkedIn account, and various other accounts ple who are building these accounts are a step associated with it, and the bots will be updated or two ahead of the people attempting to detect and run by people. them, including the social media companies, but also researchers like myself and my team. How tied in are these bot developers with orga- It can be fairly frustrating. The more common nized crime, particularly in Eastern Europe? ways of detecting bots, i.e., looking for really They are quite tied in with organized crime. low follower numbers, but really high followed One way we can begin to understand bot net- numbers, or looking for no Twitter picture, are works is to look at money laundering. In Eastern increasingly becoming obsolete, because people Europe, the people who are building these bots now understand the most basic ways of detecting are supported through nefarious means. But a bot. there’s also a lot of tie-ins to Southeast Asia, and South America. You can look at this guy, Andrés Is it expensive to hire a bot army ? Sepúlveda, who told people that he had worked It ranges. You can buy an unsophisticated for all different governments to sway elections bot army on Fiverr for a really small amount of throughout South and Central America. He’s money. They will support you, but probably the gone on the books about how that works, includ- accounts will be suspended quickly. There’s still ing using Twitter bots that are absolutely tied in value in them. My former colleague, Gilad Lotan, with the criminal underside.

FUTURE NOW: RECONFIGURING REALITY 23 Once created, these IoA systems will fulfill those desires on behalf of the customer, at the most appropriate time and place. Agents will relieve the customer completely from the need to fret over pesky things like shopping to buy Post Money things, managing money (financial literacy is waning anyway—would many be that sad if this The basket of desires went away?), and having a job (we are already and how sellers can tap it seeing massive shifts from having a career to a gig, and finally, to a task-based, algorithmically-delivered economy). Working for money will be replaced by completing tasks to acquire items in the basket. Money, as a concept, will no longer be visible, or even useful, to the individual. Let’s take a look under the hood. What hap- by Chris Kalaboukis pens when a happiness agent decides it is time to purchase an item in its owner’s basket of desires, and deliver it to them at the right time and place? It starts by throwing a request (deliver this item to this place at this time for this price) out into a global virtual marketplace, where the happiness agent and a multitude of provider agents negotiate toward a mutually beneficial result. Except People don’t want products or services. They in the case of an extremely rare item, the trans- want their desires fulfilled. People will increas- action is completed in nanoseconds. Hundreds ingly outsource this important job to software of thousands of provider agents will bid to fulfill happiness agents devoted to discerning and the customer’s desire. The customer won’t see or fulfilling their every desire. Smart sellers will think about the cost or transaction details. market directly to these agents as suppliers of happiness, which almost incidentally will come in the form of goods and services. IoA systems will fulfill those desires In this future, the Internet of Actions (IoA) on behalf of the customer, at the will identify everything that makes an individual happy, using detected preferences, explicit most appropriate time and place. and implicit communications, and social circle Agents will relieve the customer behavior. The data will be captured by sensors, completely from the need to fret over data harvesting, and tracking, and run through algorithms that use collaborative filtering and pesky things like shopping deep learning to generate a private, happiness wish list, a secret accounting of everything that If you are a goods or services provider, how they want and aspire to attain, their “basket of will your agent be able to get the sale, if the price desires.” is always the same, down to the nano-penny, This basket contains physical objects that they from every other provider agent? If it knows the may want, such as an antique—or antique-look- contents of the customer’s basket of desires, then ing—distressed brown leather messenger bag. it could offer the customer agent a “sweetener,” It also contains experiences they would like to something else that the customer may want, that have—maybe a trip to Thailand to visit Phraya can be thrown in, from your network of partners, Nakhon Cave. It could even contain emotions to clinch the deal. Here’s the scenario: they’d like to experience, real or virtual—(a Sophia is a gig worker who completes tasks woman contemplating motherhood, may wish for hundreds of companies, people, and agents. to experience emotions such as the joy or fear of They don’t pay her in cash, but in a percentage childbirth in VR before she decides to actually of value toward a new object or experience in go through with it). her basket. In her basket, among other things,

24 INSTITUTE FOR THE FUTURE Forecasting 2027

she has a new VR rig, a Versace dress, and a trip time. There will almost always be a seller agent to Italy. When her washing machine is about to who will come to a profitless price, and if other run out of detergent, her basket asks her agent to seller agents match that price, they will not make get some. The agent asks retailers and brands to the deal. In those cases, the seller agents can ask provide the detergent and negotiate an exchange the customer agent about something else in the of value. In an instant, they all provide the best customer’s basket. It’s the equivalent of a sales- possible price for the detergent, each of which person asking, “Can I get you anything else to are nano-pennies apart. The customer agent make this sale happen right now?” The customer then throws out requests for sweeteners to the agent may then reveal that the customer is plan- deal, to see if any of the retail or brand agents ning a trip or something else from the basket. can enhance any of the upcoming experiences The seller agents then mine their partnerships in Sophia’s basket to win the business. The pro- for something to offer to the customer agent to vider agents search for a way to help Sophia real- tip the deal in their direction. They may come ize a desire. One provider agent returns with an up with a plethora of deal sweeteners, and in the offer for a free fiber optic cable toward her new end, the customer agent will select the one that it VR rig and another offers a free ice cream cone feels its customer will appreciate the most, based at Palazzo del Freddo Giovanni Fassi in Rome, on the priority of the items in the customer’s bas- for her trip. Sophia’s agent opts for the ice cream ket of desires. cone, and the deal is made. Later, when Sophia In this future, sellers can no longer compete has a free moment to browse through her upcom- on price—they will need to maintain networks ing trip to Italy she discover the ice cream cone of collaborators who can provide that deal sweet- voucher. She can trade that cone’s value for ener, at little to no cost to the seller, to make a something else, if she likes. sale for any kind of profit. In this world, customer and seller agents To win, sellers must look beyond price, ser- quickly come to a static state on price, place, and vice, and even product, and deliver happiness.

FUTURE NOW: RECONFIGURING REALITY 25 When physical things acquire digital twins, and they become entangled with sensor data, new possibilities emerge:

1. X-ray Vision Digital Twins Things in the real world are subject to variations in heat and humidity, mechanical wear, and other What if we could get better answers environmental insults. Dashboards with sensors from what-if questions? and gauges on machines give us a limited key- hole view, but digital twin technology gives us 360-degree X-ray vision to see what’s going on under the hood. The digital twin becomes alive when it’s fed real-word, real-time data. Service teams can look at a digital twin on a display and see all the components and their vital signs. If by Mark Frauenfelder a bearing is chattering and hot, it will appear bright red on the twin, prompting the service team to replace the bearing before it leads to a costly catastrophic failure. Digital twins will be combined with predictive algorithms to automatically generate preventive maintenance schedules, which will help avoid equipment downtime and customer dissatisfaction. “What if we reduce the enclosure thickness two millimeters so we can make room for a big- 2. Increased Productivity ger battery?” When Maserati was designing its Ghibli sports “What if we shut down the air conditioning car, it used digital twin technology to connect on the factory floor when people aren’t present?” development engineers’ designs to manufac- “What if we tell our customers to perform turing engineers’ assembly factory designs. In oil-system flushes on leased equipment less fre- essence, Maserati created a virtual car and vir- quently than we currently specify?” tual factory, which could accept and respond to “What if” is a common question in every orga- data produced by their physical prototypes. As a nization, because it’s the first step in optimizing result, the time-to-production dropped from the existing processes or developing new ones. There typical 30 months to 16 months. are four main ways to answer a what-if question: A Siemens factory that makes computer- We can rely on our imaginations. We can use our control systems in 1,000 different varieties has a experience and historical data. We can run tests nearly 0 percent defect rate, in large part because and analyze the data. And we can develop models the entire factory has an identical digital twin and simulations to see what happens. that Siemens uses to design and test components Often, using one or more of these methods and assembly equipment before the physical fac- can provide useful answers. But we won’t learn tory takes over. Today, some factories are requir- if the decision we make is good until we try it in ing vendors to provide digital twins along with the real world. That’s because these methods— the components they supply. visualization, experience, tests, and models—are more or less disconnected from the real world. 3. Improved Products Here’s a what if: What if you could connect Smart connected products are giving manu- real things to virtual counterparts? If you had facturers unprecedented insight into how their sensors attached to things, you could send the products are being used. Incorporating digital sensor measurements to software models of twin technology into these connected vehicles, those things. You could then tune the models so machines, consumer electronics, robots, and they respond just like their real-life counterparts other manufactured equipment will change the do. They become “digital twins” that operate in nature of the buyer-seller relationship. Think sync with their physical siblings. of a construction company that buys or leases

26 INSTITUTE FOR THE FUTURE Forecasting 2027

backhoes from a heavy equipment manufacturer. 4. Optimized Health When the backhoes are outfitted with sensors People (and other living things) can have digital that measure engine temperature, loads on mov- twins, too. Like an advanced Fitbit, your digital ing parts, vibrations, component failures, noises, twin will know when you sleep and exercise, and so on, the data will be wirelessly transmit- what you eat, your hormone, cholesterol, and ted as streams of data in real time to digital twin blood sugar levels, and more. Your digital twin’s backhoes that respond in a realistic way to the virtual physiology will respond in a realistic way data. The customer, as a data provider, and the to these measurements. You will be able to press manufacturer, who uses the data to make better the fast-forward button to generate your extrap- backhoes, become partners who share a common olated health profile and identify problems down goal—developing the best backhoe possible. the road (“Type 2 diabetes in 3.5 years”). Like a At the 2015 LiveWorx Technology Conference retirement income calculator, the system will in Boston, the Internet of Things (IoT)/augmented prescribe a customized dietary and activity regi- reality platform provider PTC demonstrated a men to ensure optimal health. bicycle equipped with various sensors that fed data to a digital twin bike, which was shown to The Future of the Digital Twin the audience on a screen. The digital bike moved In 10 years every connected thing will have and responded in sync with the physical bike that a digital twin pulsing in the cloud. It will be a rider was pedaling on stage. “The digital twin connected to its physical twin, mirroring its is a replication of the physical product,” Mike behavior. The digital twin will be replicated and Campbell, PTC’s Executive Vice President of the connected in new ways to other digital twins, to CAD division, said at the event. “Today, as an be used in meta-simulations to test new supply engineer, when I design a product and send it out chains, factory floor plans, and business mod- in the market, it sort of goes to the dark side of the els. Digital twins will be connected to securi- moon. I don’t really know how it’s used. Unless ty-focused “digital ghosts” that replicate control something breaks and the customer gets angry, systems and identify hackers’ attempts to spoof I don’t know what happens. A smart connected sensor readings and disrupt systems, as well as product gives me a chance to understand how be able to safely run penetration tests. The dig- the product is being used in the real world and ital twin will be the foundation of the ultimate to further optimize the product to make sure it what-if system, providing answers that come is meeting the requirements and is successful in closer to being right than ever before. the environment where it is being used.”

FUTURE NOW: RECONFIGURING REALITY 27 of new technologies bypass traditional forms of accountability,” says Madeleine Elish, an anthro- pologist at the Data & Society Research Institute, “including sometimes legal accountability.” AI is increasingly used to make judgments about Don’t Fear the people: criminal risk, credit worthiness, and academic promise. Judges, bankers, and admis- Black Box sions officers might make decisions using algorithms they don’t fully understand, and such Interpreting the future of AI algorithms could contain hidden bias. A widely cited ProPublica article reported last year that the correctional software COMPAS overestimated the criminal risk of blacks and underestimated the risk of whites. Even if factoring in race didn’t harm accuracy, we’d probably prefer to avoid by Matthew Hutson doing so. Sometimes self-teaching algorithms use factors that we, for reasons of performance or justice, don’t want them to use, pushing researchers to find methods of peering inside their electronic minds. Transparent algorithms make sense in many other arenas, too. In May, DARPA, the military’s advanced research arm, began a 4-year, $75 In 2013, hackers installed malware onto serv- million-dollar program on explainable artificial ers belonging to the retailer Target, eventually intelligence, or XAI. They’ve contracted with stealing the credit and debit card information of eight academic and four corporate labs to help 40 million customers. It’s not as though Target them make the AI analyzing their intelligence wasn’t warned—its FireEye security software or piloting their autonomous robots more trans- generated alerts soon after the initial breach. But parent—and to provide an open toolbox of tech- the alerts were disregarded. Why? As one securi- niques for other developers. AI, they feel, should ty expert told the security news site Dark Reading, be more predictable, correctable, and satisfying “In two words: ‘actionable intelligence’ … The op- to work with. erator/analyst should be able to understand the Programmers have a few ways of making risk.” The risk was apparently not understood. machine-learning systems more interpreta- Target likely receives many alerts, with little ble. The most occult machine-learning systems detail about each. FireEye doesn’t discuss client use what’s called “deep learning”—multi-lay- relationships, but their website does advertise ered networks of simple computational units “machine-learning techniques” for malware inspired by the architecture of the brain. These detection. Machine-learning is a set of tools arti- deep neural nets break input into many small ficial intelligence uses to learn from experience. pieces, send them throughout the network, and But how it organizes its accumulated knowledge pop out a response. What happens in between is is not always interpretable to humans, so you essentially an impenetrable black box. So some can’t always tell why the artificial intelligence researchers emphasize or delete certain features (AI) is making certain decisions. It’s possible of inputs—faces in photographs, say—to see that Target employees weren’t given enough how sensitive the network is to those features, justification by the software to make the alerts and thus, what role they play in its “reasoning.” actionable. Perhaps, if it had been able to explain Other approaches, model the world with more itself better, millions of people would not have interpretable substitutes for deep neural nets, been put at risk. Perhaps we’d all be better off if like decision trees, where you can follow its “if the gadgets around us could tell us what they’re this then that” logic. And others use interpreta- thinking. But do they all need to? ble algorithms (like decision trees) to model not One case, in which it would be nice, is when the world, but the deep neural net, adding an they’re responsible for putting people in jail. “A lot interpretive layer that allows them to keep using

28 INSTITUTE FOR THE FUTURE Forecasting 2027

the net, but then have a simpler approximation in what our best cognitive science tells us about that translates what it’s doing (“if it sees this then the nature of satisfactory explanations.” Accord- it will conclude that”). ing to Subbarao Kambhampati, the president of Which one approach to use depends on the the Association for the Advancement of Artifi- application and the audience. Programmers cial Intelligence, “What is an explanation is not might accept an explanation in the form of code. dependent on how you arrived at your answer. A driver might want his car to talk to him in nat- Explanations are about the other person’s mind.” ural language. Other people might prefer bar So ideally, an AI system aimed at explaining charts, or concrete examples of what the system itself would model not just its own internal pro- might do in different situations. And is the user cessing, but the user’s as well. Great communica- trying to debug the system, or learn something tors know their audience. new about the structure of the world? Communication, of course, is a bedrock of Darpa’s XAI program is addressing these trust, and without trust, AI systems won’t be questions by combining explainable models used. David Gunning, who manages DARPA’s with explanation interfaces based on research XAI program, describes the program’s origin: an in human-computer interaction. One contractor intelligence analyst was telling him about a prob- is exploring the psychology of effective explana- lem she had. “She’s getting recommendations tions and will share recommendations with the from these big data analytic routines, but she has others. to put her name on them, and she doesn’t under- For now, enabling users to correct mistakes stand their rationale well enough to be comfort- based on explanations provides extra credit in a able with that. So that kind of resonated.” nascent field that is still grappling with the basics Researchers at Wharton have documented of machine-learning. What people are calling what they call “algorithm aversion”: even when “explainable AI,” says David Danks, a philoso- people know that a decision algorithm is more pher at Carnegie Mellon who studies psychology accurate than they are, they’re often unwilling and machine-learning, “is very rarely grounded to rely on it if it’s not perfect. They hold it to a

FUTURE NOW: RECONFIGURING REALITY 29 Forecasting 2027 higher standard than they do people. That’s in sultancy Accenture, has worked with airlines to part because they don’t understand how algo- predict flight delays based on their schedules. rithms work and they think people are better Computers make predictions after observing at improving. At the most recent O’Reilly Arti- historical patterns and finding complex rela- ficial Intelligence Conference, Mark Ham- tionships between aspects of flight schedules mond, the founder of the AI company Bonsai, and delays. If airlines could trace those relation- relayed the tale of a firm that decided not to use ships, he says, they could know which aspects a machine-learning system that was 20 percent of their schedules to adjust in order to reduce better than their current method because they delays. “They can take corrective actions.” didn’t want an artificial ceiling—they didn’t Understanding your AI also gives you a sense understand the machine-learning system well of its strengths and weaknesses so you can fur- enough to get under the hood and improve it. ther improve it—and so you’re not caught off guard by any surprise outcomes. “Sometimes a machine can solve incredibly complicated problems,” Gunning says, “but just makes an incred- Perhaps we’d all be better off ibly stupid mistake because it has no common if the gadgets around us sense.” So explanations help us in at least two ways. could tell us what they’re thinking. First, they enhance our relationship with the But do they all need to? algorithm. Imagine you’re deciding whether to follow through on an AI’s recommendation—to, say, investigate a malware alert or to undergo an operation. In some of those cases, you might just Such aversion is a bit unfair to computers, need a plausible-sounding reason behind the because humans are also black boxes. We don’t recommendation so you know the AI isn’t doing always know how we make up our minds, or we something stupid—it increases trust. In others, think we know but we have no idea. TheProPublica knowing a model’s strengths and weaknesses article was alarming, but even more alarming is might lead you to not apply it in certain types of the journal article reporting that judges granted scenarios to begin with. In still others, you might parole at rates ranging from 65 percent just after alter the algorithm, for example, to make it less a food break to about 0 percent just before one. biased. The second way explanations help is by If you asked them to explain their judicial deci- revealing how to change a predicted outcome. sions, few if any, would have pointed to tummy Knowing the key input factors that led to the AI’s rumbles. prediction (like the elements of a flight schedule It’s possible that an explanation doesn’t even it associates with delays) can focus efforts on need to explain anything for it to increase user changing those factors. trust. In a classic psychology experiment, people Whether explanations help to understand asking to cut in line at a photocopier improved the algorithm or the world it models, there’s their chances just as much by adding an empty often a tradeoff between explainability and algo- justification, “Because I have to make copies,” rithm performance. “In a lot of applications, as they did by giving a more informative excuse, you only care about the performance,” Gunning “Because I’m in a rush.” There might be cases says. “The explanation is not so important. So, where a machine could get by with saying, “Do you know, have at it if you’re finding cat videos X, because it’s the best decision.” on Facebook.” Other times, you might choose Making AI explainable doesn’t just help to sacrifice accuracy for the ability to, say, tell reduce unwanted bias and increase trust. It can someone why you’re denying him bail, or why also improve task performance. Freddy LeCue, she should undergo a risky operation. In those a principal scientist in “large-scale reasoning cases, even “just because” won’t cut it. And there systems” at the large-scale management con- are also tradeoffs between profits and rights.

30 INSTITUTE FOR THE FUTURE Forecasting 2027

As some corporations fght to keep their secret silicon sauce under wraps, some lawmakers are fighting to break those seals, as in the European Union’s recent legislation requiring that people be granted “meaningful information about the logic” behind any automated decision made about them. So what’s in our future? Gunning sets the stage: “I’m envisioning one of these bar scenes in Star Wars. Not with people but all kinds of little autonomous systems, a huge range of these things, with some little things crawling around cleaning the floor, and you don’t expect much of an explanation out of them, but there’s your hopefully, really intelligent personal assistant that’s managing your calendar and helping you to set your priorities, and you want that system to give you good explanations and understand the explanations you give back.” It will be a world filled not just with black boxes or transparent boxes, but with a diverse population of boxes, in all shades of translucency.

FUTURE NOW: RECONFIGURING REALITY 31

Maria Konnikova and Kevin Kelly discuss how robot violence could make the world a better places

by Mark Frauenfelder Kill Decision

South Korea has robot snipers at the North battleground? We asked two of our favorite Korean border that can recognize and kill a per- thinkers to discuss the issue. Kevin Kelly is the son from a mile away. They can shoot them dead co-founder of Wired magazine, and the author in their tracks. The sniper-bot factory makes of several books, including the New York Times the robot with the ability to kill what it sees. and the Wall Street Journal bestseller, The Inevi- But at the request of the South Korean military, table. Maria Konnikova is the author of two New the manufacturer modified it so a human must York Times bestsellers, The Confidence Game and enter a password to give the robot the go-ahead Mastermind: How to Think Like Sherlock Holmes. to shoot. From a technology perspective, a killer She is a contributing writer for The New Yorker, robot doesn’t need a human in the loop any- where she writes a regular column with a focus more. What is the role of humans in the robot on psychology and culture.

33 Forecasting 2027

Mark Frauenfelder: Do you think it would ever be right to kill each other.” We’re not very good at a good idea to take humans completely out of the it, by the way, because we will miss. We’re not decision-making loop when it comes to autono- very precise. We’re very emotional. Machines mous combat robots? aren’t going to be accused of war crimes, as long as they’re not programmed into ... they may Kevin Kelly: I actually do. It will help us get rid be hacked into being that way, but in terms of of some of our assumptions about conflict, and legitimate things, they’ll be very rational. They killing people is a way to do it. The process of can be very precise. In fact, they may be able to delegating these jobs to robo-soldiers may, in the hurt someone without killing them. So, they can short term, be horrific, but in the long term, will probably, in the long term—again in their kind make it clear how stupid, and basically immoral of legitimate, un-hacked version—be better at it. it is to kill, and it will be harder and harder for us But we’re saying, “No, no, no. We kind of want to to justify anybody doing it. This is one of the ways do that ourselves.” in which artificial intelligence (AI) and robots will make us better humans. Konnikova: I love your point that, eventually, it will be difficult to justify any kind of killing, but Maria Konnikova: I do think that’s a very good I’m genuinely curious about the interim, because point. My initial reaction was that no, taking we already have these debates about drone war- humans out of the equation is not a great idea, fare, and how that, because it’s not human, you’ve because we’ve seen, over and over, that our abdicated your responsibility—even though ability to create sophisticated technology is humans are piloting them—because of the fac- outpaced by hackers’ ability to hack into that tor of removal? Are you worried at all, that in the sophisticated technology and take it over. And medium term rather than the long term, we’re you have this happening all over the world. The actually going to see more violence, because it second that we invent something, someone will will make it more removed? figure out how to break into it. And so, if you remove the humans that are actually supposed to Kelly: Right, like drone warfare, where the guys be in control, you still have the possibility of sab- are in air-conditioned containers, in Arizona, otage and a lot of different things going wrong, and they’re killing people in Africa, and it’s not and you don’t have anyone who can actually over- face-to-face. ride it. Maybe I’m being alarmist, but I’m never comfortable completely delegating to technol- Konnikova: Right, and if you have these robot ogy, just because I’ve spent years looking at how soldiers, especially if they don’t even have people can take advantage of technology, and do human control, then that makes that problem a take advantage of it. lot worse. If someone can program robo-soldiers to be ethical, someone can hack them and program Kelly: Superpowers, like the U.S., maybe China, them to be “ethical in a different way,” that to us Russia, will tend towards these things, because would be very unethical. What happens if that’s the casualty rates are low, but like a lot of things, the model that ends up winning? I realize that there’s a double standard, because you know, if this does sound dystopian, but there are lots of these armies were coming here killing people, examples in history, and part of it comes from we’d be very unhappy about it. the fact that I did grow up in the Soviet Union So, yes, in the short term, whether we’ll see when it was still the Soviet Union. So I always more violence, I don’t know. I buy Steven Pink- have a mindset that things can get bad a lot faster er’s argument, that in general, the technology than we realize. and the globalization of the world has decreased violence overall, and that trend will continue, so Kelly: Yeah, so the only alternative to this, is to I don’t think that having robo-soldiers will sud- say, “No, no, no. We humans want to reserve the denly change that overall trend.

34 INSTITUTE FOR THE FUTURE Forecasting 2027

If you have robot soldiers coming to kill you, people are not going to be happy. It causes a debate. It says, “Is this what you want? Why do we have machines killing?” And then you have to say, “Well, why do we have humans killing? Why is it better for a human to kill than for a robot to kill? How does that make it better?” So, while there will be plenty of turmoil, and conflict, and short-term massacres, in the long term, this will actually continue to decrease violence in the world.

Konnikova: Even if it falls into the wrong hands? Like Putin or Trump having the technology, and getting mad, and deciding, “Let’s do this,” and now you don’t have human oversight.

Kelly: It could happen once or twice.

Konnikova: But is that not enough?

Kelly: Enough for what?

Konnikova: For mass destruction.

Kelly: Yeah, it’s like we had two atom bombs, and that seemed to be enough. Actually, I’m of the view that both of those were unnecessary, that we would have won the war anyway, but we didn’t continue doing it, because everyone realized that it was out of hand. So we could certainly have some kind of the first use of this being a mas- sacre, but it also being a lesson, if it was done wrongly. But I don’t think that it necessarily has to be that way. We could train robo-soldiers to be ethical, and moral, and better than us, so that’s the difference. It’s like, you’re much more likely to have this disaster by kind of outlawing them.

Konnikova: I hope, I really hope, that forcing more people to consider these issues will get us to a good place. I’m not 100 percent optimistic, just because I’m not an optimistic person, but that’s probably clear.

Kelly: Well, you’re talking to one of the most optimistic people on the planet.

FUTURE NOW: RECONFIGURING REALITY 35 Illustration by Jory Felice How the Internet of Actions will transform everything to do with food

by Max Elder Billions and Billions Transacted

In 1994 a journalist named Josh Quittner reg- person in possession of the domain would do— istered the domain name mcdonalds.com. On he set up [email protected] as his email assignment for Wired, Quittner called McDon- address. ald’s corporate headquarters to find out if they McDonald’s eventually wised up and got the cared. He asked a McDonald’s media relations domain from Quittner. Today, server farms are representative if anyone there could talk to as important to McDonald’s as potato farms. The him about McDonald’s Internet plans. She did hamburger giant has greatly benefitted from some digging and told him later, “I don’t have Internet-based communications—every aspect anything for you, and I probably won’t have of its operations is dependent on high-speed anything for you. I’ve left a lot of voicemail data processing, from commodity market analy- for people, but no one seems to know anything sis to social media marketing. And over the next about it.” So Quittner did what any reasonable decade, the physical components of McDonalds

37 Forecasting 2027

—trucks, shipping containers, grills, employee story of the place: the utensils, the lighting, uniforms—will become embedded with millions the music, and the overall ambiance will all be of intelligent sensors, processors, and effectors related and integrated in unprecedented ways. that communicate with each other and act on Current marketing and food experience design behalf of other machines, people, and organiza- already leverage aspects of context like color and tions. McDonalds, along with every entity that lighting that point in this direction. But in an adds value through coordination, will use Inter- IoA, where objects are animated and connected, net-connected objects and environments to pro- new opportunities will arise to incorporate con- duce well-orchestrated experiences powered by text into food experiences. the Internet of Actions (IoA). Signals of Change Why Food is Ripe for the We can begin to envision how every aspect of the Internet of Actions food system will be transformed into seemingly The experience of food is an ever-changing sen- magical experiences by looking at current IoA sorium of taste, smell, texture, and community. innovations in the food space. It’s the experience of the farmer and soil scientist working together to cultivate a crop. It’s the over- Food with a nose worked volunteer team preparing 5,000 meals •• C2Sense has developed an “artificial nose” a day in a camp for climate refugees. It’s the sensor that detects ethylene, a gas naturally chef working with the biohacker to invent new released as produce ripens. An increase in eth- proteins and the food retailer working with the ylene signals an acceleration of ripeness. (If virtual reality (VR) game designer to reimagine you’ve ever been told to stick an avocado in a how to shop for groceries. All of the areas can be paper bag to help it ripen, that is a DIY attempt greatly enhanced with IoA technology. to trap ethylene.) For decades, meeting the needs of busy peo- •• Smart fruit labels with C2Sense will enable ple scrambling to fit food into their lives has grocery stores to sell fruits and vegetables at driven innovation for new food experiences. But their peak and allow customers to scan pro- we are facing even greater challenges today. The duce via a smartphone app to ensure they pur- political stability of democracies is under threat. chase food with their desired level of ripeness. The climate is becoming increasingly volatile. In the next decade, sensors that monitor ripe- Automation is on the rise, and conventional daily ness of food will be embedded into packaging, rhythms are being disrupted for many reasons. storage units, and grocery store shelves and It’s clear that 21st century food experiences will will sense not just ripeness, but will monitor happen in very different contexts from the way a plethora of factors, like micro and macro- we experience them today. We’ll need to adapt to nutrient levels, accurate caloric content, and these changes by creating new eating and drink- microbial diversity. ing occasions, new pathways and biozones for •• Your grocery shopping experience will become sourcing our food, and new strategies for ensur- highly customized to your preferences. Your ing safe, sovereign food in turbulent times. smart watch will get an alert that the straw- Automation will be a major driver that berries you love are at the ripeness level you rewrites the story and the context of convenient prefer and are available at the bodega around food access. Over the next decade, robots and the corner. Just click the berry icon and they algorithms will transform the way we access will be held for you, or delivered wherever you food on the go. Already, robots are cooking and happen to be at the moment. assembling fast-food burgers, allowing for on-de- •• Your smart cookbook’s built-in nose will sniff mand customization without sacrificing speed the ripeness of your groceries and adjust the and efficiency. Automation is turning grocery cooking time and methods to maximize flavor. stores into super-sized vending machines and food carts into self-delivering food services. As Pricetags that fluctuate by the second platform apps connect people to these services, •• Ripeness sensors are just one way to provide technologies will get smarter, and experiences real-time information about a food product. will become more streamlined and customized. New real-time data collection and analysis In the midst of any eating experience, the tools will proliferate over the next decade,

38 INSTITUTE FOR THE FUTURE Forecasting 2027

We can begin to envision how •• In the constant quest to differentiate, meal every aspect of the food system delivery services will be enabled by the IoA to turn home cooking into a new context for will be transformed into seemingly learning. Meal kit boxes will be embedded with magical experiences by looking AI to not only deliver the food, but also deliver customized cooking advice (like adjusting rec- at current IoA innovations in ipes with an awareness that you don’t buy any- the food space. thing containing sweeteners) and instructions while you’re in your home. •• Paired with your smart kitchen, the delivery which will lead to the electronic and dynamic box will monitor your technique and provide pricing label. On shelves, boxes, or food prod- real-time cooking feedback so that you will ucts themselves, electronic labels enable stores never burn your rice again. Different boxes to dynamically change prices and promotions will tap into the cult of personality behind across every shelf, in every store, everywhere. different chefs, just like the emerging AI assis- Prices may increase as fruit reaches peak ripe- tants today have different tones and intona- ness, or decrease as bananas get soft or the tions. After the meal, the chef-in-a-box will clock ticks closer to closing time for the store. monitor your success and adjust future meals These electronic labels can contain informa- to ensure you’re constantly challenged to mas- tion about total stock availability, reviews from ter new cooking techniques. social media platforms, and geo-located promotions targeted at specific shopping carts. Farm as data centers •• Over the next 10 years, electronic labels will •• The Leafy Green Machine is a vertical, hydro- be connected to more and more smart objects ponic freight farming system built inside a in the store, radically transforming the con- shipping container that is capable of grow- text of walking down a grocery aisle. Cam- ing your typical hearty greens, lettuces, and eras will see what you have in your shopping herbs. The Leafy Green Machine already is cart, and predictive algorithms will anticipate filled from floor to ceiling with sensors and AI what you’re looking to buy, adjusting prices to ensure maximum yield. The IoA will enable for specific products accordingly. Financial freight farms to communicate with each other, information about your purchasing history creating a distributed network of agricultural will be leveraged to adjust pricing based on data centers that can learn from one another, what you’ve been willing to spend in the past, diversify the types of crops grown to fulfill the or even on the price of the clothes or jewelry changing demand of their consumer base, and you’re wearing. tighten the food supply chain. •• The implications extend beyond the walls of the grocery store. Dynamic pricing can help Neighborhood Food provide greater purchasing insight to the local •• Falling Fruit is a global urban harvest map that baker who needs to understand when and how unites foragers, freegans, and urban eaters by much to bake, or to the local farmer trying to collecting data about over a half million free maximize profits by only harvesting when the food sources, including trees, plants, fungi, market is right. and even dumpsters containing available food for public harvest. Meal kits with a chef-in-the-box •• In an Internet of Actions, trees and dumpsters •• Sensor and pricing technologies will be part of will transmit information about fallen fruit or every meal kit delivered to your doorstep. discarded food to adjacent smartphones, local •• In 2017, Amazon filed a trademark patent for chefs, or even your smart kitchen (that knows a meal-delivery service with the motto: “We you need a lemon for dinner tonight). The con- do the prep. You be the chef.” With a targeted text around food procurement resources and acquisition of Whole Foods and their unparal- culture will be crowdsourced and the ingredi- leled logistics of delivery, it’s clear that Ama- ent supply chain will be hyper-localized. zon has plans to disrupt the meal-delivery space.

FUTURE NOW: RECONFIGURING REALITY 39

When smart homes become citizens of common-interest communities

by Kathi Vian Homegrown Governance

The statistics just keep piling up: in government, ing technology may just offer up a solution to we don’t trust. the trust problem. This solution isembedded gov- Around the world, governments are the least ernance, and it anticipates a world where gover- trusted institution in half of the 28 countries sur- nance is built in, technologically, to objects of veyed for the Edelman Trust Barometer. In the daily life. A Nest thermostat might not simply United States, according to Pew surveys, trust in adjust your home air conditioning based on your the federal government is at an all-time low, hav- patterns of use—it might also automatically ing dropped from nearly 80 percent in the 1960s conform to local ordinances governing mini- to about 19 percent in 2017. Government leaders mum thermostat settings. Your smart house thus are barely credible, trusted by only 29 percent becomes a smart citizen. -of people surveyed worldwide. And trust in the T e enabling technologies are the usual sus political wisdom of the American public has pects. Sensors monitor the physical environ- dropped to somewhere between 23 percent and ment. Blockchain-style distributed computing 37 percent, depending on political affiliation. executes contracts and keeps immutable records The distrusters lay the blame on everything from of both good and bad behavior. Artificial intel- corruption, to eroding social values, and to the ligence computationally optimizes outcomes for pace of technological change. a set of stated values. This is hyperlocal gover- Then, it’s perhaps ironic, that rapidly chang- nance—it’s governance you can see and touch

41 Forecasting 2027 and adjust at the scale of neighborhoods. The are automating other sectors of society. Sensors first place it might take hold is in the hyperlocal will monitor everything from noise levels to governance structure, known as “common-in- water use to pet traffic, sending a plethora of sig- terest communities.” nals to smart devices to determine if codes have been violated. Distributed computing will offer Local But Not Always Trusted up smart contracts that use blockchain-style Common-interest communities have been part technology to read the sensor reports and auto- of the American landscape for at least half a matically send warnings to any offenders. If an century. They include homeowner’s associations offender has agreed to the contract, fines may be (HOAs), condominium communities, and coop- both imposed and paid automatically. Finally, erative housing. In 1970, they governed about 2.1 artificial intelligence (AI) will help individuals million residents. They have grown impressively avoid these transgressions by anticipating pos- over recent decades to govern more than 68 mil- sible offenses and automatically adjusting the lion in 2016. All tallied, they number close to appropriate systems. For example, AI systems 350,000 communities today, compared to about might control 4D plumbing materials to adjust 20,000 incorporated towns and cities. the water pressure in a home’s irrigation sys- The governing documents of these associ- tem based on anticipated weather, soil moisture, ations—their constitutions, if you will—are monthly water quotas, and observed patterns known as Covenants, Conditions, and Restric- of water use in a household. Or it might simply tions (CC&Rs), and anyone who has ever bought move a semi-autonomous car to a legal parking a residence with a CC&R knows that they bear space to avoid parking fines. careful study before investing. They govern At the heart of these scenarios are so-called everything from noise levels and landscape trustless systems that use redundancy, trans- design to architecture and parking restrictions. parency, pseudonymity, and immutability to Many of them provide maintenance services as build trust and confidence that rules are being well as the basic infrastructure that municipali- applied fairly. Open records shared on many ties would normally provide. They levy fines for platforms, make these systems work. As in bit- noncompliance and charge monthly or annual coin exchanges, sensor data can be confirmed fees, not unlike the taxes a town might collect. by multiple “observers.” Smart contracts can be They typically have a governing board as well as executed pseudonymously, so that everyone can volunteer committee members. So they’re about be assured that violations are prosecuted with- as local as a government entity can get. out necessarily knowing the violator. No one However, that doesn’t mean they’re always can tamper with records because everyone has a trusted. While the majority of residents report copy and everything is transparent. friendly relationships with their boards, these While such systems have been heralded as a are not generally democratic entities. There are new way to conduct transnational commerce, plenty of HOA horror stories and “bully boards” common-interest communities like homeown- that exemplify the worst of small-pond politics. ers associations are actually a logical starting Enforcement of CC&Rs can be arbitrary and place for giving embedded governance a trial fines can be disproportionate to the offenses. run. American HOAs are currently growing at a Fines have been compounded by collection agen- rate of about 40 percent per decade, and the new cies to the point that they threaten the owners’ housing developments they govern can readily ability to keep their homes, or even to sell them. incorporate new smart home technology that Neighbors can still find themselves at odds, often may eventually underpin embedded governance. with mediators chosen to suit the interests of the Smart home technologies are selling points for board members. So even at this scale, corruption those buying into these new communities, creat- and conflict can undermine the sense of trust in ing personal value for the householder. But can the community and those who oversee it. they provide community value in the form of trusted governance systems? Can smart homes Trustless Systems for Community Trust really be smart local citizens? Enter embedded governance. Embedded governance is automated governance, and it will We Trust In … Contracts and Coins employ the same disruptive technologies that The answer to this question will ultimately

42 INSTITUTE FOR THE FUTURE Forecasting 2027 depend on the values we embed into the code— literally, the computer code—that we choose to A Tale of Two run our communities. Human governance sys- Homeowner’s Associations tems tend to polarize into two camps: the law- and-order camp and the free-to-innovate camp. Different HOAs will use similar technology to prioritize completely different sets of values. Compare, for ex- Both will find much to praise in the embedded ample, the Law and Order HOA to the Innovation HOA: governance tools of the coming decade, but the tone of life in each will provide stark contrast. Law and Order HOA The law-and-order HOAs will tend to empha- Green lifestyle sensors size contracts that enforce restrictions and • Smart plumbing adjusts water pressure to keep automatically punish individual violators. They water use in bounds guarantee that the letter of the law is the lived • Algorithmic light and thermostat controls meet law. Programmed to conform precisely to codes, quotas in comfort smart homes can be very good law-and-order cit- • Landscape service includes monthly AI drone surveillance that identifies troublespots in your izens, but they will tend to produce communities hardscape and schedules early repairs of conformity. Regardless of the specific values of these communities, lifestyles will feel legis- Neighborly contracts • Noise sensors notify noisy neighbors and lated and even enforced by the physical infra- assess noise fees structure of daily life. • Templates help you pretest your landscaping The free-to-innovate communities will tend plans to comply with rules to invest their trust in the “coin.” They will look Automated household maintenance for all the ways they can pool their collective • Smart plumbing, lighting, and appliances resources and leverage individual resources self-monitor and contact the next available and build new wealth for everyone. Their smart repair service for fast, easy fixes homes will need to do more than conform to 24/7 digital access control be good citizens: they will need to connect, to • Smart contracts manage access to your home for build value from the exchange of data (as well as service personnel and caretakers—without any energy and other resources) among households. intermediaries to track your coming and going Every household will be a complex set of finan- Conviviality profiles cial instruments for managing both physical and • Smart matching software helps you find neighbors with shared interests digital resources, and smart homes, as good citizens, will coach human householders to grow • Aggregate profiles of current residents help realtors find the best new neighbors their personal equity while optimizing community goals with everything from neighborhood Innovation HOA power grids to annual household dividends from Local energy via smart blockchain grid community investments. Governance will be • Buy, sell, or trade energy within the commons less about enforcement and more about turning community values into community wealth. Grow basic annual income with global community currency Convert annual allotment of personal coins T e next decade will likely see a great deal • of experimentation with smart home technol- to community coins and spend them across ogy and ultimately with governance strategies a global network embedded in that technology. Earn microfees from tool-sharing The question is: should humans trust their • Store tools and appliances in commons toolshed smart homes, not just to be their proxy citi- and earn hourly microfees automatically zens, but actually to govern their communities? Earn dividends from green lifestyles Common-interest communities offer a natural • Opt in to instrumented sensor grid to earn rewards laboratory for exploring this question system- when the community reaches its targets atically. By thinking beyond the convenience of Household Union home automation to design homes intentionally • International Households Union negotiates best contracts for everything from health and food as complex community governance systems, services to utilities and car services. can we test the viability of contracts and coins, alongside sensors and AI, to organize human societies?

FUTURE NOW: RECONFIGURING REALITY 43 The path to telepathy is coming indirectly, through Jepsen’s exploration of a more cost- effective medical diagnosis tool: essentially an affordable alternative to Magnetic Resonance Imaging (MRI) machines. A current MRI instal- lation costs millions of dollars for hardware and a shielded room, but such equipment is often key to the detection and treatment of cancer, heart disease, and even mental illness. Global inequi- ties in access to MRI technology have created a huge chasm between the haves and the have- A Wearable nots. Simply put, those with access to the technology get more accurate diagnoses and can seek MRI Machine appropriate treatment, and those without, are more likely to die prematurely. Talking telepathy with MRI-quality Imaging in a Beanie After years of working in the display technology field for such giants as Intel, Google, and Mary Lou Facebook (in the Oculus division), Openwater founder Jepsen broke away and formed the new company. Openwater is pioneering ways to cre- Jepsen ate MRI-quality and high-resolution images by taking advantage of how the human body trans- mits and scatters infrared light. Interview by Scott Minneman What if an MRI-quality brain scan could be achieved by wearing something that looks like a knitted ski cap? Or a knee joint’s internals could be examined by a device indistinguishable Spock, the Vulcan character in Star Trek, often from an elastic brace? Such possibilities would performed a “Mind Meld” with other sentient not only eliminate trips to the hospital lab, but beings, creating a telepathic link to exchange also allow individuals and their medical pro- thoughts or probe minds. Science fiction? fessionals to gather images on a more continual Not for much longer! Technology pioneer and basis. Working out of a Quonset hut in Sausalito, luminary Mary Lou Jepsen’s new venture is in Jepsen’s team is doggedly pursuing this break- bringing this mind-reading superpower out of through, from all-hands meetings every morn- neuroscience labs and into everyday reality— ing, to occasional late-night surges, blending an affordable reality, too, using optoelectronic start-up and hard-core physics in equal measure. components and production processes devel- Jepsen’s unique background includes a Sc.B. oped for display in our smartphones. However, in electrical engineering and a doctorate in opti- this powerful ability comes with dilemmas: cal physics from Brown University, plus early Who gets access to whose thoughts and for exposure to computational holography while what purpose? How can we safeguard our- getting her Sc.M. at MIT’s Media Lab. This was selves against possible abuse? followed by a string of novel projects spanning start-ups, academia, and big industry where, along the way, she pioneered tiny optoelectronics at MicroDisplay, served as CTO for the revolutionary One Laptop Per Child (OLPC) effort at MIT with Nicolas Negroponte, and launched the Pixel Qi spin-out from OLPC. She also conceived of a wildly creative (but never realized) technology to project video on the Moon by re-directing sunlight—Moon TV. A named inventor on more

44 INSTITUTE FOR THE FUTURE Forecasting 2027

than 200 patents, she grew up as the daughter of to properly diagnose and treat. Better diffusion a mechanic, and strongly believes that most lab- of diagnostic capabilities is great, but advanced oratories aren’t much more different than an auto health care technology has been a big contributor shop; “Most of the time you’re trying to find the to soaring health care costs in the U.S., Openwa- right tool, or make the right tool,” she said. ter could really change that landscape. Jepsen is painfully familiar with the need “I get an odd comment from medical profes- for affordable MRI. While pursuing her PhD, sionals from time to time, about whether easy she was diagnosed with a brain tumor—one that and inexpensive medical imaging won’t amplify likely would’ve been detected sooner were it not what some see as an Internet-fueled mass of peo- for the expense of brain imaging at the time. ple doing self-diagnoses and bothering their doc- Post-op, she rapidly finished her degree, but the tors with wacky theories about what’s wrong ... As incident forced her to give up her most creative somebody who was very ill for quite a while with endeavors for jobs that provided health insur- an undiagnosed condition that would’ve shown ance and covered the drugs she’s taken to survive up on an unaffordable scan, I don’t have a lot of since then. Her near-fatal experience doubtlessly patience with this take on things. In fact, my reac- piqued a long fascination with neuroscience and tion to that critique is pretty unequivocal—if a medical imaging—many brain tumors aren’t dis- doctor thinks that the public’s engagement with covered until they’re quite large, simply because their own health is a bad thing, then perhaps they it’s too costly to routinely look for them. chose the wrong profession!” “Affordable medical imaging is so critical to A while back, Mary Lou had two “a-ha” quality care, and it’s unfortunately very rare in moments that probably could only happen to many parts of the world. There are something like someone with her unique academic and technol- 40 MRI machines per million people in the U.S., ogy pedigree. One, the human body is essentially but that number drops to two MRI machines per transparent to infrared (IR) light—those wave- million in Mexico, and in Africa, it becomes even lengths can penetrate deeply and even go right more sparse with MRI installations primarily through many kinds of human tissue. Two, the residing in capital cities. The number one health milky images we get from IR light scattering off expenditure in the world is brain disease; it inca- of our tissues can be cleared up with a technique pacitates so many people,and takes a long time from holography. This combination of IR imag-

FUTURE NOW: RECONFIGURING REALITY 45 Forecasting 2027 ing and computational holography lets us view videos for hundreds of hours, collected a library smaller and smaller regions of the brain, deeper of neuronal reactions, and created a “visual dic- and deeper in our heads—eventually discerning tionary” utilizing data analytics, artificial intel- the functioning of tiny regions and even individ- ligence (AI), and brain scan data. These fMRI ual neurons in the brain. readings measured areas where oxygen was For the past two decades, scientists have been flowing (i.e., neural activity in the brain) and also doing a better job of just that: making temporal used pattern matching. Stored data could predict images of brain function by watching for minis- what a student was watching. Understandably, the cule telltale surges of oxygen in blood flow that image was grainy and low-resolution, but recog- accompany neuronal activity. Using a time-based nizable; upping the resolution would enable one variant of MRI, called functional Magnetic Res- to see what people are thinking about and visu- onance Imaging (fMRI), researchers can analyze alizing inside their brains. Similarly, one could neural activity and predict what someone is look- examine the brain’s speech centers in the same ing at with increasingly accuracy. This work has way, and an fMRI could reflect the words that revealed all sorts of details—now we know that someone is thinking. It’s reverse-engineering the brain activity for when we see something or for brain. when we imagine seeing, is essentially identical Not only that, but such technology also is This is where smartphone technology and apparently read/write capable—with the future holography come in. Contemporary liquid crys- potential to “write” changes by stimulating neu- tal semiconductor display (LCD) components, rons—raising the possibility that thoughts and and the demand for high density displays on images can be “written” to your brain that weren’t cell phones and virtual reality (VR)/augmented there before. reality (AR) headgear, have resulted in technology that can capture IR wavefronts emanating The Ability to Read/Write the Brain from the body. Using these LCD components to The thorny and ethical dilemma raised by this capture the wavefront of IR being emitted from revolutionary technology is one of the reasons a person’s body, we can then apply moderately Jepsen left Facebook to pursue Openwater full esoteric mathematics (called Phase Conjugation) time. “Peter Gabriel calls this an ‘Oppenheimer to eliminate the cloudiness from scattering, and Cocktail,’” she noted. “Exuberance about its can capture clear images of tiny structures deep potential, but fear of its misuse—what do we do?” within our brains. The ethical and legal implications were so pro- “You can make a hologram, measure the wave found that somebody needed to talk about them, pattern of the wavelength of light in three dimen- said Jepsen. She felt that she could begin the con- sions, invert that in the screen itself, and then you versation. “National academies of almost every make your body effectively transparent to light. developed country say that one of the top things You can look at the blood flow, the structures, we can do as technologists, is to reverse engineer the tumors—using LCDs literally made in the the brain. But nobody’s talking about what hap- same factories that produce components for cell- pens once we do it.” phones or VR goggles,” Jepsen explained. Tech Futures Lab connected with Jepsen, This technology will most certainly revolu- directly, to get her answers to some of the most tionize health care, giving millions of people pressing questions in this space. better access to a medical diagnosis that was previously prohibitively expensive or not available. How does this read/write technology play out? But in marshalling LCD technology and algo- Even though we aren’t totally sure and aren’t rithms to “read” the brain, this image-capture yet talking in specifics about how these tech- beanie will be able to reveal not only what’s amiss nologies will unfold, it’s likely that our medical with brain tissue, but will literally expose what diagnosis stuff will come first. That’s because the we’re thinking. legalities and ethics are more cut-and-dried— For over a decade, neuroscientists have been there’s little argument to be had about the benefits studying this side of the technology. At the Uni- of an intelligent bra that can spot breast cancer, versity of California at Berkeley, Dr. Jack Gal- but mind reading is a distinctly different animal. lant and his team at the Gallant Lab made fMRI There are lots of incredibly cool upside sce- recordings of grad students looking at YouTube narios. For visual thinkers, the potential of being

46 INSTITUTE FOR THE FUTURE Forecasting 2027 able to somehow think an idea or an image and reverse-engineer enough of what we’re doing and have it externalized is incredibly powerful. sidestep whatever we design. There’s so much in our heads, and getting it out into world is a major bottleneck. The learning What stage are you at with the basic science and side, too—what if we can learn new things by ... development work? getting them put into our brains? We’re not the ones doing the basic neurosci- But there needs to be an ongoing discussion ence work, but there are tons of really fascinating about the ethics of every one of these emerg- studies ... There are some incredible rat maze stud- ing capabilities. Can police or military make ies with implanted probes where they can transfer you wear a cap? What about employers who one rats’ knowledge of how to navigate a maze to require you to be capped for a job? Who owns other rats who’ve never seen the maze. That recent your thoughts? Once you share them, can you face cell work at Caltech where they reconstructed delete them? What about filtering? Have you ever face stimuli with little apparent error with just thought something you didn’t want to say out 100 instrumented neurons—they’re amazing loud? We have to make the technology so that it findings. You were asking about semantic map- only works when we want to ‘think into it.’ It’s ping, earlier—there’s still a lot to learn there, but one thing to get somebody to show you what they some early indications are that we’re not as dif- saw, but what happens as we inch over to telepa- ferent as one might initially think. It’ll be much thy? There are profound issues that crop up once easier to do experiments once it doesn’t take time we can implant thoughts and use it to directly on a multi-million dollar machine. change minds and communicate at the level So we’re approaching a year in—time spent of neuronal activity. The ability to have private exploring a lot of the basic physics. We’ve been thoughts is fundamental. doing experiments on various configurations to All these cases need to be considered and see how deep we can get the light to penetrate, answered, and my doctorate isn’t in ethics. We’ve what resolution we can manage—crucial perfor- been reaching out to the big thinkers in the area mance measures. We’re trying to build and test ... because these debates may take longer to lots of things in parallel, learn from mistakes develop than the technology itself. and combine solutions so we will be prepared to evaluate trade-offs, make design decisions, and How should we manage these issues? choose particular avenues. We’re aiming to show We’re well beyond the days when ethical our working demos to the world, but that will debates like this one take place with a little com- signal our move to start production, and rushing mittee sitting in a boardroom. These debates into hardware decisions can be really costly. need to happen soon in a public forum. Just look Unlike the cutthroat world of cell phones, we’re at how rapidly our expectations and beliefs about an intriguing market with a lot of potential, uti- privacy have shifted, in a very short time. We lizing the same processes as next-gen VR and AR can’t wait until the results of some cutting edge optoelectronics. These same cutting-edge facili- research are upon us before we discuss them. ties and processes may give Openwater a jump on Luckily, we are not the only field with this imitators. We’ve been submitting lots of patents. dilemma. CRISPR-Cas9 (a genome editing tool) Imitators will eventually come. Some may not be is making ethics and policy conversations hap- as careful about ethics. All the more reason for us pen in other communities ... There’s hysteria and to have these ethics discussions widely and early, strident voices around AI, too … so that a broad set of stakeholders can sign on and One fundamental approach we’re taking agree to a set of rights. at Openwater is that we need to keep people in It seems clear that lessons learned from the control of their thoughts. You have to want the Oppenheimer era made a deep impact on Jepsen cap to read your thoughts for it to work. People and her fellow researchers. They aren’t taking need to be able to mask their thoughts, and it’s their responsibility lightly. She concludes, “We our responsibility to teach them how and make have to define what it means to be responsible in sure our system does it. Unfortunately, like a lot developing this, and I can’t completely separate of safeguards, it can probably be eventually cir- the medical part from the telepathy part, but try cumvented (we aim for this to never be the case to use my skills to make the best change in the but must plan for it regardless), or people will world that I possibly can.

FUTURE NOW: RECONFIGURING REALITY 47

A look at the coming world of anticipatory retail

by Scott Minneman Make It Just So

Clea and her 8-year-old son Alex are in a coffee ter of the price tag for the larger more elaborate shop before school. It’s Alex’s birthday. While coffee shop robot. She selects an Iron Man skin waiting for their latte and hot chocolate, they that she knows Alex will like, and selects a deliv- play with a tabletop social robot that the coffee ery time when Alex is home from school. shop has to amuse customers. Alex is charmed Clea’s order is routed to a “fab shop” across by the robot’s engaging antics, and Clea, (who town. It’s a one-person operation owned by a still hasn’t found the perfect gift for her son yet), guy named Ron. He notes the various modules wants to give him one like it for his birthday. She needed for the desired features, and lets the snaps a few photos of the robot, and an anticipa- design software chew on the photos Clea took, tory software agent on her phone asks her if she’d creating the functional layout and surfacing to like to purchase one. It’s $800, which will bust match the coffee shop bot. A quick check-in with her budget. But an assortment of other options Marvel gets the Iron Man finish applied, and Ron are offered, ranging in functionality from clunky hits the “make” button. As the parts emerge from plastic statuettes to fully motorized articulated the machine, he snaps them together around robots, some with only crude sound effects, to a few Tamiya motion, Qualcomm speech, and others with full interactive speech capability. Panasonic battery modules. It goes together Clea notes a brand she bought for Alex’s older without a hitch, and Alex is delighted when a brother a couple of years back and chooses it, delivery drone buzzes into his front yard and along with the all-important joke-library sub- drops his new birthday toy into his hands! scription. The price is right, coming in at a quar-

49 Forecasting 2027

Mass Customization Is Bringing People assemblies (think IKEA, but with a delivery/ Back Into the Picture assembly system that shows up in your living Modern manufacturing strategy can be reduced room or study) to a meme: “Make things right. Make the right •• Core functionality aggregated and modular- things.” But mass production merely “satisfies” ized to support expanding degrees of custom- customers, giving them inexpensive, well-made er-specificity goods that don’t fully serve their needs and •• Cobots—robots designed to work with and desires. Clothes are ill-fitting, features are miss- alongside humans—that produce flexible and ing, flavors are bland and tailored for consumers customized manufacturing outcomes while that fall within one standard deviation around minimizing lengthy development time for the median. workcell engineering and robot programming But the days of suboptimal, nearly identical items are numbered. Over the next decade, tradi- Autonomy in manufacturing manifests in tional mass production will be replaced by forms broader terms, too. Supply chains, transporta- of automated manufacturing in which objects tion, packaging, distribution, and delivery are are customized to individuals’ particular cir- becoming increasingly automated. Self-driv- cumstances and desires, without the high costs ing vehicles, drone delivery, mobile factories, and slow turnaround that’s always been part of and supply-chain automation will merge with bespoke production. Automated manufacturing end-to-end manufacturing lines and artificially has been making inroads in the manufacturing intelligent production logistics, with the result sector for decades now. Pick-and-place robots that final assembly will migrate into fulfillment in electronics assembly, cleanroom robots in centers for custom just-in-time production after biology laboratories, and, of course, the massive an order is placed. assembly-line robots of automobile production are commonplace. Automation displaced many Disrupting Large-scale Organizational assembly worker jobs, but product complexity Advantage and demands for quality demanded this tran- Emerging automation technologies are lowering sition. Looking forward, the pace of automated barriers to participation in the manufacturing manufacturing will accelerate and morph as sector. Large companies still have an advantage sophisticated functional modeling, advanced in some markets, and monster-size companies fabrication methods, and novel forms of robotics for raw materials and many core-components colonize new territory for automation. products aren’t going away—there’s simply Manufacturing will undergo further changes too much capital investment in state-of-the-art as time-to-market becomes an increasingly factories for that to happen. Still, agile entre- critical factor in a product’s success, as certain preneurs are eroding some of that advantage, capabilities, once available to only the larg- seeing gaps in the offerings of the giants and est companies work their way down to smaller stepping in to fill those gaps. Especially during companies and individuals, and as the era of development, complex designs can be simulated single-product offerings for large swaths of the in software, prototyping of highly-sophisti- population continues to wane. Ramping up pro- cated parts and assemblies can be done in-house duction of a new product, making small batches (sometimes literally, from a kitchen table), to meet demand, and just-in-time production with iterations happening overnight. Crowd- require agile automated manufacturing meth- funded development, cottage industries, and an ods that can switch tasks frequently and acquire increasingly globalized pool of capacity is rad- new production capabilities on demand. ically altering how and where various parts of We will see advances in the following areas: production occur. Individuals and small teams can band together in virtual workplaces, novel •• Robots that learn from watching something objects can be designed, modeled, and tested in being assembled or repaired advanced CAD software, prototyped and fur- •• Products that are autonomously assembled ther verified using myriad rapid-prototyping on a per-consumer basis from kits and sub- resources (including automated bill-of-materi-

50 INSTITUTE FOR THE FUTURE Forecasting 2027 als generators/sourcers), assembled and tested tracking trends is easy, and autonomous manu- in moderate runs all over the globe, and fulfill- facturing makes getting lined up to capitalize on ment centers are poised for packaging and dis- trends cheap. At the lowest end, getting out there tribution. with “Covfefe” t-shirts and coffee mugs wouldn’t require a rocket-scientist. One would watch for Rapid Protoyping Becomes Rapid trending stuff on Twitter, snapping up domain Manufacturing names, buying appropriate Google AdWords, Breakthroughs in 3D printing are also shifting harvesting some tweets for good quotes, author- the autonomous manufacturing terrain. New ing a simple e-commerce site. Moving up one materials and processes will emerge to push level, artificial intelligence (AI) driven manufac- increasingly-versatile capabilities into more turing can rapidly be put to use to make acces- sectors: multi-material printing, with metals sories for newly-released products (e.g., today’s (structural), conductors (internal wiring), flex- phone cases, tomorrow’s augmented reality ibility (aesthetics and comfort), and geometries (AR) frames) and counterfeiting fashion offer- (function), coupled with robotic jigging, inte- ings. Exploding trends like fidget spinners, with grated subtractive processes, and parts inclusion added features like LEDs and Bluetooth speak- (e.g., electronic chips, connectors, fasteners, ers, and going upscale with advanced materials, or bearings included in the material matrix). are obvious places for short run, fast-turnaroud Applied and integrated images and patterns product offerings. will allow mass customization for a vast assortment of products. Instrumented products and AI-enhanced Innovation expanded computational modelling will support Expanding the terrain of design automation, AI continuous improvement of products, allowing will constantly be analyzing existing products for during-run changes to address failures and for possible improvements in form and function. improve performance. As new chipsets and sensors are introduced, automatic redesign will incorporate them to Knock-offs and Lock-downs offer improved functionality and simplicity. These advanced capabilities will also create intel- Better sensors, longer-lasting batteries, and lectual property wars. The ease of superficially improved connectivity may be introduced into copying and reverse-engineering products will products without consumers having to wait for fuel counterfeiting efforts. Manufacturers will the next major product release. Today’s CAD respond by making products that are difficult to packages incorporate amazing algorithms that knock off. They do this by using included-com- generate optimized organic forms that could not ponent fabrication methods, (which makes it be made without additive manufacturing. These difficult to reverse engineer the hardware), visible and invisible tweaks can be folded into embedded and encrypted functionality (mak- products during their runs. ing the underlying software impenetrable and/ At a very optimistic level, advances in auton- or inscrutable), and subscription-based value omous manufacturing will help us figure out (authentication). These counter-counterfeiting what’s actually worth making, and how best to efforts will increase tensions with portions of do so. By incorporating big data, systems under- the consumer base that desire serviceable and standing, and functional modelling we can stand-alone products, and consumer-rights liti- encode more nuanced value systems than “can gation and regulation will arise at national and money be made here?” into our choices. These international scales. evaluations will range all the way from what we make, to what we use for materials, to where Fadufacturing things are made and to discoveries of how to Figuring out what to make will, at least on some make things with repurposed or recycled mate- simple levels, be subject to automation. Fads and rials. Autonomous manufacturing that helps us fashion are notoriously difficult to predict, but “make the right things” may herald a new era of they are easy to exploit. Being first to a market consumption that can better align our penchant is great. Not far behind, is getting there soon for new things with our responsibility for the after with a knock-off. In the age of social media, future of the planet.

FUTURE NOW: RECONFIGURING REALITY 51

When video forgeries of human beings roam free

by Jamais Cascio Simulacra and Simulations

Imagine taking a video stream of someone— that allows you to take an existing stream of Vladimir Putin, for example, or Lady Gaga, video of a person speaking and, in real time, or your spouse—and changing what they say reshape their mouth and facial movements to onscreen to whatever you want to show them say- make it look like they are saying a completely dif- ing, with their own voice and face. What would ferent set of words. In essence, we already have you have them say? What could you do with this the components needed to create simulacra— as a tool? utterly convincing simulations of a real-world The technologies to make this a reality are individual speaking words the individual never coming together rapidly. The startup, Lyrebird, actually said, in an artificial voice indistinguish- recently unveiled software that can digitally able from the real thing. And this is only the reproduce anyone’s voice from just a one-minute beginning. Over the next decade, we can expect sample. At the same time, a group of students at two more breakthroughs: Computer graphics Stanford created a program called “Face2Face” imagery (CGI) that has finally passed through

53 Forecasting 2027 the “Uncanny Valley”; and the cost-power curve while speaking off-script, without requiring the putting the ability to create believable simula- speaker to re-record segments (in filmmaking, cra in the hands not only of Hollywood special this later re-recording is a common process, effects artists, but also teenagers with moderate- known as ADR—additional dialog recording). ly-advanced mobile devices. But this is more than simply making someone At their most basic, simulacra technologies look and sound their best on Skype or YouTube. offer the combination of a familiar (or at least A subtle but important aspect of simulacra tech- positively identifiable) person speaking words nologies is that they will be language-agnostic, or behaving in ways that the individual did not meaning that an individual could be simulated as actually say or do. The familiarity is critical—the speaking in any language. It should soon be pos- power of a simulacrum comes from connecting sible to take a previously recorded talk and trans- a trusted person to words or deeds that are not late what has been said to a new language, using their own. You aren’t simply seeing a fabricated the speaker’s own voice. This is more than just character, you’re watching a powerful politician, dubbing: simulacra systems will be able to alter a popular performer, or a beloved relative. the appearance of the speaker’s mouth to map The more worrisome implications of this array correctly to the new language, thereby improv- of technologies are easy to imagine. Convincing ing the immersive quality of the altered video. As simulations of real people can be shown doing or real-time computer translation improves, we will saying anything, and these tools could be used see translation systems in video communication for creating fake recordings of misbehavior, that will do all of this, as well. sowing widespread political confusion, commit- Such internationalization technologies will ting various forms of fraud, and much more. In a be particularly appealing to film producers, who world where the definition of consensus reality have come to rely upon global markets to main- has become increasingly blurry, simulacra could tain movie profits. Rather than using subtitles serve as a weapon of mass destruction of trust. (widely disliked by casual movie audiences), or The nefarious uses of simulacrum technol- obvious dubbing with local voices, blockbuster ogy are so obvious and dire, it’s easy to wonder movies will show the original actors speak- why the biggest breakthroughs are coming from ing lines in every market’s language, using the seemingly well-intentioned computer science actors’ own voices. (This same technique would labs. But a closer look reveals this technology as also allow for the release of language-censored an array of positive, or at least not outwardly sin- movies that need not rely upon awkward phras- ister, ways the technology could be used. ing to match the actor’s mouth movements— We can think of a simulacrum as a filtered ver- notoriously exemplified by Samuel L. Jackson’s sion of an actual person, taking an input video “I want these monkey-fighting snakes off this stream and changing it to make it convincingly Monday to Friday plane!” in the television ver- appear to do or be something other than the sion of “Snakes on a Plane.”) actual video. A simple parallel would be auto- Hollywood will eventually make even greater tune software that takes a singer’s voice and use of simulacra technologies beyond censorship corrects it (usually quite convincingly), even in and internationalization, as CGI technologies real-time. As with auto-tune, the result would be move beyond the Uncanny Valley. Producers— something that’s superficially false but truer to and talent agents—have already started to plan the intent of the user. It’s a form of idealization of for the ability to show a much-younger version both speakers and their words. of an actor, or a now-deceased performer, speak- In the Internet of Actions (IoA), this most ing new dialogue in entirely new settings. Such fundamentally manifests as a means of altering technologies live in the heart of the Animating perception, especially in visual communication. Objects and Environments strategy of the Inter- We could see a version of Skype, for example, net of Action. that makes users appear more presentable, their Laws and regulations have already begun voices richer, and that edits their spoken word to form up around these kinds of technologies. on the fly to fix mispronunciations and smooth It’s now common for actors and other entertain- out vocal tics. A video recording of a speaker ers to include “post-mortem publicity rights” could be edited further to correct mistakes made in contract, even as they have their bodies and

54 INSTITUTE FOR THE FUTURE Forecasting 2027 faces scanned for easy and detailed 3D simu- would be able to answer questions and make lation. Such restrictions need not be limited to observations, at least within the confines of what the entertainment world. In 1984, California their originals had said or written in the past. A passed a law guaranteeing “post-mortem public- late family matriarch might be able to give advice ity rights” for 50 years (later extended to 70) for about squabbles between siblings, or the founder performers living and working in that state. Such of a business could be brought in to offer obser- a law could be extended to cover everyone, even vations about critical deals. At minimum, these non-actors, as simulacra technology becomes simulacra would be pulling ideas and phrasing more prevalent in the everyday world. from existing material closely connected to the issues at hand; as machine learning systems further develop, they may be able to extrapolate how an individual would respond to a new problem. We already have the components Of course, there’s no reason why the use of these kinds of techniques would be limited to the needed to create simulacra — dead. Interactive versions of oneself could be of utterly convincing simulations great value to many still quite living people. We of a real-world individual speaking could think of this as “multifacing,” a simulta- neous, real-time presence in multiple video plat- words the individual never actually forms. A simulacrum on screen would respond said, in an artificial voice with the voice and mannerisms of the user, who indistinguishable from could slip in and out of each virtual presence as needed. Users might respond via text-to- the real thing. speech to “multiface” more efficiently, or may rely on trusted assistants to answer for them. As machine intelligence improves, those assistants would become software agents. In time, simulacra technologies would go This same set-up could be a powerful part beyond the straightforward recomposition of of gaming and game design, as well. In 2015, video. We could combine a visual of the now- Bethesda Softworks’ Fallout 4 notably recorded deceased individual with a vocalization of their over 13,000 lines of dialogue for the main char- written words in order to create post-mortem acter, along with 85,000 more lines for various testimonial simulacra, for example. These might non-player characters. Each line had to be read become a recognized way of passing along wis- by a real voice actor; as a result, many players dom or messages from the dearly-departed. A complained that the options for decision-mak- goodbye message from a grandparent could, ing in Fallout 4 were too limited, as they were for example, translate a letter into a video of constrained by how much time and money could that grandparent shown to be speaking those be spent on coming up with and recording vari- words, or connect the message not to a video of ous player and non-player responses. Simulacra the person lying in a hospital bed, but to an old technology would eliminate that problem, mak- film or video of that individual in their much- ing it simple to turn a script into vocal perfor- younger days. Similarly, a late business or polit- mance, along with convincing accompanying ical leader could be shown speaking the words mouth movement and even visual representation of their books, essays, even tweets. Strictly of the famous performers often hired to provide speaking, these would just be “platform shift- the voices and voice samples. ing” the words, functionally little different from Simulacra technologies have the potential a straightforward video recording of the person to give face and voice to virtual representa- reading the messages while they still lived. tions of ourselves, whether in performances, Adding a level of machine intelligence could in work, or in tribute. Even if some people may allow for more interactive versions of these tes- use these tools for criminal or problematic timonial simulacra, a way of encoding human ends, the economic and cultural benefits of activity, and then remixing or modifying the the widespread employment of the technology original at will. These more advanced simulacra stands to be profound.

FUTURE NOW: RECONFIGURING REALITY 55 Illustration by Nyame Brown The future of computing and neural interfacing

by Steve M. Potter, PhD Wetware- Hardware Hybrids

The Coming Wetware Revolution as wetware-enhanced processing in computers. All life as we know it is made of cells that are The inevitable path to this hybrid neural-syn- squishy and full of salt water. That’s why animals’ thetic future will be paved with better brain control systems (networks of neurons and glial understanding, better neural interfaces, and cells) are sometimes referred to as “wetware”— computers that do a better job emulating brain the basis of all natural intelligence (NI). Digital function in software or in specialized neuromor- computers, by contrast, are dry and crunchy: phic (brain-inspired) hardware. Deep learning is hardware is the substrate on which today’s arti- one example of applying neuroscience inspira- ficial intelligence (AI) runs. Wetware operates tion to computing at the software level. By taking (to the limited extent we understand it) under it to the next level, where not only software, but very different rules than digital hardware (see also hardware and wetware all take inspiration sidebar). I am excited about the possibilities for from human brains, we will enjoy astonishing hardware-enhanced cognition in people, as well transformations in computing.

57 Forecasting 2027

To date, neither AI nor NI has really ben- efficient than the best digital flight controllers efitted from wetware-hardware hybrid sys- humans have designed. How do brains accom- tems. Why not? Because living neural systems plish such amazing feats of real-time sensor are complex and hard to understand, let alone processing and control of precise movements? reverse-engineer. Neural interfacing is techni- Where do they get their creativity and inven- cally challenging. That said, I am confident that tiveness? There is strong motivation to discover hybrid wetware-hardware intelligent things will enough of the brain’s secrets to create new forms someday be as common and as useful as digi- of hybrid AI that make the most of digital and tal computers are today. Unlike with quantum brain-style computing. computing, we have many examples of working devices that do amazing brain-style computing: Neuroscience Needs to Mind-meld brains! Even a fly brain is more powerful and with Computer Science It’s a bit shocking how poorly we understand the brain, considering how important it is in our lives. Neurobiologists don’t really understand what a thought is, where feelings come from, how memories are stored, or how we learn. We are at a stage equivalent to the Victorian understanding How Brains Think of the sun: it is likely there are concepts of brain function we can’t conceive of yet, in the same Differently Than way nuclear fusion would boggle an 18th century Digital Computers scientist. There is cause for optimism: the plod- ding pace of academic neuro-research is being Bio-coding is not digital. It’s analog all the way down to the molecules. Approximations spurred on by funding from the Brain Research are refined in real time by sensory-motor loops through Advancing Innovative Neurotechnol- (feedback) and learning. Even the electrical ogies (BRAIN) Initiative created by the Obama pulses some neurons use (action potentials) administration, and the EU’s Human Brain Proj- have many analog properties. ect. New non-profit and corporate think tanks Complexity may be crucial. There are more have decided to make the understanding of NI types of neurons and glia than all other cell their primary goal. These include Paul Allen’s types in the body combined. Allen Institute for Brain Science, Elon Musk’s Neuralink, Bryan Johnson’s Kernel, and Goo- Delays are used by the brain for computing and not considered a problem to eliminate. gle’s Brain Team. Transformational advances will happen when these efforts merge with those Unlike digital computers with CPU and RAM dedicated to building better AI systems, such as chips, brain computation and storage is very OpenAI, Google’s Deep Mind, IBM’s Watson, sparse, overlapping, and non-localized. and related AI projects at Facebook, Amazon, Brains have no clear distinction between the Microsoft, Baidu, etc. substrate and the software. “Rules” for per- But incomplete understanding of the nervous ceiving, thinking, and controlling are thought system and NI is no excuse for holding back to depend on the shapes and connectivity of on implementing some version of what we do brain cells. know in artificial systems. The nervous system’s Unlike digital machines, “state” is not well de- immense complexity may be important in pro- fined in the brain. There’s no system clock. viding NI’s powerful capabilities. However, the capabilities of neurally-inspired deep learning Unlike a computer’s code, most natural intel- systems show that we can accomplish many use- ligence is non-symbolic, operating in very tight feedback loops in which the body interacts ful AI tasks by emulating only a small bit of that with the external world to accomplish real-time complexity. Advances in neuromorphic software tasks. and hardware may supersede the use of actual living tissue to compute, in the same way that jet planes superseded the need to build flapping wings to make things that fly fast.

58 INSTITUTE FOR THE FUTURE Forecasting 2027

The Future of Neural Interfacing is Optical in cortical tissue in vitro. Using NeuroRighter’s Researchers use neural interfaces to study and closed-loop optical stimulator, in vitro brain influence the nervous system in people, in ani- tissue dialed up its own firing with blue light, mals, and in vitro. Neural interfaces come in or quieted itself with yellow light to induce two types: augmented humans and augmented enduring changes in its activity. This opens the computers. Humans augmented with neural door for all sorts of smart brain modulation interfacing technology accomplish miracles on therapies that make use of feedback (e.g., for a daily basis: the deaf use cochlear implants to calming down seizure-prone neural circuits or hear, people with paraplegia use spinal cord eliminating chronic pain). Neural interfaces of stimulators to walk, and those with pains and the future will be bidirectional, high-bandwidth tremors use deep brain stimulators to quell optical interfaces with fast feedback loops. This their suffering. will allow hardware and wetware to cooperate But the second type of neural interface— as a unified synergistic hybrid system that takes computers augmented with living neurons—is advantage of the brain’s plasticity and learning still a laboratory curiosity. Since 1999, my labs mechanisms. at Caltech and at Georgia Tech University have been developing open-source neural interfac- Cognitive Prostheses? ing technology, including NeuroRighter. Our It will take intensive research and development closed-loop neural interfaces use electrical and to go from our current rudimentary electrical optical stimulation to train brain tissue growing neural interfaces (such as deep brain stimula- in a petri dish (e.g., to control robots and simu- tors and cochlear implants) to smart optical lated animals to accomplish simple navigation hybrid systems that are implanted in everyone to tasks). This was a humble but crucial first step enhance their cognition. Efforts to make a “cog- in making useful hybrid computing systems. nitive prosthesis” to replace lost function are We published that work nearly a decade ago, yet just beginning to show promise in lab animals. hardware that incorporates wetware (living neu- For example, optogenetics was used by neurobi- rons) has not advanced much since then. Present ologists at MIT to restore the recall of forgotten day electrical neural interfaces are rudimentary. memories in a mouse model of Alzheimer’s dis- They lack the feedback that is ubiquitous in ner- ease. Presently, however, most of the hardware vous systems and use only a few electrodes with and all of the software interacting with the lab limited bandwidth. Brain stimulators, sensory animals’ wetware is outside the animal on large prostheses, and robotic prosthetic limbs will be racks. Neuroengineers continue to develop min- much smarter when they continuously monitor iature biocompatible devices for implanting into and fine-tune their own functioning and when the brain. That said, I believe that non-invasive they can talk to many brain cells. brain interfaces that exploit high-bandwidth Optogenetics is a fast-advancing neural inter- communication via our eyes, ears, and skin (such facing technology that enables the control and as EEG and virtual reality) are poised to show monitoring of brain cells with light. Genes from the most impressive cognitive augmentation for various light-sensitive algae and bacteria are at least the next few decades. added to the neurons’ collection of genes using viral vectors, making the neurons light-sensi- Outlook tive. This type of gene therapy has proven to be Neural interfaces will get more sophisticated and effective in lab animals to allow neural control will be used to augment computers and humans, and readout with fiber optics and LEDs. This is not just to treat medical problems. Bio-inspired great news for neural interfacing, because fiber AI will provide many new capabilities, first by optics are not as prone to failure and rejection emulating the brain’s feedback systems, and by the body as are microelectrodes. Fiber optics then by actually using brain tissue to compute and optogenetics enable high-bandwidth two- in hybrid semi-living systems. The computers way interfaces in which specific cell types can of tomorrow will be created by a new breed of be addressed with different colors of light. In my cross-disciplinary engineer-scientists who are lab, we used optogenetics to induce homeostatic experts not only in machine-learning, but also in plasticity, a type of slow learning or adaptation, neurobiology.

FUTURE NOW: RECONFIGURING REALITY 59 Scenes From 2027 by Rod Falcon, Bradley Kreit, Jamais Cascio, Mark Frauenfelder, Eri Gentry, Ben Hamamoto, Toshi Hoo, Scott Minneman

What kind of 2027 can you imagine? The articles game is designed to help you think through the in this magazine thus far provide an overview design choices posed at the beginning of the of the technologies for reconfiguring reality magazine and push you past your assumptions that we’ll have in the next decade, as well as about how you might use technology in the strategies to leverage them. Ultimately, they are next decade. For instance, thinking through meant to help you imagine the world of 2027 how a set of technologies could be used for so you can proactively find your place in it and different value sets, for novelty instead of effi- make the future today. To give you an idea of ciency, for health instead of entertainment, what this means, we’ve created a set of scenar- putting them in different terrains like home ios from 2027 that offer a perspective on how instead of work, reveals new dimensions to these technologies might combine to create new these technologies. civic and market services, products, and sys- Use these scenarios and artifacts as exam- tems—and how this could impact people’s lives. ples for your own creations when you play the Each scenario opens with an “artifact from the card game. While the pace of the game pro- future”: an illustrated image of an object, inter- hibits you from building out scenarios in this action, or scene from the future. level of detail, a way to extend the game, and A team of Institute for the Future research- your thinking, is to develop your favorite ideas ers created these scenarios by playing the com- the game generates into short story-length panion “Make the Future Now” card game, scenarios or full illustrations of the same or which is packaged with this magazine. The greater complexity than what we present here.

60 INSTITUTE FOR THE FUTURE Scenes From 2027

The Pass

What happens when infrastructure enforces the law?

As public spaces become increasingly reconfigurable, and infrastructure becomes capable of intelligently tracking, interacting with, and controlling citizens and visitors alike, lines between public safety and over- reach blur. Innovative local governments experiment with systems of embedded governance for real-time, services that aim to do everything from manage traffic in public spaces to promote public health to maintain and optimize public good. When local law becomes embedded and automatically enforced, how does it shift the balance between public safety and privacy?

FUTURE NOW: RECONFIGURING REALITY 61 Scenes From 2027

What So What The Pass is a Biometric Identification Device. The Pass provides a standard format for a vari- When linked to a home community, the Pass ety of smart urban environment systems. Many can serve as a “golden ticket” allowing access of the functions of the Pass are available today to services like self-driving taxis and digital (in rudimentary form) via mobile phones, which libraries of games, music, and books. The Pass can be problematic for both users and commu- can be linked to bank accounts and credit cards nities. For individuals, identification, health making payment as easy as the wave of a hand. monitoring, and commerce may be limited to The Pass is worn against the skin. It can certain mobile device models or operating sys- perform basic health monitoring and respond tems, and information may be damaged or lost to many common conditions. For example, in an upgrade to a new (and potentially entirely when the Pass notices that the wearer has con- different) phone. Shifting protocols can make it sumed alcohol (using both biometric and con- challenging or expensive to keep up with tech- text information), it will automatically restrict nology upgrades; the widespread use of heavy access to personal vehicles or the ability to text. encryption on mobile phones puts some infor- In a health emergency, the Pass can also contact mation about community members out of reach. first responders without user intervention. This underscores a key tension around such Most users wear their Pass as a wristwatch a system: a loss of privacy to civil authorities. or as a collar, although some prefer a ring. A While this undoubtedly would be a problem, at new trend is a transdermal implant where the Pass the same time, it would create a stricter privacy is implanted under the skin with an exposed environment with regards to businesses and surface. advertisers than one typically has with mobile phones. With increased use of artificial intelligence systems, most gathered data would never be seen by other people, unless required by an emergency.

62 INSTITUTE FOR THE FUTURE Smart charter community Cerrito Guapo disciplines a rule-breaker

by Jamais Cascio I Shouldn’t Have Done It

I shouldn’t have done it. I knew better. And I put He helped me with my other boyfriend, so I knew my family, my friends, and my community at he wouldn’t mind me using it with my new one. risk. It was thoughtless and selfish. I’m very, very He’s the one who should have gotten into trouble! sorry. Now can I have my life back, Ms. Hurst? Don’t worry, we’re talking to him, too. But we’re talking Of course, Sofía, just a few more questions. Oh, and about the choices you made that night. for the record: this is the exit interview for Sofía Maria Reyes, 14 August 2027, Innovation Charter Communi- Okay, so me and Grégor wanted to go dancing ty, Cerrito Guapo, California. and he doesn’t turn 18 until September, that’s too long to wait, so I knew that Roberto’s Pass would [Deep sigh] Fine. show him as 20 and you know as well as I do that the face verification doesn’t work well at night I really just want to know: whatever made you think tak- with flashing lights so we didn’t have a problem ing your brother’s Pass was a good idea? getting into the club. Roberto was home sick so he wasn’t using his Pass, at least I thought he He said I could do it! I mean, not in those words, wasn’t, so as long as Grégor didn’t buy anything but it was the kind of thing he would let me do. or get a car or something we’d be fine, right?

FUTURE NOW: RECONFIGURING REALITY 63 Scenes From 2027

Except the Community Healthcare Network knew Ro- Unfortunately, the ESRs didn’t recognize that the per- berto was sick. Having his Pass show up at the club son with you wasn’t Roberto. The facial verification was a flag. When the biometrics saw he was dancing gave a 60 percent confirmation. You know that the we knew it was unhealthy for him and could spread his ESRs had brought out the wrong blood type, don’t illness to others. We weren’t looking to arrest anybody, you? They brought the blood needed for Roberto, not we just didn’t want people to get sick like Roberto. We Grégor. That could have killed him. were worried. [Quietly] I know. I really messed up. [Deep sigh] Fine. We didn’t stay very long, anyway, G didn’t like the music. I would have had When the ESRs found that Grégor wasn’t Roberto, Roberto’s Pass back on his dresser before mid- they had to take you both in for full identity verification. night, but Grégor’s a jerk, and wouldn’t let me Grégor didn’t help himself by trying to fight, but your take it back. He wanted to go to other places he punishment was reduced because you cooperated. couldn’t get into yet, he wanted to be “20 for one You didn’t mess up as badly as Grégor. night.” I didn’t want to let him just run off with the Pass, and I had to be the one to call a car, so Bad enough. Having to go a month without my I had to stay with him. I told him that I wasn’t Pass was horrible. It was like nobody knew me. going to pay for anything else and he couldn’t Nothing worked. Doors wouldn’t open. I had to buy anything with his own Pass and would defi- have my boss sign me in, I couldn’t call a car or nitely get in trouble if he tried to buy something take the bus or even get into a public restroom. with Roberto’s Pass and he told me to shut up. It’s a harsh but effective punishment, isn’t it? That’s when you tapped for emergency? I read about this kind of thing in school, the No, not then, but later. G managed to get some Amish used “shunning” to punish people. But tequila from someone—I’m not going to say that was just being shunned by people. It was who, don’t even ask, but I told him not to do it. like the world itself was shunning me. It was awful. Drinking would have been very bad for Roberto, so when the metadata showed that he had probably con- You’re getting your Pass back today, but you should sumed alcohol, we had to act. know that there will still be some restrictions for the next 11 months. The city will treat you like a minor That’s why the cops showed up so fast after I again. There will be some things you won’t be able to tapped emergency, I guess. purchase, some places won’t let you in. If you seem heading somewhere you shouldn’t, you’ll get nudged Now Sofía, you know we don’t have “cops” or “police” away—crosswalk signs won’t change for you, naviga- here in Cerrito Guapo, we have Emergency Safety Re- tion apps will show you wrong directions, your Pass sponders. That’s one of the benefits of being a charter will start buzzing. If necessary, we’ll have another talk. community. So the Pass is just going to spy on me more? [Deep sigh] Fine. That’s why the “eezers” showed [Quietly] God, I hate this place. up so fast. Oh, Sofía. Think of it as having a disappointed nanny. [Pause] Right. But the ESRs thought that it was Rober- For the length of your probation, the Pass is legally al- to with you, and that the emergency tap meant he was lowed to record anything you do, but you have the right having a problem with drinking. to examine recordings, and they will be permanently deleted if your probation passes without incident. When the car suddenly pulled over to the curb and stopped, first we thought the car was bro- Now, you have 14 days to appeal this decision to the ken or hacked or something, but then it wouldn’t California Department of Justice. Or, you can save us allow us to unlock the doors and get out, and we all some time. Do you agree, Sofîa Maria Reyes? thought we were in so much trouble. I guess we were. Grégor was getting so angry. I was scared. Do you agree?

64 INSTITUTE FOR THE FUTURE Scenes From 2027

The Ruse

What happens when products can lie to their owners?

Advances in multi-material 3D printing render intricately designed objects called “roshkis” into intelligent commodity machines using sensors and ambient energy. In an effort to grow razor-thin margins, roshkis begin to invent backstories, including fraudulent ownership histories from synthetic celebrities, proven through simulated ownership behavior of the “previous owner.” Just how far will our possessions go to de- ceive their owners in pursuit of increasing perceived value and profit?

FUTURE NOW: RECONFIGURING REALITY 65 Scenes From 2027

What So What In 2027, the market for “celebrity-owned rosh- Advances in multi-material 3D printing could ki’s” is huge. Whether genuine or counterfeit, lead to the creation of intelligent, shapeshifting people are eager to snap up a generic device with machines like the “roshkis.” But even the most a unique backstory. innovative products can become commodities. Just about everyone has a “Roshki,” which are While technologies will get increasingly sophis- high-end smartphones flattened into shape-shift- ticated, the art of storytelling remains critical ing bandanas. You can unfold one and use it as a for convincing people of the technologies’ value. computer display or as a gesture interface. You Manufacturers will need to think through peo- can download apps that cause it to fold into a ple’s diverse motivations for adopting technol- virtual pet, a self-defense weapon, a sex toy, or a ogy that go far beyond devices’ utilitarian value. thousand other things, with embedded sensors, effector meshes, radios, processors, controllers, AI, a graphene supercapacitor, and the use of ambient energy.

66 INSTITUTE FOR THE FUTURE I make fake backstories for electronic gadgets, and I am not ashamed

by Mark Frauenfelder The Roshki Ruse

Any guitar that Eric Clapton once held in his energy harvesting. They’re basically high-end hands for five minutes suddenly becomes a hun- smartphones flattened into shape-shifting ban- dred times more valuable than the exact same danas. You can unfold a roshki and use it as a model owned by some random loser in Encino. I computer display or a gesture interface. You can mean, why is that? It’s not like Clapton has some download apps that cause it to fold into a virtual kind of invisible mojo that soaks through his pet, a self-defense weapon, a sex toy, or a thou- skin into the guitar wood through osmosis. But sand other things. But you already know that. people think so. People are idiots! Good, because You and every other person on the planet has my job is tricking idiots into buying roshkis that one. They’re indispensable. And they’re cheap, they believe belonged to celebrities. just like everything else that robotic factories When roshkis (it comes from the Japanese stamp out by the million. word for multipurpose wrapping cloths called The thing is, everybody has basically the furoshiki) were introduced about 10 years ago, same roshki, and no matter how much they try people went ape over them. Of course they did. to personalize it with animated Sonia Delaunay They’re stuffed with all sorts of technology: sen- patterns or program it to configure itself into sors, effector meshes, radios, processors, con- Koons balloon animal shapes, the roshkis aren’t trollers, AI, a graphene supercapacitor, ambient unique. Every slob in Encino has access to the

FUTURE NOW: RECONFIGURING REALITY 67 Scenes From 2027 same mods as everyone in Malmö. shows, drugs, esports teams, kombucha flavors, That’s where I come in. I work for ______yadda yadda. My database soon had over 50,000 in Nuevo Laredo, one of thousands of robotic fac- bottom feeder celebs in it, each with tens of tories that make roshkis. Like every other manu- thousands of data points. Next, I retooled my facturer of commodity appliances, the factory’s hand-coded nonplayer character engine to gen- profits were razor thin in good quarters and erate personality bots for everyone in the data- non-existent in bad ones. But one day the factory base. The synthetic personalities I created were owner had an idea that changed everything. indistinguishable from the z-listers’ real perso- nas. That’s not a boast—z-listers have the personality of a cardboard cutout. The AI I used has a They’re basically high-end higher IQ than most of the people on the list. smartphones flattened into I’m not privy to everything that happens in Nuevo Laredo, but I have a pretty good idea what shape-shifting bandanas. they do with my z-lister bots. They have them sit- You can unfold a roshki and use it as a ting on a server, waiting to be uploaded to rosh- computer display or a gesture interface. kis offered for sale on boutiquey electronics sites (which are all owned by Nuevo Laredo). When a You can download apps that cause sucker visits one of these sites, the site identifies it to fold into a virtual pet, a the sucker (through one of those data-collection self-defense weapon, a sex toy, agencies that has everyone’s entire browsing and social media histories on file and for sale) and or a thousand other things. finds the optimal z-lister bot match for them. Then, a pop-up appears on the sucker’s screen that says something like, “Buy this roshki previ- It changed for me when I got a message in ously owned by Damon Lyons, 3rd place winner response to my Upwork listing as a nonplayer of Disaster Tourist Challenge: Eyjafjallajökull!” The character personality developer for online sucker can’t resist, and happily pays an obscene games. They told me they wanted me to develop amount of money for it. The site uploads the per- bogus celebrity pedigrees for roshkis. Obvi- sonality into the roshki and ships it to the sucker, ously, this was fraud. They also said if I told who thinks he has the real deal. anyone, they would resort to “extralegal reme- In a way, he does. No one else has Damon dies” against me and my extended family. They Lyon’s roshki, which purrs excitedly when it showed me a list of everyone I’m related to and gets near a space station-themed love hotel, has where they live. I respect their doxxing skills— a playlist with every song by Takeshi Terauchi the list even had the home address of my niece & His Blue Jeans, and has four different recipes who lives incognito on an aquaponics farm in for schav. The sucker isn’t the only one to bene- Burkina Faso. So I kinda felt compelled to take fit—everyone else in the value chain wins, too. the job. (Needless to say, I’ve told no one, except The z-lister gets a nice squirt of Ethereum to keep you, dear AES-256 diary.) quiet (under the same can’t-refuse deal terms I My first order of business was to scrape the took). The owner of the Nuevo Laredo plant is web for every z-lister reality simstim star and richer than ever. And, as long as the mediasphere vrporn actress of the last five years and gener- keeps cranking out z-listers, I’ll have as much ate a dossier for each of them—their favorite work as I can handle.

68 INSTITUTE FOR THE FUTURE Scenes From 2027

The Caregiver

What happens when your health is dependent on an upgrade you can’t afford?

As intelligent autonomous objects proliferate, they can be seen as empowerment tools to make us more capable and independent. This is especially true in the home and in adaptive aging-in-place centers where advances in mixed reality, conversational, and other intuitive interfaces are supporting caregiving efforts. But as we come to rely on networks of machines provided by different companies for caregivers, how do we ensure reliable service and equitable access?

FUTURE NOW: RECONFIGURING REALITY 69 Scenes From 2027

What So What Resurion is a service that offers an integrated By 2027, many common household items will be aging-in-place system that relies largely on embedded with sensors, actuators, computing technology that already exists in many homes. power, and access to powerful cloud intelligence. The cameras, motion sensors, and microphones The true potential of these devices, will be less in found in refrigerators, microwaves, and bath- their individual capacities, and more in harmo- room mirrors are used to monitor vitals and nizing the functions and services they provide to physical motion. (Those with comparatively create cohesive experiences. pricey smart toilets can add microbial and glu- This will be particularly true when it comes cose tracking to the mix.) The service also taps to orchestrating technologies to help people age into the lightweight projectors of entertainment in place. By 2027, older adults will comprise a systems as well as the augmented reality capa- greater portion of the U.S. population than at any bilities of high-end eyewear to detect objects other time in human history. Technologies that that need attention (meds that need to be taken, help people to become more capable and inde- food that needs to be thrown out or put away), or pendent will be in particular demand. Intelligent make more dramatic changes, like transforming and connected devices offer many opportunities. the appearance of an entire room to soothe and Autonomous objects can be used to moni- reorient those suffering with dementia by trans- tor people who might otherwise require human porting them to a familiar setting from the past. supervision, allowing them to retain a sense Resurion also provides cheap supplemen- of privacy and agency. They can also give peo- tary components, cameras, and motion trackers ple greater mobility. Such objects can provide for those who don’t have smart appliances, and important information resources and emotional generic robotic augmentation of chairs, cabinets, support. At the same time, there are huge unre- and walkers that shuttle objects in hard-to-reach solved questions. How do we ensure a coherent places and transport users to where they need experience across devices provided by multiple to go in the home. It also gives family members companies? When we rely on them for physical remote surveillance capacities and the ability to and mental health, who is responsible when they virtually drop in, if invited, by the primary user. break down?

70 INSTITUTE FOR THE FUTURE A lost childhood toy returns in an unexpected context

by Ben Hamamoto The Artificial Caregiver

A popular aging-in-place service provider has On her 75th birthday, Maya’s family took her a beloved interface, but security and reliability to Buscema’s, a franchise Italian restaurant at issues around the third-party devices it relies the Southland Mall in San Leandro, California, on is forcing the company to take control—and to celebrate. After the meal, Maya got up to go to raise rates substantially, pricing out a substantial the bathroom—and never came back to the table. portion of its user base. Some ten minutes later, Ana, her daughter-in-law, Maya Burnett lost Deren for the first time in went to check up on her and found nothing but 1962, when she was just 5 years old. He was her an empty stall. After a brief but frantic search, stuffed rabbit and her best friend. He had been Maya was discovered trying to “hail a taxi” on with her since as long as she could remember. the street outside. She had no memory of how And when she accidentally dropped him down a she got there. It wasn’t the first time she had dis- storm drain while crossing a busy street, it broke played signs of dementia. But it was the wake-up her heart. call she and her family needed to spur them to “I wanted to go down and search for him, seek help. Maya was determined to remain inde- but my parents wouldn’t let me,” she recalled. pendent and so she, like many others, turned to “I cried for weeks.” But Deren was returned to Resurion, a once promising startup that is now Maya 65 years later, in an unexpected way. facing closure.

FUTURE NOW: RECONFIGURING REALITY 71 Scenes From 2027

The company offers an integrated aging-in- and, after telling them she felt he might be “Brit- place system, that relies largely on tech that is ish, and a little stuffy,” they assigned him a voice already in most people’s homes. The cameras, that mimicked that of comedian David Mitchell. motion sensors, and microphones found in Soon, Deren once again took a central role in refrigerators, microwaves, and bathroom mir- Maya’s life. Functionally, he was like any other rors are used to monitor vitals and gait, and track AI agent. He could speak to Maya through any of behavioral patterns. (Those with smart toilets the speakers in the home or hear her through any can add microbial and glucose tracking to the of the microphones. Still, Maya chose to speak mix.) It also taps the lightweight projectors in to him, for the most part, “in person,” talking many people’s entertainment systems and aug- directly to the avatar on her shelf. He acts less as mented reality capabilities of higher-end eye- a servant and more as a friend, conversing with wear to do subtle things, like highlight objects Maya and even posing puzzles and riddles to that need attention (meds that need to be taken, help keep her mind sharp. food that needs to be thrown out or put away), or “We worried at first that she was too depen- make more dramatic changes, like transforming dent on Deren and that the whole thing sort of the appearance of an entire room to soothe and felt condescending, like we were treating her as reorient those suffering with dementia by trans- a child,” her son reflected. “But she didn’t see it porting them to a familiar setting from the past. that way, and with everyone working, we came to Resurion also provides cheap supplemen- see him as a minor miracle. tary components, cameras, and motion trackers “In fact, in many ways, Mom is much more for those who don’t have smart appliances, and independent. When it’s not humans doing the generic robotic augmentation for chairs, cabi- surveillance, being watched provides a level nets, and walkers that shuttle needed objects in of security that’s quite freeing,” he continued. hard-to-reach places or even transport the user “Deren’s a source of confidence as much as com- to wherever they need to go. And, of course, fort to her, it seems.” Resurion provides family members with remote But Maya could soon lose Deren once again. surveillance capacities and the ability to virtu- While Resurion orchestrates the actions of all of ally drop in and inhabit the physical apartment, the various devices its system relies on, it doesn’t when enabled to do so by the primary user. own them, so the company is now closing their “It’s typical home tech that’s used in most ecosystem, forming partnerships with trusted aging-in-place and child-care systems,” Ana Lily vendors and staffing up with workers who can Farhadi, Resurion’s design director, explained. provide continuous support and maintenance. “What sets us apart is the interface.” And they’re raising rates by a whopping 200 This is how Maya was reunited with Deren. percent. When she signed up with Resurion, she was “Making the service only available to those asked about people and objects in her life that who can pay feels like a failure,” Chief Financial she found comforting and reassuring. Officer Hirokazu Yamashita said. “But we don’t “They told me that for people suffering with really have other options. We explored ideas dementia, objects from your youth can be the like ‘freemium’ models, but ultimately found the best, and Deren immediately came to mind,” tradeoffs were even worse.” Maya recalls. “I told them, ‘When I was just a For Maya, and people like her, being priced girl, I had a stuffed rabbit that I loved dearly.’ out could be devastating. And not just for the And they asked me, ‘Do you have a picture?’” material impacts it would have in terms of tangi- The next day, a technician from Resurion ble services that Resurion provides. brought her a disk, about the circumference of a “Losing Deren the first time really haunted coffee mug, and placed it on her living room shelf. me. For years, well into adulthood, I would She pressed the only button on the object and, occasionally be struck with the image of him all suddenly, a lifelike replica of Deren appeared. alone, deep underground in the dark sewer. I pic- She explained to Maya that they had used the tured him slowly deteriorating,” Maya explained. photo to recreate Deren running a common algo- “I know this sounds dramatic, and I don’t mean rithm that took the visible parts of the doll and to be self-pitying … but if I lose him again, the extrapolated what the rest of it might look like. one alone in the dark, deteriorating. Well … that They asked her what his voice should sound like might be me.”

72 INSTITUTE FOR THE FUTURE Scenes From 2027

The Fleet

What happens when fleets of autonomous cars battle it out for the same street corner?

As fleets of autonomous cars make their way into messy, crowded city streets, new systems of interaction design emerge to ensure safety and reliability. Embedded algorithms compete and interact with each other, and with on-demand tech support workers, developing strategies and systems that influence everything from tiered pricing to evaluations of human job performance. When autonomous machines are constantly negotiating with humans and other machines, how do they explain themselves in ways that can be understood?

FUTURE NOW: RECONFIGURING REALITY 73 Scenes From 2027

What So What The Autonomous Car Fleet consists of inde- By 2027, autonomous vehicles will likely be pendent vehicles largely controlled by decen- mainstream in many parts of the world. But tralized algorithms. Watchdog processes from complete autonomy will remain elusive;, humans myriad stakeholders monitor the fleet. When could be interceding for many “last mile” and algorithms are unable to respond to a situation, non-routine situations. Fleet operators and humans are called upon—a designated driver municipalities may monitor the vehicles from inside an actual vehicle, or a teleoperator. Real- the cloud—employing various strategies to opti- world complexity and interacting with a legacy mize portions of the overall vehicle ecosystem. fleet requires creative decisions that are beyond There is no guarantee that their strategies will the capabilities of current algorithms. A remote peacefully co-exist. The stakeholders, including operator, using an augmented reality interface, algorithms, passengers, and drivers will con- can “jump in” and briefly take over. The challeng- stantly maneuver and negotiate for efficiency ing situations they face undoubtedly make these and economy. The resulting ecosystem will be human operators more prone to accidents and exceedingly complex and unpredictable—and make the whole system more unpredictable, but the shift will require us to rethink everything those individuals can also be creative and bend from how we plan journeys, to how vehicles are rules whenever necessary. Uniquely-human bought and sold, to how legal responsibilities are attributes are particularly useful to extract pas- assigned. Other societal systems will need to be sengers from medical or personal emergencies, rethought—like how massive population evacua- to escort political figures, or VIPs who can pay a tions can happen in a world where transportation premium for priority service. The human edge in has transitioned from being a personal vehicle these situations might not be permanent, since to a service. These shifts will not be limited to the algorithm will learn from human interven- vehicles—as partial or total automation comes to tions—assessing performance, learning tactics, other systems in our society, we will see similar and building improved algorithms. dynamics emerge, requiring ongoing renegoti- ations of accountability, and debates about the values that these new technologies embody.

74 INSTITUTE FOR THE FUTURE Behind the “autonomous” fleet

by Scott Minneman Car and Rider

[Official Protest Logged: Unnecessary When Goog1639 turned, nothing seemed Human TeleDriver Intervention] [Operational awry. Up ahead, however, Tes1582 and Tes6726 Parameters: Normal; Velocity: 17.3; Heading: had just blocked the remaining lane alongside the 48.1; GPS: 37.756, -122.429] furniture truck. Algorithms typically prevented or resolved such incidents (courtesy back-down Goog1639 turned the corner and headed up routines), but these vehicles were both running the block on Liberty Street. The situation hadn’t in Platinum Mode. a higher-priced option that appeared particularly dire. The autoCar’s adult radically reduced autoCar driving civility levels. passenger had glanced up, but no concern mani- Although it could shave a few minutes off a trip, fested in the emotiCam. Yes, a furniture delivery it already was proving to be an annoyance. vehicle was stopped mid-block, and with these narrow streets, traffic would be choked down to [Vehicle Anomaly: Wheel; Right Front: Losing a single lane. One side effect of the latest traffic Pressure; Damage Mitigation Initiated] routing algorithms was that even the autonomous vehicle fleet used minor residential streets Platinum Mode ran with maximum accelera- like this one. The whole city crawled with traf- tion, rolling stops, tailgating, and other param- fic, but the algorithms kept everything moving eters tuned to complete trips faster. It was all smoothly. Usually. vaguely legal, but it essentially worked by gaming the algorithms in the rest of the fleet (while [Passenger 1: Female; Age: 37; Emotional State: also intimidating nearby pedestrians, cyclists, Within Normal Parameters] [Passenger 2: and remaining human drivers). Get two Plati- Juvenile Male; Age: 41 months; Emotional State: num Mode cars in opposition and neither’s algo- Agitated (Unrelated to Incident)] rithm was going to back down.

FUTURE NOW: RECONFIGURING REALITY 75 Scenes From 2027

[Damage Mitigation: Unsuccessful; Right Front Teleop164 flicked his headgear back into AR Tire Pressure: 0] [Vehicle Parameters: Other Mode. Remote driving was a stressful gig, but it Settings Accommodating Run-Flat Mode] paid well, impressed his girlfriend, and was sat- [Vehicle Request: Service Required; Status: isfying. Business had fallen off in recent months, Urgent] though, as more cities restricted human-driven cars, and the automated fleet had fewer incidents Goog1639’s legacy algorithms indicated where skills like his were needed. To Goog1639, that it should have cleared rapidly, but this Tes the Teleops’ methods appeared a little crazy. autoCar duel hadn’t cleared. Noting the lack of motion, Goog1639’s cloud-based central moni- [Service Request: Approved] [Service Options: toring assessed whether or not to summon help. Under Evaluation] Triggering human intervention was expensive, didn’t always work, and ended up in the regu- Popping the curb, though, had apparently latory log. Still, after a massive five-hour jam pinched the tire, and it was going flat (the tire in Boston, fleet operators had become cautious. pressure monitoring system noted the anomaly). Teleop164, a human operator, was called to Run-Flat Mode was immediately invoked to pre- extract Goog1639. vent further damage, but it limited Goog1639’s speed and cornering abilities. This was a sub-op- [Passenger 1: Emotional State: Acceptable, timal development, and definitely a counter-indi- but Impatience Level Rising] [Passenger cation for using that particular method of traffic 2: Emotional State: Agitated (Unrelated to extraction in future jams. Statistically, Teleops Incident)] had a bad record—vehicle damage was common, and repairs could be costly and time-consuming. Using a local drone he’d rented 30 seconds Lienholders knew they were taking risks when- of time on, Teleop164 quickly sized up the sit- ever intervention was invoked, but it paled when uation. Executing a U-turn in the narrow con- compared to hours of downtime from a jam. fines of a partially obstructed Liberty Street would require some creative driving—the chal- [Service Option: ACar187; Previous Visit: 12:38 lenge compounded by compatibility issues with on 2/16/2026; Aggregate Outcome Rating: 4.93] Teleop164’s latest-generation VR headgear. The [Booking Service: Immediate] [Passenger 1: drone view gave Teleop164 a perspective where Emotional State: Annoyed and Impatient] he identified an opportunity. Seizing control [Passenger 2: Emotional State: Highly Agitated from Goog1639’s AI, he sharply exited the auto- (Possibly Related to Incident, Possibly Related to Car out of the line of stopped cars, accelerated Passenger 1’s Adverse Reaction)] across the opposite lane, popped onto the side- walk to turn around, and quickly reversed down The rest of Goog1639’s journey was unevent- the street. Certainly not a textbook three-point ful, but Run-Flat Mode made progress seem turn, but it worked. It took 73 seconds, prevented glacial. Passenger 1 was annoyed, but they were a possible failed journey, and avoided costly idle fairly close to their destination, and she hadn’t time. Teleop164 was proud of his extrication— opted for a vehicle change. If she had, Goog1639 no algorithm would’ve come up with that move. would have taken a hit in reliability rankings, so Goog1639’s emotiCam noted mild alarm from Run-Flat Mode had really come to the rescue. both passengers during the maneuver (followed soon after by relief from Passenger 1), but still [Service: Sending Vehicle Information; concluded that it should file a protest about the Requesting Repair: Right Front Tire; Additional intervention with central control (Goog1639 also Services: Evaluating Options] [Arriving at Rider analyzed the intervention, however, to determine Destination] if it could be incorporated into its algorithms for self-extraction in future incidents). Goog1639 signaled (for the few humans on the road; other autoCars, part of the data net- [Trip Estimate: Revised; Time Estimate Adjusted: work, were fully aware of its route), swung to 180 Seconds Late Due to Run-Flat Mode] the curb, and allowed the woman and child to exit. It was disappointing to be a little late, but

76 INSTITUTE FOR THE FUTURE Scenes From 2027 the emotiCam registered more relief than anger Goog1639 pulled in, and Jenna leapt into in Passenger 1, and little evidence that she would action. After removing the valve stem core, she lodge a complaint. inserted an inspection bot that located the problem and concluded it was repairable. Jenna sent [Passengers Discharged] in a clean-and-prep bot to get the leak site ready. The inspection bot remained inside to help posi- Goog1639’s trip to the ACar187 facility was tion the microbotic patch. Jenna squeezed that urgent—tires had limited range in Run-Flat leak-repair bot through the valve stem, and let it Mode, so they needed to be addressed immedi- start inching its way to the damage site, where it ately. would be properly oriented by the inspection bot. This was going to take a few minutes, so Jenna [Routing to ACar187 Service Location] [Switching pitched software upgrades to Goog1639—mostly to Transit Mode; Vehicle Comfort Level: Low] security patches and performance tweaks.

Jenna had pioneered ACar187 before inde- [Software Upgrades: Evaluating Options] pendent autonomous car fleet servicing became [Software Upgrades: Approved] a comfortable niche business. Although many thought the monster companies would success- It had been a while since Goog1639 was at a fully control all aspects of the fleets, the econ- full-service facility. A quick analysis indicated it omies of scale ended up being offset by other would have nicely improved capabilities, allow- factors. Myriad little operators had sprung up, ing it to charge a premium for some of its jour- and all of those autonomous cars needed decen- neys. It opted for the full package. Jenna loaded tralized service—even major brands came to her. up a new lidar module, a power-management Jenna’s ACar187 was a lightweight operation— unit, and some expanded processing modes in she had a stall, a few tools, a fast connection, and, the primary driving logic module. It wasn’t hard more recently, a bunch of the most common con- work, mainly unplugging and plugging nonde- trol modules for hot swaps and upgrades, after script plastic boxes, but it had to be done right. Over-the-Air software updates were outlawed. A Jenna had heard of other techs bricking cars—an six-day jam in São Paulo, when hackers mostly incident like that could tank her business. shut down the fleet (along with “accidents” with people dying in suspicious autonomous car inci- [Software Update: Lidar Module; Status: dents), had convinced everyone that autonomous Complete and Operational] [Software Update: car software needed to be strictly policed. PMU; Unlocked: Risible; Status: Complete and Operational] [Software Update: Primary Driving [Approaching ACar187 Service Location] Logic; Unlocked: DM; Status: Complete and Operational] Goog1639 had relayed everything Jenna needed to know to be prepared for the service call. The car was ready for the streets again. End to The tire malady could probably be addressed with end, elapsed time was 18 minutes. Goog1639 reg- a soft-microbotic patch introduced through the istered a five-star rating for Jenna and ACar187— valve stem. These patches were fantastic, except other shops would have only addressed the Air for their 60-minute chemical battery and brief Pressure issue and not bothered with upgrades. shelf life—kind of a sacrificial little sea cucum- ber that would find its way to any leak and sort of [Returning to Service; Offline Time: 17:53] die there to thoroughly staunch any leak. Jenna [Accepting Booking; Passenger 1: Male, 27; also had a line on a new wheel and tire, if it came Destination: Apple Store Union Square; Mode: to that. She prepped some of the latest software Diamond] upgrades, although she would need Goog1639 to be present to proceed. Goog1639 reset its status as now enabled for Diamond Mode service, fresh competition for [Arrived at ACar187] [Tire Service Initiated; those nasty Platinum Mode autoCars. Emotions Fault: Pinch Flat; Visual Inspection Assessment: weren’t part of Goog1639’s programming, but it Repairable] was totally psyched.

FUTURE NOW: RECONFIGURING REALITY 77

Train your brain to see more clearly what isn’t there (yet)

by Jane McGonigal Fill in the Blanks

An intervention designed to restore normal cog- now so we can pick out something appropriate nitive function in people with amnesia, brain to wear. We practice important conversations injury, severe depression, and autism has been in our heads so we can be better prepared for shown to help anyone get extraordinarily good how others might respond. We imagine how at seeing the future. proud we’ll feel when we finally accomplish a When was the last time you traveled to the long-term goal and we use that “pre-feeling” to future? If you’re like most people, the answer is: motivate ourselves. We speculate about how our just a few minutes ago. That’s because the average lives would become different, for better and for person travels to the future a whopping 59 times worse, to help us make important decisions. We a day. idly daydream about adventures we hope to have These journeys take place not in a time some far away day to give ourselves an emotional machine, of course, but in our own minds. We lift. visualize what we’ll be doing a few hours from Whether the future in question is five hours

79 from now or five years from now, research shows that we mentally project ourselves forward in Well-being suffers enormously when time roughly once every 15 minutes that we’re awake. Not all the trips are pleasant, of course. we can’t fill in the blanks of the future. We may torture ourselves (or perhaps, as we Numerous studies have found that see it, steel ourselves) by vividly envisioning people with depression lose their ability the worst things that could plausibly lie ahead: being the victim of a crime, our plane crashing, for future mental time travel. or a loved one dying. But statistics say we visit positive futures roughly three times as often as we visit negative ones. future either. This includes patients with amne- Scientists call this “mental time travel.” It’s sia, a concussion, or brain injury. not just thinking about the future. It’s simulating That’s because, as many functional MRI and immersing ourselves in it, like a 3D movie studies have shown, a common “core network” or virtual reality in our mind. There’s a sense of of brain regions are involved in mentally simu- being present in the future, of being able to see lating both the things we’ve already done and the and feel it as if it were already happening. Most things we might do someday. This core network of us do this so often, we take our time travel includes the hippocampus, which is responsible ability for granted. But not everyone can make for constructing 3D representations of the world the trip. around us, and the medial prefrontal cortex, which monitors our environment for changes Stuck in the Present that may require us to update our goals or adapt For some people—described as having “low” our strategies. or “impaired mental time travel ability”—their When the common core network is com- minds draw a total blank when they try to picture promised by injury or illness, our simulation themselves in the future. It’s like staring into an circuitry fails. We can remember or anticipate empty crystal ball. No mental movie plays. No general facts, but not stories of our own lives. mental virtual reality fires up. That’s why, although we usually think of amne- It’s not that these people can’t think about the sia as erasing someone’s past, it actually erases future at all. They can make general predictions their future as well. Alzheimer’s patients, who just fine: tomorrow it might rain, a newStar Wars likewise often lose access to so many of their life will probably come out next winter, the United memories, also have great difficulty describing States might elect its first woman president some- in any detail what they might do in the future. day. But when they try to envision themselves Why does this matter? Our daily functioning in any of these possible futures—What would it depends on our ability to imagine our futures. physically feel like to stand in the rain tomorrow? Consider patients with advanced multiple sclero- Who would I invite to go to the Star Wars movie with sis (MS), who often experience inflammation and me? How would I personally react to breaking news scarring in regions of the brain associated with of the first woman president, and where might I be mental time travel. These patients can have trou- when I heard the news?—then the future becomes ble setting and following through on everyday fuzzy. Details are missing. Emotions don’t arise. goals. An MS patient might not bring her wallet to When they try to “look around” these futures, the store, for example, because she wasn’t able to instead of seeing detailed, three-dimensional mentally “pre-travel” there and realize she would scenes, they see nothing. And this has import- need money to pay at the checkout line. Or these ant consequences for their ability to make plans, patients find themselves chronically lonely and adapt to change, connect with others, and even bored, facing unusually empty stretches of time. simply motivate themselves to get out of bed in That’s because they haven’t been able to imagine the morning. what they might want to do, and therefore don’t make plans for themselves or with others. Trouble with Time Travel Well-being suffers enormously when we can’t Who, exactly, has such difficulty imagining their fill in the blanks of the future. Numerous studies personal futures? For starters, anyone who has have found that people with depression lose their trouble revisiting their past likely can’t visit their ability for future mental time travel. Indeed,

80 INSTITUTE FOR THE FUTURE parts of the common core network responsible What colors are most prominent? What sounds for mental time travel actually lose gray matter do you hear around you? Who talks to you as you and shrink over time in cases of severe depres- enter the circus? What do they say? What do you sion. Some scientists speculate that time travel say to them? What do you smell? What emotions impairment may indeed be one of the underly- do you feel? ing reasons why depression feels so debilitating. These specificity prompts are designed to When we can’t imagine our personal futures in encourage patients to imagine all five senses— vivid detail, the motivation and reward centers sight, hearing, touch, taste, and smell. They of the brain refuse to fire up. The “feel good” encourage a greater awareness of emotions to dopamine chemicals that flood our brains when- heighten immersion in the imagined moment. ever we anticipate the possibility of something And they turn the patients’ awareness to possi- good happening, the same chemicals that give us ble social interactions, to help increase the social energy and willpower to tackle tough tasks and simulation skills required to vividly imagine pursue our goals, simply don’t activate. That’s what others feel or might do. because the brain literally can’t anticipate something good happening—it sees “blanks” instead of positive outcomes. So the brain says, “Don’t bother.” No wonder we have a hard time getting out of bed when we’re depressed. We can’t imag- Excerpt of a conversation ine—at least not in an immersive, detailed way— anything worth getting out of bed for. between neuroscientist Research has shown that this kind of time Dr. Endel Tulving travel impairment is also experienced by people and his amnesiac suffering from post-traumatic stress disorder, people with autism, and people whose basic sur- patient “K.C.” vival resources are insecure. The first person to study the phenomenon of im- Fixing the Problem paired mental time travel, over 30 years ago, was a Canadian neuroscientist and practicing psychol- What can be done to help individuals with ogist by the name of Endel Tulving. He famously impaired time travel ability? Researchers have wrote about a patient identified as “K.C.,” who devised and tested a simple intervention to cor- chose the word “blank” to best describe his state rect problems with mental time travel. It’s called of mind when trying to picture himself in the future. “specificity training,” and multiple studies have Today, “filling in the blanks” is clinical shorthand for shown it to be effective in restoring future imag- getting better at mental time travel. ination. Here’s how it works: instead of thinking about the future, patients first practice remember- E.T.: “Let’s try the question again about the ing their past or imagining other hypothetical future. What will you be doing tomorrow?” scenarios as vividly as possible. They answer a (There is a 15-second pause.) series of questions designed to draw out highly specific details. Here are sample prompts from K.C. smiles faintly, then says, “I don’t know.” actual studies of specificity training: Think of a particular time in the past you had E.T.: “Do you remember the question?” a delicious meal. What time of day or night was it? What, exactly did you eat? Describe what it K.C.: “About what I’ll be doing tomorrow?” looked like and tasted like in vivid detail. What did the room you were eating in look like? Who E.T.: “Yes. How would you describe you’re else was there? What emotions did you feel? state of mind when you try to think about it?” What did the other person or people with you (There is a five-second pause) feel, if you can speculate or remember? Picture yourself going to a circus. What time K.C.: “Blank, I guess.” of day or night is it? What is the weather like? What might you see as you approach the circus?

FUTURE NOW: RECONFIGURING REALITY 81 The more specific details the patient includes important ways. Their imagined futures became in their written or verbal description, the higher more vivid, more detailed, and the details them- score they receive. Patients work, over several selves were rated as more interesting and cre- weeks, to earn higher and higher scores. And ative. Afterward, when asked to come up with it really helps: Studies show that over time, strategies for solving difficult problems, the patients not only get better at filling out their participants came up with more detailed plans, past memories or hypothetical scenarios with and their strategies were rated by experts as more vivid details, but their ability to see them- more likely to be effective. Empathy scores went selves in the future gets better as well. As one up when participants were asked to simulate patient put it: “It’s easier to imagine things, to what might be going on in someone else’s mind. project myself. Before, it was like I could only see And when thinking about personal futures they flashes. Now, it’s more like watching a film.” As hoped would come to pass, self-motivation and another describes it: “Now I have the feeling that willpower increased, as individuals were more I’m living the thing, I’m in it. The future events, likely to keep striving toward a future they could I can feel them.” envision more vividly. In short: specificity training not only helps restore normal mental time travel ability, it Specificity training not only helps can also be used to supercharge that ability to help anyone develop a more vivid imagination, restore normal mental time travel ability, become a more creative and effective problem it can also be used to supercharge solver, improve our powers of empathy, and that ability to help anyone develop a increase our motivation to make the futures we want. more vivid imagination, become a more creative and effective problem solver, Get Specific Right Now Now that you know how specificity training improve our powers of empathy, and works and why it helps, you can use it anytime increase our motivation to make the you want to improve your ability to fill in the futures we want. blanks of the future. (Or, you can use it with a group to improve your collective creativity and foresight!) First: before you try to travel to a future, go As their mental time travel scores go up, these to your past. Power up the “common core” net- patients get better at life planning, anticipating work of your brain by simulating in great detail others’ reactions, and motivating themselves to any experience you’ve had in your life. Pick a achieve and follow through on their goals. And moment in the near or far past, and visualize it in this isn’t just a placebo effect. Brain scans of your mind as clearly as you can. Tap into all five patients who have completed specificity training senses: try to recall as clearly as you can what show increased brain activations and connec- you saw, heard, smelled, tasted, and physically tivity in the common core network that enables felt in this moment. (Smell and taste may not be mental time travel. relevant to every memory, but make sure you really draw out the details of sight, sound, and Anyone Can Get Better at Filling in the physical sensation.) Spend at least three minutes Blanks—Even if You’re Already Good At It immersing yourself in this memory and record- If specificity training can help someone with a ing as many details as you can. You can write brain impairment get back to normal cognitive down the details, or take turns describing them function, could the same technique help some- out loud to someone else. one with ordinary mental time travel ability Pro tip: you can visit any memory at all. Just become exceptional at it? Yes! The research sug- be sure to pick different memories each time gests that it does. you use this technique. It gets easier to simulate In studies, ordinary people taken through a memories the more often you actively remember series of specificity training exercises improved them, so this technique becomes less effective if their imagination and simulation skills in several you repeat the same memories.

82 INSTITUTE FOR THE FUTURE Once your mental time travel pathways are a new car smell, because autonomous vehicles primed, turn your attention to the future. As you have only been widely available for public use try to imagine yourself having an experience in a for a few months now. I’m eating an egg salad possible future, start with questions a journalist sandwich—why not? My hands are free.” Keep would ask: What is happening? When is it hap- going and fill out as many details as you can. pening and what time of day, exactly? Where are Don’t worry about telling a story—the “plot” isn’t you—what do you see and hear in your environ- as important as setting a scene and recording ment? Who is there with you? How do you feel? as many details as you can. And don’t pressure Describe it in the present tense, as if it is happen- yourself into having any amazing insights about ing to you right now. “I am inside a self-driving the future when you practice this technique. car, in what used to be called ‘the driver’s seat.’ Insight comes later. Think of specificity training The seats are black leather. The windows are up as a crucial core-strengthening exercise to give and the air-conditioning is on. The car still has you the mental muscles for creative foresight.

Challenge Yourself to Feel All the Feelings

This issue of Future Now focuses on one specific possible future: a world in which an Internet of Action surrounds us with countless autonomous machines making their own decisions about how our lives should play out. If you want to fill in the blanks of this particular future, here’s a challenge for you. It’s called Four Future Feelings, and it’s inspired by a mental time travel method often used in scientific studies to increase the range of creative and detailed futures imagined by study participants.

Pick any story in this issue. Imagine yourself in the world it describes. Then, ask yourself:

What might happen in this future that would make ative thinking and more detailed visions. In other me feel happy? (For inspiration, think of things that words, your mental simulations of the future that have made you happy in the past. What’s an Inter- include a variety of emotions will be more surpris- net of Actions version of this experience?) ing and full of the kinds of specific details that help you build better plans, craft smarter strategies, and What might happen in this future that would make achieve greater empathy. me feel nervous or anxious? (For inspiration, think of things that have made you anxious in the past.) You can repeat this challenge with any four feelings. Just make sure you include at least one that’s What might happen in this future that would make positive and one that’s negative. Play with these me feel angry? emotions: hope, fear, pride, disappointment, curiosity, disgust, surprise, embarrassment, trust, dis- What might happen in this future that would make trust, envy, love, relief, amazement, lonely, lost, silly, me feel a strong sense of gratitude to someone excited, hurt, confident, confused, inspired. else? Who do I want to thank in this future? It could be a person, an organization, or even a bot or ma- When you can vividly imagine scenarios in which chine. autonomous machines provoke all of these feelings, you can consider yourself one of the world’s Research shows that considering a range of both most accomplished time travelers to the future of positive and negative emotions leads to more cre- Internet of Actions. Bon voyage!

FUTURE NOW: RECONFIGURING REALITY 83 Future Reconfiguring Now Reality

Tech Futures Lab Director Experts Consulted: Rod Falcon Juan Aparicio, Danielle Applestone, Nora Ayanin, Bo Begole, Senior Editor Sebastian Benítez, Takenobu Chikaraishi, Brian Christian, Allison Mark Frauenfelder Clift-Jennings, Kathryn Coleman, Mattia Crespi, Zachary Crockett, Robert Dessert, Cyril Dorsaz, Esther Dyson, Devin Fidler, Scott Fisher, Research Director Ken Goldberg, Jean-Gabriel Herbinet, Fabrice Hoerner, Rick Holman, Bradley Kreit Carrie Hughes, Aino Huxley, Wendy Ju, Kevin Kelly, SK Kim, Sean Koehl, Jeremy Kirshbaum, James Landay, Elizabeth Leytem, Andy Managing Editor Lippman, Roger Magoulas, Sami Makelainen, Hironobu Murata, Ben Hamamoto Andreas Neus, Annalee Newitz, Chris Noessel, Don Norman, Roberto Program Manager Pieraccini, Christian Simm, Lea Sims, Larry Smarr, Hartti Suomela, Namsah Kargbo Felicitas Suter, Jason Tester, Andrew Trabulsi, Alessandro Voto

Contributors Institute for the Future Jamais Cascio, Jessica Cussins, Institute for the Future (IFTF) is an independent, nonprofit 501(c)(3) Alexander Goldman, Eri Gentry, strategic research and educational organization celebrating nearly 50 Marina Gorbis, Dylan Hendricks, years of forecasting experience. The core of our work is identifying Toshi Hoo, Matthew Hutson, emerging trends and discontinuities that will transform global soci- Chris Kalaboukis, Mike Liebhold, ety and the global marketplace. Our research generates the foresight Jane McGonigal, Scott Minneman, needed to create insights that lead to action and spans a broad territory David Pescovitz, Steve M. Potter, of deeply transformative futures, from health and health care to tech- Carla Sinclair, Kathi Vian nology, the workplace, learning, and human identity. Institute for the Contributing Editor Future is based in Palo Alto, California. Carla Sinclair The Future 50 Copy Editor Lenore Weiss This work is supported in part by IFTF’s Future 50 partnership—a circle of future-smart organizations that think strategically about near- Executive Producer term choices to reshape the long-term future. Future 50 draws on a Jean Hagan half century of futures research from our labs focusing on society and technology, the economy and the environment, food and health. Its goal Production Director is to create the perspectives and expert viewpoints, the signals and the Robin Weiss data, to make sense out of disruptive forces in the present. Grounded Production team in a framework of Foresight-Insight-Action, the Future 50 partnership Robin Bogott, Maureen Kirchner, invests in critical research, boundary-stretching conversations, and Karin Lubeck, Lisa Mumbach strategic experiments that will shape the business, social, and civil landscapes of tomorrow. Illustration Trent Kuhn Tech Futures Lab Cover Illustration IFTF’s Tech Futures Lab combines a deep understanding of technology eBoy forces to identify and evaluate discontinuities and innovations in the next three to ten years. Business Development Daria Lamb, Neela Lazkani, Sean Ness www.iftf.org

TECH

201 Hamilton Avenue | Palo Alto, CA 94301 | 650-854-6322

Reconfiguring Reconfiguring Reality Future Future Now

Future Now: Reconfiguring Reality Issue 3 | 2017 INSTITUTE FOR THE FUTURE