OUT OF THE BLUE iAM 1/4 .ME EDITORIAL

The Digital Revolution is all around us. The impact of digitization on Under the overarching theme iAM, we have developed four established businesses, institutions, and start-ups is profound, and different concepts. is opening up new dimensions to deliver products and services and .me: How individualism and personalization take on new to interact with customers. dimensions with digital. However, perhaps even deeper than the industry changes are .now: How digital is changing our perception of time and our use the underlying changes that digital is having on our personal of resources and professional lives. In a positive feedback cycle, users foster the development of innovations and, at the same time, change .here: How digital is broadening the concepts of place behaviors and preferences upon using them. The expectation of and presence technology to make our lives easier and faster is growing. As digital continues changing what we do, the question arises: is digital .mind: How digital is affecting our minds and our attention changing who we are? The material is crowdsourced and linked for further reference Out of the Blue digs into these changes: new behaviors, new and reading. We are only scratching the surface and invite you to preferences, and new social mores that are emerging with such a join us on this journey. Both .here and .mind are scheduled for a strength and transformative power that we could be witnessing the January release. dawn of a new concept of the self: the i.AM.

Knowing how individuals evolve in the digital revolution will be – Oliver Wyman’s Communications, Media and Technlogy practice critical for business and institution. Out of the Blue’s purpose is to contribute to this knowledge.

2 “Are our devices turning us into a new kind of human?”

– Amber Case, cyborg anthropologist, TED Radio

3 iAM

Digital is an integral part of our lives. Technology is blurring the lines between the digital, physical and biological spheres while reshaping our perception of space and time and even the concept of our identity or persona. The expectation of technology to make our lives easier and faster is growing. As digital continues changing what we do, the question arises: is digital changing who we are?

4 iAM INDEX

.ME How individualism and personalization take on new dimensions with digital.

.NOW How digital is changing our perception of time and our use of resources

.HERE How digital is broadening the concepts of place and presence

.MIND How digital is affecting our minds and our attention

Ogilvy & Mather China Center For Psychological Research, Shenyang Phone Wall Campaign. Used under permission

5 How individualism and personalization .ME take on new dimensions with digital Digital is reinforcing an overall trend towards individualism. Over the past decade, the smartphone has become our digital umbilical cord, the physical support and connection with our digital selves. It has made computing truly personal. Now, technological advancements in artificial intelligence, machine learning, voice- based interactions, and emotional tracking technologies promise to capture the “digital me”: to deliver customized content and personalized interaction in ways that can scale to reach millions.

6

“Our goal is to build a personal Google for each and every user. Not a single Google that we all can use, but an individualized Google for everyone.”

– Sundar Pichai, CEO Google, introducing Google Assistant

7 In the UK, adult smartphone users spent nearly 2 hours online each day using a smartphone compared to just over an hour on laptops and PCs

In the US, smartphones make up the largest share (37%) of connected devices in the home. M-commerce contributes about 1 in every 6 dollars spent via digital commerce

In China, 92.5% of Internet users use their smartphones to pay rent, buy things, and even monitor school home work around the clock. On average web users spent 3.8 hours a day on Internet

Within the EU, there are 75 mobile broadband SIM cards per 100 people; up from 34 in 2012

In July 2016, Apple announced that it has sold its one billionth iPhone

8 ME AND MY MOBILE HOW THE MOBILE MADE COMPUTING PERSONAL

More people get online using mobile than any other technology: In 2015, mobile messaging Between 2010 and 2014, the number of apps outgrew usage of social media. unique mobile subscribers increased by 120% worldwide. In Q216, more than 90% of Facebook’s 1.71 billion users At the end of 2015, 3.24 billion people were on mobile devices. 84% (44% of the global population) were of the company’s $6.2 billion connected to the Internet. advertising revenue came from mobile. Our smartphone is the first device that is 100% personal, and it is becoming our primary computer: We are spending more time on our smartphones – even at home and in the office – and we increasingly use our mobiles manage our personal and professional lives, including purchasing products and services, consulting information and making payments.

Sources: CNNIC, Ofcom, ComScore, European Commission, Business Insider

9 IT IS ALL ABOUT ME HOW INTERNET IS EVOLVING INTO THE “INTERNET OF ME”

Ubiquitous information, connectivity, social networks, and an overall trend towards individualism have given people of all ages the freedom to construct their identities more freely than ever, and to share what they do and who they are with others online. Our willingness and appetite to share who we are and what we like is changing the nature and traffic of the Internet. The “Internet of we,” in which we went online primarily to find information on certain topics, has evolved into the “Internet of me,” where individuals are primarily the topic of interest and the hub of data flows. We are sharing our personal experiences and information like never before, and we spend several hours a day uploading, tweeting, and curating digital information about ourselves. Our online behaviour is amassing a wealth personal information that offers companies and organizations endless possibilities to track, analyse, and predict individual behaviour. By using data to unveil our inner identities companies are offering highly customized and personalized services.

“What we've learned over time is: it's not who they are in a superficial sense – like gender, age, even geography. It's not even what they tell you. It's what they do.”

– Todd Yellin, VP, Product Innovation, Netflix

10 Looking to the future, the next big step will be for the very concept of the “device” to fade away. Over time, the computer itself – whatever its form factor – will be an intelligent assistant helping you through your day. We will move from mobile first to an AI first world.”

– Sundar Pichai, CEO Google

DISCOVERING THE NEW .ME (1/2) THE REVIVAL OF ARTIFICIAL INTELLIGENCE TO CUT THROUGH THE CLUTTER

Among digital abundancy we are drowning in too many apps and too much data. We are always connected, toggling between different devices. Our THE DAWN OF A POST DEVICE ERA? photo collections keep growing, music and media is available at a click, and Global PC shipments reached their historical high messaging apps buzz for our attention. of 95.4 million units in 2011. Since then, global In this setting, contextual awareness and predictive analysis – underpinned demand has gradually declined. by artificial intelligence and machine learning fed by the wealth of personal Smartphone sales recorded their highest growth information stored in the cloud – point toward more intuitive, personal, and in 2010. Double digit growth followed in the relevant “information discoveries” as they are needed. period between 2011-2015. Gartner estimates the Although it might be difficult to envision today, the central role of the smartphone growth for 2016 to slow down to 7%. in our lives might fade away. According to Bob Donell of Techanalsyis, we are entering a new era of computing where the role of hardware and devices becomes secondary, and where voice-based interactions, driven by deep-learning algorithms in the cloud will enter the main stage.

Sources: Fastcompany, Gartner, Statista.com

11 DISCOVERING THE NEW .ME (2/2) THE REVIVAL OF ARTIFICIAL INTELLIGENCE FOR PERSONALIZATION AT MASS SCALE

Advancements in artificial intelligence and machine learning are Over the past year, tech companies have raced to open up their also opening up the doors for personalization at an unprecedented artificial intelligence software as open source and integrated scale, potentially reaching millions of customers. machine learning capabilities within their cloud service offerings. Companies of all sizes now have an increasing set of tools to use Use cases for artificial intelligence range from personalized artificial intelligence to create personal and smart experiences at recommendations for discovery, customer segmentation, mass scale. programmatic advertising, marketing forecasting, and prediction to helping customers navigate vast amounts of information, customer service, acquisition, and retention.

TECH COMPANIES AI TOOLS and OFFERINGS Google: In November 2015, Google Facebook: In January 2015, Facebook IBM: offers a wide range of cognitive released open source machine learning made several of its AI tools available on the computing services, including Watson software package TensorFlow Torch open source library Microsoft: In April 2016, Microsoft added IBM: In November 2015, IBM transferred AirBnB: In June 2015, AirBnB released “Cognitive Services”, a series of 22 APIs its SystemML machine-learning software Aerosolve, its open-source machine- to extend their machine learning service to Apache Software Foundation, learning tool on which Airbnb’s pricing within its cloud service offering Azure which supports several major open- algorithm relies, on the Github code- Amazon: In April 2015, Amazon launched source projects sharing platform Amazon Machine Learning within Amazon Web Services.

12 BENEFICIAL INTELLIGENCE HOW COMPANIES USE MACHINE LEARNING TO PERSONALIZE OFFERINGS

Personal finance Digit startup enables personalized Language learning app Duolingo personalizes micro savings learning experiences Digit promises its users to “save money, without thinking about Language learning app Duolingo uses a task-based, gamified, it.” The company connects to a user’s checking account, analyzes multiple-choice format to get users up to speed on learning the user’s spending habits and income history, and uses an a language. algorithm to automatically set aside small amounts of money – on average between $2 and $18 – at times when the user is least The founders used machine learning to figure out from users likely to miss it. themselves how they learned a second language best. The service is free, but Digit keeps the interests of the savings The learning app now has over 120 million users, and is using generated by its users. In March 2016, Digit said it was saving its machine learning to adjust language learning for different users users $14 million a month. and cultures, including automatic adjustments of the difficulty level, depending on personal performance and user interface tweaks for positive enforcement.

M-commerce app Wish personalizes product listings AI powered lunchbot Forkable offers tailored menu plans for team lunches The Wish shopping app connects shoppers in the United States and Europe directly with primarily Chinese wholesalers and Workers enter their dietary requirements and preferences and retailers that sell at steep discounts. The company uses machine Forkable’s “lunchbot AI” creates what the company calls “the learning algorithms and natural language processing to fine- equivalent of a Spotify playlist” for each person’s lunch. tune and curate what products are shown to which customer based on browsing behavior, showing only a handful of highly Users can replace, customize and cancel orders using the app’s targeted products. web interface that are requested automatically and scheduled to arrive at a specified time. Wish expects to reach $2 billion in sales in 2016, and has 100 million registered users. The system is designed to manage differing employee allowances and budgets.

US retailer Macy’s uses AI shopping assistant backed by Major music streaming services personalizes music discovery IBM’s Watson Spotify, Apple Music and Google Music Play use a combination Macy’s On Call is a mobile tool that will help shoppers find of taste prediction algorithms and human curation combined their way round 10 US stores piloting the system. The system with data analysis to help users navigate the vast ocean of music uses Watson’s Natural Language Classifier API and Satisfi, an streaming catalogues and differentiate in a commoditized music intelligent engagement platform, to understand and respond space. Read more to input. The service will be customized by location to tell customers where the shoe department is in a particular store, or whether that branch has a certain size in stock.

13 We curate songs purely based on what we think the listeners are going to like and what we see in the data.”

– Doug Ford, Spotify CASE STUDY

HUMAN CURATION AND MACHINE LEARNING FOR MUSIC DISCOVERY How curated playlists help users navigate the vast ocean of streamed music and keep them coming back for more

On-demand music streaming is becoming increasingly popular, and although paying subscriber growth has been slow, users are listening to more music On-demand audio music streaming overtook video than ever, largely thanks to personalization features such as playlists that use a music streaming for the first time in the US in the combination of machine learning and human curation to cater for a wide array first six months of 2016 according to Nielsen. of music interests. Spotify says 50% of its more than 100 million Around 100 music curators are working at Apple, Google, and Spotify, to users globally are listening to its human- assemble, name, and update tens of thousands ready-made playlists for every curated playlists (not counting those in conceivable genre, activity, or mood. Spotify has around 4,500 playlists, Apple “Discover Weekly”). Music has 14,000 and Google Play Music has tens of thousands. According to an industry estimate, 1 out of The music curators have access to a suite of proprietary tools that streamline every 5 plays across all streaming services today the maintenance process of playlists, such as performance trackers of a playlist happens inside of a playlist. (age range, gender, region, time of day, subscriber tier, etc.), and breakdown analysis of individual songs in the playlist (number of plays, average play length, number of skips, number of saves/likes, etc.).

Source: Buzzfeed, Techinsider

14 CASE STUDY

HUMAN CURATION AND MACHINE LEARNING FOR MUSIC DISCOVERY How Spotify leverages user generated content, taste predictive algorithms, and human curation to “connect the right music with the right ears,” for millions of users on a weekly basis

By using big data analysis and deep learning, Spotify has been •• In August 2016, Spotify launched “Release Radar”, a weekly able to draw a detailed map of every user’s taste that is feeding personalized playlist of the artists that people follow mixed with automated personalized weekly playlists: new discoveries based on the listening habits •• “Discover Weekly”, is a weekly personalized “mixtaple of fresh The enabling technology partly comes from The Echo Nest, a music music” of 30 songs based on collaborative filtering algorithms. intelligence platform and MIT Media Lab startup from 2005 acquired The service was launched in July 2015 and now has40 million by Spotify in 2014, that has synthesized billion of data points for unique users with 5 billion tracks streamed. more than 37 million songs to power smart music applications. •• “Fresh Finds” is a service aimed at “understanding the brand new,” launched in the form of six generic genre playlists, updated every Wednesday. To feed the updates, Spotify crawls new trending artists on music blogs and review sites on the web. The identified artists are then filtered through a real-time analysis of Spotify listening behavior of around 50,000 early adopters who tend to discover new artists before they break. The identified songs are then sorted and ordered by Spotify music curators into six different genre buckets

Sources: The Verge, Fivethirtyeight, Quartz, Brian Whitman blog (Principal Scientist, Spotify), Fast Company, Spotify

15 The end result is a playlist of songs designed to feel like it was handcrafted for you by a good friend. I see personalization as a huge part of not just attracting users, but keeping them.”

– Matthew Ogle, product owner, Discover Weekly

CASE STUDY

HUMAN CURATION AND MACHINE LEARNING FOR MUSIC DISCOVERY How personalization and customer engagement has resulted in lower churn, and how Spotify might use customer insights and big data to launch its own music label

The bet on recommendations seems to pay off in an increasingly Quartz envisions a future in which Spotify uses its giant database commoditized music world where players have access to the same and clever algorithms to find up-and-coming bands for a music music catalogues: According to a recent report by Cowen & Co, label of its own, similar to how Netflix produces its own content the monthly churn of Spotify’s 35 million paid subscribers is 2.2%, based on Big Data insights into its users’ tastes and preferences. compared to Apple Music that has monthly churn rate of 6.4% for its 15 million paid subscribers*. Both “Discover Weekly” and “Fresh Finds” are helping up-and- coming artists: Some bands have landed on the Fresh Finds playlist after being played only 10 times on the platform, and more than 8,000 artists on Spotify had half of their total listeners find them on Discover Weekly.

Sources: Cowen & Co, Quartz Note: Updated subscriber figures as of September 2016: Spotify 40 million paying customers, Apple Music 17 million paying customers

16 The complexity is too much. We need to tame it. We need to be able to make it much more natural for people to get things done, vs. this thing about let me remember the 20 apps I need to get anything done.”

– Sataya Nedella, CEO Microsoft

LET’S JUST TALK (1/2) THE REVIVAL OF ARTIFICIAL INTELLIGENCE FOR “CONVERSATION AS A PLATFORM”

Artificial intelligence is also spurring new user interfaces based on conversation. The growing popularity of messaging apps combined with advancements Virtual agents and chatbots will be the top in AI has seen the revival of the chatbot for “contextual commerce” or consumer applications of artificial intelligence “conversational commerce” that seek to sell services and products based on over the next five years, according to a the conversations users have within an messaging app. Based on the success of consensus poll by TechEmergence, a marketing Chinese WeChat, that was among the first to invite business to use chat based research firm for AI and machine learning. conversations with its users. Facebook, Microsoft, and Google have been introducing chatbots recently. At Microsoft’s worldwide partner conference in July 2016, Microsoft’s CEO Sataya Nedella said that chatbots will “fundamentally revolutionize how computing is experienced by everybody.” According to Nadella, the change will be gradual and bots will start off by augmenting apps, but in time, human language will be taught to all computers and become the new interface.

Source: TechEmergence

17 20 percent of queries on Google’s mobile app and on Android devices are voice searches.”

– Sundar Pichai, CEO, Google Google I/O keynote, May 2016

LET’S JUST TALK (2/2) THE REVIVAL OF ARTIFICIAL INTELLIGENCE FOR VOICE INTERACTION

If recent advancements in speech recognition using deep learning keep their current pace, chances are high that we will once again rely mainly on our voice MACHINES ARE ALMOST AS GOOD AS when interacting with our smartphones (and other devices). PEOPLE AT SPEECH RECOGNITION Voice-operated virtual assistants such as Apple’s Siri, Microsoft’s Cortana, and Over the past year, tech companies have Google Now are readily available on most smartphones. Newer devices, like reported major advancements in word error Amazon’s Alexa, offer a simple way to look up information, switch on the lights, rates for speech recognition using deep learning turn on the music, and build shopping lists with your voice. and neural networks: Voice-based services can be can be accessed by the simplest of devices, with In February 2016, Chinese Baidu reported an little more than an audio input, an audio output, and a wireless connection since error rate of 3.7% for short phrases using its all the computing power takes place in the cloud. Deep Speech 2. In April 2016, the IBM Watson team reported a word error rate of 6.9%. In June 2015, Google reported an 8% error rate, down from 15% in 2014, using neural networks that are now more than 30 layers deep. In comparison, the transcription of human error rate is around 4%.

Sources: MIT Technology Review, Silicon Valley AI Lab, IBM Watson Developer Community, Venturebeat

18 BOTS & VIRTUAL ASSISTANTS HOW COMPANIES ARE USING TWO-WAY TEXT-BASED OR VERBAL CONVERSATIONS TO RESPOND TO CONSUMER NEEDS IN A MATTER OF SECONDS

Microsoft Facebook In 2014, Microsoft launched its intelligent personal assistant In April 2016, Facebook launched chatbots for businesses Cortana as a key ingredient for “makeover” of the future within its Messenger app, to allow businesses to deliver operating systems for Windows Phone and Windows. In 2015, automated customer support, e-commerce guidance, content, Cortana was also launched for Xbox, iOS and Android. and interactive experiences through chatbots. The platform is primarily powered by Facebook’s Applied Machine Learning team Microsoft has also developed Xiaoice, a chatbot created by a and Wit.ai, a startup Facebook acquired in 2015. team of software developers and psychological experts to create a balance of IQ (intelligence) and EQ (emotional intelligence). In August 2015, Facebook launched a beta of a personal chatbot Xiaoice, that has the personality of a 17-year-old girl, has more assistant named M in San Francisco, powered by a combination than 40 million registered users, and approximately 25% of them of AI and humans, so called “M trainers”, that complement AI to (10 million people) have said “I love you” to her. perform trickier judgment calls and tasks that AI can’t cope with. In March 2016, Microsoft was forced to withdraw its infamous M can suggest and book reservations, find birthday gifts, and chatbot Tay after Twitter users taught it to be racist. retrieve information. Read more

Google Amazon Google started investing in voice recognition, language In June 2015, Amazon officially launched its Alexa Voice Service, understanding and machine translation long before most of that powers its wireless home speaker Echo, the Alexa skills set, its rivals. and the Alexa Fund. At its May 2016 developer conference, Google launched The Alexa platform is open to developers to allows others to build Google Assistant, a service that is integrated in its new Alexa skills and integrate Alexa into their own products. In June conversational user interface products: Allo, its new 2016, Amazon said that its internal Alexa team had more than independent chat bot app, as well as Google Home, its 1,000 employees and that tens of thousands of developers are smart portable speaker that is powered by voice assistance building skills for Alexa. technology (launched in September 2016). The Alexa Fund has invested in 16 startups, with a focus on smart The Google Assistant is seen as an important step in Google’s home and wearable products to date. Over the next year, the ambition to extend its search functions across all kinds of Alexa Fund will be expanding investments into startups that focus devices and interfaces. on robotics, developer tools, healthcare, accessibility, and more. Read more

19 What we are seeing is a clear uptake of activity. The majority of interactions with the bot are placing an order. The vast majority are new customers for our brand. We are reaching the early-adopter audience that the general advertising doesn’t hit. We are reaching the younger demographics.”

– Chris McCann, President, 1-800-Flowers.com CASE STUDY

FACEBOOK CHATBOTS Leveraging technology to enable conversation a with 1 billion audience

In September 2016, there were more than 30.000 chatbots on the Facebook chatbots platform, with more than 300 million people using its audio and MONTHLY ACTIVE USERS MESSAGING APPS video-calling features every month. Examples of companies that have used Facebook Messenger: 1 billion (July 2016) chatbots include: Whatsapp (owned by Facebook): 1 billion Disney: Disney launched a bot for release of Zootopia, allowing fans to chat with (Feb 2016) the movie’s lead character, Officer Judy Hopps, extending the story to the real world by enlisting fans to help solve multiple ZPD (Zootopia Police Department) WeChat: 762 million (Q116) cases. Millions of messages were sent, with an average chat duration time of LINE: 218 million (April 2016) several minutes. Kik: 175 million NBA: The NBA’s bot on Messenger was focused on providing fans instant Snapchat: 150 million (June 2016) access to highlights for the NBA finals and draft 2016. The bot had over 350,000 interactions. Shopify: Shopify offers the platform components needed for its merchants to send post-sale receipts and shipping updates in Messenger. More than 16,000 Shopify merchants offer this service to their customers, with nearly 200,000 order status updates sent every single day.

Source: Facebook, messengerblog.com, The Verge

20 CASE STUDY

MYKAI PERSONAL BANKING BOT Using chatbots to enable life-style banking

Personal finance chatbot maker Kasisto, a spinoff of Siri-maker SRI International, aims to make financial services more ubiquitous and enable lifestyle banking by DIGIBANK IN INDIA, AN ALL DIGITAL embedding financial decision-making in consumers’ everyday life. EXPERIENCE POWERED BY THE KAI VIRTUAL ASSISTANT CHATBOT In June 2016, the company launched two chatbots for intelligent banking conversations on Facebook Messenger, Slack, and via SMS in the US: MyKAI for In April 2016, Singaporean retail bank DBS personal finances and KAI Banking to help banks interact with their customers on launched a mobile-only bank, offering a savings messaging platforms. account with a 7% interest and an unmetered debit card. By using MyKAI, people are able to check their account balances, categorize and track their spending, pay with Venmo and in the future monitor their investment The bank relies 100% on smartphones, it has no portfolio. MyKAI has been trained to answer more than 1,000 questions about branches, and there is no paper-work involved for financial terminology and the user’s personal finance. opening up an account. Instead, an account can be activated from any of the 500-plus Cafe Coffee Day The KAI chatbots for banks has been used by Digibank in India and will be outlets across India, and customer authentication implemented by Royal Bank of Canada. is done by a biometrics-enabled ID. The customer service is provided by a 24x7 AI- driven Virtual Assistant from Kaisto, trained to answer over 10,000 questions, from customer on- boarding to user education to customer service. It also includes an intelligent, intuitive budget optimizer that helps customers be smarter about

Image: Mykai press kit their money.

21 CASE STUDY

DONOTPAY: “THE WORLDS FIRST ROBOTIC LAWYER” How a Stanford student used artificial intelligence to provide personalized simple legal advice and help people understand their rights

In autumn 2015, 19-year-old Stanford student Joshua Browder Browder plans to expand the artificial intelligence lawyer, which can launched DoNotPay, a chatbot that asks the user a series of also work out compensation for delayed flights, to help vulnerable questions designed to work out whether a parking ticket can be groups navigate complicated legal systems, including people who appealed within 30 seconds. Examples of the questions asked are are HIV-positive and refugees in foreign countries. The latter will use whether there were clear parking restriction signs or if the driver IBM Watson to translate Arabic and English. was travelling to the hospital urgently. After determining that an appeal is viable, it then walks the user through the steps of appeal. The service is available in London and New York, and has been used by over 250,000 people. Of the $4 million worth of tickets overturned, about 150,000 were in London. It is currently only available through the DoNotPay website, but Browder is in talks with Facebook to incorporate it in the Messenger app.

Source: The Guardian, Venture Beat

22 Alexa may be Amazon’s most loved invention yet – literally – with over 250,000 marriage proposals from customers and counting.”

– Jeff Bezos, CEO, Amazon

CASE STUDY

AMAZON ECHO Echo, the first device with voice as main interface

Amazon’s Echo is a 24-cm tall cylindrical wireless device that doubles as smart home appliance and portable Bluetooth speaker. It is the first computer for which The most popular items being ordered through the main interface is voice. The only physical controls are an on-off switch, a Alexa are coffee, batteries, pet food, and button to mute the microphone, and a volume knob. paper towels. Echo connects to Wi-Fi and the Alexa Voice Service to play music, provide information, news and weather updates, order Amazon items, control compatible smart home devices, and activate “Alexa skills” developed by third parties by using voice commands. Amazon is now working on endowing Alexa with conversational skills. In September 2016, the company announced a $2.5 million prize to spur the development of conversational artificial intelligence. The company will sponsor up to 10 university student teams with $100,000 each, and will award $500,000 to the best team. Additionally, a prize of $1 million will be awarded to the winning team’s university if their socialbot achieves the grand challenge of conversing coherently and engagingly with humans on popular topics for 20 minutes. The Echo is currently available in the US and the UK. Customer response has been overwhelmingly positive with over 40,500 reviews and an average customer rating of 4.4 stars.

Source: MIT Technology Review, Press release: The Alexa Price, Press release: Amazon Q316 results Conversational artificial intelligence – The Alexa price Image: Amazon press kit

23 As technology is increasingly applied to situations where it must interact with everyone, it is all the more vital that it does so in a way that is courteous and respectful of people’s feelings.”

–  Dr. Rosalind Picard, Affective Computing Group, MIT Media Lab

THE VALUE OF CAPTURING THE EMOTIONAL.ME (1/2) THE DAWN OF ARTIFICIAL EMOTIONAL INTELLIGENCE

As our expectations continue to rise, and we increasingly expecting technology (and companies) to be empathetic, to understand how we feel, to simplify our According to study by Forrester, emotion and lives, and to help us to become the people we want to be. Solving emotional how an experience makes a customer feel, is the intelligence is a natural path to solve true machine learning and make machines number one factor in customer loyalty - ahead understand us better. of effectiveness and ease - across 94% of the industries studied. Better cameras, computer vision algorithms, emotion-sensing facial analysis software, and semantic analysis are helping machines get better at reading the important clues our body language and speech reveal about our mood and intent, including the movement of our eyes, our gestures, the way we talk, and even how we move our heads. Companies have a lot to gain by connecting with customers’ emotions: An eight year research project at Harvard identified 300 universal motivating emotions, and validated their impact on consumer behavior through extensive surveys and data collection. Twenty-five important emotional motivators, including desires to “stand out from the crowd,” “have confidence in the future,” and “enjoy a sense of well-being,” significantly affected customer value across all the categories studies. Fully connected customers were found to spend twice as much annually as highly satisfied customers.

Sources: Harvard Business Review, Forrester Image: Frank Behrens, Flickr Creative Commons. Original photo has been cropped

24 Algorithms are noise-free. People are not. When you tell team leaders that there is 50% variability [in expert judgment] when they expected 5% or 10%, then they’re willing to take an algorithm.“

– Daniel Kahneman, Nobel-prize winning behavioral economist and phycologist

THE VALUE OF CAPTURING THE EMOTIONAL.ME (2/2) AI FOR SMARTER PROFESSIONAL AND PERSONAL DECISIONS

Behavioral economists and philosophers argue that artificial intelligence We make extremely poor decisions can help us overcome our human flaws, such as overconfidence, limited about how we should manage attention, and cognitive biases, which inevitably cause errors in our judgment and decision making. relationships. We have little idea what job to focus on and when to quit. According to Daniel Kahneman, Nobel-prize winning behavioral economist and psychologist, much of human error is not even attributable to a We don’t know what to spend our systematic cause or biases, but to “noise.” Noise is random, unpredictable, and impossible to explain. By using algorithms, organizations can temper money on. We get holidays wrong, human judgment with “disciplined thinking.” have no clue how to repair friendships or handle tricky employees, and According to philosopher Alain De Botton, artificial intelligence can give us a helping hand in managing our personal lives too, since “our fumble as to how to reconcile with emotional frailties far outweigh our incapacities in raw mathematics or data our parents.” management.” The six areas that artificial emotional intelligence are set to revolutionize include self-knowledge, education, news media, art, shopping, –Alain De Botton, author and philosopher and relationships.

Source: Knowledge Wharton, Wired Image: Riccardo Cuppini, Flickr Creative Commons

25 ARTIFICIAL EMOTIONAL INTELLIGENCE HOW COMPANIES ARE USING SPEECH RECOGNITION, COMPUTER VISION, EMOTION-SENSING FACIAL ANALYSIS SOFTWARE, AND SENSORS TO UNDERSTAND AND RESPOND TO USER’S EMOTIONS

iDAvatars offer conversational avatars for customer service MIT Personal Robotics Group helps children learn Spanish Sophie is a highly advanced, emotionally intelligent Avatar who MIT’s Personal Robotics Group led has created Tega, a cuddly provides patient care and data feedback for healthcare clients by social robot that uses an Android device to process movement, talking, listening, comforting, soothing, and relating to patients. perception, and thinking and can respond appropriately to individual children’s behaviors as it teaches Spanish to Kate is a Virtual Assistant who provides 5-star customer preschoolers. Over time, it learned how the cues influenced a service for financial, insurance, education, healthcare, retail, student’s engagement, happiness, and learning successes. As the government, and other clients. Kate guides a customer through sessions continued, it ceased to simply mirror the child’s mood and even the applications, instructions and forms and answers began to personalize its responses in a way that would optimize questions and charting progress along the way. Read more each student’s experience and achievement. Read more

Emotional robot Pepper greets guests and keep SRI International is working on a new generation of virtual people company assistants that respond to users’ emotions Softbank Japan has developed an emotional robot called Pepper SRI International, the research lab that made Apple’s Siri, has together with French company Aldebaran that uses facial developed a technology called SenSay Analytics, that is designed recognition to pick up on sadness or hostility, and voice recognition to identify emotional state based on a variety of cues, including to hear concern. In Japan, over 7,000 Pepper robots have been typing patterns, speech tone, facial expressions, and body used to greet guests, answer questions and play with kids. movements. 2016 will see the launch of Pepper in Europe and US for business For example, the virtual assistant might be able to tell from a applications. Read more patient’s pattern of speech if he or she were becoming confused, then slow down. Read more

Chatbot Karim delivers physiological support to Biometric bracelet help users “to become the best version Syrian refugees of themselves” Silicon Valley startup X2AI has created an artificially intelligent Zenta is a wearable biometric bracelet that uses advanced chatbot called Karim that can have personalised text message sensing technology, and machine learning algorithms to build a conversations in Arabic to help people with their emotional holistic view of the wearer’s emotional and physical wellbeing. problems. Zenta constructs a personalized digital profile and provides the As the user interacts with Karim, the system uses natural user with actionable insights on how to maximize their personal language processing to analyse the person’s emotional well-being. Read more state and returns appropriate comments, questions and recommendations. Read more

26 CASE STUDY

EMOTIONAL ROBOT PEPPER SET TO CONQUER THE WORLD Pepper, the emotional robot

Softbank Japan has developed an emotional robot called In May 2016, Pepper was launched in Europe at a price of €19,900, Pepper together with French company Aldebaran that uses including professional services. A partnership has already been closed facial recognition to pick up on sadness or hostility, and voice with several European companies. Usage cases include Pepper as a recognition to hear concern. patient companion, retail assistant, and receptionist. The physical shape of Pepper was deliberately design to incentivize Pepper will be available in the US at the end of 2016. Softbank engagement. Its height, shape, and gesticulating arms are all and Google released an Android SDK to help Pepper understand designed to show empathy. Americans better. In Japan, over 7,000 have been sold to families and businesses. Pepper robots have been used Softbank and Nescafe stores to welcome guests, answer questions and collect data. SoftBank has also closed a partnership with Honda to use the artificial-intelligence system from Pepper to build a car that can read a driver’s emotions and “talk” to them to keep them company.

Sources: Pepper Partnerships in Europe, Wired Image: Softbank Robotics

27 CASE STUDY

EMOTION TRACKING SMART BRACELETS Wearables that focus on emotional well-being over physical health to help you “become the best version of yourself”

A number of companies, with varying success, are launching wearable smart bracelets that make recommendations to alleviate stress and unhappiness, sending EMOTIONAL TRACKING TO IMPROVE their wearers off for a walk or a meditation when emotions run high. THE PERFORMANCE OF FINANCIAL Emotion-tracking smart bracelet SenceBand by biotech company Planexta Sence TRADERS tracks 64 different emotional states from the wearer’s wrist. It uses a clinical-grade Andrew Lo, director of the Laboratory for Financial ECG (EKG)-tracking technology to track advanced heart rate analysis, including Engineering at the Massachusetts Institute of heart rate variance (HRV). The SenceHub uses algorithms to turn the raw data into Technology, has been conducting simulations of actionable analytics, and sends of notifications about the emotional states of the stressful situations facing financial traders. The user of the people in his/her network. ScenceBand is available for preorder on simulations used wrist watch sensors to measure Indiegogo and Kickstarter for $129. the pulse and perspiration of traders to warn them to step away from their desks to avoid high Announced in late 2015, the Feel Wristband by Sentio Solutions uses four integrated risk behavior when emotions were running high. sensors that measure the body’s skin temperature, blood volume pressure, and The information could also be monitored by risk galvanic skin response. Based on the data it gathers, the app graphs a visualization managers to spot problems brewing on a specific of emotion levels throughout the day and offers personalized suggestions for stress desk, such as unauthorized trading, before too reduction and self-improvement. Feel can be preorder at a price of $149. much damage is done. Meanwhile Vinaya, the London-based manufacturer of smart jewelry and other Likewise, Humanyze, a startup founded by MIT wearables, that built the AI and machine learning for its devices in-house, went into graduates has created a sensor-laden badge administration in December 2016. The company’s crowdfunded biometric bracelet that transmits data on speech, activity, and Zenta, that raised over $270,000 on Indiegogo and was scheduled to ship in April stress patterns. 2017 will not ship, and the company is reportedly planning a restructuring to focus on B2B instead of B2C. 28 FOOD FOR THOUGHT

PRIVACY PROTECTION FOR .ME A personal privacy assistant to manage your data

Technology may also provide us a helping hand in coming to terms The privacy assistants alert users about practices the users may with a problem it caused in the first place – how to manage our not feel comfortable with and even occasionally nudge users to privacy and personal data in the digital age. reconsider the implications of some of their privacy decisions. Research at Carnegie Mellon has shown that when people learn A team at Carnegie Mellon University is creating a personalized exactly how many times apps share information, they rapidly act to privacy assistant app, which uses natural language processing, limit further sharing. machine learning, crowdsourcing, and a private interface design to learn the privacy preferences of their users over time, with Another potential use case of the assistant would be to recognize the ultimate aim of making it simpler for users to manage their when an app is leveraging user data without asking permission to smartphone app permissions. do so. By using machine learning, the assistant analyzes a user’s response to a small number of questions, focusing on the particular apps they have on their phones. Based on the results, the agents are capable of semi-automatically configuring many settings and making privacy decisions on their user’s behalf.

Sources: CIO.com, Carnegie Mellon University

29 By design, our feeds are largely ephemeral, and we treat them that way, parcelling out seemingly insignificant moments – until, all of a sudden, we stop. That collection of moments, once thought to be of infinite supply, will now outlive your physical form, freezing in time the flow of your whole existence.”

– Ghost in the Machine, Jenna Wortham, New York Times

FOOD FOR THOUGHT

DIGITAL MOURNING, MEMORIES AND MINDCLONES How do we cope with digital death and digital legacies?

Technology is reshaping our relation to death. First, we die tangibly and cease to exist in the physical world. But what happens to our VIRTUAL GRIEF AND VIRTUAL DEATH digital selves and to our digital legacies? As our digital legacies outlive our physical selves, technology is Viktor Mayer-Schönberger of the Oxford Internet Institute says that offering new ways to mourn and share memories of someone we might be producing more memories than we can cope with: that has passed away. “Forgetting is built into the human brain. So for thousands of years In May 2015, Eugenia Kuyda, co-founder and CEO of a we’ve developed ways to preserve special memories.” Today, though, Russian artificial intelligence startup Luka, developed a it is quicker and easier to save every bit of our vast digital trails than it chatbot that lets anyone talk to her dearly departed friend is to sort and discard what we don’t want. Roman Mazurenko, a tech entrepreneur who died in a car In her book, Virtually Human: The Promise and the Peril of Digital accident in November 2015. The Roman chatbot is a dialogue Immortality, futurist Martine Rothblatt argues that our “mindfiles” model that uses smaller datasets of past conversations with (i.e. our uploaded thoughts, memories, preferences, beliefs) create Roman (photos, texts, articles) on top of a neural net. Users a virtual existence that software engineers can “mindware” and can text or chat with @Roman like they would have done use to create humanlike consciousness in computer software, or when he was alive–and the Roman AI replies like Roman a cyberconsciousness. In the next decade or two, these efforts would have done in real life. will result in the first digital copies of our identities, which will be The Hereafter Institute evaluates a person’s digital our “mindclones”. afterlife using new technologies. It offers multiple digital preservation options, including 3D body scanning, wearable memorialization, and the embedding of personal data into everyday objects.

Sources: The Atlantic, New Scientist, LACMA Unframed, The Verge

30 Far from giving us a stronger sense of our own identities, the 1 million odd selfies taken every day across the world (the average millennial is expected to take 25,700 selfies in his or her lifetime) will only propagate insecurities and provoke precisely the kind of neurotic and self-questioning behavior that characterizes adolescence.”

– Elsa Goodhardt, author of the book I Selfie, Therefore I Am FOOD FOR THOUGHT

NARCISSISM, WEALTH DISTRIBUTION, AND DIGITAL DIVIDE How do we cope with the flipsides of individualism?

Warning voices point towards the rise of a modern narcissism, with the Narcissism has overtaken altruism as the selfie as the ultimate expression of self-promotion, and the opportunities driving force behind social connection. lost in using social media to drive social connection. Popular culture fuels this behavior, Despite advancements in advancements in technology and productivity, the reinforcing the idea that being social gap in income and wealth distribution around the world is increasing: is actually about self-promotion.” According to Oxfam, the gap between the global richest and the global poorest has widened. In 2015, 62 people had the same wealth as the – Peter Kim, Chief Digital Officer, Cheil Worldwide poorest half of humanity, compared 388 people in 2011. According Allianz, the distribution of wealth is becoming increasingly unequal in developed countries with Gini coefficients well above 70, particularly in the US (80.52) Sweden (79.9), UK (75.72), Austria (73.59) and Germany (73.34). According to a report published by the World Economic Forum, the digital divide between developed countries and many emerging economies is increasing because of the high cost of digital access.

Sources: Telegraph, Oxfam, Allianz, World Economic Forum Image: 'Selfi #6: Flawless' by Dean Christensen, used under permission

31 DIVE DEEPER

TEXT VIDEO & AUDIO

Affective Computing, Rosalind Picard, Affective Computing The Post Device Era, Bob O’Donnell Group, MIT Media Lab Emotional Intelligence, The School of Life AI’s Language Problem, MIT Technology Review, August 2016 Deep Learning in Practice: Speech Recognition and Beyond, The New Science of Customer Emotions, Harvard Business Review, Andrew Ng Baidu November 2015 Recommender Systems, Harvard University 20+ Emotion Recognition APIs That Will Leave You Impressed, and Concerned, Nordic APIS, December 2015 Me, My Selfie & I: Narcissism or Empowerment? Institute of Ideas Baidu’s Deep-Learning System Rivals People at Speech How Computers Are Learning to Be Creative, TED talk, Recognition, MIT Technology Review, December 2015 Blaise Agüera y Arcas, principal scientist at Google Why Do So Many Digital Assistants Have Feminine Names? HereafterLife, Gabriel Barcia-Colombo & Sarah Rothberg, The Atlantic, March 2016 Hereafter Institute The Real Story of How Amazon Built the Echo, Are Our Devices Turning Us Into A New Kind Of Human? Bloomberg, April 2016 TED Radio, Amber Case Mind vs. Machine, Brian Christian, The Atlantic, November 2011 Digital legacy: The Fate of Your Online Soul, New Scientist, April 2011 For Sympathetic Ear, More Chinese Turn to Smartphone Program, The New York Times, July 2015 Art, Death, and Memorial: A Participatory Event with Gabriel Barcia-Colombo, LACMA, the LA County Museum of Art, August 2016 Speak Memory – When Her Best Friend Died, She Rebuilt him Using Artificial Intelligence, The Verge, October 2016

32

Out of the Blue is a periodic publication from the Oliver Wyman CMT (Communications, Media and Technology) Research and Knowledge Management Team aimed at highlighting trends within the communication, media and technology space.

The Out of the Blue Team

Communications, Media and Technology Knowledge Management Mia Burner, Senior Knowledge Manager Borja Xercavins, Senior Consultant

Editorial Lorenzo Miláns del Bosch, Partner, Madrid Iria Aparicio, Manager, Knowledge Management

Marketing Kate Wildman

Design Paloma Sanchez Michael Tveskov Melissa Ureksoy

For more information, please visit www.oliverwyman.com

Copyright© Oliver Wyman