A Continuous Bond. Speculating on the future of Conversational Agents Eric Tron Gianet, 2020 Thesis project — Interaction Design Master Malmö University Supervisor: Clint Heyer Examiner: Susan Kozel Abstract As Digital Personal Assistants get increasingly present in our lives, repositioning Conversational Interfaces within Interaction Design could be beneficial. More contributions seem possible beyond the commercial vision of Conversational Agents as digital assistants. In this thesis, Design fiction is adopted as an approach to explore a future for these technologies, focusing on the possible social and ritual practices that might arise when Conversational Agents and Artificial Intelligence are employed in contexts such as mortality and grief. As a secondary but related concern, it is argued that designers need to come to find ways to work with Artificial Intelligence and Machine Learning, uncovering the “AI blackbox”, and understanding its basic functioning, therefore, Machine learning is explored as a design material. This research through design project presents a scenario where the data we leave behind us are used after we die to build conversational models that become digital altars, shaping the way we deal with grief and death. This is presented through a semi-functional prototype and a diegetic prototype in the form of a short video. 1. Introduction 1 1.1 Opening 1 1.2 Aim of the research 1 2. Theory 2 2.1 Talking to machines 2 2.1.1 More than just digital servants 4 2.2 Being serious about fiction 6 2.3 Shall we talk about death? 7 2.3.1 Death and culture 7 2.3.2 Grief and mourning 8 2.3.3 Death in design 10 2.3.4 Death & grief in Sci-fi 13 3. Methods and Process 18 3.1 Concept Ideation 18 3.1.1 Scenario 21 3.2 Exploration 21 3.2.1 Pen & Paper Sketching 22 3.2.2 Cloning myself (sort of…) 24 3.2.3 Sketching some of the interactions 26 3.3 Prototyping 28 3.3.1 (Semi-)Functional Prototype 29 3.3.2 Diegetic Prototype 30 4. Discussion and Reflections 31 5. Conclusions 33 6. Acknowledgements 34 7. GDPR and Ethical considerations 34 8. References 35 1 1. Introduction​ 1.1 Opening ​ Conversational Interfaces have a long history of study and research, but recently a new generation of conversational agents (CAs) — i.e. dialogue systems that use natural language processing to interact and respond using human language ​(Feine et al., 2019) — has appeared, with popular products such as Google Home, Amazon’s Alexa and Apple’s Siri. This diffusion has resulted in the call for a new kind of user experience (UX) design able to meet the need to create UX for these new conversational interfaces ​(Moore et al., 2018, 2017)​. Arguably, Interaction Design (IxD) and Human-Computer Interaction (HCI) should act in a similar way ​(Munteanu et al., 2014, 2020; S. Reeves et al., 2018)​. One way interaction designers could do this is to explore new uses for these technologies, and beyond that of digital personal assistants (DPAs), which is largely the major field of use. Other researchers and designers have successfully used design fiction as an approach to explore conversational interfaces. However, this has been done either to be critical towards certain aspects of today's mainstream CAs ​(Søndergaard & Hansen, 2018) or without deviating from the idea of CAs as personal assistants anyway ​(​Our Friends Electric - Superflux​, n.d.)​. In this thesis project, I argue that Design Fiction could do much more and could serve the need of expanding the collective imagining ​(Dourish & Bell, 2014) to progress CAs beyond current applications. 1.2 Aim of the research ​ The work of this thesis will hence consist of an exploration that makes use of Design Fiction to expand and progress the concept of speech-based conversational agents beyond current mainstream applications. Being Machine Learning the backbone of today's CAs, it is explored as well to some extent as a design material. Since this speculative exploration could take many forms, a more defined focus will concern a conversational agent for dealing with grief. The focus on grief and death has been picked — after an initial brainstorming on possible areas to explore — because they hold a strong importance that encompass cultural and social practices, that can be of interest for interaction designers and be a fertile ground for speculations. A more precise research question has thus been framed as follows: What might conversational agents that support bereaved individuals be like? The work adopts a design fiction approach to explore a design space for speech based conversational agents beyond their status-quo usage. 2 As a final note, this thesis aims to be a Research through Design project that doesn't so much want to create a vision for a potentially novel product for the intelligent personal assistants market, but rather tries to “widens our perspective and extends the concerns we, as designers, should include in our praxis” ​(Krogh et al., 2015) 2. Theory​ This section tries to introduce and unpack theoretical elements that will provide a base for the design process. To begin with, conversational agents are introduced through a series of perspectives that aim to highlight the need to expand them beyond their status-quo. Next, the reasons why engaging with fiction can be fruitful for designers and researchers are unpacked and the approach of design fiction is presented. Finally, the themes of death and grief are explored through a glimpse to the psychoanalytical debate and to how different cultures deal with something that could erroneously be considered universal. Throughout this section, some projects, studies, and fictional works are introduced as canonical examples that will be analyzed later providing, thus scaffolding to the ideation of a concept. 2.1 Talking to machines ​烙 The history of conversational interfaces can be traced back to the ’60s with text-based dialog systems. From the late ’80s, different study fields started researching and deploying interactive systems based on speech ​(McTear et al., 2016, p. 51)​. Nowadays, a new generation of text and voice CAs, enabled by the renaissance of Artificial Intelligence and by advances in machine learning (ML) and natural language processing (NLP), has appeared ​(McTear et al., 2016, p. 16; Moore et al., 2018, p. 2) and these systems are now becoming ubiquitous ​(Perez, 2019)​. How these technologies are referred to in the literature is very inconsistent. Dialogue systems, voice user interfaces, intelligent personal assistants, digital personal assistants and conversational agents, are often used interchangeably, though the use of the latter often indicates these systems as social entities ​(Feine et al., 2019, p. 140)​. More simplistically, in this thesis “conversational agent” is used as a broader term compared to “digital personal assistant” used instead to identify dialogue systems that have as their application that of personal assistant. Conversational interfaces have been explored in different fields of research as, for instance, health ​(Moore et al., 2018, p. 33) or education for children ​(Xu & Warschauer, 2019)​. However, the main application for these technologie is by far that of digital assistants, whose market size is expected to reach USD 25.63 billion by 2025 ​(​Global IVA 2019-2025​, 2019)​. This is something well represented by the increasing proliferation of DPAs such as Amazon Alexa, Google Assistant and Apple's Siri ​(Perez, 2019)​. These are totem or speaker shaped devices, or even assistants inside our phones, which allow users to pronounce voice commands to interact with the services offered, such as playing music, setting reminders and alarms, doing web searches, sending messages and 3 making calls and interfacing with other smart devices such as controlling the lights in the house. Image 1 - From left to right: Google Home, Amazon Alexa, Apple Homepod Research on social robots and embodied conversational agents seems to highlight the importance of social dialogue in developing trust ​(Bickmore & Cassell, 2005)​, perceive empathy ​(Looije et al., 2010) and provide emotional support ​(Sabelli et al., 2011)​. Moreover, Feine et al. ​(2019) highlight how “many studies have shown that humans react socially to CAs when they display social cues” (p. 138). However, Clark et al. (2019) made a series of interviews to understand what people value in conversations and how these same characteristics vary with CAs, and revealed how their interviewees saw the interactions with CAs in a purely functional way. There seems to be an underlying refusal to accept the possibility of satisfying relational and social needs with "something inherently machine-like" ​(L. Clark et al., 2019, p. 7)​. This apparent contradiction with what said before, as the same authors seem to suggest, may be rooted in the task-oriented/transactional orientation of existing products on the market that hegemonize the collective imagination and that results in difficulty of envisaging technologies that do not exist ​(L. Clark et al., 2019)​. Moreover, something that interaction designers who strive to work with conversational interfaces plausibly need to start thinking about is finding ways to work with Machine Learning in their practice, as NLP e ML are key technologies on which modern CAs rely. Tools that allow to train models and create ML applications through GUI with minimal effort or knowledge and that can provide accessibility to machine learning for creators and designers are already starting to emerge ​(​AutoML​, n.d., ​Lobe​, n.d., ​RunwayML​, n.d.) and platforms such as Dialogflow specifically allows to built conversational experiences unlocking Natural Language Processing capabilities with minimal requirements of coding ​(​Dialogflow​, n.d.)​. However, these tools are currently based on the present usage and function of commercial CAs and this could reinforce the limits in imagining and creating alternative uses for these technologies. Furthermore, as a related but secondary concern, it has been argued that designers “need a hands-on understanding of AI to help influence the next generation of AI technology and applications” ​(van Allen, 2018)​.