Eliciting and Analysing Users' Envisioned Dialogues with Perfect Voice Assistants
Total Page:16
File Type:pdf, Size:1020Kb
Eliciting and Analysing Users’ Envisioned Dialogues with Perfect Voice Assistants Sarah Theres Völkel Daniel Buschek Malin Eiband [email protected] [email protected] [email protected] LMU Munich Research Group HCI + AI, LMU Munich Munich, Germany Department of Computer Science, Munich, Germany University of Bayreuth Bayreuth, Germany Benjamin R. Cowan Heinrich Hussmann [email protected] [email protected] University College Dublin LMU Munich Dublin, Ireland Munich, Germany ABSTRACT 1 INTRODUCTION We present a dialogue elicitation study to assess how users envision Voice assistants are ubiquitous. They are conversational agents conversations with a perfect voice assistant (VA). In an online sur- available through a number of devices such as smartphones, com- vey, N=205 participants were prompted with everyday scenarios, puters, and smart speakers [16, 68], and are widely used in a number and wrote the lines of both user and VA in dialogues that they imag- of contexts such as domestic [68] and automotive settings [9]. A ined as perfect. We analysed the dialogues with text analytics and recent analysis of more than 250,000 command logs of users in- qualitative analysis, including number of words and turns, social teracting with smart speakers [1] showed that, whilst people use aspects of conversation, implied VA capabilities, and the influence them for functional requests (e.g., playing music, switching the light of user personality. The majority envisioned dialogues with a VA on/off), they also request more social forms of interaction with as- that is interactive and not purely functional; it is smart, proactive, sistants (e.g., asking to tell a joke or a good night story). Recent and has knowledge about the user. Attitudes diverged regarding reports on smart speakers [42] and in-car assistant usage trends [41] the assistant’s role as well as it expressing humour and opinions. corroborate these findings, emphasising that voice assistants are An exploratory analysis suggested a relationship with personality more than just speech-enabled “remote controls”. for these aspects, but correlations were low overall. We discuss Moreover, voice assistants are perceived as particularly appeal- implications for research and design of future VAs, underlining the ing if adapted to user preferences, behaviour, and background [19, vision of enabling conversational UIs, rather than single command 20, 22, 45]. Since conversational agents tend to be seen as social “Q&As”. actors in general [58], with users often assigning them personali- ties [69], personality has been highlighted as a promising direction CCS CONCEPTS for designing and adapting voice assistants. For example, Braun et • Human-centered computing ! Empirical studies in HCI; al. [9] found that users trusted and liked a personalised in-car voice Natural language interfaces. assistant more than the default version, especially if its personality matched their own. Although efforts have been made to generate KEYWORDS and adapt voice interface personality [49], commercially available Adaptation, conversational agent, dialogue, personality, voice assis- voice assistants have so far taken a one-size-fits-all approach, ig- tant noring the potential benefits that adaptation to user preferences arXiv:2102.13508v2 [cs.HC] 6 Apr 2021 may bring. ACM Reference Format: Systematically adapting a voice assistant to the user is chal- Sarah Theres Völkel, Daniel Buschek, Malin Eiband, Benjamin R. Cowan, lenging: People tend to show individual differences in preferences and Heinrich Hussmann. 2021. Eliciting and Analysing Users’ Envisioned for conversations when asked about their envisioned version of Dialogues with Perfect Voice Assistants. In CHI Conference on Human Factors in Computing Systems (CHI ’21), May 8–13, 2021, Yokohama, Japan. ACM, a perfect voice assistant [83]. Personalisation also harbours cer- New York, NY, USA, 15 pages. https://doi.org/10.1145/3411764.3445536 tain dangers, as an incorrectly matched voice assistant may be less accepted by a user than a default [9]. Current techniques for Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed generating personalised agent dialogues tend to take a top-down for profit or commercial advantage and that copies bear this notice and the full citation approach [9, 45], with little user engagement. That is, different on the first page. Copyrights for components of this work owned by others than the versions of voice assistants are developed and then contrasted in author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission an evaluation, without investigating how they should behave in and/or a fee. Request permissions from [email protected]. specific tasks or contexts. CHI ’21, May 8–13, 2021, Yokohama, Japan To overcome these problems, we present a pragmatic bottom-up © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-8096-6/21/05...$15.00 approach, eliciting what users envision are dialogues with perfect https://doi.org/10.1145/3411764.3445536 voice assistants: Concretely, in an online survey, we asked N=205 CHI ’21, May 8–13, 2021, Yokohama, Japan Völkel et al. participants to write what they imagined to be a conversation be- termed alignment. This alignment also occurs frequently in human- tween a perfect voice assistant and a user for a set of common use human interaction but within human-machine dialogue is thought cases. In an exploratory approach, we then analysed participants’ re- to be driven by a user’s attempt to ensure communication success sulting dialogues qualitatively and quantitatively: We examined the with the system [7]. This phenomenon can also be leveraged in share of speech and interaction between the interlocutors, as well conversational design with recent work indicating that users as- as social aspects of the dialogue, voice assistant and user behaviour, cribe high likability and integrity to a voice user interface that and knowledge attributed to the voice assistant. We also assessed aligns its language to the user [47]. In contrast to prior work, which relationships of user personality and conversation characteristics. examined user conversation with voice assistants given the cur- Specifically we address these research questions: rent technological status quo, we take a different approach: We let (1) RQ1: How do users envision a dialogue between a user and a users freely imagine a conversation they consider to be perfect, perfect voice assistant, and how does this vision vary? using their desired conversation style, syntax, and wording, thus (2) RQ2: How does the user personality influence the envisioned exploring what users actually do want when given the choice. conversation between a user and a perfect voice assistant? Our contribution is twofold: First, on a conceptual level, we 2.2 Human and Conversational Agent propose a new approach so as to engage users in voice assistant Personality design. Specifically, it allows designers to gain insight into what Personality describes consistent and characteristic patterns which users feel is the design of a dialogue with a perfect voice assistant. determine how an individual behaves, feels, and thinks [51]. The Second, we present a set of qualitative and quantitative analyses Big Five (also Five-Factor model or OCEAN) is the most prevalent and insights into users’ preferences for personalised voice assistant paradigm for modelling human personality in scientific research dialogues. This provides much needed information to researchers and has five broad dimensions [23, 26, 27, 35, 39, 40, 50–53]: and practitioners on how to design voice assistant dialogues and Openness reflects an individual’s inclination to seek new experi- how these vary and might thus be adapted to users. ences, imagination, artistic interests, creativity, intellectual curios- ity, and an open-minded value and norm system. 2 RELATED WORK Conscientiousness reflects a tendency to be disciplined, orderly, Below we summarise work on the characteristics of today’s human- dutiful, competent, ambitious, and cautious. agent conversation, human and conversational agent personality, Extraversion reflects a tendency to be friendly, sociable, assertive, and adapting the agent to the user. dynamic, adventurous, and cheerful. Agreeableness reflects a tendency to be trustful, genuine, helpful, 2.1 Human-Agent Conversation modest, obliging, and cooperative. reflects an individual’s emotional stability and re- Despite the promise that the name Conversational Agent implies, Neuroticism several studies show that conversations with voice assistants are lates to experiencing anxiety, negative affect, stress, and depression. highly constrained, falling short of users’ expectations [21, 48, 68, A plethora of work in psychology and linguistics has examined the role of personality in human language use [12, 13, 25, 33, 54, 59, 70]. Existing dialogues with voice assistants are heavily task ori- 65, 67, 71, 73]. This relationship is most pronounced for . ented, taking the form of adjacency pairs that revolve around re- Extraversion questing and confirming actions [17, 34]. Voice assistant research For example, extraverts tend to talk more, use a more explicit and is increasingly interested in imbuing speech