Exploring voice interactions in an open world video game environment Robin Harnesk [email protected] Interaktionsdesign Bachelor 22.5HP Spring 2019 Supervisor: Henrik Svarrer Larsen 2 of 38 Acknowledgements I want to thank my supervisor Henrik Svarrer for all his support throughout this degree project. A tremendous thank you to all people attending interviews, observations and user testing. Finally, I would like to thank my fiancée for putting up with me during the period of writing this essay. 3 of 38 Abstract The use of voice interactions in home voice assistants are becoming mainstream and it’s being used to everything from information seeking to personal management. Even though games utilising voice has been praised for its innovation it has never been anything other than a novelty and research has been scarce. This study explores how a voice user interface (VUI) can be designed to reduce in-game interruptions in an open world video game environment. Through literature research, interviews, observations and prototypes aimed towards people playing video games and users of VUIs, this thesis proposes a set of design qualities for future implementation. The findings show that designing for voice interactions in an open world video game is, among other things, a matter of providing context-based triggers together with a compressed and modular approach that does not require the user to shift their attention from the video game. Keywords: Voice User Interface, Open World Video Game, Peripheral Interaction, Home Voice Assistant, Interaction Design 4 of 38 1 Introduction ......................................................................... 6 1.1 Purpose ................................................................................................ 6 1.2 Delimitations ................................................................................... 7 1.3 Target group .................................................................................... 7 2 Background ........................................................................... 7 2.1 What are voice user interfaces? ....................................................... 7 2.2 Open world games ........................................................................... 7 2.3 Voice interactions in video games ................................................... 8 2.4 Theory and related work .................................................................. 9 3 Methods .............................................................................. 11 3.1 Design process ................................................................................ 11 3.2 Literature studies ........................................................................... 13 3.3 Ethical considerations ................................................................... 13 3.4 Semi-structured interviews ........................................................... 14 3.5 Observations .................................................................................. 14 3.6 Brainstorming and Bodystorming ................................................. 15 3.7 Prototyping .................................................................................... 15 4 Design Process .................................................................... 16 4.1 Semi-structured interviews ........................................................... 16 4.2 Observations .................................................................................. 19 4.3 Design opportunity ........................................................................ 22 4.4 Brainstorming ................................................................................ 22 4.5 Bodystorming ................................................................................ 24 4.6 “Wizard of Oz” - Prototyping ......................................................... 25 5 Discussion ........................................................................... 32 5.1 Self-evaluation ............................................................................... 34 6 Conclusion ......................................................................... 34 7 References .......................................................................... 36 5 of 38 Figure list Figure 1: Implementation of voice interactions in Google Stadia ................. 11 Figure 2: Illustration of Double Diamond Model ......................................... 12 Figure 3: Double diamond with integrated design process .......................... 16 Figure 4: Affinity diagram clustering insights from interviews .................... 18 Figure 5: Participant playing Red Dead Redemption 2 .............................. 20 Figure 6: The journey of interactions to set a waypoint in-game. ................ 21 Figure 7: Part of the ideas generated during brainstorming session ............ 23 Figure 8: Illustration showing order of interactions and its end goal .......... 25 Figure 9: Back-end view of one of three commands of the prototype. ......... 27 Figure 10: Synergy between user (1) and researcher (2) .............................. 27 Figure 11: Icon appearing in the bottom left indicating that voice interactions are available .................................................................................................. 28 Figure 12: Illustration showing video output for the player and researcher controlling video output by interacting with video software. ....................... 29 Figure 13: Annotation sketch of the in-game player view ............................. 30 Figure 14: Flow of interaction for setting a waypoint ................................... 31 Figure 15: The journey of voice interactions to set an in-game waypoint .... 32 6 of 38 1 Introduction Home voice assistants are becoming more mainstream and the last ten years have seen a substantial rise of this technology in everyday life (Myers, Furqan, Nebolsky, Caro and Zhu, 2018). It is being used to everything from information seeking to personal management and Helft (2016) reports that a total of 20% of all Google searches are done using voice interactions and Canalys (2018) reports that global smart speaker shipments increased with 187% in the second quarter of 2018. Despite this data showing an increase in use of voice interactions, the amount of empirical research within the field of human-computer interaction (HCI) has been scarce in terms of usage in everyday situations. 4 in 10 have gaming as an interest and 86% of internet users play games on a gaming console at least once a month (GlobalWebIndex, 2019). Video games implementing speech recognition have been praised for its innovation but never moved past the novelty aspect, and the advances in voice recognition technology in consumer products like Microsoft Kinect have seen a rise during the last ten years (Globalme, 2018). This has led to player-to-player voice interactions receiving notable attention in the field of HCI but voice interactions in games where the voice is used an input, has not. (Carter, Allison, Downs & Gibbs, 2015). Open world video games is characterised by its non-linear gameplay and freedom to explore. This makes the genre among the most popular genres in gaming (Invision Game Community, 2019). It also has its fair share of menus the user has to navigate through next to their main gaming activity. It is essential that these menus are efficient and provide no interruptions for the user during gameplay (Kulshreshth & LaViola, 2014). 1.1 Purpose The aim of this study is to explore how voice interactions can be used to reduce in-game interruptions in an open world video game environment and understand how voice interactions in video games are perceived as an input by the intended target group. Through practice of design methods grounded in interaction design, seek to get an understanding of the experience when using a voice user interface and playing open world video games. The study aims to contribute to designers within the field of HCI that seek to gain a better understanding of how to design for voice interactions in a video game environment. 1.1.1 Research question How can voice user interfaces be designed to reduce in-game interruptions in an open world video game? 7 of 38 1.2 Delimitations This thesis work will not go into extensive research regarding the technical aspects of speech recognition but rather focus on the value of upcoming and future design of voice user interfaces for voice interactions in video games. It will set out to look deeper into the player experiences that go beyond examining speech recognition technology. This study will be targeting open world video games as it blends qualities of multiple genres into one. There is no intention of having the design solutions being implemented but they will work as a foundation and discussion for future design with focus on the user experience. The research for this thesis is limited to potential users located in Sweden, due the need of having field research and testing done in person. Additionally, one might need to take into account that Sweden is a country of early adopters and users in this study might have a different relationship to new technology compared to other countries. This allows for research with people who already have an idea of what the technology can do. 1.3 Target group The target group for this study includes people who play video games and want to add another layer of interactivity to their games. Even though users of home voice assistants