DOCSLIB.ORG

Reducing Cognitive Load and Improving Warfighter Problem

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Reducing Cognitive Load and Improving Warfighter Problem fpsyg-11-554706 November 11, 2020 Time: 15:25 # 1 ORIGINAL RESEARCH published: 17 November 2020 doi: 10.3389/fpsyg.2020.554706 Reducing Cognitive Load and Improving Warfighter Problem Solving With Intelligent Virtual Assistants Celso M. de Melo1*, Kangsoo Kim2,3, Nahal Norouzi4, Gerd Bruder3 and Gregory Welch2,3,4 1 Computational and Information Sciences, CCDC US Army Research Laboratory, Playa Vista, CA, United States, 2 College of Nursing, University of Central Florida, Orlando, FL, United States, 3 Institute for Simulation and Training, University of Central Florida, Orlando, FL, United States, 4 Department of Computer Science, University of Central Florida, Orlando, FL, United States Recent times have seen increasing interest in conversational assistants (e.g., Amazon Edited by: Alexa) designed to help users in their daily tasks. In military settings, it is critical to design Varun Dutt, assistants that are, simultaneously, helpful and able to minimize the user’s cognitive Indian Institute of Technology Mandi, load. Here, we show that embodiment plays a key role in achieving that goal. We India present an experiment where participants engaged in an augmented reality version Reviewed by: Ion Juvina, of the relatively well-known desert survival task. Participants were paired with a voice Wright State University, United States assistant, an embodied assistant, or no assistant. The assistants made suggestions Michail Maniadakis, Foundation for Research verbally throughout the task, whereas the embodied assistant further used gestures and Technology - Hellas (FORTH), and emotion to communicate with the user. Our results indicate that both assistant Greece conditions led to higher performance over the no assistant condition, but the embodied *Correspondence: assistant achieved this with less cognitive burden on the decision maker than the Celso M. de Melo [email protected]; voice assistant, which is a novel contribution. We discuss implications for the design [email protected] of intelligent collaborative systems for the warfighter. Specialty section: Keywords: intelligent virtual assistant, collaboration, cognitive load, embodiment, augmented reality This article was submitted to Cognitive Science, a section of the journal INTRODUCTION Frontiers in Psychology Received: 22 April 2020 In the near future, humans will be increasingly expected to team up with artificially intelligent Accepted: 23 October 2020 (AI) non-human partners to accomplish organizational objectives (Davenport and Harris, 2005; Published: 17 November 2020 Bohannon, 2014). This vision is motivated by rapid progress in AI technology that supports a Citation: growing range of applications, such as self-driving vehicles, automation of mundane and dangerous de Melo CM, Kim K, Norouzi N, tasks, processing large amounts of data at superhuman speeds, sensing the environment in ways Bruder G and Welch G (2020) that humans cannot (e.g., infrared), and so on. This technology is even more relevant given Reducing Cognitive Load the increasing complexity and dynamism of modern workplaces and operating environments. and Improving Warfighter Problem Solving With Intelligent However, the vision for AI will only materialize if humans are able to successfully collaborate with Virtual Assistants. these non-human partners. This is a difficult challenge as humans, on the one hand, do not fully Front. Psychol. 11:554706. understand how AI works and, on the other hand, are often already overburdened by the task doi: 10.3389/fpsyg.2020.554706 (Lee and See, 2004; Hancock et al., 2011; Schaefer et al., 2016). Frontiers in Psychology| www.frontiersin.org 1 November 2020| Volume 11| Article 554706 fpsyg-11-554706 November 11, 2020 Time: 15:25 # 2 de Melo et al. Reducing Cognitive Load With Assistants This challenge is even more critical in military domains, where on users’ cognitive load from conversational assistants is still not warfighters are under increased (physical and cognitive) pressure well understood. and often face life and death situations (Kott and Alberts, 2017; A fundamental limitation of conversational assistants, TRADOC, 2018; Kott and Stump, 2019). Since the potential however, is their lack of embodiment and consequent costs of failure are higher, warfighters tend to be even more limited ability to communicate non-verbally. Non-verbal reluctant to trust and collaborate with AI (Gillis, 2017). For these communication plays an important role in regulating reasons, it is important to gradually increase exposure of AI social interaction (Boone and Buck, 2003). Expressions of technology to the warfighter through training that explains how emotion, additionally, serve important social functions such AI works, its strengths, but also its limitations. Complementary, as communicating one’s beliefs, desires, and intentions to it is fundamental that, during mission execution, AI is capable others (Frijda and Mesquita, 1994; Keltner and Lerner, 2010; of actively promoting trust and collaboration by communicating van Kleef et al., 2010; de Melo et al., 2014). The information naturally, effectively, and efficiently with the warfighter, while conveyed by non-verbal cues, therefore, can be very important minimizing the warfighter cognitive load. in building trust and promoting cooperation with humans Cognitive load is commonly defined as the difference (Bickmore and Cassell, 2001; Frank, 2004). Non-verbal between the cognitive demands of the task and the user’s communication in technology systems is typically achieved available cognitive resources (e.g., attention and memory) through robotic (Breazeal, 2003) or virtual agents (Gratch (Hart and Wickens, 1990; Nachreiner, 1995; Mackay, 2000). et al., 2002). Because these systems are embodied, they support Several factors have been identified that influence cognitive load non-verbal communication with users, including expression of but, broadly, it is possible to distinguish between exogenous emotion. Research shows that embodied conversational assistants factors (e.g., task difficulty) and endogenous factors (e.g., can have positive effects in human–machine interaction (Beale information processing capabilities for perceiving, planning, and and Creed, 2002; Beun et al., 2003; Kim et al., 2018b; Shamekhi decision making). Furthermore, individual factors, such as skill et al., 2018), including building rapport (Gratch et al., 2006) and experience, influence cognitive load. The importance of and promoting cooperation with users (de Melo et al., 2011; de controlling cognitive load for task performance has been studied Melo and Terada, 2019, 2020). In the context of problem-solving across several domains, with several studies suggesting that tasks, pedagogical agents have been shown to be able to improve increased cognitive load can undermine performance (Cinaz learning and task performance (Lester et al., 1997; Atkinson, et al., 2013; Dadashi et al., 2013; Vidulich and Tsang, 2014; 2002). These improvements are typically achieved through Fallahi et al., 2016). More recently, there has been increasing gestures that focus the user’s attention or through affective cues interest on the impact that technology has on worker’s cognitive serving specific pedagogical purposes, such as motivating the load. Much research has focused on the relationship between user (Schroeder and Adesope, 2014). Embodied conversational automation and user’s boredom and mental underload (Ryu assistants, therefore, hold the promise of having at least all and Myung, 2005), and its negative consequences on users’ the benefits in terms of task performance as (non-embodied) ability to react in a timely manner during real or simulated conversational assistants, while having minimal impact on the flight (Shively et al., 1987; Battiste and Bortolussi, 1988), user’s cognitive load. air combat (Bittner et al., 1989), air traffic control (Block With increased immersion afforded to the user (Dey et al., 2010), and so on. However, another concern is that et al., 2018; Kim et al., 2018a), augmented reality (AR) has technology may increase user’s cognitive load due to the the potential to further enhance collaboration with AI. AR change of routine processes, stress due to the transition, and technology supports superimposition of virtual entities alongside general lack of understanding of how the technology works the physical space in the user’s field of view. Therefore, first, (Mackay, 2000). interaction with AI systems can occur as the user is fully Intelligent virtual assistants offer a promising route to immersed in the task and, second, virtual interfaces—such promote collaboration between humans and AI, while controlling as embodied assistants—can be integrated seamlessly in the the impact on the user’s cognitive load (Gratch et al., 2002). workspace. Increased user influence in AR environments often Conversational assistants—i.e., intelligent virtual assistants that occurs through an increased sense of social presence (Blascovich can communicate through natural language—have, in particular, et al., 2002). Kim et al.(2018b, 2019) investigated the effects of an been experiencing considerable commercial success—e.g., Apple embodied conversational assistant on the sense of social presence Siri, Amazon Alexa, and Microsoft Cortana (Hoy, 2018). The and confidence in the system and found that both factors basic premise is that advances in natural language processing positively impacted users’ perception of the system’s ability to be technology (Hirschberg and Manning, 2015) enable more aware
Load more

Recommended publications
  • Explainable Embodied Agents Through Social Cues: a Review
    Explainable Embodied Agents Through Social Cues: A Review SEBASTIAN WALLKÖTTER∗ and SILVIA TULLI∗, Uppsala University and INESC-ID - Instituto Superior Técnico GINEVRA CASTELLANO, Uppsala University, Sweden ANA PAIVA, INESC-ID and Instituto Superior Técnico, Portugal MOHAMED CHETOUANI, Institute for Intelligent Systems and Robotics, CNRS UMR 7222, Sorbonne Université, France The issue of how to make embodied agents explainable has experienced a surge of interest over the last three years, and, there are many terms that refer to this concept, e.g., transparency or legibility. One reason for this high variance in terminology is the unique array of social cues that embodied agents can access in contrast to that accessed by non-embodied agents. Another reason is that different authors use these terms in different ways. Hence, we review the existing literature on explainability and organizeitby(1) providing an overview of existing definitions, (2) showing how explainability is implemented and how it exploits different social cues, and (3) showing how the impact of explainability is measured. Additionally, we present a list of open questions and challenges that highlight areas that require further investigation by the community. This provides the interested reader with an overview of the current state-of-the-art. CCS Concepts: • Computing methodologies → Intelligent agents; Reasoning about belief and knowledge; • Computer systems organization → Robotics; • General and reference → Metrics; • Human-centered computing → User models. Additional Key Words and Phrases: transparency, interpretability, explainability, accountability, expressive behaviour, intelligibility, legibility, predictability, explainable agency, embodied social agents, robots ACM Reference Format: Sebastian Wallkötter, Silvia Tulli, Ginevra Castellano, Ana Paiva, and Mohamed Chetouani. 2019. Explainable Embodied Agents Through Social Cues: A Review.
    [Show full text]
  • R.E.I.N.A. Towards Pervasive Interface Agents That Transcend The
    R.E.I.N.A. Towards Pervasive Interface Agents that Transcend the Physical-Digital Worlds by Elena Chong Loo Kodama B.S., Rose-Hulman Institute of Technology (2016) Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning in partial fulfillment of the requirements for the degree of Master of Science in Media Arts and Sciences at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY September 2020 ○c Massachusetts Institute of Technology 2020. All rights reserved. Author................................................................ Program in Media Arts and Sciences August 17, 2020 Certified by. Joseph A. Paradiso Alexander W Dreyfoos (1954) Professor Program in Media Arts and Sciences Thesis Advisor Accepted by . Tod Machover Academic Head, Program in Media Arts and Sciences 2 R.E.I.N.A. Towards Pervasive Interface Agents that Transcend the Physical-Digital Worlds by Elena Chong Loo Kodama Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning on August 17, 2020, in partial fulfillment of the requirements for the degree of Master of Science in Media Arts and Sciences Abstract Our generation is spending more time in front of computer screens, in part due to the onset of the COVID-19 pandemic. In front of our screens, we see multiple notes, fold- ers, windows, and applications that somehow replicate a metaphoric desk. The way we navigate this digital system has not changed much in the past four decades. How- ever, in the last two years, the technological landscape is showing sign of a potential shift that could enable novel ways of navigating the physical and digital information spaces.
    [Show full text]
  • Conversational Agents in Software Engineering: Survey, Taxonomy and Challenges
    Conversational Agents in Software Engineering: Survey, Taxonomy and Challenges QUIM MOTGER and XAVIER FRANCH, Department of Service and Information System Engineering (ESSI), Universitat Politècnica de Catalunya (UPC), Spain JORDI MARCO, Department of Computer Science (CS), Universitat Politècnica de Catalunya (UPC), Spain The use of natural language interfaces in the field of human-computer interaction is undergoing intense study through dedicated scientific and industrial research. The latest contributions in the field, including deep learning approaches like recurrent neural networks, the potential of context-aware strategies and user-centred design approaches, have brought back the attention of the community to software-based dialogue systems, generally known as conversational agents or chatbots. Nonetheless, and given the novelty of the field, a generic, context-independent overview on the current state of research of conversational agents covering all research perspectives involved is missing. Motivated by this context, this paper reports a survey of the current state of research of conversational agents through a systematic literature review of secondary studies. The conducted research is designed to develop an exhaustive perspective through a clear presentation of the aggregated knowledge published by recent literature within a variety of domains, research focuses and contexts. As a result, this research proposes a holistic taxonomy of the different dimensions involved in the conversational agents’ field, which is expected to
    [Show full text]
  • Reducing Task Load with an Embodied Intelligent Virtual Assistant for Improved Performance in Collaborative Decision Making
    Reducing Task Load with an Embodied Intelligent Virtual Assistant for Improved Performance in Collaborative Decision Making Kangsoo Kim* Celso M. de Melo† Nahal Norouzi‡ Gerd Bruder§ University of Central Florida US Army Research Laboratory University of Central Florida University of Central Florida Gregory F. Welch¶ University of Central Florida ABSTRACT the overhead caused by the collaboration. A large body of literature Collaboration in a group has the potential to achieve more effective in social psychology and collaborative learning has investigated this solutions for challenging problems, but collaboration per se is not phenomenon and identified the importance of reducing task load in an easy task, rather a stressful burden if the collaboration partners collaborative situations while still emphasizing the improvement of do not communicate well with each other. While Intelligent Virtual task performance at the same time. Assistants (IVAs), such as Amazon Alexa, are becoming part of our With the advent of advanced artificial intelligence (AI) and natu- daily lives, there are increasing occurrences in which we collaborate ral language processing (NLP), intelligent virtual assistants (IVAs) with such IVAs for our daily tasks. Although IVAs can provide have experienced dramatic technological achievements in both re- important support to users, the limited verbal interface in the current search and commercial application fields [55]. While IVAs, such state of IVAs lacks the ability to provide effective non-verbal social as Amazon Alexa, Google Assistant, and Apple Siri, are becoming cues, which is critical for improving collaborative performance and part of our daily lives, there are increasing occurrences in which reducing task load. we collaborate with such IVAs for our daily tasks [54, 63].
    [Show full text]
  • A Multimodal Approach for Programming Intelligent Environments †
    technologies Article ParlAmI: A Multimodal Approach for Programming Intelligent Environments † Evropi Stefanidi, Michalis Foukarakis, Dimitrios Arampatzis, Maria Korozi *, Asterios Leonidis * and Margherita Antona Institute of Computer Science, Foundation for Research and Technology—Hellas (FORTH), Heraklion, 70013 Crete, Greece; [email protected] (E.S.); [email protected] (M.F.); [email protected] (D.A.); [email protected] (M.A.) * Correspondence: [email protected] (M.K.); [email protected] (A.L.) † This paper is an extended version of our paper published in Proceedings of the 11th International Conference on Pervasive Technologies Related to Assistive Environments (PETRA 2018), Corfu, Greece, 26–29 June 2018; pp. 50–57. Received: 10 December 2018; Accepted: 7 January 2019; Published: 11 January 2019 Abstract: The proliferation of Internet of Things (IoT) devices and services and their integration in intelligent environments creates the need for a simple yet effective way of controlling and communicating with them. Towards such a direction, this work presents ParlAmI, a conversational framework featuring a multimodal chatbot that permits users to create simple “if-then” rules to define the behavior of an intelligent environment. ParlAmI delivers a disembodied conversational agent in the form of a messaging application named MAI, and an embodied conversational agent named nAoMI employing the programmable humanoid robot NAO. This paper describes the requirements and architecture of ParlAmI, the infrastructure of the “Intelligent Home” in which ParlAmI is deployed, the characteristics and functionality of both MAI and nAoMI, and finally presents the findings of a user experience evaluation that was conducted with the participation of sixteen users.
    [Show full text]
  • Making New Friends: a Study in Companionship
    Making New Friends: A Study in Companionship Matthew Gabriel Maharaj Department of Graphic Design College of Design North Carolina State University April 30th, 2020 Submitted in partial fulfillment for the degree of Masters of Graphic Design Making New Friends: A Study in Companionship Matthew Gabriel Maharaj Department of Graphic Design College of Design North Carolina State University April 30th, 2020 Submitted in partial fulfillment for the degree of Masters of Graphic Design Denise Gonzales Crisp | Committee Chair Professor of Graphic Design, Director of Graduate Programs Helen Armstrong | Committee Reader Associate Professor of Graphic Design Matthew Peterson | Committee Reviewer Associate Professor of Graphic Design Acknowledgements Thank you to my teachers: Deb, Helen, Derek, Scott, and Matt for imparting your wisdom and knowledge to me. Thank you to Denise for taking a chance and admitting me into the program. Thank you to my cohort for all the memories and photographs. Thank you to my family and friends for all the love and support. Thank you to my loving wife Gloria for being my companion. Contents 1 Introduction 11 2 Problem Statement and Justification 13 3 Assumptions and Limitations 3.1 Assumptions 17 3.2 Limitations 17 4 Annotated Bibliography 19 4.1 Table Bibliography 23 5 Conceptual Framework And Research Question 5.1 Conceptual Framework 27 5.2 Research Question and Subquestions 31 5.3 Table of Definitions 33 6 Methods 35 7 Results 7.1.1 Precedents 37 7.2 Studies 43 7.2.1 Sub Question 1 47 7.2.2 Sub Question 2 53 7.2.3 Sub Question 3 57 8 Discussion 8.1 Design Principles 65 8.2 Future Work 69 8.3 Conclusion 71 9 References 73 Abstract Loneliness is a growing problem in the world; more people every year struggle to find a sense of comfort with those around them.
    [Show full text]
  • Preferred Virtual Assistant Gender and Realism in Self-Service
    Gendering the Machine: Preferred Virtual Assistant Gender and Realism in Self-service Jeunese Payne 1, Andrea Szymkowiak 1, Paul Robertson 1, and Graham Johnson 2 1University of Abertay Dundee, Dundee DD1 1HG, UK [email protected],[email protected], [email protected] 2NCR Discovery Centre, 3 Fulton Road, Dundee DD2 4SW, UK [email protected] Abstract. A virtual agent is a human-like character that is designed to assist us- ers in interactions with technology and virtual worlds. Research into the pre- ferred visual characteristics of a virtual agent has focused on education-based agents, gaming avatars, and online help assistants. However, findings from these studies are not necessarily generalizable to other technologies, such as self-service checkouts (SSCO). This paper describes data from 578 participants, looking at the gender preferences of Virtual Assistants (VA) in a SSCO context and the impact of VA realism depending on user gender. Due to female partici- pants’ preference for female VAs, and an overall preference for three- dimensional characters, a realistic, female VA should be used in SSCO. The re- sults are discussed in terms of similarity-attraction theory and social role theory. Keywords: Interface Agent, Virtual Assistant, Gender, Visual Realism, Self- service, Psychology, Preference 1 Introduction Technology plays a central role in service delivery, providing several channels through which consumers can be reached, e.g., the Internet, mobile technologies and kiosks [ 1- 2]. Many of these channels are self-service – technologies that enable users to direct the service they receive, e.g., cash withdrawals, shopping transactions etc., with minimal direct input from staff [ 3].
    [Show full text]
  • Explainable Embodied Agents Through Social Cues: a Review Sebastian Wallkötter, Silvia Tulli, Ginevra Castellano, Ana Paiva, Mohamed Chetouani
    Explainable Embodied Agents Through Social Cues: A Review Sebastian Wallkötter, Silvia Tulli, Ginevra Castellano, Ana Paiva, Mohamed Chetouani To cite this version: Sebastian Wallkötter, Silvia Tulli, Ginevra Castellano, Ana Paiva, Mohamed Chetouani. Explainable Embodied Agents Through Social Cues: A Review. ACM Transactions on Human-Robot Interaction, ACM, 2021, 10 (3), 10.1145/3457188. hal-03152010 HAL Id: hal-03152010 https://hal.sorbonne-universite.fr/hal-03152010 Submitted on 25 Feb 2021 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Explainable Embodied Agents Through Social Cues: A Review SEBASTIAN WALLKÖTTER∗ and SILVIA TULLI∗, Uppsala University and INESC-ID - Instituto Superior Técnico GINEVRA CASTELLANO, Uppsala University, Sweden ANA PAIVA, INESC-ID and Instituto Superior Técnico, Portugal MOHAMED CHETOUANI, Institute for Intelligent Systems and Robotics, CNRS UMR 7222, Sorbonne Université, France The issue of how to make embodied agents explainable has experienced a surge of interest over the last three years, and, there are many terms that refer to this concept, e.g., transparency or legibility. One reason for this high variance in terminology is the unique array of social cues that embodied agents can access in contrast to that accessed by non-embodied agents.
    [Show full text]
  • Embodied Pedagogical Agents from Visual Impact to Pedagogical Implications
    Embodied Pedagogical Agents From Visual Impact to Pedagogical Implications Magnus Haake Doctoral Thesis Dept. of Design Sciences Lund University Sweden To Agneta – my everything … … and to Sebastian and Igis – who, with their childish egoism, never have given me a real chance to confuse the reality of life with the virtuality of academia. Embodied Pedagogical Agents From Visual Impact to Pedagogical Implications Magnus Haake DOCTORAL THESIS Department of Design Sciences Faculty of Engineering Lund University Sweden Magnus Haake: Embodied Pedagogical Agents From Visual Impact to Pedagogical Implications © Magnus Haake 2009 Div. of Ergonomics Dept. of Design Sciences Faculty of Engineering Lund University Sweden http://www.design.lth.se/ ISSN 1650-9773 Publication 32 ISRN LUTMDN/TMAT --1032--SE EAT 2009 ISBN 978-91-628-7804-7 Format: 154 x 233 mm Typefaces: Adobe Garamond Print: E-husets tryckeri, Lund, Sweden Embodied Pedagogical Agents: From Visual Impact to Pedagogical Implications Acknowledgements I have had a poston that s becomng rare n Sweden. As a faculty employed doctoral student wth no specfic project fundng, I have found myself wth a marvellous opportunty to pursue research out of pure curosty. For ths I am very much oblged to my supervsor Gerd Johansson that once offered me ths poston. Regardng the otherwse most pecular Swedsh academc system, t seems to me that most people carry out ther research not thanks to, but rather n spte of, the system. Of course I also want to gve my regards to the people around me at the depart- ment of Desgn Scences n Lund, as well as to the people at LUCS (Cogntve Scence Lund Unversty).
    [Show full text]
  • Affective Interfaces of Embodied Conversational Agents
    Affective Interfaces of Embodied Conversational Agents Studies of Hardware and Character Interfaces Yi-Chen Hsu DOCTORAL THESIS School of Design Studies College of Fine Arts The University of New South Wales October 2011 ORIGINALITY STATEMENT ‘I hereby declare that this submission is my own work and to the best of my knowledge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgement is made in the thesis. Any contribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project's design and conception or in style, presentation and linguistic expression is acknowledged.’ Signed Date 01/05/2012 COPYRIGHT STATEMENT ‘I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstract International (this is applicable to doctoral theses only).
    [Show full text]