A Unified Framework for Individualized Avatar-Based Interactions" (2014)
Total Page:16
File Type:pdf, Size:1020Kb
University of Central Florida STARS Faculty Bibliography 2010s Faculty Bibliography 1-1-2014 A Unified rF amework for Individualized Avatar-Based Interactions Arjun Nagendran University of Central Florida Remo Pillat University of Central Florida Adam Kavanaugh University of Central Florida Greg Welch University of Central Florida Charles Hughes University of Central Florida Find similar works at: https://stars.library.ucf.edu/facultybib2010 University of Central Florida Libraries http://library.ucf.edu This Article is brought to you for free and open access by the Faculty Bibliography at STARS. It has been accepted for inclusion in Faculty Bibliography 2010s by an authorized administrator of STARS. For more information, please contact [email protected]. Recommended Citation Nagendran, Arjun; Pillat, Remo; Kavanaugh, Adam; Welch, Greg; and Hughes, Charles, "A Unified Framework for Individualized Avatar-Based Interactions" (2014). Faculty Bibliography 2010s. 5645. https://stars.library.ucf.edu/facultybib2010/5645 Arjun Nagendran* A Unified Framework for Remo Pillat Adam Kavanaugh Individualized Avatar-Based Greg Welch Interactions Charles Hughes Synthetic Reality Lab University of Central Florida Abstract This paper presents a framework to interactively control avatars in remote environ- ments. The system, called AMITIES, serves as the central component that connects people controlling avatars (inhabiters), various manifestations of these avatars (surro- gates), and people interacting with these avatars (participants). A multiserver–client architecture, based on a low-demand network protocol, connects the participant environment(s), the inhabiter station(s), and the avatars. A human-in-the-loop metaphor provides an interface for remote operation, with support for multiple inhabiters, multiple avatars, and multiple participants. Custom animation blending routines and a gesture-based interface provide inhabiters with an intuitive avatar control paradigm. This gesture control is enhanced by genres of program-controlled behaviors that can be triggered by events or inhabiter choices for individual or groups of avatars. This mixed (agency and gesture-based) control paradigm reduces the cognitive and physical loads on the inhabiter while supporting natural bidirectional conversation between participants and the virtual characters or avatar counterparts, including ones with physical manifestations, for example, robotic surrogates. The associated system affords the delivery of personalized experiences that adapt to the actions and interactions of individual users, while staying true to each virtual charac- ter’s personality and backstory. In addition to its avatar control paradigm, AMITIES provides processes for character and scenario development, testing, and refinement. It also has integrated capabilities for session recording and event tagging, along with automated tools for reflection and after-action review. We demonstrate effective- ness by describing an instantiation of AMITIES, called TeachLivE, that is widely used by colleges of education to prepare new teachers and provide continuing profes- sional development to existing teachers. Finally, we show the system’s flexibility by describing a number of other diverse applications, and presenting plans to enhance capabilities and application areas. 1 Introduction The use of virtual characters and associated environments has been widely adopted in training and rehabilitation scenarios over the last several decades. These virtual characters/environments generally offer the flexibility to recre- ate specific scenarios and events, while doing so in a controlled and consistent manner. Traditionally, virtual characters have autonomous agency—they are Presence, Vol. 23, No. 2, Spring 2014, 109–132 doi:10.1162/PRES_a_00177 © 2014 by the Massachusetts Institute of Technology *Correspondence to [email protected]. Nagendran et al. 109 110 PRESENCE: VOLUME 23, NUMBER 2 driven by a computer program. Advances in artifi- cial intelligence (such as natural language processing and decision trees) have helped create realistic inter- action scenarios (e.g., Rizzo et al., 2013). However, there are still several research challenges associated with open-ended interactions. For example, hampered or interrupted flow during bidirectional conversation can result in a reduced sense of scenario plausibility, and processing errors such as speech recognition errors, repeated responses, or inappropriate responses can detract from the experience or cause harm. To address these and other issues, control of virtual characters may involve a human who inhabits (i.e., controls) the charac- ter. The character that is being controlled by a human is Figure 1. Components of the proposed system for avatar-mediated referred to as an avatar. More formally, a virtual avatar individualized interactions. is described as a perceptible digital representation whose behaviors reflect those executed, typically in real time, by a specific human being (Bailenson & Blascovich, 2004). groups. This paper is an extended version of our work In a more general sense, avatars can have physical (e.g., presented at the Virtual Reality Software and Technol- robotic), as well as virtual manifestations. The term ogy Conference (VRST; Nagendran, Pillat, Kavanaugh, human surrogate is also used when the avatar is intended Welch, & Hughes, 2013) in which we described the to represent the human at some remote destination. In AMITIES system architecture without focusing on the this context, persons who drive their remote counter- individual components that form the underlying basis parts (avatars) are referred to as inhabiters (Nagendran, for AMITIES. AMITIES can be thought of as a bind- Pillat, Hughes, & Welch, 2012) although the term ing system between three components that are typically interactor is also used when the inhabiter is a highly involved during interactions: (1) the avatars; (2) their trained professional capable of embodying many dif- inhabiters; and (3) the participants. This paper addresses ferent, disparate avatars. People who interact with the the role of AMITIES in bringing together these compo- avatars are referred to as participants—these can be nents for improved avatar-mediated interactions (e.g., active participants who directly influence an interaction see Figure 1) and presents an instantiation of AMITIES or passive participants who merely observe the interac- as a case study. tion with an intent to either gain knowledge, analyze The system provides an interface for each of these performance, or provide guidance to active participants three components by leveraging technological affor- during the interactions. Further distinctions of par- dances and avatar mediation to create scenarios that ticipants and the roles they may assume is provided in establish, maintain, and preserve user beliefs that Section 3.3 of this paper. are critical to the interaction. In essence, the system In this paper, we present a framework and its systems attempts to preserve place illusion (a sense of “being architecture that forms the central component to medi- there/this is happening in my space”) and situational ating individualized avatar-based interactions. We call plausibility (a sense of “this event is possible”), both our system AMITIES, for Avatar-Mediated Interac- of which have been shown to influence human percep- tive Training and Individualized Experience System. tions (Slater, 2009), particularly in virtual-reality-based The acronym has dual meaning, as the word “amities” environments. The AMITIES system features digital (derived from Old French) indicates peaceful relation- puppetry (Hunter & Maes, 2013; Mapes, Tonner, & ships, friendships, and harmony between individuals or Hughes, 2011) blended with autonomous behaviors Nagendran et al. 111 and a network interface to allow inhabiters to con- nings, 1995; Baylor, 2011) are very flexible as they can trol multiple virtual characters seamlessly from remote be replicated easily, can be used during any hour of the locations. The system uses a network-efficient proto- day, and are cost-effective human representations. Since col during control, thereby minimizing the required avatars are directly controlled by humans, they rely less bandwidth and hence any associated latencies. Ren- on the capabilities of the agent’s artificial intelligence dering is in the domain of each recipient station and so engine and can convincingly simulate social scenarios perceptible lag is avoided. At the user end, the system and adaptively steer conversations (Blascovich et al., offers the flexibility for several observers to be involved 2002; Ahn, Fox, & Bailenson, 2012). (passively) during a training session, extending the On the other hand, a recent metastudy comparing training impact to additional users. Additionally, the the effectiveness of agents and avatars (Fox et al., 2010) system allows multiple interactors and flexible assign- found that avatars elicit stronger levels of social influence ments to support many-to-many, many-to-one, and compared to agents. Similar results were found in game one-to-many interactor–character scenarios. Within environments (Lim & Reeves, 2010). AMITIES is another component that is of value during While having free-speech conversation with vir- avatar-mediated interactions. This is called the activ- tual characters is desirable in virtual environments, it ity storage/retrieval unit. This subcomponent supports is difficult to achieve this through intelligent agents trainers in the processes