University of Central Florida STARS Electronic Theses and Dissertations, 2020- 2020 Interactions Between Humans, Virtual Agent Characters and Virtual Avatars Tamara Griffith University of Central Florida Part of the Graphics and Human Computer Interfaces Commons, and the Interpersonal and Small Group Communication Commons Find similar works at: https://stars.library.ucf.edu/etd2020 University of Central Florida Libraries http://library.ucf.edu This Doctoral Dissertation (Open Access) is brought to you for free and open access by STARS. It has been accepted for inclusion in Electronic Theses and Dissertations, 2020- by an authorized administrator of STARS. For more information, please contact [email protected]. STARS Citation Griffith,amar T a, "Interactions Between Humans, Virtual Agent Characters and Virtual Avatars" (2020). Electronic Theses and Dissertations, 2020-. 52. https://stars.library.ucf.edu/etd2020/52 INTERACTIONS BETWEEN HUMANS, VIRTUAL AGENT CHARACTERS AND VIRTUAL AVATARS by TAMARA S GRIFFITH B.S. University of Central Florida, 1992 B.S. University of Central Florida, 2003 M.S. University of Central Florida, 2011 A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the School of Modeling, Simulation, and Training in the College of Engineering and Computer Science at the University of Central Florida Orlando, Florida Spring Term 2020 Major Professor: Patricia Bockelman © 2020 Tamara S. Griffith iii ABSTRACT Simulations allow people to experience events as if they were happening in the real world in a way that is safer and less expensive than live training. Despite improvements in realism in simulated environments, one area that still presents a challenge is interpersonal interactions. The subtleties of what makes an interaction rich are difficult to define. We may never fully understand the complexity of human interchanges, however there is value in building on existing research into how individuals react to virtual characters to inform future investments. Virtual characters can either be automated through computational processes, referred to as agents, or controlled by a human, referred to as an avatar. Knowledge of interactions with virtual characters will facilitate the building of simulated characters that support training tasks in a manner that will appropriately engage learners. Ultimately, the goal is to understand what might cause people to engage or disengage with virtual characters. To answer that question, it is important to establish metrics that would indicate when people believe their interaction partner is real, or has agency. This study makes use of three types of measures: objective, behavioral and self-report. The objective measures were neural, galvanic skin response, and heart rate measures. The behavioral measure was gestures and facial expressions. Surveys provided an opportunity to gain self-report data. The objective of this research study was to determine what metrics could be used during social interactions to achieve the sense of agency in an interactive partner. The results provide valuable feedback on how users need to see and be seen by their interaction partner to ensure non-verbal cues provide context and additional meaning to the dialog. This study provides insight into areas of future research, offering a foundation of knowledge for further exploration and lessons learned. This can lead to more realistic experiences that open the door to human dimension training. iv This is dedicated to my family, who has been patient and supportive with me throughout this process. In addition, I appreciate the never-ending support of my work family, who has also provided support and some of the most amazing opportunities an individual could hope for. Finally, I’d like to express my deepest gratitude to my dissertation committee. Your involvement and support have been priceless. v ACKNOWLEDGMENTS The completion of this dissertation will be a culmination of many years of work. As with anyone facing this moment, there have been highs and lows. I would never have made it to this place without the unending support from my dissertation committee who invested many hours of their personal time to ensure I was on the right path. Dr. Patricia Bockelman stepped into the role of my committee chair and immediately I felt that I had gained a best friend, albeit one who could tell me honestly when I didn’t have my head screwed on right. She was able to ensure research rigor with a firm but kind hand. I have been honored to benefit from her guidance. Dr. Cali Fidopiastis has also been a strong supporter. She offered not just her profound knowledge in neurological measurements, but also her insight into what makes sense in the topics that matter to me. Her strength and confidence come from years of practical experiences in the commercial and research worlds, the value of which cannot be measured. And like icing on the cake, Dr. Joan Johnston, a coworker and friend, has brought her vast knowledge of research design, analysis, and analytical tools. Her guidance cleared my confusion and pulled back the curtains of intimidation. She fostered a passion for examining data that I didn’t think lived within me. These women have been so inspiring and so complimentary of one another’s styles; I cannot imagine being here without them and I cannot thank them enough for their guidance and friendship. vi TABLE OF CONTENTS LIST OF FIGURES ..................................................................................................................................... xi LIST OF TABLES ................................................................................................................................... xviii LIST OF ACRONYMS ............................................................................................................................. xix CHAPTER ONE: BACKGROUND ............................................................................................................ 1 Interpersonal Interactions ...................................................................................................................... 3 Virtual Characters ................................................................................................................................. 4 Theories on Virtual Character Interactions ........................................................................................... 5 Army Training Applications ................................................................................................................. 7 Research Problem ................................................................................................................................. 8 Biometric Measures of Social Interactions ........................................................................................... 8 Behavioral Measures ........................................................................................................................... 10 Objectives ........................................................................................................................................... 10 Research Questions ............................................................................................................................. 11 Potential Ethical Issues ....................................................................................................................... 13 Mixed Methods Study ......................................................................................................................... 14 CHAPTER TWO: LITERATURE REVIEW ............................................................................................ 16 Modeling and Simulation .................................................................................................................... 16 Avatars and Agents ............................................................................................................................. 17 Virtual Characters in Training ............................................................................................................ 19 vii Character Realism Theories ................................................................................................................ 22 Behavioral Realism/Fidelity ............................................................................................................... 26 Agency ................................................................................................................................................ 27 Behavioral/Gestures ............................................................................................................................ 28 Neuroscience as a Tool for Assessing Social Interactions/Game Theory ........................................... 28 Neuroeconomic Research and Measurement ...................................................................................... 31 Mixed Methods Research.................................................................................................................... 34 Current Study Design .......................................................................................................................... 36 CHAPTER THREE: METHOD ................................................................................................................ 38 Participants .........................................................................................................................................