Robotic Faces: Exploring Dynamical Patterns of Social Interaction
Total Page：16
File Type：pdf, Size：1020Kb
ROBOTIC FACES: EXPLORING DYNAMICAL PATTERNS OF SOCIAL INTERACTION BETWEEN HUMANS AND ROBOTS Casey Bennett Submitted to the faculty of the University Graduate School in partial fulfillment of the requirements for the degree Doctor of Philosophy in the School of Informatics and Computing Indiana University May 2015 ii Accepted by the Graduate Faculty, Indiana University, in partial fulfillment of the requirements for the degree of Doctor of Philosophy. Doctoral Committee _________________________________ Selma Sabanovic, Ph.D. _________________________________ Randall Beer, Ph.D. _________________________________ Kay Connelly, Ph.D. _________________________________ David Crandall, Ph.D. April 13, 2015 iii Copyright © 2015 Casey Bennett iv For Dhari, for always being the light in the darkness. v Casey Bennett EXPLORING DYNAMICAL PATTERNS OF SOCIAL INTERACTION BETWEEN HUMANS AND ROBOTS The purpose of this dissertation is two-fold: 1) to develop an empirically-based design for an interactive robotic face, and 2) to understand how dynamical aspects of social interaction may be leveraged to design better interactive technologies and/or further our understanding of social cognition. Understanding the role that dynamics plays in social cognition is a challenging problem. This is particularly true in studying cognition via human-robot interaction, which entails both the natural social cognition of the human and the “artificial intelligence” of the robot. Clearly, humans who are interacting with other humans (or even other mammals such as dogs) are cognizant of the social nature of the interaction – their behavior in those cases differs from that when interacting with inanimate objects such as tools. Humans (and many other animals) have some awareness of “social”, some sense of other agents. However, it is not clear how or why. Social interaction patterns vary across culture, context, and individual characteristics of the human interactor. These factors are subsumed into the larger interaction system, influencing the unfolding of the system over time (i.e. the dynamics). The overarching question is whether we can figure out how to utilize factors that influence the dynamics of the social interaction in order to imbue our interactive technologies (robots, clinical AI, decision support systems, etc.) with some "awareness of social", and potentially create more natural interaction paradigms for those technologies. In this work, we explore the above questions across a range of studies, including lab-based experiments, field observations, and placing autonomous, interactive robotic faces in public spaces. We also discuss future work, how this research relates to making sense of what a robot "sees", creating data-driven models of robot social behavior, and development of robotic face personalities. Keywords: Affective Communication; Emotion; Facial Expressions; Human-Robot Interaction; Robot Design; Social Interaction vi Table of Contents 1. Introduction …... 1 2. The Robotic Face Platform …... 8 3. Deriving Minimal Features for Human-Like Facial Expressions in Robotic Faces …... 18 4. The Effects of Culture and Context on Perceptions of Robotic Facial Expressions …... 49 5. Context Congruency and Robotic Facial Expressions: Do Effects on Human Perceptions Vary across Culture? …... 78 6. A Month in the Museum: Interaction Patterns with a Robotic Face in the Wild …... 93 7. Comparing Human Interaction with a Robotic Face in-the-lab vs. in-the-wild: An Empirical Study …... 118 8. Future Directions …... 135 8.2. Temporal Dynamics (e.g. rhythmicity, synchronicity) in Human-Robot Social Interaction: Towards Developing Future Models to Guide Interactive Robot Behavior …... 136 8.3. Making Sense of What a Robotic Face “Sees”: Machine Learning and Sparse Visual Data …... 139 8.4. Robotic Face Personality and a “Sense of Self”: Clues from Human Borderline Personality Disorder …... 144 9. Discussion …... 152 10. References …... 157 11. Curriculum Vitae 1 Chapter 1 Introduction 1.1 Problem At its core, the purpose of this dissertation is two-fold: 1) to develop an empirically-based design for an interactive robotic face, and 2) to understand how dynamical aspects of social interaction may be leveraged to design better interactive technologies and/or further our understanding of social cognition. In this chapter, we explain the importance of delving into these two challenges. The motivation for such work starts from a basic premise: that social interaction is a “system” that subsumes the individual interactors and components of any interaction. Even when we interact with technology, much of what shapes the interaction goes beyond the design of the technology itself. Studying such a system requires the ability to rigorously and consistently manipulate aspects of the system. Technologies such as interactive robots (e.g. human-robot interaction, HRI) can afford such abilities, but only if robotic technologies are designed in an empirical way, allowing for replicable experimentation, i.e. “robotic science.” 1.2 Question The primary driving question here is: what makes an interaction “social”? There are corollaries to this primary question: what is social cognition? Where does it come from? Why do we humans and other animals exhibit such a capacity? Why does interacting with other items, such as tools or technology, not exhibit “social” features? Social interaction is a dynamic process influenced by a number of factors, but what are the factors, and what role do they play? Can we imbue our technologies with features or dynamical properties that make them interact more socially, and/or that encourage people to ascribe more social characteristics to them? It is outside the scope of the current work to answer all those questions. Indeed, it may take lifetimes of work to ever do so, if we even can do so. Rather, our focus here is on beginning to drive 2 towards potential lines of evidence that may shed light on them. The roots of sociality. We do so from the perspective of human-robot interaction (HRI), using robots as tools to study social cognition, and the dynamics thereof, affords particular benefits. For instance, one advantage of using a robot as one of the interactors (averse to two humans) is that it allows us to “get inside the mind” of one of the interactors and purposely manipulate the interaction in a consistent manner across human subjects. Understanding the role that various factors – such as environmental context or culture – play in social cognition is a challenging problem. This is particularly true in studying cognition via HRI, which entails both the natural social cognition of the human and the “artificial intelligence” of the robot. Clearly, humans interacting with humans have some sort of awareness of the social nature of the interaction – their behavior in those cases differs from that when interacting with inanimate objects such as tools. Humans have some “awareness of social”, some sense of other agents (Froese & Di Paolo, 2011). However, it is not clear how or why, short of positing a special “module” of social cognition in the mammalian brain. Moreover, the synchronization that occurs between human interactors, whether the product of coupled oscillators in some dynamical system or otherwise, presents challenges. Constructing an emergent adaptivity into a robot in order to enable it to step into such a dynamical system (as one of the interactors) demands robot behavior that is emergent itself, i.e. “designed for emergence.” However, as noted elsewhere, we have no idea how to systematically do so (Pfeiffer & Bongard, 2007). Hemmed in, we still are, by our Von Neumann computing paradigm. A number of papers exist that have explored the dynamics of interaction patterns in human-robot interaction through a variety of temporal models – oscillating dynamical systems, Markov decision processes, etc. (e.g. Michalowski et al., 2007; Kahn et al., 2008; Kahn et al., 2010b). The work proposed here builds on this, exploring how such interaction patterns vary across culture, context, and individual characteristics of the human interactor. These factors are subsumed into the larger interaction system, influencing the unfolding of the system over time (i.e. the dynamics). Such influences should be detectable in the way people respond to the robot, and shape common patterns in the interaction data. In other words, they should be inherent in the sociality of the interaction. The overarching question is 3 whether we can figure out how to utilize dynamical aspects of the social interaction in order to imbue our interactive technologies (robots, clinical AI, decision support systems, etc.) with some “awareness of social”, and potentially create more natural interaction paradigms for those technologies. In plain language, interaction is a system, and if we want to design interactive technologies, we are really designing the system, not the technology itself: “We must go beyond the view that defines interaction as simply the spatio-temporal coincidence of two agents that influence each other. We must move towards an understanding of how their history of coordination demarcates the interaction as an identifiable pattern with its own internal structure, and its own role to play in the process of understanding each other and the world.” (De Jaegher & Di Paolo, 2007, pp. 492) 1.3 Minimalist Robotics and “Robotic Science” as an Approach for Studying Social Interaction A principle goal in this work is taking an empirical approach to designing robots, with a