A Dancing Robot for Rhythmic Social Interaction
Total Page:16
File Type:pdf, Size:1020Kb
A Dancing Robot for Rhythmic Social Interaction Marek P. Michalowski Selma Sabanovic Hideki Kozima Robotics Institute Rensselaer Polytechnic Inst. National Inst. of Information Carnegie Mellon University 110 8th St. and Communications Tech. 5000 Forbes Ave. Troy, NY 12180, USA Hikaridai 3-5, Seika, Soraku Pittsburgh, PA 15213, USA [email protected] Kyoto 619-0289, Japan [email protected] [email protected] ABSTRACT This paper describes a robotic system that uses dance as a form of social interaction to explore the properties and im- portance of rhythmic movement in general social interaction. The system consists of a small creature-like robot whose movement is controlled by a rhythm-based software system. Environmental rhythms can be extracted from auditory or visual sensory stimuli, and the robot synchronizes its move- ment to a dominant rhythm. The system was demonstrated, and an exploratory study conducted, with children interact- ing with the robot in a generalized dance task. Through a behavioral analysis of videotaped interactions, we found that the robot’s synchronization with the background music had an effect on children’s interactive involvement with the robot. Furthermore, we observed a number of expected and unexpected styles and modalities of interactive exploration and play that inform our discussion on the next steps in the design of a socially rhythmic robotic system. Figure 1: Keepon dancing with a child. Categories and Subject Descriptors aesthetic experience). Regardless of motivation or applica- H.1.2 [Models and Principles]: User/machine systems; tion, the success of a face-to-face human-robot social inter- I.2.9 [Artificial Intelligence]: Robotics; J.4 [Social and action depends strongly on the robot perceiving the spatial Behavioral Sciences]: Psychology, Sociology; J.5 [Arts and temporal properties of the interaction and behaving in and Humanities]: Performing arts a coordinated manner. Robotics researchers have devoted many years of work to interactive skills such as turn-taking, General Terms gestures, and emotive expression, all of which are important Design, Experimentation, Human factors for effective communication. However, an increasing amount of attention is being directed to the idea that the comfort and “naturalness” of an interaction may depend on a foun- Keywords dation of far more subtle and fine-grained coordination be- Human-robot interaction, Social robotics, Children, Dance tween the sounds and movements of the two interactors. Dance, as a human art form, can be seen as a representa- 1. INTRODUCTION tion of the imitation and pattern1 that we find pleasurable Socially interactive robots are being designed and built in social interactions. Anthropology and sociology litera- for the purposes of exploration (performing scientific exper- ture seems to support this view, with the observation that iments to learn about human behavior or cognition), service analysis of common social interactions often reveals the coor- (assisting in labor, communication, or access to informa- dinated production of sound and movement in what might tion), and influence (education, therapy, entertainment, or metaphorically be described as a “dance.” What we call dancing, usually to the accompaniment of music, in general consists of more structured, regularized, and exaggerated be- havioral rhythms than those observed in social interactions, Permission to make digital or hard copies of all or part of this work for so it may be a fruitful area in which to develop and test sys- personal or classroom use is granted without fee provided that copies are tems that would allow social robots to perceive and exhibit not made or distributed for profit or commercial advantage and that copies rhythmic behaviors and participate in an interactive pro- bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific 1 permission and/or a fee. Aristotle’s Poetics proposes these as the two innate moti- HRI’07,March8-11,2007,Arlington,Virginia,USA. vations for creating and enjoying art; we note that they are Copyright 2007 ACM 978-1-59593-617-2/07/0003 ...$5.00. also fundamental to both dance and social interaction. 89 cess of coordination and synchrony. We believe that inves- synchrony in human interaction, it is clear that difficulties tigating the effects of rhythmic synchrony in dance-oriented in establishing interactional synchrony can make face-to-face human-robot social interactions will be informative in de- interpersonal communication and interaction difficult, if not signing robots that can perceive and behave according to impossible; in fact, marked asynchronies within an individ- natural rhythms in more open-ended interactions. ual’s own behaviors [8], along with abnormal entrainment In this paper, we review literature that emphasizes the in interpersonal interactions [6], are often characteristic of importance of rhythmic synchrony (the coupling of rhyth- pathologies such as autism and schizophrenia. mic patterns in verbal and nonverbal behavior) in social in- Attention to the rhythmic characteristics of nonverbal in- teraction, and we review robots that have made progress teraction has not been widely adopted in social robotics toward this end. We then present the system we have built, research. General ideas of turn-taking in conversation are which can dance not only to music, but with a human dance widely implemented, but fine-grained rhythmic perception partner in a coordinated way. The system consists of a and synchrony by a robot has been difficult to develop. robot, Keepon [17], and a software architecture that ex- Ogawa et al. [21] recognize the importance of rhythmic en- tracts rhythm from auditory and visual sensor data. To trainment to non-verbal cues such as nodding and gesture in our knowledge, this system is novel both in the abstraction vocal communcation. In their InterRobot humanoids, which of rhythmic perception from a particular modality and in are used in pairs for embodied telecommunication between the agility and quality of its movement. Children were ob- remote interactors, they use nonverbal rhythmic cues such as served interacting with the system during a laboratory open facial expressions and bodily movements that are automat- house event for members of the community (fig. 1), with ically synchronized to the speech input of a remote human the aim of studying the ways in which children would phys- interactor. Andry et al. [1], with the view of interpersonal ically and rhythmically interact with the robot. This was coordination as a method of learning, use synchronized im- an exploratory study of a novel software architecture used itation as a way for robots to learn sensory-motor corre- to control a well-established, dynamic hardware platform in spondences and rhythmic motion sequences. The humanoid order to observe rhythmic interaction between people and robot Nico [9] and the robot Haile [25] perform drumming the robot. While the task was centered around dancing, the synchronized to another person or a conductor as an ex- interactions remained unconstrained so that we were able to ample of the kind of rhythmic entrainment that might be observe a wide range of interaction styles that will inform fu- applied to other social tasks. ture work. We expected that the rhythmic synchrony of the In existing projects, rhythmic interaction is generally based robot’s movements with the music and the children would on auditory cues and limited perception of embodied move- have an effect on the quality of interactions and the rhyth- ment. However, robots should be able to “tune in” to the mic behaviors of children. Our analysis and observations bodily rhythms of their interaction partners as well as to suggest that, indeed, this was the case; that music provided generate such nonverbal behaviors (e.g. nodding, sway- a powerful environmental cue for the negotiation of rhythmic ing, gesturing) themselves. Tanaka et al. [23] created non- behavior relative to that of the robot; and that the robot’s interactive and interactive (posture mirroring) dance modes responsiveness to people’s behaviors positively affected their for a QRIO robot in a playroom with children. They found engagement with the robot. We additionally discovered pos- a significant difference between the two conditions with re- sible gender differences in the degree of rhythmic movement spect to the time children spent with the robot, yet this and touching behavior with the robot. experiment concerned contingency rather than the type of rhythmic synchrony we are developing. We have observed 2. RHYTHM IN SOCIAL INTERACTION similar differences between contingent and non-contingent behaviors in rhythmic interaction between children and a The past four decades have witnessed the emergence and robot-like puppet [19], and furthermore observed a higher development of a line of research spanning disciplines such incidence of rhythmic play by children when the robot be- as anthropology, psychology, and cognitive science in under- haved contingently. standing the organizing principles for interaction between Since both self-synchrony (rhythmic matching of move- intelligent embodied agents. We now understand interac- ments and vocalizations) and interactional synchrony are tion not as a sequential process of give-and-take, but as a important for normal human-human interaction [15, 8], these dynamic process of coordinated activity between constantly capabilities should be considered