AISB 2004 Convention: Motion, Emotion and Cognition Proceedings of the AISB 2004 Symposiium on Language, Speech and Gesture for Expressive Characters 29 March – 1 April, 2004 ICSRiM, University of Leeds, Leeds LS2 9JT, UK www.leeds.ac.uk/aisb www.icsrim.org.uk AISB 2004 Convention 29 March – 1 April, 2004 ICSRiM, University of Leeds, Leeds LS2 9JT, UK www.leeds.ac.uk/aisb www.icsrim.org.uk Proceedings of the AISB 2004 Symposium on Language, Speech & Gesture for Expressive Characters Published by The Society for the Study of Artificial Intelligence and the Simulation of Behaviour http://www.aisb.org.uk ISBN 1 902956 39 0 Contents The AISB 2004 Convention ……………………………………………………………………. ii K. Ng Symposium Preface ……………………………………………………………………………. iii R. Aylett, M. Cavazza, P. Olivier Expressive speech characteristics in the communication with artificial agents ……..………… 1 Kirsten Fischer A probabilistic hierarchical framework for active appearance model ..…….………………… 12 Lukasz Zalewski & Shaogang Gong Modelling character emotion in an interactive virtual environment ..………………………… 20 El Jed Mehdi, Pallamin Nico, Dugdale Julie and Pavard Bernard Artistically based computer generation of expressive motion .……..………………………… 29 Michael Neff & Eugene Fiume Speaking and acting - interacting language and action for an expressive character ………… 40 Sandy Louchart, Daniela Romano, Ruth Aylett & Jonathan Pickering Expressive characters and a text chat interface …………………....………………………… 48 Marco Gillies, Barry Crabtree & Daniel Ballin Speaking with emotions .…………………………………………………………………… 58 Elisabetta Bevacqua, Maurizio Mancini & Catherine Pelachaud Reusing motion data to animate visual speech .……..……………………...………………… 66 James D. Edge Manuel A. Sánchez Lorenzo & Steve Maddock Influences on Embodied Conversational Agent's Expressivity …...…..……………………… 75 Vincent Maya, Myriam Lamolle & Catherine Pelachaud Developing a virtual ballet dancer to visualise choreography .…………………..…………… 86 Royce Neagle, Kia Ng & Roy Ruddle Virtual human signing as expressive animation .……………………………………………… 98 John Glauert, Richard Kennaway, Ralph Elliott & Barry-John Theobald Defining the gesticon: language & gesture coordination for interacting embodied agents ... 107 Brigitte Krenn & Hannes Pirker Artificial companions for older people .……………………………………………………… 116 Gopal Vaswani, David Benyon, Stewart Cringean, Oli Mival & Greg LePlatre “To be or seem to be; that is the question” .…………………………..……………………… 124 Christopher Newell & Alistair Edwards To tell or not to tell...building an interactive virtual storyteller .……………..……………… 132 Andre Silva, Guilherme Raimundo, Celso de Melo & Ana Paiva Pragmatics of Body-Speech Coordination in Multi-Modal Expression …...………………… 136 Satinder P. Gill & Masahito Kawamori i The AISB 2004 Convention On behalf of the local organising committee and all the AISB 2004 programme committees, I am delighted to welcome you to the AISB 2004 Convention of the Society for the Study of Artificial Intelligence and the Simulation of Behaviour (SSAISB), at the University of Leeds, Leeds, UK. The SSAISB is the oldest AI society in Europe and it has a long track record of supporting the UK AI research community. This year, the underlying convention theme for AISB 2004 is “Motion, Emotion and Cognition”, reflecting the current interest in such topics as: motion tracking, gesture interface, behaviours modelling, cognition, expression and emotion simulation and many others exciting AI related research topics. The Convention consists of a set of symposia and workshop running concurrently to present a wide range of novel ideas and cutting edge developments, together with the contribution of invited speakers: • Prof Anthony Cohn Cognitive Vision: integrating symbolic qualitative representations with computer vision; • Prof Antonio Camurri Expressive Gesture and Multimodal Interactive Systems; • Dr David Randell Reasoning about Perception, Space and Motion: a Cognitive Robotics Perspective; and • Dr Ian Cross The Social Mind and the Emergence of Musicality; not to mention the many speakers invited to the individual symposia and workshop, who will made the Convention an exciting and fruitful event. The AISB 2004 Convention consists of symposia on: • Adaptive Agents and Multi-Agent Systems; • Emotion, Cognition, and Affective Computing; • Gesture Interfaces for Multimedia Systems; • Immune System and Cognition; • Language, Speech and Gesture for Expressive Characters; and the • Workshop on Automated Reasoning. The coverage is intended to be wide and inclusive all areas of Artificial Intelligence and Cognitive Science, including interdisciplinary domains such as VR simulation, expressive gesture, cognition, robotics, agents, autonomous, perception and sensory systems. The organising committee is grateful to many people without whom this Convention would not be possible. Thanks to old and new friends, collaborators, institutions and organisations, who have supported the events. Thanks the Interdisciplinary Centre of Scientific Research in Music (ICSRiM), School of Computing and School of Music, University of Leeds, for their support in the event. Thanks to the symposium chairs and committees, and all members of the AISB Committee, particularly Geraint Wiggins and Simon Colton, for their hard work, support and cooperation. Thanks to all the authors of the contributed papers, including those which were regretfully not eventually accepted. Last but not least, thanks to all participants of AISB 2004. We look forward to seeing you soon. Kia Ng AISB 2004 Convention Chair ICSRiM, University of Leeds, School of Computing & School of Music, Leeds LS2 9JT, UK [email protected] www.kcng.org ii Proceedings of the AISB 2004 Symposium on Language, Speech & Gesture for Expressive Characters Symposium Preface Research into expressive characters, for example embodied conversational agents, is a growing field, while new work in human-robot interaction has also focussed on issues of expressive behaviour. With recent developments in computer graphics, natural language engineering and speech processing, much of the technological platform for expressive characters – both graphical and robotic – is in place. However, progress has been hampered by the need to integrate work in various sub-fields of psychology, in natural language processing, speech and in computer graphics, carried out by many different groups in communities that rarely intersect. Other areas, such as integrating gesture and facial expression and affective state with language and speech, are less developed but vital to progress. The symposium aims to bring together psychologists, experts in natural language and speech technologies, researchers in embodied agents (graphical and robotic), affective computing and computer graphics and animation researchers The Organising Committee Chairs: Ruth Aylett, University of Salford, UK Marc Cavazza, University of Teeside, UK Patrick Olivier, University of York & Lexicle Limited Programme Committee: Ruth Aylett, University of Salford, UK Daniel Ballin, BTExact Technologies, Radical Multimedia Lab, UK Paul Brna, University of Northumbria, UK Marc Cavazza, University of Teeside, UK Suresh Manandhar, University of York, UK Dominique Noel, As An Angel SA, France Patrick Olivier, Lexicle & University of York, UK Catherine Pelachaud, University of Paris 8, France Thomas Rist, DFKI, Germany Judy Robertson, University of Edinburgh, UK Daniela Romano, University of Sheffield, UK Takenobu Tokunaga, Tokyo Institute of Technology, Japan iii Artistically Based Computer Generation of Expressive Motion Michael Neff? Eugene Fiume? ?Department of Computer Science University of Toronto neff|[email protected] Abstract Understanding how to create the right movement for a specific character is a significant open problem in computer animation. This same problem, however, has been at the heart of the work of performance artists for hundreds of years. In this work, we try to learn from the lessons contained in the performing arts literature so that we can apply them to creating rich, engaging, animated characters. Three classes of movement properties are identified: those relating to shape, those relating to transitions and those relating to timing. Computational models of some of these properties have been developed and are briefly detailed. A software framework is also presented that allows these properties to be applied independently to a character’s movement. The resulting animation system allows the key aesthetic aspects of movement to be quickly varied and allows motion to be easily customized for a particular character. 1 Introduction directly control how a motion is modified in order to cap- ture the nuances of a specific character’s movements. A talented actor, dancer or animator can take a few ab- Our work learns and applies lessons from the arts lit- stract notions of a character, and from those build a rich, erature. Through studying the performance literature consistent, concrete piece of movement that captures the from theatre, animation, mime, theatre anthropology and essence of that character. This ability to take an abstract movement theory, we have built a list of key movement idea and give it concrete form is the heart of a performer’s properties that significantly affect the aesthetic aspects of art. In the research field of computer animation, we have a character’s movement. These have been divided into barely begun to understand how this mapping works: how three main categories: those that affect the character’s