Socially Intelligent Agents and the Primate Social Brain --Towards A

Home , Memetics

Socially Intelligent Agents and The Primate Social Brain - Towards a Science of Social Minds Kerstin Dautenhahn Adaptive Systems Research Group Faculty of Engineering & Information Sciences University of Hertfordshire Hatfield, Herts ALl0 9AB, United Kingdom K. Daut enhah~@herts. ac. uk http : //homepages.feis. herts, ac. uk/~comqkd

Abstract Introduction: Socially Intelligent Agents - The Humanin the Loop

This article puts research on socially intelligent agents (SIA) in the broader context of how humans (and As Reeves and Nass have shown (RN96) humans tend other primates) perceive and interact with the social to treat computers (and media in general) as peo- world. Phylogenetic (evolutionary) and ontogenetic ple. I believe that this ’media equation’ (media equals (developmental) issues are discussed with respect real life) is particularly relevant for socially intelli- the social origin of primate and human intelligence gent agents (SIA’s) research with the ’human in the and human culture. Implications for designing artifacts and for the evolvability of human societies are loop’, namely studying the relationship between so- outlined. A theory of empathy is presented that is cially intelligent agents and humans as designers, users~ based on current research on the primate social brain. observers, assistants, collaborators, competitors, cus- Research projects that investigate some of these is- tomers, or friends. In order to acknowledge the ’hu- sues are reviewed. I argue that Socially Intelligent man in the loop’ I suggested in (Dan98) a list of design Agents (SIA) research, although strongly linked software and robotic engineering, goes beyond a soft- guidelines for SIA technology, identifying the following ware engineering paradigm: it can potentially serve as roles of humans and suggesting that a balanced de- a paradigm for a science of social minds. A system- sign of socially intelligent agents need to address these atic and experimental investigation of human social roles: Humans are embodied agents, humans are ac- minds and the way humans perceive the social world tive agents, humans are individuals, humans are social can result in truly social artifacts, that are integrated in humansociety. beings, humans are storytellers, humans are anima- tors, humans are autobiographic agents, humans are observers. Not unsurprisingly, SIA research is more than other agent research strongly inspired and mo- "Once there lived a robot called Rob, he was made tivated by findings outside software engineering and by a mad professor called Brain-Box on Jan the 3rd computer science, in particular the humanities, so- 2,000,000 in Germany. Everybody thought it was bril- cial sciences, and natural sciences. As such, SIA re- liant and on Jan 30th the same year it ran away and search is different from the field of agent-based com- it ran away to England and terrorised England people puting research that views agents primarily as a sob- looked to see who or what had done it. The robot had killed someone and taken her brain they got the po- ware engineering paradigm (cf. (JSW98), (WJK00)). lice out and they looked into it the police didn’t know In contrast, SIA research benefits from viewing agents who or what had done it. One day someone called Tod in the larger picture of autonomous agents as defined saw Rob and called the police to come and get Rob but by Franklin and Graesser ((FG97)): "An autonomous when the police cam Rob had gone. Tod got a fine for supposedly lying to the police. The next year the police agent is a system situated within and a part of the en- had still not found Rob. Sometime in the last year Rob vironment that senses that environment and acts on had fallen o~ something and he was found in pieces in it, over time, in pursuit of its own agenda and so as to a junk yard. Some people phoned the police the police affect what it senses in the future". This definition is came out and he had gone. The village dug a massive very attractive since it applies easily both to natural pit to try and catch Rob. Now Rob was as high as and artificial systems. Franklin and Graesser (F097) siz houses he was eating a lot o/junk metal. One day rob was walking along and he fell in the pit, the people also propose a natural kinds taxonomy of agents~ dis- found him in the pit and he was killed." (Christopher, tinguishing between biological, robotic and computa- 9 years old, (BD99)). tional agents as instances of the class of autonomous

35 agents. In a different paper Stan Franklin argues for Fogg calls the study of planned persuasive effects of the stud), of autonomousagents as embodiedartificial computer technologies captology. Figure 1 shows the intelligence (Fra97)1. I believe that this was an im- functional triad of computer persuasion. Following portant step, namely viewing (autonomous) agents Fogg’s terminology SIA’s might fall under the cate- vehicles and subjects for the investigation of artificial gory of ’social actors’, where agents can adopt ani- (and human)intelligence, and in this paper I like to ar- mate characteristics, play animate roles, and follow so- gue that socially intelligent agents can similarly serve cial dynamicsfor the purpose of creating relationships as tools and vehicles for the study of artificial (and with humansand invoke social responses. In this sense, human) social minds. SIA’s are persuasive technologies and therefore issues Socially Intelligent Agents research is concerned of design, credibility (TF99), and ethics of persuasive with agents and human-agent interactions whereby the technology (BN99) also apply to SIA’s technology, agents showaspects of human-style social intelligence. particular to the new generation of highly interactive ’social’ software and robotic agents, manyof them de- Socially intelligent agents are biological or ar- scribed2. in this volume and elsewhere. tificial agents that show elements of (human- style) social intelligence. The term artificial social intelligence refers then to an instantiation of Social Actor human-stylesocial intelligence in artificial agents. Creates relationships (Dau98)

Social intelligence can be natural (humans) or artificial (computational or robotic agents), but within the context of human-style social interaction and behaviour. Please note that social intelligence in this sense does not makeclaims on howintelligent the agent needs to be: often simple strategies can be socially very effective! Although SIA research is primarily interested in human-style behaviour and interactions, humansocial intelligence has a history, both evolutionary as well as developmental. Below I therefore discuss a few findings from primatology and developmental psychology Tool Medium and their implications for SIA research. Increases abilities Provides experience The social, ethical, cultural, as well as cognitive implications of SIA technology are important issues to consider. Even on the level of the individual human, Figure 1: The functional triad of computer persuasion, interactions with SIA’s can influence a humanbeings redrawn from (Fog99). attitudes, behaviour and minds, and in this way em- power as well as manipulate humans (see discussion in (ND00). In (Fog99) B.J. Fogg discusses computers The Life-Like Agents Hypothesis as persuasive technologies. In contrast to other non- As discussed in the previous section, SIA’s are often persuasive technologies, "persuasive computing tech- designed to ’imitate’ life. Based on what I called previ- nology is a computing system, device, or application ously the ’Life- Like Agents Hypothesis’ this approach intentionally designed to change a person’s attitudes can be characterised as follows (Dan99c): or behaviour in a predetermined way." Furthermore, 2Examplesof collections of articles on SIA research 1 Embodimentis here consideredas ’embodiedin the sit- are: K.Dautenhahn, C. Numaoka(guest editors): "So- uated sense of being autonomousagents structurally cou- cially Intelligent Agents", Special Issues of AppliedArti- pled with their environment’.In this sense software agents ficial Intelligence, Vol 12 (7-8), 1998, and Vol 13 (3) 1999, can be as embodiedas robotic and biological agents. Tom K.Dautenhahn(2000) HumanCognition and Social Agent Quick developeda definition of embodimentbased on struc- Technology, John Benjamins Publishing Company-,B. Ed- tural coupling and mutual perturbation between an agent mondsand K. Dautenhahn(guest editors): Social Intel- and its environment,a definition that applies to different ligence, special issue of Computationaland Mathematical kinds of systems, including autonomousagents, and which Organisation Theory, 5(3), 1999, K. Dautenhahn(guest allows to measuredifferent degrees of embodimentquanti- itor): Simulation Modelsof Social Agents, special issue of tatively (QDNR99b),(QDNR99a), (QDNR99c). Adaptive Behaviour, 7(3-4), 1999

36 "Artificial social agents (robotic or software) Attitudes towards Socially Intelligent which are supposed to interact with humans are Agents: Anthropomorphism and most successfully designed by imitating life, i.e. Behaviour Reading making the agents mimic as closely as possible animals, in particular humans. This comprises According to the Social Intelligence Hypothesis, dis- both ’shallow’ approaches focussing on the pre- cussed in more detail below, the evolution of primate sentation and believability of the agents, as well as intelligence is linked with an increase of the complexity ’deep’ architectures which attempt to model faith- of primate social life ((BE98), (WE97)). The argument fully animal cognition and intelligence. Such life- suggests that during the evolution of humanintelli- like agents are desirable since 1) The agents are gence a transfer took place from social to non-social supposed to act on behalf of or in collaboration intelligence so that hominid primates could transfer with humans; they adopt roles and fulfill tasks their expertise from the social to the non-social domain normally done by humans, thus they require hu- (see review in (Gig97)). An interesting aspect of manforms of (social) intelligence, 2) Users prefer kind of transfer is given by Mithen (Mit96), who ex- to interact ideally with other humansand less ide- plains the evolution of anthropomorphic thinking with ally with human-like agents. Thus, life-like agents an accessibility between the domains of social intelli- can naturally be integrated in human work and gence and natural history intelligence so that "people entertainment environment, e.g. as assistants or could be thought of as animals, and animals could be pets, and 3) Life-like agents can serve as models thought of as people", (Mithen 1996, p. 224). for the scientific investigation of animal behaviour The attribution of humanmotivation, characteris- and animal minds". tics, or behaviour to inanimate objects, animals or natural phenomena is usually called anthropomorphism (see The American HeritageQDictionary of the En- glish Language). Anthropomorphism is often dis- Argument (3) is certainly valid and need not missed as a curiosity or unscientific phenomenonand discussed here. However, arguments (1) and (2) only relatively few scientific work outside philoso- not as straightforward as they seem. Designing life- phy have experimentally addressed the issues of how like agents that closely mimic human appearance or and why people tend to adopt an intentional stance behaviour can unnecessarily restrict and narrow the (Den71), (Den87), namely treating non-humam apparent and actual functionality of an agent. Sim- jects and animals as intentional objects (what seems ilarly, imagine mobile phones were designed so that to be based on the human mindreading or social they had the shape of old-fashioned dial-operated tele- competence system, see discussion below). It is of- phones. It then could be disturbing or at least puzzling ten suggested that physical likeness, familiarity, phy- for people to find out that the mobile phone might logeny and/or cultural stereotypes are important fac- have more functionalities (e.g. sending and receiving tors. Well knownis the study by Eddy et al. (EGP93) email, browsing the Webetc.) than the original model. who investigated peoples tendency to anthropomor- Thus, ’new’ designs not imitating any other previously phise animals (see summary in (Wat97)). The study existing object might better suit a piece of technol- identified two primary mechanisms why people at- ogy that is combining functionalities in a novel way or tribute similar experiences or cognitive abilities to ani- has new functionalities. A social interface agent (e.g. mals, based on 1) the degree of physical similarity, and in an e-commerce context) presented, with humanoid 2) the degree of an existing attachment bond (familiar- appearance and behaviour might have the advantage ity). Dogs and cats are more familiar to most people of evoking an initial feeling of ’familarity’ in a human than frogs, primates are physically (and behaviourally) customer, but 1) humancustomer’s are then likely to similar to humans. expect the agent to show other humancharacteristics This study seems to support the above mentioned and functionalities, humanknowledge, personality and Li/e-Like Agents Hypothesis, namely that humanoid other characteristics of humansin general (including agents that look like humans should be more believ- that it understands jokes and possesses commonsense able and successful as social interaction partners for knowledge), and sales agents in particular, and 2) new humans than non-humanoid agents (assuming that hu- or different functionalites that the real agent does not mans mostly enjoy interacting with other humans). possess need to be integrated in a plausible way in the However, other evidence suggests that not physical agent’s behaviour, without breaking the suspense of similarity, but behaviour in context matters. Mitchell disbelief (Mat97) (see (ND00)for further discussion and Hamm(MH97) provided undergraduate students these issues). with narratives depicting different mammalianagents (including humans) showing behaviour that suggested St. Margarets Junior School in Durham, UK. A num- jealousy or deception. The students were then asked ber of working hypotheses were addressed with respect to answer questions on particular psychological charac- to howthe children portrayed robots. In three studies terisations of the agents. The narratives varied accord- the children were asked a) to draw a picture of a robot, ing to species, context in which an agent’s behaviour b) to write a story about the robot they had drawn. occurred, and the degree of emphasis that the nar- These studies were observational. The third study had rative was about a particular species of animal (or the format of an informal, guided and filmed interview human). The behaviour was constant in all narra- while the children were in a group interacting with two tives. Mitchell and Hammfound that variations in mobile robots (see figure 2) that were running in an en- context influenced the psychological characterisations, vironment with a lightsource. The robots were simple but variations in species and emphasis did not, i.e. behaviour-based vehicles (Bra84). the psychological characterisations of all species were almost always similar: "Nonscientists (and some sci- entists as well) apparently use a mammal’sbehavior- in-context (whether human or not) as evidence of its psychological nature, regardless of the mammal’sphys- ical similarity, familiarity, or phylogenetic closeness to humans, or the mammal’scultural stereotype; psychological terms are not used specifically for humans, but rather are depictive of behaviour-in-context". Inter- estingly, the notion that behaviour matters more than appearance in ascribing intentionality is supported by an experimental study published in 1944 (HS44) that convincingly demonstrates the effects of the ’intentional stance’. Here, humansubjects created elaborate narratives about intentional agents when asked to describe movements of moving geometric shapes shown in a silent film. Other studies along research done by Mitchell and Hammand Heider and Simmel could confirm whether this also applies to non-mammalianani- Figure 2: a) The experimental set up and the two au- mals. A particular challenge would be to include com- tonomous, mobile fischertechnik robots, b) Drawings putational and robotic agents in such studies. I sug- of 8-year and 9-year olds, (BD99). gest that behaviour-reading might apply also to inanimate objects such as robots. Every robotics researcher who has ever given a demonstration of autonomous mobile robots to a general audience can confirm how readily humans view robots as people, cf. (Bra84), (BD99). The importance of behaviour expression Results of study a (pictures) showe.g. that the chil- agent building has been recognised e.g. by Phoebe Sen- dren tend to give the robot humanoid faces. Figure 2 gets (Sen98), (Sen00). Her argument is that ’doing shows examples of a variety of drawings by 8/9-year right thing’ (the classical approach of AI approaches to olds, portraying robots. In study b (stories), one re- agent control) needs to be complementedby paying at- sult was that the children tend to put the robots in fa- tention to ’doing the thing right’, in particular creating miliar settings, doing familiar tasks. The robots were believable transitions between agent behaviours. significantly often put in a social context. Study c (interview) showed a clear tendency to anthropomorphise Attitudes Towards Agents: A Case Study the robots, e.g. "I don’t think it likes the light.". The with Robots children also often talked to the robots as if they were animals or small children. Other findings of this case In (BD99) Kate Bumby and Kerstin Dautenhahn in- study, e.g. with respect to attribution of gender or vio- vestigated children’s attitudes towards robots, a brief lence is reported in more detail in (BD99). This single summary is given here. We were interested to find case study cannot answer the question of how children out how children interact and describe robots. Thirty in that age range in general think about robots, but eight children (ages seven to eleven, 21 males, 17 fe- the results give some indication that confirm findings males, BC1 socioeconomic category) were studied at along the lines of studies with computers (RN96). Societies~ of Social Animals all highly social. Eventhe ’solitary’ life style of Pongo Swarm Intelligence: Social Insects Don~t pygmaeus or orangutans, (who nevertheless seem to be highly social in their ability to recognise and interact have Friends with each other) is rather a secondary adaptation to The term ’societies’ is generally applied both to hu- particular environment which demandsa spatially ’dis- man and other animal societies, including social in- tributed’ social organisation. The Social Intelligence sects. Social insects (e.g. termites, bees, ants) are very Hypothesis suggests that primate intelligence primar- well studied and two important theoretical concepts ily evolved in adaptation to social complexity, i.e. in are used to understand coordination in social insect order to interpret, predict and manipulate conspecifics societies, namely self- organisation and stigmergy. Re- (see overview in (BE98), (WE97)). Thus, there cently, models of swarmintelligence and their applica- two important aspects to humansociality: it served as tions to problems like combinatorial optimisation and an evolutionary constraint which led to an increase of routing in communications networks have been studied brain size in primates, this in return led to an increased extensively (BDT99). The concept stigmergy describes capacity to further develop social complexity. a class of mechanisms mediating animal-animal interactions, based on the description of insect behaviour Although it is still unknownwhy hominids needed as stimulus-response (S-R) sequences (even for solitary or chose to live in social groups, this feedback prin- ciple soon led to the development of highly sophisti- species). Stigmergy is based on indirect communica- cated levels of organisation and control in humanso- tion, communication via the environment, and an ex- cieties. In (Rus93) four levels of primate social organ- ample of collective behaviour. isation are discussed which might serve as models for Primate Intelligence: Getting to Know the evolution of primate societies: a) the ’shrew’-type Each Other pre- primates: solitary, manyoffspring, insectivores, e.g. Purgatorius, a 70- million-year-old fossil, b) the In primate societies, and different from membersof so- ’mouse-lemur’-type primates: bush-living, nocturnal, cial insect societies, an individual is not only socially strong mother-daughter bonding (stable matrilines), situated (being part of and surrounded by a social en- social learning (offspring learns from mother), solitary vironment) but also socially embedded (ED98) which males and social groups of mothers and daughters, e.g. means that the agent needs to pay attention to other the 50-million-year-old fossil Shoshoniuscooperi, c) the agents and their interactions individually. Particularly ’Lemur catta’-type diurnal lemurs: appearing about humanprimates are specialised in predicting, manipu- 54 million years ago, social groups (troops), domi- lating and dealing with highly complex social dynam- nant females, submissive males, stable matrilines, oc- ics (involving direct relationships as well as third-party casionally consort bonds between single male and fe- relationships); they possess language as an effective male, e.g. Adapidae, d) the ’chimpanzee’-type lemur- means of preserving group coherence, ’social grooming’ ape: appearing about 24 million years ago, groups of (Dun93), (BD97) and communicate about themselves dominant males and submissive females, stable fami- and others in terms of stories (Dau99b). Humansare lies of mothers and their offspring, male power coali- not only dealing with very complex relationships but tions, e.g. Dryopithecus. The social organisation of seem to have mental ’models’ of themselves, others and recent species of apes shows variations of this pat- the social world (cf. (Whi91), (BC95)). Humans, tern: of harem-structures (gorilla), solitary lifestyle ferent from social insects live in individualised societies (orangutan). Such stages of social organisation can (as do some other species of birds and mammals). related to behavioural as well as cognitive capacities of increasingly complexsocial field and an increasing need primates. to effectively communicatewith each other were likely to have been among the important constraints in the The terms ’theory of mind’ and mindreading are usually used in order to discuss whether an animal is able evolution of humanminds. .Minds are certainly attributed to membersof Homo to reflect on its ownmental states (e.g. desires, inten- sapiens (and as some evidence suggests several other tions and beliefs) and those of others. Researchers have hominid species might have existed with ’minds’), studied whether humans might have particularly spe- but other candidates exist amongmammals (e.g. non- cialised in a theory of mind (PW78), (PP95a). ever, as Richard Byrne pointed out (Byr97), the Social humanapes, dolphins, elephants) and birds (e.g. par- Intelligence Hypothesis might account for the evolution rots and members of the crow family). Interestingly, of primate intelligence, but not for the specific human species which we describe as possessing a ’mind’ are kind of intelligence. Here, narrative psychology and 3This section is based on (Dau00b). studies on the development of autobiographic memory

39 and a ’self’ might offer an explanation: evidence sug- in which these agents ’grow up’. Thus, it’s up to us gests that ’stories’ are the most efficient and natural whether the stories of the future will be nightmares, human way to communicate, in particular to commu- fairy-tales, comedies or adventures. nicate about others (Brugl). Dennett (Den89) even "Oncethey were a robot, that lived in the country. He gards the ’self’ as a ’centre of narrative gravity’. Narra- cameto this cliff, he thoughtit wastoo steep for him to tivity, the capacity to communicatein terms of stories go downso he went downthe steps. He went onto the is therefore regarded an efficient means to communi- beach oh yeah the robot’s nameis Shaped and Shaped cate social matters, and the origin of narratives might went to play in the sea and after a few minuteshe fell to the ground and got washed into shore because the therefore have been a crucial milestone in the evolu- water has gone into his body." (Becky - 8 years old, tion of primate social intelligence (RM95). The Nar- (BD99)). rative Intelligence Hypothesis (Dau99b) proposes that the evolutionary origin of communicatingin stories was Primate Culture: We are not alone correlated with increasing social dynamics amongour The terms anonymousand individualised societies are humanancestors (see figure 3), in particular the ne- used in biology in order to describe two different types cessity to communicateabout third-party relationships of social organisation. Social insects are the most (which in humansreaches the highest degree of sophis- prominent example of anonymous(eusocial) societies tication amongall apes, cf. gossip). where group members do not recognise each other as individuals but rather as group members4. We do not observe bees or termites searching for missing members of their colony. Although individuals adopt spe-

.C cific roles in a colony they do not show individuality "6 ¢:h or ’personality’. The situation is quite different in individualised so-

Lac~c b r~no s, o~md Ir.aroi og 8e- cieties which primate societies belong among. Here gO we find complex recognition mechanisms of kin and group members. This gives rise to complex kinds of social interaction and the development of various forms of social relationships and networks. On the memalo:~ rc~t~fioo 0 behavioural level long-lasting social bonding, attachment, alliances, dynamic (not genetically determined)

~or o! hor~mds hierarchies, social learning, developmentof traditions =’6 etc. are visible signs of individualised societies. In humans the evolution of language, culture and an elaborate cognitive system of mindreading and empathy are characteristics of humansocial intelligence in individualised societies (Dau97). As a consequence Figure 3: Evolution of the Social Mind, modified from the latter, humans are not only paying attention to (Byr97), see text for explanation. other agents and their interactions individually (interactions between distinct personalities), but they use A number of current research is devoted to build- their mental capacities to reason about other agents ing narrative software, virtual or physical environ- and social interactions. It is at present unclear to what ments (e.g. (GC97), (MMP99), (BC00), (BID+99), extent the social intelligence of membersof other an- (MAD+00), (BBA+00)). Supporting human narrative imal species, in particular very social species like ele- intelligence is expected to impact human minds and phants, Grey parrots, non-human apes and cetaceans, our notions of sociality and what we call our ’selves’. In is similar to or different from our own. Similarly, the parallel, investigations into autonomousstory-telling issues of cultural and ’memetic’ evolution is highly agents can result in agents (robotic or software) with controversial. The concept of memes, first introduced genuine narrative minds, being able to tell us interest- by Dawkins (Daw76) comprises ideas, fashions, skills ing stories, listen to and understand our stories, and and other components of human culture. Humancul- make us laugh. A first attempt to a bottom-up ap- ture and the memetic transmission of knowledge, ideas proach to narrative intelligence for autonomousagents 4Note that African naked mole-rats, mammals,show is described in (CD00), (DC00). The kind of stories a eusocial organisation similar to social insects (SJAgl). these agents might tell us will be shaped by the social Thus, the eusocial form of organisation has evolved inde- field and the cultural environment of humansocieties pendentlyin different taxa of animals.

4O and skills is often regarded unique to humansocieties. different chimpanzee communities, data which cannot According to Donald’s discussion of the evolution of be explained by ecological differences of the habitats, culture and cognition (Don93) modern humans have and comprising dozens of different behaviours includ- three systems of memoryorganisation (mimetic skill, ing tool usage, grooming and courtship behaviours language and external symbols) not available to our (WGM+99). Possibly the kind of mechanisms that primate relatives, and these ’inventive capacities’ re- are necessary for and support culture (e.g. cognitive sult in languages, gestures, social rituals, images etc. mechanisms, language, imitation) might be different in According to Tomasello et al. (TKR93)cultural learn- different animal species. As Frans del Waal concludes ing is a uniquely humanform of social learning. Cul- (dW99): "The ’culture’ label befits any species, such tural learning requires three social-cognitive processes as the chimpanzee, in which one community can read- which emerge in human ontogeny: imitative learning, ily be distinguished from another by its unique suite instructed learning (teaching) and collaborative learn- of behavioural characteristics. Biologically speaking, ing. Similarly, Blackmore(Bla99) argues that only so- humans have never been alone - now the same can be phisticated forms of imitation which are characteristic said of culture.". of humans but not non-human primates, were a neces- The striking similarity of cultural transmission of sary prerequisite for memetic replication which leads novel behaviour exhibited by Japanese macaque mon- to human culture. keys and chimpanzees and what we call human cul- Others argue that culture as such is unlikely to be ture questions the uniqueness of humansocieties. Note a feature unique to humansocieties and that the ac- that this behaviour is observed in monkeys, which do quisition of novel behaviours in ’proto-cultures’ can be not show complex forms of social learning like imita- observed in animals. To give an example: traditions tion, and do not seem to possess higher-level ’cogni- have been observed among troops of Japanese macaque tive’ capacities necessary for complex social forms of monkeys (Huf96): Japanese macaques or Macaca fus- ’primate politics’ shown by non-human apes and hu- cata showseveral examplesof the acquisition of innova- mans. However, many non-human primates are very tive cultural behaviours, e.g. sweet potato washing and good social learners (widely using non-imitative forms wheat-washing was invented in 1953 by a young female of social learning, e.g. stimulus enhancementor social and subsequently spreading to older kin, siblings, and facilitation). Reader and Laland (RL99) therefore playmates, eventually to other membersof the troop. gue that the meme concept can and should also be Other observed cultural behaviours are fish eating (as applied to cultural transmission amongnon-human an- many newly acquired food sources initially spreading imals. Animal societies can appear in various forms. from peripheral males to adult females, then from older Humansocieties, human culture and human minds re- to younger individuals), and stone handling or stone flect in manyways their evolutionary origin in animal play (initially spread only laterally amongindividuals societies, animal culture and animal minds. Consider- of the same age). Subsequently all these behaviours ing humanculture in an evolutionary context, linking were passed downfrom older to younger individuals in it to precursors in non-humanprimate societies might successive generations (tradition phase). These exam- help a better understanding of humanculture. ples clearly show the influence of social networks on the transmission phase of novel behaviour: the nature Implications for Evolvability of Human of the behaviour and social networks determine how Societies the behaviours are initially transmitted, depending on According to Kirschner and Gerhart evolvability can who is likely to be together in a certain context and be defined as "the capacity to generate heritable phe- therefore is exposed to the novel behaviour. Innovative notypic variation" (KG98). As outlined above, in pri- behaviours of the kind described here have been inde- mate non-human societies we already observe precur- pendently observed at different sites. Various factors sors of humanculture (e.g. social learning, traditions). have been discussed which influence cultural transmis- For reasons still under dispute our human ancestors sion: environmental factors, gender, and age, and other were required to deal with increasingly complex social social and biological life history variables. For exam- dynamics. Mental capacities evolved which allowed the ple. unlike potato or wheat washing, stone handling evolution of increasingly complex mechanismsof social declines when individuals mature. control, which in return increased the complexity of With respect to cultural transmission in non-human primate societies. apes, recent evaluations of long-term field studies of Based on what the previous sections discussed about chimpanzees or Pan troglodytes give compelling evi- primate societies and culture, the following require- dence for cultural behavioural variants (traditions) ments for mental capacities and social skills which facilitated the evolution of primate culture are proposed: inventing new, more efficient ways of "social grooming" (exceeding the communicative capacities of lan- ¯ Individualised societies: The capacity to identify guage). Another interesting issue discussed by Dunbar and recognise individual group members. (Dun93), (BD97)is that language is 2.8 times more ficient as a mechanismof social bonding in comparison ¯ Social Networks: the capacity to establish, maintain, to physical grooming. The suggestion is therefore that remember and utilise social networks. Three basic humanconversational group sizes should be limited to elements5 in the primate social field are: about 3.8 (which means one speaker and 2-3 listeners). -Forming direct (one-to-one) relationships with Data on small group sizes confirm this hypothesis. group members I showed above that biological evolution led to two -Identifying third-party relationships (relation- distinctively different forms of social organisation in ships among other group members) animal societies (anonymous and individualised societies). It appears that individualised societies were - Recognition of conspecifics as membersin a group necessary (but not sufficient) prerequisite for the evo- hierarchy/social structure (e.g. structures of kin- lution of culture, providing a social environment which ship, allies, dominancehierarchies, etc.) supported the evolution of complex forms of social ¯ Efficient mechanisms of social bonding, either via learning (in particular imitation). The capacity for physical grooming (in on-human primate societies) phenotypic, cultural evolution seems correlated with or via language and communication in narratives particular mental capacities and social skills (see list as efficient ways of ’social grooming’, important for above) which facilitated the evolution of complex forms maintaining the coherence of social groups at differ- of primate societies and primate culture. Primate so- ent levels of social organisation. cial behaviour is well studied, we know less about the social life and mental capacities of non-primate ¯ Social learning: the capacity to use others as ’social species (crows, parrots, cetaceans, elephants, and oth- tools’ (as explained in (Dau95)), via social learning ers). However, when searching for animal culture, mechanisms with varying degrees of what the an- highly social animals in individualised societies are imals learn from each other (cf. social facilitation good candidates. Ants don’t imitate, they don’t learn versus imitation) from each other, primates do. Memes, as the repli- cators of culture, seem to require ’a social host~, and Thus, culture and other characteristics of humanso- memesare transmitted along social networks and de- cieties cannot be separated from specific environmen- pending on interactions its ’host’ is engaged in. These tal (including social) constraints and mental capacities seem to be the natural constraints under which cul- which evolved as adaptations for dealing with such ture is able to evolve in primate societies. The ’magic constraints. Specific adaptations then turned out to numbers’ 150 and 3.8 indicate strong limitations and be prerequisites in the evolution of more sophisticated constraints for the future development of humansoci- forms of primate societies and culture. Although new eties. Systematic investigations that take these cogni- forms of media seem to substantially expand the social tive constraints into consideration could provide a ba- life of humans, even today the same mental capaci- sis for social agent technology that meets the cognitive ties which were involved in the evolution of the human demands of human primates. social animal now pose cognitive limits on the complexity and number of social encounters. Our primate Social Robots in Rehabilitation: the social brain has a limit on the number of individuals Cases of Autism who we can maintain direct social relationships with, namely relationships based on direct social knowledge Autism (around 150), correlated with the relative size of the Although we use the term autism throughout this pa- human neocortex (Dun93), (BD97). This figure can per it is more appropriate to use the term autistic identified consistently in various ancient and present spectrum disorders (ASD) which acknowledges the fact humancultures. This number is significantly larger, that autism occurs in differing degrees and in a vari- namely more than double that observed in any pop- ety of forms. The National Autistic Society (NAS00) ulation of non-humanprimates. Unless drastic (tech- lists the following triad of impairments: 1. Social in- nological) enhancements of humancognitive capacities teraction (difficulty with social relationships, for ex- are invented, this number could only be exceeded by ample appearing aloof and indifferent to other people, 5This is not supposedto be an exhaustive list. 6This section is based on (Dau00a). inappropriate social interactions, inability to relate to to what extent they want to connect to the world of others in a meaningful way, impaired capacity to un- non-autistic people. derstand other’s feelings or mental states). 2. Social communication (difficulty with verbal and non-verbal Brief project Description and Related communication, for example not really understanding Work the meaning of gestures, facial expressions or tone of voice). 3. Imagination (difficulty in the development The AURORAproject develops an autonomous, mo- of play and imagination, for example having a lim- bile robot as a therapeutic tool for children with autism ited range of imaginative activities, possibly copied and (Dau99c), (WD99), (DW00). Conceptually, this pursued rigidly and repetitively). proach is strongly related to SeymourPapert’s con- structionist approach towards learning (PapS0). Such In addition to this triad, repetitive behaviour pat- an approach focuses on active exploration of the envi- terns and a resistance to change in routine can gen- ronment, namely improvisational, self-directed, ’play- erally be observed, associated with a significantly re- ful’ activities in appropriate learning environments duced repertoire of activities and interests, stereotypi- (’contexts’) which can be used as ’personal media’. cal behaviour, and a tendency of fixation to stable envi- the mid- 1960ies Papert and his colleagues at the MIT ronments. Depending on what is included in ’autism’, AI LAB developed the programming language LOGO rates of occurrence are given which range between 5- which has been widely used in teaching children. A 15 in 10000. Instead of a physical handicap which remote controlled device (a ’turtle’ robot) was devel- prevents people from physically interacting with the oped which is moving according to a set of LOGOin- environment, people with autism have great difficulty structions, cf. the LEGO/LOGOArtificial Life Toolkit in making sense of the world, in particular the social for children (Res89). In 1976 Sylvia Weir and Ricky world. Autism can but need not be accompanied by Emanuel (WE76) published research which used such learning disabilities. At the higher functioning end of a LOGOlearning environment to catalyse communica- the autistic spectrum we find people with Asperger tion in an autistic child. They report on their experi- Syndrome. Some of them manage to live indepen- ence with a seven-year-old autistic boy and the positive dently as adults and to succeed in their profession, but effects of his explorations in controlling a LOGOtur- only by learning and applying explicit rules in order to tle on his behaviour. A more recent approach using overcomethe ’social barrier’ (Gra95), (GS96), (Sch97). more interactive rather than remote-controlled tech- Instead of picking up and interpreting social cues ’nat- nology for rehabilitation of autistic children is taken in urally’ they can learn and memorise rules about what the Affective Social Quotient (ASQ) project, (Blo99). kind of behaviour is socially appropriate during inter- Here, embedded technology is used to support autis- action with non-autistic people. Autism is not, as has tic children in learning about social-emotional cues. long been assumed in public, a voluntary decision to Short ’emotionally charged’ video clips are used to- retract from the world: people with autism do not have gether with a set of physical stuffed ’dolls’ (embodying the choice to live socially or not, the decision has been one emotional expression) through which the child can made for them. Twodifferent viewpoints exist on how interact with the movies. By touching the doll the child to connect the autistic with the non- autistic world: ei- can match a doll with a video clip. A child can explore ther efforts are undertaken to teach people with autism emotional situations by picking up dolls with certain the skills they need to survive in the world of ’normal’ emotions, or the system can prompt the child to pick people, or it is suggested that they might be happier up dolls that go with certain clips. A therapist is able living separately in a world specifically designed for to control and monitor the interactions. The system them. From all what we know about the way indi- shows that human-intensive, repetitive aspects of ex- viduals with autism feel (see books written by Tem- isting behavioural therapy techniques can potentially ple Grandin and others), they are painfully aware of be automated. their ’being different’ from other people, and express In recent years the AURORAproject described be- the wish to be part of the ’world outside’. Accepting low and work by Francois Michaud (Michaud, this the differences, empowering people with autism, and volume; (MCLL00), (MLL+00)) who develops inter- linking their world with the world that non-autistic esting interactive robotic designs, is taking up this people are living in poses many challenges. In order line of work. Since end of 1998 the project AU- to understand people with autism we have to under- RORA(AUtonomous RObotic platform as a Reme- stand better the causes of autism, and can find ways to dial tool for children with Autism) investigates how empowerthem, including computer and robotic tech- an autonomous mobile robot can be developed into nology, so that they have the choice of whether and a remedial tool in order to encourage children to be- comeengaged in a variety of different interactions that ’playing’ with an interactive autonomousrobot as it is possess features which are important elements of hu- used in the AURORAproject, 2) the robot can engage mansocial behaviour (eye-contact, joint-attention, ap- the child in interactions which demonstrate important proach, avoidance, following, imitation games etc.). aspects of human-humaninteraction (e.g. eye-contact, The children who are interacting with the robot are turn-taking, imitation games), and 3) (as a long term between 8-12 years of age, including children who are therapeutic goal), while slowly increasing the robot’s non-verbal, i.e. they cannot use language or usually behaviour repertoire and the unpredictability of its ac- do not use language. In the rehabilitation of children tions and reactions, the robot can be used to guide the with autism therapeutic issues (e.g. eye contact, joint children towards more realistic and ’complex’ forms of attention, turn taking, reading mental states and emo- social interactions resembling human-humaninterac- tions) are usually addressed in constrained teaching tion. This approach is based on two areas of theoretical sessions (HBCH99). In contrast, robot-human inter- work, namely mindreading and interaction dynamics. actions in the AURORAproject are it unconstrained These issues and their implications for the AURORA and unstructured, the children are allowed to interact project are described in the following two sections. with the robot in whatever body position they prefer (e.g. lying on the floor, crawling, standing, cf. figure Mindreading 4, they are also free to chose howthey interact with the robot (touching, approaching, watching from a dis- Generally, humans are from an early age on at- tance, picking it up etc.). Interference is only necessary tracted to self-propelled objects which are moving au- if the child is about to damagethe robot or if the child tonomously and seemingly with ’intention’ (Dau97). (by pressing buttons) switches off the robot so that (PP95b) a theory of humansocial competence is pre- needs to be restarted. Such conditions are much dif- sented that consists of three units: the first unit (in- ferent from other projects on robot-human interaction tentional system) identifies self- propelled movements wtfich are based on structured and constrained set ups in space and interprets them as intentional, engaged (e.g. KISMET,or the ROBOTAdolls, see this vol- in goal-directed behaviour, such as escaping from con- ume) where the human is expected to interact with finement, making contact with another intentional ob- the robot while adopting a particular position and ori- ject, overcominggravity (e.g. seeking to climb a hill). entation towards the robot (e.g. sitting face-to-face Animate and inanimate objects are distinguished since in close distance to an interactive robot that is not only animate objects can moveboth in space and time moving in space). The particular challenges faced in without the influence of other objects. Movementin the AURORAproject, in the broader context of re- place is interpreted as animate but not intentional. habilitation, together with a more detailed discussion The second unit is the social system which specifies of therapeutical issues involved, is given in (WD99), the changes that the intentional objects undergo. It (DW00). allows to interpret relations e.g. as possession or group membership. The third unit is the theory of mind sys- Theoretical Background and Working tem, which outputs explanation, states of mind, per- Hypotheses ception, desire, belief, and its variations. These mental states are used to explain the actions. Premack and The AURORAproject deliberately uses a non- Premack’s theory of human social competence shows humanoid robot, based on the observation that chil- great similarity with Baron-Cohen’ssuggestion of four dren with autism prefer a predictable, stable environ- mechanisms underlying the human mindreading sys- ment and that many people with autism have diffi- tem (BC95). The first mechanismis the intentionality culty interpreting facial expressions and other social detector that interprets motion stimuli (stimuli with cues in social interactions. Consequently, they often self- propulsion and direction) in terms of the men- avoid social interactions since people appear unpre- tal states of goal and desire. These primitive men- dictable and confusing. Generally, using a robot as tal states are basic since they allow making sense of a remedial toy takes up the challenge of bridging the universal movementsof all animals, namely approach gap between the variety and unpredictability of human and avoidance, independent of the form or shape of social behaviour (which often appears frightening to the animal. The ID mechanism works through vision, children with autism) and the predictability of repet- touch and audition and interprets anything that moves itive and monotonous behaviour which children with with self-propelled motion or produces a non-random autism prefer and which can be performed by mobile sound as an object with goals and desires. The sec- robots (see discussion in (Dan99c)). We hypothesise ond mechanism as part of Baron-Cohen’s mindread- that a child with autism 1) is sufficiently interested in ing system is the eye-direction detector (EDD)which

44 works only through vision. The EDDdetects the presence of eye-like stimuli, detects the direction of eyes, and interprets gaze as seeing (attribution of perceptual states). This mechanismallows interpreting stimuli in terms of what an agent sees. ID and EDDrepresent dyadic relations (relations between two objects, agent and object or agent and self) such as ’Agent X wants Y: or ’Agent X sees Y’, however they not allow to establish the link between what another agent sees and wants and what the self sees and wants. Sharing per- ceptions and beliefs is beyond the ’autistic universe’, it requires the additional mechanisms SAM(shared- attention-mechanism, allows to build triadic representations: relations between an agent, the self, and a Figure 4: An autistic boy playing with the Labo-1 mo- third object) and TOM(theory-of-mind mechanism). bile robot which was kindly donated by Applied AI ID, EDD, SAMand TOMmake up a fully developed Systems Inc. The child is not afraid to let the robot humanmindreading system as it exists in biologically come physically very close to his body, including the normal children above the age of four. In normal de- face. velopment, from birth to about 9 months a child can only build dyadic representations based on ID and basic functions of EDD.Prom about 9 to 18 months SAM comes on board and allows triadic representations that make joint attention possible. SAMlinks EDDand ID, so that eye direction can be read in terms of basic mental states. Prom about 18 to 48 months TOMcomes on board, triggered by SAM.The arrival of TOMis visible e.g. through pretend play. Note, that earlier mechanisms are not replaced by newer ones, they still con- tinue to function. According to Simon- Baron’s analysis children with autism possess ID and EDD. TOM is missing in all children with autism while some of them possess SAM.Referring to this theoretical framework, the working hypotheses (section 2.4) studied Figure 5: The child frequently ’reaches out’ to the the AURORAproject clearly address the ID and EDD robot, ’testing’ its front sensors and eliciting the mechanisms. In the same way as biologically normal robot’s response to approach or avoid. After 20 min- children above 4 years of age detect, are attracted to, utes the teacher ended the interaction since the child and interpret autonomous, self-propelled objects such had to go back to class. as robots as ’social agents’, we hypothesise that children with autism can accept a mobile robot as a social agent. the infant 1) to identify people as opposed to other objects, and 2) to use the hke-me-test in order to dis- Interaction Dynamics tinguish between different persons. Motivated by this research we suggested a conceptual framework in or- The second strand of theories which the AURORA der to classify different and increasingly complex dy- project is influenced by concerns interaction dynam- namics in robot-human interactions (DW00). Within ics between babies and their caretakers as studied in this framework, robot-human interactions in the AU- developmental psychology, e.g. (Bu179), (UBKV89), RORAproject are designed where synchronisation of (.Me196), (MM99).A more detailed account of these movements, temporal coordination, and the emergence sues in the general context of robot-human interaction, of imitation games are used as important mechanisms and their relevance in the AURORAproject is given for making ’social contact’ between the robot and the in (DW00), we can only present a brief summaryhere. child. It is hoped that such an approach which fo- Infants seem to detect specific temporal and structural cuses on interaction dynamics rather than cognitive aspects of infant-caretaker interaction dynamics. It is reasoning mechanismscan incrementally facilitate and suggested that turn-taking and imitation games allow strengthen temporal aspects which are so fundamental to the development of social competence and the seems to be specialised in (Bro97). ability to socially interact with people (cf. (Hal83). Inspired by autism research and Barrett-Lennard’s cyclic/phasic model of empathy, (BLS1), (BL93), I AURORA: Preliminary Results gested in (Dau97) and (Dau99a) to distinguish between Initial trials in the AURORAproject stressed the in- two different mechanisms: a) empathic resonance, an dividual nature of the specific needs of children with immediate, direct way of re-experiencing, and b) bio- autism, but the), also showed that most children re- graphical reconstruction, namely reading another per- sponded very well and with great interest to the au- sons mind by re-constructing the other’s autobiograph- tonomous robot, see figures 4, 5. In a recent series ical context (who that person is, where he comes from, of comparative trials where the children were play- what the relationship is with oneself, what behaviour ing with the robot (condition 1) and also (separately) might be expected etc.). Barrett-Lennard’s empathy with a passive non-robotic toy (condition 2) children cycle is a process between two people, involving ex- showed greater interest in interactions with the robot pressing and receiving empathy. than with the "nanimate’ toy (quantitative data will be published in a forthcoming publication by Werry and Dautenhahn). Also, children often showed in- hltersubjective experience creased interest in the front part of the robot where the pyro-electric sensor is attached, a sensor with strongly eye-like features (eye- like shape, located at the dis- Mind- tal end of the robot’s preferred direction of movement, prominent position raised above the chassis, direction of the sensor changing according to ’gaze’). These ob- % servations seem to confirm our hypothesis that interactions in the AURORAproject can successfully built on mechanisms of intentionality detection (ID mechanism) and eye-direction- detection (EDDmechanism). Please note that mobile robots are only seen as potentially one form of therapy, which might complement other forms of therapies (see review in (DW00)). interesting line for future research is to study the application of virtual environments for children with autism, as discussed in (Dan00a). A particular problem we encounter in the AURORA / project is that (with few exceptions) we cannot ask our subjects, the), do not give verbal feedback, techniques like interviews or questionnaires are impossible. This puts particular emphasis on the analysis of behaviour and interaction. I believe that the field of socially intelligent agents has a huge potential in education, therapy Intersubjective experience and rehabilitation. However, new design and evalua- tion techniques and methodologies need to be developed (cf. (MJL00), (MAD+00). Figure 6: A Theory of empathy based on issues discussed in this paper, see text for explanation. Empathy In (Dau97) (see also (Dau99a)) I discussed empathy Recent experiments on empathic accuracy (the abil- a fundamental, experiential mechanism with humans ity to read and understand reliably another person’s use to bond and understand each other. Also, empa- intentions, beliefs, etc.), as well as neurophysiologi- thy can be considered as a means of social learning via cal experiments with monkeys point towards an excit- bonding with other people. According to Wispe empa- ing possibility howempathic resonance might actually thy is a wayof ’knowing’, as opposedto ’relating’ which be grounded in biological mechanisms: Neurons were occurs in sympathy (Wis86). Brothers considers em- found in area F5 of the monkey brain that discharge path)- (Bro89) a biological phenomenon,an ’emotional when the monkey grasps or manipulates objects, but communication’ system that the human social brain also when the monkeyobserves an experimenter mak- 46 ing a similar gesture ((GFFR96), (RFGF98)). reasoning about another person’s beliefs, desires, goals, (Arb01) speculates that all primates (including emotions etc. and biographical reconstruction. mans) might share the mirror system as a neurobio- Empathydoes not only occur in face-to-face contact logical mechanismunderlying imitation (note that im- with another person, it can also be evoked by reading itation can be shown for humans and other apes, but a book or watching a movie, i.e. without any feed- is difficult to confirm for monkeys, who are neverthe- back from the character/person we might empathise less good social learners, (VF01)). Some researchers with. If I empathise with a human being (whether even suggest that the mirror system in F5 (analogous real, enacted, fictional or imagined) empathy is nev- to Broca’s area in humans, important for language), ertheless based on my assumption that the other hu- is ’grounding’ language in gestures and body language man is to some extent ’like me’. An important chal- (Arb01), (RA98). Although the findings of neurophys- lenge is then to create empathic relationships with non- iological studies in monkeysneed to be confirmed for humanartifacts. Here, as shown in figure 7 we cannot humans and further understood for all primates and assume an ’understanding’ from the artifact. How- non-primate animals, it is suggested that mirror neu- ever, our own behaviour-reading and expression mech- rons could be the basis for a simulation theory of empa- anisms still work, and non-biological socially intelli- thy (GG98). Previously, in discussions on how people gent agents could exploit this, see (Dau95), (BDH98), ascribe mental states to themselves and others, a sim- (BS00), (BA00), and other research projects on social ulation theory was opposed to a theory theory (Go192), robots described in this volume and elsewhere. (Got92). Supporters of the simulation theory favour a process of "putting oneself in the other’s place’, as opposed to (detached) reasoning about other’s beliefs, emotions etc. The importance of the mirror system in this context is that (in support of the simulation theory) it could be nature’s solution - at least in somepri- mates - to solving the correspondence problem for era- pathy and creating intersubjective experience by creating a commonshared context and shared understanding of actions and affordances. Figure 6 shows mechanisms and processes which were discussed above and how they might fit into a theory of empathy. Here, two persons are linked via a commonsocial ’currency’, namelyfacial expressions, body language, gestures, im- "’Social Currency" ? itation games~ interactions dynamics, spatial-temporal dynamics as they are are studied in proxemics (the stud)’ of human’s perception and use of space, cf. (Hal68) (Hal83)), etc. These are important automatic empathy that creates intersubjective experience and physiological synchrony (LR97). This is accompanied by a cognitive, controlled mechanism o/ empathy, a more deliberative inference making (HW97), what we called biographical reconstruction. The theory sketched in figure 6 needs to be confirmed by the discover), mirror neurons e.g. for facial expressions and other gestures, as speculated in (Bro97), p. 78. Si- mon Baron-Cohen’s theory of mindreading nicely fits in this framework, as well as Mitchell and Hamm’sdis- Figure 7: Empathising with agents? cussion of behaviour reading, see above. ID and EDD might (in normally developed humans) play a strong role in behaviour reading (although they do not seem Quo Vadis ? to be necessary, since empathy does not rely on the visual channel alone, cf. blind people, or Mitchell and I argued in this paper that Socially Intelligent Agents Hamm’sstudy of behaviour reading with narratives (SIA) research, although strongly linked to software (MH97)). TOMmight be part of controlled empathy, and robotic engineering, goes beyond a software engineering paradigm: it can potentially serve as a

47 paradigm for a science of social minds. This paper Robert A. Barton and Robin I.M. Dunbar. Evolution gave some indications of a few research questions that of the social brain. In Andrew Whiten and Richard W. I believe are important. A systematic and experimen- Byrne, editors, Machiavellian Intelligence II Extensions tal investigation of human social minds and the way and Evaluations, chapter 9, pages 240-263. Cambridge University Press, 1997. humans perceive the social world can result in truly social artifacts, socially intelligent agents that are in- K. Bumby and K. Dautenhahn. Investigating children’s attitudes towards robots: A case study. In tegrated in human society, e.g. social robots that meet Proc. CT99, The Third International Cognitive Tech- the cognitive and social needs of humans. Such social nology Conference, August, San Francisco, available at agents might become more and more similar to us, in http://www, cogtech, org/CTgg, pages 391-410, 1999. ways which could even make it difficult to distinguish A. Billard, K. Dautenhahn, and G. Hayes. Experiments between them and human beings, cf. (Fon97), (Fon00), on human-robot communication with robota, an imita- (Dau00c). tive learning and communication doll robot. Technical Report CPM-98-38, Centre for Policy Modelling, Manch- "Once there was a robot called Jig Jag and Jig Jag lived ester Metropolitan University, UK, 1998. in the countryside. One day Jig Jags lights started to flash, that meant that the robot had an idea. "I think Eric Bonabeau, Marco Dorigo, and Guy Theraulaz. I will go for a walk", so Jig Jag went into a field with SwarmIntelligence: From Natural to Artificial Systems. some sheep in it and the silly robot tried to talk to the Oxford University Press, New York, Oxford, 1999. sheep, "Silly, silly, Jig Jag". Next Jig Jag saw some R. W. Byrne and A. Whiten (Eds.). Machiavellian intel- cows in the next field, so silly Jig Jag tried to talk ligence. Clarendon Press, 1998. to the cows! After that Jig Jag went to the shops, he wanted to buy some bolts and oil. So Jig Jag went into Aaron F. Bobick, Stephen S. Intille, James W. Davis, the hardwareshop, but poor Jig Jag set the alarm off. Freedom Baird, Claudio S. Pinhanez, Lee W. Campbell, So Jig Jag went into another hardware store across the Yuri A. Ivanov, Arjan Schtte, , and Andrew Wilson. The road. So the robot tried to get into the shop but again kidsroom: A perceptually-based interactive and immer- Jig Jag set the alarm off. So poor Jig Jag had to go sive story environment. Presence, 8(4):369-393, August home empty handed." (Lanren - 8 years old, (BD99)). 1999. Godfrey T. Barrett-Lennard. The empathy cycle: refine- Acknowledgements ment of a nuclear concept. Journal of Counseling Psy- chology, 28(2):91-100, 1981. I like to thank Katherine Bumby and Iain Werry whose work is summarised in this paper and helped me to Godfrey T. Barrett-Lennard. The phases and focus of empathy. British Journal of Medical Psychology, 66:3-14, develop my ideas. The AURORAproject is supported 1993. by an EPSRC grant (GR/M62648). Susan Blackmore. The Meme Machine. Oxford University References Press, 1999. Michael Arbib. The mirror system, imitation, and the evo- Katherine H. Blocher. Affective Social Quest (ASQ). lution of language. In K. Dautenhahn and C. L. Nehaniv Teaching emotion recognition with interactive media and (Eds.), Imitation in Animals and Artifacts. MITPress (in wireless expressive toys. Master’s Thesis for Master of press), 2001. Science in Media Technology Massachusetts Institute of Technology, MIT, USA, 1999. Alan H. Bond and Matt Ashton. The robot as person. Workshopon Interactive Robotics and Entertainment D. Berdichevsky and E. Neunschwander. Toward and (WIRE-2000), Pittsburgh, April 2000, 2000. ethics of persuasive technology. Communications of the ACM,42(5):51-58, 1999. Steve Benford, Benjamin B. Bederson, Karl-Petter Akesson, Victor Bayon, Allison Druin, Par Hansson, Valentin Braitenberg. Vehicles: Experiments in Synthetic Juan Pablo Hourcade, Rob Ingram, Helen Neale, Claire Psychology. MIT Press, Cambridge, 1984. O’Malley, Kristian T. Simsarian, Danae Stanton, Yngve Leslie Brothers. A biological perspective on empathy. Sundbald, and Gustav Taxen. Designing storytelling tech- American Journal of Psychiatry, 146(1):10-19, 1989. nologies to encourage collaboration between young children. Proc. CHI 2000, April 1-6, 2000, The Hague, The Leslie Brothers. Friday’s footprint: how society shapes the Netherlands, 2000. human mind. Oxford University Press, New York, Oxford, 1997. Simon Baron-Cohen. Mindblindness: An Essay on Autism and Theory of Mind. A Bradford Book, The MIT Press, Jerome Bruner. The Narrative Construction of Reality. Cambridge, London, 1995. Criticial Inquiry, 18(1):1-21, 1991. M. Umaschi Bers and J. Cassell. Children as designers Cynthia Breazeal and Brian Scassellati. Infant-like social of interactive storytellers "let me tell you a story about interactions between a robot and a human caretaker. To myself". In Kerstin Dautenhahn, editor, HumanCogni- appear in Special issue of Adaptive Behavior on Simula- tion and Social Agent Technology, chapter 16, pages 61- tion Models of Social Agents, guest editor Kerstin Daut- 83. John Benjamins Publishing Company, 2000. enhahn, 2000.

48 M. Bullowa. Before Speech. Cambridge University Press, Daniel C. Dennett. The intentional stance. MIT Press, 1979. 1987. R. W. Byrne. Machiavellian intelligence. Evolutionary Daniel C. Dennett. The origins of selves. Cogito, 3, pp. Anthropology, 5:172-180, 1997. 163-73, Autumn 1989. Reprinted in Daniel Kolak and R. Martin, eds., Self and Identity: Contemporary Philosoph- Steven Coles and Kerstin Dautenhahn. A robotic story- ical Issues, Macmillan, 1991, 1989. teller. Proc. SIRS2000, 8th Symposium on Intelligent Robotic Systems, The University of Reading, England, Merlin Donald. Precis of origins of the modern mind: 18-20 July 2000, pp. 393-401, 2000. Three stages in the evolution of culture and cognition. Behavioral and Brain Sciences, 16:737-791, 1993. Kerstin Dautenhahn. Getting to know each other - artificial social intelligence for autonomousrobots. Robotics R. I. M. Dunbar. Coevolution of neocortical size, group and Autonomous Systems, 16:333-356, 1995. size and language in humans. Behavioral and Brain Sci- ences, 16:681-735, 1993. Kerstin Dautenhahn. I could be you - the phenomenolog- ical dimension of social understanding. Cybernetics and Frans B.M. de Waal. Cultural primatology comes of age. Systems, 25(8):417-453, 1997. Nature, 399:635-636, 1999. Kerstin Dautenhahn and Iain Werry. Issues of robot- Kerstin Dautenhahn. The art of designing socially intel- humaninteraction dynamics in the rehabilitation of chil- ligent agents: science, fiction and the humanin the loop. dren with autism. To be published in Proc. FROMAN- Applied Artificial Intelligence Journal, Special Issue on IMALS TO ANIMATS, The Sixth International Confer- Socially Intelligent Agents, 12(7-8):573-617, 1998. ence on the Simulation of Adaptive Behavior (SAB2000), Kerstin Dautenhahn. Embodiment and interaction in so- 11 - 15 September 2000, Paris, France, 2000. cially intelligent life-like agents. In C. L. Nehaniv, editor, B. Edmonds and K. Dautenhahn. The contribution Computation for Metaphors, Analogy and Agents, pages of society to the construction of individual intelligence. 102-142. Springer Lecture Notes in Artificial Intelligence, Techical Report CPM-98-42, Centre for Policy Modelling, Volume 1562, 1999. Manchester Metropolitan University, UK, 1998. Kerstin Dautenhahn. The lemur’s tale - story-telling in T. J. Eddy, G. G. Gallup, and D. J. Povinelli. Attribution primates and other socially intelligent agents. Proc. Nar- of cognitive states to animals: Anthropomorphismin com- rative Intelligence, AAAI Fall Symposium 1999, AAAI parative perspective. Journal of Social Issue, 49(1):87 Press, Technical Report FS-99-01, pp. 59-66, 1999. 101, 1993. Kerstin Dautenhahn. Robots as social actors: Aurora Stan Franklin and Art Graesser. Is it an agent, or just and the case of autism. In Proc. CT99, The Third Inter- a program?: A taxonomy for autonomous agent. In Pro- national Cognitive Technology Conference, August, San ceedings of the Third International Workshop on Agent Francisco. pages 359-374, 1999. Theories, Architectures, and Languages, published as In- Kerstin Dautenhahn. Design issues on interactive envi- telligent Agents III, pages 21 35. Springer-Verlag, 1997. ronments for children with autism. Proc. The 3rd Inter- B. J. Fogg. Introduction: Persuasive technologies. Com- national Conference on Disability, Virtual Reality and As- munications of the ACM,42(5):27-29, 1999. sociated Technologies, ICDVRAT2000, 23-25 September Leonard N. Foner. Entertaining agents: a sociological 2000, Alghero, Sardinia, Italy, 2000. case study. In W. Lewis Johnson, editor, Proc. of the First Kerstin Dautenhahn. Evolvability, culture and the pri- International Conference on Autonomous Agents, Marina mate social brain. In C. L. Nehaniv, editor, Proceedings del Rey, CA, USA, February 5-8, pages 122-129, 1997. of the Evolvability Workshopat the Seventh International Leonard N. Foner. Are we having fun yet? using so- Conference on the Simulation and Synthesis of Living Sys- cial agents in social domains. In Kerstin Dautenhahn. tems (Artificial Life VII), pages 23-26. 2000. editor, HumanCognition and Social Agent Technology, Kerstin Dautenhahn. Reverse engineering of societies - a chapter 12, pages 323-348. John Benjamins Publishing biological perspective. Proc. AISB SymposiumStarting Company, 2000. from Society - the application of social analogies to com- Stall Franklin. Autonomous agents as embodied ai. Cy- putational systems, a symposium at AISB-00, 17th-20th bernetics and Systems, 28(6), 1997. April 2000, University of Birmingham, England, pp. 21- 31, 2000. Jennifer W. Glos and Justine Cassell. Rosebud: A place for interaction between memory,story, and self. In Richard Dawkins. The Selfish Gene. Oxford University Jonathon P. Marsh, Chrystopher L. Nehaniv, and Barbara Press: 1976. Gorayska, editors, Proceedings of the Second Internatwnal Kerstin Dautenhahn and Steven Coles. Robotic story- Conference on Cognitive Technology, pages 88-97. IEEE tellers: A bottom-up computational framework for the Computer Society Press, 1997. stud)’ of narrative intelligence in autonomous agents. V. Gallese, L. Fadiga, L. Fogassi, and G. Rizzolatti. Ac- Proc. Narrative and Interactive Learning Environments tion recognition in the premotor cortex. Brain, 119:593- Edinburgh, Scotland 30th August - 1st September 2000, 609, 1996. pp. 51-59, 2000. V. Gallese and A. Goldman. Mirror neurons and the sim- D. C. Dennett. Intentional systems. Journal of Philoso- ulation theory of mind-reading. Trends in Cognitive Sci- phy, 68:87-106, 1971. ences, 2(12):493-501, 1998. Gerd Gigerenzer. The modularity of social intelligence. In F. Michand, A. Claver, G. Lachiver, and M. Lucas. De- Andrew Whiten and Richard W. Byrne, editors, Machi- signing toy robots to help autistic children - an open de- avellian Intelligence H Extensions and Evaluations, chap- sign project for electrical and computer engineering educa- ter 10, pages 264-288. Cambridge University Press, 1997. tion. Proc. American Society for Engineering Education, June 2000, 2000. Alvin I. Goldman. In defense of the simulation theory. Mind and Language, 7(1-2):104-119, 1992. A. Meltzoff. The human infant as imitative generalist: a 20-year progress report on infant imitation with implica- Robert M. Gordon. The simulation theory: objections tions for comparative psychology. In B. G. Galef and C. M. and misconceptions. Mind and Language, 7(1-2):11-33, Heyes, editors, Social Learning in Animals: the Roots of 1992. Culture, pages 34~370. Academic Press, New York, 1996. Temple Grandin. Thinking in pictures. Doubleday Pub- R. W. Mitchell and M. Hamm.The interpretation of ani- lisher, New York, London, Toronto, Sydney, Auckland, mal psychology: Anthropomorphism or behavior reading? 1995. Behaviour, 134:173-204, 1997. T. Grandin and M. M. Scariano. Emergence: Labeled Steven Mithen. The prehistory of the mind. Thames and autistic. "~¥arner Books, 1996. Hudson Ltd, London, England, 1996. E. T. Hall. Proxemics. Current Anthropology, 9(2-3):83- Stacy C. Marsella, W. Lewis Johnson, and C. LaBore. In- 95, 1968. teractive pedagogical drama. In Proceedings of the Fourth E. T. Hall. The Dance o/Life: The Other Dimension o/ International Conference on Autonomous Agents, June 3- Time. Anchor Books, Doubleday, 1983. 7, Barcelona, Spain, pag~ 301-308. ACMPress, 2000. Patricia Howlin, Simon Baron-Cohen, and Julie Hadwin. F. Michaud, P. Lepage, J.-D. Leroux, M. Clarke, F. Be- Teaching Children with Autism to Mind-Read. John Wiley langer, Y Brosseau, and D. Neu. Mobile robotic toys and Sons, 1999. for autistic children. Proc. International Symposiumon Robotics, Montral, May 2000, 2000. F. Heider and M. Simmel. An experimental study of apparent behavior. American Journal of Psychology, 57:243- A. N. Mettzoff and M. K. Moore. Persons and represen- 259, 1944. tation: why infant imitation is important for theories of human development. In J. Nadel and G. Butterworth, ed- Michael A. Huffman. Acquisition of innovative cultural itors, Imitation in Infancy, pages 9-35. Cambridge Uni- behaviors in nonhuman primates: a case study of stone versity Press, 1999. handling, a socially transmitted behaviour in japanese macaques. In Cecilia M. Heyes and Bennett G. Galef Jr., I. Machado, C. Martinho, and A. Paiva. Once upon a editors, Social learning in animals, chapter 13, pages 267- time. Proc. Narrative Intelligence, AAAI Fall Symposium 289. Academic Press, 1996. 1999, AAAIPress, Technical Report FS-99-01, pp. 115- 119, 1999. S. D. Hodges and D. M. Wegner. Automatic and controlled empathy. In W. Ickes, editor, Empathic accuracy, NAS. http:~r.oneworld.org/autism_uk/index.html. pages 311-339. The Guildford Press, 1997. Last referenced on 14th of February, 2000, 2000. Chrystopher L. Nehaniv and Kerstin Dautenhahn. Liv- N. R. Jennings, K. Sycara, and M. Wooldrige. A roadmap ing with socially intelligent agents: a cognitive technology of agent research and development. Autonomous Agents view. In Kerstin Dautenhahn, editor, HumanCognition and Multi-Agent Systems, 1:7-38, 1998. and Social Agent Technology, chapter 16, pages 415-426. Marc Kirschner and John Gerhart. Evolvability. Proe. John Benjamins Publishing Company, 2000. Natl. Acad. Sci, 95:8420-8427, 1998. Seymour Papert. Mindstorms: Children, Computers, and R. ~,V. Levenson and A. M. Ruef. Physiological aspects Powerful Ideas. Basic Books, New York, 1980. of emotional knowledge and rapport. In W. Ickes, edi- Daniel J. Povinelli and Todd M. Preuss. Theory of mind: tor, Empathic accuracy, pages 44-72. The Guildford Press, evolutionary history of a cognitive specialization. Trends 1997. in Cognitive Neurosciences, 18:418-424, 1995. Jaime Montemayor, Houman Alborzi, Allison Druin, Jim David Premack and Ann James Premack. Origins of hu- Hendler, Deborah Pollack, Jessica Porteous, Lisa Sher- man social competence. In Michael S. Gazzaniga, editor, man. Asmara Afework, Jack Best, Joe Hammer, Alean- The cognitive neurosciences, pages 205-218. A Bradford der Kriskal, Abigail Lal, Thomas Plaisant Schwenn, Lau- Book, The MIT Press, 1995. ren Sumida~ and Rebecca V~ragner. From pets to sto- rykit: Creating new technology with an intergenerational D. Premack and G. Woodruff. Does the chimpanzee have design team. Proceedings 2000 Workshop on Interactive a theory of mind? Behavioral and Brain Sciences, 4:515- Robotics and Entertainment (WIRE-2000) in cooperation 526, 1978. with AAAI, April 30 - May 1, 2000, The Robotics Insti- tute Carnegie Mellon University Pittsburgh, Pennsylva- T. Quick, K. Dantenhahn, C. Nehaniv, and G. Roberts. The essence of embodiment: A framework for understand- nia, USA, 2000. ing and exploiting structural coupling between system and .Michael Mateas. Computational subjectivity in virtual environment. Proc. CASYS’99,Third International Con- world avatars. In Socially Intelligent Agents, pages 43-45. ference on Computing Anticipatory Systems, HEC, Lige, AAAIPress, Technical report FS-97-02, 1997. Belgium, August 9 -14, 1999.

5O T. Quick, K. Dautenhahn, C. Nehaniv, and G. Roberts. Elisabetta Visalberghi and Dorothy Fragaszy. Do mon- On bots and bacteria: Ontology independent embodi- keys ape? - ten years after. In K. Dautenhahn and C. L. ment. Proc. ECAL99, Fifth European Conference on Ar- Nehaniv (Eds.), Imitation in Animals and Artifacts. MIT tificial Life, Switzerland, September, pp. 339-343, 1999. Press (in press), 2001. T. Quick, K. Dautenhahn, C. Nehaniv, and G. Roberts. Stuart N. K. Watt. Seeing things as people: anthropomor- Understanding embodiment, system-environment cou- phism and common-sense psychology. Unpublished PhD pling and the emergence of adaptive behaviour. In: A. thesis, Department of Psychology, The Open University, Drogoul, J.-A. Meyer(eds.) Intelligence artificielle situ~e, 1997. Hermes Science Publications, Paris, 1999, pp. 13-31, 1999. Iain Werry and Kerstin Dautenhahn. Applying robot G. Rizzolatti and M. A. Arbib. Language within our technology to the rehabilitation of autistic children. grasp. Trends in Neurosciences, 21(5):188-194, 1998. Proc. SIRS99, 7th International Symposiumon Intelligent Robotic Systems ’99, 1999. Mitchel Resnick. LEGO, LOGO,and Life. In C. G. Lang- ton, editor, Proc. of an Interdisciplinary Workshopon the S. Weir and R. Emanuel. Using LOGOto catalyse com- Synthesis and Simulation of Living Systems, Los Alamos, munication in an autistic child. Technical report, DAI New Mexico, September 1987, pages 397-406, 1989. Research Report No. 15, University of Edinburgh, 1976. G. Rizzolatti, L. Fadiga, V. Gallese, and L. Fogassi. Pre- Andrew Whiten and R. W. Byrne (Eds.). Machiavellian intelligence ii.: Evaluations and extensions. Cambridge motor cortex and the recognition of motor actions. Cog- University Press, 1997. nition and Brain Research, 3:131-141, 1998. A. Whiten, J. Goodall, W.C. McGrew, T. Nishida, Simon M. Reader and Kevin N. Laland. Do animals have V. Reynolds, Y. Sugiyama, C.E.G. ~tin, R.W. Wrang- memes? Journal of Memetics - Evolutionary Models of ham, and C. Boesch. Cultures in chimpanzees. Nature, Information Transmission, 3(2), 1999. 399:682-685, 1999. Stephen John Read and Lunn Carol Miller. Stories are Andrew Whiten. Natural Theories of Mind. Basil Black- fundamental to meaning and memory: for social crea- well, 1991. tures, could it be otherwise? In Robert S. Wyer, editor, Knowledge and Memory: the Real Story, chapter 7, Lauren Wispe. The distinction between sympathy and pages 139-152. Lawrence Erlbaum Associates, Hillsdale, empathy: to call forth a concept, a word is needed. Jour- New Jersey, 1995. nal of Personality and Social Psychology, 50(2):314-321, 1986. B. Reeves and C. Nass. The Media Equation. Cambridge University Press, 1996. M. Wooldridge, N. R. Jennings, and D. Kinny. The gaia methodology for agent-oriented analysis and design. Robert Jay Russell. The Lemurs’ Legacy. The Evolution Journal of Autonomous Agents and Multi-Agent Systems, of Power, Sex, and Love. G.P. Putnam’s Sons, New York, 3(3):285-312, 2000. 1993. Susanne Sch/ifer, editor. Sterne, .4pfel und rundes Glas. Mein Leben mit Autismus. Verlag Freies Geistesleben und Urachhaus GmbH, Stuttgart, Germany, 1997. Phoebe Sengers. Do the thing right: an architecture for action-expression. In Katia P. Sycara and Michael Wooldrige, editors, Proc. of the Second International Con- ference on Autonomous Agents, Minneapolis/St. Paul, USA, May 9-13, pages 24-31, 1998. Phoebe Sengers. Narrative intelligence. In: HumanCog- nition and Social Agent Technology, Ed. Kerstin Dauten- hahn, John Benjamins Publishing Company, 2000. Paul W. Sherman, Jennifer U.M. Jarvis, and Richard D. Alexander, editors. The Biology of the Naked Mole-Rat. Princeton University Press, Princeton, N.J, 1991. S. Tseng and B. J. Fogg. Credibility and computing technology. Communications of the ACM, 42(5):39-44, 1999. M. Tomasello, A.C. Kruger, and H.H. Rather. Cultural learning. Behavioral and Brain Sciences, 16(3):495-552, 1993. I. Uzgiris, J. Benson, J. Kruper, and M. Vasek. Estab- lishing action-environment correspondences: Contextual influences on imitative interactions between mothers and infants. In J. Lockman and N. Hazen, editors, Action in Social Context, pages 103-127. New York: Plenum, 1989.