View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by University of Hertfordshire Research Archive Facilitating Play Between Children with Autism and an Autonomous Robot Doroth´ee C. M. Fran¸cois October 2008 Submitted to the University of Hertfordshire in partial fulfilment of the requirements for the degree of Doctor of Philosophy Adaptive Systems Research Group, School of Computer Science, Faculty of Engineering and Information Sciences, University of Hertfordshire Facilitating Play Between Children with Autism and an Autonomous Robot Doroth´ee C. M. Fran¸cois University of Hertfordshire PhD Thesis October 2008 Abstract This thesis is part of the Aurora project, an ongoing long-term project investiga- ting the potential use of robots to help children with autism overcome some of their impairments in social interaction, communication and imagination. Autism is a spec- trum disorder and children with autism have different abilities and needs. Related research has shown that robots can play the role of a mediator for social interaction in the context of autism. Robots can enable simple interactions, by initially providing a relatively predictable environment for play. Progressively, the complexity of the interaction can be increased. The purpose of this thesis is to facilitate play between children with autism and an autonomous robot. Children with autism have a potential for play but often encounter obstacles to actualize this potential. Through play, children can develop multidisciplinary skills, involving social interaction, communication and imagination. Besides, play is a medium for self-expression. The purpose here is to enable children with autism to experience a large range of play situations, ranging from dyadic play with progressively better balanced interaction styles, to situations of triadic play with both the robot and the experimenter. These triadic play situations could also involve symbolic or pretend play. This PhD work produced the following results: • A new methodological approach of how to design, conduct and analyse robot- assisted play was developed and evaluated. This approach draws inspiration from non-directive play therapy where the child is the main leader for play and ii the experimenter participates in the play sessions. I introduced a regulation process which enables the experimenter to intervene under precise conditions in order to: i) prevent the child from entering or staying in repetitive behaviours, ii) provide bootstrapping that helps the child reach a situation of play she is about to enter and iii) ask the child questions dealing with affect or reasoning about the robot. This method has been tested in a long-term study with six children with autism. Video recordings of the play sessions were analysed in detail according to three dimensions, namely Play, Reasoning and Affect. Re- sults have shown the ability of this approach to meet each child’s specific needs and abilities. Future work may develop this work towards a novel approach in autism therapy. • A novel and generic computational method for the automatic recognition of human-robot interaction styles (specifically gentleness and frequency of touch interaction) in real time was developed and tested experimentally. This method, the Cascaded Information Bottleneck Method, is based on an information theo- retic approach. It relies on the principle that the relevant information can be progressively extracted from a time series with a cascade of successive bottle- necks sharing the same cardinality of bottleneck states but trained successively. This method has been tested with data that had been generated with a phy- sical robot a) during human-robot interactions in laboratory conditions and b) during child-robot interactions in school. The method shows a sound reco- gnition of both short-term and mid-term time scale events. The recognition process only involves a very short delay. The Cascaded Information Bottleneck is a generic method that can potentially be applied to various applications of socially interactive robots. • A proof-of-concept system of an adaptive robot was demonstrated that is re- sponsive to different styles of interaction in human-robot interaction. Its im- pact was evaluated in a short-term study with seven children with autism. The recognition process relies on the Cascaded Information Bottleneck Method. The robot rewards well-balanced interaction styles. The study shows the potential of the adaptive robot i) to encourage children to engage more in the interaction and ii) to positively influence the children’s play styles towards better balanced interaction styles. It is hoped that this work is a step forward towards socially adaptive robots as well as robot-assisted play for children with autism. iii Acknowledgements This PhD research was supported by a research scholarship of the University of Hertfordshire. The work was partially conducted within the EU Integrated Project RobotCub (Robotic Open-architecture Technology for Cognition, Understanding and Behaviours) and was partially funded by the European Commission through the E5 Unit (Cognition) of FP6-IST under contract FP6-004370. I would like to thank my three supervisors, Kerstin Dautenhahn, Daniel Polani and Stuart Powell, for their their guidance and support. Thank you to Ren´ete Boekhorst who introduced me to statistics for behavioural sciences, and to Chrysto- pher L. Nehaniv for his useful comments on the thesis. I am grateful to my viva’s examiners, Michael Hillman and Rod Adams, for their useful comments and suggestions during the viva. The Adaptive Systems Research Group is a multidisciplinary team and it was a very good experience to be part of it. In particular, I would like to thank Megan Davis, Assif Mirza, Alexander Klyubin, Michael Walters, Arnaud Blanchard and Joe Saunders. I am also grateful to everyone at the school where I conducted the play sessions. Thank you to the teachers and carers from the school who welcomed me and showed great interest in my research. My thanks also go to the parents for accepting their child to take part in the play sessions and for their enthusiasm for my work, and of course to the children who participated in the play sessions. The play sessions with the children were a rich and unforgettable personal experience. Many thanks also go to all my friends. These several years in the UK were a great opportunity to share and learn about different cultures with my friends from Oxford and London. Thank you to Frances, Xavier, Ade, Lily, William, Lisa, Rickard, Maki, Ross, Rani, Ludovic and Sascha for the great moments we shared. Thank you to Ga¨etane, too. And thank you to Sarah, who has always been very attentive despite the miles that separate us. I would like to thank my family for their support. Thank you to my brother, Benjamin, my parents, Dominique and Hugues, and my stepparents, Inge and Guy, who encouraged me all along. Finally, I would like to thank Mathieu, for the wonderful years we have together. Contents 1 Introduction 1 1.1 Motivation ................................. 2 1.2 MethodologyandPracticalEffort. 3 1.3 ContributiontoKnowledge . 4 1.4 Publications resulting from this work . .... 5 1.5 Outlineofthethesis ............................ 6 2 Robot-Assisted Therapy and Education 8 2.1 Human-RobotInteraction . 8 2.2 Child-RobotInteraction . .. .. .. .. .. .. 9 2.3 Robot-Assisted Therapy and Education . 11 2.3.1 Robotic devices as tools for physical rehabilitation ....... 11 2.3.2 The robot as a peer or a playmate for therapy and education . 12 2.4 Methodology ................................ 14 2.4.1 Safety ................................ 14 2.4.2 Short-term / Long-term studies . 14 2.4.3 Behavioural based data analyses . 15 2.4.4 Self-reporting through questionnaires or semi-structured inter- views ................................ 15 2.5 Summary .................................. 15 3 Autism and Play 17 3.1 Autism ................................... 17 3.1.1 Mainimpairments ......................... 17 3.1.2 Autobiographical accounts . 18 3.2 Play ..................................... 20 3.2.1 Whatisplay?............................ 20 3.2.2 Children with autism and play . 21 3.2.3 Whyfocusonplay? ........................ 22 3.3 Approaches in learning in psychology . 23 3.3.1 Behaviouristapproach . 23 3.3.2 Piaget’s constructivist approach . 24 3.3.3 Vygotsky’s influence (socio-constructivist approach) ...... 25 iv CONTENTS v 3.3.4 Bruner’sapproach . 25 3.4 Summary .................................. 26 3.5 ResearchQuestions............................. 27 4 A Novel Approach in Robot-Assisted Play 29 4.1 Introduction................................. 29 4.2 Non-directivePlayTherapy . 30 4.3 RelatedWork................................ 32 4.3.1 Non-directive play therapy for children with autism . 32 4.3.2 Robot-mediated therapy in the context of autism . 33 4.4 Method ................................... 34 4.4.1 Participants............................. 34 4.4.2 Artifact ............................... 35 4.4.3 ProceduresandMeasures . 36 4.4.4 CodingandReliability. 45 4.5 Results.................................... 45 4.6 Discussion.................................. 67 4.7 Conclusion ................................. 71 5 Real-Time Recognition of HRI styles 73 5.1 Introduction................................. 73 5.1.1 Motivation ............................. 73 5.1.2 Criteriatodescribeaninteraction . 74 5.1.3 RelatedWork............................ 75 5.1.4 ArtifactandSensors