Socially Interactive Robots as Mediators in Human-Human Remote Communication Fotios Papadopoulos Submitted to the University of Hertfordshire in partial fulfilment of the requirements of the degree of Doctor of Philosophy July 2012 Adaptive Systems Research Group, School of Computer Science, Faculty of Engineering and Information Sciences, University of Hertfordshire This page was left blank intentionally 2 Abstract This PhD work was partially supported by the European LIREC project (Living with robots and interactive companions) a collaboration of 10 EU partners that aims to develop a new generation of interactive and emotionally intelligent companions able of establishing and maintaining long-term relationships with humans. The project takes a multi-disciplinary approach towards investigating methods to allow robotic companions to perceive, remember and react to people in order to enhance the companion’s awareness of sociability in domestic environments. (e.g. remind a user and provide useful information, carry heavy objects etc.). One of the project's scenarios concerns remote human-human communication enhancement utilising autonomous robots as social mediators which is the focus of this PhD thesis. This scenario involves a remote communication situation between two distant users who wish to utilise their robot companions in order to enhance their communication and interaction experience with each other over the internet. The scenario derived from the need of communication between people who are separated from their relatives and friends due to work commitments or other personal obligations. Even for people that live close by, communication mediated by modern technologies has become widespread. However, even with the use of video communication, they are still missing an important medium of interaction that has received much less attention over the past years, which is touch. The purpose of this thesis was to develop autonomous robots as social mediators in a remote human-human communication scenario in order to allow the users to use touch and other modalities on the robots. This thesis addressed the following research questions: Can an autonomous robot be a social mediator in human-human remote communication? How does an autonomous robotic mediator compare to a conventional 3 computer interface in facilitating users’ remote communication? Which methodology should be used for qualitative and quantitative measurements for local user-robot and user-user social remote interactions? In order to answer these questions, three different communications platforms were developed during this research and each one addressed a number of research questions. The first platform (AIBOcom) allowed two distant users to collaborate in a virtual environment by utilising their autonomous robotic companions during their communication. Two pet-like robots, which interact individually with two remotely communicating users, allowed the users to play an interactive game cooperatively. The study tested two experimental conditions, characterised by two different modes of synchronisation between the robots that were located locally with each user. In one mode the robots incrementally affected each other’s behaviour, while in the other mode, the robots mirrored each other’s behaviour. This study aimed to identify users’ preferences for robot mediated human-human interactions in these two modes, as well as investigating users’ overall acceptance of such communication media. Findings indicated that users preferred the mirroring mode and that in this pilot study robot assisted remote communication was considered desirable and acceptable to the users. The second platform (AiBone) explored the effects of an autonomous robot on human- human remote communication and studied participants' preferences in comparison with a communication system not involving robots. We developed a platform for remote human- human communication in the context of a collaborative computer game. The exploratory study involved twenty pairs of participants who communicated using video conference software. Participants expressed more social cues and sharing of their game experiences with each other when using the robot. However, analysis of the interactions of the participants with each other and with the robot show that it is difficult for participants to 4 familiarise themselves quickly with the robot while they can perform the same task more efficiently with conventional devices. Finally, our third platform (AIBOStory) was based on a remote interactive story telling software that allowed users to create and share common stories through an integrated, autonomous robot companion acting as a social mediator between two people. The behaviour of the robot was inspired by dog behaviour and used a simple computational memory model. An initial pilot study evaluated the proposed system's use and acceptance by the users. Five pairs of participants were exposed to the system, with the robot acting as a social mediator, and the results suggested an overall positive acceptance response. The main study involved long-term interactions of 20 participants in order to compare their preferences between two modes: using the game enhanced with an autonomous robot and a non-robot mode. The data was analysed using quantitative and qualitative techniques to measure user preference and Human-Robot Interaction. The statistical analysis suggests user preferences towards the robot mode. Furthermore, results indicate that users utilised the memory feature, which was an integral part of the robot’s control architecture, increasingly more as the sessions progressed. Results derived from the three main studies supported our argument that domestic robots could be used as social mediators in remote human-human communications and offered an enhanced experience during their interactions with both robots and each other. Additionally, it was found that the presence of intelligent robots in the communication can increase the number of exhibited social cues between the users and are more preferable compared to conventional interactive devices such as computer keyboard and mouse. 5 Acknowledgments I would like to thank a number of people that without their help, this PhD research would not be possible. I thank my principal supervisor, Prof. Kerstin Dautenhahn who supported, guided and helped me all these years, my secondary supervisor Dr. Wan Ching Ho who inspired and help me during the implementation, testing and evaluation of my studies and who also helped me in my social life. I would also like to thank the laboratory STRI team who help and gave me advices whenever I had problems and enquires about technical stuff. Additionally, I would like to thank Dr. Adam Miklosi who helped me design natural dog-like robot behaviours for my experiments. I am also grateful of Dr. Farshid Amirabdollahian for his very helpful and constructive comments and recommendations during my first and second viva examinations who helped me form an adequate and feasible research plan. Additionally, I would like to thank all of my friends and colleagues who stood by me and helped me with various phases of my research and in particular, Dr. Kheng Lee Khoey, Kyron Du Casse, Dimitris Zampelis, Michael Bowler and more. I am particularly grateful to my partner Eleni Lamprianidou who has been of immense practical help, showing an interest in all stages of the work, coding a vast number of videos as an external observer, and finally proof-reading an earlier version of this thesis and most of my published work Finally, I would like to thank all the users who participated in our experiment and all the colleagues for their useful feedback during the development and testing. This work was partially supported by the European Commission (EC) and is currently funded by the EU FP7 ICT-215554 project LIREC (Living with Robots and Interactive Companions). The authors are solely responsible for the content of this thesis. It does not represent the opinion of the EC, and the EC is not responsible for any use that might be made of data appearing therein. 6 Contents CHAPTER 1: INTRODUCTION ................................................................................................................... 13 1.1 INTRODUCTION .................................................................................................................................. 13 1.2 RESEARCH QUESTIONS ......................................................................................................................... 19 1.3 METHODOLOGY ................................................................................................................................. 20 1.4 CONTRIBUTION TO KNOWLEDGE ............................................................................................................. 21 1.5 OVERVIEW OF THE THESIS CONTENT ........................................................................................................ 22 1.6 PUBLICATIONS LIST ............................................................................................................................. 24 CHAPTER 2: BACKGROUND AND RELATED RESEARCH ............................................................................. 26 2.1 HUMAN-ROBOT-INTERACTION (HRI) ...................................................................................................... 26 2.2 HUMAN-COMPUTER INTERACTION (HCI) ................................................................................................