Aalborg Universitet Sustaining Emotional Communication when Interacting with an Android Robot Vlachos, Evgenios DOI (link to publication from Publisher): 10.5278/vbn.phd.hum.00037 Publication date: 2015 Document Version Publisher's PDF, also known as Version of record Link to publication from Aalborg University Citation for published version (APA): Vlachos, E. (2015). Sustaining Emotional Communication when Interacting with an Android Robot. Aalborg Universitetsforlag. Ph.d.-serien for Det Humanistiske Fakultet, Aalborg Universitet https://doi.org/10.5278/vbn.phd.hum.00037 General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. ? Users may download and print one copy of any publication from the public portal for the purpose of private study or research. ? You may not further distribute the material or use it for any profit-making activity or commercial gain ? You may freely distribute the URL identifying the publication in the public portal ? Take down policy If you believe that this document breaches copyright please contact us at [email protected] providing details, and we will remove access to the work immediately and investigate your claim. Downloaded from vbn.aau.dk on: October 05, 2021 SUSTAINING EMOTIONAL COMMUNICATION WHEN INTERACTING WITH AN ANDROID ROBOT AN WHEN INTERACTING WITH COMMUNICATION EMOTIONAL SUSTAINING SUSTAINING EMOTIONAL COMMUNICATION WHEN INTERACTING WITH AN ANDROID ROBOT BY EVGENIOS VLACHOS DISSERTATION SUBMITTED 2015 EVGENIOS VLACHOS SUSTAINING EMOTIONAL COMMUNICATION WHEN INTERACTING WITH AN ANDROID ROBOT by Evgenios Vlachos Dissertation submitted . Thesis submitted: December 15, 2015 PhD supervisor: Associate Prof. Henrik Schärfe Aalborg University, Denmark PhD committee: Prof. Peter Øhrstrøm (chairman) Aalborg University, Denmark Principal Researcher Shuichi Nishio ATR Kyoto, Japan Associate Prof. Anne Gerdes University of Southern Denmark, Denmark PhD Series: Faculty of Humanities, Aalborg University ISSN (online): 2246-123X ISBN (online): 978-87-7112-434-7 Published by: Aalborg University Press Skjernvej 4A, 2nd floor DK – 9220 Aalborg Ø Phone: +45 99407140 [email protected] forlag.aau.dk © Copyright: Evgenios Vlachos, except where otherwise stated. This thesis has been submitted for assessment in partial fulfillment of the Ph.D. de- gree. The thesis is based on nine scientific papers; seven published, one accepted for publication, and one submitted for publication. Parts of the papers are used direc- tly, or indirectly in the extended summary of the thesis. As part of the assessment, co-author statements have been made available to the assessment committee and are also available at the Faculty. The thesis is not in its present form acceptable for open publication, but only in limited and closed circulation as copyright may not be ensured. Printed in Denmark by Rosendahls, 2015 CV CV Evgenios Vlachos holds a M.Sc. in Electronic Automation (2009) from National and Kapodistrian University of Athens, Greece, and a M.A. in Human Centered Informatics (2012) from Aalborg University, Denmark. After his graduation he became a Scientific Assistant at the Department of Communication and Psychology, Aalborg University, and was affiliated with the Geminoid-DK project. Since 2013, he has been a regular reviewer for the International Journal of Social Robotics, and since 2014 he has been a Board Member of the International Conference on Social Computing and Social Media (SCSM) one of the Conferences jointly held under one management and one registration in the context of the International Conference on Human-Computer Interaction (HCII). In 2014, he was also enrolled at the Ph.D. programme in Human Centered Communication and Informatics at Aalborg University where he continued his research on Social Human-Robot Interaction. In the summer of 2015, Evgenios Vlachos engaged in a research stay at Hiroshi Ishiguro Laboratories (HIL) at Advanced Telecommunications Research Institute International (ATR) in Japan. He has until now presented his work in international conferences around the world (China, Denmark, Greece, Italy, Japan, Romania, U.S.A), and has published articles in books, conference proceedings and journals. He has received a Best Paper Award at the Ph.D. Course Writing and Reviewing Scientific Papers (Spring 2, 2015) at Aalborg University, a Best Interactive Presentation Springer Award at the International Conference on Social Robotics (ICSR2012), and three scholarship grants from the Greek State Scholarships Foundation for excellent scholastic performance. His research interests include emotional affordances in human-robot interaction, social robotics, nonverbal behavior, user behavior understanding, maintaining durable interactions with robots, robot-ethics, and societal impacts of social robots. I SUSTAINING EMOTIONAL COMMUNICATION WHEN INTERACTING WITH AN ANDROID ROBOT ENGLISH SUMMARY The more human-like a robot appears and acts, the more users will have the belief of communicating with a human partner rather than with an artificial entity. However, current robotic technology displays limitations on the design of the facial interface and on the design of believable Human-Robot Interaction (HRI), therefore when directly interacting with such robots discomfort might be created in recognizing either the robotic expression of emotion, and/or the probable following action of the robot. Failure of the roboticist to meet the expectations that rise from the anthropomorphic appearance of an android related to its actions, perception and intelligence, or failure to identify which robot type is qualified to perform a specific task, might lead to disruption of HRI. This study is concerned with the problem of sustaining emotional communication when interacting with an android social robot, and consists of a number of rather diverse contributions to this field of research. These contributions include new results and methods in relation to the perception of robots both prior to and after HRI, the evaluation and assessment of robotic platforms and robot properties in relation to specific tasks, the importance of designing android robotic interfaces after actual humans, and the ethics of such a persuasive technology social robots are. The teleoperated android Geminoid-DK, whose appearance resembles a specific actual human, was used as a robotic platform for the conducted experiments (including user studies, field studies, laboratory studies, online surveys, and pre/post surveys). In the first part of this dissertation, an overview over the purpose and the goals of this study is given, accompanied by background knowledge of the related fields of knowledge, and a reflection on the theories and methods followed. During this first part, the contribution of the dissertation is summarized, and positioned in relation to the nine research papers that follow in the second part. The second part of this dissertation contains: (a) a classification of robots based on the dimensions of Intelligence (Autonomy-Control), and Perspective (Tool-Medium), (b) connections made between social robots and persuasive technology, and between robots and the user’s sense of place attachment, (c) an introduction to the Geminoid Reality including the advances in geminoid technology, (d) a methodology of mapping and evaluating genuine human facial expressions of emotion to androids, (e) a case of designing android faces after specific actual persons who portray facial features that are familiar to the users, and also relevant to the notion of the robotic task, in order to increase the chance of sustaining emotional interaction, (f) an open-ended evaluation method pertaining to the interpretation of Android facial expressions (g) a study on how users’ perception and attitude can change after direct interaction with a robot, (h) a study on how androids can maintain the focus of attention during short-term dyadic interactions, and (i) a state-of-the-art report on android hands. Throughout the dissertation, the main focus has been to understand the underlying problems that could cause disruption in human-robot communication, and to II ENGLISH SUMMARY provide meaningful insights on how to prevent them from happening. In conclusion, the eight main steps for sustaining emotional communication with an android robot are: (1) Prior to interactions, evaluate the properties of the robot, (2) Prior to interactions, assess the attitude of the users towards the robot, (3) Prior to interactions, know what tasks the robot can satisfy according to its appearance, morphology, and abilities, (4) Robotic speech, lips synchronization, facial expressions and movements need to be aligned, (5) The robot should avoid abrupt movements towards the user, (6) The robot needs to be as capable as it appears to be, (7) Robot appearance matters less if the situation is engaging, (8) The gender of the robot affects the interactions. Even though the presented eight recommendations need further investigation, and validation, which will happen soon as more android robots are produced and put into actual use in real life situations, I believe that by following them communication with androids will become more meaningful. III SUSTAINING EMOTIONAL COMMUNICATION WHEN INTERACTING WITH AN ANDROID ROBOT DANSK RESUME1 Jo mere menneskelignende en robot fremstår