Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16) Affective Personalization of a Social Robot Tutor for Children’s Second Language Skills Goren Gordon,a,b,c Samuel Spaulding,a,c Jacqueline Kory Westlund,a,c Jin Joo Lee,a,c Luke Plummer,a Marayna Martinez,a Madhurima Das,a Cynthia Breazeala,c a Personal Robots Group, MIT Media Lab, 20 Ames Street E15-468, Cambridge, MA 02139 b Curiosity Lab, Industrial Engineering Department, Tel-Aviv Univerisity, 6997801, Israel c{ggordon,samuelsp,jakory,jinjoo,cynthiab}@media.mit.edu Abstract sonalize the curriculum to each student (e.g., the order and type of questions asked or the content presented), fewer have Though substantial research has been dedicated towards us- addressed the equally critical aspect of personalizing the tu- ing technology to improve education, no current methods are as effective as one-on-one tutoring. A critical, though rela- toring interaction to the affective state of the student. If a tively understudied, aspect of effective tutoring is modulating student is discouraged by the material and disengages from the student’s affective state throughout the tutoring session in the tutoring system, personalization of the educational con- order to maximize long-term learning gains. We developed tent may be ineffective - the student is not attending to it an integrated experimental paradigm in which children play all. Recent innovations in affective sensing technology, such a second-language learning game on a tablet, in collabora- as McDuff, Kaliouby, and Picard (2012), have allowed re- tion with a fully autonomous social robotic learning compan- searchers to begin using students’ affective responses as in- ion. As part of the system, we measured children’s valence put to intelligent tutoring systems, such that the system can and engagement via an automatic facial expression analysis respond to students’ affective states Woolf et al. (2009); Ar- system. These signals were combined into a reward signal royo et al.; Vanlehn et al.. However, so far, these systems that fed into the robot’s affective reinforcement learning al- gorithm. Over several sessions, the robot played the game generally incorporate simple rule-based extensions to deal and personalized its motivational strategies (using verbal and with students’ affective states, and are based on virtual tu- non-verbal actions) to each student. We evaluated this sys- tors, not physically embodied robots Nye, Graesser, and Hu tem with 34 children in preschool classrooms for a duration (2014). Additional prior work has shown that a physically of two months. We saw that (1) children learned new words embodied tutor may be more effective than a virtual tutor from the repeated tutoring sessions, (2) the affective policy Leyzberg, Spaulding, and Scassellati (2014). personalized to students over the duration of the study, and In this work, we present an integrated affective tutoring (3) students who interacted with a robot that personalized its system that uses an integrated child-tablet-robot setup Gor- affective feedback strategy showed a significant increase in don and Breazeal; Jeong et al. (2014); Kory and Breazeal. valence, as compared to students who interacted with a non- personalizing robot. This integrated system of tablet-based The supportive affective behavior of a robotic tutor is au- educational content, affective sensing, affective policy learn- tonomously learned and personalized to each student over ing, and an autonomous social robot holds great promise for multiple interactive tutoring sessions. The system is com- a more comprehensive approach to personalized tutoring. posed of four primary components: 1. a novel, fully autonomous social robot platform (called Introduction Tega), which was specifically designed to be engaging for Socially assistive robotics (SAR) is an emerging field which children, is robust enough to work continuously for sev- strives to create socially interactive robots that aid people in eral hours, and is portable in order to be deployed in the different areas of their lives, such as education and care for field; the elderly Tapus, Maja, and Scassellatti (2007); Williams 2. a novel, educational Android tablet app that allows for (2012); Fasola and Mataric (2013). Educational assistive general curriculum generation and seamless integration robots are designed to support children’s learning and devel- with the social robot; opment, e.g., in the classroom Movellan et al. (2009); Chang et al. (2010) or in one-on-one tutoring sessions Saerbeck et 3. an Android smartphone that uses the commercial Affdex al. (2010); Kory, Jeong, and Breazeal; Fridin (2014). How- SDK to automatically analyze facial expressions in real- ever, children may learn in different ways and at different time McDuff, Kaliouby, and Picard (2012); and paces. In order to teach children most effectively, one must 4. a cognitive architecture that integrates and feeds affec- personalize the educational interaction to each child Van- tive information from Affdex and educational information Lehn (2011). While many intelligent tutoring systems per- from the tablet into an affective reinforcement learning al- Copyright c 2016, Association for the Advancement of Artificial gorithm, which determines the social robot’s verbal and Intelligence (www.aaai.org). All rights reserved. non-verbal behavior. 3951 The integration of all these components was enabled by us- problems Arroyo et al.. In order to foster engagement, the ing the Robot Operating System (ROS) throughout the setup tutor uses an empathy-based affective behavior system: the Quigley et al.. emotional actions of the tutor are intended to mirror the (es- We evaluated this system in a real world experimental timated) emotional state of the user. For example, if a child paradigm. Native English-speaking preschool children (ages appears bored, the tutor might also display signs of boredom 3-5) interacted with the system to learn second language before suggesting a new topic or problem to keep the student vocabulary (Spanish) in their own classroom over a two- engaged. month period. We first show that the tutoring setup facil- Recent efforts to develop affect-aware tutoring systems itates these children’s learning of new Spanish words. We have culminated in a number of major systems that have then analyze the performance of the affective reinforcement been extensively studied, including the Wayang Tutor Ar- learning algorithm to show that it personalized to specific royo et al. and Affective Meta-Tutor Vanlehn et al. projects) children - i.e., the algorithm adapted in different ways for which have been extensively studied. Yet much of the work each child. An analysis of the effects of the robot’s behav- on affect and modeling in the ITS literature focuses on mod- ior on children’s detected valence and engagement shows els to infer affect. Typically, once affective states are de- that only children’s positive valence is robustly and signif- tected or identified, they trigger simple behavioral rules - a icantly changed immediately following a set of non-verbal tutor might change its facial expression or offer a supportive actions by the robot. Consequently, we compared the va- comment. However, these rules are hardcoded by the devel- lence of children who interacted with either a personalized opers and remain fixed throughout the deployment. or a non-personalized robot tutor. We found that the change On the other hand, the research and development of so- in valence between the first and last sessions over the two- cial robot tutors has recently been flourishing Movellan et al. month period was significantly different between the two (2009); Leyzberg, Spaulding, and Scassellati (2014); Desh- conditions. That is, in the non-personalized condition, posi- mukh et al.; Kanda et al. (2004). For example, RUBI-4 is tive valence decreased, while in the personalized condition, a humanoid robot with articulated arms, an expressive face, positive valence increased. These results, obtained using an and a tablet embedded in its midsection that played simple integrated system that combines educational content, affec- vocabulary games with preschool children Movellan et al. tive sensing, and an expressive social robot deployed in a (2009). Personalization of robot tutors, even via simple algo- real-world, long-term interaction study, shows that affec- rithms, has been shown to greatly increase tutoring effective- tive personalization of social robotic tutors can positively ness compared to non-personalized robot tutors as well com- influence the students’ affect in constructive and meaningful pared to virtual robots Leyzberg, Spaulding, and Scassel- ways. lati (2014). While more sophisticated learning approaches to personalized robot tutors have been studied Gordon and Related Work Breazeal, understanding how to effectively personalize tutor Intelligent Tutoring Systems (ITSs) refer to a wide variety of behavior to support positive student affective is still an open computer-based educational tools. Common features of an problem. ITS include the ability to change its behavior in response to student input, provide help in the form of a hint or additional Interaction Design instruction, and conduct some form of evaluation of the user. VanLehn VanLehn (2011) distinguishes between two broad We created a complete interaction scenario for evaluating classes of computer-based tutors. First, ‘Computer-Aided our affective social robotic tutor. The educational goal was Instruction’