ORIGINAL RESEARCH published: 04 October 2017 doi: 10.3389/fnbot.2017.00053 A Developmental Learning Approach of Mobile Manipulator via Playing Ruiqi Wu 1, Changle Zhou 1, Fei Chao 1*, Zuyuan Zhu 2, Chih-Min Lin 1, 3 and Longzhi Yang 4 1 Fujian Provincal Key Lab of Brain-Inspired Computing, Department of Cognitive Science, School of Informatics, Xiamen University, Xiamen, China, 2 Department of Computer Science, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom, 3 Department of Electrical Engineering, Yuan Ze University, Tao-Yuan, Taiwan, 4 Department of Computer and Information Sciences, Faculty of Engineering and Environment, Northumbria University, Newcastle upon Tyne, United Kingdom Inspired by infant development theories, a robotic developmental model combined with game elements is proposed in this paper. This model does not require the definition of specific developmental goals for the robot, but the developmental goals are implied in the goals of a series of game tasks. The games are characterized into a sequence of game modes based on the complexity of the game tasks from simple to complex, and the task complexity is determined by the applications of developmental constraints. Given a current mode, the robot switches to play in a more complicated game mode when it cannot find any new salient stimuli in the current mode. By doing so, the robot gradually achieves it developmental goals by playing different modes of games. In the experiment, the game was instantiated into a mobile robot with the playing task of picking up toys, and the game is designed with a simple game mode and a complex game mode. A developmental algorithm, “Lift-Constraint, Act and Saturate,” is employed to drive the Edited by: mobile robot move from the simple mode to the complex one. The experimental results Patricia Shaw, Aberystwyth University, show that the mobile manipulator is able to successfully learn the mobile grasping ability United Kingdom after playing simple and complex games, which is promising in developing robotic abilities Reviewed by: to solve complex tasks using games. Eiji Uchibe, Advanced Telecommunications Keywords: developmental robotics, mobile manipulator, robotic hand-eye coordination, neural network control, Research Institute International, Japan sensory-motor coordination Daniele Caligiore, Consiglio Nazionale Delle Ricerche (CNR), Italy 1. INTRODUCTION *Correspondence: Fei Chao Intelligent robots have been widely applied to support or even replace the work of humans in many [email protected] social activities, such as assembly lines, family services, and social entertainment. These robots are made intelligent by many methods proposed in the literature, with the most common ones Received: 05 June 2017 being mathematical modeling and dynamics models, such as Yan et al. (2013), Galbraith et al. Accepted: 19 September 2017 (2015) and Grinke et al. (2015). These methods utilize predefined cognitive architectures in the Published: 04 October 2017 intelligent systems, which cannot be used for significant changes during the interaction within the Citation: environment. If the intelligent system is applied in a new environment, the intelligent systems must Wu R, Zhou C, Chao F, Zhu Z, be reconstructed. Also the complexity of the model increases exponentially as the complexity of the Lin C-M and Yang L (2017) A Developmental Learning Approach of task increases. In addition, it is still very challenging in the field of robotics to allow the robot to Mobile Manipulator via Playing. learn complex skills and incorporate a variety of skills in an intelligent system. Front. Neurorobot. 11:53. Asada et al. (2001), Lungarella et al. (2003), and Weng (2004) attempt to let the robot learn doi: 10.3389/fnbot.2017.00053 intricate skills using the so-called developmental robotics approaches. These approaches enable Frontiers in Neurorobotics | www.frontiersin.org 1 October 2017 | Volume 11 | Article 53 Wu et al. Developmental Learning of Mobile Manipulator via Playing robots to gradually develop multiple basic skills and thus learn developmental strategy of robots. Section 4 describes the to handle complex tasks (Berthouze and Lungarella, 2004; Jiang experimentation and analyzes the results. Section 5 concludes the et al., 2014). In other words, the learning target of a complex paper and points out possible future work. set of skills are divided into the learning of a number of stage targets (Wang et al., 2013; Zhu et al., 2015), and the robot achieves the ultimate learning goal by completing a series of 2. DEVELOPMENTAL ROBOTICS sub learning goals. This method reduces the difficulty for the robot to learn new skills (Shaw et al., 2014), and gives the robot As a research method with an interdisciplinary background the ability to accumulate learning, where the basic skills learned of developmental psychology, neuroscience, computer science, during the development process are reserved so as to arrive at etc. (Earland et al., 2014; Law et al., 2014a; Gogate, 2016), the final skill (Lee et al., 2013). When a robot uses the method developmental robotics aims to provide solutions in the of developmental robotics to learn new skills, the target in every design of behavior and cognition in the artificial intelligence developmental phase must be clearly defined (Stoytchev, 2009). systems (Marocco et al., 2010; Baillie, 2016; Salgado et al., However, this is practically very challenge for those phases with a 2016). Developmental robotics is inspired by the developmental large number of complex tasks, thereby limiting the applicability principles and mechanisms observed during the development of of developmental robotics. infants and children (Chao et al., 2014a), and thus the main idea It has been observed by infant development researchers that of developmental robotics is to let a robot imitate a human’s infants and young children, when developing skills, do not need development process (Adolph and Joh, 2007; Oudeyer, 2017). to define specific developmental goals (Adolph and Joh, 2007), The robot achieves sensory-movement and cognitive ability of and mergence is the primary form for infants to acquire skills incremental acquisition according to the inherent development (Morse and Cangelosi, 2017). In particular, a play phenomenon principles and through real-time interaction with the external often accompanies the process of an infant skill development environment (Cangelosi et al., 2015). Developmental robotics (Cangelosi et al., 2015), which has led to one infant development focuses on two primary challenges in the field of robotics: (1) theory that infants develop relevant skills during play. The play learning new knowledge and skills from a constantly changing of the early infant is driven primarily by intrinsic motivation environment; and (2) understanding their relationship with their (Oudeyer et al., 2007; Baldassarre and Mirolli, 2013; Caligiore physical environment and other agents. et al., 2015), and an infant’s development goal is implied in the Guerin et al. (2013) suggested in developmental robotics that game that the infant plays. This theory has not been applied most patterns need to be learned from a few patterns and the and verified in developmental robotics. Therefore, a robotic described knowledge must be developed gradually, by alluding to developmental model that combines the infant developmental the general mechanism of sensory-movement development and theory and developmental robotics is proposed herein. In this the knowledge description in action-object relationships. Law model, the learning skills of a robot are artificially viewed as game et al. (2014a) achieved stage development on an iCub robot. playing by an infant, and the developmental target is implied They successfully built a development model for infants from in the game goals. Then, a method of developmental robotics is birth to 6 months, which is driven by a new control system. ustilised by the model to accomplish the robot’s skill development Starting from uncontrolled movements and passing through and learning. The proposed system not only reduces the difficulty several obvious stages of behavior, the iCub robot, like an infant, of robot learning and allows accumulate learning, but also finally reaches out and simply manipulates the object. Cangelosi mitigates the limitation of applicability as discussed above by et al. (2015) used a method of action-centering to perform a clearly defining goals in the developmental method. large number of synchronous comparisons with similar human In contrast to other developmental learning methods (Yang development and artificial systems. They discovered that human and Asada, 1996; Berthouze and Lungarella, 2004), the proposed development and artificial developmental systems share some approach embeds the role of play in early infant development into common practices from which they can learn. These studies the developmental learning approach. Through two game modes, inspired the establishment of the proposed robotic systems our robot developed mobile reaching and grasping abilities with reported in this paper using the key features and important no external reward existing in the two game modes. The robot theories in human infant development. merely uses its learning status to switch from one game mode into One of the two most important research focuses in the field of next one. Such approach also adopts the intrinsic motivation- developmental robotic is the development of skills corresponding