University of Plymouth PEARL https://pearl.plymouth.ac.uk 04 University of Plymouth Research Theses 01 Research Theses Main Collection 2009 Development of Cognitive Capabilities in Humanoid Robots Tikhanoff, Vadim http://hdl.handle.net/10026.1/2807 University of Plymouth All content in PEARL is protected by copyright law. Author manuscripts are made available in accordance with publisher policies. Please cite only the published version using the details provided on the item record or document. In the absence of an open licence (e.g. Creative Commons), permissions for further reuse of content should be sought from the publisher or author. DEVELOPMENT OF COGNITIVE CAPABILITIES IN HUMANOID ROBOTS by VADIM TIKHANOFF A thesis submitted to the University of Plymouth in partial fulfilment for the degree of DOCTOR OF PHILOSOPHY School of Computing Communications and Electronics Faculty of Technology August 2009 l'.,iverslty of Plymouth L.tr~.J n· i.:J.COJ35323=fX 1 -n::fcs·Y { ~ - 8qz. ITK.I Development of Cognitive Capabilities in Humanoid Robots by Vadim Tikhanoff Abstract Bu ilding intelligent systems with human level of competence is the ultimate grand challenge for science and technology in general, and especially for the computational intelligence community. Recent theories in autonomous cognitive systems have focused on the close integration (grounding) of communication with perception, categorisation and action. Cognitive systems are essential for integrated multi-platform systems that are capable of sensing and communicating. This thesis presents a cognitive system for a humanoid robot that integrates abilities such as object detection and recognition , which are merged with natural language understanding and refined motor controls. The work includes three studies; (1) the use of generic manipulation of objects using the NMFT algorithm, by successfully testing the extension of the NMFT to control robot behaviour; (2) a study of the development of a robotic simulator; (3) robotic simulation experiments showing that a humanoid robot is able to acquire complex behavioural, cognitive, and linguistic skills through individual and social learning. The robot is able to learn to handle and manipulate objects autonomously, to cooperate with human users, and to adapt its abilities to changes in internal and environmental conditions. The model and the experimental results reported in this thesis, emphasise the importance of embodied cognition, i.e . the humanoid robot's physical interaction between its body and the environment. Table of Contents Abstract .... .... ... .. .... ..................... ................ ... ... ..... .. .. ........ ........................... ... .. .... ... 4 Table of Contents .. .............................. .. .... .. .............. .. .. ..... ........... ... ......... ... .... ... ..... 5 List of Figures .. ... .. .. .. .. ... ............. ............. ............................ ................... .. ........... .... 7 List of T abies .. .. ..... .... .... .. .......... ... ......... ... ... ...... ......... ....... ................... .. .. .............. 10 Acknowledgements ... ... ..... ......... .... ...................... ..................................... ... ........ .. 12 Author's declaration ...... ..... .. .. ... .... .. ......... ...... ................. .. ............ .................. ....... 14 1 Introduction ..................................................................................................... 1 1 .1 Motivation ........... ........ .. ........... ..... .... .................. ... ... .... .. .... ... ..................... 1 1.2 Scope .. .... ........... ....... .. .... ..... .. ...... .... ........ .. .. ........ ............ ......... .......... ........3 1.3 Thesis Outline ............ ... ............. .......... .... .. ..... ... ........... .... .. .................... .... 3 2 Cognitive Robotics .................................... ...................................................... s 2.1 Cognitive Robotics Approaches ... .. .. .. ........ ... ....... .............................. ....... 10 2.1 .1 Developmental Robotics .... .............. ................. .... ......... .... .......... ... ... 10 2.1. 2 Epigenetic Robotics ..................... ................................... ..... ............... 12 2.1. 3 Evolutionary Robotics ......... ... .. ............. ............ ..... ............................. 13 2.2 Human robot interaction ....... .... ..... ...... ..... .......................... ... ... ... .... .... ... ... 15 2.3 Developmental robotic models of language learning ............. ... .......... .... .. 21 2.4 Robotics and simulators ....... ..... .. ........ .. ......................... ...... .. .. ............... .. 27 2.5 The iCub humanoid robotic platform .. ..... .... .... ............. ....... .................... ..33 2.5.1 Why a humanoid robot in cognitive robotic research .................... ...... 34 2.5.2 The iCub Robot ....... ...... ... .... ....... ...... ..... ........ ............... .... .......... ... .... 36 2.5.3 Hardware specifications ........ ........ .. .. ........ ... .... ............ .... .... ........ ...... 39 2.5.4 Software Architecture: YARP ... ......................... .... ............................ .42 3 Artificial Intelligence for cognitive robotics ................................................ 47 3.1 Artificial Intelligence ......... .... ............ ........... ........ .. ................................. .. .48 3.2 Neural Networks .......... .. ................................. ... ..... .. ..... ..... ... .. ................. 55 4 A robotic simulation model of action and language learning: Neural Modelling Field Theory ........................................................................................ 69 4.1 The robotic model ....... .. ....... ....... ....... .... .. .... .. .... ........ .......... ... ... ..... .. ........ 71 4.2 Overview of the NMFT Algorithm ... .. ... .. ...... .. ...... ..... .. .. ... ................... .... ... 74 4.3 The NMFT Model for Robotic Experiments ..... ......... .... .................... ... .. .... 76 4.4 NMFT Simulations ..... .............................. ... .......... ....... ... .. ..... .......... .. ... .... 78 4.5 Language and Cognition Integration through Modelling Field Theory ....... 79 4.5.1 Simulation 1: Incremental addition of feature ..................... ................. 80 4.5.2 Simulation 11 : Categorisation of robotic actions ....... .... ... ... .... ............. 82 4.5.3 Simulation Ill: Scaling up with complex stimuli sets ............................ 84 4.6 Scaling up of Action Repertoire in Linguistic Cognitive Agents ... .. ..... ... .... 85 4.6.1 Simulation IV: Classification and categorization of actions for building sensorimotor concept-models ......... .............. ................ .. ... ................. ......... ... 85 4.6.2 Simulation V: Incremental Feature - lexi con acquisition .................... 88 4.6.3 Simulation VI: Progressive learning of basic gestures into composite actions 91 5 Cognitive Humanoid Robot: the iCub platform .......................................... 95 5.1 The birth of the iCub Simulator ............................. ............................... ..... 96 5.1 .1 iCub Simulator Development.. ........................ ..... .... ..... ..... ... .............. 96 5.1.2 iCub Simulator Communication protocols ................................. .. ....... 99 5.1.3 iCub Body Model ................................................. .................. ........... 1 01 5.1.4 iCub Simulator testing: Kinematic and dynamic analysis ................. 109 5.1 .5 Current uses of the iCub simulator in cognitive robotics projects ..... 114 6 Cognitive Experiments ................................................................................118 6.1 Experiment overview ............................. ... .......... ... ... .. ............ .. ...... ... .. .... 118 6.2 Vision .. .. ...... ..... ... ... .... .. ... .......... ... ............. ... ...... .. .. .... ... .......................... 121 6.2. 1 Depth discontinu ities .......... ................... ... ... .... .... .. .... ......... .. .. ... ..... .. 124 6.2.2 Colour lnformation ............. ... ................................. ... .......... .... ........ .. 131 6.2.3 Template tracking ................................................... ......................... 132 6.3 Motor Control .......................................................................................... 135 6. 3. 1 Reaching .......... .... ........ .. ........................ .......................................... 137 6.3.2 Grasping ... ............ ... ......................... ... .... ....... .. ................................ 148 6.4 Worki ng with Speech .. ... ................................ .. .... .... ..... ..... ..................... 157 6.4.1 Speech categorisation learning experimental resu lts .. ....... .............. 162 6.5 En active cognition: experiments ... ... ....................... ................................ 167 6.5.1 Experiments on grounding speech and vision .. .... .. ......... ................. 168 6.5.2 En active cognition experiment .................... ........................... .. ........ 175 7 Discussion and conclusions .. ... .... ........................ ... .................. .... .... ............ 185 7.1 Discussion .... .. ....... ........ ................ ............... .................... ....... .... .. ........