Cognitive Development and The iCub Humanoid Robot
Second EUCogII Members Conference "Development of Cognition in Artificial Agents“
Zürich 29 January 2010 iCub
Giulio Sandini, IIT & U. Genoa Rolf Pfeifer, U. Zurich José Santos-Victor, IST Giorgio Metta, IIT & U. Genoa Harold Martinez, U. Zurich Alexandre Bernardino, IST Lorenzo Natale, IIT & U. Genoa Gabriel Gomez, U. Zurich Ricardo Beira, IST Francesco Nori, IIT & U. Genoa Alexandre Schmitz, U. Zurich Bruno Damas, ST Paul Fitzpatrick, IIT & U. Genoa Yvonne Gustain, U. Zurich Jonas Hornstein, , IST Francesco Orabona, IIT & U. Genoa Jonas Ruesch, U. Zurich Luís Vargas, IST Matteo Brunettini, IIT & U. Genoa Kerstin Dautenhahn, U. Hertfordshire Ricardo Nunes, IST Alessandro Scalzo, IIT Chrystopher L. Nehaniv, U. Hertfordshire Hugo Alves, IST Marco Maggiali, IIT Marco Randazzo IIT Hatice Kose-Bagci, U. Hertfordshire Nuno Conraria, IST Roberto Puddu, IIT Frank Broz, U. Hertfordshire Julio Gomes, IST Gabriele Tabbita, IIT Naeem Assif Mirza, U. Hertfordshire Matteo Tajana, IST Walter Fancellu, IIT Dorothée François, U. Hertfordshire Giovanni Saponnaro, IST Bruno Bonino, IIT Lars Olsson, U. Hertfordshire Christian Wressengger, IST Fabrizio Larosa, IIT Qiming Shen, U. Hertfordshire Dario Figueira, IST Claudio Lorini, IIT Cecilia Laschi, SSSA Rodrugo Ventura, IST Luciano Pittera, IIT Paolo Dario, SSSA Miguel Praça, IST Davide Dellepiane, IIT Fernando Gamarra, SSSA Jonas Ruesch, IST Mattia Salvi, IIT Davide Zambrano, SSSA Luís Montesano, IST Luca Rivano, IIT Egidio Falotico, SSSA Manuel Lopes, IST Ravinder Dahiya, IIT Alberto Parmiggiani, IIT Maria Chiara Carrozza, SSSA Luciano Fadiga, U. Ferrara Matteo Fumagalli, IIT Giovanni Stellin, SSSA Laila Craighero, U. Ferrara Alexander Schmitz, IIT Giovanni Cappiello, SSSA Andrey Olyniyck, U. Ferrara Diego Torazza, IIT Aude Billard, EPFL Livio Finos, U. Ferrara Nikos Tsagarakis, IIT & U. Sheffield Auke Ijspeert, EPFL Giovanni Ottoboni, U. Ferrara Darwin Caldwell, IIT & U. Sheffield Sarah Degallier, EPFL Claes von Hofsten, U. Uppsala Francesco Becchi, TeleRobot Ludovic Righetti, EPFL Kerstin Rosander, U. Uppsala Paolo Pino, TeleRobot S. Gay, EPFL Olga Kochukova, U. Uppsala Giulio Maggiolo, TeleRobot Helena Gronqvist, U. Uppsala Gabriele Careddu TeleRobot John Gray, U. Sheffield David Vernon, U. Genoa U. of Genoa/IIT Scuola S. Anna U. of Zurich U. of Uppsala U. of Ferrara Giulio Sandini Cecilia Laschi Rolf Pfeifer Claes von Hofsten Luciano Fadiga Giorgio Metta Paolo Dario
U. Hertfordshire IST - Lisbon U. Sheffield/IIT EPFL Telerobot S.r.l Kerstin Dautenhahn José Santos-Victor Darwin Caldwell Aude Billard Francesco Becchi Chrystopher L. Nehaniv Alexandre Bernardino Nikos Tsagarakis Auke Ijspeert Head Design (IST) Face/Cover Design (IST) Facial Expressions (IST) Smooth Pursuit (U. Uppsala, IST, SSSA)
Perception and anticipation of the upcoming motion
4‐week‐old
9‐week‐old Smooth Pursuit (U. Uppsala, IST)
“Uppsala drum” – where we can rotate the base and/or the target
Manuel Lopes, Alexandre Bernardino, José Santos- Victor, Claes von Hofsten and Kerstin Rosander. Biomimetic Eye-Neck Coordination. IEEE - International Conference on Development and Learning, Shanghai, China, 2009. Predictive tracking with temporarily-occluded objects (U. Uppsala, U. Zürich, SSSA)
4-month-old children move gaze ahead of time to the place where they expect the object to appear
9‐week‐old lag= 740 ms 17‐week‐old lag = 15 ms Smooth Pursuit “Infant” Gaze (U. Uppsala & IST) Attention
• Posner task (IST & U. Ferrara) Attention
• Infant Gaze (U. Uppsala) Attention
• Infant Gaze (U. Uppsala) Attention
Multi-Modal Bottom-Up Attention System (IST & UZH)
Salience Egocentric
flicker faces
skin
Multimodal Saliency-Based Bottom-Up Attention A Framework for the Humanoid Robot iCub, Jonas Ruesch, et al. ICRA 2008. Attention
•Top-down (IST) • Representation and detection familiar objects • Learning triggered by depth (proximity based) segmentation
From Pixels to Objects: Enabling a spatial model for humanoid social robots. Dario Figueira, et al. ICRA’09 Attention
(IST & UZH) Reaching and Grasping
(U. Genoa, IIT, IST) Reaching and Grasping
(U. Genoa, IIT, IST) Crawling
Kinematic studies of crawling children as they crawled, went from sitting to crawling, and crawling to sitting (U. Uppsala, EPFL)
Crawling to sitting Crawling
Kinematic studies of crawling children as they crawled, went from sitting to crawling, and crawling to sitting (U. Uppsala, EPFL)
Crawling with one hand occupied Crawling
Kinematic studies of crawling children as they crawled, went from sitting to crawling, and crawling to sitting (U. Uppsala, EPFL) (U. Uppsala, EPFL) Wiki Software
Hardware
Drawings iCub production
15 iCubs completed and 5 more in production
Mind as Motion, Port & Van Gelder
GOFAI Cognitive Systems
Functionalist & Dualist Emergent, Embodied, & Enactive
Mechanisms are independent Embodiment plays a constitutive of the instantiation role in the process of cognition
Breaking the ‘here-and-now barrier’
Bond of Union M. C. Escher, 1956 Breaking the ‘prior knowledge barrier’
z
Bond of Union M. C. Escher, 1956 The fire-hose of experience
z
Bond of Union M. C. Escher, 1956 Cognition: guide actions
– Missing information – Uncertain information – LATE information
• Adapt (make sense of the world) • Anticipate (predict what might happen) Cognitive systems
– Anticipate – Assimilate Learn & develop – Adapt
– Predict future events when selecting actions – Learn from what actually happens – Modify subsequent predictions – Autonomously • What makes an action the right one to choose?
• What type of behaviour does cognition enable?
• What motivates cognition?
• How is perception guided?
• How are actions selected?
• What makes cognition possible?
• Cognitive skills can improve, but what do you need to get started?
• What drives the developmental process? Embodiment
Meaning (inter-agent epistemology)
Development Meaning emerges through shared consensual experience mediated by interaction
Bond of Union M. C. Escher, 1956
Bond of Union M. C. Escher, 1956 Types of Embodiment Why Humanoid Robotics is Special
Historical Embodiment
Organismic Embodiment
Organismoid Embodiment
Shared epistemology ⇒ Physical Embodiment compatible embodiment
Structural Coupling
From: T. Ziemke, 2003 The problem of disparate embodiment & interaction histories The problem of disparate embodiment & interaction histories Development
Progressive ontogenetic acquisition of anticipatory capabilities
– Cognition cannot short-circuit ontogeny
– Necessarily the product of a process of embodied development
– Initially dealing with immediate events t
– Increasingly acquiring a predictive capability t
Cognition and perception are functionally-dependent on the richness of the action interface Luc Steels:
1. AI through design 2. AI through statistical machine learning 3. Self-generated AI (AI by orchestrating the processes that generate it) Self-Generated AI … HOW?
Phylogeny (Cognitive Architecture)
Ontogenesis (Learning & Development + Motivations) Co-Determination / Structural Coupling
BUT … simple coupling between sensor and motor surfaces
Perturbation is only effected by the environment
[Note: this ideogram and similar ones to follow were introduced in Maturana and Varela 1987] Cognitive system: operationally-closed system with a nervous system
Nervous system facilitates a highly-plastic mapping between sensor and motor surfaces Perturbation by both environment and system (of receptors & NS) “Development is the result of a process with two foci, one in the central nervous system and one in the subject’s dynamic interactions with the environment” Claes von Hofsten t
t Anticipation / Planning / Deliberation / Prediction INTERACTION
A shared activity in which the actions of each agent Influence the actions of the other agents in the same interaction Resulting in a mutually-constructed pattern of shared behaviour [Ogden et al.] COGNITION & SENSE-MAKING
Is a process whereby the issues that are important for the continued existence of the cognitive entity are brought forth: co-determined by the entity as it interacts with the environment in which it is embedded PERCEPTION, ACTION, and COGNITION form a single process of self-organization in the specific context of environmental perturbations of the system THE SPACE OF PERCEPTUAL POSSIBILITIES
Is predicated not on an objective environment, but on the space of possible actions that the system can engage in whilst still maintaining the consistency of its coupling with the environment Emergent Systems
Self- Organization Embodiment
Co- Co- determination development
Phylogeny Ontogeny Humanoid Emergent Systems
Self- Organization Embodiment
Co- Co- determination development
Phylogeny Ontogeny Humanoid
Morphology is a constitutive component of both co-determination and co-development Consequently, a plastic morphology is important: the embodiment shouldn’t be static Emergent Systems
Self- Organization Embodiment
Co- Co- determination development
Phylogeny Ontogeny Humanoid
Cognitive Increasing complexity Development in action space
Increasing degree of Prospection
Each with own Network of limited encodings cooperating/competing circuits
Learning: to tune phylogenetic skills Self-modification Model Generation Development: change system dynamics Cognitive Architecture new action spaces The iCub Phylogeny
Exploratory motives Novel regularities in the world
Potential of own actions
agent interaction: Social motives mutually-constructed patterns of behaviour
Unsupervised Learning Supervised Reinforcement Mechanism to rehearse Moderate hypothetical scenarios actual behaviour
Action-Perception Embodiment Couplings Level 2 APIs: Prospective Action Behaviours Coordinated operation: Ontogenic Development Cognitive Architecture Level 1 APIs: perception/action behaviours Based on Innate perception/action primitives own phylogenic learning loose federation of behaviours configuration model Level 0 APIs: data acquisition & motor control Software Multiple YARP processes Architecture Running on multiple processors
Gbit ethernet
HUB iCub Embedded DSP DSP DSP DSP Systems
Sensors & Actuators simulated sensory signals
Motor/Sensory Sensory/Motor hetero-associative hetero-associative Prospection by memory memory action simulation
Perturbation simulated motor signals
Motivation (Amygdala) Action Selection Modulation circuit: (Basal Ganglia) Auto-associative homeostatic action selection Memory by disinhibition of (Hippocampus) perceptuo-motor skills
Phylogenetic self-organizing perceptuo-motor skills
The RobotCub Cognitive Architecture for the iCub Key Issue: Development
Learning: to tune phylogenetic skills Self-modification Development: change system dynamics new action spaces
– Improve the predictive performance
– Increase the space of viable actions Key Issue: Internal Simulation
Three purposes of internal simulation: Mechanism to rehearse Moderate hypothetical scenarios 1. Prediction: future events actual behaviour
2. Reconstruction: explaining observed events (imagining a causal chain leading to that event)
3. Imagination: internal simulation as a way of imagining new ideas Key Issue: Exogenous and Endogenous
Exploratory motives Exogenous: curiosity Novel regularities in the world Potential of own actions Sensori-motor learning
Endogenous: experimentation
Experience-based generative development & operation Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention Control iCub Selection Gaze Interface
Reach & Grasp Egosphere
A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention Control iCub Selection Gaze Interface
Reach & GraspGaze Control Egosphere (IST, SSSA, U. Uppsala) A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion Reach Attention Control iCub Selection & Grasp Gaze Interface (U. Genoa, IIT, IST) Reach & Grasp Egosphere
A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention CrawlingControl iCub Selection Gaze Interface (EPFL)
Reach & Grasp Egosphere
A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data Attention Sub-system (IST)
Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention Control iCub Selection Gaze Interface
Reach & Grasp Egosphere
A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention Control iCub Selection Gaze Interface
Reach & Grasp Egosphere Bhattacharyya distance A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention Control iCub Selection Gaze Interface
Reach & Grasp Egosphere
A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Alternative work on episodic memory at Aural, visual, and proprioceptive sensory data the University of Hertfordshire P A
Procedural Affective Action Memory State Selection P A P A P
Episodic Memory Loco- motion P Attention Control iCub Selection Gaze Interface
Reach Prediction & Reconstruction& Grasp by presenting Egosphere (Pi, ~, ~) or (~, ~, Pk) A Priori Feature Values Also possible Exogenous Endogenous Salience Salience (Pi, Aj,Vergence ~), (~, Aj, Pk), or (Pi, ~, Pk)
Aural, visual, and proprioceptive sensory data Learning affordances at IST
Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention Control iCub Selection Gaze Interface
Reach & Grasp Egosphere
A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Interaction Histories at the Aural, visual, and proprioceptive sensory dataUniversity of Hertfordshire Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention Control iCub Selection Gaze Interface
Reach & Grasp Egosphere
A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data SIFT-based object salience at IST
Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention Control iCub Selection Gaze Interface
Reach & Grasp Egosphere
A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data Procedural Affective Action Memory State Selection
CuriosityEpisodic & Memory Loco- Experimentation motion
Attention Control iCub Selection Gaze Interface x Level of Curiosity ∑ & Experimentation Reach & Grasp C, E Egosphere New Event | Expired Event ⇒ Curiosity Spike A Priori Feature PredictedValues Event ≡ Recalled Event ⇒ Experimentation Spike
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data Procedural Affective Action Memory State Selection
Episodic Memory Loco- motion
Attention Control iCub Selection Gaze Interface
Reach & Grasp argmaxEgosphere (Curiosity Level, Experimentation Level)
A Priori Feature Values
Exogenous Endogenous Salience Salience Vergence
Aural, visual, and proprioceptive sensory data /episodicMemory/action:i
/i:map /remoteEgoSphere
/o:position /pos /remoteEgoSphere attentionSelection controlGaze2
/i:gaze /o:velocity /vel /o:status
/mapVisual/rgb_in
/icub/cam/right /right/out /right/view (exogenous) /right/map /mapVisual/map_in cameraCalib egoSphere salience /map_out /conf
/mapObject/bottle_in
/cartesianImage:i /salience:o
/logpolarImage:i Endogenous From /episodicMemory/image:o Salience /head:i /segmentedImage:o From /icub/head/state:o To viewer /weights:o To viewer /mode:i proceduralMemory /action:o /pos controlGaze2
/imageId:i /imageId:o
/imageId:i /imageId:o
/icub/cam/right /image:i /recalledImage:o To viewer logPolarTransform episodicMemory logPolarTransform /head:i
/action:i /retrievedImage:o
TO /endogenousSalience/logpolarImage:i From /icub/head/state:o
From /attentionSelection/o:position
/imageId:i /state:o /state:i /mode:o affectiveState actionSelection /mode:i A thesis for discussion:
“Cognition emerges during development in a close interplay of experience, of the social and physical environment and of the neuronal mechanisms of growth.
An understanding of cognition cannot be achieved without an understanding of the development of cognition.
It is thus an necessity for artificial cognitive systems to take development on board”
Key Research Topics for the Future
Imagination
Self-Modification
Plasticity
Redundancy
Internal Dynamics
Social motives
The value system: creating order Thank You! Giulio Sandini, IIT & U. Genoa Rolf Pfeifer, U. Zurich José Santos-Victor, IST Giorgio Metta, IIT & U. Genoa Harold Martinez, U. Zurich Alexandre Bernardino, IST Lorenzo Natale, IIT & U. Genoa Gabriel Gomez, U. Zurich Ricardo Beira, IST Francesco Nori, IIT & U. Genoa Alexandre Schmitz, U. Zurich Bruno Damas, ST Paul Fitzpatrick, IIT & U. Genoa Yvonne Gustain, U. Zurich Jonas Hornstein, , IST Francesco Orabona, IIT & U. Genoa Jonas Ruesch, U. Zurich Luís Vargas, IST Matteo Brunettini, IIT & U. Genoa Kerstin Dautenhahn, U. Hertfordshire Ricardo Nunes, IST Alessandro Scalzo, IIT Chrystopher L. Nehaniv, U. Hertfordshire Hugo Alves, IST Marco Maggiali, IIT Marco Randazzo IIT Hatice Kose-Bagci, U. Hertfordshire Nuno Conraria, IST Roberto Puddu, IIT Frank Broz, U. Hertfordshire Julio Gomes, IST Gabriele Tabbita, IIT Naeem Assif Mirza, U. Hertfordshire Matteo Tajana, IST Walter Fancellu, IIT Dorothée François, U. Hertfordshire Giovanni Saponnaro, IST Bruno Bonino, IIT Lars Olsson, U. Hertfordshire Christian Wressengger, IST Fabrizio Larosa, IIT Qiming Shen, U. Hertfordshire Dario Figueira, IST Claudio Lorini, IIT Cecilia Laschi, SSSA Rodrugo Ventura, IST Luciano Pittera, IIT Paolo Dario, SSSA Miguel Praça, IST Davide Dellepiane, IIT Fernando Gamarra, SSSA Jonas Ruesch, IST Mattia Salvi, IIT Davide Zambrano, SSSA Luís Montesano, IST Luca Rivano, IIT Egidio Falotico, SSSA Manuel Lopes, IST Ravinder Dahiya, IIT Alberto Parmiggiani, IIT Maria Chiara Carrozza, SSSA Luciano Fadiga, U. Ferrara Matteo Fumagalli, IIT Giovanni Stellin, SSSA Laila Craighero, U. Ferrara Alexander Schmitz, IIT Giovanni Cappiello, SSSA Andrey Olyniyck, U. Ferrara Diego Torazza, IIT Aude Billard, EPFL Livio Finos, U. Ferrara Nikos Tsagarakis, IIT & U. Sheffield Auke Ijspeert, EPFL Giovanni Ottoboni, U. Ferrara Darwin Caldwell, IIT & U. Sheffield Sarah Degallier, EPFL Claes von Hofsten, U. Uppsala Francesco Becchi, TeleRobot Ludovic Righetti, EPFL Kerstin Rosander, U. Uppsala Paolo Pino, TeleRobot S. Gay, EPFL Olga Kochukova, U. Uppsala Giulio Maggiolo, TeleRobot Helena Gronqvist, U. Uppsala Gabriele Careddu TeleRobot John Gray, U. Sheffield David Vernon, U. Genoa