Autotelic Principle: the Role of Intrinsic Motivation in the Emergence And
Total Page:16
File Type:pdf, Size:1020Kb
Autotelic Principle : the role of intrinsic motivation in the emergence and development of artificial language Miquel Cornudella Gaya To cite this version: Miquel Cornudella Gaya. Autotelic Principle : the role of intrinsic motivation in the emergence and development of artificial language. Artificial Intelligence [cs.AI]. Université Paris sciences et lettres, 2017. English. NNT : 2017PSLEE082. tel-01765234v2 HAL Id: tel-01765234 https://tel.archives-ouvertes.fr/tel-01765234v2 Submitted on 7 Mar 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THÈSE DE DOCTORAT de l’Université de recherche Paris Sciences et Lettres PSL Research University Préparée à l’École Normale Supérieure Autotelic Principle: the role of intrinsic motivation in the emergence and development of artificial language Autotelic Principle : le rôle de la motivation dans l’apparition et le développement d’un langage artificiel École doctorale no540 ECOLE DOCTORALE TRANSDISCIPLINAIRE LETTRES/SCIENCES Spécialité SCIENCES DU LANGAGE COMPOSITION DU JURY : M. OUDEYER Pierre-Yves DR INRIA, Président du Jury M. DOMINEY Peter Ford DR CNRS, Rapporteur M. DE BOER Bart Professeur, Vrije Universiteit Brussel, Rapporteur Soutenue par Mme. BOUVERET Myriam Miquel Cornudella Gaya Maître de conférences HDR le 19 Décembre 2017 Université de Rouen, Examinatrice M. VAN TRIJP Remi Chercheur, Sony Computer Science Dirigée par Thierry Poibeau Laboratories Paris, Encadrant M. POIBEAU Thierry DR LATTICE-CNRS, Directeur Abstract This thesis studies the role of intrinsic motivation in the emergence and develop- ment of communicative systems in populations of artificial agents. To be more specific, our goal is to explore how populations of agents can use a particular mo- tivation system called autotelic principle to regulate their language development and the resulting dynamics at the population level. To achieve this, we first propose a concrete implementation of the autotelic principle. The core of this system is based on the balance between challenges, tasks to be done to achieve a goal, and skills, the abilities the system can employ to accomplish the different tasks. The relation between the two elements is not steady but regularly becomes destabilised when new skills are learned, which al- lows the system to attempt challenges of increasing complexity. Then, we test the usefulness of the autotelic principle in a series of language evolution experiments. In the first set of experiments, a population of artificial agents should develop a language to refer to objects with discrete values. These experiments focus on how unambiguous communicative systems can emerge when the autotelic principle is employed to scaffold language development into stages of increasing difficulty. In the second set of experiments, agents should agree on a language to communicate with about colour samples. In this part, we explore how the motivation system can regulate the linguistic complexity of interactions for a continuous domain and examine the value of the autotelic principle as a mechanism to control several language strategies simultaneously. To summarise, we have shown through our work that the autotelic principle can be used as a general mechanism to regulate complexity in language emergence in an autonomous way for discrete and continuous domains. Résumé Dans cette thèse nous étudions le rôle de la motivation intrinsèque dans l’émer- gence et le développement des systèmes communicationnels. Notre objectif est d’explorer comment des populations d’agents artificiels peuvent utiliser un sys- tème de motivation computationnel particulier, appelé l’autotelic principle, pour réguler leur développement linguistique et les dynamiques qui en résultent au niveau de la population. Nous proposons d’abord une mise en œuvre concrète de l’autotelic princi- ple. Le noyau de ce système repose sur l’équilibre des défis, des tâches à ac- complir afin d’atteindre un objectif, et des compétences, les capacités que le système peut utiliser pour accomplir les différentes tâches. La relation entre les deux éléments n’est pas stable mais se déstabilise régulièrement lorsque de nou- velles compétences sont acquises, ce qui permet au système de tenter des défis de plus grande complexité. Ensuite, nous testons l’utilité de ce système de moti- vation dans une série d’expériences sur l’évolution du langage. Dans le premier ensemble d’expériences, une population d’agents artificiels doit développer une langue pour se référer à des objets ayant des caractéristiques discrètes. Ces expé- riences se concentrent sur la façon dont les systèmes communicatifs non ambigus peuvent émerger lorsque l’autotelic principle est utilisé pour réguler le développe- ment du langage en étapes de difficulté croissante. Dans le deuxième ensemble d’expériences, les agents doivent créer un langage artificiel pour communiquer sur des couleurs. Dans cette partie, on explore comment le système de motiva- tion peut contrôler la complexité linguistique des interactions pour un domaine continu et on examine aussi la validité de l’autotelic principle en tant que mé- canisme permettant de réguler simultanément plusieurs stratégies linguistiques de difficulté similaire. En résumé, nous avons démontré à travers de notre travail que l’autotelic principle peut être utilisé comme un mécanisme général pour réguler la complexi- té du langage développé de manière autonome en domaines discrets et continus. A la meva mare Acknowledgements First of all, I want to thank Luc Steels, Thierry Poibeau and Remi van Trijp for the opportunity they gave me. For the last years I could work in a privileged environment and meet outstanding researchers during my time as a member of the language team at Sony CSL Paris. I want to thank my thesis director, Thierry, for his continuous support and guidance during the writing of this manuscript. I’m particularly thankful to Luc who, using one of his analogies, passed me the ball of the autotelic principle and let me play with it. None of what is written here would have been possible without his ground breaking work. Special thanks go to Pieter Wellens and Joris Bleys, whose previous research on the autotelic principle and the colour domain have been an inspiration and a basis for this thesis. I also want to thank the members of my PhD jury, Prof. Dr. Peter Ford Dominey, Prof. Dr. Bart de Boer, Prof. Dr. Pierre-Yves Oudeyer, Prof. Dr. Myriam Bouveret, Dr. Remi van Trijp and Prof. Dr. Thierry Poibeau as their comments and suggestions improved the quality of this manuscript significantly. I’m undoubtedly grateful to all the people that worked in the Babel, the framework used in the experiments in this thesis, in the past. As Isaac Newton famously said: "If I have seen further, it is by standing on the shoulders of giants". Thanks a lot to all my colleagues in Barcelona, Brussels, Paris and Tokyo: Emília, Andrea, Maria, Jorge, Katrien, Pieter, Yana, Aurélien, Michael, Paul, Remi and Mica. Among them, I had the chance to share the office for nearly three years with an extremely talented researcher: Paul, it has been a pleasure! I also want to thank all the people at Sony CSL, in particular the former director François Pachet and Sophie Boucher, who has always been there ready to help me. A special thanks to my friends in Paris: Dani, Timotée, Marco, Alba, Emily, all the people from Cerise and to my parisian wife, Emmanuel. I am extremely lucky to have met Nicolas, Jeanne, Élise, Françoise and Pierre, who have become part of my family. Voldria acabar amb unes paraules d’agraïment per a la gent que ha viscut aquesta etapa des de la distància. Tinc la sort d’estar envoltat de gent m’ajuda i recolza a cada pas que faig. A la meva família i amics1, moltes gràcies! The research reported in this thesis has been financially supported by Sony Computer Science Laboratories Paris and by a CIFRE grant (agreement no. 2013/0730). 1Menció especial pel Chris per ajudar-me a corregir l’anglès macarrònic. Contents 1 Introduction 1 1.1 Situating this work: language evolution . .2 1.2 Selectionist theory of language evolution . .4 1.2.1 Language as a Complex Adaptive System . .4 1.2.2 Language systems and language strategies . .5 1.2.3 Linguistic selection . .5 1.2.4 Self-organisation of language systems . .6 1.2.5 Semiotic cycle: the importance of context . .7 1.3 Managing complexity . .8 1.3.1 Motivation as a force . .9 1.3.2 Computational approaches to motivation . .9 1.3.3 Linguistic complexity . 10 1.4 Main hypothesis . 11 1.5 Objectives . 11 1.6 Structure of the thesis . 11 I Technical Background 15 2 Language games and technical background 17 2.1 Language games . 17 2.2 Incremental Recruitment Language . 19 2.2.1 Meaning as a network . 19 2.2.2 Execution of an IRL-network . 20 2.2.3 Building meaning . 21 2.3 Fluid Construction Grammar . 21 2.3.1 Representing linguistic structures . 21 2.3.2 Transient Structures . 23 2.3.3 Constructions . 23 2.3.4 Construction application process . 25 2.3.5 An example of how FCG works . 27 2.4 Summary . 29 CONTENTS II The Autotelic Principle 31 3 History of Motivation: psychological and AI approaches 33 3.1 Psychological theories on motivation . 33 3.1.1 Drive-reduction theory . 34 3.1.2 Operant conditioning . 35 3.1.3 Reconsidering motivation . 36 3.1.4 Motivation and balance . 37 3.1.5 Personal causation, or the opposition of intrinsic and ex- trinsic motivation .