Tilburg University Meaning development versus predefined meanings in language evolution models Vogt, P. Published in: Proceedings of the International Joint Conference on Artificial Intelligence 2005 Publication date: 2005 Link to publication in Tilburg University Research Portal Citation for published version (APA): Vogt, P. (2005). Meaning development versus predefined meanings in language evolution models. In L. Kaelbling, & A. Saffiotti (Eds.), Proceedings of the International Joint Conference on Artificial Intelligence 2005 (pp. 1154-1159). IJCAI. General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal Take down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim. Download date: 27. sep. 2021 Meaning development versus predefined meanings in language evolution models ¢¡ £ Paul Vogt Language Evolution and Computation, School of Philosophy, Psychology and Language Sciences University of Edinburgh, 40 George Square, Edinburgh EH8 9LL, U.K. £ Induction of Linguistic Knowledge / Computational Linguistics Tilburg University, P.O. Box 90153, 5000 LE Tilburg, The Netherlands [email protected] To appear in the proceedings of IJCAI-05. ¤ c IJCAI. Abstract it takes a number of generations until compositionality arises, in studies where the syntax co-develops with the seman- This paper investigates the effect of predefining tics compositionality arises from the first generation [Steels, semantics in modelling the evolution of composi- 2004; Vogt, 2005]. Why is this difference? In this paper, the tional languages versus allowing agents to develop answer is sought by starting off with an implementation of these semantics in parallel with the development of Vogt’s model, which was based on Kirby’s model, but without language. The study is done using a multi-agent predefined semantics, and then comparing this with a model model of language evolution that is based on the in which the semantics is predefined, as in Kirby’s model. Talking Heads experiment. The experiments show The next section presents some background relating to that when allowing a co-evolution of semantics Kirby’s and Vogt’s model. Section 3 presents the model with language, compositional languages develop which is our starting point. The results are presented in Sec- faster than when the semantics are predefined, but tion 4 and discussed in Section 5. compositionality appears more stable in the latter case. The paper concludes that conclusions drawn from simulations with predefined meanings, which 2 Background most studies use, may need revision. In the context of this paper, compositionality is defined as a representation of which the meaning of the whole can be de- 1 Introduction scribed as a function of the meaning of its parts. For instance, the meaning of the expression “red apple” is a function of the The field of evolutionary linguistics is a rapidly growing field meaning of “red” and the meaning of “apple”. As a conse- in contemporary cognitive science. Many studies are based quence, it is possible to substitute one part with another to on computational modelling, where the researchers typically form a new meaning as in the expression “green apple”. In study aspects of language evolution using models that inte- contrast, there are holistic representations in which the mean- grate AI techniques such as multi-agent systems, evolutionary ing of the whole cannot be described as a function of the computation, machine learning, natural language processing meaning of its parts. For instance, the expression “kick the and robotics. See [Cangelosi and Parisi, 2002] for an exten- bucket” in the meaning of dying is a holistic phrase. sive overview. It has been repeatedly shown that compositional structures One of the most prominent aspects of human language that can arise from initially holistic structures (i.e. structures with is researched concerns the origins and evolution of gram- no compositionality) using the iterated learning model (ILM) matical structures, such as compositionality. Composition- [Brighton, 2002; Kirby, 2002; Smith et al., 2003]. In the ILM ality refers to representations (typically utterances in lan- the population at any time consists of adults and children.1 guages) in which the meaning of the whole is a function of The children learn from the linguistic behaviour of adults. the meaning of its parts. Studies into the origins and evo- After a learning episode (or iteration), the adults are removed lution of compositionality have yielded models that can suc- from the population, the children becomes new adults and cessfully explain how compositionality may emerge. Most new children enter the population and the process repeats. models have semantic structures built in, so the agents only Kirby and others have shown that, given an induction mech- have to acquire a mapping from signals to these mean- anism that can induce compositional structures, an initially ings, together with their syntactic structures [Brighton, 2002; holistic language can change into a compositional one after Kirby, 2002; Smith et al., 2003]. Only few models have con- a number of iterations, provided the language is transmitted sidered how compositional structures can arise through a co- through a bottleneck. evolution between syntax and semantics, where the semantics The transmission bottleneck entails that children only ob- are grounded through interactions with the world and develop serve a part of the expressible meanings of the language. As- from scratch [Steels, 2004; Vogt, 2005]. suming the children are equipped with a learning mechanism Naturally, the two approaches yield different results. Whereas in [Brighton, 2002; Kirby, 2002; Smith et al., 2003] 1In most ILMs there is only one adult and one child. to discover and store compositional structures whenever pos- action result sible, these structures tend to remain in the language because 1 sensing the environment context they allow an agent to communicate about previously unseen 2 select topic topic meanings. Suppose, for instance, you only have observed 3 discrimination game meaning the expressions “ab”, “ad” and “cb” meaning p(m), q(m) 4 decoding expression and p(n) resp.. If you have no way to discover a composi- 5 encoding topic tional structure, you would not be able to express the mean- 6 evaluate success feedback ing q(n). However, if you have the ability to acquire com- 7 induction grammar positional structures such as S -> A/x B/y, where A/m -> a or A/n -> c, and B/p(x) -> b or B/q(x) -> Table 1: A brief outline of the guessing game. Step 2 and 4 d, you would be able to form the sentence “cd” to express are performed only by the speaker, steps 5 and 7 only by the the meaning q(n). If an agent has acquired the language hearer, and all other steps by both agents. through a bottleneck, it may have to produce expressions about previously unseen meanings when it has become an adult. If there is no bottleneck, the children are expected to the agents’ world. Language development is controlled by have learnt the entire language, so no compositionality is re- agents playing a series of guessing games, cf. [Steels, 1997]. quired and typically does not evolve. The description of the model provided below lacks many de- In Kirby’s model, all agents (adults and children alike) are tails and motivations. Unfortunately this is unavoidable due equipped with the same predefined predicate argument se- to the lack of space in this paper. For further details, consult mantics. Naturally this is unrealistic, since it is widely ac- [Vogt, 2005]. knowledged that human children are born without innate se- The guessing game (GG) is briefly outlined in Table 1. The mantics. The question is therefore: to what extent does pre- game is played by two agents: a speaker and a hearer. Both defining the agents’ semantics influence the results of such agents sense the situation (or context) of the game. The con- simulations? text consists of a given number of geometrical coloured To investigate this question, Vogt [2005] implemented a objects. For each object ¡£¢ , the agents extract a feature vec- ¢¦¥¨§ © © © © © simulation based on Kirby’s model, but without predefining tor ¤ , where are features in a specified the semantics. In Vogt’s model (described below), the seman- quality dimension [Gardenfors,¨ 2000]. These dimensions re- tics of individual agents develop in parallel with the language late to a sensed quality, which in the current implementation © © © learning. This way, the semantics of adult agents differ from is one of the components of the rgb colour space ( ) the children’s. or a shape feature © indicating the shape of the object. Vogt [2005] showed that, even without a bottleneck, rela- The speaker selects one object from the context as the topic tively high levels of compositionality developed very early in ¡ of the game (step 2, Table 1) and plays a discrimination the simulation. It was hypothesised that this rapid emergence game [Steels, 1997] to find a category that distinguishes the § ¡ ¡ ¢ ¦ ¡ of compositionality has to do with the statistical nature of the topic from the other objects in the context input given to the agents (both from the environment and sig- (step 3). To this aim, each individual agent constructs an on- nals). Furthermore, it was shown that under certain condi- tology containing categorical features (CF), which are points ! © tions, e.g., when a bottleneck on transmission was absent, in some quality dimension " .