Tilburg University Meaning Development Versus Predefined

Tilburg University

Meaning development versus predefined meanings in language evolution models Vogt, P.

Published in: Proceedings of the International Joint Conference on Artificial Intelligence 2005

Publication date: 2005

Link to publication in Tilburg University Research Portal

Citation for published version (APA): Vogt, P. (2005). Meaning development versus predefined meanings in language evolution models. In L. Kaelbling, & A. Saffiotti (Eds.), Proceedings of the International Joint Conference on Artificial Intelligence 2005 (pp. 1154-1159). IJCAI.

General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal Take down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Download date: 27. sep. 2021

Meaning development versus predeﬁned meanings in language evolution models

¢¡ £

Paul Vogt

Language Evolution and Computation, School of Philosophy, Psychology and Language Sciences

University of Edinburgh, 40 George Square, Edinburgh EH8 9LL, U.K. £ Induction of Linguistic Knowledge / Computational Linguistics Tilburg University, P.O. Box 90153, 5000 LE Tilburg, The Netherlands [email protected] To appear in the proceedings of IJCAI-05. ¤ c IJCAI.

Abstract it takes a number of generations until compositionality arises, in studies where the syntax co-develops with the seman- This paper investigates the effect of predefining tics compositionality arises from the first generation [Steels, semantics in modelling the evolution of composi- 2004; Vogt, 2005]. Why is this difference? In this paper, the tional languages versus allowing agents to develop answer is sought by starting off with an implementation of these semantics in parallel with the development of Vogt’s model, which was based on Kirby’s model, but without language. The study is done using a multi-agent predefined semantics, and then comparing this with a model model of language evolution that is based on the in which the semantics is predefined, as in Kirby’s model. Talking Heads experiment. The experiments show The next section presents some background relating to that when allowing a co-evolution of semantics Kirby’s and Vogt’s model. Section 3 presents the model with language, compositional languages develop which is our starting point. The results are presented in Sec- faster than when the semantics are predefined, but tion 4 and discussed in Section 5. compositionality appears more stable in the latter case. The paper concludes that conclusions drawn from simulations with predefined meanings, which 2 Background most studies use, may need revision. In the context of this paper, compositionality is defined as a representation of which the meaning of the whole can be de- 1 Introduction scribed as a function of the meaning of its parts. For instance, the meaning of the expression “red apple” is a function of the The field of evolutionary linguistics is a rapidly growing field meaning of “red” and the meaning of “apple”. As a conse- in contemporary cognitive science. Many studies are based quence, it is possible to substitute one part with another to on computational modelling, where the researchers typically form a new meaning as in the expression “green apple”. In study aspects of language evolution using models that inte- contrast, there are holistic representations in which the mean- grate AI techniques such as multi-agent systems, evolutionary ing of the whole cannot be described as a function of the computation, machine learning, natural language processing meaning of its parts. For instance, the expression “kick the and robotics. See [Cangelosi and Parisi, 2002] for an exten- bucket” in the meaning of dying is a holistic phrase. sive overview. It has been repeatedly shown that compositional structures One of the most prominent aspects of human language that can arise from initially holistic structures (i.e. structures with is researched concerns the origins and evolution of gram- no compositionality) using the iterated learning model (ILM) matical structures, such as compositionality. Composition- [Brighton, 2002; Kirby, 2002; Smith et al., 2003]. In the ILM ality refers to representations (typically utterances in lan- the population at any time consists of adults and children.1 guages) in which the meaning of the whole is a function of The children learn from the linguistic behaviour of adults. the meaning of its parts. Studies into the origins and evo- After a learning episode (or iteration), the adults are removed lution of compositionality have yielded models that can suc- from the population, the children becomes new adults and cessfully explain how compositionality may emerge. Most new children enter the population and the process repeats. models have semantic structures built in, so the agents only Kirby and others have shown that, given an induction mech- have to acquire a mapping from signals to these mean- anism that can induce compositional structures, an initially ings, together with their syntactic structures [Brighton, 2002; holistic language can change into a compositional one after Kirby, 2002; Smith et al., 2003]. Only few models have con- a number of iterations, provided the language is transmitted sidered how compositional structures can arise through a co- through a bottleneck. evolution between syntax and semantics, where the semantics The transmission bottleneck entails that children only ob- are grounded through interactions with the world and develop serve a part of the expressible meanings of the language. As- from scratch [Steels, 2004; Vogt, 2005]. suming the children are equipped with a learning mechanism Naturally, the two approaches yield different results. Whereas in [Brighton, 2002; Kirby, 2002; Smith et al., 2003] 1In most ILMs there is only one adult and one child. to discover and store compositional structures whenever pos- action result sible, these structures tend to remain in the language because 1 sensing the environment context they allow an agent to communicate about previously unseen 2 select topic topic meanings. Suppose, for instance, you only have observed 3 discrimination game meaning the expressions “ab”, “ad” and “cb” meaning p(m), q(m) 4 decoding expression and p(n) resp.. If you have no way to discover a composi- 5 encoding topic tional structure, you would not be able to express the mean- 6 evaluate success feedback ing q(n). However, if you have the ability to acquire com- 7 induction grammar positional structures such as S -> A/x B/y, where A/m -> a or A/n -> c, and B/p(x) -> b or B/q(x) -> Table 1: A brief outline of the guessing game. Step 2 and 4 d, you would be able to form the sentence “cd” to express are performed only by the speaker, steps 5 and 7 only by the the meaning q(n). If an agent has acquired the language hearer, and all other steps by both agents. through a bottleneck, it may have to produce expressions about previously unseen meanings when it has become an adult. If there is no bottleneck, the children are expected to the agents’ world. Language development is controlled by have learnt the entire language, so no compositionality is re- agents playing a series of guessing games, cf. [Steels, 1997]. quired and typically does not evolve. The description of the model provided below lacks many de- In Kirby’s model, all agents (adults and children alike) are tails and motivations. Unfortunately this is unavoidable due equipped with the same predefined predicate argument se- to the lack of space in this paper. For further details, consult mantics. Naturally this is unrealistic, since it is widely ac- [Vogt, 2005]. knowledged that human children are born without innate se- The guessing game (GG) is briefly outlined in Table 1. The mantics. The question is therefore: to what extent does pre- game is played by two agents: a speaker and a hearer. Both defining the agents’ semantics influence the results of such agents sense the situation (or context) of the game. The con- simulations? text consists of a given number of geometrical coloured

To investigate this question, Vogt [2005] implemented a objects. For each object ¡£¢ , the agents extract a feature vec-

¢¦¥¨§ © © © © © simulation based on Kirby’s model, but without predeﬁning tor ¤ , where are features in a speciﬁed the semantics. In Vogt’s model (described below), the seman- quality dimension [Gardenfors,¨ 2000]. These dimensions re-

tics of individual agents develop in parallel with the language late to a sensed quality, which in the current implementation

learning. This way, the semantics of adult agents differ from is one of the components of the rgb colour space ( ) the children’s. or a shape feature © indicating the shape of the object. Vogt [2005] showed that, even without a bottleneck, rela- The speaker selects one object from the context as the topic

tively high levels of compositionality developed very early in ¡ of the game (step 2, Table 1) and plays a discrimination

the simulation. It was hypothesised that this rapid emergence game [Steels, 1997] to ﬁnd a category that distinguishes the

¡ ¡ ¢ ¦ ¡ of compositionality has to do with the statistical nature of the topic from the other objects in the context input given to the agents (both from the environment and sig- (step 3). To this aim, each individual agent constructs an on-

nals). Furthermore, it was shown that under certain condi- tology containing categorical features (CF), which are points

! ©

tions, e.g., when a bottleneck on transmission was absent, in some quality dimension " . Each feature is cate-

this compositionality was unstable. In those cases, the lan- gorised with that categorical feature ! such that the distance

# #

& guages gradually (though sometimes suddenly) transformed © %$ is smallest.

into holistic languages. One possible explanation for such Combining the CFs of each quality dimension constructs a

¢ ¢ ¢

¡ ' ( a transition was based on the ontogenetical development of category ' for object . The category is a point in a -

meaning. dimensional space. If the category for the topic ¡ is different The study in this paper investigates the effect of predeﬁn- from the categories of all other objects in the context, the DG

ing the semantics as opposed to semantic development with succeeds. If no distinctive category is found, then each fea- ©

respect to the rapid development of compositionality and to ture of the topic’s feature vector ¤) is used as an exemplar the stability of the compositional structures. More concrete, a to form new CFs ! , which are added to the ontology. This number of simulations will be presented in which the model way, the ontology is built up from scratch. is gradually changed from the model reported by Vogt [2005] The distinctive category resulting from the DG serves as

to a model with predeﬁned semantics as reported in Kirby ¡ the ‘whole meaning’ of , where the whole meaning can [2002]. The aim is to verify the two possible explanations be considered as a holistic semantic category. Semantic cat- mentioned. egories are represented by conceptual spaces [Gardenfors,¨

2000]. A conceptual space is spanned by ( quality dimen- '¢

3 The model sions and contain ( -dimensional categories formed from

The model is implemented in Vogt’s Talking Heads simula- ( CFs . The holistic semantic category is spanned by tor THSim [Vogt, 2003].2 In this simulation, a population of all 4 quality dimensions. Other semantic categories include agents can develop a language that allows them to commu- any possible conﬁguration of quality dimensions, such as a nicate about a set of geometrical coloured objects that form colour conceptual space (spanned by the rgb qualities), a 1-dimensional shape space and a ‘redgreen’ space spanned 2Available at http://www.ling.ed.ac.uk/˜paulv/thsim.html. by the r and g components of the rgb space. The whole

¢¡

¦ ¤ ¦ ¤ ¦ ¤ §©¨

S -> greycircle/ £¥¤ or that decodes the meaning holistically. The former case

S -> A/rgb B/s is true if the speaker wanted to decode, for instance, mean-

A -> red/ £¥

£- £ ¤ § ©¨

ing using the grammar of Table 2. The part

¢ 3254

B -> square/ £¥

£¥ can be decoded using composition ,

254

¤¥ ¨

B -> triangle/ £ where is the undecoded part. A new rule of the form B ->

264

§ ¨ circle/ £-¤ is then constructed to ﬁll in . In the model,

Table 2: A constructed example grammar. In the example S, ¡

expressions are constructed at random from a limited alpha-

# # D

A B ¡

and are non-terminals. is a holistic rule that rewrites to ¥

7 8¦ 9 :<;%9=;%>

bet 7 of size and with length , where ,

¤¦ ¤¦ ¤ § ¨

an expression greycircle with meaning £¥¤¦ .

"CB ?@ A9

with a frequency distribution to implement a bias is a compositional rule that rewrites to non-terminals A toward short strings. and B that form linguistic categories associated with the se- The decoded expression is uttered to the hearer, which tries mantic categories indicated by the quality dimensions rgb to encode this expression (step 5). The hearer does not know

(for the colour space) and s (for the shape space). Each non-

which object ¡ is the topic, and its aim is to guess this terminal rewrites to another expression and meaning. (Note using the verbal hint received. To do this, the hearer ﬁrst

that an expression can be associated with more than one

plays a DG for each individual object , thus resulting meaning as in .) in a set of possible meanings for the expression. Then the hearer searches its grammar for compositions that can parse meaning can thus be formed by a holistic category, of by a the expression and of which the meaning matches one of the possible meanings. If there are more than one, the hearer

combination of more lower dimensional categories. There are

" two constraints in the current model: (1) All quality dimen- selects the one with highest score # and guesses that sions should be used exactly once. (2) Only combinations of the object of the corresponding meaning is the topic. 2 semantic categories are allowed. During their development, This information is passed back to the speaker, who veriﬁes agents learn (or discover) which semantic categories are use- if the guess is correct and provides the hearer with feedback ful to form a compositional language. (In one experiment, regarding to the outcome (step 6). If the game succeeds, the

though, these are predeﬁned.) weights of used rules are increased, while competing ones are

¥GF $'FK"L

EIHJ E

inhibited using the equation E , where is

A composition may be of only one holistic rule (e.g.,

¢¡

L ¥

a weight ( or ), and if the weight is increased and

¥ ¥

in Table 2), or of several rules, such as

¥M

, which combines a colour with shape (note that L if it is inhibited. (Note that a rule is competing if it can

the operator is used to connect terminal rules ( and ) decode an expression for the topic or encode the expression

received. Meanings compete when they belong to the same

with compositional rules such as ). Each rule has a rule

used rule, but are not the ones used, see, e.g., .) If the weight and – for the terminal rules – each possible mean-

game fails, the speaker provides the hearer with the correct ¢ ing of a rule has the weight weight , both indicating the effectiveness of a rule and meaning.3 These weights are used topic and the hearer will then try to induce new knowledge

(step 7). to compute a score indicating the effectiveness of the rule Induction proceeds in up to three of the following steps (if

based on past experiences. If the rule is a terminal rule, ¢

one step succeeds, induction stops):

¥ ¢ ¥

; if is a compositional rule (such as ), .

Given the composition , one can calculate the composi- 1. Exploitation. If there is a composition that partially

#" ¥%$'&)(+* !

tion score . When a rule/meaning pair is decodes the expression, the remaining gaps are ﬁlled.

¢ used successfully in the guessing game, the weights and For instance, if the hearer hears “redcircle” and the

are increased, while the weights of competing rules/meanings topic (indicated by the speaker) has distinctive category

¤ §©¨

are inhibited. The score is used to select among com- £¥ , then it can decode the expression par-

N 83264

peting compositions. tially using composition , covering meaning

When the speaker obtained a distinctive category from the £¥ . In that case, the hearer will add the rule B

§ ¨ DG, it tries to decode an expression (step 4, Table 1). To -> circle/ £-¤ to its grammar.

this aim, the speaker searches its grammar , for composi- 2. Chunking. If exploitation fails, the hearer will investi- tions of rules that match the distinctive category (or meaning).

gate whether it can chunk the expression-meaning pair

¤¦ ¤¦ ¤ § ©¨ So, the meaning £¥¤¦ can be decoded into the ex-

¡ based on stored (holistic) expression-meaning pairs re-

£©¨

pression greycircle using and meaning £- . . ceived in the past. (These pairs are stored in an instance into redsquare using . (Note that elements

base.) This is done by looking for alignments in the

such as and are CFs with value 1 and dimension r and received expression with stored expressions and align- s respectively.) If there is more than one composition that ments in the distinctive category with stored meanings. can decode a meaning, the composition with the highest score

If more chunks are possible, the chunk that has occurred

!" # is selected. This implements a selection mechanism most frequently or that has the largest common string is favouring effective compositions. actually pursued. If there is no composition that decodes the meaning, a new

rule is constructed that either decodes a part of the meaning, For instance, if the instance base contains an entry with

¤PO ¤PO ¤ § ©¨ expression-meaning pair “pinkcircle”- £¥ 3

The actual implementation of /10 is more complex, but for the and the hearer tries to induce knowledge from purpose of this paper, the current explanation sufﬁces. receiving the expression-meaning pair “greencircle”-

no bottleneck bottleneck

¤ § ¨ £ , the following alignment is present:

4 1 1

£ ©

¤ § ¨ “greencircle”- £ . Based on this alignment the following new rules are added: S -> 0.8 0.8

0.6 0.6

£ ¤O ¤O ¨

C/rgb D/s, C -> pink/ £- , C ->

¨ £¥¤ § ¨ green/ £ and D -> circle/ . 0.4 0.4 compositionality compositionality 3. Incorporation. If chunking fails too, the hearer 0.2 0.2

will incorporate the expression-meaning pair holis- 0 0 0 50 100 150 200 250 0 50 100 150 200 250 tically. For instance, if the expression-meaning iterations iterations

1 1

¤ : ¤ : ¤O ¨ pair is: “wateva”- £ , the rule S ->

0.8 0.8

¤ : ¤ : ¤O ¨ wateva/ £ is added to the grammar. At the end of the induction step, a post-operation called 0.6 0.6 0.4 0.4

generalise and merge is performed to generalise even further compositionality compositionality and to eliminate redundancies. Redundancies can occur when 0.2 0.2 new rules are created that are basically similar to already ex- 0 0 0 50 100 150 200 250 0 50 100 150 200 250 isting ones. The above example of chunking, for instance, iterations iterations yielded the construction of the rule S -> C/rgb D/s with 1 1

additional ﬁller rules. This rule, however, is in essence the 0.8 0.8 same as rule from Table 2. Therefore, this new rule is re- 0.6 0.6 moved (merged) and the non-terminals of the additional ﬁller 0.4 0.4 rules are replaced with the letters A and B. In addition, given a compositionality compositionality

0.2 0.2

§ ¨

newly added rule B -> circle/ £-¤ and the rules of Ta- ¡ 0 0 ble 2, rule can be chunked (generalised) as well, leading 0 50 100 150 200 250 0 50 100 150 200 250

iterations iterations

¤ ¦ ¤ ¦ ¨

to the new rule A -> grey/ £-¤¦ . (Note that the ¡ old rules such as remain in the grammar.) Figure 1: Compositionality as a function of iterations for ex- The language game model is integrated with the iterated ¡ periments 1 (top), 2 (middle) and 3 (bottom). All experiments learning model. In this model, the population contains ¡ were done both without (left) and with (right) a transmission adults and children. Whenever a GG is played, the speaker bottleneck. Each line represents one simulation run. is selected from the adult population and the hearer from the child population. After a given number of guessing games, no bottleneck bottleneck all adults are replaced by the children, and new children enter 2 250 2 250

the population. Such an iteration then repeats.

¢ ¢ ¢

1 0.74 ¢ 0.01 0.16 0.11 0.69 0.04 0.93 0.01

¢ ¢ ¢

2 0.68 ¢ 0.01 0.76 0.14 0.64 0.04 0.93 0.02

¢ ¢ ¢ 4 Results 3 0.30 ¢ 0.03 0.35 0.26 0.48 0.03 0.91 0.02 A number of experiments were done with the above model, three of which are reported here. Each condition was car- Table 3: The results of the first three experiments, both with- ried out both without and with a transmission bottleneck. If out and with bottleneck. For each condition compositionality a transmission bottleneck was imposed, a set of 60 objects is given for the end of the 2nd iteration and the end of the were selected at random from the world of 120 objects at the 250st iteration. start of each iteration. These 60 objects were then the objects about which the agents could communicate during that Experiment 1 The first experiment was done with exactly iteration. the same model as described in the previous section. Here all Each experiment contained a population of 3 adults and meanings developed from scratch in parallel to the develop- 3 children. The experiments were run for 250 iterations of ment of the grammar. 6000 guessing games each. At the end of an iteration, the population was tested before the adults were replaced by the Experiment 2 In experiment two, all agents were given a children and new children entered the population. In each repertoire of categorical features. With these CFs, the agents test phase, 200 contexts were constructed. In every context could form categories right from the start, but they did not one object was chosen as topic, and all agents then tried to have any idea how to combine the different dimensions to encode an expression, which in turn all other agents tried to form conceptual spaces as the basis of the semantic struc- decode. From this phase, various measures were calculated of tures. which compositionality is the measure reported in this paper. Compositionality is calculated as the ratio between the num- Experiment 3 In the third experiment, the semantic structure ber of compositional rules used (both encoded and decoded) was predefined. All agents were given CFs as in experiment and the total number of utterances produced and interpreted. 2 and an additional constraint that semantic categories (i.e. The testing phase always used the entire world of 120 objects. conceptual spaces) were either colours, shapes or the whole (holistic) meaning. 4Alignments are only allowed if they appear at the start or the end of an expression, and if they are connected. Figure 1 and Table 3 show the results of these three experi-

¥ ¦ § ¡©¨ § ments both without (left) and with (right) a transmission bottleneck. The development of compositionality in experiment r gbs 0.297 1 is shown in the top graphs. As can be seen, in both cases g rbs 0.200

compositionality emerged to a rather high level in the ﬁrst few b rgs 0.256

¡ ¢ ¤ ¤ §£¢ ¢ ¤ iterations ( ¤PO without and with a bottle- rg bs 0.117 neck, see Table 3, row 1). After that, compositionality tended rb gs 0.144 to be instable and eventually died away in favour of holistic gb rs 0.117 languages if no bottleneck was present (this was conﬁrmed in rgb s 0.075 simulations run for 500 iterations). With a transmission bot-

Table 4: Probability of ﬁnding co-occurring structures in di- tleneck, compositionality rapidly kept rising to a stable value ¥

mension in two different games, assuming the values in

4 of around 0.93. Only one simulation showed a sudden dra- ¦

differ. Assuming that can have any possible value in

4 ¥

matic decrease in compositionality around iteration 200, but ¥

the space of (indicated by ) and any value in

¡ ¡ ¡

§ § 4 ¥ ¦ soon recovered to its previous level. ¦

¥ "

the space of , then and

§ 4 4 ¦

When the agents have predeﬁned CFs, but had to learn the §

¥ ¥ § " "

semantic structures (experiment 2, middle graphs), the initial . The values are based on levels of compositionality were slightly lower than in exper- the distribution of feature values, which are not presented in iment 1. In the case where there was no bottleneck, compo- this text. sitionality remained in most cases at the same level, and in some cases even increased to levels near 1. When there was a bottleneck, compositionality rose in the same way to similar lems the agents learn is to discover regularities in the seman- levels as in experiment 1. tic space (which reﬂects the feature space of the objects) in combination with regularities in the signal space. It was hy- In the case where the semantics, including their structures, pothesised that the rapid development of compositionality in were predeﬁned (experiment 3, bottom graphs), the results experiment 1 (and 2) was caused by the statistical properties were quite different from the other two experiments. In both of the semantic space on one hand and the signal space on the cases, the initial levels were substantially lower, although – other [Vogt, 2005]. Because signals tend to be short and are in contrast to the other experiments – compositionality was

constructed from a limited alphabet, the likelihood of ﬁnd-

¤ ¢ ¤ ¤ lower without a bottleneck ( ¤ ) than with a bottle-

ing alignments in this space is high. Since this aspect is not

> ¢ ¤ ¤ neck ( ¤ ). When no bottleneck was imposed, com- varied, all experiments had the same likelihood. positionality increased to 0.8 in one simulation run, it rose in a few other cases, but often it remained at the same level The chances for ﬁnding regularities in the semantic space, or decreased. When a bottleneck was imposed, the composi- however, is much higher when no structure is predeﬁned. In tionality revealed a similar increase as in the two other exper- experiment 3, the structure for combining the rgb space with

iments with a bottleneck. the shape s space was imposed. Given there are 12 colours

: and 10 shapes, there are 8: different ways to form semantic structures of these 120 objects. However, when 5 Discussion no structure is imposed, the combinations r-gbs, g-rbs, The experiments in this paper investigate the hypothesis that b-rgs, rg-bs, rb-gs and bg-rs are possible ways to the statistical nature of the semantic structure can explain combine the 4 quality dimensions as well. Since, for in-

the rapid development of compositionality and the hypothesis stance, in the r dimension there are 40 different objects (4

that the instability of compositionality in some circumstances §

had to do with the ontogenetical development of meaning. In a rough estimation, the probability of ﬁnding a one of

these objects in game is proportional to . The probability

The experiments further investigate the effect of imposing a §

bottleneck on the transmission of language on the develop- §

ment of compositionality. is B . So, the probability of ﬁnding a co-occurring

structure in the r dimension with in two consecutive

¨§

All experiments reveal that when a bottleneck is imposed, §

BM >

compositionality develops rapidly to a high and stable degree, games is: ¤¥ . Now, if we look at objects of the

thus conﬁrming the results achieved earlier by Kirby and oth- same colour, e.g., blue (i.e. with and ),

then the¡ chance of ﬁnding a blue object in the ﬁrst game is ¡

ers [Brighton, 2002; Kirby, 2002; Smith et al., 2003]. When §

no bottleneck is imposed, the behaviour of the different ex- and a different blue object in the second game is

§ § ¨§

B BG § periments is more different from each other, which will be , so finding this co-occurrence is ¤ . discussed in the remainder of this section. Table 4 shows the co-occurrence probabilities for all ob- There are two types of observations: First, when no seman- jects involving similar values in one or more dimensions of tic structure is imposed (experiments 1 and 2), high levels of the colour space. The values shown are the probabilities compositionality are achieved within two iterations. Second, summed over all possible values in a conceptual space. As when no categorical features are predefined (experiment 1), the table shows, when considering the entire colour space compositionality is unstable, whereas in other cases, compo- rgb, the co-occurrence probability is much lower than when sitionality is either stable (experiment 2) or may emerge at a searching for a co-occurring structure in the r space. So later stage (experiment 3). clearly, when an agent is able to construct any possible Let me start discussing the first observation. The prob- combination of semantic categories as in experiment 1, the chances of finding co-occurring structures is much higher are confirmed in all tested conditions. Hence, the assumption than when it is restricted only to using rgb-s as in exper- they made on predefining semantics to simplify their models iment 3. Hence, compositionality in experiment 1 emerges appears valid. However, predefining the meanings – either faster than in experiment 3. with or without structure – can affect the results achieved in Interestingly, when there is a bottleneck on the transmis- simulations of language evolution to a great extent, so one sion of language and stable compositional systems emerge, should be very careful in interpreting results achieved with the most dominant rules (i.e. the type of rules that were used such ungrounded models. This does not mean that the current most frequently) combine the colour space rgb and the shape grounded model on language evolution is sufficiently realis- space s. When there is no bottleneck, the most frequently tic to make hard claims regarding the evolution of language. used compositional rule combined r with gbs. This can be However, the model is more realistic than ungrounded mod- understood by realising that when half of the objects are dis- els in that in this model children do not have the same full- carded, the co-occurrence probability in the rgb-s structure fletched semantic structures as adults. is more or less maintained, while the other probabilities are affected differently in each iteration. This is because the dis- Acknowledgements tribution of colours and shapes is less skewed than the distri- This work is sponsored by a EC Marie Curie Fellowship and butions in the other dimensions. (Recall that each iteration a a VENI grant provided by the Netherlands Organisation for different set of objects is selected to learn from.) As a result, Scientific Research (NWO). The author thanks Andrew and the rgb-s structure can be learnt most reliably. Kenny Smith and three anonymous reviewers for comments on earlier versions of this manuscript. The second observation that compositionality is unstable in the absence of bottlenecks when categorical features are not predefined can be explained by realising the consequences of References the gradual development of CFs. The foremost difference is [Brighton, 2002] H. Brighton. Compositional syntax from that at the start of an iteration adults have a well developed cultural transmission. Artificial Life, 8(1):25–54, 2002. set of CFs, while the children have none. During develop- [Cangelosi and Parisi, 2002] A. Cangelosi and D. Parisi, ed- ment, the children gradually acquire a similar set of CFs, but itors. Simulating the Evolution of Language. Springer, initially these CFs are overgeneralisations of the final cate- London, 2002. gories. Nevertheless, the children may – in certain situations – categorise some objects distinctively, thus successfully fin- [Gardenfors,¨ 2000] P. Gardenfors.¨ Conceptual Spaces. Brad- ishing a discrimination game. ford Books, MIT Press, 2000. As a consequence, these children can successfully learn the [Kirby, 2002] S. Kirby. Learning, bottlenecks and the evo- words associated with these categories. However, since the lution of recursive syntax. In Ted Briscoe, editor, Lin- CFs are overgeneralisations of the adults’ CFs, the children’s guistic Evolution through Language Acquisition: Formal CFs may be associated with more than one word. This is and Computational Models. Cambridge University Press, even more the case when realising that different adults can 2002. have different words to name a CF. When the children ac- [Smith et al., 2003] K. Smith, H. Brighton, and S. Kirby. quire more fine grained CFs, the words may become associ- Complex systems in language evolution: the cultural ated with different CFs than was the case for the adults. This emergence of compositional structure. Advances in Com- way, the words can drift through the conceptual space. When plex Systems, 6(4):537–558, 2003. the language is compositional, the movement in one semantic category affects a larger part of the language than it would [Steels, 1997] L. Steels. The synthetic modeling of language when the language is holistic. Hence, holistic languages are origins. Evolution of Communication, 1(1):1–34, 1997. more stable and thus easier to learn. When the CFs are prede- [Steels, 2004] L. Steels. Constructivist development of fined, there is less or no reason for overgeneralisations. This, grounded construction grammars. In W. Daelemans, ed- in turn, allows children to acquire any compositional struc- itor, Proceedings Annual Meeting of Association for Com- ture of their adults more reliably, which – in part – explains putational Linguistics, 2004. why compositionality is more stable (experiment 2) or even [Vogt, 2003] P. Vogt. THSim v3.2: The Talking Heads sim- emerges (experiment 3). The results of experiment 3 are very ulation tool. In W. Banzhaf, T. Christaller, P. Dittrich, J. T. similar to those reported in [Smith et al., 2003], while the re- Kim, and J. Ziegler, editors, Advances in Artificial Life - sults of experiment 2 is highly dissimilar. It is yet unclear Proceedings of the 7th European Conference on Artificial whether the predefined semantics of Smith et al. are more Life (ECAL). Springer Verlag Berlin, Heidelberg, 2003. similar to the ones used in experiment 2 or 3, though the results suggest that the semantics are more similar to the one [Vogt, 2005] P. Vogt. The emergence of compositional struc- used in experiment 3. tures in perceptually grounded language games. Submitted for publication, 2005. Concluding, both hypotheses tested in this paper are confirmed. In addition, the major findings of [Brighton, 2002; Kirby, 2002; Smith et al., 2003] that compositionality emerges under the pressure of a transmission bottleneck