Imitation As a Mechanism of Cultural Transmission

Home , Imitation

Imitation as a Mechanism of Chris Marriott*,** Cultural Transmission University of Calgary James Parker** University of Calgary

Jörg Denzinger** University of Calgary

Abstract We study the effects of an imitation mechanism on a population of animats capable of individual ontogenetic learning. An urge to imitate others augments a network-based reinforcement learning strategy used in the control system of the animats. We test Keywords populations of animats with imitation against populations without for their ability to find, and maintain over generations, successful Social learning, imitation, cultural transmission, pandaemonium, animat foraging behavior in an environment containing three necessary resources: food, water, and shelter. We conclude that even simple imitation mechanisms are effective at increasing the frequency of success when measured over time and over populations of animats.

1 Introduction

Animats [37] are the artificial analogue of animals. Animats, thus, count as autonomous situated agents (see [15]). There are many definitions of agents, but we will proceed very generally by defining an agent to be a quadruple Ag = (Sit, Act, Dat, fAg)[14].Sit is a set of environmental situations that the agent can find itself in. These situations are differentiated by how the agent views its environment, that is, two distinct situations that present themselves to the agent as indistinguishable would be counted as one element of Sit. Act is the set of actions that the agent can perform. For ease we will include in this set both basic actions that cannot be decomposed into simpler actions, and complex actions that may consist of chains (or decision trees) of actions. We will use the term behavior to refer to long term action sequences, especially when they lead to the accomplishment of a noted goal. Action will be used to refer to a single action. Dat is a set of possible configurations of the internal data areas of the agent. Finally, the function fAg is a function fAg : Sit × Dat → Act that takes pairs of situations and internal data configurations and produces an action. We consider only agents where Sit, Act,andfAg are static over the lifetime of the agent. Animats can exist in artificial environments (as in simulations like [37]) or real environments (as in robotics [24]). Traditional animat experiments test phylogenetic adaptive processes, that is, adaptive processes rooted in the genetic evolution of the animats. Agents capable of only phylogenetic adaptation will have static agent definitions, that is, Sit, Act, Dat as well as fAg will all be static over the lifetime of the agent. Phylogenetic adaptation can be carried out in a population of such agents only if the success of an individual can be measured quantitatively. The adaptation occurs by first evaluating the agents for success (either continuously or after some predetermined period). Then the most successful agents are selected to produce the next generation of agents. These agents will pass their traits onto

* Contact author. ** Department of Computer Science, University of Calgary, 2500 University Dr. NW, Calgary, AB, Canada, T2N 1N4. E-mail: [email protected] (C.M.); [email protected] (J.P.); [email protected] (J.D.)

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

the next generation (with possible variations), while the unsuccessful agents will not pass on their traits. In this way the various parameters of Ag can be optimized within the population, though across individuals. With the introduction of artificial neural networks (ANNs) as control systems for animats (see, e.g., [38]), a real possibility of animats displaying both phylogenetic and ontogenetic adaptive processes emerged. Ontogenetic adaptive processes operate over the lifetime of the animal (or animat) and result in phenotypic plasticity (the ability of the expressed phenotype to vary relative to the encoded genotype as a result of the organismʼs interaction with the environment). In recent decades the role of ontogenetic adaptation on behavior selection and the interplay between ontogenetic and phylogenetic adaptation has been studied in some detail [16, 23, 25]. Specific results include the now well-known Baldwin effect, where ontogenetic adaptation increases selective pressure, leading to an increased rate of evolution; and the hiding effect, where ontogenetic adaptation decreases selective pressure due to the organismsʼ ability to overcome genetic drawbacks with lifetime adaptation [25]. In agents capable of ontogenetic adaptation, Dat is dynamic during the lifetime of the agent. Typ- ically, ontogenetic alterations to Dat are aimed at increasing long term success, especially when the mechanisms that alter Dat are subjected to phylogenetic adaptation. That is, phylogenetic adaptation will select for individuals that increase their success through their ontogenetic processes. If agents exist that decrease their success through ontogenetic adaptation, then they will be unlikely parents of the next generation, and thus over time we expect agents to develop ontogenetic processes aimed at increasing their individual success. We can define a social environment to be one in which the agent encounters other (homogeneous or heterogeneous) agents. When situated in a social environment, agents that are capable of lifetime learning may also engage in sociogenetic adaptation. Sociogenetic adaptation relies on at least one mechanism for transferring the learned behavior of one organism to another (these mechanisms of cultural transmission can lead to cultural learning [32, 34]). Sociogenetic adaptation is rapidly becoming the assumed missing step in the simulation of humanlike intelligence [20, 28, 32, 36, 40]. Most mechanisms of cultural transmission and the adaptations they can lead to are not yet well studied in the ALife community (see [1, 7] as examples of experiments oriented toward social learning), though social learning and cooperative behavior have become more popular in recent years [2, 20, 28, 40]. Formalizing our notion of sociogenetic adaptation, we note that it will be a subcategory of ontogenetic adaptation, but one deserving of its own characterization and study. When other agents occupy the environment, it is possible for the agent to alter its Dat by observations of other agents. In particular, when elements from the Dat of one agent are transferred to another agentʼs Dat (with possible variation), they are engaged in cultural transmission. As we will see, the mechanisms of cultural transmission can vary from a direct transfer (akin to telepathy and occurring most prominently in artificial agents and approximated by language-using humans) or indirect transfer (through the observation of anotherʼs actions and the application of simple inferences to generate changes to the Dat). These inferences will be based on (explicit or implicit) assumptions that the other agent is in a similar Sit and then integrating hypotheses about what Dat element must be present to result in the observed action in that Sit. We will consider differences in these mechanisms in more detail below. As in ontogenetic adaptation, typically the transferred behaviors, and the associated behavior gen- erating Dat elements, are those that increase success (again, this is especially true if the transmission mechanisms are subjected to phylogenetic adaptation). Thus, the best of the learned behaviors will tend to be retained by agents in the population, and the spread of these behaviors throughout the population lead to what biologists and anthropologists call a “culture” or “tradition.” The evolution of these behaviors, that is, their long term improvement due to selection and variation in cultural transmission, is termed cultural evolution. In this article we will study the extent to which imitation can serve as a mechanism of cultural transmission supporting cultural learning and cultural evolution in animats. Against a population of imitating animats we maintain a control group of animats only capable of ontogenetic (non-sociogenetic) learning.

22 Artificial Life Volume 16, Number 1

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

2 Social Learning

Social learning, cultural transmission, sociogenetic adaptation, and cultural evolution have been growing in popularity in the artificial life and robotics fields [1, 2, 7, 20, 28, 40], following a slow-building momentum in sociobiology and philosophy over the last decades [6, 9, 11, 22, 32, 33, 35, 36]. Though these terms are often used interchangeably, they represent different aspects of the phenomenon in question. Among other aspects, these terms are used to identify (i) mechanisms of information transmission allowing for learned behavior to be transferred from one agent to another, and (ii) the ability of successful behaviors to multiply and spread in populations via the mechanisms of (i) and thus leading to a new level of evolution of behavior—the societal. In this article, when the mechanisms of transfer are of particular interest, the term social learning or cultural transmission is used. When the evolution of culturally transmitted behaviors is of interest, the term sociogenetic adaptation or cultural evolution is used. Behaviors typically have several different means of moving from body to body. These transmission mechanisms generally either are genetically coded or are learned ontogenetically by the organisms that deploy them. Examples of such mechanisms observed in biology include: automatic signaling, as in metabolic signaling or insect signaling; simple non-linguistic vocalizations, and gestures as in nonhuman primates; imitation, as in humans and possibly other primates; and genuine language-using communication, as in humans. Imitation is considered by cognitive ethologists [33, 35] to be one of the most powerful mechanisms of cultural transmission. As Whiten puts it, “[t]he most thorough way in which an animal may learn from the actions of another is to imitate or copy it. Such a copying process operating across a whole community could lead to population-level similarities of behavior—a ‘culture’ or ‘tradition’ in biologistsʼ terminology.” These mechanisms, and others, have been classified into two groups by Whiten. Both groups can lead to what Whiten calls “cultural transmission”—though only the more sophisticated mechanisms are called “social learning,” while for the less sophisticated mechanisms the term “social influence” is reserved. The distinction here is between the way in which behavior mimicry emerges in the population. In “social learning” the agent can be said to learn from other agents, whereas in “social influence” the learning involved is merely ontogenetic adaptation influenced by the changes in the environment due to the presence of other agents. For instance, Whiten identifies one mechanism of social influence he calls “exposure.” This mechanism results in similar behaviors between two agents due to the fact that both agents are exposed to the same environmental stimuli by being part of the same social organization. Since the two agents are assumed to have similar learning processes, the similar environmental stimuli lead to similar behavior. On the surface the similarities in behavior might seem to be due to a more sophisticated social interaction, but after scrutiny it is found to merely be the result of similarities in environment and develop- ment. We can formalize this notion by noting that changes in behavior due to “exposure” will be purely ontogenetic, involving no transfer of information from one agentʼs Dat to another. Indeed, in cases of “exposure” the other agents are not required for the behavior to be learned, nor does the presence of others increase the speed or level of success of the learned behavior. Whiten also discusses two mechanisms of “social learning” relevant to our experiment. These are stimulus enhancement and imitation. Stimulus enhancement occurs when agent A brings agent Bʼs attention to some object of interest (usually food or a predator) and thus allows agent B to direct the appropriate behavior toward the object. The simplest version of this mechanism merely directs the attention of other agents and then relies on simpler mechanisms of social influence, like “exposure,” to kick in. Whiten notes, however, that merely allowing agent Bʼsattentionto“pass over” the object of interest might not always be sufficient. Indeed, he notes “. . .that stimulus enhancement of a feeding site may occur only if bird B sees bird A actually succeeding in gaining food there” [35]. We can distinguish cases of stimulus enhancement from cases of exposure by identifying the role the “teacher” agent plays in bringing the “student” agentʼs attention to the locus. Formally, the learning agentʼs Dat is altered due to information in the model agentʼs Dat, but this transfer need not include an appreciation of this fact by either of the agents. For instance, the learning agent may exploit the knowledge represented

Artificial Life Volume 16, Number 1 23

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

in the modelʼs Dat merely by following or staying close to the model agent. In this way the learning agent can ensure that it is exposed to the appropriate stimulus to learn from. This is differentiated from “exposure” in that the “exposure” of the learning agent is dependent on the information stored in the model agentʼs Dat. If the model agent were not present, or if the model agent did not possess relevant information in its Dat, then the behavior would not be learned or would be learned less efficiently. Imitation, in contrast, involves learning part of the relevant behavior from other agents. This is distinct from the first two cases, because in stimulus enhancement merely the objects of the behavior are identified. In imitation, the behavior of the learning agent B is matched in some way to the behavior of agent A. As Whiten notes, the behavior matching need not (and indeed rarely is) exact. This is due to the complexity of behavior matching and measuring. A behavior can be measured in many different ways [35]. For instance, we can measure the bodily shape and orientation of the behavior. These aspects may be the same while other aspects, such as the speed of performance, may differ. Indeed, we might expect the learning agent to execute the behavior in a slower and more deliberate fashion as it learns the behavior. Thus, one way we can categorize cases of imitation is by the level of fidelity (along each measurable dimension) with which the behavior is reproduced in the learning agent. Traditionally, the evidence sought for mechanisms of imitation in primates is high- fidelity mimicry of novel behavior [18]. Imitation is seen by ethologists such as Whiten to require that the learning agents represent the model agentʼs frame of mind. The learning agent must appreciate the model agentʼs goal-directed behavior as goal-directed, and must also identify with the goal as being one it shares. This means that Sit and/or Dat must include the means for considering other agents as agents (in psychology this is primarily referred to as possessing a “theory of mind,” and many biologists and roboticists have adopted this term for their own use [26]). In stimulus enhancement, the learning agent needed only to consider the model agent as a tool for learning. There need not have been any appreciation of the toolʼsownintentionality. In imitation, this appreciation is necessary. The above treatment of social learning and imitation is presented using the traditional methodology of ethologists, which some would call the mental representation model [18] or the information-processing model [30]. Under this methodology, ethologists seek to study and understand the nature of the representational—or, in the philosophersʼ terms, intentional [9]—processes of individual cognition. The mental representation model traditionally focuses on cognition as an individual information- processing enterprise facilitated by manipulation of mental representations of the world. Thus, there is a focus on inputs, outputs, and the information-processing stage between the two. Likewise, the traditional treatment of coordinated behavior (as in social learning and communication) are processes of discrete information transfer. Messages (deliberate or not) are seen as both inputs and outputs, and must be encoded by the sender and decoded by the receiver. This model is traditionally practiced by ethologists; for example, it is dominant in [26, 32, 35], and has been directly motivated as a methodology by Dennett in [8] (reprinted with reflections in [9]), with the motivation extended to the study of human behavior in [10, 12, 13]. Recently a shift of perspective (though some would say “paradigm” [18, 30]) has occurred among ethologists and other cognitive researchers in related fields [4, 5, 18, 20, 30, 39]. Advocates of the situated perspective recognize that embodied [39] cognition does not have discrete input and output phases— both systems operate continuously in tandem. This perspective will see even a simple task like reaching for an object in a drastically different way than someone operating from a pure traditional one. Reaching might traditionally be modeled as a three-stage exercise, where first, input is gathered as to the trajectory of the object, its distance, and so on; second, an ultimate action plan is laid out in detail; and third, the plan is executed in one discrete action of reaching. From the situated perspective, we see the reaching act as a sequence of rapid cues and behavioral adjustment consisting of many cycles of input and output. At each stage the position of the arm relative to the object is roughly estimated based on visual and kinesthetic cues. A simple action is suggested, something like “move a little to the left,” and then is executed. Through many iterations, this rapid back and forth can accomplish the reaching task with very little offline planning and analysis. Or, in the language of the situated perspective, the coupling of the system and the world does most of the “information processing,” so no ultimate action plan must be calculated offline by the agent.

24 Artificial Life Volume 16, Number 1

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

The traditional perspective and the situated perspectives need not be competitors [4]. Instead we can see them as defining a spectrum of explanations of agent behavior. The traditional perspective necessarily occupies a high-level view of the behavior, focusing on offline information processing involved in the task. The situated perspective, in contrast, occupies a low-level view of the behavior, focusing on the online coupling of the agent and environment as the behavior is executed. The two perspectives must be combined to gain a complete picture of the behavior, from the high-level decisions to take action down to the low-level details of how that action is carried out. Consider, for example, the familiar task of shopping for groceries. Traditional explanations of shopping behavior will focus on inputs to the task, like empty cupboards and future menu plans, and on how to produce the appropriate output, a list of needed items. A pure traditional explanation would stop as though the shopping task ended when the list was written. Situated explanations will focus on how grocery shoppers use the organization of the supermarket to shorten the distances they must travel, and how they exploit their environment when they place items in their cart or bags. A purely situated explanation would begin as though shopping lists and scheduled shopping trips sprang out of the ether. The only complete explanation of the behavior is one that includes both of these perspectives. Social learning can on the one hand be characterized as discrete instances of cultural transmission, and on the other hand be seen as involving interactive, co-constructed behavior. The pure traditional perspective will see cultural transmission as a process of information encoding in the sender, and a process of decoding in the receiver. The situated perspective will focus on the rapid behavioral cues exchanged during the transmission process. As above, we can see that the situated perspective allows us to recognize that much of the encoding and decoding in transmission is accomplished by social coupling. In our discussion below we will attempt to provide a balanced analysis respecting both of these perspectives. It is our stated desire to study cultural transmission by imitation. However, at this point, it is important to note that the use of terms across the disciplines of this multidisciplinary field is not consistent. Lindblom and Ziemke [20] note that “AI researchers . . . tend to interpret the term imitation in a rela- tively wide sense, whereas primatologists are much more restrictive, arguing that imitation is the most advanced social learning mechanism.” That is, AI researchers tend to not distinguish as finely between different mechanisms of social learning (or in some cases even social influence), using the term “imitation” to broadly refer to any mechanism in which the studentʼs behavior has been adapted via observations of the teacherʼs behavior. As we have seen, primatologists differentiate these mechanisms more finely, reserving the term “imitation” for a sophisticated social learning mechanism in which appreciation of the intentionality of the model subject plays a necessary role. We have outlined the biologistsʼ notion of imitation above, but our experimental setup, embed- ded as it is in the artificial life methodology, relies more readily on the broader computer science definition. However, aware of this distinction as we are, we will address its implications in the discussion that follows. This means that we will not assume prior to our experiments that our mechanism of imitation involves the awareness of intentionality in others, as is required under the primatolo- gistsʼ usage.

3 Pandas: Pandaemonium-Controlled Animats

The world inhabited by our animats (we call them pandas, since they are based on the pandaemonium model [17, 29]) is a 9 × 9 discrete grid containing objects ∈ {food, water, cave, tree, panda} (depicted in Figure 1). The world obeys the following rules of evolution (meaning, in this sense, change over time):

• No object can occupy the same cell as a tree. • All objects are stationary except pandas.

Artificial Life Volume 16, Number 1 25

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

Figure 1. Panda environment.

• Water, caves, and trees are static (have no state changes) in the simulation. • Food is depleted when used, and grows over time.

Furthermore, the pandas in the system have limited ability to move about and change the environment:

• Pandas can move one cell in one of eight directions {N,NE,E,SE,S,SW,W,NW}. • Pandas have four properties: food energy, water energy, rest energy, health. • Every round, the panda loses a fixed amount of food energy, water energy, rest energy. • If any of the pandaʼs properties drops below 0, the panda dies. • A dead panda is removed from the system and replaced with food. • A panda can use food, water, cave to replenish food energy, water energy, rest energy, respectively. • A panda can create a new panda by spawning or mating. • A panda can only spawn or mate when it has an excess of all energy types. • A panda can only mate with another mating panda. • Pandas can only interact with objects in the same cell. • A panda can fight with another panda, reducing the target pandaʼs health. • A pandaʼs health automatically recovers, at an energy cost.

From this world characterization we can define the quadruple specifying panda: P = {Sit, Act, Dat, fP}. A panda has only a one-cell perception range, meaning it can view a 3 × 3 subsection of the world with the panda at the center. A panda also has a limited internal sense allowing it to monitor its energy levels. All perception is done utilizing specialized input daemons. (Daemons, roughly, are nodes in a network not unlike the neurons in ANNs, with the exception that daemons typically

26 Artificial Life Volume 16, Number 1

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

are assigned, and play, a specific role in the network, whereas in general a neuronʼs role is unassigned and unknown.) The input daemons that form the input layer of the panda include nine daemons for each object in the environment {see food, see water, see cave, see tree, see panda}: one daemon for each of the cells in the perceptual range of the panda. That is, there is a specific see food daemon for each cell, allowing the panda to represent differently food seen to the north and food seen to the south (and all other directions, including seeing food in the current cell). Furthermore, the input layer contains daemons {food energy, water energy, rest energy, hungry, thirsty, tired}. The activity (identical to neuronal activity in ANNs) of the energy daemons represents directly the amount of energy of that type that the panda has. The daemons {hungry, thirsty, tired} are warning daemons that are activated when the respective energy level drops below a threshold; the activity level is directly proportional to the difference between the energy level and the threshold. Thus, we can define Sit to be the set of all legal 3 × 3 perspectives on legal world configurations and all legal energy level combinations. The pandaʼs actions are defined by its interactive capacities listed above. The move actions are {move north, move northeast, move east, move southeast, move south, move southwest, move west, move northwest}. Act also includes the interactive actions {eat, drink, rest, spawn, mate, fight}. Most of these actions have prerequisites that make the action possible. The move actions are possible only if the target cell does not contain a tree. The resource gathering actions {eat, drink, rest} can only be carried out in a cell containing the appropriate resource {food, water, cave}, respectively. Spawn and mate can only be taken if the pandaʼs energy levels are all above a threshold. Finally, mate and fight can only be taken if the panda is in a cell with other pandas. The functioning of our control system is complex, and a precise, detailed account is not relevant for this article, so we will focus only on the high-level organization of Dat relevant to our discussion. However, it is important to motivate our selection of a nonstandard agent architecture. We have elected to work with a prototype architecture based on the pandaemonium model because we believe that it is superior to artificial neural networks along a couple of dimensions. First, though ANNs are capable of ontogenetic learning, the number of training epochs required to train an ANN even for simple tasks is enormous. Most animals, in contrast, show rapid learning and flexibility even in novel situations. Our pandaemonium model, being able to integrate reinforcement learning strategies, has a vastly more powerful ontogenetic learning capacity than standard ANNs. One major result of this is a decrease in the number of training generations. Typically, an evolved ANN will go through hundreds of generations before successful individuals appear in the population. In data-bearing runs (see below) our pandas are successful from the very first generation. Secondly, and primarily due to the ontogenetic shortcomings just noted, ANNs have no obvious and easy way to test social learning. The artificial social learning mechanisms developed for ANNs by [7] involve an artificial pairing of teacher and student, as well as a necessary training period, both of which are absent in natural systems (excepting human institu- tions). Because the pandaemonium model does not suffer from the same shortcomings, we believe that it is a reasonable model to test our hypotheses about social learning and cultural transmission. The internal data areas of a panda consist of a network of various types of daemons, connected by various types of connections and organized into functional layers. We have already introduced the input layer, consisting of input daemons, whose role is identical to that of the input neurons in an ANN, that is, to represent specific salient features of the environment or the panda itself. Each input daemon represents a single feature of the environment, and together the active input daemons char- acterize a situation. Related to the input daemons are episode daemons resident in the episode layer. An episode daemon is a coalition of input daemons representing a (partial) situation (i.e., an element of Sit) as a unit. (This is in contrast to the distributed representation in the input layer.) Episode daemons function in the state space mapping algorithms utilized by the panda and also in reinforcement learning. Episode daemons are also created dynamically over the lifetime of a panda as they are needed to represent new situations. The episode layer is initially empty. Drive daemons serve as one intermediary between input daemons and action daemons, and thus play a role in action selection. Drive daemons respond to circumstance represented by the input daemons

Artificial Life Volume 16, Number 1 27

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

by providing a motivation to action. This is done by spreading activity to appropriate action daemons along special connections called drive connections. Drive daemons are connected via a positively weighted connection to those actions that if taken would satisfy the drive, or alternatively via a negatively weighted connection to those actions that would frustrate the drive (see [21]). Specifically, a panda will have drive daemons {seek food, seek water, seek rest, cease feeding, cease drinking, cease resting, reproduce}. The first six drive daemons play a role in resource management. The seek drives provide motivation to perform actions that will gather the associated resource and are activated by the warning input daemons for the same resource. Thus, if a panda is low on food energy,thenthe hungry input daemon provides activity to the seek food daemon, which in turn provides activity to the eat action daemon (see below). The cease drives are used to end gathering activities once the pandaʼs associated energy level has risen above a threshold. These drives are activated by the associated resource energy level, and provide negative motivation to the gathering action for the same resource. Again, if a panda has been eating to counteract its hunger, then the food energy activity will rise with each eat action. As the food energy level rises, it will provide increasing activity to the cease feeding drive, which in turn will inhibit the eat action daemon. Once sufficient food energy has been gathered, the panda will cease feeding and may attend to other drives that may be in greater need. The additional need for cease daemons is created by the interaction of different learning mechanisms within the panda. Specifically, without the cease daemons, pandas will quickly learn by reinforcement that eating (say) generates positive reinforcement. Any other immediate action (say moving away from the food source) will cost energy instead of increasing it, and thus, other things being equal, the panda will feed at the feeding spot until it dies of thirst or exhaustion. The cease daemons provide the additional negative reinforcement that will allow the panda to break out of such loops. Finally, the reproduction drive provides motivation for spawn or mate actions when all resource needs have been satisfied. Re- production is activated by all three resource-level input daemons with a high threshold, and provides activity to the spawn and mate action daemons. Action daemons represent the basic actions. The role of action daemons in the system is limited, since the action selection algorithm operates primarily on episode-action daemons (coalitions consisting of an episode daemon and an action daemon; see below). The primary role of an action daemon in the system is to provide activity to those episode-action daemons that feature the action daemon as a member. Episode-action daemons are the most important daemons in action selection, and thus play a role analogous to action or output neurons in ANNs. Episode-action daemons are also used in the state space mapping and reinforcement learning strategies used by the pandas. Episode-action daemons represent a particular action taken from a particular episode. This action will lead to a (possibly new) resulting episode, and the episode-action daemon is used to connect the earlier episode to the later one. In this way, the network will construct over time a map of the state space, and the actions needed to traverse it. This map is also used to store reinforcement data, using the difference in total energy as the primary reinforcement. Reinforcement data is stored in connections between episode-action daemons and simulates standard Q-learning techniques (see [31]). Furthermore, episode-action daemons are connected to one another with predecessor, successor, and inhibiting connections (see [21]). Predeces- sor connections spread activity from an episode-action daemon e back toward episode-actions that might be taken to make e executable (that is, to episode-action daemons that lead to the episode in the coalition e). Successor connections operate similarly, but by spreading activity forward to episode- actions that will become executable by taking e. Inhibitor connections spread negative activity from an episode-action daemon to others that would prevent this episode-action from becoming executable. These connections are used primarily to spread activity through the episode-action layer, creating both paths leading from the current episode forward and paths leading from goal episodes backward. The activity in these paths is typically greatest along shortest paths and is augmented by the reinforcement connections. This way, paths through the episode space with the highest expected reinforcement over the shortest time will typically have the most activity. The cognitive cycle of a panda begins with taking input from the environment and activating input daemons, and ends with an action selected. Activity spreads from layer to layer in the following order. First, input daemons are activated by the environment. These daemons spread activity to the current

28 Artificial Life Volume 16, Number 1

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

episode daemon (which is created if it does not already exist). Input daemons also provide activity to the drive daemons at this point. Next, the drive daemons provide (or inhibit) activity to action daemons that will satisfy (or frustrate) them. Then episode daemons and action daemons provide activity to any episode-action daemons that contain them (recall that episode-action daemons are special coalitions between an episode daemon and an action daemon). This will ensure two things: Currently executable episode-action daemons in the network will gain activity from the current episode, and goal state episode-action daemons will gain activity from action daemons. At this point, the episode-action daemons that are either currently executable or represent goal states will be the “brightest” in the network. Finally, episode-action daemons spread their activity to each other, creating paths through the network. Generally, unless one episode-action is the clear favorite, this cycle is repeated many times before an action is selected. This allows the network to evolve to the point where the “best” path has the most activity. Finally, an action is selected using a soft max rule applied to the executable episode-action daemons. That is, the activities of all executable episode-action daemons are used to construct a probabi- listic mixture of actions, and one action is selected at random from this mixture. Using the relative activity of the daemons to create the mixture ensures that “better” actions have a higher probability of being selected. The benefit of a soft max rule over a simple max rule is that it will tend to encourage exploration of the environment. When an action has been selected, it is executed in the system and the cycle begins anew with the resulting situation. Pandas can engage in all three types of adaptation. The genome of a panda encodes both “phys- ical” properties (determining how the panda interacts with its environment) and “mental” properties (determining how the panda maintains its internal organization). Using standard mutation and crossover mechanisms, pandas are capable of passing their characteristics to future generations via either the spawn or the mate actions. Most of the ontogenetic systems were described above, but to emphasize, pandas create a state space map from their experience, and this map is used to navigate the actual world. Most of the initial parameters of the pandaʼs network are encoded in the pandaʼs genetic code, and thus are subject to phylogenetic adaptation. The reinforcement learning and state space mapping functions of the network are ontogenetic mechanisms. In our control group of pandas these ontogenetic mechanisms are non-socially oriented. Our imitating pandas are equipped with additional daemons that enable cultural transmission. In particular, they have input daemons that directly represent the actions of others. For each action a in Act there is an additional input daemon See Ac- tion a. These additional daemons are then connected (via an imitation drive daemon) to the action daemon a in the panda. This means that if an action is observed, it will motivate the panda to take the same action. This additional motivation is not guaranteed to produce the same result in the observing panda, but it does provide an increased chance that the panda will imitate those in its environment. Other than this functionality, the imitating pandas are identical to non-imitating pandas.

4 Experimental Setup

To study the effects of the imitation drive on our populations of pandas, we have gathered two sets of experimental data. The first set consists of data gathered from pandas that are mere ontogenetic learners. These pandas have no imitation drive and are unable to perceive directly (through input daemons) the actions of others (though they can still indirectly perceive the results of these actions insofar as they are reflected in the observable environment). This data will serve as control data against which we will measure the performance of our imitating pandas, for which data is gathered in a second set. In our experiment we wish to assess the frequency of success in the population as opposed to the level of success (which is common in other experiments). To do this, we must first select a criterion for success that can be measured easily and absolutely (each panda is either a success or a failure). As is common in animat experiments, we use the life span of our pandas as an indicator of their success. To measure frequency of success we must be able to assess the moment (i.e., the simulation round)

Artificial Life Volume 16, Number 1 29

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

in which a panda reaches success, so that we can easily assess the length of time between occurrences of success. To do this, we introduce a parameter of our simulation that we call the elder age. The elder age is an arbitrarily selected age (measured in rounds) at which we assign success. A panda that reaches this age before dying is considered successful, and the round at which it reaches the elder age is used for the measurement of frequency. Since there is no choice of value for the elder age parameter that is optimal a priori, we conduct parallel assessments over a range of possible values. For the current experiment, the elder age values used are those in the range {1000, 2000, . . ., 10,000}. Clearly, the learning task assigned the pandas increases in difficulty as the elder age is increased (i.e., it is more difficult to reach age 10,000 than to reach age 1000). The experiment is conducted as follows. A run is seeded with a single Adam panda whose genome is determined by an XML file. The genome of all Adam pandas across runs (in both data sets) are identical (up to possible mutations and necessary difference between imitators and non-imitators). It is the responsibility of each Adam panda to populate the environment. It is common for an Adam panda to fail to live up to this responsibility, either by failing to produce offspring before its death, or because its lineage ceases with the extinction of the species. These runs can be categorized into false starts and data-bearing runs. False starts are runs in which no frequency data is gathered relative to any elder age value. These runs are ignored and are not reflected in the data presented below. Data-bearing runs, though possibly resulting in the extinction of the lineage, have produced at least two elders relative to a particular elder age. That is, the Adam panda or one of its offspring lives to be considered a success relative to the smallest elder age value (1000 in the current experiment), and at least one other panda (born after the first elder, but not necessarily a direct descendant of the first elder) also reaches this elder age. In this case, the number of rounds between these elder events is measured and recorded. Thus, such a run has generated data relative to at least the smallest elder age value. False start runs are quite common (up to 90–95%—data not shown), due to the steep demands placed on the Adam panda in the motherhood state, but among data-bearing runs it is common for a successful population to emerge. A successful population is one that will apparently run forever (of course, we say “apparently,” since we must eventually stop all runs). We choose to stop a run artificially (that is, when there are still pandas living in the simulation) if we have gathered a fixed amount of frequency data for the largest elder age (10,000). Runs were conducted sequentially, ending either nat- urally with the extinction of the lineage or artificially as just described, until we had gathered a sufficient amount of data for analysis at all elder age values. In our experiment, we were forced to introduce a maximum population size at any particular moment (10 pandas in this experiment). This was necessary for two reasons. First, the computational demands of our prototype pandaemonium control system are somewhat high, and a maximum population keeps these demands reasonable. Aside from this, there were experimental reasons to keep the population low for the imitating pandas. The more pandas in a particular area, the greater the effect of the imitation urge on each of them. Thus, an overpopulated environment results in significant noise for our imitators. The maximum population size ensured that this noise was not too overpowering. To enforce the population limit we merely blocked all replication actions when the population was at its maximum. This did not prevent the pandas from carrying out replication actions, but these actions had no results (neither creating new pandas, nor exacting an energy cost when applicable). Thus, when the maximum population was reached, no new pandas were created until at least one of the existing pandas died. The necessity of this limit led to other needs as well. The first was the need for a pruning age. Since we did not want the population limit to lead to stagnation in the population, we forcibly killed pandas when they reached the venerable age of 20,000 rounds. We chose this high value for the sake of the imitating pandas, allowing elders of age 10,000 to remain in the population, and thus giving other pandas the opportunity to pick up their successful behaviors through imitation. The interaction between these two necessitated adaptations can be seen to introduce noise into the frequency data that we gather. Consider a state in which the maximum population is reached and sustained for x rounds. During these rounds all potential births are blocked and so no potential elders can be born. This can introduce an artificial increase of up to x rounds between potential elders. Sensitive to this potential problem, we

30 Artificial Life Volume 16, Number 1

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

elected to use a secondary method of measuring the frequency of success that would not be subject to this noise. That is, we also elected to measure the number of births between successive elders. This measure of frequency corresponds to the frequency of success within the population rather than over time. Notice that the artificial periods in which births are blocked will not affect this measure of frequency, since no pandas are born in these periods. In summary, we record both the number of rounds and number of births between elder events for the elder age values ∈ {1000, 2000, . . . , 10,000}. From this data we can perform simple statistical analysis. We chose to consider the medians of these data sets rather than the means and standard deviations, due to the asymmetrical distribution of the data, where most data points lie close to 0. We considered two possible outcomes of our experiment, yielding two hypotheses. The first hypothesis is the positive one which suggests that due to the cultural transmission made possible by our imitation mechanism, we will see the emergence of a culture. The possibility for observably distinct cultures in our experiment is quite small, due to the simplicity of both our pandas and their environment; however, evidence for the positive hypothesis would be increased frequency of success in imitating pandas over non-imitating pandas. The competing hypothesis is negative in that it would suggest that the imitation mechanism we have chosen is insufficient to support true cultural transmission, and instead merely introduces (detrimental) noise into the other ontogenetic processes. Evidence for this noise, and thus the negative hypothesis, would be decreased frequency of success in imitating pandas with respect to non-imitating pandas. Both of these hypotheses are tested against the null hypothesis, which states there are no statistically significant differences in frequency of success between our two groups of pandas.

5 Data

Figures 2 and 3 present the median frequency of success as measured in rounds and births, respectively. Since the data displayed a near-linear correlation with the elder age value (especially in the case of imitating pandas), we have chosen to normalize the data. To create the normalized frequency data, we perform the following steps: (1) take the median number of rounds (births) between elders, (2) divide the median by one-thousandth of the elder age (e.g., the median for elder age value 2000 is di- vided by 2), and (3) take the reciprocal of the resulting value to convert into a frequency measurement. This allows the differences between the imitating and non-imitating pandas, as well as trends within a single data set, to be better visualized. At first glance, the data clearly supports the positive hypothesis. The median frequencies for imitating pandas are consistently higher than the frequencies for non-imitating pandas. Statistical analysis shows

Figure 2. Elders per 1000 rounds (normalized).

Artificial Life Volume 16, Number 1 31

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

Figure 3. Elders per 10 births (normalized).

that most of these higher frequencies are significant for a = .05 and many are significant for lower values of a (Figures 4 and 5 display the significance values for each set of data). In general, this supports the hypothesis that imitation can lead to cultural transmission and thus to higher frequency of success among a population. This supports the common wisdom about imitation. What may be more inter- esting, however, are the instances where this wisdom fails. For elder age 1000, we note that the data is not statistically significant. In this case, the null hypothesis is not rejected, and thus the evidence does not support the positive hypothesis. We suggest that this is due to the simplicity of the learning task assigned to the pandas. That is, we suppose that the learning task is simple enough that mere ontogenetic learning, or even just good luck, is sufficient to lead to success. We hypothesize that when this is the case, the imitation does not provide sufficient additional help in performing the task. This might suggest why imitation is not more prevalent in the animal kingdom. Indeed, if this is the case, it would support recently popular hypotheses in primate cognitive ethology that posit difficult tasks (i.e., complex foraging and social interaction) as the primary selection pressure leading to the adaptation of mechanisms of cultural transmission and primate intelligence [3, 19, 36]. Though further investigation is needed to confirm these suspicions, we can see this as also supporting the common wisdom about imitation. That is, imitation can lead to higher frequency of success at specific learning tasks, but typically only when the learning task is sufficiently difficult.

Figure 4. Round frequency significance.

32 Artificial Life Volume 16, Number 1

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

Figure 5. Birth frequency significance.

More difficult to explain is the lack of significant differences in frequency (as measured in rounds) for the high elder age values of 9000 and 10,000. We consider two possible explanations. The first is that the noise mentioned above, introduced by the maximum population, causes the discrepancies. This explanation is supported in part by the existence of statistically significant differences in frequency when measured in births. The second possible explanation is supported by anecdotal evidence gathered in preliminary testing. We noted that the frequency of success of imitating pandas was considerably lower when successful pandas were pruned too early (indeed, in one test we found the imitating pandas had a statistically significant lower frequency of success than non-imitators—data not shown). That is, if the successful pandas are not allowed to remain in the simulation long enough to pass their successful behavior to others, then this hindered the ability of the population to maintain successful behavior among their members. We hypothesize that the combination of these two effects led to the lack of significant differences in frequency for the high elder age values.

6 Discussion

As mentioned earlier, we are able to give explanations of our pandasʼ behavior from both the traditional information-processing perspective and the situated perspective. Pandas in our simulation have no mechanism for high-level transfer of behaviors; instead they possess only a low-level motivation to imitate those around them. This means any convergence of high-level behaviors must emerge from the dynamics of this low-level rule. Thus it is reasonable to begin our discussion from the situated perspective and focus on what emerges from this simple rule. The first observed effect of the pandasʼ imitation mechanism is a tendency for pandas to group together. The motivation to mimic observed behavior means that, other things being equal, a panda will feel pressure to move in the same direction as pandas in its immediate vicinity. This pressure will be particularly strong in newborn pandas, since they have no experience to motivate them in other directions. Emerging from this tendency is the formation of loose herds of pandas. A panda herd can be seen as a type of distributed organism engaging in distributed cognition in the sense of [18]. The herd, collectively, can occupy a larger area than a single panda and thus can have a larger collective visual range than any individual. In addition, the herd has a distributed decision-making process. Each pandaʼs action is treated as an implicit vote on how those around it should act. The more frequent a particular vote, the more it will contribute to a pandaʼs action selection. Ultimately, the pandaʼs action selection will emerge from the interaction between this social pressure from the herd and its individual motivations. A perfectly stable herd is one in which all the pandas vote (and thus act)

Artificial Life Volume 16, Number 1 33

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

the same, moving in unison from resource to resource. Most herds do not occupy such ideal conditions, instead consisting of pandas voting (and acting) differently. As an example of competing motivations in the herd, we will consider a large herd that is completing its gathering activities at a resource site. The first panda in the herd whose motivations would normally be sufficient to break the gathering cycle is subjected to social pressure exhibited by the other pandas that are still motivated to gather. Soon, however, its own natural motivations become more powerful than the imitative motivation and the panda moves away from the resource cell. Once this occurs, its dissenting vote is registered by pandas in the herd that are still gathering. Among these pandas, some will still want to gather, whereas others might be ready to move on, yet remain under the thrall of the herd. Both groups of pandas will gain motivation from the dissenting panda to also dissent, but since it has only one vote, it will only pull those pandas that are close to the edge. Slowly, a cascade of dissenters will follow the first away from the lure of the resource site, possibly leaving a small group of gatherers be- hind. During a slow migration, the herd is seen to move slowly away from the resource, while individual pandas are seen shuffling back and forth. A herd seeking new resources operates a distributed search. Individual pandas will venture out in various directions, motivated by either experience or exploration. Resources found by the leading edge of the herd act as attractors for individual motivations, and this transfers to the herd via imitative motivations. During the search, the herd can dissolve into smaller parts due to conflicting motivations and random wandering. Lone wanderers may succumb without the security of the herd, though typically they soon find themselves within range of another herd. Since resource sites serve as attractors, they also serve as a common place for pandas to gather into a new herd and for smaller herds to merge. The distributed decision making of the herd insulates it from ignorance and error. Pandas that wander away from a resource prematurely are unlikely to gain a large following and commonly wander back. Pandas making random exploratory jaunts will find it hard to offset the momentum of the herd. Con- versely, experienced pandas, or merely lucky pandas, will be able to draw the herd in the direction of the next resource even when it is out of sight. Inexperienced pandas in the herd benefit from the experience distributed among the other members of the herd. Experienced pandas benefit less in this way, but do still benefit from being insulated against their own errors and explorations. These benefits may alone be enough to account for the greater frequency of success observed in the data; however, this was not the only observed effect of the motivation to imitate. In addition to herding, an effect similar to stimulus enhancement occurred in a number of cases. Pandas that witness other pandas gathering a resource are motivated to gather that same resource, even if they did not have sufficient natural motivation. As mentioned above, this led to long gathering sessions in large herds. Similar to this was an effect on reproductive behavior. The simplest reproductive behavior was to spawn: to asexually create a clone offspring at a marginal energy cost. Alternatively, if two pandas were to coordinate their behavior to mate with each other, they could sexually create an offspring with genetic crossover at no energy cost. In early stages of the simulation, traditions of repeated spawning were witnessed, where others in the herd spawned in response to the influence of the herd. These spawning explosions would stop with the population cap and as pandas ran out of energy. Later, in successful populations, this tradition of spawning changed to a tradition of mating. This can be explained as the result of two pressures. First, the energy cost of spawning would negatively effect the reinforcement learning algorithms in pandas that had spawned often. This is a natural pressure for older pandas to shift to mating instead of spawning. Secondly, once the mating practice became favored by some pandas, it would serve as social pressure for other pandas in the herd to also mate. Once a tradition of mating emerged, it was rare for a tradition of spawning to reemerge except on isolated occasions. The analysis so far has been fine-grained. So far, we have encountered clear examples of social influence, though we have not isolated clear examples of transmission. The low-level motivation to mimic does not have the trappings of traditional transmission. However, at a higher level, we can see that individual pandas tend to share behavior patterns with the others. We can only say this at a high level, since when finely observing their action sequences, pandas rarely behave exactly the same. For instance, we might make the coarse observation that pandas in the herd all traveled from resource

34 Artificial Life Volume 16, Number 1

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

site A to resource site B despite the fact that on fine observation they all took different paths. As is common in traditional ethology, we ignore the fine differences and focus on the coarse similarities. Due to the herding patterns, the behavior of individual pandas converged on high-level traditions. Traditions were maintained in the population due to high-level transmission. This transmission was facilitated by the social coupling that emerged from rapid repetitions of imitative behavior. The traditional question that remains to be answered was on what order of intentionality the pandas operate during occasions of transmission. Recall that, according to Whiten [35], for our mechanism to count as imitation the behavior mimicry must be the result of the imitating panda acknowledging the intentionality of the other pandas. Consider the case of panda P1 following panda P2 from resource site A to resource site B.IfP1 is truly imitating P2, we attribute to P1 the understanding that P2 shares the goal of reaching resource site B and that P2 has the knowledge of how to get there. If P1 is operating under a lower-order approximation to true imitation, then we need not make such sophisticated attributions. To determine which is correct by study of displayed behavior alone, we must construct experiments where the subject is forced into behaviors that either verify or falsify the higher-order hypothesis. Thus, if we are to determine whether they actually do participate in higher-order intentionality when imitating, we must consider such reveal- ing experiments. In this discussion, we will consider only a thought experiment, though one that could easily be carried out. Consider the following test. We wish to determine whether the pandas are really acting in response to their understanding of the other pandas as agents. To do this, we might disembody one of the witnessed behaviors. That is, we might provide input to the panda as though there were another panda in the vicinity performing an action, but without any such panda. For argumentʼs sake we will consider the mate action. If the panda truly operates on the higher order, it should notice that this phantom mate action is not accompanied by an actual mating panda, and thus the panda should choose not to mate. Given the details of how the panda operates, however, if the false signal is sufficiently activated, it will not matter whether there is an actual mating panda present or not; the panda will still mate. This means that the panda does not have the fine understanding we wish to attribute to it, since it cannot distinguish disembodied messages from real ones. Thus, we are confident that our pandas can be tricked by this experiment into “imitating” nothing, demonstrating that they fail to distinguish agents from non-agents. This thought experiment is similar to one that can be conducted with social insects to verify their low-order mechanisms. Many ants signal one another with pheromones. Like our automatic signaling in pandas, these pheromones are automatically emitted by the ants while performing their role in the colony. Other ants that detect the pheromones automatically interpret them as messages suggesting actions. For instance, an ant may find a pheromone trail that it interprets as, roughly, “food source in this direction,” and if the ant is a forager (or foraging at the moment), then it will follow the path. As in our thought experiment, we can demonstrate that these insects are not truly communicating via the pheromone channel, because the order on which the communication operates is only the first order of intentionality. This can be done by taking a cotton swab dipped in the appropriate pheromone and drawing a trail from the colony to some distant point. Despite not being created by another ant, this trail will be just as compelling as a legitimate one for any foraging ants leaving the colony. Though this artificial path will not be reinforced by other ants (as would a legitimate path leading to a food source), it will still trick many ants into following it. From this we conclude that the ants have evolved a complex first-order approximation to a true communication channel, one that can be used without higher-order intentions. This alternative is sufficient to get the ants to benefit from the cultural transmission, but can at best be considered a type of stimulus enhancement. Our conclusion regarding the pandas must be similar. These pandas can easily be tricked by artificial signals introduced into their environment, just as the ants can. This reveals that the pandas also operate via a fragile first-order approximation to true imitation. By having the capacity to directly perceive the actions of others, the pandas can forgo a complex interpretation process necessary for interpreting the actions of others as intentional. Thus, a similar conclusion is warranted regarding pandas, that is, their behavior mimicry is best considered a first-order type of transfer, like stimulus enhancement.

Artificial Life Volume 16, Number 1 35

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

The herding and stimulus enhancement effects may be the primary reasons for the increased frequency of success among imitators. As has been known for some time [27], herding can be accomplished in animals with very simple rules for individual behavior. These simple rules are simpler to evolve and more efficient (in time and energy) than complex rules that would govern true imitation. Since herding can approximate the benefits of true imitation, we can see why herding, flocking, and schooling are so common in the natural world. The benefit to the individual that maintains a close proximity to more experienced conspecifics is great enough to provide evolutionary pressure for this behavior. The Baldwin effect will take over, ensuring that these behaviors are integrated into the genome. Furthermore, once the benefits of herding have evolved in the population, it is unlikely that there will be further, strong pressure to evolve true imitative behaviors unless there is a high probability of cheating. For example, if ants were frequently the victims of copycat insects—insects that spread ant pheromones to lure the ants into their dens, where they would dine on the mindlessly parading treat— then there would be significant pressure for the ants to upgrade their communication mechanisms. However, if such pressure is absent, the first-order approximation to true communication is the most efficient way to gain the advantages of cultural transmission.

7 Conclusion

In conclusion, we have studied the effect of a simple imitation mechanism on the frequency of success within a population of animats. The experiment clearly supports the standard wisdom that mechanisms of cultural transmission can increase the frequency of success in a population. Our mechanism, when scrutinized using ethological categorizations, is best categorized as a first-order mechanism like stimulus enhancement. This is a simple mechanism of cultural transmission not requiring higher-order intentions, but one that we have demonstrated can lead to the emergence of a culture. Further experiments need to be performed to move from this simple mechanism to the more sophisticated mechanisms that primatologists would call true imitation. Such experiments must face the difficult task of attributing higher-order intentionality to artificial agents, something that has never been convincingly done to our knowledge. It is noted that such attributions are still considerably controversial even when applied to primates, the best nonhuman candidates in the animal world. Though this task is difficult, it is presumably not impossible so long as the agents are indeed worthy of higher-order attributions.

References 1. Acerbi, A., & Parisi, D. (2006). Cultural transmission between and within generations. Artificial Societies and Social Simulation, 9(1). 2. Alissandrakis, A., Nehaniv, C., Dautenhahn, K., & Saunders, J. (2006). Evaluation of robot imitation attempts: Comparison of the systemʼs and the humanʼs perspectives. In Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction. Association for Computing Machinery. 3. Byrne, R., & Whiten, A. (Eds.) (1988). Machiavellian intelligence: Social expertise and the evolution of intellect in monkeys, apes, and humans. Oxford, UK: Oxford University Press. 4. Clark, A. (1997). Being there—Putting brain, body and world together again. Cambridge, MA: MIT Press. 5. Clark, A. (2005). Beyond the flesh: Some lessons from a mole cricket. Artificial Life, 11, 233–244. 6. Dawkins, R. (1976). The selfish gene. Oxford, UK: Oxford University Press. 7. Denaro, D., & Parisi, D. (1997). Cultural evolution in a population of neural networks. In M. Marinaro & R. Tagliaferri (Eds.), Proceedings of Neural Nets Wirn-96 (pp. 100–111). Berlin: Springer. 8. Dennett, D. (1983). Intentional systems in cognitive ethology: The “Panglossian paradigm” defended. Behavior and Brain Sciences, 6,343–390. 9. Dennett, D. (1987). The intentional stance. Cambridge, MA: MIT Press. 10. Dennett, D. (1991). Consciousness explained. New York: Little, Brown. 11. Dennett, D. (1995). Darwinʼs dangerous idea. New York: Simon and Schuster.

36 Artificial Life Volume 16, Number 1

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 C. Marriott et al. Imitation as a Mechanism of Cultural Transmission

12. Dennett, D. (2003). Whoʼs on first?: Heterophenomenology explained. Journal of Consciousness Studies, 10,10–30. 13. Dennett, D. (2007). Heterophenomenology reconsidered. Phenomenology and the Cognitive Sciences, 6, 247–270. 14. Denzinger, J., & Ennis, S. (2002). Being the new guy in an experienced team—Enhancing training on the job. In Proceedings of Autonomous Agents and Multi-agent Systems (pp. 1246–1253). 15. Franklin, S., & Graesser, A. (1997). Is it an agent, or just a program?: A taxonomy for autonomous agents. In J. Müller, M. Wooldridge, & N. Jennings (Eds.), Intelligent agents III, agent theories, architectures, and languages (pp. 141–155). 16. Hinton, G., & Nowlan, S. (1987). How learning can guide evolution. Complex Systems, 1, 495–502. 17. Jackson, J. (1987). Idea for a mind. ACM SIGART Newsletter, 181,23–26. 18. Johnson, C. (2001). Distributed primate cognition: A review. Animal Cognition, 4, 167–183. 19. Kummer, H., Daston, L., Gigerenzer, G., & Silk, J. (1997). The social intelligence hypothesis. In P. Weingart, S. Mitchell, P. Richerson, & S. Mausen (Eds.), Human nature: Between biology and social sciences (pp. 157–179). 20. Lindblom, J., & Ziemke, T. (2003). Social situatedness of natural and artificial intelligence: Vygotsky and beyond. Adaptive Behavior, 11,79–96. 21. Maes, P. (1990). Situated agents can have goals. In P. Maes (Ed.), Designing autonomous agents. Cambridge, MA: MIT Press. 22. Mesoudi, A., Whiten, A., & Laland, K. (2006). Towards a unified science of cultural evolution. Behavioral and Brain Sciences, 29, 329–383. 23. Nolfi, S., & Floreano, D. (1999). Learning and evolution. Autonomous Robots, 7,89–113. 24. Nolfi, S., & Floreano, D. (2000). Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. Cambridge, MA: MIT Press/Bradford Books. 25. Paenke, I., Kawecki, T., & Sendhoff, B. (2006). On the influence of lifetime learning on selection pressure. In L. Rocha, L. Yaeger, M. Bedau, D. Floreano, R. Goldstone, & A. Vespignani (Eds.), Artificial Life X. Cambridge, MA: MIT Press. 26. Povinelli, D., Bering, J., & Giambrone, S. (2000). Toward a science of other minds: Escaping the argument by analogy. Cognitive Science, 24,509–541. 27. Reynolds, C. (1987). Flocks, herds and schools: A distributed behavioral model. Computer Graphics, 21,25–34. 28. Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3, 233–242. 29. Selfridge, O. G. (1959). Pandemonium: A paradigm for learning. In Symposium on the Mechanism of Thought Processes. Her Majestyʼs Stationery Office. 30. Shankar, S., & King, B. (2002). The emergence of a new paradigm in ape language research. Behavior and Brain Sciences, 25, 605–656. 31. Sutton, R., & Barto, A. (1998). Introduction to reinforcement learning. Cambridge, MA: MIT Press. 32. Tomasello, M. (1999). The cultural origins of human cognition. Cambridge, MA: Harvard University Press. 33. Tomasello, M., & Call, J. (1997). Primate cognition. Oxford, UK: Oxford University Press. 34. Tomasello, M., Kruger, A. C., & Ratner, H. H. (1993). Cultural learning. Behavioral and Brain Sciences, 16, 495–552. 35. Whiten, A. (2000). Primate culture and social learning. Cognitive Science, 24, 477–508. 36. Whiten, A., & Byrne, R. W. (1997). Machiavellian intelligence II—Extensions and evaluations. Cambridge, UK: Cambridge University Press. 37. Wilson, S. W. (1987). Classifier systems and the animat problem. Machine Learning, 2, 199–228. 38. Yaeger, L. (1994). Computational genetics, physiology, metabolism, neural systems, learning, visions and behavior or Polyworld: Life in a new context. In C. G. Langton (Ed.), Artificial Life III. Reading, MA: Addison-Wesley. 39. Ziemke, T. (2003). What is this thing called embodiment? In R. Alterman & D. Kirsch (Eds.), Proceedings of the 25th Annual Meeting of the Cognitive Science Society (pp. 1305–1310). San Mateo, CA: Lawrence Erlbaum. 40. Zlatev, J. (2001). The epigenesis of meaning in human beings, and possibly in robots. Minds and Machines, 11, 155–195.

Artificial Life Volume 16, Number 1 37

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/artl.2009.Marriott.014 by guest on 01 October 2021