From: AAAI Technical Report SS-96-01. Compilation copyright © 1996, AAAI (www.aaai.org). All rights reserved. EVOLUTIONARY COMPUTING IN COOPERATIVE MULTI-AGENT ENVIRONMENTS Lawrence Bull and Terence C Fogarty Faculty of Computer Studies and Mathematics University of the West of England, Bristol, BS161QY, England E-mail: {l_bull, tcf} @btc.uwe.ac.uk Abstract Weuse a two-agent trail following task to demonstrate the new operators. Thefields of Artificial Intelligenceand Artificial Life have The paper is arranged as follows: both focused on complexsystems in which agents must section two describes the simulated task and the rule- cooperateto achievecertain goals. In our workwe examine based frameworkused with the GA. Sections three to the performanceof the genetic algorithm whenapplied to five describe the use of the new operators and finally systems of this type. That is, we examinethe use of we discuss our results. population-basedevolutionary computing techniques within cooperative multi-agent environments.In extending the genetic algorithmto such environmentswe introduce three 2 A Multi-Agent Trail Following Task macro-leveloperators to reduce the amountof knowledge requireda priori; the joining of agents(symbiogenesis), the 2.1 The Tracker Task transfer of genetic material between agents and the speciation of initially homogeneousagents. Theseoperators Jefferson et al. [1989] presented a modelbased on the are used in conjunction with a generic rule-based trail following behaviour of ants, the Tracker task. framework,a simplified version of Pittsburgh-styleclassifier Their artificial ants (neural networksand finite state systems, which we alter to allow for direct systemic machines)must evolve the ability to follow a winding communicationto evolve betweenthe thus represented broken trail across a toroidal rectilinear grid agents. In this paperwe use a simulatedtrail followingtask environment,the John Muir trail. The trail is designed to demonstratethese techniques,finding that they can give improvedperformance. such that it becomesincreasingly difficult toward the end. The ants’ fitness is determined by howmuch of the trail (max89) they cover in a given lifetime of 200 1 Introduction time steps. Oncea cell of the trail is covered it is Nature is full of examples of both inter and deleted to encourage movement.We use these aspects intraspecies cooperation; from the workings of ant of Jefferson et al.’s modelbut alter the ant analogyto that of two single tracked vehicles learning to follow colonies to the cleaning symbiosis seen between the Pederson shrimp and the fish of the Bahamas. The the John Muir trail. Weemphasize that this task is used simply to introduce the new operators. Each vehicle fields of Artificial Intelligence and Artificial Life have consequently focused on these phenomenaas a means has a classifier system [Smith 1980]controller able to detect whetherthe trail is in the two locations in front of dealing with complexsystems in which agents must cooperate to achieve certain goals. In this paper we of it and return one of five actions - turn on forward/ backwardone or two revolutions, or do nothing - at introduce three macro-level operators to enhance the each discrete time step. Onerevolution will movethe use of population-based evolutionary computing vehicle one square in the appropriate direction. This techniques within cooperative multi-agent means that individually the vehicle can only move environments: symbiogenesis, gene transfer and forwards or backwards, intimating that they can speciation. That is, we extend the genetic algorithm potentially only follow any part of the trail whichlays (GA) [Holland 1975] to include processes found in their original path. In this modela pair of vehicles nature which can only be implementedby population- initially stand next to one another on the start position based search techniques within multi-agent systems. indicated in figure 1, facing the trail. Wemaintain that The operators are used in conjunction with a generic whenever both vehicles occupy the same location in rule-based frameworkto represent each agent, which their world, they are next to each other (on the same we alter to allow for direct communicationto evolve. 22 side of each other as at the start), whichthey are also parallel, messagepassing, rule-based systems running able to sense. in discrete time steps. The rules are in the form of The cooperative advantage comes condition/action sets, usually specified over the fromthe fact that by coupling together the vehicles are alphabet {0,1,#} (#represents a match all wildcard), able to turn on the spot, giving them the capacity to with an associated strength. All external input and completethe trail. Weallow that if both vehicles are internal messagepassing is done via a blackboard-like ever at the samelocation, they are coupled, and one’s messagelist. The classifiers use the GAfor their rule motor is on forward whilst the other’s is on backward, discovery; each system is represented by its rules then they turn on the spot in the appropriate direction, strung together to form a traditional GAgenome. To For exampleif the vehicle on the left turns on forward use this framework within multi-agent systems we two revolutions and the vehicle on the right turns on have altered the rule structure so that each action backwardtwo revolutions, and they are coupled, then contains an extra address "tag" and give each classifier they turn right 180° on the spot. Wesay that coupled system agent an address. Thenon a given time step all vehicles cannot drag one another, therefore if one or active rules of all agents also have their actions posted both are off neither vehicle moves.Similarly if in the onto the message list of the classifier system(s) above examplethe left vehicle had turned on forward specified by their tag; direct communicationis able to evolve through an external version of the rule-bases’ I0 message passing mechanism.A full description of the use and results from implementing our framework on this task can be foundin [Bull & Fogarty 1994]; due to limited space the evolution of communicationwill not be discussed here. Wegive each agent twelve bi- conditional rules for this task (unless otherwise stated), resulting in a genomeof 228 genes. We will now introduce our first macro-level operator which effects the identifiable nodes/agents of a given multi-agent system. 3 Symbiogenesis 3.1 A Macro-level Operator Figure 1: The John Muir trail in a 32x32 toroidal grid. Scores for reaching various Symbiogenesis is the name given to evolutionary landmarksare indicated - grey cells are not part of the trail; they markthe fastest innovation through the establishment of symbiotic route of traversal to the reader (they appear associations. Symbiosis is the phenomenonin which whiteto the trail followers). organismsof different species live together, resulting in a raised level of fitness for one or more of the organisms, i.e. heterogeneous cooperation. one° revolution, they would have only turned right 90 on the spot as the right cannot movethe left during its Left Right Left Right second revolution. Each single tracked vehicle scores 0 0 I every time it movesonto an element of the trail and both score if they moveonto an dement at the same time, whether or not they are coupled; trail is only deleted to encourage movement. We use a generational GA, with Figure 2: Over time the population space roulette wheel selection, allele mutation (rate 0.01), of the separate agents can be invaded by multi-point crossover (rate 0.01), with populations the joined configuration (Left&Right). 5000 individuals (unless otherwise stated). Endosymbiosis is the name given to symbiotic 2.2 Evolving Communicating Classifier Systems relationships in whichpartners are contained within a host partner. A large number of endosymbioses are Pittsburgh-style classifier systems [Smith 1980] are hereditary, wherein the host’s endosymbiontsare 23 passed directly to its offspring, e.g. through evaluatedon its ownside of the trail. Agentsare paired transmission in the egg cytoplasm, as seen in insects. with their corresponding opposite for evaluation This joining together of heterogeneous organismscan (left[x] is paired with fight[x], where 0<x<pop_size) be viewed as resulting in a newsuper-organism [Allee and each receive their ownfitness measure. At the end et al. 1949] with the combinedabilities of the partners of the first and each successive generation a macro- involved. This phenomenonis nowwidely accepted as level operator probability Psy is tested to see if a the mechanism by which eukaryotic cells evolved symbiogenesis event will take place (the reverse [Margulis 1970]. process can also occur to stop drift). A symbiogenesis Within multi-agent systems it can event leads to two randomly chosen agents being be the case that someagents are highly interdependent. joined and forming a competingconfiguration (figure Under these conditions the configuration/behaviour of 2), where each super-agent receives the combined one agent has a large, often derogatory [Kauffman fitness of its two parts. Selection under the GAis run 1993], effect on the performanceof one or moreother over the total population"space" as usual except that if agents in the system. Wesuggest that multi-agent a separate agent is chosen to be created a systems containing these kinds of agents will benefit complementarypartner offspring is also created to froma joining process since they will no longer suffer allow for the fact that joined agents have a combined from the oscillatory dynamics of selfish interest; fitness rating and that an equal numberof left and fight joined agents will act as one super-agent and will agents must exist. Duringthe application of the usual therefore maximise their performance for both micro-level GAoperators of recombination and allele mutation joined genomes of the chosen parents are md0~l~lc og~t~ 90.0, i , I ’ i ¯ i ’ treated as if they were separate; each child of the G---Obolt 80.0F- o--io meon joined species is a mix of the corresponding genomes 70.01- carded by its parents.