The Cognitive Aptitude of Swarming Agents

H. Van Dyke Parunak and Sven A. Brueckner Vector Research Center of TTGSI 3520 Green Court, Suite 250 Ann Arbor, MI 48105 USA +1 734 302 {4684, 4683} {van.parunak, sven.brueckner}@newvectors.net

Abstract probes this issue. It draws together insights that we have previously published in a wide range of more Swarming models of computation, inspired by focused studies, blending them into a coherent social animals such as ants, are increasingly popular argument that swarming agents are a reasonable and in applications that require decentralized, distributed practical approach to constructing systems with human management of diverse entities in dynamic cognition. environments. However, the relative simplicity of We are not arguing that the human mind in fact digital ants compared with more conventional AI swarms. We strongly suspect that it does, but our claim agents sometimes leads to skepticism concerning the in this paper is much more modest. As builders of level of cognitive sophistication that such a system can computer systems to solve real-world problems, we produce. We explore three lines of argument that recommend swarming as a reasonable way to achieve swarms can execute cognition as well as other cognitive performance at least as acceptable as that computational methods: their intrinsic computational produced by other architectures. Aerospace engineers power, their ability to model the aggregate behavior of design successful airplanes without using the flapping human populations, and their representational wings that dominate natural flight. Cognitive behavior alignment with accepted models of individual need not be limited to architectures that mimic known cognition. Then we address practical issues in cognitive processes in humans. engineering swarms for realistic applications. We begin (Section 2) by summarizing one particular swarming idiom, marker-based stigmergic 1. Introduction coordination, that we have found particularly fruitful. The next four sections develop the argument that swarming agents using this mechanism have high There is growing interest among computer scientists cognitive aptitude, in three stages. in algorithms for distributed decentralized computing Section 3 argues that a population of stigmergic that are patterned on the behavior of social insects. agents is formally at least as powerful as a Turing These algorithms are most commonly used for machine, and thus we should not be surprised if it can applications similar to those addressed by those exhibit cognitive behavior as complex as any other animals, such as path planning (e.g., robot navigation computerized system. In principle, a swarm should be [23], packet routing for communications networks [8], able to do as well as any other computational cognitive TSP optimization[3]), sorting (data clustering [7]), or model at mimicking cognition. foraging (Kennedy’s swarm intelligence [9]). The next two sections make this abstract possibility We have applied these techniques to tasks that more concrete by comparing swarms with human would appear to require a higher level of cognition, cognition at two levels. Section 4 explores how a such as coordination of multiple agents [17] or battle swarm of simple agents can often generate group-level prediction [14]. The success of these systems is at first behavior similar to that of human groups (reflecting glance surprising, given the “obvious” difference in group cognition), while Section 5 explains why a cognitive capacity between ants and people, or swarm can faithfully mimic individual cognition. between simple swarming software agents and more It is one thing to claim that a swarm can exhibit complex agents that model the beliefs, desires, and high cognitive aptitude, and quite another to advocate intentions (BDI) of a human reasoner. This paper the use of swarms (rather than more direct cognitive node or one of its immediate neighbors. By analogy with insect prototypes, we sometimes refer to these variables as “digital pheromones.” The environment executes a set of processes on the fields. These typically include aggregation of contributions from multiple agents (a form of information fusion), propagation to neighboring nodes (a smoothing operation), and evaporation over time (thus discarding obsolete information). To reason over time, we maintain a set of field maps at discrete time steps (a “book of fields”) for the entire graph. Figure 1: Ghosts of one entity augment a labeled Each agent has state and a decision function. The scalar field reflecting their presence, and sense the state includes the agent’s location (typically a node of presence of ghosts of other entities through their the environment, though continuous coordinates are respective fields. also possible), and may also include domain-specific models) to implement cognitive tasks. Section 6 variables such as strength, wealth, or influence. The discusses some of the engineering aspects of executing inputs to the decision function are two vectors: the cognitive tasks with swarming. agent’s local state and the strengths of the fields on the agent’s current node and its neighbors in the graph. 2. Stigmergic Swarming The decision function has three outputs: changes in the agent’s state, the amount by which to augment the We define “swarming” as “useful self-organization various fields on the current node, and the next node to of multiple entities through local interactions” [11]. which the agent should move. Thus the decision This term is applicable to a number of different function maps from scalars to scalars, and is typically mechanisms, including interactions of individually implemented as a set of arithmetic equations, which intelligent agents [26], particle swarms [4], and can combine local field strengths using different coordination fields [10]. For concreteness, we focus the weights or even nonlinearly. discussion in this paper on swarming achieved by This basic machinery can be elaborated in a number marker-based stigmergy. “Stigmergy” refers to of ways. We most commonly use it in the polyagent coordination of multiple agents by means of a shared modeling construct [16], which represents each entity environment that they can sense and modulate. Particle in the domain by a set of agents (thus, a polyagent). A swarms and coordination fields are versions of persistent avatar manages a stream of transient ghosts , stigmergy in which the relevant characteristics of the each of which explores an alternative future for the environment are the locations and states of the agents entity in a simulated world. As the ghosts of different themselves. In marker-based stigmergy, the agents avatars interact via the fields they generate ( Figure 1), deposit and sense markers. they explore alternative futures for their individual A commonly cited example of marker-based entities, complete with the full range of possible stigmergy is the use of pheromones, chemical markers, interactions that might result from the alternative by social insects in tasks such as nest construction and futures of other entities. Each ghost dies after limited path marking. We generalize and formalize this lifetime, so avatars can generate them continuously process for application to real-world problems. Our without the need for garbage collection. systems include an environment, and a set of agents Stigmergic swarming is an attractive architecture that are localized in the environment and move over it. for domains with certain characteristics, including The environment has three components. • diversity of agent types, since the deposition and Structurally, it is a graph (a set of nodes with edges sensing of local scalar variables is a minimal among them). Depending on the application, the edges interface that places very few restrictions on an may be directed (as in a hierarchical task network) or agent’s inner architecture; undirected (as in a lattice representing the tiling of a • decentralized execution, since the feedback loops spatial region). The structure of the graph is an intrinsic to the architecture are conducive to self- important way to encode domain knowledge in a organizing and self-stabilizing behavior without a stigmergic system. A set of scalar variables is single authority; associated with each node in the environment, thus • distributed implementations that enable defining fields over the environment. Agents can applications to scale linearly with problem size, augment or decrement these variables when they are since each agent need interact only with a local located at the node, and sense them when they are at a neighborhood of environment nodes, and each of swarming agents can offer a credible model of the node need deal only with its neighbors and the aggregate behavior of a group of humans. This thesis is agents currently located on it; supported by two lines of evidence: the existence of • dynamic problems, since changes in the real universality in multi-agent modeling, and the observed world can be registered in the environment as they stigmergic behavior of real people. happen and appear the same to the agents as the changes being made by other agents, and the self- 4.1. Universality in Multi-Agent Modeling organizing dynamics of the system adjust to accommodate changed conditions. We use the term “universality” in analogy with its These benefits are attractive, but only if the use in statistical physics, where it refers to a curious architecture has the computational power to satisfy behavior of the critical exponents associated with a problem requirements. We now consider reasons to continuous phase transition. Near the transition believe that it does. temperature Tc, all relevant physical quantities (e.g., magnetic susceptibility, specific heat, isothermal α 3. Swarming Agents and Computation compressibility) vary as | T – Tc| . The exponent α depends on the physical quantity in question, and may Our first argument for the cognitive aptitude of be positive (in which case the quantity vanishes at Tc) swarming agents is that one can construct a Turing or negative (in which case it diverges). Remarkably, machine from a proper subset of the structure defined the values of the critical exponents are relatively in the previous section. Let the environment consist of independent of the material being studied. For a linear graph, equipped with a single binary variable. example, the exponent for the liquid-gas coexistence The aggregation function consists of replacing the curve is the same (about 0.333) for Ne, A, Kr, Xe, N 2, current value with the new one. There is no O2, CO, and CH 4, although these substances differ propagation or evaporation. We have a single agent widely in atomic weight, molecular structure, and the with a single state variable, defined over a finite subset details of their electrochemical interactions. of the natural numbers. Its decision function is a look- Generally, two physical systems will exhibit up table from its state and the value of the field (0 or1) universality if their interactions have the same spatial in its current node, to a triplet: the value to deposit in dimensionality and interaction symmetry, as captured the current node, the new state that the agent should in the Hamiltonian (or generalized energy) of the enter, and which of the node’s two neighbors it should overall system. The aspects of the interaction captured move to next. in the symmetry group neutralize the detailed The Church-Turing thesis asserts that any algorithm characteristics of the molecules and of their can be computed by a Turing machine. Specifically, a interactions, so that the difference between (say) Ne Turing machine is capable of executing any algorithm and CH 4 no longer affects the behavior of the system proposed by a cognitive scientist or AI researcher as near the critical point. We extend the notion of embodying human cognition. Since a single stigmergic universality from its simple, precise meaning in agent can emulate any Turing machine, such an agent statistical mechanics to include any system of can execute any such algorithm. interacting elements whose qualitative or quantitative Strictly speaking, we do not consider a single agent system-level behavior includes characteristics that are as “swarming.” When we allow multiple agents to invariant under changes in the individual behavior and execute over the same environment, we have coupled detailed interaction of the elements. multiple Turing machines. Such a system is known as Universality is a manifestation of Ashby’s Law of an “interaction machine,” and can model ongoing Requisite Variety [1]. As usually stated, this law processes, which are beyond the capability of a single asserts that the greater the variety of actions in a Turing machine [5, 25]. Thus a stigmergic swarm can control system, the greater the variety of disturbances it compute anything that a Turing machine can compute, is able to compensate. A corollary [21] is that the and even more. amount of appropriate selection that a controller can perform is limited by the amount of information 4. Swarming Agents and Group Cognition available. In a physical system, a system’s spatial dimensions and the symmetry of the Hamiltonian limit the information to which the molecules can respond, The formal relation of stigmergic swarming to a restricting the scope of their actions. Stated in this Turing machine is of theoretical interest, but we can form, the application to multi-agent systems is offer much more practical evidence of the cognitive straightforward. If agent interactions limit the aptitude of swarming agents: in aggregate, a population information available to individuals or their ability to 5.1. Swarms can Out-Perform People respond nonrandomly to that information, any sophistication above the level appropriate to the Our polyagent modeling construct [16] is based on environmental information will not make a difference structured stigmergic swarms, and is the heart of a in the system’s behavior, and in fact will introduce battle prediction system that we have tested in noise that can limit system efficiency. wargames in which the decisions of human We have documented universality in multi-agent commanders were played out in a battlefield simulator systems [18], where the system-level results are [14]. The commander for each side has at his disposal a invariant even when the processing details differ team of pucksters, human operators who set waypoints nontrivially. Universality is exploited, though not for individual units in the simulator. Each puckster is named explicitly, in fields such as vehicular or responsible for four to six units. At each turn of the pedestrian traffic modeling. Human drivers and battle, the commanders provide the pucksters with pedestrians perform complex cognitive tasks as they orders, the simulator moves the units, determines firing travel, but models that usefully predict their aggregate actions, and resolves the outcome of conflicts. behavior often reduce their behavior to a small set of We conducted two series of experiments to compare variables, such as a minimum and maximum velocity, our predictions with those made by experienced a minimum acceptable separation between vehicles, military officers who were observing the wargames. and a fixed rate of acceleration and deceleration, or an The first experiment sought to detect the emotional even simpler kinetic gas model [22, 24]. state of the fighters. One puckster responsible for four In summary, in a complex, constrained units is designated the “emotional” puckster. He selects environment, population level behavior may be two of his units to be cowardly (“cowards”) and two to relatively independent of the complexity of individual be irritable (“Rambos”). He does not disclose this agents, and the difference between the processing assignment during the run. He moves each unit capabilities of our simple swarming agents and more according to the commander’s orders until the unit complex agents may be irrelevant to the cognitive encounters circumstances that would trigger the realism of their aggregate behavior. emotion associated with the unit’s disposition. Then he manipulates cowards as though they are fearful 4.2. Human-Human Stigmergy (avoiding combat and moving away from the enemy), and moves Rambos into combat as quickly as possible. Perhaps the ultimate evidence of universality is that Our software receives position reports on all units, in environments with complex constraints, people every twenty seconds. often behave stigmergically, eschewing direct highly Neither swarming and humans can detect Rambos. symbolic communications with relatively simple signs Their risk-prone behavior leads to their deaths before deposited and sensed in a shared environment [12]. they generate a signature that we can detect. However, Though people are capable of much more complex we can predict cowards. We played eleven separate reasoning and coordinating behavior, they themselves wargames, with two cowards in each game, or a total often engage in stigmergic swarming, making the of 22. Figure 2 shows the cumulative number of success of swarming agents in emulating collective cowards detected as a function of the progress of the human cognition much less surprising. Cowards Found vs Percent of Run Time

5. Swarming Agents and Individual 14

Cognition 12

10 Not only can a population of simple agents effectively mimic a human collective, but they can 8

provide a credible model of the cognitive behavior of 6 individual humans. We introduce this point by summarizing empirical evidence that swarming 4 Human Cowards Found (out of 22) 2 agents can perform as well as or better than humans ARM-A

in real-world tasks. Then we show how two widely- 0 respected models of human cognition (dynamic field 0% 20% 40% 60% 80% 100% theory and neural networks) can be mapped onto the Percent of Run Time (Wall Clock) stigmergic model. Figure 2: Swarming vs. Human: Emotion Detection. games. Though humans detect one more coward æ æææ æ than swarming, through most of the game they lag 500 æ æ by about 15 minutes. 400 The second experiment compared the accuracy æ of our predictions of unit locations with those of 300 æ æ æ ææ æææ æ æææ æææææææææ human observers, in runs in which we are told the 200 æææ ææææææææ locations of only 20% of the units at each turn. We use a CEP (“circular error probable”) measure of 100 accuracy, the radius of the circle that one would have to draw around each prediction to capture Swarming H0 Staff H0 Swarming H15 Staff H15 50% of the actual unit locations. The higher the Figure. 3: Box-and-whisker plots of swarming and Staff CEP measure, the worse the accuracy. predictions at 0 and 15 minute horizons. Y-axis is CEP In 18 runs, we generated 1405 predictions at radius in meters; lower values indicate greater accuracy. each of two time horizons (0 and 15 minutes), while computational power of stigmergic swarming by staff generated 102 predictions. Figure. 3 plots the CEP showing that its mechanisms are powerful enough to measures, in meters, of these predictions. Our median construct interacting Turing machines. A more focused CEP score even at 15 minutes is lower (better) than example of this argument observes that swarming either Staff median. The Wilcoxon test shows that the mechanisms correspond point by point to a difference between the H15 scores is significant at the computational model of cognition that is recognized in 99.76% level, while that between the H0 scores is the computational psychology community, the significant at more than 99.999%. Dynamic Field Theory of Busemeyer and Townsend Thus we have empirical evidence that a stigmergic [2]. This theory was developed to account for a number swarm can perform at least as well as, and arguably of experimental features of human cognition. Each of better than, humans in a complex predictive task. its parameters has a counterpart in our stigmergic swarms (Table 1). Thus we could use swarming agents 5.2. Swarms can Emulate Dynamic Field to model any specific scenario that is cast in terms of Theory Dynamic Field Theory. In addition to covering the phenomena that

Dynamic Field Theory was devised to explain, Section 3 offered a general argument for the Table 1: Comparison of Stigmergic Swarm and Dynamic Field Theory Parameter Function Polyagent Feature Direction of preference is based on Weights in field combining function provide δ = mean reward & punishment (weighted differential impact of various environmental valence input differently) factors. Preference strength varies from trial Actions are chosen stochastically, allowing σ2 = variance to trial different results from the same inputs. θ = inhibitory Speed-Accuracy tradeoff: preference Field accumulation from earlier ghosts affects threshold accumulates over trials behavior of later ones. Static fields (e.g., terrain) can bias the behavior of z = initial Preference evolution can be anchored the agents as dynamic fields (e.g., laid down by anchor point at a non-neutral point adversaries) accumulate Serial position effects: the order in s = growth Fields evaporate, forgetting earlier inputs and which stimuli are presented can decay rate giving more prominence to later ones. change the outcome. Nonlinearities in field combining function allow Consequences become more salient as c = goal gradient the importance of certain fields to increase or decision approaches. decrease nonlinearly with their strength. Agents act in time; fields evolve over time; pages h = time unit Preferences evolve over time. in book of field maps record system evolution. stigmergic swarming offers a richer semantics, from the input and output avatars. encoded in the structure of its environment. For The swarming analog to connections between example, our rTÆMS formalism [15] allows the neurons are field paths (sequences of environmental representation of a complex process as a hierarchical cells with higher field levels than their neighbors) task network, over which swarming agents formed by a process analogous to the foraging representing multiple performers can coordinate their behavior of biological ants [6]. Ants have two actions. behaviors, depending on whether they are searching for Thus stigmergic swarming is, in principle, food or carrying it back to the nest, and use two equipotent with dynamic field theory. different pheromone flavors. Searching ants deposit searching pheromone as they move from the nest in a 5.2. Swarms can Emulate a Neural Network random walk that is biased up the gradient of any food pheromone that they sense. Food pheromone is not the As another example of how conventional cognitive smell of food, but a second flavor of pheromone models can be reduced to stigmergic swarming, we deposited by ants that have found food, as they climb outline an approach to constructing a multi-layer neural the gradient of searching pheromone to return to the network using swarming agents. The architecture nest. The interaction of these two sets of behaviors described here has not been implemented, but is leads to the formation of a path between the nest and a offered as a thought exercise to show how more food source. traditional cognitive models can be cast in terms of our In analogous fashion, we imagine that input avatars mechanisms. are the source of a kind of food (“Demand”) desired by Table 2 summarizes the analogies between this output avatars, while output avatars are the source of a construction and conventional neural networks. We different kind of food (“Product”) desired by input describe how to implement each element of a neural avatars. The ghosts of each avatar swarm out to form network using stigmergic swarms, then explain how to paths to avatars of the opposite kind. Input ghosts train and operate such a system. increment local values of the D(emand) field as they Each neuron in the network is represented by a seek to climb the P(roduct) field gradient, and when polyagent, an avatar that can generate a swarm of they reach an avatar that can provide them with ghosts. The avatars are of three kinds, corresponding to Product, they then climb the D gradient back to their input, output, and hidden neurons. home avatar. Similarly, ghosts from output avatars • Input avatars are bound to the network’s input climb D gradients while incrementing the P field, and channels. when they reach a supplier of Demand, return along • Output avatars are bound to the network’s output the P gradient while incrementing the D field. channels. This mechanism by itself will lead to the formation • Intermediate avatars serve the of a network of paths connecting role of hidden neurons, Table 2: Mapping between Sti gmergic input and output avatars, allowing more complex Swarming and Neural Networks analogous to the connections in a functions than can be Neural Net Swarming neural network. The rate of ghost implemented by a two-layer movement along the path network. Neurons Avatars corresponds to signal strength Input and output avatars are modulated by connection weight Connections fixed in number by the Field paths in a conventional network. The btw neurons requirements of the network, and stronger the field gradients along the path, the higher the rate of occupy fixed locations on a Signals Ghosts rectangular lattice. Intermediate ghost transit along the path will avatars are transient. They can be Connection be, corresponding to the strength generated anywhere in the lattice weight / Signal Ghost rate of a signal along a connection in by several different processes: strength a neural network. random appearance at a fixed rate; In a conventional network, Field signals coming into a neuron are initiation by an output avatar that is Input accumulation at summed and the output unable to match patterns in the summation training set; or by lattice cells avatar computed through a (typically based on local field patterns. They sigmoidal ) function. We can do Ghost the same thing with swarming disappear when they are not found Output function generation and exploited by ghosts exploring ghosts. The swarming analog to function summation of inputs is the aggregation of field deposits from 2. It incorporates the Max Rate many ghosts in the vicinity of an physical location of the neurons avatar. The analog to the output into the computation (again in function is provided by sigmoidal keeping with natural neural functions at each avatar that processing). translate from local field levels to 3. It dynamically determines ghost generation rate. Figure 4 both the number and the location illustrates the function at an input of hidden neurons, accommodating avatar. The higher the local level of to the complexity of the required GhostGeneration Rate the Demand field (fixed in the case input-output mapping. of an Input avatar by the level of This construction, while only a the input signal at that node), the Demand Min rate > 0 (ensures exploration) Gedanken experiment at this point, faster it generates Demand ghosts Demand Field Level illustrates that the mechanisms of that spread out, incrementing the D stigmergic swarming are capable Figure 4: Sigmoid function relating field as they seek avatars sending of constructing systems that in out P ghosts. The sigmoid shape of level of D field to generation of other contexts are recognized as the curve defines a maximum and Demand ghosts capable of cognitive performance. minimum level of ghost generation. Importantly, the minimum is greater than zero, ensuring some ghost 6. Engineering Cognitive Swarming generation for the purpose of maintaining exploration even at very low levels of demand. Output avatars have From the perspectives of computational power, a corresponding function, relating level of Product to collective realism, and even representational adequacy, generation of Product ghosts. swarming agents guided by marker-based stigmergy Intermediate avatars have both a Demand and a are fully adequate to model human cognition. In spite Product function. The input to each curve (Demand or of this theoretical equipotency, some important Product) is the current level of the corresponding engineering issues remain. A Turing machine can be pheromone at the avatar’s location. The aggregation of programmed to execute any algorithm, but no one field increments from multiple ghosts corresponds to would think of using the Turing model to write a real- the summation of inputs in a conventional neuron. world application. Roman numerals are fully adequate Intermediate avatars attract ghosts from other avatars, to represent the natural numbers, but no one today forming one or more intermediate layers of neurons. would think of doing arithmetic with them. In the same To train such a network, we clamp the input and way, we need to consider the engineering aspects of output levels for each pattern in turn. The Input avatars implementing cognitive systems using swarming. We send out Demand ghosts at a rate depending on input develop this thought along four lines: our extensive level, while Output avatars send out Product ghosts at a body of experience in constructing real-world rate depending on the clamped output level. swarming systems, a growing body of theory in Intermediate avatars send out both kinds of ghosts at understanding how to engineer swarming systems, minimum levels. The emergent patterns of field paths recognition of the engineering benefits of swarming and ghost traffic along them are analogous to both over more centralized approaches, and the ability to node connectivity and link weight. hybridize swarming and conventional cognition in a The network operates in the same fashion, except single system. that the output nodes aren’t clamped. Training patterns Our experience , since 1984, includes construction can be interspersed with normal operation to adapt the of more than fifty systems based on swarming net to changing requirements. Field strength evaporates concepts. Most are documented in our online over time to forget obsolete paths, but at a rate that is publications.1 These systems include applications in slow compared to presentation rate of training patterns manufacturing (including automotive, semiconductor, and fast compared to the change in set of relevant and shipbuilding), supply-chain management, patterns. intelligence analysis, and military operations. They In addition to the parallels outlined in Table 2, this involve design, information management, command construction has three features not usually present in and control, prediction, and simulation and modeling. artificial neural networks. Swarming applications have gone far beyond simple 1. It replaces signal amplitude with the frequency of ghost traffic (thus more faithfully imitating what happens with biological neurons). 1 http://www.newvectors.net/staff/parunakv/papers.htm demonstrations of ant colony Multiple classical AI systems. behavior. They are doing heavy Processes 4. The modularity of our lifting in complex real-world Coupled Processes code has repeatedly allowed us to human problems. transition seamlessly back and Along the way, we have been Autocatalytic forth between simulation and Potential developing a growing body of field hardware, running the same theory to guide in the practical code in both environments. engineering of complex swarming Finally, we have developed applications. Figure 5 illustrates Function ways to hybridize swarming with three successive restrictions that more classical cognitive must be imposed on a set of mechanisms [20]. We have processes in order to achieve argued that swarming by itself is useful self-organization and fully capable of high-level emergence [11, 19]. Figure 5: Zeroing in on Swarming cognition, but there are often • The various processes Functionality practical reasons to integrate it must be coupled with one another so that with more conventional cognitive architectures. information can flow among them. The field- Different aspects of a reasoning problem can based coordination of swarming stigmergic sometimes be handled more directly in one architecture agents is a particularly efficient mechanism for than the other, as summarized in Figure 6. Each such coupling. swarming entity executes a fairly simple algorithm, • This coupling must form closed feedback loops and the coordination of many such entities allows that allow autocatalysis , in which the processing of large amounts of data, but the underlying interaction of the agents leads to a reduction of reasoning process can be difficult to explain to system entropy in some regions of their state humans. Classical AI allows centralized expression of space. Again, stigmergic interaction is a natural complex reasoning, and its structures are easier for way to ensure the closed loops that provide humans to understand, but it is often intractable for autocatalysis. We have found principles from large amounts of data. We have demonstrated a variety statistical physics particularly helpful in of hybrid architectures in which the two approaches understanding how to generate, monitor, and complement one another, including manage autocatalysis [13]. • Using a swarm as a subroutine called by a • Not all organization that emerges from symbolic reasoner, autocatalysis satisfies the desired function . • Using a symbolic reasoner as a subroutine Techniques to achieve functionality include called by a swarm, generating behavioral diversity, assessing the • Using a symbolic reasoner as a human interface fitness of alternative behaviors, and providing a for a swarm, mechanism for selecting those behaviors that • Combining swarming and symbolic agents to are most fit. provide the symbolic agents with a more Recognizing the benefits of swarming over other technologies often helps justify a Symbolic Reasoning swarming approach to a particular problem. In •Cognitively clear •Combinatorially •Rich Models our experience, four benefits repeatedly emerge. intractable •Good for Exploitation 1. The small modules into which swarming systems are naturally decomposed are BDI provides models, Complexity of Swarming easier to design, test, and maintain than Filtering enables reduces data, Reasoning enables BDI

swarming to Zooming large integrated bodies of code. Volume to zoom in. 2. Stigmergic coordination is robust to filter . of Data small parameter changes, and tends to Focusby degrade gracefully as its operating Focusby assumptions are compromised. • Combinatorially • Cognitively tractable 3. Swarming systems lend themselves to opaque • Abundant Data training against real-world examples • Good for Exploration (via synthetic evolution), a process that Swarming Reasoning is faster and less costly than the explicit knowledge engineering required by Figure 6: Complementary Strengths and Weaknesses of Swarming and Symbolic Reasoning. realistic environment, [5] D. Goldin and P. Wegner. The Interactive Nature • Swarming over symbolic structures produced of Computing: Refuting the Strong Church-Turing Thesis. and interpreted by a symbolic reasoner, Minds and Machines , 18(1):17-38, 2008. http://www.cs.brown.edu/people/dqg/papers/strong-cct.pdf . • Integrating multiple symbolic reasoners through a stigmergic environment. [6] S. Goss, S. Aron, J. L. Deneubourg, and J. M. In our experience, this hybrid approach is much Pasteels. Self-organized Shortcuts in the Argentine Ant. more promising than religious wars seeking to Naturwissenschaften , 76:579-581, 1989. anathematize and then eliminate opposing approaches. [7] J. Handl and B. Meyer. Improved ant-based 7. Conclusion clustering and sorting in a document retrieval interface. In Proceedings of Parallel Problem Solving from Nature (PPSN VII) , Springer, 2002. The simplicity of the computational mechanisms http://dbk.ch.umist.ac.uk/handl/HandlMeyerPPSN2002.pdf . involved in stigmergic swarms and their inherently distributed, decentralized, dynamic nature makes them [8] M. Heusse, S. Guérin, D. Snyers, and P. Kuntz. attractive from a software engineering perspective, but Adaptive Agent-Driven Routing and Load Balancing in often raises questions about their ability to execute Communication Networks. Advances in Complex Systems , complex cognitive tasks. This concern can be allayed 1:234-257, 1998. by considering four observations that we have made over the course of more than two decades of building [9] J. Kennedy, R. C. Eberhart, and Y. Shi. Swarm and deploying such systems. Intelligence . San Francisco, Morgan Kaufmann, 2001. 1. From a pure computational perspective stigmergic swarming is at least as powerful as a [10] M. Mamei, F. Zambonelli, and L. Leonardi. Turing machine. Distributed Motion Coordination with Co-Fields: A Case 2. Stigmergic swarms can faithfully capture the Study in Urban Traffic Management. In Proceedings of 6th IEEE Symposium on Autonomous Decentralized Systems collective cognitive behavior of a population of (ISADS 2003) , Pisa, Italy, IEEE CS Press, 2003. humans. http://polaris.ing.unimo.it/Zambonelli/PDF/isads.pdf . 3. In addition, they can be configured to produce cognitive behavior comparable to that of an [11] H. V. D. Parunak. Making Swarming Happen. In individual human reasoner. Proceedings of Swarming and Network-Enabled C4ISR , 4. A growing body of empirical examples and Tysons Corner, VA, ASD C3I, 2003. engineering principles is available to guide in http://www.newvectors.net/staff/parunakv/MSH03.pdf . the practical implementation of cognitive applications that use stigmergic swarming. [12] H. V. D. Parunak. A Survey of Environments and Mechanisms for Human-Human Stigmergy. In D. Weyns, F. 8. References Michel, and H. V. D. Parunak, Editors, Proceedings of E4MAS 2005 , vol. LNAI 3830, Lecture Notes on AI , pages 163-186. Springer, 2006. [1] W. R. Ashby. Requisite variety and its implications www.newvectors.net/staff/parunakv/HumanHumanStigmerg for the control of complex systems. Cybernetica , 1(2):83-99, y2005.pdf . 1958. [13] H. V. D. Parunak. Monitoring and Managing [2] J. R. Busemeyer and J. T. Townsend. Decision Intelligence in Distributed Systems. In Proceedings of AAAI Field Theory: A Dynamic-Cognitive Approach to Decision 2007 Fall Symposium, Regarding the Intelligence in Making in an Uncertain Environment. Psychological Review , Distributed Intelligent Systems (RIDIS) , Gaithersburg, MD, 100(3):432-459, 1993. AAAI, 2007. http://mypage.iu.edu/~jbusemey/psy_rev_1993.pdf . www.newvectors.net/staff/parunakv/AAAIFS07RIDIS.pdf.

[3] M. Dorigo and T. Stuetzle. Ant Colony [14] H. V. D. Parunak. Real-Time Agent Optimization . Cambridge, MA, MIT Press, 2004. Characterization and Prediction. In Proceedings of International Joint Conference on Autonomous Agents and [4] J. T. Feddema, R. Robinett, D. Schoenwald, C. Multi-Agent Systems (AAMAS'07), Industrial Track , Lewis, J. Wagner, and E. Parker. Analysis and Control for Honolulu, Hawaii, pages 1421-1428, ACM, 2007. Distributed Cooperative Systems. Presentation at Swarming http://www.newvectors.net/staff/parunakv/AAMAS07Fitting. Entities – Joint C4ISR DSC Study Plan Conference , Johns pdf Hopkins University Applied Physics Laboratory, Laurel, MD, 2002. [15] H. V. D. Parunak, T. Belding, R. Bisson, S. the Fourth International Workshop on Engineering Self- Brueckner, E. Downs, R. Hilscher, and K. Decker. Organizing Systems (ESOA'06) , Hakodate, Japan, Springer, Stigmergic Modeling of Hierarchical Task Networks. In 2006. Proceedings of the Tenth International Workshop on Multi- http://www.newvectors.net/staff/parunakv/ESOA06Hybrid.p Agent-Based Simulation (MABS 2009, at AAMAS 2009) , df . Budapest, Hungary, pages (forthcoming), 2009. http://www.newvectors.net/staff/parunakv/MABS09rTAEM [21] Principia Cybernetica. Law of Requisite Variety. S.pdf 2003. Web page, http://pespmc1.vub.ac.be/ASC/LAW_VARIE.html . [16] H. V. D. Parunak and S. Brueckner. Concurrent Modeling of Alternative Worlds with Polyagents. In [22] M. Pursula. Simulation of Traffic Systems - An Proceedings of the Seventh International Workshop on Multi- Overview Agent-Based Simulation (MABS06, at AAMAS06) , Hakodate, Journal of Geographic Information and Decision Analysis , Japan, Springer, 2006. 3(1):1-8, 1999. http://www.newvectors.net/staff/parunakv/MABS06Polyage http://publish.uwo.ca/~jmalczew/gida_5/Pursula/Pursula.htm nts.pdf . l.

[17] H. V. D. Parunak, S. Brueckner, and J. Sauter. [23] J. A. Sauter, R. Matthews, H. V. D. Parunak, and Digital Pheromones for Coordination of Unmanned Vehicles. S. Brueckner. Evolving Adaptive Pheromone Path Planning In Proceedings of Workshop on Environments for Multi- Mechanisms. In Proceedings of Autonomous Agents and Agent Systems (E4MAS 2004) , New York, NY, pages 246- Multi-Agent Systems (AAMAS02) , Bologna, Italy, pages 434- 263, Springer, 2004. 440, ACM, 2002. http://www.newvectors.net/staff/parunakv/E4MAS04_UAV www.newvectors.net/staff/parunakv/AAMAS02Evolution.pd Coordination.pdf . f.

[18] H. V. D. Parunak, S. Brueckner, and R. Savit. [24] M. Treiber and D. Helbing. Macroscopic Universality in Multi-Agent Systems. In Proceedings of simulation of widely scattered synchronized traffic states. J. Third International Joint Conference on Autonomous Agents Phys. A: Math. Gen. , 32:L17-L23, 1999. http://xxx.uni- and Multi-Agent Systems (AAMAS 2004) , New York, NY, augsburg.de/abs/cond-mat/9901119 . pages 930-937, ACM, 2004. http://www.newvectors.net/staff/parunakv/AAMAS04Univer [25] P. Wegner. Interactive Foundations of Computing. sality.pdf . Theoretical Computer Science , 192(2):315-351, 1998. http://www.cs.brown.edu/people/pw/papers/finaltcs.ps . [19] H. V. D. Parunak and S. A. Brueckner. Engineering Swarming Systems. In F. Bergenti, M.-P. [26] F. Weiskopf, T. Gion, D. Elkiss, H. Gilreath, J. Gleizes, and F. Zambonelli, Editors, Methodologies and Bruzek, and R. Bamberger. Control of Cooperative, Software Engineering for Agent Systems , pages 341-376. Autonomous Unmanned Aerial Vehicles. In Proceedings of Kluwer, 2004. First AIAA Technical Conference and Workshop on UAV, http://www.newvectors.net/staff/parunakv/MSEAS04.pdf . Systems, Technologies, and Operations , pages AIAA Paper 2002-3444, 2002. [20] H. V. D. Parunak, P. E. Nielsen, S. Brueckner, and R. Alonso. Hybrid Multi-Agent Systems. In Proceedings of