Market-based Coordination Strategies for Large-scale Multi-Agent Systems ∗ MyungJoo Ham Gul Agha Open System Laboratory, Department of Computer Science, University of Illinois at Urbana-Champaign 201 N. Goodwin Ave., Urbana, IL 61801, USA Email: fham1, [email protected] Abstract: This paper studies market-based mechanisms for dy- get and robot agents combined. However, note that the auctions namic coordinated task assignment in large scale agent sys- that are used in this paper yield sub-optimal assignments [3]. tems carrying out search and rescue missions. Specifically, the Previous research [1][4] and other similar work [2][5] effect of different auction mechanisms and swapping are stud- [6][8][9][10][12][13] studied the problem with small-scale ex- ied. The paper describes results from a large number of sim- periments. However, small-scale experiments can be easily bi- ulations of homogeneous agents, where by homogeneous we ased by specific experimental parameters. Another limitation mean that agents in a given simulation use the same strategy. is that they do not establish the scalability of mechanisms. The information available to agents and their bidding strategies These limitations motivate us to run large-scale simulations are used as simulation parameters. The simulations provide in- in which the strategies and experimental parameters are var- sight about the interaction between the strategy used by indi- ied. We study different auction mechanisms, non-cooperative vidual agents and the market mechanism. Performance is eval- heuristic method, which resembles swarm intelligence [11], uated using several metrics: mission time, distance traveled, as a control and swapping. Different bidding strategies, which communication and computation costs, and workload distribu- weigh the utility, cost, and popularity of a target, are used. Ex- tion. Some of the results obtained include: limiting information perimental parameters are varied in order to test the robustness may improve performance, different utility functions may af- and characteristics of the mechanisms. fect the performance in non-uniform ways, and swapping may Limiting sensing and communication ranges can provide help improve the efficiency of assignments in dynamic envi- more scalability in real applications because of the broadcast- ronments. ing cost. Perhaps more surprisingly, the results suggest that Keywords: auction, market-based approach, multi-agent sys- limiting the ranges improves performance in most of the met- tem, task assignment. rics. In theory, assuming a fixed order of synchronized auctions, 1. Introduction all auction mechanisms would yield the same results. However, in this paper, the assumption is not met and it is easy to see how New types of physical agents being developed include robots, different auction mechanisms may produce different results. small unmanned aerial vehicles (micro-UAVs), and unmanned Two robots may swap tasks to reduce their costs. Dynam- underwater vehicles (UUVs). Such physical agents will be use- icity of the environment and asynchrony of auctions create the ful for surveillance, search and rescue, mine sweeping and need for swapping although swapping causes additional delays other applications. This paper focuses on a two dimensional and messages. search and rescue (SR) problem involving pursuer robots and The rest of this paper is organized as follows. Section 2 de- mobile targets. In SR, the number of tasks will generally ex- scribes previous research on multi-agent coordinations. Sec- ceed that of agents and a task may require multiple agents. tion 3 describes the methods studied. Section 4 describes the Thus, efficient methods, which can enable coordination be- results. Section 5 analyzes these results. Finally, Section 6 dis- tween the agents, are required. Note that the SR problem is cusses the conclusions and directions for future research. computationally intractable. Even a simplified version of the SR problem, the vehicle routing problem [7], is NP-hard. 2. Related Work To address the SR problem, two mechanisms are investi- gated: auctions and swapping–both fully distributed and asyn- In principle, a centralized approach can provide results that are chronous. These methods have reasonable computational com- equal to or better than those of distributed approaches [3][10]. plexity. Each target requires a number of auctions to find robots However, it has a single point of failure and requires fully to serve. In practice, this is a small number. If the communica- connected networks. More critically, it is not scalable. tion and sensing ranges are bounded, the mechanisms require A number of distributed approaches have also been pro- constant time for each round of auctions. If the ranges are un- posed. Most of the proposed mechanisms are market-based. bounded, O(n) time is required, where n is the number of tar- Some of the mechanisms are offline algorithms, i.e., the as- signment is done once at the beginning [13]. Obviously, offline ∗ This research has been supported in part by NSF under grant CNS 05-09321 and by ONR under DoD MURI award N0014-02-1-0715. We thank Liping algorithms cannot adapt to dynamic environments. Other re- Chen and Rajesh Kumar at UIUC, Tom Brown at Google, and the anonymous search has studied online mechanisms [1][2][6][8][9]. Swarm reviewers of the paper for their invaluable comments and advice. intelligence [11], analogous to the behavior of social insects, is of the bidders. An auction of t is finished if it has enough an example of non-market based mechanism. number of bidders (≥reqt) after Tround. Different degrees of dynamicity have been experimented; When an auction for a target agent t is started, the tasks may be static, passive or dynamic. Static tasks do not auctioneer accepts bids higher than pricet of Eq. (1). change their utility or cost: [2][3][6][13]. Passive tasks may be modified only by the action of robots: [5][8][9]. Dynamic tasks ( can be modified by themselves: [1][4]. Preemption to change min (min bid; max rej bid) ; asnt > 0; price = (1) an agent’s attention and adaptation become important in order t max rej bid; otherwise. to let agents respond to the change of dynamic environment. The previous research [1] proposed forward/reverse auction asnt is the number of bidders, min bid is the lowest bid, and swapping for task allocation with physical agents in dy- and max rej bid is the larger value of the highest rejected namic environment. However, the main weakness of [1] is that bids and the initial price. If asnt>reqt, the lowest bid is re- the experiments were not sufficient–only one execution was jected until asnt=reqt. The rejected bidder bids for t again carried out for each method in a small-scale experiment with if it can outbid with the updated conditions. Otherwise, it a single metric (mission time). Moreover, the effect of the bid- searches for another target. ding strategy was not examined. An auction is stopped after Tround, a specified time pe- Distributed multi-robot coordination research based on a ge- riod from the beginning of the auction. If the auction does ographic area usually uses small-scale experiments. The prob- not result in a sufficient number of bidders, the bidders are lem size is extended to show the scalability, test various coor- released and the auction is paused and restarts after a ran- dination methods, and experiment more concretely with more dom interval (random(0:1; 1:0)·Tround) in the experiments. metrics and various experimental parameters. The behavior of After the pause, the auction restarts. This delay allows the agents, such as bidding, cost evaluation, and the dimensions environment to evolve (e.g. more robots to become free). and mobility of agents are configured by the values similar to Reverse Auction. In contrast to a forward auction, where those in [4], which are based on the physical simulations in [1]. robots increase prices to attract targets, in a reverse auc- tion, targets decrease prices to attract robots. The price is 3. Method cut when the auction is paused. The price is also cut if a bidder retracts when asn ≤req . Unlike a forward auction, A number of simplifying assumptions are made. Agents are on t t higher bids do not raise prices. a bounded rectangular Euclidean plane. Agents in a given sim- Forward/Reverse Auction (F/R). Using both forward and ulation are homogeneous. Robots observe every target within reverse auction in order to reduce auction delay with the sensing range and notify others in the communication range equivalent auction results has been proposed by [1][3]. about its own observed targets. Targets move around with pre- Forward/reverse auction is implemented by running for- defined patterns, which are not predicted by robots. Although ward auction during normal operations and reverse auction other algorithms such as roaming algorithms and movement when the auction is paused or a bid is retracted. prediction may improve the performance, they are not studied, Sealed-bid Auction based on F/R Auction. Unlike auctions as their benefits are likely to be marginal and the purpose is to with actual fund transactions, robots do not actually pay focus on the effect of coordination mechanisms. anything to win auctions. Thus, target price may not be In order to be served, a target t requires multiple dedicated a cost factor. Given this, a sealed-bid auction is designed req (>1) robots to be present nearby (<0.2m) at the same t based on a forward/reverse auction. The expected profit is time. t distributes its utility util evenly util =req to each of t t t f (t)−cost(r; t). However, bids are rejected based on the the req robots; the req +1’th cannot receive the payoff. util t t price as other auction methods do. Each instance of the problem is defined as a mission; a mis- sion is complete when every target agent has been served.