Performance Evaluation of a Multi-Robot Search & Retrieval System: Experiences with MinDART Paul E. Rybski, Amy Larson, Harini Veeraraghavan, Monica LaPoint, and Maria Gini Department of Computer Science and Engineering, University of Minnesota, 200 Union St. S.E., Minneapolis, MN 55455-0159 Abstract. The costs of developing robot teams can be reduced if they are designed to exploit swarm techniques. In this methodology many simple homogeneous units are used instead of more complex ones. The challenge lies in selecting an appropriate control strategy for the individual units. Complexity in design costs both money and time, therefore a control strategy should be just complex enough to perform the task successfully in a variety of environments, relative to some performance measure. To explore the effects of control strategies and environmental factors on performance, we have conducted two sets of foraging experiments using real robots (the Minnesota Distributed Autonomous Robotic Team). The first set of experiments tested the efficacy of localization capabilities, in addition to the effects of team size and target distribution. The second set tested the efficacy of simple communication. We found that more complex control strategies do not necessarily improve task completion times, however they can reduce variance in performance measures. 1. Introduction Designing a distributed robotic system using swarm techniques, whereby simple homogeneous units solve a complex task, is an attractive engi- neering solution for many reasons [6]. Analogous to software design, there are obvious advantages to this modular approach, including re- ducing a complex task to a simpler, more manageable one. There also arises a natural redundancy in the resultant system, as well as the abil- ity to scale to task with minimal tractability issues [14]. The difficulty lies in determining an appropriate robot control strategy for a given task. Complexity comes at a price, both in money and time, and by minimizing the complexity, thus the cost, we hope to create a simple, c 2003 Kluwer Academic Publishers. Printed in the Netherlands. paper.tex; 12/07/2003; 0:43; p.1 2 Rybski, Larson, Veeraraghavan, LaPoint, Gini efficient, and tractable system. One application that highlights this point is explosive ordinance disposal (EOD) using teams of mobile robots [27]. Since this application is potentially hazardous to the robots (the explosives could detonate near or inside the robot), the key design issue is a very cheap and disposable robot that still accomplishes the task. In a general sense, we believe this is a difficult issue to address, because, not only is the efficacy of a control strategy dependent on the specific task, but also on the environment in which the task is performed. Additionally, care must be taken to avoid highly inefficient solutions caused by redundant actions taken by each of the individual agents. We are interested in determining what level of improvement of task performance we can expect relative to the sophistication of the control strategy (i.e. the capabilities of the robot), and how this is affected by team size and by the environment in which the robots work. To explore this question, we built simple robots with simple control strategies to perform a task (foraging). Then, we enhanced our robots with capa- bilities (i.e. localization and communication) that we believed would improve the robot's performance. For comparison, we conducted a se- ries of experiments with these real robots, the Minnesota Distributed Autonomous Robot Team (MinDART) shown in Figure 1. Although there are many tasks that can serve as a testbed, we chose foraging, a task studied by many other researchers, and on which solutions and results can be compared more easily. In our version of the task, robots locate a target in an enclosed arena, pick up the target, and then drop it off at a designated home base. The arena contains some obstacles, and the distribution of the targets varies. The task is complete when all the targets have been collected. The simplest control strategy for foraging is random walk, in which the most difficult implementation issues are obstacle avoidance and maximal coverage of the search arena. Reactive behaviors can be used to avoid obstacles and random direction changes at random intervals can increase the probability of complete coverage. This strategy is an attractive choice for very simple robotic hardware because it can be programmed with only a few lines of code and requires little sensor paper.tex; 12/07/2003; 0:43; p.2 MinDART : A Multi-Robot Search & Retrieval System 3 Figure 1. The Minnesota Distributed Autonomous Robotics Team (MinDART) with the infrared targets in front and colored landmarks in back. The MinDART robots searched for the infrared emitting targets in a search and retrieval task. Landmarks were used for homing and localization. bandwidth. A more complex control strategy can reduce the random- ness of the robot's search, but it will require additional time to process information necessary for a deliberate search. However, the advantage of this processing should be a decrease in variance of the time to complete the task. In our first set of experiments, we considered the efficacy of using localization versus random walk, in addition to the performance effects of team size, and target distribution. In the second set, we compared control strategies using communication (a simple recruiting mecha- nism) against random walk. In this set of experiments, we evaluated how the duration of the communication affected the performance of the team. Our experimental results show that for simple robots such as the MinDART, deliberative strategies, which can help reduce the amount of time a robot randomly searches its environment, do help in decreasing the variance of the team's performance. However, this decrease in vari- ability comes at the expense of not improving the mean time to solve the paper.tex; 12/07/2003; 0:43; p.3 4 Rybski, Larson, Veeraraghavan, LaPoint, Gini task. Instead of spending time wandering randomly, the robots spend time deliberating either on determining their locations or on recruiting other robots. By doing the latter, more consistent performance can be achieved. We would like to emphasize that all of our experiments were con- ducted with real robots. We contend that a rigorous study of system design warrants physical robots, as opposed to simulated, to examine the unforeseen effects of an embodied control strategy. We will further discuss this issue of real versus simulated in the following section. 2. Related Work Most research with multiple robots has focused on various forms of col- laborative work as detailed, for instance, in [3, 8, 12]. While collabora- tion may be essential for some tasks, we are interested in studying tasks that can be done by a single robot, but where using multiple robots can potentially increase performance by decreasing the time to complete the task and/or by increasing the reliability. Sample tasks include mapping a large area [36], placing a distributed sensor network [13], cleaning up trash [28], and box-pushing [19, 25]. Foraging is a widely used testbed application for distributed systems, for example [9, 11, 15] and our previous work [30, 31]. In addition to these experimental studies, researchers have developed predictive models of foraging behavior. In [24], a probabilistic model was devel- oped and verified using simulation and some real robot experiments. Similarly in [22], a mathematical model was developed and used to study the effects of interference among robots. Mathematical models called optimal foraging theory are used to model foraging behaviors of animals [10, 33]. The effect of group size on performance has been well studied. For instance, [16] presents a quantitative analysis of the tradeoffs between group size and efficiency in collective search tasks. The analysis can be used to predict the optimal number of robots required to com- plete a task in the most efficient way. The study was done only using simulation. paper.tex; 12/07/2003; 0:43; p.4 MinDART : A Multi-Robot Search & Retrieval System 5 There have also been a handful of studies to evaluate the efficacy of communication strategies applied to the foraging task. Balch and Arkin [4] conducted an extensive investigation into the impact of vari- ous communication strategies on three separate tasks, including forag- ing. Communication strategies were based on indirect communication based on cues from the environment. (This form of communication, called stygmergy in the biology literature, is commonly used in robotics, for example in [2, 5, 26].) Sugawara et al. [34, 35] also looked at the effects of indirect communication in regards to collection patterns and team efficiency using both experimental results and a mathematical model. The conclusion of both is that communication can improve performance but that the length of the communication strategy greatly affects the performance. There is a critical duration at which the per- formance is maximized. Any duration greater or less than this will only serve to decrease the performance of the team. It is reasonable to assume that communication will assist in foraging, since it is a strategy that has evolved in nature. It is widely known that bees \dance" to communicate the direction of pollen sources [32] and ants communicate the location of prey with pheromone trails [17]. To our knowledge, biologically-inspired communication strategies for foraging on small scale robots have yet to provide performance im- provements as predicted by the above mentioned work. In those studies, the majority of the experiments were conducted in simulation and were not fully implemented on real robots. Simulation is important to establish the potential of control and communication strategies, but our results serve as a caution to designers, that simula- tion may be deceiving. Easton and Martinoli [14] had similar findings in a stick pulling experiment using Khepera robots and its simulation environment Webots.