Stigmergic Landmark Foraging
Total Page:16
File Type:pdf, Size:1020Kb
Stigmergic Landmark Foraging Nyree Lemmens Karl Tuyls Faculty of Humanities and Sciences, MICC, Faculty of Industrial Design, Eindhoven Maastricht University University of Technology P.O. Box 616, 6200 MD P.O. Box 513, 5600 MB Maastricht, The Netherlands Eindhoven, The Netherlands [email protected] [email protected] ABSTRACT through unfamiliar worlds. Instead, for navigation, they use a strat- In this paper, we describe a nature-inspired optimization algorithm egy named Path Integration (PI). Bees are able to compute their based on bee foraging behavior. This algorithm combines the high present location from their past trajectory continuously and, as a performance of bee path-integration navigation with ant-like stig- consequence, can return to their starting point by choosing the di- mergic behavior in the form of landmarks. More precisely, each rect route rather than retracing their outbound trajectory [15, 21]. individual landmark can be created at any walkable state in the For recruitment, bees employ a direct strategy by dancing in the environment and contains a collection of direction markers with nest. Their dance communicates distance and direction towards a which visiting agents can find their way in an unknown environ- destination [27]. ment. A landmark can either be represented by an agent or any In previous research, we introduced a foraging algorithm in- other information distributing object (e.g., a RFID). Essentially, we spired by bees and compared it to an Ant System [17]. Our compar- implement ant recruitment behavior based on pheromone. How- ison showed that the bee-inspired, non-pheromone-based algorithm ever, instead of using attracting or repelling pheromone in every clearly outperformed the ant-inspired, pheromone-based algorithm state of the environment, we only update directional information at in relatively unobstructed environments; more precisely, the bee al- key locations in the environment. The resulting algorithm, which gorithm was able to collect all items present in the environment in we call Stigmergic Landmark Foraging (SLF), proves to be very about three times less time steps than the ant algorithm [17]. How- efficient in terms of building and adapting solutions. ever, we also found that in more constrained and/or dynamical en- vironments, such as the Deneubourg Bridge [10], the bee algorithm tends to get stuck behind obstacles [17]. Unlike the ant algorithm, Categories and Subject Descriptors the bee algorithm is only able to learn from any mistakes at one Swarm Intelligence [Multi-Agent Systems]: Landmark navigation location (i.e., the hive) and as such is less adaptive. To account for the adaptability problem we introduced inhibition Keywords pheromones, resulting in the algorithm ‘Bee System with inhibi- tion Pheromones’ (BSP) [16]. More precisely, agents can place Swarm Intelligence, Multi-agent Systems, Landmark navigation pheromones in cells that can be considered undesirable, for in- stance cells that lead to a dead end. In other words, this algorithm 1. INTRODUCTION is a hybrid, integrating some features of Ant Systems into a bee- In previous work we presented Bee System as an alternative to inspired algorithm. The major advantage of this algorithm is that Ant Colony Optimization (ACO) in foraging domains [17]. Forag- its performance is equal to our initial Bee System algorithm while ing is an interesting problem in the domain of Multi-Agent Systems being at least as adaptive as the ant-inspired algorithm. However, (MAS). The problem entails that a group of agents has to gather the major downside of this approach is that inhibition pheromones objects in a complicated, potentially dynamical environment. Ants need to be simulated somehow, which is not a trivial issue, espe- deposit pheromone on the path they take during travel. Using this cially in embodied systems. trail, they are able to navigate towards their nest or food. Ants em- Therefore, in [18], we explored the possibility of using land- ploy an indirect recruitment strategy by accumulating pheromone marks instead of pheromones. More precisely, we extended our trails. When a trail is strong enough, other ants are attracted to it previously introduced Bee System [17] with landmark navigation (i.e., recruitment) and will follow this trail towards a destination. with which agents can learn landmarks that together function as This is known as an autocatalitic process. More precisely, the more routes. Each landmark represents a segment in the total route [6] ants follow a trail, the more that trail becomes attractive for being and indicates what follow-up action to take to get to the next seg- followed. Short paths will eventually be preferred. Since ants fol- ment. Moreover, each agent is able to contribute to the success low pheromone trails towards their destination, it may be clear that of a landmark route. The more agents follow a route, the more the ant’s navigation strategy is also guided by pheromone [11]. that route becomes attractive for being followed. Summarizing, In contrast, non-pheromone-based algorithms are inspired by the [18] presents a stigmergic landmark algorithm. With respect to behavior of (mainly) bees and do not use pheromones to navigate pheromone-based algorithms, such a solution is easier to realize in the physical world since landmarks are either already physically CiteCite as: StigmergicStigmergic Landmark Landmark Foraging, Foraging, Nyree Nyree Lemmens Lemmens, and Karl Karl Tuyls, Tuyls, available, or could easily be placed, in the environment by means Proc.Proc. of 8th8th Int.Int. Conf. on Autonomous Autonomous Agents Agents and and Multiagent Multiagent Systems Sys- of, for example, RFID tags. In comparison to BSP, the algorithm is tems (AAMAS 2009), Decker, Sichman, Sierra and Castelfranchi (eds.), (AAMAS 2009), Decker, Sichman, Sierra and Castelfranchi (eds.), May, at least as efficient when applied to static environments. However, May, 10–15, 2009, Budapest, Hungary, pp. XXX-XXX. Copyright10–15, 2009,c Budapest,2009, International Hungary, Foundationpp. 497–504 for Autonomous Agents and due to its higher computation time, the algorithm has longer run MultiagentCopyright © Systems 2009, International (www.ifaamas.org). Foundation All rightsfor Autonomous reserved. Agents and Multiagent Systems (www.ifaamas.org), All rights reserved. 497 AAMAS 2009 • 8th International Conference on Autonomous Agents and Multiagent Systems • 10–15 May, 2009 • Budapest, Hungary times. familiar route and known global landmarks are present. Thus, bees In this paper, we present Stigmergic Landmark Foraging (SLF) preferably attend to global landmarks [26]. which extends the algorithm from [18] so that it can be applied In recent years, researchers have started to use landmark nav- in dynamic environments. The extension addresses the route-loop igation behavior to create robots which can navigate in unknown problem that may emerge in dynamic environments. More pre- environments autonomously. In [1], a single robot is presented cisely, due to landmark decay, a route may be broken. When an which builds an internal map of the environment in order to nav- agent relinks the route it may be attached to a previous visited igate through it. Although the robot’s control system is designed landmark which may result in a route loop. Moreover, we test a as a MAS, it is solely used to determine which control system (for new recruitment heuristic in order to further improve efficiency. To example, the pilot system) of the robot get access to the robot. By show the performance of SLF, we compare it to BSP in a set of ba- using a bidding mechanism the most urgent control system gets sic static experiments. Furthermore, to illustrate the performance access. In [12, 13], a single robot is able to navigate through an en- difference with the new heuristic, we apply SLF to a set of environ- vironment based on image matching. The robot is allowed to take ments with multiple, different quality goals. In such experiments, snapshots (i.e., images) while recording positional information. By quality may represent goal priority. A high-priority goal should mathematically comparing two snapshots, the robot is able to cal- have a higher preference than a low-priority goal. Finally, we show culate an average displacement vector with which it can move to- the algorithm’s adaptability capabilities in a dynamic environment wards a goal. Together these vectors make a displacement field and its robustness to goal-quality assessment errors. with which a robot can move from any point in the environment The remainder of this paper is structured as follows. First, we towards its destination. [20] also demonstrates an image matching present an overview of related work in the area of landmark nav- approach to landmark navigation and shows its functionality in a igation. Next, we present our new landmark algorithm and show robot with analog electronic hardware. [14] presents a single robot its effectiveness in a set of experiments. Finally, we conclude and that uses the Perception-Action architecture which is a neural com- look at future research. putation architecture proposed to solve a wide variety of control problems requiring learning capabilities. By using this architec- 2. RELATED WORK ture, the robot is able to learn how to reach any goal from any other landmark by moving around the goal. [22] presents a robot with a In this paper, we focus on landmark navigation in large groups. self-organizing mechanism to build a representation of the environ- More precisely, we focus on swarm communities like bee colonies. ment based on landmark recognition. The robot first starts with a Individuals of such colonies are limited in their means to navigate map-building effort and once the map is complete the map interpre- over typically unknown environments due to limitations in, for ex- tation starts which consists of map planning and execution. Note ample, brain capacity. However, where the individual fails, the that all these researches are based on single robots.