Self–Organised Multi Agent System for Search and Rescue Operations Panteha Saeedi A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy of the University College London. Department of Computer Science University College London August 18, 2010 2 Human beings are members of a whole, In creation of one essence and soul. If one member is afflicted with pain, Other members uneasy will remain. If you have no sympathy for human pain, The name of human you cannot retain. Saadi of Shiraz Abstract Autonomous multi-agent systems perform inadequately in time critical missions, while they tend to explore exhaustively each location of the field in one phase with out selecting the pertinent strategy. This research aims to solve this problem by introducing a hierarchy of exploration strategies. Agents explore an unknown search terrain with complex topology in multiple predefined stages by performing pertinent strategies depending on their previous observations. Exploration inside unknown, cluttered, and confined environments is one of the main challenges for search and rescue robots inside collapsed buildings. In this regard we introduce our novel exploration algorithm for multi–agent system, that is able to perform a fast, fair, and thorough search as well as solving the multi–agent traffic congestion. Our simulations have been performed on different test environments in which the complexity of the search field has been defined by fractal dimension of Brownian movements. The exploration stages are depicted as defined arenas of National Institute of Standard and Technology (NIST). NIST introduced three scenarios of progressive difficulty: yellow, orange, and red. The main concentration of this research is on the red arena with the least structure and most challenging parts to robot nimbleness. Acknowledgements I would like to acknowledge the valuable contribution of many people who have helped me directly or indirectly in the accomplishment of this work. My supervisor, Soren Aksel Sorensen, has played a key role in guiding my first steps in the world of research. I am greatly appreciating Stephen Hailes for his kind support and valuable advice and Lica Capra for her generous guidance. I am also grateful to Michele Sama for his continues support and constructive comments all through my thesis. I would also like to thank Mohammad and Steffen for their interesting and helpful discussion on various topics related to this work. Many thanks to all those not named here and who showed interest in my work and guided me through this thesis . Last but not least, I take this opportunity to express my profound gratitude to my beloved parents, for their constant support and love all through my life. Panteha Saeedi Contents 1 Introduction 14 1.1 Research Challenges . 14 1.2 Motivation . 15 1.3 Research Hypothesis & Objectives . 15 1.4 System Design & Implementation . 16 1.5 Contributions . 16 1.6 Scope & Assumptions . 17 1.7 Thesis Structure . 17 2 Background 19 2.1 USAR Five–Phase Plan . 20 2.1.1 Global Survey of the Site . 20 2.1.2 Local Surveying the Site . 21 2.1.3 Microscopic Surveying the Site . 21 2.1.4 Rescue Operation and Recovery . 23 2.2 Simulated Search Fields . 24 2.2.1 Fractal Dimension . 24 2.2.2 Box–Counting Dimension . 26 2.2.3 Related Work . 27 2.2.4 Maze Generation Technique . 28 2.3 Exploration Strategies . 28 2.3.1 Advantages & Disadvantages . 29 2.3.2 Heuristic & Probabilistic Strategies . 29 2.3.3 Probabilistic Exploration Strategies . 30 2.4 Uncertainty . 32 2.4.1 MDP and POMDP Preliminaries . 33 2.4.2 POMDP–Solver . 36 2.5 Exploration Formation . 37 2.6 Summary . 39 6 Contents 3 System Simulation 40 3.1 Agents . 41 3.1.1 Eight Segmented Vision . 41 3.1.2 Agent Tasks . 42 3.1.3 Brownian Random Walk . 42 3.2 Simulated 2D Search Field . 44 3.2.1 Search Field Generator . 44 3.2.2 Discriminative Complexity Index . 45 3.2.3 Probability Mass Function . 48 3.3 Complexity Index Validation . 49 3.4 Discussion . 53 4 Performance Metric Analysis 54 4.1 Preliminary Remarks about Metric . 55 4.2 Multi–Agent Exploration System Taxonomy . 55 4.2.1 Team–Level Metrics . 56 4.2.2 Individual–Level Metrics . 57 4.3 Expected Utility . 57 4.3.1 Effectiveness of Coverage . 58 4.3.2 Parameter Evaluation . 60 4.4 Evaluated Benchmark . 61 4.4.1 Ridge Regression . 62 4.4.2 Predicted Time Series . 64 4.5 Discussion . 65 5 Performance Aware Agents 66 5.1 Reward Function . 67 5.2 Frontier–Based Technique . 68 5.3 Exploration Without Positional Uncertainty . 69 5.4 Exploration With Positional Uncertainty . 73 5.4.1 Belief System by Bayes Theorem . 73 5.4.2 Dual–mode Controller . 74 5.4.3 24 States Approximate–POMDP . 75 5.4.4 Exploration Evaluation . 77 5.5 Frame Size Setting . 79 5.6 Initial Congestion . 79 5.7 Kidnap Recovery . 81 5.8 Discussion . 82 7 Contents 6 Architectural Design of Multi Agent System 83 6.1 Emergent and Self–Organised System . 84 6.2 Tethered Search of Limited–Sensing Multi–agent (TeSLiSMA) . 85 6.2.1 Macro–Level Behaviours . 86 6.2.2 Micro–Level Behaviours . 88 6.2.3 Local Memory Grid Generation . 89 6.2.4 Multi–Agent Spreading Techniques . 92 6.2.5 Local Memory Merging . 93 6.2.6 Spreading Techniques Evaluation . 94 6.3 Exploring a Large Area with One Team . 96 6.3.1 RF Communication . 97 6.3.2 Pebble Tracking . 97 6.4 Area Percentage Coverage . 98 6.5 Exploration Efficiency Coverage . 99 6.6 Discussion . 100 7 Multi Agent System Evaluation 101 7.1 Multi–Agent Exploration Comparison . 101 7.2 Multi–agent Performance Improvement . 106 7.3 Toward Real Robot Testing . 107 7.3.1 Player/Stage . 108 7.3.2 Applying TeSLiSMA al. on Player/Stage . 109 7.4 Conclusion & Future work . 111 8 Conclusion 112 8.1 Critical Evaluation . 112 8.1.1 Efficiency . 113 8.1.2 Thoroughness . 113 8.1.3 Fairness . 114 8.1.4 Robustness . 114 8.2 Practical Limitation . ..