Self-Adaptive and Self-Organised Planning and Decision-Making for Multi-Robot Systems vorgelegt von M.Sc. Informatik Christopher-Eyk Hrabia ORCID: 0000-0002-5220-1627 von der Fakult¨atIV { Elektrotechnik und Informatik der Technischen Universit¨atBerlin zur Erlangung des akademischen Grades Doktor der Ingenieurwissenschaften { Dr.-Ing. { genehmigte Dissertation Promotionsausschuss: Vorsitzender: Prof. Dr. Oliver Brock Gutachter: Prof. Dr. Dr. h. c. Sahin Albayrak (TU Berlin) Gutachter: Prof. Dr.-Ing. Uwe Meinberg (BTU Cottbus-Senftenberg) Gutachter: Prof. Dr. Mihhail Matskin (KTH Royal Institute of Technology) Tag der wissenschaftlichen Aussprache: 14.11.2019 Berlin 2019 Abstract Robots are leaving the friendly, well-structured world of automation and are facing the challenges of a dynamic world. The uncertain conditions in the dynamic world call for a high degree of robustness and adaptivity for individual robots as well as interactions between multiple robots and other system entities. The uncertainty makes it difcult for designers and engineers to anticipate all conditions, interactions, and side efects a system will have to deal with while the system is specifed and developed. Furthermore, implicit and explicit coordination is required to perform a joint goal with multiple entities in a multi-robot system. Enabling scalability for multi-robot applications can be especially supported by means of implicit and decentralised coordination approaches. Nevertheless, robots that adapt to the dynamic environment and coordinate themselves still have to pursue their given tasks or goals. This thesis researches how multi-purpose, mobile, multi-robot systems can be en- hanced to operate more adaptively in dynamic environments. This is done by analysing and exploring the combination of so far separated research directions of goal-driven decision-making and planning as well as self-adaptation and self-organisation. The presented hybrid decision-making and planning framework is integrated into the popular robotic middleware Robot Operating System (ROS). The solution combines symbolic planning with reactive behaviour networks, automated task delegation, rein- forcement learning, and pattern-based selection of suitable self-organisation mechanisms. On that account, it brings together the advantages of bottom-up and top-down oriented architectures for task-level control of multi-robot systems. The developed framework enables a coherent and integrated design and implementation of decision-making and planning as well as coordination application logic within one software ecosystem that features a common domain model and a modular architecture. This results in a simpli- fcation of the development of adaptive multi-purpose multi-robot systems by avoiding system discontinuities and enabling a holistic view on the actual implementation. The presented approach has been successfully evaluated in various research projects and international competitions in the feld of robotics and multi-agent systems. iii Zusammenfassung Roboter verlassen ihre etablierte, strukturierte Welt der Automatisierung und stellen sich den Herausforderungen der dynamischen Umwelt. Die zum Teil unbekannten Bedingun- gen in der dynamischen Umwelt erfordern ein hohes Maß an Robustheit und Adaptivit¨at fur¨ den einzelnen Roboter, als auch fur¨ die Interaktion mehrerer Roboter untereinan- der. Diese Bedingungen machen es fur¨ Entwickler schwierig alle Zust¨ande, Interaktionen und Seitenefekte fur¨ ein System im voraus zu spezifzieren. Zus¨atzlich muss fur¨ eine gemeinsame Erfullung¨ von Zielen durch mehrere Roboter eine explizite oder implizite Koordination erfolgen. Hier kann vor allem eine implizite und dezentrale Koordination eine gute Skalierbarkeit unterstutzen.¨ Trotz eines Fokusses auf Adaptivit¨at, mussen¨ die Robotersysteme aber zugleich auch gegebene Ziele und Aufgaben erfullen.¨ Diese Dissertation erforscht wie multifunktionale, mobile Multi-Roboter-Systeme ver- bessert werden k¨onnen, um adaptiver und robuster in dynamischen Umgebungen zu operieren. Dazu wird speziell eine Kombination aus den bisher unabh¨angig betrachteten Forschungsrichtungen der zielgerichteten Planung und Entscheidungsfndung, als auch der Selbst-Adaptation und Selbst-Organisation untersucht. Das vorgestellte hybride Entscheidungsfndungs- und Planungsframework ist dabei in- tegriert in die weitverbreite Robotik-Middleware Robot Operating System (ROS). Spezi- ell kombiniert die realisierte L¨osung symbolische Planung mit reaktiven Verhaltensnetz- werken, automatischer Aufgabenverteilung, verst¨arkendes Lernen und musterbasierte Auswahl von Selbstorganisationsmechanismen. Auf dieser Grundlage vereint das Sys- tem die Vorteile von Bottom-Up- und Top-Down-Architekturen fur¨ die Steuerung von Multi-Roboter-Systemen auf Aufgabenebene. Weiterhin erm¨oglicht die modulare Archi- tektur und das ubergreifenden¨ Dom¨anenmodell innerhalb eines Software¨okosystems ein einheitliches und integriertes Design der Planungs-, Entscheidungsfndungs-, und Koor- dinationslogik. Das resultiert in einer Vereinfachung der Entwicklung von adaptiven und multifunktionalen Multi-Roboter-Systemen durch die Vermeidung von Systembruchen¨ in einem holistischen Ansatz. Das vorgestellte System wurde erfolgreich in verschiedenen Forschungsprojekten und internationalen Wettk¨ampfen aus dem Bereich Multi-Agenten- und Multi-Roboter-Sys- teme evaluiert. v Acknowledgments First of all, I wish to acknowledge my supervisor Prof. Dr. Dr. h. c. Sahin Albayrak for providing the opportunity, the environment, and the infrastructure that enabled me to pursue my research at DAI-Labor, Technische Universit¨atBerlin. Furthermore, I would also like to express my very great appreciation to Prof. Dr.-Ing. Uwe Meinberg and Prof. Dr. Mihhail Matskin for taking over the role as external reviewers. In particular, I am especially thankful for all enthusiastic colleagues that collaborated with me in various research activities, provided feedback, and shared their knowledge with me during our common time in the working groups RAS (Robotics and Autonomous Systems) and ACT (Agent Core Technologies), namely Nils Masuch, Michael Burkhardt, Orhan-Can G¨or¨ur,Dr. rer. nat. Yuan Xu, Martin Berger, Christian Rackow, Tuguldur Erdene-Ochir, and Dr.-Ing. Jan Keiser. Some colleagues from RAS and ACT deserve special acknowledgement. For this rea- son, I would like to ofer my special thanks to the head of my group Dr.-Ing. Axel Heßler, who challenged me with uncomfortable questions but always supported me in all my research attempts, ideas, and own ways of thinking. Furthermore, I am particularly grateful for the assistance of Prof. Dr.-Ing. Johannes F¨ahndrich for his altruistic support, guidance, and constructive feedback at the very fnal stage of my dissertation. Likewise, the advice of Dr.-Ing. Marco L¨utzenberger put me on the right track at the beginning of my dissertation process. Furthermore, special thanks to my colleague Dr.-Ing. Tobias K¨uster,who was the most committed reviewer of all my research writing over the years. Moreover, my grateful thanks to my colleague Denis Pozo for sharing and discussing all our fears and doubts about pursuing a Ph.D. Not to forget, I am particularly grateful for all the insights and ideas I could develop through the discussions and provided feedback during the supervision of many Bach- elor's and Master's theses. It was a pleasure to mentor Stephan Wypler, Phillip Erik Rieger, Tanja Katharina Kaiser, Alexander Wagner, Patrick Marvin Lehmann, Vito Felix Mengers, and Michael Franz Ettlinger. Most importantly, very special thanks to my wife Anne, for her love, overall support in my life, and acceptance of all the necessary extra hours that allowed me to pursue my dissertation. vii Table of Contents Table of Contents ..................................... ix List of Figures ....................................... xiii List of Tables ....................................... xv List of Listings ....................................... xvii Acronyms ......................................... xviii Publications ........................................ xxi I. Introduction ................................1 1. Motivation ......................................2 2. Problem Statement ..................................3 3. Research Questions ..................................6 4. Structure of the Document ..............................8 II. Analysis .................................. 13 5. Foundational Concepts ................................ 15 5.1. Autonomous Agents and Multi-Agent Systems . 16 5.2. Emergence and Swarm Intelligence . 18 5.3. Swarm Robotics . 20 5.4. Self-Organisation and Self-Adaptation . 22 5.5. Common Characteristics of Emergence, Swarm Intelligence, Self-Adaptation, and Self-Organisation . 24 6. Self-Organisation Mechanisms and their Application ................ 25 7. Related Surveys about Emergence, Self-Organisation and Self-Adaptation ..... 28 8. Self-Adaptation and Self-Organisation Software Development Methodologies ... 31 9. Middleware Solutions and Frameworks ........................ 35 10. Mechanism Design and Implementation ....................... 41 10.1. Evolution, Learning and Simulation . 42 10.2. Functional Languages . 43 10.3. Application-specifc Approaches . 44 10.4. Declarative Approaches . 45 ix Table of Contents 10.5. Analysis, Verifcation and Validation . 48 11. Planning and Decision-Making in Robotics ..................... 49 12. Task Allocation, Decomposition, and Delegation .................. 55 12.1.