TECHNISCHE UNIVERSITÄT MÜNCHEN The
Total Page:16
File Type:pdf, Size:1020Kb
TECHNISCHE UNIVERSITÄT MÜNCHEN Lehrstuhl für Informatik XXIII Sensor Based Robotic Systems and Intelligent Assistance Systems The Cognitive Robot Abstract Machine: A Framework for Cognitive Robotics Lorenz Mösenlechner Vollständiger Abdruck der von der Fakultät für Informatik der Technischen Universität München zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften (Dr. rer. nat.) genehmigten Dissertation. Vorsitzende: Univ.-Prof. Gudrun Klinker, Ph.D. Prüfer der Dissertation: 1. Univ.-Prof. Michael Beetz Ph.D., Universität Bremen 2. Univ.-Prof. Dr. Alin Albu-Schäffer Die Dissertation wurde am 19.08.2015 bei der Technischen Universität München ein- gereicht und durch die Fakultät für Informatik am 26.02.2016 angenommen. Abstract With the development of mobile robotic platforms that can operate in domestic do- mains such as a human household, for instance Willow Garage’s PR2, solving tasks such as cleaning up or cooking food has become possible. The software that controls these robots became more and more complex and recent research provides solutions for navigation, manipulation and perception. With rising complexity of the tasks to be performed, the requirements for a high level executive that orchestrates arm mo- tions, navigation and perception changed towards a tight integration of planning, reasoning and action execution. This thesis describes CRAM, a complete framework for implementing cognitive high-level robot control programs to enable robotic agents to execute their tasks more robustly, reliably and flexibly. CRAM combines a powerful language for programming robot actions with sophisti- cated reasoning mechanisms. This allows the programmer to state plans more ab- stractly by letting the robot make decisions based on its current belief about the en- vironment and the predicted future course of actions. More specifically, besides the plan language, CRAM provides a Prolog-like reasoning engine and many components to solve problems such as where to stand to perform actions, is an object visible to the robot from a certain location, from where can an object be seen or grasped but also components that allow the robot to reason about what it did, when and why. In addi- tion to providing the means for implementing high-level control programs, the system also provides components for predicting the outcome of a plan. This, in combination with the other reasoning modules, allows the robot to base its decisions not only on knowledge about the current state of the world but to integrate the future course of actions. III Kurzfassung Mit der Entwicklung mobiler Roboterplattformen wie Willow Garages PR2, die in häuslichen Umgebungen, wie zum Beispiel einem Haushalt, operieren können, wurde die Ausführung anspruchsvoller Aufgaben wie putzen oder kochen mit diesen Robo- tern möglich. Die Software, die diese Roboter steuert, ist immer komplexer gewor- den und aktuelle Forschungsergebnisse bieten Lösungen für Navigation, Manipulati- on und Perzeption. Mit zunehmender Komplexität der zu erfüllenden Aufgaben ha- ben sich auch die Anforderungen an eine übergeordnete Kontrollschicht hin zu einer Integration von Planungsalgorithmen, logischem Schließen und der Ausführung von Aktionen verändert. In der hier vorliegenden Arbeit wird CRAM beschrieben, ein kom- plettes Framework um kognitive Kontrollprogramme für Roboter zu entwickeln, die es ihnen erlauben, ihre Aufgaben robuster, zuverlässiger und flexibler zu erfüllen. CRAM kombiniert eine Programmiersprache zur Robotersteuerung mit Mechanismen zum automatischen Ziehen logischer Schlussfolgerungen. Damit kann der Program- mierer Pläne abstrakter schreiben und dem Roboter bestimmte Entscheidungen über- lassen, die dieser basierend auf seinem Umgebungsmodell und einer Vorhersage zu- künftiger Aktionen treffen kann. Neben einer Programmiersprache zum Schreiben von Plänen enthält CRAM eine prologartige Sprache zur Lösung logischer Probleme sowie eine Vielzahl anderer Komponenten, die es ermöglichen, Probleme, wie zum Beispiel wo der Roboter stehen soll um bestimmte Aktionen auszuführen, ob ein Ob- jekt an einer bestimmten Stelle sichtbar ist oder wo der Roboter stehen soll, um ein bestimmtes Objekt zu sehen oder zu greifen, zu lösen. Daneben entält CRAM Kom- ponenten, die dem Roboter Schlüsse über seine Aktionen erlauben, insbesondere was er getan hat, wann und aus welchem Grund. Zusätzlich enthält CRAM Komponen- ten um die Effekte und Ergebnisse von Plänen vorherzusagen, was es dem Roboter in Kombination mit den anderen Inferenzalgorithmen ermöglicht, Entscheidungen nicht V nur basierend auf Wissen über den aktuellen Zustand der Umgebung, sondern auch unter Einbeziehung zukünftiger möglicher Weltzustände zu treffen. VI Acknowledgements The work presented in this thesis would not have been possible without the support of my advisor Michael Beetz. I want to thank him for his great vision, the many mo- tivating and fruitful discussions and the support I got from him. Furthermore, I want to thank all my colleagues who provided a great and unique working atmosphere, for the many interesting and enriching discussions I had with them. I especially want to thank Moritz Tenorth, Ulrich Klank, Dominik Jain, Nico Blodow, Thibault Kruse for their friendship and for the many things I could learned from them. They helped me to become a better scientist and a better software engi- neer. I also owe thanks to my former students Nikolaus Demmel, Tobias Rittweiler and Gayane Kazhoyan for helping me with the implementation of CRAM and for giving me valuable feedback on its design. I also want to thank Elena, my wife, for enriching my live and for the support and the motivation she gave me during my academic career. And last but not least I want to thank my family and my friends for their encourage- ment and support. VII Contents Abstract III KurzfassungV Acknowledgements VII ContentsIX List of Resources XIII 1 Introduction1 1.1 Problem Description...............................4 1.2 System Architecture............................... 10 1.3 Contributions................................... 16 1.4 Reader’s Guide................................... 18 2 The CRAM Software Architecture 21 2.1 The CRAM Plan Language............................ 23 2.1.1 Language Syntax............................. 24 2.1.2 Definition of Terms........................... 32 2.1.3 Fluents................................... 35 2.1.4 CRAM Plan Language Expressions.................. 46 2.2 Reasoning and the CRAM Prolog Interpreter................ 66 2.2.1 Implementation of the Prolog Interpreter.............. 69 2.2.2 Implementation of Predicates..................... 70 2.3 Symbolic Plan Parametrization......................... 74 2.3.1 Designator Concepts........................... 74 2.3.2 Designator Resolution.......................... 81 IX Contents 2.4 Process Modules.................................. 85 2.4.1 The CRAM Process Module Interface................. 88 2.4.2 Synchronous and Asynchronous Action Execution........ 91 2.5 The CRAM Plan Library............................. 95 2.5.1 Transparent Plans............................ 95 2.5.2 Goals in the Current CRAM System................. 98 2.6 Related Work.................................... 102 2.7 Discussion..................................... 103 3 CRAM Reasoning Extensions 105 3.1 Physics-Based Reasoning............................. 106 3.1.1 Physics Engine Integration and the World Database....... 110 3.1.2 Visibility Computation......................... 126 3.1.3 Reachability Reasoning......................... 131 3.1.4 Belief State Representation using CRAM’s World Database... 139 3.2 Reasoning about Plan Execution........................ 143 3.2.1 Representation of Execution Traces................. 145 3.2.2 Querying Execution Traces....................... 148 3.2.3 Examples for Reasoning on Execution Traces............ 157 3.3 Related Work.................................... 160 3.4 Discussion..................................... 161 4 Parameterizing Plans 163 4.1 Density Maps for Sampling Solution Candidates.............. 165 4.1.1 Implementation of Density Maps................... 169 4.2 Generation of Height and Orientation Values................ 189 4.3 The Density Map Generator API and Implementation Details...... 191 4.3.1 The Internal Representation of Density Maps and Generators. 192 4.3.2 Prolog API for Integration in Designator Resolution....... 194 4.3.3 Integration in Designator Resolution................. 197 4.4 Related Work.................................... 198 4.5 Discussion..................................... 198 X Contents 5 Temporal Projection of Plans 201 5.1 Projection of Plans and Generation of Timelines.............. 203 5.1.1 The Projection Environment...................... 203 5.1.2 Process Modules for Projection.................... 205 5.2 Handling of Time in Projection......................... 209 5.3 Definition and Matching of Behavior Flaws................. 211 5.4 Location Designator Optimization Using Projection............ 217 5.5 Related Work.................................... 217 5.6 Discussion..................................... 218 6 Conclusion 221 Bibliography 231 Index 239 XI List of Resources Figures 1.1 The PR2 (left) and TUM Rosie (right).....................2 1.2 In our example, silverware, mugs and plates are stored in two different drawers at the counter...............................5 1.3 Table set for breakfast for two persons. This is the final configuration for the example in this Section..........................9