The Pennsylvania State University The Graduate School Department of Computer Science and Engineering A LANGUAGE-BASED SOFTWARE FRAMEWORK FOR MISSION PLANNING IN AUTONOMOUS MOBILE ROBOTS A Thesis in Computer Science and Engineering by Mirza A. Shah c 2011 Mirza A. Shah Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science August 2011 The thesis of Mirza A. Shah was read and approved∗ by the following: John Hannan Associate Professor of Computer Science and Engineering Thesis Adviser Mahmut Kandemir Professor of Computer Science and Engineering Raj Acharya Professor of Computer Science and Engineering Head of the Department of Computer Science and Engineering ∗Signatures on file in the Graduate School. iii Abstract Autonomous mobile robots have many uses in both commercial and military ap- plications. Creating the computer software that implements the autonomy of these robots is a daunting task, which has resulted in the development of many tools across both industry and academia to help mitigate development complexity and costs in the form of software libraries, frameworks, and application programmer interfaces. This thesis describes a new software framework, the Modular Planning Framework and Lan- guage (MPFL), for developing the mission planning aspect of a mobile robot's autonomy. MPFL is unique in that it looks at the problem of vehicle planning as a programming language with a corresponding runtime compiler that processes the language. This ap- proach to autonomy development leverages concepts from programming language theory to create a planning framework that supports component reuse, strong data type ver- ification, and the ability to reason about complex autonomy solutions in a piecemeal fashion. iv Table of Contents List of Tables :::::::::::::::::::::::::::::::::::::: x List of Figures ::::::::::::::::::::::::::::::::::::: xi Acknowledgments ::::::::::::::::::::::::::::::::::: xiii Chapter 1. Introduction and Roadmap ::::::::::::::::::::::: 1 1.1 The Future is Autonomous . 1 1.2 Autonomy and Robotics . 1 1.2.1 Commercial and Research Autonomous Robots . 1 1.2.2 Military Autonomous Robots . 5 1.3 How Autonomous Robots Work . 6 1.3.1 Sensors, Actuators, and Payloads . 9 1.3.2 Data Fusion, Reasoning, and Perception . 10 1.3.3 Planning and Response . 11 1.3.3.1 Reactive versus Deliberative Planning . 12 1.3.4 Learning . 13 1.3.5 Putting it all Together . 14 1.4 The Development and Challenges of Mission Planning in Mobile Robots 14 1.5 Roadmap . 15 Chapter 2. Background Concepts :::::::::::::::::::::::::: 16 2.1 The Relationship Between Computation and Artificial Intelligence . 16 2.2 The Relationship Between AI and Autonomy . 17 2.3 The Relationship Between Intelligence, Natural Languages, and Pro- gramming Languages . 18 2.4 The Relationship Between AI and Programming Languages . 19 2.4.1 Symbol Processing, Languages, and Semantics . 20 2.4.2 Knowledge Representation as Types . 20 2.4.2.1 Composite types and abstract data types . 21 2.4.2.2 Hierarchical Knowledge Representation Through Sub- typing . 21 2.4.2.3 Reasoning Through Type Inferencing . 21 2.4.3 Hierarchical Error Handling through Exceptions . 22 2.4.4 The Power of Abstraction - Sapir-Whorf Revisited . 23 2.5 The Relationship Between AI and Domain-Specificity . 23 2.6 The Relationship Between AI and Domain-Specific Programming Lan- guages . 24 2.6.1 Libraries versus Domain-Specific Languages . 26 2.6.2 Looking at Software Development as a Programming Lan- guage Design Problem . 27 v 2.7 The Relationship between Intelligence, Planning, and Scheduling . 27 2.8 Planning as Processes . 28 2.8.1 Processes and Operating Systems . 28 2.8.2 Modeling Planning in Terms of Processes . 29 2.8.3 Processes and Programming Language Semantics . 30 2.9 Segue: The Big Picture . 30 Chapter 3. Overview of the Modular Planning Language and Framework :::: 31 3.1 Redundancies in Autonomy Development . 31 3.2 Overview of the Modular Planning Framework and Language . 32 3.2.1 Planning is to Schedules as Compiling is to Machine Code . 32 3.2.2 Expressiveness of the Mission Specification Language . 34 3.2.3 Customization and Reusability Via Plugins . 35 3.2.4 A Software Framework . 36 3.3 Segue: A Language of Planning . 37 Chapter 4. A Language of Planning ::::::::::::::::::::::::: 38 4.1 The Requirements of a Language for Planning . 38 4.2 The MPFL Mission Specification Language . 39 4.2.1 Language Overview . 39 4.2.2 Plans and Plan Instances . 40 4.2.2.1 Plan is a Type . 40 4.2.2.2 Plan Instance Constructors and Other Types . 41 4.2.2.3 Explicit Parameter Naming and Unit Specification . 43 4.2.3 User-defined Plans . 45 4.2.3.1 User-defined Plan Syntax . 45 4.2.3.2 Do Expression and Plan Operators . 46 4.2.3.3 Creating Plan Instances from User-Defined Plans . 48 4.2.3.4 The Sortie Plan Instance and Children Plan Instances 49 4.2.3.5 Multiple Instances of User-Defined Plans . 50 4.2.3.6 Circular Dependencies . 50 4.2.4 Run-time Value Lookups . 50 4.2.4.1 An Important Note on the Safety of Lookup Calls . 51 4.2.5 Conditional Expressions . 51 4.2.5.1 Meaning of Conditional Expression . 52 4.2.5.2 Fluctuation of Condition . 53 4.2.5.3 Defining the Condition . 53 4.2.5.4 Pitfalls of Conditional Expressions . 53 4.2.6 Declaring and Binding Constraints . 54 4.2.6.1 Meaning of a Constraint . 55 4.2.6.2 Binding to an Entire Subexpression . 55 4.2.6.3 Binding Multiple Constraints and Constraint Inter- section . 56 4.2.6.4 Constraints are Hierarchical . 56 4.2.7 Handling Errors: Infeasibilities and Conflicts . 57 vi 4.2.7.1 Infeasibility Errors . 57 4.2.7.2 Conflict Errors . 57 4.2.7.3 Graceful Degradation and Exception Handling . 57 4.2.7.4 Infeasible and Conflict Handlers . 58 4.2.7.5 Handlers Are Hierarchical . 60 4.2.7.6 Case Signature Matching and Plan Instance Chains 61 4.3 Segue: Making it Work . 64 Chapter 5. Understanding and Utilizing MPFL ::::::::::::::::::: 66 5.1 Overview . 66 5.2 How MPFL is Meant to Be Used . 66 5.3 The Simplified MPFL Compiler/Runtime Engine . 67 5.4 Parser . 68 5.4.1 Grammar Specification . 68 5.4.2 Syntactic Enforcement of Types . 72 5.5 Plan Instance Tree (PIT) Builder . 73 5.5.1 Static Semantics . 73 5.5.2 Intermediate Representation - Plan Instance Tree . 74 5.5.3 Formal Specification of Type Rules . 78 5.5.4 Formal Specification of Run-time Type Rules . 79 5.5.5 The Next Step . 80 5.6 Lifetime State Transition (LST) Evaluator . 80 5.6.1 Lifetime State . 81 5.6.2 Usage of Lifetime State and the MPFL Model of Planning . 82 5.6.3 The Lifetime State Transition Evaluator . 82 5.6.4 LST Transition Rules . 83 5.6.5 Formal Semantics of LST Evaluator . 85 5.6.5.1 Value Evaluator (Partial Set of Rules) . 87 5.6.5.2 ChangeOnState Evaluator (Complete Set of Rules) . 87 5.6.5.3 CurrentSerial Evaluator (Complete Set of Rules) . 88 5.6.5.4 CurrentXor Evaluator (Complete Set of Rules) . 88 5.6.5.5 Current Evaluator (Complete Set of Rules) . 89 5.6.5.6 NextSerial Evaluator (Complete Set of Rules) . 89 5.6.5.7 NextXor Evaluator (Complete Set of Rules) . 91 5.6.5.8 LST Evaluator (Complete Set of Rules) . 93 5.6.6 Purpose of Formal Specification . 94 5.7 Planner Invocation (PI) Evaluator . 95 5.7.1 Plugins and Initialization . 95 5.7.2 Knowledge Base . 96 5.7.2.1 Formal Semantics of Lookup Calls . 96 5.7.3 Planners . 97 5.7.4 Planners and the MSL . 97 5.7.5 Planner Graph . 98 5.7.6 Planner Isolation and Component Reusability . 98 5.7.7 The Planner API . 98 vii 5.7.7.1 The Class planInstance . 99 5.7.7.2 The Class planner . 101 5.7.8 The PI Evaluator Algorithm . 101 5.7.8.1 Top-Down Traversal . 102 5.7.8.2 Bottom-Up Traversal . 103 5.7.9 Schedules and Verification . 104 5.8 Putting it All Together - The Basic MPFL Compiler engine . 104 5.9 The Complete MPFL Compiler/Runtime Engine . 105 5.9.1 Planners Raise Exceptions . 105 5.9.2 Handling Errors via the Exception Handler (EH) Evaluator . 106 5.10 Replanning As Recompilation . 107 5.11 Segue: Building an Autonomous Robot with MPFL . 107 Chapter 6. A Complete Demonstration System ::::::::::::::::::: 108 6.1 Demonstration System Architecture . 108 6.2 Demo Client Application . 108 6.3 AUV Simulator and Map Display . 109 6.4 The Planner Hierarchy . 110 6.5 Demo UseAutopilot Planner . 110 6.5.1 Callback on ready to running . 111 6.5.2 Callback on forcerun to running . 111 6.5.3 Callback on running to complete . 111 6.5.4 Callback on ask for subproblems . 111 6.5.5 Schedule Encoding . 111 6.5.6 Callback build schedule . 112 6.5.6.1 Representing and Rating a Solution . 112 6.5.6.2 Comparing Solutions . 113 6.5.6.3 Finding the Solution - Genetic Algorithm . 113 6.5.6.4 Performance . 114 6.5.6.5 Failure to Converge . 114 6.5.6.6 Accounting for Blocked Instances . 114 6.5.6.7 Improvement over Time and Solution Caching . 114 6.6 Demo Transit Planner . 115 6.6.1 Callback on ready to running . 115 6.6.2 Callback on forcerun to running . 115 6.6.3 Callback on running to complete . 115 6.6.4 Callback on ask for subproblems .