DEPENDABLE PATH PLANNING for AUTONOMOUS CONTROL AUTONOMOUS for PATH PLANNING DEPENDABLE Autonomous Control

Mälardalen University Licentiate Thesis 288 Lan Anh Trinh Trinh Anh Lan Dependable Path Planning For

DEPENDABLE PATH PLANNING FOR AUTONOMOUS CONTROL AUTONOMOUS FOR PATH PLANNING DEPENDABLE Autonomous Control

Lan Anh Trinh

Address: P.O. Box 883, SE-721 23 Västerås. Sweden ISBN 978-91-7485-451-0 2019 Address: P.O. Box 325, SE-631 05 Eskilstuna. Sweden E-mail: [email protected] Web: www.mdh.se ISSN 1651-9256 1

Mälardalen University Press Licentiate Theses No. 288

DEPENDABLE PATH PLANNING FOR AUTONOMOUS CONTROL

Lan Anh Trinh

2019

School of Innovation, Design and Engineering 2

To my family. 4

Acknowledgments

I would like to express my thanks to many persons who have directly and indi- rectly helped me on the path that leads to this dissertation. First and foremost, I especially thank my supervisors, Professor Dr. Mikael Ekstrom¨ and Dr. Baran Cur¨ ukl¨ u,¨ who have guided and supported my research with warm encouragement and insightful advice. I also thank for their great helps in life so that I can understand more about Swedish culture, overcome all difﬁculties, settle down and feel that Sweden is my second mother land. I also especially thank all MDH gangs, PhD students in Malardalen Uni- versity, who are very good, friendly, and enthusiastic. Even though we do not communicate much, you guys always give me nice advices when needed. Many thanks to all of my friends in Robotics Group, who have been always very nice and friendly teammates, Fredrik, Carl, and others, especially two my ofﬁce-mates Gita and Branko, who have helped me a lots in both researches and life. Many next thanks to DPAC leaders who have proposed an excellent project so that I have learnt a lots and enjoyed working within the project. I am always thankful to administrative ladies who have made all the complicated administrative procedures to become easy and smooth. Last but not least, I would like to especially thank my family in Vietnam, my husband, and my little Sam for their unconditional supports, endless love, and encouragement. They are my accompany at every stage in my educa- tion and life. They are the ones I can always share the burden of stress. My motivation is also contributed by the love from them and their belief for this iv 5

v dissertation. I want to dedicate this thesis to them whom I love most in my life.

Lan Anh Trinh Vaster¨ as,˚ December, 2019 6 7

Sammanfattning

Att byta fran˚ automatisk till autonom kontroll har visat sig vara ett av de vik- tigaste teknikskiften vid utvecklingen av robotar idag. Den autonoma kon- trollen gor¨ det mojligt¨ for¨ en robot att ha mer frihet samt mojlighet¨ till di- rekt interaktion med manniskor¨ och andra robotar. Att ha en palitlig˚ plattform for¨ autonom kontroll blir daavg˚ orande¨ nar¨ vid byggandet av ett sadant˚ system. Tillforlitligheten¨ hos en robotagent representeras av ett antal huvudat- tribut sasom˚ tillganglighet,¨ dvs systemets kontinuerliga drift over¨ ett tidsinter- vall, tillforlitlighet,¨ dvs systemets form¨ aga˚ att tillhandahalla˚ korrekta tjanster,¨ och sakerhet,¨ dvs. robotagenten maste˚ sakerst¨ alla¨ sakra¨ kontroller for¨ att undvika katastrofala konsekvenser for¨ anvandare,¨ andra robotar och slutligen miljon.¨ Med tanke paatt˚ banplanering ar¨ en av nyckelkomponenterna i ett autonomt kontrollsystem for¨ robotagenter, syftar arbetena som presenteras i denna avhan- dling att bygga en palitlig,˚ dvs saker,¨ palitlig˚ och effektiv banplaneringsal- goritm for¨ en grupp robotar som delar sitt arbetsutrymme med manniskor.¨ For¨ det forsta¨ foresl¨ as˚ en ny metod for¨ banplanering av flera robotagenter baserat padipolfl˚ odesf¨ alt.¨ For¨ att initialisera vagarna¨ fran˚ en startpunkt till ett mal˚ anvands¨ banplanering med Theta* - algoritmen for¨ en uppsattning¨ av agenter. For¨ att hantera statiska hinder for¨ en robot som ar¨ pav˚ ag¨ till malet˚ definieras ett statiskt flodesf¨ alt¨ langs¨ den konfigurerade banan. Ett dipolfalt¨ beraknas¨ sedan for¨ att undvika kollision med andra agenter och personer. I detta tillvagag¨ angss˚ att¨ antas varje robotagent vara en kalla¨ till ett magnetiskt dipolfalt,¨ i vilket det magnetiska momentet ar¨ riktat i agentens rorelseriktning,¨ vii 8

viii med en styrka som ar¨ proportionell mot dess hastighet. De magnetiska dipol- dipolinteraktionerna mellan dessa agenter genererar darepulsiva˚ krafter for¨ att hjalpa¨ dem att undvika kollision. Samtidigt anvands¨ felanalys med Petri-nat¨ av flera robotar for¨ att forst¨ asamarbetet˚ mellan flera robotagenter for¨ att losa¨ de gemensamma uppgifterna. Darefter¨ appliceras Petri-natet¨ tillsammans med vagplaneringen¨ for¨ att undvika kollisioner genom synkroniserade rorelser¨ hos robotar genom exempelvis en korsning. Under tiden har kontinuerligt flervags-¨ planering undersokts¨ for¨ att stodja¨ feltoleransen for¨ sokplaneringsalgoritmen.¨ Detta for¨ att hantera dodl¨ aget¨ dar¨ agenten tar mycket lang˚ tid att nam˚ alet˚ eller till och med inte kan gora¨ det. Agenten har forsetts¨ med olika planer- ade vagar¨ till malet˚ och vaxlar¨ proaktivt mellan dessa nar¨ det behovs¨ for¨ att undvika dodl¨ aget.¨ Slutligen har hela ramverket implementerats i en allmant¨ anvand¨ plattform, robotoperativsystem (ROS), och utvarderats¨ genom Gazebo- simulator. 9

Abstract

Changing from automatic to autonomous control has emerged as the main shift on the development of robots nowadays. The autonomous control allows robot to have more freedom as well as direct interactions with human and other robots. Having a dependable platform for autonomous control becomes crucial when building such a system. The dependability of a robotic agent is presented by main attributes including availability, i.e. the continuous operations of the system over a time interval, reliability, i.e. the ability of the system to provide correct services, and safety, i.e. the robotic agent must ensure safe controls to avoid any catastrophic consequences on users, other robots, and finally the environment. Considering path planning is one of the key components of an autonomous control system for robotic agents, the works presented in this thesis aim at building a dependable, i.e. safe, reliable and effective, path planning algorithm for a group of robots that share their working space with humans. Firstly, the method for path planning of multiple robotic agents is proposed based on a novel dipole flow field idea. The any angle-path planning with Theta* algorithm is employed to initialise the paths from a starting point to a goal for a set of agents. To deal with static obstacles while a robot is going on the way to its goal, a static flow field along the configured path is defined. A dipole field is then calculated to avoid the collision of agents with the other agents and human subjects. In this approach, each robotic agent is assumed to be a source of a magnetic dipole field in which the magnetic moment is aligned with the moving direction of the agent, with a strength proportional the ix 10

x velocity. The magnetic dipole-dipole interactions between these agents generate repulsive forces to help them to avoid collision. Meanwhile, the fault analysis of multiple robots with Petri net is demonstrated to understand the cooperation of multiple robotic agents to solve the shared tasks. Thereafter, the Petri net is applied together with the path planning to address the collision avoidance by synchronising the movement of robots through a cross. Continu- ously, the multiple path planning has investigated to support fault tolerance for the path planning algorithm. This is to deal with the deadlock situation where the agent takes very long time to reach the goal or even is not able to do so. The agent is equipped with different paths to the goal and proactively switch among the plans whenever needed to avoid the deadlock. Finally, the whole framework has been implemented by a widely used platform, robot operating system (ROS), and evaluated through Gazebo simulator. 11

List of Publications

Papers Included in the Licentiate Thesis1

Paper A Fault Tolerant Analysis for Dependable Autonomous Agents Using Colored Time Petri Nets Authors: Lan Anh Trinh, Baran Cur¨ ukl¨ u,¨ Mikael Ekstrom¨ Status: Published in 9th International Conference on Agents and Artiﬁcial In- telligence, ICAART 2017.

Paper B Toward Shared Working Space of Human and Robotics Agents Through Dipole Flow Field for Dependable Path Planning Authors: Lan Anh Trinh, Mikael Ekstrom,¨ Baran Cur¨ ukl¨ u¨ Status: Published in Frontiers in Neurorobotics, volume 12, May 2018.

Paper C Petri Net Based Navigation Planning with Dipole Field and Dy- namic Window Approach for Collision Avoidance Authors: Lan Anh Trinh, Mikael Ekstrom,¨ Baran Cur¨ ukl¨ u¨ Status: Published in 6th International Conference on Control, Decision and Information Technologies, CODIT 2019.

Paper D Multi-Path Planning for Autonomous Navigation of Multiple Robots in a Shared Workspace with Humans

1The included papers have been reformatted to comply with the thesis layout xi 12

xii

Authors: Lan Anh Trinh, Mikael Ekstrom,¨ Baran Cur¨ ukl¨ u¨ Status: Submitted. 13

xiii

Additional Peer-Reviewed Publications, not Included in the Licentiate Thesis

• ”Dipole Flow Field for Dependable Path Planning of Multiple Agents”. Lan Anh Trinh, Mikael Ekstrom,¨ Baran Cur¨ ukl¨ u.¨ Workshop on Shared Autonomoy, IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017.

• ”Failure Analysis for Adaptive Autonomous Agents using Petri Nets”. Mirgita Frasheri, Lan Anh Trinh, Baran Cur¨ ukl¨ u,¨ Mikael Ekstrom.¨ The 11th International Workshop on Multi-Agent Systems and Simulation (MAS&S’17).

• ”Dependability for Autonomous Control with a Probability Approach”. Lan Anh Trinh, Baran Cur¨ ukl¨ u,¨ Mikael Ekstrom.¨ Newsletter in ERCIM News 109, Autonomous Vehicles. 14 15

Contents

I Thesis 1

1 Introduction 3

2 Backgrounds 7 2.1 DEPENDABILITY AND PETRI NET ...... 7 2.1.1 Dependability ...... 7 2.1.2 Petri nets ...... 10 2.2 PATH PLANNING AND THETA* ...... 11 2.3 DYNAMIC WINDOW APPROACH ...... 14

3 Related Works 17 3.1 DEPENDABILITY FOR AUTONOMOUS CONTROL ...... 17 3.2 PATH PLANNING ...... 19 3.3 MULTIPLE PATH PLANNING ...... 23

4 Research Overview 27 4.1 RESEARCH METHODOLOGY ...... 27 4.2 HYPOTHESIS ...... 29 4.3 RESEARCH QUESTIONS ...... 31 xv 16

xvi Contents

5 Thesis Results 33 5.1 CONTRIBUTIONS AND RESULTS ...... 33 5.2 SUMMARY OF PAPERS ...... 36

6 Conclusions and Future Works 41 6.1 GENERAL CONCLUSIONS ...... 41 6.2 FUTURE WORKS ...... 42

Bibliography 45

II Included Papers 55

7 Paper A: Fault Tolerance Analysis for Dependable Autonomous Agents Using Colored Time Petri Nets 57 7.1 INTRODUCTION ...... 58 7.2 RELATED WORKS ...... 60 7.3 BACKGROUND PROPOSED APPROACH ...... 62 7.3.1 Background of Petri Nets ...... 62 7.3.2 Fault Tolerance with Cooperative Agents ...... 64 7.3.3 CPN Models of Agents for Fault Tolerance Analysis of Centralised Systems ...... 64 7.3.4 CPN Models of Agents for Fault Tolerance Analysis of Distributed Systems ...... 66 7.4 RESULTS ...... 67 7.4.1 Fault Tolerance Analysis of Centralized Systems . . . 67 7.4.2 Fault Tolerance Analysis of Distributed Systems . . . 70 7.5 CONCLUSIONS ...... 71 Bibliography ...... 72

8 Paper B: Toward Shared Working Space of Human and Robotics Agents Through Dipole Flow Field for Dependable Path Planning 81 8.1 INTRODUCTION ...... 83 17

Contents xvii

8.2 METHODOLOGY ...... 89 8.2.1 Autonomous Agent Architecture ...... 89 8.2.2 Path Planning with Dipole Flow Field ...... 91 8.3 EXPERIMENTS ...... 102 8.3.1 Static Flow Field ...... 103 8.3.2 Dipole Flow Field for Crossing Scenarios of Two Agents104 8.3.3 Dipole Flow Field for Multi-Agent and Human-Agent Interaction ...... 106 8.4 CONCLUSION AND DISCUSSION ...... 109 Bibliography ...... 113

9 Paper C: Petri Net Based Navigation Planning with Dipole Field and Dynamic Window Approach for Collision Avoidance 121 9.1 INTRODUCTION ...... 123 9.2 BACKGROUNDS ...... 125 9.2.1 Petri Net (PN) Model ...... 125 9.2.2 Global Path Planning and Theta* Algorithm ...... 126 9.2.3 Dynamic Window Approach ...... 128 9.3 PETRI NET PLANNING AND OBSTACLE AVOIDANCE WITH DIPOLE FIELD AND DWA ...... 129 9.3.1 Overall System ...... 129 9.3.2 Petri Net Planning ...... 130 9.3.3 Dipole Field for Obstacle Avoidance ...... 131 9.4 IMPLEMENTATION AND EXPERIMENTAL RESULTS . . 133 9.4.1 ROS-based Implementation ...... 133 9.4.2 Crossing Scenario of Two Robots ...... 134 9.4.3 Crossing Scenario of Three Robots ...... 137 9.4.4 Scenario of Robots and Humans Sharing Working Space.137 9.5 CONCLUSIONS ...... 138 Bibliography ...... 140 18

xviii Contents

10 Paper D: Multi-Path Planning for Autonomous Navigation of Mul- tiple Robots in a Shared Workspace with Human 145 10.1 INTRODUCTION ...... 147 10.2 RELATED WORKS ...... 149 10.2.1 Multiple Path Planning ...... 149 10.2.2 Collision Avoidance ...... 151 10.3 MULTI-PATH PLANNING WITH OBSTACLE AVOIDANCE 152 10.3.1 Preliminaries ...... 152 10.3.2 Problem Formulation ...... 153 10.4 EXPERIMENTS ...... 158 10.4.1 ROS-based Implementation ...... 158 10.4.2 Two Robots Crossing Narrow Corridor Scenario . . . 159 10.4.3 Scenario with Robots/Humans Together ...... 159 10.5 CONCLUSIONS ...... 160 Bibliography ...... 162 19

Thesis

1 20 21

Chapter 1

Introduction

Until recently, robots have played a critical role in the manufacturing industry where the automatic robots perform repetitive and sometimes heavy task. However, technological advancements in recent years have resulted in a shift of attention from pre-programmed automatic solutions to (semi)-autonomous systems that can operate in unstructured environments, and even coexist with humans. Autonomous control system allows the robot to have freedom of movement as well the ability to directly interact with humans as well as other robots in a collaborative environment. Due to a direct interaction between robot and human, there have been critical needs of setting a high safety level of autonomous robots to prevent any dangerous actions to a user. Meanwhile, a robot must complete the task with correct outputs and within time. Those issues lead to a general deﬁnition of dependability. Dependability, originally, is devised from software development areas and can be stated by Avizienis et al. [1] as ”the ability to deliver service that can justiﬁably be trusted”. The dependability of a system is evaluated by attributes including availability, reliability, safety, integrity and, maintainability. Those attributes could be evaluated by quantitative or qualitative measurements. For instance, reliability is measured by the probability that the system performs correctly during the time

3 22

4 Chapter 1. Introduction interval. The dependability of a system is assessed by one, several, or all attributes mentioned above. Having such dependability attributes is crucial for an autonomous system. Path planning is a core component of a robot system to help the robot to find an optimal path (with respect to a cost function) from a start to a goal. The path is also collision free, meaning that the robot is able to avoid collisions with static obstacles and moving objects in its working environment. Although path planning is well known by numerous researches since 1990s, many solutions of the problem has been restricted to a static environment or a single robot. Recently, there have been raising new challenges for path planning of autonomous controls within Industry 4.0 context. Given examples of autonomous driving cars or autonomous delivery drones, the robotic agents need to deal with changing environments and moving objects, that introduces a high level of uncertainty into the path planning problem. In industry areas, a robot may share the working space with human and other robots, leading to a requirement of cooperation of multiple robots to address the navigation tasks in a global scale. Considering both the importance of solving the path planning in the new context and dependability for autonomous controls, this thesis targets toward a dependable path planning. The main aim is to provide a correct and reliable path planning algorithm for multiple robots and to ensure a safe path planning service to avoid any catastrophic consequences on users, other robots, and finally the environment. Improvement in the robot path planning also con- tributes to other robotic fields. In particular, a motion task of a robot can be constructed as a sequence of path planning in high dimensional configuration space. As a consequence solving path planing induces to another solution to perform a motion operation. The implementation of dependable path planning starts with the understanding of the threats to the system dependability which includes failures, errors, and faults. A failure happens when the service provided by a system does not comply with its specifications. This results in an error, which affects the services, and will/may cause a fault. This fault is thus the main root lead- 23

5 ing to the failure of the system. Therefore, the implementation of a dependable system focuses on different means including fault prevention, fault removal, fault analysis, and fault tolerance. The main objective of this thesis is to develop a comprehensive framework of dependable path planning for multiple robots sharing their working space with humans with appropriate properties and means. The main focuses of the dependability properties are centred on availability, reliability, and safety with the aim toward a correct, reliable, and safe path planning algorithm. With regards to the aforementioned means, the development of the system have been organised by the different stages. In the beginning, the general definition of dependability has been investigated with the specific studies on Petri Nets (PN) as a tool to analyse and predict the faults when multiple robots are working together to perform complicated tasks. Continuously, the path planning framework is designed with the two main modules including global path planning and local path planning, and with external modules to collect information from surrounding environment and to control the movements of robots (the global path planning is to find the optimal path from a starting point to a goal while the local path planning is to help a robot to avoid any collisions with other robots and human in the shared working space). A new dipole flow field approach has been proposed as the realisation of the designed path planning system. As the work in this thesis be- longs to a long term research profile, Dependable Platforms for Autonomous system and Control (DPAC), hosted at Malardalen¨ University to implement dependability for different platforms for autonomous systems and their control, a part of the works to improve the dependability of a navigation system, the fault analysis to verify the proposed path planning algorithm, has been shared and addressed by Gu et al. [2]. With regards to fault prevention which requires the system design should be aligned with a standard framework, the implementation of the system has been transferred into a widely applicable platform, robot operating system (ROS). Continuously, PN is developed for congestion control when several robots are routed to pass through the same place. Finally 24

6 Chapter 1. Introduction relying on a single global path planning could lead to a dead - or live lock situation where the robot takes a very long time to reach a goal or even is not able to do so. This happens in the case of multiple robots moving in a narrow area and they repeat the same moving trajectories through that area again and again without ﬁnding the path to the goal. Fault tolerance with multiple path planning is implemented to help the robots to predict when the dead-or live lock happens and proactively avoid the collisions with them by switching to alternative global paths to the goal.

Thesis Overview

This thesis is divided into two main parts.

• The ﬁrst part is a summary of my research: The objectives of the work are introduced in Chapter 1; Brief basic deﬁnitions and fundamental backgrounds are presented in Chapter 2; The comprehensive surveys of related works are presented in Chapter 3; The methodology and the path to conduct this research are described in Chapter 4; Chapter 5 sum- marises the contributions and results of the thesis; Finally Chapter 6 con- cludes the thesis with further discussion about future works.

• The second part of this thesis is a collection of four publications (paper A-D) presenting the main contributions of the thesis in details. 25

Chapter 2

Backgrounds

This chapter provides the backgrounds related to the research conducted in this thesis. The overview of dependability and Petri nets are introduced in Section 2.1. The details of Theta* algorithm are described on Section 2.2. Finally, the essentials of dynamic window approach are summarised in Section 2.3.

2.1 DEPENDABILITY AND PETRI NET

2.1.1 Dependability

Recently, robots are being widely used as assistant technology in numerous applications where a robot not only performs its tasks alone but also collabo- rates with other robots and humans to complete a complicated task. To trust in collaborative robots, the concept of dependability is deﬁned. Basically, the assessment of system dependability is based on one or several basic attributes including availability, reliability, safety, integrity, and maintainability. The brief deﬁnitions of those attributes are given as follows.

• Availability: The system is always available or ready to provide correct services.

7 26

8 Chapter 2. Backgrounds

• Reliability: This attribute presents the continuity of correct services.

• Safety: The robot system avoids any catastrophic consequences on the users, other robots and the environments.

• Integrity: Avoid improper system alteration. It prevents any unautho- rised access, or modiﬁcation of program or data.

• Maintainability: Ability for a process to undergo modifications and re- pairs. Within the scope of this work, there dependable properties including availability, reliability, and safety are focused. Availability and reliability could be quantifiable by direct measurements. For instance, the measurements for reliability and availability are evaluated with respect to time or the number of system runs. As availability states that the system is always available or ready to provide correct services, it is defined as the probability that the system is functioning correctly at a given instant. Reliability presents the continuity of correct services, so that the system’s reliability at time t is the probability that the time to failure (τ) is greater than t or the probability that the system is functioning correctly during the time interval (0, t]. Meanwhile, safety is conventionally evaluated by qualitative measurements, e.g. safety level or risk factors related to failures. The safety attribute for the robot system is assessed by the ability to avoid any collisions with users, other robots and the environments. The implementation of dependable properties starts with the understanding of the threats affecting the system dependability. There are three main defini- tions with respect to threat that includes failures, errors, and faults. The failure happens when the service provided by a system does not comply with its spec- ification which is, in other word, the expectation of the system functions or services. The error affects the services and leads to the failure of the system where the cause of an error is a fault. The taxonomy showing the relationship between dependability and its components is given in Figure 2.1. As the fault is the root of every failures appearing inside or outside the system, the methods developed to protect the system dependability focus on four 27

2.1 DEPENDABILITY AND PETRI NET 9

Figure 2.1: Dependability taxonomy.

means: Fault prevention deals with appropriate tools by the use of development methodologies, good implementation techniques, and good practices to design the system; Fault removal utilises verification tools through a log-file to identify, diagnose, and remove faults; Fault forecasting estimates the future incidence and likely consequence of faults with safety or risk analysis; Finally fault tolerance is implemented with redundant hardware or software to provide recovery mechanisms if a system fails to perform a task. The first three means usually get involved with the development stages of the system while the last one relates to the operations of the system. The main aim of this work will focus on the development of above means to improve the dependability of an autonomous path planning system. - Fault prevention: encourages the use of standard techniques. In this work, the path planning system is designed step-by-steps following previous works on algorithm developments, analysing with standard tools like PN, testing the system with Gazebo simulators, and finally transferring the implementation of the system on standardised middle ware ROS (Robot operating system). 28

10 Chapter 2. Backgrounds

- Fault removal and Fault forecasting: is to analyse and predict the future incidence and likely consequence to faults. In this thesis, Petri nets is investigated to understand the faults of an autonomous system and a part of the developed path planning system is analysed using UPPAAL tools by Gu et al. [2]. - Fault tolerance: is to avoid service failures in the presence of faults with error detection and system recovery mechanisms. Error detection is necessary to locate any errors during running time of the system. Once an error is found, system recovery mechanisms is activated. For instance, two different decision algorithms are running simultaneously. If one is failed, the other is loaded and replaced the failed one. In this work, multiple path planning algorithm is developed for the sake of fault tolerance.

2.1.2 Petri nets

Various approaches to fault analysis, such as Petri Net (PN), fault tree analysis (FTA), failure modes effects and criticality analysis (FMECA), and hazard operability (HAZOP) have been developed [3]. This research focuses on PN, as this framework provides not only a probability approach for fault analysis and fault prevention in both development and operational stages of designing a system but also for mitigation of the implementation progress. For instance, as an extension of PN, a stochastic Petri net can be combined with Markovian models to evaluate the probability of current state and the probability of future fault events for a fault prognosis process. In this work (Paper A), a coloured time PN is utilised for fault tolerance analysis of multi-agents in a complex and collaborative context. PN is deﬁned in mathematical aspect as a bipartite graph of a set of tuples (P,T,W ), where P = {p1, p2, ..., p|P |} and T = {t1, t2, ..., t|T |} are disjoint sets of places and transitions, |P | and |T | are the number of elements of P and T respectively, and W ⊆ (P × T ) ∪ (T × P ) is a set of arcs connecting from a place to transition and vice versa. The arcs that go out from a place to a transition are named as the input places of transitions. While the 29

2.2 PATH PLANNING AND THETA* 11 ones running out from a transition to place are the output places of transitions. The weights are added to the input and output ﬂows of each transition. With regards to a set of output weights O and a set of input weights I, PN is described as a set of ﬁve tuples G(P, T, W, O, I). Places in a PN may con- sist of a number of marks named tokens. The number of available tokens at place p is represented by M(p), and M is marking vector that denotes availability of tokens at all place p ∈ P . The marking M is expressed as a vector [M(p1),M(p2), ..., M(pi), ..., M(p|P |)], in which pi is a place, |P | is the number of places in PN, and M(pi) is the number of tokens at the place pi. Let

O be a two dimensional matrix of weights O(pi, tj) from the place pi to the transition tj. I is similarly deﬁned by the weight I(tj, pi) from the transition tj to the place pi. It is noted that 1 ≤ j ≤ |T |, where |T | is the total number of transition. Thus, a change of the marking vector of a transition from M to M 0 is given by a ﬁnite sequence of transitions

M 0(p) = M(p) + I(t, p) − O(p, t), ∀p. (2.1)

It is said that the marking M 0 is reached by the marking M by ﬁring t. In general, the marking M 0 is reachable from M by a ﬁnite sequence of k transitions

σ = ti1 ti2 ...tik . With an initial marking M0, the full description of a PN consists of six tuples G(P,T,W,I,O,M0). A full graph of all possible markings and transitions, i.e. a reachability set, is described by state-space analysis. As the number of vertices and edges of the state-space graph increase dramatically with regards to the number of places and transitions, the state-space analysis is limited to a small PN.

2.2 PATH PLANNING AND THETA*

In general, the path planning system of an robot is divided into local path planning and global path planning. The local path planning controls the movements of a robot within a local area once the route of the robot to the goal has been 30

12 Chapter 2. Backgrounds established. Meanwhile, the global path planning needs information regarding the environment e.g. as a scanned map beforehand to generate a global path from a starting point to a desired goal. The global path planning mainly applies for a static environment. This means that in order to compute the global path to a goal for the robot, the presence of the other robots and human, i.e. dynamic objects, are not taken into account, but only static obstacles in the working space are concerned. Conventionally, the algorithms used for planning the global paths are graph/map/grid search-based algorithms. The Dijkstra algorithm [4] and its extension A* [5] are well-known approaches for searching in a map. Recent researches in this field has lead to any-angle path planning algorithms, which are able to find an optimal path in any direction. This algorithm improves A* by adding a line-of-sight (LOS) detection to each search iteration to find a less zigzaggy path than those generated by A* and its other variants. The primary difference between the Theta* and the others is that the Theta* algorithm determines the parent of a node to be any node in the searching space. Thence, the LOS detection feature is purposed to help decrease the undesired expanding by checking for whether the offspring node and the parent are in a straight line, i.e. line-of-sight. By this means, the path found by Theta* is not a connection of adjacent nodes but a connection of line-of-sight ones. The pseudo codes of the Theta* is described in Algorithm 1.

As a heuristic-based search algorithm, Theta* approximates the length of the shortest path based on cost evaluation. The cost evaluation is conducted from the f-value, i.e. the lowest cost from the starting node to the last node, s, in a path, referred to as f(s), and a heuristic value called h-value which is the cost estimation from the node s the goal. The estimated cost of the cheapest node through node s is, thus expressed by f(s0) = f(s) + h(s, s0). (2.2)

In this work, the heuristic function is simply deﬁned as the Euclidean distance i.e., h(s, s0) = w.Euclidean(s, s0) where w is a weight that determines the 31

2.2 PATH PLANNING AND THETA* 13

Algorithm 1 Theta* algorithm procedure THETA*(sstart, sgoal) open ← ∅, closed ← ∅, g[sstart] ← 0 parent[sstart] ← sstart open.insert(sstart, g[sstart] + h[sstart]) while open 6= ∅ do s ← open.pop() if s = sgoal then return ”path found” . The found path is stored in parent[] end if closed ← closed ∪ {s} for s0 ∈ neighbor(s) do if s0 6∈ closed then g[s0] ← ∞ parent[s0] ← NULL end if 0 gold ← g[s ] COSTEVALUATION(s, s0) 0 if g[s ] < gold then if s0 ∈ open then open.remove(s0) end if open.insert(s0, g[s0] + h[s0]) end if end for end while return ”no path found” end procedure procedure COSTEVALUATION(s, s0) if LOS(parent[s], s0) then . LOS check between parent[s] and s0 if f(parent[s]) + h(parent[s], s0) < f[s0] then parent[s0] ← parent[s] f[s0] ← f(parent[s] + h(parent[s], s0) end if else if f[s] + h(s, s0) < f[s0] then parent[s0] ← s f[s0] ← f[s] + h(s, s0) end if end if end procedure 32

14 Chapter 2. Backgrounds size of the area to search for the optimal path around the straight-line between s and s0. With w > 1, Theta* is able to reduce the searching area but may return a longer path, therefore the value w = 1 is used in this work to search for the shortest path. It is assumed that the straight line distance between two nodes would never be longer than any other path connecting them. However, the shortest path generated by the A* algorithm is connected by the neighbouring grid nodes, and thus entails many turning points to the robot. The path found by Theta* is a sequence of LOS nodes so that it is smoother with few turns and closer to a straight-line path between the start and the goal. The algorithm for LOS function is implemented with a drawing-line algorithm in graphics to optimise processing time and is referred to approach proposed by [6].

2.3 DYNAMIC WINDOW APPROACH

Dynamic window approach (DWA) is a local collision avoidance method based on sampling approaches. The DWA was introduced by Fox et al. [7] to use multiple constraints of velocity limits, of acceleration limits, and of following the predeﬁned global path into the local path planning. The local searching space is reduced to dynamic windows in a three-step progress. Firstly, DWA consid- ers the robot’s trajectories as circular trajectories or curvatures determined by a set of translation and rotation velocities (vt, vr). Secondly, only admissible pairs of (vt, vr) corresponding to their trajectories are considered if the robot is able to move forward without colliding with obstacles. Finally, the dynamic window limits the admissible velocities to those that the robot can safely reach to the goal in a short time with optimised accelerations. To do so a following objective function given by equation (10.4) is used to score the admissible ve- ∗ ∗ locities. Only the pair of (vt , vr ) is selected if the objective function reaches 33

2.3 DYNAMIC WINDOW APPROACH 15 to maximum,

∗ ∗ (vt , vr ) = argmaxF (vt, vr) = argmaxf αH(vt, vr)+ (vt,vr ) (vt,vr ) X βD(vt, vr) + γV (vt, vr) + ωiΩi(vt, vr) . (2.3) i

In this equation, H(vt, vr) is a goal heading which is maximised if the robot directly moves towards from the starting point to the goal; D(vt, vr) is a distance from the robot to the closest obstacles on the robot’s trajectory; V (vt, vr) is the forward velocity; and Ωi(vt, vr) are optional cost functions. The smooth function f(·) is used to smoothen the weighted sum of the above components,

α, β, γ and ωi are the weight of each component. 34 35

Chapter 3

Related Works

Related works are categorised into three main ﬁelds including dependability (Section 3.1), path planning (Section 3.2) and multiple path planning (Section 3.3).

3.1 DEPENDABILITY FOR AUTONOMOUS CONTROL

The open challenges for development of the dependability of an autonomous system involves to the fact that the autonomous system must be able to recog- nise, work with, and adapt to human or other robot behaviours in a unstructured and unknown environment. For daily application, Care-O-bot, an intelligent home care robot used to assist elderly people, was introduced by Birgit Graf and Martin Hagel¨ [8]. The proposed home care system was equipped with extensive sensors for motion detection and with alternative levels of safety to prevent accidents caused by a person being hit by the robot. Meanwhile, a dependable platform has been the most concern in industrial robots to create col- laborating working environment between human and robot in manufacturing factories. For instance, ABB has been working on an open robotics platform

17 36

18 Chapter 3. Related Works

[9] with fault analysis tools in order to monitor and ﬁnd the failure robot causes with a testing procedure. Close to the works presented in this thesis, there has been a rise on the development of self-driving cars from big companies like Tesla, Google, Uber or Volvo Car. To minimise the risks of causing accidents to human, autonomous cars would have to tested to be driven up to millions of miles in different conditions of city trafﬁc, busy highways, etc., that could require a lot of time, even more than ten years, to complete [10]. Researches in dependability is investigated to provide a better methodology to demonstrate safety and reliability of self-driving car [11]. As aforementioned, a fault is the root of every failure occurring inside or outside of autonomous system. Different means are introduced to achieve the dependability with respect to the analysis of faults.

• Fault prevention deals with the use of appropriate tools. This can be ac- complished by use of development methodologies, good implementation techniques, good software and good practices to design the system by component-based system such as LAAS [12], RAX [13] or standardised middleware like Robot Operating System (ROS) [14], OROCOS [15], etc.

• Fault removal deals with identifying, diagnosing and removing faults. Fault identifying is performed with verification tools including dynamic verification (run test to find faults through log-file and analysis) [16] and static verification [17] (system analysis, model checking, etc.). Besides, during the operation stage, all failures must be logged for the mainte- nance cycle.

• Fault forecasting deals with estimating the future incidence and likely consequence of faults. Fault forecasting is widely used under the robot developments. If the robot system is implemented with uniﬁed modelling language (UML) [18], then the safety analysis is studied with Hazard and Operability Studies (HAZOP) [19] or Fault Tree Analysis 37

3.2 PATH PLANNING 19

(FTA) [20]. Other common approaches are Bayesian network and Petri net.

• Fault tolerance is implemented by using redundant hardware and software [21, 22, 23].

In the beginning of this work, PN has been studied to understand the fault analysis of the multi-agent system in a complex and collaborative context. The PN framework is chosen because it provides solution for fault analysis and fault prevention in both development and operational stages of a system. Re- lated works PN with dependability are found in [24, 25, 26], PN for system modelling and analysis in [27, 28, 29, 30, 31], PN with collaborative robots in [29, 32, 33, 34], PN to design and implement robots in [35, 34, 36, 30], PN and fault tolerance in [26, 37, 18], and PN with ROS in [38]. In the later part of this work, PN has been utilised to organise the path planning tasks of multiple robotic agents solve conﬂicts and collisions among them.

3.2 PATH PLANNING

With regards to the dependability properties, there are several main goals that we aim to achieve with a dependable path planning algorithm. First, the algorithm is able to correctly drive all robotic agents to reach their goals with optimal paths. Here, the optimal path is deﬁned as a path with a minimum cost calculated based on the total length of the path, the energy used by a robot to complete the path, or the complexity of the terrain over the path, etc. Second, for safety reasons the robot needs to avoid collisions with other robots and humans in its working space. Lastly, the robots are also required to complete the moving tasks within time limits. This means that if the robots are not able to reach the goals or take a long time to do so, it is considered that the robots are in the deadlock situation and failed to complete a path planning task. Re- searches related to the ﬁrst two issues are discussed in this section. So far, numerous path planning approaches have been proposed to address path plan- 38

20 Chapter 3. Related Works ning for multiple robots. Most of them have been focused on searching to find a path from a source to a destination in either static or dynamic map. Mean- while, a family of path planning algorithms address the problem of avoiding moving obstacles with field-based approaches. Regarding the global path planning to find a path from a starting point to a goal, one of the most conventional yet still effective approaches for the navigation of an agent in a large map is related to Dijkstra and its extension of A* searching algorithm [39, 40], and incremental search [41]. In detail, the A* algorithm improves the Dijkstra’s algorithm by approximating the cost- to-go function with heuristic knowledge to reduce the searching space to the goal. Meanwhile, incremental search algorithms seek for the shortest paths by utilising the results of similar searches to make the search faster, instead of solving each search problem separately. By applying incremental search on top of the A*, Koenig et al. [42] developed lifelong planning A* (LPA*) as an initial variant of A*, in order to address path planning for dynamic graphs with changing edge costs. In the D* algorithm [43], incremental search is applied to repeatedly update the shortest paths between the current position of a robot and a goal, during the robot’s approach to the goal. Koenig and Likhachev [44] improved the D* by LPA* and alternatively Sun et al. [45] developed dynamic fringe saving A* to reuse the OPEN and CLOSED lists from previous A* searches. Although different variants of A* are able to address a graph change due to the moving of a robot to a new vertex, or the updates of edge costs, those algorithms still face difficulties to deal with moving obstacles. As stated by Hu and Brady [46], a probabilistic approach is necessary to model the uncertainties of mobile obstacles in the environment. However, the complexity of path planning will be significantly increased if either the cost of the edges, or the links of the graph are presented by random variables. In order to handle the uncertainties of observed obstacles, a field-based approach is of the way to combine the global path planing with obstacle avoidance. The field consists of a repulsive field to push the agent away from the obstacles, and an attractive field to pull the agent towards the goal. For in- 39

3.2 PATH PLANNING 21 stance, Ok et al. [47] proposed Voronoi uncertainty field which is built from Voronoi diagram from the start to the goal to create the attractive field and the repulsive field from the robot to the obstacles. The works of Wang and Chirikjian [48] and later Golan et al. [49] presented an artificial potential field based on the exchanges of heat flow. If obstacles are visualised as hot objects, the target is then presented as the cold one and the temperature is discretised at each location on the grid. The temperature gradient solved by partial differential equation generates the appropriate forces to drive the robot. One of the big issues of using the potential field is that the repulsive field may push the agent to reach other obstacles or statures with the attractive field. Due to these problems, the agent may be trapped into a local optimum or loose its way toward the goal. To mitigate the local converge to a local optimal, some additions to the potential field have been introduced. Valbuena and Tanner [50] proposed the way of adding velocity constraints, meanwhile Garc´ıa-Delgado et al. [51] extended the repulsive function with the change of magnitude dependent on the angle between the attractive force and the obstacle. The main aim is to avoid the cancellation of the repulsive and attractive forces when applied in opposite orientations. However, the interactions of the agents with the environment, especially changes in the map, were not clearly addressed in above mentioned works. Besides, most of field-based navigation approaches lack the global information of a feasible path to the destination that could actually help avoid a trap that would lead to a local optimum.

Controlling the speed and directions of a robot are also key factors, which plays a role to provide the robot a collision free path. Owen and Montano [52, 53] deﬁned velocity space to estimate the arrival time of moving objects to a region of potential collisions and thereby potential solutions to avoid these collisions. The velocity space in which the motion of the robot, as well as static and moving objects are mapped, is applied to predict when the collision may happen and when the robot may escape from the collision. Damas and Santos- Victor [54] developed a map of forbidden velocity zones which is constructed as a limit on the velocity of the robot to avoid collision with obstacles. When 40

22 Chapter 3. Related Works the robot moves into the forbidden zones, it may adjust its speed to avoid the obstacles. Berg et al. [55], Wilkie et al. [56], and Berg et al. [57] further integrated the acceleration while Lee et al. [58] concerned the shape of the robots as an ellipse for obstacle avoidance. Yoo and Kim [59] proposed a modified uni-vector field to present obstacles with respect to relative their velocities and positions where the gaze control which concerned the error of localisation and the distances to surrounding obstacles was also combined into the system to find the best moving trajectory. Belkhouche [60] introduced virtual plane to present moving objects with information of velocity into stationary ones. As a consequence, path finding in a dynamic environments is converted to a simpler problem of navigation in a static environment. However, it is noted that, it is not always optimal to use velocity planning when to drive the robot. Using only velocity control for path planning usually results in oscillatory motion. Given a typical differential drive mobile robot, there are a number of constraints on the linear and angular velocities, as well as the acceleration, in order to save energy for extending operation time, and finding the path to the goals with few turns. To the best of our knowledge, these concerns have not been investigated extensively in combination with obstacle avoidance in dynamic environment. In order to address the above mentioned issues, in this thesis, a novel method for path planning of mobile agents, in the shared working environment of human and agents, called as the dipole flow field, is proposed. The dipole flow field combines both global and local path planning in a unique framework. For global planning, the method applies any-angle path planning algorithm of Theta* [61] to generate smooth paths with few turns, from a starting point to a goal for a pool of agents. Although different A* variants of any-angle path planning haven been proposed, such as A* post smoothing, block A* [62] and field D* [63], the Theta* is able to provide the most optimal path with simple and effective implementation [64]. As the computations of the Theta* algorithm is costly for a big map, the algorithm is updated when there is a sig- 41

3.3 MULTIPLE PATH PLANNING 23 nificant change on the static map of the environment. To cope with dynamic movements of the agents, a static flow field along the planned path is defined to attract the agent back to continue reaching the goal even when the agents may be deviated from the planned path. In addition, a dipole field is used to avoid the collision of the agents with others and human within the shared working space. To the best of our knowledge, most conventional approaches attempt to generate the pushing forces based only on the location of the agents, whereas in this work, it is assumed that, those should be better aligned with both moving directions and velocity magnitudes of different agents. The generated dipole field is able to push other agents far away based on their respective moving directions and the velocity magnitude of the agents. Static flow field and dipole field are combined to assure a dependable path of each agent from the starting point to the goal without colliding with each other.

3.3 MULTIPLE PATH PLANNING

The last problem mentioned in the previous section deals with the deadlock situation where a robot is blocked by human or other robots to reach the goal. In order to address the problem, the mobile robotic agents are equipped with multiple global paths to the goal. Multiple robots can communicate to exchange information about the planned paths and negotiate to find the optimal paths for all robots in a global scale. This section overviews literature researches related to this issue. For the graph-based solutions, the robots move on a connected graph from a vertex to its neighbors in one time step to reach their goals. A conflict happens when two robots occupying on a single vertex at the same time and the main aim of the path planning is to find a conflict-free solution satisfied by all robots. An extra cost function, namely sum-of-cost, like the total maximum time for all robots to reach their goals (or the sum of all path cost) are introduced as an optimal condition for the search. As the problem is shown to be non-deterministic polynomial-time (NP) hard [65], numerous approaches have 42

24 Chapter 3. Related Works chosen to seek for a close optimal solution to reduce processing time. A*-based optimal search finds a non-conflict solution among all combinations of assign- ing k-agent into the graph. To avoid the exponential grows of the state-space with respect to the number of agents, different methods have been applied, e.g. independence detection (ID) [66] to include robots only when necessary or operator decomposition (OD) [66] to treat a move as an operator. Alternative to A*, the increasing cost tree search (ICTS) [67] proposed two-layers including high-level and low-level searching where the lower is used as a goal test of the higher. Another solution different from A*, conflict based search (CBS) is introduced by Sharon et al. [68]. In CBS, the agents are constrained by a triple of parameters including the agent, occupying vertex, and time step. It means that the agent at the particular time step is refused to occupy an occupied vertex. The path is found only if all agent’s constraints are satisfied. The searching is completed when the paths for every agents are resolved. Beside above solutions, there have been suboptimal solutions for multiple agent path planning. For instance, hierarchical cooperative A* (HCA*) [69] introduced a reservation table which is used to store the path assigned into an agent. The other agents according to their priority in the group search for their paths not registered in the reservation table and update the table accordingly after the paths are found. In an improved version of HCA* like Windowed- HCA* (WHCA*) [69], the reservation table is only applied for a limited time slot, i.e., window, when the other agents are rejected to reserve to the table. Later, in the work of Bnaya and Felner [70], a dynamic window focused around conflicts and agents likely to be involved to a conflict are prioritised to be processed next. In overall, the heuristic search A* and its variants are still costly computational solutions. There have been researches developed to reduce the running time of the search-based algorithms with rule-based algorithms. Specific rules are defined for the movement of the agents to reduce searching time. Yet, the resulted paths from the rule-based algorithms are not always optimal. Alternatively, in the work of Yu and Lavalle [71], the path planning problem for multiple 43

3.3 MULTIPLE PATH PLANNING 25 agents is modeled as a network ﬂow and the collision-free paths are found by the integer linear programming (ILP) solver. Most of conventional works on multiple path planning are based on an assumption of a working environment without the presence of human. Due to the safety reasons, the operation of robot will be terminated when a human enters the working zones or crosses the moving trajectories of robot. In this thesis work, a new method of multiple path planning is proposed to consider the human and other uncontrolled moving objects as factors into the path planning problem. This helps to enhance autonomous functions of robot navigation to allow more ﬂexibility of robots to continue working even with the presence of other robots in unfamiliar environment. 44 45

Chapter 4

Research Overview

In this chapter, the overall reviews of this research are presented. Section 4.1 shows the way to formulate the research. Hypotheses of the works are stated in Section 4.2. Finally, the research questions to be addressed thoroughly the thesis are given in Section 4.3.

4.1 RESEARCH METHODOLOGY

To tackle the problem of implementing dependable path planning algorithms for an autonomous control system, the conventional research methodology of inductive reasoning has been followed. Firstly, the comprehensive surveys of researches related to dependability, autonomous controls, and conventional path planning algorithms have been conducted. As a consequence, a new path planning algorithm using dipole ﬂow ﬁeld is derived to navigate multiple robots to their goals considering the presence of humans and obstacles. PN is also investigated to analyse the fails of the systems with multiple robotic agents working together to complete a task and the roles of a backup plan for fault tolerance. From the theory ideas, hypotheses are stated and experiments are performed to evaluate those hypotheses. The experiments are tested with

27 46

28 Chapter 4. Research Overview sophisticated and comprehensive simulators like Gazebo. With experimental results, the theory, research questions and hypotheses can be evaluated and re- vised/reformulated for the next evaluation (Figure 4.1). For instance, there is a drawback with a single path planning leading to a deadlock situation where robots are not able to reach their goals. A new theory of multiple path planning has been deﬁned to address this problem and new research circle is repeated. Once achieving positive results in simulation environment, a real implementation is transformed into a real robot, like ROS-based robot Husqvarna Research Platform (HRP).

Figure 4.1: Research methodology. 47

4.2 HYPOTHESIS 29

4.2 HYPOTHESIS

The research proposed in this work will be investigated to develop a dependable navigation system for a group of autonomous robotic agents, which are able to plan their moving path to reach their goals and to avoid collision with the other moving (and static) subjects, especially human involved in a shared working environment. Correspondingly, hypotheses are defined at every stage of researches. First, the studies about dependability and a method to analyse the faults of a system of multiple robotic agents to work on shared tasks are performed. Inside the system, a group of agents are designed to solve a complicated task, which consists of a sequence of small subtasks. Each subtask can be solved by several agents therefore if one agent fails, there is another agent able to replace the failed agent to solve the unfinished subtasks. However, the time to finish the complicated task may be prolonged if the time to recover the failed agent is longer. For such a scenario, the design of the autonomous agent will be modelled by a network of nodes using PN to test the following hypothesis. Hypothesis 1: In a system where a complicated task can be divided into small subtasks and a subtask can be processed by at least two robotic agents, the time to complete the complicated task is longer if the recovering time of a failed robotic agent is longer. In the second stage of this research, a new navigation algorithm for multiple robotic agents based on dipole flow field is proposed. The method employs the any angle-path planning with Theta* algorithm to initialise the paths from a starting point to a goal for a set of agents. To deal with the movements of the agents, a static flow field along the configured path is defined. This field is used by the agents to navigate and reach their goals even if the planned trajectories are changed. Secondly, a dipole field is calculated to avoid the collision of agents with other agents and human subjects. In this approach, each agent is assumed to be a source of a magnetic dipole field in which the magnetic moment is aligned with the moving direction of the agent. The magnetic dipole- 48

30 Chapter 4. Research Overview dipole interactions between these agents generate repulsive forces to help them to avoid collision. The second hypothesis is stated to evaluate the proposed path planning approach as follows: Hypothesis 2: Using the path planning algorithm with the combination of the dipole field and flow field, it is possible to correctly drive the robot to its desired goals and avoid the collisions of the robot with humans, other robots and obstacles on the moving way (the collision happens if the distance between a robot and the target is smaller than a predefined threshold). With regards to the studies about PN and the new path planning algorithm with dipole field, the research in this proposal is continued with the uses of PN for congestion control. Obviously, the navigation of several robots into one narrow area may make them to block each other and harder to find the way out from the area or to take longer time to reach their goals. To address the congestion of routing multiple robots into one place, a group of robots should proactively define effective routes to avoid conflicts with each other. As PN is an effective tool in dependable autonomous control to resolve conflicting problems, it is able to apply PN to plan the paths of multiple robots with a delay to avoid routing many robots into a same place so that the robot may not need to turn around with longer paths to approach to their goals. The next hypothesis to be tested is formulated as: Hypothesis 3: Considering intersections of paths as conflict resources among robots, it is able to use PN to synchronise the movements of robots to allow robots to cross through an intersection one by one. The above-mentioned path planning algorithms have mainly addressed the navigation problem for multiple robots with a single path planning. Relying on a single global path planning could lead to a deadlock situation where a robot takes a very long time to reach a goal or even is not able to do so. This happens in the case of multiple robots moving in a narrow area. Since the local path planning to avoid obstacle only takes into account the collision of other robots in the vicinity, a robot must turn back to the predefined path to reach a goal. However, if two robots are routed through a very narrow area, like a corridor, 49

4.3 RESEARCH QUESTIONS 31 and enter it through two opposite sites, robots may face a deadlock situation where they repeat the same moving trajectories through that area again and again without ﬁnding the path to the goal. Considering human as a big obstacle due to safety reason could block a robot to enter a small way to follow its global path. Therefore, it is important for a robot to prepare different solutions to reach a goal and the robot needs to proactively switch between solutions whenever necessary to avoid dead lock situation. The fourth hypothesis is stated as: Hypothesis 4: If multiple path approach/algorithm is used, it is able to avoid the deadlock situation or the livelock situation, i.e. agents ”dance” back and forth when directly opposing each other. so that robots are able to reach the goals within time limits.

4.3 RESEARCH QUESTIONS

The following research questions (RQs) are going to be addressed within the scope of this research:

1. RQ1: How to deﬁne the dependability within the context of an autonomous control system? What methods can be used to analyse the dependability of a system? Which means are used to implement a dependable system of multiple agents that will operate in a shared working space with humans? The answers for those questions provide understanding on the main char- acteristics of a dependable autonomous control system and ﬁnd a right tool to develop that system in general.

2. RQ2: How to deﬁne a dependable path planning algorithm for multiple robots, which help to provide safe and reliable paths for a system of multiple robots? RQ2 is a followup of RQ1 to apply dependability properties on a path planning algorithm. The answers of this question should be put in a 50

32 Chapter 4. Research Overview context where human and robots are moving together on the same work- places. Therefore, safety is an important factor that needs to be addressed in the navigation algorithm to avoid the collisions between robots and human. The complexity of environment is also considered to reach a robust solutions that are not limited to a restricted space.

3. RQ3: How to avoid a deadlock situation through multiple path planning and congestion control? The answers to this question aim toward a path planning with fault tolerance. It means if a robotic agent fails to do a task of moving to the target (deadlock situation), there should be a backup plan for it to recover its navigation activities. Also, the uses of congestion control could help a group of robotic agents to proactively synchronise their movements through a cross intersection to avoid congestion.

4. RQ4: How to integrate the proposed path planning algorithm on real robots? What case studies with real autonomous robots can be used to test the whole system? Many challenges related to a realistic environment need to be addressed. They include how to choose a right framework to utilise existing developed solutions, how to detect and monitor the moving trajectories of humans and other objects, how to build a map and localise the robot in there, and how to develop a communication method to share information among robots etc. 51

Chapter 5

Thesis Results

5.1 CONTRIBUTIONS AND RESULTS

The works done by the thesis have led to four main contributions to the researches of dependable path planning. First, the general theory about dependability has been investigated to find the fundamental properties and means to achieve a dependable path planning algorithm. Second, a new path planning algorithm with dipole flow field has been developed. Third, the new application of PN on path planning algorithm to avoid congestion has been proposed. The final contribution is multiple path planning with its role to improve a dependable path planning with fault tolerance. Dependability The general theory about dependability has been investigated to find the fundamental properties and means to achieve a dependable autonomous system and later utilise those for path planning algorithms. A PN framework is designed for failure analysis of an autonomous system that concerns the cooperation of multiple robots to address a complicated task. In the proposed system, it is assumed that there is a pool of robotic agents

A = (a1, a2, ..., aN ), where N is the number of agents. The agents collaborate to solve a complex task Tt in which the tas Tt is divided into multiple subtasks

33 52

34 Chapter 5. Thesis Results

(t1, t2, ..., tK ). A parameter, τp, is called as processing time which is the average time of an agent ai to complete a subtask. The agent fails with probability modelled by an exponential distribution exp(λe), where λe is an average fail mean. When the agent fails, it is required to recover the normal activity of the agent. Therefore, a parameter, τr, is defined as an average of recovering time for the agent to be back to normal activity. Thus, the tuples (τp, λe, τr) are configurable parameters of the system, which are used to evaluate the performance of fault analysis and the performance of fault tolerance. A simple case study was presented with three agents (a1, a2, a3) working together to solve a complicated task which is divided into three subtasks (t1, t2, t3). Each agent can take care several subtasks (e.g., a1 can handle t1 and t2). For fault tolerance, the definition of peer agent is introduced. The two agents are peer agents if they are able to solve a common task. Thanks to the availability of multiple peer agents, once agent fails to do a subtask, other peer agents with the similar function are used to continue the unfinished tasks. In our simulation, the ar- chitectures of agents are modelled for both centralised and distributed systems. The simulation results have showed the correlation of the processing time to finish a complicated task with the failure rates of an agent. Besides, the experiments revealed that for distributed agents, the communication protocol also played an important role on the fault tolerance success of the whole system.

Path planning algorithm A new path planning algorithm with dipole flow field has been developed. The algorithm is able to process path planning in real-time by developing a navigation field so that the movements of agents is just simply controlled by the forces generated from this field. The attractive forces that drive the agents toward their desired goals are created by a static flow field. Simultaneously, the repulsive forces that prevent agent-agent, and human-agent, collisions are generated by a magnetic field of dipoles. The combination of the static flow field and dipole field forms a force to determine the moving directions of the agents at a specific time instance. The evaluation of the proposed approach with the static flow field, dipole field and their combinations are conducted with distinctive experiments. 53

5.1 CONTRIBUTIONS AND RESULTS 35

For a path planning algorithm, it is crucial to evaluate three dependable properties including reliability, availability, and safety. The reliability is assessed by the correctness of the path planning algorithm, i.e. robots are pre- cisely navigated to the defined goal. The availability is presented by the continuity of path planning services, e.g. not facing a dead-lock situation so that the robots are no longer to be able to operate. Final, safety is qualitative by the ability of robots to avoid collisions with any static and moving obstacles. With static flow field, experimental results have shown that an agent is able to move to its goals in a binary map of static obstacles with a small number of re-initialising the global path using the Theta* algorithm. Within the combined dipole-flow field, the robotic agents are well routed to their destinations, while possible collisions with other agents and human are taken into account to protect robots and environments from damages caused by a hit and to ensure safety to operating users. However, routing several robots into a narrow area, e.g., a corridor, might lead to a dead-live lock situation where robots try to avoid collisions with each other but not able to escape from the trapped region. One solution to address this problem is locating potential areas that have high possibility to cause traffic jams and controlling the movements of robots through that area with the following contribution.

Congestion control with PN The application of PN is utilised to control movement of the robots when they are routed through a cross or a narrow area. The cross/intersection region of the moving paths are considered as a conflict resource so that a PN is an appropriate approach to avoid traffic congestion among robot. A PN is applied to control the movements of robots at every intersection by organising one by one passing through a cross. Those PN analyses and controls are available with robots when they are able to communicate and share information with each other. For humans and other moving objects, the previous work with dipole field is integrated with dynamic window approach is implemented based on the local planning of the system to help robots to avoid collisions. By regarding the velocity and direction of such dynamic obstacles as a source of a virtual magnetic dipole moments, the dipole-dipole interaction 54

36 Chapter 5. Thesis Results between the moving objects will generate repulsive forces to prevent collisions. Analyses and experiments demonstrated with the Gazebo simulator have revealed that the PN control is able to synchronise the movements of multiple robots passing through the intersection, which helps to shorten the travelling paths of the robots. Meanwhile, the dipole ﬁeld implemented on DWA is able to advance the local path planning with an ability to avoid moving humans and other robots in the shared workspace. Multiple path planning The next contribution is the proposed multiple path planning and its role to improve dependable path planning with fault tolerance. Relying on a single global path planning could lead to a dead - or live lock situation which happens when multiple robots moving in a narrow area and they repeat the same moving trajectories through that area again and again without ﬁnding the path to the goal. Thanks to the uses of back-up planned paths, the robot is able to switch to another planned path when the currently planned path could lead to the block. This helps to improve the availability of the system with the ability to provide correct services even when the system is failed to complete their planning tasks.

5.2 SUMMARY OF PAPERS

The contributions and results are detailed in four articles (Paper A-D) that are summarised as follows

Paper A Fault tolerance has become more and more important in the development of autonomous systems with the aim to help the system to recover its normal activities even when some failures happen. Yet, one of the concerns is how to analyse the reliability of a fault tolerance mechanism with regards to the collaboration of multiple agents to complete a complicated task. To do so, an approach of fault tolerance analysis with the coloured time Petri net framework is proposed in this work, where a task can be represented by a tree of different concurrent and dependent sub-tasks to assign to agents. Different sub-tasks 55

5.2 SUMMARY OF PAPERS 37 and agents are modelled by colour tokens in Petri network. The time values are added to evaluate the processing performance of the whole system with respect to its ability to solve a task with fault tolerance ability. The coloured time Petri nets are then tested with simulation of centralised and distributed systems. Finally the experiments are performed to show the feasibility of the proposed approach. From the basics of this study, a generalised framework in the future can be developed to address the fault tolerance analysis for a set of agents working with a sophisticated plan to achieve a common target.

Paper B Recent industrial developments in autonomous systems, or agents, which assume that humans and the agents share the same space or even work in close proximity, open for new challenges in robotics, especially in motion planning and control. In these settings, the control system should be able to provide these agents a reliable path following control when they are working in a group or in collaboration with one or several humans in complex and dynamic environments. In such scenarios, these agents are not only moving to reach their goals, i.e. locations, they are also aware of the movements of other entities to find a collision-free path. Thus, this paper proposes a dependable, i.e. safe, reliable and effective, path planning algorithm for a group of agents that share their working space with humans. Firstly, the method employs the Theta* algorithm to initialise the paths from a starting point to a goal for a set of agents. As Theta* algorithm is computationally heavy, it only reruns when there is a significant change of the environment. To deal with the movements of the agents, a static flow field along the configured path is defined. This field is used by the agents to navigate and reach their goals even if the planned trajectories are changed. Secondly, a dipole field is calculated to avoid the collision of agents with other agents and human subjects. In this approach, each agent is assumed to be a source of a magnetic dipole field in which the magnetic moment is aligned with the moving direction of the agent. The magnetic dipole-dipole interactions between these agents generate repulsive forces to help them to avoid collision. The effectiveness of the proposed approach has 56

38 Chapter 5. Thesis Results been evaluated with extensive simulations. The results show that the static flow field is able to drive agents to the goals with a small number of requirements to update the path of agents. Meanwhile, the dipole flow field plays an important role to prevent collisions. The combination of these two fields results in a safe path planning algorithm, with a deterministic outcome, to navigate agents to their desired goals.

Paper C This paper presents a novel path planning system for multiple robots working in an uncontrolled environment in the presence of humans. The approach combines the use of Petri net to plan the movement of multiple robots to prevent the risk of congestion caused by routing several robots into a narrow region, together with a dipole ﬁeld with dynamic window approach to avoid collisions of a robot with dynamic obstacles. By regarding the velocity and direction of both humans and robots as a source of magnetic dipole moment, the dipole-dipole interaction between the moving objects will generate repulsive forces to prevent collisions. The whole system is presented on robot operating system platform so that its implementation can be extendable into real robots. Experimental results with Gazebo simulator demonstrates the effectiveness of the proposed approach.

Paper D Path finding for multiple robots is one of most important problems in robotics when to find a way to move robots from their starting positions to reach their respective goals without collisions. However, in the case of a complex environment with the presence of humans and other unpredictable moving objects, fixing a single path to the goal may lead to a situation where there are a lot of obstacles and the robots may fail to realise the moving plan. To address this issue, a new approach of using multiple path planning where each robot has different options to choose its path to the goal is introduced in this paper. The information about planned moving paths are shared among the robots in the working domain, combined with obstacle avoidance constraints in local ranges, and formulated as an optimisation problem. Solution of the 57

5.2 SUMMARY OF PAPERS 39 problem leads to the optimal moving plans of robots. The effectiveness of the proposed approach is demonstrated by experimental results. 58 59

Chapter 6

Conclusions and Future Works

6.1 GENERAL CONCLUSIONS

Nowadays, path planning as well as navigation are becoming an important components of an autonomous robot system. Yet solving the path planning problem is facing more and more challenges due to new requirements and com- plexities of autonomous robots to cooperate with each others and even with human to do a task. What are the key measurements to evaluate whether a path planning system is good or bad? What are the tools to analyse a path planning algorithm? What are the means to achieve a better path planning system? etc. Those questions have led to the works performed in this thesis toward dependable path planning algorithms. A technical researches about dependability and its roles in autonomous control has been conducted. Corresponding, three main dependable properties including reliability, availability, and safety are are considered for a path planing algorithm within the scope of this thesis. To aim at a dependable path planning system, the roots to affect those dependable properties are determined by the faults that could happen due to human errors, wrong

41 60

42 Chapter 6. Conclusions and Future Works designs, or unexpected effects from surrounding environments. Although it is infeasible to achieve a perfect system free from any faults, different means have been revealed to improve the dependability of a system including fault prevention, fault removal, fault forecasting, and fault tolerance. From above information, the pathway towards the dependable path planning algorithm has been established to drive the researches in the rest of this work. A fault analysis with a group of multiple robots working together to complete a shared task has been investigated using PN. The thesis is continuous with the developments of a safe, reliable, and effective path planning algorithm for a group of robotic agents that share their working space with humans: The method for path planning of multiple robotic agents is based on a novel dipole flow field including static flow field and dynamic dipole field. The results show that the static flow field is able to drive agents to the goals with a small number of requirements to update the path of agents. Meanwhile, the dipole field plays an important role to prevent collisions. The combination of these two fields results in a safe path planning algorithm, with a deterministic outcome, to navigate agents to their desired goals. PN is later applied to manage the path planning tasks of multiple robots to reduce congestion. The researches of this work is continued with multiple path planning with an aim to help multiple robots avoid the deadlock situation. The key dependable properties including reliability, availability, and safety are evaluated through extensive simulations with Gazebo. Finally, the implementation of the proposed algorithm has been transferred into ROS platform that can be extended to be used in real robots. In overall, the works presented in this thesis have established a pathway and achieved important im- provements to reach toward dependable path planning system.

6.2 FUTURE WORKS

Promising research directions have been planned for the next stage of this work as follows. Implementation on real robots. In this thesis, the design for the depend- 61

6.2 FUTURE WORKS 43 able path planning algorithm and the theoretical analysis are developed with ROS and evaluated through Gazebo simulation. In the next stage, the imple- mentations of the proposed path planning algorithm will be performed on real robots, e.g., Husqavar platform, Husky, or Turtlebot 3. Besides, the demon- strations in real world scenarios will be planned. Extension to distributed algorithm. So far, some parts of this work including PN congestion control and multiple path planning have been implemented by a centralised control manner. All robots need to communicate to a central node to share dynamic information and receive the optimal commands. However, this poses a risk of loosing control if the central node stops working. Therefore, the distributed implementation is important where each robot in the system will have its own control unit so that the robot itself can make decision without dependent on others. The dependability of the whole system is still guaranteed even a part of robots fail with navigation tasks. Extension of fault tolerance/congestion control mechanism. Currently, alternative paths to the goal have been established to provide a backup path if the robot no longer follows the current path. The works can be improved by utilising different algorithms to generate global paths, even including random factors (e.g. RRT algorithm) to provide more variations of the routes to the goal. A hardware solution for fault tolerance will be also considered. Meanwhile congestion control should focus more on potential areas with high probability of trafﬁc jams instead of applying control on every intersections to reduce complexity of the system. Machine learning in path planning. Last but yet important, learning techniques, such as deep reinforcement learning, can be investigated to make path planning decision more accurate by learning experiences from collected data. Besides, once the system is distributed, ﬁnding a right decision will be a heavy task for each independent robot. The path planning task can be mitigated by searching information for learning/collecting data. In another aspect, applying learning methods for data fusion from extensive sensors could also help to achieve a higher level of dependable path planning for a multi-robot system. 62 63

Bibliography

[1] A. Av ˇi zienis, J. Laprie, B. Randell, and C. Landwehr. Basic concepts and taxonomy of dependable and secure computing. IEEE Transaction on Dependable and Secure Computing, 1:11–33, 2004.

[2] R. Gu, R. Marinescu, C. Seceleanu, and K. Lundqvist. Formal veriﬁcation of an autonomous wheel loader by model checking. In Formal Methods in Software Engineering, June 2018.

[3] S. Bernardi, J. Merseguer, and D. C. Petriu. Model-Driven Dependability Assessment of Software Systems. SPRINGER, 2013.

[4] E. W. Dijkstra. A note on two problems in connexion with graphs. Nu- merische Mathematik, 1:269–271, 1959.

[5] P. E. Hart, N. J. Nilsson, and B. Raphael. A formal basis for the heuristic determination of minimum cost paths.

[6] A. Nash, K. Daniel, S. Koening, and A. Felner. Theta*: Any angle path planning on grids. Journal of Intelligent Robot System, 39:533–579, 2010.

[7] D. Fox, W. Burgard, and S. Thrun. The dynamic window approach to collision avoidance. In IEEE Robotics and Automation Magazine, pages 23–33, 1997.

45 64

46 Bibliography

[8] G. Birgit and H. Martin. Dependable interaction with an intelligent home care robot. In In proceedings of ICRA - Workshop of Technical Challenge for Dependable Robots in Human Environments, pages 21–26, 2001.

[9] M. Goran, A. Johan, and N. Christer. A dependable real-time platform for industrial robotics. In International Conference on Software Engineering, ICSE ’03, 2003.

[10] Kalra, Nidhi, and S. M. Paddock. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? In Research Report, Santa Monica, CA: RAND Corporation, 2016.

[11] P. Koopman and M. Wagner. Challenges in autonomous vehicle testing and validation. In SAE World Congress, 2016.

[12] R. Alami, R. Chatila, S. Fleury, M. Ghallab, and F. Ingrand. An architecture for autonomy. International Journal of Robotics Research, 17(4):315–337, 1998.

[13] N. Muscettola, P. P. Nayak, B. Pell, and B. C. Williams. Remote agent: To boldly go where no ai system has gone before. Artiﬁcial Intelligence, 103(1–2):5–47, 1998.

[14] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng. Ros: An open-source robot operating system. In Workshop on open source software, IEEE International Conference on Robotics and Automation (ICRA), pages 5–11, 2009.

[15] H. Bruyninckx. Open robot control software: the orocos project. In In IEEE International Conference on Robotics and Automation (ICRA), pages 2523–2528, 2001.

[16] D. Powell, J. Arlat, H. N. Chu, F. Ingrand, and M. O. Killijian. Testing the input timing robustness of real-time control software for autonomous system. In In European Dependable Computing Conference (EDCC), pages 73–83, 2012. 65

Bibliography 47

[17] M. O’Brien, R. C. Arkin, D. Harrington, D. Lyons, and S. Jiang. Au- tomatic veriﬁcation of autonomous robot missions. In In International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR), Springer, pages 462–473, 2014.

[18] B. Lussier, A. Lampe, R. Chatila, F. Ingrand, O. Killijian, M., and D. Powell. Fault tolerant planning: towards dependable autonomous robots. In Research Report, LAAS-CNRS, 2015.

[19] B. Petr and G. Thomas. A novel hazop study approach in the rams analysis of a therapeutic robot for disabled children. In Computer Safety, Reliability, and Security, pages 15–27, 2010.

[20] K. Vaurio, J. Fault tree analysis of phased mission systems with repairable and non-repairable components. Reliablility Engineering and System Safety, 74(2):169–180, 2001.

[21] E. Troubitsyna and K. Javed. Towards systematic design of adaptive fault tolerance systems. In The Sixth International Conference on Adaptive and Self-Adaptive Systems and Applications, IARIA, 2014.

[22] D. S. Scott and B. S. Fred. Automated analysis of fault-tolerance in distributed systems. Journal of Formal Methods in System Design, 26, 2005.

[23] B. Lussier, A. Lampe, R. Chatila, F. Ingrand, O. Killijian, M., and D. Powell. Fault tolerance in autonomous systems: How and how much? In In Proceedings of the 4th IARP/IEEE-RAS/EURON Joint Workshop on Technical Challenge for Dependable Robots in Human Environments, 2005.

[24] M. Kohl´ık. Dependability models based on petri nets and markov chains. In Information Science and Computer Engineering, 1st Class, Full-time study, 2009.

[25] M. Malhotra and K.S. Trivedi. Dependability modeling using petri-nets. IEEE Transactions on Reliability, 44(3), 1995. 66

48 Bibliography

[26] S. Bernardi, A. Bobbio, and S. Donatelli. Petri nets and dependability. LNCS Springer, 2004.

[27] B. Samanta and B. Sarkar. Application of petri nets for systems modeling and analysis. In OPSEARCH, 2012.

[28] D. Bera. Petri nets for Modeling Robots. PhD Dissertation, 2014.

[29] M. Thabet, A. Montebelli, and V. Kyrki. Learning movement synchro- nization in multi-component robotic systems. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 249–256, Stockholm, Sweden, May 16–21 2016.

[30] L. Iocchi, M. T. Lazaro, L. Jeanpierre, A.-I. Mouaddib, and H. Sahli. Coaches cooperative autonomous robots in complex and human popu- lated environments. In LNCS Springer, 2015.

[31] G. Kim, W. Chung, S.-K. Park, and M. Kim. Experimental research of navigation primitive selection using generalized stochastic petri nets (gspns) for a tour-guide robot. In Proceedings of the IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems (IROS), Alberta, Canada, August 2–6 2005.

[32] R. Lill and F. Saglietti. Model-based testing of cooperating robotic systems using coloured petri nets. In ERCIM/EWICS Workshop on Depend- able Embedded and Cyber-physical Systems, 2013.

[33] W. Sheng, Q. Yang, and N. Xi. Modeling, analysis and design for multi- robot exploration based on petri nets. In IEEE/ASME International Con- ference on Advanced Intelligent Mechatronics, 2005.

[34] L. Iocchi, L. Jeanpierre, M. T. Lazaro, and A.-I. Mouaddib. A practical framework for robust decision-theoretic planning and execution for service robots. In The 26th International Conference on Automated Planning and Scheduling, pages 486–494, 2016. 67

Bibliography 49

[35] G. Yasuda. Discrete event behavior-based distributed architecture design for autonomous intelligent control of mobile robots with embedded petri nets. Advances in Chaos Theory and Intelligent Control, Springer, 37, 2016.

[36] L. Iocchi, L. Jeanpierre, M. Teresa Lazaro, and A.-I. Mouaddib. Person- alized short-term multi-model interaction for social robots assisting users in shopping malls. In LNCS Springer, 2015.

[37] E. Miyagi, P. and A. M. Riascos, L. Modeling and analysis of fault- tolerant systems for machining operations based on petri nets. Control Engineering Practice, 14, 2006.

[38] J-C. Fabre, M. Lauer, M. Roy, M. Amy, W. Excoffon, and M. Stoicescu. Towards resilient computing on ros for embedded applications. In 8th Eu- ropean Congress on Embedded Real Time Software and Systems (ERTS), 2016.

[39] T. H. Cormen, C. E. Leisercon, R. L. Rivest, and C. Stein. Introduction to Algorithms, Third Edition. The MIT Press, 2009.

[40] D. S. Yershov and S. M. LaValle. Simplicial dijkstra and A* algorithms for optimal feedback planning. In Proceedings of the IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), pages 3862– 3867, San Francisco, CA, USA, September 25-30 2011.

[41] S. Koenig, M. Likhachev, Y. Liu, and D. Furcy. Incremental heuristic search in AI. Journal of AI Magazine, 25:99–112, 2004.

[42] S. Koenig, M. Likhachev, and D. Furcy. Lifelong planning A*. Journal of Artiﬁcial Intelligence, 155:93–146, 2004.

[43] A. Stentz. Optimal and efﬁcient path planning for partially-known environments. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 3310–3317, San Diego, CA, USA, May 8–13 1994. 68

50 Bibliography

[44] S. Koenig and M. Likhachev. Fast replanning for navigation in unknown terrain. IEEE Transactions on Robotics, 21:354–363, 2005.

[45] X. Sun, W. Yeoh, and S. Koenig. Dynamic fringe-saving A*. In Proceed- ings of the 8th International Conference on Autonomous Agents and Mul- tiagent Systems, pages 891–898, Budapest, Hungary, May 10–15 2009.

[46] H. Hu and M. Brady. Dynamic global path planning with uncertainty for mobile robots in manufacturing. IEEE Transactions on Robotics and Automation, 13(5):760–767, October 1997.

[47] K. Ok, S. Ansari, B. Gallagher, W. Sica, F. Dellaert, and M. Stilman. Path planning with uncertainty: Voronoi uncertainty ﬁelds. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 4581–4586, Karlsruhe, Germany, May 06–10 2013.

[48] Y. Wang and G. S. Chirikjian. A new potential ﬁeld method for robot path planning, April 2000.

[49] Y. Golan, S. Edelman, A. Shapiro, and E. Rimon. Online robot navigation using continously updated artiﬁcial temperature gradients. IEEE Robotics and Automation Letters, 2(3):1280–1287, 2017.

[50] L. Valbuena and H. G. Tanner. Hybrid potential ﬁeld based control of differential drive mobile robots. Journal of Intelligent Robot Systems, 68(3):307–322, December 2012.

[51] L. A. Garc´ıa-Delgado, J. R. Noriega, D. Berman-Mendoza, A. L. Leal- Cruz, A. Vera-Marquina, R. Gomez-Fuentes,´ A. Garc´ıa-Juarez,´ A. G. Rojas-Hernandez,´ and I.E. Zald´ıvar-Huerta. Repulsive function in potential ﬁeld based control with algorithm for safer avoidance. Journal of Intelligent Robot Systems, 80(1):59–70, October 2015.

[52] E. Owen and L. Montano. Motion planning in dynamic environments using the velocity space. In Proceedings of the IEEE/RSJ International 69

Bibliography 51

Conference on Intelligent Robots and Systems (IROS), pages 2833–2838, Edmonton, Canada, August 2–6 2005.

[53] E. Owen and L. Montano. A robocentric motion planner for dynamic environments using the velocity space. In Proceedings of the IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems (IROS), pages 4368–4374, Beijing, China, October 9–15 2006.

[54] B. Damas and Jose´ Santos-Victor. Avoiding moving obstacles: the forbidden velocity map. In Proceedings of the IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS), pages 4393–4398, St. Louis, USA, October 11-15 2009.

[55] J. V. D. Berg, M. Lin, and D. Manocha. Reciprocal velocity obstacles for real-time multi-agent navigation. In Proceedings of the IEEE Inter- national Conference on Robotics and Automation (ICRA), pages 1928– 1935, Pasadena, CA, USA, May 19–23 2008.

[56] D. Wilkie, J. V. D. Berg, and D. Manocha. Generalized velocity obstacles. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5573–5578, St. Louis, USA, October 11–15 2009.

[57] J. V. D. Berg, J. Snape, S. J. Guy, and D. Manocha. Reciprocal collision avoidance with acceleration-velocity obstacles. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 3475–3482, Shanghai, China, May 9–13 2011.

[58] B. H. Lee, J. D. Jein, and J. H. Oh. Velocity obstacles based local collision avoidance for holonomic elliptic robot. Journal of Autonomous Robots, 41:1347–1363, 2017.

[59] J.-K. Yoo and J.-H. Kim. Navigation framework for humanoid robots integrating gaze control and modiﬁed-univector ﬁeld method to avoid dynamic obstacles. In Proceedings of the IEEE/RSJ International Confer- 70

52 Bibliography ence on Intelligent Robots and Systems (IROS), pages 1683–1689, Taipei, Taiwan, October 18–22 2010.

[60] F. Belkhouche. Reactive path planning in a dynamic environment. IEEE Transactions on Robotics, 25(4):902–911, August 2009.

[61] A. Nash, K. Daniel, S. Koening, and A. Felner. Theta*: Any angle path planning on grids. Journal of Intelligent Robot System, 39:533–579, 2010.

[62] P. Yap, N. Burch, R. Holte, and J. Schaeffer. Block a*: Database- driven search with applications in any-angle path-planning. In Proceed- ings of the Twenty-Fifth AAAI Conference on Artiﬁcial Intelligence, page 120125, San Diego, CA, USA, May 7-11 2011.

[63] D. Ferguson and A. Stentz. Using interpolation to improve path planning: The ﬁeld D* algorithm. Journal of Field Robotics, 23:79–101, 2006.

[64] T. Uras and S. Koenig. An empirical comparison of any-angle path- planning algorithms. In In Proceedings of the Symposium on Combi- natorial Search (SOCS), page 206210, 2015.

[65] J. Yu and S. M LaValle. Structure and intractability of optimal multi-robot path planning on graphs. In Proceedings of the 27th AAAI Conference on Artiﬁcial Intelligence, pages 1443–1449, 2013.

[66] T. S. Standley. Finding optimal solutions to cooperative pathﬁnding problems. In Proceedings of the 24th AAAI Conference on Artiﬁcial Intelli- gence, pages 173–178, 2010.

[67] G. Sharon, R. Stern, M. Goldenberg, and A. Felner. The increasing cost tree search for optimal multi-agent pathﬁnding. Artiﬁcial Intelligence, 195:470–495, 2013.

[68] G. Sharon, R. Stern, A. Felner, and N. R. Sturtevant. Conflict based search for optimal multi-agent pathfinding. Artificial Intelligence, 219:40–66, 2015. 71

[69] D. Silver. Cooperative pathﬁnding. In Proceeding of the First AAAI Con- ference on Artiﬁcial Intelligence and Interactive Digital Entertainment, pages 117–122, June 01–03 2005.

[70] Z. Bnaya and A. Felner. Conﬂict-oriented windowed hierarchical cooperative a*. In In IEEE International Conference on Robotics and Au- tomation (ICRA), pages 3743–3748, Hong Kong, China, 31 May–7 June 2014.

[71] J. Yu and S. M. LaValle. Planning optimal paths for multiple robots on graphs. In In IEEE International Conference on Robotics and Automation (ICRA), pages 3612–3617, Karlsruhe, Germany, May 6–10 2013.