Pursuit Evasion from Multiple Pursuers Using Speed Fluctuation

Pursuit Evasion from Multiple Pursuers Using Speed

Fluctuations

A thesis submitted to the

Graduate School of University of Cincinnati

In partial fulfillment of the requirements for the degree of

Master of Science

In the School of Electronic and Computing Systems

Of the College of Engineering and Applied Science

February 2013

Deepika Prasad

B.Tech. in Electronics and Communication Engineering

Visveswaraiah Technological University, Belgaum, India

Thesis Advisor and Committee Chair: Dr. Raj Bhatnagar

Abstract

An intelligent path planning algorithm for an evader whose objective is to escape capture for as long as possible was designed and implemented on Khepera III robots. To achieve the chase a pursuer algorithm was designed and implemented on two Khepera III robots. This pursuit evasion problem is an unpredictable and dynamically changing environment in which the difficulties faced by the evader when planning an escape route due to the pursuers are corral and capture conditions. The evader considers both the pursuer’s locations and increases or decreases its speed according to the distance between itself and either of the pursuers. It also comes to a stop or reverses into a previous location when need to avoid capture arises. The successful execution of the evader algorithm is analyzed by comparing the results of this speed fluctuation algorithm with one devoid of change in speed. The implementation of the algorithm over

Khepera III robots aimed at representing the successfulness of this technique in practical cases, as the work in this field is still relatively new.

iii

Acknowledgements

I would like to take this opportunity to express my gratitude to my thesis advisor, Dr. Raj

Bhatnagar, whose guidance, encouragement and patience, inspired me to work on this project. I am extremely grateful to him for his support and co-operation which helped me in successfully completing my Master’s degree. I would also like to thank Dr. Carla Purdy and Dr. Karen Davis for their presence on my thesis committee.

I would also like to thank Robert Montjoy for his technical support in the research lab.

Most importantly, I would like to thank my family and friends for their love and support. I would like to dedicate this work to them.

Deepika Prasad

Table of Contents

Abstract ...... ii

Acknowledgements ...... iv

List of Algorithms ...... ix

List of Figures ...... x

List of Tables ...... xii

Introduction ...... 1

1.1 Introduction ...... 1

1.2 Motivation ...... 3

1.3 Our Approach ...... 4

1.4 Organization of the Thesis ...... 7

Literature Review...... 8

2.1 Introduction ...... 8

2.2 Real-Time Path Searching Robots ...... 9

2.3 Distributed Problem Solving ...... 11

2.4 Pursuit Evasion Games...... 12

2.5 Evasion Strategies ...... 13

Implementation and Specification ...... 15

3.1 Technical Specifications ...... 15

3.1.1 Hardware Specifications ...... 15

3.1.2 Software Specifications ...... 17

3.2 Environment Constraints ...... 17

Pursuit Evasion ...... 19

4.1 Introduction ...... 19

4.2 Terminology ...... 20

4.3 Pursuer Path Planning ...... 23

4.3.1 Main Method for Path Planning of Pursuer ...... 24

4.3.2 Corralling Module ...... 27

4.3.3 Collision Avoidance ...... 29

4.4 Path Planning for Evader...... 31

4.4.1 Evader Path Planning with Speed Fluctuation ...... 33

4.4.2 Evader’s Path Planning Without Speed Fluctuation...... 43

4.4.3 Evader’s Path Planning Considering Nearest Pursuer ...... 44

4.5 Shared Modules ...... 47

4.5.1 Communication Module ...... 47

4.5.2 Stop Algorithm Module ...... 48

4.6 Flow Chart ...... 49

4.6.1 Pursuer Flow Chart ...... 49

4.6.2 Evader’s Flow Chart ...... 52

4.7 Summary ...... 57

Results and Analysis ...... 58

5.1 Introduction ...... 58

5.2 Test Cases ...... 60

5.2.1 Scenario 1 ...... 60

5.2.2 Scenario 2 ...... 64

5.2.3 Scenario 3 ...... 70

Conclusions and Future Work ...... 72

6.1 Conclusion ...... 72

6.2 Future Work ...... 73

References ...... 75

vii

viii

List of Algorithms

Algorithm 4. 1: Pursuer main algorithm ...... 26

Algorithm 4. 2: Algorithm for corralling the evader ...... 28

Algorithm 4. 3: Collision avoidance algorithm ...... 31

Algorithm 4. 4: Evader’s main algorithm ...... 35

Algorithm 4. 5: Update direction for evader algorithm ...... 37

Algorithm 4. 6: Check path algorithm ...... 40

Algorithm 4. 7: Check speed algorithm ...... 42

Algorithm 4. 8: Update direction considering nearest pursuer ...... 46

List of Figures

Figure 1.1: Grid with pursuers and evaders ...... 5

Figure 3. 1: Khepera III robot ...... 16

Figure 3. 2: The grid with the coordinates and directions ...... 18

Figure 4. 1: Pursuer’s chasing a stationary target ...... 24

Figure 4. 2: Pursuer’s corralling the evader ...... 27

Figure 4. 3: Using priority to avoid collision ...... 30

Figure 4. 4: Evader path planning ...... 34

Figure 4. 5: Evader updating its direction of motion ...... 36

Figure 4. 6: Situation for path check ...... 39

Figure 4. 7: Situation to increase speed ...... 41

Figure 5. 1: Scenario 1 - agent placement ...... 60

Figure 5. 2: Path of pursuers and evader for scenario 1 ...... 61

Figure 5. 3: Scenario 1 without speed change ...... 63

Figure 5. 4: Scenario 2 agent placement ...... 64

Figure 5. 5: Path for pursuers and evader for scenario 2 ...... 65

Figure 5. 6: Continuation of path for scenario 2 ...... 66

Figure 5. 7: Scenario 2 without speed change ...... 68

Figure 5. 8: Path planning of agents when considering only nearest neighbor ...... 69

Figure 5. 9: Scenario for the evader to be caught immediately ...... 71

Figure 5. 10: Scenario for the evader to be caught immediately ...... 71

List of Tables

Table 5. 1: Path of pursuers and evader for scenario 1 ...... 62

Table 5. 2: Scenario 1 path coordinates with evader without changing speed ...... 63

Table 5. 3: Path coordinates for pursuers and evader for scenario 2 ...... 67

Table 5. 4: Path coordinates without speed change ...... 68

Table 5. 5: Path coordinates using nearest pursuer for scenario 2 ...... 70

xii

Chapter 1

Introduction

1.1 Introduction

There are many challenges in planning the paths of independent intelligent agents in an arbitrary terrain. These challenges are especially severe when there is no central coordinator informing each agent about the global situation and guiding them along the optimal routes. In this thesis we have addressed the situation of an evader-agent and multiple pursuer-agents in which the evader plans its path to avoid the pursuers as much as possible. Also, each agent is guided only by the grid-lines on the terrain in determining its current position instead of a central camera observing and informing every agent.

Developing such path planning algorithms for sets of Distributed Artificial Intelligent

(DAI) agents is a research focus that has been addressed since the past couple of decades and is still being actively pursued [10]. A DAI agent can be either a mobile robot (as in the case of this thesis) or a simulation of a software program that acts on an unknown environment in an autonomous manner to provide distributed solution for a complex problem. In the last half decade, many motion and path planning algorithms have been developed to perform individual simple activities such as lawn mowing, garbage collection, maze navigation and entertainment

[34, 28] and also to perform complex cooperative activities such as space exploration, robotic

1 surgery, video games, unmanned swarm aerial vehicles and battlefield landmine clearance [5, 8,

20, 25]. Many static world problems, such as maze traversal and finding shortest or most cost effective path for given origin and destination points, have been solved by researchers using motion planning algorithms [25, 42, 3]. A real-world environment is highly complex, unpredictable, and dynamic due to factors such as varying terrain, obstacles and constantly changing target locations. The complication of path planning in such a dynamic world increases, when an agent is being followed and it needs to avoid capture such as in the case of path planning for evader/target in a predator – prey domain that demands high precision, awareness and speed.

Pursuit evasion games (PEGs), a group of pursuers tracking and capturing a group of evaders, has been a topic of interest and studied since 1950s in varied situations, such as decreasing or increasing the lifetime of the evaders which in turn implies different termination conditions [4, 31]. The PEGs are designed to replicate practical experiences and tasks that are dangerous for human beings, for example, search and capture mission (law enforcement vehicle trying to capture culprit’s vehicle), search and rescue operations (area survey after earthquake or flooding) and avoiding capture (avoiding detection or capture in search and rescue missions in war zones) [11]. A pursuer needs to process a target’s location and static or dynamic environment obstacles in order to plan a path to intercept the target and successfully capture it.

Routing paths for multiple pursuers will cause chaos if there are no proper coordination strategies for de-centralized DAIs. The de-centralized agents do not have a central system that has prior knowledge of maps of the region. Each pursuer agent needs to have a high level of synchronization and planning to avoid coming in the way of the other pursuers, and to surround

2 the evader to capture it. Each individual member of a DAI environment should be powerful enough to perform analysis of the dynamically changing environment it is placed in in real time and decide its path.

1.2 Motivation

One of the first known pursuit evasion game problem in game theory was posed by Rufus

Isaacs, in 1954, called as the “Homicidal Chauffeur Problem” [15], which places a slow but highly maneuverable pedestrian against a fast, less maneuverable motor car. This pursuit problem is a continuous time differential game which has been used for research in various situations. Most significant situations researched are to check the circumstances under which the car will always hit the pedestrian or the pedestrian avoids being hit by the car [31]. Torrence D.

Parsons, in 1976, introduced deterministic search and rescue PEGs on finite graphs. The problem addressed was to find a lost man using a rescue team moving on the edges of a known graph. The rescue team members are able to stop or change directions anywhere on edges thus breaking down the environment into a collection of nodes. The ultimate goal achieved was to rescue the lost man in minimum amount of time or resources [7].

From the viewpoint of physical systems, the PEG is an extension of studies on traffic problems [36, 19], leading to applications in law enforcement and military applications. Over the years, researchers have developed computer simulations to study various aspects of pursuit evasion games, such as, pursuer with higher velocity, fluctuating speeds for both pursuer and evader, minimum time needed to capture, or minimum number of pursuers needed for capture. In real-time, aspects such as exploring and knowing the environment, hardware specification of

3 robot, and physical dimensions of the terrain play key roles in obtaining the optimum solution for any of the simulation study.

This thesis investigates strategies employed by the target to avoid capture from multiple pursuers for as long as possible and validates them with physical robots moving in a grid-marked terrain. Possible options for the target are to maximize the average distance between itself and the pursuers and/or to change its velocity. The pursuers, on the other hand, have to minimize the distance between themselves and the target, corner the target, and then ultimately capture the target by occupying the same location as the evader and avoiding collisions with each other.

1.3 Our Approach

Autonomous robots when placed in an unknown environment should be able to devise cost-effective and collision-free paths to perform required tasks. This requires that they should be able to locate themselves, move around the terrain and avoid obstacles. This is accomplished by first simplifying the environment around the robots by dividing the area into cells of a grid. The grid and the cell boundaries are to help an agent determine its location in a terrain. This information is a substitute for a centralized system keeping track of every agent and informing it of its current coordinates. In the system implemented, the robot independently determines its location based on the grid marks it has traversed. This is a simple model of a situation in which a robot may determine its location based on the landmarks in a terrain. So, if a robot can perform the required tasks in a grid, it can replicate the performance in any kind of surroundings as long as it knows the terrain and its landmarks.

In the thesis, the robots were placed in the simplest form of settings, a two dimensional grid with eight rows and four columns. Each agent was provided with their respective initial position coordinates and the directions they are facing. Once the agent decides on its optimum path, it navigates itself towards the set goal. The robot encounters the black border of the cell before moving into the next cell, indicating a change in location coordinates. This change is tracked by using IR sensors placed at the bottom of each robot and its current coordinates are changed to the new cell location. The tracking of coordinates helps the robot to be cognizant of the grid’s boundary. Also, to further simplify the problem, robots are allowed only to move in axis parallel directions, that is diagonal motion in the grid is not allowed. Figure 1.1 shows the grid on which the agents were placed. The black lines separate one cell from the next and also mark the boundary.

P P 1 2 Figure 1.1: Grid with pursuers and evaders

Pursuit evasion environment with pursuers and evaders is a behavior based environment model which has two aspects to it – pursuer behavior and evader behavior. First is the pursuer whose objective is to chase and corral the evader and second is the victim whose purpose is to

5 escape the assault. The implementation of design for pursuers with a stagnant target was conducted first and later movement for evader was put into practice. The pursuers follow a greedy policy algorithm where the direction of the pursuer is determined by calculating the direction in which the probability of intercepting the evader’s path is maximized. In this path planning, another factor that plays an important part is avoiding collision between pursuers, which is implemented by a priority system assigned to each pursuer.

The evader needs to counter this attack and think of a path where the probability of it being intercepted or surrounded by any of pursuers is minimized. In figure 1.1, the blue circle represents an evader whereas the red circles represent the pursuers and the arrows indicate the directions of their goal cells. The evader algorithm uses various methods to accomplish this. The main method is used to assign priority to each cell around the evader’s current location based on maintaining its safety. A single pursuer can catch a target only if targets speed or agility is lesser than those of itself [19]. A group of pursuers have an advantage to coordinate, cooperate and corral the target to capture it. In such a case, speed fluctuation, coming to a complete stop for a time until it can get a chance to avoid cornering, and finally reversing into a spot where it cannot be captured and can recalculate its path are some of the methods implemented.

The algorithm for evader with constant speed was implemented on various scenarios. The evader algorithm with speed fluctuation was implemented on scenarios in which the evader was caught. The results of both the methods were compared to show that the algorithm with speed fluctuation module helps the evader to survive the capture for a longer amount of time. It is seen

6 that using change in speeds for evader helps its escape in most of the situations. Results and comparison for some such scenarios are discussed in chapter 5.

1.4 Organization of the Thesis

The rest of this thesis is organized as follows: Chapter Two reviews previous pursuit- evasion game research. In Chapter Three, the technical specification of the robots used and the constraints applied on them are discussed. Chapter Four describes an algorithm each for pursuer and evader for the pursuit-evasion scenario. Chapter Five discusses results and analysis for these algorithms. Chapter Six gives conclusions and describes areas for future research.

Chapter 2

Literature Review

2.1 Introduction

Robots, nowadays, are being programmed to move collectively on land, in air and under water to perform specific tasks. Pursuer evasion domain is being used extensively to study a multitude of online applications, like, computer and video games, differential games [4, 18], and real world complex scenarios, such as, communication, cooperation and coordination in swarm behavior in capture missions, missile guidance systems, law enforcement vehicle trying to capture culprits vehicle, detection and diffusing a bomb, avoiding detection or capture in search and rescue missions in war zones [8, 18]. It is also used for research in evolutionary ecology or building population models [27, 24]. According to the goals set, robots have to transfer knowledge, work together and make decisions and respond in a fraction of a second.

In real time, a single pursuer can catch a target only if target’s speed or agility is lesser than those of the pursuer’s. Otherwise, in finite or infinite space, the target could evade capture indefinitely by having enough speed to avoid being trapped in a dead-end path [16]. Another approach to capture a target is to have more than one pursuer so that they coordinate to surround the target. Most of the solutions in pursuer evader problems deal with the evaders capture

8 strategies. Until now some of the strategies considered for a pursuer path planning are based on fitness value where the fitness value increases as the pursuer gets closer to the evader [14], or cost effective target catching where a cost is awarded every time the pursuer captures a target and this value is used for learning [17]. One of the key characteristics assumed in all these strategies is that the target is stationary. Since, this is not the case with real world evader where it calculates an exit path continuously and tries to escape capture, this thesis deals with a continuously moving evader. The challenge faced was to come up with a dynamic path planning algorithm for evader when the pursuers are moving continuously until the final condition of capturing the evader is satisfied. In the below sections, strategies for an evader and a simple multi-agent pursuer strategy are discussed in detail along with the algorithms employed.

2.2 Real-Time Path Searching Robots

Real-time autonomous robot applications require the robot to execute tasks independently without human interaction in an unknown environment. For a robot to be autonomous, it should be able to multi-task until the objective is accomplished. Interacting and sensing changes in the local environment, computing the plausible next move, executing the move in the physical world, calculating the following move at the same time, and communicating with other agents are some of the multi-tasks performed by the agent. These learning, planning and execution tasks need to interleave to overcome the speed and time constraints. The real-time agent appears responsive to the environmental changes regardless of the scale of the problem when the decision is made at an instance of time or the actions to be performed are in constant time per move.

Real-time search algorithms may have a-priori knowledge of the environment around them. Search algorithms like A*[13] or Dijkstra’s algorithms [9] were some of the first to use a- priori known maps to perform complete searches to obtain the best path in a given map. Various versions of these algorithms have been successfully used in visibility graphs [39], state abstraction hierarchies [6] and route waypoints [33]. In the real world scenarios, the knowledge of obstacles or hurdles or maps is unavailable beforehand. So a complete search algorithm or pre-computation technique cannot be used in such circumstances. To overcome such complications, incremental search methods were researched, such as dynamic A* (D*) [35] and

D* Lite [21]. These algorithms were implemented in real time applications such as DARPA’s

Unmanned Ground Vehicle program and Mars rover [12, 38]. The disadvantage of using these robots is that they are huge and take more time to execute its movements than the speed of planning of the application. So, fast movement and planning are not achieved in a rapidly changing environment.

The challenge now was to come up with an algorithm which is dynamic in nature. The algorithms for pursuers and evader designed, exchange each other’s locations at every time- instance. According to change in path of an evader, the new path decision for pursuer is made at a moment’s notice or vice versa, making the agents work in real-time. Another challenge was to work with a robot which fulfilled certain requirements such as minimal planning time per move, least learning time, low development cost, robustness, ability to move in rugged terrain and for local sensors to assess and learn its surroundings [40]. The Khepera III robot developed by Swiss

Federal Institute of technology in Lausanne, Switzerland is used around the world as a standard for academic research for the above mentioned assets [1].

2.3 Distributed Problem Solving

As the complexity of the problem assigned to a robot increases, it becomes necessary for it to be more intelligent and powerful which depends on the on-chip memory and battery power.

Any robot has a maximum limit of memory, speed or power that it can use. To overcome this problem and to make the agent more efficient, distributed problem solving approach is utilized.

The complex problem is divided into smaller simpler subtasks, each of which is assigned to a different agent. The agents communicate and coordinate to accomplish the tasks, and when all the assigned subtasks are completed, the required results on the whole will produce a solution for the initial objective. Using a group of agents to obtain an essential outcome increases efficiency, reduces time, solves difficulties far complex than which can be solved by a single robot [32]. In an unknown surrounding, a group of robots need to form a decentralized system with no central system directing each robot to accomplish their task. This makes it easier for them to react to changes, make decisions or deviate from their course of action at a moment’s notice and communicate with each other the changes to finally achieve the results.

In a distributed problem solving group, physical attributes of the terrain, number of agents used for the purpose, uncertainty of the environment and robot’s physical features contributes towards the level of intelligence for generating a plan [2, 26]. Due to the small size of the grid, two pursuer-agents were used to capture a single evader. Each agent, placed in the environment, marks its position on the 2-dimensional grid, comes up with a path on the grid according to its role, and avoids collision by interacting with other agents on the grid. The multiple pursuers on the grid use the distributed decentralized problem solving method to round up the single moving target and efficiently capture the latter moving at constant speed.

2.4 Pursuit Evasion Games

Rufus Isaacs in 1964 proposed the first study on pursuit evasion games when he published a series of RAND Corporation technical reports [31] and called them the “Homicidal

Chauffeur Problem”. In the game problem, he projected a geometrically modeled environment in which a car called the homicidal pursuer P driven at a high speed and a big turning radius tries to run over a pedestrian which is the evader E moving at a slower speed and a small turning radius [31]. In this two player game he mathematically solved for the condition for pursuer to capture the evader, and the condition being:

√

where, -- is the vehicle P’s minimum turning radius

, and and are corresponding speeds of Pursuer and Evader [31, 15].

This is an example of mathematical solution for a single pursuer capturing a single target.

Over the years, various cases of this same problem with varying boundaries and diverse conditions have been studied leading to complex and challenging problems to obtain the analytical expression [29, 7]. Later, computational systems and simulations were developed to verify these results.

The cases where there are multiple chasers and/or escapees are much less studied in comparison. Predator prey models where a single stagnant evader is captured by multiple pursuers [14, 17, 22, 30] or multiple pursuers capturing multiple evaders [19, 37] or pursuer

12 pursuing a moving evader [16] have been studied. Lately, research in predator-prey models has increased to develop some applications in game theory, robotics and multi-agent systems [20].

In this thesis, a single moving target chased by multiple pursuers was considered. The objective of the target is to anticipate the enclosure and take steps to avoid capture. The objective of the pursuer is to exist in the same cell as the target at a given point of time by pursuit and corralling the target.

2.5 Evasion Strategies

An agent evades pursuit by maximizing the distance between itself and the pursuers. The evader is said to be caught when a pursuer occupies the same space, location, cell or node as the evader [19]. Some game theory research also considers evader is caught if it is in the line of sight of the pursuer [11]. Pursuit evasion games until now focus on the questions of minimizing number of pursuers, time taken to capture, or distance for the evader’s capture with different assumptions about movement or capture. If the evader’s speed is same as a pursuer’s, then the evader will surely escape. The problem arises when the evaders speed is constant and there are multiple pursuers, where the pursuers get an advantage in numbers. The latter can communicate, coordinate and surround the evader leading to its capture. In such a situation, evader must have a faster deducting capacity of pursuer’s future moves and higher speed to avoid capture. Very little work is done on target’s success in evading a pursuit.

Pursuit evasion games have been studied by placing pursuers and targets on a grid or a graph made up of vertexes/nodes and edges with some given rules of movement [14, 19]. The

13 evader studies the nearest pursuer and calculates the probability for each node around it that takes it further away from the considered pursuer [19]. This method may not be accurate as the evader is taking into account only the nearest pursuer where the resultant node may take it closer to another pursuer, leading to its capture. There have also been algorithms using Electronic

Counter Measures (ECM) which can intentionally disrupt the sensing capability of their opponent by using aerosols, deception and blip enhancement or flares and decoys [23]. This technique, even though very innovative, faces with the problem of transporting the ECMs on the agent. Carrying ECMs on small agent used for search and rescue missions in a war zone is not feasible. So these small robots have to depend on speed to avoid capture. All the methods discussed up till now calculate and move the pursuer and evader alternately, like in a game of chess, giving the agent a definite goal destination and time to calculate its next move. In real world scenario, such is not the case. An evader is not given time to think and make a move as in that time the pursuer is already moving towards the former.

Pursuers and evader move continuously on the grid not giving the other enough time to calculate its next move. The evader should be very fast in getting the pursuers locations and calculating its route on the map. Since there is more than one pursuer, the evader faces with the challenge of keeping track of all the pursuers on the grid and see that it is not cornered. A case where the evader is small but fast and distances itself from all the pursuers on the grid is taken into account. Also a method to dodge caging of the pursuers is implemented by changing the evader’s speed.

Chapter 3

Implementation and Specification

3.1 Technical Specifications

Khepera III robot (Figure) manufactured by K-Team S.A. 2006, and produced by Swiss

Federal Institute of technology in Lausanne, Switzerland, have multitude of advantages and are being used for research purposes worldwide [1]. Efficient in navigation and control, minimal planning time per move, robust, ability to move in rugged terrain and local sensors to assess and learn its surroundings are some of the advantages [40]. The hardware and software specification for the robot provided by K-Team is listed below [1]. Figure 3.1 shows the front and side view of the Khepera III robot.

3.1.1 Hardware Specifications

 We have used the Korebot II embedded with the standard embedded Linux operating

system which supports C/C++ cross compiler for C/C++ programming so that the robot

can be programmed to operate autonomously.

 Processor: DsPIC 30F5011 at 600MHz

 Ram: 64 MB

 Size: Khepera III with its small body can be used in missions where not being detected is

of key importance. It had a diameter measuring 130 mm and 70 mm height.

 Motors: 2 DC brushed servo motors controlled by separate PID controllers connected to 2

wheels is used for motion. The speed can be individually controlled with incremental

encoders at roughly 22 pulses per mm of robot motion. It can reach a maximum speed of

0.5 m/s.

 Sensors: Khepera III has 9 Infra-red proximity and ambient light sensors with up to 30cm

range and 5 Ultrasonic sensors with range 20cm to 4 meters to help with the research

work. Of the 9 light sensors, 2 Infra-red are ground proximity sensors help in line

following and edge detection applications.

 Communication: Khepera III has multiple choices for communication.

1. RS232 on a standard serial port

2. Via USB with KoreBot up to 115kbps

3. Wireless Ethernet with KoreBot and a Wi-Fi card

We used the wireless Ethernet with Wi-Fi card for the research.

Figure 3. 1: Khepera III robot

3.1.2 Software Specifications

 Development Environment for Autonomous Application: GNU C/C++ compiler, C

programming language, ARM-Linux-gcc cross compiler.

 Program execution: The algorithm for the robots motion control, communication and

result analysis was programmed on a desktop PC with Ubuntu as the operating system.

 An FTP client for file transfers to and from the computer and robots, executing

programs, analyzing outputs.

3.2 Environment Constraints

There are certain environmental constrains that have been applied to all the agents placed on the grid.

 No cameras mounted on top of the arena were used to capture images as time taken to

process such images affect the overall performance by increasing the execution time. It is

an important task for the robot to keep track of its and the other robots current

coordinates correctly at all times. The IR sensors play a major role in helping the robot

visualize its surroundings and keep track of its position on the grid. Factors such as the

brightness of the light and the material used for the grid effect the IR sensor reading.

 The robots are placed on an 8x4 grid and their mobility in the grid is restricted to

orthogonal motion only.

 Wireless Ethernet with KoreBot and a Wi-Fi card was used to establish connection

between the robots and communicate their present locations.

 No manual intervention is provided to the robot, so there is a need to develop robust,

efficient and intelligent autonomous path planning algorithm.

Figure 3.2 illustrates the 8x4 grid with the coordinates used for each block on the grid, the x and y axis and the north (N), south (S), east (E), and west (W) directions assumed by the agents for solving the stated problem.

3 N

2 W E

1 Y direction Y S

(0 , 0) 1 2 3 4 5 6 7

X direction Figure 3. 2: The grid with the coordinates and directions

Chapter 4

Pursuit Evasion

4.1 Introduction

An intelligent path planning algorithm for an evader to escape capture and the counter path planning algorithm for pursuers are presented and discussed to move on an 8x4 grid. Three agents have been used for implementing these algorithms where two of the agents are pursuers and one is an evader. The agents are placed randomly on the grid, loaded with the algorithms, and given their respective coordinate locations and directions they are facing. Each agent according to the algorithm loaded onto it behaves as a pursuer or an evader. The same algorithm is loaded onto the two pursuers where they must synchronize to capture the evader and avoid collision. The pursuer algorithm is divided into one main module and two sub modules concentrating on the above mentioned problems. An evader faces two difficulties, one avoiding corral and other avoiding capture, which have been addressed separately in three modules which are called by the main evader module whenever needed. The agent has to know its destination for it to do the path planning. The goal coordinates for a pursuer is the cell in which the evader is present and for an evader is the cell next to its present cell that takes it away from the pursuers.

Since all the agents move at the same time, the destination keeps changing dynamically.

The algorithm plans the best and intelligent path for an evader on the grid. The execution stops once any one of the pursuers share the same cell as the evader and the latter is said to be caught, which brings all the agents on the grid to an instant stop. All the above mentioned techniques to solve the issues faced by the pursuers and evader are discussed in section 4.3 and 4.4 respectively and their corresponding flowcharts are presented in section 4.5.

4.2 Terminology

The following terminology and definitions given below is used often in the study of the predator- prey domain.

1. Agent – Any single robot programmed to perform some activity.

2. Pursuer/Predator – The robot whose objective is to chase, corral and capture another

robot. There are two pursuers in this thesis to help with the corralling process.

3. Target/Prey/Evader – The robot whose objective is to avoid corral and capture by other

robots. There is one target robot which increases the risk of capture.

4. Grid/Predator-Prey Domain/terrain – The 8x4 square boxes forming the physical layout

on which the robot traverses and calculates its position on the map.

5. Direction – The orientation of the robots in the grid is kept track of by the direction. The

four directions (Up, Down, Right and Left) are represented as North, South, East and

West. Updating the direction of the robot helps it to make appropriate turn decisions in

the future. Each direction is given a constant value for this purpose -- NORTH = 0, EAST

= 1, SOUTH = 2, WEST = 3. Movement in each direction signifies:

a. NORTH = y increment.

b. SOUTH = y decrement.

c. EAST = x increment.

d. WEST = x decrement.

6. Start coordinates – The initial location from where the robot starts traversing the maze.

This information consists of the two-dimensional coordinates on the 8x4 grid, starting

with (0, 0) till (7, 3).

7. Ground Sensing – Robots use Infra-red sensor information to locate and navigate

themselves on the grid. When a cell boundary is crossed, the corresponding X and Y

coordinates are updated. When a grid boundary is detected, robots take necessary actions

to be within the grid.

8. Path – The sequential array of cells on the grid generated for the pursuer or target using

strategies to obtain the objective of the pursuer and target. It keeps track of the next cell

to be visited by the robot.

9. Dynamic Obstacles – This term is related to the pursuer, when both the pursuers are

chasing the target, they might move into each other and form moving obstacles for the

other pursuer. The pursuers need to keep track of each other’s location to avoid running

into the other’s path and collision.

10. Priority – When the robot comes with a dynamic obstacle, priority is assigned to each

robot according to its location on the grid to decide which pursuer will continue in its

path and which will recalculate its path.

11. Blocked Condition – The conditions applied on target robot when it is present on certain

areas on the grid from which it cannot move forward. The evader is blocked when it is:

 Stuck at a border with the two pursuers coming at it without giving it area to

move away from both of them and not getting caught. Here it may wait until one

of the pursuers move closer to give it enough space to get away.

 Stuck at a corner with the two pursuers coming at it without giving it area to move

away from both of them and not getting caught. Here the evader is blocked and it

cannot get away.

12. Target Caught – When the pursuer occupies the same block or grid location as the target,

or pursuer’s location coincides with the location of the target, then the target is said to be

caught.

4.3 Pursuer Path Planning

The pursuer or evader algorithms are implemented such that all agents are aware of each other’s locations at any given point of time. The objective is to get an intelligent escape route for an evader. To fulfill this objective, a path planning algorithm was implemented for the pursuers to follow and capture the evader. The path for the pursuer is planned one cell at a time such that the pursuers try to minimize the distance between themselves and the target. The path planning and communication between agents is done while the agents are in motion, which made it important for the agents to multitask. The planning terminates only when the target is captured.

When there is a single pursuer on the grid, the movement is decided based on reducing the distance between itself and the target to zero first in the x direction then in the y direction. Since two pursuers are considered, making catching the target tricky. Some of the difficulties faced and solved are how to avoid collision, how to recalculate path, how to corral the evader.

Each agent on the grid uses the two ground sensors to keep track of change in cell coordinates and accordingly its location co-ordinates are changed. It gets its sense of direction using the North=0, South=2, East=1 and West=3 which also helps it to change the x or y coordinate while moving, and this was explained in the previous section. The agent turns 90 to accurately maintain path in only x and y direction. Due to the 90 turn constraint, the new direction of the agent is calculated either by subtracting or adding one to the old direction. Any agent with the knowledge of start co-ordinates, direction it is facing and a destination can move on the grid without human intervention [19].

The major modules that help solve the above mentioned pursuer problems are path planning with corralling module, collision avoidance module, along with some common modules 23 for all the agents such as communication module and evader caught module. Each of these modules is discussed in detail in later sections.

4.3.1 Main Method for Path Planning of Pursuer

P P 1 2 Figure 4. 1: Pursuer’s chasing a stationary target

A pursuer placed on the grid, allowed to move only orthogonally, has to move in either the x or y direction towards the target in order to capture the latter. Due to the length of the grid being more than the width, the agent prefers to move lengthwise first and then widthwise. The time taken to traverse to the border in a straight line is less than the time taken to reach the border with a lot of turns. So, the pursuer chooses x or y direction to traverse according to its position on the grid, its role and priority. Since the evader moves constantly, the destination of the pursuer changes dynamically. The decision of a turn or a new path needs to be made instantly when the evader’s path changes. Once a direction is chosen, the agent keeps moving in the said direction until either of two conditions is encountered

 Firstly border is reached and the agent can’t move further in that direction.

 Secondly moving in the particular direction takes the pursuer away from the evader. 24

Once the evader is caught, a signal is communicated to all the agents on the grid to stop their movement.

4.3.1.1 Algorithm for the Main Method

The algorithm 4.1 is for a pursuer which fulfills the pursuer’s objective of corralling and capturing the evader. The algorithm begins by providing the agent with its initial coordinates and the direction in which it is facing. All the agents share each other’s location to place itself and others on the grid map by calling COMMUNICATE_LOCATION() module. Since there are two pursuers, each needs to decide either to follow evader in x or y direction, so that they corner the evader. Each pursuer is given a priority according to the cell it is occupying. The goal location of the pursuer is the evader’s present cell co-ordinates. UPDATE_DIRECTION() module sets the priority of the pursuer, sets the destination and decides the direction of motion. Deciding to move in a direction is not enough. The pursuer must also check if its new path coincides with the path of the other pursuer. The AVOID_COLLISION() module checks if in the future cell the pursuers may collide. Only when there is no collision, does the agent start to move in the said direction towards the evader. The pursuer while in motion continuously updates the destination location, calculates a path, and checks for collision in each cell it occupies. These modules are run until the evader is caught by either of the pursuers. If pursuer A catches evader, it communicates a constant value instead of its location by calling PREY_CAUGHT() module indicating the capture, resulting in termination of the algorithm in all the agents on the grid. Section 4.6.1.1 shows the flowchart of the discussed algorithm.

1 BEGIN 2 Input: Start x, y coordinates; Direction robot is currently facing 3 Initialize Khepera III Motor Controller 1 and 2; 4 COMMUNICATE_LOCATION (); 5 while (target is not caught) do /* pursuer and prey exist in same block on grid*/ 6 COMMUNICATE_LOCATION () /* communicates each other’s location on the grid */ 7 turn_towards_direction = UPDATE_DIRECTION () /* calculates the new x or y direction which will lead towards the prey*/ 8 if (! AVOID_COLLISION ()) do /*checks collision in next 2 steps on the path with other pursuer. Returns true if no collision occurs.*/ 9 goto step 6; 10 end 11 Turn towards the new direction; 12 If (! pursuer either reaches border or one of the co-ordinates is equal to the targets) do 13 Start moving in straight line; 14 COMMUNICATE_LOCATION () when a grid line is crossed; 15 end 16 end 17 If (target is caught) do 18 PREY_CAUGHT (); 19 STOP; 20 end

Algorithm 4. 1: Pursuer main algorithm

4.3.2 Corralling Module

The corralling module addresses the issue of the decision to move the agent in x or y direction which is dependent on location of evader as well as the other pursuer. Each pursuer is assigned a priority according to its position in the grid, which determines the direction to move.

The priority decides if the agent will chase the evader in the x or the y direction. Since the two pursuers come at the evader from two different directions as shown in figure 4.2, this algorithm helps in surrounding the evader and cornering it. The priority is also helpful in deciding who gets to continue in its path and who has to calculate an alternate path when faced with the collision problem. The algorithm also sets the goal coordinates of the agent equal to the coordinates of the evader. Since the evader’s location changes dynamically, this step is important to make a new direction decision. The new direction in which the agent needs to proceed is decided based on the destination location and priority allocated to it.

P 1

E P 2 Figure 4. 2: Pursuer’s corralling the evader

4.3.2.1 Algorithm for Update Direction

1 BEGIN 2 Set destination coordinates = coordinates of Prey; 3 if (current_coordinates > predator_coordinates) do

4 Priority = 1; 5 else do

6 Priority = 2; 7 end 8 if (Priority == 1) do

9 move in y direction; 10 else do

11 move in x direction; 12 end 13 if (moving in x direction) do 14 if (prey is to the right of predator) do // move in positive x direction

15 turn_towards = EAST;

16 else // move in negative x direction 17 turn_towards = WEST;

18 end 19 else if (moving in y direction) do

20 if (prey is above predator) do //positive y direction

21 turn_towards = NORTH; 22 else //negative y direction

23 turn_towards = SOUTH; 24 end

25 end orithm for corralling the evader 26 STOP

Algorithm 4. 2: Algorithm for corralling the evader

The priority allocated to a pursuer is based on its position on the grid. The pursuer with a higher x coordinate moving in x direction or y co-ordinate moving in y direction is given a lower priority. The priority is used for making decision in two circumstances. First is the initial position where the pursuer with the higher coordinates is given lower priority and has to move in the y direction. Second is to avoid collision while in motion, where the priority set is the same but the pursuer with lower priority must stop and recalculate its path to avoid collision. For example, if one pursuer A is at the (0, 0) and the pursuer B is at (4, 0), as seen in figure 5.4, B gets lower priority and starts to move in y direction towards the target and A moves in the x direction. The priority set here is also used in the AVOID_COLLISION() module to decide which pursuer stops and calculates an alternate route. The UPDATE_DIRECTION() module performs the above steps and comes up with the new direction of the agent keeping in mind the goal coordinates, priority awarded to it and the boundary conditions. Section 4.6.1.2 shows the flowchart of the discussed algorithm.

4.3.3 Collision Avoidance

When there is more than one pursuer in a terrain, there is a very high chance their paths will overlap leading to both the pursuers occupying the same cell at the same time. In such a case collision is bound to happen. To avoid such a collision, one of the pursuers should stop its movement and calculate an alternate route to surround the target. The priority, set in the previous module, is used to make the decision of which pursuer will recalculate its path. The pursuer with lower priority needs to re-route and the higher priority pursuer can continue in its route. For example, a pursuer A is at (3, 2) and pursuer B is at (4, 3) as shown in figure 4.3 with both

29 moving into (3, 3). Here B gets a lower priority and has to re-route a new path according to the evader’s position on the grid and A can continue onto new location (3, 3).

E P 2

P 1

Figure 4. 3: Using priority to avoid collision

4.3.3.1 Algorithm for Collision Avoidance

The module begins by getting the coordinates and direction of the other pursuer.

Knowing the direction helps to figure out the future position of any agent. If NORTH, y coordinate is incremented, if EAST x coordinate is incremented, if SOUTH y coordinate is decremented and if WEST x coordinate is decremented. Using this formula, the future location of the agent itself and other pursuer is calculated. These future locations are used to check if they coincide leading to a collision. If the collision is going to occur, the pursuer with the lower priority must stop, pick a new direction and chase the target from the new direction. If not, the pursuers can continue in their respective paths. This check is performed by each pursuer individually at each cell it occupies on the grid. Section 4.6.1.3 shows the flowchart of the discussed algorithm.

1 BEGIN

2 get other_predator_coordinates, Direction;

3 Calculate next_location_of_other_predator

4 Calculate next_location_of_agent

5 If (next_location_of_other_predator == next_location_of _agent) do

6 if (Priority == 1) do

7 STOP;

8 Goto main_method () and break to re-calculate route;

9 else do

10 move in the specified direction;

11 end

12 STOP

Algorithm 4. 3: Collision avoidance algorithm

4.4 Path Planning for Evader

If an evader with same or higher speed than a group of pursuers moving in a straight line in an infinite plane away from the said group can go interminably without being caught. The efficiency of an evader algorithm can be tested when the evader is placed in a confined space with multiple pursuers. In this thesis, the confined space is 8x4 grid where a single target tries to evade two pursuers. Since the pursuer does not wait and give time for the evader to make a path decision and move into the new location, the evaders path planning has to be continuous, adaptable and powerful.

When the evader gets the location of the pursuers, it can map them on the grid with respect to itself, making it easier for its own path planning. Evader algorithm, like pursuer, uses

North=0, South=2, East=1 and West=3 to maintain its sense of direction and can only move orthogonally on the grid taking 90 turns. When there is only a single pursuer present, evader can pick either x or y direction to move and begin moving in that direction away from the pursuer.

This forward movement comes to a stop when the agent encounters a border or a corner. When there are two pursuers, a third condition to exit the forward movement comes into play. That is if it is being corralled or if its future location is going to coincide with any one of the pursuers (as that condition leads to evaders capture).

For a pursuer, the final destination is the cell occupied by the evader at that point of time.

Unlike a pursuer, an evader does not have a final destination provided to it except for the boundary conditions and its objective to avoid capture. Using the data given to the evader, it must come up with a destination cell for itself. The evader has to decide on the best possible cell location around its present location that it can visit in one step time. It allocates the cell with the highest probability to get away from all the pursuers present (the best possible cell) on the grid as its destination coordinates and moves into the new cell. These steps are repeated at every cell visited by the evader. So, for an evader, as like pursuer, the final destination keeps changing dynamically.

All mentioned solutions for the problems faced for an evader are solved using the Evader path planning algorithm. The algorithm is divided into pursuit evasion main module to decide a capture free path from both the pursuers, check path module to see if it gets caught in the path

32 decided and speed check module that sees if capture is avoided by changing the speed of the evader. As in a pursuer, evader uses the communication module to locate itself and other agents on the grid and algorithm stopping module to indicate evaders capture and immediate stopping of the movement of all robots on the grid. This algorithm is compared with two base algorithms.

One in which the speed of the evader is not varied by not implementing the speed check module and the other in which evader considers only one predator at a time. Each of these modules are discusses in detail in the below sections.

4.4.1 Evader Path Planning with Speed Fluctuation

The evader path planning is divided into three basic modules. First is the pursuit evasion path planning module which decides a capture free path from both the pursuers. Second is the check path module to see if it gets caught in the decided path. Lastly the speed check module that sees if capture can be avoided by changing the speed of the evader. These modules help the evader to move on the 8x4 enclosed space grid and avoid capture from two pursuers.

4.4.1.1 Evader’s Path Planning Main Method

An evader algorithm tracks and updates its own location on the grid to check for the border limits. Every time the agent crosses a black line on the grid indicates moving into the next cell. It also continuously tracks the positions of both the pursuers and calculates the direction of movement which will take it away from both the pursuers. The direction is chosen by considering the location of the pursuers and considering the cell with the highest probability of escaping the pursuers. Once evader picks a direction of motion it continues to move in that path

33 in a straight line as a traversing in a straight line takes less time than reaching destination with a lot of turns. The forward motion continues until it reaches a border, a corner or a cell in which collision with one of the pursuers in its future path is unavoidable. The main method calls other modules to perform the above task with logic, intelligence and speed.

P P 1 2 Figure 4. 4: Evader path planning

4.4.1.1.1 Algorithm for Evader’s Main Method

All the agents are placed randomly on the grid. Similar to the pursuer’s algorithm, an evader’s algorithm starts with providing the agent with its location on the grid and the direction it is facing. The coordinate details of all the agents are communicated to each other using

COMMUNICATE_LOCATION(). Turn_towards is the new direction in which the agent is supposed to turn and move on. It can have only 4 values 0, 1, 2, and 3 same as direction. The turn_towards direction gets calculated by the probability of a cell around the evader which takes it further away from both pursuers and is achieved by calling UPDATE_DIRECTION() method.

After the new direction is determined, a check is made to see if none of the pursuer’s path coincide with the evader’s new path by calling the CHECK_PATH() method. If the path is clear

the speed for the evader is set according to the distance between the evader and the pursuers by

calling CHECK_SPEED() method. Only after all these checks does the evader move into the

new cell. All these checks need to be performed at every cell visited by the evader. And the cycle

is repeated until the evader is caught. The success of the algorithm is measured by the amount of

time the evader can plan a path and keep moving on the grid without getting caught. Section

4.6.2.1 shows the flowchart of the discussed algorithm.

1 BEGIN 2 Input: Starting x, y coordinates; Direction robot is currently facing

3 Initialize Khepera III Motor Controller 1 and 2; 4 COMMUNICATE_LOCATION (); 5 while (Evader is not caught) do /* pursuer and prey do not exist in same block on grid*/ 6 COMMUNICATE_LOCATION () /* communicates each other’s location on the grid */ 7 turn_towards_direction = UPDATE_DIRECTION () /*calculates the new x or y direction which will lead towards the prey*/

8 if (CHECK_PATH ()) do /*checks capture in next 2 steps on its path with all pursuers. Also checks for border conditions. Returns true if future block is safe to proceed.*/

9 Turn towards the new direction; 10 speed = CHECK_SPEED ();

11 Move to next block; 12 else do 13 move 1 block back; 14 end

15 end 16 If (target is caught) do 17 PREY_CAUGHT (); 18 STOP;

19 end

Algorithm 4. 4: Evader’s main algorithm

4.4.1.2 Updating Direction for Evader

P 1

P 2 Figure 4. 5: Evader updating its direction of motion

Computing the direction of motion for an evader is complex as it needs to take into account the positions of both the pursuers and develop a path. It is easy to program the evader’s new direction when both the pursuers come at it from the same direction. When pursuers come from two different sides, the evader should be intelligent enough to move on the other axis, or wait until the pursuers change their position giving it a chance to move away. For this reason, the algorithm keeps track of the previous location of the pursuers and calculates the directions from which they are attacking. The evader also has the border coordinates programed into the algorithm so that it does not move out of bounds. For example, if the evader is at (1, 0) and a pursuer is at (0, 4) moving in WEST direction, the evader is at the border and cannot move in the negative x direction so the evader needs to turn towards NORTH. Whereas, if the same pursuer at the same location is moving in NORTH direction, needs to turn SOUTH and move out of the pursuer’s way.

4.4.1.2.1 Algorithm for Updating Direction

6 if (both predators to the left) do 7 turn_towards = EAST; 8 else if (both predators to the right) do 9 turn_towards = WEST; 10 else if (one is above and other is below) do 11 if (prey is closer to x_maximum boundary) do 12 turn_towards = WEST;

13 else do 14 turn_towards = EAST; 15 end 16 end 17 else if (not at y border) do 18 if (both predators are above) do 19 turn_towards = SOUTH;

20 else if (both predators are below) do 21 turn_towards = NORTH; 22 else if (one is to the left and other is to the right) do 23 if (prey is closer to y_maximum boundary) do 24 turn_towards = SOUTH; 25 else do 26 turn_towards = NORTH;

27 end 28 end 29 else if (at a border and predators at either side) do 30 wait until safe to move; 31 else if (corner) do 32 if (predator_1 is closer) do 33 turn towards predator_2;

24 else do 25 turn towards predator_1; 36 end 37 end 38 STOP

Algorithm 4. 5: Update direction for evader algorithm 37

It is important for the evader to know the direction of movement of both the predators to know from which direction it’s being attacked. By evaluating the increment or decrement in the x or y coordinates of an agent their direction of motion can be evaluated. The previous locations of both the predators are stored to calculate their individual direction of movement. By checking the location of evader, locations of both the pursuers and directions of the pursuers the new direction for the evader is calculated.

This algorithm module is divided into three parts. First case is when the evader is in the middle of the grid. Here it needs to maximize the distance between itself and the pursuers by choosing a safe axis and start to move in that direction until it reaches a boundary or is cut off by a pursuer. If both pursuers are to the left, move right, if pursuers above move below, if pursuers are to right move left and if pursuers are below move up. This case also deals with pursuers being on either side of the pursuer, where the evader chooses the axis in which pursuers do not surround it. For example, when evader is at (2, 2), pursuer 1 is at (4, 2) and pursuer 2 is at (0, 2) the evader moves SOUTH as evader is further away from y_maximum boundary. Second case is when the evader is at a border and it needs to make a decision of which side to turn in the other axis depending on the position of the pursuers. If pursuers at on one side of the evader, the evader moves in the opposite direction. The problem faced by evader is when the pursuers are on either side of the evader and evader is at the boundary. In this cornering case, evader waits until either of the pursuers changes their position to give it a scope to move away. Third and final case is when it is at one of the corners of the grid. Here evader evaluates the direction in which the pursuers are moving towards it and proceed in the perpendicular direction. If evader faces the corner condition where the pursuers are attacking it from perpendicular directions, it moves towards the pursuer which is further away from evader’s current position. The

UPDATE_DIRECTION module handles all these cases and comes up with the best possible block the evader can move into. The figure below shows all these cases and a possible movement for the evader. Section 4.6.2.2 gives the flowchart of the discussed algorithm.

4.4.1.3 Check Path

P 1

E P 2 Figure 4. 6: Situation for path check

The evader at all times needs to place the pursuers are on the grid with respect to itself. It should also keep track of which direction the pursuer is moving in by storing the previous location occupied by the pursuer. Using this information, evader should check if in its future cell the pursuer might intercept its path. This is achieved by calculating the future blocks of the evader and the pursuers and comparing them individually. By knowing the present location of an agent and the direction it is going to move the future cells are calculated. If an agent is moving in the NORTH direction y is incremented, if in SOUTH direction y is decremented, if moving in

EAST direction x is incremented and finally if in WEST direction x is decremented. This module is exactly like the collision avoidance module of the pursuer with the difference that the evader is always the agent with the lower priority because of which it must stop in its path and re-calculate

39 an escape route. For example, if an evader is at (0, 1) moving towards EAST and a pursuer is at

(0, 3) moving towards the evader, the evader needs to stop or even move back one step (to give it time to calculate a new route) and turn NORTH for the escape route.

4.4.1.3.1 Algorithm for Check Path

1 BEGIN

2 Get predator_1_coordinates, Direction_1, predator_2_coordinates, Direction_2;

3 Calculate next_location_of _predator_1 and next_location_of _predator_2;

4 Calculate next_location_of _agent

5 If (next_location_of _predator_1 == next_location_of _agent or next_location_of _predator_2 == next_location_of _agent) do

6 STOP;

7 Goto main_method () and break to re-calculate route;

8 else do

9 move in the specified direction;

10 end

11 STOP

Algorithm 4. 6: Check path algorithm

The module starts with passing the directions and coordinated of the two pursuers and the evader. Using these values the future cells to be visited by the pursuers and the evader are calculated separately. The future cells going to be occupied by the pursuers are compared with the present and future cells occupied by the evader. If either of the pursuers intercepts the future path of the evader, the evader stops, takes a step back and re-routes its path around the grid. If

40 there is no capture detected on its path the evader turns towards the new direction and starts moving on the intended path. Section 4.6.2.3 gives the flowchart of the discussed algorithm.

4.4.1.4 Speed Check

An important module in the evader algorithm is the speed check, as this helps the evader to move away from a tight situation. It calculates the distance between the evader and the pursuer and according to the distance speeds up to reach a destination where it is safe form the pursuers.

All the agents on the grid start off with the same initial speed. Where the pursuers have constant speed, the evader has variable speed. The increase in speed is calculated using the distance the prey has to travel and the distance between itself and pursuers. For example, if the pursuer is at

(2, 1) and the evader is at (2, 0), the evaders needs to put at least 2 blocks distance between itself and the pursuer to be able to avoid capture. This algorithm is compared, in future sections, with an algorithm which out speed check module to show how the algorithm is improved by including this module.

P 1

E P 2 Figure 4. 7: Situation to increase speed

4.4.1.4.1 Algorithm for Check Speed

1 BEGIN

2 Get predator_1_coordinates, predator_2_coordinates;

3 Calculate dist1 and dist2; /*dist1 and dist2 are the distance between evader and pursuer1 and pursuer2 respectively. */

4 minimum dist = 2; /* the minimum distance is the distance in block away from either

of the pursuers where the evader is safe from capture*/

5 If (dist1 <= 2 or dist2 <= 2) do

6 if (dist1 <=2) do

7 speed = (dist1+2*(min_dist – dist1)) * start_speed;

8 else do

9 speed = (dist2+2*(min_dist – dist2)) * start_speed;

10 end

11 end 12 STOP

Algorithm 4. 7: Check speed algorithm

The module begins obtaining the coordinates of the pursuers and the evader. The individual distance between the evader and the pursuers is calculated. The distance is calculated by adding the number of blocks in x and y axis between the evader and pursuer as the agents are allowed orthogonal movement only by using the following equation,

( ) ( )

Any agent will have to either go fully in y axis after which on x axis or vice versa. Since the evader needs to stay away from both the pursuers equally, the new speed measure is

42 calculated if either of the pursuers is nearby. The minimum distance between the pursuer and evader where the evader is safe to move in slow speed is considered to be 2 cells away. The increase in speed is inversely proportional to the distance between the two agents. Section 4.6.2.4 gives the flowchart of the discussed algorithm.

4.4.2 Evader’s Path Planning Without Speed Fluctuation

The evader path planning algorithm was designed without the speed fluctuation to compare and analyze the outcome in each scenario tested on the agents on the grid. This algorithm is divided into two modules similar to the evader’s path planning algorithm with speed fluctuations. First is the pursuit evasion path planning module which decides a capture free path from both the pursuers. Second is the check path module to see if it gets caught in the decided path. The speed check module is not implemented in this algorithm. These modules help the evader to move on the 8x4 enclosed space grid and avoid capture from two pursuers.

The algorithm starts with all the agents being placed on the grid and provided with initial cell coordinates and facing direction. The agents set up a communication link between individual pairs and share the location and direction details with each other. A cell is chosen according to the probability of the evader maximizing its distance from the average distance between both the pursuers. The evader agent turns towards the new direction and starts moving in that direction until boundary is encountered or a pursuer cuts its path. The change in cell location is tracked by crossing the black line on the grid. At every cell visited by the evader, the future path is checked to see if pursuer will coincide with the evader. If there is such a future occurrence, the evader pauses in the present cell and recalculates its path and proceeds to move in the new path.

In certain scenarios the evader is caught when implementing this algorithm. The algorithm with speed fluctuation is tested with same scenarios to prove that speed fluctuation lets the evader to stay on the grid longer than when implementing without speed fluctuation. The results of scenarios implemented with and without the speed fluctuations are discussed and analyzed in chapter 5.

4.4.3 Evader’s Path Planning Considering Nearest Pursuer

The evader path planning algorithm with speed fluctuation is compared with another base condition case where the evader decides the probability of the cells by considering only the nearest pursuer [19]. The pursuers on the grid have the same algorithm where they chase, corral and capture the evader. The evader, on the other hand, wants to escape its nearest pursuer. The distance between a pursuer and evader is calculated by adding number of cells in x and y axis between them. Then a pursuer with the minimum distance is the nearest to the evader [19]. If there are two pursuers equally near the evader, the one with the higher coordinates is chosen as the nearest neighbor. After decision is made on the nearest pursuer, probability is allocated to all the cells around the evader to minimize the capture from that single evader. All other modules in this algorithm are similar in working to the ones with speed fluctuation.

This algorithm is not efficient as considering one pursuer at a time increases the risk of placing the evader closer to the other pursuer on the grid. In certain scenarios, evader does not even move from its initial cell as it considers one then the other pursuer alternatively as the

44 nearest pursuer. Such a scenario is discussed, compared and analyzed with the speed fluctuating algorithm in chapter 5.

4.4.3.1 Algorithm for Updating Direction

The evader path planning algorithm with speed fluctuations considers the positions of all the agents on the grid to calculate the best path. Another alternative to decide the evader’s path is to consider only the nearest pursuer. The module starts with obtaining the location coordinates of both the pursuers and their respective directions are calculated using the present and previous coordinates. The distance between a pursuer and a evader is calculated using the formula

( ) ( )

The distance is calculated in such a manner since no diagonal motion is allowed. The pursuer with the minimum d or lower coordinate values is chosen as the nearest pursuer. For example, if evader is at (2, 1) and the pursuers are at (0, 0) and (4, 0) respectively. The distance between (evader, pursuer 1) pair and (evader, pursuer 2) pair is same with the value 3 cells.

Pursuer 1 has lesser coordinate values than pursuer 2; the nearest pursuer to the evader is chosen to be pursuer 2. So evader has to maximize the distance between itself and the nearest pursuer chosen. There is only one pursuer to consider at a point of time it becomes easier for the evader to decide its path. As the grid is longer in length than width, the evader tries to move along the length first then the width. This restriction and considering only one pursuer puts the evader at a high risk of moving into a cell closer or occupied by the other pursuer on the grid. Section

4.6.2.5 gives the flowchart of the discussed algorithm.

1 BEGIN 2 Get predator_1_coordinates, predator_2_coordinates; 3 Calculate directions of both predators using the predators present and old coordinate; 4 Store old_predator_1_coordinates = predator_1_coordinates and old_predator_2_coordinates = predator_2_coordinates; 5 if (predator_1_coordinates are nearer to evader or predator_1_coordinates > predator_2_coordinates) do 6 destination_coordinates = predator_1_coordinates; 7 destination_direction = predator_1_direction; 8 else if (predator_2_coordinates are nearer to evader or predator_2_coordinates > predator_1_coordinates) do 9 destination_coordinates = predator_2_coordinates; 10 destination_direction = predator_2_direction; 11 end 12 If (not at x border) do 13 if (destination_coordinates to the left) do 14 turn_towards = EAST; 15 else if (destination_coordinates to the right) do 16 turn_towards = WEST; 17 end 18 else if (not at y border) do 19 if (destination_coordinates is above) do 20 turn_towards = SOUTH; 21 else if (both predators are below) do 22 turn_towards = NORTH; 23 end 24 else if (corner) do 25 choose perpendicular direction to destination_direction; 26 end 27 STOP

Algorithm 4. 8: Update direction considering nearest pursuer

4.5 Shared Modules

In case of a single agent performing tasks, there is no need for any kind of communication. When there is more than one agent in an area, communication is necessary so that there is no collision or no two agents try to perform the same task. All the agents may it be a pursuer or an evader share a common method to communicate their locations to each other. The other shared functionality between a pursuer and an evader is the module used to terminate the all the algorithms called when the evader is caught. This method when called instantly stops the movement and running of the algorithm in all the agents.

4.5.1 Communication Module

In a predator-prey domain research up till now, the locations of the pursuers and evaders are communicated to all the agents in many different ways to obtain a distributed intelligent path.

To name a few ways, wires or cables or sensors are used to detect other agents around them, photographic images are processed to get the required information or Wi-Fi can be used to indicate the locations. A wireless Ethernet with KoreBot and a Wi-Fi card combination has been used for the purpose of locating all the agents on the grid. The pursuer needs the locations to avoid collision with other pursuers and to catch the evader, whereas the evader needs to know the location of all pursuers to move away from those locations.

Multitasking is useful in writing programs that simultaneously manage several different input and output devices, or sensors and effectors in the case of a robot.

A Server and Client model is designed using threads and implemented using the UDP

Protocol over TCP/IP protocol. Threads were used for simultaneous executing of the procedures

47 so that the exchange information was performed without any delay. Server thread is used to receive the updated location of other agents. It is called in a while loop to run continuously to receive the updated information at any moment. The client thread is called to send the location of the agent to other agents. This thread runs only when the location needs to be updated and as soon as its purpose is fulfilled, the thread is terminated.

Figure 4.7: Communication between agents

4.5.2 Stop Algorithm Module

Both pursuer and evader algorithms terminate when any one of the pursuer catches the evader. When the evader is caught, the stop algorithm module or the PREY_CAUGHT method is called. In the method, the location of the agent is set to 1000 and communicated to all other agents on the grid indicating the capture of the evader and the algorithm running on the agent is terminated. When the other agents receive 1000 as the location of any one agent on the grid knows the evader is caught also terminate the algorithm and come to a stop.

4.6 Flow Chart

4.6.1 Pursuer Flow Chart BEGIN 4.6.1.1 Main Method

Get Start (x, y) coordinates and direction.

Send current coordinates to other agents.

Get other agents coordinates.

Is Prey caught Ye s Send stop coordinates. by agent or Stop motors other pursuer?

N STOP Do UPDATE_DIRECTION() get new direction to move in.

No AVOID_COLLISI ON()?

Yes

1. Turn towards new direction 2. Update direction

Move in the present direction

Read ground sensors. Change coordinates

when new block entered

No Is new (x,y) border or

x=prey_x or y=prey_y?

Yes 49

4.6.1.2 Flow Chart For Corralling Model

BEGIN

Destination coordinates = coordinates of Prey

Is current_coordinates > s predator_coordinates?

Priority is low

Priority is high Move in y Direction

Move in x Direction

Prey to above of pursuer? N

Prey to right of o pursuer? Yes New_direction = NORTH N Yes o New_direction = SOUTH New_direction = EAST

New_direction = WEST

Send New_direction to main_method

STOP

4.6.1.3 Flow Chart For Avoid Collision

BEGIN

Get other predator coordinates

Calculate future location of agent and other predator

N Do new coordinates coincide?

Ye s N o Is the priority low? Return true

Yes

STOP Movement

Return false

STOP

4.6.2 Evader’s Flow Chart

4.6.2.1 Evader’s Main Module

BEGIN

Get Start (x, y) coordinates and direction.

Send current coordinates to other agents.

Get other agents coordinates.

Is Prey s Send stop coordinates. Stop motors caught? N

Do UPDATE_DIRECTION() get new direction to move in. STOP

No CHECK_PATH()?

Yes Move back 1 block

1. Turn towards new direction 2. Update direction

speed = CHECK_SPEED()

Move in the present direction

Read ground sensors. Change coordinates when new block entered

4.6.2.2 Flow Chart For Updating Direction Of Evader

BEGIN

Get predator_1_coordinates, predator_2_coordinates

Calculate direction1 and direction2

old_predator_1_coordinates = predator_1_coordinates; old_predator_2_coordinates = predator_2_coordinates;

At x border?

No Yes Both pursuers to left? At Y border? No

No Yes Yes Both pursuers above? New_direction = EAST Is corner? New_direction = WEST No Yes Yes New_direction = SOUTH No Predator 1 New_direction = NORTH closer?

wait until safe to Yes move

Move towards predator

Move towards predator 2

Send New_direction to main_method

STOP53

4.6.2.3 Flow Chart For Check Path

BEGIN

Get predator 1 coordinates and Direction; Get predator 2 coordinates and direction;

Calculate future location of agent and both predators

No Do new coordinates coincide?

Yes

STOP Movement Return true

Return false

STOP

4.6.2.4 Flow Chart For Check Speed

BEGIN

Get predator 1 coordinates; Get predator 2 coordinates; Set min_dist=2;

Calculate dist1 and dist2 using predator and prey

Is dist1 <= 2 or

dist2 <= 2? N

s N Is dist1 <= 2?

Speed = start speed Ye s Speed = (dist1+2*(min_dist – dist1) ) * start_speed;

Speed = (dist2+2*(min_dist – dist2) ) * start_speed;

Return speed

STOP

4.6.2.5 Evader’s Update Direction Module Using Nearest Neighbor

BEGIN

Get predator_1_coordinates, predator_2_coordinates

Calculate direction1 and direction2; Calculate dist1 and dist2 using predator and prey co-ordinates

Is (dist1 < dist2) or (predator_1_coordinate>predator_ 2_coordinate) No Yes

destination_coordinates = predator_2_coordinates; destination_coordinates = predator_1_coordinates; destination_direction = predator_2_direction;

destination_direction = predator_1_direction;

No At x border? Destination to left? No Yes No Yes At Y border? New_direction = EAST

New_direction = WEST No Yes Destination is above? Destination_direction

is on x axis Yes No

New_direction = SOUTH Yes Move in x axis New_direction = NORTH Move in y axis

Send New_direction to main_method

STOP 56

4.7 Summary

In this chapter, the strategies for the algorithms and sub-modules for pursuers and evader were designed and discussed in sections 4.4 and 4.5. The corresponding flowcharts for the discussed algorithms, is given in section 4.6 in the same order. Next chapter deals with the implementation of these algorithms on Khepera III robots placed on a grid. The paths planned individually for the two pursuers and evader is discussed for various scenarios.

Chapter 5

Results and Analysis

5.1 Introduction

The algorithms for two pursuers and evader discussed in the previous chapter were implemented on three Khepera III robots on a two-dimensional grid. The robots were placed randomly on the grid and tested for the practical working of the algorithm. The algorithms were applied to various scenarios to study the success of evader’s escape. The results for two such scenarios covering all the possible strategy decisions thus produced are examined, compared and analyzed in this chapter. The placement of the agents, initial direction of the agent, time taken to establish contact and communicate corresponding locations, dynamic obstacle avoidance, and velocity variation play an important role in generating an intelligent path for the agent.

Certain initial conditions were applied to all the agents to ensure proper coordination and comparable behavior. Firstly, the grid was gauged with x and y axis forming the 2D location coordinates of the agents. The direction of the agents on the grid was established according to

North=0, South=2, East=1 and West=3 with the grid boundaries ranging from (0, 0) to (7, 3).

Secondly, all the agents placed on the grid are aware of the boundaries of the grid and that no diagonal movement is allowed. Each agent is provided with their initial location coordinates and

58 the direction they are facing for them to obtain a sense of direction and place themselves on the grid. The agents turn with the same speed and always make 90degree turns. Thirdly, Wi-Fi was used to set up a communication between the agents to help them place each other on the grid.

Lastly, all the agents are given the same starting speed (which changes for the evader with the proximity of the pursuers). Once a direction for movement was chosen, the agent kept moving in that direction until the border or a corner is reached or to avoid an eminent collision. The decision is biased since the time taken to stop and turn many times over is more than moving in a straight line. Infrared sensors of the Khepera III robots help track the black lines of the grid indicating change in cell location by incrementing or decrementing the coordinates. The distance between any pair of agents is calculated by adding the number of blocks between them in x and y axis.

The algorithm was tested on a number of scenarios in which two different test cases representing all the strategies are presented in this chapter, one where the pursuers are unequally distant from the evader and the other where the pursuers are equally distant but at either side of the evader. As the pursuer and evader algorithms run continuously and simultaneously, the paths generated for the pursuers and evader is discussed concurrently for every single time step. A single time step is considered the time taken by the agent to move from one block to another at the starting speed. Each of these test case results is compared with base algorithms for the evader to validate the efficiency of the algorithm with speed fluctuation. One base algorithm used for the comparison being without varying velocity and the other considering only the nearest pursuer to demonstration the importance of the modules used in the pursuit evasion algorithm.

5.2 Test Cases

In this section two main scenarios are described covering all strategies designed for evader to avoid capture by two pursuers. The success of these results are compared and analyzed with the results obtained by running the base algorithms.

5.2.1 Scenario 1

Figure 5.1 shows the first scenario which is a simple case where the pursuers are in line with the evader and are placed at unequal distance from the latter. The initial coordinates of the pursuer 1 are (0, 3) and direction it is facing is EAST. The initial coordinates of the pursuer 2 is

(2, 0) with the direction as NORTH. Initial coordinates (2, 3) facing EAST is given to the evader.

P E 1

P 2 Figure 5. 1: Scenario 1 - agent placement

Fgure 5.2 shows the path taken by the pursuers and the evader. The initial locations of the agents are marked where the red and orange lines representing pursuer 1 (P1) and 2’s (P2) path respectively and the blue lines representing the evader’s (E) path. Each arrow represents one

60 time step. Any coordinates specified together follow coordinate of pursuer 1(P1) followed by coordinate of Pursuer 2 (P2) and then coordinate of Evader (E).

All agents set up a pair wise connection between each other and share each other’s location coordinates. The two pursuers are allocated a priority each with P2 with lower priority moving in the y axis. They start moving towards the evader’s initial location (2, 3). In the first time step, since P1 is 2 blocks away from E, which is not a safe distance for the evader, it doubles it’s speed to cover 2 blocks in one time step and reaches (4, 3) when P1 and P2 reach

(1,3) and (2,1) respectively. In the next time step, all agents move one block in the direction they were moving and reach (2, 3), (2, 2) and (5, 3) respectively. Here P2 cannot continue in its present direction to avoid collision in future block with P1. So to avoid collision P2 recalculates its route and moves into (3, 2) and P1 continues in its path into (3, 3) and E move into (6, 3). All agents move one step more reaching (4, 3), (4, 2) and (7, 3) respectively.

E 3 P1

P2 0

0 1 2 3 4 5 6 7 Figure 5. 2: Path of pursuers and evader for scenario 1

Now E encounters the corner condition, where it checks the pursuers location and direction in which they are moving and turns towards SOUTH moving into (7,2) and the pursuers move into (5,3) and (5,2) respectively. Evader now is again in the unsafe zone with number of blocks between itself and P2 is 2. So it speeds up and moves into (7, 0). P2 moves into (6, 2) and P1 moves into (6, 3). Evader is at another corner, so it decides to move towards

WEST into (6, 0) at normal speed. Pursuer 2 to corral the evader with the lower priority turns

SOUTH and moves into (6, 1) and P1 continues in its motion into (7, 3). With speed change in evader, these scenarios repeat themselves and the evader is able to avoid capture. Table 5.1 shows the time step path coordinates of pursuer 1, pursuer 2 and evader for the first 16 time steps. The same kind of steps repeat themselves leading to an infinite loop as the termination clause (evader caught) is not reached.

Time Step Pursuer 1’s Position Pursuer 2’s Position Evader’s Position 0 0,3 2,0 2,3 1 1,3 2,1 4,3 2 2,3 2,2 5,3 3 3,3 3,2 6.3 4 4,3 4,2 7,3 5 5,3 5,2 7,2 6 6,3 6,2 7,0 7 7,3 6,1 6,0 8 7,2 6,0 4,0 9 7,1 5,0 2,0 10 6,1 4,0 1,0 11 5,1 3,0 0,0 12 4,1 2,0 0,1 13 3,1 1,0 0,2 14 2,1 0,0 0,3 15 1,1 0,1 1,3 16 1,2 0,2 2,3 Table 5. 1: Path of pursuers and evader for scenario 1

5.2.1.1 Results Comparison and Analysis

Figure 5.3 shows the path planning of both the pursuers and evader when the evader cannot change its speed. Table 5.2 shows the path coordinates for pursuer 1, pursuer 2 and evader for the same condition. At this point, it is seen that the evader and pursuers path planning starts out same as the previous algorithm discussed. But without the change in speed the evader gets caught in the 7th time step when pursuer 2 moves into (7, 2) before evader can move out of this fix. Comparing the table 5.1 and 5.2 it is seen that an evader with change in speed is able to survive longer without capture.

E 3 P1

0 P2

0 1 2 3 4 5 6 7 Figure 5. 3: Scenario 1 without speed change

Time Step Pursuer 1’s Position Pursuer 2’s Position Evader’s Position 0 0,3 2,0 2,3 1 1,3 2,1 3,3 2 2,3 2,2 4,3 3 3,3 3,2 5.3 4 4,3 4,2 6,3 5 5,3 5,2 7,3 6 6,3 6,2 7,2 Table 5. 2: Scenario 1 path coordinates with evader without changing speed

5.2.2 Scenario 2

P P 2 1 Figure 5. 4: Scenario 2 agent placement

Figure 5.4 shows another scenario where the pursuers P1 and P2 are placed at equal distance to the evader attacking it from opposite directions. The pursuer 1 is positioned at (0, 0) facing EAST, pursuer 2 is at (4, 0) facing NORTH and the evader is placed at (2, 1) facing

NORTH. Like in scenario 1, the figure 5.5 shows the path taken by the pursuers and the evader with the red, orange and blue lines representing pursuer 1 (P1), 2’s (P2) and evader’s (E) path respectively. Each arrow represents the movement of each agent in one time step where a single time step is the amount of time needed for an agent to move from one block to another with the starting speed. For the analysis of the path planning of the evader and pursuers, any coordinates specified together follow coordinate of pursuer 1(P1) followed by coordinate of Pursuer 2 (P2) and then coordinate of Evader (E).

The pursuers and evader as mentioned above are placed initially at (0, 0), (4, 0) and (2, 1) and once the algorithms start running all agents move continuously and simultaneously. In time step 1, the pursuers decide from which side to corral the evader, since P1 gets allocated higher

64 priority due to its location on the grid it turns towards EAST and moves into (1, 0), P2 moves

NORTH into (4, 1). Since both the pursuers are placed below the evader, E moves NORTH into

(2, 2). In the next step, the agents continue in their path into (2, 0), (4, 2) and (2, 3) cells respectively. Now, P1 is in the same x coordinate as evader and it decides to turn NORTH and move into (2, 1) whereas P2 moves into (4, 3). The evader, on the other hand, encounters boundary condition and makes a decision to turn WEST and moves into (1, 3). In time step 4, P2 is at the border and to follow the evader turns WEST and moves into (3, 3), P1 goes to (2, 2) and evader to (0, 3).

1 E

0 P1 P2

0 1 2 3 4 5 6 7 Figure 5. 5: Path for pursuers and evader for scenario 2

The evader encounters the corner condition and decides to move SOUTH into (0, 2).

Since P1 and P2 are at (2, 2) and (3, 3) about to move into (2, 3), a collision is about to happen.

P2 having higher coordinates than p1 gets lower priority and has to break and re-route a new path to capture the evader. Whereas, P1 has the same x coordinates as evader, so it breaks as well and recalculates a new direction. As the evader moved into (0, 2), P1 turns WEST wards and

65 moves into (1, 2), whereas P2 turns towards SOUTH and moves into (3, 2). At this point of time,

P1 is at an unsafe distance from E, so the speed of evader is varied and the evader moves into (0,

0) and P1 and P2 move into (0, 2) and (3, 1). For the evader, distance from the P1 is still not at a safe distance and P1 is right behind E, evader turns towards EAST and moves into (2, 0). P1 and

P2 move into (0, 1) and (3, 0) cells respectively. The evader checks its future path and realizes that because of speed change it has moved too fast and is about to move into the same cell as P2 and get captured. At this point, evader doesn’t have time to recalculate a new route and make a turn. So it reverses into the previous location (1, 0) to get enough time to re-route, this is represented by the doted blue line in the Figure 5.5. After re-routing, evader by increasing its speed moves into (1, 1) before it gets captured by the Pursuers 1 and 2 who have moved into (0,

0) and (2, 0).At this point, P1 is at (0, 0), P2 at (2, 0) chasing the evader at (1, 1).

1 E

0 P1 P2

0 1 2 3 4 5 6 7

Figure 5. 6: Continuation of path for scenario 2

Figure 5.6 shows the continuation of the problem from here on, where it is seen that the evader is again in a position on the grid to form an infinite loop to evade capture as in the

66 previous scenario. The table 5.3 shows the coordinates of all the agents on the grid for the first

16 time step. At step 7, the evader moves into (2, 0) but to avoid capture reverses into (1, 0). The final position of the evader after the time step 7 is (1, 0) therefore the coordinates are in the brackets indicate an interim position of the agent. It can be seen from the coordinates in the table

5.3 that the agents will keep revolving around the grid as the evader is not caught.

Time Step Pursuer 1’s Position Pursuer 2’s Position Evader’s Position 0 0,0 4,0 2,1 1 1,0 4,1 2,2 2 2,0 4,2 2,3 3 2,1 4,3 1.3 4 2,2 3,3 0,3 5 1,2 3,2 0,2 6 0,2 3,1 0,0 7 0,1 3,0 (2,0) 1,0 8 0,0 2,0 1,1 9 1,0 2,1 1,3 10 1,1 2,2 0,3 11 1,2 2,3 0,2 12 0,2 1,3 0,0 13 0,1 1,2 2,0 14 0,0 1,1 3,0 15 1,0 2,1 4,0 16 2,0 3,1 5,0 Table 5. 3: Path coordinates for pursuers and evader for scenario 2

5.2.2.1 Results Comparison and Analysis

In this scenario, evader with the help of reversing into the previous cell and accelerating for a small amount of time is able to dodge the pursuers for at least 16 time steps. For comparison, figure 5.7 shows the paths of evader and pursuers when the same initial positions are given to the agents where evader is unable to change its speed. It is observed that the evader and pursuers path planning starts out same as the algorithm discussed in section 5.2.2 for the first

6 time steps. It is seen that at step 7, the evader is caught by P2 at location (1, 0) since the evader is unable to speed up and move out of the cell before it is caught. The location coordinates for the agents are as shown in table 5.4. Comparing the table 5.3 and 5.4 it is seen that an evader with change in speed is able to survive longer without capture.

1 E

0 P1 P2

0 1 2 3 4 5 6 7 Figure 5. 7: Scenario 2 without speed change

Time Step Pursuer 1’s Position Pursuer 2’s Position Evader’s Position 0 0,0 4,0 2,1 1 1,0 4,1 2,2 2 2,0 4,2 2,3 3 2,1 4,3 1.3 4 2,2 3,3 0,3 5 1,2 3,2 0,2 6 0,2 3,1 0,1 7 0,1 3,0 0,0 8 0,0 2,0 1,0 Table 5. 4: Path coordinates without speed change

Figure 5.8 shows the paths of evader and pursuers with the same initial positions but the evader considers only the nearest pursuer to decide its escape path. Here, initial positions of pursuers and evader are (0, 0), (4, 0) and (2, 1). In the first time step, the evader considers the distance between itself and each pursuer to decide which pursuer is nearest. Since both the pursuers are equally distant from the evader it considers the pursuer with higher coordinates that is P2 as the nearest pursuer. Since P2 is to the right of evader, it turns towards WEST and starts to move into (1, 1). In the meantime, P1 moves into (1, 0) and P2 moves into (4, 1). In the next time step, evader calculates nearest pursuer to be P1. So it turns around to face EAST and move into (2, 1). P1 on the other hand, had the same x coordinates as the evader, so it turns towards

NORTH and moves into (1, 1). P2 has the same y coordinates as the evader so it turns towards

WEST and moves into (3, 1). The evader is surrounded and cannot make a move to escape and gets caught.

1 E

0 P1 P2

0 1 2 3 4 5 6 7 Figure 5. 8: Path planning of agents when considering only nearest neighbor

Time Step Pursuer 1’s Position Pursuer 2’s Position Evader’s Position 0 0,0 4,0 2,1 1 1,0 4,1 1,1 2 1,1 3,1 2,1 Table 5. 5: Path coordinates using nearest pursuer for scenario 2

Table 5.5 shows the path coordinates for pursuer 1, pursuer 2 and evader for the same algorithm discussed above. It is seen that, when evader considers only one pursuer it moves only in the x axis direction and places it in a cell closer to the second pursuer. The evader moves in a circle in its initial cell and finally gets caught. By considering only nearest pursuer, the evader gets caught in the 3rd time step when pursuer 2 moves into (2, 1) not giving the evader a chance to move out of this fix. Comparing the table 5.3 and 5.5 it is seen that an evader considering both the pursuers on the grid is able to survive longer without capture.

5.2.3 Scenario 3

Initially all three agents need to establish a connection between each other and communicate their locations. This only takes a fraction of a second, but this small amount of time is very important as if the coordinates of pursuers are not communicated in time to evader, the evader will not be able to make a move and will be caught if the pursuers are in the next cell.

One such scenario is depicted in the figure 5.8.

Another scenario where the evader is not given a scope to route an escape is depicted in the figure 5.9. Here even though the evader has enough time to obtain the locations and process them, the pursuers move into the next spot corralling the evader into the corner and the evader is caught.

P E 1

P 2 Figure 5. 9: Scenario for the evader to be caught immediately

P 1

E P 2 Figure 5. 10: Scenario for the evader to be caught immediately

Chapter 6

Conclusions and Future Work

6.1 Conclusion

In this thesis, an algorithm each for a pursuer and an evader moving on an 8x4 grid was successfully designed and implemented. The algorithm was tested on two agents pursuing a lone target. Each agent autonomously executes the algorithm over the global information available and generates an intelligent path favoring the common goal. When each agent moves it updates it position, communicates the new position to others on the grid, receives the latest location of its neighbors and based on all this information makes a decision on the next cell to be visited. A turning constraint was applied on all the agents of being able to turn only 90degrees so that no diagonal motion was allowed.

The two pursuer agents aided with the knowledge of each other’s current location were successfully able to avoid collision. And with the awareness of location of the prey are able to corral and capture the evader moving with a constant speed by coming at it from two different directions. The key objective was to obtain pursuit evasion for as long as possible by using agents with turning constraints and adding acceleration and deceleration to the evader. The evader according to the whereabouts of pursuers can increase its speed, come to a complete stop

72 or even reverse into the previous cell to avoid capture. The results and analysis section discusses the scenarios with and without acceleration and the resulting escape and capture of the evader respectively.

In the said predator-prey domain along with all the constraints applied on the agents, it was seen that the evader was able to escape capture for as long as 30 to 35 steps after which one of the agents run out of memory to process the algorithm and stop behaving in the required manner. Also it was seen that, the initial placement of the agents on the grid mattered in evader’s capture that is any scenario where the pursuers were placed two or less cells away from the evader, the evader was captured. Due to wear and tear of the robots, the 90degree turns weren’t perfect all the time causing the agent to move into other cells. To avoid this, manual adjustments for each robot from time to time was needed.

6.2 Future Work

Path planning for evader and pursuers can be extended by evolving other ideas from this research work.

 First extension would be to increasing the difficulty in the terrain such as placing the

agents in a ground or a floor of a room with static obstacles without providing the

locations of the obstacles. By using the IR sensors around the robot, evader should be

able to find an intelligent path to escape getting captured.

 Increasing the difficulty of the evader by adding more pursuers. The chance of an evader

escaping capture reduces with the increase in number of pursuers present on the grid [19].

On the other hand, number of evaders on the grid can also be increased to study the time

taken by the pursuers to capture all the evaders on the grid. This model can also be used

to study the effects of swarms and how the evaders or pursuers work together to obtain

the final goal [8].

 A comparative study can be made of predator-prey models using different modes of

communication such as a GPS system, central camera mounted on the top, or using a map

and discuss the best possible means of communications for various scenarios [3].

References

[1] K-Team. http://www.k-team.com/

[2] D. E. Appelt, “Planning natural-language utterances to satisfy multiple goals,” Technical Note 259, SRI International, 1982

[3] Almeida, J. M., Martins, A., Silva, E. P., and F. L. Pereira, “Simultaneous control, navigation and target tracking for robotic formations,” Proceedings of the 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp. 291, Sept, 2006.

[4] M. H. Breitner, “The genesis of differential games in light of isaacs contributions,” Journal of Optimization Theory and Applications, Vol. 123, No. 4, pp. 523-559, Mar, 2005.

[5] Bodhale, D., Afzulpurkar, N., and N. T. Thanh, “Path planning for a mobile robot in a dynamic environment,” Proceedings of the 2008 IEEE International Conference on Robotics & Biomimetics (ROBIO 2008), pp. 2115-2120, February, 2009.

[6] Botea, A., M¨uller, M., and J. Schaeffer, “Near optimal hierarchical path-finding,” Journal of Game Development, Vol. 1, No. 1, pp. 7–28, 2004.

[7] Borie, R., Tovey, C., and S. Koenig, “Algorithms and complexity results for pursuit-evasion problems,” Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence, pp. 59-66, 2009.

[8] Bruce, J., and M. Veloso, “Real-time multi-robot motion planning with safe dynamics,” Multi-Robot Systems. From Swarms to Intelligent Automata, Volume III, pp. 159-170, Springer Netherlands, 2005.

[9] E. W. Dijkstra, “A note on two problems in connection with graphs,” Numerische Mathematik, Vol. 1, pp. 269–271, 1959.

[10] J. Ferber, “Multi-agent system: an introduction to distributed artificial intelligence,” Harlow: Addison Wesley Longman, 1999.

[11] Hespanha, J. P., Kim, H. J., and S. Shankar, “Multiple-agent probabilistic pursuit-evasion games,” Proceedings of the 38th Conference on Decision & Control, Phoenix, Arizona, pp. 2432- 2437, Dec, 1999.

[12] Herbert, M., McLachlan, R., and P. Chang, “Experiments with driving modes for urban robots,” In Proceedings of the SPIE Mobile Robots, 1999.

[13] Hart, P., Nilsson, N., and B. Raphael, “A formal basis for the heuristic determination of minimum cost paths,” IEEE Transactions on Systems Science and Cybernetics, Vol 4, No. 2, pp. 100–107, 1968.

[14] Haynes, T., and S. Sen, “Evolving behavioral strategies in predators and prey,” Adaptation and Learning in Multi-agent Systems, Lecture Notes in Computer Science, Springer Verlag, Berlin, Vol 1042, pp. 113-126, 1996.

[15] R. Isaacs, “Differential games,” Dover Publications, Mineola, NY, 1965

[16] Ishida, T., and R. E. Korf, “Moving target search,” Proceedings of the International Joint Conference on Artificial Intelligence, pp. 204-210, 1991.

[17] E. de-Jong, “Multi-agent coordination by communication of evaluations,” Artificial Intelligence Lab, Vrije University, Brussel, Brussel, Belgium, 1997.

[18] Jin, S., Dechev, D., and Z. Qu, “Parallel particle swarm optimization (PPSO) on the coverage problem in pursuit-evasion games,” Proceedings of the 2012 Symposium on High Performance Computing, Article No. 12, ISBN: 978-1-61839-788-1, 2012.

[19] Kamimura, A., and T. Ohira, “Group chase and escape,” New Journal of Physics, Vol 12, May, 2010.

[20] Kim, K. H., Ko, K. W., Kim, J. G., Lee, S. H., and H. S. Cho, “The development of a micro robot system for robot soccer game,” Proceedings of the 1997 IEEE International Conference on Robotics & Automation, volume 1, pp. 644-649, April, 1997.

[21] Koenig, S., and M. Likhachev, “D* lite,” In Proceedings of the National Conference on Artificial Intelligence, pp. 476–483, 2002.

[22] Krapivsky, P. L., and S. Redner, “Kinetics of a diffusive capture process: lamb besieged by a pride of lions,” Journal of Physics A: Mathematical and General, Vol. 29, No. 17, Apr, 1996.

[23] Kachroo, P., Shedied, S. A., and H. Vanlandingham, “Pursuit evasion: the herding non- cooperative dynamic game—the stochastic model,” IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, Vol. 32, No. 1, pp. 37-42, Feb, 2002.

[24] X. Li, “A real-coded predator-prey genetic algorithm for multi-objective optimization,” Evolutionary Multi-criterion Optimization, Lecture Notes in Computer Science, Vol 2632, No. 69, pp. 207-221, 2003.

[25] LaValle, S. M., Gonzalez-Banos, H. H., Becker, C., and J. C. Latombe, “Motion strategies for maintaining visibility of a moving target,” Proceedings of the 1997 IEEE International Conference on Robotics & Automation, volume 1, pp. 731-736, April, 1997.

[26] McArlhur, D., Steeb, R., and S. Cammarata, “A framework for distributed problem solving,” Proceedings of the National Conference on Artificial Intelligence, Pittsburg, PA, pp. 181-184, 1982.

[27] Mccauley, E., Wilson, W. G., and A. M. De-Roost, “Dynamics of age-structured and spatially structured predator-prey interactions: individual-based models and population-level formulations,” The American Naturalist, Vol. 142, pp. 412-442, September 1993.

[28] Nelson, A. L., Grant, E., Galeotti, J. M., and S. Rhody, “Maze exploration behaviors using an integrated evolutionary robotics environment,” Robotics and Autonomous Systems, Vol. 46, Issue 3, pp. 159-173, 31 March, 2004

[29] P. J. Nahin, “Chases and escapes: the mathematics of pursuit and evasion,” Published by Princeton University Press, Princeton, NJ, 2012.

[30] Oshanin, G., Vasilyev, B., Krapivsky, P. L., and J. Klafter, “Survival of an evasive prey,” Proceedings of the National Academy of Sciences of the United States of America, Vol. 106, No. 33, pp. 13696–13701, Aug, 2009.

[31] Patsko, V. S., and V. L. Turova, “History of the homicidal chauffeur problem,” http://home.imm.uran.ru/kumkov/HHCP/index.html, last visited on Feb, 2007.

[32] Quottrup, M. M., Bak, T., and R. I. Zamanabadi, “Multi-Robot planning: a timed automata approach,” Proceedings of the 2004 ICRA IEEE International Conference on Robotics and Automation, Vol. 5, pp. 4417-4422, May 2004.

[33] Reece, D., Krauss, M., and P. Dumanoir, “Tactical movement planning for individual combatants,” In Proceedings of the 9th Conference on Computer Generated Forces and Behavioral Representation, 2000.

[34] Reggente, M., Mondini, A., Ferri, G., Mazzolai, B., Manzi, A., Gabelletti, M., Dario, P., and A. J. Lilienthal, “The dustbot system: using mobile robots to monitor pollution in pedestrian area,” Chemical Engineering Transactions, Vol. 23, pp. 273-278, 2010.

[35] A. Stenz, “The focussed D* algorithm for real-time replanning,” In Proceedings of the International Conference on Artificial Intelligence, pp. 1652–1659, 1995.

[36] Sugiyama, Y., Fukui, M., Kikuchi, M., Hasebe, K., Nakayama, A., Nishinari, K., Tadaki, S., and S. Yukawa, “Traffic jams without bottlenecks—experimental evidence for the physical mechanism of the formation of a jam,” New Journal of Physics, Vol 10, March, 2008.

[37] Shakernia, O., Kim, H. J., Shim, D. H., and S. Sastry, “Probabilistic pursuit-evasion games: theory, implementation, and experimental evaluation,” Proceedings of the 2002 IEEE International Conference on Robotics and Automation, Vol. 18, No. 5, pp. 662-669, Oct, 2002.

[38] Thayer, S., Digney, B., Diaz, M., Stentz, A., Nabbe, B., and M. Hebert, “Distributed robotic mapping of extreme environments,” In Proceedings of the SPIE: Mobile Robots XV and Telemanipulator and Telepresence Technologies VII, 2000.

[39] S. Woodcock, “Can AI SDKs help?,” Game Developer magazine, 2000.

[40] Wang, L. F., Tan, K. C., and V. Prahlad, “Developing khepera robot applications in a webots environment,” Proceedings of the 2000 IEEE International Symposium on Micromechatronics and Human Science, pp. 71-76, 2000.

[41] H. Yamaguchi, “A cooperative hunting behavior by mobile-robot troops,” The International Journal of Robotics Research, Vol. 18, No. 9, pp. 931-940, Sept, 1999.

[42] Zheng, J., Yu, H., Liang, W., and P. Zeng, “A distributed and optimal algorithm to coordinate the motion of multiple mobile robots,” WCICA 2008. 7th World Congress on Intelligent Control and Automation, pp. 3027, June, 2008.