1 A Comprehensive Review of Shepherding as a Bio-inspired Swarm-Robotics Guidance Approach Nathan K. Long, Student Member, IEEE, Karl Sammut, Senior Member, IEEE, Daniel Sgarioto, Matthew Garratt, Senior Member, IEEE, and Hussein Abbass, Fellow, IEEE Accepted Article: Copyright 2020 IEEE. Personal use of system, where each agent is responsible for its own processing this material is permitted. Permission from IEEE must be and functionality, allowing simpler agents to be designed obtained for all other uses, in any current or future media, rather than an all-encompassing complex single agent [3]. including reprinting/republishing this material for advertising Therefore, decentralization makes robotic systems easier to or promotional purposes, creating new collective works, for build and program, as well as introducing robustness, flexibil- resale or redistribution to servers or lists, or reuse of any ity, and scalability into the multi-robot system [2]. copyrighted component of this work in other works. Controlling multiple coordinated robotic agents presents a complex problem which can be categorized into high- Abstract—The simultaneous control of multiple coordinated level path planning and low-level single agent dynamics. robotic agents represents an elaborate problem. If solved, how- While various swarm control techniques have been devel- ever, the interaction between the agents can lead to solutions to sophisticated problems. The concept of swarming, inspired by oped, including using neighbourhood topologies [4], artificial nature, can be described as the emergence of complex system- potential functions [5], [6], and different forms of adaptive level behaviors from the interactions of relatively elementary control [7], [8]; one method, termed shepherding, has emerged agents. Due to the effectiveness of solutions found in nature, bio- which divides the two control functions into separate entities. inspired swarming-based control techniques are receiving a lot of Vo, Harrison and Lien [9] defined the motion planning of a attention in robotics. One method, known as swarm shepherding, is founded on the sheep herding behavior exhibited by sheepdogs, group of robots using an external entity as the group control where a swarm of relatively simple agents are governed by problem, describing it as underactuated and dynamic, which a shepherd (or shepherds) which is responsible for high-level elicits multi-agent cooperation. guidance and planning. Many studies have been conducted on Shepherding, as a swarm control technique, was inspired by shepherding as a control technique, ranging from the replication sheepdogs and sheep, where the shepherding problem can be of sheep herding via simulation, to the control of uninhabited vehicles and robots for a variety of applications. A comprehensive defined as the guidance of a swarm of agents from an initial review of the literature on swarm shepherding is presented in location to a target location [10]. Essentially, the shepherd is order to reveal the advantages and potential of the approach to responsible for the high-level path planning of the swarm and be applied to a plethora of robotic systems in the future. task allocation, while the swarm members themselves function Index Terms—Autonomous vehicles, Herding, Shepherding, solely on single agent dynamics. Swarm guidance, Swarm intelligence, Swarm robotics, Uninhab- It should be noted that while previous literature has referred ited aerial vehicle (UAV). to the group of agents guided by the shepherd as a flock or herd, this review uses the term swarm for two reasons. I. INTRODUCTION First, the modeling of a flock is mostly done using swarm NSPIRED by animal swarms in nature, swarm robotic intelligence principles and rules. Second, the term swarm systems have the potential to offer simple, dynamic solu- generalizes the common characteristics exhibited by a flock, a arXiv:1912.07796v4 [cs.RO] 30 Apr 2020 I tions to complex problems. Swarms are capable of emulating herd, a school of fish, and other biologically or behaviorally and augmenting multi-agent dynamic systems to complete inspired terminologies used in the literature. intricate, objective-based behaviors with relatively simple local Many applications for multi-robotic teams controlled via agents [1]. This manifestation of intelligent behavior without shepherding have been discussed, covering a wide selection external regulation is described as an emergent property [2]. of domains. Of interest, these include the use of robotic Swarm robotic systems permit decentralization of the control agents to herd sheep [11], [12]; crowd control [13]; oil spill cleanups [14]; protecting aircraft from bird strikes [15]–[17]; N. K. Long is with the School of Engineering and Information Technology, disaster relief and rescue operations [18]; fishing [14]; manag- University of New South Wales, Canberra, ACT 2612, as well as the Maritime Division of the Australian Defence Science and Technology (DST) Group, ing wildlife distribution and protection [19]; and the handling VIC 3207, Australia. e-mail: [email protected]. of micro-organisms [20]. Furthermore, shepherding has been K. Sammut is with the College of Science and Engineering at Flinders examined for security and military procedures, such as un- University, Tonsley, SA 5042, and with DST Group Maritime Division, SA 5111, Australia. inhabited vehicle (UxV) maneuvering in combat [21], [22], M. Garratt, and H. Abbass are with the School of Engineering and searching terrains for intruders [18], and mine collection and Information Technology, University of New South Wales, Canberra, ACT distribution [14]. 2612, Australia. D. Sgarioto is in the Maritime Division of DST Group, Port Melbourne, This review paper covers the majority of shepherding litera- VIC 3207, Australia. ture, as a swarm control method, to date, with the objective of 2 using previous research to fortify the notion that shepherding • Driving: Once fetching is complete and the flock are can act as simple, yet effective, means for controlling a wide driven around the handler, the sheepdog is expected to variety of swarm robotic systems in the future. drive the flock away from the handler towards the first The survey is organized as follows. Section II gives a drive gates. brief overview of real techniques and assessments of sheepdog • Shedding: When the flock passes through the second shepherding methods. This is used to build a base knowledge drive gates, the sheepdog is expected to turn the sheep of the biological origins of technical shepherding. Section towards the shedding ring, where the handler and the III outlines the early studies on shepherding as a swarm sheepdog sort and separate the flock into a specified control technique which laid the foundations for the research number of sheep. that followed. Section IV details the classic shepherd herding • Penning: The flock is driven into a small enclosure, sheep inspired studies. Section V describes the specific control where the handler is responsible for closing the gate once techniques used for shepherding in more detail. Section VI all sheep are inside. illustrates alternative behaviors used in shepherding, while • Singling: Similar to, but more challenging than, shedding, Section VII highlights the shepherding studies into actually where a specific single sheep needs to be separated from using UxVs and robots, including those which involve animal the flock and driven away. shepherding. Section VIII describes the research into human- • Gathering/Collecting: The collecting behaviour is when shepherd interactions, and Section IX showcases how machine the sheepdog brings the flock to the handler. learning and computational intelligence techniques have been Another list of sheepdog commands were defined by Ben- applied to shepherding research. Finally, Section X articulates nett and Trafankowski [24] below, where each command some of the research challenges facing shepherding swarm represents an expected behaviour from the sheepdog. control. • Stay: Dog halts and remains still. II. SHEEPDOG TRIALS • Lie Down: Dog lies down. The International Sheep Dog Society’s Rules for Trials [23] • Easy, Steady, or Take Time: The dog moves slowly. are designed to measure the ability of a sheepdog to perform • Walk-Up: The dog approaches the nearest sheep and the basic skills required for shepherding. From a modeling, stops at a distance sufficiently large enough to not exert machine learning and artificial intelligence (AI) perspective, any force on the sheep to flee. these rules offer four main advantages. The first advantage • Look Back: The dog leaves the current group of sheep is the ground work that has evolved over many centuries and searches for others. in this domain, which decomposes the shepherding problem • Get Back or Get Out: The dog moves back away from into the basic sub-skills required for shepherding. The second the flock. advantage is in the possible use of the scenarios used in • That will do: The dog returns to the handler. these competitions as test benchmarks for shepherding. The • Come-bye: The dog circles the flock in a clock-wise third advantage lies in the nature of these competitions, where manner. the sheepdog needs to interact with a handler (a human) to • Away to Me: The dog circles the flock in a counter- shepherd the sheep. Replacing a biological sheepdog with a clock-wise manner. smart autonomous system that acts as a smart artificial sheep- The above commands could be seen as the basic skills dog would mean that the scenarios used in the competition required by a sheepdog to herd. The higher order planning is could also be used for testing the efficacy of the human-AI conducted by the handler, who then decides which command interaction. Last, but not least, the rules include a point-based to issue to the sheepdog. As such, the sheepdog could be seen system to assess the dogs which could be useful for the design to exhibit a basic level of autonomy once given a particular of reward functions for an AI to shepherd. command, while the handler is the smarter entity that decides The rules list and test for the basic skills required by a which command to issue and when.