Flock Heterogeneity and Its Applications

Flock Heterogeneity and its Applications

Geoﬀ Nagy

M.Sc., University of Manitoba, 2016 B.Sc., University of Manitoba, 2013

Thesis Submitted in Partial Fulﬁllment of the Requirements for the Degree of Doctor of Philosophy

in the School of Computing Science Faculty of Applied Science

Copyright in this work rests with the author. Please ensure that any reproduction or re-use is done in accordance with the relevant national copyright legislation. Approval

Name: Geoﬀ Nagy

Degree: Doctor of Philosophy (Computer Science)

Title: Flock Heterogeneity and its Applications

Examining Committee: Chair: Mo Chen Assistant Professor

Richard Vaughan Senior Supervisor Professor

Richard Zhang Supervisor Professor

Hang Ma Supervisor Assistant Professor

Nick Sumner Internal Examiner Associate Professor Computing Science Simon Fraser University

Dylan Shell External Examiner Associate Professor Computer Science and Engineering Texas A&M University

Date Defended: August 31, 2021

ii Abstract

Although the vast majority of synthetic flocking models assume that agents in a flock are homogeneous, biologists have shown that this is not the case in real flocks. Mixed-species flocks, for example, are quite common, as are differences in interaction behaviours between agents in a flock. This heterogeneity presents a barrier to our understanding of flocking behaviour, and must be taken into consideration when developing models of such behaviour. This thesis makes the following contributions. First, it describes a software tool for generating photo-realistic images of synthetic flocks for the purpose of training a neural network to learn individual-level attributes such as bird species, position, depth, and flapping phase. Although a quantitative evaluation of this tool is not available at the time of writing, qualitative analysis shows that the output from this tool shares many features in common with photos of real flocks. The second contribution of this thesis is to describe and evaluate the performance of an engineered flocking controller that exploits the pairwise flocking phenomenon—the tendency for birds of certain species to flock in stable monogamous pairs. Advantages of this engineered controller include more stable flock formations that are less likely to break up in the presence of obstacles, as well as a reduced number of tracking interactions overall between agents. This controller is evaluated using both simulated and real drones (on a small team of novel low-cost drones called the µBee). The aforementioned advantages are found to be present in both settings at multiple scales. These results show that an engineered pairwise flocking controller may have useful real-world applications in settings where flock stability or limited computational resources available for tracking flock mates are important factors. Keywords: biologically-inspired robotics, drones, flocking

iii Dedication

This thesis is dedicated to my father, Steve Nagy. Thanks, Dad, for always being there and encouraging me to achieve great things.

iv Acknowledgements

Acknowledgements are hard to write—inevitably, I’ll end up forgetting someone! If your name is not listed below, and you believe that it should be, you can reach me at [email protected]. But since that email address will likely be rudely yanked out from underneath me by the time I’m no longer a student, I wouldn’t hold my breath. First and foremost, I would like to thank my advisor, Richard Vaughan. I will miss our travels, particularly those in the UK. I fondly remember repeated instances of let’s try this pub on nights where you, Sepehr, and I were already stumbling back to the Nightingale House. I’ve learned a lot from you and I am immensely grateful for your guidance over these past ﬁve years. By some strange twist of fate, it might be possible that you’ll still be my boss somewhere after I graduate, but that’s probably not very likely. Wait a second... Also, a very big thank you to both Richard Zhang and Hang Ma for supervising my thesis progress, Nick Sumner for his role as internal examiner, and Dylan Shell as my external examiner. Thank you all for taking the time to support me. My graduate student colleagues (including those long-graduated), too numerous to men- tion here, are also owed signiﬁcant gratitude. Thank you very much for comments on early drafts of my papers, NSERC applications, and other work. However, some of you I am grateful I will never travel with again, and you know who you are. On a serious note, to Suzie, I owe immeasurable gratitude. Everything I do becomes easier with your love and support. Thank you for your endless encouragement and for always being there for me. It means the world to me. Lastly, thank you to my family and friends for putting up with my anti-social behaviours during my graduate school career. My studies actually had nothing to do with that, so you can expect more of the same.

v Table of Contents

Approval ii

Abstract iii

Dedication iv

Acknowledgements v

Table of Contents vi

List of Tables ix

List of Figures x

1 Introduction 1 1.1 Flocks and Heterogeneity ...... 1 1.2 Studying Flocks with Computer Vision ...... 2 1.3 Pairwise Flocking ...... 3 1.4 Thesis Structure ...... 5

2 Previous Work 6 2.1 Introduction ...... 6 2.2 Filming Flocks ...... 6 2.3 Biologically-Inspired Algorithms and Robots ...... 9 2.4 Flocking Algorithms ...... 12 2.5 Conclusion ...... 13

3 The BirdGen Tool 15 3.1 Introduction ...... 15 3.2 Motivation ...... 16 3.3 Design ...... 18 3.3.1 Virtual Environment ...... 18 3.3.2 3D Models and Flocking Characteristics ...... 18 3.3.3 Flocking Controller ...... 20

vi 3.4 Output ...... 22 3.5 Comparison Against Real Flocks ...... 24 3.5.1 Visual Comparison ...... 24 3.5.2 Training Performance on a Neural Network ...... 26 3.6 Future Work and Conclusion ...... 27

4 Pairwise Flocking Simulations 28 4.1 Introduction ...... 28 4.2 Flocking Controller ...... 28 4.3 Evaluation ...... 29 4.3.1 Simulator and Environment ...... 29 4.3.2 Experiments and Performance Criteria ...... 31 4.4 Results ...... 33 4.5 Discussion ...... 35 4.5.1 Applications and Limitations ...... 37 4.6 Future Work and Conclusion ...... 39

5 The µBee Drone 40 5.1 Introduction ...... 40 5.2 Main Speciﬁcations and Development ...... 42 5.2.1 3D Printing the µBee Frame ...... 42 5.2.2 Electronic Components and Hardware ...... 42 5.3 Support for Motion Capture Systems ...... 44 5.4 Flight Control Loop ...... 46 5.5 Flight Panel Application ...... 50 5.6 Wireless Protocol ...... 50 5.7 Future Work and Conclusion ...... 52

6 Pairwise Flocking with the µBee 53 6.1 Introduction ...... 53 6.2 Drone Command Center ...... 54 6.2.1 Interface and Modules ...... 54 6.2.2 Simulation Accuracy ...... 56 6.3 Veriﬁcation of Pairwise Flocking Correctness ...... 58 6.3.1 Pairwise Flocking Controller Port ...... 58 6.3.2 Results ...... 59 6.4 Physical µBee Experiments ...... 60 6.4.1 Flying Multiple µBees ...... 60 6.4.2 Experiment Setup ...... 64 6.4.3 Experiment #1 ...... 66

vii 6.4.4 Experiment #2 ...... 68 6.4.5 Overall Discussion ...... 71 6.4.6 Limitations and Future Work ...... 72 6.5 Evalution of the µBee ...... 72 6.6 Conclusion ...... 73

7 Conclusion 75

Bibliography 77

Appendix A Code 83

viii List of Tables

Table 3.1 Individual bird parameters used by BirdGen...... 20

Table 4.1 Experiment #1 numbers of paired and unpaired agents...... 32

Table 5.1 Relevant 3D printing parameters used in Cura 4.8.0 to slice the µBee frame for export to an AnyCubic Mega S 3D printer...... 43 Table 5.2 The µBee PCB’s fabrication parameters...... 46 Table 5.3 Message types defined by the µBee wireless communications protocol. 51 Table 5.4 Flight parameters used to tune the µBee’s flight, by flight parameter group. All values are 4-byte floating-point...... 52

Table 6.1 Breakdown of the number of trials for the experiments used to verify the correctness of the pairwise flocking behaviour implemented in the DCC...... 59 Table 6.2 Average number of sub-flocks at the conclusion of each trial...... 69 Table 6.3 Average number of tracking connections during all trials, grouped by configuration...... 70 Table 6.4 Average length of each trial, grouped by configuration...... 70

ix List of Figures

Figure 1.1 The main forces involved in a classical Boids controller. Adapted from http://www.red3d.com/cwr/boids/...... 2 Figure 1.2 Paired agents A and B using the pairwise flocking controller. Agent A tracks the neighbours on A’s side of the formation, plus Agent B. Agent B tracks agents on the opposite side, plus Agent A. Both agents travel closer together than other non-paired agents...... 4 Figure 1.3 High-resolution photographs of corvid flocks by Jolles et al. [26] reveal that such flocks can be partially composed of pairs...... 4

Figure 2.1 Top-down illustration of the 3-camera setup used by Kelly et al. [28]. 7 Figure 2.2 Pairs are present in this ﬂock of corvids. Used with permission from Guillam McIvor...... 8 Figure 2.3 The main forces involved in a classical Boids controller. Adapted from http://www.red3d.com/cwr/boids/...... 12

Figure 3.1 Sample output image generated by BirdGen. Ground truth 3D poses of all simulated birds are known and can be used for training a neural network...... 15 Figure 3.2 Representation of a 4-camera setup for imaging flocks of birds. . . . 16 Figure 3.3 The sketches provided by Guillam McIvor and the resulting 3D models and rigging...... 19 Figure 3.4 Sample output image generated by BirdGen...... 21 Figure 3.5 A photograph of a real flock of jackdaws. Note that BirdGen does not render images of trees...... 21 Figure 3.6 The 12 channels of PNG output generated by a single camera-frame. Best viewed on a computer screen or high-quality print-out. . . . . 25 Figure 3.7 Left column: Images generated by compositing successive photographs of corvid flocks (used with permission from Hangjian Ling). Right column: Images generated by BirdGen without clearing the OpenGL framebuffer, resulting in a similar effect...... 26

Figure 4.1 Agents A and B track mutually-exclusive sets of neighbours on their respective sides of the pair formation...... 30

x Figure 4.2 A screenshot of the pairwise ﬂocking simulator. Colour-corrected for clarity and printer-friendliness...... 31 Figure 4.3 Agents A and B are a stable pair, and are compressed into a single node which now contains all of A’s and B’s connections...... 32 Figure 4.4 Sub-ﬂocks after every trial, for varying values of AT , CP , and pairing percentages. In all results here, the number of neighbours tracked by

single agents, ns, is 7, and the number of neighbours tracked by

paired agents, np, is 3...... 34 Figure 4.5 Snapshots from a sample run with no pairing enabled, at 0, 6, 12, and 18 seconds into the simulation. The first sub-flock exits the obstacle field in the final frame...... 35 Figure 4.6 Snapshots from a sample run with half of the agents paired, at 0, 5, 10, and 15 seconds into the simulation. The flock exits the obstacle field in the final frame...... 35

Figure 4.7 The average number of sub-ﬂocks over time for all trials where ns = 7

and np = 3. Error bars denote 95% confidence intervals...... 36 Figure 4.8 Average flock diameter of the (a) original flock and (b) the compressed flock, normalized with respect to the number of sub-flocks. Error bars denote 95% confidence intervals. These values are relatively unchanged over time since they are normalized with respect to the number of sub-flocks; thus, they are an indicator that flock diameter in general is dependent on the number of paired agents. . 37 Figure 4.9 The difference in average degree of the compressed and original flock

graphs, where ns = 7 and np = 3. With the obvious exception of the case with 0% pairing, mean differences at each time step were found to be statistically significant at p < 0.05 using an independent t-test. Error bars denote 95% confidence intervals for the difference of means. 38 Figure 4.10 The average number of sub-flocks at the conclusion of every trial,

where np = 3 and ns varies. Error bars denote 95% conﬁdence intervals. 39

Figure 5.1 Digital render and photograph of the µBee drone...... 40 Figure 5.2 Schematic of the µBee PCB...... 44 Figure 5.3 The top and bottom layers of the µBee PCB. Having only 2 layers helped to keep costs low. The PCB measures 35.5 mm × 46.5 mm. 45 Figure 5.4 Marker pins are snapped into place from underneath the attachment. Reﬂective markers are screwed onto the pins. The attachment can then be mounted onto the µBee...... 47 Figure 5.5 Digital render and photograph of the µBee drone with markers attached...... 48

xi Figure 5.6 Interactive tool used to place marker mounting holes on the µBee attachment...... 49 Figure 5.7 The µBee Flight Panel application for conﬁguring ﬂight parameters. 51

Figure 6.1 4 µBee drones flying in a Vicon motion capture-equipped workspace. 53 Figure 6.2 Organizational structure of the Drone Command Center tool. . . . 54 Figure 6.3 3D flight paths and velocities of the box flight test for (a) a simulated µBee and (b) a real µBee...... 57 Figure 6.4 Average number of sub-flocks present at the conclusion of each trial with 0, 5, 11, 17, 22 paired agents. Error bars denote 95% confidence intervals...... 60 Figure 6.5 Average number of total connections between agents at the conclusion of each trial with 0, 5, 11, 17, 22 paired agents. Error bars denote 95% confidence intervals...... 61 Figure 6.6 Number of sub-flocks after every trial, for varying values of AT , CP , and number of paired agents...... 62 Figure 6.7 10 µBee drones sitting in their handy 3D-printed carrying case. . . 63 Figure 6.8 Shows the new triangular marker configuration and how position and orientation vectors are computed. A clustering algorithm finds groups of 3 markers whose pairwise edge lengths fall into defined ranges. The average position of markers A, B, and C yield the center position of the drone. The left vector is computed as the unit vector from marker C to B. The forward vector is computed as the unit vector between the average marker position and marker A. The up vector comes out of the page and is defined as the cross product of the left and forward vectors...... 64 Figure 6.9 Visual outline of the flow of detected marker data in the DCC, including both real and simulated markers...... 65 Figure 6.10 The drone detection quality map displayed in the DCC...... 65 Figure 6.11 A Galton board. Svjo, CC BY-SA 3.0, https://creativecommons. org/licenses/by-sa/3.0, via Wikimedia Commons ...... 66 Figure 6.12 Placement of the drones and obstacles in the 11 × 5 meter Vicon space...... 67 Figure 6.13 Timelapse of a run with no drones paired, at 0, 10, 20, and 30 seconds into the trial. Note that a single drone has escaped the flock. Single drones departing the flock was typical for this configuration. . . . . 67 Figure 6.14 Timelapse of a run with 2 drones paired, at 0, 10, 20, and 30 seconds into the trial...... 68

xii Figure 6.15 Histogram of the number of sub-flocks present at the end of each trial, for non-paired and paired configurations...... 69 Figure 6.16 Histogram of the number of sub-flocks present at the end of each trial, for non-paired, paired, and paired catch-up configurations. . . 70

xiii Chapter 1

Introduction

1.1 Flocks and Heterogeneity

Displays of collective animal behaviour such as flocking, schooling, and herding, have fasci- nated humans for centuries. Variations of these phenomena exist throughout nature but no complete explanation for such behaviour exists. As late as the 1930s, scientists even turned to the supernatural to explain them [63]. The first feasible generative model of collective group motion came from computer graphics designer Craig Reynolds in 1987 [55]. He demonstrated that the displays of collective behaviour we observe in nature could be imitated by virtual agents following three simple interaction rules: attraction to nearby agents, repulsion from agents that are too close, and velocity alignment with nearby agents. (Agents that fall within these ranges of influence will be referred to as neighbours, with respect to some focal agent.) The net effect of these rules is to create behaviours and interactions that resemble the flocking behaviours we see in nature. Unfortunately, it is currently unknown if these flocking rules are truly those that are employed by birds or other animals that exhibit flocking behaviour; it is possible that they use entirely different sets of rules that yield a visually similar result. In fact, we can be reasonably certain that most existing flocking algorithms do not accurately represent the behaviours of real flocks in at least one respect: the vast majority of flocking algorithms (with some exceptions [21, 44]) make the assumption that all agents in a flock are homogeneous, despite biologists having previously shown that this is not always the case. For example, it has been shown that bird flocks can be composed of more than one species [26], such as the mixed flocks of jackdaws (Corvus monedula) and rooks (Corvus frugilegus). Additionally, these two species of birds will preferentially flock alongside a mate [26, 34]. These pairs are believed to be stable monogamous pair-bonds, and this phenomenon will be referred to as pairwise flocking. This is one example of flock heterogeneity, where it is clear that the assumption that all agents follow an identical set of interaction rules is false. Thus, in order to develop a more comprehensive understanding of flocking behaviours, we must take flock

1 (a) Separation steers agents (b) Alignment guides agents (c) Cohesion moves agents to- away from collisions. towards average local head- wards average local position. ing.

Figure 1.1: The main forces involved in a classical Boids controller. Adapted from http: //www.red3d.com/cwr/boids/. heterogeneity into account. However, flock heterogeneity has been little-studied and this thesis aims to make a contribution in this area. Throughout this thesis, the term flocking will be used to describe any of these collective group behaviours commonly referred to as flocking, schooling, or herding. While these terms carry slightly different meanings1, for the purpose of this thesis the term “flocking” will be used to refer to the general phenomena of collective group motion. Participants in these groups will be generally referred to as agents. There are two main related themes to this thesis. The first is that of studying flocks of birds using stereomatching computer vision techniques, which is discussed in Section 1.2. The second theme is the pairwise flocking phenomenon and attempts to engineer this behaviour using real and simulated drones, and this is discussed in Section 1.3.

1.2 Studying Flocks with Computer Vision

One way of improving our understanding of flocking behaviours is through statistical analysis of naturally-occurring flocks, made possible by image stereomatching techniques. Image stereomatching—or simply stereomatching—is the simultaneous use of multiple points of view to compute the distance and angle to an object. An intuitive example of this is our use of both eyes to determine how far away an object is. Using this approach, an array of cameras captures a group of animals, and stereomatching algorithms are used to determine the position of each agent over time in 3D space. In general, this is done by projecting each detected individual from each camera as a line into 3D space or onto the planes of the other cameras; where lines intersect (or are close to intersecting) an object is assumed to exist [14, 37]. A camera calibration process is also required. This technique has been successfully applied to flocks of birds [4, 5, 7, 34, 33, 35], swarms of insects [28, 45], and analysis of fluid dynamics [37, 47]. Related non-stereomatching techniques have been used

1A school, for example, implies a group of ﬁsh while a ﬂock implies birds, etc.

2 for fish [38, 1, 51] and horses [23]. The type of animal being studied dictates the exact configuration used: small tanks can be used to study insects or fish, but flocks of birds present an additional challenge because of their size and speed. For this, outdoor arrays of cameras are used to capture a larger volume. As discussed in Chapter 3, the stereomatching approach has several limitations, the main one being that only positional information is obtained. Other information, such as an agent’s species, is not. The lack of individual-level species information places restrictions on what we can learn. This is a critical limitation of current approaches since it is known that flocks can contain more than one species, and flock heterogeneity in general is known to affect the behaviour of the flock [26]. Deep neural networks, however, have been shown to be highly capable of performing species discrimination tasks when provided with 2D RGB images [53, 32]. When used in conjunction with stereomatching techniques, it may be possible to detect each bird’s position as well as its species, or any other information that is relevant to the analysis, if the network has been trained to learn it. Training such a network would require a larger amount of hand-annotated data than is practical, given the sheer number of birds that may be present in a single camera frame. Therefore, the first part of this thesis focuses on the theme of generating large quantities of synthetic flock data using computer software, for the purpose of training a deep network to recognize relevant flock characteristics at the individual level. Chapter 3 will introduce this data generation tool and discuss its development and output.

1.3 Pairwise Flocking

The second part of this thesis concerns the pairwise flocking phenomenon described in Section 1.1. It has provided the inspiration for a pairwise flocking controller that has been developed for teams of drones. As mentioned previously, agents in a traditional Boids flock track a number of nearby agents (neighbours) in order to maintain the flock and their position within it. The core idea of the pairwise flocking controller is to mutually pair some of these agents. Agents in such pairs travel closer together, and each agent in a pair treats their partner as a proxy for other nearby agents, without the need for those agents to be tracked explicitly as they would be using a traditional Boids controller. Thus, each agent in a pair is responsible only for tracking neighbours alongside each half of the pair formation, plus their partner. This essentially splits the neighbour tracking load evenly between them. This concept of an engineered pairwise flocking controller is a novel contribution of this thesis, and is illustrated in Figure 1.2. Using fleets of both real and simulated drones, this thesis will show how this controller provides several advantages to a group of agents. These advantages include greater stability in the face of obstacles designed to break up the flock, and a simultaneous reduction of the number of tracking interactions between members overall. It is important to note that

3 B A

Figure 1.2: Paired agents A and B using the pairwise flocking controller. Agent A tracks the neighbours on A’s side of the formation, plus Agent B. Agent B tracks agents on the opposite side, plus Agent A. Both agents travel closer together than other non-paired agents. these advantages are known to be present only in this context; there is no evidence to support the possibility that real flocks experience these same advantages by flocking in pairs. Additionally, the motivation for pairwise flocking in nature is currently unknown2.

Figure 1.3: High-resolution photographs of corvid ﬂocks by Jolles et al. [26] reveal that such ﬂocks can be partially composed of pairs.

2Personal correspondence with an expert on bird ﬂocks has suggested that pairwise ﬂocking exists as a mechanism to protect against the high cost of losing one’s mate.

4 1.4 Thesis Structure

This thesis will take the following structure. Chapter 2 will present previous work as it relates to the topic of flocking and, more generally, biologically-inspired algorithms and robots. Chapter 3 will discuss a software tool called BirdGen, which generates mass quantities of synthetic bird flocking images for the purpose of training a deep neural network to recognize birds, their species, and other properties at a distance. In Chapter 4 an engineered pairwise flocking controller will be presented, as well as simulation results showing how pairwise flocking can be advantageous when employed in fleets of drones. Chapter 5 will introduce a novel, low-cost research drone, the µBee, that has been used to further validate findings with regards to pairwise flocking on a real drone fleet. The µBee pairwise flocking experiments themselves and their results are discussed in Chapter 6. Concluding remarks are made in Chapter 7.

5 Chapter 2

Previous Work

2.1 Introduction

In order to provide proper context for the work described in this thesis, it is helpful to describe some related work upon which it builds. As discussed in Chapter 1, this thesis is focused on two themes: (1 ) studying flocks using stereographic imagery, and (2 ) the development and evaluation of a pairwise flocking controller using real and simulated drones. To provide the reader with the relevant background, this chapter will first cover advances in filming and analyzing collective animal behaviour (not only that of birds, but of fish and insects as well) using a variety of techniques. Section 2.3 will discuss a wide range of biologically-inspired robotic behaviours and algorithms, with a special focus on flocking algorithms later in the chapter (Section 2.4). Concluding remarks are made in Section 2.5.

2.2 Filming Flocks

Faria et al. [16] used a single-camera setup to measure the ability of a robotic fish to recruit single individuals, and ten-fish shoals, from a refuge area of a water tank. The water tank measured 86 cm wide × 18 cm long × 5 cm deep and the camera was mounted on top of this tank, facing downwards. Fish positions were digitally labelled using computer software, allowing for statistical analysis. The authors determined that a robotic fish was able to recruit single individuals, but not an entire shoal, to the refuge area. Marras et al. [38] used a similar setup with two cameras—one underneath, and one above—to determine under what conditions a real fish would be enticed to swimming near a robotic one. In experiments studying live zebrafish attraction to robotic conspecifics, Abaid et al. [1] and Polverino et al. [51] used a similar setup with a single top-mounted downwards-facing camera. Jiang et al. [25] similarly used a single camera above a water tank to examine groups of 2 and 5 rummy-nose tetras when executing collective U-turns. They observed that fish were influenced by 1-2 nearest neighbours (typically in front of the focal fish), although

6 these neighbours often changed according to ﬁsh preference. There were also periods where no interaction partners were considered.

Figure 2.1: Top-down illustration of the 3-camera setup used by Kelly et al. [28].

Swarms of insects are easily studied because of their small size and limited space requirements. Kelly et al. [28] described a laboratory setup containing a swarm of midges surrounded by cameras on three of the four sides of a 91 cm cubical enclosure (shown in Figure 2.1). Using stereoscopic algorithms, they were able to extract the 3D positions of the midges in the enclosure and perform statistical analysis on the swarm’s shape and behaviour. The authors determined that the midge swarm, in addition to experiencing a central force that bound the swarm together, was made up of several different “sub-swarms” that exhibited coordinated motions not correlated with the motions of other sub-swarms. Additional work by Ni et al. [45] showed how midge swarms exhibit elastic properties: when two attractive ground plates (“swarm markers”) were moved away from each other from underneath a swarm, the midges gradually split into two sub-swarms that maintained stable positions over each marker, while also experiencing an attractive force to each opposite sub-swarm. Jolles et al. [26] were the first to provide empirical evidence for the existence of pairs in corvid flocks. This was determined by photographing flocks of corvids from the ground, and analyzing the photos in an image manipulation program. Although their work did not consider 3-dimensional bird data, further studies involving the use of ground-based arrays of cameras confirmed the existence of stable pairs in corvid flocks [34]. Additional work [33] has

7 shown that paired birds in such flocks are subject to “spring-like” forces that keep the pair together, and that birds in a pair likely exert considerable cognitive effort in order to track one another. The authors claim that this indicates a potential conflict of interest within members of such flocks, in which paired members desire to maintain social bonds at the cost of reduced sensitivity to the environment as a whole. This contrasts with the engineered pairwise flocking controller that will be discussed later in this thesis, which assumes that the cognitive load associated with tracking a partner is identical to that of tracking any other agent. Ling et al. [35] further demonstrate that the attractive forces associated with corvid flocks are highly anisotropic, with birds experiencing attraction more strongly in a direction perpendicular to that of the bird’s motion. They also showed that repulsion between agents is effected by sideways turns, rather than changes in speed. The authors argue that this is most likely due to the physics and locomotion constraints imposed on birds in flight, and that future models of flocking behaviour should take these factors into account for greater accuracy.

Figure 2.2: Pairs are present in this ﬂock of corvids. Used with permission from Guillam McIvor.

Earlier research focusing on the flocking behaviour of European starlings used three cameras mounted approximately 30m above the ground to image the flocks [4]. Similar to the work on fish by Jiang et al. [25], the authors demonstrated that the social forces governing flocking behaviours were determined by topological—rather than Euclidean—distance. That is, rather than considering all agents within some metric distance, the starlings only

8 considered the nearest 6-7 agents to maintain a position within the flock. They further demonstrated, in simulation, that the use of such topological distance neighbourhoods resulted in a flock that was less likely to break up into smaller sub-flocks in the face of repulsive obstacles or active predators, when compared to the use of Euclidean distance neighbourhoods. This concept of “flock stability”, along with the use of computer simulations to test the stability of virtual flocks of agents, are core elements of this thesis and are discussed further in Chapters 4 and 6. Additional work by the same research group has shown that computer models of spatially-balanced interaction neighbourhoods—i.e., the selection of interaction partners in a uniformly radially-distributed manner—yields optimal cohesion and improved flock stability [7]. The ability to gather and analyze data from real flocks is a key requirement to being able to understand flocking behaviours that are observed in nature. A limitation of these approaches, however, is that they provide positional data only. It is known, for example, that flocks of real birds can contain more than one species, and this heterogeneity cannot be ignored if we are to fully understand the interaction rules governing flock behaviour. Chapter 3 discusses these issues in greater detail and presents a tool intended to help improve the quality and type of flocking data that can be obtained after recording real flocks in the field.

2.3 Biologically-Inspired Algorithms and Robots

A key element of this thesis is the development and evaluation of a biologically-inspired controller for drone behaviour, i.e., the pairwise flocking controller initially described in Chapter 1. To provide useful context, this section will describe previous work as it relates to biologically-inspired algorithms and robots. Flocking algorithms, specifically, feature promi- nently in this thesis and are discussed in their own section (2.4). The path-optimization ability of ants has provided perhaps one of the best-known ex- amples of biological inspiration for algorithms. When foraging for food, ants will release small amounts of a slowly-evaporating chemical pheromone that can be detected by other ants [17]. As paths to food become more travelled, these paths accumulate more pheromone and are seen as more attractive by other ants, encouraging them to take that path. Since shorter paths accumulate pheromone faster than longer ones, the ants will eventually con- verge to the shortest path. This is an example of stigmergy, where agents communicate indirectly by modifying a shared environment [18], and it has provided inspiration for numerous applications. Dorigo [11] demonstrated how simulated pheromone-depositing behaviour could be used to find optimal or near-optimal solutions to instances of the Travelling Salesman problem1.

1Although this work is written in Italian, a full description can be found in [12].

9 His algorithm works by treating cities as vertices on a graph, with edges representing connections between them; virtual ants travel the network and lay “pheromone” on the edges that they travel. Much like in the real world, the virtual pheromone loses potency over time, and after several iterations the shortest paths eventually contain more pheromone. Due to the time required to find solutions for problem sizes larger than approximately 30 cities, Dorigo et al. developed an improved version to address performance issues [13]. Schoonderwoerd et al. [62] developed an ant-inspired approach for balancing loads on a telecommunications network, and has the property that it can be adjusted to load-balance dynamically-changing networks rather than only static ones. Vaughan et al. [68] demonstrated a physical implementation of the pheromone-laying behaviour using several Pioneer robots tasked with foraging for mock resources inside a building. Pheromone was simulated in the form of virtual “breadcrumbs” that were com- municated over a wireless network, indicating the heading and rough distance to a certain goal location. A unique feature of this work is that the robots relied heavily on odometry integration for localization, which was reset upon reaching a known goal location. To address the problem of robots colliding when travelling in opposite directions on the same path, Sadat et al. [60] created a modified version of this algorithm which encouraged agents to modify their trajectory to the left side of the interfering trail if a breadcrumb was encoun- tered that pointed towards a different goal location. Further improvements are described by Song et al. [64], in which agents are able to untangle intersecting trails by gradually moving the longer ones around the shorter ones. A final improvement by Abdelaal et al. [2] allows agents to choose one of two “lanes” in which to travel for any given direction, greatly increasing the throughput of the system. Notably, this is similar to how real ants form travel lanes when transporting resources [10], although in Abdelaal et al.’s work the number of lanes is fixed. Phototaxis, the attraction to sources of light, is another phenomenon frequently seen in nature that has provided inspiration for robotic or algorithmic behaviour. The very first example of robotic phototaxis was demonstrated in 1950 by Grey Walter [69], in which mechanical “tortoises” Elmer and Elsie were equipped with light sensors and an analog mechanical design that caused the two robots to drive towards light sources. Omnidirectional force sensors allowed the robots to navigate around obstacles. Kube and Bonabeau [30] developed a robotic behaviour that allowed ant-inspired cooperative transport of a brightly- lit box to a brightly-lit goal position, demonstrating how phototaxis could be used to perform useful tasks2. Their approach was made even more effective by the addition of individual- level, random adjustments to the push angle that robots would make, similar to how real ants make small adjustments until they are able to collectively move their prey [66]. Rubenstein

2Note that Stilwell and Bay [65] had previously demonstrated successful cooperative transport in simulation, but phototaxis was not used.

10 et al. [59] also demonstrated how a group of kilobots [58], equipped with rings to enable a rudimentary form of grasping, could be used to cooperatively push a larger object towards a light source. They also validated their work using teams of r-one robots [39] that were lined with Velcro3 to enable grasping. Bignell et al. [6] demonstrated in simulation how a swarm of small phototactic robots could be used to push small boxes into several different 2D geographical shapes such as circles, squares, and five-pointed stars. Their approach involved the use of a programmable light field that served to both power the robots and influence their motion. Another source of inspiration for robotic behaviours has been the synchronization abili- ties exhibited by groups of insects, such as the chirping of crickets or synchronized flashes of swarms of fireflies. Christensen et al. [9] demonstrated that a group of s-bots [42] could synchronize their LED flashes and treat a lack of synchronized flashes as an indicator that one or more robots had become non-operational. Robots whose LEDs did not flash were considered to have suffered a failure. The authors note that the use of omnidirectional cameras and line-of-sight provides directional information about the locations of non-functional robots, and the team’s ability to detect those robots was dependent on their prior level of synchronization. An additional study by Perez-Diaz et al. [50] investigated the effects of robot speed, field of view, and line of sight on the ability of a robot team to similarly synchronize their LED flashes. After performing experiments in the ENKI robot simulator [36], they concluded that synchronization was possible in a wide variety of environments (including ones containing obstacles), but less occlusion, combined with greater robot fields-of-view, was preferable. Purely physical phenomena—as opposed to animal-driven behaviours—have also provided roboticists with inspiration. The Brazil Nut Effect is an excellent example, in which shaking a box of mixed nuts tends to leave the smaller pieces at the bottom and the larger ones on top. This particular phenomenon (no pun intended) is explained by Rosato [57] as follows: due to the fact that the smaller particles require less space to travel to the bottom of the container, they subsequently fill voids that the larger particles cannot occupy. Groß et al. [19] exploit this effect to sort swarms of robots in simulation into both top-down and circular formations, where robots occupy positions determined by their assigned “rank” (a property analogous to particle size). This can be achieved without any inter-robot knowledge of another robot’s rank: rather, a robot’s rank determines the distance it prefers to maintain between itself and other robots. Combining a variety of virtual forces (repulsion from nearby robots, attraction to a shared global origin or direction, and randomized motion mimicking the shaking of a box of nuts) results in the robots sorting themselves by rank within the desired formation. Chen et al. [8] validated this work by porting the robot

3Another biologically-inspired technology, although not robotic.

11 (a) Separation steers agents (b) Alignment guides agents (c) Cohesion moves agents to- away from collisions. towards average local head- wards average local position. ing.

Figure 2.3: The main forces involved in a classical Boids controller. Adapted from http: //www.red3d.com/cwr/boids/. logic to the physical e-puck platform [41], and demonstrated that it works on a swarm of up to 20 robots.

2.4 Flocking Algorithms

Craig Reynolds is credited with developing the first algorithmic implementation of virtual flocks [55]. His algorithm (named Boids) works by applying a combination of three social forces to each agent in the virtual flock: (a) repulsion away from agents that are too close, (b) velocity alignment with nearby agents, and (c) attraction to the center of other nearby agents. Combining these forces results in behaviour that resembles the motions seen in real flocks of birds. Unlike the topological neighbourhoods used by real birds [4] or fish [25] to select interaction partners, Boids use an Euclidean neighbourhood within which all other agents are considered.4 These forces are depicted in Figure 2.3. Additional work by Reynolds builds on these three social forces to include behaviours such as path finding, obstacle avoidance, pursuit and evasion, and more [56]. OpenSteer, an open-source implementation of these behaviours, is maintained at https://github.com/meshula/OpenSteer. Reynolds notes that a naïve implementation of his algorithm is O(N 2); a single simulation step can be very costly in terms of performance if the number of agents is very large. To address this issue, Reynolds presented an updated, multi-processor implementation of his original algorithm with spatial partitioning [54]. In this work, the environment is voxelized and each voxel is assigned to a processor that is responsible for updating the agents contained within it. An additional improvement called skipThink yields further performance gains, where groups of agents take turns updating their steering vectors on a frame-by-frame basis. A similar approach is used by Treuille et al. [67], whereby agent steering updates occur at a lower rate than the rendering itself.

4The virtual ﬂocks presented in this thesis (speciﬁcally, in Chapters 4 and 6), however, use a combination of nearest-neighbour (topological) and Euclidean distance factors when choosing interaction partners.

12 Additional research [46] has shown how GPU multi-processing can be used to further increase the performance of the original Boids implementation. Passos et al. [49] described a spatial partitioning method that sorts agents into grid cells. Perception of other agents is limited to those in adjacent cells (i.e., their Extended Moore neighbourhood [61]) only. The authors argue that this spatial sorting is ideal given the spatial locality inherent in multi-processor algorithms. Their approach also has the property that it exploits a rough approximation of topological distance interactions, similar to how birds [4] and fish [25] interact, as mentioned previously. The authors were able to achieve a frame rate of roughly 25 frames per second while simulating over 1 million agents in this manner. They were able to achieve 10 frames per second during a simulation of 32,000 agents on a consumer tablet device [27]. Hartman and Benes [21] proposed another extension to the Boids algorithm to imitate the sudden large-scale changes of direction sometimes exhibited by real flocks. They observed that these motions can be triggered by an agent suddenly “shooting off” from the flock, causing the other flock members to follow. This is implemented as a “change of leadership” force that is a function of the agent’s position within the flock. Agents along the outer front edge of the flock are more likely to change direction and take the rest of the flock with them. More recent work has shown that such changes of direction in real flocks are likely caused by evasive maneuvers designed to avoid collisions, since it less costly to change flight direction than flight speed [35]. Previous work by the author in Nagy et al. [44] forms a key part of this thesis: inspired by observations of pairwise flocking [26], Nagy et al. introduced a flocking algorithm that incorporates several properties of real flocks, such as the topological interaction neighbourhoods seen in real flocks and the tendency for birds of certain species to fly in pairs. In this work, agents are optionally assigned a mate, analogous to real-world flock pairings, to which such agents will fly more closely. Rather than tracking 6-7 nearest neighbours [4], however, each bird in a pair tracks only 3 neighbours on the opposite side of their mate, plus the mate themselves. This reduces the number of tracking interactions of the flock overall, but also has the property of decreasing the tendency of the flock to split into multiple sub-flocks in the presence of obstacles designed to disturb the flock.

2.5 Conclusion

This chapter has provided background knowledge with respect to the two central themes of this thesis: (1 ) stereographic analysis of ﬂocks of birds, and (2 ) the development and evaluation of a pairwise ﬂocking controller for simulated and real drones. Subsequent chapters will now discuss these topics and present motivations for, and design considerations and evaluations of, novel approaches that have been developed in support of these themes. The

13 next chapter begins by discussing a specialized tool called BirdGen that has been developed to assist researchers in analyzing ﬂocks of birds that have been recorded in the ﬁeld.

14 Chapter 3

The BirdGen Tool

3.1 Introduction

Figure 3.1: Sample output image generated by BirdGen. Ground truth 3D poses of all simulated birds are known and can be used for training a neural network.

This chapter will describe a software tool for generating mass quantities of images of computer-simulated flocks, developed as part of a larger collaboration whose goal is to gain further understanding of flocking behaviours seen in nature. The output of this tool includes standard RGB images similar to those that would be captured by a typical camera array setup (depicted in Figure 3.2), as well as all relevant labels (species, depth, etc.) that a deep network should learn from those images. The overall goal of this tool is to generate large quantities of synthetic flock data of a sufficiently realistic quality such that expert-labelled real data would be necessary only in small amounts for fine-tuning, or not necessary at all.

15 3.2 Motivation

The collective, coordinated behaviour observed in many groups of animals is a subject of a great deal of research. One of the main goals of this research is to understand the interaction rules governing the group behaviours that we observe. This is often accomplished by using arrays of cameras to capture the group of interest, and then using stereo-matching algorithms [14, 37] to determine the 3D position of each agent over time for statistical analysis. This approach has been successfully applied to swarms of insects [28, 45] and flocks of birds [4, 34], and related techniques have been used to study schools of fish [38, 1, 51] and herds of horses [23]. Flocks of birds, however, present an interesting challenge because of their speed and the amount of space that they occupy. Many previous works have demonstrated success in using ground-based arrays of cameras to capture flocks of birds. The first of these involved the study of flocks of starlings [5] and further work demonstrated that starling interaction neighbourhoods are constructed using nearest-neighbour distance rather than Euclidean distance [4]—i.e., starlings interact with a fixed number of flock mates rather than all flock mates that fall within a fixed radius. Additional work [7] has shown that this behaviour results in flocks that are less likely to split up in the presence of predators or obstacles; this concept of flock stability is a core theme of this thesis and will be covered in greater detail in Chapter 4. Ling et al. [34] demonstrated a portable camera system for imaging large flocks of jackdaws and rooks, and analyzed the impact of pairwise flocking on the behaviour of these flocks [33]. Follow-on work [35] examined the anisotropy present in flocking forces, and its impact on flock behavior and the shape of the flock overall. In a typical ground-based camera array setup, multiple cameras are pointed upwards to view a common volume of space [34]. Such a setup is depicted in Figure 3.2.

Figure 3.2: Representation of a 4-camera setup for imaging ﬂocks of birds.

16 As birds fly through the volume imaged by the cameras, their 3D positions within the volume can be determined using stereo-matching algorithms [14, 37, 47, 28, 34]. In general, this is done by projecting each camera’s bird detections into 3D space; where these lines intersect (or are close to intersecting), a bird is assumed to exist. However, these techniques capture positional information only. Data such as an agent’s species, for example, is not detected. This presents a barrier to our understanding of collective behaviour since groups of animals can contain more than one type of agent [26]. The assumption that all agents in a group are homogeneous can lead us into drawing incorrect conclusions about the behaviour of such a group, or make it more difficult to determine the interaction rules that govern such behaviour. An additional downside of this technique is that it requires a lengthy (roughly 15 minute) camera calibration process [34] using a large drone with a pair of highly-visible markers attached. This calibration process is required to recover camera extrinsics, which are necessary to project 2D bird detections into 3D space. In some cases, however, it may be impractical to use such a drone, either due to local regulations or unfavourable weather conditions. Even when conditions are ideal, crashes can occur due to the added flight instability associated with the attached markers. Transporting and charging a large drone presents further, although minor, inconveniences. A possible solution is to use high-resolution GPS, an accelerometer, and a digital compass to recover the extrinsics of each camera, and recover the depth (distance from the camera) of each bird detection. This would relax the requirement that the camera array remains stationary, and also remove the need for calibration. While depth sensing is possible with sensors such as the Microsoft Kinect, these sensors do not perform well outdoors and do not work at the scales required for capturing flocks of birds. Deep learning-based approaches may hold the answer to both the species classification and depth sensing problems: previous work has shown that deep neural networks are adept at both species classification [53, 32] and depth sensing [15, 31] when only provided with RGB images as input. The downside to deep network-based approaches is the large amount of training data that is required. While it is technically possible to hand-annotate hundreds of thousands of images of flocks with individual species labels, this would take a great deal of time and must be done by an expert capable of distinguishing between bird species at extreme ranges1. Adding to this difficulty is the fact that there is no accurate way for an expert to determine a bird’s depth at these distances using only RGB images. However, both of these issues can be solved by embracing synthetic data. The BirdGen tool is designed for this purpose and is described in subsequent sections.

1Correspondence with an expert in jackdaw and rook anatomy and social behaviour has also revealed that one can make only rough guarantees when determining bird species at the individual level. Flock composition is often inferred by location or by spotting individuals that are unambiguously one species or another.

17 3.3 Design

3.3.1 Virtual Environment

At its simplest, BirdGen is a Boids-inspired [55] simulation specifically tailored to simulate the appearance and behaviour of single- and mixed-species flocks of jackdaws and rooks (see Chapter 2). Similar to real field experiments, it simulates the typical 4-camera configuration shown in Figure 3.2 and generates photo-realistic output from each. Each pair of cameras is 50 meters apart, with 8 meters of space in between each camera in a pair. The cameras point towards a common location at the center and are inclined 60 degrees from the ground plane, to give a clear view of the simulated sky. BirdGen requires the user to specify several different parameters in order to configure the data generation to suit their needs. These include (a) how many flocking sequences to generate, (b) length of each flocking sequence in frames, assuming 60 frames per second, (c) minimum and maximum ranges for the number of birds in each flock, randomly sampled from this range, and (d) a set of backdrops to use and options to simulate a moving sky.

3.3.2 3D Models and Flocking Characteristics

The desire to use synthetic data to train a deep network meant that such data needed to resemble real flocks of birds as closely as possible. Realism at both the individual level and collective behaviour level was therefore a crucial goal. A generic 3D animated bird model was acquired from an online resource [48]. With the help of an expert on rook and jackdaw behaviour and anatomy (Dr. Guillam McIvor2 from the University of Exeter), two modified versions of this model were developed: one jackdaw and one rook. Considerable feedback was taken into consideration to match the proportions and flapping characteristics of both species of bird 3. Hand-drawn sketches of each species’ bone structure and flapping motions were used as reference images; these images can be overlaid on the user’s workspace in Blender, a free 3D modelling application4, to aid in animating and modelling. Those sketches and the resulting models are shown in Figure 3.3. The completed models were exported as Quake .MD25 files such that bird size and flapping speed (both of which vary by species) could be set programmatically by BirdGen. Flapping speed, bird size, and flight speed were inferred from previous work [34] and used

2https://guillmcivor.wordpress.com/

3Conversations were conducted during a ﬁeld research visit in 2019, and subsequent conversations took place over email.

4https://www.blender.org/

5http://tfc.duke.free.fr/coding/md2-specs-en.html

18 (a) Sketch of the bone structure and ﬂapping (b) Sketch of the bone structure and ﬂapping motions of a jackdaw. motions of a rook.

(c) A screenshot of the animated jackdaw (d) A screenshot of the animated rook model model in Blender with bone rigging. in Blender with bone rigging.

Figure 3.3: The sketches provided by Guillam McIvor and the resulting 3D models and rigging.

19 Parameter Jackdaw Rook Wingspan (m) [0.34, 0.39] [0.45, 0.47] Flapping freq. mean (Hz) 4.2 3.4 Flapping freq. std. dev. (Hz) 0.3 0.4 Flight speed (m/s) 15 15

Table 3.1: Individual bird parameters used by BirdGen.

in BirdGen’s flocking parameters. Where ranges are given, values are randomly chosen from a linear probability distribution in that range. They are listed in Table 3.1. Additional logic takes into consideration various rules regarding flock composition and size. (Where no references are specified the information was gathered from email correspondence with Guillam McIvor, mentioned previously, specifically for the purpose of configuring the simulated flocks.) These rules are:

• Mixed species ﬂocks of jackdaws and rooks occur with 25% probability; otherwise, a ﬂock will be composed entirely of jackdaws or entirely of rooks.

• For mixed ﬂocks, if the ﬂock size is 200 or greater, there is a 90% probability that any given bird will be a jackdaw.

• For mixed flocks, if the flock size is less than 200, the probability of being jackdaw- dominated (as opposed to rook-dominated) is 50%. In the case of jackdaw-dominated flocks, birds will be jackdaws with 75% to 95% probability (chosen randomly at the beginning of flock creation). In the case of rook-dominated flocks, birds will be jackdaws with 5% to 25% probability (chosen randomly at the beginning of flock creation).

• Birds are initialized in a circular disk pattern 30 meters in diameter, at a height of 21 to 27 meters [34], selected randomly. Individual bird altitude offsets are assigned using a Gaussian distribution with a standard deviation of 3.3 meters, to reflect the fact that bird flocks are actually quite flat, despite their appearance to the contrary to the ground-based observer.

• The probability that any given bird is paired with another of the same species is determined by a random probability between 5% and 80%, inclusive [33], determined at the beginning of ﬂock creation.

3.3.3 Flocking Controller

A Boids-like controller [55] provided the initial inspiration for ﬂocking in BirdGen. This was further tuned several times until an acceptable level of realism was achieved. Notable changes from the original Boids controller include linearly-scaling forces and the inclusion

20 Figure 3.4: Sample output image generated by BirdGen.

Figure 3.5: A photograph of a real flock of jackdaws. Note that BirdGen does not render images of trees. of pairs that are believed to be indicative of the monogamous pair-bonds seen in flocks of rooks and jackdaws [26]. The final version of the bird flocking controller is described below. Like the original Boids algorithm, it is defined by three social forces: attraction, repulsion, and alignment. BirdGen defines these forces by the equations,

21  |d | d × (1.0 − i,j ), if |d | < r and i 6= j i,j rattract i,j attract fattract = (3.1) 0, otherwise,  |di,j | −[di,j × (1.0 − ), if |di,j| < rrepulse and i 6= j rrepulse frepulse = (3.2) 0, otherwise,  |di,j | vj × (1.0 − ), if |di,j| < ralign and i 6= j ralign falign = (3.3) 0, otherwise,

where di,j is the vector from bird i to bird j, vi is the velocity of bird i, and rx describes the radius of social force x. The •b operator denotes vector normalization and |•| is the L2 (i.e., Euclidean distance) norm. Paired birds are also subject to the following force, which ensures that they ﬂy close together:

 |d | d × (1.0 − i,j ), if |d | > r and i 6= j i,j rmate i,j mate fmate = (3.4) 0, otherwise.

The ﬁnal output force fsum is computed by Equation 3.3.3,

fsum = (fattract × gattract × gsocial) + (frepulse × grepulse × gsocial)+

(falign × galign × gsocial) + (fmate × gmate) ,

where g describes the gain multipliers associated with each force. gsocial is intended to scale down non-pairing forces when paired birds are too far apart, allowing them to ﬂy back together. It is set according to

 0.5, if |di,j| > rmate and i 6= j gsocial = (3.5) 1.0, otherwise.

The final force fsum is normalized and scaled by a constant bird speed, and this becomes the new velocity vector of the bird. While no formal analysis has been conducted, informally, the resulting flock behaviours bear a strong resemblance to those of mixed-species flocks of jackdaws and rooks.

3.4 Output

BirdGen generates 12 channels of PNG data per camera-frame and several text ﬁles. The directory structure of BirdGen’s output is described below.

22 • cameras Text ﬁles containing the projection and view matrices of each of the 4 cameras in column-major format.

• ground_truth Text ﬁles containing the unique ID and world-space positions of every bird for every frame.

• cameras 8-bit RGB PNGs of the simulated camera images, for every frame of each of the 4 cameras. These are meant to simulate the real-world images that would be captured by a 4-camera array. See Figure 3.1 for an example.

• label 16-bit greyscale PNGs of labelled simulated camera images, for every camera and every frame. These output channels can be used as training inputs to a deep neural network. The use of 16 bits per pixel means that each pixel can take a value in the range [0, 216).

– px Images where pixel values containing birds refer to the pixel oﬀset to the bird’s origin along the horizontal axis. All other pixels are black. The image origin (0, 0) is at the bottom left of the image. – py Same as px, except for the vertical axis. – dp Depth of each bird from the camera, where depth is deﬁned as the shortest distance to the camera plane. This is formally described by Equation 3.6,

FcC · (PB − PC ), (3.6)

where FcC is the forward vector of the camera, PB is the position of the bird, and

PC is the position of the camera. All coordinates are in world space. – qx, qy, qz, qw Normalized (from 0.0 to 1.0) components of a quaternion representing the bird orientation in the world frame with respect to the camera world orientation. This output quaternion is formally described by Equation 3.7,

Γ(C · M ) + {1, 1, 1, 1}| bV B , (3.7) {2, 2, 2, 2}|

where CV is the camera view matrix, MB is the bird model matrix, and Γ(•) is the operator that extracts a quaternion from a 4x4 transformation matrix. Informally, Equation 3.7 simply scales the bird quaternion (which is expressed in camera space) elements into the range [0.0, 1.0]. – fp Flap phase of the bird (wrapping from 0.0 to 1.0), where 0.0 and 1.0 represent the wings in their top-most position, and 0.5 represents the wings in their bottom-most position. In terms of species discrimination, this was a particularly important label since jackdaws and rooks ﬂap their wings at diﬀerent frequencies.

23 Although these frequencies do overlap slightly between the two species, ﬂapping speed can be used as a useful indicator as to whether or not a particular individual unambiguously belongs to one species or another. – jd Jackdaw mask, where all jackdaw bird pixels are rendered as 0.5 greyscale and all other pixels are black. – rk Same as jd, except for rooks. – id Images where bird pixel values correspond to the unique ID of each bird. IDs are assigned in the range [1, 216). All other pixels are black. This channel is only used for debugging and reconstruction error analysis.

Figure 3.6 shows all 12 channels of output for a sample camera-frame.

3.5 Comparison Against Real Flocks

3.5.1 Visual Comparison

Figure 3.7 shows side-by-side, time-overlayed images of real ﬂocks (left column) and synthetic ﬂocks generated by BirdGen (right column). Note that the following features are present in both columns of images.

• The presence of pairs. These are visible as trails that appear in close proximity and are nearly parallel to one another.

• Several small groups exhibiting coordinated motion. Both sets of images contain sub-groups of individuals which appear to move in roughly the same direction.

• The existence of ‘glide’ flight phases. Both sets of images contain trails in which a bird’s wings are fully extended for a certain length of time without flapping. This was implemented in BirdGen as a ‘gliding’ phase in which a simulated bird fully extends its wings for 0.5 seconds. This occurs after a delay of at least 2 seconds since the last gliding phase, once the bird’s wings are in the extended position as a result of regular flapping.

• Similar wing ﬂapping frequency. Both the real photos and synthetic images have a frame rate of 60 fps. Examining the wing positions of a single bird trail, and counting the number of consecutive bird images b in between fully-extended positions, enables 60 the calculation of wing beat frequency using the formula b . Visual examination of various real bird trails in this manner reveals wing beat frequencies between roughly 3.5 Hz and 5.5 Hz. Recall from Table 3.1 that jackdaws and rooks in BirdGen were assigned ﬂapping frequencies sampled from gaussian distributions having means of 4.2 Hz (0.3 Hz) and 3.4 Hz (0.4 Hz), respectively. This is also visible in the synthetic images.

24 (a) The simulated scene as (b) Screen-space x- (c) Screen-space y- captured by one of the cam- coordinates of the bird coordinates of the bird eras. centers. centers.

(d) Linear depth of the bird (e) Quaternion x-component (f) Quaternion y-component centers, clamped at 65535 of bird orientation with re- of bird orientation with re- (200 meters). spect to the camera. spect to the camera.

(g) Quaternion z-component (h) Quaternion w-component (i) Circular ﬂap phase of of bird orientation with re- of bird orientation with re- birds, where 0 and 65535 rep- spect to the camera. spect to the camera. resent extended wings.

(j) Jackdaw mask. Only jack- (k) Rook mask. Image is blank (l) IDs of birds. These range daws are present in this ﬂock. because no rooks are present. from 0 to 65535 and are too dark to see in this image.

Figure 3.6: The 12 channels of PNG output generated by a single camera-frame. Best viewed on a computer screen or high-quality print-out.

25 Figure 3.7: Left column: Images generated by compositing successive photographs of corvid flocks (used with permission from Hangjian Ling). Right column: Images generated by BirdGen without clearing the OpenGL framebuffer, resulting in a similar effect.

• Straight lines and sudden changes of direction. Both sets of images show combinations of relatively straight trajectories as well as trails which exhibit sudden coordinated changes of direction.

3.5.2 Training Performance on a Neural Network

As mentioned in Section 3.1, BirdGen was developed as part of a larger collaboration whose goal is to further understand real flocking behaviour. At the time of writing, the neural network tool for which BirdGen is intended is not yet complete. Because of this, a more empirical evaluation of BirdGen—such as how well the network performs when trained on the simulated flocks, and how much real data (if any) is required for fine-tuning—is not

26 presented here. While results will be forthcoming, they will not be available in time for completion of this thesis.

3.6 Future Work and Conclusion

A limitation of BirdGen is that it does not take into account physical effects such as motion blur or atmospheric fog, both of which are present in the real images seen in Figure 3.7, but are absent in the synthetic ones. Future versions could include effects such as these to increase visual realism, for the sake of more accurate neural network training samples. This chapter has described a software program for generating mass quantities of realistic bird flocking images. It was developed using feedback from an expert on jackdaw and rook behaviours and appearance. The goal of this tool is to provide a method for generating training data of a sufficient quality such that it can be used to train a deep neural network to recognize individual birds in a real flock, as well as their species, distance from the camera, and other properties.

27 Chapter 4

Pairwise Flocking Simulations

4.1 Introduction

As mentioned in Chapter 2, the vast majority of computer-based flocking models assume that agents in a flock are homogeneous. (Exceptions include Hartman and Benes’ work on leadership changes [21] as well as the original pairwise flocking controller first described by the author in [44].) However, studies of real flocks have revealed two ways in which flocks are heterogeneous. The first is the existence of mixed-species flocks of jackdaws and rooks, and the second is the existence of pairs, which is a central theme of this thesis. This chapter will present a simulated pairwise flocking controller and show how it results in the following two advantages: (a) improved flock stability, which is the tendency of a flock to maintain fewer connected structures instead of splitting into many sub-flocks, and (b) the reduction of the number of nearby agents that must be considered by some focal agent in order to maintain its position in the flock. The work described in this chapter was first presented in [44] and was the first to empirically explore and evaluate the advantages of such a controller. It is currently unknown how or why certain species of birds prefer to flock in pairs. Al- though the pairwise flocking phenomenon seen in nature has provided the initial inspiration for the controller described in this chapter, it should be understood that the motivation and results given here may not necessarily be applicable to real flocks. No claims are being made regarding the mechanisms or motivations that underlie this behaviour in nature.

4.2 Flocking Controller

The flocking controller described in this chapter mimics the pairwise behaviour described in Chapter 2. In addition to tracking a fixed partner, each agent also tracks a fixed number of nearest neighbours, i.e., those within some topological distance. This is in line with previous work demonstrating that flock members track some approximate number of nearest neighbours (e.g., 6-7 for birds [4, 7], or 1-2 for fish [25]) as opposed to all neighbours that fall within some Euclidean distance (such as in the classic Boids model [55]).

28 A conventional forces model generates flocking behaviour, whereby agents experience virtual attraction, repulsion, and alignment forces with each neighbour N within a fixed 1 topological distance. Each force is scaled by , where dN is the topological distance to dN agent N. Topological distance is defined as the nearest-neighbour distance in Euclidean space. For example, the agent nearest some focal agent A has a topological distance of 1 to agent A. The second-closest agent to A has a topological distance of 2, and so on. The magnitude of each of the three forces is normalized with respect to the number of tracked nearest neighbours and applied to a final steering vector. Repulsion is only enabled when two agents are within a fixed metric distance of one another. Agents using this controller may also optionally have a fixed partner, henceforth referred to as a mate. This is a reciprocal, permanent relationship and is analogous to the stable pairings described in Chapter 2. Agents in a pair are subject to the same virtual forces as the other agents in the flock, with the exception that they experience a shorter repulsion radius to their mates. This enables the pair to flock closer together, mimicking the behaviour seen in nature. Flocking closer to a mate in this fashion is enabled or disabled by a boolean variable called close pairing (CP ). Paired agents are also optionally subject to the constraint that they may only choose interaction partners on the side of their pairwise formation that is opposite to their mate, as depicted in Figure 4.1. The intuition here is that a paired agent should treat their mate as a proxy for all other agents on that side of them. This ensures that the two paired agents always track mutually-exclusive sets of neighbours. This also reduces the total number of partners that a paired agent would need to track, roughly by half. This is enabled or disabled by a boolean variable called asymmetric tracking (AT ). Agents can see 360° around them, but have a maximum view range beyond which connections to other agents are not established. This allows a flock to break up in the presence of obstacles or flock instability, or possibly merge later on. Implementing a limited view range in this manner allows for the virtual flocks to divide or merge, as seen in real flocks [34, 33]. Additionally, paired agents cannot be “pulled apart” in the presence of conflicting forces. An agent’s mate is always considered to have a topological distance of 1. This, combined with the fact that paired-agent connections are also not subject to a limited connection range, ensures that the pair always stays together.

4.3 Evaluation

4.3.1 Simulator and Environment

To evaluate the performance of the flocking controller described in Section 4.2, a custom flocking simulator was developed in C++. The main goal was to determine how various configurations of pairwise flocking parameters affected overall flock stability, particularly in

29 B A

Figure 4.1: Agents A and B track mutually-exclusive sets of neighbours on their respective sides of the pair formation. the presence of external forces, e.g., obstacles, that were meant to disturb the flock. Previous work by Ballerini et al. [4] and Camperi et al. [7] also used computer simulations to test the stability of simulated flocks of birds. Figure 4.2 shows a screenshot from a typical run1. The 2D media library SIGIL2, which was used to build the simulated world, operates in floating-point pixel units; the distances and sizes reported here are in those units. Although flocks of real birds are distributed in three dimensions, they can be treated roughly as 2D since such flocks are usually relatively flat and move parallel to the ground plane, despite the appearance of the contrary when seen from the ground [5]. Agents were implemented as particles with a constant velocity of 90 pixels per second. The environment itself is infinite and unbounded: there are no surrounding barriers and the environment is non-toroidal. It contains a field of circular obstacles, each with a radius of 10 to 50 pixels, chosen randomly. They were placed on a uniform 12 × 12 grid, with each cell measuring 200 square pixels, which is centered at the environment’s origin. Each obstacle exerts a strong repulsive force on any agent that overlaps it, causing the agent to steer around it. This provides an environment designed to test the stability of a flock; weakly-connected or unstable flocks will more easily be broken up by the obstacle field as they navigate through it. Agents were randomly spawned within a 600 × 600 pixel area centered at the origin, and were assigned a random initial orientation. Paired agents were spawned within 15 pixels of

1A video demonstration can be found at https://youtu.be/LMIJSa2m1S8

2www.libsigil.com

30 Figure 4.2: A screenshot of the pairwise flocking simulator. Colour-corrected for clarity and printer-friendliness. one another. These values were chosen to ensure that the flock achieved a relatively stable, single structure at the onset of the simulation. The simulator can be configured with the following relevant parameters: the number of single agents s, the number of agent pairs p, the number of nearest neighbours tracked by single agents ns, and the number of nearest neighbours tracked by paired agents (not including their mate) np.

4.3.2 Experiments and Performance Criteria

In Experiment #1, 100 trials were run for each of the following 5 configurations, for varying values of ns and np, as seen in Table 4.1. In all configurations, the total agent population was 44 and both CP and AT were enabled. Each trial was run for 30 simulated seconds, since visual observation revealed that this was sufficient time for all agents to exit the obstacle field in most cases. However, exiting the obstacle field was not a specific goal. Fixed random seeds were used for each trial to aid in reproducibility and debugging. An image of a typical trial is shown in Figure 4.2. The simulator recorded a number of statistics, which are described below.

31 s p Percentage Paired 44 0 0% 32 6 27% 22 11 50% 12 16 73% 0 22 100%

Table 4.1: Experiment #1 numbers of paired and unpaired agents.

B A

Figure 4.3: Agents A and B are a stable pair, and are compressed into a single node which now contains all of A’s and B’s connections.

A second experiment (Experiment #2) was run to measure the effects on flock stability of varying CP and AT . Similar to Experiment #1, 100 trials were run, each having the same 5 levels of agent pairing percentages as described above. In both experiments fixed values of ns = 7 and np = 3 were used. For the purposes of evaluation, it is convenient to represent the simulated flock as a graph (or several graphs, in the case of multiple sub-flocks), where agents are vertices and edges represent tracking interactions between them. Since connections in this implementation are based on topological distance, edges are directed. However, evaluation of flocking performance only requires consideration of the undirected version of the flock graph. In all sections that follow, this is the version of the flock graph that is considered. In addition, the compressed graph of the flock is maintained. This is a special representation of the flock graph in which all paired agents are each merged into a single vertex, as shown in Figure 4.3. Because the implementation guarantees that paired agents will (a) never become unconnected, and (b) always track mutually-exclusive sets of neighbours

32 (when AT = true), paired agents can always be considered to be a single vertex containing the edges of both agents in a pair. This is a useful statistic to consider because it takes into account the fact that information flow between agents in a pair will always be present, without artificially inflating measurements of information flow across the network. For example, the uncompressed network diameter (i.e., the length of the shortest path between the two most distance vertices) of a single chain of 4 agents with 2 paired agents is 3. However, a truer (i.e., non-inflated) representation of the network diameter is given by the compressed graph (containing only 3 agents with a diameter of 2) since it is known that the 2 paired agents always share the same information. Similar to [4, 7], one of the main performance criteria was the ability of the flock to maintain as few sub-flocks as possible, particularly in the presence of objects that may exert a repulsive force on its members (such as an attacking predator). It has been proposed [29] that one of the primary functions of flocking in nature is to reduce the risk of predation, and a smaller number of sub-flocks is considered preferable over both (a) a larger number of smaller flocks, or (b) single agents becoming separated from a larger flock. To this end, the simulator measures the number of connected graph components (i.e., sub-flocks) every second. The simulator also records the diameter (i.e., the length of the shortest path between the two most distance vertices) and degree (i.e., average number of connections) of the compressed and uncompressed flock graphs. These statistics can be taken as indicators of information flow through the flock. Greater information flow, regardless of the statistic used to measure it, is more desirable because it indicates that a greater number of flock members are exerting influence on other members, thus helping to keep the flock together. For example, a lower diameter is more desirable because, all other properties being equal, it indicates a higher level of graph connectivity and thus a more stable flock. Where multiple sub-flocks are present the diameter of each is measured, and normalized by the number of agents in that sub-flock over the total number of agents, to produce an average weighted diameter.

4.4 Results

The first row of Figure 4.4 shows the number of sub-flocks at the conclusion of each trial where ns = 7, np = 3, AT = true, and CP = true, for five levels of agent pairing. The values of ns = 7 and np = 3 are meaningful because (a) ns = 7 is the currently accepted maximum number of tracked nearest neighbours in real bird flocks [4, 7], and (b) np = 3 is approximately half of this. Note that in the 100%-paired, AT = true, and CP = true configuration in Figure 4.4, a single flock is maintained throughout all trials. As the number of paired agents decreases over the experimental configurations, the flocks are separated into an increasing number of sub-flocks.

33 h omlzddaee side mle stenme fpie gnsincreases. agents paired 4.8b, of Figure number in the shown as diameter and smaller graph 4.8a, indeed compressed is Figure the diameter in for normalized However, different the case. greatly diam- the the be indeed expect to is implies not configurations this sub-flocks would experimental of one re- any number normalization, with between higher this results eters a of intuitively, diameter Because since, the the diameters. sub-flocks average normalizes by of smaller This multiplied number sample. are the time here to each spect reported at values sub-flocks diameter of The number respectively. field. time, obstacle over the graphs navigate they as separation to prone more are flocks the n of tracking combinations different with each trial, agents, single by tracked agents, paired neighbours by of tracked number neighbours the of of number here, values the results varying all for trial, In every ages. after Sub-flocks 4.4: Figure s

7 = CP = 0,AT = 0 CP = 1,AT = 0 CP = 0,AT = 1 CP = 1,AT = 1 iue .aad48 hwteaeaedaee fteoiia n opesdflock compressed and original the of diameter average the show 4.8b and 4.8a where Figures trials all of duration the over sub-flocks of number average the shows 4.7 Figure each of conclusion the at sub-flocks of number the show 4.4 Figure of rows remaining The

Frequency Frequency Frequency Frequency 100 100 100 100 40 40 40 40 80 80 80 80 60 60 60 60 20 20 20 20 0 0 0 0 1 1 1 1 and %Pie 7 ard5%Pie 3 ard10 Paired 100% Paired 73% Paired 50% Paired 27% Paired 0% and 2 2 2 2 3 3 3 3 n p ls pairing close 4 4 4 4 3 = 5 5 5 5 6 6 6 6 o v eeso gn arn.A h ubro ardaet decreases, agents paired of number the As pairing. agent of levels ﬁve for , 7 7 7 7

Frequency Frequency Frequency Frequency 100 100 100 100 40 40 40 40 80 80 80 80 60 60 60 60 20 20 20 20 0 0 0 0 1 1 1 1 2 2 2 2 aeo h tblt fteﬂc si aiae h environment. the navigates it as ﬂock the of stability the on have 3 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7 7 7

Frequency Frequency Frequency Frequency 100 100 100 100 40 40 40 40 80 80 80 80 60 60 60 60 20 20 20 20 0 0 0 0 1 1 1 1 AT 2 2 2 2 34 and 3 3 3 3 4 4 4 4 CP 5 5 5 5 6 6 6 6 n oso h ﬀcsthat eﬀects the show to , p 7 7 7 7 s3. is ,

Frequency Frequency Frequency Frequency 100 100 100 100 40 40 40 40 80 80 80 80 60 60 60 60 20 20 20 20 0 0 0 0 1 1 1 1 AT 2 2 2 2 3 3 3 3 , 4 4 4 4 CP 5 5 5 5 n arn percent- pairing and , 6 6 6 6 7 7 7 7

Frequency Frequency Frequency Frequency 100 100 100 100 40 40 40 40 80 80 80 80 60 60 60 60 20 20 20 20 0 0 0 0 1 1 1 1 2 2 2 n 2 asymmetric s 3 3 3 s7 and 7, is , 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7 7 7 Figure 4.5: Snapshots from a sample run with no pairing enabled, at 0, 6, 12, and 18 seconds into the simulation. The first sub-flock exits the obstacle field in the final frame.

Figure 4.6: Snapshots from a sample run with half of the agents paired, at 0, 5, 10, and 15 seconds into the simulation. The flock exits the obstacle field in the final frame.

Figure 4.9 shows the difference in average degree between the compressed and the original flock graph over time. Note that as the percentage of paired agents increases, the difference in degrees increases as well. There is no difference in degree where the number of paired agents is zero, since both the original and the compressed graphs are identical in this case. Figure 4.10 shows the number of sub-flocks present at the conclusion of each trial, where np = 3 and ns varies. Examination of the averages and confidence intervals shows that the flock is still not entirely stable even when the number of neighbours tracked by single agents is as high as 15. Interestingly, Figure 4.10 also reveals that even with a low value of ns (i.e., less than 6 or 7), the flock is still reasonably stable if a sufficiently high percentage of agents are paired.

4.5 Discussion

These results show that pairwise flocking can provide an advantage for the type of flocking controller presented in this chapter. Figures 4.4 and 4.7 show that a greater ratio of paired to unpaired agents—even those which track roughly half as many nearest neighbours as unpaired ones—reduces the average number of connections in the flock, yet still results in greater stability. The reason for the increased stability can be seen in Figures 4.8 and 4.9. Figure 4.8b shows that with more paired agents the diameter of the flock decreases, resulting in shorter paths between agents on average. This is because, as mentioned in Section 4.3.2, a mated pair can be considered to be a single agent that carries connections of both agents in a pair. This observation is further reinforced by Figure 4.9, which shows that the degree of

35 Average Number of Sub Flocks Over Time

0% paired 1.8 27% paired 50% paired 73% paired 100% paired 1.6

1.4 Average Sub Flocks 1.2

1.0

5 10 15 20 25 30 Time (sec.)

Figure 4.7: The average number of sub-flocks over time for all trials where ns = 7 and np = 3. Error bars denote 95% confidence intervals. the compressed flock graph increases with respect to the original flock graph as a greater percentage of agents are paired. Figure 4.4 illustrates the effects of asymmetric tracking and close pairing. Of the two, asymmetric tracking has the greatest positive effect on the stability of the flock. With asymmetric tracking enabled, close pairing only provides a minimal improvement. Asym- metric tracking is beneficial because it allows each pair to track mutually-exclusive sets of neighbours, as described in Section 4.2. Without it, paired agents may have overlapping interaction neighbourhoods and this can contribute to a lower average degree of the compressed flock graph. As shown in the portions of Figure 4.4 where asymmetric tracking is disabled, the flock stability decreases as the number of paired agents increases. This is due to the fact that paired agents track roughly half as many neighbours as unpaired ones—thus, more paired agents implies fewer interactions overall. Figure 4.4 also shows that with asymmetric tracking disabled, the stability of the flock actually improves slightly if close pairing is disabled. Disabling the latter allows paired agents to maintain a greater distance from each other. The advantage here lies in the fact that physically spread-out pairs are more likely to track mutually-exclusive sets of neighbours (due to agents’ limited view range) than pairs which flock closer together. These results indicate that for maximum benefit, both asymmetric tracking and close pairing should be enabled, at least for flocking controllers similar to the one presented here. Where asymmetric tracking is unavailable, stability can be improved by disabling close pairing. Importantly, it is not always desirable for a flock to maintain a single structure

36 Diameter Over Time Diameter (Compressed) Over Time 11 11 0% paired 0% paired 10 27% paired 10 27% paired 50% paired 50% paired 9 73% paired 9 73% paired 100% paired 100% paired 8 8

7 7

6 6 Diameter (Normalized) Diameter (Normalized) 5 5 5 10 15 20 25 30 5 10 15 20 25 30 Time (sec.) Time (sec.) (a) Average ﬂock diameter (b) Average compressed ﬂock diameter

Figure 4.8: Average flock diameter of the (a) original flock and (b) the compressed flock, normalized with respect to the number of sub-flocks. Error bars denote 95% confidence intervals. These values are relatively unchanged over time since they are normalized with respect to the number of sub-flocks; thus, they are an indicator that flock diameter in general is dependent on the number of paired agents. under all circumstances, particularly where obstacles are present. The simulated obstacles in this work are simply meant to test the stability of the flock under various configurations. It is worth mentioning that without these obstacles a flock is much less likely to break up. These results show that it can be beneficial for agents to form pairs when flocking. When they do, the average degree of the flock is effectively increased. Agents can afford to track fewer neighbours and enjoy a more stable flock. This is useful in physical implementations of flocking systems (e.g. UAVs), since occlusion and limited fields-of-view can place limitations on the number of neighbours that can be tracked. Computational limitations with respect to vision-based tracking, in particular on embedded systems, may also restrict the number of neighbours that can be tracked in real time. Pairwise flocking presents a means to partially overcome these limitations. This assumes, however, that the computational cost associated with tracking a fixed partner is at most as demanding as that of tracking any other interaction partner. Thus, the advantages associated with this controller are highly dependent on the tracking method(s) employed by the flocking agents.

4.5.1 Applications and Limitations

This work has a number of real-world applications. First, it demonstrates that stable flocking behaviours can be achieved with reduced tracking effort, which is particularly desirable in mobile or embedded systems due to their limited computational power. Using this pairwise flocking controller, agent flocking simulations can be built that require smaller interaction neighbourhoods, improving simulation performance. Also, physical flocking systems with

37 Compressed Degree Minus Degree Over Time

4.0 0% paired 27% paired 3.5 50% paired 73% paired 3.0 100% paired

2.5

2.0 Degree 1.5

1.0

0.5

0.0

5 10 15 20 25 30 Time (sec.) Figure 4.9: The difference in average degree of the compressed and original flock graphs, where ns = 7 and np = 3. With the obvious exception of the case with 0% pairing, mean differences at each time step were found to be statistically significant at p < 0.05 using an independent t-test. Error bars denote 95% confidence intervals for the difference of means.

UAVs can be constructed with simpler tracking components. For example, rather than using an expensive 360° field-of-view camera to track all nearby neighbours, a UAV only needs a single 180° camera to track nearby neighbours on one side, and a lower-resolution camera on the opposite side to track its mate, which can be assumed to be physically closer. Orthogonal to this argument is the benefit of having a “wingman” in general: having a fixed partner or team allows for the easier coordination of tasks and enables increased redundancy in the case of failures [20, 43]. Pairwise flocking appears to work well for the particular type of controller presented here. It is possible that other types of controllers may not enjoy the same benefits. For the same reason, it is impossible to provide exact parameters (e.g., ns, np) for pairwise flocking that would be optimal for all controller types. This work relies on the fact that an agent can always be tracked by its mate, and that this connection is never broken. These assumptions may not always be valid. An erratically moving UAV may be more difficult to track, and fast travel speeds and increased agent maneuverability may increase the probability that a mate could become lost. In this case, a recovery mechanism should be executed to locate the agent, join the flock as an unpaired agent, or find a new mate if the original is unrecoverable. This is explored further in Chapter 6.

38 Sub Flocks Over Neighbours Tracked by Single Agents 6 0% paired 27% paired 50% paired 5 73% paired 100% paired

3 Sub Flocks

2 3 4 5 6 7 8 9 10 11 12 13 14 15 Number of Tracked Neighbours

Figure 4.10: The average number of sub-ﬂocks at the conclusion of every trial, where np = 3 and ns varies. Error bars denote 95% conﬁdence intervals.

4.6 Future Work and Conclusion

Future work on this topic includes examining the benefits of stable 3- or even n-way relationships. Due to the fact that it may be difficult to find optimal values of np and ns for a given flocking system, it would be useful to develop a controller that can learn them over time as members interact, or controllers which dynamically adjust the number of connections, or even the percentage of paired agents, on-the-fly. This chapter has described a pairwise flocking behaviour that results in a reduced tracking load for individual paired agents, and increases the stability of the flock as a whole in the presence of obstacles. This is inspired by observations of real birds. Although there is no evidence that real flocks perform this exact behaviour (or experience the advantages of doing so shown in this chapter), the pairwise flocking controller simply exploits this possibility to reduce computational load and improve the stability of the simulated flock. These findings could have practical benefits for real-world flocking robot applications. Chapter 6 validates that these effects can be seen in a real-world robot setting.

39 Chapter 5

The µBee Drone

5.1 Introduction

(a) Digital render of the µBee drone. (b) Photograph of the µBee drone.

Figure 5.1: Digital render and photograph of the µBee drone.

This chapter describes a small, low-cost educational and research drone called the µBee, shown in Figure 5.1. The use of the Greek symbol µ, indicating “micro”, reflects its development philosophy: in all aspects of its design, the µBee was optimized for size, simplicity, and low cost. Complete firmware and hardware specifications are open-source and available at https://gitlab.com/geoff-nagy/ubee. It was built to satisfy the need for a low-cost and lightweight drone in research and educational settings, as well as validate the performance of the pairwise flocking controller on a team of real drones, as seen in Chapter 6. The µBee was built with the following goals.

• Low cost. A lower price tag allows larger fleets to be built. A fleet of 10 drones, for example, including several spare parts, 20 LiPo batteries, and 35 reflective motion

40 capture markers, costs $1,085.45 USD (shipping and taxes excluded). Without factor- ing in the cost of the optional reﬂective markers, each drone (including spare parts) costs less than $100 USD. In comparison, the popular Crazyﬂie drone1, which features a similar form factor and has been used in previous research (e.g., [52]), costs $195 USD including spare parts.

• Small size and weight. Not including rotors, the µBee frame measures approximately 85 mm (3.35 in) on a side. Without the 9 g 300 mAh LiPo battery, the drone weighs 25 g. Its small size and weight means it can be easily transported and stored. Additionally, less room is required for ﬂight. A disadvantage of its small size is its inability to ﬂy outdoors except in low-wind conditions.

• Simplicity of repair and construction. The µBee’s frame was designed using the computer-aided design (CAD) software program Fusion 3602 and can be 3D printed (see Section 5.2.1). Because it is 3D-printable, the frame can be easily replaced if damaged. The frame is durable—initial prototyping and the crashes inevitably associated with early development have resulted in a sturdy design that does not easily break, even during full speed collisions. Although the main printed circuit board (PCB) can be ordered with the surface-mount device (SMD) components pre-populated, no components have a footprint smaller than the 0603 format (1608 metric), and thus any component can be (re-)soldered by hand if necessary. Construction of a µBee with the SMD components already assembled on the PCB can be completed in roughly 30 minutes and only requires through-hole soldering skills. (Soldering the SMD components onto a blank PCB by hand, if desired, takes several hours.) All other components are screwed on or press-ﬁt into place.

• Support for motion capture attachments. Using the four screws holding the PCB to the board, it is possible to add attachments that allow the drone to be tracked using a motion capture system. As depicted in Figure 5.4, this is achieved by attaching a 3D-printed plate through which 3D-printed pins can be inserted. Reﬂective motion capture markers are screwed onto those pins. Motion capture support is described further in Section 5.3. Other attachments, such as cameras, could also be mounted using a similar design if desired.

• Educational value. An additional goal was to provide a platform that would be useful to educators teaching robotics or real-time systems. An autonomous drone exempliﬁes both of these, and the core ﬂight control source code is a mere few hun- dred lines of well-documented C code that is within reach of a third- or fourth-year

1https://store.bitcraze.io/collections/kits

2https://www.autodesk.ca/en/products/fusion-360/overview

41 robotics or computer science undergraduate student. A similarly-accessible, custom wireless radio protocol is used, with possible applications for teaching basic digital wireless communication.

5.2 Main Speciﬁcations and Development

5.2.1 3D Printing the µBee Frame

The frame of the µBee measures approximately 85 mm on a side and is 17 mm tall. Ultimaker Cura 4.8.03 was used to slice the model for export and printing. When 3D printing the frame with an AnyCubic Mega S4, using the print settings described in Table 5.1, the total print time for one frame is 3 hours and 18 minutes. Although this print time can be reduced by half by printing at a resolution of 0.2 mm instead of 0.1 mm, the finer printing resolution of 0.1mm results in a smoother appearance and more accurate tolerances with respect to components that must be mounted vertically. The material used was polylactic acid (PLA). The printed frame weighs approximately 11 g, using 3.63 m of PLA filament with a strand diameter of 1.75 mm. Since the top of the µBee’s frame is flat, the model can be printed upside down without supports.

5.2.2 Electronic Components and Hardware

At the µBee’s heart is the popular 8-bit Microchip (formerly Atmel) ATmega328P5. The advantages associated with this microcontroller are its low cost (less than $3 USD) and large community support, partially due to its use in the Arduino platform. Its primary weakness is its limited computational power and memory. Without an external oscillator, the chip is limited to 8 MHz. It features only 32 KB of flash memory, 1 KB of EEPROM, and 2 KB of SRAM [40]. As an 8-bit microcontroller, it performs floating-point operations quite slowly. Despite these limitations, the flight control firmware runs at a rate of approximately 190 Hz. This update rate is sufficient for maintaining stable flight (see Chapter 6). The µBee uses the RFM69 915 MHz radio transceiver6 to communicate with a control station or handheld remote control. The SMD version of this transceiver costs $7 USD and works over significant distances (at least 100 m through significant obstacles, as determined during early µBee development tests), making it an ideal choice for a low-cost drone. It has a power output of +13 dBm, a low-current receive mode that consumes only 16 mA, and supports bit rates up to 300 KBs [22]. This transceiver was chosen instead of a Bluetooth

3https://ultimaker.com/software/ultimaker-cura

4https://www.anycubic.com/products/anycubic-i3-mega-s

5https://www.microchip.com/wwwproducts/en/ATmega328p

6https://www.hoperf.com/modules/rf_transceiver/RFM69W.html

42 Parameter Value Material PLA Filament diameter 1.75 mm Layer height 0.1 mm Initial layer height 0.3 mm Line width 0.3 mm Wall thickness 1.0 mm Top/bottom thickness 1.0 mm Optimize wall print order Yes Infill density 20% Infill pattern Grid Nozzle temperature 200 °C Build plate temperature 60 °C Print speed 50 mm/s Generate support No Build plate adhesion type Skirt Skirt line count 2 Total print time 3 hours 18 minutes Used filament weight 11 g Used filament length 6.63 m

Table 5.1: Relevant 3D printing parameters used in Cura 4.8.0 to slice the µBee frame for export to an AnyCubic Mega S 3D printer.

module to avoid connection difficulties that often arise with Bluetooth. The downside of this decision is that the base station (a laptop or workstation) must use a separate USB device to communicate with a µBee, rather than using any built-in Bluetooth capabilities. However, it is worth noting that this custom radio setup has resulted in no wireless communications problems whatsoever. The base workstation setup used to control a fleet of drones is described in Chapter 6. A TDK InvenSense MPU6050 7 inertial measurement unit (IMU) breakout provides gyroscopic and linear acceleration information to the ATMega328P via an inter-integrated circuit (I2C) connection. Activating the MPU6050’s built-in digital low-pass filter [24] was necessary to reduce the effects of vibrations on orientation sensing while flying. When powered by a 3.7 V 300 mAh LiPo battery, the µBee achieves flight times of at most 3.5 minutes. The ATmega328P monitors battery levels via a voltage divider to ensure that the battery does not overdischarge. A Texas Instruments LM3670MF-3.3 buck converter8 provides a constant 3.3 V to the onboard components. This is a requirement since the RFM69 and the MPU6050 accept 3.3 V logic only. A blue LED on the main board

7https://invensense.tdk.com/products/motion-tracking/6-axis/mpu-6050/

8https://www.ti.com/product/LM3670

43 indicates a low battery condition or the reception of wireless commands. The MPU6050 board comes with a white LED already mounted; this provides a power-on indicator. The electronics schematic was designed using Diptrace Schematic Capture9, and is shown in Figure 5.2. The PCB was designed using Diptrace PCB Layout, and is shown in Figure 5.3. The ﬁnal version of the pre-assembled PCB was manufactured by PCBWay10. Only 2 PCB layers were required; this helped to keep fabrication costs low. Fabrication parameters are speciﬁed in Table 5.2.

Figure 5.2: Schematic of the µBee PCB.

5.3 Support for Motion Capture Systems

As mentioned in Section 5.1, the design of the µBee supports attachments that can be screwed into place on top of the PCB. Reflective motion capture markers can be mounted onto these attachments, allowing a motion capture system to detect the µBee’s position and orientation. Figure 5.4 shows how these markers are mounted onto the attachment. Figure 5.5 shows both a computer-generated render and a photo of the µBee with reflective markers attached. Because the µBee is so lightweight (34 g including the battery), special care was taken to choose a configuration of markers that would neither weigh it down nor unbalance it during flight. The marker attachment was designed to respect the following constraints, to maximize both (a) detection by a motion capture system, and (b) flight performance.

9https://diptrace.com/

10https://www.pcbway.com/

44 (a) The top layer of the µBee PCB. (b) The bottom layer of the µBee PCB..

Figure 5.3: The top and bottom layers of the µBee PCB. Having only 2 layers helped to keep costs low. The PCB measures 35.5 mm × 46.5 mm.

Note that these two goals compete with one another: adding markers contributes to greater motion capture visibility, but negatively aﬀects ﬂight performance.

• Only three markers should be used. The marker attachment with three 9.4mm B&L Engineering reflective pearl markers mounted weighs approximately 3 g. At- tached to a 34 g µBee, this represents a weight increase of approximately 9%. Although a minimum of four non-coplanar arrangements of reflective markers is required to unambiguously determine the orientation of an object, weight considerations dictated that only three markers be used. A marker configuration was chosen such that only two orientation solutions are possible: upside down, or right side up. Since during regular flight a drone should not be upside down, this solution can be discarded.

• The markers’ center of gravity should be as close to the origin as possible. This helps to ensure balanced ﬂight.

• Markers should be at least 35 mm apart. Experiments with a Vicon motion capture system (see Chapter 6) have revealed that pairs of reﬂective markers closer than 35 mm cannot be consistently resolved into separate detections. Therefore the conﬁguration should ensure that all inter-pair distances are at least this far apart.

• The minimum diﬀerence over all pairs of intra-pair distances should be maximized. Ensuring that each pair of markers is as dissimilar (in terms of distance) to the others as possible lowers the probability of ambiguity in orientation detection.

• All three markers should be placed on a plane parallel to the ground. While this is not a strict requirement, it simpliﬁes the mounting of the markers and their weight distribution.

45 Parameter Value Size 35.5 mm × 46.5 mm Layers 2 Thickness 0.8 mm Min hole size 0.3 mm or greater Silkscreen White Surface ﬁnish HASL with lead Via process Tenting vias Material FR-4: TG130 Min track/spacing 20% Solder mask Black Gold ﬁngers No °C HASL to ENIG No °C Finished copper 1 oz cu Fabrication cost for 15 boards $18.00 USD

Table 5.2: The µBee PCB’s fabrication parameters.

To help satisfy these constraints, a custom interactive tool was developed. This tool allowed the manual placement of all three markers on an image of a µBee attachment and computes the following statistics.

• The center of gravity of all markers.

• Optionally, the automatic placement of the third marker if the positions of the ﬁrst two are known, such that the center of gravity is located at the origin.

• The minimum absolute diﬀerence over all pairs of marker intra-pair distances.

• The positions of the markers in millimeters, for use in CAD designs.

This application is shown in Figure 5.6 and resulted in the marker conﬁguration shown in Figure 5.5.

5.4 Flight Control Loop

The firmware running on the ATmega328P is responsible for maintaining stable flight, responding to wireless commands, and monitoring battery voltage. Special care was taken to ensure that the firmware was not too complex; at 3.3 V, the ATMega328P runs at 8 MHz [40] and excessive computation would limit the update rate of the main control loop, resulting in unstable flight. Stable flight is maintained by five individually-tuned proportional-integral-derivative (PID) controllers. Two of these are responsible for leveling the drone along its pitch and roll axes. The remaining three PID controllers stabilize the drone’s rotational velocity along each

46 Figure 5.4: Marker pins are snapped into place from underneath the attachment. Reflective markers are screwed onto the pins. The attachment can then be mounted onto the µBee. of its three axes. Despite the slow speed at which the ATmega328P performs these floating- point calculations, an update rate of approximately 190 Hz (5.1 ms) was achieved. This is sufficient for stable flight. The flight stabilization algorithms alone constitute approximately 60% of the entire update loop. Additional flight parameters control the relative scaling of yaw, pitch, and roll demands, as well as manual trim adjustment. All parameters are stored in the ATMega328P’s electronically-erasable programmable read-only memory (EEPROM). The µBee’s orientation detection is computed using a combination of accelerometer and gyroscope readings from the MPU6050. Gyroscope readings measure changes in rotational velocity and are subject to drift over the long term. Accelerometers, on the other hand, provide absolute orientation but are noisy in the short term. A complimentary filter [3] was implemented to provide accurate orientation readings (see Listing 5.1). This continually integrates gyroscope readings and uses the constant compAlpha to balance between the accelerometer readings (which are noisy) and the integrated gyroscope readings (which are subject to drift).

47 (a) Digital render of the µBee drone with (b) Photograph of the µBee drone with mark- markers attached. ers attached.

Figure 5.5: Digital render and photograph of the µBee drone with markers attached.

Listing 5.1: Source code for µBee orientation sensing.

// retrieve raw accelerometer and gyro data from the MPU6050 mpu6050.getAcceleration(&ax, &ay, &az); mpu6050.getRotation(&gx, &gy, &gz);

// compute pitch and roll in degrees from raw linear accelerometer values accelAngleX = atan2Approx(ay, az) * RAD_TO_DEG; accelAngleY = atan2Approx(ax, az) * RAD_TO_DEG;

// compute pitch, roll, and yaw rate from the gyro pitchRate = (gx / 131) * seconds; rollRate = -(gy / 131) * seconds; yawRate = -(gz / 131);

// integrate gyro values gyroX = pitch + pitchRate; gyroY = roll + rollRate;

// pass accelerometer and integrated gyro value into a complimentary filter pitch = (compAlpha * gyroX) + (oneMinusCompAlpha * accelAngleX); roll = (compAlpha * gyroY) + (oneMinusCompAlpha * accelAngleY);

48 Figure 5.6: Interactive tool used to place marker mounting holes on the µBee attachment.

The complimentary filter was chosen because it is easy to implement and computa- tionally fast. More complex methods such as a Kalman filter would severely impact the firmware’s update rate due to the required matrix math. Note that the atan2Approx function above is a fast approximation for the atan2 function acquired from https://gist. github.com/volkansalma/297223711. Battery voltage levels are monitored using the ATmega328P’s analog-to-digital converter. When voltage levels read below approximately 3.8 V, the µBee’s LED flashes every

11This code is based on the fast ﬁxed-point atan2 function described in http://dspguru.com/dsp/tricks/ fixed-point-atan2-with-self-normalization/.

49 second. This indicates that the battery has reached nominal voltage. When voltage levels fall to approximately 3.4 V, the µBee cannot generate suﬃcient thrust to stay aloft and will drift down to a gentle landing as a result. Conveniently, this is the voltage level below which a LiPo cell can be damaged by overdischarge. For this reason no behaviour to handle low-battery conditions was needed. Drone ﬂight performance characteristics as measured by a motion capture system is described in Chapter 6.

5.5 Flight Panel Application

Flight parameters were tuned iteratively using a custom C# .NET application, shown in Figure 5.7. This µBee Flight Panel application allows the user to send or receive all flight parameters to and from a µBee drone. The Flight Panel also allows for real-time IMU plotting (where the µBee continually transmits its orientation; this was useful for solving issues relating to orientation sensing) and a radio range test. A feature is also available for re- calibrating the µBee’s IMU. µBee ID numbers, used to address commands to specific drones, are also assigned using the Flight Panel. Where applicable, commands can be broadcast to all drones. Drone IDs range from 0 to 254, and 255 is the broadcast ID. Individual drone parameters can also be loaded from or saved to file.

5.6 Wireless Protocol

The µBee firmware uses the RFM69 interface written by Low Power Labs 12. This library provides digital packet-based wireless communication with built-in checksums, with a maximum packet length of 61 bytes. The µBee firmware implements a simple protocol on top of this functionality: all wireless messages begin with 2 bytes denoting the message type and the number of remaining bytes left in the message, followed by any additional data that is required. The µBee wireless protocol defines 13 types of messages. These are described in Table 5.3. As referenced in this table, there are 3 different parameter groups that are used to configure the µBee’s flight performance. These are given in Table 5.4. Radio commands are sent via serial to a USB device constructed specifically for the µBee drone. This device uses the same RFM69 radio as the µBee and is responsible for both re-transmitting serial commands to the drone via radio, as well as handling reception of radio messages and transmitting them via serial to the controlling application. The same device is used in the experiments described in Chapter 6 to control a small fleet of drones.

12https://github.com/LowPowerLab/RFM69

50 Figure 5.7: The µBee Flight Panel application for conﬁguring ﬂight parameters.

Message Description Additional bytes Flash LED Instruct to flash LED ID Set ID Instruct to change ID ID, and new ID Set parameters Update flight parameters ID, param group, and values (Table 5.4) Read parameters Read flight parameters ID, param group Flight command Send flight demand ID, yaw, throttle, roll, and pitch Flight commands Send flight demands Multiple flight commands Acknowledgment Data received None Non-ACK Data reception failed None Recalibrate IMU Re-level using IMU ID Start IMU TX Start TXing IMU values ID End IMU TX Stop TXing IMU values ID IMU state Denotes IMU values Pitch, roll, and yaw rate Data packet Flight param packet Param group, and values (Table 5.4)

Table 5.3: Message types deﬁned by the µBee wireless communications protocol.

51 Flight parameter group Description Bytes Levelling parameters PID parameters for pitch and roll 32 Rotation rate parameters PID parameters pitch, roll, and yaw rate 48 Response and trim Control gains; trim; compl. ﬁlter α (see Section 5.4) 24

Table 5.4: Flight parameters used to tune the µBee’s flight, by flight parameter group. All values are 4-byte floating-point.

5.7 Future Work and Conclusion

Although the µBee can be flown using a hand-held controller without the use of a motion capture system, this is difficult due to the degrees of freedom (throttle, yaw, pitch, and roll) that an operator must control. Additional sensors could be added to a future version of the µBee that remove the need to control all four of these at once. The addition of a downwards-facing infrared sensor, for example, could be used to detect and stabilize altitude automatically without constant manual correction from the operator. A digital compass could also be added to stabilize yaw more reliably. To keep costs low and the design simple, these components were not included. This chapter has described a low-cost drone for use in research and education. Other design goals included small size (85 mm on a side) and weight (25 g without a battery), simplicity of construction and maintenance, and support for motion capture systems. A fleet of 10 µBees, including spare parts and reflective tracking markers, costs approximately $1085 USD. Complete hardware and firmware specifications are available at https://gitlab.com/geoff-nagy/ubee. The real world performance of the µBee is described in the next chapter.

52 Chapter 6

Pairwise Flocking with the µBee

6.1 Introduction

Figure 6.1: 4 µBee drones ﬂying in a Vicon motion capture-equipped workspace.

Chapter 4 presented an empirical evaluation of a pairwise flocking controller, and showed how it can result in increased flock stability despite fewer tracking interactions overall. However, this evaluation was conducted in simulation and does not capture important elements that must be considered in a physical implementation. Among these include the use of imperfect sensors and real-world dynamics. This chapter presents an implementation and evaluation of the pairwise flocking controller on a team of real µBee drones (the drone is described in Chapter 5), and shows that the effects of pairwise flocking—increased flock stability and reduced tracking interactions—are present in that setting. This work also serves to demonstrate the µBee as a feasible platform for conducting multi-drone research.

53 or...

real Vicon marker motion capture data

Abstract Experiment Abstract Global Sensor simulated marker Main Interface Logic Module Module motion capture data simulated drones and world Abstract Drone Pairwise Flocking Communication simulated radio Experiment Control Interface

or... real radio

or... Figure 6.2: Organizational structure of the Drone Command Center tool.

6.2 Drone Command Center

6.2.1 Interface and Modules

The Drone Command Center (DCC) is a software application built with the goal of developing, testing, and running behaviours for teams of simulated or physical drones1. It has been designed in such a way that it can be extended to work with virtually any kind of global sensing system (e.g., a Vicon motion capture system), communications interface (e.g., digital radio), or drone behaviour. This is made possible through a variety of abstract software modules. Figure 6.2 shows the high-level organization of the DCC. The directions of the arrows in the figure denote the flow of information between connected modules. The DCC is composed of four main modules: (1) the graphical interface presented to the user, (2) an abstract module for handling drone and experiment control logic, (3) an abstract module for processing global sensor readings enabling drone detection, and (4) an abstract interface for sending flight or configuration commands to drones. The high level of abstraction means that any combination of modules can be used, and new modules of any type can be implemented to support a wide variety of behaviours, sensors, or communication schemes. The DCC is invoked on the command-line and requires three parameters:

• GUI mode. The DCC can be run headless or with a GUI enabled. In headless mode simulations can be run faster than real-time.

1https://gitlab.com/geoff-nagy/dcc

54 • World file. The path to a file specifying the environment, sensors, and communication modules to be used. Simulated drones to be spawned in the environment can be specified in the world file.

• Trial ﬁle. The path to a ﬁle specifying the behaviour module that all drones will run, and any other parameters required by the behaviours.

Configuration files are written in a custom C-esque data format called Gumdrop2 and are passed to the DCC as command-line parameters. A sample world file is shown in Listing 6.1.

Listing 6.1: An example world conﬁguration ﬁle specifying the simulated versions of modules that should be used by the DCC. world { name = "simulated_autolab" size = {11.0, 5.0}# in meters simulated = true# enable virtual world

sensor { name = "simulated_vicon"# DCC-simulated Vicon sensor size = {11.0, 5.0}# range of sensor, in meters }

comms { name = "simulated_ubee_radio"# simulated version of real radio device = ""# can be left blank in sim mode delay_ms = 100# comms delay in milliseconds }

drones { # ... } }

2https://gitlab.com/geoff-nagy/gumdrop

55 6.2.2 Simulation Accuracy

One of the goals of the DCC was to provide a simulated platform within which real drone behaviours could be developed and tested (see Section 6.2.1). It was therefore important that the performance of the simulated drones matched that of the real drones to some degree of accuracy with respect to the intended behaviours and tasks. In order to develop the pairwise flocking behaviour module for a team of real µBees, a sufficiently accurate physics model of the µBee was required for initial testing in simulation. Since the pairwise flocking behaviour does not feature any extreme or aerodynamically demanding maneuvers, several simplifying assumptions were made during the development of this model.

• No extreme maneuvers. Acrobatic motions (such as ﬂips or ﬂying sideways) or high speeds would not be required.

• Relatively constant altitude. Drones would not need to change altitude frequently 3.

• Inﬁnite battery life. Since pairwise ﬂocking trials were assumed to be 60 seconds in length or less [44], limited battery life would not be a concern. Decreasing motor output associated with lower battery voltage levels was not modelled for the same reason. (During the physical experiments described in Section 6.4.2, batteries were changed at regular intervals.)

• Collision physics were not needed. Recovery from crashes (either with the environment or other drones) was not a core element of this work. Accurate collision or recovery physics would not be necessary.

For these reasons, a simple second-order physics model was implemented. This model includes parameters for setting drone weight, number of rotors, thrust, acceleration, and velocity. Individual roll, pitch, yaw, and throttle demands were assigned by higher-level logic using the abstract communication interface, and these demands were converted into forces applied to the drone physics model to approximate the physics of rotor thrust. Air resistance was modelled as a constant that dampened velocity with each simulated time step. A simple flight task was devised to test the accuracy of the simulated model with respect to the real-world flight performance of a µBee. In both cases, the drone was flown in a fixed pattern between different waypoints modelled by 8 vertices of a rectangular prism measuring 1 m wide × 1 m long × 0.75 m tall. Flight control was achieved with two sets of PID controllers: a position-based PID controller provided error feedback based on the difference between the drone’s desired and actual position, and these outputs were fed into a second PID controller that determined the drone’s target velocity, limited to 0.4 meters

3Although the goal is not to directly imitate the flight behaviour of birds, previous work has shown that bird flocks are mostly flat anyways [5].

56 (a) Simulated ﬂight. (b) Real ﬂight.

Figure 6.3: 3D flight paths and velocities of the box flight test for (a) a simulated µBee and (b) a real µBee. per second for precision and safety. A base level of thrust—equivalent approximately to the level of thrust necessary to maintain a stable hover—was continually applied, and a minimum thrust level was enforced at all times. 3D plots of the paths and velocities of both the simulated and real drones for approximately 3 minutes each (a conservative maximum flight time when using a 300 mAh LiPo battery, as discussed in Chapter 5) are shown in Figure 6.3. Radio commands were sent to the µBee using the same USB radio interface described in Chapter 5. 3D tracking of the µBee was done using using a Vicon motion capture system. The available space measured 11 × 5 meters and was surrounded by 10 wall-mounted Vicon Bonita 10 cameras4. These cameras function by emitting infrared (IR) light, which is reflected back from diffuse tracking markers mounted on the object(s) to be tracked. The IR light reflected from these markers is detected by the Bonita 10 cameras as a series of circular blobs, and a server connected to all 10 cameras performs stereographic matching of the incoming images to resolve those reflections into 3D marker positions. The Vicon server broadcasts 3D poses for pre-defined marker configurations (or, optionally, individual marker positions) on the local network at a maximum rate of 250 Hz. Markers were mounted on the µBee in the configuration described in Section 5.3. Visual examination of the real and simulated flights paths shows that there is a rea- sonable level of agreement with respect to the target path and velocity (0.4 m/s) between the simulated and real µBee. No parameter changes (including PID gains) in the box flight behaviour module were made in between simulated and real flights. This indicates that despite the physics limitations of the simulated µBee model mentioned above, the simulation

4The lab space actually measures 13 × 7 meters, but the outer 1 meter perimeter is not captured by the Vicon.

57 provides qualitatively similar performance to that of a real µBee within the context of the box flight test. Since the frequent changes of altitude made this test more demanding than the pairwise flocking experiments, the accuracy of the simulator was deemed acceptable for testing the pairwise flocking behaviours.

6.3 Veriﬁcation of Pairwise Flocking Correctness

6.3.1 Pairwise Flocking Controller Port

The pairwise ﬂocking controller from Chapter 4 was ported into the DCC as an additional behaviour module with changes to accommodate actions such as take-oﬀ, altitude stabilization, and landing. An additional set of enhancements, partly inspired by directions for future work (see Section 4.5.1), was also implemented:

• Pairs can become separated. In the original implementation, paired agents could never lose track of one another. This assumption has been removed. In this implementation, paired agents which lose sight of one another revert to tracking 7 nearest neighbours (i.e., they act like single agents) until their mate is located again. No eﬀorts are made by separated members of a pair to search for their mate.

• Obstacles can obscure agent fields of view. This behaviour was not present in the original implementation; obstacles would not prevent agents from establishing tracking connections by blocking lines of sight. Allowing obstacle occlusion adds a greater challenge to flock stability as an additional source of potentially broken connections between agents. Obstacles are made of a larger “soft” radius that exerts a repulsive force, and a smaller “hard” radius that occludes agents’ fields of view.

• Agent view range can be varied. This was implemented to examine the effect that agent view range—beyond which connections to other agents are not made—has on the stability of the flock and the effectiveness of pairwise flocking.

Before physical experiments were conducted, experiments in simulation were run to verify that the DCC version of the pairwise flocking behaviour worked correctly with respect to the goals of the original implementation. These goals were: (a) greater flocking stability, and (b) fewer agent connections overall, as the number of paired agents in a flock increased. The simulated environment implemented in the DCC was similar to the one described in Chapter 4. The number and distribution of single versus paired agents was also identical (44 agents, with 0, 5, 11, 17, or 22 paired agents present). The environment was unbounded and infinite, and contained an 11 × 11 grid of obstacles, each cell spaced 2 meters apart, which the agents attempted to avoid by steering around them. Drones were placed randomly within 1 meter cells in an 8 × 8 grid centered around the origin. Paired drones were spawned in adjacent cells. A custom Python script generated 6,000 trial files, as outlined in Table 6.1.

58 A second Python script was used to run multiple instances of these trials concurrently, with the option of stopping and resuming trial execution on demand.

Parameter Values Count Number of pairs 0, 5, 11, 17, 22 5 Close pairing (CP ) false, true 2 Asymmetric tracking (AT ) false, true 2 View range 1.5 m, 2 m, 2.5 m, 3 m, 3.5 m, 4 m 6 Trials per conﬁguration 50 50 Total count 6,000

Table 6.1: Breakdown of the number of trials for the experiments used to verify the correctness of the pairwise ﬂocking behaviour implemented in the DCC.

6.3.2 Results

Figure 6.4 shows the average number of sub-flocks present at the conclusion of the experiments, grouped by the number of agents initially paired in the flock, with an agent view range of 400 cm. A one-way ANOVA (F = 17.67, p « 0.01) indicated a statistically significant difference between at least one group. To determine between which groups a statistically significant difference existed, a post-hoc Tukey’s multiple comparison of means with a statistical significance level of α= 0.05 was run. This test revealed a statistically significant difference between the 0 and 11; 0 and 17; 0 and 22; 5 and 11; 5 and 17; and, 5 and 22 paired groups. These results show that the number of sub-flocks at the end of each trial decreases, on average, as the number of paired agents increases. Thus, a higher percentage of initially-paired agents corresponds to greater flock stability. Figure 6.5 shows the average number of total connections between agents present at the conclusion of the experiments, grouped by the number of agents initially paired in the flock, with an agent view range of 400 cm. A one-way ANOVA (F = 12060.15, p « 0.01) indicated a statistically significant difference between at least one group. A post-hoc Tukey’s multiple comparison of means with a statistical significance level of α= 0.05 revealed statistically significant differences between all groups. These results show that the number of connections between agents at the end of each trial decreases, on average, as the number of paired agents increases. Note that despite the fewer number of tracking interactions as the number of paired agents increases, the flocks experience greater stability (as shown in the previous paragraph). Figure 6.6 shows histograms depicting the number of sub-flocks at the conclusion of each trial, with different combinations of the AT and CP flags, to show the effects that asymmetric tracking and close pairing have on the stability of the flock. These results are similar to those in Section 4.4, and show the positive effect that asymmetric tracking and close pairing have on flock stability when the number of paired agents in the flock

59 Sub Flocks at Trial End (View Range 400 cm) 1.75

1.50

1.25

1.00

0.75 Number of Sub Flocks 0.50

0.25

0.00 0 5 11 17 22 Number of Paired Agents

Figure 6.4: Average number of sub-flocks present at the conclusion of each trial with 0, 5, 11, 17, 22 paired agents. Error bars denote 95% confidence intervals. increases. With these flags disabled, flock stability decreases as the number of paired agents increases, since in those settings paired agents are further away from their mates and are not guaranteed to track mutually-exclusive sets of neighbours. As discussed previously in Section 4.5, this results in reduced useful information being shared between paired agents since they may end up tracking some partners in common.

6.4 Physical µBee Experiments

6.4.1 Flying Multiple µBees

This section describes the pairwise flocking experiments on a team of real µBee drones. Although the previous simulations of pairwise flocking were performed with 44 drones and an infinitely large environment, factors such as limited physical space, the number of available drones, and hardware capabilities meant that a smaller-scale experiment was needed to test the effectiveness of pairwise flocking. A limitation of the proprietary marker detection algorithm used by the Vicon system is that it will not detect multiple instances of the same predefined marker configuration. Early tests also revealed that defining multiple instances of the same marker configuration

60 Total Number of Connections (View Range 400 cm)

300

250

200

150

100 Number of Connections

0 0 5 11 17 22 Number of Paired Agents

Figure 6.5: Average number of total connections between agents at the conclusion of each trial with 0, 5, 11, 17, 22 paired agents. Error bars denote 95% confidence intervals. will not result in reliable detections where multiple instances of those configurations are present. The limited space available for marker placement on each µBee also placed limits on the number of unique predefined marker configurations that could be constructed. For this reason, a custom detection algorithm was written that would be capable of detecting multiple instances of a simple triangular marker configuration. This configuration, and how drone poses are computed, is shown in Figure 6.8. Because each drone would use the same marker configuration, it was necessary to write a tracking system that would associate these detections with µBee ID numbers over time. This detection and tracking algorithm is scalable to any number of µBee markers. An overview of the detection and tracking approach is shown in Figure 6.9. The Vicon system provides raw marker detection data, which is resolved into µBee detections. These detections are passed into a tracking system which is responsible for associating drone IDs with marker detections over time. It does this by matching each new detection with the closest detection recently discovered nearby with a known ID, within a certain distance limit. Any drone continuously not matched within 0.4 meters from its last successful detection for 3 seconds is considered lost. The value 0.4 meters was chosen because this represents roughly the minimum safe distance between two drones before collision or air disturbance becomes a

61 fdoe eysml o h srsnei sbsdo eaie ahrta boue initial absolute, than rather relative, on based is it since user positions the drone for simple very configuration drones 2-dimensional association any trial of and in the placement order, makes in approach that IDs This in environment top-to-bottom). of (left-to-right, the list reading-order in a placed system are tracking themselves dominating drones the became The to of file. providing disturbance association by Initial air done experiments. or is during collision IDs occurred drone testing that never IDs close during of so determined Mis-association ranges factors. was at risk IDs which only currents drone possible air of induce be association can to nearby Incorrect flying flight). drones affect two that adversely revealed tests flight (early risk of values varying for agents. trial, paired every after of sub-flocks of Number 6.6: Figure utpedoe erya oe liue asdsgicn i disturbance. air significant caused altitudes lower at nearby drones multiple once. only made was mistake This crashes. proce- entire the experiments during throughout The altitude meters meters. target 0.5 0.5 drones’ approximately the approximately matching of of roughly height resolution dure, 11 a a entire at with the suspended drone through pattern was moved lawnmower A drone manually a workspace. was in and Vicon-equipped space rod the meter horizontal 5 meter in 2 spots” a “blind to attached potential was any reveal and rithm

6 5 ∼ CP , ∼ AT CP , ∼ AT ∼ CP , AT CP , AT aeattd f07mtr a sdi h culeprmnswe twsdsoee htflying that discovered was it when experiments actual the in used was meters 0.7 of altitude catastrophic in base result will A ordering ID provided their to respect with drones the place correctly to Failure ipets a eie ovsal a h ffcieeso h e eeto algo- detection new the of effectiveness the map visually to devised was test simple A

Frequency Frequency Frequency Frequency 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 0 0 0 0 Sub-Flocks (Pairs=0,AT=false,CT=false) Sub-Flocks (Pairs=0,AT=false,CT=true) Sub-Flocks (Pairs=0,AT=true,CT=false) Sub-Flocks (Pairs=0,AT=true,CT=true) 1 1 1 1 %Pie 7 ard5%Pie 3 ard10 Paired 100% Paired 73% Paired 50% Paired 27% Paired 0% 2 2 2 2 Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks 3 3 3 3 4 4 4 4 5 5 5 5 5 . 6 6 6 6 7 7 7 7

Frequency Frequency Frequency Frequency 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 0 0 0 0 Sub-Flocks (Pairs=5,AT=false,CT=false) Sub-Flocks (Pairs=5,AT=false,CT=true) Sub-Flocks (Pairs=5,AT=true,CT=false) Sub-Flocks (Pairs=5,AT=true,CT=true) 1 1 1 1 2 2 2 2 Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks 3 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7 7 7

Frequency Frequency Frequency Frequency 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 0 0 0 0 Sub-Flocks (Pairs=11,AT=false,CT=false) Sub-Flocks (Pairs=11,AT=false,CT=true) Sub-Flocks (Pairs=11,AT=true,CT=false) Sub-Flocks (Pairs=11,AT=true,CT=true) 1 1 1 1 2 2 2 2 62 Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks 3 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7 7 7

Frequency Frequency Frequency Frequency 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 0 0 0 0 Sub-Flocks (Pairs=17,AT=false,CT=false) Sub-Flocks (Pairs=17,AT=false,CT=true) Sub-Flocks (Pairs=17,AT=true,CT=false) Sub-Flocks (Pairs=17,AT=true,CT=true) 1 1 1 1 2 2 2 2 Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks 3 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 6 AT nadtoa DCC additional An . 7 7 7 7 ,

CP Frequency Frequency Frequency Frequency 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 0 0 0 0 Sub-Flocks (Pairs=22,AT=false,CT=false) Sub-Flocks (Pairs=22,AT=false,CT=true) Sub-Flocks (Pairs=22,AT=true,CT=false) Sub-Flocks (Pairs=22,AT=true,CT=true) 1 1 1 1 n number and , 2 2 2 2 Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks Number ofSub-Flocks 3 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7 7 7 × Figure 6.7: 10 µBee drones sitting in their handy 3D-printed carrying case. behaviour divided the workspace into an 11 × 5 meter grid with a cell size of 0.5 meters, and for each drone detection that occurred in a cell, incremented a counter for that cell. The resulting map is shown in Figure 6.10. Red cells denote few or no detections, yellow denotes intermediate counts, and green cells indicate high detection counts. The results indicate that drone detection quality within the entire Vicon workspace (with the exception of all 4 corners) at a height of approximately 0.5 meters was likely to be sufficiently high with respect to the experiments to be performed. For the physical pairwise flocking experiments described in the following section, detection and ID association occurred at a rate of 200 Hz. Although the Vicon system is capable of detection rates as high as 250 Hz, in practice this resulted in a significant number of missed frames, possibly due to network congestion. To avoid overburdening the network a slightly lower update rate was used. Agent logic updates and radio commands were issued from the USB radio at a rate of 30 Hz. Hardware limitations prevented the possibility of faster radio updates, and limited the number of drones that could be flown at once. While the fastest reliable serial data rate of 38400 baud was used (communication tests revealed that baud rates faster than this were extremely unreliable), and a radio packet transmission rate of approximately 20 ms per 64-byte packet (50 Hz) was possible, in practice the interaction of both of these communication schemes in the same pipeline resulted in communication rates slower than this. Faster baud rates may be possible in future work with the integration of a faster microcontroller powering the USB radio device, which was limited to 8 MHz in its

63 Figure 6.8: Shows the new triangular marker configuration and how position and orientation vectors are computed. A clustering algorithm finds groups of 3 markers whose pairwise edge lengths fall into defined ranges. The average position of markers A, B, and C yield the center position of the drone. The left vector is computed as the unit vector from marker C to B. The forward vector is computed as the unit vector between the average marker position and marker A. The up vector comes out of the page and is defined as the cross product of the left and forward vectors. current configuration. For example, a 20 MHz crystal, e.g., the ZTT-20.00MX7, may help to further improve possible communication speeds.

6.4.2 Experiment Setup

Although the pairwise flocking experiments described previously in this thesis (Sections 4.1 and 6.3) featured unbounded environments, experiments with physical drones would need to be conducted in the 11 × 5 meter Vicon space. For this reason, and because of the communications limitations described in the previous section, the number of drones was reduced to 4. Early tests with various obstacle configurations (and repulsive walls designed to keep agents confined) revealed that even with pairwise flocking disabled, the flock was unlikely to break up in an environment this size. Inspired by a combination of previous work on flock stability [4, 7], and the function of a Galton board (Figure 6.11), a different flocking task was devised in which the flock would travel from one end of the environment to the other. A wedge-shaped group of obstacles placed on the far side would exert forces on the flock causing it to bisect and break into smaller pieces. An example figure showing the placement of drones and obstacles in a sample trial is shown in Figure 6.12.

7https://www.digikey.ca/en/products/detail/ecs-inc/ZTT-20-00MX/124531

64 Sensing module

simulated Vicon marker detections

uBee tracking and ID uBee marker detection Behaviour module or assocation

real Vicon marker detections

Figure 6.9: Visual outline of the ﬂow of detected marker data in the DCC, including both real and simulated markers.

Figure 6.10: The drone detection quality map displayed in the DCC.

The majority of the workspace concrete floor was covered in at least one layer of car- peting or blankets to reduce damage caused by crashes. Drones were placed on flat takeoff surfaces in a rectangular configuration. For each trial, drone starting positions varied no more than 2.5 cm from their desired positions indicated in Figure 6.12. At the beginning of each trial, the drones would take off and begin flocking. All drones were subject to an “eastwards” force, analogous to gravity on a Galton board, that would encourage the drones to fly towards the far side of the lab. As described in Section 6.3.1, obstacles would exert a repulsive force inside the partially-transparent portion of the obstacle, and the solid portions of each obstacle would also occlude drones’ fields of view and prevent tracking interactions. Drones had a maximum view range of 1.5 meters. Horizontal drone speeds were fixed at 0.2 meters per second for safety. Trials were ended manually, and the drones commanded to land, when any drone reached within 1 meter of the far east wall, which was visually indicated by the DCC GUI. Because the µBee possesses no sensors other than an IMU, all

65 Figure 6.11: A Galton board. Svjo, CC BY-SA 3.0, https://creativecommons.org/ licenses/by-sa/3.0, via Wikimedia Commons computations, drone flocking logic, and neighbour sensing were conducted off-board using the DCC tool. (In essence, the drones flew blind as commanded by the flocking behaviour module.) To reduce the risk of collisions, each drone maintained a base altitude of 0.7 meters plus an individual offset of (0.1 × i) meters for the ith drone. To reduce the effects of flight performance variations caused by differences in construction or battery charge, drones were rotated by one position in a clockwise manner after every successful trial. Where pairwise flocking was enabled, pairs were intentionally assigned such that they were always at the greatest risk of being split by the wedge-shaped formation of obstacles. In Figure 6.12, for example, drone 8 would be paired with 7, and 3 with 5. Crashes, although infrequent, did occasionally occur due to tracking loss, loss of control due to low battery, or other malfunctions (such as a prop becoming detached). Batteries were swapped and charged on a rotating basis in groups of 4, including a cool-down period before charging. Trials typically lasted between 30 and 40 seconds. To reduce the effects of low battery power, batteries were swapped after every 3 trials. However, variations in battery performance as well as wear on batteries due to repeated use became an issue in later trials, and batteries were swapped more frequently as experiments progressed.

6.4.3 Experiment #1

An experiment was conducted with two configurations: one in which no drones were paired, and another in which all drones were paired. Pairs were assigned in the manner described in the previous section. For every configuration, 12 trials were run. This allowed 3 complete rotations of drone starting positions. Statistics such as the number of sub-flocks and

66 Figure 6.12: Placement of the drones and obstacles in the 11 × 5 meter Vicon space.

Figure 6.13: Timelapse of a run with no drones paired, at 0, 10, 20, and 30 seconds into the trial. Note that a single drone has escaped the flock. Single drones departing the flock was typical for this configuration. the number of connections were recorded by the DCC every second. Figure 6.15 shows a histogram of the number of sub-flocks at the end of each trial, for each of the two configurations.

Results

The mean number of sub-flocks at the end of every non-paired configuration was 2.33 (S = 0.943), while the mean number of sub-flocks at the end of every paired configuration was slightly higher, at 2.58 (0.759). Levene’s test indicated equal variances (F = 1.09, p = 0.31). An independent t-test for equal variances indicated no significant difference between the two conditions (t = -0.68, p = 0.50). The total number of connections maintained during every non-paired trial was 6.48 (3.30), and the total number of connections maintained during the pairwise trials was 4.02 (1.36). Levene’s test indicated unequal variances (F = 287.2, p « 0.001) between the two configurations, and an independent t-test for unequal variances indicated a statistically significant difference between them (t = 13.627, p « 0.001).

67 Figure 6.14: Timelapse of a run with 2 drones paired, at 0, 10, 20, and 30 seconds into the trial.

Discussion

These results indicated that in the lab setting, pairwise flocking appeared to have no significant effect on flock stability, although the number of connections maintained by members of the flock was reduced. This is an interesting result because this is the first scenario in which pairwise flocking appears to have no significant effect on flock stability. This may have been due to the intentionally adversarial experiment setting: the eastwards force on the drones, combined with the formation of obstacles designed specifically to split up a flock, might have been enough to overcome the advantages originally associated with pairwise flocking. To test this theory, a second experiment was designed. It is described in the following section.

6.4.4 Experiment #2

In the second experiment, paired drones were given the ability to catch up to their partner if they became separated by too great a distance. When members of a pair were separated by more than 1.0 meters, they ignored social forces from all other interaction partners except their assigned mate. The eastwards force was also ignored when catching up. This was designed to ease the adversarial conditions of the environment and allow pairs to re- main intact, while not directly affecting the behaviour of other unpaired flock members. This behaviour was directly inspired by observations of real corvid flocks, in which pairs experience an elastic social force that keeps them together. The overall intended effect is a strengthening of the social forces that keeps pairs connected. The results from the original two configurations (pairwise flocking disabled, pairwise flocking enabled) were used as controls against which to judge the effectiveness of pairwise flocking with the new catch-up behaviour enabled.

Results

Figure 6.16 shows a histogram of the number of sub-flocks at the end of all three configurations for comparison. Table 6.2 shows the resulting mean and standard deviations of the number of sub-flocks. Notably, the mean number of sub-flocks in the pairwise catch-up configuration is lower than in all other configurations. Levene’s test indicates unequal variances between the non-pairwise and pairwise with catch-up configurations (F = 9.14, p = 0.006).

68 Number of Sub-Flocks 10 No Pairs 2 Pairs

Frequency 4

0 1 2 3 4 Number of Sub-Flocks

Figure 6.15: Histogram of the number of sub-ﬂocks present at the end of each trial, for non-paired and paired conﬁgurations.

Conﬁguration Mean Std. dev. No pairs 2.33 0.94 2 pairs 2.58 0.76 2 pairs catch-up 2.08 0.49

Table 6.2: Average number of sub-ﬂocks at the conclusion of each trial.

An independent t-test for unequal variances indicated no statistically significant difference between them (t = 0.78, p = 0.45). Table 6.3 shows the average number of connections maintained in all configurations. Levene’s test indicated unequal variances (F = 484.3, p « 0.001) between the non-paired and paired catch-up configurations, and an independent t-test showed a statistically significant difference between them (t = 13.72, p « 0.001). Table 6.4 shows the average length of time it took to complete the trials for each configuration. Note that this statistic was not reported in previous experiments because those trials were intentionally designed to have a fixed length. The µBee trials, on the other hand, only ended when a drone reached the far side of the lab. Levene’s test revealed unequal variances (F = 4.81, p < 0.04) between the non-paired and paired catch-up configurations.

69 Number of Sub-Flocks 10 No Pairs 2 Pairs 2 Pairs with Catch-up 8

Frequency 4

0 1 2 3 4 Number of Sub-Flocks

Figure 6.16: Histogram of the number of sub-ﬂocks present at the end of each trial, for non-paired, paired, and paired catch-up conﬁgurations.

Conﬁguration Mean Std. dev. No pairs 6.48 3.30 2 pairs 4.02 1.36 2 pairs catch-up 4.10 1.01

Table 6.3: Average number of tracking connections during all trials, grouped by conﬁgura- tion.

Conﬁguration Mean Std. dev. No pairs 32.66s 2.49s 2 pairs 32.08s 3.42s 2 pairs catch-up 37.42s 5.87s

Table 6.4: Average length of each trial, grouped by conﬁguration.

An independent t-test for unequal variances indicated a statistically signiﬁcant diﬀerence between them (t = -2.47, p < 0.03).

70 Discussion

The results of the second experiment indicate that pairwise flocking performed much more effectively with the catch-up behaviour enabled. The flocks were less likely to break up in this configuration than without the catch-up behaviour. Also, the number of connections maintained overall was lower compared to when pairwise flocking was not used. In the vast majority of the trials with pairwise catch-up enabled, flocks were most likely to split in two (as seen in Figure 6.16). Also, unlike the previous two configurations, the flock never broke up into more than 3 sub-flocks. Although maintaining a single flock structure is clearly more desirable, maintaining two equally-sized sub-flocks is an outcome that is preferable to multiple sub-flocks (or single agents) leaving the main flock, which occurred regularly when the catch-up behaviour was not enabled. Visual observation confirmed that in all cases where the pairwise flocking catch-up behaviour was active and the flock split into two sub-flocks, both pairs were still intact. This is also desirable because the forces joining pairs are effectively stronger due to the catch-up behaviour, and are less likely to break up further than if those agents were joined only by regular non-paired tracking connections. This implies that maintaining unbroken connections between pairs is a critical requirement if pairwise flocking is to be used to improve flock stability. The results also show that the time to complete the trials was longest in the pairwise catch-up configuration than the non-pairwise configuration. This is likely due to the fact that pairs which may have otherwise become separated instead performed backtracking maneuvers to ensure that they did not lose their mate. This is also in agreement with visual observation of drone behaviours during the experiments.

6.4.5 Overall Discussion

The results from the physical drone experiments show that pairwise flocking succeeds in maintaining or improving flock stability, while reducing the number of tracking interactions between agents, compared to conditions where pairwise flocking was not used. The flocking scenario described here—the wedge-shaped obstacle formation, combined with the eastwards force—made it difficult for the drones to maintain a single formation in all cases. Issues associated with real-world sensors were also present: failed drone detections due to imperfect sensing and missed Vicon data frames all contributed additional challenges. A final challenge is that of real-world dynamics and control: drones, like any robot system, are subject to the laws of physics and have inherent limitations associated with actuation and control. Pairwise flocking, for example, works best when connections between pairs are maintained, and so the physical control outputs of the drones must be able to accommodate actions that allow this. These experiments also help to demonstrate the scalability of pairwise flocking. While experiments described in previous sections (4.3.2, 6.3) show the effectiveness of pairwise

71 flocking for large simulated flocks, the results presented here show that pairwise flocking works as intended on real, smaller flocks. For these reasons, it is likely that a pairwise flocking implementation on a larger fleet of physical drones would see the same advantages as well. Additionally, these results show that pairwise flocking can be useful in two different settings: an environment with regularly-spaced obstacles (Section 6.3), and an environment with a constant force—such as wind—pushing the flock against a group of obstacles (Sec- tion 6.4.2). It is possible that pairwise flocking may have applications in other settings as well where flock stability or limited computational resources available for partner tracking are important factors. A YouTube video showing the pairwise flocking is available at https://youtu.be/qpV_ P0xzJcI.

6.4.6 Limitations and Future Work

Although these experiments successfully demonstrate the advantages and feasibility of pairwise flocking in different scenarios, a number of limitations should be addressed in future work. The first of these limitations was the use of global sensing. A more realistic study would involve the use of on-board sensing, i.e., each drone would have sensor(s) for detecting nearby interaction partners. A second limitation was available hardware and lab space; this placed constraints on the scale of the physical experiments, which used 4 drones in an 11 × 5 meter space. Performing sensing and flocking logic on-board would also remove the need for a centralized approach altogether, lifting the limitations associated with motion capture technology and its limited space (potentially allowing evaluation outdoors) and the speed of radio commands to the drones.

6.5 Evalution of the µBee

In Chapter 5, the µBee was introduced as a micro-scale drone designed with a number of goals in mind: small size and weight, low cost, simplicity of repair and construction, support for motion capture-controlled ﬂight, and educational value. These goals are evaluated below with respect to the drone’s performance during the experiments described above.

• Low cost. While the low cost of $1,085.45 USD for a ﬂeet of 10 drones and a number of spare parts (not including shipping and taxes) makes the µBee attractive price- wise, maintenance was an ongoing issue. The most common issues included broken propellers or damaged motors due to crashes, and it became necessary to purchase additional replacement parts. Protective netting or softer padding would have greatly reduced the damage caused by crashes. An alternative solution would have been to develop 3D-printable propeller guards, but this was not done due to weight considerations. The motors also tended to get very hot during operation, contributing to wear and tear that eventually resulted in motors needing to be replaced; in the future, the

72 viability of lower-powered motors with a similar form factor should be evaluated. All of these factors made maintenance an additional unforeseen cost. The cost of additional propellers and motors totaled $274.79 USD, although this included more spare propellers than were needed. Nearly all spare motors were used. Additional work should be done to determine if the µBee is more cost-eﬀective than other more expensive (but less maintenance-costly) drones, for various ﬂeet sizes.

• Small size and weight. The drone’s weight with the final triangular marker configuration and 300mAh LiPo battery was 37 g. This is sufficient for 3.5 minutes of sustained flight, although batteries were changed more frequently than this during trials to reduce the possibility of performance issues caused by low batteries.

• Simplicity of repair and construction. As described in Section 6.4.2 crashes, though infrequent, did occur. (The exact number was not recorded.) As a result, propellers or motors needed to be replaced on occasion. The simple design of the µBee meant that repairs were quick and could be performed without any specialized tools: damaged motors could be slid out from their mounts, and propellers could be removed and press-fit onto motor shafts by hand. The frames and PCBs themselves were sturdy: only the frame landing gear needed repairs (once), and the PCBs and their components were undamaged. Marker mounts were damaged on two occasions, but the application of hot glue was sufficient. It was not necessary to 3D print any replacement parts. If a particular repair was deemed too difficult to investigate (occasionally motor failures were difficult to debug, in terms of which motor(s) had failed), a spare µBee was substituted. For this reason, it was convenient to have multiple spare µBees on hand.

• Support for motion capture attachments. The triangular marker configuration, combined with the detection and tracking algorithms, proved to work reliably enough for a small fleet of drones to be controlled in the Vicon-equipped workspace. The attachments and 9.5 mm reflective markers contributed approximately only 3 g of additional weight.

• Educational value. An additional goal was to build a drone that would be useful to educators teaching robotics or real-time systems. This was not evaluated in this thesis. A formal evaluation of the drone’s value with respect to an undergraduate curriculum, perhaps by future students working with the µBee, still needs to be conducted in order to be convinced of its educational value.

6.6 Conclusion

This chapter described an evaluation of a pairwise flocking implementation on a small fleet of real µBee drones. These experiments demonstrated that pairwise flocking increases the

73 stability of a real ﬂock while reducing the total number of connections between drones. This chapter also demonstrated the feasibility of the µBee as a small, low-cost drone that can be used in a multi-drone research setting.

74 Chapter 7

Conclusion

The mechanisms underlying the fascinating displays of flocking behaviour are still unknown. Although many attempts have been made to synthetically generate collective behaviours that resemble flocking to some degree, recent studies have shown that commonly held assumptions regarding agent homogeneity are incorrect. (For example, in mixed-species corvid flocks of jackdaws and rooks, the socially-dominant rooks are more likely to be at the lead- ing edge of the flock [26].) This exposes a weakness in current generative flocking models, which generally assume that all agents obey the same set of rules. This also emphasizes the importance of including information such as individual-level species classification when studying the behaviour of real flocks in the field. Technology used in current field studies of flocks provides positional information only. Chapter 2 provided an overview of previous work covering both studies of flocking, schooling, and swarming behaviour in general, as well as attempts to imitate this behaviour in simulation. Chapter 3 introduced a novel tool for generating mass quantities of synthetic flocking data, which includes such information as agent species, for the purpose of training a deep neural network to learn individual-level attributes such as flap phase, species, or distance from a camera. Ideally, a properly-trained neural network should be able to extract this information from sequences of flocking images to provide a much richer set of data for statistical analysis with respect to individual flocking behaviours and flock heterogeneity in general. While a quantitative evaluation of the BirdGen tool is not available at the time of writing, qualitative analysis of the generated images shows that it produces output that has many features in common with actual flocks. Chapter 4 explored another aspect of flock heterogeneity: the tendency of certain species of birds to flock in stable pairs, and how this behaviour can be leveraged in a flocking controller to increase the stability of the flock despite maintaining fewer overall tracking connections between agents. Experiments conducted on simulated flocks indicated that flocks are more likely to break up into fewer components as the number of paired agents increases. While no claims are made about the underlying mechanisms or advantages of this behaviour

75 in real flocks, these experiments have shown that this engineered pairwise flocking behaviour possesses these benefits. The remainder of this thesis evaluated the engineered pairwise flocking controller on a team of real drones on a smaller scale. Chapter 5 introduced a novel low-cost drone called the µBee, designed to meet various requirements regarding small size, low weight, and simplicity with respect to repair, maintenance, and overall design. As described in Chapter 6, a small fleet of µBee drones running the pairwise flocking controller also exhibits the same advantages as seen in simulation, although on a smaller scale: a tendency for the flock to break up into fewer pieces when pairwise flocking is enabled, despite maintaining fewer tracking interactions overall. These experiments demonstrated that the advantages of pairwise flocking previously seen in simulation are also present in a flock of drones where sensing is imperfect and agents are subject to real-world dynamics.

76 Bibliography

[1] Nicole Abaid, Tiziana Bartolini, Simone Macrì, and Maurizio Porfiri. Zebrafish re- sponds differentially to a robotic fish of varying aspect ratio, tail beat frequency, noise, and color. Behavioural brain research, 233(2):545–553, 2012. [2] Alaa Eldin Abdelaal, Maram Sakr, and Richard Vaughan. Lost highway: a multiple- lane ant-trail algorithm to reduce congestion in large-population multi-robot systems. In 2017 14th Conference on Computer and Robot Vision (CRV), pages 161–167. IEEE, 2017. [3] A-J Baerveldt and Robert Klang. A low-cost and low-weight attitude estimation system for an autonomous helicopter. In Proceedings of IEEE International Conference on Intelligent Engineering Systems, pages 391–395. IEEE, 1997. [4] Michele Ballerini, Nicola Cabibbo, Raphael Candelier, Andrea Cavagna, Evaristo Cis- bani, Irene Giardina, Vivien Lecomte, Alberto Orlandi, Giorgio Parisi, Andrea Procac- cini, et al. Interaction ruling animal collective behavior depends on topological rather than metric distance: Evidence from a field study. Proceedings of the national academy of sciences, 105(4):1232–1237, 2008. [5] Michele Ballerini, Nicola Cabibbo, Raphael Candelier, Andrea Cavagna, Evaristo Cis- bani, Irene Giardina, Alberto Orlandi, Giorgio Parisi, Andrea Procaccini, Massimiliano Viale, et al. Empirical investigation of starling flocks: a benchmark study in collective animal behaviour. Animal behaviour, 76(1):201–215, 2008. [6] Adam Bignell, Lily Li, and Richard Vaughan. Open-loop collective assembly using a light field to power and control a phototaxic mini-robot swarm. In 2019 International Conference on Robotics and Automation (ICRA), pages 5851–5857. IEEE, 2019. [7] Marcelo Camperi, Andrea Cavagna, Irene Giardina, Giorgio Parisi, and Edmondo Sil- vestri. Spatially balanced topological interaction grants optimal cohesion in flocking models. Interface focus, 2(6):715–725, 2012. [8] Jianing Chen, Melvin Gauci, Michael J Price, and Roderich Groß. Segregation in swarms of e-puck robots based on the brazil nut effect. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1, pages 163–170. International Foundation for Autonomous Agents and Multiagent Sys- tems, 2012. [9] Anders Lyhne Christensen, Rehan OGrady, and Marco Dorigo. From fireflies to fault-tolerant swarms of robots. IEEE Transactions on Evolutionary Computation, 13(4):754–766, 2009.

77 [10] Iain D Couzin and Nigel R Franks. Self-organized lane formation and optimized traﬃc ﬂow in army ants. Proceedings of the Royal Society of London. Series B: Biological Sciences, 270(1511):139–146, 2003.

[11] Marco Dorigo. Optimization, learning and natural algorithms. PhD Thesis, Politecnico di Milano, 1992.

[12] Marco Dorigo. The ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics-Part B, 26(1):1–13, 1996.

[13] Marco Dorigo and Luca Maria Gambardella. Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Transactions on evolutionary computation, 1(1):53–66, 1997.

[14] Th Dracos. Particle tracking in three-dimensional space. In Three-dimensional velocity and vorticity measuring and image analysis techniques, pages 209–227. Springer, 1996.

[15] David Eigen and Rob Fergus. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In Proceedings of the IEEE international conference on computer vision, pages 2650–2658, 2015.

[16] Jolyon J Faria, John RG Dyer, Romain O Clément, Iain D Couzin, Natalie Holt, Ashley JW Ward, Dean Waters, and Jens Krause. A novel method for investigating the collective behaviour of ﬁsh: introducing ‘roboﬁsh’. Behavioral Ecology and Sociobiology, 64(8):1211–1218, 2010.

[17] Simon Goss, Serge Aron, Jean-Louis Deneubourg, and Jacques Marie Pasteels. Self- organized shortcuts in the argentine ant. Naturwissenschaften, 76(12):579–581, 1989.

[18] Plerre-P Grassé. La reconstruction du nid et les coordinations interindividuelles chezbellicositermes natalensis etcubitermes sp. la théorie de la stigmergie: Essai d’interprétation du comportement des termites constructeurs. Insectes sociaux, 6(1):41–80, 1959.

[19] Roderich Groß, Stéphane Magnenat, and Francesco Mondada. Segregation in swarms of mobile robots based on the brazil nut eﬀect. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4349–4356. IEEE, 2009.

[20] Tyler Gunn and John Anderson. Dynamic heterogeneous team formation for robotic urban search and rescue. Journal of Computer and System Sciences, 81(3):553–567, 2015.

[21] Christopher Hartman and Bedrich Benes. Autonomous boids. Computer Animation and Virtual Worlds, 17(3-4):199–206, 2006.

[22] Hope Microelectronics. RFM69 ISM Transceiver Module [DATASHEET], 2006. Available at https://www.hoperf.com/data/upload/portal/20191105/ RFM69%20Specification.pdf, Rev: V1.0.

[23] Sota Inoue, Shinya Yamamoto, Monamie Ringhofer, Renata S Mendonça, Carlos Pereira, and Satoshi Hirata. Spatial positioning of individuals in a group of feral horses: a case study using drone technology. Mammal Research, pages 1–11, 2018.

78 [24] InvenSense. MPU-6000 and MPU6050 Product Speciﬁcation [DATASHEET], 2013. Available at https://invensense.tdk.com/wp-content/uploads/2015/02/ MPU-6000-Datasheet1.pdf, Rev: 3.4.

[25] Li Jiang, Luca Giuggioli, Andrea Perna, Ramón Escobedo, Valentin Lecheval, Clément Sire, Zhangang Han, and Guy Theraulaz. Identifying inﬂuential neighbors in animal ﬂocking. PLoS computational biology, 13(11):e1005822, 2017.

[26] Jolle W Jolles, Andrew J King, Andrea Manica, and Alex Thornton. Heterogeneous structure in mixed-species corvid ﬂocks in ﬂight. Animal Behaviour, 85(4):743–750, 2013.

[27] Mark Joselli, Erick Baptista Passos, Jose Ricardo Silva Junior, Marcelo Zamith, Este- ban Clua, Eduardo Soluri, and Nullpointer Tecnologias. A ﬂocking boids simulation and optimization structure for mobile multicore architectures. Proceedings of SBGames, pages 83–92, 2012.

[28] Douglas H Kelley and Nicholas T Ouellette. Emergent dynamics of laboratory insect swarms. Scientiﬁc reports, 3:1073, 2013.

[29] Jens Krause, Graeme D Ruxton, Graeme D Ruxton, Ibls Graeme Ruxton, et al. Living in groups. Oxford University Press, 2002.

[30] C Ronald Kube and Eric Bonabeau. Cooperative transport by ants and robots. Robotics and autonomous systems, 30(1-2):85–101, 2000.

[31] Yevhen Kuznietsov, Jorg Stuckler, and Bastian Leibe. Semi-supervised deep learning for monocular depth map prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6647–6655, 2017.

[32] Kenneth Lai, Xinyuan Tu, and Svetlana Yanushkevich. Dog identiﬁcation using soft biometrics and neural networks. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2019.

[33] H Ling, G E McIvor, K van der Vaart, R T Vaughan, A Thornton, and N T Ouellette. Costs and beneﬁts of social relationships in the collective motion of bird ﬂocks. Nature Ecology and Evolution, (3):943–948, 2019.

[34] Hangjian Ling, Guillam E Mclvor, Geoﬀ Nagy, Sepehr MohaimenianPour, Richard T Vaughan, Alex Thornton, and Nicholas T Ouellette. Simultaneous measurements of three-dimensional trajectories and wingbeat frequencies of birds in the ﬁeld. Journal of The Royal Society Interface, 15(147):20180653, 2018.

[35] Hangjian Ling, Guillam E Mclvor, Kasper van der Vaart, Richard T Vaughan, Alex Thornton, and Nicholas T Ouellette. Local interactions and their group-level conse- quences in ﬂocking jackdaws. Proceedings of the Royal Society B, 286(1906):20190865, 2019.

[36] Stephane Magnenat, Markus Waibel, and Antoine Beyeler. Enki: The fast 2d robot simulator. URL http://home. gna. org/enki, 2011.

79 [37] Jakob Mann, Søren Ott, and Jacob Sparre Andersen. Experimental study of relative, turbulent diﬀusion. 1999.

[38] Stefano Marras and Maurizio Porﬁri. Fish and robots swimming together: attraction towards the robot demands biomimetic locomotion. Journal of The Royal Society Interface, 9(73):1856–1868, 2012.

[39] James McLurkin, Andrew J Lynch, Scott Rixner, Thomas W Barr, Alvin Chou, Kath- leen Foster, and Siegfried Bilstein. A low-cost multi-robot system for research, teaching, and outreach. In Distributed Autonomous Robotic Systems, pages 597–609. Springer, 2013.

[40] Microchip. ATmega328P [DATASHEET], 2015. Avail- able at https://ww1.microchip.com/downloads/en/DeviceDoc/ Atmel-7810-Automotive-Microcontrollers-ATmega328P_Datasheet.pdf, Rev: 7810D–AVR–01/15.

[41] Francesco Mondada, Michael Bonani, Xavier Raemy, James Pugh, Christopher Cianci, Adam Klaptocz, Stephane Magnenat, Jean-Christophe Zuﬀerey, Dario Floreano, and Alcherio Martinoli. The e-puck, a robot designed for education in engineering. In Pro- ceedings of the 9th conference on autonomous robot systems and competitions, volume 1, pages 59–65. IPCB: Instituto Politécnico de Castelo Branco, 2009.

[42] Francesco Mondada, Giovanni C Pettinaro, Andre Guignard, Ivo W Kwee, Dario Flo- reano, Jean-Louis Deneubourg, Stefano Nolﬁ, Luca Maria Gambardella, and Marco Dorigo. Swarm-bot: A new distributed robotic concept. Autonomous robots, 17(2- 3):193–221, 2004.

[43] Geoﬀ Nagy and John Anderson. Active team management strategies for multi-robot teams in dangerous environments. In Canadian Conference on Artiﬁcial Intelligence, pages 385–396. Springer, 2017.

[44] Geoﬀ Nagy, Alex Thornton, Hangjian Ling, Guillam McIvor, Nicholas T. Ouellette, and Richard Vaughan. Computational and structural advantages of pairwise ﬂocking. In Proceedings of the 2nd IEEE International Symposium on Multi-Robot and Multi-Agent Systems, (MRS’19), New Brunswick, NJ, August 2019.

[45] Rui Ni and Nicholas T Ouellette. On the tensile strength of insect swarms. Physical biology, 13(4):045002, 2016.

[46] John Nickolls, Ian Buck, and Michael Garland. Scalable parallel programming. In 2008 IEEE Hot Chips 20 Symposium (HCS), pages 40–53. IEEE, 2008.

[47] Nicholas T Ouellette, Haitao Xu, and Eberhard Bodenschatz. A quantitative study of three-dimensional lagrangian particle tracking algorithms. Experiments in Fluids, 40(2):301–313, 2006.

[48] Panther-One. Bird model. https://opengameart.org/content/bird-animated, 2017. Acquired and modiﬁed in accordance with the terms of the Creative Commons Attribution 3.0 License.

80 [49] Erick Baptista Passos, Mark Joselli, Marcelo Zamith, Jack Rocha, E Clua, Anselmo Montenegro, Aura Conci, and Bruno Feijo. Supermassive crowd simulation on gpu based on emergent behavior. In Proceedings of the Seventh Brazilian Symposium on Computer Games and Digital Entertainment, pages 70–75. Citeseer, 2008.

[50] Fernando Perez-Diaz, Ruediger Zillmer, and Roderich Groß. Fireﬂy-inspired synchronization in swarms of mobile agents. In Proceedings of the 2015 International Con- ference on Autonomous Agents and Multiagent Systems, pages 279–286. International Foundation for Autonomous Agents and Multiagent Systems, 2015.

[51] G Polverino, N Abaid, V Kopman, S Macrì, and Maurizio Porfiri. Zebrafish response to robotic fish: preference experiments on isolated individuals and small shoals. Bioin- spiration & biomimetics, 7(3):036019, 2012.

[52] James A Preiss, Wolfgang Honig, Gaurav S Sukhatme, and Nora Ayanian. Crazyswarm: A large nano-quadcopter swarm. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 3299–3304. IEEE, 2017.

[53] Dhruv Rathi, Sushant Jain, and S Indu. Underwater ﬁsh species classiﬁcation using convolutional neural network and deep learning. In 2017 Ninth International Confer- ence on Advances in Pattern Recognition (ICAPR), pages 1–6. IEEE, 2017.

[54] Craig Reynolds. Big fast crowds on ps3. In Proceedings of the 2006 ACM SIGGRAPH symposium on Videogames, pages 113–121. ACM, 2006.

[55] Craig W Reynolds. Flocks, herds and schools: A distributed behavioral model. In Pro- ceedings of the 14th annual conference on computer graphics and interactive techniques, pages 25–34, 1987.

[56] Craig W Reynolds. Steering behaviors for autonomous characters. In Game developers conference, volume 1999, pages 763–782. Citeseer, 1999.

[57] Anthony Rosato, Katherine J Strandburg, Friedrich Prinz, and Robert H Swendsen. Why the brazil nuts are on top: Size segregation of particulate matter by shaking. Physical Review Letters, 58(10):1038, 1987.

[58] Michael Rubenstein, Christian Ahler, and Radhika Nagpal. Kilobot: A low cost scalable robot system for collective behaviors. In 2012 IEEE International Conference on Robotics and Automation, pages 3293–3298. IEEE, 2012.

[59] Michael Rubenstein, Adrian Cabrera, Justin Werfel, Golnaz Habibi, James McLurkin, and Radhika Nagpal. Collective transport of complex objects by simple robots: theory and experiments. In Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems, pages 47–54. International Foundation for Autonomous Agents and Multiagent Systems, 2013.

[60] Seyed Abbas Sadat and Richard T Vaughan. So-lost-an ant-trail algorithm for multi- robot navigation with active interference reduction. In ALIFE, pages 687–693, 2010.

[61] Palash Sarkar. A brief history of cellular automata. Acm computing surveys (csur), 32(1):80–107, 2000.

81 [62] Ruud Schoonderwoerd, Owen E Holland, Janet L Bruten, and Leon JM Rothkrantz. Ant-based load balancing in telecommunications networks. Adaptive behavior, 5(2):169–207, 1997.

[63] Edmund Selous. Thought transference (or what?) in birds. Birds, London, Constable. Sheldrake Robert (1987), New Science of Life: Hypothesis of Formative Causation, London, Paladin, 1931.

[64] Zhao Song, Seyed Abbas Sadat, and Richard T Vaughan. Mo-lost: Adaptive ant trail untangling in multi-objective multi-colony robot foraging. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 3, pages 1199–1200. International Foundation for Autonomous Agents and Multiagent Systems, 2012.

[65] Daniel J Stilwell and John S Bay. Toward the development of a material transport system using swarms of ant-like robots. In [1993] Proceedings IEEE International Conference on Robotics and Automation, pages 766–771. IEEE, 1993.

[66] John H Sudd. The transport of prey by an ant, pheidole crassinoda em. Behaviour, pages 295–308, 1960.

[67] Adrien Treuille, Seth Cooper, and Zoran Popović. Continuum crowds. In ACM Trans- actions on Graphics (TOG), volume 25, pages 1160–1168. ACM, 2006.

[68] Richard T Vaughan, Kasper Stoy, Gaurav S Sukhatme, and Maja J Mataric. Lost: Localization-space trails for robot teams. IEEE Transactions on Robotics and Au- tomation, 18(5):796–812, 2002.

[69] W Grey Walter. An imitation of life. Scientiﬁc American, 182(5):42–45, 1950.

82 Appendix A

Code