A visual tracking system for honeybee 3D flight trajectory reconstruction and analysis A thesis submitted to The University of Manchester for the degree of Doctor of Philosophy in the Faculty of Science and Engineering 2020 Cong Sun School of Engineering Department of Electrical & Electronic Engineering Table of Contents Abstract ................................................................................................................................................................. 7 Declaration .......................................................................................................................................................... 8 Copyright statement ........................................................................................................................................ 9 Acknowledgements ........................................................................................................................................ 10 1. Chapter 1 – Introduction .................................................................................................................... 11 1.1 The main character of the story - bees ............................................................................... 12 1.1.1 Characteristics and behaviours ............................................................................................ 12 1.1.2 The unquestionable value ...................................................................................................... 16 1.1.3 Current situation and threats ............................................................................................... 17 1.1.4 Strategies to maintain effective pollination .................................................................... 19 1.2 Main contributions ............................................................................................................................ 20 1.3 Chapter summary ............................................................................................................................... 21 2 Chapter 2 – Literature review .......................................................................................................... 24 2.1 An overview of insect behaviour ................................................................................................. 24 2.2 Ground-based passive observation methods ......................................................................... 27 2.3 Airborne netting and sampling platforms ................................................................................ 28 2.4 Entomological radar .......................................................................................................................... 29 2.4.1 Traditional scanning radar / marine radar ..................................................................... 30 2.4.2 Vertical-looking radar (VLR) ................................................................................................. 31 2.4.3 Conventional entomological scanning radar (without harmonic transponders) ...................................................................................................................................................................... 34 2.4.4 Harmonic radar........................................................................................................................... 35 2.4.5 Radio-frequency identification............................................................................................. 39 2.5 Optical imaging systems .................................................................................................................. 42 2.5.1 In-field visual tracking with conventional instrumentation .................................... 43 2.5.2 Insect Monitoring with Light Detection and Ranging (LiDAR) ............................... 45 2.5.3 Laboratory observation ........................................................................................................... 47 2.5.4 Object tracking inspired by modern computer vision algorithms ........................ 49 2.5.4.1 Overview of modern object tracking ......................................................................... 49 2.5.4.2 Small object tracking ........................................................................................................ 51 2.6 Summary ................................................................................................................................................ 52 3 Chapter 3 - Relevant Theory............................................................................................................. 54 3.1 Gaussian mixture model .................................................................................................................. 54 3.2 Kalman filter ......................................................................................................................................... 57 3.3 Hungarian algorithm ......................................................................................................................... 59 3.4 3D space triangulation & epipolar geometry .......................................................................... 60 4 Chapter 4 – Hardware configuration and data collection .................................................... 66 4.1 Exploration of the imaging principle .......................................................................................... 66 4.2 First experiment (2D determination) ........................................................................................ 71 4.3 Camera parameters ........................................................................................................................... 73 4.4 Simulation of 3D flights .................................................................................................................... 75 4.5 Second experiment session (3D determination) ................................................................... 82 4.6 Conclusion ............................................................................................................................................. 86 5 Chapter 5 – Software development ............................................................................................... 88 5.1 Definition of the problem ................................................................................................................ 88 5.2 Processing procedure of the software ....................................................................................... 90 5.3 Flowchart of the software processing procedure ................................................................. 91 5.4 Software development environment ......................................................................................... 92 5.5 Camera calibration ............................................................................................................................. 92 5.6 Pre-processing of the raw video .................................................................................................. 96 5.7 Background subtraction .................................................................................................................. 97 5.8 Morphological operations & further filtering ...................................................................... 103 5.9 Software simulation ....................................................................................................................... 109 5.10 Epipolar geometry system and 3D triangulation ............................................................ 113 5.11 Motion estimation and the generation of flight tracks .................................................. 116 5.12 Quantified system evaluation .................................................................................................. 119 5.13 Conclusion ....................................................................................................................................... 121 6 Chapter 6 – Results and data analysis ....................................................................................... 122 6.1 Initial cleaning and reduction of data ..................................................................................... 122 6.2 Analysis of individual flights ....................................................................................................... 126 6.3 Error analysis .................................................................................................................................... 136 6.4 Analysis of the entire cluster ...................................................................................................... 138 6.4.1 Analysis of flight tendency .................................................................................................. 138 6.4.2 Density distribution of worker bees at hive entrance ............................................. 148 6.5 Conclusion .......................................................................................................................................... 154 7 Chapter 7 - Conclusion ..................................................................................................................... 155 7.1 Overview ............................................................................................................................................. 155 7.2 Summary of system development and findings .................................................................