DEVELOPMENT OF AN AUTOMATED DEBRIS DETECTION SYSTEM FOR WILD BLUEBERRY HARVESTERS USING A CONVOLUTIONAL NEURAL NETWORK TO IMPROVE FRUIT QUALITY by Anup Kumar Das Submitted in partial fulfilment of the requirements for the degree of Master of Science at Dalhousie University Halifax, Nova Scotia October 2020 © Copyright by Anup Kumar Das, 2020 DEDICATION I would like to dedicate my M.Sc dissertation to my parents (Bablu Kumar Das and Fulkumary Das). Author Anup Kumar Das ii TABLE OF CONTENTS LIST OF TABLES ....................................................................................................................... viii LIST OF FIGURES ....................................................................................................................... xi ABSTRACT ................................................................................................................................. xiii LIST OF ABBREVIATIONS USED .......................................................................................... xiv ACKNOWLEDGEMENTS ........................................................................................................ xvii CHAPTER 1: INTRODUCTION ................................................................................................... 1 1.1 Objectives ................................................................................................................................. 3 CHAPTER 2: REVIEW OF LITERATURE .................................................................................. 4 2.0 Wild Blueberry Cropping System ............................................................................................. 4 2.1 Wild Blueberry Harvesting System .......................................................................................... 4 2.2 Factors Affecting Wild Blueberry Fruit Quality ....................................................................... 5 2.2 Machine Learning Over Deep Learning ................................................................................... 6 2.2.1 Supervised Deep Learning ..................................................................................................... 7 2.3 Convolutional Neural Network ................................................................................................. 8 2.3.1 AlexNet .................................................................................................................................. 8 2.3.2 ZFNet ..................................................................................................................................... 9 2.3.3 GoogleNet .............................................................................................................................. 9 2.3.4 VGGNet ............................................................................................................................... 10 2.3.5 ResNet .................................................................................................................................. 10 iii 2.3.6 DenseNet .............................................................................................................................. 11 2.4 Modern Object Detection Algorithm YOLOv3 ...................................................................... 12 3.0 Conclusion .............................................................................................................................. 14 CHAPTER 3: TRAINING AND OPTIMIZING A CNN FOR DEBRIS DETECTION DURING MECHANICAL WILD BLUEBERRY HARVESTING ............................................................. 15 Abstract ......................................................................................................................................... 15 3.0 Introduction ............................................................................................................................. 16 3.1 Materials and Methodology .................................................................................................... 19 3.1.1 Data Collection and Preparation .......................................................................................... 19 3.1.2 Image Augmentation ............................................................................................................ 20 3.1.3 You Only Look Once Version 3 (YOLOv3) Family ........................................................... 23 3.1.4 YOLO Parameter Settings ................................................................................................... 24 3.1.5 YOLO Training and Testing ................................................................................................ 25 3.1.6 Model Evaluation ................................................................................................................. 26 3.1.7 Statistical Analysis ............................................................................................................... 27 3.0 Result and Discussion ............................................................................................................. 27 3.2.1 Effects of Data Augmentation on Accuracy ........................................................................ 30 3.2.2 T-test on Testing Images of T1 Dataset ............................................................................... 31 3.2.3 T-test on Testing Images of T2 Dataset ............................................................................... 32 3.0 Conclusions ............................................................................................................................. 34 iv CHAPTER 4: DEVELOPMENT OF A REAL-TIME DEBRIS DETECTION SYSTEM (HARDWARE & SOFTWARE) FOR MECHANICAL WILD BLUEBERRY HARVESTER . 36 Abstract ......................................................................................................................................... 36 4.0 Introduction ............................................................................................................................. 37 4.1 Material and Methodology ...................................................................................................... 40 4.1.1 Video Acquisition Hardware Development ......................................................................... 40 4.1.2 Camera Acquisition Software Development ....................................................................... 41 4.1.3 Determine the Minimum Processing Time Required for a Debris Detection System to Perform in Real-time..................................................................................................................... 43 4.1.4 Determine Detection Speed of CNNs for Developing Debris Detection System ................ 44 4.1.4.1 Data Collection and Preparation ....................................................................................... 44 4.1.4.2 Hardware Setup ................................................................................................................. 46 4.1.4.3 Darknet Installation ........................................................................................................... 47 4.1.4.4 Measurement of Prediction Time...................................................................................... 48 4.1.4.5 Statistical Analysis for Determining the Effect of Models and Hardwire on Prediction Time ....................................................................................................................................................... 48 4.1.5 Validation of Models ........................................................................................................... 49 4.1.6 Development of a Debris Detection System ........................................................................ 50 4.2 Result and Discussion ............................................................................................................. 51 4.2.1 Validation of Models ........................................................................................................... 53 4.2.1.1 Evaluation of Model-1 Under Different IoU and Confidence Thresholds ....................... 53 4.2.1.2 Evaluation of the Model-2 Under Different IoU and Confidence Thresholds ................. 55 v 4.3 Conclusion .............................................................................................................................. 58 CHAPTER 5: EVALUATION OF OPTIMIZED CNN MODEL FOR DEBRIS DETECTION ON THE IMAGES CAPTURED DURING HARVESTING ............................................................. 60 Abstract ......................................................................................................................................... 60 5.0 Introduction ............................................................................................................................. 60 5.1 Material and Methodology ...................................................................................................... 62 5.1.1 Data Collection .................................................................................................................... 62 5.1.3 Statistical Analysis ............................................................................................................... 64 5.2 Results and Discussion ........................................................................................................... 64 5.3 Conclusion .............................................................................................................................