An Autonomous Unmanned Aerial Vehicle-Based Imagery System Development and Remote Sensing Images Classification for Agricultural Applications
Total Page:16
File Type:pdf, Size:1020Kb
Utah State University DigitalCommons@USU All Graduate Theses and Dissertations Graduate Studies 12-2009 An Autonomous Unmanned Aerial Vehicle-Based Imagery System Development and Remote Sensing Images Classification for Agricultural Applications Yiding Han Utah State University Follow this and additional works at: https://digitalcommons.usu.edu/etd Part of the Aerospace Engineering Commons Recommended Citation Han, Yiding, "An Autonomous Unmanned Aerial Vehicle-Based Imagery System Development and Remote Sensing Images Classification for Agricultural Applications" (2009). All Graduate Theses and Dissertations. 513. https://digitalcommons.usu.edu/etd/513 This Thesis is brought to you for free and open access by the Graduate Studies at DigitalCommons@USU. It has been accepted for inclusion in All Graduate Theses and Dissertations by an authorized administrator of DigitalCommons@USU. For more information, please contact [email protected]. AN AUTONOMOUS UNMANNED AERIAL VEHICLE-BASED IMAGERY SYSTEM DEVELOPMENT AND REMOTE SENSING IMAGES CLASSIFICATION FOR AGRICULTURAL APPLICATIONS by Yiding Han A thesis submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE in Electrical Engineering Approved: Dr. HuiFang Dou Dr. YangQuan Chen Major Professor Committee Member Dr. Donald Cripps Dr. Byron R. Burnham Committee Member Dean of Graduate Studies UTAH STATE UNIVERSITY Logan, Utah 2009 ii Copyright c Yiding Han 2009 All Rights Reserved iii Abstract An Autonomous Unmanned Aerial Vehicle-Based Imagery System Development and Remote Sensing Images Classification for Agricultural Applications by Yiding Han, Master of Science Utah State University, 2009 Major Professor: Dr. HuiFang Dou Department: Electrical and Computer Engineering This work concentrates on the topic of remote sensing using a multispectral imag- ing system for water management and agriculture applications. The platform, which is a light-weight inexpensive runway-free unmanned aerial vehicle (UAV), namely, AggieAir, is presented initially. A major portion of this work focuses on the development of a light- weight multispectral imager payload for the AggieAir platform, called GhostFoto. The imager is band-reconfigurable, covering both visual red, green, and blue (RGB) and near infrared (NIR) spectrum, and interfaced with UAV on-board computer. The development of the image processing techniques, which are based on the collected multispectral aerial images, is also presented in this work. One application is to perform fully autonomous river tracking for applications such as river water management. Simulation based on aerial mul- tispectral images is done to demonstrate the feasibility of the developed algorithm. Other effort is made to create a systematic method to generate normalized difference vegetation index (NDVI) using the airborne imagery. The GhostFoto multispectral imaging system based on AggieAir architecture is proven to be an innovative and useful tool. (72 pages) iv To my family and friends. v Acknowledgments I would like to thank my advisor, Dr. Dou, for her generous guidance, advice, financial support, and tremendous help when I am stuck with problems. Without her motivation and insights this work would have never been complete. Also, I would like to thank Dr. Chen for giving me this opportunity to join CSOIS and the UAV team, constantly motivating me, generously supporting me financially in the beginning of my master’s program, and the extraordinary insights that he has brought to my work. Also, I would like to thank my committee member, Dr. Cripps, for his comments on my work. I would like to thank all the CSOIS UAV members, without whom my work would have been impossible. I would like to thank Haiyang Chao for being a great role model for me, guiding and helping me in my work, bringing me up to speed initially in the UAV team, and all the late-night pre-flight tests he participated in. I would like to thank Calvin Coopmans for helping me to understand Linux and programming Gumstix, creating those brilliant ideas about AggieAir architecture with me, and all the late-night work he did with me on different projects. I would like to thank Austin Jensen for inviting me into the UAV team in the first place, and all the great work he did for us in managing the team, and the countless flight tests in which he participated. I would like to thank Di Long for building the airframes and hundreds of backup parts for us, and for participating in the flight tests for UAV competition. I would like to thank Hu Sheng for building and donating his Tiger plane for us to participate in the UAV competition. I would also like to thank Chris Hall and Daniel Morgan for their support and advice. Also, for the other members of the CSOIS, the help that they gave me on other projects is truly appreciated. I would like to thank Shayok Mukhopadhyay for the countless late-night work he did with me for the Smartwheel projects and Sumo robots. He has been such a great friend and helped me enormously with my English. I would also like to thank Shelley Rounds for helping me in the Mechatronics lab projects and teaching me the American culture, Varsha Bhambhani for helping me on the Smartwheel project, and Dr. Yan Li for helping me to understand Fractional Order vi Calculus. I would like to thank all the Chinese scholars that visited CSOIS during my master’s program, for supporting me the whole time and sharing the great Chinese meals with me. I would also like to thank my friends and roommates who supported me and made my time at Utah State University enjoyable. Above all, I would like to thank my family in China, especially my mother, for their constant selfless support and unwavering belief in me. Last, but not least, I would like to thank the Utah Water Research Lab for providing funding for this project. Yiding Han vii Contents Page Abstract ....................................................... iii Acknowledgments ............................................... v List of Tables ................................................... ix List of Figures .................................................. x Acronyms ...................................................... xii 1 Introduction ................................................. 1 1.1 Overview ..................................... 1 1.1.1 Unmanned Aerial Vehicle . 1 1.1.2 Remote Sensing and Agriculture . 2 1.2 Motivation . 2 1.3 GhostFoto Multispectral Remote Sensing Platform . 3 1.3.1 Development Background . 3 1.3.2 Hardware Development for GhostFoto . 3 1.3.3 Software Development for GhostFoto . 4 1.3.4 Image Processing . 5 1.4 Contribution and Organization . 5 2 AggieAir Miniature UAV Architecture ........................... 7 2.1 AggieAir System Overview . 9 2.1.1 Airframe . 9 2.1.2 On-Board Electronics . 10 2.1.3 Ground Controls . 12 2.2 Paparazzi . 14 2.2.1 Paparazzi TWOG Board . 14 2.2.2 Ground Station . 14 2.3 Gumstix On-Board Computer . 16 2.3.1 Functionality . 16 2.3.2 Data Flow . 17 2.4 gRAID Image Processing . 19 3 GhostFoto: A Multispectral Remote Sensing Platform .............. 21 3.1 Background and Expectations . 21 3.2 GhostFoto Hardware . 22 3.2.1 Cameras . 22 3.2.2 Near Infrared Camera . 25 3.3 GhostEye ..................................... 26 viii 3.3.1 gPhoto . 27 3.3.2 Periodic Image Capturing . 27 3.3.3 Calculation of Capture Interval . 30 3.3.4 State Machine . 32 3.3.5 Multithreading Architecture . 33 3.3.6 Image Geo-Referencing . 36 3.3.7 Logging and Debugging . 37 3.3.8 Implementation in Gumstix . 37 3.3.9 Configure GhostEye . 37 4 Image Processing ............................................. 39 4.1 RiverTracking .................................. 39 4.1.1 Motivation . 39 4.1.2 Water Area Recognition . 40 4.1.3 River Tracking Mission . 40 4.1.4 River Identification . 42 4.1.5 Way Points Generation . 45 4.1.6 Results .................................. 47 4.2 Vegetation Recognition . 48 4.2.1 Normalized Difference Vegetation Index . 48 4.2.2 White Panel Calibration . 50 5 Conclusion ................................................... 53 5.1 Contribution ................................... 53 5.2 FutureWork ................................... 54 5.2.1 Thermal Infrared Camera . 54 5.2.2 Collaborative Remote Sensing . 55 5.2.3 Implementation of OpenJAUS . 56 5.2.4 General-Purpose GPU-Based Image Processing . 56 References ...................................................... 58 ix List of Tables Table Page 3.1 Specifications of PowerShot SX100 IS and SX110 IS. 24 3.2 Starting transmission wavelength of different NIR filters. 26 4.1 Multispectral imager settings for river tracking mission. 42 x List of Figures Figure Page 2.1 AggieAir system level overview. 8 2.2 Unicorn airframe. 9 2.3 AggieAir 72 inches UAV layout. 10 2.4 Electronic devices inside the main bay. 11 2.5 Underneath the aircraft. 12 2.6 Paparazzi TWOG board. 15 2.7 Paparazzi Center. 15 2.8 Paparazzi Ground Control Station (GCS). 16 2.9 Gumstix Verdex computer-on-module with expansion board. 17 2.10 Data flow of AggieAir airborne system. 18 2.11 Aerial imagery mosaic created by gRAID under WorldWind. 20 3.1 GhostFinger-DC imager using Pentax Optio E10 camera. 22 3.2 Canon PowerShot SX100 IS with and without the cover. 24 3.3 Canon PowerShot SX100 IS and Canon PowerShot SX110 IS. 25 3.4 CCD sensor inside Canon PowerShot SX100 IS, covered by visible light filter. 26 3.5 Software architecture on which GhostEye is based. 28 3.6 Imager footprint. 31 3.7 Flowchart of GhostEye control structure. 32 3.8 Statuses defined in GhostEye state machine. 34 3.9 Multithread architecture of GhostEye. 35 3.10 Inter-thread communication between GhostEye threads. 35 xi 3.11 Inter-thread communication to store geo-referencing data. 36 4.1 NIR and RGB images of a reservoir . 41 4.2 Flight plan of river tracking mission. 42 4.3 NIR and RGB images of river segment. 44 4.4 Histogram of the NIR image. 45 4.5 Binary images of the river. 46 4.6 Dynamic way points generated from the aerial NIR images.