Long-range Outdoor Monocular Localization with Active Features for Ship Air Wake Measurement by Brandon J. Draper B.S., Aerospace Engineering, Univ. of Maryland, College Park (2016) Submitted to the Department of Aeronautics and Astronautics in partial fulfillment of the requirements for the degree of M•s-tttr BBJchdM of Science in Aerospace Engineering at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY February 2019 © Massachusetts Institute of Technology 2019. All rights reserved. Author..-Signature redacted . ................ Departme~t of Aeronautics and Astronautics October 1, 2019 Signature redacted Certified by .......... Jonathan P. How R. C. Maclaurin Professor of Aeronautics and Astronautics, MIT Thesis Supervisor --- . / Signature redacted Accepted by .... \..._..,/ Sertac Karaman Associate Professor of Aeronautics and Astronautics, MIT MASSACHUSETTSINSTITUTE Chair, Graduate Program Committee OF TECHNOLOGY MAR 12 2019 LIBRARIES ARCHIVES 2 Long-range Outdoor Monocular Localization with Active Features for Ship Air Wake Measurement by Brandon J. Draper Submitted to the Department of Aeronautics and Astronautics on October 1, 2019, in partial fulfillment of the requirements for the degree of Bachelor of Science in Aerospace Engineering Abstract Monocular pose estimation is a well-studied aspect of computer vision with a wide ar- ray of applications, including camera calibration, autonomous navigation, object pose tracking, augmented reality, and numerous other areas. However, some unexplored areas of camera pose estimation remain academically interesting. This thesis provides a detailed description of the system hardware and software that permits operation in one application area in particular: long-range, precise monocular pose estimation in feature-starved environments. The novel approach to pose extraction uses special hardware, including active LED features and a bandpass-interference optical filter, to significantly simplify the image processing step of the Perspective-n-Point (PnP) problem. The PnP problem describes the calculation of pose from n extracted image points corresponding to n known 3D world points. The proposed application method operates in tandem with a tethered unmanned aerial vehicle (UAV) and mobile ground control station (GCS). The integrated localization and flight system serves as a plat- form for future U.S. Navy air flow research. Indoor tests at the RAVEN flight space of MIT's Aerospace Controls Lab and outdoor tests at a grass strip runway demonstrate the system's efficacy in providing an accurate and precise pose estimate of the UAV relative to the mobile GCS. Thesis Supervisor: Jonathan P. How Title: R. C. Maclaurin Professor of Aeronautics and Astronautics, MIT 3 4 Acknowledgments I would first like to thank my advisor, Jon How, both for the opportunity to advance myself here at MIT and for providing insight and advice at each obstacle I faced in the project. Your guidance proved invaluable in the completion of the project and in my personal development. I have learned such a tremendous amount in such a short time. You have helped me to improve both technically and analytically, giving me the skills that provide access to much greater success in the future. I would like to thank Ben Cameron, John Walthour, Will Fisher, and many others of Creare, LLC. for their technical expertise and roles in the completion of the project. Thank you to all of my fellow ACL members for their support during my time at MIT. Justin Miller took me under his wing when I first joined the lab, helping me transition into grad school and offering support in the early stages of the project. Brett Lopez has provided extensive hardware support, and has propped me up when- ever I felt particularly discouraged. Michael Everett was always my go-to when it came to Linux and Robotic Operating System (ROS) issues, and has played a vi- tal role in the completion of the project. Lastly, my UROP, Ryan Scerbo, helped tremendously in the final stages of the project, implementing software updates and providing much-needed flight test support. Thanks to my parents, who encourage me to set the bar high and give it my all. You're the reason for my ambition and dedication, and I owe you much of my success. I want to thank my incredible partner, Dema Tzamaras, for her unending sup- port, patience, and resilience through our long-distance relationship. Thank you for encouraging me when I faltered, being the rock that I relied on, and being a constant source of joy. I hope I bring as much happiness to your life as you do to mine. Lastly, I would be remiss not to acknowledge the sources of funding for the project and my educational pursuits at MIT: Creare, LLC., the Small Business Technology Transfer (STTR), and the Office of Naval Research. 5 6 Contents 1 Introduction 15 1.1 Project Overview ...... ....... ...... ...... .... 15 1.2 M otivation ....... ............. ............ 16 1.3 Related Work ..... ........ ........ ......... 17 1.3.1 Operational Environment Overview ...... ..... .... 18 1.3.2 Non-camera-based Localization Methods .... ........ 18 1.3.3 Camera-based Localization Methods ........... ... 20 1.4 Thesis Contributions .... ............ ........... 21 2 Monocular Camera Model and Perspective-n-Point Problem 23 2.1 Monocular Camera Model .... .................... 23 2.1.1 Pinhole Camera Model .......... ........... 24 2.1.2 Real Lens Calibration ...................... 25 2.2 Perspective-n-Point Problem ........ ............... 27 2.2.1 PnP Description ......................... 27 2.2.2 The OpenCV Iterative PnP Method .... ....... ... 29 3 Monocular Localization System 31 3.1 System Requirements .. ......................... 31 3.2 Monocular Localization Choice ... ........ ......... 32 3.3 H ardware .......... ............ ........... 32 3.3.1 Active Features ........ .................. 33 3.3.2 C am era ....... ........ ........ ....... 34 7 3.3.3 Power Considerations ...... ..... ...... ...... 35 3.4 Software .. ..... ..... ..... ...... .... ..... .. 36 3.4.1 Initial Centroid Extraction ...... ...... ...... .. 37 3.4.2 First Centroid Filter . ..... ..... .... ..... ... 39 3.4.3 Second Centroid Filter ... ..... .... ..... ..... 39 3.4.4 Assign Points to Rows .... ........ ....... ... 40 3.4.5 Solve PnP ...... ........ ........ ....... 41 3.4.6 Summary .. ....... ........ ........ .... 41 4 Flight System 43 4.1 Hardware ....... ...... ...... ...... ...... .. 43 4.1.1 GCS Hardware ... ........ ........ ....... 43 4.1.2 UAV Hardware ...... ........ ........ .... 45 4.1.3 Special Hardware Considerations ...... ....... ... 46 4.2 Software ..... ...... ....... ...... ...... .... 48 4.2.1 GCS Software . ........ ........ ....... ... 49 4.2.2 UAV Software . ..... ..... ..... .... ..... .. 50 4.2.3 Waypoint Commands in a Relative System . ..... ..... 52 4.2.4 A Note on Communications .. ..... .... ..... ... 54 4.2.5 Summary . .... .... .... .... .... ... .... 56 5 Testing and Results 57 5.1 Monocular Localization System Ground Tests ... .... .... .. 57 5.1.1 Vicon ..... ...... ...... ...... ...... .. 58 5.1.2 Outdoor . ...... ...... ..... ...... ...... 66 5.2 Flight System Ground Tests ....... ...... ...... .... 70 5.2.1 Software-in-the-Loop Simulations .. .... ..... ..... 70 5.2.2 Hardware Validation ..... ..... .... ..... .... 70 5.3 Integrated System Tests . ..... ...... .... ..... .... 72 5.3.1 Initial Verification ..... ..... .... ..... ..... 72 5.3.2 Stationary GCS Flights .... .... .... .... .... .. 73 8 5.3.3 Mobile GCS Flights. ................... .... 77 5.3.4 Summary ....... ............ .......... 80 6 Conclusion 81 6.1 Summary ................................. 81 6.2 Future Work. ..................... .......... 82 A Equipment 83 9 10 List of Figures 1-1 Project Overview . ....... ...... ...... ...... ... 16 2-1 Pinhole Camera Model ... ........ ......... ...... 25 2-2 Camera Distortion Examples .... .......... ......... 26 2-3 Camera Calibration Process ... ......... ........ ... 27 2-4 P3P Problem Diagram ........ ........ ......... 28 3-1 Sunlight Energy Spectrum at Sea Level ................. 33 3-2 Mobile Ground Station Beacon Mounting .......... ...... 34 3-3 Camera Setting Comparison ..... ............ ...... 35 3-4 Optical Filter Transmission Spectrum .................. 36 3-5 Pose Extraction Block Diagram ... ............ ...... 37 3-6 Trucated Search Space and Feature Extraction Process .. ...... 38 4-1 Hardware Interconnect Diagram . .................... 44 4-2 Ground Control Station Hardware .. ............ ...... 45 4-3 Assembled UAV .. ........ ......... ........ ... 45 4-4 Anti-vibration Mount and Plot ... ............ ...... 47 4-5 Electromagnetic Interference Shielding . ............. ... 48 4-6 Tether Tensioner . ............ ............. ... 49 4-7 Ground Control Station Console ..... ............. ... 50 4-8 ROS Architecture for Flight ............. .......... 51 4-9 Yaw Command Generation Example ..... ............ 52 4-10 Mission Transform Example ............. .......... 53 11 5-1 1:35 Scale Model ......... .. ..... ... .... .... 5 9 5-2 Vicon Ground Test 1: Position .. ... .... ... .... .... 6 1 5-3 Vicon Ground Test 1: Orientation .. ..... ... .... .... 6 2 5-4 Vicon Ground Test 2: Position . .. ... ... ... ... ... .. 6 3 5-5 Vicon Ground Test 2: Orientation .... .... .... .... ... 6 4 5-6 Noise Filter Example ....... ... ... ... ... ... ... 6 5 5-7 Briggs Field Beacon Installation .. .... .... .... .... 6 7 5-8 Outdoor Validation: Position . ... .... .... .... ... 68 5-9 Outdoor Validation: