Quadrotor Flight in Constrained Indoor Environments by Antonio Matus-Vargas Dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Science in Computational Science at the Instituto Nacional de Astrof´ısica, Optica´ y Electr´onica Tonantzintla, Puebla, M´exico November, 2020 Supervisors: Dr. Gustavo Rodr´ıguez G´omez Dr. Jos´eMart´ınez Carranza Coordination of Computational Science INAOE c INAOE 2020 All rights reserved The author hereby grants to INAOE permission to reproduce and to distribute copies of this thesis document in whole or in part. 2 Quadrotor Flight in Constrained Indoor Environments by Antonio Matus-Vargas Submitted to the Coordination of Computational Science on November 26, 2020, in partial fulfillment of the requirements for the degree of Doctor of Science in Computational Science Abstract Nowadays, advances in sensor, actuator, and processing technologies have permitted to shrink rotorcraft down to a micro-scale. Thanks to their hovering capability and high maneuverability, micro rotorcraft, classified as micro aerial vehicles (MAVs), are well suited for operating within constrained indoor environments. The autonomous navigation of these vehicles in such environments is a challenge for two main reasons. First, due to payload and energy restrictions, a MAV is equipped with the least quantity of sensors. Second, operation within a constrained environment will place a MAV close to objects, which will induce aerodynamic interactions. The most studied aerodynamic interaction is the ground effect, which can be explained as the cushion of air under the vehicle when it flies closely over a rigid surface. In this thesis, we propose a novel approach for rejecting disturbances induced by the ground effect. To retrieve the position of the vehicle, we employ a monocular camera, which is a lightweight and energy-efficient sensor. Also, we provide experimental evidence that theground effect induces sensor faults. For that reason, we develop the approach asafault- tolerant control scheme, which consists of a detection strategy and mitigation strategy. We assume a hierarchical control structure for trajectory tracking. Specifically, we assume that the nominal control structure consists of an external PD controller and an internal PI controller. In experimental tests, we found that the sensor faults occur on the inner loop, which we counteract in the outer loop by switching between control actions. In a novel approach, we use a metric monocular SLAM algorithm for detecting internal faults. We design the fault detection scheme as a logical process that depends on the weighted residual between inner and outer estimations. Furthermore, we propose two control strategies for fault mitigation. The first combines the external PD controller and a function of the residual (FTC 1). The second treats the sensor fault as an actuator fault and compensates with a sliding mode action (FTC 2). In either case, we utilize onboard sensors only. We evaluate the effectiveness of the strategies in simulations and experiments. The experimental results show that FTC 1 surpasses FTC 2. Finally, we explore the usage of optical flow estimations for the detection and mitigation of the ground effect. 3 Thesis Supervisor: Dr. Gustavo Rodríguez Gómez Title: Full Researcher and Associate Professor Thesis Supervisor: Dr. José Martínez Carranza Title: Full Researcher and Associate Professor 4 Acknowledgments The completion of this thesis would not have been possible without the help of a large group of people. First, I would like to extend my deepest gratitude to my supervisors, Gustavo Rodríguez Gómez and José Martínez Carranza. I am grateful to Gustavo for bringing order out of the chaos of my research ideas and for all the superb advice. I thank José for the great discussions, which helped me to develop and explain my research both in spoken and written form. Also, I would like to express my appreciation to all members of my committee for their valuable comments that enhanced this thesis. In my work, I have received invaluable support from Oyuki Rojas Perez, Aldrich Cabrera Ponce, Arturo Cocoma Ortega, David Carrillo López, and Roberto Munguía Silva. Furthermore, I am deeply indebted to my parents, aunt, brother, and sister for always being there for me; and to my friends for the good times over the years. Finally, I have to admit that I lack the poetic skills to express, in a suitable way, my gratitude to Kristel, my fiancée. She always brightens up my day and hopefully will do so for years to come. This research has been funded by the Consejo Nacional de Ciencia y Tecnología (Conacyt), in the framework of the Programa Nacional de Posgrados de Calidad (PNPC) as a scholarship registered to the CVU number 540945. Also, I gratefully acknowledge financial support over the final months by the Instituto Nacional de Astrofísica, Óptica y Electrónica (INAOE). 5 6 Contents 1 Introduction 19 1.1 Justification and Motivation . 21 1.2 Problem Statement . 23 1.2.1 Research Questions . 23 1.2.2 Hypothesis . 23 1.3 Objectives . 24 1.4 Research Methodology . 24 1.5 Contributions . 25 1.5.1 Publications . 27 1.6 Thesis Outline . 28 2 Theoretical Framework 29 2.1 Multirotor UAVs . 29 2.2 Hardware . 30 2.3 Quadrotor Concepts . 34 2.4 Attitude Representations . 35 2.4.1 Rotation Matrix . 36 2.4.2 Euler Angles . 39 2.4.3 Quaternions . 41 2.4.4 Rotation Vector . 44 2.5 Control Systems . 45 2.6 SLAM . 47 2.6.1 Metric Monocular SLAM . 49 7 2.7 Optical Flow . 50 2.8 Deep Learning . 53 2.8.1 Convolutional Neural Networks . 55 3 Literature Review 59 3.1 Full-scale Helicopter . 59 3.1.1 Hovering Flight . 59 3.1.2 Forward Flight . 62 3.2 Model Helicopter . 65 3.2.1 Control . 65 3.2.2 Flow Studies . 66 3.2.3 Modeling . 67 3.3 Multirotor . 68 3.3.1 Control . 69 3.3.2 Modeling . 75 3.3.3 Applications . 85 3.4 Other Proximity Effects . 86 3.5 Discussion . 88 3.6 Chapter Conclusion . 90 4 Control Optimization 93 4.1 Deterministic Procedures . 94 4.1.1 Related Work . 95 4.1.2 Quadrotor Model . 96 4.1.3 Mathematical Problem . 97 4.1.4 Conjugate Gradient Methods . 99 4.1.5 Optimal Control . 101 4.1.6 Controllers Implementation . 102 4.1.7 Results and Analysis . 105 4.1.8 Summary . 107 4.2 Stochastic Procedure . 108 8 4.2.1 Background on Multirotor Control . 109 4.2.2 Multirotor Model . 110 4.2.3 Position Control Algorithm . 112 4.2.4 Attitude Control Algorithm . 113 4.2.5 Cost Function . 114 4.2.6 Stochastic Test Signals . 116 4.2.7 Results and Analysis . 118 4.2.8 Summary . 122 4.3 Ground Effect Simulation . 123 4.3.1 Optimization with Reduced Vehicle Model . 123 4.3.2 Optimization with Full Vehicle Model . 125 4.4 Chapter Conclusion . 127 5 Discontinuous Control 129 5.1 Multi-controller Structure . 129 5.1.1 Drone Overview . 131 5.1.2 Proposed Control Structure . 132 5.1.3 Experiments . 136 5.1.4 Summary . 140 5.2 Fault-Tolerant Control . 141 5.2.1 Related Work . 142 5.2.2 Methods and Materials . 144 5.2.3 Results and Discussion . 154 5.2.4 Summary . 162 5.3 Optical Flow and GE . 163 5.3.1 Dataset . 164 5.3.2 Training . 165 5.3.3 Detection Results . 167 5.3.4 Focus of Expansion . 168 5.4 Chapter Conclusion . 170 9 6 Conclusion 173 6.1 Answers to the Research Questions . 174 6.2 Hypothesis and Objectives Assessment . 175 6.3 Limitations and Future Research . 176 10 List of Figures 1-1 Qualitative schematic view of flow vectors. The pressure above the vehicle is indicated as Pa, the pressure below as Pg, and the net force due to the ground effect as fe....................... 22 1-2 Outline of the research methodology. 25 2-1 UAV classification, based on [56]. 30 2-2 Main components of a quadcopter. 31 2-3 Common quadrotor configurations (top view), a) “x” style, b) “+” style. 34 2-4 Degrees of freedom of a quadrotor. 35 2-5 Block diagrams of a) the open-loop and b) the closed-loop control sys- tems. 46 2-6 Architecture of a typical SLAM system. The back end can provide feedback to the front end for loop closure detection and verification. 48 2-7 Geometric configuration to generate the synthetic depth image. 49 2-8 Snapshots that illustrate the functioning of the metric monocular SLAM; the top row provides samples from camera frames with the tracked fea- tures (green markers); the bottom row displays three views of the map with keyframes (blue pyramids, the first one is green), and map points (red ones are being used for tracking, while black ones are ignored). 50 2-9 Two visualization techniques of the motion field transforming I1 in I2 [49]. 52 2-10 A simple CNN architecture, comprised of four layers. 56 11 3-1 Increase in rotor thrust versus distance from the ground for a variety of helicopters [109]. The legends OH-6A, OH-58A, UH-1H, and UH-1C are codenames for full-scale helicopters. 60 3-2 Measurements and predictions of rotor power versus forward speed when operating near the ground [109]. 63 3-3 The flow patterns for a fan in GE represented as a source (a)anda dipole (b) [82]. ..