Sonar Attentive Underwater Navigation in Structured Environment Hashim Kemal Abdella Thesis submitted for the Degree of Doctor of Philosophy Heriot-Watt University School of Engineering and Physical Sciences Institute of Sensors, Signals and Systems July 2018 © This copy of the thesis has been supplied on condition that anyone who consults it is understood to recognise that the copyright rests with its author and that no quotation from the thesis and no information derived from it may be published without the prior written consent of the author or the University (as may be appropriate). ABSTRACT One of the fundamental requirements of a persistently Autonomous Underwater Vehicle (AUV) is a robust navigation system. The success of most complex robotic tasks depends on the accuracy of a vehicle’s navigation system. In a basic form, an AUV estimates its po- sition using an on-board navigation sensors through Dead-Reckoning (DR). However DR navigation systems tends to drift in the long run due to accumulated measurement errors. One way of mitigating this problem require the use of Simultaneous Localization and Map- ping (SLAM) by concurrently mapping external environment features. The performance of a SLAM navigation system depends on the availability of enough good features in the environment. On the contrary, a typical underwater structured environment (harbour, pier or oil field) has a limited amount of sonar features in a limited locations, hence exploitation of good features is a key for effective underwater SLAM. This thesis develops a novel attentive sonar line feature based SLAM framework that im- proves the performance of a SLAM navigation by steering a multibeam sonar sensor, which is mounted on a pan and tilt unit, towards feature-rich regions of the environment. A sonar salience map is generated at each vehicle pose to identify highly informative and stable regions of the environment. Results from a simulated test and real AUV experiment show an attentive SLAM performs better than a passive counterpart by repeatedly visiting good sonar landmarks. To my parents Acknowledgements First of all I would like to say ‘Alhamdulillah’. Everything happens for a reason and a lot of things have happened to get me here. There are many people who have helped with my research over the last few years, yet it all started when I joined the Ocean Systems Laboratory. Hence, I would like to express my sincere appreciation to my supervisor, prof. David Lane, for giving me the opportunity. A special thank you to Dr. Keith Brown for tirelessly going through my drafts and providing valuable feedbacks. A big thank you to colleagues in the Ocean Systems Laboratory for all the encouragement and moral support. Endless thanks to my family who have always been a constant source of love, support, encouragement and inspiration. Finally, I am indebted to my wife, Zuleyha, thank you for all your contributions to our family. Funding This work has been supported by the FP7-ICT-2011-7 European project PANDORA-Persistent Autonomy through Learning, Adaptation, Observation and Re-planning (Ref. 288273). ACADEMIC REGISTRY Research Thesis Submission Name: School: Version: (i.e. First, Degree Sought: Resubmission, Final) Declaration In accordance with the appropriate regulations I hereby submit my thesis and I declare that: 1) the thesis embodies the results of my own work and has been composed by myself 2) where appropriate, I have made acknowledgement of the work of others and have made reference to work carried out in collaboration with other persons 3) the thesis is the correct version of the thesis for submission and is the same version as any electronic versions submitted*. 4) my thesis for the award referred to, deposited in the Heriot-Watt University Library, should be made available for loan or photocopying and be available via the Institutional Repository, subject to such conditions as the Librarian may require 5) I understand that as a student of the University I am required to abide by the Regulations of the University and to conform to its discipline. 6) I confirm that the thesis has been verified against plagiarism via an approved plagiarism detection application e.g. Turnitin. * Please note that it is the responsibility of the candidate to ensure that the correct version of the thesis is submitted. Signature of Date: Candidate: Submission Submitted By (name in capitals): Signature of Individual Submitting: Date Submitted: For Completion in the Student Service Centre (SSC) Received in the SSC by (name in capitals): Method of Submission (Handed in to SSC; posted through internal/external mail): E-thesis Submitted (mandatory for final theses) Signature: Date: Please note this form should be bound into the submitted thesis. Academic Registry/Version (1) August 2016 5 Contents List of Figures iv List of Tablesv Nomenclature 1 Introduction1 1.1 Problem Statements and Thesis Aim.....................3 1.1.1 Problem Statements.........................3 1.1.2 Thesis Aims and Objectives.....................3 1.1.3 Evaluation Metrics..........................4 1.2 Platform and Test Environments.......................4 1.2.1 UWSim Simulator..........................5 1.2.2 Nessie VII AUV...........................5 1.2.3 OSL Cartesian Robot and Indoor Tank................6 1.2.4 HWU Wave-tank...........................6 1.3 Thesis Layout.................................7 2 Literature Review9 2.1 Underwater Navigation............................9 2.1.1 Common Underwater Navigation Sensors.............. 10 2.1.2 Navigation and Localization Through State Estimation....... 14 2.2 Underwater SLAM.............................. 20 2.2.1 State of the Art of Underwater SLAM................ 20 2.2.2 Large Scale Underwater SLAM................... 22 2.2.3 Robust SLAM and Outlier Rejection................. 24 2.3 Sonar Sensors and Feature Extraction.................... 26 2.3.1 Feature Extraction.......................... 26 2.4 Active Vision................................. 35 2.4.1 Applications of Active Vision.................... 36 2.4.2 Techniques of Active Vision..................... 40 i 2.5 Focus of Attention.............................. 44 2.5.1 Categories of Attention....................... 44 2.5.2 Computational Models of Visual Attention............. 45 2.5.3 Focus of Attention for Mobile Robotics............... 52 2.6 Summary................................... 55 3 Sonar Feature Extraction 57 3.1 Multibeam Sonar Characteristics and Simulation.............. 57 3.1.1 BlueView Multibeam Sonars..................... 58 3.1.2 Characteristics of Sonar Images................... 58 3.1.3 Simulation of Sonar Images..................... 64 3.2 Sonar Line Feature Extraction........................ 68 3.2.1 Line Features Representation..................... 68 3.2.2 Sonar Image Processing....................... 70 3.2.3 Hough-Transform Line Extraction.................. 71 3.2.4 Incremental Line Fitting....................... 73 3.2.5 Split and Merge Line Fitting..................... 73 3.2.6 Least Square Line Fitting...................... 74 3.3 Uncertainty Estimation............................ 76 3.4 Comparison of Line Extraction Algorithms................. 77 3.4.1 Experiment Set-up.......................... 77 3.4.2 Result and Discussion........................ 80 3.5 Summary................................... 82 4 Underwater SLAM Navigation 83 4.1 Bayesian Framework of SLAM........................ 83 4.1.1 State Representation......................... 84 4.1.2 Probabilistic SLAM Recursion.................... 84 4.2 Extended Kalman Filter SLAM........................ 85 4.2.1 State Transition Model........................ 88 4.2.2 Sonar Observation Model...................... 89 4.3 Unscented Kalman Filter SLAM....................... 92 4.3.1 Unscented Transform and Sigma Points............... 94 4.3.2 UKF-SLAM Recursion....................... 94 4.4 SLAM Experiment and Results........................ 97 4.4.1 Simulated SLAM Test........................ 97 4.4.2 Marina Dataset............................ 99 4.5 Probability Hypothesis Density Filter.................... 103 4.5.1 Gaussian Mixture-PHD Filter.................... 104 ii 4.5.2 PHD Filter vs KF for Sonar Based Mapping............. 107 4.6 Summary................................... 111 5 Focus of Attention for Underwater Navigation 112 5.1 Sonar Based Salience Map.......................... 112 5.1.1 Grid Point Generation........................ 114 5.1.2 Angle of Incidence Weight...................... 115 5.1.3 Overlap Weight............................ 116 5.1.4 Distance Weight........................... 117 5.1.5 Combined Sonar Salience...................... 118 5.2 Sonar Focus of Attention (SFoA)....................... 119 5.2.1 Inhibition of Large Rotation (IoLR)................. 120 5.2.2 Attention Salience Map........................ 120 5.3 Attention Based Active SLAM........................ 122 5.4 Experimental Results............................. 122 5.4.1 Simulated Tests............................ 122 5.4.2 OSL tank experiment......................... 134 5.4.3 Wave-tank test............................ 149 5.5 Summary................................... 153 6 Conclusion 154 6.1 Summary and Contributions......................... 154 6.2 Review of Contributions........................... 155 6.3 Future Work.................................. 156 Bibliography 158 iii List of Figures 1.1 Nessie VII OSL research AUV.........................5 1.2 Ocean system lab indoor test tank and Cartesian robot............6