MSC AI INDIVIDUAL PROJECT IMPERIAL COLLEGE LONDON DEPARTMENT OF COMPUTING Automation and Intelligent Control in High Performance Sailing Boats Final Report Submitted in partial fulfilment of the requirements for the MSc Degree in Artificial Intelligence of Imperial College London Author: Supervisor: Charles Metz (CID: 01825073) Dr Pedro Baiz Second Supervisor: Dr Eric Topham Second Marker: Prof Julie McCann Date: September 4, 2020 Abstract This study is a continuation of the work of Birk Ulstad, Roman Kastusik and Stanis- las Hannebelle on the application of machine learning methods to the intelligent steering of sailing boats. The purpose of the study is to investigate models that reli- ably reproduce the behaviour of a sailing boat in its sea environment. These digital twins of the sailboat consist of timeseries forecasting models that predict the values of various variables that define the state of the boat for the following second. This allows a virtual simulation of how the angle of the boat’s rudder affects the boat’s course and its state, which is the basis for Reinforcement Learning algorithms to learn intelligent control of the rudder. Detailed background research provides an overview of relevant developments in the field of timeseries forecasting. The models investigated here are LSTM-based deep neural networks as well as models derived from first principles. The improvement of the architecture and hyperparameters of the models using Bayesian optimisation is discussed. A significant improvement of the models compared to a previous approach is achieved. While adequate model hyperparameters can be found for a given dataset of a given boat, it is found that they are not easily generalisable across different data collecting protocols. Finally, a framework with which to obtain and assess accurate forecasting models is proposed. Keywords: Sailing, Autopilot, Digital Twin, Deep Learning, Recurrent Neural Net- works, LSTM, Bayesian Optimisation, Timeseries Forecasting 1 Acknowledgements I would like to thank Roman Kastusik and Dr Eric Topham, who during almost five months took time for daily online meetings. Despite otherwise very full agendas, they were always available for enquiries and made me benefit very much from their experience, ideas and constructive criticism. I am particularly grateful that they did so during an era marked by a crown-shaped predator, on top of which they allowed me to gain insight into what was happening at their company T-DAB. I would also like to thank Dr Pedro Baiz, who contributed his input and knowledge to the project on a weekly basis; his feedback proved valuable in many ways. I would also like to thank Stanislas Hannebelle: he took the time to explain various aspects of the project to me. Finally, I would like to thank Prof Julie McCann, who took the time to review this and another report and to answer several questions. All in all, I am extremely grateful that this project could take place despite the circumstances. 2 Contents 1 Introduction6 1.1 Motivation.................................6 1.2 Entities involved in the project.....................6 1.3 Outline..................................7 2 Background9 2.1 Overview of previous work........................9 2.1.1 Supervised learning approach..................9 2.1.2 RL approach........................... 10 2.1.3 Refinement of the existing approaches............. 11 2.2 Available datasets............................. 11 2.2.1 Types of boats........................... 12 2.2.2 Concise 8............................. 14 2.2.3 Virgin Media Business (VMB).................. 16 2.2.4 Unknown 1 and Unknown 2................... 16 2.3 Previous work by S. Hannebelle..................... 17 2.3.1 Data Pre-Processing....................... 17 2.3.2 Data Cleaning and Splitting................... 23 2.3.3 Supervised Learning Process................... 23 2.4 Previous work by R. Kastusik...................... 25 2.4.1 State estimator.......................... 25 3 Literature Review 32 3.1 Autonomous sailboats.......................... 32 3.1.1 RoboSail Project......................... 32 3.1.2 Other research on autonomous sailboats............ 33 3.2 Timeseries Forecasting: Evolution and State of the Art...................... 34 3.3 Deep Learning for Timeseries Forecasting.......................... 35 3.4 Generative Adversarial Networks for Timeseries Forecasting........................ 38 3.5 Hybrid Models of Dynamic Systems................... 40 3.6 Conclusion and resulting scope of the study............................ 41 3.6.1 Conclusion............................ 41 3 CONTENTS CONTENTS 3.6.2 Scope of the study........................ 42 4 Data 45 4.1 Gathering................................. 45 4.1.1 Challenges encountered..................... 45 4.1.2 Implications on conversion of datasets............. 46 4.2 Cleaning.................................. 47 4.3 Distribution................................ 48 4.4 Selection................................. 51 4.4.1 Choice of datasets........................ 51 4.4.2 Data splitting........................... 51 4.5 Preprocessing............................... 55 4.5.1 Normalisation........................... 55 4.5.2 Rearranging the Timeseries................... 56 5 Models 58 5.1 1 Model for n Features.......................... 58 5.2 n Models for n Features......................... 59 5.3 Deterministic models........................... 59 5.3.1 Motivation............................ 59 5.3.2 Formulae............................. 60 5.4 Prediction Time Horizon......................... 62 5.5 Evaluation Metrics............................ 62 6 Experiments 64 6.1 1 Model for n features.......................... 65 6.1.1 Motivation and Hypothesis................... 65 6.1.2 Experiments............................ 65 6.1.3 Observed time window...................... 66 6.1.4 Search space........................... 66 6.1.5 Bayesian Optimisation...................... 67 6.2 N models for n features: Model 1.................... 67 6.2.1 Motivation and Hypothesis................... 67 6.2.2 Experiments............................ 68 6.2.3 Model 1 hyperparameters.................... 68 6.3 N models for n features: Model 2.................... 69 6.3.1 Motivation and Hypothesis................... 69 6.3.2 Experiments............................ 69 6.4 Deterministic models........................... 70 6.4.1 Motivation and hypotheses................... 70 6.4.2 Experiments............................ 70 6.5 Transferability of model hyperparameters between boats and datasets....................... 71 6.5.1 Motivation and hypotheses................... 71 6.5.2 Experiments............................ 72 4 CONTENTS CONTENTS 7 Results and Discussion 74 7.1 1 Model for n Features.......................... 74 7.1.1 Results............................... 74 7.1.2 Discussion............................. 75 7.2 N Models for n Features: Model 1.................................. 79 7.2.1 Results............................... 79 7.2.2 Discussion............................. 80 7.3 N Models for n Features: Model 2.................................. 82 7.3.1 Results............................... 82 7.3.2 Discussion............................. 83 7.4 Deterministic Models........................... 84 7.4.1 Results............................... 85 7.4.2 Discussion............................. 85 7.5 Transferability of models between boats and datasets............................ 90 7.5.1 Concise 8 (Atlantic)....................... 91 7.5.2 Unknown 1 (transat 1)..................... 94 8 Conclusion and Future Work 98 8.1 Conclusion................................ 98 8.2 Future Work................................ 100 8.2.1 Framework to create a reliable RL simulation environment.. 100 8.2.2 Further directions of work.................... 102 9 Ethical considerations 105 A Ethics checklist 107 B Cleaning of abnormal segments and of segments containing tacks 109 B.1 Tack detection model........................... 109 C Reinforcement Learning Framework 110 C.1 Deep RL agent.............................. 110 D Conversion from nkz to csv 113 Bibliography 114 5 Chapter 1 Introduction 1.1 Motivation Modern racing sailboats are masterpieces of engineering: from materials science to communication technology and aerodynamics, they combine state-of-the-art tech- nology and science. One area seems to be somewhat excluded from this rapid devel- opment, namely that of sailing autopilots. During races they are estimated to take over 95% of the steering, but do so with about 80% of the performance of a human skipper. Hence, there is a large potential to reduce this discrepancy between man and machine by using novel machine learning (ML) methods. Reinforcement Learn- ing (RL) is particularly interesting for this purpose, as it can theoretically result in algorithms that outperform human behaviour. However, it can only deliver satisfac- tory results if a satisfactory RL simulation environment is available to train it. Such a training environment corresponds to a time series forecasting model that is able to predict the next values of the measures that define the state of the sailboat and of its environment. Indeed, the accurate prediction of the latter variables allows to provide feedback to the RL algorithm about the consequences of its actions. The present project is concerned with the optimisation of this RL simulation environ- ment, i.e. with the elaboration of accurate time series forecasting models. This study is the fourth in a series