Machine Learning of Atmospheric Turbulence in a Variational Multiscale Model MSc Thesis Martin Janssens Machine Learning of Atmospheric Turbulence in a Variational Multiscale Model MSc Thesis by Martin Janssens to obtain the degree of Master of Science at the Delft University of Technology, to be defended publicly on Tuesday August 27, 2019 at 10:00 AM. Student number: 4275780 Project duration: September 16, 2018 – August 28, 2019 Supervisor: Dr. S. J. Hulshoff, TU Delft Thesis committee: Dr. R. P.Dwight, TU Delft Dr. B. Chen, TU Delft An electronic version of this thesis is available at http://repository.tudelft.nl/. Cover image: The Starry Night, Vincent van Gogh Abstract Today’s leading projections of climate change predicate on atmospheric General Circulation Models (GCMs). Since the atmosphere consists of a staggering range of scales that impact global trends, but computational constraints prevent many of these scales from being directly represented in numerical simulations, GCMs require “parameterisations” - models for the influence of unresolved processes on the resolved scales. State- of-the-art parameterisations are commonly based on combinations of phenomenological arguments and physics, and are of considerably lower fidelity than the resolved simulation. In particular, the parameter- isation of low-altitude stratocumulus clouds that result from small-scale processes in sub-tropical marine boundary layers is widely considered the largest source of uncertainty that remains in contemporary GCMs’ prediction of the temperature response to a global increase in CO2. Improvements in the capacity of machine learning algorithms and the increasing availability of high- fidelity datasets from global satellite data and local Large Eddy Simulations (LES) have identified data-driven parameterisations as a high-potential option to break the deadlock. However, early contributions in this field still rely on inconsistent multiscale model formulations and are plagued by instability. To sketch a clearer picture on the sources of the accuracy and instability of data-driven parameterisations, this work proposes a framework in which no assumptions on the model form are made, building on a recent MSc thesis by Michel Robijns at the TU Delft. It uses Artificial Neural Networks (ANNs) to infer exact projections of the unresolved scales processes on the resolved degrees of freedom. These “interaction terms” naturally arise from Varia- tional Multiscale (VMS) model formulations. The resulting VMM-ANN framework limits error to the data- driven interaction term approximations, offering explicit insight into their functioning. The model is assessed in the context of a statistically stationary convective boundary layer turbulence problem, which is further reduced to a one-dimensional, forced inviscid Burgers’ equation. Simple, feedfor- ward ANNs with relatively local input stencils that are trained on error-free data a priori to inserting them in forward simulations (offline) are capable of predicting the interaction terms of this problem much better than traditional, algebraic VMS closures in offline settings at various levels of discretisation; they also generalise well to uncorrelated instances of the turbulence. However, this performance does not translate to simula- tions of forward problems. It is shown that the model suffers from instability due to i) ill-posed nonlinear solution procedures and ii) self-inflicted error accumulation. These correspond to two dimensions of for- ward simulations that are not accounted for by offline training on error-free data. The first instability mode can in some situations be dealt with by reformulating the VMM-ANN model architecture; the second is con- jectured to demand training strategies that account for the self-induced errors. Finally, despite scaling well, the framework is still found to be computationally expensive compared to a state-of-the-art model. In all, appreciable challenges therefore remain in order to capitalise on the promise offered by ANNs to improve the parameterisation of clouds in GCMs in particular and turbulence in general. iii Preface This thesis is the culmination of my MSc with the Aerodynamics group at the faculty of Aerospace Engineering at Delft University of Technology. It also brings me to a juncture where I conclude my engineering aspirations and continue on the path of atmospheric research. The document you are currently reading lives on this juncture. It attempts to marry recent progress on data-driven models intended for simulating turbulent flows encountered in engineering to the modelling of processes that drive the large scales of the atmosphere. In this context, I have endeavoured to contribute the thoughts of an educated engineer to the development of the atmospheric models upon which the prediction of climate change, one of the defining challenges of my generation, rests. I am grateful to be able to continue to devote my time to the understanding of and the solutions to this challenge in the coming years. This study would never have been possible without the considerable insight, vision, numerical modelling expertise and skillful supervision of Dr. Steven J. Hulshoff. Steve, I am incredibly privileged to have been the beneficiary of uncountable one-liners and animated discussions that ran way over the time you had for them, but also of your solely constructive support, expedient feedback and remarkable investment in the work. The successful completion of this thesis would also not have been possible without Prof. Harm J.J. Jonker and Prof. Pier Siebesma of the Geosciences and Remote Sensing group at the TU Delft, who initially gave the right perspective for this work and very kindly provided both a test case and its data. I am truly thankful to you both. Thanks also to Michel Robijns for building the foundation upon which this work rests, and for our interesting conversations in the early stages of the project. There are many more people to whom I am deeply indebted. First and foremost, I would like to thank my parents for their unconditional love and support throughout this journey. The values you instilled in me and the conditions you provided for me to succeed are the single most important reason for my successful completion of this work. A special thanks goes to my father, who committed to proofread every single word of this document. Carol, thank you, thank you for not only sticking with me through the long nights and weekends I should have spent with you rather than working, but for keeping me sane and grounded. I cannot wait to take on the world with you. Finally, I am profoundly thankful to all my friends and family; you gave colour to my years as a student. Rens, thanks for turning each of the long, dark hours of our MSc into a lively experience, you are a truly remarkable friend and probably the only person who knows what finishing this work means. Robin, thank you for being a guiding force and inspiration since the very first day of my studies; Max, thank you for making the Fellowship bearable and for teaching me how to enjoy life outside it. I cannot wait to continue to spend good times together with you in the future. Thanks also to Arent, Joël, Joris, the “Belgjes” and everyone else who made the basement a slightly warmer place. Martin Janssens Delft, August 2019 v Contents List OF Figures XI List OF TABLES XV AcronYMS XVII List OF Symbols XIX 1 Introduction 1 1.1 Background............................................1 1.2 Research Objective and Research Questions...........................2 1.3 Thesis Outline..........................................2 2 Atmospheric Modelling3 2.1 Problem Formulation.......................................3 2.1.1 Mathematical Model....................................3 2.1.2 Energy Spectrum of Atmospheric Turbulence.......................4 2.2 General circulation models....................................6 2.2.1 Modelling Approach....................................6 2.2.2 Applications........................................6 2.2.3 Assumptions and Approximations.............................7 2.2.4 Discussion.........................................8 2.3 Global Cloud-Resolving Models..................................9 2.3.1 Modelling Approach.................................... 10 2.3.2 Applications........................................ 10 2.3.3 Assumptions and Approximations............................. 10 2.3.4 Discussion......................................... 11 2.4 Superparameterisation: The Multiscale Modelling Approach................... 11 2.4.1 Heterogeneous Multiscale Methods............................ 12 2.4.2 Modelling Approach.................................... 12 2.4.3 Applications........................................ 14 2.4.4 Assumptions and Approximations............................. 14 2.5 Four Improvement Areas for SP.................................. 14 2.5.1 Model Consistency..................................... 15 2.5.2 Low Cloud Simulation Accuracy.............................. 17 2.5.3 Computational Cost.................................... 17 2.5.4 Scale Separation...................................... 18 2.6 Summary and Outlook...................................... 19 3 A VARIATIONAL Multiscale AND Machine LEARNING Modelling FRAMEWORK 21 3.1 Conceptual Modelling Framework................................ 21 3.2 Variational Multiscale Methods.................................. 22 3.2.1 Modelling Approach.................................... 23 3.2.2 Model Equations.....................................