A new deterministic Ensemble Kalman Filter with one-step-ahead smoothing for storm surge forecasting Thesis by Naila Raboudi In Partial Fulfillment of the Requirements For the Degree of Masters of Science King Abdullah University of Science and Technology Thuwal, Kingdom of Saudi Arabia November, 2016 2 EXAMINATION COMMITTEE PAGE The thesis of Naila Raboudi is approved by the examination committee Committee Chairperson: Ibrahim Hoteit Committee Members: Omar Knio, Shuyu Sun, Boujemaa Ait-El-Fquih 3 ©November, 2016 Naila Raboudi All Rights Reserved 4 ABSTRACT A new deterministic Ensemble Kalman Filter with one-step-ahead smoothing for storm surge forecasting Naila Raboudi The Ensemble Kalman Filter (EnKF) is a popular data assimilation method for state- parameter estimation. Following a sequential assimilation strategy, it breaks the problem into alternating cycles of forecast and analysis steps. In the forecast step, the dynamical model is used to integrate a stochastic sample approximating the state analysis distribution (called analysis ensemble) to obtain a forecast ensemble. In the analysis step, the forecast ensemble is updated with the incoming observation using a Kalman-like correction, which is then used for the next forecast step. In realistic large-scale applications, EnKFs are implemented with limited ensembles, and often poorly known model errors statistics, leading to a crude approximation of the forecast covariance. This strongly limits the filter performance. Recently, a new EnKF was proposed in [1] following a one-step-ahead smoothing strategy (EnKF-OSA), which involves an OSA smoothing of the state between two successive analysis. At each time step, EnKF-OSA exploits the observation twice. The incoming observation is first used to smooth the ensemble at the previous time step. The resulting smoothed ensemble is then integrated forward to compute a "pseudo forecast" ensemble, which is again updated with the same observation. The idea of constraining the state with future observations is to add more information in the estimation process in order to mitigate for the sub-optimal character of EnKF-like methods. The second EnKF-OSA "forecast" is computed from the smoothed ensemble and should therefore provide an improved background. 5 In this work, we propose a deterministic variant of the EnKF-OSA, based on the Sin- gular Evolutive Interpolated Ensemble Kalman (SEIK) filter. The motivation behind this is to avoid the observations perturbations of the EnKF in order to improve the scheme's behavior when assimilating big data sets with small ensembles. The new SEIK-OSA scheme is implemented and its efficiency is demonstrated by perform- ing assimilation experiments with the highly nonlinear Lorenz model and a realistic setting of the Advanced Circulation (ADCIRC) model configured for storm surge forecasting in the Gulf of Mexico during Hurricane Ike. Key words : Data assimilation, Kalman filter, Ensemble Kalman filter, Singular Evolutive Interpolated Ensemble Kalman filter, Smoothing-based filtering. 6 ACKNOWLEDGEMENTS Because I owe a particular debt of respect to all those who have listened, advised and criticized me, I would like to hereby express them, from my heart, the deepest gratitude through these short lines. Many people assisted me in one way or an other with the elaboration of this work. This Master project is made under the sincere guidance of my advisor, Prof. Ibrahim Hoteit, to whom I would like to extend a special note of gratitude for his generous technical and moral support, for being always patient and willing to assist me with my work and for all the time and effort that he spent helping me. It is also my duty to express my special thanks and appreciations to Dr. Boujemaa Ait El Fquih who has willingly assisted me with his abilities, for all the time and effort he helped me out with and for his valuable guidance, endless support and understanding spirit. I would especially like to express my gratitude towards my amazing family for the love, support, and constant encouragement I have gotten over the years. I undoubtedly could not have done this without their kind co-operation and encouragement which helped me in the completion of this project. I am also indebted to KAUST for the education and the facilities that it is providing. I want also to acknowledge with much appreciation the crucial role of all my professors who educated me during my stay here. Last but not least, many thanks and gratitudes are extended to the highly esteemed members of the examination committee, Prof. Omar Knio and Prof. Shuyu Sun, who accepted to judge my work and who I wish hopefully to be satisfied. 7 TABLE OF CONTENTS Examination Committee Page 2 Copyright 3 Abstract 4 Acknowledgements 6 List of Figures 9 List of Tables 12 1 Introduction 13 2 Kalman Filtering 18 2.1 Introduction . 18 2.2 Bayesian filtering . 18 2.3 The Kalman Filter (KF) . 21 2.4 The (stochastic) Ensemble Kalman filter (EnKF) . 25 2.5 EnKF limitations and and auxiliary methods . 27 2.5.1 EnKF limitations . 27 2.5.2 Methods to mitigate EnKF limitations . 29 2.6 Discussion . 34 3 Ensemble Kalman Filtering with One-Step-Ahead Smoothing 35 3.1 Introduction . 35 3.2 The Kalman filtering with one-step-ahead smoothing (KF-OSA) . 35 3.2.1 Generic algorithm . 36 3.2.2 KF-OSA equations . 37 3.3 Ensemble formulation (EnKF-OSA) . 39 3.3.1 Smoothing step . 40 3.3.2 Analysis step . 41 3.4 Summary of the EnKF-OSA algorithm . 42 8 3.5 Discussion . 43 4 Deterministic Ensemble Kalman Filtering with One-Step-Ahead Smooth- ing 44 4.1 Introduction . 44 4.2 The Singular Evolutive Interpolated Kalman filter (SEIK) . 45 4.3 SEIK filter with One-Step-Ahead smoothing (SEIK-OSA) . 48 4.3.1 Smoothing step . 48 4.3.2 Analysis step . 51 4.4 Summary of the SEIK-OSA algorithms . 63 4.5 Conclusion . 65 5 Numerical Experiments & Results 66 5.1 Introduction . 66 5.2 Numerical experiments with the Lorenz-96 model . 66 5.2.1 Experimental setting . 66 5.2.2 Results and discussion . 69 5.3 Numerical experiments with a storm surge model . 88 5.3.1 An overview of ADCIRC model . 88 5.3.2 Experimental design and implementation . 89 5.3.3 Results and discussion . 96 5.4 Summary and conclusions . 99 6 Concluding Remarks 101 References 104 Appendices 108 9 LIST OF FIGURES 5.1 Time-averaged RMSE as a function of the localization radius (x axis) and inflation factor (y axis). The two filters are implemented with 10 members and assimilation of observations from (top) all model vari- ables (middle) half and (bottom) quarter of the variables at every 4 model time steps (or 24 h in real time). A logarithmic color scale is used to emphasize the low RMSE values. The minimum-averaged RM- SEs are indicated by asterisks, and their associated values are given in the title. White boxes indicate divergence of the filter. 70 5.2 Same as Fig. (5.1), but for 20 ensemble members. 71 5.3 Time-averaged RMSE as a function of the localization radius (x axis) and inflation factor (y axis). The two filters are implemented with 20 members and assimilation of observations every (top) 1 model time step (middle) 2 and (bottom) 4 model time steps. A logarithmic color scale is used to emphasize the low RMSE values. 71 5.4 Same as Fig. (5.1), but for 20 ensemble members and F = 6 (instead of 8) in filters. 73 5.5 Time-averaged RMSE as a function of the localization radius (x axis) and inflation factor (y axis). All filters are implemented with 10 mem- bers and assimilation of observations from (top) all model variables (middle) half and (bottom) quarter of the variables at every 4 model time steps (or 24 h in real time). A logarithmic color scale is used to emphasize the low RMSE values. The minimum-averaged RMSEs are indicated by asterisks, and their associated values are given in the title. White boxes indicate divergence of the filter. 74 5.6 Same as Fig. (5.5), but for 20 ensemble members. 75 5.7 Same as Fig. (5.5), but for 40 ensemble members. 76 5.8 Minimum average RMSE for all tested filters (EnKF-Reg, EnKF-OSA, SEIK-Reg and SEIK-OSA) as a function of the ensemble size. left (All) middle (half) and right (quarter) observations are observed every 4 model time steps. 77 10 5.9 Same as Fig. (5.5), but for 20 ensemble members and assimilation every model time step. 79 5.10 Same as Fig. (5.5), but for 20 ensemble members and assimilation every two model time steps. 80 5.11 Same as Fig. (5.5), but for 20 ensemble members and assimilation every six model time steps. 81 5.12 Minimum average RMSE for all tested filters (EnKF-Reg, EnKF-OSA, SEIK-Reg and SEIK-OSA) as a function of the frequency of observa- tions. (left) All (middle) half and (right) quarter observations are observed every every 4 model steps with Ne=20. 81 5.13 Same as Fig. (5.5), but for 20 ensemble members and observational error variance equal to 0:1......................... 83 5.14 Same as Fig. (5.5), but for 20 ensemble members and observational error variance equal to 2. 84 5.15 Minimum average RMSE for all tested filters (EnKF-Reg, EnKF-OSA, SEIK-Reg and SEIK-OSA) as a function of the measurement error variance. (left) All (middle) half and (right) quarter observations are assimilated every 4 model steps with Ne=20. 85 5.16 Same as Fig. (5.16), but for 20 ensemble members and assimilation every 4 model time steps.