Towards High-order Methods for Stochastic Differential Equations with White Noise: A Spectral Approach by Zhongqiang Zhang A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Division of Applied Mathematics at Brown University PROVIDENCE, RHODE ISLAND May 2014 Abstract of \Towards High-order Methods for Stochastic Differential Equations with White Noise: A Spectral Approach" by Zhongqiang Zhang, Ph.D., Brown University, May 2014 We develop a recursive multistage Wiener chaos expansion method (WCE) and a recursive multi- stage stochastic collocation method (SCM) for numerical integration of linear stochastic advection- diffusion-reaction equations with multiplicative white noise. We show that both methods are com- parably efficient in computing the first two moments of solutions for long time intervals, compared to a direct application of WCE and SCM while both methods are more efficient than the standard Monte Carlo method if high accuracy is required. Both methods belong to Wong-Zakai approximation, where Brownian motion is truncated using its spectral expansion before any discretization in time and space. For computational convenience, WCE is associated with the Ito formulation of underlying equations and SCM is associated with the Stratonovich formulation. We apply SCM using Smolyak's sparse grid construction to obtain the shock location of the one-dimensional piston problem, which is modeled by stochastic Euler equations with multiplica- tive white noise. We show numerically that SCM is efficient for short time simulations and for small magnitudes of noises and quasi-Monte Carlo methods are efficient for moderate large-time simulations. We also illustrate the efficiency of SCM through error estimates for a linear model problem. We further investigate the effect of a spectral approximation of Brownian motion, rather than a piecewise linear approximation, for both spatial and temporal noise. For spatial noise, we con- sider semilinear elliptic equations with additive noise and show that when the solution is smooth enough, the spectral approximation is superior to the piecewise linear approximation while both approximations are comparable when the solution is not smooth. For temporal noise, we use this spectral approach to design numerical schemes for stochastic delay differential equations under the Stratonovich formulation. We show that the spectral approach admits higher-order accuracy only for higher-order schemes. Besides equations with coefficients of linear growth, we also consider stochastic ordinary differ- ential equations with coefficients of polynomial growth. We formulate a basic relationship between local truncation error and global error of numerical methods for these equations, and apply this relationship for our explicit balanced scheme to obtain the convergence order. © Copyright 2014 by Zhongqiang Zhang This dissertation by Zhongqiang Zhang is accepted in its present form by the Division of Applied Mathematics as satisfying the dissertation requirement for the degree of Doctor of Philosophy. Date George Em Karniadakis, Advisor Date Boris Rozovskii, Co-advisor Recommended to the Graduate Council Date Michael V. Tretyakov, Reader Date Marcus Sarkis, Reader Approved by the Graduate Council Date Peter M. Weber, Dean of the Graduate School iii Acknowledgments I would like to thank my advisor Professor George Em Karniadakis and co-advisor Professor Boris L. Rozovskii for their guidance and support during my study at Brown University. Their passion and great vision on scientific research in applied mathematics have inspired me to explore my research field in depth. Professor George Em Karniadakis always reminds of the essence of applied mathematics and shares with me his rich knowledge and research experience, which have helped me appreciate applications beyond mathematics itself. Professor Boris L. Rozovskii introduced his works to me and has encouraged me to be more rigorous in my work. My gratitude to them is actually beyond any words. It is my great fortune to collaborate with Professor Michael V. Tretyakov of University of Nottingham at England. Without valuable discussions with him and invaluable advice from him, I would not have this dissertation in current form, several chapters of which are papers we published together in the last few years. He carefully checked every point leaving nothing to chance, which greatly helped me improve my presentation in clarity and accuracy. I express my sincere thanks to my committee members and readers, Professor Marcus Sarkis and Professor Michael V. Tretyakov, for their precious time in carefully reading my thesis and serving as committee members. I would like to thank Professor Johnny Guzm´anfor discussing finite element methods for elliptic equations with me and Professor Hongjie Dong for his help on energy estimates of some parabolic equations with minimal regularity. I also thank Professor Chau-Hsing Su for his help on the stochastic piston problem. My heart is full of gratitude to people at the Crunch group: Heyrim Cho, Minseok Choi, Mingge Deng, Huan Lei, Xuejin Li, Zhen Li, Yuhang Tang, Daniele Venturi, Alix Witthoft, Xiu Yang, Yue Yu, Summer Zheng and many others for their help and support in life and in research. I would also like to thank Xingjie Li at Brown University, Wanrong Cao at Southeast University, Guang Lin at iv Pacific Northwest National Laboratory, and Xiaoliang Wan at Louisiana State University for many helpful discussions. Many thanks are also due to Ms. Madeline Brewster, Ms. Camille O. Dickson, Ms. Lisa Eklund, Ms. Stephanie Han, Ms. Jean Radican and Ms. Laura Leddy for their timely help. Finally, I have to express my heartfelt thanks to my parents and my sisters for their love and encouragement. I remain indebted to my wife, Fei Yu, for her emotional support and love. I am also grateful to my father- and mother-in-law for their love and support, especially during my hard times. This dissertation is dedicated to my deceased mother. This work was supported by OSD/MURI grant FA9550-09-1-0613, NSF/DMS grant DMS- 0915077, NSF/DMS grant DMS-1216437 and also by the Collaboratory on Mathematics for Meso- scopic Modeling of Materials (CM4) which is sponsored by DOE. v Contents Acknowledgments iv 1 Introduction 1 1.1 Review of numerical methods for SPDEs . .2 1.1.1 Direct semi-discretization methods for parabolic SPDEs . .3 1.1.2 Wong-Zakai approximation for parabolic SPDEs . .6 1.1.3 Preprocessing methods for parabolic SPDEs . .8 1.1.4 Stochastic Burgers and Navier-Stokes equations . 11 1.1.5 Beyond parabolic SPDEs . 13 1.1.6 Stability and convergence of existing numerical methods . 14 1.1.7 Conclusion . 18 1.2 Approximation of Brownian motion . 18 1.2.1 Piecewise linear approximation . 18 1.2.2 Other approximations . 21 1.3 Integration methods in random space . 22 1.3.1 Monte Carlo method and its variants . 22 1.3.2 Wiener chaos expansion method . 23 1.3.3 Stochastic collocation method . 24 1.4 Objectives of this work . 25 vi Part I: Temporal White Noise 29 2 Wiener chaos methods for linear stochastic advection-diffusion-reaction equa- tions 29 2.1 Introduction . 30 2.2 WCE of the SPDE solution . 31 2.3 Multistage WCE method . 33 2.4 Algorithm for computing moments . 39 2.5 Numerical tests in one dimension . 43 2.5.1 Test problems . 43 2.5.2 Application of WCE algorithms to the model problem . 44 2.5.3 Numerical results . 48 2.5.4 Comparison of the WCE algorithm and Monte Carlo-type algorithms . 50 2.6 Numerical tests with passive scalar equation . 55 2.7 Summary . 61 3 Stochastic collocation methods for differential equations with white noise 63 3.1 Introduction . 64 3.2 Sparse grid for weak integration of SDE . 67 3.2.1 Smolyak's sparse grid . 67 3.2.2 Weak-sense integration of SDE . 70 3.2.3 Illustrative examples . 75 3.3 Recursive collocation algorithm for linear SPDE . 84 3.4 Numerical experiments . 88 4 Comparison between Wiener chaos methods and stochastic collocation methods 96 4.1 Introduction . 97 vii 4.2 Review of Wiener chaos and stochastic collocation . 99 4.2.1 Wiener chaos expansion (WCE) . 100 4.2.2 Stochastic collocation method (SCM) . 101 4.3 Error estimates . 103 4.4 Numerical results . 106 4.5 Proofs . 115 4.5.1 Proof of Theorem 4.3.1 . 115 4.5.2 Proof of Theorem 4.3.3 . 120 4.6 Appendix: derivation of Algorithm 4.2.1. 123 5 Application example: stochastic collocation methods for 1D stochastic piston problem 125 5.1 Introduction . 126 5.2 Theoretical Background . 128 5.2.1 Stochastic Euler equations . 130 5.3 Verification of the Stratonovich- and Ito-Euler equations . 132 5.3.1 A splitting method for stochastic Euler equations . 133 5.3.2 Stratonovich-Euler equations versus first-order perturbation analysis . 134 5.3.3 Stratonovich-Euler equations versus Ito-Euler equations . 135 5.4 The Stochastic Collocation Method . 137 5.5 Summary . 142 5.6 Appendix: a first-order model driven by different Gaussian processes . 145 Part II: Spatial White Noise 150 6 Semilinear Elliptic Equations with Additive Noise 150 6.1 Introduction . 150 viii 6.2 Main results . 153 6.2.1 Semidiscrete and fully discrete schemes . 154 6.2.2 Strong and weak convergence order . 155 6.2.3 Examples of other PDEs . 158 6.3 Numerical results . 160 6.4 Proofs . 163 6.4.1 Regularity of the solution to (6.1.1). 164 6.4.2 Proofs for strong convergence order . 166 6.4.3 Proofs for weak convergence order . 167 6.4.4 Proof of Theorem 6.2.9 . 172 6.4.5 Proof of Theorem 6.2.5 . 174 6.4.6 Proof of Theorem 6.2.7 . 175 Part III: Stochastic Ordinary Differential Equations 180 7 A Fundamental Limit Theorem for SDEs with non-Lipschitz coefficients 180 7.1 Introduction .