SIMULATING GAUSSIAN RANDOM FIELDS AND SOLVING STOCHASTIC DIFFERENTIAL EQUATIONS USING BOUNDED WIENER INCREMENTS A thesis submitted to the University of Manchester for the degree of Doctor of Philosophy in the Faculty of Engineering and Physical Sciences 2013 Phillip Taylor School of Mathematics Contents Abstract 9 Declaration 10 Copyright Statement 11 Publications 12 Acknowledgements 13 List of abbreviations 14 Notation 15 1 Introduction 20 1.1 Part I: Simulation of random fields . 20 1.2 Implementation of the circulant embedding method in the NAG Fortran Library . 22 1.3 Part II: Solution of SDEs . 24 I Simulating Gaussian Random Fields 26 2 Stochastic Processes and Random Fields 27 2.1 Random Variables . 27 2.1.1 The Gaussian distribution . 30 2.2 Stochastic Processes . 33 2.2.1 Martingales . 34 2.2.2 The Wiener process . 35 2 2.2.3 The Brownian bridge process . 36 2.2.4 Fractional Brownian Motion . 37 2.2.5 Simulating the Wiener process . 38 2.3 Random Fields . 40 3 Linear Algebra 44 3.1 Discrete Fourier Transforms . 45 3.2 Toeplitz and Circulant Matrices . 48 3.3 Embeddings of Toeplitz matrices into circulant matrices . 49 3.4 Block Toeplitz and Block Circulant Matrices . 54 3.4.1 Additional notation for BSTSTB and symmetric BTTB matrices 56 3.4.2 Notation for padding . 57 3.5 Embeddings of BSTSTB and symmetric BTTB matrices into BSCSCB and symmetric BCCB matrices . 60 3.6 Matrix Factorisations . 61 4 The Circulant Embedding Method 64 4.1 Introduction . 64 4.2 The circulant embedding method in one dimension . 67 4.3 The circulant embedding method in two dimensions . 69 4.4 Example: Simulating Fractional Brownian Motion using the circulant embedding method . 73 4.5 Approximation in the case that a non-negative definite embedding ma- trix cannot be found . 75 4.6 Cases for which the embedding matrix is always non-negative definite . 76 4.7 Implementation in the NAG Fortran Library . 77 4.7.1 The NAG Library routines . 78 4.7.2 List of variograms for G05ZNF and G05ZRF ............ 82 4.8 Numerical examples . 84 4.8.1 Notes on the numerical examples . 85 4.8.2 Stringent tests . 93 3 II Solving Stochastic Differential Equations 94 5 Background Material for SDEs 95 5.1 Stochastic Differential Equations . 95 5.1.1 Stochastic integration . 95 5.1.2 Definition of SDEs . 96 5.1.3 The It^oformula . 98 5.1.4 Existence and uniqueness of solutions for SDEs . 99 5.1.5 Notation for stochastic integrals . 99 5.1.6 The It^o{Taylor and Stratonovich{Taylor Theorems . 103 5.1.7 Numerical schemes for solving SDEs . 105 5.1.8 Local truncation errors for numerical schemes . 110 5.1.9 Types of global error and order of a numerical scheme . 111 5.1.10 Stopping times . 113 5.2 Important results for pathwise error analysis of SDEs . 113 5.2.1 The Burkholder{Davis{Gundy inequalities and Kolmogorov{ Centsovˇ Theorem . 114 5.2.2 Bounds for stochastic integrals . 115 5.2.3 Local truncation error bounds for explicit numerical schemes . 118 5.2.4 The Dynkin formula . 119 5.2.5 Modulus of continuity of stochastic processes . 120 6 Existing Variable Stepsize Methods 123 6.1 Numerical Schemes . 123 6.1.1 Explicit numerical schemes . 124 6.1.2 Implicit numerical schemes . 126 6.1.3 Balanced schemes . 127 6.2 Variable stepsize methods . 127 6.2.1 Example to show non-convergence of the Euler{Maruyama scheme with a variable stepsize method . 128 6.2.2 Variable stepsize methods based on the Euler{Maruyama and Milstein schemes . 131 6.2.3 Variable stepsize methods based on other schemes . 135 4 6.3 Two examples of variable stepsize methods . 135 6.3.1 Method proposed by Gaines and Lyons . 136 6.3.2 Method proposed by Lamba . 140 6.3.3 A comparison of the two methods . 144 7 Simulating a Wiener Process 146 7.1 Deriving the PDE to solve for bounded diffusion . 147 7.1.1 Background . 147 7.2 Simulating the exit time for a one-dimensional Wiener process . 151 7.2.1 Separation of variables . 152 7.2.2 Using a Green's function . 154 7.3 Simulating the exit point of a Wiener process if τ = Tinc ........ 162 7.3.1 Separation of variables . 162 7.3.2 Using a Green's function . 164 7.4 Simulating the exit point and exit time of a one-dimensional Wiener process with unbounded stepsize and bounded stepsize . 166 7.5 Simulating the exit time of a multi-dimensional Wiener process . 168 7.5.1 Simulating the exit point of a multi-dimensional Wiener process if τ = Tinc .............................. 171 7.5.2 Simulating the exit point of the multi-dimensional Wiener pro- cess if τ < Tinc ............................ 172 7.5.3 The final algorithm for simulating the bounded Wiener process . 173 7.6 Using bounded increments to generate a path of the Wiener process . 174 8 Solving SDEs using Bounded Wiener Increments 176 8.1 Background . 177 1 8.1.1 Pathwise global error of order γ − 2 for numerical schemes in Sγ 179 8.1.2 Pathwise global error of order γ for numerical schemes in Sγ .. 181 8.2 The Bounded Increments with Error Control (BIEC) Method . 181 8.3 Numerical results . 183 8.3.1 The numerical methods used . 183 8.3.2 The numerical results . 184 8.3.3 Conclusions from the numerical experiments . 196 5 8.3.4 Future directions . 198 Bibliography 200 Word count 38867 6 List of Figures 4.1 Four realisations of the one-dimensional random field with variogram (4.11), with σ2 = 1, ` = 0:01 and ν = 0:5. 86 4.2 Four realisations of the one-dimensional random field with variogram (4.11), with σ2 = 1, ` = 0:01 and ν = 2:5. 86 4.3 Four realisations of the one-dimensional random field with variogram (4.11), with σ2 = 1, ` = 0:002 and ν = 0:5. 87 4.4 Four realisations of the one-dimensional random field with variogram (4.11), with σ2 = 1, ` = 0:002 and ν = 2:5. 87 4.5 Plots of the variogram (4.11) for the different values of ` and ν..... 88 4.6 Four realisations of fractional Brownian motion with Hurst parameter H = 0:2.................................... 88 4.7 Four realisations of fractional Brownian motion with Hurst parameter H = 0:8.................................... 89 4.8 Four realisations of the two-dimensional random field with variogram 2 (4.10), with σ = 1, `1 = 0:25, `2 = 0:5 and ν = 1:1. 89 4.9 Four realisations of the two-dimensional random field with variogram 2 (4.10), with σ = 1, `1 = 0:5, `2 = 0:25 and ν = 1:8. 90 4.10 Four realisations of the two-dimensional random field with variogram 2 (4.10), with σ = 1, `1 = 0:05, `2 = 0:1 and ν = 1:1. 90 4.11 Four realisations of the two-dimensional random field with variogram 2 (4.10), with σ = 1, `1 = 0:1, `2 = 0:05 and ν = 1:8. 91 4.12 Plots of the variogram (4.10) in one dimension for different values of ` and ν..................................... 91 6.1 An illustration of the Brownian tree, for K = 4. 137 7 6.2 An illustration to show how each element of the Wiener increments ∆W 2k−1;`+1 and ∆W 2k;`+1 are generated from ∆W k;`, for s = (k − ∆tmax ∆tmax 1) 2` , t = k 2` .............................. 138 7.1 The possible cases for the exit points of X(t) from G, with m = 1, X(0) = 0 and G = (−r; r).......................... 150 7.2 The possible cases for the behaviour of X(t), with m = 1, X(0) = 0 and G = (−r; r). .............................. 150 8.1 Numerical results over 1000 samples for geometric Brownian motion with d = m = 1, a = 0:1 and b = 0:1.................... 187 8.2 Numerical results over 1000 samples for geometric Brownian motion with d = m = 1, a = 1:5 and b = 0:1.................... 188 8.3 Numerical results over 1000 samples for geometric Brownian motion with d = m = 1, a = 0:1 and b = 1:2.................... 189 8.4 Numerical results over 1000 samples for geometric Brownian motion with d = m = 1, a = 1:5 and b = 1:2.................... 190 8.5 Numerical results over 1000 samples for geometric Brownian motion with d = m =2................................ 191 8.6 Numerical results over 1000 samples for (8.29) with a = 1 and b = 0:1. 192 8.7 Numerical results over 1000 samples for (8.29) with a = 1 and b = 1. 193 8.8 Numerical results over 1000 samples for (8.31) with a = 0:2. 194 8.9 Numerical results over 1000 samples for (8.31) with a = 2. 195 8 The University of Manchester Phillip Taylor Doctor of Philosophy Simulating Gaussian Random Fields and Solving Stochastic Differential Equations using Bounded Wiener Increments January 28, 2014 This thesis is in two parts. Part I concerns simulation of random fields using the circulant embedding method, and Part II studies the numerical solution of stochastic differential equations (SDEs). A Gaussian random field Z(x) is a set of random variables parameterised by a variable x 2 D ⊂ Rd for some d ≥ 1 which, when sampled at a finite set of sampling points, produces a random vector with a multivariate Gaussian distribution. Such a random vector has a covariance matrix A which must be factorised as either A = RT R for a real-valued matrix R, or A = R∗R for a complex-valued matrix R, in order to take realisations of Z(x).