Stochastic Processes and Applications
Total Page:16
File Type:pdf, Size:1020Kb
STOCHASTIC PROCESSES AND APPLICATIONS G.A. Pavliotis Department of Mathematics Imperial College London November 11, 2015 2 Contents 1 Stochastic Processes 1 1.1 Definition of a Stochastic Process . ............ 1 1.2 StationaryProcesses .... ...... ..... ...... ...... .. ......... 3 1.3 BrownianMotion .................................. ...... 10 1.4 Examples of Stochastic Processes . ............ 13 1.5 The Karhunen-Lo´eve Expansion . ........... 16 1.6 Discussion and Bibliography . ........... 20 1.7 Exercises ....................................... ..... 21 2 Diffusion Processes 27 2.1 Examples of Markov processes . .......... 27 2.2 The Chapman-Kolmogorov Equation . ........... 30 2.3 The Generator of a Markov Processes . ........... 34 2.4 Ergodic Markov processes . ......... 37 2.5 The Kolmogorov Equations . ......... 39 2.6 Discussion and Bibliography . ........... 46 2.7 Exercises ....................................... ..... 47 3 Introduction to SDEs 49 3.1 Introduction.................................... ....... 49 3.2 The Itˆoand Stratonovich Stochastic Integrals . .................. 52 3.3 SolutionsofSDEs................................. ....... 57 3.4 Itˆo’sformula................................... ........ 59 3.5 ExamplesofSDEs .................................. ..... 65 3.6 Lamperti’s Transformation and Girsanov’s Theorem . .................. 68 3.7 Linear Stochastic Differential Equations . ................. 70 3.8 Discussion and Bibliography . ........... 73 3.9 Exercises ....................................... ..... 74 4 The Fokker-Planck Equation 77 4.1 Basic properties of the Fokker-Planck equation . ................. 77 4.2 Examples of the Fokker-Planck Equation . ............. 81 i 4.3 Diffusion Processes in One Dimension . ............. 87 4.4 The Ornstein-Uhlenbeck Process . ............ 90 4.5 TheSmoluchowskiEquation . ......... 96 4.6 Reversible Diffusions . ..........101 4.7 Eigenfunction Expansions . ..........106 4.8 MarkovChainMonteCarlo. ........108 4.9 Reduction to a Schr¨odinger Operator . ..............110 4.10 Discussion and Bibliography . ............114 4.11Exercises ...................................... 118 Appendix A Frequently Used Notation 123 Appendix B Elements of Probability Theory 125 B.1 Basic Definitions from Probability Theory . ...............125 B.2 RandomVariables................................. .......127 B.3 Conditional Expecation . ..........131 B.4 The Characteristic Function . ............132 B.5 Gaussian Random Variables . .........133 B.5.1 Gaussian Measures in Hilbert Spaces . ..........135 B.6 Types of Convergence and Limit Theorems . ............138 B.7 Discussion and Bibliography . ...........139 Index 141 Bibliography 145 ii Chapter 1 Introduction to Stochastic Processes In this chapter we present some basic results from the theory of stochastic processes and investigate the properties of some of the standard continuous-time stochastic processes. In Section 1.1 we give the definition of a stochastic process. In Section 1.2 we present some properties of stationary stochastic processes. In Section 1.3 we introduce Brownian motion and study some of its properties. Various examples of stochastic processes in continuous time are presented in Section 1.4. The Karhunen-Loeve expansion, one of the most useful tools for representing stochastic processes and random fields, is presented in Section 1.5. Further discussion and bibliographical comments are presented in Section 1.6. Section 1.7 contains exercises. 1.1 Definition of a Stochastic Process Stochastic processes describe dynamical systems whose time-evolution is of probabilistic nature. The pre- cise definition is given below.1 Definition 1.1 (stochastic process). Let T be an ordered set, (Ω, , P) a probability space and (E, ) a F G measurable space. A stochastic process is a collection of random variables X = X ; t T where, for { t ∈ } each fixed t T , X is a random variable from (Ω, , P) to (E, ). Ω is known as the sample space, where ∈ t F G E is the state space of the stochastic process Xt. The set T can be either discrete, for example the set of positive integers Z+, or continuous, T = R+. The state space E will usually be Rd equipped with the σ-algebra of Borel sets. A stochastic process X may be viewed as a function of both t T and ω Ω. We will sometimes write ∈ ∈ X(t), X(t,ω) or X (ω) instead of X . For a fixed sample point ω Ω, the function X (ω) : T E is t t ∈ t → called a (realization, trajectory) of the process X. Definition 1.2 (finite dimensional distributions). The finite dimensional distributions (fdd) of a stochastic k process are the distributions of the E -valued random variables (X(t1), X(t2),...,X(tk)) for arbitrary positive integer k and arbitrary times t T, i 1,...,k : i ∈ ∈ { } F (x)= P(X(ti) 6 xi, i = 1,...,k) with x = (x1,...,xk). 1The notation and basic definitions from probability theory that we will use can be found in Appendix B. 1 From experiments or numerical simulations we can only obtain information about the finite dimensional distributions of a process. A natural question arises: are the finite dimensional distributions of a stochastic process sufficient to determine a stochastic process uniquely? This is true for processes with continuous paths 2, which is the class of stochastic processes that we will study in these notes. Definition 1.3. We say that two processes Xt and Yt are equivalent if they have same finite dimensional distributions. Gaussian stochastic processes A very important class of continuous-time processes is that of Gaussian processes which arise in many applications. Definition 1.4. A one dimensional continuous time Gaussian process is a stochastic process for which E = R and all the finite dimensional distributions are Gaussian, i.e. every finite dimensional vector (X , X ,...,X ) is a (µ , K ) random variable for some vector µ and a symmetric nonnegative t1 t2 tk N k k k definite matrix Kk for all k = 1, 2,... and for all t1,t2,...,tk. From the above definition we conclude that the finite dimensional distributions of a Gaussian continuous- time stochastic process are Gaussian with probability distribution function n/2 1/2 1 1 γ (x) = (2π)− (detK )− exp K− (x µ ), x µ , µk,Kk k −2 k − k − k where x = (x1,x2,...xk). It is straightforward to extend the above definition to arbitrary dimensions. A Gaussian process x(t) is characterized by its mean m(t) := Ex(t) and the covariance (or autocorrelation) matrix C(t,s)= E x(t) m(t) x(s) m(s) . − ⊗ − Thus, the first two moments of a Gaussian process are sufficien t for a complete characterization of the process. It is not difficult to simulate Gaussian stochastic processes on a computer. Given a random number generator that generates (0, 1) (pseudo)random numbers, we can sample from a Gaussian stochastic pro- N cess by calculating the square root of the covariance. A simple algorithm for constructing a skeleton of a continuous time Gaussian process is the following: Fix ∆t and define t = (j 1)∆t, j = 1,...N. • j − N N N N N N Set Xj := X(tj) and define the Gaussian random vector X = Xj . Then X (µ , Γ ) • j=1 ∼N N N with µ = (µ(t1),...µ(tN )) and Γij = C(ti,tj). 2In fact, all we need is the stochastic process to be separable See the discussion in Section 1.6. 2 Then XN = µN + Λ (0,I) with ΓN = ΛΛT . • N We can calculate the square root of the covariance matrix C either using the Cholesky factorization, via the spectral decomposition of C, or by using the singular value decomposition (SVD). Examples of Gaussian stochastic processes Random Fourier series: let ξ , ζ (0, 1), i = 1,...N and define • i i ∼N N X(t)= (ξj cos(2πjt)+ ζj sin(2πjt)) . j=1 Brownian motion is a Gaussian process with m(t) = 0, C(t,s) = min(t,s). • Brownian bridge is a Gaussian process with m(t) = 0, C(t,s) = min(t,s) ts. • − The Ornstein-Uhlenbeck process is a Gaussian process with m(t) = 0, C(t,s) = λ e α t s with • − | − | α, λ> 0. 1.2 Stationary Processes In many stochastic processes that appear in applications their statistics remain invariant under time transla- tions. Such stochastic processes are called stationary. It is possible to develop a quite general theory for stochastic processes that enjoy this symmetry property. It is useful to distinguish between stochastic pro- cesses for which all finite dimensional distributions are translation invariant (strictly stationary processes) and processes for which this translation invariance holds only for the first two moments (weakly stationary processes). Strictly Stationary Processes Definition 1.5. A stochastic process is called (strictly) stationary if all finite dimensional distributions are invariant under time translation: for any integer k and times t T , the distribution of (X(t ), X(t ),...,X(t )) i ∈ 1 2 k is equal to that of (X(s+t ), X(s+t ),...,X(s+t )) for any s such that s+t T for all i 1,...,k . 1 2 k i ∈ ∈ { } In other words, P(X A , X A ...X A )= P(X A , X A ...X A ), s T. t1+s ∈ 1 t2+s ∈ 2 tk+s ∈ k t1 ∈ 1 t2 ∈ 2 tk ∈ k ∀ ∈ Example 1.6. Let Y0, Y1,... be a sequence of independent, identically distributed random variables and consider the stochastic process Xn = Yn. Then Xn is a strictly stationary process (see Exercise 1). Assume furthermore that EY = µ< + . Then, by the strong law of large numbers, Equation (B.26), we have that 0 ∞ N 1 N 1 1 − 1 − X = Y EY = µ, N j N j → 0 j=0 j=0 3 almost surely. In fact, the Birkhoff ergodic theorem states that, for any function f such that Ef(Y ) < + , 0 ∞ we have that N 1 1 − lim f(Xj)= Ef(Y0), (1.1) N + N → ∞ j=0 almost surely. The sequence of iid random variables is an example of an ergodic strictly stationary processes. We will say that a stationary stochastic process that satisfies (1.1) is ergodic. For such processes we can calculate expectation values of observable, Ef(Xt) using a single sample path, provided that it is long enough (N 1).