Lecture 2: Itô Calculus and Stochastic Differential Equations Simo Särkkä Aalto University, Finland (visiting at Oxford University, UK) November 14, 2013 Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 1 / 34 Contents 1 Introduction 2 Stochastic integral of Itô 3 Itô formula 4 Solutions of linear SDEs 5 Non-linear SDE, solution existence, etc. 6 Summary Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 2 / 34 SDEs as white noise driven differential equations During the last lecture we treated SDEs as white-noise driven differential equations of the form dx = f(x; t) + L(x; t) w(t); dt For linear equations the approach worked ok. But there is something strange going on: The use of chain rule of calculus led to wrong results. With non-linear differential equations we were completely lost. Picard-Lindelöf theorem did not work at all. The source of all the problems is the everywhere discontinuous white noise w(t). So how should we really formulate SDEs? Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 4 / 34 Equivalent integral equation Integrating the differential equation from t0 to t gives: Z t Z t x(t) − x(t0) = f(x(t); t) dt + L(x(t); t) w(t) dt: t0 t0 The first integral is just a normal Riemann/Lebesgue integral. The second integral is the problematic one due to the white noise. This integral cannot be defined as Riemann, Stieltjes or Lebesgue integral as we shall see next. Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 6 / 34 Attempt 1: Riemann integral In the Riemannian sense the integral would be defined as Z t X ∗ ∗ ∗ L(x(t); t) w(t) dt = lim L(x(t ); t ) w(t )(tk+1 − tk ); n!1 k k k t0 k ∗ where t0 < t1 < : : : < tn = t and tk 2 [tk ; tk+1]. ∗ Upper and lower sums are defined as the selections of tk such ∗ ∗ ∗ that the integrand L(x(tk ); tk ) w(tk ) has its maximum and minimum values, respectively. The Riemann integral exists if the upper and lower sums converge to the same value. Because white noise is discontinuous everywhere, the Riemann integral does not exist. Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 7 / 34 Attempt 2: Stieltjes integral Stieltjes integral is more general than the Riemann integral. In particular, it allows for discontinuous integrands. We can interpret the increment w(t) dt as increment of another process β(t) such that Z t Z t L(x(t); t) w(t) dt = L(x(t); t) dβ(t): t0 t0 It turns out that a suitable process for this purpose is the Brownian motion— Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 8 / 34 Brownian motion Brownian motion 1 Gaussian increments: 1.8 1.6 ∆β ∼ ( ; ∆ ); 1.4 k N 0 Q tk 1.2 1 where ∆β = β(t ) − β(t ) and 0.8 k k+1 k 0.6 ∆tk = tk+1 − tk . 0.4 0.2 2 0 Non-overlapping increments are 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 independent. Q is the diffusion matrix of the Brownian motion. Brownian motion t 7! β(t) has discontinuous derivative everywhere. White noise can be considered as the formal derivative of Brownian motion w(t) = dβ(t)/dt . Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 9 / 34 Attempt 2: Stieltjes integral (cont.) Stieltjes integral is defined as a limit of the form Z t X ∗ ∗ L(x(t); t) dβ = lim L(x(t ); t )[β(tk+1) − β(tk )]; n!1 k k t0 k ∗ where t0 < t1 < : : : < tn and tk 2 [tk ; tk+1]. ∗ The limit tk should be independent of the position on the interval ∗ tk 2 [tk ; tk+1]. But for integration with respect to Brownian motion this is not the case. Thus, Stieltjes integral definition does not work either. Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 10 / 34 Attempt 3: Lebesgue integral In Lebesgue integral we could interpret β(t) to define a “stochastic measure” via β((u; v)) = β(u) − β(v). Essentially, this will also lead to the definition Z t X ∗ ∗ L(x(t); t) dβ = lim L(x(t ); t )[β(tk+1) − β(tk )]; n!1 k k t0 k ∗ where t0 < t1 < : : : < tn and tk 2 [tk ; tk+1]. ∗ Again, the limit should be independent of the choice tk 2 [tk ; tk+1]. Also our “measure” is not really a sensible measure at all. ) Lebesgue integral does not work either. Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 11 / 34 Attempt 4: Itô integral The solution to the problem is the Itô stochastic integral. ∗ The idea is to fix the choice to tk = tk , and define the integral as Z t X L(x(t); t) dβ(t) = lim L(x(tk ); tk )[β(tk+1) − β(tk )]: n!1 t0 k This Itô stochastic integral turns out to be a sensible definition of the integral. However, the resulting integral does not obey the computational rules of ordinary calculus. Instead of ordinary calculus we have Itô calculus. Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 12 / 34 Itô stochastic differential equations Consider the white noise driven ODE dx = f(x; t) + L(x; t) w(t): dt This is actually defined as the Itô integral equation Z t Z t x(t) − x(t0) = f(x(t); t) dt + L(x(t); t) dβ(t); t0 t0 which should be true for arbitrary t0 and t. Settings the limits to t and t + dt, where dt is “small”, we get dx = f(x; t) dt + L(x; t) dβ: This is the canonical form of an Itô SDE. Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 13 / 34 Connection with white noise driven ODEs Let’s formally divide by dt, which gives dx dβ = f(x; t) + L(x; t) : dt dt Thus we can interpret dβ/dt as white noise w. Note that we cannot define more general equations dx(t) = f(x(t); w(t); t); dt because we cannot re-interpret this as an Itô integral equation. White noise should not be thought as an entity as such, but it only exists as the formal derivative of Brownian motion. Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 14 / 34 Stochastic integral of Brownian motion Consider the stochastic integral Z t β(t) dβ(t) 0 where β(t) is a standard Brownian motion (Q = 1). Based on the ordinary calculus we would expect the result β2(t)=2—but it wrong. If we select a partition 0 = t0 < t1 < : : : < tn = t, we get Z t X β(t) dβ(t) = lim β(tk )[β(tk+1) − β(tk )] 0 k " X 1 = lim − (β(t ) − β(t ))2 2 k+1 k k # 1 + (β2(t ) − β2(t )) 2 k+1 k Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 16 / 34 Stochastic integral of Brownian motion (cont.) We have X 1 1 lim − (β(t ) − β(t ))2 −! − t 2 k+1 k 2 k and X 1 1 lim (β2(t ) − β2(t )) −! β2(t): 2 k+1 k 2 k Thus we get the (slightly) unexpected result Z t 1 1 β(t) dβ(t) = − t + β2(t): 0 2 2 This is unexpected only if we believe in the chain rule: d 1 dx x2(t) = x: dt 2 dt But it is not true for a (Itô) stochastic process x(t)! Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 17 / 34 Itô formula Itô formula Assume that x(t) is an Itô process, and consider arbitrary (scalar) function φ(x(t); t) of the process. Then the Itô differential of φ, that is, the Itô SDE for φ is given as @φ X @φ 1 X @2φ dφ = dt + dxi + dxi dxj @t @xi 2 @xi @xj i ij @φ 1 n o = dt + (rφ)T dx + tr rrTφ dx dxT ; @t 2 provided that the required partial derivatives exists, where the mixed differentials are combined according to the rules dx dt = 0 dt dβ = 0 dβ dβT = Q dt: Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 18 / 34 Itô formula: derivation Consider the Taylor series expansion: @φ(x; t) X @φ(x; t) φ(x + dx; t + dt) = φ(x; t) + dt + dxi @t @xi i 1 X @2φ + dxj dxj + ::: 2 @xi @xj ij To the first order in dt and second order in dx we have dφ = φ(x + dx; t + dt) − φ(x; t) @φ(x; t) X @φ(x; t) 1 X @2φ ≈ dt + dxi + dxi dxj : @t @xi 2 @xi @xj i ij In deterministic case we could ignore the second order and higher order terms, because dx dxT would already be of the order dt2. In the stochastic case we know that dx dxT is potentially of the order dt, because dβ dβT is of the same order. Simo Särkkä (Aalto) Lecture 2: Itô Calculus and SDEs November 14, 2013 19 / 34 Itô formula: example 1 Itô differential of β2(t)=2 1 2 If we apply the Itô formula to φ(x) = 2 x (t), with x(t) = β(t), where β(t) is a standard Brownian motion, we get 1 dφ = β dβ + dβ2 2 1 = β dβ + dt; 2 as expected.