Ordinary Differential Equation

Ordinary Differential Equations ISC-5315 – 1 Ordinary differential equation

1 References

• Sachin Shanbag, lecture notes (see class website)

• Wikipedia (see class website)

2 Basics

Explicit differential equation (1) dy y0 = = F [t] (Quadrature) (1) dt Explicit differential equation (2)

y0 = F (y, t) (2)

Implicit differential equation

F [y, y0, t) = 0 (We will not look these) (3)

3 Higher order ODEs can be reduced to a system of first order ODEs, for example

y00(t) + y(t) = 0 (4) with initial conditions y(0) = 0, y0(0) set z = y0, then we can express as a system of two ODEs

z0(t) + y(t) = 0 (5) y0(t) − z(t) = 0 with initial conditions z(0) = 0, y(0). In general an n-order ODE can be reduced to a system of n 1-order ODEs.

4 Systems of equations

 0      y1 a11 a12 ... a1m y1 0 y  a21 a12 ... a1m y2  2 =     (6) ...  ......  ... 0 yn an1 an2 ... anm yn y0 = Ay (7)

y(0) = y0 (8)

1 Peter Beerli; March 31, 2011 Ordinary Differential Equations ISC-5315 – 2

5 Boundary value problems

(We will not look at boundary value problems, only at initial value problems in ACS 1) Boundary value problems (BVP)

y00(t) + y(t) = 0 with y(0) = 0 y(π/2) = 2 (9)

Initial value problem (IVP)

y00(t) + y(t) = 0 with y(0) = 0 y0(0) = 0 (10)

Typically only scalar equations are used

6 Uniqueness

Multiple solutions to some ODEs exist, for example

y0(t) = py(t) with y(0) = 0

Solution 1:

y(t) = 0 (11)

Solution 2:

t2 y(t) = (12) 4 7 Existence

Solutions may not exist for all values of t

y0(t) = [y(t)]2 with y(0) = 

Solution:

1 1 y(t) = exists only for t 6= (13) −1 − t 

8 Lipschitz continuity

In mathematical analysis, Lipschitz continuity, named after Rudolf Lipschitz, is a strong form of uniform continuity for functions. Intuitively, a Lipschitz continuous function is limited in how fast it can change: For every pair of points on the graph of this function, their secant line-segment’s slope has absolute-value no greater than a definite real number; this constant, which is the same for all the secants, is called the function’s “Lipschitz constant” (or “modulus of uniform continuity”).

2 Peter Beerli; March 31, 2011 Ordinary Differential Equations ISC-5315 – 3

A function

n+1 n f(x, t): R → R (14) is Lipschitz continuous if, for all (y1, t) and (y2, t), there exists an L, such that |f(y , t) − f(y , t)| 2 1 ≤ L. (15) |y2 − y1| Figure 1: For a Lips- If f(y, t) can be differentiated with respect to y, then chitz continuous function, there is a double cone

∂f(y, t) L = max (16) (shown in white) whose D ∂y vertex can be translated along the graph, so that Examples the graph always remains entirely outside the cone. f(y, t) = y2, y ∈ [−3, 7] ⇒ L = 14 (17) p f(y, t) = y2 + 5, y ∈ R ⇒ L = 1 (18) Small L implies a slowly varying function of y.

8.1 Indicator of uniqueness and existence It informs us about uniqueness and existence. A theorem without proof: If f(y, t) is Lipschitz continuous, then the initial value

0 y (t) = f(y, t) y(0) = y0 (19) has a unique differentiable solution on [a, b] for each y0, which depends continuously on y0 |y(t) − yˆ(t)| ≤ eLt|y(0) − yˆ(0)|. (20)

Bounded perturbation of the ODE, such as

0 y (t) = f(t, yˆ + r(t, yˆ)y ˆ(0) =y ˆ0 (21) with

r(t, yˆ) ≤ kMk (22) then M |y(t) − yˆ(t)| ≤ eLt|y(0) − yˆ(0)| + (eLt − 1) (23) L Example: Recall that we had multiple solutions for the ODE

y0(t) = py(t) y(0) = 0. (24) the function is discontinuous near zero ∂f(t, y) 1 = (25) ∂y 2py(t) Therefore the function is not Lipschitz continuous.

3 Peter Beerli; March 31, 2011 Ordinary Differential Equations ISC-5315 – 4

8.2 Lipschitz continuity examples 8.2.1 Continuous functions that are Lipschitz continuous: √ The function f(x) = x2 +5 defined for all real numbers is Lipschitz continuous with the Lipschitz constant K = 1, because it is everywhere differentiable and the absolute value of the derivative is bounded above by 1. Likewise, the sine function is Lipschitz continuous because its derivative, the cosine function, is bounded above by 1 in absolute value. The function f(x) = |x| defined on the reals is Lipschitz continuous with the Lipschitz constant equal to 1, by the reverse triangle inequality. This is an example of a Lipschitz continuous function that is not differentiable. More generally, a norm on a vector space is Lipschitz continuous with respect to the associated metric, with the Lipschitz constant equal to 1.

8.2.2 Continuous functions that are not (globally) Lipschitz continuous: The function f(x) = x2 with domain all real numbers is not Lipschitz continuous. This function becomes arbitrarily steep as x approaches ∞. It is however locally Lipschitz continuous. √ The function f(x) = x defined on [0, 1] is not Lipschitz continuous. This function becomes infinitely steep as x approaches 0 since its derivative becomes infinite.

8.2.3 Differentiable functions that are not (globally) Lipschitz continuous. The function f(x) = x3/2 sin (1/x) with x 6= 0 and f(0) = 0, restricted on [0, 1], gives an example of a function that is differentiable on a compact set while not locally Lipschitz because its derivative function is not bounded. See also the first property below.

9 Numerical solutions of ODEs

Consider an explicit scalar ODE

0 y (t) = f(y, t) y(0) = y0. (26)

We can discretize the t domain {t0, t1, ..., tk, ..., tn}. We compute approximate numerical solutions at the gridpoints tk

yk = y(tk) ∼ Yk = Y (tk). (27) where Yk are the exact solutions evaluated at grid points. We explore simple methods based on Taylor series and Euler methods.

9.1 Taylor series 0 Given y (t) = f(y, t) with initial values y(t0) = y0 we can consider a Taylor series expansion of y around t0 (t − t )2 y(t) = y + (t − t )y0(t ) + 0 y00(t ) + ... (28) 0 0 0 2! 0

4 Peter Beerli; March 31, 2011 Ordinary Differential Equations ISC-5315 – 5 recall that

df ∂f ∂f ∂y y00(t) = = + (chain rule) (29) dt ∂t ∂y ∂t ∂f ∂f y00(t) = + f (30) ∂t ∂y

A general expression for order-n method

h2 hn y(t ) = y(t ) + hy0(t ) + y00(t ) + ... + y(n)(t ) (31) i i−1 i−1 2! i−1 n! i−1 the error is O(hn+1)

hn+1  = |Y − y | = y(n+1)(ξ) withξ ∈ [t , t ] (32) G i i (n + 1)! i−1 i

Example:

y(t) = t − y y(0) = 1, (33) differentiate y0 repeatedly

y0(t) = t − y y0(0) = −1 y00(t) = 1 − y0 y00(0) = 2 y(3)(t) = −y00 y(3)(0) = −2 y(4)(t) = −y(3) y(4)(0) = 2

4th order Taylor series solution

t2 t4 y(t) = 1 − t + (2) + fract33!(−2) + (2) (34) 2! 4! t3 t4 = 1 − t + t2 − + (35) 3 12

Exact solution

Y (t) = t + 2e−t − 1 (36)

A comparison of the exact solution with the approximate Taylor series solution is shown in figure 2. In summary we can attest that the Taylor approximation delivers great results for small h values. The differentiation of f(t, y) can be laborious; it is problem specific and need to be repeated for dif- ferent f(t, y), can be automated. The error analysis of the Taylor approximation is straightforward and one can get an upper bound of the error. In practice only low order Taylor series formulae are used (Runge-Kutta formulae take their inspiration from higher order Taylor series).

5 Peter Beerli; March 31, 2011 Ordinary Differential Equations ISC-5315 – 6

1.00

0.95

0.90

0.85

0.80

0.75

0.70

0.0 0.2 0.4 0.6 0.8 1.0

Figure 2: The blue curve represent the exact formula (36) and the red dashed curve reflects the Taylor approximation (35).

9.2 Euler method 9.2.1 Forward Euler method

0 Given y (t) = f(y, t) with initial values y(t0) = y0 we can discretize the first derivative with time step

h = tn − tn−1 (37) as y − y y0 = n n−1 = f(t , y ) = f (38) h n−1 n−1 n−1 and then use a time marching algorithm

yn = yn−1 + fn−1 (39) (40)

Example: Say, we have

f(t, y) = −2t3 + 12t2 − 20t + 8.5; y(0) = 1 (41)

Using Mathematica we can construct the following function for the Euler method: euler[y0_,h_]:=Module[ {yy={},ii={},i=0}, AppendTo[yy,y0]; While[i<4, AppendTo[yy,yy[[-1]]+h (8.5-20 i+12 i^2-2 i^3)]; AppendTo[ii,i]; i+= h

6 Peter Beerli; March 31, 2011 Ordinary Differential Equations ISC-5315 – 7

]; AppendTo[ii,i]; Transpose[{ii,yy}] ] and with different intervals sizes we get different accuracies. Figure 3 shows results for an h = 0.01 and h = 0.1 in comparison with the exact solution.

5

4

3

2

1 2 3 4

Figure 3: The blue curve represent the exact formula and the blue dashed curve reflects the Euler approxi- mation with h = 0.01, and green with h = 0.1

We see that the accuracy of the Euler approximation is quite good with small intervals but stinks with large intervals, looking at the algorithm, we that it is equivalent to Riemann’s sum (quadra- ture). The rectangular rule certainly is not the most sophisticated approach.

10 Evaluation of solutions

We saw with the Euler method that we need to check our results. Without analytical results this is rather difficult and we need to develop a language to discuss such issues. We need to be able to explore the local truncation or discretization error (e.g. h = 0.01 and h = 0.1 in the Euler method). We need to worry about the global error because we sum over larger numbers of small steps that propagate the local error.

• Consistency: Does the discrete approximation approach the exact solution as h goes to zero? (local discretization error)

• Convergence: Given consistency, how quickly does the discrete approximation approach the exact solution.

• Stability: How sensitive is the solution to perturbations in the initial conditions or the function?

• Stiffness

7 Peter Beerli; March 31, 2011 Ordinary Differential Equations ISC-5315 – 8

10.1 Consistency

Let y(ti) represent the exact solution to the discrete equations. Let Y (ti) represent the exact solution to the IVP. Now ask whether y(ti) approach Y (ti) as h is reduced. Define a difference operator for a numerical method (for example for the Euler method we have y(t ) − y(t ) N (y ) = n n−1 − f(t , y ) (42) h n h n−1 n−1

Let y = {y0, y1, ..., yN } and Y = {Y0,Y1, ..., YN } then

Nh(y) = 0 (43)

Nh(Y) = d 6= 0 (44) For the forward Euler method we then can rephrase y − y N (y) = n n−1 − f (45) h h n−1 the local truncation error d is the residual, when the difference operator is applied to the exact solution Y − Y N (Y ) = n n−1 − f = d (46) h h n−1

5

0

-5

1 2 3 4

Figure 4: Left: Local truncation error is the difference between the red and the green slope. Right: Differences in the approximated derivatives used in Euler using h = 0.1 (red) versus the analytical derivatives. where d is the local truncation error. for a consistent method d goes to zero as h goes to zero. and the discrete solution approaches the continuous solution.

10.2 Convergence Convergence describes rate at which the discrete solutions approaches continuous (true) solution. The global error is

n = |yn − Y (tn)| withe0 = 0 (47) a method Nh converges with order p, if for all n

p n = O(h ) (48)

8 Peter Beerli; March 31, 2011 Ordinary Differential Equations ISC-5315 – 9

10.3 Stability 10.4 Stiffness

9 Peter Beerli; March 31, 2011