Math2221 Higher Theory and Applications of Differential Equations

Math2221 Higher Theory and Applications of Diﬀerential Equations

School of Maths and Stats, UNSW Session 2, 2016

Version: January 20, 2017 Part I

Linear ODEs Introduction

In first year, you studied second-order linear ODEs with constant coefficients. We will see that the techniques you learned can be extended to handle higher-order linear ODEs with variable coefficients. A key idea, used repeatedly throughout the remainder of the course, is linear superposition. Outline

Linear diﬀerential operators

Diﬀerential operators with constant coeﬃcients

Wronskians and linear independence

Methods for inhomogeneous equations

Solution via power series

Singular ODEs

Bessel and Legendre equations Linear diﬀerential operators

In linear algebra, you have seen the advantages arising from the compact notation Ax = b for a system of linear equations. A similarly compact notation is of great value when dealing with a linear diﬀerential equation

Lu = f.

Here, L is an operator (or transformation) that acts on a function u to create a new function Lu. Notation

Given coefficients a0(x), a1(x),..., am(x) we define the linear differential operator L of order m,

m j Lu(x) = aj(x)D u(x) j=0 (1) X m m−1 = amD u + am−1D u + ··· + a0u,

where Dju = dju/dxj (with D0u = u).

We refer to am as the leading coeﬃcient of L. For simplicity, we assume that each aj(x) is a smooth function of x.

The ODE Lu = f is said to be singular with respect to an interval [a, b] if the leading coeﬃcient am(x) vanishes for any x ∈ [a, b]. (Also say L is singular on [a, b].) Linearity

An operator of the form (1) is indeed linear: for any constants c1 and c2 and any (m-times diﬀerentiable) functions u1 and u2,

L(c1u1 + c2u2) = c1Lu1 + c2Lu2.

Hence, the set of solutions to the homogeneous equation Lu = 0 forms a vector space. Example Lu = (x − 3)u000 − (1 + cos x)u0 + 6u is a linear differential operator of order 3, with leading coefficient x − 3. Thus, L is singular on [1, 4], but not singular on [0, 2]. Example N(u) = u00 + u2u0 − u is a nonlinear differential operator of order 2. Linear initial-value problem Consider a general mth-order linear differential operator

m j Lu = aj(x)D u. j=0 X

Given f(x) and m initial values ν0, ν1,..., νm−1 we seek u = u(x) satisfying Lu = f on [a, b], (2) with

0 (m−1) u(a) = ν0, u (a) = ν1, . . . , u (a) = νm−1. (3)

Theorem Assume that the ODE Lu = f is not singular with respect to [a, b], and that f is continuous on [a, b]. Then the IVP (2) and (3) has a unique solution. Homogeneous problem

Theorem Assume that the linear, mth-order diﬀerential operator L is not singular on [a, b]. Then the set of all solutions to the homogeneous equation Lu = 0 on [a, b] is a vector space of dimension m. Proof. Let V = { u : Lu = 0 on [a, b] } and deﬁne the linear m transformation Θ : V → R by

Θu = [u(a), u0(a), . . . , u(m−1)(a)]T .

Uniqueness of solutions means that Θ is one-one, and existence means that Θ is onto. Hence, Θ is an isomorphism, and therefore the vector space V has dimension m. General solution

If {u1, u2, . . . , um} is any basis for the solution space of Lu = 0, then every solution can be written in a unique way as

u(x) = c1u1(x) + c2u2(x) + ··· + cmum(x) for a 6 x 6 b. (4) We refer to (4) as the general solution of the homogeneous equation Lu = 0 on [a, b].

Linear superposition refers to this technique of constructing a new solution out of a linear combination of old ones. Of course, this trick works only because L is linear.

Example The general solution to u00 − u0 − 2u = 0 is

−x 2x u(x) = c1e + c2e . Inhomogeneous problem

Consider the inhomogeneous equation Lu = f on [a, b], and ﬁx a particular solution uP.

For any solution u, the diﬀerence u − uP is a solution of the homogeneous equation because

L(u − uP) = Lu − LuP = f − f = 0 on [a, b].

Hence, u(x) − uP(x) = c1u1(x) + ··· + cmum(x) for some constants c1,..., cm, and so

u(x) = uP(x) + c1u1(x) + ··· + cmum(x), a 6 x 6 b,

uH(x)

is the general solution| of the inhomogeneous{z equation} Lu = f. Reduction of order

If we know (somehow) one solution u = u1(x) to a second-order, linear, homogeneous ODE

u00 + p(x)u0 + q(x)u = 0,

then, to ﬁnd a second (linearly independent) solution, substitute u = v(x)u1(x) into the ODE and rearrange to obtain

00 0 00 0 0 (u1 + pu1 + qu1)v + u1v + (2u1 + pu1)v = 0. =0 This is just| a ﬁrst{z-order,} linear ODE for the derivative of the unknown factor v(x): put w = v0 then

0 0 u1w + (2u1 + pu1)w = 0. Reduction of order (continued)

Writing the ODE for w in the standard form

0 0 −1 w + (2u1u1 + p)w = 0, we seek an integrating factor 0 −1 2 I(x) = exp (2u1u1 + p) dx = u1 exp( p dx), so that d R R (Iw) = Iw0 + I0w = Iw0 + (2u0 u−1 + p)w = 0. dx 1 1 Then Iw = C for some constant C, and so C v = dx. I(x) Z Example 00 0 3x For the ODE u − 6u + 9u = 0, take u1 = e and find v. Differential operators with constant coefficients

If L has constant coeﬃcients, then the problem of solving Lu = 0 reduces to that of factorizing the polynomial having the same coeﬃcients. Some complications occur if the polynomial has any repeated roots. Characteristic polynomial

Suppose that aj is constant for 0 6 j 6 m, with am 6= 0. We deﬁne the associated polynomial of degree m,

m j m m−1 p(z) = ajz = amz + am−1z + ··· + a1z + a0, j=0 X so that, formally, L = p(D).

Since Djeλx = λjeλx we have

λx m m−1 λx λx p(D)e = amλ +am−1λ +···+a1λ+a0 e = p(λ)e ,

and so p(D)eλx = 0 ⇐⇒ p(λ) = 0. Factorization

By the fundamental theorem of algebra,

k1 k2 kr p(z) = am(z − λ1) (z − λ2) ··· (z − λr)

where λ1, λ2,..., λr are the distinct roots of p, with corresponding multiplicities k1, k2,..., kr satisfying

k1 + k2 + ··· + kr = m.

Lemma j λx j−1 λx (D − λ)x e = jx e for j > 0. Lemma (D − λ)kxjeλx = 0 for j = 0, 1,..., k − 1. Proof

An elementary calculation gives

(D − λ)xjeλx = jxj−1eλx + xjλeλx − λxjeλx = jxj−1eλx,

as claimed, and then

(D − λ)2xjeλx = (D − λ)jxj−1eλx = j(j − 1)xj−2eλx, . . (D − λ)jxjeλx = j!eλx, (D − λ)j+1xjeλx = j!(D − λ)eλx = 0,

k j λx so (D − λ) x e = 0 for all k > j + 1, that is, for j 6 k − 1. Basic solutions

Lemma If (z − λ)k is a factor of p(z) then the function u(x) = xjeλx is a solution of Lu = 0 for 0 6 j 6 k − 1. Proof. Write p(z) = (z − λ)kq(z), so that q(z) is a polynomial of degree m − k. It follows that

p(D) = (D − λ)kq(D) = q(D)(D − λ)k

and so for 0 6 j 6 k − 1, p(D)xjeλx = q(D)(D − λ)kxjeλx = q(D)0 = 0. General solution

Theorem For the constant-coeﬃcient case, the general solution of the homogeneous equation Lu = 0 is

r kq−1 l λqx u(x) = cqlx e , q=1 l=0 X X

where the cql are arbitrary constants.

k Since (z − λq) q is a factor of p(z),

r kq−1 r kl−1 l λqx Lu = cqlLx e = cql × 0 = 0. q=1 l=0 q=1 l=0 X X X X Linear independence is shown in the Technical Proofs. Distinct real roots

Example From the factorization

D4 − 2D3 − 11D2 + 12D =( D + 3)D(D − 1)(D − 4)

we see that the general solution of

u0000 − 2u000 − 11u00 + 12u0 = 0

is −3x x 4x u = c1e + c2 + c3e + c4e . Repeated real root

Example From the factorization

D4 + 6D3 + 9D2 − 4D − 12 =( D − 1)(D + 2)2(D + 3)

we see that the general solution of

u0000 + 6u000 + 9u00 − 4u0 − 12u = 0

is x −2x −2x −3x u = c1e + c2e + c3xe + c4e . Complex root

Example From the factorization

D3 − 7D2 + 17D − 15 = (D2 − 4D + 5)(D − 3) =( D − 2 − i)(D − 2 + i)(D − 3)

we see that the general solution of

u000 − 7u00 + 17u0 − 15u = 0

(2+i)x (2−i)x 3x u(x) = c1e + c2e + c3e 2x 2x 3x = c4e cos x + c5e sin x + c3e . Simple oscillator

Second-order ODEs arise naturally in classical mechanics. Consider a particle of mass m that moves along the x-axis with velocity v = x˙ = dx/dt under the inﬂuence of

an external applied force = f(t), a frictional resistance force = −r(v)v, a restoring force = −k(x)x.

Newton’s second law,

d2x m x¨ = m = f(t) − r(v)v − k(x)x, dt2 leads to a second-order diﬀerential equation

m x¨ + r(x˙) x˙ + k(x)x = f(t). Simplest case is when r(v) = r0 > 0 and k(x) = k0 > 0 are constant, giving a linear ODE with constant coeﬃcients:

m x¨ + r0 x˙ + k0x = f(t).

Typically interested in the case when the applied force is periodic with frequency ω; for example,

f(t) = F sin ωt.

The general solution x(t) = xH(t) + xP(t).

We now show that xH(t) → 0 as t → and that xP(t) exists. Since xP(t + T) = xP(t) for T = 2π/ω, it follows that the solution x(t) always tends to a periodic∞ function with frequency ω, regardless of the initial values x(0) and x˙(0). The characteristic equation is

2 mλ + r0λ + x0 = m(λ − λ+)(λ − λ−), and the roots are √ −r ± ∆ λ = 0 where ∆ = r2 − 4mk . ± 2m 0 0

If ∆ > 0, then λ− < λ+ < 0 so, for any constants A and B,

λ+t λ−t xH(t) = Ae + Be → 0 as t → .

∞ If ∆ = 0 then λ− = λ+ < 0 so again xH(t) → 0.

√ p If ∆ < 0 then ∆ = i |∆| so Re λ+ = Re λ− < 0, and again xH(t) → 0. Since Re λ± 6= 0 the particular solution has the form

xP(t) = C cos ωt + E sin ωt, and we ﬁnd that

2 mx¨P + r0x˙P + k0xP = −mω C + r0ωE + k0C cos ωt 2 + −mω E − r0ωC + k0E sin ωt, which equals F sin ωt iﬀ

2 (k0 − mω )C + r0ωE = 0, 2 −r0ωC + (k0 − mω )E = F.

This 2 × 2 system has a unique solution since its determinant is

2 2 2 (k0 − mω ) + (r0ω) > 0. Wronskians and linear independence

We introduce a function, called the Wronskian, that provides us with a way of testing whether a family of solutions to Lu = 0 is linearly independent. The Wronskian also turns out to have several other uses. Wronskian

The Wronskian of the functions u1, u2,..., um is the m × m determinant

i−1 W(x) = W(x; u1, u2, . . . , um) = det[D uj].

For instance, if m = 2 then

u1 u2 0 0 W(x) = 0 0 = u1u2 − u2u1, u1 u2 and when m = 3,

u1 u2 u3 0 0 0 W(x) = u1 u2 u3 . 00 00 00 u1 u2 u3 Of course, W(x) is deﬁned only when the functions are diﬀerentiable m − 1 times. Wronskian

Example 2x 2x The Wronskian of the functions u1 = e , u2 = xe and −x u3 = e is

2x 2x −x e xe e 3x W = 2e2x e2x + 2xe2x −e−x = 9e .

4e2x 4e2x + 4xe2x e−x

Example x x The Wronskian of the functions u1 = e cos 3x and u2 = e sin 3x is

x x e cos 3x e sin 3x 2x W = = 3e . ex cos 3x − 3ex sin 3x ex sin 3x + 3ex cos 3x Linearly dependent functions

Lemma If u1,..., um are linearly dependent over an interval [a, b] then W(x; u1, . . . , um) = 0 for a 6 x 6 b.

Example x The functions u1 = cosh x, u2 = sinh x and u3 = e are linearly dependent because

ex + e−x ex − e−x cosh x + sinh x = + = ex. 2 2 Their Wronskian is

x cosh x sinh xe

W = sinh x cosh x ex = 0.

cosh x sinh xe x Proof (for m = 3)

Assume that u1, u2, u3 are linearly dependent on the interval [a, b], that is, there exist constants c1, c2, c3, not all zero, such that

c1u1(x) + c2u2(x) + c3u3(x) = 0 for a 6 x 6 b. Diﬀerentiating, it follows that

0 0 0 c1u1(x) + c2u2(x) + c3u3(x) = 0, 00 00 00 c1u1 (x) + c2u2 (x) + c3u3 (x) = 0,

so       u1(x) u2(x) u3(x) c1 0 0 0 0 u1(x) u2(x) u3(x) c2 = 0 for a 6 x 6 b. 00 00 00 u1 (x) u2 (x) u3 (x) c3 0 This 3 × 3 matrix must be singular and thus W(x) = 0. Wronskian satisﬁes a ﬁrst-order ODE

Lemma If u1, u2,..., um are solutions of Lu = 0 on the interval [a, b] then their Wronskian satisﬁes

0 am(x)W (x) + am−1(x)W(x) = 0, a 6 x 6 b.

Example The second-order ODE

u00 + 3u0 − 4u = 0

x −4x has solutions u1 = e and u2 = e . Their Wronskian is

x −4x e e −3x = −5e , ex −4e−4x

and satisﬁes W 0 + 3W = 0. Proof (for m = 2)

0 0 Diﬀerentiating W = u1u2 − u1u2 we have

0 0 0 00 00 0 0 00 00 W = (u1u2 + u1u2 ) − (u1 u2 + u1u2) = u1u2 − u1 u2,

0 00 00 0 0 a2W + a1W = a2(u1u2 − u1 u2) + a1(u1u2 − u1u2) + a0(u1u2 − u1u2) =0 00 0 = u1(a2u2| + a1{zu2 + a}0u2) 00 0 − (a2u1 + a1u1 + a0u1)u2 = u1(Lu2) − (Lu1)u2 = 0. Linear independence of solutions

Theorem Let u1, u2,..., um be solutions of a non-singular, linear, homogeneous, mth-order ODE Lu = 0 on the interval [a, b]. Either W(x) = 0 for a 6 x 6 b and the m solutions are linearly dependent, or else W(x) 6= 0 for a 6 x 6 b and the m solutions are linearly independent. Proof

The Wronskian satisﬁes

0 am−1 W + pW = 0 for a 6 x 6 b, where p = . am Deﬁne an integrating factor I(x) = exp p(x) dx 6= 0, Z so that I0 = Ip and hence

(IW)0 = IW 0 + IpW = I(W 0 + pW) = 0.

Thus, I(x)W(x) = C for some constant C.

Either C = 0 in which case W(x) = 0 for all x ∈ [a, b], or else C 6= 0 in which case W(x) is never zero for x ∈ [a, b]. (Assume now that m = 3.) We already know that

u1, u2, u3 linearly dependent =⇒ W ≡ 0.

Hence, to complete the proof it suﬃces to show

W(a) = 0 =⇒ u1, u2, u3 linearly dependent.

If W(a) = 0, then there exist c1, c2, c3, not all zero, such that       u1(x) u2(x) u3(x) c1 0 0 0 0 u1(x) u2(x) u3(x) c2 = 0 at x = a, 00 00 00 u1 (x) u2 (x) u3 (x) c3 0 so the function u(x) = c1u1(x) + c2u2(x) + c3u3(x) satisﬁes

0 00 Lu = 0 for a 6 x 6 b, with u(a) = u (a) = u (a) = 0. The solution of this initial-value problem is unique, so u(x) ≡ 0 and thus u1, u2, u3 are linearly dependent. Interlacing of zeros

Theorem (Sturm separation theorem) Assume a2(x) 6= 0 for a 6 x 6 b. If u1 and u2 are linearly independent solutions of

00 0 a2(x)u + a1(x)u + a0(x)u = 0

then u1 has exactly one zero between any two successive zeros of u2 in the interval [a, b].

Example

The functions u1(x) = cos x and u2(x) = sin x are linearly independent solutions of u00 + u = 0 on any interval [a, b]. Example What about u00 − u = 0? Proof

Suppose a 6 α < β 6 b with

u2(α) = 0 = u2(β) and u2(x) 6= 0 for α < x < β.

Case 1: W(x) > 0 for a 6 x 6 b and u2(x) > 0 for α < x < β. Thus,

0 u2(α + h) u2(α) = lim > 0, h→0+ h 0 u2(β + h) u2(β) = lim 6 0. h→0− h

0 0 Since W = u1u2 − u1u2, we see

0 0 0 < W(α) = u1(α)u2(α) and 0 < W(β) = u1(β)u2(β),

so u1(α) > 0 and u1(β) < 0. Proof (continued)

Case 2: W(x) > 0 for a 6 x 6 b and u2(x) < 0 for α < x < β. Similar argument shows u1(α) < 0 and u1(β) > 0.

Case 3: W(x) < 0 for a 6 x 6 b and u2(x) > 0 for α < x < β. Similar argument shows u1(α) < 0 and u1(β) > 0.

Case 4: W(x) < 0 for a 6 x 6 b and u2(x) < 0 for α < x < β. Similar argument shows u1(α) > 0 and u1(β) < 0.

In all cases, u1(α) and u1(β) have opposite signs, and so by the Intermediate Value Theorem u1(ξ) = 0 for at least one ξ ∈ (α, β).

Finally, suppose for a contradiction that u1 has more than one zero in (α, β), say u1(ξ) = 0 = u1(η) with α < ξ < η < β. Then, by interchanging the roles of u1 and u2, it follows that u2 must have a zero in (ξ, η), contradicting the assumption that u2(x) > 0 for α < x < β. Methods for inhomogeneous equations

In first year, you learned the method of undetermined coefficients for constructing a particular solution uP to an inhomogeneous second-order linear ODE Lu = f in some simple cases. We will study this method systematically for higher-order linear ODEs with constant coefficients.

We also discuss variation of parameters, a technique that applies for general L and f, but which requires the evaluation of possibly very diﬃcult integrals. Superposition of solutions

We now consider methods for ﬁnding a particular solution uP satisfying LuP = f. First note that if

f(x) = c1f1(x) + c2f2(x)

and if we know uP1 and uP2 satisfying

LuP1 = f1 and LuP2 = f2,

then we can put

uP(x) = c1uP1(x) + c2uP2(x),

because by the linearity of L,

LuP = c1LuP1 + c2LuP2 = c1f1 + c2f2 = f. Polynomial solutions Let L = p(D) be a linear diﬀerential operator of order m with constant coeﬃcients. Theorem Assume that L1 6= 0, or equivalently a0 = p(0) 6= 0. For any integer r > 0, there exists a unique polynomial uP of degree r such r that LuP = x . For simplicity, we prove the result only for the case m = 2. Thus,

00 0 Lu = a2u + a1u + a0u,

where a0, a1, a2 are constants with a2 6= 0 and a0 6= 0.

Look for uP in the form

r xj u (x) = c . P j j! j=0 X We ﬁnd that

r xj−2 r xj−1 r xj Lu = a c + a c + a c P 2 j (j − 2)! 1 j (j − 1)! 0 j j! j=2 j=1 j=0 X X X xr xr−1 = a c + a c + a c 0 r r! 0 r−1 1 r (r − 1)! r−2 xj + a c + a c + a c , 2 j+2 1 j+1 0 j j! j=0 X which equals xr if and only if

a0cj + a1cj+1 + a2cj+2 = 0, 0 6 j 6 r − 2, a0cr−1 + a1cr = 0,

a0cr = r!.

This upper triangular system is uniquely solvable because a0 6= 0. An example

Let Lu = 3u00 − u0 + 2u and suppose we want a particular solution to Lu = 8x3. The theorem ensures that

2 3 uP(x) = C + Ex + Fx + Gx

works for some C, E, F, G. In fact,

2 3 LuP = (2C − E + 6F) + (2E − 2F + 18G)x + (2F − 3G)x + 2Gx

2C − E + 6F = 0, 2E − 2F + 18G= 0, 2F − 3G= 0, 2G = 8

2 3 and back substitution gives uP = −33 − 30x + 6x + 4x . Exponential solutions Theorem Let L = p(D) and µ ∈ C. If p(µ) 6= 0, then the function eµx u (x) = P p(µ)

µx satisﬁes LuP = e . Proof. Follows at once because p(D)eµx = p(µ)eµx. Example A particular solution of

u00 + 4u0 − 3u = 3e2x

is e2x u = . P 3 Product of polynomial and exponential

Theorem Let L = p(D) and assume that p(µ) 6= 0. For any integer r > 0, there exists a unique polynomial v of degree r such that µx r µx uP = v(x)e satisﬁes LuP = x e . Proof. Again, for simplicity, we prove the result only for m = 2. Put v = e−µxu so that u = veµx, and observe that

00 0 2 0 µx Lu = a2(v + 2µv + µ v) + a1(v + µv) + a0v e 00 0 2 µx = a2v + (a1 + 2a2µ)v + (a2µ + a1µ + a0)v e .

µx 2 Thus, Lu = e q(D)v where q(z) = a2z + (a1 + 2a2µ)z + p(µ), and our earlier result yields the desired v satisfying q(D)v = xr because q(D)1 = q(0) = p(µ) 6= 0. An example

Consider 2u00 + u0 − 3u = 9xe−2x. Here, p(z) = 2z2 + z − 3 −2x so p(−2) = 3 6= 0 and a particular solution uP = (Cx + E)e exists. In fact, we ﬁnd that

−2x p(D)uP = (3Cx − 7C + 3E)e

so 3C = 9 and − 7C + 3E = 0. Thus, C = 3 and E = 7, giving

−2x uP = (3x + 7)e . Polynomial solutions: the remaining case Theorem Let L = p(D) and assume p(0) = p0(0) = ··· = p(k−1)(0) = 0 (k) but p (0) 6= 0 where 1 6 k 6 m − 1. For any integer r > 0, there exists a unique polynomial v of degree r such that k r up(x) = x v(x) satisﬁes LuP = x . For simplicity, we discuss only the case m = 2 and k = 1; thus,

2 0 p(z) = a2z + a1z with a1 = p (0) 6= 0.

Write r xj v(x) = c j (j + 1)! j=0 X so that

r xj+1 r+1 xj u (x) = xv(x) = c = c . P j (j + 1)! j−1 j! j=0 j=1 X X Since r+1 xj−1 r xj u0 (x) = c = c P j−1 (j − 1)! j j! j=1 j=0 X X and r xj−1 r−1 xj u00(x) = c = c , P j (j − 1)! j+1 j! j=1 j=0 X X we have

xr r−1 xj Lu = a c + a c + a c . P 1 r r! 2 j+1 1 j j! j=0 X

The assumption that a1 6= 0 ensures a unique solution to the lower triangular system

a1cj + a2cj+1 = 0, 0 6 j 6 r − 1, a1cr = r!. An example Let Lu = u000 + 2u00 and seek a particular solution to Lu = 12x2. The theorem ensures that

2 2 2 3 4 uP = x (C + Ex + Fx ) = Cx + Ex + Fx

works for some C, E, F. In fact,

2 LuP = (4C + 6E) + (12E + 24F)x + 24Fx

4C + 6 E = 0, 12E + 24F = 0, 24F = 12

and back substitution gives

x2 u = (3 − 2x + x2). P 2 Exponential times polynomial: remaining case Lemma If u(x) = w(x)eµx then

m zj p(D)u = eµxq(D)w where q(z) = p(j)(µ) . j! j=0 X

Theorem Let L = p(D) and assume p(µ) = p0(µ) = ··· = p(k−1)(µ) = 0 (k) but p (µ) 6= 0, where 1 6 k 6 m − 1. For any integer r > 0, there exists a unique polynomial v of degree r such that k µx r µx uP(x) = x v(x)e satisﬁes LuP = x e . Proof. Since q(j)(0) = p(j)(µ) for all j, there is a unique polynomial v of degree r such that w(x) = xkv(x) satisﬁes q(D)w = xr and hence µx µx r p(D)uP = e q(D)w = e x . Proof of Lemma

m m k k p(D)weµx = a Dkweµx = a Djw Dk−jeµx k k j k=0 k=0 j=0 X X X m k k! = a DjwD k−jeµx k j!(k − j)! k=0 j=0 X X m Djw m = a k(k − 1)(k − 2) ··· (k − j + 1)µk−jeµx j! k j=0 k=j X X m Djw m = a k(k − 1)(k − 2) ··· (k − j + 1)µk−jeµx j! k j=0 k=j X X m Djw m Djw = p(j)(µ)eµx = eµx p(j)(µ) j! j! j=0 j=0 X X = eµxq(D)w. An example Consider the ODE

Lu = 12e2x where Lu = u000 − 4u00 + 4u0.

Here, L = p(D) for p(z) = z3 − 4z2 + 4z = z(z − 2)2, so p(2) = p0(2) = 0 but p00(2) 6= 0. Thus, we try

2 2x uP = Cx e

and ﬁnd

0 2 2x 00 2 2x uP = C(2x + 2x )e , uP = C(2 + 8x + 4x )e , 000 2 2x uP = C(12 + 24x + 8x )e ,

2x so LuP = 4Ce and we require 4C = 12. Therefore, a particular solution is 2 2x uP = 3x e . Complex conjugate roots Consider

Lu ≡ p(D) ≡ u000 + u0 + 10u = 13ex sin 2x.

Here, p(z) = z3 + z + 10 = [(z − 1)2 + 4](z + 2) so p(1 ± 2i) = 0, and e2ix − e−2ix ex sin 2x = ex 2i is a linear combination of e(1+2i)x and e(1−2i)x. Therefore put

(1+2i)x (1−2i)x x uP(x) = Cxe + Exe = xe F cos 2x + G sin 2x .

We ﬁnd that if F = −3/4 and G = −1/2 then

x x x LuP = (−8F + 12G)e cos 2x + (−12F − 8G)e sin 2x = 13e sin 2x,

so xex u = − 3 cos 2x + 2 sin 2x. P 4 Variation of parameters What if f is not a polynomial times an exponential, or if L does not have constant coeﬃcients? Consider a linear, second-order, inhomogeneous ODE with leading coeﬃcient 1:

Lu = u00(x) + p(x)u0(x) + q(x)u(x) = f(x). (5)

Let u1(x) and u2(x) be linearly independent solutions to the homogeneous equation and let W(x) = W(x; u1, u2) denote their Wronskian. Thus,

Lu1 = 0, Lu2 = 0, W 6= 0.

We seek v1 and v2 such that

u(x) = v1(x)u1(x) + v2(x)u2(x)

is a (particular) solution to Lu = f. Variation of parameters (continued) To simplify the expression

0 0 0 0 0 u = v1u1 + v1u1 + v2u2 + v2u2

0 0 we impose the condition v1u1 + v2u2 = 0, then (as if v1 and v2 were constant parameters)

0 0 0 u = v1u1 + v2u2.

A short calculation now shows

0 0 0 0 0 0 0 0 Lu = v1Lu1 + v2Lu2 + v1u1 + v2u2 = v1u1 + v2u2,

since by assumption Lu1 = 0 = Lu2. Conclusion: u = v1u1 + v2u2 satisﬁes Lu = f if

0 0 v1u1 + v2u2 = 0, 0 0 0 0 v1u1 + v2u2 = f. 0 0 Thus, we have a pair of equations for the unknown v1 and v2. In matrix form

0 u1(x) u2(x) v1(x) 0 0 0 0 = , u1(x) u2(x) v2(x) f(x) so 0 0 v1(x) 1 u2(x) −u2(x) 0 0 = 0 , v2(x) W(x) −u1(x) u1(x) f(x) or in other words, −u (x)f(x) u (x)f(x) v0 (x) = 2 and v0 (x) = 1 . 1 W(x) 2 W(x)

Example Find the general solution to

3u00 − 6u0 + 30u = ex tan 3x. Solution via power series

If L has variable coeﬃcients, then we cannot expect in general that the solution of Lu = 0 is expressible in terms of elementary functions like polynomials, trigonometric functions, exponentials etc. Power series provide a ﬂexible way to represent u in this case. Constructing a series solution

Consider the initial-value problem

Lu = (1 − x2)u00 − 5xu0 − 4u = 0, u(0) = 1, u0(0) = 2.

Look for a solution in the form of a power series

k 2 u(x) = Akx = A0 + A1x + A2x + ··· . ∞ k=0 X Formal calculations show that

k Lu = (k + 2)[(k + 1)Ak+2 − (k + 2)Ak]x , ∞ k=0 X and the initial conditions imply A0 = 1 and A1 = 2. Convergence? Since Lu is identically zero iﬀ the coeﬃcient of xk vanishes for every k, we obtain the recurrence relation k + 2 A = A for k = 0, 1, 2, . . . . k+2 k + 1 k Thus,

A0 = 1, A1 = 2, A2 = 2, A3 = 3, . . . ,

giving u(x) = 1 + 2x + 2x2 + 3x3 + ··· . Since k+2 Ak+2x k + 2 2 2 lim k = lim x = x , k→ Akx k→ k + 1 2j 2j+1 the ratio test shows∞ that j=0 A2jx∞ and j=0 A2j+1x converge for x2 < 1 but diverge for x2 > 1. P∞ P∞ General case Consider a general second-order, linear, homogeneous ODE

00 0 Lu = a2(x)u + a1(x)u + a0(x)u = 0.

Equivalently, u00 + p(x)u0 + q(x)u = 0, where a (x) a (x) p(x) = 1 and q(x) = 0 . a2(x) a2(x)

Assume that aj is analytic at 0 for 0 6 j 6 2, and that a2(0) 6= 0. Then p and q are analytic at 0, that is, they admit power series expansions

k k p(z) = pkz and q(z) = qkz for |z| < ρ, ∞ ∞ k=0 k=0 X X for some ρ > 0. Formal expansions

If k u(z) = Akz ∞ k=0 X then we ﬁnd that

Lu(z) = (2A2 + p0A1 + q0A0)

+ (6A3 + 2p0A2 + p1A1 + q0A1 + q1A0)z + ··· ,

n−1 where, on the RHS, the coeﬃcient of z for a general n > 1 is

n−1 (n + 1)nAn+1 + (n − j)pjAn−j + qjAn−1−j . j=0 X Convergence theorem

Given u(0) and u0(0), we put

0 A0 = u(0) and A1 = u (0),

and compute recursively

−1 n−1 A = (n − j)p A + q A , n 1. n+1 n(n + 1) j n−j j n−1−j > j=0 X Theorem If the coeﬃcients p(z) and q(z) are analytic for |z| < ρ, then the formal power series for the solution u(z), constructed above, is also analytic for |z| < ρ. Previous example

Earlier we considered

Lu = (1 − x2)u00 − 5xu0 − 4u = 0, u(0) = 1, u0(0) = 2.

In this case, −5z 2k+1 p(z) = 2 = −5 z 1 − z ∞ k=0 X and −4 2k q(z) = 2 = −4 z 1 − z ∞ k=0 X are analytic for |z| < 1, so the theorem guarantees that u(z), given by the formal power series, is also analytic for |z| < 1. Expansion about a point other than 0

Suppose we want a power series expansion about a point c 6= 0, for instance because the initial conditions are given at x = c.

A simple change of the independent variable allows us to write

k k u = Ak(z − c) = AkZ where Z = z − c. ∞ ∞ k=0 k=0 X X Since du/dz = du/dZ and d2u/dz2 = d2u/dZ2, we obtain the translated equation

d2u du + p(Z + c) + q(Z + c)u = 0. dZ2 dZ

Now compute the Ak using the series expansions of p(Z + c) and q(Z + c) in powers of Z. Example

We construct a power series solution about z = 1 to Airy’s equation: u00 − zu = 0. Put Z = z − 1 and ﬁnd that u00 − zu = u00 − (Z + 1)u equals

k−2 k+1 k k(k − 1)AkZ − AkZ − AkZ ∞ ∞ ∞ k=0 k=0 k=0 X X X k k k = (k + 2)(k + 1)Ak+2Z − Ak−1Z − AkZ ∞ ∞ ∞ k=0 k=1 k=0 X X X k = (2A2 − A0) + (k + 2)(k + 1)Ak+2 − Ak−1 − Ak Z . ∞ k=1 X Thus, the coeﬃcients must satisfy 2A2 − A0 = 0 and

(k + 2)(k + 1)Ak+2 − Ak−1 − Ak = 0 for all k > 1, so A A + A A = 0 and A = k−1 k for k 1. 2 2 k+2 (k + 2)(k + 1) >

We ﬁnd that

(z − 1)2 (z − 1)3 (z − 1)4 u(z) = A 1 + + + + ··· 0 2 6 24 (z − 1)3 (z − 1)4 + A (z − 1) + + + ··· . 1 6 12 Singular ODEs

Recall that our basic existence and uniqueness theorem for Lu = f assumes that L is not singular, that is, the leading coeﬀecient of L does not vanish on the interval of interest. However, some important applications lead to singular ODEs so we must now address this case. Singular ODEs of second order

Consider

00 0 a2(x)u + a1(x)u + a0(x)u = 0 for a 6 x 6 b,

and suppose that a2(x0) = 0 for some x0 with a < x0 < b, but a2(x) 6= 0 if x 6= x0. Put

bj(y) = aj(x) and v(y) = u(x) where y = x − x0,

so that, with c = a − x0 < 0 < d = b − x0,

00 0 b2(y)v + b1(y)v + b0(y)v = 0 for c 6 y 6 d.

Since y = 0 when x = x0, we have b2(0) = 0.

In this way, it suﬃces to consider the case when the leading coeﬃcient vanishes at the origin. Cauchy–Euler ODE

A second-order Cauchy–Euler ODE has the form

Lu = ax2u00 + bxu0 + cu = f(x),

where a, b and c are constants, with a 6= 0. This ODE is singular at x = 0.

Noticing that Lxr = ar(r − 1) + br + cxr, we see that u = xr is a solution of the homogeneous equation (f = 0) iﬀ ar(r − 1) + br + c = 0. Factorization

Suppose ar(r − 1) + br + c = a(r − r1)(r − r2). If r1 6= r2 then the general solution of the homogeneous equation Lu = 0 is

r1 r2 u(x) = C1x + C2x , x > 0.

Lemma If r1 = r2 then the general solution of the homogeneous Cauchy–Euler equation Lu = 0 is

r1 r1 u(x) = C1x + C2x log x, x > 0.

Example Solve x2u00 − xu0 + u = 0. Example Solve 2(x − 2)2u00 − 3(x − 2)u0 − 3u = 0. Proof of the lemma r Since r1 = r2 the function F(x, r) = x satisﬁes

2 00 0 2 r ax F + bxF + cF = a(r − r1) x , where the dash means ∂/∂x. Put

∂F ∂ r log x r log x r v(x) = = e = e 1 log x = x 1 log x ∂r ∂r r=r1 r=r1 and observe that 2 2 00 0 2 ∂ ∂F ∂ ∂F ∂F ax v + bxv + cv = ax + bx + c ∂x2 ∂r ∂x ∂r ∂r r=r1 2 ∂ 2 ∂ F ∂F = ax + bx + cF ∂r ∂x2 ∂x r=r1 ∂ 2 r = a(r − r1) x ∂r r=r1 r 2 r = 2a(r − r1)x + (r − r1) x log x = 0. r=r1 More general singular ODEs

A number of important applications lead to ODEs that can be written in the Frobenious normal form

z2u00 + zP(z)u0 + Q(z)u = 0,

where P(z) and Q(z) are analytic at z = 0:

k k P(z) = Pkz and Q(z) = Qkz , |z| < ρ. (6) ∞ ∞ k=0 k=0 X X

Notice that u00 + p(z)u0 + q(z)u = 0 but p(z) = z−1P(z) and q(z) = z−2Q(z) are not analytic at z = 0 unless P(0) = 0 and Q(0) = Q0(0) = 0.

So in general we cannot expect a solution u(z) to be analytic at z = 0. A clue We can think of an ODE in Frobenius normal form as a Cauchy–Euler ODE with variable coeﬃcients.

For z near 0 we have P(z) ≈ P0 and Q(z) ≈ Q0 so we might expect u(z) to behave like a solution of

2 00 0 z u + P0zu + Q0u = 0.

We therefore consider the indicial polynomial

I(r) = r(r − 1) + P0r + Q0 = (r − r1)(r − r2).

If r1 6= r2 then the approximating Cauchy–Euler ODE has the r r general solution c1z 1 + c2z 2 , so it is natural to seek a solution in the form

r k k+r u(z) = z Akz = Akz , |z| < ρ, with A0 6= 0. ∞ ∞ k=0 k=0 X X An example

Consider Lu = 2z2u00 + 7zu0 − (z2 + 3)u = 0. Here, P(z) = 7/2 and Q(z) = −(z2 + 3)/2 are trivially analytic at z = 0 (since they are polynomials).

The approximations P(z) ≈ P0 = 7/2 and Q(z) ≈ Q0 = −3/2 lead to the Cauchy–Euler equation z2u00 + (7/2)zu0 − (3/2)u = 0 or

2z2u00 + 7zu0 − 3u = 0.

Thus, the indicial polynomial is

2r(r − 1) + 7r − 3 = 2r2 + 5r − 3 = (2r − 1)(r + 3)

so r1 = 1/2 and r2 = −3. Using

k+r 0 k+r−1 u = Akz , u = (k + r)Akz , ∞ ∞ k=0 k=0 X X 00 k+r−2 u = (k + r)(k + r − 1)Akz , ∞ k=0 X we ﬁnd that

Lu = (2z2u00 + 7zu0 − 3u) − z2u

k+r = 2(k + r)(k + r − 1) + 7(k + r) − 3 Akz ∞ k=0 X k+r+2 − Akz . ∞ k=0 X Since

2(k + r)(k + r − 1) + 7(k + r) − 3 = 2(k + r)2 + 5(k + r) − 3 = [2(k + r) − 1][(k + r) + 3] and k+r+2 k+r Akz = Ak−2z ∞ ∞ k=0 k=2 X X it follows that

r r+1 Lu = (2r − 1)(r + 3)A0z + (2r + 1)(r + 4)A1z

k+r + (2k + 2r − 1)(k + r + 3)Ak − Ak−2 z . ∞ k=2 X Conclusion: u is a solution if r ∈ {1/2, −3} with

A A = 0, A = k−2 for all k 2. 1 k (2k + 2r − 1)(k + r + 3) >

First solution: r = 1/2 with

A A = 0, A = k−2 for all k 2, 1 k k(2k + 7) > so A A A A A = 0 ,A = 1 = 0, A = 2 = 0 ,... 2 22 3 39 4 60 1320 and z2 z4 u(z) = A z1/2 1 + + + ··· . 0 22 1320 Second solution: r = −3 with A A = 0, A = k−2 for all k 2, 1 k k(2k − 7) > so A A A A A = − 0 ,A = − 1 = 0, A = − 2 = 0 ,... 2 6 3 3 4 4 24 and z2 z4 u(z) = A z−3 1 − + + ··· . 0 6 24

General solution of Lu = 0:

z2 z4 u(z) = Az1/2 1 + + + ··· 22 1320 z2 z4 + Bz−3 1 − + + ··· . 6 24 General case

Now consider z2u00 + zP(z)u0 + Q(z)u = 0 for P(z) and Q(z) satisfying (6). Formal manipulations show that Lu(z) equals

k−1 r k+r I(r)A0z + I(k + r)Ak + [(j + r)Pk−j + Qk−j]Aj z , ∞ k=1 j=0 X X

so we deﬁne A0(r) = 1 and

−1 k−1 A (r) = [(j + r)P + Q ]A (r), k 1, k I(k + r) k−j k−j j > j=0 X provided I(k + r) 6= 0 for all k > 1. Choice of exponent

The preceding calculations show that the series

k+r F(z; r) = Ak(r)z ∞ k=0 satisﬁes X z2F00 + zP(z)F0 + Q(z)F = I(r)zr, with I(r) = r(r − 1) + P0r + Q0 = (r − r1)(r − r2).

Assume, with no loss of generality, that Re r1 > Re r2. It follows that I(k + r1) 6= 0 for all integers k > 1, and therefore u1(z) = F(z; r1) is (formally) a solution.

If r1 − r2 is not a whole number, then a second, linearly independent solution is u2(z) = F(z; r2). Roots diﬀering by an integer

2 Suppose that r1 = r2. In this case, I(r) = (r − r1) and so

2 00 0 2 r z F + zP(z)F + Q(z)F = (r − r1) z .

The function v = ∂F/∂r satisﬁes

2 00 0 r 2 r z v + zP(z)v + Q(z)v = 2(r − r1)z + (r − r1) z log z,

and the RHS is zero if r = r1, so a second, linearly independent solution is

∂F 0 k+r1 k+r1 u2(z) = (z; r1) = Ak(r1)z + Ak(r1)z log z . ∂r ∞ ∞ k=0 k=0 X X u1(z) log z | {z } Even worse complications if r1 = r2 + n for an integer n > 1. Concluding remark about the indicial equation

We can ﬁnd r1 and r2 easily without any manipulation of series, to quickly determine the qualitative behaviour of solutions as z → 0. Example To see that all nontrivial solutions of the ODE

z2u00 + 3(e3z − 1)u0 + 15(cosh z)u = 0.

are unbounded as z → 0, use Taylor expansion to write

(3z)2 (3z)3 z2 z4 z2u00+3 3z+ + +··· u0+15 1+ + +··· u = 0. 2! 3! 2! 4!

The indicial polynomial is

r(r − 1) + 9r + 15 = r2 + 8r + 15 = (r + 3)(r + 5),

which has roots r1 = −3 < 0 and r2 = −5 < 0. Bessel and Legendre equations

We wrap up this part of the course with two particularly important examples of second-order, linear ODEs with variable coeﬃcients. Both occur several times later in the course. Friedrich Wilhelm Bessel 1784–1846 Bessel equation

The Bessel equation with parameter ν is

z2u00 + zu0 + (z2 − ν2)u = 0.

This ODE is in Frobenius normal form, with indicial polynomial

I(r) = (r + ν)(r − ν),

and we seek a series solution

k+r u(z) = Akz . ∞ k=0 X We assume Re ν > 0, so r1 = ν and r2 = −ν. Recurrence relation We ﬁnd that if

(r + 1 + ν)(r + 1 − ν)A1 = 0, (k + r + ν)(k + r − ν)Ak + Ak−2 = 0, k > 2. then

2 00 0 2 2 r z u + zu + (z − ν )u = (r + ν)(r − ν)A0z . Taking r = ν gives −A A = k−2 for k 2, k k(k + 2ν) >

so with A0 arbitrary and A1 = 0 we obtain

(z/2)2 (z/2)4 u(z) = A zν 1 − + 0 1 + ν 2(2 + ν)(1 + ν) (z/2)6 − + ··· . 3!(3 + ν)(2 + ν)(1 + ν) Bessel function With the normalisation 1 A = 0 2νΓ(1 + ν)

the series solution is called the Bessel function of order ν and is denoted (z/2)ν (z/2)2 (z/2)4 J (z) = 1 − + − ··· . ν Γ(1 + ν) 1 + ν 2!(1 + ν)(2 + ν)

From the functional equation Γ(1 + z) = zΓ(z) we see that

(z/2)ν (z/2)ν+2 (z/2)ν+4 (z/2)ν+6 J (z) = − + − + ··· ν Γ(1 + ν) Γ(2 + ν) 2!Γ(3 + ν) 3!Γ(4 + ν)

and so (−1)k(z/2)2k+ν Jν(z) = . ∞ k!Γ(k + 1 + ν) k=0 X Bessel function of negative order

If ν is not an integer, then a second, linearly independent, solution is (−1)k(z/2)2k−ν J−ν(z) = . ∞ k!Γ(k + 1 − ν) k=0 X For an integer ν = n ∈ Z, since Γ(n + 1) = n! we have

(−1)k(z/2)2k+n Jn(z) = . ∞ k!(k + n)! k=0 X Also, since 1/Γ(z) = 0 for z = 0, −1, −2, . . . , we ﬁnd that Jn and J−n are linearly dependent; in fact,

n J−n(z) = (−1) Jn(z). Bessel functions of integer order Neumann function The Neumann function (or Bessel function of the second kind) is

J (z) cos νπ − J (z) Y (z) = ν −ν , if ν 6∈ . ν sin νπ Z

For n ∈ Z, L’Hospital’s rule shows that if ν → n then Yν(z) tends to a ﬁnite limit Yn(z) = lim Yν(z). ν→n

The functions Jν and Yν are linearly independent solutions of Bessel’s equation for all complex ν. As z → 0 with ν ﬁxed, (z/2)ν J (z) ∼ , ν∈ / {−1, −2, −3, . . .}, ν Γ(ν + 1) 2 −Γ(ν) Y (z) ∼ log z, Y (z) ∼ , Re ν > 0. 0 π ν π(z/2)ν Neumann functions of integer order Interlacing of zeros Legendre equation The Legendre equation with parameter ν is (1 − z2)u00 − 2zu0 + ν(ν + 1)u = 0. This ODE is not singular at z = 0 so the solution has an ordinary Taylor series expansion k u = Akz . ∞ k=0 X The Ak must satisfy

(k + 1)(k + 2)Ak+2 − [k(k + 1) − ν(ν + 1)]Ak = 0

for k > 0, and since k(k + 1) − ν(ν + 1) = (k − ν)(k + ν + 1), the recurrence relation is (k − ν)(k + ν + 1) A = A for k 0. k+2 (k + 1)(k + 2) k > General solution

We have u(z) = A0u0(z) + A1u1(z) where ν(ν + 1) (ν − 2)ν(ν + 1)(ν + 3) u (z) = 1 − z2 + z4 − ··· 0 2! 4! and

(ν − 1)(ν + 2) u (z) = z − z3 1 3! (ν − 3)(ν − 1)(ν + 2)(ν + 4) + z5 − ··· . 5!

Suppose now that ν = n is a non-negative integer. If n is even then the series for u0(z) terminates, whereas if n is odd then the series for u1(z) terminates. Legendre polynomial

The terminating solution is called the Legendre polynomial of degree n and is denoted by Pn(z) with the normalization

Pn(1) = 1.

The ﬁrst few Legendre polynomials are

1 3 P0(z) = 1, P3(z) = 2 (5z − 3z),

1 4 2 P1(z) = z, P4(z) = 8 (35z − 30z + 3),

1 2 1 5 3 P2(z) = 2 (3z − 1),P5(z) = 8 (63z − 70z + 15z).

Notice that Pn is an even or odd function according to whether n is even or odd. Behaviour of Legendre polynomials Part II

Dynamical Systems Introduction

In this part of the course, we survey several topics in the theory of diﬀerential equations (DEs). None is discussed in any depth. Instead, the aim is to introduce you to a few of the key themes in this subject, some of which are developed further in our third-year mathematics courses. Outline

Examples and terminology

Existence and uniqueness

Practical solution methods

Linear dynamical systems

Higher-order, linear, scalar equations

Stability

Final remarks on nonlinear DEs Examples and terminology

We begin with some examples of how systems of diﬀerential equations arise in applications, and see how all such problems can be formulated as a ﬁrst-order system dx = F(x). dt Such a formulation leads to a natural geometric interpretation of a solution. Predator and prey populations Lotka–Volterra equations

Simpliﬁed ecology with two species:

F(t) = number of foxes at time t, R(t) = number of rabbits at time t.

Assume populations large enough that F and R can be treated as smoothly varying in time.

In the 1920s, Alfred Lotka and Vito Volterra independently proposed the preditor–prey model dF = −aF + αFR, F(0) = F , dt 0 dR = bR − βFR, R(0) = R . dt 0 Here, a, α, b and β are non-negative constants. Zero predation case

If α = 0 = β then the system uncouples to give dF dR = −aF and = bR dt dt so −at bt F(t) = F0e and R(t) = R0e .

Interpretation: if the foxes fail to catch any rabbits, then the foxes starve (F → 0) and the rabbits multiply without limit (R → ).

∞ Interaction terms

Rewrite equations as 1 dF = −a + αR, F dt 1 dR = b − βF. R dt So relative rate of increase in fox popula- αR = tion due to predation on rabbits and relative rate of decrease in rabbit pop- βF = . ulation due to predation by foxes Periodic solutions

Example with a = 1.0, α = 0.5, b = 1.5 and β = 0.75. Vector ﬁeld

Any ﬁrst-order system of N ODEs in the form dx 1 = F (x , x , . . . , x ), x (0) = x , dt 1 1 2 N 1 10 dx 2 = F (x , x , . . . , x ), x (0) = x , dt 2 1 2 N 2 20 . . . . dx N = F (x , x , . . . , x ), x (0) = x , dt N 1 2 N N N0 can be written in vector notation as dx = F(x), x(0) = x . dt 0 The system of ODEs is determined by the vector N N ﬁeld F : R → R . Predator–prey example

Recall dF = −aF + αFR, F(0) = F , dt 0 dR = bR − βFR, R(0) = R . dt 0 So deﬁning F −aF + αFR F0 x = , F(x) = , x0 = R bR − βFR R0

we have dx dF/dt −aF + αFR = = = F(x), dt dR/dt bR − βFR

with x(0) = x0. Geometric viewpoint Think of the trajectory x(t) as a parametric curve in the phase N space R . Then dx/dt = F(x) means that along any trajectory, Fx(t) always points in the forward tangent direction to the trajectory, with the speed of x(t) equal to the magnitude of F. Non-autonomous ODEs

A system of ODEs of the form dx = F(x). dt is said to be autonomous.

For example, we have just seen that the Lotka–Volterra equations are autonomous.

In a non-autonomous system, F may depend explicitly on t: dx = F(x, t). (7) dt Equivalent autonomous system

Given a non-autonomous system (7), let

x F(x, t) y = and G(y) = . t 1

If x = x(t) is a solution of (7), then y = y(t) is a solution of the autonomous system

dy dx/dt F(x, t) = = = G(y), dt dt/dt 1

and vice versa.

Therefore, suﬃcient (in principle) to develop theory for the autonomous case. Second-order ODE

Consider an initial-value problem for a general (possibly non-autonomous) second-order ODE

d2x dx dx = f x, , t , with x = x and = y at t = 0. dt2 dt 0 dt 0

If x = x(t) is a solution, and if we let y = dx/dt, then

dy d2x dx = = f x, , t = f(x, y, t), dt dt2 dt

that is, (x, y) is a solution of the ﬁrst-order system

dx = y, x(0) = x , dt 0 dy = f(x, y, t), y(0) = y . dt 0 Simple oscillator as ﬁrst-order system

The second-order ODE

mx¨ + r0x˙ + k0x = f(t)

is equivalent to the (non-autonomous) pair of ﬁrst-order ODEs

x˙ = v, 1 v˙ = f(t) − r v − k x. m 0 0 That is, dx = F(x, t), dt where x v x = and F(x, t) = −1 . v m f(t) − r0v − k0x Immune response to a viral infection

Human T-cell Simian virus Roy Anderson and Robert May, Infectious Diseases of Humans: Dynamics and Control, Oxford University Press, 1991.

Simple model of infection with dependent variables

V(t) = density of the virus, E(t) = density of the eﬀector cells.

Suppose that in the absence of infection, behaviour of eﬀector cells (e.g., lymphocytes) described by

dE = Λ − µE, dt for constants Λ > 0 (recruitment rate from bone marrow) and µ > 0 (death rate). If E0 is the initial density of eﬀector cells, then Λ E(t) = E e−µt + (1 − e−µt) 0 µ so E(t) → Eˆ ≡ Λ/µ as t → .

∞ The presence of the virus causes E to grow, so that now dE = Λ − µE + VE, dt for a constant > 0 (coeﬃcient of proliferation). In the absence of an immune response (E = 0) the virus population obeys dV = rV, dt rt for a constant r > 0 (intrinsic growth rate), so V = V0e .

However, for E > 0, dV = rV − σVE = (r − σE)V, dt for a constant σ > 0. Provided

r > σEˆ = σΛ/µ, we have dV/dt > 0 so infection can occur (V can grow). Complete 2 × 2 dynamical system: dE = Λ − µE + VE, E(0) = E dt 0 dV = rV − σVE, V(0) = V . dt 0

Example with parameters

Λ = 1.0, µ = 0.5, = 0.01, r = 1.25, σ = 0.01, and initial conditions E0 = 2.0 and V0 = 1.0.

Notice Eˆ = Λ/µ = 2 so r > σEˆ. Numerical solution Phase portrait Existence and uniqueness

The most fundamental question about a dynamical system dx = F(x, t) dt is

For a given initial value x0, does a solution x(t) satisfying x(0) = x0 exist, and if so is this solution unique? Answer is yes, whenever the vector ﬁeld F is Lipschitz. Lipschitz constant

Deﬁnition The number L is a Lipschitz constant for a function f :[a, b] → R if |f(x) − f(y)| 6 L|x − y| for all x, y ∈ [a, b].

Example 2 Consider f(x) = 2x − x + 1 for 0 6 x 6 1. Since f(x) − f(y) = 2(x2 − y2) − (x − y) = 2(x + y)(x − y) − (x − y) = (2x + 2y − 1)(x − y)

we have |f(x) − f(y)| = |2x + 2y − 1| |x − y| so a Lipschitz constant is L = max |2x + 2y − 1| = 3. x,y∈[0,1] Lipschitz implies (uniformly) continuous

We say that the function f :[a, b] → R is Lipschitz if a Lipschitz constant for f exists.

Theorem If f is Lipschitz then f is (uniformly) continuous. Proof. Suppose L is a Lipschitz constant for f :[a, b] → R. Given > 0, if δ = /L then

|f(x) − f(y)| 6 L|x − y| < whenever |x − y| < δ, so f is (uniformly) continuous on [a, b]. Continuous does not imply Lipschitz Example Consider the (uniformly) continuous function √ f(x) = 3 + x for 0 6 x 6 4. In this case, if x, y ∈ (0, 4] then √ √ √ √ √ √ x + y f(x) − f(y) = x − y = x − y × √ √ x + y x − y = √ √ x + y

so if a Lipschitz constant L exists then

|f(x) − f(y)| 1 L = √ √ > |x − y| x + y

for arbitrarily small x and y, a contradiction. Continuously diﬀerentiable implies Lipschitz

Deﬁnition k 0 00 (k) A function f : I → R is C if f, f , f ,..., f all exist and are continous on the interval I. Theorem For any closed and bounded interval I = [a, b], if f is C1 on I then 0 L = maxx∈I |f (x)| is a Lipschitz constant for f on I. Proof. Given a 6 x < y 6 b, the Mean Value Theorem says that there exists a number c (depending on x and y) such that

f(x) − f(y) = f0(c)(x − y) with x < c < y,

0 so |f(x) − f(y)| = |f (c)| |x − y| 6 L|x − y|. Equivalent integral equation

Consider an initial-value problem for a (scalar) ODE,

dx = f(x) for t > 0, with x(0) = x . (8) dt 0 If x = x(t) is a solution then

t x(t) = x0 + f x(s) ds. (9) Z0

Conversely, if x = x(t) is a continuous function satisfying the (Volterra) integral equation (9) then x is a solution of the initial-value problem (8). Picard iterates

We try to solve (9) by ﬁxed point iteration, letting

x1(t) = x0, t x2(t) = x0 + f x1(s) ds, 0 Z t x3(t) = x0 + f x2(s) ds, Z0 and in general,

t xk(t) = x0 + f xk−1(s) ds for k > 1. (10) Z0 Increments

Subtracting t xk(t) = x0 + f xk−1(s) ds Z0 from t xk+1(t) = x0 + f xk(s) ds Z0 gives

t xk+1(t) − xk(t) = f xk(s) − f xk−1(s) ds, Z0 so if L is a Lipschitz constant for f then

|xk+1(t) − xk(t)| 6 L xk(s) − xk−1(s) ds, t > 0. Z0 Estimating the increments Put

δk(t) = |xk+1(t) − xk(t)| and ∆k(t) = max δk(s). 06s6t We have t δk(t) 6 L δk−1(s) ds Z0 so for 0 6 t 6 T, t t δ2(t) 6 L δ1(s) ds 6 L ∆1(T) ds 6 ∆1(T)Lt, 0 0 Z t Z t 2 1 2 δ3(t) 6 L δ2(s) ds 6 L ∆1(T)Ls ds = ∆1(T)L × 2 t , 0 0 Z t Z t 2 1 2 3 1 3 δ4(t) 6 L δ3(s) ds 6 L ∆1(T)L 2 s ds = ∆1(T)L × 3! t , 0 0 . Z Z . Convergence of the Picard iterates By induction on k,

(Lt)k−1 δ (t) ∆ (T) for |t| T. k 6 1 (k − 1)! 6

The telescoping sum xk(t) − x0(t) = x1(t) − x0(t) + x2(t) − x1(t) + ··· + xk(t) − xk−1(t) k−1 = xj+1(t) − xj(t) j=0 X converges uniformly for |t| 6 T because

j−1 (Lt) Lt |xj+1(t) − xj(t)| 6 ∆1(T) = ∆1(T)e . ∞ ∞ (j − 1)! j=0 j=1 X δj(t) X | {z } Thus, we can deﬁne x(t) by the uniformly convergent series

x(t) = lim xk(t) = x0 + xj+1(t) − xj(t) . k→ ∞ j=0 X ∞ In turn, f xk(s) converges uniformly to f x(s) for |s| 6 T, so by sending k → in (10) we have

t x(t)∞ = lim xk+1(t) = x0 + lim f xk(s) ds k→ k→ 0 t Z = x0 +∞ lim f xk(s) ds∞ 0 k→ Z t = x0 + f x∞(s) ds, Z0 dx and therefore = fx(t) with x(0) = x . dt 0 Lipschitz vector ﬁeld

N N A vector ﬁeld F : S → R is Lipschitz on S ⊆ R if

kF(x) − F(y)k 6 Lkx − yk for all x, y ∈ S. Here, N 1/2 2 kxk = xj j=1 X N denotes the Euclidean norm of the vector x ∈ R .

We say that F(x, t) is Lipschitz in x if

kF(x, t) − F(y, t)k 6 Lkx − yk. Local existence and uniqueness

Theorem N Let x0 ∈ R , ﬁx r > 0 and τ > 0, and put

N S = { (x, t) ∈ R × R : kx − x0k 6 r and |t| 6 τ }. If F(x, t) is Lipschitz in x for (x, t) ∈ S, and if

kF(x, t)k 6 M for (x, t) ∈ S, then there exists a unique C1 function x(t) satisfying

dx = F(x, t) for |t| min(r/M, τ), with x(0) = x . dt 6 0

Proof. See Technical Proofs handout. Example of non-uniqueness

The initial-value problem dx = 3x2/3 for t > 0, with x(0) = 0, dt has inﬁnitely many solutions, namely, for any a > 0,

0, 0 t a, x(t) = 6 6 3 (t − a) , t > a.

In this case, f(x) = 3x2/3 is not Lipschitz on any neighbourhood of 0 because

|f(x) − f(0)| 3x2/3 − 0 = = 3x−1/3 → as x → 0. |x − 0| x

∞ Example of local, but not global, existence The (separable) initial-value problem dx = 1 + x2 for t > 0, with x(0) = 1, dt has a unique solution

π 3π π x(t) = tan t + for − < t < . 4 4 4

2 Applying the theorem with f(x) = 1 + x , x0 = 1, r > 0 and τ = , we ﬁnd that

|f(x)| M = r2 + 2r + 2 for |x − x | r, ∞ 6 0 6 and min(r/M, τ) equals √ r r 2 − 1 π 2 6 2 √ = < . r + 2r + 2 r + 2r + 2 r= 2 2 4 Distinct trajectories cannot intersect

Suppose that the trajectories of two solutions x1(t) and x2(t) intersect, that is, x1(t1) = x2(t2)

for some t1 and t2. If we deﬁne

y1(t) = x1(t1 + t) and y2(t) = x2(t2 + t),

then, for j ∈ {1, 2}, y˙ j(t) = x˙ j(tj + t) = F xj(tj + t) = F yj(t) .

Since y1(0) = x1(t1) = x2(t2) = y2(0), uniqueness implies that y1(t) = y2(t) for all t, or in other words

x1(t1 + t) = x2(t2 + t) for all t.

Therefore, the two solutions trace out the same trajectory in phase space. Practical solution methods

This course emphasises analytical (paper and pencil) methods for solving diﬀerential equations, but such an approach works only for relatively simple problems. We now take a brief look at computer approximations of the kind that are widely used for direct numerical simulations in scientiﬁc and industrial modelling (Math2301 and Math3101). Discrete-time approximation

Initial-value problem in 1D: dx = f(x) for 0 < t < T, with x(0) = x . dt 0 Fix P > 0 and a step size ∆t = T/P. Let

tp = p ∆t for p = 0, 1, 2,..., P,

so 0 = t0 < t1 < t2 < ··· < tP = T.

Aim: compute numbers X1, X2,..., XP such that

x(tp) ≈ Xp for 1 6 p 6 P. Finite diﬀerence approximation

Since dx ∆x x(t + ∆t) − x(t) = lim = lim , dt ∆t→0 ∆t ∆t→0 ∆t if ∆t is small (and thus P is large), then

x(t + ∆t) − x(t) dx ≈ = fx(t). ∆t dt

Also, when t = tp,

x(tp + ∆t) = x(tp+1) ≈ Xp+1 and x(tp) ≈ Xp,

which suggests we require X − X p+1 p = f(X ). ∆t p Euler’s method Rearranging: Xp+1 = Xp + f(Xp) ∆t.

Thus, given x0 we let X0 = x0 and calculate

X1 = X0 + f(X0) ∆t,

X2 = X1 + f(X1) ∆t,

X3 = X2 + f(X2) ∆t, . .

XP = XP−1 + f(XP−1) ∆t. Easily programmed on a computer. For instance, using Julia, Dt = T / P X = zeros(P+1); X[1] = x0 for p = 1:P X[p+1] = X[p] + f(X[p]) * Dt end Example

Consider dx = ax for t > 0, with x(0) = x , dt 0 at which has the solution x = x0e . Euler’s method is X − X p+1 p = aX for p = 0, 1, 2, . . . , with X = x , ∆t p 0 0

so Xp+1 = (1 + a ∆t)Xp and therefore

p Xp = (1 + a ∆t) x0.

If we send ∆t → 0 and p → , keeping t∗ = p ∆t ﬁxed, then p at∗ (1 + a ∆t)p =∞ 1 + → eat∗ , p

showing that Xp ≈ x(t∗) = x(tp). Euler’s method with a = 1, P = 10 and T = 1 Digression: order notation

Deﬁnition We write φ(y) = Oψ(y) as y → a, if there exist constants C > 0 and δ > 0 such that

|φ(y)| 6 C|ψ(y)| whenever |y − a| 6 δ.

This notation can be useful if ψ(y) is much simpler than φ(y).

Interpret φ(y) = γ(y) + Oψ(y) as φ(y) − γ(y) = Oψ(y). Taylor expansions

Recall that if f is Cm+1 then then

(x − a)2 f(x) = f(a) + f0(a)(x − a) + f00(a) + ··· 2 (x − a)m + f(m)(a) + R (x), m! m where the remainder term is x (x − y)m R (x) = f(m+1)(y) dy m m! Za

Theorem If f is Cm+1 on a neighbourhood of a, then

m+1 Rm(x) = O (x − a) as x → a. Proof. (m+1) There exist constants C and δ such that |f (y)| 6 C for |x − a| 6 δ. If 0 6 x − a 6 δ, then x (x − y)m C |R (x)| C dy = (x − a)m+1, m 6 m! (m + 1)! Za and similarly if −δ 6 x − a < 0. Example As x → 0,

sin x = x + O(x3), 1 2 4 cos x = 1 − 2 x + O(x ), 1 2 3 log(1 + x) = x − 2 x + O(x ), 1 = 1 − x2 + O(x4). 1 + x2 How accurate is Euler’s method?

Taylor expansion shows that if x(t) is C2, then as ∆t → 0,

x(t + ∆t) = x(t) + x˙(t) ∆t + O(∆t2) = x(t) + fx(t) ∆t + O(∆t2),

so in particular when t = t0,

2 x(t1) = x(t0) + f x(t0) ∆t + O(∆t ).

Thus, after the ﬁrst step of Euler’s method, since X0 = x0 = x(t0),

X1 = X0 + f(X0) ∆t = x(t0) + f x(t0) ∆t 2 = x(t1) + O(∆t ).

What happens after p steps? Theorem (Math2301)

Assume that we compute Xp ≈ x(tp) using Euler’s method. If L is a Lipschitz constant for f, then for 0 6 tp 6 T,

1 Ltp |Xp − x(tp)| 6 2 tpe ∆t max |x¨(t)| 06t6T 1 LT 6 C ∆t where C = 2 Te max |x¨(t)|. 06t6T

Thus, for Euler’s method we have

Xp = x(tp) + O(∆t) as ∆t → 0.

Example How many steps are needed to ensure the error is less than 10−4 if −4 T = 5, L = 1 and |x¨(t)| 6 2? Suﬃcient to satisfy C ∆t 6 10 . T TeLT 10−4 ⇐⇒ P T 2eLT × 104 ≈ 37, 103, 290. P 6 > A more eﬃcient method

Since dx/dt = f(x), the chain rule gives

d2x dx x¨ = = f0(x) = f0(x)f(x) dt2 dt so by Taylor expansion,

1 2 3 x(t + ∆t) = x(t) + x˙(t) ∆t + 2 x¨(t) ∆t + O(∆t ) 1 0 2 3 = x(t) + f x(t) ∆t + 2 f x(t) f x(t) ∆t + O(∆t ) We therefore deﬁne the Taylor method of order 2 by

1 0 2 Xp+1 = Xp + f(Xp) ∆t + 2 f (Xp)f(Xp) ∆t . Euler step correction term | {z } | {z } Comparison We saw earlier that for Euler’s method,

Xp = x(tp) + O(∆t) as ∆t → 0.

Can show in a similar way that for the Taylor method of order 2,

2 Xp = x(tp) + O(∆t ) as ∆t → 0.

Example When T = 1 we have ∆t = 1/P, so

P ∆t ∆t2 10 0.1 0.01 100 0.01 0.0001 1000 0.001 0.000001 10000 0.0001 0.00000001 100000 0.00001 0.0000000001 Example

Consider dx = cos2 x for t > 0, with x(0) = 0, dt which has the solution x = tan−1(t). Since

f(x) = cos2 x and f0(x) = −2 cos x sin x,

the Taylor method of order 2 is

1 0 2 Xp+1 = Xp + f(Xp) ∆t + 2 f(Xp)f (Xp) ∆t 2 2 3 = Xp + ∆t cos Xp − ∆t cos Xp sin Xp

for p = 0, 1, 2, . . . , with X0 = 0. Comparison using P = 8 steps Systems of ODEs

N N Euler’s method works in the same way if F : R → R and dx = F(x) for 0 < t < T, with x(0) = x . dt 0

Put X0 = x0 and compute vectors Xp ≈ x(tp) by

Xp+1 = Xp + ∆t F(Xp) for p = 0, 1, 2,..., P − 1.

Similarly, for the Taylor method of order 2,

1 2 0 Xp+1 = Xp + ∆t F(Xp) + 2 ∆t F (Xp)F(Xp).

0 N×N Here, F (x) = [∂Fi/∂xj] ∈ R denotes the Jacobian matrix. Immune response example

Recall dE = Λ − µE + VE, E(0) = E dt 0 dV = rV − σVE, V(0) = V . dt 0 In this case,

Λ − µE + VE −µ + V E F(E, V) = , F0(E, V) = rV − σVE −σV r − σE

and the Taylor method of order 2 is

E E Λ − µE + V E p+1 = p + ∆t p p p Vp+1 Vp rVp − σVpEp ∆t2 −µ + V E Λ − µE + V E + p p p p p . 2 −σVp r − σEp rVp − σVpEp How to avoid computing F0 In 1895, Carl Runge proposed the scheme

Φ1 = F(Xp),

Φ2 = F(Xp + ∆t Φ1), 1 Xp+1 = Xp + 2 ∆t (Φ1 + Φ2), which requires two evaluations of F per step, but no knowledge of F0. Since

0 2 Φ2 = F(Xp) + F (Xp)(∆t Φ1) + O(∆t ) 0 2 = F(Xp) + ∆t F (Xp)F(Xp) + O(∆t )

we have

1 2 0 3 Xp+1 = Xp + ∆t F(Xp) + 2 ∆t F (Xp)F(Xp) +O(∆t ). Taylor method of order 2 | {z } Thus, up to O(∆t3), Runge’s method agrees with the Taylor method of order 2, so in both cases we expect

2 Xp = x(tp) + O(∆t ) for p = 1, 2,..., P.

In 1901, Martin Kutta proposed a more elaborate scheme,

Φ1 = F(Xp), 1 Φ2 = F(Xp + 2 ∆t Φ1), 1 Φ3 = F(Xp + 2 ∆t Φ2), Φ4 = F(Xp + ∆t Φ3), 1 Xp+1 = Xp + 6 ∆t Φ1 + 2Φ2 + 2Φ3 + Φ4 , and showed that it behaved like a Taylor method of order 4:

4 Xp = x(tp) + O(∆t ) for p = 1, 2,..., P. Convergence rate

Denote the maximum error using P steps by

EP = max kXp − x(tp)k, 06p6P and suppose that, for some positive constants C and r,

r EP ≈ C ∆t .

r −r Since ∆t = T/P it follows that EP ≈ CT P , so

r −r EP/2 CT (P/2) r ≈ r −r = 2 EP CT P and thus r ≈ log2(EP/2/EP). Simple test problem For the system dx = 2x + 3y, x(0) = 2, dt dy = −3x + 2y, y(0) = −1, dt we obtain the following maximum errors and convergence rates (taking T = 1).

P Euler RK2 RK4 20 4.64e+00 3.02e-01 4.44e-04 40 2.21e+00 1.07 7.77e-02 1.958 2.81e-05 3.981 80 1.06e+00 1.05 1.97e-02 1.979 1.77e-06 3.993 160 5.19e-01 1.03 4.96e-03 1.989 1.11e-07 3.997 320 2.56e-01 1.02 1.25e-03 1.995 6.92e-09 3.999 640 1.27e-01 1.01 3.12e-04 1.997 4.33e-10 3.999 Linear dynamical systems

Linear diﬀerential equations are generally much easier to solve than nonlinear ones. Hence, most of this course is devoted to linear problems. Fortunately, linear DEs suﬃce for describing many important applications.

Also, understanding how the solutions of linear ODEs behave can help us understand nonlinear ODEs. First-order, linear systems of ODEs

We say that the N × N, ﬁrst-order system of ODEs dx = F(x, t) dt is linear if the RHS has the form

F(x, t) = A(t)x + b(t)

for some N × N matrix-valued function A(t) = [aij(t)] and a vector-valued function b(t) = [bi(t)].

The system is autonomous precisely when A and b are constant. Global existence and uniqueness

We have a stronger existence result in the linear case.

Theorem If A(t) and b(t) are continuous for 0 6 t 6 T, then the linear initial-value problem dx = A(t)x + b(t) for 0 t T, with x(0) = x , dt 6 6 0

has a unique solution x(t) for 0 6 t 6 T.

We now investigate the special case when A is constant and b(t) ≡ 0: dx = Ax. dt General solution via eigensystem If Av = λv and we deﬁne x(t) = eλtv, then

dx = λeλtv = eλt(λv) = eλt(Av) = A(eλtv) = Ax dt that is, x is a solution of dx/dt = Ax. If Avj = λjvj for 1 6 j 6 N, then the linear combination

N λjt x(t) = cje vj (11) j=1 X is also a solution because the ODE is linear and homogeneous. Provided the vj are linearly independent, (11) is the general N solution because given any x0 ∈ R there exist unique cj such that

N x(0) = cjvj = x0. j=1 X Example Consider dx = −5x + 2y, x(0) = 5, dt dy = −6x + 3y, y(0) = 7. dt In this case, −5 2 1 1 A = , λ = −3, v = , λ = 1, v = , −6 3 1 1 1 2 2 3 so the general solution is 1 1 x(t) = c e−3t + c et , 1 1 2 3

and the initial conditions imply c1 = 4 and c2 = 1, so −3t t 1 1 x = 4e + e , x(t) = 4e−3t + et and 1 3 y = 4e−3t + 3et. Exponential of a matrix Recall that the Taylor series

zk z2 z3 ez = = 1 + z + + + ··· ∞ k! 2! 3! k=0 X converges for all z ∈ C (Math2621). N×N By analogy, given A ∈ C , we deﬁne (Math2601)

Ak A2 A3 eA = = I + A + + + ··· . ∞ k! 2! 3! k=0 X This series always converges because, for any matrix norm (Math2301),

k k A kA k kAk kAk ke k 6 6 = e . ∞ k! ∞ k! k=0 k=0 X X Term-by-term diﬀerentiation

Given A and x0, let

t2A2 tkAk x(t) = etAx = I + tA + + ··· + + ··· x . 0 2! k! 0

Since d tk−1Ak etA = 0 + A + tA2 + ··· + + ··· dt (k − 1)! tk−1Ak−1 = A I + tA + ··· + + ··· = AetA, (k − 1)!

we have dx = AetAx = Ax and x(0) = Ix = x . dt 0 0 0 But how can we calculate etA explicitly? Diagonalising a matrix

Deﬁnition N×N A square matrix A ∈ C is diagonalisable if there exists a N×N −1 non-singular matrix Q ∈ C such that Q AQ is diagonal. Theorem N×N A square matrix A ∈ C is diagonalisable if and only if there N exists a basis {v1, v2,..., vN} for C consisting of eigenvectors of A. Indeed, if

Avj = λjvj for j = 1, 2,..., N,

−1 and we put Q = [v1 v2 ··· vN], then Q AQ = Λ where   λ1  ..  Λ =  .  . λN Matrix powers

Consider a diagonalisable matrix A. Since Q−1AQ = Λ, it follows that A has an eigenvalue decomposition

A = QΛQ−1.

Thus

A2 = (QΛQ−1)(QΛQ−1) = QΛ(Q−1Q)ΛQ−1 = QΛIΛQ−1 = QΛ2Q−1

and A3 = A2A = QΛ2Q−1QΛQ−1 = QΛ3Q−1. In general, we see by induction on k that

Ak = QΛkQ−1 for k = 0, 1, 2,.... Since Λ is diagonal, so is     λ1 λ1  λ2   λ2  2     Λ =  ..   ..   .   .  λN λN  2  λ1  λ2   2  =  ..  ,  .  2 λN and in general,

 k  λ1  λk  k  2  Λ =  ..  for k = 0, 1, 2, 3,....  .  k λN Example If −5 2 A = −6 3 then −3 0 1 1 1 3 −1 Λ = ,Q = ,Q−1 = 0 1 1 3 2 −1 1 so 1 1 1 (−3)k 0 3 −1 Ak = QΛkQ−1 = 2 1 3 0 1 −1 1 1 (−1)k × 3k+1 − 1 (−1)k+1 × 3k + 1 = . 2 (−1)k × 3k+1 − 3 (−1)k+1 × 3k + 3 Polynomial of a matrix

For any polynomial

2 m p(z) = c0 + c1z + c2z + ··· + cmz

and any square matrix A, we deﬁne

2 m p(A) = c0I + c1A + c2A + ··· + cmA .

When A is diagonalisable, Ak = QΛkQ−1 so

−1 −1 2 −1 p(A) = c0QIQ + c1QΛQ + c2QΛ Q + ··· m −1 + cmQΛ Q 2 m −1 = c0QI + c1QΛ + c2QΛ + ··· + cmQΛ Q 2 m −1 = Q c0I + c1Λ + c2Λ + ··· + cmΛ Q = Qp(Λ)Q−1. Lemma For any polynomial p and any diagonal matrix Λ,   p(λ1)  p(λ2)    p(Λ) =  ..  .  .  p(λN)

Theorem If two polynomials p and q are equal on the spectrum of a diagonalisable matrix A, that is, if

p(λj) = q(λj) for j = 1, 2,..., N, then p(A) = q(A). Example Recall that −5 2 A = −6 3 has eigenvalues λ1 = −3 and λ2 = 1. Let

p(z) = z2 − 4 and q(z) = −2z − 1, and observe

p(−3) = 5 = q(−3) and p(1) = −3 = q(1).

We ﬁnd 9 −4 p(A) = A2 − 4I = = −2A − I = q(A). 12 −7 Exponential of a diagonalisable matrix

Theorem If A = QΛQ−1 is diagonalisable, then

eλ1   eλ2  A Λ −1 Λ   e = Qe Q and e =  ..   .  eλN

Proof. eA equals

Ak QΛkQ−1 ΛkQ−1 Λk = = Q = Q Q−1. ∞ k! ∞ k! ∞ k! ∞ k! k=0 k=0 k=0 k=0 X X X X Example Again put −5 2 A = . −6 3 We have 1 1 1 e−3 0 3 −1 eA = QeΛQ−1 = 2 1 3 0 e1 −1 1 1 3e−3 − e −e−3 + e = . 2 3e−3 − 3e −e−3 + 3e

Notice tA = Q(tΛ)Q−1 so

1 1 1 e−3t 0 3 −1 etA = QetΛQ−1 = 2 1 3 0 et −1 1 1 3e−3t − et −e−3t + et = . 2 3e−3t − 3et −e−3t + 3et A (maybe) simpler method

The following trick lets you compute eA without ﬁnding Q.

If a polynomial p has the property

λj p(λj) = e for j = 1, 2,..., N,

then eA = QeΛQ−1 = Qp(Λ)Q−1 = p(A).

For example, we can choose p with degree 6 N − 1 using the Lagrange interpolation formula,

N N λ − λr p(λ) = eλj . λj − λr j=1 r=1 X Yr6=j A 3 × 3 example

Problem: ﬁnd etA for  6 2 2 A = −2 8 4 . 0 1 7

Can show that A = QΛQ−1 where

 1 2 2 6 0 0 Q = −1 0 1 and Λ = 0 7 0 , 1 1 1 0 0 8 so tA = Q(tΛ)Q−1 for all t. Fix t and put

(λ − λ2)(λ − λ3) (λ − λ1)(λ − λ3) p(λ) = eλ1t + eλ2t (λ1 − λ2)(λ1 − λ3) (λ2 − λ1)(λ2 − λ3) (λ − λ1)(λ − λ2) + eλ3t (λ3 − λ2)(λ3 − λ2) (λ − 7)(λ − 8) (λ − 6)(λ − 8) = e6t + e7t (6 − 7)(6 − 8) (7 − 6)(7 − 8) (λ − 6)(λ − 7) + e8t (8 − 6)(8 − 7) so that

p(6) = e6t, p(7) = e7t, p(8) = e8t. Thus,

e6t etA = p(A) = (A − 7I)(A − 8I) − e7t(A − 6I)(A − 8I) 2 e8t + (A − 6I)(A − 7I) 2 −1 0 2  −4 2 6 −2 2 4 6t 7t 8t = e  1 0 −2 − e  0 0 0 + e −1 1 2 , −1 0 2 −2 1 3 −1 1 2 which equals

−e6t + 4e7t − 2e8t −2e7t + 2e8t 2e6t − 6e7t + 4e8t  e6t − e8t e8t −2e6t + 2e8t  . −e6t + 2e7t − e8t −e7t + e8t 2e6t − 3e7t + 2e8t Digression: higher-order, linear, scalar equations

Consider once again

m j Lu = aj(x)D u. j=0 X We are now ready to prove the result stated earlier:

Theorem Assume that the ODE Lu = f is not singular with respect to [a, b], and that f is continuous on [a, b]. Then the IVP (2) and (3) has a unique solution. Equivalent ﬁrst-order system

Suﬃcient to look at the case m = 3,

000 00 0 Lu = a3u + a2u + a1u + a0u.

(General mth order case follows in the same way.)

0 00 Writing y1 = u, y2 = u and y3 = u we see that Lu = f if and only if dy 1 = y , dx 2 dy 2 = y , (12) dx 3 dy3 1 = f − a0y1 − a1y2 − a2y3 . dx a3

Can now see why to expect trouble if L is singular, that is, if the leading coeﬃcient a3 vanishes at any x in the interval of interest. Equivalent ﬁrst-order system: matrix version

We can write Lu = f as dy = A(x)y + b(x), dx where, when m = 3,

 0 1 0  A(x) =  0 0 1  , −a0/a3 −a1/a3 −a2/a3  u   0  y(x) = u0  , b(x) =  0  . 00 u f/a3 Existence and uniqueness

Theorem Assume that L is not singular on [a, b] and that f is continuous on [a, b]. Then the IVP (2) and (3) has a unique solution. Proof. For m = 3, the IVP (2) and (3) is equivalent to

ν  dy 0 = A(x)y + b(x), y(0) = ν dx  1 ν2

0 00 where y1 = u, y2 = u , y3 = u and

 0 1 0   0  A(x) =  0 0 1  , b(x) =  0  . −a0/a3 −a1/a3 −a2/a3 f/a3 Stability

In many applications we are interested to know how the solution x(t) behaves as t → , and might not care much about the precise details of the transient behaviour for finite t. A lot can be discovered without the need∞ to solve the DE, which is just as well because finding x(t) explicitly is often difficult or impossible, especially for nonlinear problems. Equilibrium points

N We say that a ∈ R is an equilibrium point for the dynamical system dx/dt = F(x) if F(a) = 0. In this case, the solution of dx = F(x) for all t, with x(0) = a, dt is just the constant function x(t) = a. Equilibrium points for the viral infection model Recall dE = Λ − µE + VE, dt dV = rV − σVE. dt Here, (E, V) is an equilibrium point iﬀ

Λ − µE + VE = 0, rV − σVE = 0.

Thus, the only equilibrium points are at Λ E = and V = 0, µ and at r 1 σΛ E = and V = µ − . σ r Stable equilibrium

Deﬁnition An equilibrium point a is stable if for every > 0 there exists δ > 0 such that whenever kx0 − ak < δ the solution of dx = F(x) for t > 0, with x(0) = x dt 0 satisﬁes kx(t) − ak < for all t > 0. (In particular, x(t) must exist for all t > 0.)

In this case, if x0 → a then x(t) → a, uniformly for t > 0.

In particular, a trajectory stays close to a stable equilibrium point a if it starts out suﬃciently close to a. Asymptotic stability

Definition N Let D be an open subset of R that contains an equilibrium point a. We say that a is asymptotically stable in D if a is stable and, whenever x0 ∈ D, the solution of dx = F(x) for t > 0, with x(0) = x dt 0 satisfies x(t) → a as t → . In this case D is called a domain of attraction for a. ∞ Linear, constant-coefficient case

Consider dx = Ax + b with x(0) = x . (13) dt 0 Since Ax + b = 0 ⇐⇒ x = −A−1b, the only equilibrium point is a = −A−1b. Moreover,

tA x(t) = a + e (x0 − a)

because dx = AetA(x − a) = A(x − a) = Ax − Aa = Ax + b dt 0 and x(0) = a + I(x0 − a) = x0. Criteria for stability

Theorem Let A be a diagonalisable matrix with eigenvalues λ1, λ2,..., λN. The equilibrium point a = −A−1b is of (13) 1. stable if and only Re λj 6 0 for all j. 2. asymptotically stable if and only Re λj < 0 for all j. N In the second case, the domain of attraction is the whole of R . Proofs

1. Using the eigenvalue decomposition A = QΛQ−1, we have

tA tΛ −1 x(t) − a = e (x0 − a) = Qe Q (x0 − a).

λ t (Re λ )t If Re λj 6 0 then 0 < |e j | = e j 6 1 for all t > 0, so

N N tΛ 2 λjt 2 2 2 ke wk = e wj 6 |wj| = kwk , j=1 j=1 X X

implying that a is stable. Otherwise, Re λj > 0 for at least one j and eλjt → as t → .

2. For asymptotic∞ stability∞ it is necessary and suﬃcient that eλjt → 0 as t → , for all j.

∞ Example: stable (but not asymptotically stable)

Consider dx 2 5 = Ax where A = . dt −1 −2

Eigenvalues λ1 = i and λ2 = −i so Re λ1 = 0 = Re λ2, hence stable. Eigenvalue decomposition A = QΛQ−1 where

i 0 5 5 Λ = ,Q = [v v ] = , 0 −i 1 2 i − 2 −i − 2 1 1 − 2i −5i Q−1 = , 10 1 + 2i 5i

and thus cos t − 2 sin t 5 sin t etA = QetΛQ−1 = . − sin t cos t − 2 sin t Example: asymptotically stable

Consider dx 14 −9 = Ax where A = . dt 30 −19

Eigenvalues λ1 = −1 and λ2 = −4 so Re λ1 < 0 and Re λ2 < 0, hence asymptotically stable. Eigenvalue decomposition A = QΛQ−1 where

−1 0 3 1 Λ = ,Q = [v v ] = , 0 −4 1 2 5 2 2 −1 Q−1 = , −5 3

and thus 6e−t − 5e−4t −3e−t + 3e−4t etA = QetΛQ−1 = . 10e−t − 10e−4t −5e−t + 6e−4t Example: unstable

Consider dx −26 36 = Ax where A = . dt −18 25

Eigenvalues λ1 = 1 and λ2 = −2 so Re λ1 > 0 and Re λ2 < 0, hence unstable. Eigenvalue decomposition A = QΛQ−1 where

1 0 4 3 Λ = ,Q = [v v ] = , 0 −2 1 2 3 2 −2 3 Q−1 = , 3 −4

and thus −8et + 9e−2t 12et − 12e−2t etA = QetΛQ−1 = . −6et + 6e−2t 9et − 8e−2t Linearization

Suppose that x0 is close to an equilibrium point a. If dx = F(x) for all t, with x(0) = x , (14) dt 0 then for small t the diﬀerence y = x − a is small and satisﬁes dy dx = = F(x) = F(a + y) ≈ F(a) + F0(a)y. dt dt

This suggests that if y0 = x0 − a and y is the solution of the linear dynamical system dy = F(a) + F0(a)y for all t, with y(0) = y , dt 0 then x(t) ≈ a + y(t) for small t. In particular, we can infer stability properties of (14) at an equilibrium point a from the eigenvalues of A = F0(a). Viral infection model

Recall that,

dE/dt Λ − µE + VE = F(E, V) = . dV/dt rV − σVE

Thus, −µ + V E F0(E, V) = , −σV r − σE so at the ﬁrst equilibrium point,

Λ −µ Eˆ E = = E,ˆ V = 0, F0(Λ/µ, 0) = . µ 0 r − σEˆ

Hence, λ1 = −µ < 0 and λ2 = r − σEˆ, which means that this ˆ equilibrium point is stable iﬀ r 6 σE. Damped pendulum

The angular deﬂection θ of a damped pendulum satisﬁes

mθ¨ + rθ˙ + k sin θ = 0,

where θ = 0 is the “down” position of the bob. For simplicity, take m = 1, so that we have the equivalent ﬁrst-order system dθ = φ, dt dφ = −k sin θ − rφ. dt The equilibrium points are

(θ, φ) = (nπ, 0)

for n ∈ {..., −2, −1, 0, 1, 2, . . .}. Writing d θ φ = F(θ, φ) = dt φ −k sin θ − rφ we see that 0 1 F0(θ, φ) = , −k cos θ −r and the matrix 0 1 A = F0(nπ, 0) = (−1)n+1k −r has eigenvalues given by

λ −1 2 n det(λI − A) = = λ + rλ + (−1) k = 0. (−1)nk λ + r When n is even, we solve λ2 + rλ + k = 0 to obtain √ √ 1 −r ± i 4k − r2, 0 r < 2 k, 2 √ 6 √ λ = λ = − k, r = 2 k, ±  √ √  1 2  2 −r ± r − 4k , r > 2 k. 

However, when n is odd, we solve λ2 + rλ − k = 0 to obtain

1 p 2 λ = λ± = 2 −r ± r + 4k , r > 0. Conclusion: equilibrium point at (θ, φ) = (nπ, 0) classiﬁed as follows.

r = 0 √ Re λ+ = Re λ− = 0 stable n even 0 < r 6√2 k Re λ+ = Re λ− < 0 asymptot. stable r > 2 k λ− < λ+ < 0 asymptot. stable n odd r > 0 λ− < 0 < λ+ unstable

Makes sense because θ = nπ is the “down” position if n is even, but is the “up” position if n is odd. Final remarks on nonlinear DEs

We conclude this part of the course by discussing a technique for solving, or at least partially solving, some nonlinear problems. Importantly, this technique allows us to deal with the 1D motion of a particle moving under the inﬂuence of a conservative force. First integrals

Deﬁnition N A function G : R → R is a ﬁrst integral (or constant of the motion) for the system of ODEs

dx = F(x) dt if Gx(t) is constant for every solution x(t). By the chain rule,

d N ∂G dx dx Gx(t) = j = ∇G(x) · = ∇G(x) · F(x). dt ∂xj dt dt j=1 X Geometric Interpretation: G is a ﬁrst integral iﬀ

∇G(x) ⊥ F(x) for all x. Simple example The function G(x, y) = x2 + y2 is a ﬁrst integral of the linear system of ODEs dx = −y, dt dy = x. dt In fact, putting −y F(x, y) = x we have 2x −y ∇G · F = · = (2x)(−y) + (2y)(x) = 0, 2y x or equivalently, dG ∂G dx ∂G dy = + = (2x)(−y) + (2y)(x) = 0. dt ∂x dt ∂y dt Partial solutions

In eﬀect, a ﬁrst integral provides a partial solution of the ODE.

Putting C = G(x0), we know that x(t) is conﬁned to the surface G(x) = C.

If N = 2, the equation G(x1, x2) = C implicitly gives x2 = g(x1), so dx 1 = F (x , x ) = F x , g(x ) dt 1 1 2 1 1 1

and F1 becomes a known function of x1 alone. If we can then evaluate dx t = 1 F Z 1 to obtain t = t(x1), then implicitly we know x1 = x1(t) and ﬁnally x2 = g x1(t) . A class of second-order ODE

Consider d x x˙ x¨ = f(x) or equivalently = , (15) dt x˙ f(x)

and suppose that −V(x) is an indeﬁnite integral of f, that is, −dV/dx = f(x). Since

d x˙ 2 d dV dx = x˙ x¨ and V(x) = = −f(x)x,˙ dt 2 dt dx dt if x = x(t) is a solution of (15) then

d x˙ 2/2 + V(x) = x¨ − f(x)x˙ = 0, dt

1 2 so the function G(x, x˙) = 2 x˙ + V(x) is a first integral. Example: undamped pendulum The angular deflection of an undamped pendulum satisfies

mθ¨ + k sin θ = 0, (16)

or r k θ¨ + ω2 sin θ = 0, ω = , m which has the form dV θ¨ = f(θ) = − dθ with f(θ) = −ω2 sin θ and V(θ) = −ω2 cos θ. Thus,

d 1 θ˙ 2 − ω2 cos θ = θ˙θ¨ + ω2θ˙ sin θ = θ˙θ¨ + ω2 sin θ = 0 dt 2 and so every solution of (16) satisﬁes

1 ˙ 2 2 2 θ − ω cos θ = C.

Partial solutions in higher dimensions

Suppose N > 2 and we know several (functionally independent) ﬁrst integrals G1, G2,..., Gk. Then the solution x(t) must lie on the intersection of the surfaces Gj(x) = Cj for 1 6 j 6 k, where Cj = Gj(x0).

We necessarily have k 6 N − 1.

If k = N − 1, then the implicit function theorem gives xj = gj(x1) for 2 6 j 6 N, and thus F1(x) becomes a known function of x1 so, as before dx t = 1 . F Z 1 In principle, we then know x1 = x1(t) and hence xj = gj x1(t) for 2 6 j 6 N.

However, a system might not have any ﬁrst integrals. Lorenz equations

Edward N. Lorenz, Journal of Atmospheric Sciences 20:130–141, 1963.

The 3 × 3 system of ODEs dx = −σx + σy, dt dy = rx − y − xz, dt dz = xy − bz. dt is a very simplified model of convection in a thin layer of fluid heated uniformly from below and cooled uniformly from above. The dependent variables give the coefficients of 3 Fourier modes. Lorenz equations

For r > 1, the system has three equilibrium points at

(0, 0, 0), pb(r − 1), pb(r − 1), r − 1, −pb(r − 1), −pb(r − 1), r − 1.

The parameter choices 8 r = 28, σ = 10, b = , 3 and initial conditions 1 x = y = z = , 0 0 0 2 lead to the trajectory shown on the next page. Lorenz equations Lorenz equations

Since the solution does not lie on a smooth surface we conclude that no ﬁrst integral exists.

The Lorenz equations also exhibit a sensitive dependence on initial conditions, as illustrated in the following video.

http://www.youtube.com/watch?v=FYE4JKAXSfY Part III

Initial-Boundary Value Problems in 1D Introduction

We have seen that an initial-value problem for a (nonsingular) linear ODE Lu = f always has a unique solution. However, matters are not so simple for a boundary-value problem: a solution might not exist, or if one exists it might not be unique. One aim of this part of the course is to understand when these diﬀerent outcomes occur with the help of some concepts adapted from linear algebra.

We also begin our study of partial diﬀerential equations (PDEs) by considering two important examples, the wave equation and the heat equation, in one spatial variable. Outline

Two-point boundary value problems

Existence and uniqueness

Inner products and norms of functions

Self-adjoint diﬀerential operators

The vibrating string

Heat equation Two-point boundary value problems

In an mth order initial-value problem we specify m initial conditions at the left end of an interval. In an mth order boundary-value problem, we again specify m conditions involving the solution and its derivatives, but some apply at the left end and some at the right end.

We will consider only second-order linear boundary-value problems, so exactly one boundary condition will apply at each end of the interval. Initial conditions vs boundary conditions

Consider the second-order ODE

u00 + u = 0 for 0 < x < π,

whose general solution is

u(x) = A cos x + B sin x.

We obtain a unique solution by specifying initial conditions, that is, specifying u(0) and u0(0).

What if we instead specify boundary conditions at x = 0 and at x = π? Three possibilities

Example A unique solution u(x) = sin x exists satisfying

u0(0) = 1 and u(π) = 0.

Example No solution exists satisfying

u(0) = 0 and u(π) = 1.

Example Inﬁnitely many solutions u(x) = C sin x exist satisfying

u(0) = 0 and u(π) = 0. Linear two-point boundary value problem We want to solve

Lu = f for a < x < b, with B1u = α1 and B2u = α2, (17) where 00 0 Lu = a2u + a1u + a0u is a 2nd-order linear diﬀerential operator, and the boundary functionals have the form

0 B1u = b11u (a) + b10u(a), 0 B2u = b21u (b) + b20u(b).

Example

u00 − u = x − 1 for 0 < x < log 2, u = 2 at x = 0, u0 − 2u = 2 log 2 − 4 at x = log 2. Existence and uniqueness

Having seen a couple of examples, we now investigate the conditions that determine whether or not a linear boundary-value problem has a unique solution. Linearity (again)

Since L, B1 and B2 are all linear, the solutions of the homogeneous BVP

Lu = 0 for a < x < b, with B1u = 0 and B2u = 0, (18)

form a vector space: if u1 and u2 are solutions of (18) then so is u = c1u1 + c2u2 for any constants c1 and c2.

Any two solutions of the inhomogeneous problem (17) diﬀer by a solution of the homogeneous problem, that is, if u1 and u2 satisfy (17) then u = u1 − u2 satisﬁes (18).

If u1 satisfies (17) and u2 satisfies (18) then u = u1 + cu2 satisfies (17) for any constant c. Existence and uniqueness

The preceding facts imply the following result. Theorem (Uniqueness) The inhomogeneous BVP (17) has at most one solution iﬀ the homogeneous BVP (18) has only the trivial solution u ≡ 0.

To investigate existence, suppose that the general solution of the homogeneous equation Lu = 0 is uH = c1u1(x) + c2u2(x), and suppose that uP(x) is a particular solution of the inhomogeneous equation Lu = f. The general solution of Lu = f will then be

u(x) = uH(x) + uP(x) = c1u1(x) + c2u2(x) + uP(x). Existence and uniqueness (continued)

To satisfy the boundary conditions in (17) we must choose c1 and c2 so that

B1(c1u1 + c2u2 + uP) = α1,

B2(c1u1 + c2u2 + uP) = α2,

Since B1 and B2 are linear, the inhomogeneous BVP (17) has at least one solution iﬀ the 2 × 2 linear system

B u B u c α − B u 1 1 1 2 1 = 1 1 P (19) B2u1 B2u2 c2 α2 − B2uP

T has at least one solution [c1, c2] . Similarly, u(x) = c1u1(x) + c2u2(x) is a solution of the homogeneous BVP (18) iﬀ c1 and c2 satisfy

B u B u c 0 1 1 1 2 1 = . B2u1 B2u2 c2 0

The 2 × 2 matrix on the left is non-singular iﬀ this homogeneous linear system has only the trivial solution c1 = c2 = 0.

Hence, the following is true. Theorem If the homogeneous problem (18) has only the trivial solution, then for every choice of f, α1 and α2 the inhomogeneous problem (17) has a unique solution. Inner products and norms of functions

If the homogeneous problem (18) admits nontrivial solutions, then the inhomogeneous problem (17) might or might not have any solutions, depending on the data f, α1 and α2. To formulate a condition that guarantees existence we require a short digression that introduces some ideas from functional analysis. Inner product and norm

The inner product hf, gi of a pair of continuous functions f, g :[a, b] → R is deﬁned by

b hf, gi = f(x)g(x) dx. Za The corresponding norm of f is deﬁned by

b 1/2 kfk = phf, fi = [f(x)]2 dx . Za We say that f and g are orthogonal if hf, gi = 0. Example If (a, b) = (−1, 1), f(x) = x, g(x) = cos πx, then r 2 hf, gi = 0, kfk = , kgk = 1. 3 Thus, f and g are orthogonal over the interval (−1, 1). Example Later we will show that the Legendre polynomials are orthogonal over the interval (−1, 1):

1 Pn(x)Pk(x) dx = 0 if n 6= k. Z−1 Key properties

The inner product and norm for functions behave like the dot n product and norm for vectors in R .

For any continuous functions f, g, h and any constant α: 1. hf + g, hi = hf, hi + hg, hi; 2. hf, g + hi = hf, gi + hf, hi; 3. hαf, gi = αhf, gi = hf, αgi; 4. hf, gi = hg, fi; 5. kfk = 0 if and only f(x) = 0 for a < x < b. Cauchy–Schwarz inequality

Theorem |hf, gi| 6 kfkkgk. Proof. If f = 0 or g = 0 then hf, gi = 0 and kfkkgk = 0 so the result reduces to 0 6 0. Otherwise, f 6= 0 and g 6= 0 so kfk= 6 0 and kgk= 6 0, and we may deﬁne α = 1/kfk and β = 1/kgk. Since

2 0 6 kαf ± βgk = hαf ± βg, αf ± βgi = α2kfk2 ± 2αβhf, gi + β2kgk2 = 21 ± αβhf, gi,

we see that −1 6 αβhf, gi 6 1 so

−1 −1 |hf, gi| 6 α β = kfkkgk. Triangle inequality

Corollary kf + gk 6 kfk + kgk. Proof.

kf + gk2 = hf + g, f + gi = kfk2 + 2hf, gi + kgk2 2 2 6 kfk + 2kfkkgk + kgk = (kfk + kgk)2. Self-adjoint diﬀerential operators

We now introduce a class of second-order, linear diﬀerential operators that possess a number of special properties. Operators of this type are of great importance in applications, as well as enjoying a rich theory that will be developed here and later in the course. Integration by parts

Consider a second-order,linear diﬀerential operator

00 0 Lu = a2u + a1u + a0u.

Integrating by parts,

b b 0 b 0 (a1u )v dx = u(a1v) a − u(a1v) dx, a a b Z Z b 00 0 0b 00 (a2u )v dx = u (a2v) − u(a2v) a + u(a2v) dx, Za Za so

0 0 b hLu, vi = u (a2v) − u(a2v) + u(a1v) a 00 0 + u, (a2v) − (a1v) + a0v . Adjoint operator and Lagrange identity

Thus, deﬁning the formal adjoint

∗ 00 0 L v = (a2v) − (a1v) + a0v 00 0 0 00 0 = a2v + (2a2 − a1)v + (a2 − a1 + a0)v

and the bilinear concomitant

0 0 P(u, v) = u (a2v) − u(a2v) + u(a1v),

we have the Lagrange identity

∗ b hLu, vi = hu, L vi + P(u, v) a. The diﬀerentiated version of the Lagrange identity is d (Lu)v = u L∗v + P(u, v). dx Example If Lu = 3xu00 − (cos x)u0 + exu then

L∗v = (3xv)00 + [(cos x)v]0 + exv = 3xv00 + (6 + cos x)v0 + (ex − sin x)v and

P(u, v) = u0(3xv) − u(3xv)0 − uv cos x = 3x(u0v − uv0) − (3 + cos x)uv. Higher order diﬀerential operators Consider an mth order, linear diﬀerential operator m j Lu = aj(x)D u. j=0 X Integrating by parts j times gives the identity

b b j j j aj(x)D u(x)v(x) dx = u(x)(−1) D [aj(x)v(x)] dx Za Za j k−1 j−k k−1 b + (−1) (D u)D (ajv) a, k=1 X so, summing over j, the Lagrange identity holds with m ∗ j j L v = (−1) D [aj(x)v], j=0 X m j k−1 j−k k−1 P(u, v) = (−1) (D u)D (ajv). j=1 k=1 X X Formal self-adjointness

The operator L is formally self-adjoint if L∗ = L.

Theorem A second-order, linear diﬀerential operator L is formally self-adjoint iﬀ it can be written in the form

Lu = −(pu0)0 + qu = −pu00 − p0u0 + qu, (20)

in which case the Lagrange identity takes the form

(Lu) v − u (Lv) = −p(x)(u0v − uv0)0, (21)

or in other words, the bilinear concomitant is

P(u, v) = −p(x)(u0v − uv0). Proof

If L = −(pu0)0 + qu = −pu00 − p0u0 + qu then

L∗v = (−pv)00 + (p0v)0 + qv = −(p00v + 2p0v0 + pv00) + (p00v + p0v0) + qv = −(pv00 + p0v0) + qv = −(pv0)0 + qv = Lv,

so L is formally self-adjoint. Conversely, if 00 0 ∗ Lu = a2u + a1u + a0u and L = L, then

00 0 a2u + a1u + a0u 00 0 0 00 0 = a2u + (2a2 − a1)u + (a2 − a1 + a0)u

for all u. Choosing u ≡ 1 and then u ≡ x, it follows that

00 0 0 a0 = a2 − a1 + a0 and a1 = 2a2 − a1. 0 Thus, a2 = a1, so by putting p = −a2 and q = a0 we have 0 a1 = −p , implying that L has the form

Lu = −pu00 − p0u0 + qu = −(pu0)0 + qu.

Moreover, in this case, the Lagrange identity takes the form

b hLu, vi = −(pu0)0v + quv dx a Z b b 0 b 0 0 = −(pu )v a + pu v dx + quv dx a a Z b Z b 0 0 b 0 0 = −(pu )v + u(pv ) a − u(pv ) + quv dx a a b Z Z 0 0 b 0 0 = −p(u v − uv ) a + u −(pv ) + qv dx Za 0 0 b = −p(u v − uv ) a + hu, Lvi. Example Consider the Bessel equation

x2u00 + xu0 + (x2 − ν2)u = f(x).

Dividing both sides by x gives Lu = −x−1f(x) where

Lu = −(xu0)0 + (ν2x−1 − x)u.

Example The Legendre equation

(1 − x2)u00 − 2xu0 + ν(ν + 1)u = f(x) has the form Lu = −f(x) with

Lu = −[(1 − x2)u0]0 − ν(ν + 1)u. Transforming to self-adjoint form If we can evaluate the integrating factor

a (x) p(x) = exp 1 dx , a (x) Z 2 00 0 then we can transform a2u + a1u + a0u = f(x) to self-adjoint form: pa pa −pf(x) −pu00 − 1 u0 − 0 u = , a2 a2 a2 pa −pf(x) −(pu00 + p0u0) − 0 u = , a2 a2 −(pu0)0 + qu = f˜(x)

where q = −pa0/a2 and f˜= −pf/a2. Example Write the Cauchy–Euler ODE ax2u00 + bxu0 + cu = f(x) in self-adjoint form. Self-adjointness and boundary operators

Lemma Any formally self-adjoint operator L = −(pu0)0 + qu satisﬁes the identity

2 hLu, vi − hu, Lvi = Biu Riv − Riu Biv , (22) i=1 X for all u and v, where p(a)u(a) p(a)u0(a) R1u = or R1u = − b11 b10 and p(b)u(b) p(b)u0(b) R2u = − or R2u = . b21 b20 Proof

Suppose b11 6= 0 and b21 6= 0. At x = a, 0 0 0 pv +p(u v − uv ) = b11u + b10u b11 pu 0 − b11v + b10v b11

= (B1u)(R1v) − (R1u)(B1v),

and at x = b, 0 0 0 pv −p(u v − uv ) = b21u + b20u − b21 pu 0 − − b21v + b20v b21

= (B2u)(R2v) − (R2u)(B2v).

A similar argument works if b11 = 0 or b21 = 0. Necessary condition for existence If u is a solution of the bvp (17), and if v is a solution of the homogeneous problem

Lv = 0 for a < x < b,

B1v = 0 at x = a, (23)

B2v = 0 at x = b,

then on the one hand

hLu, vi − hu, Lvi = hf, vi − hu, 0i = hf, vi

and on the other hand, by (22),

hLu, vi − hu, Lvi = α1 R1v − R1u × 0 + α2 R2v − R2u × 0.

Hence, the data f, α1 and α2 must satisfy

hf, vi = α1 R1v + α2 R2v. (24) Suﬃcient condition for existence?

It can be shown that, provided a2(x) 6= 0 for a 6 x 6 b, the condition (24) is also suﬃcient for the existence of u, i.e., u exists iﬀ (24) holds for all v satisfying (23). Example The 2-point BVP

u00 + u = f for 0 < x < π,

u = α1 at x = 0,

u = α2 at x = π,

has a solution iﬀ π f(x) sin(x) dx = α1 + α2. Z0 In this case, if u is one solution then every other solution is of the form u + C sin x for some constant C. Fredholm alternative

Theorem Either the homogeneous problem (18) has only the trivial solution v ≡ 0, in which case the inhomogeneous problem (17) has a unique solution u for every choice of f, α1 and α2, or else the homogeneous problem admits non-trivial solutions, in which case the inhomogeneous problem (17) has a solution u iﬀ f, α1 and α2 satisfy (24) for every solution v of the homogeneous problem (18).

In the latter case, u + Cv is also a solution of (17) for any constant C. Domain of a self-adjoint operator

Deﬁnition Let D be a subspace of a vector space V with inner product h·, ·i, and let L be a linear operator on V with domain D. We say that L is self-adjoint if

hLu, vi = hu, Lvi for all u, v ∈ D.

The identity (22) shows that L = −(pu0)0 + qu is self-adjoint if D 2 is the set of C functions u :[a, b] → R satisfying the homogeneous boundary conditions

B1u = 0 at x = a,

B2u = 0 at x = b. Example: Bessel operator

Recall the Bessel operator

0 0 2 −1 Lu = −(xu ) + (ν x − x)u for 1 6 x 6 2, and let 0 B1u = u ,R1u = u, at x = 1, 0 B2u = 2u − u, R2u = −u, at x = 2, 2 so that with hf, gi = 1 f(x)g(x) dx the Lagrange identity implies R hLu, vi − hu, Lvi = B1u R1v − R1u B1v + B2u R2v − R2u B2v.

Thus, if we let D denote the vector space of C2 functions u :[1, 2] → R satisfying B1u = 0 at x = 1 and B2u = 0 at x = 2, then hLu, vi = hu, Lvi for all u, v ∈ D. The vibrating string

Our ﬁrst example of an initial-boundary value problem for a partial diﬀerential equation is a classical model for a vibrating string, such as in a musical instrument like a piano. This example reveals a connection between Fourier series and acoustics. Notation and assumptions

Consider a stretched string having the following properties: 1. the string is fixed at x = 0 and x = `; 2. the string is uniform, perfectly elastic and offers no resistance to bending; 3. the weight of the string is negligible compared to tension; 4. the motion of the string is purely transverse and the deflection is small.

Let

u = u(x, t) = transverse deﬂection at position x and time t., T = T(x, t) = magnitude of tension, θ = θ(x, t) = tangent angle. Newton’s laws

Let ρ be the linear density (mass per unit length) of the string.

For the piece of string between x and x + ∆x,

longitudinal momentum = 0, ∂u transverse momentum = ρ∆x . ∂t No motion in the longitudinal direction so

T(x + ∆x, t) cos θ(x + ∆x, t) = T(x, t) cos θ(x, t), (25)

whereas for the transverse motion ∂ ∂u ρ∆x = T(x + ∆x, t) sin θ(x + ∆x, t) − T(x, t) sin θ(x, t). ∂t ∂t (26) Wave equation We deduce from (25) that

T cos θ = T0

for some T0 independent of x, and we deduce from (26) that

∂2u (T sin θ)(x + ∆x, t) − (T sin θ)(x, t) ρ = . ∂t2 ∆x Taking the limit as ∆x → 0 gives

∂2u ∂ ∂ ρ = (T sin θ) = T (tan θ). ∂t2 ∂x 0 ∂x But tan θ = ∂u/∂x so u satisﬁes the wave equation

∂2u ∂2u ρ = T . ∂t2 0 ∂x2 Initial-boundary value problem (IBVP)

p Put c = T0/ρ and suppose that the string is initially at rest with a known deﬂection u0(x), then

∂2u ∂2u − c2 = 0, 0 < x < `, t > 0, ∂t2 ∂x2

u(0, t) = 0, t > 0, u(`, t) = 0, t > 0, (27)

u(x, 0) = u0(x), 0 < x < `, ∂u (x, 0) = 0, 0 < x < `. ∂t Separation of variables

Look for solutions having a simple form:

u(x, t) = X(x)T(t).

Such a u satsiﬁes the PDE iﬀ

XT 00 − c2X00T = 0.

Thus, we want T 00 X00 = . c2T X Since T 00/(c2T) is a function of t only, and X00/X is a function of x only, there must be a separation constant λ such that

T 00 X00 = −λ = . c2T X A homogeneous BVP Since

u(0, t) = X(0)T(t), u(`, t) = X(`)T(t), ∂u (x, 0) = X(x)T 0(0), ∂t we also want

X(0) = 0, X(`) = 0, T 0(0) = 0.

Thus, X should satisfy

X00 + λX = 0 for 0 < x < `; X = 0 at x = 0; X = 0 at x = `.

One solution is X ≡ 0, but non-zero solutions exist for special choices of λ. Case λ > 0 Write λ = ω2 where ω > 0. Thus, X00 + ω2X = 0, which has the general solution

X = A cos ωx + B sin ωx.

Since X(0) = A the ﬁrst boundary condition implies A = 0 so X = B sin ωx. Since X(`) = B sin ω` the second boundary condition implies

B sin ω` = 0.

If B = 0, then X ≡ 0. If B 6= 0, then sin ω` = 0 so ω` = nπ for some n ∈ {1, 2, 3, . . . }. In this case, ω = nπ/` so

nπ2 nπx λ = and X = B sin . ` ` Case λ = 0

If λ = 0 then X00 = 0 which has the general solution

X = Ax + B.

Since X(0) = B, the ﬁrst boundary condition implies B = 0 so X = Ax.

Since X(`) = A`, the second boundary condition implies

A` = 0.

But ` > 0 so A = 0 and thus X ≡ 0. Case λ < 0

Write λ = −ω2 where ω > 0.

Thus, X00 − ω2X = 0, which has the general solution

X = A cosh ωx + B sinh ωx.

Since X(0) = A the ﬁrst boundary condition implies A = 0 so X = B sinh ωx.

Since X(`) = B sinh ω` the second boundary condition implies

B sinh ω` = 0.

If B = 0, then X ≡ 0.

If B 6= 0, then sinh ω` = 0 so ω` = 0, which is impossible if ω > 0. Non-trivial solutions

Conclusion: get a useful solution only when λ = λn and X = BXn, where

nπ2 nπ λ = and X (x) = sin x for n = 1, 2, 3,.... n ` n `

Recall that for T(t), we require T 00 + λc2T = 0 and T 0(0) = 0. 2 2 Since λnc = (nπc/`) , the general solution is

nπc nπc T(t) = C cos t + E sin t , ` `

and the initial condition implies that E = 0, so T = CTn where nπc T (t) = cos t . n ` Modes We call nπ nπc u (x, t) = X (x)T (t) = sin x cos t n n n ` `

the nth normal mode of vibration or pure harmonic. The smallest number τn > 0 such that un(x, t + τn) = un(x, t) for all x and t is called the period of the nth normal mode. Since nπc τ = 2π ` n we see that 2` τ = . n nc The corresponding frequency is 1 nc = . τn 2` Summary

Each mode un satisﬁes all four homogeneous equations in (27),

∂2u ∂2u n − c2 n = 0, ∂t2 ∂x2 u (0, t) = 0, n (28) un(`, t) = 0, ∂u n (x, 0) = 0, ∂t but since nπ u (x, 0) = sin x (29) n ` the inhomogeneous initial condition will not be satisﬁed unless u0(x) happens to be sin(nπx/`). Superposition principle Each equation in (28) is linear and homogeneous, so any superposition of normal modes,

nπ nπc u(x, t) = An sin x cos t , (30) ∞ ` ` n=1 X also satisﬁes (28). We can now hope that (30) is the general solution of the initial-boundary value problem (27): for any given u0 we can ﬁnd A1, A2, . . . such that the remaining condition holds, i.e.,

u(x, 0) = u0(x), 0 6 x 6 `. But this just means that

nπ u0(x) = An sin x , 0 6 x 6 `. (31) ∞ ` n=1 X Fourier sine coeﬃcients

Thus, the solution to (27) is given by (30) with

2 ` nπx A = u (x) sin dx. (32) n ` 0 ` Z0 Example If ` = 1 and the initial deﬂection is given by

1 2x, 0 < x < 2 , u0(x) = 1 2(1 − x), 2 < x < 1, we ﬁnd that

k−1 8 (−1) u(x, t) = 2 2 sin (2k − 1)πx cos (2k − 1)πct . π ∞ (2k − 1) k=1 X Partial sums Heat equation

Our second example of an initial-boundary value problem is the application that led, in 1807, to Fourier’s original paper that introduced the idea of expanding an arbitrary function in a trigonometric series. Jean-Baptiste Joseph Fourier 1768–1830 Heat conduction in 1D

Let u(x, t) = temperature at position x and time t. Here, we assume that the temperature does not vary in the y and z directions, and that the physical parameters

K = thermal conductivity, ρ = mass density, σ = speciﬁc heat,

are all constant (and positive).

Fourier’s law of heat conduction asserts that the heat ﬂux q in the x-direction is given by ∂u q = −K ∂x Conservation law for thermal energy

In a (ﬁxed) region x1 < x < x2, the thermal energy per unit of cross sectional area equals

x2 ρσu dx. Zx1

The heat ﬂux entering the region at x = x1 equals q(x1), and the heat ﬂux entering the region at x = x2 equals −q(x2).

The rate of change of thermal energy must equal the total heat ﬂux entering the region, so

d x2 ρσu dx = q(x ) − q(x ), dt 1 2 Zx1 x2 ∂u x2 ∂q ρσ dx = − dx. ∂t ∂x Zx1 Zx1 Heat equation

x2 ∂u ∂q ρσ + dx = 0. ∂t ∂x Zx1 By Fourier’s law, q = −K∂u/∂x, so

1 x2 ∂u ∂2u ρσ − K dx = 0. x − x ∂t ∂x2 2 1 Zx1 The LHS equals the average value of the integrand over the interval (x1, x2), which tends to the pointwise value at x = x1 if we let x2 → x1. Since the choice of x1 is arbitrary, we conclude that u must satisfy the heat equation, ∂u ∂2u ρσ − K = 0, ∂t ∂x2 or equivalently, K u − au = 0, where a = . t xx ρσ Part IV

Generalised Fourier Series Introduction

We have seen how a solution to the vibrating string problem can be constructed using the Fourier sine series expansion of the initial data. In this part of the course, we will study abstract Fourier expansions that include the familiar trigonometric Fourier series as just one special case. The connection with diﬀerential equations comes about via the theory of Sturm–Liouville eigenproblems.

These generalised Fourier series will allow us to solve partial diﬀerential equations by separating variables in curvilinear coordinates. Outline

Complete orthogonal systems

Sturm–Liouville problems

Fourier–Bessel series

Schr¨odingerequation Complete orthogonal systems

We now introduce a more general class of inner product, and investigate properties of orthogonal functions. Expanding a function as a linear combination of orthogonal function leads naturally to the notion of a generalised Fourier series. Weighted inner product

If w :(a, b) → R satisﬁes

w(x) > 0 for a < x < b,

then we deﬁne the inner product with weight function w by

b hf, giw = hf, gwi = f(x)g(x)w(x) dx, Za and the corresponding norm by

s b p 2 kfkw = hf, fiw = [f(x)] w(x) dx. Za Two functions f and g are orthogonal with respect to w over the interval (a, b) if hf, giw = 0. Example: Tchebyshev polynomials

The Tchebyshev polynomials T0, T1, T2, . . . are deﬁned recursively by

T0(x) = 1, T1(x) = x,

Tn+1(x) = 2xTn(x) − Tn−1(x) for n = 1, 2, 3,....

One can verify by induction on n that

Tn(cos θ) = cos(nθ).

Hence, the substitution x = cos θ shows that if n 6= j then

1 T (x)T (x) n√ j dx = 0 1 − x2 Z−1 and so Tn√is orthogonal to Tj with respect to the Tchebyshev weight 1/ 1 − x2. The space L2

Let L2(a, b, w) denote the set of all (Lebesgue measurable) functions f :(a, b) → R such that kfkw < .

∞ The Cauchy–Schwarz inequality states that if f, g ∈ L2(a, b, w) then hf, giw exists and is ﬁnite, with

|hf, giw| 6 kfkwkgkw.

Furthermore, L2(a, b, w) is a vector space with

kλfkw = |λ|kfkw, kf + gkw 6 kfkw + kgkw. Orthogonal set of functions

A set of functions S ⊆ L2(a, b, w) is said to be orthogonal if every pair of functions in S is orthogonal and if no function is identically zero on (a, b).

We say that S is orthonormal if, in addition, each function has norm 1.

Lemma If S is orthogonal then S is linearly independent. Proof

Suppose that φ1, φ2,..., φN ∈ S satisfy

N Cjφj(x) = 0 for a 6 x 6 b. j=1 On the one hand,X N φk, Cjφj = hφk, 0iw = 0 for 1 6 k 6 N, j=1 w X

and on the other hand, because φk is orthogonal to φj for j 6= k,

N N φk, Cjφj = Cjhφk, φjiw = Ckhφk, φkiw, j=1 w j=1 X X 2 so Ckhφk, φkiw = 0. Since hφk, φkiw = kφkkw > 0, it follows that Ck = 0 for 1 6 k 6 N. Hence, S is linearly independent. Generalised Pythagorus theorem

Lemma If {φ1, . . . , φN} is orthogonal then, for any C1,..., CN ∈ R,

N 2 N 2 2 Cjφj = C kφjk . j w j=1 w j=1 X X Proof.

N N N N LHS = Cjφj, Ckφk = CjCkhφj, φkiw j=1 k=1 w j=1 k=1 X X X X N 2 = + = Cj hφj, φjiw + 0 = RHS. j=k j6=k j=1 X X X Generalised Fourier coeﬃcients

Lemma If f is in the span of an orthogonal set of functions {φ1, φ2, . . . , φN} in L2(a, b, w), then the coeﬃcients in the representation N f(x) = Ajφj(x) j=1 X are given by hf, φjiw Aj = 2 for 1 6 j 6 N. (33) kφjkw

We call Aj the jth Fourier coeﬃcient of f with respect to the given orthogonal set of functions. Proof

Since N f(x) = Ajφj(x), j=1 X if 1 6 k 6 N then N N hf, φkiw = Ajφj, φk = Ajhφj, φkiw j=1 w j=1 X X 2 = Akhφk, φkiw + = Akkφkkw + 0, j6=k X so hf, φkiw Ak = 2 . kφkkw Classical Fourier sine coeﬃcients The sequence of functions nπx φ (x) = sin , n = 1, 2, 3, . . . , n ` is orthogonal with respect to the weight w(x) = 1 over the interval (a, b) = (0, `), that is,

` nπx jπx hφ , φ i = sin sin dx = 0 if n 6= j. n j ` ` Z0 Since ` nπx ` kφ k2 = sin2 dx = , n ` 2 Z0 the Fourier (sine) coeﬃcients of a function f(x) are

hf, φ i 2 ` jπx A = j = f(x) sin dx. j kφ k2 ` ` j Z0 Least-squares approximation

Consider approximating a function f ∈ L2(a, b, w) by a function in the span of an orthogonal set {φ1, φ2, . . . , φN}, that is, ﬁnding coeﬃcients Cj such that

N f(x) ≈ Cjφj(x) for a 6 x 6 b. j=1 X

We seek to choose the Cj so that the weighted mean-square error

N 2 b N 2

f − Cjφj = f(x) − Cjφj(x) w(x) dx

j=1 w a j=1 X Z X is as small as possible. Least-squares approximation

Lemma For all C1, C2,..., CN, the weighted mean-square error satisﬁes

N 2 N 2 2 2 f − Cjφj = kfk − A kφjk w j w j=1 w j=1 X X N 2 2 + (Cj − Aj) kφjkw, j=1 X

and so achieves its unique minimum when Cj equals the jth Fourier coeﬃcient of f, that is, when

hf, φjiw Cj = Aj = 2 for 1 6 j 6 N. kφjkw Proof

N 2 N N

f − Cjφj = f − Cjφj, f − Ckφk

j=1 w j=1 k=1 w X X X N N = hf, fiw − f, Ckφk − Cjφj, f k=1 w j=1 w X X N N + Cjφj, Ckφk j=1 k=1 w X X N N 2 = kfkw − Ckhf, φkiw − Cjhφj, fiw k=1 j=1 X X N N + CjCkhφj, φkiw. j=1 k=1 X X Since

N N N 2 2 CjCkhφj, φkiw = + = Cj kφjkw + 0 j=1 k=1 j=k j6=k j=1 X X X X X 2 and hf, φjiw = Ajkφjkw, we can complete the square as follows:

N 2 N N 2 2 2 f − Cjφj = kfk − 2 Cjhf, φjiw + C kφjk w j w j=1 w j=1 j=1 X X X N N 2 2 2 2 = kfkw − 2 CjAjkφjkw + Cj kφjkw j=1 j=1 X X N 2 2 2 2 2 = kfk2 + Cj − 2CjAj + Aj − Aj kφjkw j=1 X N N 2 2 2 2 2 = kfk2 − Aj kφjkw + Cj − Aj kφjkw. j=1 j=1 X X Conclusion: Let f ∈ L2(a, b, w) and put

hf, φjiw Aj = 2 . kφjkw

If f belongs to the span of S = {φ1, . . . , φN}, then

N f(x) = Ajφj(x), a 6 x 6 b, j=1 X but if f does not belong to the span of S then

N f(x) ≈ Ajφj(x), a 6 x 6 b, j=1 X is the best approximation (in the weighted L2-norm) to f(x) by a function in the span of S. Example of least-squares approximation Deﬁne f :[0, 1] → R by

1, 0 < x < 1/2, f(x) = 0, 1/2 < x < 1,

and let φn(x) = sin nπx. Then

hf, φ i 1 A = j = 2 f(x) sin jπx dx j kφ k2 j Z0 1/2 2B = 2 sin jπx dx = j , jπ Z0 where

0, j ≡ 0 (mod 4), jπ B = 1 − cos = 1, j ≡ 1 or 3 (mod 4), j 2  2, j ≡ 2 (mod 4),

 4 2 1 f(x) ≈ A φ (x) = sin πx + sin 2πx + sin 3πx . j j π 3 j=1 X Convergence of least-squares approximation

N f(x) ≈ Ajφj(x) j=1 X Gibbs phenomenon

100 f(x) ≈ Ajφj(x) j=1 X Legendre polynomials

Recall that

1 2 P0(z) = 1, P1(z) = z, P2(z) = 2 (3z − 1), 1 3 P3(z) = 2 (5z − 3z). We will see later that these functions are orthgonal with respect to

1 hf, gi = f(x)g(x) dx, Z−1 that is, hPn,Pji = 0 if n 6= j. It can also be shown that

1 2 kP k2 = P (x)2 dx = . n n 2n + 1 Z−1 Example of polynomial least-squares approximation Let 1, 0 < x < 1/2, f(x) = 0, −1 < x < 0 or 1/2 < x < 1. We ﬁnd that j 0 1 2 3 4

hf, Pji 1/2 1/8 −3/16 −19/128 15/256 2 kPjk 2 2/3 2/5 2/7 2/9

Aj 1/4 3/16 −15/32 −133/256 135/512 so

4 4 hf, Pji f(x) ≈ AjPj(x) = 2 Pj(x) kPjk j=0 j=0 X X 1 3 15 133 135 = P (x) + P (x) − P (x) − P (x) + P (x). 4 0 16 1 32 2 256 3 512 4

Orthogonal sequences of functions

Now let S = {φ1, φ2, φ3,...} be a countably inﬁnite orthogonal set in L2(a, b, w). Given any f ∈ L2(a, b, w) we write

f(x) ∼ Ajφj(x) ∞ j=1 X

to indicate that Aj is the jth Fourier coeﬃcient of f, i.e.,

hf, φjiw Aj = 2 for all j > 1. kφjkw Example

Fourier sine series: for f ∈ L2(0, π),

2 π f(x) ∼ Aj sin jx, Aj = f(x) sin jx dx. ∞ π j=1 0 X Z

Example

Fourier–Legendre series: for f ∈ L2(−1, 1),

2j + 1 1 f(x) ∼ AjPj(x),Aj = f(x)Pj(x) dx. ∞ 2 j=0 −1 X Z Example Fourier–Tchebyshev series: recall that

3 T0(x) = 1, T1(x) = x, T2(x) = 2x−1, T3(x) = 4x −3x, . . . are orthogonal√ over (−1, 1) with respect to the weigth function w(x) = 1/ 1 − x2. The substitution x = cos θ implies that

1 2 2 Tj(x) dx π, j = 0, kTjk = √ = w 1 − x2 π/2, j 1, Z−1 > so for f ∈ L2(−1, 1, w),

A hf, T i 2 1 f(x)T (x) dx 0 j √ j f(x) ∼ + AjTj(x),Aj = 2 = . 2 ∞ kTjk π 2 j=1 w −1 1 − x X Z Example Trigonometric Fourier series: the functions

1, cos x, sin x, cos 2x, sin 2x, cos 3x, sin 3x, . . . are orthogonal over (−π, π) with

π π π 12 dx = 2π and cos2 jx dx = π = sin2 jx dx, Z−π Z−π Z−π so if f ∈ L2(−π, π) then

A0 f(x) ∼ + Aj cos jx + Bj sin jx 2 ∞ j=1 X where 1 π 1 π A = f(x) cos jx dx and B = f(x) sin jx dx. j π j π Z−π Z−π Bessel inequality

For every N,

N 2 N 2 2 2 0 f − Ajφj = kfk − A kφjk 6 w j w j=1 w j=1 X X so N 2 2 2 Aj kφjkw 6 kfkw, j=1 X and therefore the Fourier coeﬃcients satisfy Bessel’s inequality

2 2 2 Aj kφjkw 6 kfkw. ∞ j=1 X Example

The Fourier sine coeﬃcients of f ∈ L2(0, π) satisfy

π 2 2 2 Aj 6 f(x) dx. ∞ π j=1 0 X Z

Example

The Fourier–Legendre coeﬃcients of f ∈ L2(−1, 1) satisfy

A2 1 j 1 2 6 f(x) dx. ∞ 2j + 1 2 j=0 −1 X Z Example √ For w(x) = 1/ 1 − x2, the Fourier–Tchebyshev coeﬃcients of f ∈ L2(−1, 1, w) satisfy

2 1 2 A0 2 1 f(x) dx + Aj 6 √ . 2 ∞ π 2 j=1 −1 1 − x X Z

Example

The trigonometric Fourier coeﬃcients of f ∈ L2(−π, π) satisfy

2 π A0 2 2 1 2 + Aj + Bj 6 f(x) dx. 2 ∞ π j=1 −π X Z Completeness

An orthogonal set S is complete if there is no non-trivial function in L2(a, b, w) orthogonal to every function in S, i.e., if the condition hf, φiw = 0 for every φ ∈ S implies that kfkw = 0.

In particular, if S is a complete orthogonal set, then every proper subset of S fails to be complete.

Example The set S = { sin jx : j > 1 and j 6= 7 } is not complete in L2(0, π) because sin 7x is orthogonal to every function in S. Equivalent deﬁnitions of completeness

Theorem If S = {φ1, φ2,...} is orthogonal in L2(a, b, w), then the following properties are equivalent: 1. S is complete;

2. for each f ∈ L2(a, b, w), if Aj denotes the jth Fourier coeﬃcient of f then N

f − Ajφj → 0 as N → ;

j=1 w X ∞ 3. each function f ∈ L2(a, b, w) satisﬁes Parseval’s identity:

2 2 2 kfkw = Aj kφjkw. ∞ j=1 X Proof that 2 ⇐⇒ 3

We saw already that

N 2 N 2 2 2 f − Ajφj = kfk − A kφjk , w j w j=1 w j=1 X X so LHS → 0 iﬀ RHS → 0. Now observe that RHS → 0 means the same thing as

2 2 2 kfkw = Aj kφjkw. ∞ j=1 X Proof that 3 =⇒ 1

Assume that Parseval’s identity holds for every f ∈ L2(a, b, w). If

hf, φjiw = 0 for all j > 1,

then Aj = 0 for all j > 1, so

2 2 2 kfkw = Aj kφjkw = 0, ∞ j=1 X showing that f = 0. Therefore S is complete. Proof that 1 =⇒ 2 Assume that S is complete. Given f ∈ L2(a, b, w), Bessel’s inequality implies that the Fourier series of f converges in L2(a, b, w), so we can deﬁne

g = f − Akφk. ∞ k=1 X Since

hg, φjiw = hf, φjiw − Akhφj, φkiw ∞ k=1 X 2 = hf, φjiw − Ajkφjkw = 0

for every j > 1, we conclude that g = 0 and therefore 2 holds:

f = Ajφj. ∞ j=1 X Least-squares error Consider N f(x) ≈ Ajφj(x) for a < x < b, j=1 X 2 where Aj = hf, φjiw/kφjkw is the jth Fourier coeﬃcient of f. We have seen that this choice of Aj minimises the norm in L2(a, b, w) of the error N eN(x) = f(x) − Ajφj(x). j=1 X Corollary

If S = {φ1, φ2, φ3,...} is a complete orthogonal sequence in L2(a, b, w), then for any f ∈ L2(a, b, w),

2 2 keNk = Ajkφjkw. ∞ j=N+1 X Proof We saw in the proof of the theorem (2 ⇐⇒ 3) that

N 2 2 2 2 keNkw = kfkw − Aj kφjkw. j=1 X Since Parseval’s identity holds,

2 2 2 kfkw = Aj kφjkw, ∞ j=1 X and therefore

N 2 2 2 2 2 keNkw = Aj kφjkw − Aj kφjkw ∞ j=1 j=1 X X 2 2 = Aj kφjkw. ∞ j=N+1 X Parseval identity

Each of our four examples of orthgonal sequences is in fact complete. Example

The Fourier sine coeﬃcients of f ∈ L2(0, π) satisfy

π 2 2 2 Aj = f(x) dx. ∞ π j=1 0 X Z

Example

The Fourier–Legendre coeﬃcients of f ∈ L2(−1, 1) satisfy

A2 1 1 j = f(x)2 dx. ∞ 2j + 1 2 j=0 −1 X Z Example √ For w(x) = 1/ 1 − x2, the Fourier–Tchebyshev coeﬃcients of f ∈ L2(−1, 1, w) satisfy

2 1 2 A0 2 1 f(x) dx + Aj = √ . 2 ∞ π 2 j=1 −1 1 − x X Z

Example

The trigonometric Fourier coeﬃcients of f ∈ L2(−π, π) satisfy

2 π A0 2 2 1 2 + Aj + Bj = f(x) dx. 2 ∞ π j=1 −π X Z Estimating the least-squares error Consider f(x) = x for 0 < x < π. This function has the Fourier sine series

(−1)n+1 f(x) ∼ 2 sin nx ∞ n n=1 X so N (−1)n+1 f(x) = 2 sin nx + e (x), n N n=1 X and the error term eN(x) satisﬁes

π n+1 2 2 2 2(−1) π eN(x) dx = keNk = . ∞ n 2 Z0 n=N+1 X 2 2 kφnk An | {z } |{z} Thus,

2 1 keNk = 2π 2 ∞ n n=N+1 X dt 2π 2π = , 6 t2 N ZN∞ and so −1/2 keNk = O(N ). In other words, the root mean square error for the approximation

sin 2x sin Nx x ≈ 2 sin x − + ··· + (−1)N+1 , 0 < x < π, 2 N tends to zero like N−1/2. Sturm–Liouville problems

Recall that the eigenvectors of a real N × N symmetric matrix N form a orthogonal basis for R . In the same way, we will see that a type of homogeneous boundary-value problem generates a sequence of functions that are orthgonal with respect to an associated weight function.

The sequence of orthogonal functions turns out to be complete, but the proof of this fact relies on results from functional analysis (Math5605) about a compact linear operator on a Hilbert space. Sturm–Liouville operator

An ODE of the form

[p(x)u0]0 + [λr(x) − q(x)]u = 0, a < x < b, (34)

is called a Sturm–Liouville equation. The coeﬃcients p, q, r must all be real-valued with

p(x) > 0 and r(x) > 0 for a < x < b.

Deﬁning the formally self-adjoint diﬀerential operator

Lu = −[p(x)u0]0 + q(x)u, (35)

we can write (34) as

Lu = λru on (a, b). Eigenfunctions and eigenvalues Any non-trivial (possibly complex-valued) solution u satisfying Lu = λru on (a, b) (plus appropriate boundary conditions) is said to be an eigenfunction of L with eigenvalue λ. In this case, we refer to (φ, λ) as an eigenpair.

Example Legendre’s equation

(1 − x2)u00 − 2xu0 + ν(ν + 1)u = 0

is equivalent to

[(1 − x2)u0]0 + ν(ν + 1)u = 0

which is of the form (34) with

p(x) = 1 − x2, q(x) = 0, r(x) = 1, λ = ν(ν + 1). Boundary conditions

Assume as before that p, q, r are real-valued with p(x) > 0 and r(x) > 0 for a < x < b.A regular Sturm–Liouville eigenproblem is of the form Lu = λru for a < x < b, 0 B1u = b11u + b10u = 0 at x = a, (36) 0 B2u = b21u + b20u = 0 at x = b,

where a and b are ﬁnite with

p(a) 6= 0 and p(b) 6= 0,

and where b10, b11, b20, b21 are real with

|b10| + |b11| 6= 0 and |b20| + |b21| 6= 0. Eigenfunctions are orthogonal We now describe how a Sturm–Liouville problem leads to an associated complete orthogonal system, and hence to a generalised Fourier expansion. Theorem Let L be a Sturm–Liouville diﬀerential operator (35). If u, v :[a, b] → C satisfy

Lu = λru on (a, b), with B1u = 0 = B2u,

and Lv = µrv on (a, b), with B1v = 0 = B2v, and if λ 6= µ, then u and v are orthogonal on the interval (a, b) with respect to the weight function r(x), i.e.,

b hu, vir = u(x)v(x)r(x) dx = 0. Za Proof

Using the Lagrange identity,

b (λ − µ)hu, vir = (λ − µ) u(x)v(x)r(x) dx a b Z b = λr(x)u(x)v(x) dx − u(x)µr(x)v(x) dx a a Z b b Z = Lu(x) v(x) dx − u(x) Lv(x) dx Za Za 2 = BjuRjv − RjuBjv = 0, j=1 X

since Bju = 0 = Bjv for j ∈ {1, 2}. By hypothesis, λ 6= µ so λ − µ 6= 0, and we conclude that hu, vir = 0. Eigenvalues are real

Theorem Let L be a Sturm–Liouville diﬀerential operator (35). If u :[a, b] → C is not identically zero and satisﬁes

Lu = λru on (a, b), with B1u = 0 = B2u,

then λ is real.

In the same way, we can prove that for a real, symmetric (or complex, Hermitian) matrix:

I eigenvectors with distinct eigenvalues are orthogonal;

I every eigenvalue is real. Proof

Since p(x), q(x), r(x) are each real,

Lu¯ = −(pu¯ 0)0 + qu¯ = −(pu0)0 + qu = Lu = λru = ¯λru,¯

so u¯ is an eigenfunction with eigenvalue ¯λ. Thus, if λ 6= ¯λ then

b b 2 0 = hu, u¯ir = u(x)u¯(x)r(x) dx = |u| r dx, Za Za implying u ≡ 0, a contradiction. We conclude λ = ¯λ, that is, λ is real. Completeness of the eigenfunctions

Theorem The regular Sturm–Liouville problem (36) has an inﬁnite sequence of eigenfunctions φ1, φ2, φ3, . . . with corresponding eigenvalues λ1, λ2, λ3, . . . and moreover:

1. the eigenfunctions φ1, φ2, φ3, . . . form a complete orthogonal system on the interval (a, b) with respect to the weight function r(x);

2. the eigenvalues satisfy λ1 < λ2 < λ3 < ··· with λj → as j → . ∞ ∞ In the same way, for every real symmetric (or complex, Hermitian) n n × n matrix A there is an orthgonal basis for R consisting of eigenvectors of A. Real trigonometric Fourier series The various trigonometric Fourier series arise from the the Sturm–Liouville ODE u00 + λu = 0. For the interval [−`, `], periodic boundary conditions

u(`) = u(−`) and u0(`) = u0(−`)

lead to the eigenfunctions jπx φ (x) = cos for j = 0, 1, 2, . . . , 2j ` jπx φ (x) = sin for j = 1, 2, . . . , 2j−1 ` and (non-distinct!) eigenvalues

jπ2 λ = 0, λ = λ = , j 1. 0 2j 2j−1 ` > Complex trigonometric Fourier series Again taking

u00 + λu = 0, u(`) = u(−`), u0(`) = u0(−`),

we can instead take

jπ2 φ (x) = eiπjx/` and λ = for j = 0, ±1, ±2,.... j j `

Here, the φj are orthogonal with respect to the complex inner product ` hf, gi = f(x)g(x) dx, Z−` so

` ijπx/` 1 −ijπx/` f(x) ∼ Aje ,Aj = f(x)e dx. ∞ 2` j=− −` X Z ∞ Half-range trigonometric Fourier series Consider u00 + λu = 0 for 0 < x < `. Homogeneous Dirichlet boundary conditions

u(0) = u(`) = 0

lead to jπx jπ2 φ (x) = sin and λ = for j = 1, 2, 3, . . . , j ` j ` whereas homogeneous Neumann boundary conditions

u0(0) = u0(`) = 0

lead to jπx jπ2 φ (x) = cos and λ = for j = 0, 1, 2, . . . . j ` j ` Least-squares error for cosine expansion

Consider once again the function

f(x) = x for 0 < x < π.

This time, instead of the sine series use the cosine series

π 2 f(x) ∼ + 2 (cos nπ − 1) cos nx 2 ∞ πn n=1 X π 4 cos(2k − 1)x ∼ − 2 , 2 π ∞ (2k − 1) k=1 X and estimate the error term eN(x) where

π N 2 f(x) = + (cos nπ − 1) cos nx + e (x). 2 πn2 N n=1 X With N = 2K − 1, 2 2 2 π keNk = 2 (cos nπ − 1) ∞ πn 2 n=N+1 X 2 2 kφnk An | 4 {zπ 1 } |{z}1 = 2 4 = 1 ∞ π (2k − 1) 2 8π ∞ (k − )4 k=K k=K 2 X X 1 dt 1 1 2 1 6 = = . 8π (t − 1 )4 24π (K − 1 )3 3π N3 ZK∞ 2 2 Thus, for this choice of f, the root mean square error using the cosine expansion is

−3/2 keNk = O(N ), versus O(N−1/2) for the sine expansion. Fourier–Bessel series

In this section, we study a singular Sturm–Liouville problem that will arise when we deal with eigenproblems for partial differential equations. The proof that the eigenfunctions are orthogonal needs to be modified because of the different type of boundary condition imposed at x = 0. A singular Sturm–Liouville problem

Consider the parameterised Bessel equation of order ν > 0, x2u00 + xu0 + (k2x2 − ν2)u = 0,

which can be written in Sturm–Liouville form as

(xu0)0 + k2x − ν2x−1u = 0,

i.e.,

ν2 p(x) = x, r(x) = x, λ = k2, q(x) = . x On the interval [0, `], we do not have a regular Sturm–Liouville problem because p(0) = 0 and q(x) is unbounded as x → 0+. A condition ensuring λ > 0

Lemma Consider the ODE

(xu0)0 + (λx − ν2x−1)u = 0 for 0 < x < `, (37)

with boundary conditions

u(x) bounded and xu0(x) → 0 as x → 0+, 0 (38) c1u + c0u = 0 at x = `,

and suppose that c0c1 > 0. 1. If a non-trivial solution u exists, then λ > 0. 2. A non-trivial solution u exists for λ = 0 only when ν = 0 and c0 = 0, in which case u is a constant. Proof

Multiply (37) by u and integrate to obtain

` (xu0)0u + (λx − ν2x−1)u2 dx = 0. Z0 Integration by parts gives

` ` (xu0)0u dx = `u0(`)u(`) − x(u0)2 dx, Z0 Z0 since the ﬁrst boundary condition ensures xu0(x)u(x) → 0 as x → 0, so

` ` ` λ xu2 dx = x(u0)2 dx + ν2 x−1u2 dx − `u0(`)u(`). Z0 Z0 Z0 The boundary condition at x = ` gives

0 0 0 u (`) u (`) 0 c1 0 2 −u (`)u(`) = −c0u(`) = c1u (`) = u (`) , c0 c0 c0 if c0 6= 0, and

0 0 u(`) u(`) c0 2 −u (`)u(`) = −c1u (`) = c0u(`) = u(`) , c1 c1 c1

0 if c1 6= 0. In either case, −u (`)u(`) > 0 because c0c1 > 0. Hence, λ > 0, proving part 1. If λ = 0 then u satisﬁes

` ` 0 2 2 −1 2 0 x(u ) dx + ν x u dx = `u (`)u(`) 6 0. Z0 Z0 For a non-trivial u, it follows that u0 ≡ 0 and ν2 = 0, so u is constant and ν = 0, proving part 2. Bessel functions

Theorem Assume that c0c1 > 0. If c0 6= 0 or ν > 0, then the eigenvalues λ1, λ2, . . . and eigenfunctions φ1, φ2, . . . of the Bessel equation (37) with boundary conditions (38) are given by

2 λj = kj and φj(x) = Jν(kjx) for j > 1,

where kj is the jth positive solution of the equation

0 c0Jν(k`) + c1kJν(k`) = 0. (39)

0 If c0 = 0 = ν, so that u (`) = 0 by (38), then we have an additional eigenvalue and eigenfunction

λ0 = 0 and φ0(x) = 1. Proof

First suppose that λ = k2 > 0. Then (37) is equivalent to the parameterised Bessel equation

x2u00 + xu0 + (k2x2 − ν2)u = 0

which has the general solution u = AJν(kx) + BYν(kx). Since Yν(kx) is unbounded as x → 0, we must have B = 0, so 0 0 u = AJν(kx) and u = AkJν(kx). Thus, at x = `,

0 0 c1u + c0u = A c1kJν(k`) + c0Jν(k`) ,

showing that the boundary condition holds iﬀ k is a solution of (39), in which case Jν(kx) is an eigenfunction.

When λ = 0, we already know that c0 = 0 = ν and u = A for some constant A. Orthogonality and normalization Theorem Assume that c0c1 > 0. The Bessel eigenfunctions, satisfying (37) and (38), have the orthogonality property

` φi(x)φj(x)x dx = 0 if i 6= j. Z0 If c1 6= 0, then

` 1 `c 2 φ (x)2x dx = 0 + (k `)2 − ν2 J (k `)2 for j 1, j 2k2 c j ν j > Z0 j 1

whereas if c1 = 0, so that φj(`) = 0 by (38), then

` `2 φ (x)2x dx = J (k `)2 for j 1. j 2 ν+1 j > Z0 ` 2 2 In the case c0 = 0 = ν, we have 0 φ0(x) x dx = ` /2. R Proof of orthogonality

0 0 2 −1 Put Lu = −(xu ) + ν x u so that Lφj = xλjφj. It suﬃces to show that hLu, vi = hu, Lvi for all u, v ∈ D, where D is the space of C2 functions satisfying the boundary conditions (38). In fact, if u, v ∈ D then x(u0v − uv0) → 0 as x → 0 so the Lagrange identity gives

hLu, vi − hu, Lvi = −`u0(`)v(`) − u(`)v0(`)

and it suﬃces to prove that u0v − uv0 = 0 at x = `. Using the 0 0 boundary conditions c1u + c0u = 0 = c1v + c0v, we have 0 0 0 0 0 0 0 0 c0u v = u (c0v) = u (−c1v ) = (−c1u )v = (c0u)v = c0uv 0 0 0 and c1u v = (−c0u)v = u(−c0v) = u(c1v ) = c1uv , so 0 0 cj(u v − uv ) = 0 for j ∈ {0, 1}. At least one of c0 and c1 is not zero, and thus u0v − uv0 = 0 at x = `. Fourier–Bessel expansion

For instance, when c1 = 0 we have

f(x) ∼ AjJν(kjx), 0 < x < `, ∞ j=1 X with ` ` 2 Aj = f(x)Jν(kjx)x dx Jν(kjx) x dx Z0 Z0 2 ` = f(x)J (k x)x dx. `2J (k `)2 ν j ν+1 j Z0 Plots when ` = 1, ν = 0, c1 = 0 Partial sums Another singular Sturm–Liouville problem Consider the Legendre equation with parameter ν,

(1 − x2)u00 − 2xu0 + ν(ν + 1)u = 0 for −1 < x < 1,

which can be written in Sturm–Liouville form as

[(1 − x2)u0]0 + ν(ν + 1)u = 0,

i.e.,

p(x) = 1 − x2, r(x) = 1, λ = ν(ν + 1), q(x) = 0.

On the interval [−1, 1] we do not have a regular Sturm–Liouville problem because p(−1) = 0 = p(1). 1 We assume without loss of generality that ν > − 2 because ν(ν + 1) = µ(µ + 1) if µ = −ν − 1. Eigenvalues are positive

Consider the ODE

[(1 − x2)u0]0 + λu = 0 (40)

with boundary conditions

u(x) bounded and (1 ∓ x)u0(x) → 0 as x → ±1. (41)

Muliplying the ODE by u and integrating (by parts) we ﬁnd

1 1 λ u2 dx = (1 − x2)(u0)2 dx, Z−1 Z−1 so if a non-trivial solution u exists then λ > 0. Legendre polynomials are orthogonal

If ν is not an integer then the boundary-value problem (40)–(41) has only the trivial solution u ≡ 0 (see the tutorial problems), whereas if ν = n is a non-negative integer, then all solutions have the form u = CPn(x),

where Pn is the Legendre polynomial of degree n, so

φn(x) = Pn(x) and λn = n(n + 1)

for n = 0, 1, 2, 3, . . . . Theorem The Legendre polynomials are orthogonal over the interval [−1, 1],

1 hPn,Pmi = Pn(x)Pm(x) dx = 0 if m 6= n, Z−1 Proof

Since r(x) = 1, it suﬃces to show that the Legendre diﬀerential operator Lu = −[(1 − x2)u0]0 is self-adjoint,

hLu, vi = hu, Lvi for all u, v ∈ D,

where D is the space of C2 functions on (−1, 1) that satisfy the boundary conditions (41).

In fact, the Lagrange identity shows that

2 0 0 1 hLu, vi − hu, Lvi = −(1 − x )(u v − uv ) −1, which is zero by (41). Plots Schr¨odingerequation

To conclude this part of the course, we use separation of variables to solve a partial diﬀerential equation that arises in quantum theory, and has a similar structure to the heat equation but with the imaginary unit i multiplying the time derivative term. As a consequence, the modes that arise do not decay exponentially but instead have a complex phase factor. Erwin Schr¨odinger1887–1961 Quantum wave function

The state of a quantum mechanical particle in 1D can be represented by a complex-valued wave function ψ(x, t). We can interpret |ψ(x, t)|2 as the probability density function for the position x of the particle, so that

b |ψ(x, t)|2 dx Za is the probability that the particle lies in the interval [a, b] at time t. Thus, the wave function must have the normalisation

|ψ(x, t)|2 dx = 1. Z−∞

∞ Time-dependent Schr¨odingerequation

In 1925, Erwin Schr¨odingerpostulated that ψ obeys the PDE

∂ψ h 2 ∂2ψ ih + − V(x, t)ψ = 0. (42) ∂t 2m ∂x2 Here, m is the mass of the particle, V(x, t) is the potential of the applied force and h = h/(2π) where

h = 6.626 070 040 × 10−34 Joule-seconds

is Planck’s constant. Separable solutions

Suppose that V does not depend on t, and look for separable solutions ψ(x, t) = u(x)T(t). We ﬁnd that ihT 0 h 2 u00 = E = − + V(x) T 2m u where the separation constant E has the physical dimensions of energy.

Thus, iE T 0 = − T h and we may take T(t) = e−iEt/h , so that

ψ(x, t) = e−iEt/h u(x). Time-independent Schr¨odingerequation

The spatial dependence of the wave function satisﬁes

h 2 u00 + E − V(x)u = 0, (43) 2m which has the form of a Sturm–Liouville ODE (34) with

h 2 p(x) = , λ = E, r(x) = 1, q(x) = V(x). 2m

If (φn,En) is the nth eigenpair, then φn and En are real, and

−iEnt/h ψn(x, t) = e φn(x)

is a solution of the time-dependent Schr¨odingerequation (42). Eigenstates

2 2 Since |ψn(x, t)| = φn(x) the eigenfunction φn must be normalised to satisfy

2 φn(x) dx = 1. (44) Z−∞

In this case, ψn is called the∞ nth eigenstate. Physically, the eigenvalue En is the energy of the particle in this eigenstate. The general solution is then a superposition

−iEnt/h ψ(x, t) = Anψn(x, t) = Ane φn(x). ∞ ∞ n=0 n=0 X X Deﬁne the complex inner product and norm with weight function r(x) = 1,

hf, gi = f(x)g(x) dx and kfk2 = |f(x)|2 dx. Z−∞ Z−∞

The φn are orthonormal∞ (hφn, φji = δnj) so the∞ amplitudes An satisfy Parseval’s identity:

1 = |ψ(x, t)|2 dx = hψ(·, t), ψ(·, t)i Z−∞ −iEnt/h −iEkt/h = ∞ Ane φn, Ake φk ∞ ∞ n=0 k=0 X X

−iEnt/h +iEkt/h = Ane Ake hφn, φki ∞ ∞ n=0 k=0 X X 2 = |An| . ∞ n=0 X Quantum mechanical harmonic oscillator For the classical harmonic oscillator, mx¨ + kx = 0, the restoring 1 2 force −kx has the potential V(x) = 2 kx , and the motion of the particle is given by p xClassical = A cos ωt + B sin ωt where ω = k/m.

1 2 1 2 2 Putting V(x) = 2 kx = 2 mω x in the time-independent Schr¨odingerequation (43) we have

h 2 u00 + E − 1 mω2x2u = 0. (45) 2m 2 To simplify the coeﬃcients, let

v(y) = u(x) where y = (mω/h )1/2x,

and obtain E 1 v00 + (2α + 1 − y2)v = 0, α = − . hω 2 Series solutions

Next we put 2 v(y) = e−y /2w(y) and ﬁnd that w satisﬁes Hermite’s equation

w00 − 2yw0 + 2αw = 0,

which admits power series solutions

k w(y) = Bky . ∞ k=0 X The recurrence relation for the coeﬃcients is 2k − 2α B = B for all k 0. k+2 (k + 2)(k + 1) k > Series solutions

Thus, w(y) = B0w0(y) + B1w1(y) where

2α 22(2 − α)α 23(4 − α)(2 − α)α w = 1 − y2 − y4 − y6 − ··· , 0 2! 4! 6! 2(1 − α) 22(3 − α)(1 − α) w = y + y3 + y5 + ··· . 1 3! 5! It can be shown that if α is not a non-negative integer then y2 y2/2 w0 and w1 are both O(e ) so v(y) is O(e ), so the normalisation condition (44) is impossible to satisfy.

If α = n is even, then w0 is a polynomial of degree n but w1 is again O(ey2 ).

If α = n is odd, then w1 is a polynomial of degree n but w0 is again O(ey2 ). Hermite polynomials

The Hermite polynomial Hn of degree n is the polynomial solution of w00 − 2yw0 + 2nw = 0, (46) normalised by the condition that the coeﬃcient of yn equals 2n. We conclude that the eigenpairs (φn,En) of (45) are given by

−y2/2 1 φn(x) = Cne Hn(y) and En = (n + 2 )hω

for n ∈ {0, 1, 2, . . .} and suitable Cn. Writing the Hermite ODE (46) in self-adjoint form,

2 2 (e−y w0)0 + 2ne−y w = 0,

we see that

−y2 Hn(y)Hk(y)e dy = 0, n 6= k. Z−∞

∞ Normalisation It can be shown that

2 −y2 n √ Hn(y) e dy = 2 n! π, Z−∞ so ∞ 1/2 2 h φ (x)2 dx = C e−y /2H (y)2 dy n n n mω Z−∞ Z−∞ 1/2 h 2 ∞ = C2∞ H (y)2e−y dy n mω n Z−∞ h 1/2 √ = C2 2nn∞! π n mω

and we therefore set

mω1/4 2−n 1/2 C = √ . n h n! π Standing waves φn 2 Probability densities |ψn| Part V

Initial-Boundary Value Problems in 2D Introduction

In this final part of the course, we use separation of variables to solve partial differential equations involving two space variables. First we see how the Fredholm alternative applies to boundary-value problems in 2D (or 3D) for a class of self-adjoint partial differential equations. Here, the second Green identity plays the role of the Lagrange identity in 1D. Next we generalise the Sturm–Liouville theory to obtain complete orthogonal systems of eigenfunctions in 2D, allowing us to construct solutions to initial-boundary value problems in 2D as generalised Fourier series.

The signiﬁcance of Bessel’s equation will ﬁnally be revealed. Outline

Elliptic diﬀerential operators

Green identities and boundary-value problems

Elliptic eigenproblems

Wave and diﬀusion equations Elliptic diﬀerential operators

We begin by defining a class of second-order, linear partial differential operators that are well-behaved in certain key respects. These operators appear as the spatial part of 3 key partial differential equations, each of which has many important applications. Reminder: notation from vector calculus

Partial derivative operator ∂j = ∂/∂xj. d For a scalar field u : R → R, the gradient is the vector field d d grad u : R → R defined by   ∂1u d ∂2u   grad u = ∇u = ∂ju ej =  .  j=1  .  X ∂du

d d For a vector field F : R → R , the divergence is the scalar d field div F : R → R defined by

d div F = ∇ · F = ∂jFj = ∂1F1 + ∂2F2 + ··· + ∂dFd. j=1 X Second-order linear PDEs in Rd The most general second-order linear partial diﬀerential operator d in R has the form d d d Lu =− ajk(x)∂j∂ku + bk(x)∂ku + c(x)u. (47) j=1 k=1 k=1 X X X Example The Laplacian is deﬁned by ∇2u = ∇ · (∇u) = div(grad u), that is,

d 2 2 2 2 2 ∇ u = ∂j u = ∂1u + ∂2u + ··· + ∂du. j=1 X Thus, −∇2u has the form (47) with

ajk(x) = δjk, bk(x) = 0, c(x) = 0. Principal part We call d d L0u = − ajk(x)∂j∂ku (48) j=1 k=1 X X the principal part of the partial diﬀerential operator (47). In many applications, L arises in the divergence form

d d Lu = − div(A grad u) = − ∂j ajk(x)∂ku, j=1 k=1 X X

where A = [aij]. We can still write such an L in the general form (47) with the same principal part (48) and with lower-order coeﬃcients

d bk(x) = − ∂jajk(x) and c(x) = 0. j=1 X Ellipticity

Deﬁnition A second-order linear partial diﬀerential operator (47) is d (uniformly) elliptic in a subset Ω ⊆ R if there exists a positive constant c such that

T 2 d ξ A(x)ξ > ckξk for all x ∈ Ω and ξ ∈ R .

Written out explicitly, the ellipticity condition looks like

d d d 2 ajk(x)ξjξk > c ξk. j=1 k=1 k=1 X X X Example 2 d The operator L = −∇ is elliptic (with c = 1) on any Ω ⊆ R , since d d d 2 2 δjkξjξk = ξk = kξk . j=1 k=1 k=1 X X X

Example 2 2 2 3 The operator L = −(∂1 + 2∂2 − ∂3) is not elliptic in R since in this case the quadratic form     1 0 0 ξ1 T 2 2 2 ξ Aξ = [ξ1 ξ2 ξ3] 0 2 0  ξ2 = ξ1 + 2ξ2 − ξ3 0 0 −1 ξ3 is negative if ξ1 = ξ2 = 0 and ξ3 6= 0. Symmetry and skew-symmetry Put 1 asy = a + a = symmetric part of a , jk 2 jk kj jk 1 ask = a − a = skew-symmetric part of a , jk 2 jk kj jk so that

sy sk sy sy sk sk ajk = ajk + ajk, akj = ajk, akj = −ajk

sy When investigating if L is elliptic, it suﬃces to look at ajk. Lemma

d d d d sy ajk(x)ξjξk = ajk(x)ξjξk. j=1 k=1 j=1 k=1 X X X X Proof

d d d d sk sk ajk(x)ξjξk = −akj(x) ξjξk j=1 k=1 j=1 k=1 X X X X d d sk = − ajk(x)ξkξj k=1 j=1 X X d d sk = − ajk(x)ξkξj j=1 k=1 X X d d sk = − ajk(x)ξjξk j=1 k=1 X X so d d sk 2 ajk(x)ξjξk = 0. j=1 k=1 X X Theorem sy Denote the eigenvalues of the real symmetric matrix [ajk(x)] by λj(x) for 1 6 j 6 d. The operator (47) is elliptic on Ω if and only if there exists a positive constant c such that

λj(x) > c for 1 6 j 6 d and all x ∈ Ω.

Example 2 2 Show that the L = −(3∂1 + 2∂1∂2 + 2∂2) is elliptic.

Example 2 2 Show that L = −(∂1 − 4∂1∂2 + ∂2) is not elliptic. Proof

sy Fix x, then the real symmetric matrix A = [ajk] is diagonalisable, d that is, R has an orthonormal basis of eigenvectors v1,..., vd and so T Q AQ = diag(λ1, λ2, . . . , λd)

where the matrix Q = [v1 v2 ··· vd] is orthogonal: QT = Q−1. Put η = QT ξ, then ξ = Qη so

d T T T T 2 ξ Aξ = (Qη) A(Qη) = η (Q AQ)η = λjηj , j=1 X and kξk2 = (Qη)T (Qη) = ηT (QT Q)η = ηT (I)η = kηk2, so

d T 2 2 2 ξ Aξ > ckξk ⇐⇒ λjηj > ckηk . j=1 X Three important PDEs

In the remainder of this course, we mainly focus on three PDEs. 1. Poisson equation (elliptic):

−∇2u = f.

2. Diﬀusion equation or heat equation (parabolic):

∂u − ∇2u = f. ∂t 3. Wave equation (hyperbolic):

∂2u − ∇2u = f. ∂t2 Here, u = u(x, t) for t > 0 and x ∈ Ω where Ω is a bounded, d open subset of R with a piecewise smooth boundary. Green identities and boundary-value problems

We saw how the Lagrange identity leads to the concept of self-adjointness, and allowed us to understand the solvability of boundary-value problems in 1D. In higher dimensions, the same role is played by the second Green identity, which we use to prove (the easy half of) the Fredholm alternative. Inner products

We denote the (real) inner product in L2(Ω) by

Ω fg dx dy, d = 2, hf, gi = hf, giΩ = fg dV = fg dx dy dz, d = 3. ZΩ RR Ω Sometimes we will require a weightedRRR inner product

hf, gir = hf, gir,Ω = fgr dV. ZΩ We will also require an inner product on a part of the boundary, Γ ⊆ ∂Ω,

hf, giΓ = fg dS, ZΓ or a weighted version

hf, gir,Γ = fgr dS. ZΓ A class of second-order elliptic operators To generalise the Sturm–Liouville theory, we consider a second-order, linear partial diﬀerential operator of the form

Lu = −∇ · (p∇u) + qu, (49)

with real-valued coeﬃcients p(x) and q(x).

Example If d = 1 then ∇ = d/dx so Lu = −(pu0)0 + qu. Example If d = 2 then

Lu = −(pux)x − (puy)y + qu ∂ ∂u ∂ ∂u = − p(x, y) − p(x, y) + qu. ∂x ∂x ∂y ∂y Ellipticity

The principal part of L is in divergence form −∇ · (A∇u) with A = [p(x)δjk] = p(x)I so

ξT Aξ = p(x)kξk2

and therefore L is elliptic on Ω iﬀ

p(x) > c > 0 for all x ∈ Ω.

Example If p(x) = 1 and q(x) = 0 then L = −∇2. Divergence theorem (Math2111)

Notation: Ω = Ω ∪ ∂Ω is the closure of Ω. Since Ω is bounded, the set Ω is compact and hence, by the Extreme Value Theorem, any continuous function f : Ω → R must be bounded.

Theorem d If a vector ﬁeld F : Ω → R is continuously diﬀerentiable then

∇ · F dV = F · n dS, ZΩ I∂Ω where n is the outward unit normal to Ω.

Notation: ∂u/∂n = n · ∇u. First Green identity

Theorem If u is C2 and v is C1 on Ω, then ∂u hLu, vi = p∇u · ∇v + quv dV − p v dS. (50) Ω ∂n ZΩ I∂Ω

Proof. Since ∇ · [(p∇u)v] = [∇ · (p∇u)]v + p∇u · ∇v we have

(Lu)v = p∇u · ∇v + quv − ∇ · [(p∇u)v]

and the result follows after applying the divergence theorem to F = (p∇u)v, because F · n = p(∂u/∂n)v. Second Green identity Theorem If u and v are C2 on Ω, then ∂v ∂u hLu, vi − hu, Lvi = p u − v dS. (51) Ω Ω ∂n ∂n I∂Ω

Proof. Since (Lu)v = p∇u · ∇v + quv − ∇ · [(p∇u)v] and u Lv = p∇v · ∇u + qvu − ∇ · [(p∇v)u], the Lagange identity generalizes to

(Lu)v − u Lv = ∇ · p(u∇v − v∇u),

and the result follows after applying the divergence theorem. Boundary conditions and self-adjointness

Suppose that ∂Ω is divided into two parts, ΓD and ΓN, where the D and N stand for Dirichlet and Neumann boundary conditions.

The second Green identity (51) shows that

hLu, viΩ = hu, LviΩ for all u, v ∈ D, (52)

if we choose D, the domain of the partial diﬀerential operator L, to be the space of C2 functions on Ω satisfying the homogeneous boundary conditions u = 0 on ΓD, ∂u = 0 on Γ . ∂n N Elliptic boundary value problem Recall L = −∇ · (p∇u) + qu and assume

p(x) > c > 0 for all x ∈ Ω, so that L is elliptic.

Given a source term f, Dirichlet data gD and Neumann data gN, we seek a solution u to the BVP Lu = f in Ω, u = gD on ΓD, (53) ∂u p = g on Γ . ∂n N N Technical requirement: for u to count as a solution it must have finite energy: |∇u|2 + |u|2 dV < . ZΩ ∞ Fredholm alternative Either the homogeneous problem ∂v Lv = 0 in Ω, v = 0 on Γ , p = 0 on Γ , (54) D ∂n N has only the trivial solution v ≡ 0, in which case the inhomogeneous problem (53) has a unique solution u for “every” choice of f, gD and gN; or else the homogeneous problem admits finitely many, linearly independent solutions v1, v2,..., vm, in which case the inhomogeneous problem (53) has a solution u iff the data f, gD and gN satisfy ∂v fv dV = g p k dS − g v dS k D ∂n N k ZΩ ZΓD ZΓN for k = 1, 2, . . . , m. Partial proof

Write the second Green identity (51) as

∂v ∂u hLu, vi − hu, Lvi = up − p v dS. Ω Ω ∂n ∂n I∂Ω

If u satisﬁes (53) and v satisﬁes (54) then, since ∂Ω = ΓD ∪ ΓN,

∂v ∂u hf, vi − hu, 0i = g p − p × 0 dS Ω Ω D ∂n ∂n ZΓD + u × 0 − gN v dS, ZΓN and so ∂v fv dV = g p dS − g v dS. D ∂n N ZΩ ZΓD ZΓN A pure Neumann problem Consider the BVP ∂u −∇2u = f in Ω, = g on ∂Ω. (55) ∂n If v is a solution of the homogeneous problem, ∂v −∇2v = 0 in Ω, = 0 on ∂Ω, ∂n then by the ﬁrst Green identity, ∂v (−∇2v) v dV = k∇vk2 dV − v dS. Ω Ω ∂Ω ∂n Z 0 Z I 0

|2 {z } so Ω k∇vk dV = 0. If Ω is connected then v must|{z} be constant, so the inhomogeneous problem (55) is solvable iﬀ the data satisfy R f dV + g dS = 0. ZΩ I∂Ω Elliptic eigenproblems

We now study second-order, self-adjoint elliptic eigenproblems. After looking at a few simple, concrete examples that can be solved by separation of variables, we show some key properties of the eigenfunctions that generalise earlier results for Sturm–Liouville problems in 1D. We then see how eigenfunctions can be used to construct a series solution to a boundary-value problem. By analogy with the Sturm–Liouville problem in 1D, we now consider the PDE

∇ · (p∇u) + (λr − q)u = 0 in Ω. (56)

Here, the coeﬃcients p, q, r are given real-valued, continuous functions on Ω, with

p(x) > const > 0 and r(x) > const > 0 for x ∈ Ω, with r(x) not identically zero.

Writing Lu = −∇ · (p∇u) + qu as before, the PDE (56) can be written in the form

Lu = λru in Ω. Boundary conditions

Again suppose that ∂Ω is divided into two parts, ΓD and ΓN, where the D and N stand for Dirichlet and Neumann boundary conditions.

If u : Ω → C is not identically zero and if Lu = λru in Ω, u = 0 on ΓD, (57) ∂u p = 0 on Γ , ∂n N then u is said to be an eigenfunction with eigenvalue λ, and we call (u, λ) an eigenpair.

The results shown earlier for Sturm–Liouville problems and Fourier series in 1D generalise in a natural way to this 2D or 3D setting. Example: eigenproblem on the unit square

The eigenvalues λkn and eigenfunctions u = φkn(x, y) for

∇2u + λu = 0, 0 < x < 1, 0 < y < 1,

with boundary conditions

u(x, 0) = u(x, 1) = 0, 0 < x < 1,

and ∂u ∂u (0, y) = (1, y) = 0, 0 < y < 1, ∂x ∂x are given by

2 2 2 λkn = (k + n )π and φkn(x, y) = cos(kπx) sin(nπy)

for k = 0, 1, 2, . . . and n = 1, 2, 3, . . . . The eigenfunction φkn(x, y) = cos(kπx) sin(nπy) Contours of φkn Example: eigenproblem on the unit disk

2 Introduce polar coordinates in R ,

x = r cos θ and y = r sin θ,

and recall that 1 ∂ ∂u 1 ∂2u ∇2u = r + . r ∂r ∂r r2 ∂θ2

Consider

2 ∇ u + λu = 0, 0 6 r < 1, −π < θ 6 π, with the boundary condition

u(1, θ) = 0, −π 6 θ < π. The eigenvalues λ = λnj are

2 λnj = knj where Jn(knj) = 0, for n = 0, 1, 2, . . . and j = 1, 2, 3, . . . .

The corresponding eigenfunctions u = φnj are

φ0j(r, θ) = J0(k0jr), C φnj(r, θ) = Jn(knjr) cos(nθ), n > 1, S φnj(r, θ) = Jn(knjr) sin(nθ), n > 1. C The eigenfunction φnj = Jn(knjr) cos(nθ) Contours of φnj Eigenproblem on a more complicated domain (Math5295)

Let

2 2 2 Ω1 = { (x, y): x + y < 3 }, 2 Ω2 = (−1, 1) ,

Ω = Ω1 \ Ω2,

and consider the eigenproblem

−∇2u = λu in Ω,

u = 0 on ΓD, ∂u = 0 on Γ , ∂n N

where ΓD denotes the inner (square) boundary and ΓN the outer (circular) boundary. Compute approximate solutions using a ﬁnite element method with 254,336 degrees of freedom. λ1 = 0.505 λ2 = λ3 = 0.680 λ4 = 1.148 and λ5 = 1.251 Eigenfunctions are orthogonal

Theorem Let L be the partial diﬀerential operator (49). If u, v are (possibly complex-valued) functions satisfying

Lu = λru, Lv = µrv, in Ω,

u = 0, v = 0, on ΓD, ∂u ∂v p = 0, p = 0, on Γ , ∂n ∂n N and if λ 6= µ, then u and v are orthogonal on Ω with respect to the weight function r, i.e.,

hu, vir = uvr dV = 0. ZΩ Eigenvalues are real

Theorem If u is a nontrivial solution of the elliptic eigenproblem (57) then λ is real. Moreover, both Re u and Im u are also solutions of (57).

Thus, it is always possible to choose purely real-valued eigenfunctions for L.

The proofs proceed exactly as in the 1D case. Proofs

The self-adjointness (52) of L implies that

(λ − µ)hu, vir = hλru, vi − hu, µrvi = hLu, vi − hu, Lvi = 0,

so if λ 6= µ then hu, vir = 0. The coeﬃcents p, q, r are real-valued so we easily see that (u,¯ ¯λ) is an eigenpair and therefore

(λ − ¯λ)hu, u¯ir = 0.

2 ¯ Since hu, u¯ir = Ω |u| r dV > 0, we conclude that λ = λ, or in other words λ is real. R Finally, because u and u¯ are eigenfunctions with eigenvalue λ, so are u + u¯ u − u¯ Re u = and Im u = . 2 2i Completeness of the eigenfunctions

If several eigenfunctions share the same eigenvalue then they are not necessarily orthogonal. In this case, however, we can always orthogonalise them using the Gram–Schmidt procedure (Math2601).

In general, the elliptic eigenproblem (57) has a sequence of eigenvalues λ = λj and corresponding eigenfunctions u = φj such that 1. the eigenvalues satisfy λ1 6 λ2 6 λ3 6 ··· with λj → as j → ;

2. the eigenfunctions φ1, φ2, φ3, . . . form a complete ∞ orthogonal∞ system on Ω with respect to the weight function r. Steklov eigenproblem

As before, let Lu = −∇ · (p∇u) + qu,

but suppose now that ∂Ω is divided into three parts: Γ, ΓD and ΓN. If u : Ω → C is not identically zero and if Lu = 0 in Ω, ∂u p = λru on Γ, ∂n (58) u = 0 on ΓD, ∂u p = 0 on Γ , ∂n N then we say that (u, λ) is a Steklov eigenpair. Example: a Steklov eigenproblem for the unit square

The eigenvalues λn and eigenfunctions u = φn(x, y) for

∇2u = 0, 0 < x < 1, 0 < y < 1, ∂u (x, 1) = λu(x, 1), 0 < x < 1, ∂y u(0, y) = u(1, y) = 0, 0 < y < 1, ∂u (x, 0) = 0, 0 < x < 1, ∂y are

λn = nπ tanh(nπ) and φn(x, y) = sin nπx cosh nπy

for n = 1, 2, . . . . The eigenfunction φn(x, y) = sin(nπx) cosh(nπy) Contours of φn Steklov eigenvalues are real The ﬁrst Green identity with v = u¯ gives ∂u hLu, u¯i = p|∇u|2 + q|u|2) dV − p u¯ dS. Ω ∂n ZΩ I∂Ω If u satisﬁes (58), then Lu = 0 in Ω so

∂u p|∇u|2 + q|u|2) dV = p u¯ dS, ∂n ZΩ I∂Ω and since ∂Ω = Γ ∪ ΓD ∪ ΓN the boundary conditions imply that ∂u ∂u p u¯ dS = (λru)u¯ dS + p (0¯) dS + (0)u¯ dS. ∂n ∂n I∂Ω ZΓ ZΓD ZΓN We conclude that λ is real, because

p|∇u|2 + q|u|2) dV = λ r|u|2 dS. ZΩ ZΓ Steklov eigenfunctions are orthogonal over Γ

If u and v are eigenfunctions with eigenvalues λ and µ, then by the second Green identity,

(λ − µ) uvr dS = (λru)v dS − u(µrv) dS ZΓ ZΓ ZΓ ∂u ∂v = p v − u dS ∂n ∂n ZΓ ∂u ∂v = p v − u dS ∂n ∂n Z∂Ω = hu, LviΩ − hLu, viΩ

= hu, 0iΩ − h0, viΩ = 0,

hu, vir,Γ = uvr dS = 0 if λ 6= µ. ZΓ Completeness?

The second type of eigenproblem (58) has a sequence of eigenvalues λ = λj and corresponding eigenfunctions u = φj such that

1. the eigenfunctions φ1, φ2, φ3, . . . are orthogonal with respect to r on Γ; 2. the eigenvalues satisfy λ1 6 λ2 6 λ3 6 ··· with λj → as j → . ∞ The eigenfunctions∞ are complete in L2(Γ, r) iﬀ the homogeneous BVP Lv = 0 in Ω, v = 0 on Γ ∪ ΓD, (59) ∂v p = 0 on Γ , ∂n N has only the trivial solution v ≡ 0 in Ω. Exceptional case If (59) has a nontrivial solution v, then by the second Green identity

∂v ∂v ∂φ φ p dS = p φ − v j dS j ∂n j ∂n ∂n ZΓ I∂Ω = hLφj, viΩ − hφj, LviΩ,

and so, since Lφj = 0 = Lv in Ω, ∂v φ p dS = 0. j ∂n ZΓ Thus, the function p ∂v w = r ∂n satisﬁes hφj, wir,Γ = 0 for all j. Boundary-value problem with gD = 0 and gN = 0 Suppose that we have homogeneous boundary data, ∂u Lu = f in Ω, u = 0 on Γ , p = 0 on Γ . D ∂n N

To construct u, ﬁnd the eigenpairs (φj, λj) satisfying ∂φ Lφ = λ rφ in Ω, φ = 0 on Γ , p j = 0 on Γ . j j j j D ∂n N Then u exists iﬀ

hf, φjiΩ = 0 whenever λj = 0,

in which case (with arbitrary constants Cj),

hf, φ i u(x) ∼ C φ + j Ω φ (x), x ∈ Ω. j j λ kφ k2 j λ =0 λ 6=0 j j r,Ω Xj Xj Boundary-value problem with f = 0 and gN = 0 Consider ∂u Lu = 0 in Ω, u = g on Γ , p = 0 on Γ . D D ∂n N

Find the Steklov eigenpairs (φj, λj) satisfying ∂φ ∂φ Lφ = 0 in Ω, p j = λ rφ on Γ , p j = 0 on Γ . j ∂n j j D ∂n N If the homogeneous problem ∂v Lv = 0 in Ω, v = 0 on Γ , p = 0 on Γ , D ∂n N has only the trivial solution v ≡ 0, then

hgD, φjir,ΓD u(x) ∼ 2 φj(x), x ∈ Ω. ∞ kφjk j=1 r,ΓD X Boundary-value problem with f = 0 and gD = 0 Consider ∂u Lu = 0 in Ω, u = 0 on Γ , p = g on Γ . D ∂n N N

Find the Steklov eigenpairs (φj, λj) satisfying ∂φ Lφ = 0 in Ω, φ = 0 on Γ , p j = λ rφ on Γ . j j D ∂n j j N Then u exists iﬀ

hgN, φjiΓN = 0 whenever λj = 0,

in which case

hgN, φjiΓN u(x) ∼ Cjφj(x) + 2 φj(x), x ∈ Ω. λjkφjk λ =0 λ 6=0 r,ΓN Xj Xj Example

Consider the following boundary-value problem on the unit square:

∇2u = 0, 0 < x < 1, 0 < y < 1, ∂u (x, 1) = 1, 0 < x < 1, ∂y u(0, y) = u(1, y) = 0, 0 < y < 1, ∂u (x, 0) = 0, 0 < x < 1. ∂y

Use the associated Steklov eigenproblem (solved previously) to show that

4 1 cosh(2k − 1)πy u(x, y) ∼ 2 2 sin(2k − 1)πx . π ∞ (2k − 1) sinh(2k − 1)π k=1 X Boundary-value problem with general data

To handle the most general situation ∂u Lu = f in Ω, u = g on Γ , p = g on Γ , D D ∂n N N we write u as a sum of three terms

u = u1 + u2 + u3,

where each term solves one of the cases treated above: ∂u Lu = f in Ω, u = 0 on Γ , p 1 = 0 on Γ , 1 1 D ∂n N ∂u Lu = 0 in Ω, u = g on Γ , p 2 = 0 on Γ , 2 2 D D ∂n N ∂u Lu = 0 in Ω, u = 0 on Γ , p 3 = g on Γ . 3 3 D ∂n N N An alternative approach

If we can ﬁnd w : Ω ∪ ∂Ω → R satisfying just the boundary conditions in (53),

∂w w = g on Γ and p = g on Γ , D D ∂n N N then we can solve (53) by putting f? = f − Lw, solving

Lu? = f? in Ω, ? u = 0 on ΓD, ∂u? p = 0 on Γ , ∂n N and then putting u = u? + w.

This will often be easier than ﬁnding three sets of eigensystems. Example Consider the PDE on the unit square,

−∇2u = x(1 − x) for 0 < x < 1 and 0 < y < 1,

subject to the boundary conditions

u(x, 0) = 1 and u(x, 1) = 2 for 0 < x < 1,

plus ∂u ∂u (0, y) = 0 = (1, y) for 0 < y < 1. ∂x ∂x Use a previously solved eigenproblem to show that

2 sin(2m − 1)πy u(x, y) ∼ y + 1 + 3 3 3π ∞ (2m − 1) m=1 X 4 cos 2jπx sin(2m − 1)πy − 5 . π ∞ ∞ j2(2m − 1) 4j2 + (2m − 1)2 j=1 m=1 X X Wave and diﬀusion equations

We now consider two time-dependent partial diﬀerential equations. Both are solved in the same way as their 1D versions. The only change is that we must use 2D eigenfunctions. Vibrating membrane

Consider a stretched membrane in the shape of a planar region Ω. Assuming that the membrane is ﬁxed along ∂Ω, has initial displacement u0(x) and is initially at rest, we have the following initial boundary value problem for the transverse displacement u = u(x, t):

∂2u − c2∇2u = 0 in Ω, for t > 0, ∂t2 ∂u (60) u = u and = 0 in Ω, when t = 0, 0 ∂t u = 0 on ∂Ω, for t > 0.

This problem is a 2D version of the vibrating string problem. The constant c2 depends on how tightly the membrane is stretched and on the density of the membrane. Reduction to a sequence of ODEs Let λ1, λ2, . . . and φ1, φ2, . . . be the eigenvalues and eigenfunctions of the Laplacian,

2 −∇ φj = λjφj in Ω, with φj = 0 on ∂Ω,

then

hu(·, t), φjiΩ u(x, t) ∼ uj(t)φj(x), uj(t) = , ∞ hφj, φjiΩ j=1 X and 2 2 ∂ u 2 2 d uj 2 2 − c ∇ u = 2 + c λjuj φj(x). ∂t ∞ dt j=1 X Thus, uj satisﬁes a second-order ODE,

2 d uj 2 hu0, φjiΩ duj 2 + c λjuj = 0, uj(0) = u0j = , (0) = 0. dt hφj, φjiΩ dt Normal modes of vibration

Solving the IVP for uj, we ﬁnd that p uj(t) = u0j cos(ωjt) where ωj = c λj,

u(x, t) ∼ u0j cos(ωjt)φj(x). ∞ j=1 X The separable solution

cos(ωjt)φj(x)

is the jth normal mode of vibration. The period and frequency of this mode are

2π 1 ωj τj = and = . ωj τj 2π Concentration and ﬂux Consider molecules of a solute moving in a stationary ﬂuid, and let

u(x, t) = mass concentration of solute, q(x, t) = ﬂux density vector of solute, f(x, t) = volume density of solute sources,

at position x and time t. Thus,

u dV = mass of solute in region Ω, ZΩ f dV = rate at which solute is added within Ω, ZΩ and, for an oriented surface Γ,

q · n dS = mass ﬂux through Γ. ZΓ Diﬀusion In 1855, Adolf Fick postulated that

q = −K∇u, (61)

where the diffusivity K depends on physical properties of the solute and the fluid medium. In a fixed region Ω, conservation of mass requires that d u dV = q · (−n) dS + f dV. dt ZΩ I∂Ω ZΩ On the LHS, d ∂u u dV = dV, dt ∂t ZΩ ZΩ and on the RHS the divergence theorem gives

q · (−n) dS = − ∇ · q dV = ∇ · (K∇u) dV. I∂Ω ZΩ ZΩ Diffusion Thus, 1 ∂u − ∇ · (K∇u) − f dV = 0. vol Ω ∂t ZΩ The LHS tends to the pointwise value of the integrand if we shrink Ω to a point, so u satisfies the diffusion equation, ∂u − ∇ · (K∇u) = f(x, t). (62) ∂t

In a steady state, the concentration u does not depend on t and so

−∇ · (K∇u) = f(x, t).

If K is independent of x, then

∇ · (K∇u) = K∇2u. Heat equation in 3D If we let

u(x, t) = temperature,K = thermal conductivity, q(x, t) = heat ﬂux vector, ρ = mass density, f(x, t) = vol. density of heat sources, σ = speciﬁc heat,

then since Fourier’s law,

q = −K∇u,

has the same form as Fick’s law (61), conservation of thermal energy leads to the heat equation, ∂u ρσ − ∇ · (K∇u) = f(x, t) ∂t in the same way that conservation of mass leads to the diﬀusion equation (62). An application: the cooling-oﬀ problem

A body Ω is initially at a temperature u0, and then cools down as heat leaves through ∂Ω. A (linearised) model of this process is

∂u − ∇2u = 0 in Ω, for t > 0, ∂t ∂u + bu = 0 on ∂Ω, for t > 0, (63) ∂n u = u0 on Ω, when t = 0,

where b > 0 is a constant. The associated eigenproblem takes the form −∇2φ = λφ in Ω, ∂φ (64) + bφ = 0 on ∂Ω. ∂n We have not previously discussed such a Robin boundary condition. Eigenvalues are strictly positive

The ﬁrst Green identity gives ∂φ λ |φ|2 dV = k∇φk2 dV − φ dS ∂n ZΩ ZΩ Z∂Ω = k∇φk2 dV + b |φ|2 dS, ZΩ Z∂Ω so λ > 0. Moreover, if λ = 0 then ∇φ = 0 in Ω and φ = 0 on ∂Ω,

implying that φ is constant on each component of Ω, and that each of the constant values is zero; that is, φ ≡ 0. Hence, every eigenvalue is strictly positive. Radially symmetric example

Suppose now that Ω is the unit ball ρ < 1, and that u0 = u0(ρ) is radially symmetric. Recall (from Math2111) that ∇2u equals

1 ∂ ∂u 1 ∂ ∂u 1 ∂2u ρ2 + sin ϕ + ρ2 ∂ρ ∂ρ ρ2 sin ϕ ∂ϕ ∂ϕ ρ2 sin2 ϕ ∂θ2 but from symmetry, u is independent of the angular variables ϕ and θ, that is, u = u(ρ, t). Thus, it suﬃces to consider radially symmetric eigenfunctions φ = φ(ρ) satisfying

1 d dφ − ρ2 = λφ for 0 < ρ < 1. ρ2 dρ dρ

Putting λ = k2 with k > 0, we conclude that φ satisﬁes a spherical Bessel equation,

(ρ2φ0)0 + k2ρ2φ = 0. A tutorial problem shows that the function v(ρ) = ρ1/2φ(ρ) satisﬁes the parameterised Bessel equation of order 1/2,

2 00 0 2 2 1 ρ v + ρv + (k ρ − 4 )v = 0, so v = AJ1/2(kρ) + BY1/2(kρ) and thus

−1/2 φ = ρ AJ1/2(kρ) + BY1/2(kρ) .

Here, B = 0 because φ is bounded at ρ = 0, and k must be chosen so that dφ + bφ = 0 at ρ = 1. dρ Recall the identity 1 d z−νJ (z) = −z−ν−1J (z) z dz ν ν+1 and put s = kρ, then d ds d ρ−1/2J (kρ) = k1/2s−1/2J (s) dρ 1/2 dρ ds 1/2 3/2 −1/2 −1/2 = −k s J3/2(s) = −kρ J3/2(kρ), so the equation for k > 0 is

−kJ3/2(k) + bJ1/2(k) = 0, which has a sequence of solutions 0 < k1 < k2 < k3 < ··· tending to , and corresponding eigenfunctions

φ (ρ) = ρ−1/2J (k ρ). ∞ j 1/2 j Solutions when b = 1. Eigenfunctions Orthogonality

Put 1 hf, gi = f(ρ)g(ρ) ρ2 dρ Z0 then hφn, φji = 0 if n 6= j, 2 2 0 0 2 since u = φn and λ = kn satisfy (ρ u ) + λρ u = 0. Note that 2 dV = 4πρ dρ so if we view φn and φj as functions on Ω, then

1 2 φnφj dV = 4π φnφjρ dρ = 0 if n 6= j. ZΩ Z0

Exercise: prove this orthogonality property using the second Green identity and (64). Fourier modes

The Fourier expansion

hu(·, t), φji u(ρ, t) ∼ uj(t)φj(ρ), uj(t) = 2 , ∞ kφjk j=1 X gives 2 ut − ∇ u ∼ u˙ j + λjuj)φj(ρ), ∞ j=1 X so we require that

u˙ j + λjuj = 0 for t > 0, with uj(0) = u0j.

Thus, −λjt uj(t) = u0je . The series

−λjt u(ρ, t) ∼ u0je φj(ρ), 0<ρ<1, t>0, ∞ j=1 X satisﬁes the PDE and boundary condition. The initial condition gives

u0(ρ) = u(ρ, 0) ∼ u0jφj(ρ), ∞ j=1 X so u0j must be the jth Fourier coeﬃcient of the initial data:

1 2 hu0, φji 0 u0(ρ)φj(ρ)ρ dρ u0j = = 1 . hφj, φji 2 2 R 0 φj(ρ) ρ dρ R As expected, u(ρ, t) → 0 as t → .

∞