18.103 Fall 2013

Orthonormal Bases

Consider an inner product space V with inner product hf, gi and norm

kfk2 = hf, fi

Proposition 1 (Continuity) If kun − uk → 0 and kvn − vk → 0 as n → ∞, then

kunk → kuk; hun, vni → hu, vi.

Proof. Note first that since kvn − vk → 0,

kvnk ≤ kvn − vk + kvk ≤ M < ∞ for a constant M independent of n. Therefore, as n → ∞,

|hun, vni − hu, vi| = |hun − u, vni + hu, vn − vi| ≤ Mkun − uk + kukkvn − vk → 0

2 2 In particular, if un = vn, then kunk = hun, uni → hu, ui = kuk .  For u and v in V we say that u is perpendicular to v and write u ⊥ v if hu, vi = 0. The Pythogorean theorem says that if u ⊥ v, then

ku + vk2 = kuk2 + kvk2 (1)

Definition 1 ϕn is called an orthonormal sequence, n = 1, 2,... , if hϕn, ϕmi = 0 for n 6= m 2 and hϕn, ϕni = kϕnk = 1.

Suppose that ϕn is an orthonormal sequence in an inner product space V . The following four consequences of the Pythagorean theorem (1) were proved in class (and are also in the text): N X If h = anϕn, then n=1 N 2 X 2 khk = |an| . (2) 1

1 N X If f ∈ V and sN = hf, ϕniϕn, then n=1

2 2 2 kfk = kf − sN k + ksN k (3)

If VN = span {ϕ1, ϕ2, . . . , ϕN }, then

kf − sN k = min kf − gk (best approximation property) (4) g∈VN

If cn = hf, ϕni, then ∞ 2 X 2 kfk ≥ |cn| (Bessel’s inequality). (5) n=1

Definition 2 A Hilbert space is defined as a complete inner product space (under the dis- tance d(u, v) = ku − vk).

Theorem 1 Suppose that ϕn is an orthonormal sequence in a Hilbert space H. Let ∞ [ VN = span {ϕ1, ϕ2, . . . , ϕN },V = VN N=1

(V is the vector space of finite linear combinations of ϕn.) The following are equivalent. a) V is dense in H (with respect to the distance d(f, g) = kf − gk),

b) If f ∈ H and hf, ϕni = 0 for all n, then f = 0. N X c) If f ∈ H and sN = hf, ϕniϕn, then ksN − fk → 0 as N → ∞. n=1 d) If f ∈ H, then ∞ 2 X 2 kfk = |hf, ϕni| n=1

∞ If the properties of the theorem hold, then {ϕn}n=1 is called an orthonormal basis or complete orthonormal system for H. (Note that the word “complete” used here does not mean the same thing as completeness of a metric space.)

Proof. (a) =⇒ (b). Let f satisfy hf, ϕni = 0, then by taking finite linear combinations, hf, vi = 0 for all v ∈ V . Choose a sequence vj ∈ V so that kvj − fk → 0 as j → ∞. Then by Proposition 1 above

2 0 = hf, vji → hf, fi =⇒ kfk = 0 =⇒ f = 0

2 N X (b) =⇒ (c). Let f ∈ H and denote cn = hf, ϕni, sN = cnϕn. By Bessel’s inequality 1 (5), ∞ X 2 2 |cn| ≤ kfk < ∞. 1 Hence, for M < N (using (2))

2 N N 2 X X 2 ksN − sM k = cnϕn = |cn| → 0 as M,N → ∞. M+1 M+1

In other words, sN is a Cauchy sequence in H. By completeness of H, there is u ∈ H such that ksN − uk → 0 as N → ∞. Moreover,

hf − sN , ϕni = 0 for all N ≥ n.

Taking the limit as N → ∞ with n fixed yields

hf − u, ϕni = 0 for all n.

Therefore by (b), f − u = 0. (c) =⇒ (d). Using (3) and (2),

N 2 2 2 2 X 2 kfk = kf − sN k + ksN k = kf − sN k + |cn| , (cn = hf, ϕni) 1

2 Take the limit as N → ∞. By (c), kf − sN k → 0. Therefore,

∞ 2 X 2 kfk = |cn| 1

Finally, for (d) =⇒ (a),

N 2 2 X 2 kfk = kf − sN k + |cn| 1

2 2 Take the limit as N → ∞, then by (d) the rightmost term tends to kfk so that kf −sN k → 0. Since sN ∈ VN ⊂ V , V is dense in H. 

3 Proposition 2 Let ϕn be an orthonormal sequence in a Hilbert space H, and

X 2 X 2 |an| < ∞, |bn| < ∞

then ∞ ∞ X X u = anϕn, v = bnϕn n=1 n=1 are convergent series in H norm and

∞ X hu, vi = anbn (6) n=1

Proof. Let N N X X uN = anϕn; vN = bnϕn. 1 1 Then for M < N, N 2 X 2 kuN − uM k = |an| → 0 as M → ∞ M

so that uN is a Cauchy sequence converging to some u ∈ H. Similarly, vN → v in H norm. Finally, N N N X X X huN , vN i = hajϕj, bkϕki = ajbkhϕj, ϕki = ajbj j,k=1 j,k=1 j=1 since hϕj, fki = 0 for j 6= k and hfj, fji = 1. Taking the limit as N → ∞ and using the continuity property (1), huN , vN i → hu, vi, gives (6).  ∞ If H is a Hilbert space and {ϕn}n=1 is an orthonormal basis, then every element can be written ∞ X f = anϕn (series converges in norm) n=1 The mapping X {an} 7→ anϕn n is a linear isometry from `2(N) to H that preserves the inner product. The inverse mapping is f 7→ {an} = {hf, ϕni}

4 It is also useful to know that as soon as a linear mapping between Hilbert spaces is an isometry (preserves norms of vectors) it must also preserve the inner product. Indeed, the inner product function (of two variables u and v) can be written as a function of the norm function (of linear combinations of u and v). This is known as polarization: Polarization Formula.

2 2 2 2 hu, vi = a1ku + ivk + a2ku + vk + a3kuk + a4kvk (7) with a1 = i/2, a2 = 1/2, a3 = −(1 + i)/2, a4 = −(i + 1)/2 Proof.

ku + ivk2 = hu + iv, u + ivi = kuk2 + hiv, ui + hu, ivi + kvk2 = kuk2 + i(hv, ui − hu, vi) + kvk2

Similarly, ku + vk2 = kuk2 + (hv, ui + hu, vi) + kvk2 Multiplying the first equation by i and adding to the second, we find that

iku + ivk2 + ku + vk2 = (i + 1)kuk2 + 2hu, vi + (i + 1)kvk2

Solving for hu, vi yields (7). 

5