ó Hilbert spaces The proof of the Hilbert basis theorem is not mathematics, it is theology. — Camille Jordan Wir müssen wissen, wir werden wissen. — David Hilbert We now continue to study a special class of Banach spaces, namely Hilbert spaces, in which the presence of a so-called “inner product” al- lows us to define angles between elements. In particular, we can intro- duce the geometric concept of orthogonality. This has far-reaching con- sequences. ó.G Deänition and Examples Definition 4.1. Let H be a vector space over K. An inner product (or a scalar product) on H is a map (· j ·) : H × H ! K such that the following three properties hold. (IP1) For all x 2 H, one has (x j x) ≥ 0, and (x j x) = 0 if and only if x = 0. (IP2) For all x, y 2 H, one has (x j y) = (y j x). (IP3) For all x, y, z 2 H and l 2 K, one has (lx + y j z) = l(x j z) + (y j z). The pair (H, (· j ·)) is called inner product space or pre-Hilbert space. Example 4.2. (a) On H = KN, N (x j y) := ∑ xkyk k=1 defines an inner product. 99 4 Hilbert spaces (b) On H = `2, ¥ (x j y) := ∑ xkyk k=1 for x = (xk) and y = (yk) defines an inner product. (c) On C([a, b]), Z b ( f j g) := f (t)g(t) dt a defines an inner product. If w : [a, b] ! R is such that there exist constants 0 < # < M, such that # ≤ w(t) ≤ M for all t 2 [a, b], then also Z b ( f j g)w := f (t)g(t)w(t) dt a defines an inner product on C([a, b]). (d) If (W, S, m) is a measure space, then Z ( f j g) := f g dm W defines an inner product on L2(W, S, m). Note that for examples (b), (c) and (d), it follows from Hölder’s in- equality that the sum, respectively the integral, is well-defined. Remark 4.3. Condition (IP1) is called definiteness of (· j ·), (IP2) is called symmetry. Note that if K = R, then (IP2) reduces to (x j y) = (y j x). (IP3) states that (· j ·) is linear in the first component. Note that it follows from (IP2) and (IP3) that (x j ly + z) = l(x j y) + (x j z). If (H, (· j ·)) is an inner product space, we set q kxk := (x j x). Lemma 4.4 (Cauchy–Schwarz). Let (H, (· j ·)) be an inner product space. Then j(x j y)j ≤ kxk · kyk (4.1) for all x, y 2 H and equality holds if and only if x and y are linearly dependent. Proof. Let l 2 K. Then 0 ≤ (x + ly j x + ly) = (x j x) + l(y j x) + l(x j y) + jlj2(y j y) = kxk2 + 2 Re(l(x j y)) + jlj2kyk2. If y = 0, then (x j y) = 0 and the claimed inequality holds true. If y 6= 0, 100 4.1 Definition and Examples − we may put l = −(x j y)kyk 2. Then we obtain j(x j y)j2 j−(x j y)j2 j(x j y)j2 0 ≤ kxk2 − 2 Re + kyk2 = kxk2 − , kyk2 kyk4 kyk2 which establishes (4.1). By (IP1), equality holds if and only if x = −ly. p Lemma 4.5. Let (H, (· j ·)) be an inner product space. Then k·k := (· j ·) defines a norm on H. Proof. If kxk = 0, then (x j x) = 0 and therefore x = 0 by (IP1). This proves (N1). For (N2), let x 2 H and l 2 K be given. Then klxk2 = (lx j lx) = ll(x j x) = jlj2kxk2. Property (N3) holds since kx + yk2 = kxk2 + 2 Re (x j y) + kyk2 ≤ kxk2 + 2kxkkyk + kyk2 = (kxk + kyk)2, where we used the Cauchy–Schwarz inequality. Definition 4.6. A Hilbert space is an inner product space (H, (· j ·)) which p is complete with respect to the norm k·k := (· j ·). Example 4.7. The inner product spaces from Example 4.2 (a), (b) and (d) are Hilbert spaces, see for example Theorem 3.105. The inner product space from Example 4.2 (c) is not complete and therefore not a Hilbert space. It suffices to note that C([a, b]) is dense in L2([a, b]) and that the given inner product gives rise to the restriction of the usual norm in L2([a, b]) to C([a, b]). Lemma 4.8 (Parallelogram identity). Let H be an inner product space. Then In fact, also a con- verse holds: If the kx + yk2 + kx − yk2 = 2kxk2 + 2kyk2 parallelogramm identity holds in for all x, y 2 H. a normed space (X, k·k), then there Proof. exists a unique 2 2 inner product (· j ·) kx + yk + kx − yk on X such that 2 = kxk2 + 2 Re (x j y) + kyk2 + kxk2 − 2 Re (x j y) + kyk2 kxk = (x j x). = 2(kxk2 + kyk2) d p p Exercise 4.9. Show that in (R , k·kp), (` , k·kp) and (L ([0, 1]), k·kp) the parallelogram identity fails if p 6= 2. Hence in this case there is no inner p product (· j ·)p such that kxkp = (x j x)p. Lemma 4.10. Let H be an inner product space. Then (· j ·) : H × H ! K is continuous. 101 4 Hilbert spaces Proof. Let xn ! x and yn ! y. Then M := supn2Nkxnk < ¥. By the Cauchy–Schwarz inequality, j(xn j yn) − (x j y)j ≤ j(xn j yn − y)j + j(xn − x j y)j ≤ Mkyn − yk + kykkxn − xk ! 0. ó.³ Orthogonal Projection In this section we show that in a Hilbert space one can project onto any closed subspace. In other words, for any point there exists a unique best approximation in any given closed subspace. For this the geometric prop- erties arising from the inner product are crucial. In fact, an inner Definition 4.11. Let (H, (· j ·)) be an inner product space. We say that product allows two vectors x, y 2 H are orthogonal (and write x ? y) if (x j y) = 0. to define angles Given a subset S ⊂ H, the annihilator S? of S is defined by between two vectors. Consider H = R2 ? := f 2 : ? 2 g and describe the S y H y x for all x S . angle between two ? vectors x, y 2 R2 If S is a subspace, then S is also called the orthogonal complement of S. with kxk = kyk = 1 in terms of the inner In an inner product space the following fundamental and classic geo- product! metric identity holds. Lemma 4.12 (Pythagoras). Let H be an inner product space. If x ? y, then kx + yk2 = kxk2 + kyk2. Proof. kx + yk2 = kxk2 + 2 Re (x j y) + kyk2 = kxk2 + kyk2. Proposition 4.13. Let H be an inner product space and S ⊂ H. (a) S? is a closed, linear subspace of H. (b) span S ⊂ (S?)?. (c) span S \ S? = f0g. Proof. (a) If x, y 2 S? and l 2 K, then, for z 2 S, we have (lx + y j z) = l(x j z) + (y j z) = 0, hence lx + y 2 S?. This shows that S? is a linear ? subspace. If (xn) is a sequence in S which converges to x, then we infer from Lemma 4.10 that (x j z) = lim (xn j z) = 0 for all z 2 S. (b) By (a), (S?)? is a closed linear subspace which contains S. Thus span S ⊂ (S?)?. (c) If x 2 span S \ S?, then, by (b), x 2 S? \ (S?)? and hence x ? x. But this means (x j x) = 0. By (IP1) it follows that x = 0. 102 4.2 Orthogonal Projection We now come to the main result of this section. Theorem 4.14. Let (H, (· j ·)) be a Hilbert space and K ⊂ H be a closed linear Give an example `2 subspace. Then for every x 2 H, there exists a unique element PKx of K such of a subspace of that that is not closed! kPKx − xk = min fky − xk : y 2 Kg. Proof. Let d := inf fky − xk : y 2 Kg. By the definition of the infimum, there exists a sequence (yn) in K with kyn − xk ! d. Applying the paral- lelogram identity 4.8 to the vectors x − yn and x − ym, we obtain 2 2 2(kx − ynk + kx − ymk ) 2 2 = k(x − yn) + (x − ym)k + kx − yn − (x − ym)k 2 2 = k2x − yn − ymk + kyn − ymk 1 2 = 4 x − (y + y ) + ky − y k2. 2 n m n m 1 2 2 Since znm := 2 (yn + ym) 2 K, we have kx − znmk ≥ d and thus In a Hilbert space the orthogonal pro- 2 2 2 2 kyn − ymk ≤ 2(kx − ynk + kx − ymk ) − 4d . jection can be more generally defined for all closed and By the choice of the sequence (yn), the right-hand side of this equation ! ( ) convex subsets. But converges to 0 as n, m ¥, proving that yn is a Cauchy sequence. then the orthogoal Since H is complete, (yn) converges to some vector PKx. Since K is closed, projection does not PKx 2 K. We have thus proved existence. need to be a linear As for uniqueness, if kz − xk = min fky − xk : y 2 Kg, then, by the map, of course. parallelogram identity, 1 2 2kx − P xk2 + 2kx − zk2 = 4 x − (P x + z) + kz − P xk2 K 2 K K and thus 1 2 4d2 = 4 x − (P x + z) + kP x − zk2 ≥ 4d2 + kP x − zk2 2 K K K proving that kPKx − zk = 0; hence PKx = z.