222 BRUCE K. DRIVER†
12. Hilbert Spaces 12.1. Hilbert Spaces Basics. Definition 12.1. Let H be a complex vector space. An inner product on H is a function, , : H H C, such that h· ·i × → (1) ax + by, z = a x, z + b y, z i.e. x x, z is linear. h i h i h i → h i (2) x, y = y, x . (3) hx 2i hx, x i 0 with equality x 2 =0iff x =0. k k ≡ h i ≥ k k Notice that combining properties (1) and (2) that x z,x is anti-linear for fixed z H, i.e. → h i ∈ z,ax + by =¯a z,x + ¯b z,y . h i h i h i We will often find the following formula useful: x + y 2 = x + y,x + y = x 2 + y 2 + x, y + y, x k k h i k k k k h i h i (12.1) = x 2 + y 2 +2Re x, y k k k k h i Theorem 12.2 (Schwarz Inequality). Let (H, , ) be an inner product space, then for all x, y H h· ·i ∈ x, y x y |h i| ≤ k kk k and equality holds iff x and y are linearly dependent. Proof. If y =0, the result holds trivially. So assume that y =0. First off notice 2 6 that if x = αy for some α C, then x, y = α y and hence ∈ h i k k x, y = α y 2 = x y . |h i| | |k k k kk k x,y h i Moreover, in this case α := y 2 . k k 2 Now suppose that x H is arbitrary, let z x y − x, y y. (So z is the “orthogonal projection”∈ of x onto y, seeFigure28.)Then≡ − k k h i
Figure 28. The picture behind the proof.
x, y 2 x, y 2 x, y 0 z 2 = x h iy = x 2 + |h i| y 2 2Re x, h iy ≤ k k − y 2 k k y 4 k k − h y 2 i ° k k ° k k k k ° x, y° 2 = °x 2 |h °i| °k k − y °2 k k from which it follows that 0 y 2 x 2 x, y 2 with equality iff z =0or 2 ≤ k k k k − |h i| equivalently iff x = y − x, y y. k k h i ANALYSIS TOOLS WITH APPLICATIONS 223
Corollary 12.3. Let (H, , ) be an inner product space and x := x, x . Then is a norm on H. Moreoverh· ·i , is continuous on H H,kwherek H his viewedi as thek·k normed space (H, ). h· ·i × p k·k Proof. The only non-trivial thing to verify that is a norm is the triangle inequality: k·k x + y 2 = x 2 + y 2 +2Re x, y x 2 + y 2 +2 x y k k k k k k h i ≤ k k k k k kk k =( x + y )2 k k k k where we have made use of Schwarz’s inequality. Taking the square root of this inequality shows x + y x + y . For the continuity assertion: k k ≤ k k k k
x, y x0,y0 = x x0,y + x0,y y0 |h i − h i| |h − i h − i| y x x0 + x0 y y0 ≤ k kk − k k kk − k y x x0 +( x + x x0 ) y y0 ≤ k kk − k k k k − k k − k = y x x0 + x y y0 + x x0 y y0 k kk − k k kk − k k − kk − k from which it follows that , is continuous. h· ·i Definition 12.4. Let (H, , ) be an inner product space, we say x, y H are orthogonal and write x h· y·i iff x, y =0. More generally if A H ∈is a set, x H is orthogonal to⊥A andh writei x A iff x, y =0for all⊂ y A. Let ∈ ⊥ h i ∈ A⊥ = x H : x A be the set of vectors orthogonal to A. We also say that a set S { H∈ is orthogonal⊥ } if x y for all x, y S such that x = y. If S further satisfies,⊂ x =1for all x S, then⊥ S is said to∈ be orthonormal.6 k k ∈ Proposition 12.5. Let (H, , ) be an inner product space then h· ·i (1) (Parallelogram Law) (12.2) x + y 2 + x y 2 =2 x 2 +2 y 2 k k k − k k k k k for all x, y H. (2) (Pythagorean∈ Theorem) If S H is a finite orthonormal set, then ⊂ (12.3) x 2 = x 2. k k k k x S x S X∈ X∈ (3) If A H isaset,thenA⊥ is a closed linear subspace of H. ⊂ Remark 12.6. See Proposition 12.37 in the appendix below for the “converse” of the parallelogram law. Proof. IwillassumethatH is a complex Hilbert space, the real case being easier. Items 1. and 2. are proved by the following elementary computations: x + y 2 + x y 2 = x 2 + y 2 +2Re x, y + x 2 + y 2 2Re x, y k k k − k k k k k h i k k k k − h i =2 x 2 +2 y 2, k k k k and x 2 = x, y = x, y k k h i h i x S x S y S x,y S X∈ X∈ X∈ X∈ = x, x = x 2. h i k k x S x S X∈ X∈ 224 BRUCE K. DRIVER†
Item 3. is a consequence of the continuity of , and the fact that h· ·i A⊥ = x A ker( ,x ) ∩ ∈ h· i where ker( ,x )= y H : y,x =0 — a closed subspace of H. h· i { ∈ h i } Definition 12.7. A Hilbert space is an inner product space (H, , ) such that the induced Hilbertian norm is complete. h· ·i Example 12.8. Let (X, ,µ) be a measure space then H := L2(X, ,µ) with inner product M M (f,g)= f gdµ¯ · ZX is a Hilbert space. In Exercise 12.6 you will show every Hilbert space H is “equiv- alent” to a Hilbert space of this form. Definition 12.9. A subset C of a vector space X is said to be convex if for all x, y C the line segment [x, y]:= tx +(1 t)y :0 t 1 joining x to y is contained∈ in C as well. (Notice that any{ vector− subspace≤ of≤X }is convex.) Theorem 12.10. Suppose that H is a Hilbert space and M H beaclosedconvex subset of H. Then for any x H there exists a unique y M⊂ such that ∈ ∈ x y = d(x, M)= inf x z . z M k − k ∈ k − k Moreover, if M is a vector subspace of H, then the point y may also be characterized astheuniquepointinM such that (x y) M. − ⊥ Proof. By replacing M by M x := m x : m M we may assume x =0. − { − ∈ } Let δ := d(0,M)=infm M m and y, z M, seeFigure29. ∈ k k ∈
Figure 29. Thegeometryofconvexsets.
By the parallelogram law and the convexity of M, y + z (12.4) 2 y 2 +2 z 2 = y+z 2 + y z 2 =4 2 + y z 2 4δ2 + y z 2. k k k k k k k − k k 2 || k − k ≥ k − k Hence if y = z = δ, then 2δ2 +2δ2 4δ2 + y z 2, so that y z 2 =0. k k k k ≥ k − k k − k Therefore, if a minimizer for d(0, ) M exists, it is unique. · | ANALYSIS TOOLS WITH APPLICATIONS 225
Existence. Let yn M be chosen such that yn = δn δ d(0,M). Taking ∈ 2 k2 k 2 → ≡ 2 y = ym and z = yn in Eq. (12.4) shows 2δm +2δn 4δ + yn ym . Passing to the limit m, n in this equation implies, ≥ k − k →∞ 2 2 2 2 2δ +2δ 4δ + lim sup yn ym . ≥ m,n k − k →∞
Therefore yn ∞ is Cauchy and hence convergent. Because M is closed, y := { }n=1 lim yn M and because is continuous, n →∞ ∈ k·k
y = lim yn = δ = d(0,M). n k k →∞ k k So y is the desired point in M which is closest to 0. Now for the second assertion we further assume that M is a closed subspace of H and x H. Let y M be the closest point in M to x. Then for w M, the function ∈ ∈ ∈
g(t) x (y + tw) 2 = x y 2 2tRe x y, w + t2 w 2 ≡ k − k k − k − h − i k k has a minimum at t =0. Therefore 0=g0(0) = 2Re x y,w . Since w M is arbitrary, this implies that (x y) M. Finally suppose− h −y Miis any point∈ such that (x y) M. Then for z −M,⊥by Pythagorean’s theorem,∈ − ⊥ ∈ x z 2 = x y + y z 2 = x y 2 + y z 2 x y 2 k − k k − − k k − k k − k ≥ k − k which shows d(x, M)2 x y 2. That is to say y is the point in M closest to x. ≥ k − k
Definition 12.11. Suppose that A : H H is a bounded operator. The adjoint → of A, denote A∗, is the unique operator A∗ : H H such that Ax, y = x, A∗y . → h i h i (The proof that A∗ exists and is unique will be given in Proposition 12.16 below.) A bounded operator A : H H is self - adjoint or Hermitian if A = A∗. → Definition 12.12. Let H be a Hilbert space and M H be a closed subspace. ⊂ The orthogonal projection of H onto M is the function PM : H H such that for → x H, PM (x) is the unique element in M such that (x PM (x)) M. ∈ − ⊥ Proposition 12.13. Let H be a Hilbert space and M H beaclosedsubspace. ⊂ The orthogonal projection PM satisfies:
(1) PM is linear (and hence we will write PM x rather than PM (x). 2 (2) PM = PM (PM isaprojection). (3) PM∗ = PM , (PM is self-adjoint). (4) Ran(PM )=M and ker(PM )=M ⊥. Proof.
(1) Let x1,x2 H and α F, then PM x1 + αPM x2 M and ∈ ∈ ∈
PM x1 + αPM x2 (x1 + αx2)=[PM x1 x1 + α(PM x2 x2)] M ⊥ − − − ∈ showing PM x1 + αPM x2 = PM (x1 + αx2), i.e. PM is linear. 2 (2) Obviously Ran(PM )=M and PM x = x for all x M. Therefore P = ∈ M PM . 226 BRUCE K. DRIVER†
(3) Let x, y H, then since (x PM x) and (y PM y) are in M ⊥, ∈ − − PM x, y = PM x, PM y + y PM y h i h − i = PM x, PM y h i = PM x +(x PM ),PM y h − i = x, PM y . h i (4) It is clear that Ran(PM ) M. Moreover, if x M, then PM x = x implies ⊂ ∈ that Ran(PM )=M. Now x ker(PM ) iff PM x =0iff x = x 0 M ⊥. ∈ − ∈
Corollary 12.14. Suppose that M H is a proper closed subspace of a Hilbert ⊂ space H, then H = M M ⊥. ⊕ Proof. Given x H, let y = PM x so that x y M ⊥. Then x = y +(x y) ∈ −2 ∈ − ∈ M + M ⊥. If x M M ⊥, then x x, i.e. x = x, x =0. So M M ⊥ = 0 . ∈ ∩ ⊥ k k h i ∩ { }
Proposition 12.15 (Riesz Theorem). Let H∗ be the dual space of H (Notation 3.63). The map j (12.5) z H ,z H∗ ∈ −→ h· i ∈ is a conjugate linear isometric isomorphism. Proof. The map j is conjugate linear by the axioms of the inner products. Moreover, for x, z H, ∈ x, z x z for all x H |h i| ≤ k kk k ∈ with equality when x = z. This implies that jz = ,z = z . Therefore H∗ H∗ j is isometric and this shows that j is injective.k k To finishkh· theik proofk wek must show that j is surjective. So let f H∗ which we assume with out loss of generality is non-zero. Then M =ker(f) —∈ a closed proper subspace of H. Since, by Corollary 12.14, H = M M ⊥,f : H/M = M ⊥ F is a linear isomorphism. This ⊕ ∼ → 28 shows that dim(M ⊥)=1and hence H = M Fx0 where x0 M ⊥ 0 . ⊕ ¯ ∈ 2 \{ } Choose z = λx0 M ⊥ such that f(x0)= x0,z . (So λ = f(x0)/ x0 .) Then for ∈ h i k k x = m + λx0 with m M and λ F, ∈ ∈ f(x)=λf(x0)=λ x0,z = λx0,z = m + λx0,z = x, z h i h i h i h i which shows that f = jz. Proposition 12.16 (Adjoints). Let H and K be Hilbert spaces and A : H K → be a bounded operator. Then there exists a unique bounded operator A∗ : K H such that →
(12.6) Ax, y K = x, A∗y H for all x H and y K. h i h i ∈ ∈ ¯ Moreover (A + λB)∗ = A∗ + λB∗,A∗∗ := (A∗)∗ = A, A∗ = A and A∗A = 2 k k k k k k A for all A, B L(H, K) and λ C. k k ∈ ∈ 28 Alternatively, choose x0 M 0 such that f(x0)=1. For x M we have f(x λx0)=0 ∈ ⊥ \{ } ∈ ⊥ − provided that λ := f(x). Therefore x λx0 M M = 0 , i.e. x = λx0. This again shows − ∈ ∩ ⊥ { } that M ⊥ is spanned by x0. ANALYSIS TOOLS WITH APPLICATIONS 227
Proof. For each y K, then map x Ax, y K is in H∗ and therefore there exists by Proposition 12.15∈ a unique vector→ zh H isuch that ∈ Ax, y K = x, z H for all x H. h i h i ∈ This shows there is a unique map A∗ : K H such that Ax, y K = x, A∗(y) H → h i h i for all x H and y K. To finishtheproof,weneedonlyshowA∗ is linear and ∈ ∈ bounded. To see A∗ is linear, let y1,y2 K and λ C, then for any x H, ∈ ∈ ∈ Ax, y1 + λy2 K = Ax, y1 K + λ¯ Ax, y2 K h i h i h i = x, A∗(y1) K + λ¯ x, A∗(y2) K h i h i = x, A∗(y1)+λA∗(y2) K h i and by the uniqueness of A∗(y1 + λy2) we find
A∗(y1 + λy2)=A∗(y1)+λA∗(y2).
This shows A∗ is linear and so we will now write A∗y instead of A∗(y). Since
A∗y, x H = x, A y H = Ax, y K = y, Ax K h i h ∗ i h i h i ¯ it follows that A∗∗ = A. he assertion that (A + λB)∗ = A∗ + λB∗ is left to the reader, see Exercise 12.1. The following arguments prove the assertions about norms of A and A∗ :
A∗ =sup A∗k =sup sup A∗k, h k k k K: k =1 k k k K: k =1 h H: h =1 |h i| ∈ k k ∈ k k ∈ k k =sup sup k, Ah =sup Ah = A , h H: h =1 k K: k =1 |h i| h H: h =1 k k k k ∈ k k ∈ k k ∈ k k
2 A∗A A∗ A = A and k k ≤ k kk k k k 2 A =sup Ah, Ah =sup h, A∗Ah k k h H: h =1 |h i| h H: h =1 |h i| ∈ k k ∈ k k sup A∗Ah = A∗A , ≤ h H: h =1 k k k k ∈ k k wherein these arguments we have repeatedly made use of the Inequality. Exercise 12.1. Let H, K, M be Hilbert space, A, B L(H, K),C L(K, M) and ¯ ∈ ∈ λ C. Show (A + λB)∗ = A∗ + λB∗ and (CA)∗ = A∗C∗ L(M,H). ∈ ∈ Exercise 12.2. Let H = Cn and K = Cm equipped with the usual inner products, i.e. z,w H = z w¯ for z,w H. Let A be an m n matrix thought of as a linear h i · ∈ × operator from H to K. Show the matrix associated to A∗ : K H is the conjugate transpose of A. → Exercise 12.3. Let K : L2(ν) L2(µ) be the operator defined in Exercise 9.12. 2 2 → Show K∗ : L (µ) L (ν) is the operator given by →
K∗g(y)= k¯(x, y)g(x)dµ(x). ZX Definition 12.17. uα α A H is an orthonormal set if uα uβ for all α = β { } ∈ ⊂ ⊥ 6 and uα =1. k k 228 BRUCE K. DRIVER†
Proposition 12.18 (Bessel’s Inequality). Let uα α A be an orthonormal set, then { } ∈ 2 2 (12.7) x, uα x for all x H. |h i| ≤ k k ∈ α A X∈ In particular the set α A : x, uα =0 is at most countable for all x H. { ∈ h i 6 } ∈ Proof. Let Γ A be any finite set. Then ⊂ 2 2 2 0 x x, uα uα = x 2Re x, uα uα,x + x, uα ≤ k − h i k k k − h ih i |h i| α Γ α Γ α Γ X∈ X∈ X∈ 2 2 = x x, uα k k − |h i| α Γ X∈ showing that 2 2 x, uα x . |h i| ≤ k k α Γ X∈ Taking the supremum of this equation of Γ A then proves Eq. (12.7). ⊂⊂ Proposition 12.19. Suppose A H is an orthogonal set. Then s = v A v 2 ⊂ ∈ exists in H iff v A v < . (In particular A must be at most a countable set.) ∈ k 2k ∞ P Moreover, if v A v < , then 2 P∈ k k 2 ∞ (1) s =P v A v and k k ∈ k k (2) s, x = v A v, x for all x H. h i P ∈ h i ∈ ∞ Similarly if Pvn n∞=1 is an orthogonal set, then s = vn exists in H iff { } n=1 ∞ 2 ∞ P vn < . In particular if vn exists, then it is independent of rearrange- n=1 k k ∞ n=1 mentsP of vn n∞=1. P { } Proof. Suppose s = v A v exists. Then there exists Γ A such that ∈ ⊂⊂ 2 P v 2 = v 1 k k ° ° ≤ v Λ °v Λ ° X∈ °X∈ ° for all Λ A Γ ,wherein the first inequality° we° have used Pythagorean’s theorem. ⊂⊂ \ ° ° 2 Taking the supremum over such Λ shows that v A Γ v 1 and therefore ∈ \ k k ≤ v 2 1+ Pv 2 < . k k ≤ k k ∞ v A v Γ X∈ X∈ 2 Conversely, suppose that v A v < . Then for all >0 there exists Γ A ∈ k k ∞ ⊂⊂ such that if Λ A Γ , ⊂⊂ \ P 2 (12.8) v = v 2 <