Functional Analysis Lecture Notes Chapter 1. Hilbert Spaces

Home , Functional analysis, Hilbert space, Orthogonal complement

CHAPTER 1. HILBERT SPACES

CHRISTOPHER HEIL

1. Elementary Properties of Hilbert Spaces Notation 1.1. Throughout, F will denote either the real line R or the complex plane C. All vector spaces are assumed to be over the ﬁeld F.

Definition 1.2 (Semi-Inner Product, Inner Product). If X is a vector space over the field F, then a semi-inner product on X is a function h·, ·i: X × X → F such that (a) hx, xi ≥ 0 for all x ∈ X, (b) hx, yi = hy, xi for all x, y ∈ X, and (b) Linearity in the first variable: hαx + βy, zi = αhx, zi + βhy, zi for all x, y, z ∈ X and α, β ∈ F.

Exercise 1.3. Immediate consequences are: (a) Anti-linearity in the second variable: hx, αy + βzi = α¯hx, yi + β¯hx, zi. (b) hx, 0i = 0 = h0, yi. (c) h0, 0i = 0.

Deﬁnition 1.4. If a semi-inner product h·, ·i satisﬁes: hx, xi = 0 =⇒ x = 0, then it is called an inner product, and X is called an inner product space or a pre-Hilbert space.

Notation 1.5. There are many diﬀerent standard notations for a semi-inner product, e.g., hx, yi = [x, y] = (x, y) = hx|yi, to name a few. We shall prefer the notation hx, yi.

Date: February 20, 2006. These notes closely follow and expand on the text by John B. Conway, “A Course in Functional Analysis,” Second Edition, Springer, 1990. 1 2 CHRISTOPHER HEIL

Exercise 1.6. Prove the following. n (a) The dot product x · y = x1y¯1 + · · · + xny¯n is an inner product on C .

(b) If w1, . . . , wn ≥ 0 are fixed scalars, then the weighted dot product hx, yi = x1y¯1w1 + n · · · + xny¯nwn is a semi-inner product on C . It is an inner product if wi > 0 for each i. (c) If A is an n × n positive semi-definite matrix (Ax · x ≥ 0 for all x ∈ Cn), then hx, yi = Ax · y is a semi-inner product on Cn, and it is an inner product if A is positive definite (meaning Ax · x > 0 for all x =6 0). The weighted dot product is just the special case where A is diagonal. (d) Show that if h·, ·i is an arbitrary inner product on Cn, then there exists a positive definite matrix A such that hx, yi = Ax · y. Hint: Let e1, . . . , en be the standard basis. Then hx, yi = y¯1hx, e1i + · · · + y¯nhx, eni. For each k, the mapping x 7→ hx, eki is linear.

Exercise 1.7. Let I be a countable index set (e.g., I = N or Z). Let w : I → [0, ∞). Let 2 2 2 `w = `w(I) be the weighted ` space deﬁned by

2 2 2 `w = `w(I) = x = (xi)i∈I : |xi| w(i) < ∞ . n Xi∈I o Show that

hx, yi = xi y¯i w(i) i∈I X 2 deﬁnes a semi-inner product on `w. If w(i) > 0 for all i then it is an inner product. If w(i) = 1 for all i, then we simply call this space `2. The series deﬁning hx, yi converges because of the Cauchy–Schwarz inequality.

Example 1.8. Let (X, Ω, µ) be a measure space (X is a set, Ω is a σ-algebra of subsets of of X, and µ: Ω → [0, ∞] is a countably additive measure). Deﬁne

L2(X) = f : X → F : |f(x)|2 dµ(x) < ∞ , X n Z o where we identify functions that are equal almost everywhere, i.e., f = g a.e. ⇐⇒ µ{x ∈ X : f(x) =6 g(x)} = 0. Then hf, gi = f(x) g(x) dµ(x) ZX deﬁnes an inner product on L2(X). Other notations for L2(X) are L2(µ), L2(X, µ), L2(dµ), L2(X, dµ), etc. 2 2 The space `w(I) is a special case of L (X), where X = I and µ is a weighted counting measure on I. CHAPTER 1. HILBERT SPACES 3

Exercise 1.9. Every subspace of an inner product space is itself an inner product space (using the same inner product). 2 2 Hence every subspace of `w or L (X) is also an inner product space. For example, V = L2(Rn) ∩ C(Rn) = {f ∈ L2(Rn) : f is continuous} is a subspace of L2(Rn) and hence is also an inner product space. List some other subspaces 2 2 of some particular `w or L (X) spaces.

Notation 1.10 (Associated Semi-Norm or Norm). If h·, ·i is a semi-inner product on X, then we will write kxk = hx, xi1/2. We will see later that k · k deﬁnes a semi-norm on X, and that it is a norm on X if h·, ·i is an inner product on X.

Lemma 1.11 (Polar Identity). If h·, ·i is a semi-inner product on X, then for all x, y ∈ X we have kx + yk2 = kxk2 + 2 Re hx, yi + kyk2. Proof. Using the fact that z + z¯ = 2Re z, we have kx + yk2 = hx + y, x + yi = kxk2 + hx, yi + hy, xi + kyk2 = kxk2 + 2 Re hx, yi + kyk2.

Theorem 1.12 (Cauchy-Bunyakowski-Schwarz Inequality). If h·, ·i is a semi-inner product on X, then ∀ x, y ∈ X, |hx, yi| ≤ kxk kyk. Moreover, equality holds if and only if there exist scalars α, β ∈ F, not both zero, such that kαx + βyk = 0. Proof. If x, y ∈ X and α ∈ F, then 0 ≤ kx − αyk2 = hx − αy, x − αyi = hx, xi − αhy, xi − α¯hx, yi + αα¯hy, yi = kxk2 − αhy, xi − α¯hx, yi + |α|2 kyk2. Write hy, xi = beiθ where b ≥ 0 and θ ∈ R (θ = 0 if F = R). Set α = e−iθt where t ∈ R. Then for this α we have 0 ≤ kxk2 − e−iθtbeiθ − eiθtbe−iθ + t2 kyk2 = kxk2 − 2bt + kyk2 t2. This is a quadratic polynomial in t. In order for this polynomial to be everywhere nonneg- ative, it can have at most one real root, which means that the discriminant must be ≤ 0, i.e., (−2b)2 − 4 kyk2 kxk2 ≤ 0. 4 CHRISTOPHER HEIL

Hence |hx, yi|2 = |hy, xi|2 = b2 ≤ kxk2 kyk2. Exercise: Supply the proof for the case of equality.

Corollary 1.13. If h·, ·i is a semi-inner product on X, then kxk = hx, xi1/2 deﬁnes a semi- norm on X, which means that: (a) kxk ≥ 0 for all x ∈ X, (b) kαxk = |α| kxk for all x ∈ X and α ∈ F, (c) Triangle Inequality: kx + yk ≤ kxk + kyk for all x, y ∈ X.

If h·, ·i is an inner product on X then k · k is a norm on X, which means that in addition to (a)–(c) above, we also have:

(d) kxk = 0 =⇒ x = 0. Proof. (a), (b), (d) Exercises. (c) Using the Polar Identity, we have kx + yk2 = hx + y, x + yi = kxk2 + 2 Re hx, yi + kyk2 ≤ kxk2 + 2 |hx, yi| + kyk2 ≤ kxk2 + 2 kxk kyk + kyk2

2 = kxk + kyk . Deﬁnition 1.14 (Distance). Let h·, ·i be an inner product on X. Then the distance from x to y in X is d(x, y) = kx − yk.

Exercise 1.15. d(·, ·) deﬁnes a metric on X.

Deﬁnition 1.16. Many of the results in this chapter are valid not only for inner product spaces, but for any space which possesses a norm. Assume that X is a vector space. A semi-norm on X is a function k · k: X → [0, ∞) such that statements (a)-(c) above hold. If, in addition, statement (d) holds, then k · k is a norm, and X is called a normed space, normed linear space, or normed vector space.

Deﬁnition 1.17 (Convergence). Let X be a normed linear space (such as an inner product ∞ space), and let {fn}n=1 be a sequence of elements of X. CHAPTER 1. HILBERT SPACES 5

(a) We say that {fn} converges to f ∈ X, and write fn → f, if

lim kf − fnk = 0, n→∞ i.e.,

∀ ε > 0, ∃ N > 0 such that n > N =⇒ kf − fnk < ε.

(b) We say that {fn} is Cauchy if

∀ ε > 0, ∃ N > 0 such that m, n > N =⇒ kfm − fnk < ε.

Exercise 1.18. Let X be a normed linear space. Prove the following. (a) Reverse Triangle Inequality: kxk − kyk ≤ kx − yk.

(b) Continuity of the norm: xn → x =⇒ k xnk → kxk.

xn → x, yn → y =⇒ hxn, yni → hx, yi.

(d) All convergent sequences are bounded, and the limit of a convergent sequence is unique. (e) Cauchy sequences are bounded. (f) Every convergent sequence is Cauchy. (g) There exist inner product spaces for which not every Cauchy sequence is convergent.

Deﬁnition 1.19 (Hilbert Space). An inner product space H is called a Hilbert space if it is complete, i.e., if every Cauchy sequence is convergent. That is, ∞ {fn}n=1 is Cauchy in H =⇒ ∃ f ∈ H such that fn → f. The letter H will always denote a Hilbert space.

Deﬁnition 1.20 (Banach Space). A normed linear space X is called a Banach space if it is complete, i.e., if every Cauchy sequence is convergent. We make no assumptions about the meaning of the symbol X, i.e., it need not denote a Banach space. A Hilbert space is thus a Banach space whose norm is associated with an inner product.

n 2 2 Example 1.21. C , `w (w strictly positive), and L (X) are all Hilbert spaces (using the default inner products).

n 2 Exercise 1.22. Prove that C and `w are Hilbert spaces. 6 CHRISTOPHER HEIL

∞ Deﬁnition 1.23 (Convergent Series). Let {fn}n=1 be a sequence of elements of a normed ∞ linear space X. Then the series n=1 fn converges and equals f ∈ X if the partial sums s = N f converge to f. That is, N n=1 n P P N kf − sN k = f − fn → 0 as N → ∞. n=1 X

∞ Exercise 1.24. Let X be an inner product space, and suppose that the series n=1 fn converges in X. Show that if g ∈ X, then ∞ ∞ P

fn, g = hfn, gi. n=1 n=1 DX E X Note that this is NOT an immediate consequence of the deﬁnition of inner product. You must use both the fact that the inner product is linear in the ﬁrst variable AND the continuity of the inner product (Exercise 1.18) to prove this result.

∞ Exercise 1.25 (Absolutely Convergent Series). Let X be a Banach space and let {fn}n=1 ∞ ∞ be a sequence of elements of X. Prove that if n=1 kfnk < ∞ then the series n=1 fn converges in X. We say that such a series is absolutely convergent. P P Hint: You must show that the sequence of partial sums {sN } converges. Since X is a Banach space, you just have to show that this sequence is Cauchy.

Exercise 1.26 (Unconditionally Convergent Series). Let X be a Banach space and let ∞ ∞ {fn}n=1 be a sequence of elements of X. The series f = n=1 fn is said to converge uncon- ∞ ditionally if every rearrangement of the series converges. That is, f = n=1 fn converges unconditionally if for each bijection σ : N → N the seriesP ∞ P

fσ(n) n=1 X ∞ converges. It can be shown that in this case, the series will converge to f, i.e., f = n=1 fσ(n) for every permutation σ. ∞ P Prove that if a series f = n=1 fn converges absolutely, then it converges unconditionally. In ﬁnite dimensions the converse is true, but we will see later that the converse fails in inﬁnite dimensions (see Exercise 4.16).P

∞ Proposition 1.27. Let X be a Banach space and let {fn}n=1 be a sequence of elements of ∞ X. Then the series f = n=1 fn converges unconditionally if and only if it converges with respect to the net of finite subsets of N, i.e., if P ∀ ε > 0, ∃ finite F0 ⊆ N such that ∀ finite F ⊇ F0, f − fn < ε. n∈F X

CHAPTER 1. HILBERT SPACES 7

Deﬁnition 1.28 (Topology). Let X be a normed linear space. (a) The open ball in X centered at x ∈ X with radius r > 0 is

B(x, r) = Br(x) = {y ∈ X : kx − yk < r}.

(b) A subset U ⊆ X is open if ∀ x ∈ U, ∃ r > 0 such that B(x, r) ⊆ U.

Deﬁnition 1.29 (Limit Points, Closure, Density). Let X be a normed linear space and let A ⊆ X.

(a) A point f ∈ A is called a limit point of A if there exist fn ∈ A with fn =6 f such that fn → f. (b) The closure of A is the smallest closed set A¯ such that A ⊆ A¯. Speciﬁcally, A¯ = {F ⊆ X : F is closed and F ⊇ A}.

Exercise 1.30. (a) The closure of A equals the union of A and all limit points of A:

A¯ = A ∪ {x ∈ X : x is a limit point of A} = {z ∈ X : ∃ yn ∈ A such that yn → z}.

(b) If X is a normed linear space, then the closure of an open ball B(x, r) is the closed ball B(x, r) = {y ∈ X : kx − yk ≤ r}. (c) Prove that A is dense if and only if ∀ x ∈ X, ∀ ε > 0, ∃ y ∈ A such that kx − yk < ε.

Example 1.31. The set of rationals Q is dense in the real line R.

Proposition 1.32. Let X be a normed linear space and let F ⊆ X. Then F is closed ⇐⇒ F contains all its limit points. Proof. ⇒. Suppose that F is closed but that there exists a limit point f that does not belong to F . By deﬁnition, there must exist fn ∈ F such that fn → f. However, f ∈ X \ F , which is open, so there exists some r > 0 such that B(f, r) ⊆ X \ F . Yet there must exist some fn with kf − fnk < r, so this fn will belong to X \ F , which is a contradiction. ⇐. Exercise. 8 CHRISTOPHER HEIL

Exercise 1.33. In ﬁnite dimensions, all subspaces are closed sets. This is not true in inﬁnite dimensions. (a) Prove that

c00 = {x = (x1, . . . , xN , 0, 0, . . . ) : N > 0, x1, . . . , xN ∈ F} 2 2 is a subspace of ` (N) that is not closed. Prove that c00 is dense in ` (N). (b) Prove that ∞ c0 = {x = (xk)k=1 : lim xk = 0} k→∞ is a dense subspace of `2(N).

(c) Prove that Cc(R), the space of continuous, compactly supported functions on R, is a subspace of L2(R) that is dense and not closed. The support of a function f : R → C is the closure in R of the set {x ∈ R : f(x) =6 0}. Thus a function is compactly supported if nonzero only within a bounded subset of R. (d) Let E be a (Lebesgue) measurable subset of Rn. Let M = {f ∈ L2(Rn) : supp(f) ⊆ E}. Prove that M is a closed subspace of L2(Rn).

Proposition 1.34. Let H be a Hilbert space and let M be a subspace of H. Then M is itself a Hilbert space (using the inner product from H) if and only if M is closed. Let X be a Banach space and let M be a subspace of X. Then M is itself a Banach space (using the norm from X) if and only if M is closed. Proof. ⇒. Suppose that M is a Hilbert space. Let f be any limit point of M, i.e., suppose fn ∈ M and fn → f. Then {fn} is a convergent sequence in H, and hence is Cauchy in H. Since each fn belongs to M, it is also Cauchy in M. Since M is a Hilbert space, {fn} must therefore converge in M, i.e., fn → g for some g ∈ M. However, limits are unique, so f = g ∈ M. Therefore M contains all its limit points and hence is closed. ⇐. Exercise.

Exercise 1.35. Find examples of inner product spaces that are not Hilbert spaces.

Remark 1.36. In constructing the examples for the preceding exercise, you will probably look for examples of inner product spaces X which are subspaces of a larger Hilbert space H. Is every inner product space a subspace of a larger Hilbert space? The answer is yes, it is also possible to construct a Hilbert space H ⊇ X, called the completion of X.

Theorem 1.37 (Parallelogram Law). If X is an inner product space, then ∀ f, g ∈ X, kf + gk2 + kf − gk2 = 2 kfk2 + kgk2 . Proof. Exercise. CHAPTER 1. HILBERT SPACES 9

Exercise 1.38. Show that `p for p =6 2 and Lp(R) for p =6 2 are not inner product spaces under the default norms (show that the Parallelogram Law fails).

Exercise 1.39. Suppose that X is a Banach space over the complex ﬁeld C, and the norm k · k of X satisﬁes the Parallelogram Law. Prove that 1 hf, gi = kf + gk2 − kf − gk2 + ikf + igk2 − ikf − igk2 4 is an inner product on X, and that kfk2 = hf, fi.

2. Orthogonality Deﬁnition 2.1. Let H be a Hilbert space. (a) Two vectors f, g ∈ H are orthogonal (written f ⊥ g) if hf, gi = 0.

(b) A sequence of vectors {fi}i∈I is an orthogonal sequence if hfi, fji = 0 whenever i =6 j.

(c) A sequence of vectors {fi}i∈I is an orthonormal sequence if it is orthogonal and each vector is a unit vector, i.e., 1, i = j, hfi, fji = δij = (0, i =6 j.

Exercise 2.2. (a) If {fi}i∈I is an orthogonal sequence of nonzero vectors, then it is finitely linearly independent, i.e., every finite subset is linearly independent. ∞ (b) Let I = N. Define en = (δmn)m=1, i.e., en is the sequence having a 1 in the nth ∞ 2 component and zeros elsewhere. Show that {en}n=1 is an orthonormal sequence in ` (called the standard basis of `2). 2πinx (c) Let X = [0, 1] and let µ be Lebesgue measure. Define en(x) = e . Show that 2 {en}n∈Z is an orthonormal sequence in L [0, 1].

Theorem 2.3 (Pythagorean Theorem). If f1, . . . , fn ∈ H are orthogonal, then n n 2 2 fk = kfkk . k=1 k=1 X X Proof. Exercise.

Deﬁnition 2.4 (Convex Set). Let X is a vector space and K ⊆ X, then K is convex if x, y ∈ K, 0 ≤ t ≤ 1 =⇒ tx + (1 − t)y ∈ K. Thus, the entire line segment between x and y is contained in K (including the midpoint 1 1 2 x + 2 y in particular). 10 CHRISTOPHER HEIL

Exercise 2.5. (a) Every subspace of a vector space X is convex. (b) If X is a normed linear space, then open and closed balls in X are convex.

Deﬁnition 2.6. Let X be a normed linear space and let A ⊆ H. The distance from a point x ∈ H to the set A is dist(x, A) = inf{kx − yk : y ∈ A}.

Theorem 2.7 (Closest Point Property). If H is a Hilbert space and K is a nonempty closed, convex subset of H, then given any h ∈ H there exists a unique point k0 ∈ K that is closest to h. That is, there is a unique point k0 ∈ K such that

kh − k0k = dist(h, K) = inf{kh − kk : k ∈ K}. Proof. Let d = dist(h, K) = inf{kh − kk : k ∈ K}.

By definition, there exist kn ∈ K such that kh − knk → d, and furthermore we have d ≤ kh − knk for each n. Therefore, if we fix any ε > 0 then we can find an N such that 2 2 2 2 n > N =⇒ d ≤ kh − knk ≤ d + ε . By the Parallelogram Law, 2 2 2 2 k(h − kn) − (h − km)k + k(h − kn) + (h − km)k = 2 kh − knk + kh − kmk . Hence, k − k 2 1 kh − k k2 kh − k k2 k + k 2 m n = k(h − k ) − (h − k )k2 = n + m − h − m n . 2 4 n m 2 2 2 m n m n Ho wever, k +k ∈ K since K is convex, so kh − k +k k ≥ d and therefore 2 2 k + k 2 − h − m n ≤ −d2. 2

Also, if m, n > N then 2 2 2 2 kh − knk , kh − kmk ≤ d + ε . Therefore, k − k 2 d2 + ε2 d2 + ε2 m n ≤ + − d2 = ε2. 2 2 2

So, kkm − knk ≤ 2ε for all m, n > N, which says that the sequence {kn} is Cauchy. Since H is complete, this sequence must converge, i.e., kn → k0 for some k0 ∈ H. But kn ∈ K for all n and K is closed, so we must have k0 ∈ K. Since h − kn → h − k0, we have

kh − k0k = lim kh − knk = d, n→∞

and thus kh − k0k ≤ kh − kk for every k ∈ K. Exercise: Prove that k0 is the unique point closest to h by assuming that there exists another point k0 that satisﬁes kh−k0 k ≤ kh−kk for every k ∈ K, and derive a contradiction. 0 0 0 k0+k0 Draw a picture and think about the midpoint 2 . CHAPTER 1. HILBERT SPACES 11

Notation 2.8 (Notation for Closed Subspaces). Since we will often deal with closed subspaces of a Hilbert space, we declare that the notation M ≤ H means that M is a closed subspace of the Hilbert space H. The letter H will always denote a Hilbert space.

Theorem 2.9. Let M ≤ H, and ﬁx h ∈ H. Then the following statements are equivalent. (a) h = p + e where p is the (unique) point in M closest to h. (b) h = p + e where p ∈ M and e ⊥ M (i.e., e ⊥ f for every f ∈ M). Proof. (a) ⇒ (b). Let p be the (unique) point in M closest to h, and let e = p − h. Choose any f ∈ M. We must show that hf, ei = 0. Since M is a subspace, p + λf ∈ M for any scalar λ ∈ F. Hence, kh − pk2 ≤ kh − (p + λf)k2 = k(h − p) − λfk2 = kh − pk2 − 2 Re hλf, h − pi + |λ|2 kfk2 = kh − pk2 − 2 Re λhf, ei + |λ|2 kfk2. Therefore, ∀ λ ∈ F, 2 Re λhf, ei ≤ |λ|2 kfk2. If we consider λ = t > 0, then we can divide through by t to get ∀ t > 0, 2 Re hf, ei ≤ t kfk2. Letting t → 0+, we conclude that Re hf, ei ≤ 0. Similarly, taking λ = t < 0 and letting t → 0−, we obtain Re hf, ei ≥ 0. If F = R then we are done. If F = C, then we take λ = it with t > 0 and λ = it with t < 0 to show that Im hf, ei = 0 as well. (b) ⇒ (a). Suppose that h = p + e where p ∈ M and e ⊥ M. Choose any f ∈ M. Then p − f ∈ M, so h − p = e ⊥ p − f. Therefore, by the Pythagorean Theorem, kh − fk2 = k(h − p) + (p − f)k2 = kh − pk2 + kp − fk2 ≥ kh − pk2. Hence p is the point in M that is closest to h.

Definition 2.10 (Orthogonal Projection). Let M ≤ H. (a) For each h ∈ H, the point p ∈ M closest to H is called the orthogonal projection of h onto M. (b) Define P : H → H by P h = p, the orthogonal projection of h onto M. Then the operator P is called the orthogonal projection of H onto M. Note that by definition and Theorem 2.9, h = P h + e where e = h − P h ⊥ M. 12 CHRISTOPHER HEIL

Deﬁnition 2.11 (Orthogonal Complement). Let A be a subset (not necessarily a subspace) of a Hilbert space H. The orthogonal complement of A is A⊥ = {x ∈ H : x ⊥ A} = {x ∈ H : hx, yi = 0 ∀ y ∈ A}.

Exercise 2.12. (a) If A ⊆ H, then A⊥ is a closed subspace of H (even if A itself is not). (b) {0}⊥ = H and H⊥ = {0}.

Definition 2.13 (Notation for Operators). Let X, Y be normed linear spaces. Let T : X → Y be a function (= operator = transformation). We write either T (x) or T x to denote the image of an element x ∈ X. (a) T is linear if T (αx + βy) = αT (x) + βT (y) for every x, y ∈ X and α, β ∈ F. (b) T is injective if T (x) = T (y) implies x = y. (c) The kernel or nullspace of T is ker(T ) = {x ∈ X : T (x) = 0}. (c) The range of T is range(T ) = {T (x) : x ∈ X}. (d) The rank of T is the dimension of its range, i.e., rank(T ) = dim(range(T )). In particular, T is finite-rank if range(T ) is finite-dimensional. (d) T is surjective if range(T ) = Y . (e) T is a bijection if it is both injective and surjective.

(f) We use the notation I or IH to denote the identity map of a Hilbert space H onto itself.

Exercise 2.14. Show that if T : X → Y is linear and continuous, then ker(T ) is a closed subspace of X and that range(T ) is a subspace of Y . Must range(T ) be a closed subspace?

Theorem 2.15 (Properties of Orthogonal Projections). Let M ≤ H, and let P be the orthogonal projection of H onto M. Then the following statements hold. (a) h − P h ⊥ M for every h ∈ H. (b) h − P h ∈ M ⊥ for every h ∈ H. (c) kh − P hk = dist(h, M) for every h ∈ H. (d) P is a linear transformation of H into itself. (e) kP hk ≤ khk for every h ∈ H. (f) P is idempotent, i.e., P 2 = P . (g) ker(P ) = M ⊥. (h) range(P ) = M. (i) I − P is the orthogonal projection of H onto M ⊥. CHAPTER 1. HILBERT SPACES 13

Proof. (a) Follows from the deﬁnition of orthogonal projection and Theorem 2.9. (b) Follows from (a) and the deﬁnition of M ⊥. (e) Choose any h ∈ H. Then h = P h + e with P h ∈ M and e ∈ M ⊥. Hence P h ⊥ e, so by the Pythagorean Theorem, khk2 = kP hk2 + kek2 ≥ kP hk2. (g) Suppose that h ∈ ker(P ), i.e., P h = 0. Then h = h − P h ⊥ M, so h ∈ M ⊥. Conversely, suppose that h ∈ M ⊥. Then h = 0 + h with 0 ∈ M and h ∈ M ⊥, so we must have P h = 0 and hence h ∈ ker(P ). The remaining parts are left as exercises.

Corollary 2.16 (Double Complements). If M ≤ H, then (M ⊥)⊥ = M. Proof. Choose any x ∈ M. Then hx, yi = 0 for every y ∈ M ⊥. Thus x ⊥ M ⊥, so x ∈ (M ⊥)⊥. Conversely, suppose that x ∈ (M ⊥)⊥. Since M is closed, we can write x = p + e with p ∈ M and e ∈ M ⊥. Since x ∈ (M ⊥)⊥ we have hx, ei = 0. But we also have p ∈ M ⊆ (M ⊥)⊥, so he, pi = 0. Therefore kek2 = he, ei = hx − p, ei = hx, ei − hp, ei = 0. Hence e = 0, so x = p ∈ M.

Definition 2.17 (Span, Closed Span). Let A be a subset of a normed linear space X. (a) The finite span (or linear span or just the span) of A, denoted span(A), is the set of all finite linear combinations of elements of A: n

span(A) = αkxk : n > 0, xk ∈ A, αk ∈ F . Xk=1 ∞ In particular, if A is a countable sequence, say A = {xk}k=1, then n ∞ span({xk}k=1) = αkxk : n > 0, αk ∈ F . Xk=1 (b) The closed ﬁnite span (or closed linear span or closed span) of A, denoted span(A) or ∨A, is the closure of the set of all ﬁnite linear combinations of elements of A:

∨A = span(A) = span(A) = {z ∈ X : ∃ yn ∈ span(A) such that yn → z}. Beware: This does NOT say that ∞

span(A) = αkxk : xk ∈ A, αk ∈ F . k=1 X ∞ It does not even imply that every element of span(A) can be written x = k=1 αkxk for some xk ∈ A, αk ∈ F. What does it say about span(A)? If we consider the case ∞ P of a countable sequence A = {xk}k=1, then we have n ∞ span({xk}k=1) = z ∈ X : ∃ αk,n ∈ F such that αk,nxk → z as n → ∞ . n Xk=1 o 14 CHRISTOPHER HEIL

That is, an element z lies in the closed span if there exist αk,n ∈ F such that n

αk,nxk → z as n → ∞. k=1 X ∞ In contrast, to say that x = k=1 αkxk means that n P αkxk → x as n → ∞, Xk=1 i.e., the scalars αk must be independent of n.

(c) If X is a Banach space then we say that a subset A ⊆ X is complete if span(A) is dense in X, or equivalently, if span(A) = X.

Beware of the two diﬀerent meanings that we have assigned to the word complete!

∞ 2 ∞ Exercise 2.18. Let {en}n=1 be the standard basis for ` (N). Prove that c00 = span({en}n=1) ∞ 2 and conclude that {en}n=1 is complete in ` (N).

Exercise 2.19. Let A be any subset of a Hilbert space H. (a) (A⊥)⊥ = span(A). (b) A is complete if and only if A⊥ = {0}.

Note that, by deﬁnition, A⊥ = {0} is equivalent to the statement that x ⊥ A =⇒ x = 0. Thus, this exercise says that A is complete if and only if the only element orthogonal to every element of A is 0.

3. The Riesz Representation Theorem

Deﬁnition 3.1 (Continuous and Bounded Operators). Let X, Y be normed linear spaces, and let L: X → Y be a linear operator.

(a) L is continuous at a point x ∈ X if xn → x in X implies Lxn → Lx in Y .

(b) L is continuous if it is continuous at every point, i.e., if xn → x in X implies Lxn → Lx in Y for every x. (c) L is bounded if there exists a ﬁnite K ≥ 0 such that ∀ x ∈ X, kLxk ≤ K kxk. Note that kLxk is the norm of Lx in Y , while kxk is the norm of x in X. (d) The operator norm of L is kLk = sup kLxk. kxk=1 CHAPTER 1. HILBERT SPACES 15

(e) We let B(X, Y ) denote the set of all bounded linear operators mapping X into Y , i.e., B(X, Y ) = {L: X → Y : L is bounded and linear}. If X = Y = X then we write B(X) = B(X, X). (f) If Y = F then we say that L is a functional. The set of all bounded linear functionals on X is the dual space of X, and is denoted X0 = B(X, F) = {L: X → F : L is bounded and linear}.

Exercise 3.2. Let X, X, Y be normed linear spaces. Let L: X → Y be a linear operator.

(a) L is injective if and only if ker L = {0}. (b) L is bounded if and only if kLk < ∞. (c) If L is bounded then kLxk ≤ kLk kxk for every x ∈ X (note that three diﬀerent meanings of the symbol k · k appear in this statement!). (d) If L is bounded then kLk is the smallest value of K such that kLxk ≤ Kkxk holds for all x ∈ X. kLxk (e) kLk = sup kLxk = sup . kxk≤1 x6=0 kxk (f) B(X, Y ) is a subspace of the vector space V containing ALL functions A: X → Y . Moreover, the operator norm is a norm on the space B(X, Y ), i.e., i. kLk ≥ 0 for all L ∈ B(X, Y ). ii. kLk = 0 if and only if L = 0 (the zero operator that sends every element of X to the zero vector in Y ). iii. kαLk = |α| kLk for every L ∈ B(X, Y ) and every α ∈ F. iv. kL + Kk ≤ kLk + kKk for every L, K ∈ B(X, Y ). (g) Part (f) shows that B(X, Y ) is itself a normed linear space, and we will see later that it is a Banach space if Y is a Banach space. However, in addition to vector addition and scalar multiplication operations, there is a third operation that we can perform with functions: composition. Prove that the operator norm is submultiplicative, i.e., if A ∈ B(X, Y ) and B ∈ B(Y, Z), then BA ∈ B(X, Z), and furthermore, kBAk ≤ kBk kAk. (3.1) In particular, when X = Y = Z, we see that B(X) is closed under composition. The space B(X) is an example of an algebra.

(h) If L ∈ X0 = B(X, F) then kLk = sup |Lx|. kxk=1

Exercise 3.3. Let M ≤ H, and let P be the operator of orthogonal projection onto M. Find kP k. 16 CHRISTOPHER HEIL

Exercise 3.4. Let X, Y be normed linear spaces. Prove that if L: X → Y is linear and X is finite-dimensional (take Y = Cn or Rn if you like), then L is bounded. Hint: If n x = (x1, . . . , xn) ∈ C then x = x1e1 + · · · + xnen where {e1, . . . , en} is the standard basis for Cn. Use the Triangle Inequality. If X is any finite-dimensional vector space, do the same but with an arbitrary basis for X, and use the fact (that we will prove later) that all norms on a finite-dimensional space are equivalent.

The next lemma is a standard fact about continuous functions. L−1(U) denotes the inverse image of U ⊆ Y , i.e., L−1(U) = {x ∈ X : Lx ∈ U}.

Lemma 3.5. Let X, Y be normed linear spaces. Let L: X → Y be linear. Then L is continuous ⇐⇒ U open in Y =⇒ L−1(U) open in X . Proof. ⇒. Suppose that L is continuous and that U is an open subset of Y . We will show that X \ L−1(U) is closed by showing that it contains all its limit points. −1 −1 Suppose that x is a limit point of X \L (U). Then there exist xn ∈ X \L (U) such that −1 xn → x. Since L is continuous, this implies Lxn → Lx. However, xn ∈/ L (U), so Lxn ∈/ U, −1 i.e., Lxn ∈ Y \ U, which is a closed set. Therefore Lx ∈ Y \ U, and hence x ∈ X \ L (U). Thus X \ L−1(U) is closed, so L−1(U) is open. ⇐. Exercise.

Theorem 3.6 (Equivalence of Bounded and Continuous Linear Operators). Let X, Y be normed linear spaces, and let L: X → Y be a linear mapping. Then the following statements are equivalent. (a) L is continuous at some x ∈ X. (b) L is continuous at x = 0. (c) L is continuous. (d) L is bounded. Proof. (c) ⇒ (d). Suppose that L is continuous but unbounded. Then kLk = ∞, so there must exist xn ∈ X with kxnk = 1 such that kLxnk ≥ n. Set yn = xn/n. Then kyn − 0k = kynk = kxnk/n → 0, so yn → 0. Since L is continuous and linear, this implies Lyn → L0 = 0, and therefore kLynk → k0k = 0. But 1 1 kLy k = kLx k ≥ · n = 1 n n n n for all n, which is a contradiction. Hence L must be bounded. (d) ⇒ (c). Suppose that L is bounded, so kLk < ∞. Suppose that x ∈ X and that xn → x. Then kxn − xk → 0, so

kLxn − Lxk = kL(xn − x)k ≤ kLk kxn − xk → 0, i.e., Lxn → Lx. Thus L is continuous. The remaining implications are left as exercises. CHAPTER 1. HILBERT SPACES 17

Exercise 3.7. Let X, Y be normed linear spaces. Suppose that L: X → Y is linear and satisﬁes kLxk = kxk for all x ∈ X. Such an operator is said to be an isometry or norm- preserving. Prove that L is injective and ﬁnd kLk. Find an example of an isometry that is not surjective. Contrast this with the fact that if A: Cn → Cn is linear, then A is injective if and only if it is surjective.

2 2 Exercise 3.8. (a) Define L: ` (N) → ` (N) by L(x) = (x2, x3, . . . ) for x = (x1, x2, . . . ) ∈ `2(N). Prove that this left-shift operator is bounded, linear, surjective, not injective, and is not an isometry. Find kLk. 2 2 2 (b) Define R: ` (N) → ` (N) by R(x) = (0, x1, x2, x3, . . . ) for x = (x1, x2, . . . ) ∈ ` (N). Prove that this right-shift operator is bounded, linear, injective, not surjective, and is an isometry. Find kRk. (c) Compute LR and RL. Contrast this computation with the fact that in finite dimensions, if A, B : Cn → Cn are linear maps (hence correspond to multiplication by n × n matrices), then AB = I implies BA = I and conversely.

Exercise 3.9. Let X be a Banach space and Y a normed linear space. Suppose that L: X → Y is bounded and linear. Prove that if there exists c > 0 such that kLxk ≥ ckxk for all x ∈ X, then L is injective and range(L) is closed.

Exercise 3.10. For each h ∈ H, deﬁne a linear functional Lh : H → F by

Lh(x) = hx, hi, x ∈ H. Prove the following.

(a) Lh is linear. ⊥ (b) ker(Lh) = {h} . 0 (c) kLhk = khk, so Lh is a bounded linear functional on H. Therefore Lh ∈ H = B(H, F).

0 Thus, each element h of H determines an element Lh of H . In the Riesz Representation Theorem we will prove that these are all the elements of H 0.

n n T Example 3.11 (Dual Space of C ). Consider H = C (with F = C). Let h = (h1, . . . , hn) ∈ H = Cn (think of h as a column vector). Then Lh(x) = x · h = x1h¯1 + · · · + xnh¯n deﬁnes a bounded linear functional on Cn. Suppose that L: Cn → C is any linear functional on Cn (it will necessarily be bounded n m since Cn is ﬁnite-dimensional, see Exercise 3.4). Any linear mapping from C to C is given by multiplication by an m × n matrix. So in this case there must be a 1 × m matrix (i.e., a row vector) A = a1 · · · an such that

x1 Lx = Ax = a · · · a . = x a + · · · + x a = x · h = L (x) 1 n  .  1 1 n n h x n   18 CHRISTOPHER HEIL

T where h = (a¯1, . . . , a¯n) . n 0 n Thus we see that every element of (C ) is an Lh for some h ∈ C .

Exercise 3.12. Deﬁne T : Cn → (Cn)0 by n T (h) = Lh, h ∈ C . Prove directly that T is anti-linear, i.e., T (αh + βk) = α¯T (h) + β¯T (k), and that T is an isometry, i.e., kT (h)k = khk for each h. All isometries are injective, and in the preceding example we showed that T is surjective, so T is an anti-linear isometric bijection of Cn onto (Cn)0. n n 0 n If we deﬁne S : C → (C ) by S(h) = Lh¯ , then S a linear isometric bijection of C onto (Cn)0. Such an S is called an isomorphism, and thus Cn and (Cn)0 are isomorphic.

Theorem 3.13 (Riesz Representation Theorem). For each h ∈ H, deﬁne Lh : H → F by

Lh(x) = hx, hi, x ∈ H.

0 (a) For each h ∈ H, we have Lh ∈ H and kLhk = khk. 0 (b) If L ∈ H , then there exists a unique h ∈ H such that L = Lh. 0 (c) The mapping T : H → H deﬁned by T (h) = Lh is an anti-linear isometric bijection of H onto H0. Proof. (a) See Exercise 3.10. (b) Choose any L ∈ H 0. If L = 0 then h = 0 is the required vector, so assume that L =6 0 (i.e., L is not the zero operator). Then L does not map every vector to zero, so the kernel of L is a closed subspace of H that is not all of H. Choose any z ∈/ ker(L), and write z = p + e, p ∈ ker(L), e ∈ ker(L)⊥. Note that L(p) = 0 by deﬁnition, and therefore we must have L(e) =6 0 (since L(z) =6 0). Set u = e/L(e), and note that u ∈ ker(L)⊥ and L(u) = 1. Given x ∈ H, since L is linear and L(u) = 1 we have L x − L(x) u = L(x) − L(x) L(u) = 0. Therefore x − L(x) u ∈ ker(L). However, u ⊥ ker(L), so 0 = hx − L(x) u, ui = hx, ui − hL(x) u, ui = hx, ui − L(x) kuk2. Hence, 1 u L(x) = hx, ui = x, = hx, hi = L (x) kuk2 kuk2 h where D E u h = . kuk2 CHAPTER 1. HILBERT SPACES 19

Thus L = Lh, and from part (a), we have that kLk = kLhk = khk. It remains only to show that h is unique. Suppose that we also had L = Lh0 . Then for every x ∈ H we have 0 0 hx, h − h i = hx, hi − hx, h i = Lh(x) − Lh0 (x) = L(x) − L(x) = 0. Consequently, h − h0 = 0. (c) Parts (a) and (b) show that T is surjective and that T is norm-preserving. Therefore, we just have to show that T is anti-linear. Let h ∈ H and let c ∈ F. We must show that T (ch) = cT¯ (h), i.e., that Lch = cL¯ h. This follows immediately from the fact that for each x ∈ H, we have

Lch(x) = hx, chi = c¯hx, hi = c¯Lh(x).

The proof that Lh+k = T (h + k) = T (h) + T (k) = Lh + Lk is left as an exercise.

Corollary 3.14. If L: `2(I) → F is a bounded linear functional, then there exists a unique 2 h = (hi)i∈I ∈ ` (I) such that ¯ 2 L(x) = xi hi, x = (xk)i∈I ∈ ` (I). i∈I X

Corollary 3.15. If (X, Ω, µ) is a measure space and L: L2(X) → F is a bounded linear functional, then there exists a unique h ∈ L2(X) such that

L(f) = f(x) h(x) dµ(x), f ∈ L2(X). ZX

4. Orthonormal Sets of Vectors and Bases

Definition 4.1 (Hamel Basis). Let V be a vector space. We say that {fi}i∈I is a basis for V if it is both finitely linearly independent and its finite span is all of V . That is, every vector in V equals a unique finite linear combination of the fi (aside from trivial zero terms), N i.e., every f =6 0 can be written f = k=1 ckfik for a unique choice of indices i1, . . . , iN and nonzero scalars c1, . . . , cN . Because the word “basis” is heavily overused, we shall refer to such a basis as a Hamel basis or a vePctor space basis. For most vector spaces, Hamel bases are only known to exist because of the Axiom of Choice; in fact, the statement “Every vector space has a Hamel basis” is equivalent to the Axiom of Choice. As in finite dimensions, it can be shown that any two Hamel bases for a vector space V must have the same cardinality. This cardinality is called the (vector space) dimension of V .

Example 4.2 (Existence of Unbounded Linear Functionals). We can use the Axiom of Choice to show that there exist unbounded linear functionals L: X → F whenever X is an inﬁnite-dimensional normed linear space. Let {fi}i∈I be a Hamel basis for an inﬁnite-dimensional normed linear space X, normalized so that kfik = 1 for every i ∈ I. Let J0 = {j1, j2, . . . } be any countable subsequence of I. 20 CHRISTOPHER HEIL

Define L: X → C by setting L(fjn ) = n for n ∈ N and L(fi) = 0 for i ∈ I\J0. Then extend N L linearly to all of X: if f = k=1 ckfik is the unique representation of f using nonzero scalars c , . . . , c , then define L(f) = N c L(f ). This L is a linear functional on X, but 1 N P k=1 k ik since kfjn k = 1 yet |L(fjn )| = n, we have kLk = supkfk=1 |L(f)| = ∞. Thus, if L: X → Y and Y is finite-dimensional,P we cannot conclude that L must necessarily be bounded. Contrast this with the fact that if L: X → Y and X is finite-dimensional, then L must be bounded (see Exercise 3.4).

Remark 4.3. Aside from the preceding result, Hamel bases are not going to be much use 2 2 to us. For example, consider the space ` = ` (N), and let {en}n∈N be the standard basis 2 2 for ` . Note that {en}n∈N is NOT a Hamel basis for ` ! Its finite linear span is the proper 2 subspace c00. Thus {en}n∈N is a Hamel basis for c00, but not for ` . In fact, a Hamel basis for an infinite-dimensional Hilbert or Banach space must be uncountable. The main point is that since we have a norm, there is no reason to restrict to only finite linear combinations. Given a sequence of vectors {fi}i∈N and scalars (ci)i∈N, we can consider “infinite linear combinations” ∞

ci fi. i=1 X HOWEVER, we must be very careful about convergence issues: the series above will not converge for EVERY choice of scalars ci. On the other hand, if our sequence {fi} is orthogonal, or even better yet, orthonormal, then we will see that the convergence issues become easy to deal with, and we can make sense of what it means to have a “basis” which allows “infinite linear combinations.” HOWEVER, there still remains an issue of how many vectors we need in this “basis”—for some Hilbert spaces (the separable Hilbert spaces) we will be able to use a basis that requires only countably many vectors, as was implicitly assumed in the discussion in the preceding paragraph. But for others, a “basis” may need uncountably many vectors, even allowing “infinite (but still countable) linear combinations.” In some fields (e.g., the geometry of Banach spaces literature), it is customary to only use the word basis for the case of countable bases. Conway does not follow this custom, which is fine as long as you realize that other authors use the term “basis” differently. Conway’s definition of a basis for a Hilbert space follows. The definition does not seem to say anything about linear combinations, infinite or otherwise—we shall see the connection later. To emphasize that Conway’s definition is not universally accepted, I will refer to his definition of basis as a Conway basis.

Deﬁnition 4.4 (Conway Basis). Let H be a Hilbert space. Then a Conway basis for H is a maximal orthonormal set. That is, a subset E ⊆ H is a Conway basis if (a) E is orthonormal, and (b) there does not exist an orthonormal set E0 ⊆ H such that E ⊆ E0. CHAPTER 1. HILBERT SPACES 21

Exercise 4.5. Zorn’s Lemma is an equivalent formulation of the Axiom of Choice. Learn what Zorn’s Lemma says, and use it to prove that every Hilbert space has a Conway basis. Then ask all your friends the classic riddle: What is yellow and equivalent to the Axiom of Choice?1 That one is nearly as good as the classic: What is purple and commutes?2

2 Exercise 4.6. Show that {en}n∈N is a Conway basis for ` (N).

Exercise 4.7. Let I be any set. Deﬁne `2(I) to be the set of all functions x: I → F such that x(i) =6 0 for at most a countable number of i and such that kxk2 = |x(i)|2 < ∞ i∈I X (there are at most countably many nonzero terms in the above sum, so this series with a possibly uncountable index set just means to sum the countably many nonzero terms). Then `2(I) is a Hilbert space with respect to the inner product hx, yi = x(i) y(i). i∈I X For each i ∈ I, deﬁne ei : I → F by 1, i = j, ei(j) = δij = (0, i =6 j. 2 2 Then {ei}i∈I is called the standard basis for ` (I). Prove that it is a Conway basis for ` (I).

Instead of “Conway basis,” the following gives us a terminology that does not require us to use the word basis.

Lemma 4.8. Let {fi}i∈I be a sequence of elements of H. Then:

{fi}i∈I is orthonormal and complete ⇐⇒ {fi}i∈I is a Conway basis.

Proof. ⇒. Suppose that {fi}i∈I is orthonormal and complete. Then by Exercise 2.19, there is no nonzero g ∈ H that is orthogonal to every fi. Therefore there can be no orthonormal set that properly contains {fi}i∈I. ⇐. Exercise.

Deﬁnition 4.9 (Schauder basis). Now we give the “correct” deﬁnition of a basis. Let X be a Banach space. Then a (countable!) sequence {fi}i∈N is a Schauder basis, or just a basis, for X if for each f ∈ X there exist unique scalars {ci}i∈N such that ∞

f = cifi. i=1 X 1Zorn’s Lemon! 2An abelian grape. 22 CHRISTOPHER HEIL

N Note in particular that this means that i=1 cifi → f as N → ∞. To be a little more precise, the deﬁnition of Schauder basis really includes the requirement P ∞ that if we let ci(f) denote the unique scalars such that f = i=1 ci(f)fi, then the mapping f 7→ ci(f) must be a continuous linear functional on X for each i. However, the Strong Basis Theorem proves that this is automatically true when XPis a Banach space, so we have omitted that requirement in our deﬁnition.

2 Exercise 4.10. The standard basis {en}n∈N is an orthonormal Schauder basis for ` (N).

Exercise 4.11. (See also Exercise 10 on p. 98 of Conway.) Let {fi}i∈N be a Schauder basis for a Banach space X.

(a) Prove that {fi}i∈N is complete in X, i.e., that the set of ﬁnite linear combinations is dense in X. Note: The converse is false, i.e., a complete sequence need not be a Schauder basis (see Example 4.18). On the other hand, we will see that in a separable Hilbert space, a complete orthonormal sequence IS a Schauder basis. (b) Let N

S = rifi : N > 0, Re (ri), Im (ri) ∈ Q . i=1 Prove that S is a countable, denseX subset of X. A Banach space which has a countable, dense subset is said to be separable.

Remark 4.12. Thus, if a Banach space has a Schauder basis, then it must be separable. Does every separable Banach space have a Schauder basis? This was a longstanding open problem in Banach space theory, called the Basis Problem. It was settled by Enﬂo in 1973: there exist separable, reﬂexive Banach spaces which do not possess any Schauder bases.

Exercise 4.13. Let X be a Banach space. Suppose there exists an uncountable sequence {xi}i∈I of elements of X such that kxi − xjk ≥ ε > 0 for all i =6 j ∈ I. Prove that X is not separable. Exercise 4.14. Use the preceding exercise to prove the following. (a) Prove that `∞(N) is not separable. (b) Prove that L∞(R) is not separable. (c) Prove that if I is uncountable then `2(I) is not separable. More generally, any Hilbert space which contains an uncountable orthonormal sequence is not separable.

We will show that, in a separable Hilbert space, any complete orthonormal sequence is a Schauder basis for H. If H is nonseparable, something very similar happens; we just have to be careful with what we mean by inﬁnite series with uncountably many terms (only countably many terms can be nonzero). First, however, we will present the results for separable Hilbert spaces. CHAPTER 1. HILBERT SPACES 23

Theorem 4.15. Let {en}n∈N be any orthonormal sequence in a Hilbert space H. Then the following statements hold. ∞ 2 2 (a) Bessel’s Inequality: |hf, eni| ≤ kfk . n=1 ∞ X

(b) If f = cnen converges, then cn = hf, eni. n=1 ∞ X ∞ 2 (c) cnen converges ⇐⇒ |cn| < ∞. n=1 n=1 X X ∞

(d) If cnen converges then it converges unconditionally, i.e., it converges regardless of n=1 theXordering of the indices. ∞

(e) f ∈ span({en}n∈N) ⇐⇒ f = hf, eni en. n=1 X (f) If f ∈ H, then ∞

P f = hf, eni en n=1 X is the orthogonal projection of f onto span({en}n∈N). Proof. (a) Choose f ∈ H. For each N deﬁne N

fN = f − hf, eni en. n=1 X Show that fN ⊥ e1, . . . , eN (exercise). Therefore, by the Pythagorean Theorem, we have N N N 2 2 2 2 2 kfk = fN + hf, eni en = kfN k + khf, eni enk ≥ |hf, eni| . n=1 n=1 n=1 X X X Since this is true for each ﬁnite n, Bessel’s inequality follows. (b) Exercise. ∞ 2 (c) ⇐. Suppose that n=1 |cn| < ∞. Set N N P 2 sN = cnen, tN = |cn| . n=1 n=1 X X We know that {tN } is a convergent (hence Cauchy) sequence of scalars, and we must show that {sN } is a convergent sequence of vectors. We have for N > M that N N N 2 2 2 2 ksN − sM k = cnen = kcnenk = |cn| = |tN − tM |. n=M+1 n=M+1 n=M+1 X X X

24 CHRISTOPHER HEIL

Since {tN } is Cauchy, we conclude that {sN } is Cauchy and hence converges. ⇒. Exercise. (d), (e) Exercise. (f) By Bessel’s inequality and part (c), we know that the series deﬁning P f converges, so we just have to show that it is the orthogonal projection of f onto span({en}). Check that hf − P f, eni = 0 for every n (exercise). By linearity, conclude that f − P f ⊥ span({en}). By continuity of the inner product, conclude that f − P f ⊥ span({en}) (exercise).

Exercise 4.16. Use Theorem 4.15 to construct an example of a series cnen in a Hilbert space that converges unconditionally but not absolutely (compare Exercise 1.26). P The next theorem shows that any countable orthonormal sequence that is complete must be a Schauder basis, and vice versa. Such an orthonormal Schauder basis is usually just called an orthonormal basis (or just ONB) for H.

Theorem 4.17. Let {en}n∈N be any orthonormal sequence in a Hilbert space H. Then the following statements are equivalent.

(a) {en}n∈N is a Schauder basis for H, i.e., for each f ∈ H there exist unique scalars ∞ (cn)n∈N such that f = n=1 cnen. ∞P (b) For each f ∈ H, f = hf, eni en. n=1 X (c) {en}n∈N is complete (i.e, it is a Conway basis for H).

∞ 2 2 (d) Plancherel/Parseval Equality: For each f ∈ H, kfk = |hf, eni| . n=1 X ∞

(e) Parseval/Plancherel Equality: For each f, g ∈ H, hf, gi = hf, eni hen, gi. n=1 X Proof. (b) ⇒ (e). Suppose that (b) holds, and choose any f, g ∈ H. Then

∞ ∞ ∞

hf, gi = hf, eni en, g = hf, eni en, g = hf, eni hen, gi, n=1 n=1 n=1 DX E X X where we have used Exercise 1.24 to move the inﬁnite series outside of the inner product.

(c) ⇒ (b). If {en} is complete, then its closed span is all of H, so by Theorem 4.15(e) we ∞ have f = n=1hf, eni en for every f ∈ H. P CHAPTER 1. HILBERT SPACES 25

Example 4.18. Let {en}n∈N be an orthonormal basis for a separable Hilbert space H. Deﬁne fn = e1 + en/n. Then {fn}n∈N is complete (exercise) but it is not a Schauder basis for H (harder).

Exercise 4.19. Let H be a Hilbert space. Show that H is separable if and only if there exists a countable sequence {xn}n∈N that is an orthonormal basis for H. Hint: By Exercise 4.5 we know that there exists a complete orthonormal sequence for H.

We state without proof the facts about complete orthonormal systems in nonseparable Hilbert spaces.

Theorem 4.20. Let {ei}i∈I be an orthonormal sequence in a Hilbert space H (note that I might be uncountable). Then the following statements hold.

(a) If f ∈ H then hf, eii =6 0 for at most countably many i.

2 2 (b) For each f ∈ H, |hf, eii| ≤ kfk . Xi∈I (c) For each f ∈ H, hf, eii ei converges with respect to the net of ﬁnite subsets of I i∈I (see Proposition 1.27X for the meaning of net of ﬁnite subsets).

Theorem 4.21. Let {ei}i∈I be an orthonormal sequence in a Hilbert space H. Then the following statements are equivalent.

(a) {ei}i∈I is complete (i.e., is a Conway basis for H).

(b) For each f ∈ H, f = hf, eii ei with respect to the net of ﬁnite subsets of I. i∈I X 2 2 (c) For each f ∈ H, kfk = |hf, eii| (only countably many terms are nonzero). Xi∈I 26 CHRISTOPHER HEIL

The (Hilbert space) dimension of H is the cardinality of a complete orthonormal sequence. It can be shown that any two complete orthonormal sequences must have the same cardinality. Further, H is separable if and only if it has a countable complete orthonormal sequence (which by Theorem 4.17 will be an orthonormal Schauder basis for H).

5. Isomorphic Hilbert Spaces and the Fourier Transform for the Circle

Deﬁnition 5.1. If H1, H2 are Hilbert spaces, then L: H1 → H2 is an isomorphism if L is a linear bijection that satisﬁes

∀ f, g ∈ H1, hLf, Lgi = hf, gi. ∼ In this case we say that H1 and H2 are isomorphic, and write H1 = H2. If H1 = H2 = H and L: H → H is an isomorphism, then we say that L is unitary.

Proposition 5.2. Let L: H1 → H2 be a linear mapping. Then the following statements are equivalent.

(a) L is inner-product-preserving, i.e., hLf, Lgi = hf, gi for all f, g ∈ H1.

(b) L is norm-preserving (an isometry), i.e., kLfk = kfk for all f ∈ H1. Proof. We need only show that (b) implies (a). Assume that L is an isometry, and ﬁx f, g ∈ H. Then for any scalar λ ∈ F we have by the Polar Identity and the fact that L is isometric that kfk2 + 2 Re λ¯ hf, gi + |λ|2 kgk2 = kf + λgk2 = kLf + λLgk2 = kLfk2 + 2 Re λ¯ hLf, Lgi + |λ|2 kLgk2 = kfk2 + 2 Re λ¯ hLf, Lgi + |λ|2 kgk2. Thus Re λ¯ hLf, Lgi = Re λ¯ hf, gi for every λ ∈ F. This implies (take λ = 1 and λ = i) that hLf, Lgi = hf, gi.

In particular, every isomorphism is an isometry and hence is automatically injective.

Corollary 5.3. Let L: H1 → H2 be given. Then L is an isomorphism if and only if it is a linear surjective isometry.

Theorem 5.4. All separable inﬁnite-dimensional Hilbert spaces are isomorphic.

Proof. Let H1 and H2 be separable inﬁnite-dimensional Hilbert spaces. Then H1 has a countable orthonormal basis {en}n∈N, and H2 has a countable orthonormal basis {hn}n∈N. ∞ 2 ∞ 2 If f ∈ H1 then by Theorem 4.17 we have f = n=1hf, eni en and kfk = n=1 |hf, eni| . ∞ Since this is ﬁnite and since {hn} is an ONB, the series Lf = n=1hf, eni hn converges, and 2 ∞ 2 2 P P kLfk = n=1 |hf, eni| = kfk . Exercise: Finish the proof by showing that L is a linear surjectivP e isometry. P CHAPTER 1. HILBERT SPACES 27

In particular, every separable Hilbert space is isomorphic to `2(N). The proof can be extended to show that any two Hilbert spaces which have the same dimension are isomorphic.

Example 5.5 (Fourier Series). Now we give one of the most important examples of an orthonormal basis. 2πinx We saw in Exercise 2.2 that if we deﬁne en(x) = e , then {en}n∈Z is an orthonormal sequence in L2[0, 1] (the space of square-integrable complex-valued functions on the domain [0, 1], or we can consider these functions to be 1-periodic functions on the domain R). 2 It is a fact that {en}n∈Z is actually complete in L [0, 1] and hence is an orthonormal basis for L2[0, 1]. The Fourier coeﬃcients of f ∈ L2[0, 1] are 1 −2πinx fˆ(n) = hf, eni = f(x) e dx, n ∈ Z. Z0 By Theorem 4.17, we have ˆ f = hf, eni en = f(n) en (5.1) Z Z Xn∈ Xn∈ (the series converges unconditionally, so it does not matter what ordering of the index set Z that we use to sum with). Equation 5.1 is called the Fourier series of f. However, note that the series in 5.1 converges in the norm of the Hilbert space, i.e., in L2-norm. That is, the partial sums converge in L2-norm, i.e.,

lim f − fˆ(n) en = 0, N→∞ 2 n=−N X or, 2 1 N lim f(x) − fˆ(n) e2πinx dx = 0. N→∞ Z0 n=−N X We cannot conclude from this that the equality

f(x) = fˆ(n) e2πinx Z Xn∈ holds pointwise. In fact, one of the deepest results of Fourier series is the Carleson–Hunt Theorem, which proves the conjecture of Lusin that if f ∈ Lp[0, 1] where 1 < p ≤ ∞, then the Fourier series of f converges to f a.e.

2 2 Exercise 5.6. Show that the mapping F : L [0, 1] → ` (Z) given by F(f) = fˆ = {fˆ(n)}n∈Z 2 is exactly the isomorphism constructed by Theorem 5.4 for the case H1 = L [0, 1] and 2 H2 = ` (Z). The operator F is the Fourier transform for the circle (thinking of functions in L2[0, 1] as being 1-periodic, the domain [0, 1] is topologically a circle). 28 CHRISTOPHER HEIL

Exercise 5.7. Prove the following (easy) special case of the Riemann–Lebesgue Lemma: If f ∈ L2[0, 1] then fˆ(n) → 0 as |n| → ∞. The full Riemann–Lebesgue Lemma for the circle states the same conclusion holds if we only assume that f ∈ L1[0, 1].

Deﬁnition 5.8. In honor of Fourier series, if {en}n∈I is any orthonormal basis of a separable Hilbert space H, then {hf, eni}n∈I is the sequence of (generalized) Fourier coeﬃcients of f and f = n∈I hf, eni en is the (generalized) Fourier series of f. P Exercise 5.9. The Plancherel formula with respect to the orthonormal basis {en}n∈Z for L2[0, 1] is 2 ˆ 2 kfk2 = |f(n)| . Z Xn∈ Use the Plancherel formula to derive a formula for π by applying it to the function f = χ[0,1/2) − χ[1/2,1).