LECTURE NOTES

CHAPTER 2. OPERATORS ON HILBERT SPACES

CHRISTOPHER HEIL

1. Elementary Properties and Examples First recall the basic definitions regarding operators.

Definition 1.1 (Continuous and Bounded Operators). Let X, Y be normed linear spaces, and let L: X Y be a linear . → (a) L is continuous at a point f X if f f in X implies Lf Lf in Y . ∈ n → n → (b) L is continuous if it is continuous at every point, i.e., if fn f in X implies Lfn Lf in Y for every f. → → (c) L is bounded if there exists a finite K 0 such that ≥ f X, Lf K f . ∀ ∈ k k ≤ k k Note that Lf is the of Lf in Y , while f is the norm of f in X. k k k k (d) The of L is L = sup Lf . k k kfk=1 k k

(e) We let (X, Y ) denote the set of all bounded linear operators mapping X into Y , i.e., B (X, Y ) = L: X Y : L is bounded and linear . B { → } If X = Y = X then we write (X) = (X, X). B B (f) If Y = F then we say that L is a functional. The set of all bounded linear functionals on X is the dual of X, and is denoted X0 = (X, F) = L: X F : L is bounded and linear . B { → } We saw in Chapter 1 that, for a linear operator, boundedness and continuity are equivalent. Further, the operator norm is a norm on the space (X, Y ) of all bounded linear operators from X to Y , and we have the composition propertyBthat if L (X, Y ) and K (Y, Z), then KL (X, Z), with KL K L . ∈ B ∈ B ∈ B k k ≤ k k k k Date: February 20, 2006. These notes closely follow and expand on the text by John B. Conway, “A Course in ,” Second Edition, Springer, 1990. 1 2 CHRISTOPHER HEIL

Exercise 1.2. Suppose that L: X Y is a bounded map of a X into a Banach space Y . Prove that if there →exists a c > 0 such that Lf c f for every f X, then range(L) is a closed subspace of Y . k k ≥ k k ∈

n n Exercise 1.3. Let Cb(R ) be the set of all bounded, continuous functions f : R F. Let n n → C0(R ) be the set of all continuous functions f : R F such that lim|x|→∞ f(x) = 0 (i.e., for every ε > 0 there exists a compact set K such that→ f(x) < ε for all x / K). Prove that these are closed subspaces of L∞(Rn) (under the L∞|-norm;| note that for∈a continuous we have f ∞ = sup f(x) ). Define δ : C (Rnk) k F by | | b → δ(f) = f(0). n 0 Prove that δ is a bounded linear functional on Cb(R ), i.e., δ (Cb) , and find δ . This linear functional is the delta distribution (see also Exercise 1.26∈below). k k

Example 1.4. In finite dimensions, all linear operators are given by matrices, this is just standard finite-dimensional linear . Suppose that X is an n-dimensional complex normed and Y is an m- dimensional complex . By definition of , this means that there exists a = x , . . . , x for X and a basis = y , . . . , y for Y . If x X, BX { 1 n} BY { 1 m} ∈ then x = c1x1 + + cnxn for a unique choice of scalars ci. Define the coordinates of x with respect to the basis· · · to be BX c1 . Cn [x]BX = . .   ∈ cn The vector x is completely determined by its coordinates, and conversely each vector in Cn

is the coordinates of a unique x X. The mapping x [x]BX is a linear mapping of X onto Cn ∈Cm 7→ . We similarly define [y]BY for vectors y Y . Let A: X Y be a ∈ (it is automatically∈ bounded since X is finite-dimensional). Then A transforms→ vectors x X into vectors Ax Y . The vector x is determined by its ∈ ∈ coordinates [x]BX and likewise Ax is determined by its coordinates [Ax]BY . The vectors x and Ax are related through the linear map A; we will show that the coordinate vectors [x]BX and [Ax] are related by multiplication by an m n determined by A. We call this BY × matrix the standard matrix of A with respect to X and Y , and denote it by [A]BX ,BY . That is, the standard matrix should satisfy B B [Ax] = [A] [x] , x X. BY BX ,BY BX ∈ We claim that the standard matrix is the matrix whose columns are the coordinates of the vectors Axk, i.e.,

[A]B ,B = [Ax1] [Axn]B . X Y  BY · · · Y    CHAPTER 2. OPERATORS ON HILBERT SPACES 3

To see this, choose any x X and let x = c1x1 + + cnxn be its unique representation with respect to the basis ∈ . Then · · · BX c1 . [A]BX ,BY [x]BX = [Ax1]B [Axn]BY .  Y · · ·    cn = c [Ax ] + + c [Ax ]   1 1 BY · · · n n BY = [c Ax + + c Ax ] 1 1 · · · n n BY = [A(c x + + c x )] 1 1 · · · n n BY

= [Ax]BY .

Exercise 1.5. Extend the idea of the preceding example to show that that any linear mapping L: `2(N) `2(N) (and more generally, L: H K with H, K separable) can be realized in terms of→multiplication by an (infinite but coun→table) matrix.

Exercise 1.6. Let A be an m n complex matrix, which we view as a linear transformation A: Cn Cm. The operator×norm of A depends on the choice of norm for Cn and Cm. Compute→an explicit formula for A , in terms of the entries of A, when the norm on Cn and Cm is taken to be the `1 norm. kThenk do the same for the `∞ norm. Compare your results to the version of Schur’s Lemma given in Theorem 1.23.

The following example is one that we will return to many times.

Example 1.7. Let en n∈N be an for a separable H. Then we know that every {f }H can be written ∈ ∞ f = f, e e . h ni n n=1 X Fix any of scalars λ = (λn)n∈N, and formally define ∞ Lf = λ f, e e . (1.1) n h ni n n=1 X This is a “formal” definition because we do not know a priori that the above will converge—in other words, (1.1) may not make sense for every f. 2 Note that if H = ` (N) and e N is the standard basis, then L is given by the formula { n}n∈ Lx = (λ x , λ x , . . . ), x = (x , x , . . . ) `2(N). 1 1 2 2 1 2 ∈ We will show the following (the `∞-norm of the sequence λ is λ = sup λ ). k k∞ n | n| (a) The series defining Lf in (1.1) converges for each f H if and only if λ `∞. In this case L is a bounded linear mapping of H into itself,∈ and L = λ .∈ k k k k∞ 4 CHRISTOPHER HEIL

(b) If λ / `∞, then L defines an unbounded linear mapping from the domain ∈ ∞ domain(L) = f H : λ f, e 2 < (1.2) ∈ | n h ni| ∞ n=1 n X o (which is dense in H) into H. Proof. (a) Suppose that λ `∞, i.e., λ is a bounded sequence. Then for any f we have ∈ ∞ ∞ λ f, e 2 λ 2 f, e 2 = λ 2 f 2 < , | n h ni| ≤ k k∞ h ni| k k∞ k k ∞ n=1 n=1 X X so the series defining Lf converges (because en is an orthonormal sequence). Moreover, the preceding calculation also shows that Lf{ 2}= ∞ λ f, e 2 λ 2 f 2, so we k k n=1 | n h ni| ≤ k k∞ k k see that L λ ∞. On the other hand, by orthonormality we have Len = λnen (i.e., each k k ≤ k k P en is an eigenvector for L with eigenvalue λn). Since en = 1 and Len = λn en = λn we conclude that k k k k | | k k | |

L = sup Lf sup Len = sup λn = λ ∞. k k kfk=1 k k ≥ n∈N k k n∈N | | k k The converse direction will be covered by the proof of part (b). (b) Suppose that λ / `∞, i.e., λ is not a bounded sequence. Then we can find a subsequence ∈ 1 N (λnk )k∈ such that λnk k for each k. Let cnk = k and define all other cn to be zero. Then 2 1 | | ≥ c = 2 < , so f = c e converges (and c = f, e ). But the formal series n | n| k k ∞ n n n n h ni Lf = λncnen does not converge, because P n P P ∞ ∞ ∞ P k2 c λ 2 = c λ 2 = . | n n| | nk nk | ≥ k2 ∞ n=1 X Xk=1 Xk=1 In fact, the series defining Lf in (1.1) only converges for those f which lie in the domain defined in (1.2). That domain is dense because it contains the finite span of en n∈N, which we know is dense in H. Further, that domain is a subspace of H (exercise), so{ it}is an inner- product space. The map L: domain(L) H is a well-defined, linear map, so it remains only → to show that it is unbounded. This follows from the facts that en domain(L), en = 1, and Le = λ e = λ . ∈ k k  k nk | n| k nk | n|

∞ Exercise 1.8. Continuing Example 1.7, suppose that λ ` and set δ = infn λn . Prove the following. ∈ | | (a) L is injective if and only if λ = 0 for every n. n 6 (b) L is surjective if and only if δ > 0 (if δ = 0, use an argument similar to the one used in part (b) of Example 1.7 to show that range(L) is a proper subset of H). (c) If δ = 0 but λ = 0 for every n then range(L) is a dense but proper subspace of H. n 6 (d) Prove that L is unitary if and only if λ = 1 for every n. | n| CHAPTER 2. OPERATORS ON HILBERT SPACES 5

In Example 1.7, we saw an whose domain was a dense but proper subspace of H. This situation is typical for unbounded operators, and we often write L: X Y even when L is only defined on a subset of X, as in the following example. →

Example 1.9 (Differentiation). Consider the Hilbert space H = L2(0, 1), and define an operator D : L2(0, 1) L2(0, 1) by Df = f 0. Implicitly, we mean by this that D is defined on the largest domain→that makes sense, namely, domain(D) = f L2(0, 1) : f is differentiable and f 0 L2(0, 1) . ∈ ∈ 2 Note that if f domain(D), then Df is well-defined, Df L (0, 1), and Df 2 < . Thus every vector∈ in domain(D) maps to a vector in L2(0, 1)∈ which necessarilyk hask finite∞ inx norm. Yet D is unbounded. For example, if we set en(x) = e then en 2 = 1, but 0 inx k k Den(x) = en(x) = ine so Den 2 = n. While each vector Den has finite norm, there is no upper bound to these norms.k Sincek the e are unit vectors, we conclude that D = . n k k ∞ The following definitions recall the basic notions of measures and spaces. For full details, consult a book on .1

Definition 1.10 (σ-, Measurable Sets and Functions). Let X be a set, and let Ω be a collection of subsets of X. Then Ω is a σ-algebra if (a) X Ω, ∈ (b) If E Ω then X E Ω (i.e., Ω is closed under complements), ∈ \ ∈ (c) If E , E , Ω then E Ω (i.e., Ω is closed under countable unions) 1 2 · · · ∈ k ∈ The elements of Ω are called theS measurable subsets of X. If we choose F = R then we usually allow functions on X to take extended-real values, i.e., f(x) is allowed to take the values . An extended-real-valued function f : X [ , ] is called a if ∞x X : f(x) > a is measurable for each a →R. −∞ ∞ If we choose F = C then we require{ ∈functions on X} to take (finite) complex v∈alues—there is no complex analogue of . A complex-valued function f : X C is called a measurable function if its real and imaginary∞ parts are measurable (real-valued)→ functions.

Definition 1.11 (Measure Space). Let X be a set and Ω a σ-algebra of subsets of X. Then a function µ on Ω is a (positive) measure if (a) 0 µ(E) + for all E Ω, ≤ ≤ ∞ ∈ (b) If E1, E2, . . . is a countable family of disjoint sets in Ω, then

∞ ∞ µ Ek = µ(Ek). k=1 k=1  S  X 1For example, R. Wheeden and A. Zygmund, “Measure and ,” Marcel Dekker, 1977, or G. Folland, “Real Analysis,” Second Edition, Wiley, 1999. 6 CHRISTOPHER HEIL

In this case, (X, Ω, µ) is called a measure space. If µ(X) < , then we say that µ is a finite measure. ∞ If there exist countably many subsets E1, E2, . . . such that X = Ek and µ(Ek) < for all k, then we say that µ is σ-finite. For example, Lebesgue measure on Rn is σ-finite.∞ S It is often useful to allow measures to take negative values.

Definition 1.12 (Signed Measure). Let X be a set and Ω a σ-algebra of subsets of X. Then a function µ on Ω is a signed measure if (a) µ(E) + for all E Ω and µ( ) = 0, −∞ ≤ ≤ ∞ ∈ ∅ (b) If E1, E2, . . . is a countable family of disjoint sets in Ω, then ∞ ∞ µ Ek = µ(Ek). k=1 k=1  S  X Definition 1.13 (Integration). Let (X, Ω, µ) be a measure space. (a) If f : X [0, ] is a nonnegative, measurable function, then the integral of f over X with respect to→µ is ∞

f dµ = f(x) dµ(x) = sup inf f(x) µ(Ej) , x∈Ej ZX ZX  j  X  where the supremum is taken over all decompositions E = E E of E as the union 1 ∪ · · · ∪ N of a finite number of disjoint measurable sets Ek (and where we take the convention that 0 = 0 = 0). ∞ · · ∞ (b) If f : X [ , ] and we define → −∞ ∞ f +(x) = max f(x), 0 , f −(x) = min f(x), 0 , { } − { } then f dµ = f + dµ f − dµ, − ZX ZX ZX as long as this does not have the form (in that case the integral would be undefined). + − ∞ − ∞ Since f = f +f and X f dµ always exists (either as a finite number or as ), it follows that | | | | ∞ R f dµ exists and is finite f dµ < . ⇐⇒ | | ∞ ZX ZX (c) If f : X C, then → f dµ = Re (f) dµ + i Im (f) dµ, ZX ZX ZX as long as both on the right are defined and finite.

There are many other equivalent definitions of the integral. CHAPTER 2. OPERATORS ON HILBERT SPACES 7

Definition 1.14 (Lp Spaces). Let (X, Ω, µ) be a measure space, and fix 1 p < . Then Lp(X) consists of all measurable functions f : X [ , ] (if we cho≤ose F ∞= R) or f : X C (if we choose F = C) such that → −∞ ∞ → f p = f(x) p dµ(x) < . k kp | | ∞ ZX Then Lp(X) is a vector space under the operations of addition of functions and multiplication p of a function by a scalar. Additionally, the function p defines a semi-norm on L (X). Usually we identify functions that are equal almost evkerywhere· k (we say that f = g a.e. if µ x X : f(x) = g(x) = 0), and then becomes a norm on Lp(X). {For∈ p = w6 e define} L∞(X) to be thek · kset of measurable functions that are essentially bounded, i.e.,∞ for which there exists a finite constant M such that f(x) M a.e. Then | | ≤ f ∞ = ess sup f(x) = inf M 0 : f(x) M a.e. k k x∈X | | ≥ | | ≤ is a semi-norm on L∞(X), and is a norm if we identify functions that are equal almost everywhere. For each 1 p , the space Lp(X) is a Banach space under the above norm. ≤ ≤ ∞

Exercise 1.15 (`p Spaces). Counting measure on a set X is defined by µ(X) = card(E) if E is a finite subset of X, and µ(X) = if E is an infinite subset. Let Ω = (X) (the set of all subsets of X), and show that (X, Ω∞, µ) is a measure space. Show that LPp(X, Ω, µ) = `p(X). Show that µ is σ-finite if and only if X is countable.

Exercise 1.16 (The Delta Measure). Let X = Rn and Ω = (X). Define δ(E) = 1 if 0 E and δ(E) = 0 if 0 / E. Prove that δ is a measure, and find aPformula for ∈ ∈ f(x) dδ(x). Rn Z Sometimes this integral is written informally as Rn f(x) δ(x) dx, but note that δ is a measure on Rn, not a function on Rn (see also Exercise 1.26 below). R

1 Rn Exercise 1.17. Fix 0 g L ( ), under Lebesgue measure. Prove that µ(E) = E g(x) dx defines a finite measure≤ on∈Rn. R With this preparation, we can give some additional examples of operators on Banach or Hilbert spaces.

Example 1.18 (Multiplication Operators). Let (X, Ω, µ) be a measure space, and let φ L∞(X) be a fixed measurable function. Then for any f L2(X) we have that fφ ∈is measurable, and ∈

fφ 2 = f(x) φ(x) 2 dx f(x) 2 φ 2 dx = φ 2 f 2 < , k k2 | | ≤ | | k k∞ k k∞ k k2 ∞ ZX ZX 8 CHRISTOPHER HEIL

so fφ L2(X). Therefore, the M : L2(X) L2(X) given by ∈ φ → Mφf = fφ is well-defined, and the calculation above shows that Mφf 2 φ ∞ f 2. Therefore M is bounded, and M φ . k k ≤ k k k k φ k φk ≤ k k∞ If we assume that µ is σ-finite, then we can show that Mφ = φ ∞, as follows. Choose ∞ k k k k any ε > 0. Then by definition of L -norm, the set E = x X : φ(x) > φ ∞ ε has positive measure. Since X is σ-finite, we can write X ={ F∈ where| eac|h µ(kF k) <− }. ∪ m m ∞ Since E = (E Fm) is a countable union, we must have µ(E Fm) > 0 for some m. Let ∪ ∩ 1 ∩ F = E F , and set f = 1 2 χ . Then f = 1, but M f ( φ ε) f . Hence ∩ m µ(F ) / F k k2 k φ k2 ≥ k k∞ − k k2 Mφ 2 φ ∞ ε. k Exercise:k ≥ k kFind−an example of a measure µ that is not σ-finite and a function φ such that M < φ . k φk k k∞

Exercise 1.19. Let (X, Ω, µ) be a measure space, and let φ be a fixed measurable function. Prove that if fφ L2(X) for every f L2(X), then we must have φ L∞(X). ∈ ∈ ∈ Solution. Assume φ / L∞(X). Set ∈ E = x X : k φ(x) < k + 1 . k { ∈ ≤ | | } ∞ The Ek are measurable and disjoint, and since φ is not in L (X) there must be infinitely many E with positive measure. Choose any E , k N, all with positive measure and let k nk ∈ E = Enk . Define ∪ 1 1/2 , x Enk , f(x) = k µ(En ) ∈  k 0, x / E.  ∈ Then ∞  ∞ 2 1 1 f dµ = 2 = 2 < , X | | En k µ(Enk ) k ∞ Z Xk=1 Z k Xk=1 but ∞ 2 ∞ 2 k fφ dµ 2 = 1 = , X | | ≥ En k µ(Enk ) ∞ Z Xk=1 Z k Xk=1 which is a contradiction. 

Exercise 1.20. Continuing Example 1.18, do the following. 2 (a) Determine a necessary and sufficient condition on φ which implies that Mφ : L (X) L2(X) is injective. → 2 (b) Determine a necessary and sufficient condition on φ which implies that Mφ : L (X) L2(X) is surjective. → −1 2 (c) Prove that if Mφ is injective but not surjective then Mφ : range(Mφ) L (X) is unbounded. → (d) Extend from the case p = 2 to any 1 p . ≤ ≤ ∞ CHAPTER 2. OPERATORS ON HILBERT SPACES 9

Example 1.21 (Integral Operators). Let (X, Ω, µ) be a σ-finite measure space. An integral operator is an operator of the form

Lf(x) = k(x, y) f(y) dµ(y). (1.3) ZX This is just a formal definition, we have to provide conditions under which this makes sense, and the following two theorems will provide such conditions. The function k that determines the operator is called the kernel of the operator (not to be confused with the kernel/nullspace of the operator!). Note that an integral operator is just a generalization of matrix multiplication. For, if A n m is an m n matrix with entries aij and u C , then Au C , and its components are given by × ∈ ∈ n

(Au)i = aij uj, i = 1, . . . , m. j=1 X Thus, the values k(x, y) are analogous to the entries aij of the matrix A, and the values Lf(x) are analogous to the entries (Au)i.

The following result shows that if the kernel is square-integrable, then the corresponding integral operator is bounded. Later we will define the notion of a Hilbert–Schmidt operator. For the case of integral operators mapping L2(X) into itself, it can be shown that L is a Hilbert–Schmidt operator if and only if the kernel k belongs to L2(X X). ×

Theorem 1.22 (Hilbert–Schmidt Integral Operators). Let (X, Ω, µ) be a σ-finite measure space, and choose a kernel k L2(X X). That is, assume that ∈ × k 2 = k(x, y) 2 dµ(x) dµ(y) < . k k2 | | ∞ ZX ZX Then the integral operator given by (1.3) defines a bounded mapping of L2(X) into itself, and L k . k k ≤ k k2 Proof. Although a slight abuse of the order of logic (technically we should show Lf exists before trying to compute its norm), the following calculation shows that L is well-defined and is a bounded mapping of L2(X) into itself:

Lf 2 = Lf(x) 2 dµ(x) k k2 | | ZX 2 = k(x, y) f(y) dµ(y) dµ(x) ZX ZX

k(x, y) 2 dµ(y) f(y) 2 dµ(y) dµ(x) ≤ | | | | ZX ZX  ZX  10 CHRISTOPHER HEIL

= k(x, y) 2 dµ(y) f 2 dµ(x) | | k k2 ZX ZX = k 2 f 2, k k2 k k2 where the inequality follows by applying Cauchy–Schwarz to the inner integral. Thus L is bounded, and L k .  k k ≤ k k2 The following result is one version of Schur’s Lemma. There are many forms of Schur’s Lemma, this is one particular special case. Exercise: Compare the hypotheses of the following result to the operator norms you calculated in Exercise 1.6.

Theorem 1.23. Let (X, Ω, µ) be a σ-finite measure space, and Assume that k is a measurable function on X X which satisfies the “mixed-norm” conditions × C = ess sup k(x, y) dµ(y) < and C = ess sup k(x, y) dµ(x) < . 1 | | ∞ 2 | | ∞ x∈X ZX y∈X ZX Then the integral operator given by (1.3) defines a bounded mapping of L2(X) into itself, and L (C C )1/2. k k ≤ 1 2 Proof. Choose any f L2(X). Then, by applying the Cauchy–Schwarz inequality, we have ∈ Lf 2 = Lf(x) 2 dµ(x) k k2 | | ZX 2 = k(x, y) f(y) dµ(y) dµ(x) ZX ZX

2 k(x, y) 1/2 k(x, y) 1/2 f(y) dµ(y) dµ(x) ≤ | | | | | | ZX ZX   k(x, y) dµ(y) k(x, y) f(y) 2 dµ(y) dµ(x) ≤ | | | | | | ZX ZX  ZX  C k(x, y) f(y) 2 dµ(y) dµ(x) ≤ 1 | | | | ZX ZX = C f(y) 2 k(x, y) dµ(x) dµ(y) 1 | | | | ZX ZX C f(y) 2 C dµ(y) ≤ 1 | | 2 ZX = C C f 2, 1 2 k k2 where we have used Tonelli’s Theorem to interchange the order of integration (here is where we needed the fact that µ is σ-finite). Thus L is bounded and L (C C )1/2.  k k ≤ 1 2 CHAPTER 2. OPERATORS ON HILBERT SPACES 11

Exercise 1.24. Consider what happens in the preceding example if we take 1 p p ≤ ≤p ∞ instead of p = 2. In particular, in part b, show that if C1, C2 < then L: L (X) L (X) is a bounded mapping for each 1 p (try to do p = 1 or p∞= first). → ≤ ≤ ∞ ∞

Exercise 1.25 (). Define L: L2[0, 1] L2[0, 1] by → x Lf(x) = f(y) dy. Z0 Show directly that L is bounded. Then show that L is an integral operator with kernel k : [0, 1]2 F defined by → 1, y x, k(x, y) = ≤ (0, y > x. Observe that k L2([0, 1]2), so L is compact. This operator is called the Volterra operator. ∈

Exercise 1.26 (). Convolution is one of the most important examples of integral operators. Consider the case of Lebesgue measure on Rn. Given functions f, g on Rn, their convolution is the function f g defined by ∗ (f g)(x) = f(y) g(x y) dy, ∗ Rn − Z provided that the integral makes sense. Note that with g fixed, the mapping f f g is an integral operator with kernel k(x, y) = g(x y). 7→ ∗ − (a) Let g L1(Rn) be fixed. Use Schur’s Lemma (Theorem 1.23) to show that Lf = f g is a bounded∈ mapping of L2(Rn) into itself. In fact, use Exercise 1.24 to prove Young’s∗ Inequality: If f Lp(Rn) (1 p ) and g L1(Rn), then f g Lp(Rn), and ∈ ≤ ≤ ∞ ∈ ∗ ∈ f g f g . k ∗ kp ≤ k kp k k1 In particular, L1(Rn) is closed under convolution. (b) Note that we cannot use the Hilbert–Schmidt condition (Theorem 1.22) to prove Young’s Inequality, since g(x y) 2 dx dy = , Rn Rn | − | ∞ Z Z even if we assume that g L2(Rn). ∈ (c) Prove that convolution is commutative, i.e., that f g = g f. ∗ ∗ (d) Prove that there is no identity element in L1(Rn), i.e., there is no function g L1(Rn) such that f g = f for all f L1(Rn). This is not trivial—it is easier to do if y∈ou make use of the Fourier∗ transform on∈Rn, and in particular use the Riemann–Lebesgue Lemma to derive a contradiction. 12 CHRISTOPHER HEIL

(e) Some texts do talk informally about a “delta function” that is an identity element for convolution, defined by the conditions , x = 0, δ(x) = ∞ and δ(x) dx = 1, 0, x = 0, Rn ( 6 Z but no such function actually exists. In particular, the function δ defined on the left-hand side of the line above is equal to zero a.e., and hence is the zero function as far as Lebesgue

integration is concerned. That is, we have Rn δ(x) dx = 0, not 1. The “delta function” is really just an informal use of the delta distribution (see Exercise 1.3) or the delta measure (see Exercise 1.16). Show that if we defineRthe convolution of a function f with the delta measure δ to be (f δ)(x) = f(x y) dδ(y), (1.4) ∗ Rn − Z then f δ = f for all f L1(Rn). Note that in the “informal” notation of Exercise 1.16, (1.4) reads∗ ∈ (f δ)(x) = f(x y) δ(y) dy, ∗ Rn − Z which perhaps explains the use of the term “delta function.”

Exercise 1.27. Prove that L1(Rn) is not closed under pointwise multiplication. That is, prove that there exist f, g L1(Rn) such that the pointwise product h(x) = (fg)(x) = f(x)g(x) does not belong to ∈to L1(Rn).

Exercise 1.28 (Convolution Continued). (a) Consider the space Lp[0, 1], where we think of functions in Lp[0, 1] as being extended 1-periodically to the real line. Define convolution on the circle by 1 (f g)(x) = f(y) g(x y) dy, ∗ − Z0 where the periodicity is used to define g(x y) when x y lies outside [0, 1] (equivalently, replace x y by x y mod 1, the fractional− part of x− y). Prove a version of Young’s Inequality−for Lp[0, 1].− − (b) Consider the `p(Z). Define convolution on Z by

(x y)n = xm yn−m. ∗ Z mX∈ Prove a version of Young’s Inequality for `p(Z). Prove that `1(Z) contains an identity element with respect to convolution, i.e., there exists a sequence in `1(Z) (typically denoted δ) such that δ x = x for every x `p(Z). ∗ ∈ (c) Identify the essential features needed to define convolution on more general domains, and prove a version of Young’s Inequality for that setting. CHAPTER 2. OPERATORS ON HILBERT SPACES 13

Exercise 1.29 (Convolution and the ). Let be the Fourier transform 2 2 F on the circle, i.e., it is the : L [0, 1] ` (Z) given by f = fˆ = fˆ(n) n∈Z, where F → F { } 1 fˆ(n) = f, e = f(x) e−2πinx dx, e (x) = e2πinx. h ni n Z0 (a) Prove that the Fourier transform converts convolution in to multiplication. That is, prove that if f, g L2[0, 1], then (f g)∧ = fˆgˆ, i.e., ∈ ∗ (f g)∧(n) = fˆ(n) gˆ(n), n Z. ∗ ∈ (b) Note that if g L2[0, 1], then g L1[0, 1], so by Young’s Inequality we have that f g L2[0, 1]. Holding∈ g fixed, define ∈an operator L: L2[0, 1] L2[0, 1] by Lf = f g. ∗ ∈ 2 → ∗ Since e Z is an orthonormal basis for L [0, 1], we have { n}n∈ ˆ 2 f = f(n) en, f L [0, 1]. Z ∈ Xn∈ Show that Lf = f g = gˆ(n) fˆ(n) e , f L2[0, 1]. ∗ n ∈ n∈Z X Thus, in the “Fourier domain,” convolution acts by changing or adjusting the amount that each “component” or “frequency” en contributes to the representation of the function in this basis: the weight fˆ(n) for frequency n is replaced by the weight gˆ(n) fˆ(n). Explain why this says that L is analogous to multiplication by a diagonal operator. In engineering parlance, convolution is also referred to as filtering. Explain why this terminology is appropriate. Compare this operator L to Example 1.7.

2. The Adjoint of an Operator

Example 2.1. Note that the dot product on Rn is given by x y = xTy, while the dot product on Cn is x y = xTy¯. · Let A be an m · n real matrix. Then x Ax defines a linear map of Rn into Rm, and its transpose AT satisfies× 7→ x Rn, y Rm, Ax y = (Ax)Ty = xTATy = x (ATy). ∀ ∈ ∀ ∈ · · Similarly, if A is an m n complex matrix, then its Hermitian or adjoint matrix AH = AT satisfies × x Cn, y Cm, Ax y = (Ax)Ty¯ = xTATy¯ = x (AHy). ∀ ∈ ∀ ∈ · ·

Theorem 2.2 (Adjoint). Let H and K be Hilbert spaces, and let A: H K be a bounded, linear map. Then there exists a unique bounded linear map A∗ : K H→such that → x H, y K, Ax, y = x, A∗y . ∀ ∈ ∀ ∈ h i h i 14 CHRISTOPHER HEIL

Proof. Fix y K. Then Lx = Ax, y is a bounded linear functional on H. By the Riesz Representation∈ Theorem, there existsh ai unique vector h H such that ∈ Ax, y = Lx = x, h . h i h i Define A∗y = h. Verify that this map A∗ is linear (exercise). To see that it is bounded, observe that A∗y = h = sup x, h k k k k kxk=1 |h i| = sup Ax, y kxk=1 |h i| sup Ax y ≤ kxk=1 k k k k sup A x y = A y . ≤ kxk=1 k k k k k k k k k k We conclude that A∗ is bounded, and that A∗ A . Finally, we must show that A∗ is unique.k kSupp≤ k osek that B (K, H) also satisfied Ax, y = x, By for all x H and y K. Then for each fixed∈ By we would have that hx, Byi A∗hy = 0i for every x∈, which implies∈ By A∗y = 0. Hence B = A∗.  h − i −

Exercise 2.3 (Properties of the adjoint). (a) If A (H, K) then (A∗)∗ = A. ∈ B (b) If A, B (H, K) and α, β F, then (αA + βB)∗ = αA¯ ∗ + β¯B∗. ∈ B ∈ (c) If A (H , H ) and B (H , H ), then (BA)∗ = A∗B∗. ∈ B 1 2 ∈ B 2 3 (d) If A (H) is invertible in (H) (meaning that there exists A−1 (H) such that AA−∈1 =BA−1A = I), then A∗Bis invertible in (H) and (A−1)∗ = (A∈∗B)−1. B

Remark 2.4. Later we will prove the Open Mapping Theorem. A remarkable consequence of this theorem is that if X and Y are Banach spaces and A: X Y is a bounded bijection, then A−1 : Y X is automatically bounded. → →

Proposition 2.5. If A (H, K), then A = A∗ = A∗A 1/2 = AA∗ 1/2. ∈ B k k k k k k k k Proof. In the course of proving Theorem 2.2, we already showed that A∗ A . If f H, then k k ≤ k k ∈ Af 2 = Af, Af = A∗Af, f A∗Af f A∗ Af f . (2.1) k k h i h i ≤ k k k k ≤ k k k k k k Hence Af A∗ f (even if Af = 0, this is still true). Since this is true for all f we concludek thatk ≤ kA k k Ak∗ . Thereforek k A = A∗ . Next, we hakve k A≤∗kA k A A∗ =k kA 2k. Butk also, from the calculation in (2.1), we have Af 2 A∗kAf kf≤. kTakingk k thek supremk k um over all unit vectors, we obtain k k ≤ k k k k A 2 = sup Af 2 sup A∗Af f = A∗A . k k kfk=1 k k ≤ kfk=1 k k k k k k CHAPTER 2. OPERATORS ON HILBERT SPACES 15

Consequently A 2 = A∗A . The final equality follows by interchanging the roles of A and A∗. k k k k 

Exercise 2.6. Prove that if U (H, K), then U is an isomorphism if and only if U is invertible and U −1 = U ∗. ∈ B

∞ Exercise 2.7. (a) Let λ = (λn)n∈N ` (N) be given and let L be defined as in Example 1.7. Find L∗. ∈

(b)Prove that the adjoint of the multiplication operator Mφ defined in Exercise 1.18 is the multiplication operator Mφ¯.

Exercise 2.8. Let L and R be the left- and right-shift operators on `2(N), i.e.,

L(x1, x2, . . . ) = (x2, x3, . . . ) and R(x1, x2, . . . ) = (0, x1, x2, . . . ). Prove that L = R∗.

Example 2.9. Let L be the integral operator defined in (1.3), determined by the kernel function k. Assume that k is chosen so that L: L2(X) L2(X) is bounded. The adjoint is the unique operator L∗ : L2(X) L2(X) which satisfies→ → Lf, g = f, L∗g , f, g L2(X). h i h i ∈ To find L∗, let A: L2(X) L2(X) be the integral operator with kernel k(y, x), i.e., → Af(x) = k(y, x) f(y) dµ(y). ZX Then, given any f and g L2(X), we have ∈ f, L∗g = Lf, g = Lf(x) g(x) dµ(x) h i h i ZX = k(x, y) f(y) dµ(y) g(x)dµ(x) ZX ZX = f(y) k(x, y) g(x) dµ(x) dµ(y) ZX ZX = f(y) k(x, y) g(x) dµ(x) dµ(y) ZX ZX = f(y) Ag(y) dµ(y) ZX = f, Ag . h i By uniqueness of the adjoint, we must have L∗ = A. Exercise: Justify the interchange in the order of integration in the above calculation, i.e., provide hypotheses under which the calculations above are justified. 16 CHRISTOPHER HEIL

Exercise 2.10. Let en n∈N be an orthonormal basis for a separable Hilbert space H. Define 2 { } ∗ 2 T : H ` (N) by T (f) = f, e N. Find a formula for T : ` (N) H. → {h ni}n∈ →

Definition 2.11. Let A (H). ∈ B (a) We say that A is self-adjoint or Hermitian if A = A∗. (b) We say that A is normal if AA∗ = A∗A.

Example 2.12. A real n n matrix A is self-adjoint if and only if it is symmetric, i.e., if A = AT. A complex n ×n matrix A is self-adjoint if and only if it is Hermitian, i.e., if A = AH. ×

Exercise 2.13. Show that every self-adjoint operator is normal. Show that every is normal, but that a unitary operator need not be self-adjoint. For H = Cn, find examples of matrices that are not normal. Are the left- and right-shift operators on `2(N) normal?

Exercise 2.14. (a) Show that if A, B (H) are self-adjoint, then AB is self-adjoint if and only if AB = BA. ∈ B (b) Give an example of self-adjoint operators A, B such that AB is not self-adjoint. (c) Show that if A, B (H) are self-adjoint then A + A∗, AA∗, A∗A, A + B, ABA, and BAB are all self-adjoin∈ Bt. What about A A∗ or A B? Show that AA∗ A∗A is self-adjoint. − − −

∞ Exercise 2.15. (a) Let λ = (λn)n∈N ` (N) be given and let L be defined as in Example 1.7. Show that L is normal, find a formula∈ for L∗, and prove that L is self-adjoint if and only if each λn is real. (b) Determine a necessary and sufficient condition on φ so that the multiplication operator Mφ defined in Exercise 1.18 is self-adjoint. (c) Determine a necessary and sufficient condition on the kernel k so that the integral operator L defined in (1.23) is self-adjoint.

The following result gives a useful condition for telling when an operator on a complex Hilbert space is self-adjoint.

Proposition 2.16. Let H be a complex Hilbert space (i.e., F = C), and let A (H) be given. Then: ∈ B A is self-adjoint Af, f R f H. ⇐⇒ h i ∈ ∀ ∈ CHAPTER 2. OPERATORS ON HILBERT SPACES 17

Proof. . Assume A = A∗. Then for any f H we have ⇒ ∈ Af, f = f, Af = A∗f, f = Af, f . h i h i h i h i Therefore Af, f is real. h i . Assume that Af, f is real for all f. Choose any f, g H. Then ⇐ h i ∈ A(f + g), f + g = Af, f + Af, g + Ag, f + Ag, g . h i h i h i h i h i Since A(f + g), f + g , Af, f , and Ag, g are all real, we conclude that Af, g + Ag, f is real.h Hence it equalsi itsh ownicomplexh conjugate,i i.e., h i h i Af, g + Ag, f = Af, g + Ag, f = g, Af + f, Ag . (2.2) h i h i h i h i h i h i Similarly, since A(f + ig), f + ig = Af, f i Af, g + i Ag, f + Ag, g h i h i − h i h i h i we see that i Af, g + i Ag, f = i Af, g + i Ag, f = i g, Af i f, Ag . − h i h i − h i h i h i − h i Multiplying through by i yields Af, g Ag, f = g, Af + f, Ag . (2.3) h i − h i −h i h i Adding (2.2) and (2.3) together, we obtain 2 Af, g = 2 f, Ag = 2 A∗f, g . h i h i h i Since this is true for every f and g, we conclude that A = A∗. 

Example 2.17. The preceding result is false for real Hilbert spaces. After all, if F = R then Af, f is real for every f no matter what A is. Therefore, any non-self-adjoint operator proh videsi a counterexample. For example, if H = Rn then any non-symmetric matrix A is a counterexample.

The next result provides a useful way of calculating the operator norm of a self-adjoint operator.

Proposition 2.18. If A (H) is self-adjoint, then ∈ B A = sup Af, f . k k kfk=1 |h i|

Proof. Set M = supkfk=1 Af, f . By Cauchy–Schwarz and|h the definitioni| of operator norm, we have M = sup Af, f sup Af f sup A f f = A . kfk=1 |h i| ≤ kfk=1 k k k k ≤ kfk=1 k k k k k k k k To get the opposite inequality, note that if f is any nonzero vector in H then f/ f is a f f k k unit vector, so A , M. Rearranging, we see that kfk kfk ≤ f H, Af, f M f 2. (2.4) ∀ ∈ h i ≤ k k 18 CHRISTOPHER HEIL

Now choose any f, g H with f = g = 1. Then, by expanding the inner products, canceling terms, and using∈ the factkthatk Ak =k A∗, we see that A(f + g), f + g A(f g), f g = 2 Af, g + 2 Ag, f − − − h i h i = 2 Af, g + 2 g, Af h i h i = 4 Re Af, g . h i Therefore, applying (2.4) and the Parallelogram Law, we have 4 Re Af, g A(f + g), f + g + A(f g), f g h i ≤ |h i| |h − − i| M f + g 2 + M f g 2 ≤ k k k − k = 2M f 2 + g 2 = 4M. k k k k That is, Re Af, g M for every choice of unit vectors f and g. Write Af, g = Af, g eiθ. Then eiθg ish anotheri ≤ unit vector, so h i |h i| M Re Af, e−iθg = Re eiθ Af, g = Af, g . ≥ h i h i |h i| Hence Af = sup Af, g M. k k kgk=1 |h i| ≤ Since this is true for every unit vector f, we conclude that A M.  k k ≤

The following corollary is a very useful consequence.

Corollary 2.19. Assume that A (H). ∈ B (a) If F = R, A = A∗, and Af, f = 0 for every f, then A = 0. h i (b) If F = C and Af, f = 0 for every f, then A = 0. h i Proof. Assume the hypotheses of either statement (a) or statement (b). In the case of statement (a), we have by hypothesis that A is self-adjoint. In the case of statement (b), we can conclude that A is self-adjoint because Af, f = 0 is real for every f. Hence in either case we can apply Proposition 2.18 to concludeh thati A = sup Af, f = 0.  k k kfk=1 |h i|

Lemma 2.20. If A (H), then the following statements are equivalent. ∈ B (a) A is normal, i.e., AA∗ = A∗A. (b) Af = A∗f for every f H. k k k k ∈ CHAPTER 2. OPERATORS ON HILBERT SPACES 19

Proof. (b) (a). Assume that (b) holds. Then for every f we have ⇒ (A∗A AA∗)f, f = A∗Af, f AA∗f, f − h i − h i = Af, Af A∗f, A∗f h i − h i = Af 2 A∗f 2 = 0. k k − k k Since A∗A AA∗ is self-adjoint, it follows from Corollary 2.19 that A∗A AA∗ = 0. − − (a) (b). Exercise.  ⇒

Corollary 2.21. If A (H) is normal, then ker(A) = ker(A∗). ∈ B

Exercise 2.22. Suppose that A (H) is normal. Prove that A is injective if and only if range(A) is dense in H. ∈ B

Exercise 2.23. If A (H), then the following statements are equivalent. ∈ B (a) A is an , i.e., Af = f for every f H. k k k k ∈ (b) A∗A = I. (c) Af, Ag = f, g for every f, g H. h i h i ∈

Exercise 2.24. If H = Cn and A, B are n n matrices, then AB = I implies BA = I. Give a counterexample to this for an infinite-dimensional× Hilbert space. Consequently, the hypothesis A∗A = I in the preceding result does not imply that AA∗ = I.

Exercise 2.25. If A (H), then the following statements are equivalent. ∈ B (a) A∗A = AA∗ = I. (b) A is unitary, i.e., it is a surjective isometry. (c) A is a normal isometry.

The following result provides a very useful relationship between the range of A∗ and the kernel of A.

Theorem 2.26. Let A (H, K). ∈ B (a) ker(A) = range(A∗)⊥. (b) ker(A)⊥ = range(A∗). (c) A is injective if and only if range(A∗) is dense in H. 20 CHRISTOPHER HEIL

Proof. (a) Assume that f ker(A) and let h range(A∗), i.e., h = A∗g for some g K. Then since Af = 0, we ha∈ve f, h = f, A∗g∈ = Af, g = 0. Thus f range(A∗)∈⊥, so ker(A) range(A∗)⊥. h i h i h i ∈ Now ⊆assume that f range(A∗)⊥. Then for any h H we have Af, h = f, A∗h = 0. But this implies Af = ∈0, so f ker(A). Thus range(A∈∗)⊥ ker(A).h i h i ∈ ⊆ (b), (c) Exercises. 

3. Projections and Idempotents: Invariant and Reducing Subspaces Definition 3.1. a. If E (H) satisfies E2 = E then E is said to be idempotent. ∈ B b. If E (H) satisfies E2 = E and ker(E) = range(E)⊥ then E is called a projection. ∈ B

Exercise 3.2. If E (H) is an idempotent operator, then ker(E) and range(E) are closed subspaces of H. Further,∈ B ker(E) = range(I E) and range(E) = ker(I E). − −

Lemma 3.3 (Characterization of Orthogonal Projections). Let E (H) be a nonzero idempotent operator. Then the following statements are equivalent. ∈ B (a) E is a projection. (b) E is the orthogonal projection of H onto range(E). (c) E = 1. k k (d) E is self-adjoint. (e) E is normal. (f) E is positive, i.e., Ef, f 0 for every f H. h i ≥ ∈ Proof. (e) (a). Assume that E2 = E and E is normal. Then from Lemma 2.20 we know that Ef ⇒= E∗f for every f H. Hence Ef = 0 if and only if E∗f = 0, or in other words,k kerk(E)k= kerk (E∗). But w∈e know from Theorem 2.26 that ker(E∗) = range(E)⊥. Hence we conclude that ker(E) = range(E)⊥, and therefore E is a projection. The remaining implications are exercises. 

Definition 3.4 (Orthogonal Direct Sum of Subspaces). Let Mi i∈I be a collection of closed subspaces of H such that M M whenever i = j. Then the{ ortho} gonal direct sum of the i ⊥ j 6 Mi is the smallest closed subspace which contains every Mi. This space is

Mi = span Mi . i∈I i∈I L S  CHAPTER 2. OPERATORS ON HILBERT SPACES 21

Exercise 3.5. Suppose that M, N are closed subspaces of H such that M N. Prove that M + N = m + n : m M, n N is a closed subspace of H, and that ⊥ { ∈ ∈ } M N = M + N. ⊕ Show that every vector x M N can be written uniquely as x = m + n with m M and n N. ∈ ⊕ ∈ ∈Extend by induction to finite collections of closed, pairwise orthogonal subspaces. (Un- fortunately, the analogous statement is not true for infinite collections.)

Exercise 3.6. Show that if A (H, K) then H = ker(A) range(A∗). ∈ B ⊕

Definition 3.7. Let A (H) and M H. ∈ B ≤ (a) We say that M is invariant under A if A(M) M, where ⊆ A(M) = Ax : x M . { ∈ } That is, M is invariant if x M implies Ax M. Note that it need not be the case that A(M) = M. ∈ ∈ (b) We say that M is a reducing subspace for A if both M and M ⊥ are invariant under A, i.e., A(M) M and A(M ⊥) M ⊥. ⊆ ⊆

Proposition 3.8. Let A (H) and M H be given. Then the following statements are equivalent. ∈ B ≤ (a) M is invariant under A.

(b) P AP = AP , where P = PM is the orthogonal projection of H onto M.

Exercise 3.9. Define L: `2(Z) `2(Z) by → L(. . . , x−1, x0, x1, . . . ) = (. . . , , x0, x1, x2, . . . ),

where on the right-hand side the entry x1 sits in the 0th component position. That is, L slides each component one unit to the left (L is called a bilateral shift). Find a closed subspace of `2(Z) that is invariant but not reducing under L.

Exercise 3.10. Assume that M H is invariant under L (H). Prove that M ⊥ is invariant under L∗. ≤ ∈ B 22 CHRISTOPHER HEIL

4. Compact Operators

Definition 4.1 (Compact and Totally Bounded Sets). Let X be a Banach space, and let E X be given. ⊆ (a) We say that E is compact if every open cover of E contains a finite subcover. That is, E is compact if whenever Uα α∈I is a collection of open sets whose union contains E, then there exist finitely{man}y α , . . . , α such that E U U . 1 N ⊆ α1 ∪ · · · ∪ αN (b) We say that E is sequentially compact if every sequence f N of points of E { n}n∈ contains a convergent subsequence f N whose belongs to E. { nk }k∈ (c) We say that E is totally bounded if for every ε > 0 there exist finitely many points f1, . . . , fN E such that ∈ N E B(fk, ε), ⊆ k=1

where B(fk, ε) is the open ball ofSradius ε centered at fk. That is, E is totally bounded if and only there exist finitely many points f , . . . , f E such that every 1 N ∈ element of E is within ε of some fk.

In finite dimensions, a set is compact if and only if it is closed and bounded. In infinite dimensions, all compact sets are closed and bounded, but the converse fails. Instead, we have the following characterization of compact sets. (this characterization actually holds in any complete space).

Theorem 4.2. Let E be a subset of a Banach space X. Then the following statements are equivalent. (a) E is compact. (b) E is sequentially compact. (c) E is closed and totally bounded. Proof. (b) (a).2 Assume that E is sequentially compact. Our first step will be to prove the following⇒ claim, where the diameter of a set S is defined to be diam(S) = sup f g : f, g S . {k − k ∈ }

Claim 1. For any open cover Uα α∈I of E, there exists a number δ > 0 (called a Lebesgue number for the cover) such that{if S} E satisfies diam(S) < δ, then there is an α I such that S U . ⊆ ∈ ⊆ α To prove the claim, suppose that U was an open cover of E such that no δ with { α}α∈I the required property existed. Then for each n N, we could find a set Sn E with diam(S ) < 1 such that S is not contained in an∈y U . Choose any f S . Since⊆ E is n n n α n ∈ n sequentially compact, there must be a subsequence f N that converges to an element of { nk }k∈ 2This proof is adapted from one given in J. R. Munkres, “,” Second Edition, Prentice Hall, 2000. CHAPTER 2. OPERATORS ON HILBERT SPACES 23

E, say fnk a E. But we must have a Uα for some α, and since Uα is open there must exist some →ε > 0∈such that B(a, ε) U . No∈ w choose k large enough that we have both ⊆ α 1 ε ε < and a fnk < . nk 2 k − k 2 ε The first inequality above implies that diam(Snk ) < 2 . Therefore, using this and second

inequality, we have Snk B(a, ε) Uα, which is a contradiction. Therefore the claim is proved. ⊆ ⊆ Next, we will prove the following claim. Claim 2. For any ε > 0, there exist finitely many f , . . . , f E such that 1 N ∈ N E B(fk, ε). ⊆ k=1 S To prove this claim, assume that there is an ε > 0 such that E cannot be covered by finitely many ε-balls centered at points of E. Choose any f1 E. Since E cannot be covered by a single ε-ball, we have E B(f , ε). Hence there exists∈ f E B(f , ε), i.e., 6⊆ 1 2 ∈ \ 1 f2 E and f2 f1 ε. But E cannot be covered by two ε-balls, so there must exist an f ∈ E B(kf , ε−) Bk(≥f , ε) . In particular, we have f f , f f ε. Continuing in 3 ∈ \ 1 ∪ 2 k 3 − 1k k 3 − 2k ≥ this way we obtain a sequence of points fn n∈N in E which has no convergent subsequence, which is a contradiction. Hence the claim{ is}proved. Finally, we show that E is compact. Let Uα α∈I be any open cover of E. Let δ be the { }δ Lebesgue number given by Claim 1, and set ε = 3 . By Claim 2, there exists a covering of E by finitely many ε-balls. Each ball has diameter smaller than δ, so by Claim 1 is contained in some Uα. Thus we find finitely many Uα that cover E.

(c) (b). Assume that E is closed and totally bounded, and let fn n∈N be any sequence ⇒ 1{ } of points in E. Since E is covered by finitely many balls of radius 2 , one of those balls must (1) contain infinitely many f , say fn N. Then we have n { }n∈ m, n N, f (1) f (1) < 1. ∀ ∈ k m − n k 1 (2) N Since E is covered by finitely many balls of radius 4 , we can find a subsequence fn n∈ of (1) { } fn n∈N such that { } 1 m, n N, f (1) f (1) < . ∀ ∈ k m − n k 2 (k) (k) (k) 1 N By induction we keep constructing subsequences fn n∈ such that fm fn < k for all m, n N. { } k − k ∈ (n) Now consider the “diagonal subsequence” fn n∈N. Given ε > 0, let N be large enough 1 (m) { } (n) N that N < ε. If m n > N, then fm is one element of the sequence fk k∈ , say (m) (n) ≥ { } fm = fk . Then 1 f (m) f (n) = f (n) f (n) < < ε. k m − n k k k − n k n (n) Thus fn n∈N is Cauchy and hence converges. Since E is closed, it must converge to some elemen{t of }E. 24 CHRISTOPHER HEIL

(a) (c). Exercise.  ⇒ Exercise 4.3. Show that if E is a totally bounded subset of a Banach space X, then its closure E is compact. A set whose closure is compact is said to be precompact.

Notation 4.4. We let BallH denote the closed unit sphere in H, i.e., Ball = Ball(H) = f H : f 1 . H { ∈ k k ≤ }

Exercise 4.5. Prove that if H is infinite-dimensional, then BallH is not compact.

Definition 4.6 (Compact Operators). Let H, K be Hilbert spaces. A linear operator T : H K is compact if T (Ball ) has compact closure in K. We define → H (H, K) = T : H K : T is compact , B0 { → } and set (H) = (H, H). B0 B0 By definition, a is linear, and we will see that all compact operators are bounded. Thus it will turn out that 0(H, K) (H, K). In fact, we will see that 0(H, K) is a closed subspace of (H, K). B ⊆ B B The following result givB es some useful reformulations of the definition of compact operator.

Proposition 4.7 (Characterizations of Compact Operators). Let T : H K be linear. Then the following statements are equivalent. → (a) T is compact.

(b) T (BallH ) is totally bounded.

(c) If fn n∈N is a bounded sequence in H, then T fn n∈N contains a convergent subse- quence.{ } { } Proof. (a) (b). This follows from Theorem 4.2 and Exercise 4.3. ⇔ (a) (c). Suppose that T is compact and that fn n∈N is a bounded sequence in H. By rescaling⇒ the sequence (i.e., multiplying by an appropriate{ } scalar), we may assume that f Ball for every n. Therefore T f T (Ball ) T (Ball ). Since T (Ball ) is compact, n ∈ H n ∈ H ⊆ H H it follows from Theorem 4.2 that T f N contains a subsequence which converges to an { n}n∈ element of T (BallH ). (c) (a). Exercise.  ⇒ Proposition 4.8. If T : H K is compact, then it is bounded. That is, → (H, K) (H, K). B0 ⊆ B Proof. Assume that T : H K is linear but unbounded. Then there exist vectors f H → n ∈ such that fn = 1 but T fn n. Therefore every subsequence of T fn n∈N is unbounded, and hencekcannotk converge.k Thereforek ≥ T is not compact by Proposition{ 4.7.}  CHAPTER 2. OPERATORS ON HILBERT SPACES 25

Exercise 4.9. Show that if H is infinite-dimensional then the identity operator on H is not compact. Hence a need be compact in general.

The following exercise shows that a compact operator maps an orthonormal sequence to a sequence that converges to the zero vector.

Exercise 4.10. (a) Let h N be a sequence of vectors in H, and let h H. Suppose { n}n∈ ∈ that every subsequence of hn n∈N contains a subsequence that converges to h. Prove that h h. { } n → Hint: Proceed by contradiction. Suppose that hn does not converge to h. Show that this implies that there is an ε > 0 and a subsequence hnk k∈N such that h hnk ε for every k. { } k − k ≥

(b) Suppose that T : H K is compact, and let en n∈N be an orthonormal sequence in H. Show that T e 0. → { } n → Hint: Choose any subsequence f N. Since T is compact, this sequence has a sub- { n}n∈ sequence gn n∈N such that T gn n∈N converges, say T gn h. Prove that T gn, h 0 { } { } 2 → h i → (use Bessel’s Inequality to find a bound for the ` -norm of T gn, h n∈N). Use part (a) to complete the proof. {h i}

The following exercise shows that a compact operator maps weakly convergent to convergent sequences.

Definition 4.11. Let fn n∈N be a sequence of vectors in H and let f H. We say that { } w ∈ f converges weakly to f, written f f, if n n → g H, f , g f, g as n . ∀ ∈ h n i → h i → ∞

Exercise 4.12. (a) Show that if f f, then f w f. n → n → w (b) Show that if e N is an orthonormal sequence in H, then e 0. { n}n∈ n → (c) Suppose that T (H) is compact. Show that if f w f, then T f T f. ∈ B n → n →

Exercise 4.13. Let φ L∞(Rn) be fixed, with φ = 0. Then by Exercise 1.18 we know that the multiplication operator∈ M : L2(Rn) L2(Rn6 ) given by M f = fφ is bounded. Show φ → φ that Mφ is not compact. Hint: There must exist an ε > 0 and a set E Rn with positive measure such that φ(x) ε for all x E. ⊆ | | ≥ ∈ ∞ Exhibit a measure space (X, Ω, µ) and a bounded, nonzero φ L (X) such that Mφ is compact. Hint: Consider Exercise 4.23. ∈

Exercise 4.14. Porve that if T : H K is compact and injective, then T −1 : range(T ) H is unbounded. → → 26 CHRISTOPHER HEIL

Theorem 4.15 (Limits of Compact Operators). 0(H, K) is a closed subspace of (H, K) (under the operator norm). That is, B B (a) if S, T (H, K) and α, β F, then αS + βT (H, K), ∈ B0 ∈ ∈ B0 (b) if T (H, K), T (H, K), and T T 0, then T (H, K). n ∈ B0 ∈ B k − nk → ∈ B0 Proof. (a) Exercise. (b) Assume that T are compact operators and that T T in operator norm. By n n → Proposition 4.7, it suffices to show that T (BallH ) is a totally bounded subset of K. Choose any ε > 0. Then there exists an n such that T T < ε . Now, T is compact, k − nk 3 n so Tn(BallH ) is totally bounded. Hence there exist finitely many points h1, . . . , hm BallH such that ∈ m ε Tn(BallH ) B Tnhj, 3 . (4.1) ⊆ j=1  We will show that T (BallH ) is totally boundedSby showing that m T (BallH ) B Tnhj, ε . (4.2) ⊆ j=1 S  Choose any element of T (BallH ), i.e., any point T f with f 1. Then Tnf Tn(BallH ), so by (4.1) there must be some j such that T f T h k< kε .≤Consequently, ∈ k n − n jk 3 T f T h T f T f + T f T h + T h T h k − jk ≤ k − n k k n − n jk k n j − jk ε < T T f + + T T h k − nk k k 3 k n − k k jk ε ε ε < 1 + + 1 3 · 3 3 · = ε. Hence (4.2) follows, so T is compact. 

Exercise 4.16. Another way to prove Theorem 4.15 is to apply a Cantor diagonalization argument. Fill in the details in the following sketch of this argument. Suppose that fn n∈N is a bounded sequence in H. Then since T1 is compact, there exists a (1){ } (1) subsequence fn n∈N of fn n∈N such that T1fn n∈N converges. Then since T2 is compact, { } { }(2) (1){ } (2) there exists a subsequence fn n∈N of fn n∈N such that T2fn n∈N converges (and note (2) { } { } { } that T1fn n∈N also converges!). Continue to construct subsequences in this way, and then { } (n) show that the “diagonal subsequence” T fn n∈N converges (use the fact that there exists a k such that T T < ε). Therefore{T is compact.} k − kk

Theorem 4.17 (Compositions and Compact Operators). Let H1, H2, H3 be Hilbert spaces.

(a) If A: H1 H2 is bounded and T : H2 H3 is compact, then T A: H1 H3 is compact. → → → CHAPTER 2. OPERATORS ON HILBERT SPACES 27

(b) If T : H1 H2 is compact and A: H2 H3 is bounded, then AT : H1 H3 is compact. → → →

Proof. (b) Assume that A is bounded and T is compact. Let f N be any bounded { n}n∈ sequence in H . Then since T is compact, there is a subsequence T f N that converges 1 { nk }k∈ in H2. Since A is bounded, the subsequence AT fnk k∈N therefore converges in H3. Hence AT is compact. { } (a) Exercise. 

Exercise 4.18. Prove that if T (H, K), then range(T ) is a separable subspace of K. ∈ B0 Hints: Since T (Ball ) is compact, it is totally bounded. Hence for each n N we can H ∈ find finitely many balls of radius 1/n with centers in T (BallH ) that cover T (BallH ). If we consider all these balls for every n, we have countably many balls that cover T (BallH ). Show that this implies that T (BallH ) contains a countable, dense subset. Then do the same for each ball of radius k N instead of just k = 1. Combine all of these together to get a ∈ countable dense subset of range(T ).

Definition 4.19 (Finite-Rank Operators). Recall that the rank of an operator T : H K is the dimension of range(T ). We say that T is a finite-rank operator if range(T ) is finite-→ dimensional. We set (H, K) = T (H, K) : T is finite-rank , B00 { ∈ B } and set (H) = (H, H). B00 B00 A linear, finite-rank operator need not be bounded (that is why we include the assumption of boundedness in the definition of 00(H, K) above). However, the following result shows that if a finite-rank operator is bounded,B then it is actually compact.

Proposition 4.20. If T : H K is bounded, linear, and has finite rank, then T is compact. Thus, → (H, K) (H, K). B00 ⊆ B0 Proof. Since T is bounded, T (BallH ) is a bounded subset of the finite-dimensional space range(T ). All finite-dimensional spaces are closed. Hence the closure of T (BallH ) is a closed and bounded subset of range(T ), and therefore is compact. 

This gives us the following very useful way to show that a general operator T is compact: try to construct a sequence of finite-rank operators Tn that converge to T in operator norm.

Corollary 4.21. Suppose that Tn (H, K) are finite-rank operators, T (H, K), and T T in operator norm. Then T ∈is Bcompact. ∈ B n →

Exercise 4.22. Show that if E (H) is compact and idempotent, then E has finite rank. ∈ B 28 CHRISTOPHER HEIL

Example 4.23. Let e N be an orthonormal basis for a separable Hilbert space H, and { n}n∈ let λ = (λn)n∈N be a bounded sequence of scalars. Then we know from Example 1.7 that ∞ Lf = λ f, e e n h ni n n=1 X defines a bounded operator on H. Suppose that λ 0 as n . Define n → → ∞ N L f = λ f, e e . N n h ni n n=1 X Since range(LN ) span e1, . . . , eN (must it be equality?), we have that LN is finite-rank. (Exercise: Show that⊆ L is{ not finite-rank} if there are infinitely many λ = 0.) n 6 Further, LN is a good approximation to L, because (using the ) we have ∞ 2 (L L )f 2 = λ f, e e k − N k n h ni n n=N+1 X ∞

= λ 2 f, e 2 | n| |h ni| n=N+1 X ∞ 2 2 sup λn f, en ≤ n>N | | |h i|   n=XN+1 2 2 sup λn f . ≤ n>N | | k k   It follows that LN converges to L in operator norm:

2 2 2 lim L LN lim sup λn = lim sup λn = 0. N→∞ k − k ≤ N→∞ n>N | | N→∞ | |   Since each LN is compact, we conclude that L is compact as well.

Exercise 4.24. Continuing the preceding example, prove the following. (a) Prove that if λn does not converge to zero then L is not compact. Hint: We know at least some of the eigenvectors of L. (b) Prove that, with only the assumption that λ `∞, we have ∈ f H, L f Lf. (4.3) ∀ ∈ N → That is, for each individual vector f we have Lf LN f 0, where this is the norm in H. A sequence of operators which satisfies k(4.3)−is saidkto→converge strongly or in the (SOT). Prove that strong convergence of operators does not imply convergence in operator norm, i.e., (4.3) does not imply that L L 0. k − N k → (c) Assuming that λ `∞, prove that L is self-adjoint if and only if every λ is real. ∈ n CHAPTER 2. OPERATORS ON HILBERT SPACES 29

We can characterize all the finite-rank operators, as follows.

Proposition 4.25 (Finite-Rank Operators). Let L: H K be bounded and linear. Then the following statements are equivalent. → (a) L has finite rank. (b) There exist vectors ϕ , . . . , ϕ H and ψ , . . . , ψ K such that 1 N ∈ 1 N ∈ N Lf = f, ϕ ψ , f H. (4.4) h ki k ∈ Xk=1 Proof. (a) (b). Since L has finite rank, we know that range(L) is a finite-dimensional subspace of⇒K. Every finite-dimensional subspace is closed, so we can find a finite orthonor- N mal basis ψk k=1 for range(L). Therefore, if f H then we can express Lf in terms of this orthonormal{ basis:} ∈ N N N Lf = Lf, ψ ψ = f, L∗ψ ψ = f, ϕ ψ , h ki k h ki k h ki k k=1 k=1 k=1 ∗ X X X where ϕk = L ψk. (b) (a). We have range(L) span ψ , . . . , ψ .  ⇒ ⊆ { 1 N }

Corollary 4.26. If L (H, K) has rank 1, then there exist ϕ H and ψ K such that ∈ B ∈ ∈ Lf = f, ϕ ψ, f H. h i ∈ In particular, if ϕ = ψ are unit vectors, then Lf = f, ϕ ϕ is the orthogonal projection of H onto span ϕ . h i { }

Exercise 4.27. Compute the adjoint of L given by (4.4). Conclude that the adjoint of a finite-rank operator is also finite-rank.

Exercise 4.28. Show that if A (H) and AT = T A for every finite-rank T then A = cI for some scalar c. ∈ B

Exercise 4.29. Use the idea of the Proposition 4.25 to show that if H is separable and L: H H is any bounded linear operator, then there exist finite-rank operators L that → N converge to L in the strong operator topology, i.e., Lf LN f 0 for each individual f. However, observe that if L is not compact, then we kcannot− havekL→ L in operator norm. N → The following result shows that not only is the operator norm limit of a sequence of finite- rank operators compact, but every compact operator can be realized as the operator norm limit of finite-rank operators.

Theorem 4.30. If T (H, K), then the following statements are equivalent. ∈ B 30 CHRISTOPHER HEIL

(a) T is compact. (b) There exist finite-rank operators T (H, K) such that T T . n ∈ B n → As a consequence, we have that (H, K) is a dense subspace of (H, K), i.e., B00 B0 (H, K) = (H, K) (closure in operator norm). B00 B0 Proof. (b) (a). This follows from Theorem 4.15 and the fact that all bounded finite-rank operators are⇒ compact. (a) (b). Assume that T is compact. Let R = range(T ). If R is finite-dimensional, then T⇒is finite-rank, and we are done. So, assume that R is infinite-dimensional. By Exercise 4.18, we know that R is separable, so there exists a countable orthonormal basis e N for R. For any f H we have T f R, so { n}n∈ ∈ ∈ ∞ T f = T f, e e , f H. h ni n ∈ n=1 X Define N T f = T f, e e , f H, N h ni n ∈ n=1 X and note that TN = PN T where PN is the orthogonal projection of K onto the closed subspace span e , . . . , e . By definition, we have that T converges to T in the strong { 1 N } N operator topology, i.e., TN f T f for every f. Our goal is to show more, namely, to show that T T in operator norm.→ That is, we need to show that sup T f T f 0. N → kfk=1 k − N k → Choose any ε > 0. Since T (BallH ) is totally bounded, it is covered by finitely many ε-balls centered at points in T (Ball ). Hence, there exist h , . . . , h H such that H 1 m ∈ m ε T (BallH ) B T hk, . ⊆ k=1 3   Since lim T h T h = 0 for k = 1, . .S. , m, we can find an N such that N→∞ k k − N kk 0 ε N > N , T h T h < , k = 1, . . . , m. ∀ 0 k k − N kk 3 Choose any f with f = 1 and any N > N . Then T f B T h , ε for some k, i.e., k k 0 ∈ k 3 ε T f T h < .  k − kk 3 Therefore we also have N T f T h = T (f h ), e e k N − N kk h − k ni n n=1 X N 1/2

= T (f h ), e 2 |h − k ni| n=1 X  CHAPTER 2. OPERATORS ON HILBERT SPACES 31

∞ 1/2 ε T (f h ), e 2 = T f T h < . ≤ |h − k ni| k − kk 3 n=1 X  Alternatively, this follows even more simply from the fact that ε T f T h = P T f P T h P T f T h < 1 . k N − N kk k N − N kk ≤ k N k k − kk · 3 In any case, it follows that ε ε ε T f T f T f T h + T h T h + T h T f < + + = ε. k − N k ≤ k − kk k k − N kk k N k − N k 3 3 3 This is true for every unit vector, so we have T TN ε for all N > N0. Therefore, we do indeed have T T 0. k − k ≤  k − N k → Corollary 4.31. If T (H, K), then ∈ B T is compact T ∗ is compact. ⇐⇒ Proof. Assume that T is compact. Then there exist finite-rank operators TN such that ∗ ∗ ∗ ∗ TN T . Hence TN T (why?), but each TN is finite-rank, so T is compact. The conv→erse is symmetrical.→ 

Exercise 4.32. Extend Example 4.23 as follows. Let H be a separable Hilbert space, and ∞ let e N be an orthonormal basis for H. Let λ = (λ ) N ` (N) be given. Define { n}n∈ n n∈ ∈ Len = λnen. Prove that the definition of L can be extended to all of H in such a way that L is a bounded linear operator. Prove that this operator L is compact if and only if λ 0. n → The next result shows that an integral operator with a square-integrable kernel is compact. Theorem 4.33. Let (X, Ω, µ) be a σ-finite measure space. If k L2(X X), then the integral operator ∈ × Lf(x) = k(x, y) f(y) dµ(y), f L2(X), ∈ ZX defines a compact mapping of L2(X) into itself. Further, L k . k k ≤ k k2 Proof. Note that by Theorem 1.22 we already know that L defines a bounded operator, and that L k 2. So, we need only show that L is compact. Forksimplicitk ≤ k ky, we will consider only the case where L2(X) is separable. In this case there 2 exists an orthonormal basis e N for L (X). Define { n}n∈ e (x, y) = e (x) e (y), x, y X. mn m n ∈ 2 Then it is easy to see that emn m,n∈N is an orthonormal sequence in L (X X), and with more work (exercise3) it can{ be} shown that that it is also complete and hence× forms an orthonormal basis for L2(X X). Since k L2(X X), we therefore have × ∈ × ∞ ∞ k = k, e e , h mni mn m=1 n=1 X X 3For details on this type of argument, see the “Real Analysis Review” handout on the instructor’s webpage. 32 CHRISTOPHER HEIL

where this series converges in the norm of L2(X X), and in fact it converges unconditionally. For each N N define an approximation to k ×by setting ∈ N N k = k, e e . N h mni mn m=1 n=1 X X Then k k in L2-norm. N → Now define an approximation to L by defining LN to be the integral operator with kernel kN , i.e., 2 LN f(x) = kN (x, y) f(y) dµ(y), f L (X). X ∈ 2 Z Since kN L (X X), we know that LN is bounded. Further, since the sums involved are finite, we ∈can interc×hange sums and integrals in the following calculation to obtain that

LN f(x) = kN (x, y) f(y) dµ(y) ZX N N = k, e e (x, y) f(y) dµ(y) h mni mn X m=1 n=1 Z X X N N = k, e e (x) e (y) f(y) dµ(y) h mni m n m=1 n=1 X X X Z N N = k, e f, e e (x). h mni h ni m m=1 n=1 X X This is an equality of functions, i.e., L f = N N k, e f, e e a.e. In any case N m=1 n=1 h mni h ni m we have LN f span e1, . . . , eN , so LN has finite rank. Since LN is bounded (why?), it is therefore compact.∈ { } P P Consequently, if we can show that LN L, then we can conclude that L itself is compact. → 2 Note that L LN is simply the integral operator with kernel k kN . Since k kN L (X X), we know that− L L is bounded, and that − − ∈ × − N L L k k 0 as N . k − N k ≤ k − N k2 → → ∞ Hence L L in operator norm, so L is compact.  N → For the remainder of this section, we consider eigenvalues and eigenvectors of compact operators.

Definition 4.34. Let A (H) be given. ∈ B (a) A scalar λ F is an eigenvalue of A if there exists a nonzero vector f H such that Af = λf. Equiv∈ alently, λ is an eigenvalue if ker(A λI) = 0 . ∈ − 6 { } (b) If λ F is an eigenvalue of A, then any nonzero vector in ker(A λI) is called an eigenve∈ ctor of A corresponding to the eigenvalue λ, or simply a λ-eigenve− ctor for short. Equivalently, a nonzero vector f H is a λ-eigenvector if Af = λf. ∈ CHAPTER 2. OPERATORS ON HILBERT SPACES 33

(c) If λ F is an eigenvalue of A, then ker(A λI) is called the eigenspace of A corresp∈onding to the eigenvalue λ, or simply a λ−-eigenspace for short. Every nonzero vector in the λ-eigenspace is a λ-eigenvector of A.

(d) The point spectrum σp(A) of A is the set of eigenvalues of A: σ (A) = λ F : λ is an eigenvalue of A . p { ∈ }

Exercise 4.35. Let en n∈N be an orthonormal basis for a separable Hilbert space H, and let ∞ { } λ = (λn)n∈N ` (N) be fixed. Let L: H H be the bounded operator Lf = λn f, en en defined in Example∈ 1.7. → h i P (a) Show that σ (L) = λ : n N . p { n ∈ } (b) Show that if µ is one component of λ and J = n N : λn = µ , then the µ- eigenspace of L is span e . { ∈ } { n}n∈J (c) Show that the eigenspaces of L corresponding to distinct eigenvalues are orthogonal.

Exercise 4.36. Let L (H) be given. Prove the following. ∈ B (a) If L is self-adjoint, then all eigenvalues of L are real. (b) If L is positive ( Lf, f 0 for all f), then all eigenvalues of L are nonnegative. h i ≥ (c) If L is positive definite ( Lf, f > 0 for all f = 0), then all eigenvalues of L are strictly positive. h i 6 (d) If L is unitary, then every eigenvalue λ satisfies λ = 1. | |

Exercise 4.37. Suppose that L (H) is normal. Prove that if λ = µ are distinct eigenvalues of L, then the corresp∈ondingB eigenspaces are orthogonal, i.e.,6 ker(L λI) ker(L µI). − ⊥ − While any linear operator A: Cn Cn must have an eigenvalue, bounded operators on infinite-dimensional Hilbert spaces need→ not have any eigenvalues.

Exercise 4.38. (a) Prove that the Volterra operator defined in Exercise 1.25 is compact but has no eigenvalues, i.e., its point spectrum is empty. (b) Prove that the right- R on `2(N) has no eigenvalues. (c) Prove that every scalar λ < 1 is an eigenvalue of the left-shift operator L on `2(N), and find the corresponding eigenv|ectors.| Thus, this operator has uncountably many eigenvalues. (d) Let φ(x) = x. Prove that the multiplication operator M : L2[0, 1] L2[0, 1], defined φ → by Mφf(x) = xf(x), is self-adjoint but has no eigenvalues. (e) Define i, y x, k(x, y) = ≤ i, y > x, (− 34 CHRISTOPHER HEIL

and let L: L2[0, 1] L2[0, 1] be the integral operator with kernel k. Prove that L is both → 2 compact and self-adjoint. Prove that the eigenvalues of L are λk = (2k+1)π (and only these), and find the corresponding eigenvectors.

Exercise 4.39 (Convolution). Fix g L2[0, 1], where we consider functions in L2[0, 1] to be 1-periodically extended to the real line.∈ Let T be the convolution operator 1 T f(x) = (f g)(x) = g(x y) f(y) dy. ∗ − Z0 (a) Prove that T is compact. Hint: Write T as an integral operator and show that its kernel is square-integrable. Note that the fact that [0, 1] has finite measure is important. 2πinx (b) Prove that the complex exponential functions en(x) = e are eigenvectors of T , with corresponding eigenvalues gˆ(n) (the Fourier coefficients of g).

Exercise 4.40. (a) Assume that A (H) is normal and let λ F be given. Show that A λI is normal. Use this to show ∈thatB ker(A λI) = ker(A∗ ∈λI¯ ). Conclude that if λ is −an eigenvalue of A then λ¯ is an eigenvalue of −A∗, and the corresp− onding eigenspaces are equal. Hint: Consider Corollary 2.21. (b) Find an example of a non- for which the conclusion of part (a) fails. Hint: Consider a shift operator.

The next result shows that the eigenspaces (if any) of a compact operator corresponding to nonzero eigenvalues must be finite-dimensional.

Proposition 4.41. Assume that T : H H is compact and that λ = 0 is an eigenvalue of T . Then ker(T λI) is finite-dimensional.→ 6 − Proof. Since T is bounded, we know that ker(T λI) is a closed subspace of H. Suppose that − it was infinite-dimensional. Then we could find an infinite orthonormal sequence e N in { n}n∈ ker(T λI). Each en is a λ-eigenvector, i.e., T en = λen. But then en n∈N is a bounded sequence− in H yet { } T e T e = λ e e = λ √2, k m − nk | | k m − nk | | so since λ = 0 there can be no convergent subsequences of T en n∈N, which contradicts the fact that T6 is compact. { } 

The following is one useful theoretical result which implies the existence of an eigenvalue of a compact operator T . It states that if infkfk=1 T f λf = 0, then this infimum is actually achieved, i.e., T f λf = 0 for some unit vkector−f, ork in other words, there exists a λ-eigenvector for T . k − k

Proposition 4.42. Assume that T : H H is compact and that λ = 0 is given. Then: → 6 inf T f λf = 0 = λ σp(T ). kfk=1 k − k ⇒ ∈ CHAPTER 2. OPERATORS ON HILBERT SPACES 35

Proof. Assume that inf T f λf = 0. Then we can find unit vectors f such that kfk=1 k − k n T fn λfn 0. Since T is compact, T fn n∈N has a convergent subsequence, say T fnk gk H−. Sincek →λ = 0 we have { } → ∈ 6

λfnk T fnk + T fnk 0 + g g fnk = − = . (4.5) λ  → λ λ Since the f are unit vectors, we conclude that g = 0. Moreover, since T is continuous it nk 6 follow from (4.5) that T fnk T g/λ. But we also know that T fnk g, so we conclude that T g/λ = g, or in other words→that g is a λ-eigenvector. → 

Corollary 4.43. Assume T : H H is compact and that λ = 0. If λ / σp(T ) and ∗ → 6 ∈ −1 λ¯ / σp(T ), then range(T λI) is a bounded bijection of H onto itself, and (T λI) is bounded.∈ − − Proof. Since λ is not an eigenvalue, we know that T λI is injective. Further, it follows − from the preceding proposition that infkfk=1 T f λf > 0. Hence there exists a C > 0 such that T f λf C for every unit vectork f,−and hencek k − k ≥ f H, T f λf C f . (4.6) ∀ ∈ k − k ≥ k k It follows from Exercise 1.2 than range(T λI) is a closed subspace of H. But then, since λ¯ is not an eigenvalue of T ∗, we have that− ⊥ range(T λI) = range(T λI) = ker (T λI)∗ = ker(T ∗ λI¯ )⊥ = 0 ⊥ = H. − − − − { } Thus T λI is a bounded bijection. It remains to show that (T λI)−1 is bounded. Given f H w−e have from (4.6) that − ∈ f = (T λI)(T λI)−1f C (T λI)−1f . k k k − − k ≥ k − k Rearranging, we see that (T λI)−1 1 < .  k − k ≤ C ∞

Actually, it can be shown that if T : H H is compact, λ = 0, and λ / σp(T ), then λ¯ / σ (T ∗) follows automatically. → 6 ∈ ∈ p 5. The Diagonalization of Compact Self-Adjoint Operators First let us summarize the facts that have been developed regarding the operator L intro- duced in Example 1.7 and studied in other examples in previous sections.

Theorem 5.1. Let en n∈N be an orthonormal basis for a separable Hilbert space H, and let λ = (λ ) `∞{(N}) be a bounded sequence of scalars. Define n n∈∞ ∈ ∞ Lf = λ f, e e , f H. (5.1) n h ni n ∈ n=1 X Then the following statements hold. (a) L is bounded, and L = λ . k k k k∞ 36 CHRISTOPHER HEIL

(b) L is normal, and L∗f = ∞ λ¯ f, e e . n=1 n h ni n (c) L is self-adjoint if and only if λ R for every n. P n ∈ (d) L is compact if and only if λ 0. n →

Exercise 5.2. Assume that λ 0 and that L is defined by (5.1). In the definition of L, → combine those terms corresponding to identical λn together. That is, let µ = (µk)k∈I be the sequence of distinct values in λ (so I is either 1, . . . , N if there are only finitely many { } distinct values, or I = N if there are infinitely many). If we set Jk = n N : λn = µk , then { ∈ } P f = f, e e k h ni n n∈J Xk is the orthogonal projection of H onto span en n∈Jk . Show that the operator L defined in (5.1) can be rewritten as { } Lf = µ P f, f H, k k ∈ Xk∈I with convergence of the series in the norm of H. Show further that

L = µk Pk, Xk∈I with convergence of the series in operator norm. Show that span e is the µ -eigenspace { n}n∈Jk k of L. Show that PjPk = PkPj = 0 for all j = k N, and consequently the eigenspaces corresponding to distinct eigenvalues are orthogonal.6 ∈

In this section we will prove a converse result, showing that all compact, self-adjoint operators on a Hilbert space can be represented in the form of (5.1). First, however, we need to develop some useful machinery.

Exercise 5.3. If λ is an eigenvalue of L (H), then λ L . ∈ B | | ≤ k k

Exercise 5.4. Let A be an n n complex matrix. Define its to be × ρ(A) = max λ : λ is an eigenvalue of A = max λ : λ σ (A) . {| | } {| | ∈ p } By the preceding exercise, if we choose any norm on Cn and let A be the corresponding operator norm, we have k k ρ(A) A . ≤ k k (a) Prove that if A is self-adjoint and we use the Euclidean (`2) norm on Cn, then A = ρ(A). k k (b) Prove that the same equality holds if A is normal. Find an example of a non-normal matrix for which ρ(A) < A . k k CHAPTER 2. OPERATORS ON HILBERT SPACES 37

(c) Prove that if A is any n n matrix, then (still using the Euclidean norm on Cn), × A = ρ(A∗A)1/2. k k

(d) (Harder). Prove that A is a fixed but arbitrary n n complex matrix and ε > 0 is given, then there exists a norm on Cn such that the corresp× onding operator norm of A satisfies A ρ(A) + ε. k k ≤ Although an arbitrary compact operator need not have any eigenvalues (see Exercise 4.38), the following result shows that a compact, self-adjoint operator must have at least one eigenvalue.

Proposition 5.5. If T : H H is compact and self-adjoint, then either T or T is an eigenvalue of T . → k k −k k Proof. Since T is self-adjoint, we know from Proposition 2.18 that T = sup T f, f . k k kfk=1 |h i|

Hence, there must exist unit vectors fn such that T fn, fn T . Since T is self-adjoint, all the inner products T f , f are real, so we can|hfind a subsequencei| → k k that converges either h n ni to T or to T . Call this subsequence g N, and let λ be either T or T as k k −k k { n}n∈ k k −k k appropriate. Then we have gn = 1 for every n, and T gn, gn λ. Hence, since both λ and T g , g are real, k k h i → h n ni T g λg 2 = T g 2 2λ T g , g + λ2 g 2 k n − nk k nk − h n ni k nk T 2 g 2 2λ T g , g + λ2 g 2 ≤ k k k nk − h n ni k nk = λ2 2λ T g , g + λ2 − h n ni λ2 2λ2 + λ2 = 0. → − It therefore follows from Proposition 4.42 that λ is an eigenvalue of T . 

Now we can prove that every compact, self-adjoint operator has a very simple and special form.

Theorem 5.6 ( for Compact Self-Adjoint Operators). Let T : H H be compact and self-adjoint. Then there exist nonzero real numbers λ , either finitely→ { n}n∈J many or λ 0 if infinitely many, and an orthonormal basis e N of range(T ), such that n → { n}n∈ T f = λ f, e e , f H. n h ni n ∈ n∈J X Each λn is an eigenvalue of T , and each en is a corresponding eigenvector. 38 CHRISTOPHER HEIL

Proof. Note that since T is compact, range(T ) is separable by Exercise 4.18. If T = 0 then the result is trivial, so assume that T is not the zero operator. Let H1 = H and T1 = T . By Proposition 5.5, T1 has an eigenvalue λ1 which satisfies λ1 = T1 > 0. Let e1 be a corresponding eigenvector, normalized to e1 = 1. | | k k ⊥ k k Let H2 = e1 and let T2 = T H2 (the restriction of T to H2). If T2 = 0, then stop at this point. Otherwise,{ } continue as |follows. Since span e1 is invariant under T1 (after all, e1 is an eigenvector), we know from Exer- { } ∗ cise 3.10 that H2 is invariant under T1 = T1. Exercise: Show that T2 : H2 H2 is compact and self-adjoint. Therefore T has an eigenvalue λ such that λ = T →> 0. Note that 2 2 | 2| k 2k since T2 is a restriction of T1, we have λ2 = T2 T1 = λ1 . Let e2 be a corresponding eigenvector, normalized to e = 1. Note| |thatk bykdefinition≤ k k of| H| , we have e e . Further, k 2k 2 2 ⊥ 1 λ2 is an eigenvalue of T (not just T2), and e2 is the corresponding eigenvector of T . Let H = e , e ⊥ and let T = T . If T = 0, then stop at this point. Otherwise, 3 { 1 2} 3 |H3 3 continue as before to construct an eigenvalue λ3 and eigenvector e3 (which will be orthogonal to both e1 and e2). Continuing in this process, there are two possibilities. Case 1: T = 0 for some N. In this case, since H = e , . . . , e ⊥, we have N+1 N+1 { 1 N } H = span e , . . . , e H . { 1 N } ⊕ N+1 Therefore, if f H then we can write f uniquely as ∈ N f = f, e e + v h ni n n=1 X where v H . Since T (v) = T (v) = 0, we therefore have ∈ N+1 N+1 N N T f = f, e T (e ) + T (v) = λ f, e e . h ni n n h ni n n=1 n=1 X X In this case T is finite-rank and the proof is complete. Case 2: T = 0 for any N. In this case we obtain countably many eigenvalues λ and N 6 n corresponding orthonormal eigenvectors en. Since T is compact, we have by Exercise 4.10 that λnen = T (en) 0. Since en = 1, we conclude that λn 0. → k k → ⊥ Let M = span en n∈N. Then en n∈N is an orthonormal basis for M, and H = M M . Hence, if f H {then} we can write{ f} uniquely as ⊕ ∈ ∞ f = f, e e + v h ni n n=1 X for some v M ⊥. Therefore ∈ ∞ ∞ T f = f, e T (e ) + T (v) = λ f, e e + T (v). h ni n n h ni n n=1 n=1 X X If we show that T (v) = 0, then we are done. CHAPTER 2. OPERATORS ON HILBERT SPACES 39

⊥ ⊥ Note that since span e1, . . . , eN M, we have v M span e1, . . . , eN = HN . Hence { } ⊆ ∈ ⊆ { } T (v) = T (v) T v = λ v 0 as N . k k k N k ≤ k N k k k | N | k k → → ∞ Consequently T (v) = 0. 

Since each eigenspace corresponding to nonzero eigenvalues is finite-dimensional, we can group terms corresponding to the same eigenvalue together. Alternatively, we could write a more efficient proof of the Spectral Theorem (as Conway does), by using the same argument on the distinct eigenvalues and corresponding eigenspaces, instead of one eigenvalue and eigenvector at a time. Either way, an extension of the ideas used in the preceding result gives the following expanded form of the Spectral Theorem.

Theorem 5.7 (Spectral Theorem for Compact Self-Adjoint Operators). Let T : H H be compact and self-adjoint. Then the following statements hold. → (a) T has only a finite or countably infinite number of distinct eigenvalues, and each eigenvalue is real.

(b) Let µ1, µ2, . . . = µk k∈I be the distinct nonzero eigenvalues, where either I = 1, . {. . , N or I}= N.{ Then} each eigenspace { } E = ker(T λµ ) k − k is finite-dimensional. (c) If I is infinite, then µ 0 as k . k → → ∞

(d) If Pk is the orthogonal projection onto the eigenspace Ek, then PjPk = PkPj = 0 for j = k I. That is, eigenspaces corresponding to distinct eigenvalues are orthogonal. 6 ∈ (e) We have

T = µkPk, Xk∈I where the series converges in operator norm.

(f) There exist nonzero real numbers λn n∈J , either finitely many or λn 0 if infinitely many, and an orthonormal sequence{ }e such that → { n}n∈J T f = λ f, e e . n h ni n Xn∈J The λn are obtained by repeating each µk according to its multiplicity (the dimension of the eigenspace E ). The sequence e forms an orthonormal basis for k { n}n∈J ker(T )⊥ = range(T ).

Corollary 5.8. If T : H H is compact, self-adjoint, and injective, then H is separable. → Proof. Since T = T ∗, we have that range(T ) = ker(T ∗)⊥ = 0 ⊥ = H. On the other hand, { } since T is compact we know from Exercise 4.18 that range(T ) is separable.  40 CHRISTOPHER HEIL

Example 5.9 (Diagonalization of Self-Adjoint Matrices). Let us examine what the Spectral Theorem says in finite dimensions. Let A be a self-adjoint n n matrix (i.e., symmetric if real, and Hermitian if complex). Then the Spectral Theorem× says that there exist real nonzero eigenvalues λ1, . . . , λk and corresponding orthonormal eigenvectors u1, . . . , uk such that k Ax = λ (x u ) u . (5.2) j · j j j=1 X We can extend this representation by including the zero eigenvalues of A, as follows. From (5.2), we see that the column space, or range, of A is C(A) = range(A) = span u , . . . , u . { 1 k} Since A is self-adjoint, its nullspace is the of its column space, for N(A) = ker(A) = range(A)⊥ = C(A)⊥.

Let uk+1, . . . , un be an orthonormal basis for N(A), and let λk+1 = = λn = 0. Then n · · · u1, . . . , un is an orthonormal basis for C with corresponding eigenvectors λ1, . . . , λn. Further, we have the following representations: n n x = (x u ) u and Ax = λ (x u ) u , x Fn. · j j j · j j ∈ j=1 j=1 X X Let us rewrite this representation as follows: n Ax = λ (x u ) u j · j j j=1 X λ (x u ) 1 · 1 = u| u| .  1 · · · n   .  | | λn (x un)    ·  λ1 x u1 | | . ·. = u1 un .. .  · · ·     .  u x u | | n · n      H λ1 — u1 — | | . . = u1 un .. . x  · · ·     H  | | un — un —       = UΛU Hx, where U is the matrix that has u1, . . . , un as columns, and Λ is the diagonal matrix with λ1, . . . , λn on the diagonal. On the one hand, this is nothing more than the diagonalization of A. However, this says much more: every self-adjoint matrix can be diagonalized (even if some eigenvalues are repeated), and furthermore, the eigenvector matrix is unitary (because it has orthonormal columns). We summarize this next as a theorem. CHAPTER 2. OPERATORS ON HILBERT SPACES 41

Theorem 5.10 (Diagonalization of Self-Adjoint Matrices). Let A be an n n matrix. Then the following statements are equivalent. × (a) A is self-adjoint. (b) A = UΛU ∗ where U is unitary and Λ is diagonal with real scalars on its diagonal.

(c) There exist real scalars λ1, . . . , λn an orthonormal vectors u1, . . . , un such that n Ax = λ (x u ) u , x Fn. j · j j ∈ j=1 X n (d) There exists an orthonormal basis u1, . . . , un for F consisting of eigenvectors of A with corresponding real eigenvalues λ1, . . . , λn.