Functional Analysis

July 12, 2007

Contents

1 Metric Spaces 3 1.1 OpenSets,ClosedSets...... 5 1.2 Convergence, Cauchy Sequence, Completeness ...... 7 1.3 CompletenessProofs ...... 9 1.4 CompletionofMetricSpaces ...... 10

2 Normed Spaces 13 2.1 Finite dimensional Normed Spaces or Subspaces ...... 15 2.2 Compactness and Finite Dimension ...... 18 2.3 LinearOperators ...... 22 2.4 Bounded and Continuous Linear Operators ...... 24 2.5 LinearFunctionals ...... 29 2.6 FiniteDimensionalCase ...... 32 2.6.1 LinearFunctionals ...... 33 2.7 DualSpace ...... 35

3 Hilbert Spaces 39 3.1 Representation of Functionals on Hilbert Spaces ...... 49 3.2 HilbertAdjoint ...... 53

4 Fundamental Theorems 59 4.1 Bounded, Linear Functionals on C[a, b]...... 68 4.2 AdjointOperator...... 71 4.3 ReﬂexiveSpaces ...... 75 4.4 BaireCategoryTheorem...... 78 4.5 StrongandWeakConvergence ...... 81 4.6 TheOpenMappingTheorem ...... 88 4.7 ClosedGraphTheorem ...... 89

1 5 Exam 1 93

6 Exam 2 95

7 Final Exam 96

2 1 Metric Spaces

Deﬁnition (Metric Space). The pair (X, d), where X is a set and the function

d : X X R × → is called a metric space if

1. d(x,y) 0 ≥ 2. d(x,y) = 0 iﬀ x = y

3. d(x,y)= d(y,x)

4. d(x,y) d(x, z)+ d(z,y) ≤ Example 1.1 (Metric Spaces).

1. d(x,y)= x y in R. | − | n 2 1 n 2. d(x,y) = [ (xi yi) ] 2 in R . i=1 − 3. d(x,y)= Px y in a normed space. k − k 4. Let (X, ρ), (Y,σ) be metric spaces and deﬁne the Cartesian product X Y = × 2 (x,y) x X,y Y . Then the product measure τ((x1,y1), (x2,y2)) = [ρ(x1,x2) + { | 2∈1 ∈ } σ(y1,y2) ] 2 . ¯ ¯ 5. (Subspace) (Y, d) of (X, d) if Y X and d = d Y Y . ⊂ | × 6. l∞. Let X be the set of all bounded sequences of complex numbers, i.e., x = (ξi) and ξi cx, i. Then | |≤ ∀

d(x,y) = sup ξi ηi i N | − | ∈ deﬁnes a metric on X.

7. X = C[a, b] and d(x,y)= max x(t) y(t) . t [a,b] | − | ∈ 8. (Discrete metric) 1 x = y d(x,y)= . 0 x = y 6

3 p p p 9. l . x = (ξi) l if ∞ ξi < , (p 1, ﬁxed), ∈ i=1 | | ∞ ≥ 1 P p ∞ p d(x,y)= ξi ηi . | − | i ! X=1 Problem 1. 1. Show that d¯ is a metric on C[a, b], where b d¯(x,y)= x(t) y(t) dt. | − | Za 2. Show that the discrete metric is a metric. 3. Sequence space s: set of all sequences of complex numbers with the metric

∞ 1 ξi ηi d(x,y)= | − | . (1) 2i 1+ ξ η i i i X=1 | − | Solution. 1. d¯(x,y) = 0 iﬀ x(t) y(t) = 0 for all t [a, b] because of the continuity. We have | − | ∈ d¯(x,y) 0 and d¯(x,y) = d¯(y,x) trivially. We can argue the triangle inequality as ≥ follows:: b b b d¯(x,y)= x(t) y(t) dt x(t) z(t) dt+ z(t) y(t) dt = d¯(x, z)+d¯(z,y). | − | ≤ | − | | − | Za Za Za 2. Left as an exercise. 3. We show only the triangle inequality. Let a, b R. Then we have the inequalities ∈ a + b a + b a b | | | | | | | | + | | , 1+ a + b ≤ 1+ a + b ≤ 1+ a 1+ b | | | | | | | | | | where in the ﬁrst step we have used the monotonicity of the function x 1 f(x)= = 1 , for x> 0. 1+ x − 1+ x

Substituting a = ξi ζi and b = ζi ηi, where x = (ξi), y = (ηi), and z = (ζi) we get − − ξi ηi ξi ζi ζi ηi | − | | − | + | − | . 1+ ξi ηi ≤ 1+ ξi ζi 1+ ζi ηi | − | | − | | − | If we multiply both sides by 1 and sum over from i = 1 to we get the stated 2i ∞ result.

4 1.1 Open Sets, Closed Sets Deﬁnition (Open Ball, Closed Ball, Sphere).

1. B(x , r)= x X d(x,x ) < r 0 { ∈ | 0 } 2. B¯(x , r)= x X d(x,x ) r 0 { ∈ | 0 ≤ } 3. S(x , r)= x X d(x,x )= r 0 { ∈ | 0 } Deﬁnition (Open, Closed, Interior).

1. M is open if contains a ball about each of its points.

2. K X is closed if Kc = X K is open. ⊂ − 3. B(x0; ε) denotes the ε neighborhood of x0. 4. Int(M) denotes the interior of M.

Remark 1.2 (Induced Topology). Consider the set X with the collection τ of all open subsets of X. Then we have

1. τ, X τ. ∅∈ ∈ 2. The union of any members of τ is a member of τ.

3. The ﬁnite intersection of members of τ is a member of τ.

We call the pair (X,τ) a topological space and τ a topology for X. It follows that a metric space is a topological space.

Deﬁnition (Continuous). Let X = (X, d) and Y = (Y, d¯) be metric spaces. The mapping T : X Y is continuous at x X if for every ε> 0 there is δ > 0 such that → 0 ∈ d¯(Tx,Tx ) < ε , x such that d(x,x ) < δ. 0 ∀ 0 Theorem 1.3 (Continuous Mapping). T : X Y is continuous if and only if the inverse → image of any open subset of Y is an open subset of X.

Proof.

1. Suppose that T is continuous. Let S Y be open S the inverse image of S. Let ⊂ 0 S = and take x S . We have Tx = y S. Since S is open there exists an 0 6 ∅ 0 ∈ 0 0 0 ∈ ε-neighborhood of y , say N S such that y N. The continuity of T implies 0 ⊂ 0 ∈ that x has a δ-neighborhood N which is mapped into N. Since N S we get that 0 0 ⊂ N S , and it follows that S is open. 0 ⊂ 0 0

5 2. Assume that the inverse image of every open set in Y is an open set in X. Then x X, and N (ε-neighborhood of Tx ) the inverse image N of N is open. ∀ 0 ∈ 0 0 Therefore N0 contains a δ-neighborhood of x0. Thus T is continuous.

Some more definitions: Definition (Accumulation Point). x M is said to be an accumulation point of M if ∈ (xn) M s.t. xn x. ∃ ⊂ → Definition (Closure). M is the closure of M. Definition (Dense Set). M X is in X dense if M = X. ⊂ Definition (Separable Space). X is separable if there is a countable subset which is dense in X. Remark 1.4.

1. If M is dense, then every ball in X contains a point of M.

2. R, C are separable.

3. A discrete metric space is separable if and only if it is countable.

Theorem 1.5. ℓ∞ is not separable.

Proof. Let y = (ηi) where ηi = 0, 1. There are uncountably many y’s. If we put small balls with radius 1 at the y’s they will not intersect. It follows that if M l is dense in l , 3 ⊂ ∞ ∞ then M is uncountable. Therefore l∞ is not separable. Problem 2. Show that lp, 1 p< is separable. ≤ ∞ Solution. Let M the set of all sequences of the form x = (ξ1,ξ2, ..., ξn, 0, 0, ...), where n is any positive integer and the ξis are rational. M is countable. We argue that M is dense in p p l as follows. Let y = (ηi) l be arbitrary. Then for every ε> 0 there is an n such that ∈ p ∞ p ε ηi < . | | 2 i n =X+1 Since the rationals are dense in R, for each ηi there is a rational ξi close to it. Hence there is an x M such that n ∈ εp ηi ξi < . | − | 2 i X=1 It follows that d(y,x) < ε.

6 1.2 Convergence, Cauchy Sequence, Completeness

Deﬁnition (Convergent Sequence). We say that the sequence (xn) is convergent in the metric space X = (X, d) if

lim d(xn,x) = 0 for some x X. n →∞ ∈ We shall also use the notation xn x. → Definition (Diameter). For a set M X we define the diameter δ(M) by ⊂ δ(M)= sup d(x,y). x,y M ∈ M is said to be bounded if δ(M) is finite. Lemma 1.6.

1. A convergent sequence (xn) in X is bounded and its limit x is unique.

2. If xn x and yn y in X, then d(xn,yn) d(x,y). → → → Problem 3. Prove Lemma 1.6.

Solution.

1. Suppose that xn x. Then, taking ε = 1 we can ﬁnd an N such that d(xn,x) < 1 → for all n > N. By the triangle inequality we have

d(xn,x) < 1+max d(x1,x), d(x2,x), ..., d(xN ,x).

Therefore (xn) is bounded. If xn x and xn z, then → → 0 d(x, z) d(xn,x)+ d(xn, z) 0 as n ≤ ≤ → →∞ and uniqueness of the limit follows.

2. We have d(xn,yn) d(xn,x)+ d(x,y)+ d(y,yn), ≤ and hence d(xn,yn) d(x,y) d(xn,x)+ d(yn,y). − ≤ Interchanging xn and x, yn and y, and multiplying by 1 we get − d(x,y) d(xn,yn) d(xn,x)+ d(yn,y). − ≤ Combining the two inequalities we get

d(xn,yn) d(x,y) d(xn,x)+ d(yn,y) 0 as n . | − |≤ → →∞

7 Deﬁnition (Cauchy Sequence). (xn) in (X, d) is a Cauchy sequence if ε > 0 N = ∀ ∃ N(ε) s.t. m,n>N d(xm,xn) < ε. ⇒ Deﬁnition (Complete). X is complete if every Cauchy sequence in X converges in X. Theorem 1.7. R, C are complete metric spaces. Theorem 1.8. Every convergent sequence in a metric space is a Cauchy sequence. Theorem 1.9. Let M X = (X, d), X is a metric space and let M denote the closure of ⊂ M in X. Then

1. x M iff (xn) M s.t. xn x. ∈ ∃ ∈ → 2. M is closed iff xn M and xn x imply that x M. ∈ → ∈ Theorem 1.10. Let X be a complete metric space and M X. M is complete iff M is ⊂ closed in X. Theorem 1.11. Let T be a mapping from (X, d) into (Y, d˜). T : X Y is continuous at → x X iff xn x implies Txn Tx . 0 ∈ → 0 → 0 Problem 4. 1. Prove Theorem 1.10 2. Prove Theorem 1.11 Solution.

1. Let M be complete. Then for every x M there is a sequence (xn) which converges ∈ to x. Since (xn) is Cauchy and M is complete xn x M, therefore M is closed. → ∈ Conversely, let M be closed and (xn) be Cauchy in M. Then xn x X, which → ∈ implies x M, and therefore x M. Thus M is complete. ∈ ∈ 2. Assume that T is continuous at x . Then for ε> 0 δ > 0 such that 0 ∃ d(x,x0) < δ implies d˜(Tx,Tx0) < ε.

Take a sequence (xn) such that xn x . Then N s.t. n > N we have → 0 ∃ ∀ d(xn,x0) < δ and hence n > N, d˜(Txn,Tx)) < ε. ∀ Conversely, assume that xn x) Txn Tx and show that T is continuous → ⇒ → 0 at x0. Otherwise, ε > 0 such that δ > 0, x = x0) satisfying d(x,x0) < δ ∃ 1 ∀ ∃ 6 1 but d˜(Tx,Tx ) ε. Let δ = . Then there is an xn such that d(xn,x ) < but 0 ≥ n 0 n d˜(Txn,Tx ) ε contrary to our assumption. 0 ≥

8 1.3 Completeness Proofs

Theorem 1.12. Rn, Cn are complete.

Theorem 1.13. ℓ∞ is complete.

Proof. Let (xm) be a Cauchy sequence in ℓ∞.

(m) (n) d(xm,xn) = sup xi xi < ε, for m,n>N(ε). ⇒ i | − |

for ﬁxed i, the sequence (ξ(m)) is Cauchy in R, and therefore we have that ξ(m) ⇒ i i → ξi, for i = 1, 2, 3, ... Let x = (ξi). It follows easily that x ℓ and xm x. ∈ ∞ → Theorem 1.14. The space of convergent sequences x = (ξi) of complex numbers with the metric induced from ℓ∞ is complete. Theorem 1.15. ℓp is complete if 1 p< . ≤ ∞ Theorem 1.16. C[a, b] is complete.

Proof. Let (xm) be a Cauchy sequence in C[a, b]. Then

ε> 0 N s.t. for m,n>N d(xm,xn)= max xm(t) xn(t) < ε. (2) ∀ ∃ ⇒ t [a,b] | − | ∈

for any ﬁxed t [a, b] we have that xm(t ) xn(t ) < ε. It follows that xm(t ) is ⇒ 0 ∈ | 0 − 0 | 0 Cauchy and therefore xm(t ) x(t ). Show that x(t) C[a, b] and xm x. From (2) 0 → 0 ∈ → with n we get →∞ max xm(t) x(t) ε. t [a,b] | − |≤ ∈ Therefore xm(t) converges uniformly to x(t) on [a, b]. Since the xm’s are continuous on [a, b] and the convergence is uniform x(t) is continuous, and thus belongs to C[a, b].

Remark 1.17. Note that in C[a, b] convergence is uniform convergence; we also use the terminology uniform metric for the metric generated by the “sup”-norm.

Example 1.18 (Incomplete Metric Spaces).

1. Q 2. The polynomials

3. C[a, b] with the norm k · k2

9 1.4 Completion of Metric Spaces Deﬁnition (Isometry). Let X = (X, d) and X¯ = (X,¯ d¯) be metric spaces. The mapping T : X X¯ is an isometry if d¯(Tx,Ty)= d(x,y). The metric spaces X and X¯ are isometric → if there is a bijective isometry of X onto X¯ Theorem 1.19. Let X = (X, d) be a metric space. Then there exists a complete metric space Xˆ = (X,ˆ dˆ) which has a subspace W that is isometric with X and is dense in Xˆ. Xˆ is unique except for isometries. Proof. This proof is divided into steps. ˆ ′ 1. Construction of X. Let (xn) and (xn) be equivalent Cauchy sequences, i.e.,

′ lim d(xn,x ) = 0. n n →∞ Deﬁne Xˆ to be the set of all equivalence classesx ˆ of Cauchy sequences. Deﬁne now

dˆ(ˆx, yˆ) = lim d(xn,yn) where (xn) xˆ and (yn) y.ˆ (3) n →∞ ∈ ∈ We show that limit in (3) exists and independent of the particular choice of the representatives. We have using the triangle inequality

d(xn,yn) d(xn,xm)+ d(xm,ym)+ d(ym,yn). ≤ It follows that d(xn,yn) d(xm,ym) d(xn,xm)+ d(ym,yn). − ≤ Exchanging the role of n and m we get

d(xm,ym) d(xn,yn) d(xn,xm)+ d(ym,yn). − ≤ The two inequality together yield

d(xn,yn) d(xm,ym) d(xn,xm)+ d(ym,yn). | − |≤

It follows that (d(xn,yn)) is a Cauchy sequence in R and therefore it converges, i.e., the limit in (3) exists. We show now that the limit in (3) is independent of the particular choice of repre- ′ ′ sentatives. Let (xn), (x ) xˆ and (yn), (y ) yˆ. Then n ∈ n ∈ ′ ′ ′ ′ d(xn,yn) d(x ,y ) d(xn,x )+ d(yn,y ) 0 as n , | − n n |≤ n n → →∞ which implies ′ ′ lim d(xn,yn) = lim d(x ,y ). n n n n →∞ →∞

10 2. Show dˆ is a metric on Xˆ . Left as an exercise.

3. Construction of an isometry T : X W Xˆ . With each b X we associate the → ∈ ∈ class ˆb Xˆ which contains the constant Cauchy sequence (b, b, b, ...). This deﬁnes ∈ the mapping T : X W onto the subspace W = T (X) Xˆ. The mapping T is → ∈ given by b ˆb = T b, where (b, b, ...) ˆb. According to (3) dˆ(ˆb, cˆ) = d(b, c), so T is → ∈ an isometry. T is onto W since T (X)= W . ( W and X are isometric.)

Show that W is dense in Xˆ. Letx ˆ Xˆ be arbitrary and let (xn) xˆ. For every ∈ ε ∈ ε > 0 there is an N such that d(xn,xN ) < , for n > N. Let (xN ,xN , ...) xˆN . 2 ∈ Thenx ˆN W , and by (3) ∈ ε dˆ(ˆx, xˆN ) = lim d(xn,xN ) < ε. n →∞ ≤ 2

4. Completeness of Xˆ. Let (ˆxn) be an arbitrary Cauchy sequence in Xˆ. Since W is dense in Xˆ for everyx ˆn there is az ˆn W such that ∈ 1 dˆ(ˆx , zˆ ) < . n n n

It follows using the triangle inequality and the Cauchyness of (ˆxn) that (ˆzn) is Cauchy. 1 Then (zn), where zn = T zˆn is Cauchy in X. Letx ˆ Xˆ be the class to which (zn) − ∈ belongs. We have 1 dˆ(ˆxn, xˆ) dˆ(ˆxn, zˆn)+ dˆ(ˆzn, xˆ) < + dˆ(ˆzn, xˆ) < ε ≤ n for suﬃciently large n.

5. Uniqueness of Xˆ. If(X,¯ d¯) is another complete metric space with a subspace W¯ dense in X¯ and isometric with X, then W¯ is isometric with W and the distances on X¯ and Xˆ must be the same. Hence X¯ and Xˆ are isometric.

Problem 5. Show now that dˆ is a metric on Xˆ.

Solution. Clearly, dˆ(ˆx, yˆ) 0 because of the deﬁnition (3). Also, dˆ(ˆx, xˆ) = 0, and dˆ(ˆx, yˆ)= ≥ dˆ(ˆy, xˆ), because of the properties of d.

dˆ(ˆx, yˆ) dˆ(ˆx, zˆ)+ dˆ(ˆz, yˆ) ≤ because we have d(xn,yn) d(xn, zn)+ d(zn,yn) for all n. ≤

11 Problem 6. A homeomorphism is a continuous bijective mapping T : X Y whose inverse → is continuous. If such mapping exists, then X and Y are homeomorphic.

1. Show that is X and Y are isometric, then they are homeomorphic.

2. Illustrate with an example that a complete and an incomplete metric space may be homeomorphic.

Solution. 1. If f : (X, d) (Y, dˆ) is an isometry, then dˆ(f(x),f(y)) = d(x,y) < ǫ → whenever d(x,y) < δ, so f is continuous. If (X, d) and (Y, dˆ) are isometric, then 1 there exists a bijective isometry f : X Y . It follows that f − : Y X is →1 → also an isometry, and that both f and f − are continuous. Therefore X and Y are homeomorphic.

2. Let f : ( π , π ) R be given by f(x) = tan x. f is cintinuous and bijective. − 2 2 → Moreover, f 1 : R ( π , π ) given by f 1(x) = tan 1(x) is also continuous. So, R − → − 2 2 − − is homeomorphic to ( π , π ). − 2 2

¯ ¯ d Problem 7. If (X, d) is complete, show that (X, d), where d = 1+d is complete.

Solution. It is easy to see that if (xn) is Cauchy in (X, d¯), then (xn) is Cauchy in (X, d). Since (X, d) is complete, xn x, but then we also have xn x in (X, d¯). It follows that → → (X, d¯) is complete.

12 2 Normed Spaces

Vector spaces, linear independence, ﬁnite- and inﬁnite-dimensional vector spaces.

Deﬁnition (Basis). If X is any vector space, and B is a linearly independent subset of X which spans X, then B is called a basis (or Hamel basis) for X.

Deﬁnition (Banach Space). A complete normed space.

Definition (norm). A real-valued function : X R satisfying k·k → 1. x 0 k k≥ 2. x = 0 iff x = 0 k k 3. αx = α x k k | |k k 4. x + y x + y | | ≤ | | | | Definition (Induced Metric). In a normed space (X, d) we can always define a metric by d(x,y)= x y . k − k Remark 2.1. All previous results about metric spaces apply to normed spaces with the induced metric.

Problem 8. The norm is continuous, that is : X R by x x is a continuous. k·k → 7→ k k Solution. The triangle inequality implies that

x y x y . |k k − k k| ≤ k − k

n n p 2 p Example 2.2 (Norm Spaces). R , C ,ℓ ,ℓ∞,C[a, b], [a, b], [a, b]. L L Remark 2.3 (Induced Norm is Translation Invariant). A metric induced by a norm on a normed space is translation invariant, i.e., d(x + a, y + a) = d(x,y) and d(αx, αy) = α d(x,y). For example the metric (1) is not coming from a norm. | | Theorem 2.4. A subspace Y of a Banach space X is complete iﬀ the set Y is closed in X.

Deﬁnition (Convergent, Cauchy, Absolutely Convergent).

1. (xn) is convergent if xn x 0. (x is the limit of (xn)). k − k→ 2. (xn) is Cauchy if xm xn <ǫ for m,n>N(ǫ). k − k

13 Deﬁnition (Inﬁnite series). Let (xk) be a sequence and consider the partial sums

sn = x1 + x2 + ... + xn.

If sn converges, say sn s, then ∞ xk is convergent and → k=1 P ∞ s = . Xk=1 If x + x + ... + xn + ... converges, then ∞ xk is absolutely convergent. k 1k k 2k k k k=1 Remark 2.5. Absolute convergence implies convergenceP iﬀ X is complete.

Deﬁnition (Schauder basis). If X contains a sequence (en) such that for every x X ∈ there exists a unique sequence (αn) with the property that

x (α e + ... + αnen) 0 as n , k − 1 1 k→ →∞ then (en) is called a Schauder Basis for X. The representation

∞ x = αkek Xk=1 is called the expansion of x with respect to (en). Remark 2.6. If X has a Schauder basis, then X is separable. Theorem 2.7. Let X = (X, ) be a normed space. Then there is a Banach space Xˆ and k·k an isometry A from X onto a subspace W Xˆ which is dense in Xˆ . Xˆ is unique, except ⊂ for isometries. Theorem 2.8. Let X be a normed space, where the absolute convergence of any series always implies convergence. Then X is complete.

Proof. Let (sn) be any Cauchy sequence in X. Then for every k N there exists nk k ∈ such that sn sm < 2 , (m,n > nk) and we can choose nk > nk for all k. Then k − k − +1 (snk ) is a subsequence of (sn) and is the sequence of the partial sums of xk, where x = sn , ..., xk = sn sn , .... Hence 1 1 k − k−1 P k xk x + x + 2− = x + x + 1. k k ≤ k 1k k 2k k 1k k 2k X X It follows that xk is absolutely convergent. By assumption, xk converges, say, sn k → s X. Since (sn) is Cauchy, sn s because ∈ P → P sn s sn sn + sn s . k − k ≤ k − k k k k − k Since (sn) was arbitrary, sn s shows that X is complete. →

14 2.1 Finite dimensional Normed Spaces or Subspaces Theorem 2.9 (Linear Combinations). Let X be a normed space, and let

x ,...,xn { 1 } be a linearly independent set of vectors in X. Then c> 0 s.t. ∃

α x + + αnxn c ( α + + αn ) . k 1 1 · · · k≥ | 1| · · · | | Proof. Let s = α + + αn . | 1| · · · | | If s = 0 we are done, so assume s> 0. Deﬁne

αi βi = . s Clearly we have that n βi = 1. | | i X=1 Then we can reduce the above inequality to

n βixi c. (4) ≥ i=1 X

We will prove the result for Theorem 4. Suppose Theorem 4 is not true. Then (ym) ∃ n (m) ym = βi xi i X=1 such that ym 0 as m . k k→ →∞ Note that n β(m) = 1 β(m) 1 i. Hence for each ﬁxed i the sequence i=1 | i | ⇒ | i |≤ ∀ P (m) (1) (2) βi = βi , βi ,... (m) is bounded. Therefore β1 has a convergent subsequence (B-W). Let β1 be the limit, and let (y1,m) be the corresponding subsequence of (ym)

n (m) (m) (y1,m)= γ1 x1 + βi xi i X=2 15 γ(m) β . 1 → 1 Then there is a there exist corresponding β2, (y2,m) where

β(m) β as n 2 → 2 →∞ n (m) (m) (m) (y2,m)= γ1 x1 + γ2 x2 + βi xi. i X=3 Since n is ﬁnite this process will terminate with

n (m) (yn,m)= γi xi i X=1 with (yn,m) a subsequence of (ym). By construction,

(m) γ βi i i → ∀ n βi = 1. | | i X=1 Therefore (yn,m) has a limit, say n y = βixi. i X=1 Since ym 0 by assumption we must also have yn,m 0, We know y = 0 since n k k → k k → 6 βi = 1 and xi is a basis, a contradiction. i=1 | | { } TheoremP 2.10 (Completeness). Every f.d.s.s. Y of a n.s. X is complete. In particular, every f.d.n.s. is complete.

Proof. Let (ym) be a Cauchy sequence in Y . We must ﬁnd a limit y such that

ym y → y Y. ∈ Let n = dim Y , and let e ,...,en be a basis for Y . Then m we can represent ym as { 1 } ∀ n (m) ym = αi ei. i X=1 (m) But then for each ﬁxed i sequence (αi ) is Cauchy in R. So each of these sequence converges, say (m) α αi. i →

16 Then deﬁne n y = αiei. i X=1 Clearly, y Y and ym y. ∈ →

Theorem 2.11. Every f.d.s.s. Y of a closed n.s. X is closed in X. Proof. By Theorem 2.10 Y must be complete. Therefore Y is closed in X.

Note that ﬁniteness in the previous proof is essential. Inﬁnite dimensional subspaces need not be closed in X. For example consider

X = C[0, 1]

Y = span ( 1, t, ..., tn, .. ) . { } Then the sequence (ti) converges to

0 t< 1 f(t)= 1 t = 1 which is not in Y . Deﬁnition (Equivalent Norms). and are equivalent if a,b > 0 s.t. k·k1 k·k2 ∃ a x x b x , x X. k k2 ≤ k k1 ≤ k k2 ∀ ∈ Theorem 2.12 (Finite Dimesional All Norms are Equivalent). On a f.d.n.s. X every ⇒ two norms , are equivalent. k·k1 k·k2 Proof. Let n = dim X, let e ,...,en be a basis for X with ei = 1, let x X be { 1 } k k1 ∈ represented as n x = αiei. i X=1 Then c> 0 s.t. ∃ n x c αi . k k1 ≥ | | i X=1 Also note n n x αi ei k αi k k2 ≤ | | k k2 ≤ | | i i X=1 X=1 where k = max ei . {k k2}

17 Combining these inequalities we obtain

n k 1 x c αi x , k k2 ≤ c | |≤ a k k1 i X=1 where c a = k and we obtain the inequality involving a. A similar argument yields for the inequality involving b.

Problem 9. What is the largest possible c> 0 in

n n αixi c αi ≥ | | i=1 i=1 X X

if 2 1. X = R , x1 = (1, 0), x2 = (0, 1).

3 2. X = R , x1 = (1, 0, 0), x2 = (0, 1, 0), x3 = (0, 0, 1). 1 1 Solution. Find min βixi : βi = 1 and obtain , , respectively. {k k | | } √2 √3 Problem 10. Show thatP if , P are equivalent norms on X then the Cauchy sequences k·k1 k·k2 in (X, ), (X, ) are the same. k·k1 k·k2 Solution. Suppose that (xn) is a Cauchy sequence in 2. Then for every ǫ > 0 there ǫ k · k exists N such that xn xm < for all m,n N, where b is a constant for which k − k2 b ≥ x b x . Then xn xm b xn xm <ǫ for all m,n N. It follows that (xn) k k1 ≤ k k2 k − k1 ≤ k − k2 ≥ is Cauchy in . A similar argument works in the other direction. k · k1 2.2 Compactness and Finite Dimension Deﬁnition (Compactness). A metric space (X, d) is compact if every sequence in X has a convergent subsequence. M X is compact if M is compact as a subspace of X, i.e., if ⊂ every sequence in M has a convergent subsequence which converges to an element in M.

Theorem 2.13. If M is compact then M is closed and bounded.

Proof. First we show closed. For all x M (xn) M s.t. xn x. M is compact ∈ ∃ ⊂ → ⇒ x M, hence M is closed. ∈ To show boundedness assume it is not true. Then ﬁx b M and choose yn ∈ ∈ M s.t. d(yn, b) > n. Then (yn) M is a sequence with no convergent subsequence, ⊂ contradicting the assumption that M is compact. Thus M is bounded.

18 Note the converse of this theorem is false. Consider M ℓ2 as ⊂ 2 M = xn ℓ s.t. xn = (en), (en)i = δni . ∈

Then M is bounded since xn = 1 and M is closed because there are no accumulation k k points xn xm = √2. k − k But the sequence (xn) has no convergent subsequence, so M is not compact. In some sense, there is too much room in the inﬁnite dimensional setting.

Theorem 2.14 (Compactness). Let X be a f.d.n.s. . Then M X is (sequentially) ⊂ compact iﬀ M is closed and bounded.

Proof. The “ ” case was proved by the previous theorem. For the other direction, assume ⇒ M is closed and bounded. For the other direction (“ ”), assume n = dim X, e ,...,en is a basis for X, and ⇐ { 1 } (xm) M is an arbitrary sequence. Then there is a representation ⊂ m (m) xm = ξ ei, m. i ∀ i X=1 Since M is bounded, (xm) is bounded so k 0 s.t. ∃ ≥

xm k. k k≤ So then

k xm ≥ k n k (m) = ξi ei

i=1 Xn (m) c ξ ≥ | i | i X=1 (m) (m) and therefore i, (ξ ) R is bounded and therefore i, (ξ ) has an accumulation ∀ i ⊂ ∀ i point z. Since M is closed, z M and there is a subsequence (zm) (xm) s.t. zm z, ∈ ⊂ → say, n z = ξiei. i X=1 Since (xm) was arbitrary, M is compact.

We shall need the following technical result to prove subsequent theorems.

19 Theorem 2.15 (Riesz). Let Y, Z be subspaces of X, and suppose Y is closed and is a proper subspace of Z. Then θ (0, 1) z s.t. ∀ ∈ ∃ z = 1 k k z y θ, y Y. k − k≥ ∀ ∈ Proof. Let v Z Y be arbitrary and let ∈ − a = inf v y . y Y k − k ∈ Note if a = 0 then v Y since Y is closed. Therefore a> 0. ∈ Choose θ (0, 1). Then y Y s.t. ∈ ∃ 0 ∈ a a v y . ≤ k − 0k≤ θ Let 1 z = c(v y ) where c = . − 0 v y k − 0k By construction, z = 1. We want to show z y θ, y Y . So k k k − k≥ ∀ ∈

z y = c(v y0) y k − k k − −y k = c v y − 0 − c

= c v y1 k − k for some y Y, v y a 1 ∈ ⇒ k − 1k≥ ⇒ z y = v y k − k k − 1k ca ≥ a = v y0 ka − k a ≥ θ = θ.

Theorem 2.16 (Finite Dimension and Compactness). Let X be a normed space. Then the closed unit ball M = x s.t. x 1 { k k≤ } is compact iﬀ dim X < . ∞

20 Proof. “ ”. Suppose M is compact, but dim X = . Pick x , x = 1. Then x ⇒ ∞ 1 k 1k 1 generates a closed, 1-dimensional subspace X1 $ X. Then also x2 X s.t. x2 = 1 ∃ ∈ k k and 1 x x θ = . k 2 − 1k≥ 2

Then x1,x2 generate a closed, 2-dimensional subspace X2 $ X. Then x3 X s.t. x3 = ∃ ∈ k k 1 and 1 x xi θ = , i = 1, 2. k 3 − k≥ 2 ∀ In this way construct xn s.t. 1 xn xi θ = , i = 1,...,n 1. k − k≥ 2 ∀ −

1 Then (xn) X is a sequence such that xm xn , m,n. Therefore (xn) has no ⊂ k − k ≥ 2 ∀ accumulation point, contradicting that M was compact. “ ” was proved by Theorem 2.14. ⇐ Theorem 2.17 (Continuous Preserves Compact). Let T : X Y be continuous. Then → M X compact T (M) Y compact. ⊂ ⇒ ⊂ Proof. Let (yn) T (M) be arbitrary. Then n xn s.t. yn = Txn. Then (xn) M ⊂ ∀ ∃ ⊂ has a convergent subsequence (xnk ). Since T is continuous, the image (ynk )= T (xnk ) also converges. Therefore T (M) is compact.

Corollary 2.18 (Max, Min). Let T : X R be continuous and let M X be compact. → ⊂ Then T obtains its max and min on M.

Proof. T (M) is compact T (M) is closed and bounded. Therefore ⇒ inf T (M) T (M) ∈ sup T (M) T (M). ∈

Problem 11. Show that a discrete metric space with inﬁnitely many points is not compact.

Solution. The space contains an inﬁnite sequence (xn), xn = xm (n = m) which cannot 6 6 have a convergent subsequence since d(xn,xm) = 1.

Problem 12 (Local Compactness). A metric space X is said to be locally compact if every point x X has a compact neighborhood. Show that R, Rn, C, Cn are locally compact. ∈ Solution. The closed ball B¯(x,ǫ) around an arbitrary point x is compact.

21 2.3 Linear Operators

into Deﬁnition (Linear, Domain and Range). Let X,Y be vector spaces and let T : X Y . −→ Then

1. If x,y (T ) and scalars α, β ∀ ∈ D T (αx + βy)= αT x + βTy

then T is said to be a linear operator.

2. The null space of T is (T )= x (T ) s.t. Tx = 0 N { ∈ D } 3. The domain of T , (T ), is a vector space D 4. The range of T , (T ), lies in a vector space R Note that obviously (T ) X. It is customary to write D ⊂ T : (T ) onto (T ) . D −→ R Also, T 0= T (0 + 0) = T 0+ T 0 T 0 = 0 (T ) = . ⇒ ⇒N 6 ∅ Example 2.19 (Diﬀerentiation). Let X be the space of polynomials on [a, b]. Then deﬁne onto T : X X −→ d Tx(t)= x(t). dt

Example 2.20 (Integration). Let X = C[a, b]. Then deﬁne T : X into X by −→ t Tx(t)= x(s)ds. Za into Example 2.21 (Multiplication). Let X = C[a, b]. Then deﬁne T : X X by −→ Tx(t)= tx(t).

m n Example 2.22 (Matrices). Let A = (aij). Then deﬁne T : R R by → Tx = Ax.

Theorem 2.23 (Range and Null Space). Let T be a linear operator.

1. (T ) is a vector space. R 2. If dim (T )= n< then dim (T ) n. D ∞ R ≤

22 3. (T ) is a vector space. N Proof. 1. Let y ,y (T ) and let α, β be scalars. Since y ,y (T ) , x ,x 1 2 ∈ R 1 2 ∈ R ∃ 1 2 ∈ (T ) s.t. D Tx1 = y1

Tx2 = y2. Since (T ) is vector space, αx + βx (T ), so D 1 2 ∈ D T (αx + βx ) (T ) . 1 2 ∈ R But T (αx1 + βx2)= αT x1 + βTx2 = αy1 + βy2 means (T ) is a vector space. R 2. Pick n + 1 elements arbitrarily in (T ). Then we have R yi = Txi, i = 1,...,n + 1 ∀ for some x ,...,xn in (T ). Since dim (T ) = n, this set must be linearly { 1 +1} D D dependent. Similarly, the points in the range are also linearly dependent. Therefore, dim (T ) n. R ≤ 3. Let x ,x (T ) , i.e. ,Tx = Tx = 0, and let α, β be scalars. But then 1 2 ∈N 1 2 T (αx + βx )= αT x + βTx = 0 αx + βx (T ) . 1 2 1 2 ⇒ 1 2 ∈N Therefore (T ) is a vector space. N

The next result shows linear operators preserve linear independence. into Deﬁnition (Injective (One-to-one)). T : (T ) Y is injective (or one-to-one) if D −→ x ,x (T ) s.t. x = x ∀ 1 2 ∈ D 1 6 2 Tx = Tx . 1 6 2 Equivalently, Tx = Tx x = x . 1 2 ⇒ 1 2 So therefore if T : (T ) into Y is injective then T −1 : (T ) onto (T ) D −→ ∃ R −→ D y x 0 7→ 0 y0 = Tx0 and therefore T −1Tx = x, x (T ) ∀ ∈ D T T −1y = y, y (T ) . ∀ ∈ R

23 into Theorem 2.24 (Inverses). Let X,Y be vector spaces and T : (T ) Y be linear with D −→ (T ) X and (T ) Y. D ⊂ R ⊂ onto 1. T −1 : (T ) (T ) exists iﬀ Tx = 0 x = 0. R −→ D ⇒ 2. If T −1 exists then it is linear. 3. If dim (T )= n< and T −1 exists then dim (T ) = dim (T ). D ∞ R D Proof. 1. Suppose Tx = 0 x = 0. Then ⇒ Tx = Tx T (x x ) = 0 x = x 1 2 ⇒ 1 − 2 ⇒ 1 2 So T is injective T −1 exists. Conversely, if T −1 exists then Tx = Tx x = x . Therefore 1 2 ⇒ 1 2 Tx = T 0 = 0 x = 0. ⇒ 2. We assume that T −1 exists and show that it is linear in a straightforward fashion. 3. It follows from the earlier result applied to both T and T −1.

Theorem 2.25 (Inverse of Product (Composition)). Let T : X Y and S : Y Z be → → bijections. Then (ST )−1 : Z X exists and → (ST )−1 = T −1S−1.

2.4 Bounded and Continuous Linear Operators

Deﬁnition (Bounded Linear Operator). Let X,Y be normed spaces and T : (T ) into D −→ Y, (T ) X. Then T is said to be bounded if k> 0 s.t. D ⊂ ∃ Tx k x , x (T ) . k k≤ k k ∀ ∈ D into Deﬁnition (Induced Operator Norm). Let X,Y be normed spaces and T : (T ) D −→ Y, (T ) X. Then the norms on X,Y induce a norm for operators D ⊂ Tx T = sup k k. k k x∈D(T ) x x=06 k k Equivalently, T = sup Tx . k k x∈D(T ) k k kxk=1

24 The preceding deﬁnition implies that

Tx T x , x. k k ≤ k k k k ∀ Example 2.26 (Diﬀerention is Unbounded). Let T : C[0, 1] C[0, 1],be deﬁned by ′ → n Tx(t) = x (t). Then T is linear but unbounded. Indeed, let xn(t) = t . Then xn = 1 k k and Txn = n, i.e., T is not bounded. k k Example 2.27 (Integration is Bounded). Let T : C[0, 1] C[0, 1], k C[0, 1]2, y = Tx → ∈ via 1 y(t)= k(t,τ)x(τ)dτ. Z0 Since k is continuous, k > 0 s.t. ∃ 0 k(t,τ) k . | |≤ 0 So then

y = Tx k k k k 1 max k(t,τ) x(t) dτ ≤ t [0,1] | | k k ∈ Z0 k x ≤ 0 k k Theorem 2.28 (Finite Dimension). Let X be normed and dim X = n < . Then every ∞ linear operator on X is bounded.

Proof. Let e ,...,en be a basis for X and let x X be arbitrary and represented as 1 ∈ n x = αiei. i X=1 Then n n Tx = αiT ei k αi , k k i=1 ≤ | | i=1 X X where k = maxi ,...,n T ei . Also, =1 k k n 1 αi x . | |≤ c k k i X=1 It follows that T is bounded.

Next, we illustrate a general method for computing bounds on operators.

25 Example 2.29 (Calculating Bounds on Operators). Let

n x = ξiei. i X=1 Then n Tx = ξiT ei k k i=1 nX

ξi T e i ≤ | | k k i=1 X n max T ek ξi . ≤ k=1,...,n k k | | i X=1 Then using Theorem 2.9 we can conclude

n n 1 1 x = ξiei ξi c k k c ≥ | | i=1 i=1 X X

which implies 1 Tx γ x where γ = max T ek . k k≤ k k c k=1,...,n k k

Deﬁnition (Continuity for Operators). Let T : (T ) into Y . Then T is said to be D −→ continuous at x (T ) if 0 ∈ D ε> 0 δ > 0 s.t. x x < δ Tx Tx < ε. ∀ ∃ k − 0k ⇒ k − 0k T is said to be continuous if it is continuous at every x (T ). 0 ∈ D Theorem 2.30 (Continuity and Boundedness). Let X,Y be vector spaces (T ) X, and into D ⊂ T : (T ) Y be linear. D −→ 1. T is continuous iﬀ T is bounded

2. T is continuous at a single point T is continuous everywhere ⇒ Proof. 1. The case T = 0 is trivial. Assume T = 0, then T = 0. Suppose T is bounded and 6 k k 6 let x (T ) ,ε> 0 be arbitrary. Then because T is linear, x (T ) s.t. 0 ∈ D ∀ ∈ D ε x x < δ = k − 0k T k k 26 we have that Tx Tx = T (x x ) T x x < T δ = ε. k − 0k k − 0 k ≤ k k k − 0k k k Therefore T is continuous. Conversely, suppose T is continuous at x (T ). Then ε> 0 δ > 0 s.t. 0 ∈ D ∀ ∃ x x δ Tx Tx ε. k − 0k≤ ⇒ k − 0k≤ So choose any y (T ) ,y = 0 and set ∈ D 6 y x = x + δ . 0 y k k Then y x x = δ x x = δ, − 0 y ⇒ k − 0k so k k Tx Tx = T (x x ) k − 0k k − 0 k δ = T y y k k δ = Ty y k k k k ε. ≤ Therefore we have ε Ty c y where c = . k k≤ k k δ 2. Above we only used continuity at a single point to show boundedness. Boundedness implies continuity.

into Corollary 2.31. Let T : (T ) Y be a bounded, linear operator. Then D −→ 1. (xn) (T ) ,x (T ) and xn x Txn Tx ⊂ D ∈ D → ⇒ → 2. (T ) is closed. N Proof. 1. Txn Tx = T (xn x) T xn x 0 k − k k − k ≤ k k k − k→ 2. x (T ) (xn) (T ) s.t. ∀ ∈N ∃ ⊂N xn x Txn Tx. → ⇒ → But xn (T ) Txn = 0, n Tx = 0 x (T ). Therefore (T ) is ∈ N ⇒ ∀ ⇒ ⇒ ∈ N N closed.

27 Compare to the notion of subadditivity (triangle inequality).

Remark 2.32 (Operator Norm is Submultiplicative).

1. TS T S k k ≤ k k k k 2. T n T n k k ≤ k k Remark 2.33 (Equality of Operators). Two operators T,S are said to be equal and we write T = S if (T )= (S) and Tx = Sx, x (T ) . D D ∀ ∈ D into Deﬁnition (Rescriction). Let T : (T ) Y and let B (T ). We deﬁne the restric- intoD −→ ⊂ D tion of T to B, denoted T B : B Y , as | −→

T Bx = T x, x B. | ∀ ∈ into Deﬁnition (Extension). Let T : (T ) Y and (T ) M. An extension of T is into D −→ D ⊂ Tˆ : M Y s.t. −→ Tˆ (T )= T |D Txˆ = T x, x (T ) . ∀ ∈ D Theorem 2.34 (Bounded Linear Extension). Let (T ) X, X a normed space, Y a into D ⊂ Banach space, and T : (T ) Y be linear. Then there is a unique bounded, linear D −→ extension Tˆ of T s.t. Tˆ = T . k k

Proof. Let x (T ). Then (xn) (T ) s.t. xn x. Since T is linear and bounded ∈ D ∃ ∈ D → we have Txn Tx = T (xn x) T xn x , k − k k − k ≤ k k k − k so (Txn) is Cauchy. Since Y is Banach, (Txn) converges, say

Txn y Y. → ∈ In this way, we deﬁne Txˆ = y. Because limits are unique, Tˆ is uniquely deﬁned x (T ). Of course Tˆ inherits linearity ∀ ∈ D and Txˆ = T x, x (T ) ∀ ∈ D so Tˆ is a genuine extension.

28 Finally, we need to show Tˆ is bounded. Note

Txn T xn k k ≤ k k k k and recall x x 7→ k k is continuous. Therefore Txn y = Txˆ and → Txˆ x x . ≤ k k k k

Therefore Tˆ is bounded and Tˆ T . By deﬁnition, Tˆ T so ≤ k k ≥ k k

Tˆ = T . k k

2.5 Linear Functionals Definition (Linear Functional). A linear functional f is a linear operator with (f) X D ⊂ where X is vector space and (f) K where K is the scalar field corresponding to X (R R ⊂ or C). Then into f : (f) K. D −→ into Definition (Bounded Linear Functional). Let f : (f) K be a linear functional. D −→ Then f is said to be bounded if c> 0 s.t. ∃ f(x) c x , x (f) . | |≤ k k ∀ ∈ D Definition (Norm of a Linear Functional). Let f be a linear functional. Then

f(x) f = sup | | = sup f(x) , k k x∈D(f) x x∈D(f) | | x=06 k k kxk=1 so f(x) f x . | | ≤ k k k k Theorem 2.35 (Continuity and Boundedness). Let f be a linear functional. f is continuous iﬀ f is bounded.

29 Example 2.36 (Dot Product). Let a R, a = 0 be ﬁxed and let f : Rn R be deﬁned ∈ 6 → as f(x)= x a. · Then f is linear functional and f(x) x a . | | ≤ k k k k On the other hand, if we take x = a

f(x) = a 2 f a . | | k k ⇒ k k ≥ k k Example 2.37 (Integral). Define f : C[a, b] R by → b f(x)= x(t)dt. Za Then f(x) (b a) x . | |≤ − k k Further, if x = 1 f(x) = b a f = b a. | | − ⇒ k k − Example 2.38 (Point Evaluation). Let t0 [a, b] be fixed and define f : C[a, b] R by ∈ →

f(x)= x(t0).

Choosing the sup-norm, we see

f(x) = x(t ) x , | | | 0 | ≤ k k and since f(1) = 1 = 1 , we have k k f = 1. k k Example 2.39 (Point Evaluation in 2). Let f : 2[a, b] R by L L → f(x)= x(a).

Choose the 2 norm, and let x 2[a, b] be such that x(a) = 1 butthen x rapidly decreases L ∈L to 0. Then f(x)=1 but 1 b 2 x = x(t) 2 dt k k | | Za and for any ε > 0 we can choose an x such that x < ε. Therefore there is no such k k c> 0 s.t. f(x) = 1 c x . | | ≤ k k So point evaluation in this space with this norm is unbounded.

30 Example 2.40 (Inner Product in ℓ2). Fix a ℓ2 and deﬁne f : ℓ2 R by ∈ → ∞ f(x)= xiai = x, a . h i i X=1 Then

∞ f(x) xiai | | ≤ | | i X=1 ∞ ∞ x2 a2 ≤ v i v i u i=1 u i=1 uX uX = tx a .t k k k k

Deﬁnition (Algebraic Dual). Let X be a vector space. Then we denote X∗ as the space of linear functionals on X. X∗∗ represents the dual of X∗, et cetera.

Space General Element Value at a Point X x n/a X∗ f f(x) X∗∗ g g(f)

Deﬁnition (Isomorphism = Bijective Isometry). Let X,Y be vector spaces over the same scalar ﬁeld K. Then an isomorphism T : X Y is a bijective mapping which preserves → the two algebraic operations of vector spaces, i.e.

T (αx + βy)= αT x + βTy.

So T is a bijective linear operator and we say X,Y are isomorphic. If in addition X,Y are normed spaces, then T must also preserve the norms

x = Tx . k k k k We can obtain g X picking a ﬁxed x X and setting ∈ ∗∗ ∈

g(f)= gx(f)= f(x), f X∗. ∀ ∈

This g is linear, and x X gx X . ∈ ⇒ ∈ ∗∗ Deﬁnition (Canonical Mapping/Embedding). Deﬁne C : X X by → ∗∗

x gx. 7→ Then C is injective and linear. Therefore C is an isomorphism of X and (C) X . R ⊂ ∗∗

31 An interesting and important problem is when (C) = X , i.e., X, X are isomor- R ∗∗ ∗∗ phic. Deﬁnition (Embeddable). Let X,Y be vector spaces. X is said to be embeddable in Y if it is isomorphic with a subspace of Y .

From Definition 2.5 we can see X is always embeddable in X∗∗. C is also called the canonical embedding of X into X∗∗. Definition (Algebraic Reflexive). If the canonical mapping C is surjective (and hence bijective) then (C)= X∗∗ R and X is said to be algebraically reflexive. Problem 13. If Y is a subspace of a vector space X and f is a linear functional on X such that f(Y ) is not the whole scalar field K, show f(y) = 0, y Y. ∀ ∈ a Solution. Suppose that f(y)= b = 0 for some y Y . For arbitrary a R we have b y Y a 6 ∈ ∈ ∈ and f( b y)= a. It follows that f(Y )= R; a contradiction. Problem 14. Let f = 0 be a linear functional on a vector space X, and let x X (f) 6 0 ∈ −N be fixed. Show that every x X has a unique representation ∈ x = αx + y, some y (f) . 0 ∈N Solution. Let f = 0 and consider x X N(f). Then f(x ) = c = 0. Let a = f(x) = 6 0 ∈ − 0 6 f( a x ). It follows that x a x N(f). We can write c 0 − c 0 ∈ a a x = x + y , where y = x x N(f). c 0 − c 0 ∈

2.6 Finite Dimensional Case

Let X,Y be ﬁnite dimensional vector spaces over the same ﬁeld K, and let T : X Y be → a linear operator. Let E = e ,...,en ,B = b ,... bn be bases for X,Y respectively. { 1 } { 1 } Then x X and y = Tx Y and T ek Y, k have the unique representations ∈ ∈ ∈ ∀ n x = ξkek k=1 Xn T ek = τikbi i=1 Xn y = ηibi. i X=1 32 Then calculate

y = Tx n = T ξkek k=1 ! n X = ξkT ek Xk=1 which implies

n n n y = ηibi = ξk τikbi i=1 k=1 i=1 ! X Xn n X = τikξk bi i ! X=1 Xk=1 yielding n ηi = τikξk. k X=1 So if we label TEB = (τik), x¯ = (ξk), y¯ = (ηi) then y¯ = TEBx.¯

We say TEB represents T with respect to the bases E,B.

Remark 2.41 (Finite Dimensional Linear Operators are Matrices). The above argument shows that every f.d.l.o. can be represented as a matrix.

2.6.1 Linear Functionals

Let X be a vector space, n = dim X, and let e ,...,en be a basis for X. Label { 1 }

αi = f(ei).

Then n n f(x)= f ξiei = ξiαi. i ! i X=1 X=1 (f is uniquely determined by its values αi at the n basis vectors of X).

33 Deﬁnition (Dual Basis). Deﬁne fk by

fk(ei)= δik, k = 1,...,n. ∀ Then bij f ,...,fn e ,...,en . { 1 } ↔ { 1 } Theorem 2.42 (Dimension of X∗). Let X be a vector space, n = dim X, and let E = e ,...,en be a basis for X. Then f ,...,fk is a basis for X and dim X = n. So { 1 } { 1 } ∗ ∗ then if f X then we can represent ∈ ∗ n f = αifi. i X=1 Proof. See Theorem 2.41.

Lemma 2.43 (Zero Vector). Let X be a fdvs. If x X has the property that f(x ) = 0 ∈ 0 0, f X then x = 0. ∀ ∈ ∗ 0 Proof. Let e ,...,en be a basis for X and let { 1 } n x0 = ξ0iei. i X=1 Then n f(x0)= ξ0iαi. i X=1 By assumption f(x ) = 0, f X ξ i = 0, i = 1,...,n. 0 ∀ ∈ ∗ ⇒ 0 ∀ Theorem 2.44 (Algebraic Reflexivity). A fdvs X is algebraically reflexive, i.e., X is isomorphic to X∗∗. Proof. By construction C : X into X is linear. Then −→ ∗∗ Cx = 0 (Cx )(f)= gx (f)= f(x ) = 0, f X∗ 0 ⇒ 0 0 0 ∀ ∈ and therefor x = 0 by the previous lemma. Therefore C has an inverse C −1 : (C) X. 0 R → Therefore dim (C) = dim X. Therefor C is an isomorphism and X is algebraically R reflexive.

Problem 15. Find a basis for the null space of functional f : R3 R given by → f(x)= α1ξ1 + α2ξ2 + α3ξ3 where α = 0. 1 6 3 Solution. Let (α , α , α ) with α = 0 be a ﬁxed point in R and let f(x)= aixi. The 1 2 3 1 6 vectors ( α2 , 1, 0) and ( α3 , 0, 1) form a basis for N(f). − α1 − α2 P

34 2.7 Dual Space Deﬁnition (B(X,Y )). Let X,Y be vector spaces. Then we deﬁne B(X,Y ) as the set of all bounded linear operators from X to Y .

Remark 2.45 (B(X,Y ) is a Vector Space). (αS + βT )x = αSx + βTx

Theorem 2.46 (B(X,Y ) is a Normed Space). Let X,Y be normed spaces. The vector space B(X,Y ) is also a normed space with norm

Tx T = sup k k = sup Tx . k k x∈X x x∈X k k x=06 k k kxk=1

Theorem 2.47 (B(X,Y ) is a Banach Space). If in the assumptions of the previous proof Y is Banach, then B(X,Y ) is also a Banach space.

Proof. Let (Tn) be an arbitrary Cauchy sequence in B(X,Y ). Then

ε> 0 N s.t. Tn Tm < ε, m,n >N. ∀ ∃ k − k ∀ Therefore x X, and m,n>N ∀ ∈

Tnx Tmx = (Tn Tm)x k − k k − k Tn Tm x ≤ k − k k k < ε x . k k

So ﬁx x X. We see x X that (Tnx) Y is Cauchy. Y is complete so y ∈ ∀ ∈ ⊂ ∃ ∈ Y s.t. Tnx y. We need to construct a limit for (Tn). Deﬁne T : X Y by → → Tx = y.

By construction, T is linear

lim Tn(αx + βy) = lim (αTnx + βTnx) n n →∞ →∞ = α lim Tnx + β lim Tnx n n →∞ →∞ = αT x + βTx.

The following

Tnx Tx = Tnx lim Tmx m k − k − →∞ = lim Tnx Tmx m →∞ k − k ε x ≤ k k

35 implies that Tn T is bounded. Since Tn B(X,Y ) Tn is bounded and − ∈ ⇒ T = Tn (Tn T ) − − T is also bounded. Therefore T B(X,Y ). ∈ Finally, Tnx Tx k − k ε x ≤ k k implies Tn T ε Tn T 0. k − k≤ ⇒ k − k→

Deﬁnition (Normed Dual Space X′). Let X be a n.s.. Then the set of all bounded linear functionals on X constitutes a n.s. with norm f(x) f = sup | | = sup f(x) . k k x∈X x x∈X | | x=06 k k kxk=1 This is the normed dual of X.

Theorem 2.48 (X′ is a Banach Space). Let X be a n.s. and let X′ be the dual of X. Then X′ is a Banach space. Proof. See Theorem 2.47.

Example 2.49 (Rn). The dual of Rn is Rn. n n n Proof. We claim R ∗ = R ′ and f R ∗ we have ∀ ∈ n f(x)= ξkαk, αk = f(ek). k X=1 So then n n 2 f(x) ξkαk x α , | |≤ | | ≤ k k v k k=1 uk=1 X uX which implies t n f α2. k k≤ v k uk=1 uX t If we choose x = (αk) then

n n f(x) = α2 f = α2. | | v k ⇒ k k v k uk=1 uk=1 uX uX t t 36 n n So take the onto mapping from R ′ to R

f α = (αk) = (f(ek)) 7→ which is norm-preserving, linear, and a bijection (an isomorphism), which completes the proof.

Problem 16.

1 1. Show ℓ ′ = ℓ∞

p q 1 1 2. Show ℓ ′ = ℓ where 1

1 1 1. A Schauder basis for ℓ is (ek), where ek = (δki). Then every x ℓ has a unique ∈ representation ∞ x = ξkek. k X=1 ′ ′ We consider any f ℓ1 , where ℓ1 is the dual space of ℓ1. Since f is linear and ∈ bounded, ∞ f(x)= ξkγk , where γk = f(ek). k X=1 It is easy to see that (γk) ℓ . ∈ ∞ On the other hand, for every b = (βk) ℓ , ∈ ∞

∞ 1 g(x)= ξkβk , where x = (ξk) ℓ ∈ Xk=1 ′ is a bounded linear functional on ℓ1, i.e., g ℓ1 . ∈ It is also easy see that f = c , where c = (γk) ℓ∞. It follows that the bijective ′ k k k k∞ ∈ linear mapping of ℓ1 onto ℓ deﬁned by f c is an isomorphism. ∞ 7→ 2. Take the same Schauder basis and representation for x and f as in 1. Let q be the (n) conjugate of p and consider xn = (ξk ) with

q (n) γk ξk = | | , if k n and γk = 0, γk ≤ 6 and (n) ξk =0 if k>n or γk = 0.

37 Then we have n n ∞ (n) q q 1 f(xn)= ξ γk = γk f ( γk ) p . k | | ≤ k k | | Xk=1 Xk=1 Xk=1 1 1 Using 1 p = q we get − n q 1 ( γk ) q f . | | ≤ k k Xk=1 Since n is arbitrary, letting n , we obtain →∞

∞ q 1 ( γk ) q f , | | ≤ k k k X=1 q and therefore (γk) ℓ . ∈ q p Conversely, for any b = (βk) ℓ we may deﬁne g on ℓ by ∈

∞ p g(x)= ξkβk , where x = (ξk) ℓ . ∈ k X=1 ′ Then g is linear, and boundedness follows from the Holder inequality. Hence g ℓp . ∈ From the Holder inequality we get

∞ q 1 f(x) x ( γk ) q . | | ≤ k k | | Xk=1 q It follows that f = c q, where c = (γk) ℓ and γk = f(ek). The mapping of ′ k k k k ∈ ℓp onto ℓq deﬁned by f c is linear, bijective, and norm-preserving, so it is an 7→ isomorphism.

38 3 Hilbert Spaces

Definition (Inner Product). Let X be vector space with scalar field K. , : X X K h· ·i × → is called an inner product if 1. x + y, z = x, z + y, z h i h i h i 2. αx,y = α x,y h i h i 3. x,y = y,x h i h i 4. x,x 0, x,x = 0 iff x = 0 h i≥ h i Definition (Hilbert Space). A Hilbert space is a complete inner product space. Remark 3.1.

1. A vector space X with inner product , has the induced norm and metric: h· ·i x = x,x k k h i d(x,y)= x y . k − k 2. If X is real then x,y = y,x . h i h i 3. From the deﬁnition of inner product, we obtain

αx + βy,z = α x, z + β y, z h i h i h i x,αy = α x,y h i h i x,αy + βz = α x,y + β x, z h i h i h i So the inner product is linear in the ﬁrst argument and conjugate linear in the second argument.

4. Parallelogram Equality.

x + y 2 + x y 2 = 2 x 2 + y 2 (5) k k k − k k k k k Any norm which is induced from an inner product must satisfy this equality.

5. Orthogonality. x,y X are said to be orthogonal if x,y = 0 and we write ∈ h i x y. ⊥ For subsets A, B X we say A B if ⊂ ⊥ a b a A, b B. ⊥ ∀ ∈ ∈

39 Example 3.2. 1. Rn with inner product x,y = xT y. h i 2. Cn with inner product x,y = xT y. h i 3. Real 2[a, b] with L b x,y = x(t)y(t)dt. h i Za 4. Complex 2[a, b] with L b x,y = x(t)y(t)dt. h i Za 5. ℓ2 with inner product ∞ x,y = xiyi. h i i X=1 Problem 17. 1. Show ℓp with p = 2 is not a Hilbert space. 6 2. Show C[a, b] is not a Hilbert space. Solution. p 1. Consider x = (1, 1, 0, ..., 0, ..) and y = (1, 1, 0, ..., 0, ...). Then x,y l and x p = 1 − ∈ k k y p = 2 p , and x+y p = x y p = 2. It follows that if p = 2, then the paralelogram k k k k k − k 6 equality is not satisﬁed.

t a t a 2. Let x(t)= − and y(t) = 1 − . Then we have x = y = x+y = x y = 1 b a − b a k k k k k k k − k and the paralelogram− equality is− not satisﬁed.

Problem 18. Show that x = x,x defines a norm. k k h i Solution. It follows from the definitionp of the inner product that x 0 and x = 0 iff k k ≥ k k x = 0. Also, we have using the definiton of the inner product that

1 1 2 1 αx = ( αx, αx ) 2 = (αα¯ x,x ) 2 = ( α x,x ) 2 = α x . k k h i h i | | h i | |k k Concerning the triangle inequality we get x + y 2 = x + y,x + y = x,x + x,y + y,x + y,y = x 2 + 2Re x,y + y 2 k k h i h i h i h i h i k k h i k k x 2 + 2 x y + y 2 = ( x + y )2, ≤ k k k kk k k k k k k k from which the result follows.

40 Lemma 3.3 (Continuity of Inner Product). If in an inner product space yn y and → xn x then → xn,yn x,y . h i → h i Theorem 3.4 (Completion). For any inner product space X there is a Hilbert space H and isomorphism A from X onto a dense subspace W H. The space H is unique up to ⊂ isomorphisms. Theorem 3.5 (Subspace). Let Y H with H a Hilbert space. Then ⊂ 1. Y is complete iff Y is closed in H. 2. If Y is finite dimensional then Y is complete. 3. If H is separable then Y is separable. Definition (Convex Subset). M X is said to be convex if ⊂ αx + (1 α)y M, α [0, 1], a, b M. − ∈ ∀ ∈ ∈ Theorem 3.6. Let X be an inner product space and M = a convex subset which is 6 ∅ complete. Then x X ! y M s.t. ∀ ∈ ∃ ∈ δ = inf x yˆ = x y . yˆ M k − k k − k ∈ Proof. 1. Existence. By definition of inf,

(yn) s.t. δn δ, where δn = x yn . ∃ → k − k We will show (yn) is Cauchy. Write vn = yn x. Then vn = δn and − k k vn + vm = yn + ym 2x k k k − k 1 = 2 (yn + ym) x 2δ 2 − ≥

Furthermore, yn ym = vn vm, − − ⇒ 2 2 yn ym = vn vm k − k k − k 2 2 = vn + vm + 2( vn + vm ) − k k k k k k (2δ)2 + 2(δ2 + δ2 ). ≤ − n m Since M is complete, yn y M. Then x y δ. Also, we have → ∈ k − k≥ x y x yn + yn y k − k ≤ k − k k − k = δn + yn y k − k δ + 0 → therefore x y = δ. k − k

41 2. Uniqueness. Suppose that two such elements exist, y ,y M and from above, 1 2 ∈ x y = δ = x y . k − 1k k − 2k By (5) we have

y y 2 = (y x) + (x y ) 2 k 1 − 2k k 1 − − 2 k = 2 y x 2 + 2 y x 2 (y x) + (y x) 2 k 1 − k k 2 − k − k 1 − 2 − k 1 2 = 2δ2 + 2δ2 22 (y + y ) x − 2 1 2 −

0 ≤ since 1 2 22 (y + y ) x 4δ2. 2 1 2 − ≥

Therefore y1 = y2.

Lemma 3.7 (Orthogonality). With the setup of the previous theorem, z = x y is orthog- − onal to M.

Proof. Suppose z M were false, then w M s.t. ⊥ ∃ ∈ z, w = β = 0. h i 6 and so w = 0. For any α, 6 z αw 2 = z αw, z αw k − k h − − i = z, z α z, w α [ w, z α w, w ] h i− h i− h i− h i = z 2 αβ α [β α w, w ] . k k − − − h i If we choose β α = , w, w h i and continue the equality above,

β 2 z αw 2 = δ2 | | α [0] k − k − w, w − h i < δ2 contradicting the minimality of y, δ. Therefore we must have z M. ⊥

42 Deﬁnition (Direct Sum). A vector space X is said to be direct sum of subspaces Y and Z, written X = Y Z, ⊕ if x ! y Y, z Z s.t. ∀ ∈ ∃ ∈ ∈ x = y + z.

Deﬁnition (Orthogonal Complement). Let Y be a closed subspace of X. Then the orthogonal complement of Y in X is

Y ⊥ = z X s.t. z Y . { ∈ ⊥ } Theorem 3.8 (Direct Sum). Let Y be a closed subspace of H. Then

H = Y Y ⊥. ⊕ Proof.

1. Existence. H is complete and Y is closed implies Y is complete. Further, we know Y is convex. Therefore x H y Y s.t. ∀ ∈ ∃ ∈

x = y + z, where z Y ⊥. ∈

2. Uniqueness. Assume x = y + z = y1 + z1. Then

y y Y − 1 ∈

z z Y ⊥. − 1 ∈ But Y Y = 0 y y = z z = 0. ∩ ⊥ { }⇒ − 1 − 1

Deﬁnition (Orthogonal Projection). Let Y be a closed subspace of H, so H = Y Y , ⊕ ⊥ and let x = y + z y Y ∈ z Y ⊥. ∈ Then the orthogonal projection onto Y is P : H Y given by → Px = y.

Clearly P is bounded. Further P is idempotent, that is, P 2 = P , which we call idempotent.

43 Deﬁnition (Annihilator). Let X be an inner product space and let M be a nonempty subset of X. Then the annihilator of M is

M ⊥ = x X s.t. x M = x X s.t. x, v = 0 v M . { ∈ ⊥ } { ∈ h i ∀ ∈ } Theorem 3.9. Let M, X be as the deﬁnition above and denote (M ⊥)⊥ = M ⊥⊥. Then

1. M ⊥ is a vector space.

2. M ⊥ is closed. 3. M M . ⊂ ⊥⊥ Theorem 3.10. If M is a closed subspace of a Hilbert space H then

M ⊥⊥ = M. Lemma 3.11 (Dense Set). Let M = be a subset of a Hilbert space H. Then 6 ∅ span (M)= H iﬀ M ⊥ = 0 . { } Proof.

1. . Assume x M , V = span (M) is dense in H. Then x V = H (xn) ⇒ ∈ ⊥ ∈ ⇒ ∃ ⊂ V s.t. xn x. Now x M and M V xn,x = 0. But , is continuous → ∈ ⊥ ⊥ ⊥ ⇒ h i h· ·i xn,x x,x = 0 x = 0 M = 0 , since x M was arbitrary. ⇒h i → h i ⇒ ⇒ ⊥ { } ∈ ⊥ 2. . Suppose M = 0 and write V = span (M). Then ⇐ ⊥ { } x V x M ⊥ ⇒ ⊥ x M ⊥ ⇒ ∈ x = 0 ⇒ V ⊥ = 0 ⇒ { } V = H. ⇒

Deﬁnition (Orthonormal Set/Sequence). M is said to be an orthogonal set if x,y M ∀ ∈ x = y x,y = 0. 6 ⇒ h i M is said to be orthonormal if x,y M ∀ ∈ 0 x = y x,y = δxy = 6 . h i 1 x = y If M is countable and orthogonal (resp. orthonormal) then we can write M = (xn) and we say (xn) is an orthogonal (resp. orthonormal) sequence.

44 Remark 3.12 (Pythagorean Relation, Linear Independence). Let M = x ,...,xn be an { 1 } orthogonal set. Then 1. n 2 n 2 xi = xi . k k i=1 i=1 X X

2. M is linearly independent. Example 3.13 (Orthonormal Sequences). 1. (1, 0,..., 0), (0, 1, 0,..., 0),..., (0,..., 0, 1, 0), (0,..., 0, 1) R. { }⊂ 2 2. (en) ℓ where en = δni. ⊂ 3. Let X = C[0, 2π] with x,y = 2π x(t)y(t)dt. Then the functions h i 0 (sin nt),n =R 1, 2,..., (cos nt),n = 0, 1, 2,...

form an orthogonal sequence. Remark 3.14 (Unique Representation with Orthonormal Sequences). Let X be an inner product space and let (ek) be an orthonormal sequence in X, and suppose x span ( e ,...,en ) ∈ { 1 } where n is ﬁxed. Then we can represent n n x = αkek x, ei = αkek , ei = αi ⇒ h i k=1 *"k=1 # + X n X x = x, ek ek. ⇒ h i Xk=1 Now let x X be arbitrary and take y span ( e ,...,en ), where ∈ ∈ { 1 } n y = x, ek ek. h i Xk=1 Deﬁne z by setting x = y + z z y because ⇒ ⊥ z,y = x y,y h i h − i = x,y y,y h i − h i 2 = x, x, ek ek y h i − k k D hX iE 2 = x, x, ek ek y h h i i − k k 2 = X x, ek x, ek x, ek h i h i− |h i| =X 0. X

45 Then

x 2 = y 2 + z 2 ⇒ k k k k k k 2 2 2 z = x x, ek 0 ⇒ k k k k − |h i| ≥ n 2 2 k=1 x, ek x X ⇒ |h i|2 ≤ k k2 ∞ x, ek x ⇒ Pk=1 |h i| ≤ k k

Remark 3.15 (Bessel’s Inequaltiy)P . Let X be an inner product space and let (ek) be an orthonormal sequence in X. Then the last inequality in the above remark is called Bessel’s Inequality: ∞ 2 2 x, ek x . |h i| ≤ k k k X=1 Definition (Fourier Coefficients). The sequence ( x, ek ) is called the Fourier coefficients h i of x w.r.t. (ek).

Problem 19 (Fourier Coefficients are Minimizers). Let en,...,en be an orthonormal set { } in an inner product space X (n is fixed). Let x X be an arbitrary, fixed element and n ∈ let y = βkek. Then x y depends on β ,...,βn. Show by direct calculation that k=1 k − k 1 x y is minimized iff βi = x, ei , i = 1,...,n. k − kP h i ∀ Solution. Let γi = x, ei , and y = βiei. Then h i 2 P 2 2 x y = x βiei,x βiei = x β¯iγi βiγ¯i + βi k − k − − k k − − | | D X X E X X X 2 2 2 = x γi + βi γi k k − | | | − | and this is minumum for given x and eXis iff βi =Xγi.

Problem 20 (Gramm-Schmidt). Orthonormalize the ﬁrst three terms of the sequence (1,t,t2,t3,... ) on the interval [ 1, 1] where − 1 x,y = x(t)y(t)dt. h i 1 Z− f1(t) 1 Solution. Let f1(t) = 1, then e1(t) = = . Let f2(t) = t. We have that f1(t) √2 k k f ( ), e ( ) = 0, so we just need to normalize f (t) to get e (t) = 3 t. Let f (t) = t2. h 2 · 1 · i 2 2 2 3 Easy calculation shows that f ( ), e ( ) = √2 and f ( ), e ( ) q= 0. Then f (t) = h 3 · 1 · i 3 h 3 · 2 · i 3 f ( ), e ( ) e (t)= t2 1 . Normalizing this quantity we get e (t)= 5 (3t2 1). h 3 · 2 · i 2 − 3 3 8 − q

46 Theorem 3.16 (Convergence). Let H be a Hilbert space and let (en) H be an orthonor- ⊂ mal sequence. Consider ∞ αkek. (6) Xk=1 1. The series (6) converges in the induced norm of H iﬀ

∞ 2 αk < . | | ∞ Xk=1

2. If (6) converges to x, then αk = x, ek and h i ∞ x = x, ek ek. h i Xk=1

3. For any x H the series (6) with αk = x, ek converges (in the norm of H). ∈ h i Proof. Let

n sn = αkek k=1 Xn 2 σn = αk . | | k X=1 1. For n>m

n 2 2 sn sm = αkek k − k k=m+1 nX 2 = αk | | k m =X+1 = σn σm. −

Hence (sn) is Cauchy in H iﬀ σn) is Cauchy in R.

2. Note sn, ei = αi, i = 1,...,k n. By assumption sn x, so h i ∀ ≤ →

αi = sn, ei x, ei , for i k ⇒ h i → h i ≤ αi = x, ei i = 1, 2,... ⇒ h i ∀ 3. This follows from Bessel’s inequality and 1.

47 Definition (Total Set). Let X be an inner product space and let M X. ⊂ 1. span (M)= X M is a total set. ⇒ 2. If M is an orthonormal set then M is a total orthonormal set. Remark 3.17. 1. A total orthonormal family in X is sometimes called an orthonormal basis for X. Note this is not equivalent to an algebraic basis unless X is finite dimensional. 2. In every nontrivial Hilbert space H = 0 there is a total orthonormal set. 6 { } Definition (Hilbert Dimension). The Hilbert dimension of X is the cardinality of the smallest orthonormal set, i.e., if Λ = M s.t. span (M)= H then the Hilbert dimension is n o inf M . M Λ | | ∈ Problem 21. Let X be an inner product space and let M X. Then ⊂ 1. If M is total in X then x M x = 0 (7) ⊥ ⇒ 2. If X is complete, then (7) M is total in X. ⇒ Solution. 1. Let H be the completion of X. Then, X regarded as a subspace of H, is dense in H. By assumption, M is total in X, so spanM is dense in X, and dense in H. It follows that the orthogonal complement of M in H is 0 . { } 2. If x is a Hilbert space and M = 0 , then the Dense Set Lemma implies that M is ⊥ { } total in X.

Problem 22. Show that an orthonormal set M in a Hilbert space H is total iﬀ

2 2 x, ek = x , x M. |h i| k k ∀ ∈ k X This is called Parseval’s equality.

Solution. If M is not total, by Problem 21 there is a nonzero x M in H. Since x M ⊥ ⊥ we have 0 on the left-hand side of Parseval’s Equality which is not equal to x . Hence if k k Parceval’s Equality holds for all x H, then M must be total in H. ∈ Conversely, assume M to be total in H. Consider any x H and its nonzero Fourier ∈ coeﬃcients arranged in a sequence, i.e., x, e , x, e , ...,. Deﬁne y = x, ek ek. It h 1i h 2i h i follows that x y M . Since M is total, then x = y. − ∈ ⊥ P

48 Theorem 3.18 (Separable Hilbert Space). Let H be a Hilbert space. 1. If H is separable then every orthonormal set is countable. 2. If H contains an orthonormal sequence which is total then H is separable.

3.1 Representation of Functionals on Hilbert Spaces Theorem 3.19 (Riesz). Let H be a Hilbert space. Every bounded, linear functional on H can be represented in terms of the inner product on H, i.e., f(x)= x, z h i where z is uniquely determined by f and

z = f . k k k k Proof. 1. Existence of z. If f = 0 take z = 0. Otherwise assume f = 0. Then (f) = H and 6 N 6 (f) = H (f)⊥ = 0 . Let w (f)⊥ s.t. w = 0 and set N 6 ⇒N 6 { } ∈N 6 v = f(x)w f(w)x, x H. − ∈ Then

f(v)= f(x)f(w) f(w)f(x) = 0 ⇒ − v (f) . ⇒ ∈N Since w (f) we have ⊥N 0 = v, w h i = f(x)w f(w)x, w h − i = f(x) w, w f(w) x, w h i− h i = f(x) w 2 f(w) x, w . k k − h i Then f(w) f(x) = x, w ⇒ w 2 h i k k f(w) f(x) = x, 2 w . ⇒ * w + k k Then f(w) z = w. w 2 k k

49 2. Uniqueness of z. Suppose f(x)= x, z = x, z . Then h 1i h 2i x, z z = 0, x H. ⇒ h 1 − 2i ∀ ∈ Choose x = z z . Then 1 − 2 z z , z z = 0 ⇒ h 1 − 2 1 − 2i z = z . ⇒ 1 2 3. f = z . If f = 0 then z = 0 and f = z = 0. For f = 0, z = 0. Note k k k k k k k k 6 6 f(z) = z, z h i = z 2 , and k k z 2 f z k k ≤ k k k k z f . ⇒ k k ≤ k k Also

f(x) = x, z | | |h i| x z ≤ k k k k f z . ⇒ k k ≤ k k This yields f = z . k k k k

Lemma 3.20 (Equality). Let X be an inner product space. Then

x, w = y, w w X x = y. h i h i ∀ ∈ ⇒ In particular, x, w = 0 w X x = 0. h i ∀ ∈ ⇒ Deﬁnition (Sesquilinear Form). Let X,Y be vector spaces over the same scalar ﬁeld K (R or C). A sesquilinear form (or sesquilinear functional) h on X Y is a mapping × h : X Y K × → such that x,x1,x2 X and y,y1,y2 Y and α, β K ∀ ∈ ∈ ∈ 1. h(x1 + x2,y)= h(x1,y)+ h(x2,y)

50 2. h(x,y1 + y2)= h(x,y1)+ h(x,y2) 3. h(αx,y)= αh(x,y)

4. h(x,βy)= βh(x,y)

Remark 3.21.

1. If X,Y are real (K = R), then

h(x,βy)= βh(x,y)

and h is said to be bilinear.

Theorem 3.22 (Riesz Representation). Let H1,H2 be Hilbert spaces and

h : H1 H2 K × → a bounded sesquilinear form. Then h has a representation

h(x,y)= Sx,y h i where S : H H is a bounded, linear operator. S is uniquely determined by h and 1 → 2 S = h . k k k k Proof.

1. Existence of S. Consider h(x,y) which is linear in y. If we ﬁx an x, then h(x,y) is a bounded, linear functional and we can use the Riesz theorem to ﬁnd z and represent

h(x,y)= y, z , h i

51 hence h(x,y)= z,y . (8) h i Note that z is unique, but it depends on x H . This means that (8) with variable ∈ 1 x deﬁnes an operator S : H H given by 1 → 2 z = Sx,

and we write h(x,y)= z,y = Sx,y . h i h i We must show S is linear. Observe

S(αx + βx ),y = h(αx + βx ,y) h 1 2 i 1 2 = αh(x1,y)+ βh(x2,y) = α Sx ,y + β Sx ,y , y H h 1 i h 2 i ∀ ∈ 2 S(αx + βx ) = αSx + βSx . ⇒ 1 2 1 2 2. Uniqueness of S. If h(x,y)= Sx,y = Tx,y , then S = T by the equality lemma. h i h i 3. Boundedness of S. Note ﬁrst Sx,y h = sup |h i| k k x=06 x y y6=0 k k k k Sx,Sx sup |h i| ≥ x=06 x Sx Sx=06 k k k k Sx = sup k k x=0 x 6 k k = S k k h S . ⇒ k k ≥ k k So S is bounded. Also Sx,y h = sup |h i| k k x=06 x y y6=0 k k k k Sx y sup k k k k ≤ x=06 x y y6=0 k k k k Sx sup k k ≤ x=0 x 6 k k = S k k h S . ⇒ k k ≤ k k

52 This yields S = h . k k k k

Problem 23. Let H be a Hilbert space. Show that H′ (the dual of H) is a Hilbert space with inner product , deﬁned by h· ·i1

fz,fv = z, v = v, z , h i1 h i h i where

fz(x) = x, z h i fv(x) = x, v h i ′ Solution. It is easy to verify that , 1 is an inner product on H . Since fz = z = 1 1 h· ·i ′ k k k k ( z, z ) 2 = ( fz,fz 1) 2 the norm on H is induced by the inner product , 1. We know h i h i ′ h· ·i that the normed dual is always a Banach space. It follows that H is complete, and therefore a Hilbert space.

Problem 24. Show that any Hilbert space H is isomorphic with its second dual, i.e.,

H ∼= (H′)′ = H′′. ′′ ′ Solution. Let T : H h by z Fz, where Fz : H K is deﬁned by Fz(f) = f,fz , → → → h i1 where , 1 and fz are the same as in Problem 23. By repeated applications of the Riesz h· ·i ′ representation theorem we have that F (f) = f, g for some g h and g = fz for some h i1 ∈ z H. It follows that T is surjective. ∈ ′ Suppose that z, w H and Fz = Fw. Then f,fz = f,fw for all f H and we ∈ h i1 h i1 ∈ get that fz = fw and z = w. Therefore, T is injective. It is easy to see that T is linear. Also, by the Riesz representation theorem we have that Fz = fz = z , i.e., T k k k k k k preserves norms, and inner products. It follows that T is an isomorphism.

3.2 Hilbert Adjoint Deﬁnition (Hilbert Adjoint T ). Let H ,H be Hilbert spaces and let T : H H be a ∗ 1 2 1 → 2 bounded, linear operator. Then the adjoint is T : H H s.t. ∗ 2 → 1

Tx,y = x, T ∗y , x H and x H . h i h i ∀ ∈ 1 ∈ 2 Theorem 3.23 (Existence of the Hilbert Adjoint).

1. T ∗ exists.

53 2. T ∗ is unique. 3. T = T . k ∗k k k Proof. Observe that h(y,x)= y,Tx is sesquilinear form on H H . h i 2 × 1 h(x,y) y Tx ⇒ | | ≤ k k k k T x y ≤ k k k k k k h T . ⇒ k k ≤ k k Also y,Tx h = sup |h i| k k x=06 y x y6=0 k k k k Tx,Tx sup |h i| ≥ x=06 Tx x Tx6=0 k k k k = T k k h = T . ⇒ k k k k Therefore h is a bounded sesquilinear form. Using a Riesz representation, h(x,y)= T y,x h ∗ i where T ∗ exists, is unique with norm

T ∗ = h = T . k k k k k k Also

y,Tx = T ∗y,x ⇒ h i h i Tx,y = x, T ∗y . ⇒ h i h i

Lemma 3.24 (Zero Operator). Let X,Y be inner product spaces and Q : X Y be → bounded, linear operators. Then

1. Q = 0 iﬀ Qx,y = 0 x X and y Y h i ∀ ∈ ∈ 2. Let X be complex and Q : X X. Then → Qx,x = 0 x X Q = 0. h i ∀ ∈ ⇒ Proof.

54 1. . ⇒ Q = 0 Qx = 0 x X ⇒ ∀ ∈ Qx,y = 0,y = 0 w,y = 0. ⇒ h i h i h i . ⇐ Qx,y = 0 x,y Qx = 0 x,y h i ∀ ⇒ ∀ Q = 0. ⇒ 2. Let x,y X, then v = αx + y X. Then ∈ ∈ Qv,v = Qx,x = Qy,y = 0 h i h i h i ⇒ 0 = Q(αx + y), αx + y h i = α 2 Qx,x + Qy,y + α Qx,y + α Qy,x | | h i h i h i h i = α Qx,y + α Qy,x . h i h i Now, take α = 1 then α = i to obtain the two relations

Qx,y + Qy,x = 0 h i h i Qx,y Qy,x = 0. h i − h i Then Qx,y = 0 Q = 0 by 1. h i ⇒

Remark 3.25. In the previous lemma, 2 is not necessarily true if X is real.

Theorem 3.26 (Properties of The Hilber Adjoint). Let H1,H2 be Hilbert spaces. Let S, T : H H be bounded, linear operators and let α be a scalar. Then 1 → 2 1. T y,x = y,Tx h ∗ i h i 2. (S + T )∗ = S∗ + T ∗

3. (αT )∗ = αT ∗

4. (T ∗)∗ = T 5. T T = T T = T 2 k ∗ k k ∗k k k 6. T ∗T = 0 iﬀ T = 0

55 7. (ST )∗ = T ∗S∗, assuming H1 = H2. Deﬁnition (Self-Adjoint, Unitary, Normal). Let H be a Hilbert space and T : H H. → 1. T is self-adjoint (or Hermitian) if T ∗ = T

−1 2. T is unitary if T is bijection and T ∗ = T

3. T is normal if T T ∗ = T ∗T Remark 3.27.

1. If T is self-adjoint then Tx,y = x,Ty . h i h i 2. If T is self-adjoint or normal then T is normal.

Example 3.28. Consider Cn with x,y = xT y. Let T : Cn Cn. If we specify a basis n h i → for C , we can represent T, T ∗ by matrices A, B. Then

Tx,y = (Ax)T y h i = xT AT y T x, T ∗y = x By. h i Therefore

AT = B ⇒ T B = A . ⇒ T If T is self-adjoint then A = A .

Theorem 3.29 (Self-Adjointness). Let H be a Hilbert space and let T : H H be a → bounded, linear operator. Then

1. If T is self-adjoint then Tx,x is real for all x H. h i ∈ 2. If H is complex and Tx,X is real for all x H then T is self-adjoint. h i ∈ Proof.

1. If T is self-adjoint, then for all x X we have ∈ Tx,x = x,Tx = Tx,x . h i h i h i

56 2. If Tx,x is real for all x X, then h i ∈

Tx,x = Tx,x = x, T x = T ∗x,x . h i h i h ∗ i h i Hence

0 = Tx,x T ∗x,x h i − h i = (T T ∗)x,x h − i T = T ∗ ⇒ by the zero operator lemma, since H is complex.

Theorem 3.30. Let H be a Hilbert space and let (Tn) be a sequence of bounded, linear, self-adjoint operators Tn : H H. Suppose that Tn T in norm, that is Tn T 0, → → k − k→ where is the norm on the space B(H,H). Then T is a bounded, linear, self-adjoint k·k operator on H.

Proof. We show T ∗ = T .

T T ∗ T Tn + Tn T ∗ + T ∗ T ∗ k − k ≤ k − k k − n k k n − k = T Tn +0+ T ∗ T ∗ k − k k n − k = 2 T Tn k − k 0. →

Theorem 3.31 (Unitary Operators). Let H be a Hilbert space and let U, V : H H be → unitary. Then

1. U is isometric, i.e., Ux = x x H k k k k ∀ ∈ 2. U = 1 provided H = 0 k k 6 { } −1 3. U = U ∗ is unitary 4. UV is unitary

5. U is normal.

6. If T is a bounded, linear operator on H and H is complex, then T is unitary iﬀ T is isometric and surjective.

Proof.

57 1.

Ux 2 = Ux,Ux k k h i = x, U ∗Ux h i = x,Ix h i = x 2 k k 2. Follows from 1.

3. Since U is bijective, so is U −1 and

−1 −1 −1 (U )∗ = U ∗∗ = U = (U ) .

4. UV is bijective and

−1 −1 −1 (UV )∗ = V ∗U ∗ = V U = (UV ) .

−1 −1 −1 5. U = U ∗ and UU = U U = I. 6. . Suppose that T is isometric and surjective. Isometry implies injectivity, so T is ⇒ −1 bijective. Need to show T ∗ = T . By isometry,

T ∗Tx,x = Tx,Tx h i h i = x,x h i = Ix,x h i (T ∗T I)x,x = 0 ⇒ h − i T ∗T = I. ⇒ Also,

−1 T T ∗ = T T ∗(T T ) −1 = T (T ∗T )T = I.

−1 Therefore T ∗ = T . . Conversely, T is isometric by 1 and surjective by deﬁnition. ⇐

58 4 Fundamental Theorems

Deﬁnition (Partially Ordered Set). Let M be a set with a relation . This relation is ≤ said to be a partial order if

1. a a a M ≤ ∀ ∈ 2. a b and b a a = b ≤ ≤ ⇒ 3. a b and b c a c. ≤ ≤ ⇒ ≤ We say M is a partially ordered set.

Deﬁnition (Upper Bound). Let M be partially ordered set. If u M s.t. x u x ∃ ∈ ≤ ∀ ∈ M, then u is said to be a upper bound. Such a u is not guaranteed to exist.

Deﬁnition (Maximal Element). Let M be partially ordered set. If m M s.t. m ∃ ∈ ≤ x m = x x M, then m is said to be a maximal elemtn. Such a m is not guaranteed ⇒ ∀ ∈ to exist or be unique. Example 4.1.

1. R with the usual . ≤ 2. The power set P (X) is partially ordered by set inclusion. X is the unique maximal element.

n 3. x = (xi),y = (yi) R ordered by ∈

x y xi yi i = 1,...,n. ≤ ⇔ ≤ ∀ 4. The positive integers ordered by

x y x n. ≤ ⇔ | Deﬁnition (Chain). Let M be a partially ordered set. A subset C M is said to be a ⊂ chain if a, b C a c or c a. I.e., all elements in C are comparable. ∈ ⇒ ≤ ≤ The next statement is equivalent to the axiom of choice. It retain’s its name (lemma) for historical reasons.

Lemma 4.2 (Zorn’s Lemma). Let M = be a partially ordered set. Suppose that every 6 ∅ chain C M has an upper bound. Then M has at least one upper bound. ⊂ If we decide to live with the the axiom of choice, we can use Zorn’s Lemma to prove various important results.

59 Theorem 4.3 (Hemel Basis). Every vector space X = 0 has a Hemel basis. 6 { } Proof. Let M be the set of all linearly independent subsets of X. Note that M is not empty: X = 0 x X s.t. x = 0 x M. 6 { }⇒ ∃ ∈ 6 ⇒ { }∈ Set inclusion gives a partial order on M. Let C M be a chain. Then ⊂ A = D D C [∈ is an upper bound for C. With this setup we invoke Zorn’s lemma to force the existence of a maximal element B. Let Y = span (B) and so Y X. We want to show Y = X. Assume not and let ⊂ z X Y . Then B z is linearly independent, contradicting the maximality of B. ∈ − ∪ { } Theorem 4.4 (Total Orthonormal Set). In ever Hilbert space H = 0 , there exists a 6 { } total orthonormal set.

Proof. Let M be the set of all orthonormal subsets of H. M is not empty since x X = 0 x X s.t. x = 0 M. 6 { }⇒ ∃ ∈ 6 ⇒ { x }∈ k k Set inclusion gives a partial order on M. Let C M be a chain. Then ⊂ A = D D C [∈ is an upper bound for C. With this setup we invoke Zorn’s lemma to force the existence of a maximal element F . We claim this F is total. To this end, suppose not. Then z H s.t. z = 0 and z F . z ∃ ∈ 6 ⊥ Hence F z is orthonormal, contradicting the maximality of F . ∪ { k k } Deﬁnition (Sublinear Functional). Let X be a v.s. and let p : X R. p is said to be a → sublinear functional if 1. p(x + y) p(x)+ p(y) x,y X ≤ ∀ ∈ 2. p(αx)= αp(x) x X α R, α 0. ∀ ∈ ∀ ∈ ≥ Remark 4.5. The norm on a n.s. is a sublinear functional. Theorem 4.6 (Hanh-Banach (Extensions of Linear Functionals)). Let X be a real v.s. and let p be a sublinear functional on X. Let f be a linear functional deﬁned on a subspace Z X s.t. ⊂ f(x) p(x) x Z. ≤ ∀ ∈

60 Then f has a linear extension fˆ : Z X s.t. → fˆ(x) p(x) x X ≤ ∀ ∈ fˆ(x) = f(x) x Z. ∀ ∈ Proof. 1. Let E be the set of all linear extensions g of f which satisfy g(x) p(x) x (g) . ≤ ∀ ∈ D Then f E E = . We now deﬁne the following partial order on E: g h means ∈ ⇒ 6 ∅ ≤ h is a linear extension of g, i.e. (a) (g) (h) D ⊂ D (b) h(x)= g(x) x (g). ∀ ∈ D Let C be a chain in M. Then we deﬁne the maximal elementg ˆ as follows

(a) (ˆg)= g C (g) D ∈ D (b)g ˆ(x)= g(x) g C s.t. x (g). S ∀ ∈ ∈ D Then g gˆ g C gˆ is an upper bound for C. Since C E was arbitrary, ≤ ∀ ∈ ⇒ ⊂ we invoke Zorn’s lemma to produce a maximal element fˆ. Therefore fˆ is a linear extension of f.

2. Now we want to show fˆ = X. In the manner above, assume otherwise and let D y x fˆ and consider Y = span ( ) fˆ y . If x Y then we can 1 ∈ − D 1 { D ∪ { 1}} ∈ 1 write x = y + αy ,y fˆ (and this representation is unique). Then let us deﬁne 1 ∈ D a functional g1 : Y1 R as → g1(y + αy1)= fˆ(y)+ αc

where c is any real constant. Then g1 is linear extension of f. If we can show g (x) p(x) x (g ) 1 ≤ ∀ ∈ D 1 then g E, which would contradict the maximality of fˆ and imply fˆ = X. 1 ∈ D 3. To this end, let us construct a suitable c. Let y, z fˆ . Then ∈ D fˆ(y) fˆ(z) = fˆ(y z) − − p(y z) ≤ − = p(y + y y z) 1 − 1 − p(y + y )+ p( y z) ≤ 1 − 1 −

61 and therefor p( y z) fˆ(z) p(y + y ) fˆ(y). − − 1 − − ≤ 1 − Note

(a) y1 is ﬁxed (b) y does not appear on the RHS (c) z does not appear on the LHS.

Then we can take supz (fˆ), infy (fˆ) and call these values m0,m1, respectively. ∈D ∈D Then m m . Let c be any value such that m c m . Then 0 ≤ 1 0 ≤ ≤ 1 p( y z) fˆ(z) c z fˆ − − 1 − − ≤ ∀ ∈ D c p(y + y ) fˆ(y) y fˆ . ≤ 1 − ∀ ∈ D Now let α R and consider the cases ∈ (a) α< 0. 1 1 p( y y) fˆ( y) c − − 1 − α − α ≤ 1 αp( y y)+ fˆ(y) αc − 1 − α ≤ −

⇒

g1(x) = g(y + αy1) = fˆ(y)+ αc 1 αp( y y) ≤ − − 1 − α = p(αy1 + y)= p(x).

(b) α = 0. Then x fˆ so g (x)= fˆ(x) p(x). ∈ D 1 ≤ (c) α> 0. 1 1 c p( y + y ) fˆ( y) ≤ α 1 − α 1 αc αp( y + y ) fˆ(y) ≤ α 1 − = p(x) fˆ(y) −

62 ⇒

g1(x) = fˆ(y)+ αc p(x). ≤ Then g E and we have our contradiction. Therefore fˆ is our bounded linear 1 ∈ extension.

Problem 25.

1. Show that the norm on a v.s. X is a sublinear functional on X.

2. Show that a sublinear functional p satisﬁes p(0) = 0 and p( x) p(x). − ≥− Solution.

1. Let p(x)= x . Then p(x + y)= x + y x + y = p(x)+ p(y). Also, if α 0, k k k k ≤ k k k k ≥ then p(αx)= αx = α x = α x = αp(x) k k | |k k k k . It follows that p is a sublinear functional.

2. X is not empty, 0 X. Let x X, then p(0) = p(0x) = 0p(x) = 0. Also, ∈ ∈ 0= p(0) = p(x x) p(x)+ p( x). It follows that p( x) p(x). − ≤ − − ≥−

Problem 26. If p is a sublinear functional on a real v.s. X, show that there exists a linear functional fˆ on X s.t. p( x) fˆ(x) p(x). − − ≤ ≤ Solution. Let Z = x X : x = αx + 0, α R and deﬁne the linear functional f on Z by { ∈ ∈ }

f(x)= αp(x0).

Then f(x) p(x). By the Hahn-Banach Theorem there exists a fˆ bounded linear func- ≤ tional deﬁned on X such that fˆ(x) p(x). Clearly, fˆ(x) = fˆ( x) p(x). It follows ≤ − − ≤ that p( x) fˆ(x) p(x). − − ≤ ≤ Theorem 4.7 (Hanh-Banach Generalized). Let X be a v.s. over K (R or C) and let p be a real-valued functional X s.t.

1. p(x + y) p(x)+ p(y) x,y X ≤ ∀ ∈ 2. p(αx)= α p(x) α K. | | ∀ ∈

63 Let Z X be a subspace and let f : Z K be a functional s.t. ⊂ → f(x) p(x) x Z. | |≤ ∀ ∈ Then f has a bounded linear extension fˆ s.t.

1. fˆ = X D 2. fˆ(x)= f(x) x Z ∀ ∈ 3. fˆ(x) p(x) x X. ≤ ∀ ∈

Proof.

1. K = R. Then f(x) p(x) f(x) p(x). | |≤ ⇒ ≤ Then by the previous theorem fˆ s.t. fˆ(x) p(x) x X. Also, ∃ ≤ ∀ ∈ fˆ(x)= fˆ( x) p( x)= p(x) − − ≤ − so fˆ(x) p(x). ≤

2. K = C. X complex Z complex f complex ⇒ ⇒ ⇒

f = f1 + if2

where f1,f2 are real-valued. We introduce Xr, Zr the v.s.’s obtained by restricting K to R. Now note that f linear on Z and f1,f2 real-valued means f1,f2 are real-valued linear functionals on Zr. Also,

f (x) f(x) f (x) p(x) x Zr. 1 ≤ | |⇒ 1 ≤ ∀ ∈

Then by the previous theorems fˆ extension of f from Zr to Xr s.t. ∃ 1 1

fˆ (x) p(x) x Xr. 1 ≤ ∀ ∈ Going back to Z, we observe x Z ∀ ∈

if(x) = i [f1(x)+ if2(x)]

= f1(ix)+ if2(ix) = f (x)+ if (x) − 2 1 f (x) = f (ix) ⇒ 2 − 1

64 So then x X we deﬁne ∀ ∈ fˆ(x)= fˆ (x) ifˆ (ix). 1 − 1 Note fˆ(x)= f(x) x Z. ∀ ∈ (a) We claim fˆ is a linear functional on X.

fˆ((a + ib)x) = fˆ (ax + ibx) ifˆ (iax bx) 1 − 1 − = afˆ (x)+ bfˆ (ix) i afˆ (ix) bfˆ (x) 1 1 − 1 − 1 h i = (a + ib) fˆ (x) ifˆ (ix) 1 − 1 = (a + ib)fhˆ(x) i

(b) We claim fˆ(x) p(x) x X. Recall that p(x) 0. Then ≤ ∀ ∈ ≥

fˆ(x) = 0 fˆ(x) p(x). ⇒ ≤

Assume fˆ(x) = 0. We then use the exponential form of fˆ 6 fˆ(x)= fˆ(x) eiθ.

Then iθ fˆ(x) = fˆ(x)e− ˆ iθ = f(e− x) iθ iθ = fˆ1(e− x)+ ifˆ1(ie− x) fˆ(x) R ∈ ⇒

fˆ(x) = fˆ (e iθx) 1 − iθ p(e− x) ≤ iθ = e− p(x)

= p (x)

Then fˆ is the bounded, linear extension.

65 Theorem 4.8 (Hanh-Banach (for Normed Spaces)). Let X be a n.s. and let Z X be ⊂ a subspace. Let f : Z K be a bounded, linear functional. Then there exists a bounded, → linear functional fˆ which is an extension from Z to X s.t. ˆ f = f Z X k k where

fˆ = sup fˆ(x) X x∈X kxk=1

f Z = sup f(x) . k k x∈Z | | kxk=1

Proof. If Z = 0 then f = 0 and fˆ = 0. Otherwise, let Z = 0 . Then x Z we have { } 6 { } ∀ ∈ f(x) f x . | | ≤ k kZ k k Then deﬁne p(x) f x x X. ≤ k kZ k k ∀ ∈ Notice p(x + y) = f x + y k kZ k k f ( x + y ≤ k kZ k k k k = p(x)+ p(y) p(αx) = f αx k kZ k k = α f x | | k kZ k k = α p(x). | | Then we can use the Hanh-Banach (Generalized) theorem fˆ : X K s.t. ⇒ ∃ → fˆ(x) p(x)= f x x X. ≤ k kZ k k ∀ ∈

And so ˆ ˆ f = sup f(x) f Z . X x∈X ≤ k k kxk=1

Finally, since fˆ is an extension of f ˆ f f Z . X ≤ k k

Therefore,

ˆ f = f Z . X k k

66 Remark 4.9. In the previous theorem if X = H is a Hilbert space and Z H is a closed ⊂ subspace then we can represent f (for some ﬁxed z Z) ∈ f(x) = x, z x Z h i ∀ ∈ f = z k k k k fˆ(x) = x, z x X h i ∀ ∈ Theorem 4.10 (Bounded Linear Functionals). Let X be a n.s. and let x X,x = 0. 0 ∈ 0 6 Then fˆ : X K s.t. ∃ → fˆ = 1 ˆ f( x0 ) = x0 k k Proof. Let Z X be the subspace deﬁned as ⊂ Z = x X x = αx . { ∈ | 0} Then

f(x) = f(αx0) = α x k 0k f(x) = f(αx ) | | | 0 | = α x | | k 0k = αx k 0k = x k k f = 1. ⇒ k k Therefore f has a linear extension fˆ from Z to X with norm

fˆ = f = 1. k k

ˆ ˆ By deﬁnition of f and f, f(x0)= f(x 0)= x0 . k k Problem 27. To illustrate the Hanh-Banach (for Normed Spaces) consider a functional f : R2 R deﬁned by → 2 f(x)= α1x1 + α2x2 x = (x1,x2) R . ∀ ∈ Construct its linear extension to R3 and the corresponding norms.

2 2 2 1 Solution. Let fˆ(x)= α ξ + α ξ + α ξ . Then fˆ (α + α + α ) 2 f and fˆ = f 1 1 2 2 3 3 k k 1 2 3 ≥ k k k k k k iﬀ α3 = 0.

67 2 2 Problem 28. Consider the n.s. R and let x0 R ,x0 = 0. Find a bounded, linear ∈ 6 functional fˆ : R2 R s.t. → 1. fˆ = 1

2. fˆ(x )= x 0 k 0k Solution. let fˆ = x, x0 . Then fˆ x0 = 1 and fˆ(x )= x , x0 = x . x0 x0 0 0 x0 0 h k k i k kk k k k h k k i k k 4.1 Bounded, Linear Functionals on C[a, b] Deﬁnition (Variation). Let f : [a, b] R. We deﬁne the variation of f as → n Var f = sup f(ti) f(ti 1) | − − | ( i ) X=1 where a = t

Deﬁnition (Riemann-Stieltjes Integral). Let x C[a, b] and f BV [a, b] and Pn be a ∈ ∈ partition of [a, b]. Deﬁne

ηPn = max t1 t0,...,tn tn 1 . { − − − } Consider S deﬁned by n S(Pn)= x(ti) [f(ti) f(ti 1)] . − − i X=1 Then I R s.t. ε> 0 δ > 0 s.t. ∃ ∈ ∀ ∃ ηPn < δ I S(Pn) ε. ⇒ | − |≤ We call I the Riemann-Stieltjes integral of x with respect to f and write b I = x(t)df(t). Za

68 Remark 4.11.

1. If f(t)= t then we have the usual integral.

2. If f has a integrable derivative then

b b x(t)df(t)= x(t)f ′(t)dt. Za Za 3. We have b x(t)df(t) sup x(t) Var(f). ≤ | | Za t [a,b] ∈

Theorem 4.12 (Reisz - B.L.F.’s on C[a, b ]). Every b.l.f. g on C[a, b] can be represented by a Riemann-Stieltjes integral

b g(x)= x(t)df(t) Za and g = Var(f). k k Proof. By the Hanh-Banach theorem g has an extension from C[a, b] to B[a, b] the space of bounded functions with norm

x = x(t) k k | | t [a,b] ∈X and gˆ = g . k k k k Let 1 s [0,t] xt(s)= ∈ , 0 otherwise then xt B[a, b]. Let f(a)=0 and f(t) =g ˆ(xt) t (a, b]. We want to show f is of b.v. ∈ ∀ ∈ and Var F g . ≤ k k Use exponential form for a complex quantity z, z = z e(z) where | | 1 z = 0 e(z)= . eiθ z = 0 6 Note that if z = 0 then z = ze iθ 6 | | − ⇒ z = ze(z). | |

69 We shall use the notation

εi = e(f(ti) f(ti 1)) and xti = xi. − − Then n n [f(ti) f(ti 1)] = gˆ(x1) + gˆ(xi) gˆ(xi 1) − − | | | − − | i=1 i=2 X Xn = ε1gˆ(x1)+ εi [ˆg(xi) gˆ(xi 1)] − − i=1 Xn =g ˆ ε1x1 + εi [xi xi 1] − − i=2 ! Xn gˆ ε1x1 + εi [xi xi 1] ≤ k k − − i=2 X = g 1 k k · Var f g ⇒ ≤ k k f BV [a, b] ⇒ ∈ Now, given P a partition of [a, b] deﬁne n zn = x(t0)x1 + x(ti 1 [xi xi 1] − − − i X=2 then zn BV [a, b]. Now we compute ∈ n gˆ(zn) = x(t0)ˆg(x1)+ x(ti 1) [ˆg(xi) gˆ(xi 1)] − − − i=2 Xn = x(t0)f(x1)+ x(ti 1) [f(xi) f(xi 1)] − − − i=2 n X = x(ti 1) [f(xi) f(xi 1)] − − − i X=1 Then taking a sequence of partitions (Pn) s.t. ηPn 0, we obtain → b x(t)df(t). Za We need to showg ˆ(zn) gˆ(x)= g(x) where x C[a, b]. Note zn(a)= x(a) 1 zn(z)= → ∈ · ⇒ x(a) = 0. Note ti 1

70 Note ηPn 0 zn x 0 → ⇒ k − k→ since x C[a, b] x is uniformly continuous (since [a, b] is compact). Then the continuity ∈ ⇒ ofg ˆ impliesg ˆ(zn) gˆ(x) andg ˆ(x)= f(x). → Now we want to show Var f = g . Computing, k k g(x) max x(t) Var f = x Var f | |≤ t [a,b] k k ∈ and so g Var f. k k≤ Also Var f g , so ≤ k k g = Var f. k k

Remark 4.13. f is not unique in the above theorem. However, we can make f unique by requiring:

1. f(a) = 0

2. f(t+)= f(t) (continuity from the right).

4.2 Adjoint Operator Given a b.l.o. T : X Y we are interested in constructing a new operator T × : Y X . → ′ → ′ We proceed as follows. Let g Y ,x X. Setting y = Tx we obtain a function f on X ∈ ′ ∈ by deﬁning f(x)= g(Tx). f is linear because g, T are linear. f is bounded because

f(x) = g(Tx) g T x | | | | ≤ k k k k k k f g T . ⇒ k k ≤ k k k k Therefor f X . Then ∈ ′ f(x)= g(Tx) (9) with variable g Y deﬁnes an operator called the adjoint of T , ∈ ′ × T : Y ′ X′. → Deﬁnition (Adjoint). Given a b.l.o. T : X Y the adjoint T × : Y X is given by → ′ → ′ f(x) = (T ×g)(x)= g(Tx).

71 Theorem 4.14. T × = T . k k k k Proof. We know T × is linear and f = T ×g. Then

T ×g = f g T T × T . k k k k ≤ k k k k ⇒ k k ≤ k k Now we want to show T × T . For any x X,x = 0 g Y s.t. g = 1 and k k ≥ k k 0 ∈ 0 6 ∃ 0 ∈ ′ k 0k g (Tx )= Tx . 0 0 k 0k Hence

× g0(Tx0) = (T g0)(x0) × f0 = T g0

⇒ Tx = g(Tx ) k 0k 0 = f0(x0) f x ≤ k 0k k 0k = T ×g x k 0k k 0k T × g x . ≤ k k k 0k k 0k Recall g = 1, then k 0k Tx Tx T × x T × k 0k. k 0k ≤ k k k 0k ⇒ k k≥ x k 0k But x = 0 is arbitrary, and taking sup on the RHS yields 0 6 T × T . k k ≥ k k

n n n Example 4.15. Let T : R R . Let E be a basis, and let x,y R , and let TE be the → ∈ representation of T with respect to E.

E = e ,...,en { 1 } x = (ξ1,...,ξn)

y = (η1,...,ηn)

TE = (τik)

y = TEx n ηi = τikξk. Xk=1 72 n n Let F = f1,...,fn be the dual basis of E, i.e., a basis for R ′. Then for g R ′, we can { } ∈ represent

n g = αifi i=1 X n fi(y) = fi ηkek = ηi i ! X=1 g(y) = g (TEx) ⇒ n = αiηi i=1 Xn n = αiτikξk i=1 k=1 Xn X n g (TEx) = βkξk where βk = τikαi ⇒ k i X=1 X=1 f(x) = g (TEx) ⇒ n = βkξk k=1 X × f = (TE) g ⇒ n βk = τikαi. i X=1 × Therefor, if T is represented by TE, then T is represented by the transpose of TE. Remark 4.16 (Properties of Adjoint).

1. (S + T )× = S× + T ×

2. (αT )× = αT ×

3. (ST )× = T ×S×

4. If T B(X,Y ) and T −1 exists and T −1 B(Y, X), then (T ×)−1 also exists, (T ×)−1 ∈ × −1 −1 × ∈ ∈ B(X′,Y ′) and (T ) = (T ) . Remark 4.17 (Reltaion between T × and T ). Let T : X Y, X = H ,Y = H , T : H ∗ → 1 2 1 → H , T × : H H ,f H g H 2 2′ → 1′ ∈ 1′ ∈ 2′ T ×g = f g(Tx) = f(x)

73 where H1,H2 are Hilbert spaces. Then let us represnet (Riesz)

f(x) = x,x , x H h 0i 0 ∈ 1 g(y) = y,y , y H . h 0i 0 ∈ 2 Then we can deﬁne A : H H by A f = x and A : H H by A g = y . Then 1 1′ → 1 1 0 2 2′ → 2 2 0 A , A are bijective, isometric, and conjugate-linear. So we can deﬁne T : H H by 1 2 ∗ 2 → 1 × −1 T ∗y0 = A1T A2 y0 = x0.

⇒ Tx,y = g(Tx) h 0i = f(x) = x,x h 0i = x, T ∗y h 0i Remark 4.18.

× × 1. (αT ) = αT but (αT )∗ = αT ∗

2. In the ﬁnite dimensional setting T ∗ is represented as the transpose and T ∗ is represented by the conjugate transpose:

T × = T T T T ∗ = T .

Problem 29.

1. Show (αT )× = αT ×.

2. Show (T n)× = (T ×)n.

Solution.

1. ((αT )xg)(x)= g((αT )x)= g(αT x)= αg(Tx)= α(txg)(x) = ((αT x)g)(x).

2. We have (ST )x = T xSx because ((ST )xg)(x)= g((ST )(x)) = g(S(T (x))) = (sxg)(Tx)= x x x x x x n x n 1 x x T ((S g)(x)) = (T (S g))(x) = ((T S )g)(x). It follows that (T ) = (T − ) T = = (tx)n. · · ·

74 4.3 Reflexive Spaces Definition (Reflexive (Algebraic)). A vector space X is algebraically reflexive if the canonical map C : X X is surjective (and thus bijective). Recall C is the mapping → ∗∗

x gx where gx(f)= f(x), f X∗. 7→ ∀ ∈

Definition (Reflexive (Dual)). A normed space X is called reflexive if X = X′′, i.e., if (C)= X ′. R ′ Problem 30. Let X be a n.s.. Define gx : X′ K by fixing an x X and setting → ∈

gx(f)= f(x) f X′. ∀ ∈

1. Show ﬁxed x X, gx is a b.l.f. on X and ∀ ∈ ′

gx = x . k k k k

2. C is an isomorphism X onto (C). −→ R 3. If a n.s. X is reﬂexive (i.e., (C)= X ) then X is complete. R ′′ Solution.

1. Let gx(f) = f(x), where x X is ﬁxed. Then gx(f) = f(x) f x and ∈ ′ | | | | ≤ k kk k therefore g is bounded and g x . Let fˆ X such that fˆ = 1 and fˆ(x)= x . k k ≤ k k ∈ k k k k gx(fˆ) (We established the existence of such fˆ.) Then g | | = x . It follows that x fˆ k k≥ k k k k gx = x . k k k k ′′ ′ 2. Consider C : x X , where x gx. Then C is linear, because f X we have → → ∀ ∈

gαx+βy(f)= f(αx + βy)= αf(x)+ βf(y)= αgx(f)+ βgy(f).

Then C(αx + βy) = αgx + βgy = αC(x)+ βC(y). Also, C(x) = gx = x , and k k k k k k then gx gy = gx y = x y and C is isometric. Moreover, if x = y, then k − k k − k k − k 6 gx = gy and therefore C is injective. It follows that C is an isomorphism onto its 6 range.

′′ ′ ′′ 3. X is complete being the dual of X . By assumption X is reﬂexive, hence R(C)= X . Part (ii) implies the completeness of X via isomorphism.

Theorem 4.19. Every f.d.n.s. is reﬂexive. Example 4.20.

75 1. ℓp, p[a, b], 1

Proof. The canonical map C : H H given by x gx is injective. We want to show C → ′′ 7→ is surjective, i.e., g H′′ x H s.t. g = Cx. ∀ ∈ ∃ ∈ Deﬁne A : H H by by Af = z where z is the Riesz representation of f: ′ → f(x)= x, z . h i

Then A is bijective, an isometry and conjugate-linear. We view H′ as the Hilbert space with inner product f ,f = Af , Af . h 1 2i1 h 2 1i Let g H be arbitrary. Then ∈ ′′ g(f)= f,f = Af , Af . h 0i1 h 0 i Writing f(x)= x, z , z = Af, Af = z we have h i 0 Af , Af = x, z = f(x). h 0 i h i g(f)= f(x) g = Cx. Then H is reﬂexive. ⇒ ⇒ Lemma 4.22 (Existence of a Functional). Let Y be a n.s. and let Y X be a proper, ⊂ closed subspace. Let x X Y be arbitrary. Calculate 0 ∈ −

δ = inf yˆ x0 . yˆ Y k − k ∈ Note Y closed δ > 0. Then fˆ X s.t. ⇒ ∃ ∈ ′ fˆ = 1 and fˆ(y) = 0 y Y and fˆ(x )= δ. (10) ∀ ∈ 0

Proof. Consider a subspace Z X spanned by Y and x and deﬁne a sublinear functional ⊂ 0 f on Z by f(z)= f(y + αx0)= αδ. Note δ = 0 f = 0. Then f satisﬁes (10). Now we use the Hanh-Banach theorem to 6 ⇒ 6 extend f to X. In a slight abuse of notation we refer to the extension of f as f. Then f is bounded. It is clear that α = 0 f(z) = 0. ⇒

Now to show f 1. (yn) Y s.t. yn x δ. Let zn = yn x , then f(zn)= δ. k k≥ ∃ ⊂ k − 0k→ − 0 − Then f(z) f = sup | | k k z∈Z z z6=0 k k f(zn) | | ≥ zn k k δ = zn k k 1, → f = 1. ⇒ k k Theorem 4.23 (Separability). X separable X separable. ′ ⇒ Proof. Assume X′ is separable. Then

U ′ = f f = 1 X′ { | k k }⊂ contains a countable, dense subset, say (fn) U . Since fn U n, ⊂ ′ ∈ ′ ∀

fn = sup f(x) = 1. k k x =1 | | k k

By the deﬁnition of sup we can ﬁnd a sequence (xn) X s.t. xn = 1 n s.t. ∈ k k ∀ 1 fn(xn) . | |≥ 2

Let Y = span ((xn)), then Y is separable. We want to show Y = X (because then (xn) is a countable dense subset of X).

77 ˆ ˆ ˆ Suppose Y ( X. Then f X′ s.t. f = 1, f(y) = 0 y Y . Since (xn) Y we ∃ ∈ ∀ ∈ ⊂ ˆ have f(xn) = 0 n. Then ∀ 1 fn(xn) 2 ≤ | |

= fn(xn)= fˆ(xn)

= (fn fˆ)(xn) −

fn fˆ xn ≤ − k k

= fn fˆ . −

1 Then fn fˆ and fˆ = 1 contradicts that (fn) U is dense. − ≥ 2 ⊂ ′

Corollary 4.24. X seprable and X not separable X not reﬂexive. ′ ⇒ Proof. X reﬂexive X = X X separable X separable, which contradicts our as- ⇒ ′′ ⇒ ′′ ⇒ ′ sumptions.

1 1 Example 4.25. ℓ is not reﬂexive. This is because (ℓ )′ = ℓ∞ and ℓ∞ is not separable.

4.4 Baire Category Theorem Deﬁnition (Category). Let X be a metric space and M X. ⊂ 1. M is said to be nowhere dense in X if M has no interior points.

2. M is said to be of ﬁrst category or meager if X is the countable union of nowhere dense sets.

3. M is of the second category if it is not of ﬁrst category.

Theorem 4.26 (Baire’s Theorem). If X = then X is of the second category (i.e., X is 6 ∅ not meager).

Proof. Suppose X = and X is meager. Then 6 ∅ ∞ X Mk k [=1

with Mk nowhere dense in X k. By assumption, M1 does not contain a nonempty open c ∀ c set M1 = X, M1 = X M1 is not empty and is open. Pick p1 M1 and an open ⇒ 6 ⇒ − c ∈ ball containing p , say B = B(p , ε ) M , with ε 1 . 1 1 1 1 ⊂ 1 1 ≤ 2

78 ε1 c Similarly, M2 does not contain a nonempty open set B(p1, ) ( M2. Then M2 ⇒ 2 ∩ B(p , ε1 ) = . Then 1 2 6 ∅ c ε ε B = B(p , ε ) M B(p , 1 ) with ε < 1 . ∃ 2 2 2 ⊂ 2 ∩ 1 2 2 2 1 Continuing inductively, we can ﬁnd Bk = B(pk, εk) with εk < 2k s.t.

Bk Mk = ∩ 6 ∅ εk Bk B(pk, ) Bk. +1 ⊂ 2 ⊂

1 Since ε < k we ﬁnd (pn) is Cauchy. Then because X is complete p X s.t. pn p. 2 ∃ ∈ → Also n>m ∀

d(pm,p) d(pm,pn)+ d(pn,p) ≤ εm < + d(pn,p) 2 εm → 2 c and therefore p Bm m. Since Bm Mm , we have p Mm m. Therefore p Mm = ∈ ∀ ⊂ 6∈ ∀ 6∈ ∪ X, a contradiction.

Theorem 4.27 (Uniform Boundedness). Let (Tn) B(X,Y ) with X a Banach space and ⊂ Y a normed space. Suppose x X cx s.t. ∀ ∈ ∃

Tnx cx n. k k≤ ∀

Then ( Tn ) is bounded. k k Proof. For each k deﬁne Ak = x X Tnx k n . { ∈ | k k≤ ∀ } Then we can see Ak is closed:

1. Let x Ak. Then (xi) Ak s.t. xi x. ∈ ∃ ⊂ → 2. For ﬁxed n, Tnxi k Tnx k by continuity of . k k≤ ⇒ k k≤ k·k 3. Then x Ak. ∈ By assumption x X k s.t. x Ak, so ∀ ∈ ∃ ∈ ∞ X = Ak. k[=1

79 Then since X is complete we invoke Baire to obtain Ak0 , a set which contains an open ball, say B = B(x , v) Ak . 0 0 ⊂ 0 v Let x X s.t. x = 0 be arbitrary. Set z = x0 + γx,γ = 2 x . Then ∈ 6 k k z x < r z B k − 0k ⇒ ∈ 0 Tnz k n ⇒ k k≤ ∀

Also, x B Tnx k . Then 0 ∈ 0 ⇒ k 0k≤ 0 −1 Tnx = γ Tn(z x ) k k k − 0 k −1 γ ( Tnz + Tnx ) ≤ k k k 0k 4 x k . ≤ v k k 0 We conclude 4 Tn = sup Tnx k0 = c, k k x =1 k k≤ v k k and c is independent of x.

Problem 31. Let X be the normed space of all polynomials with norm

k x = max αi , where x(t)= αkt + + α0. k k i | | · · · Show X is not complete.

Solution. Let X be the normed space of all polynomials with norm x = maxi αi (the k k | | αi’s are the coefficients). Define the linear functional fn by fn(x)= α0 + α1 + ... + αn 1. − Then fn(x) n x . Also for each fixed x we have fn(x) cx. On the other hand, for | | ≤ 2 k k n | | ≤ fn(x) x(t)=1+ t + t + ... + t , we have x = 1 and fn(x)= n x . Hence fn | x | = n. k k k k k k≥ k k It follows that the sequence ( fn ) is unbounded. The Uniform Boundedness Theorem k k implies that X is not complete.

Problem 32. If X,Y are Banach and (Tn) B(X,Y ) is a sequence, show TFAE: ⊂ 1. ( Tn ) is bounded. k k 2. ( Tnx ) is bounded x X. k k ∀ ∈ 3. ( g(Tnx) is bounded x X g Y . | | ∀ ∈ ∀ ∈ ′ Solution. Recall that in a Banach space X if a sequence (xn) is such that (f(xn)) is bounded ′ for all f X , then ( xn ) is bounded and therefore 3 implies 2. Also, 2 implies 1 by the ∈ k k Uniform Boundedness Theorem. Finally, 1 implies 3 since g(Tnx) g Tn x . | | ≤ k kk kk k

80 4.5 Strong and Weak Convergence

Deﬁnition (Strong and Weak Convergence). Let X be a normed space and let (xn) X ⊂ be a sequence.

1. (xn) is strongly convergent if x X s.t. ∃ ∈

lim xn x = 0. n →∞ k − k We denote this property with

lim xn = x or xn x. n →∞ →

2. (xn) is weakly convergent if x X s.t. f X ∃ ∈ ∀ ∈ ′

lim f(xn)= f(x). n →∞ We denote this property with w xn x. −→ w Lemma 4.28 (Weak Convergence). Let xn x. Then −→ 1. The the weak limit x is unique. w 2. xn x subsequences (xn ) (xn). k −→ ∀ k ⊂ 3. ( xn ) is bounded. k k Proof. w w 1. Suppose xn x and xn y. Then f(xn) f(x) and f(xn) f(y) f(x) = −→ −→ → → ⇒ f(y). Then 0= f(x) f(y)= f(x y) f X . Then by a previous lemma x = y. − − ∀ ∈ ′ 2. (f(xn)) R all subsequences of (f(xn)) converge and have the same limit. ⊂ ⇒ 3. (f(xn)) converges f(xn) cf n. Deﬁne gn X by ⇒| |≤ ∀ ∈ ′′

gn(f)= f(xn).

Then gn(f) = f(xn) cf , ( gn(f) ) is bounded f X . Then X complete | | | | ≤ ⇒ | | ∀ ∈ ′ ′ ( gn ) bounded by the UBT. Now gn = x and the proof is complete. ⇒ k k k k k k

Theorem 4.29 (Strong and Weak Convergence). Let X be a normed space and (xn) X. ⊂ w 1. xn x xn x. The limits are the same. → ⇒ −→

81 w 2. xn x xn x in general. −→ 6⇒ → 3. dim X < ∞⇒ w xn x xn x. −→ ⇒ → Proof. w 1. f(xn) f(x) = f(xn x) f xn x . Therefore xn x xn x. k − k k − k ≤ k k k − k → ⇒ −→ 2. We show a counterexample. Let H be a Hilbert space and (en) H and orthonormal ⊂ sequence. Then f H ,f(x)= x, z . In particular, f(en)= en, z and by Bessel’s ∀ ∈ ′ h i h i inequality ∞ 2 2 en, z z . |h i| ≤ k k n=1 X w Therefore f(en) = en, z 0 f H en 0. But (en) does not converge h i → ∀ ∈ ′ ⇒ −→ strongly because for n = m 6 2 em en = em en, em en = 2. k − k h − − i w 3. Suppose dim X = k < and xn x. Let e ,...,ek be a basis for X. Then we ∞ −→ { 1 } can represent

k (n) xn = αi ei i X=1 k x = αiei. i X=1

By assumption f(xn) f(x) f X . Take the dual basis f ,...,fk deﬁned by → ∀ ∈ ′ { 1 }

fk(ei)= δki.

Then

(n) fi(xn) = αi fi(x) = αi.

Hence (n) fi(xn) fi(x) α αi. → ⇒ i →

82 Then

k (n) xn x = (α αi)ei k − k i − i X=1 k (n) α αi ei ≤ i − k k i X=1 0 as n . → →∞

Remark 4.30.

1. In ℓ1 strong and weak convergence are equivalent.

2. In a Hilbert space H, w xn x iﬀ xn, z x, z z H. −→ h i → h i ∀ ∈

Lemma 4.31 (Weak Convergence). Let X be a normed space and let (xn) X. Then w ⊂ xn x iﬀ both: −→ 1. ( xn ) is bounded. k k 2. f(xn) f(x) f M X M total . → ∀ ∈ ⊂ ′ ∀ Proof. 1. “ ”. Weak convergence implies 1,2 by previous results. ⇒ 2. “ ”. Suppose 1,2 hold. Let f X be arbitrary and show f(xn) f(x). By 1 we ⇐ ∈ ′ → can ﬁnd c> 0 s.t.

xn c k k ≤ x c. k k ≤

Since M X is total, f X (fn) span (M) s.t. fn f. Hence ε > ⊂ ′ ∀ ∈ ′ ∃ ⊂ → ∀ 0 i s.t. ∃ ε fi f < . k − k 3c

Moreover, since fi span (M) (by2) N s.t. ∈ ∃ ε fi(xn) fi(x) < , n>N. | − | 3 ∀

83 Then

f(xn) f(x) f(xn) fi(xn) + fi(xn) fi(x) + fi(x) f(x) | − | ≤ | − | ε | − | | − | < f fi xn + + fi f x k − k k k 3 k − k k k ε ε ε < c + + c 3c 3 3c = ε.

w Therefore xn x. −→

Deﬁnition (Weak Cauchy, Weak Complete).

1. In a normed space X, a sequence (xn) X is said to be weak Cauchy if f X ⊂ ∀ ∈ ′ the sequence (f(xn)) is Cauchy.

2. A normed space X is said to be weakly complete if every weak Cauchy sequence converges weakly in X.

Problem 33. Show X reﬂexive X weakly complete. ⇒ Solution. Let (xn) be any weak Cauchy sequence in X. Then (f(xn)) converges for every ′ ′′ f X . For xn X there is a gx X such that f(xn) = gx (f). Hence (gx (f)) ∈ ∈ n ∈ n n converges, say, gx (f) g(f). Weak Cauchyness of (xn) implies the boundedness of (xn) n → and then since gxn = xn we have that g is bounded. Also, g is linear and therefore ′′ k k k k g X . Since X is reﬂexive, there is an x such that g(f) = f(x). Hence f(xn) f(x). ∈ ′ → Since f X was arbitrary, this shows that (xn) converges to x weakly. Since (xn) was ∈ any weak Cauchy sequence, X is weakly complete.

Deﬁnition (Convergence of Operators). Let X,Y be normed spaces, and let (Tn) ⊂ B(X,Y ) be a sequence. We say (Tn) is:

1. uniformly operator convergent if (Tn) converges in the norm of B(X,Y ).

2. strongly operator convergent if x X the sequence (Tnx) converges strongly ∀ ∈ in Y .

3. weakly operator convergent if x X the sequence (Tnx) converges weakly in ∀ ∈ Y .

Remark 4.32. Uniform strong weak: ⇒ ⇒ 1. Tn T 0 (Tn T )x Tn T x 0. k − k→ ⇒ k − k ≤ k − k k k→ 2. (Tn T )x 0 f(Tn T )x f (Tn T )x 0 k − k→ ⇒ k − k ≤ k k k − k→

84 2 Example 4.33 (Strong Uniform). Let X = Y = ℓ . Let Tn be “zero-overwrite” 26⇒ operator, i.e., let (xi) ℓ ∈

Tn(xi) = (0,..., 0,xn+1,xn+2,... ).

2 Then Tn(xi) 0 (xi) ℓ so (Tn) is strongly operator convergent to the zero operator. → ∀ ∈ 2 However given any n, we can construct x ℓ s.t. Tnx = x, i.e., by making the ﬁrst n ′ ∈ terms of x′ zero. Then Tnx Tn = sup k k 1. k k x=0 x ≥ 6 k k So (Tn) does not converge to 0 B(X,Y ). ∈ 2 Example 4.34 (Weak Strong). Let X = Y = ℓ . Let Tn be “zero-shift” operator, i.e., 2 6⇒ let (xi) ℓ ∈ Tn(xi) = (0,..., 0,x1,x2,... ), n zeros . 2 2 Let (zi) ℓ be another element. Deﬁne f ℓ ′ by ∈ ∈ ∞ f(x)= x, z = xizi. h i i X=1 Then

f(Tnx) = Tnx, z h i ∞ = xkzk+n. k X=1 Compute

2 2 f(Tnx) = Tnx, z | | |h i| ∞ 2 ∞ 2 xk zm ≤ | | | | m n Xk=1 X= +1 ∞ 2 zm 0. | | → m=n+1 X Then f(Tnx) 0= f(0x), i.e. (Tn) converges weakly to zero. But consider the sequence → x = (1, 0, 0,... ). Then Tnx Tmx = √2, n = m . k − k 6 So (Tn) does not converge strongly to zero. Deﬁnition (Strong, Weak and Weak* Convergence of Functionals). Let X be a normed space and let (fn) X be a sequence. ∈ ′

85 1. (fn) is strongly convergent to f X if fn f 0 and we write ∈ ′ k − k→

fn f. →

2. (fn) is weakly convergent to f X if ∈ ′

g(fn) g(f) g X′′. → ∀ ∈

3. (fn) is weak* convergent to f X if ∈ ′

fn(x) f(x) x X. → ∀ ∈ We write w∗ fn f. −→ Remark 4.35.

1. Weak weak*. ⇒ 2. Limit operators. Let (Tn) B(X,Y ) be a sequence. ∈ (a) If Tn T (uniform) then T B(X,Y ). → ∈ (b) If convergence is strong or weak, it is possible that the limit T is unbounded (not continuous) if X is not complete. This is the why we use Banach spaces.

Example 4.36. Let X be the subspace of ℓ2 of sequences with ﬁnitely many nonzero terms. Then X is not complete. Deﬁne Tn via

Tn(xi) = (x1, 2x2,...,nxn,xn+1,xn+2,... ).

Then (Tn) B(X, X) converges strong to the unbounded operator T ⊂

T (xi) = (ixi).

Lemma 4.37 (Strong Operator Convergence). Let X be a Banach space, Y a normed space, and (Tn) B(X,Y ) a sequence. (Tn) strongly operator convergent T B(X,Y ). ⊂ ⇒ ∈ Proof. Note

1. T is linear.

2. Tnx Tx x X Tnx is bounded x X Tn c, for some c R by the → ∀ ∈ ⇒ ∀ ∈ ⇒k k≤ ∈ uniform boundedness theorem.

86 Now,

Tnx Tn x k k ≤ k k k k c x . ≤ k k So Tnx c x . Taking lim on the LHS yields k k≤ k k Tx c x , k k≤ k k therefore T B(X,Y ). ∈ Theorem 4.38 (Strong Operator Convergence). Let X,Y be Banach spaces, and let (Tn) ⊂ B(X,Y ) be a sequence. (Tn) is strongly operator convergent iﬀ

1. ( Tn ) is bounded. k k 2. (Tnx) is Cauchy x M M total in X. ∀ ∈ ∀ Proof.

1. . If Tnx Tx x X then 1 follows by the UBT and 2 follows trivially. ⇒ → ∀ ∈ 2. . Suppose Tn c n. Let x X be arbitrary and show (Tnx) converges strongly ⇐ k k≤ ∀ ∈ in Y . Let ε> 0 be given. Since X = span (M), then y span (M) s.t. ∃ ∈ ε x y < . k − k 3c Also, (Tny) Cauchy N s.t. ⇒ ∃ ε Tny Tmy < . k − k 3 Then

Tnx Tmx Tnx Tny + Tny Tmy + Tmy Tmx k − k ≤ k ε −ε kε k − k k − k < c + + c 3c 3 3c = ε.

Then (Tn) is Cauchy, and since Y is complete, (Tnx) converges in Y .

Corollary 4.39. Let X be a Banach space and (fn) X a sequence which is weak* ⊂ ′ convergent to f, w∗ fn f. −→ Then f X iﬀ ∈ ′ 1. ( fn ) is bounded. k k 2. (fnx) is Cauchy x M M total in X. ∀ ∈ ∀

87 4.6 The Open Mapping Theorem Deﬁnition (Open Mapping). Let X,Y be metric spaces, T : (T ) Y , (T ) X. D → D ⊂ Then T is said to be an open mapping if B (T ) open T (B) Y open. ⊂ D ⇒ ⊂ Theorem 4.40 (Open mapping). Let X,Y be Banach spaces and T B(X,Y ) s.t. T : onto ∈ X Y . Then T is an open mapping. Hence, if T is injective then T is bijective and so −→ T −1 is continuous and so T −1 B(X,Y ). ∈ Lemma 4.41 (Open Unit Ball). Let X,Y be Banach spaces and T B(X,Y ) s.t. T : onto ∈ X Y and B = B(0, 1) X. Then T (B ) Y is open and 0 T (B ). −→ 0 ⊂ 0 ⊂ ∈ 0 Proof. Let A X. Deﬁne ⊂ 1. αA = x X x = αa, a A { ∈ | ∈ } 2. A + w = x X x = a + w, a A . { ∈ | ∈ } Consider B = B(0, 1 X, and let x X be arbitrary. Then choose k s.t. x kB , e.g., 1 2 ⊂ ∈ ∈ 1 k> 2 x . Then k k ∞ X = kB1. k[=1 Since T is surjective and linear, Y = T (X)

∞ = T kB1 ! k[=1 ∞ = kT (B1) k[=1 ∞ = kT (B1). k[=1 Then by the category theory, Y complete Y not meager. Then at least one of kT (B ) ⇒ 1 contains an open ball T (B ) contains an open ball, say ⇒ 1 B = B(y0, ε) T (B1). ∗ ⊂ Then B y0 = B(0, ε) T (B1) y0. ∗ − ⊂ − With these sets constructed, we want to show T (B ) y T (B ). 1 − 0 ⊂ 0 Let y T (B ) Y . Then y + y T (B ). ∈ 1 − 0 0 ∈ 1

88 Problem 34. Let X,Y be Banach spaces and T B(X,Y ) injective. Show that T −1 : ∈ (T ) X is bounded iﬀ (T ) is closed in Y . R → R Solution. 1. If R(T ) is closed in Y , it is complete, and boundedness follows from the Open Mapping Theorem.

1 2. Assume T − to be bounded, y R(T ) Y , (yn) in R(T ) such that yn y, and 1 1 ∈ ⊂ → xn = T yn. Since T is continuous and X is complete, (xn) converges, say, xn x. − − → Since T is continuous, yn = Txn Tx. Hence y = Tx R(T ), so that R(T ) is closed → ∈ because y R(T ) was arbitrary. ∈

4.7 Closed Graph Theorem Deﬁnition (Closed Linear Operator). Let X,Y be normed spaces and T : (T ) Y a D → linear operator with (T ) X. T is said to be closed if the graph of T , denoted ΓT D ⊂ ΓT = (x,y) x (T ) ,y = Tx { | ∈ D } is closed in X Y . × Theorem 4.42 (Closed Graph Theorem). Let X,Y be Banach spaces and let T : (T ) D → Y be a closed linear operator. If (T ) is closed in X then T is bounded. D Proof. Note that X Y is normed space with norm × (x,y) = x + y (11) k k k k k k First we show X Y is complete with norm (11). Let zn = (xn,yn) and (zn) X Y × ∈ × Cauchy. Let ε> 0 be given. N s.t. ∃ zn zm = xn xm + yn ym < ε n,m>N. k − k k − k k − k ∀ Then (xn), (yn) are Cauchy in X,Y respectively. Since X,Y are Banach, these sequences converge, say xn x X,yn y Y. → ∈ → ∈ Then setting z = (x,y), zn z X Y. → ∈ × Since (zn) was arbitrary, X Y is complete. × Now, by assumption ΓT is closed in X Y and (T ) is closed in X. Therefor ΓT, (T ) × D D are complete. Consider P :ΓT (T ) given by the map → D (x,Tx) x. 7→ Then

89 1. P is linear. Obvious.

2. P is bounded.

P (x,Tx) = x k k k k x + Tx ≤ k k k k = (x,Tx) . k k 3. P is bijective with inverse P −1 : (T ) ΓT given by the map D → x (x,Tx). 7→ Since ΓT and (T ) are complete, P −1 is bounded, say D (x,Tx) b x . k k≤ k k Then

Tx Tx + x k k ≤ k k k k = (x,Tx) k k b x , ≤ k k x (T ). ∀ ∈ D Then T is bounded, as required.

Theorem 4.43 (Closed Linear Operator). Let X,Y be normed spaces, T : (T ) Y a D → linear operator, and (T ) X. Then T is closed iﬀ D ⊂ 1. Given (xn) (T ). ⊂ D 2. If xn x and Txn y then → → x (T ) and Tx = y. ∈ D

Proof. z ΓT iﬀ sequence (zn) ΓT ∈ ∃ ⊂ zn = (xn,Txn)

s.t. zn z. → Hence, xn x and Txn y. Then z = (x,y) ΓT iﬀ → → ∈ x (T ) and y = T x. ∈ D

90 Example 4.44 (Diﬀerential Operator). Let X = C[0, 1] and let T : (T ) X be given D → by the map x x′. 7→ Note (T ) is the space of continuously diﬀerentiable functions. Then D 1. T is bounded. Obvious.

2. T is closed. Suppose xn x and Txn = x y. Then → n′ → 1 1 y(τ)dτ = lim x′ (τ)dτ n n 0 0 →∞ Z Z 1 = lim x′ (τ)dτ n n →∞ Z0 = x(t) x(0). −

⇒ 1 x(t)= x(0) + y(τ)dτ. Z0 ⇒ x (T ) ∈ D x′ = y

Remark 4.45. Boundedness does not imply closedness. To see this, let T : (T ) D → (T ) X be the identity operator where (T ) is proper, dense subset of X. Then T is D ⊂ D linear and bounded. However T is not closed. Take x X (T ) and let (xn) (T ) be s.t. ∈ − D ⊂ D

xn x. → Then Txn = xn x (T ) . → 6∈ D Lemma 4.46 (Closed Operator). Let X,Y be normed spaces, let T B( (T ) ,Y ) with ∈ D (T ) X. D ⊂ 1. If (T ) is a closed subset of X then T is close. D 2. If T is closed and Y is complete then (T ) is a closed subset of X. D Proof.

1. If (xn) (T ) and xn x and (Txn) converges then ⊂ D →

91 (a) x (T )= (T ) since (T ) is closed. ∈ D D D (b) Txn Tx since T is closed. → T is closed. ⇒ 2. For x (T ) (xn) (T ) s.t. ∈ D ∃ ⊂ D

xn x. → Since T is bounded, Txn Tx T xn xm . k − k ≤ k k k − k Therefore (Txn) is Cauchy, and since Y is complete

Txn y Y. → ∈ Since T is closed we have x (T ) and Tx = y. Hence (T ) is closed because ∈ D D y (T ) was arbitrary. ∈ D

Problem 35. Let X and Y be normed spaces and let T : X Y be a closed, linear operator. → Show the following.

1. The image B of a compact subset C X is closed in Y . ⊂ 2. The inverse image A of compact subset K Y is closed in X. ⊂ Solution. 1. Consider any a A¯. Let an a, where an A. Let cn C be such that ∈ → ∈ ∈ an = T cn. Since C is compact, (cn) has a subsequence (cnk ) which converges, say, cn c C. Also T cn a, and T c = a A because T is closed by assumption. k → ∈ k → ∈ 2. Consider any b B¯. Let bn b, where bn B. Let kn = T bn. Since K is compact, ∈ → ∈ (kn) has a subsequence (kn ) which converges, say, kn k K. Also bn b, and i i → ∈ i → T b = k K = T (B) by the closedness of T , so that b B and B is closed. ∈ ∈

92 5 Exam 1

Problem 1 (4 Points). Let M l be the subspace consisting of all sequences x = (ξi) ⊂ ∞ with at most ﬁnitely many nonzero terms. Is M complete?

Solution. Let M l , and x = (1, 1/2, 1/3, ....) = (ξi). Consider the sequence (xn), where ⊂ ∞ xn = (1, 1/2, ...., 1/n, 0, ...). Then xn M and d(xn,x) = 1/(n + 1), i.e., we have that ∈ xn x, but x M. It follows that M is not closed. → 6∈ Problem 2 (4 Points). Does

b d(x,y)= x(t) y(t) dt | − | Za deﬁne a metric or pseudometric on X if X is

1. the set of all real-valued continuous functions on [a, b]

2. the set of all real-valued Riemann integrable functions on [a, b]?

Solution. d is a metric on C[a, b], because if the integral is 0, then x(t) y(t) = 0 for all | − | t [a, b]. On the other hand d is a pseudo-metric on R[a, b], for example if x(a) =1 and ∈ x(t) = 0 everywhere else on [a, b], then the integral of x(t) is 0. | | Problem 3 (4 Points). If X is a compact metric space and M X is closed, show that ⊂ M is compact.

Solution. Let (xn) M X. Since X is compact (xn) has a convergent subsequence, ⊂ ⊂ say (xn ) M, and xn x. M is closed, so we must have x M. It follows that M is k ⊂ → ∈ compact.

Problem 4 (4 Points). Let T : X Y be a linear operator and dimX = dimY = n< . 1 → ∞ Show that R(T )= Y iﬀ T − exists. Solution.

1 1. Suppose that T exists and let e , e , ..., en a basis for X. We show that T e , T e , ..., T en − { 1 2 } { 1 2 } are linearly independent (and therefore R(T )= Y ).

Suppose that α1T e1 + ... + αnT en = T (α1e1 + ... + αnen) = 0. T is bijective by assumption, so we have that α1e1 + ... + αnen = 0, and then α1 = ... = αn = 0. It follows that T e , ..., T en are linearly independent. { 1 } 2. Suppose that R(T ) = Y , and (y1,y2, ..., yn) is a basis for Y . Then there exist x ,x , ..., xn X such that Txi = yi, for i = 1, 2, ..., n. 1 2 ∈

93 We can show that x ,x , ..., xn are linearly independent using the same argument { 1 2 } as above. It follows that x ,x , ..., xn is a basis for X. { 1 2 } Suppose that Tx = 0. Then Tx = T (α1x1 + ... + αnxn)= α1y1 + ... + αnyn = 0, and α1 = ... = αn = 0. It follows that x = 0 and T is injective. Also, T is surjective by 1 assumption. It follows that T − exists.

Problem 5 (4 Points). Show that the operator T : l l deﬁned by ∞ → ∞ ξ y = (η )= Tx, η = i , x(ξ ), i i i i

1 is linear and bounded What can you say about T − ?.

Solution. Let T : l∞ l∞, x = (ξi) l∞, Tx = (ξi/i). Then T is bounded, linear, → ∈ 1 injective, but not surjective, e.g., y = (1, 1, 1, ..) l∞ is not in R(T ). T − is only deﬁned 1 ∈ on R(T ) l . T is not bounded, because for xn = (ξni, where ξni = 1 if i = n and 0 ⊂ ∞ − otherwise we get 1 t xn k − k = n. xn k k

94 6 Exam 2

Problem 1 (4 Points). If x and y are diﬀerent vectors in a ﬁnite dimensional vector space X, show that there is a linear functional f on X such that f(x) = f(y). 6 Solution. Let x,y X, x = y, dimX = n. ∈ 6 1. Suppose that x = αy, for any α. Then x and y are linearly independent and we 6 can construct a basis x = e ,y = e , e , ..., en , and a corresponding dual basis { 1 2 3 } f ,f , ..., fn , where fi(ej)= δij. Then f (x) = 1 = f (y). { 1 2 } 1 6 1 2. If x = αy, α = 1, then we take x = e , e , e , ..., en , and f ,f , ..., fn . Then 6 { 1 2 3 } { 1 2 } f (x) = 1 = f (y)= α. 1 6 1

Problem 2 (4 Points). Let X and Y be normed spaces and Tn : X Y , n = 1, 2, 3, .., be → bounded linear operators. Show that convergence Tn T implies that for every ǫ> 0 there → is an N such that for all n > N and all x in any given closed ball we have Tnx Tx <ǫ. k − k Solution. Let B be a closed ball in X. B is bounded, i.e., if x B, then x < K for some ǫ ∈ k k K. Let N be such that Tn T < for all n > N. Then k − k K Tnx Tx = (Tn T )x Tn T x < ǫ. k − k k − k ≤ k − kk k

Problem 3 (4 Points). Show that y xn and xn x together imply x y. ⊥ → ⊥ Solution. Suppose that y xn and xn x. Then ⊥ → x,y = lim xn,y = 0. n h i →∞ h i

Problem 4 (4 Points). Let M be a total set in an inner product space X. If v,x = w,x h i h i for all x M, show that v = w. ∈ Solution. We have by assumption that v w,x = 0 for all x M. M is total in X, so h − i ∈ v w spanM¯ = X. By the continuity of the inner product we have v w, v w = − ∈ h − − i v w 2 = 0. k − k Problem 5 (4 Points). If S and T are bounded self-adjoint operators on a Hilbert space H and α and β are real, show that L = αS + βT is self-adjoint. Solution. Let L = αS + βT . Then we have

L∗ = (αS + βT )∗ = (αS)∗ + (βT )∗ =αS ¯ ∗ + βT¯ ∗ = αS∗ + βT ∗ = αS + βT = L.

95 7 Final Exam

Problem 1 (4 Points). What can you say about the reﬂexivity of lp, 1 p ? ≤ ≤∞ Solution. lp, For 1

′ Problem 2 (4 Points). Let X be a separable Banach space and M X a bounded ⊂ set. Show that every sequence of elements of M contains a subsequence which is weak* ′ convergent to an element of X .

Solution. Any sequence (fn) in M is bounded, say fn r. Since X is separable, it k k ≤ contains a countable dense subset V , which we can arrange in a sequence (xm). Since

fn(xm) fn xm r xm , | | ≤ k kk k≤ k k

we see that for ﬁxed m the sequence (fn(xm)) is bounded, so that it has a subsequence A1 which converges at x1, and A1 has a subsequence A2 which converges at x2, ...etc.; hence (fn (x)), where fn A ,fn A , ..., is a subsequence which converges at every element k 1 ∈ 1 2 ∈ 2 of V . Since V is dense in X and X is complete the statement follows.

Problem 3 (4 Points). Show that an open mapping need not map closed sets onto closed sets.

Solution. The mapping T : R2 R deﬁned by (x ,x ) (x ) is open, it maps the closed → 1 2 → 1 set (x ,x ) x x = 1 R2 onto the set R 0 which is not closed in R. { 1 2 | 1 2 }⊂ − { } Problem 4 (4 Points). Let X and Y be normed spaces and X compact. If T : X Y is 1 → a bijective closed linear operator, show that T − is bounded.

1 1 Solution. T − is closed, being the inverse of a closed operator. Hence T − : Y X, where 1 → T − is closed and X is compact, is bounded. Problem 5 (4 Points). Let f be an integrable function on the measure space (X, ,µ). B Show that given ǫ > 0, there is a δ > 0 such that for each measurable set E with µE <δ we have f < ǫ. | | ZE

96 Solution. Let f be an integrable function on the measure space (X,B,µ). Let fn(x)= f(x) if f(x) n, fn(x)= n if f(x) n, fn(x)= n if f(x) in. Then f fn 0 a.e. and | |≤ ≥ − ≤ | − |→ f fn f for all n. By the dominated convergence theorem we have | − | ≤ | |

f fn o. | − |→ Z Let ǫ> 0 be given. We can pick N such that ǫ f fn < for all N. | − | 2 Z ǫ Let δ < 2N . If E is measurable with µE <δ, then

f = (fN + (f fN )) fN + f fN δN + f fN ǫ. | | | − |≤ | | | − |≤ | − |≤ ZE ZE ZE ZE Z