Appalachian State University

Department of

Ralph Chikhany

Introduction to

c 2015 A Directed Research Paper in Partial Fulfillment of the Requirements for the Degree of Master of Arts

May 2015

Approved:

Dr William J Cook

Member

Dr William C Bauldry

Member Contents

Introduction 1

1 Metric Spaces 3 1.1 Definition and Examples ...... 3 1.2 Underlying ...... 6 1.3 Convergence and Continuity ...... 9 1.4 Completeness and Completion ...... 11 1.5 Compactness ...... 15

2 Banach Spaces 18 2.1 Normed Vector Spaces ...... 18 2.1.1 Vector Spaces ...... 18 2.1.2 Normed Spaces ...... 19 2.1.3 Bounded Linear Operators ...... 22 2.2 Banach Spaces ...... 24 2.2.1 Introduction to Differential Equations in Banach Spaces ...... 25 2.3 Linear Functionals and Duality ...... 28

3 Hilbert Spaces 31 3.1 Inner Spaces and Orthogonality ...... 31 3.2 Hilbert Spaces ...... 35 3.3 Orthonormal and Bases ...... 38 3.4 Representation Theorems ...... 41 3.5 Special Operators ...... 44 3.6 Weak-Weak∗ and Convergence ...... 48

Bibliography 51 Introduction

Functional Analysis was founded at the beginning of the 20th century to provide a general abstract domain for some problems, namely from physics, where the issue was finding functions verifying certain properties. With this incentive in mind, functional analysis can be perceived as the study of function spaces. To place this field in relation to differential and integral calculus, we can classify the study of functions in three levels, from the most elementary to the abstract:

The first level consists of studying properties of individual functions, namely their domain, extremas, concavity, asymptotes... These properties can thus be represented and perceived graphically and geometrically. This is mainly the concern of beginner calculus courses.

The second level consists of studying more general properties of collections of functions, such as continuity and convergence notions on sequences and of functions. This is mainly the concern of real analysis courses.

The third level consists of studying properties of function spaces. Those are sets of functions of given kind from one set to another. At this level, some abstract properties of collections of functions can be expressed as properties of certain function spaces.

However, one should understand that moving from finite to infinite dimensions isn’t the easiest jump for students, since we are losing the geometric intuition. Functional analysis is one of those fields that studies general concepts that might seem abstract and without practical use , but the results have turned out to be of utmost importance and efficacy for many fields, namely physics and economics, for the past century. To get familiar with the depth of some methods and key concepts, one must jump back and forth between definitions from one section to another, while relating theorems and trying to apply some previous examples to the concepts explained in a current section (for instance, one could go back and verify if the metric spaces discussed in the first chapter are normed, Banach, inner product or Hilbert, even if those are not clearly indicated in an exercise or example). And just like in any mathematics course, studying new structures often comes with the study of transformations and mappings between spaces. We will focus in this course on properties of linear transformations between spaces, namely functionals between normed spaces, which can be viewed as functions of functions.

1 The goal of this directed research project is to provide complete lecture notes with exercises for each section, for anyone willing to teach an introductory course to functional analysis at the senior undergraduate or graduate level. One could copy everything written in the next 50 or so pages verbatim, and the content would be more than enough for a semester’s worth of material, provided that the strategy of lecturing for a couple of sessions then stopping and solving problems with students for a class meeting is adopted. This is essential since courses like this tend to get really boring and frustrating for students if the same routine lecturing method is adopted in every class meeting. With this strategy kept in mind, one could interchange the number of class meetings for lecturing or problem solving. A rough schedule for the number of meetings for each section is provided below:

Section Lecture days Problems days Total 1.1 1.5 1.5 3 1.2 1 1 2 1.3 2 1 3 1.4 2.5 1.5 4 1.5 1 1 2 2.1 2 2 4 2.2 2 1 3 2.3 1 2 3 3.1 2 1 3 3.2 1.5 1.5 3 3.3 2 1 3 3.4 2 1 3 3.5 1 2 3 3.6 2 1 3

Not all exercises should be done by the professor. Most of them should be turned in for credit, but problem sessions should be aimed at helping students get started with the tricky proofs or problems. If this class is dual listed (audience is a mix of senior undergraduates and graduate students), some problems in each section (starred) should be additionally assigned to graduate students for their portion of the grade. A midterm and a final are necessary, and preferably given as a take home since a lot of definitions, theorems and proofs are being covered, and 50 minute exams are not enough to test student on all concepts. With a total of 14 sections, a suggested grade distribution could be the following: 65% for homework (5% each, drop the lowest), 15 % for the midterm and 20% for the final.

If anyone decides to teach or take this course as an independent study at Appstate, or anywhere else, and decides to use my notes and exercises, I hope to get constructive feedback and criticism, since I was learning the material while working on the next 50 pages. Happy learning/teaching!

2 Chapter 1

Metric Spaces

1.1 Definition and Examples

Definition. A metric (M, d) is an ordered pair consisting of a set M and a distance + function d : M × M → R such that for all x, y, z ∈ M: (a) d(x, y) = 0 ⇐⇒ x = y, (b) d(x, y) = d(y, x), (c) d(x, z) ≤ d(x, y) + d(y, z) (triangle inequality).

Definition. A subspace (M 0, d0) of (M, d) is a on M 0 ⊆ M with the metric d 0 0 0 0 restricted to M × M . The map d = d|M 0×M 0 is called the metric induced on M by d.

Example. The real line equipped with the usual metric d(x, y) = |x − y| is a metric space. In n general, the set F where F = Z, Q, R or C equipped with the metric v u n uX 2 d(~x,~y) = t |ξi − ηi| i=1 where ~x = ξ1, ξ2, ...ξn and ~y = η1, η2, ...ηn. This is also known as the Euclidean metric. When 2 n = 2 and F = R, we obtain the regular distance function in R pictured next.

2 Example. Sets can be equipped with multiple distinct metrics. Another metric on R is d = |ξ1 − η1| + |ξ2 − η2| and is known as the taxicab metric, since its the distance one would travel by taxi on a rectangular grid of streets. It is also pictured below.

Proof. We will prove that taxicab metric is indeed a metric. We need to verify the three conditions listed in the definition. Let ~x = (x1, x2), ~y = (y1, y2) and ~z = (z1, z2).

(a) d(x, y) = 0 ⇐⇒ |x1 − y1| + |x2 − y2| = 0 ⇐⇒ |x1 − y1| = 0 and |x2 − y2| = 0 ⇐⇒ x = y

3 2 Figure 1.1: Euclidean and Taxicab Metrics in R

(b) d(x, y) = |x1−y1|+|x2−y2| = |y1−x1|+|y2−x2| = d(y, x)

(c) d(x, y) + d(y, z) = |x1 − y1| + |x2 − y2| + |y1 − z1| + |y2 − z2|

= (|x1 − y1| + |y1 − z1|) + (|x2 − y2| + |y2 − z2|)

≥ (|x1 − y1 + y1 − z1|) + (|x2 − y2 + y2 − z2|)

= (|x1 − z1|) + (|x2 − z2|) = d(x, z)

( 0 x = y Example. Define d(x, y) = . Then (M, d) for any set M is called a discrete 1 x 6= y metric space equipped with the discrete metric.

Example. Consider the C[a, b] = {f :[a, b] → C | f continuous}, i.e. the set of all continuous functions on a given closed interval, and define

 1 Z b  p  p  |f(x) − g(x)| dx 1 ≤ p < ∞  a dp(f, g) =    max |f(x) − g(x)| p = ∞ a≤x≤b

Then (C[a, b], dp) is a metric space.

∞ p X p Example. We define the space ` of sequences ~x = (x1, x2, ...) such that |xi| converges for i=1 1 ∞ ! p X p a fixed p ≥ 1, and the metric d(x, y) = |xi − yi| . Then (`p, d) is a metric space. When i=1 p = 2 we obtain the Hilbert space, the earliest example of Hilbert spaces studied later in this course.

4 Proof. Clearly, d satisfies the first two conditions of being a metric. Proving that d satisfies the triangle inequality is not trivial. One needs to recall a couple of results. First, define q to be the conjugate exponent of p, meaning 1 1 + = 1 or (p − 1)(q − 1) = 1 p q This form helps in deriving the first of these necessary inequalities.

∞  ∞ 1/p  ∞ 1/q X X p X q • Holder’s inequality for sequence spaces: |xk yk| ≤ |xk| |yk| k=1 k=1 k=1 Note that if p = 2 then q = 2 and this yields the Cauchy-Schwarz inequality.

∞ !1/p ∞ !1/p ∞ !1/p X p X p X p • Minkowski’s inequality for sums: |xk + yk| ≤ |xk| + |yk| k=1 k=1 k=1 This last inequality helps us understand why the metric d converges (provided that we are in `p spaces). Thus:

1 ∞ ! p X p d(x, y) = |xi − yi| i=1 d(x,z) d(z,y)

1 z }| 1{ z }| 1{ ∞ ! p ∞ ! p ∞ ! p X p X p X p ≤ (|xi − zi| + |zi − yi|) = |xi − zi| − |zi − yi| i=1 i=1 i=1

Exercises

1. Check if the following functions define metrics on R:

(a) d(x, y) = x − y (b) d(x, y) = (x − y)2 (c) d(x, y) = p|x − y|

2. Show that the discrete metric is indeed a metric.

3. If (M, d) is a metric space, then prove that (M, d˜) and (M, dˆ) are metric spaces where d(x, y) d˜(x, y) = and dˆ(x, y) = ln(1 + d(x, y)) 1 + d(x, y)

4. Let (M1,D1) and (M2,D2) be two metric spaces, and M = M1 × M2. Verify that the

following functions define metrics on M, where x = (x1, y2) and y = (y1, y2):

5 (a) d(x, y) = d1(x1, y1) + d2(x2, y2) p 2 2 (b) d(x, y) = d1(x1, y1) + d2(x2, y2)

(c) d(x, y) = max[d1(x1, y1), d2(x2, y2)]

2 5. Prove that (R , d) is a metric space, with  |x − y | x = y  1 1 2 2 d(x, y) = d((x1, x2), (y1, y2)) =   |x1| + |y1| + |x2 − y2| otherwise

6*. We define the bounded sequence space

∞ ` = {(x1, x2, ...) | ∃k > 0, xi ∈ C 3 |xi| ≤ k}

∞ Prove that d(x, y) = sup |xi − yi| defines a metric on ` . i∈N 7*. Let S be any set and M consist of the finite subsets of S. Show that the following map is a metric on M:

d : E × E → [0, ∞) defined by d(A, B) = |(A − B) ∪ (B − A)|

1.2 Underlying Topology

Recall that a topology T on a set X is a family of subsets of X that satisfies the following:

1. Both the empty set and X are elements of T

2. Any union of elements of T is an element of T

3. Any intersection of finitely many elements of T is an element of T

Definition. Let (M, d) be a metric space. Then for all x ∈ (M, d) , we define the open ball around x with radius r by Br = {y ∈ M | d(x, y) < r}. Similarly, we define the closed ball c around x with radius r by Br(x) = {y ∈ M | d(x, y) ≤ r}.

We define a topology on M by T = {O ⊂ M | ∀x ∈ O ∃r > 0 3 Br(x) ⊂ O}. Consequently, open balls are open sets and closed balls are closed sets.

We now revisit a few key concept from topology that are particularly useful for us.

Definition. Let X ⊂ (M, d).

(a) The interior of X denoted X˚ is the union of all open sets contained in X.

(b) The closure of X denoted X¯ is intersection of all closed sets containing X.

6 (c) The boundary of X denoted ∂X is X¯ − X˚

(d)A neighborhood of x ∈ M is a subset of M that contains an open set containing x.

It is important to note that intuition can be misleading when considering these concepts in metric spaces. For instance, the interior of a closed ball is not necessarily the corresponding open ball. See this section’s exercises for further understanding.

Definition. Let A, B be nonempty subsets of M equipped with a metric d. Then the between A and B is d(A, B) = inf{d(a, b) | a ∈ A, b ∈ B}. Consequently, we define the distance between a point and a set by d(x, A) = d({x},A).

Definition. A subset A of M is dense in M if A¯ = M. This is equivalent to stating that for all r > 0 and x ∈ M,Br(x) ∩ A 6= φ. M is separable if it has a countable dense subset.

n Example. Q ⊂ R is countable and dense in R. So R is separable. R is separable since the set n of of all points ~x = (x1, x2, ..., xn) such that xi ∈ Q ⊂ R is dense in R .

Example. `p with 1 ≤ p < ∞ is separable.

p p p Proof. The proof for ` is more tedious. Let Sy ⊂ ` be the set of all sequences in ` of the + form ~y = (y1, y2, ..., yn, 0, 0, ...) with n ∈ N and yi ∈ Q for all i = 1 . . . n. Notice that Sy is a countable union of countable sets. We now need to show that it is dense in `p. ∞ p p X p To see that it is dense in ` , let ~x ∈ l and recall that |xi| converges. So for every ε > 0 i=1 ∞ X εp there exists n such that |x |p < . Rationals are dense in so for each x there is a y k 2 R i i k=n+1 n X εp close to it, so |x − y |p < . Therefore k k 2 k=1

n ∞ p X p X p p (d(x, y)) = |xk − yk| + |xk| < ε k=1 k=n+1 Thus d(x, y) < ε so M is dense in `p.

Example. `∞ defined in exercise 6 of section 1.1 is not separable

Proof. We need to prove that there is not countable dense subset of `∞. Consider the set S of all sequences consisting only of zeros and ones. Then there exists a bijection between S and P (N): ( 0 n∈ / A for all A, B (N we associate ~xA = (xn) such that xn = . So d∞( ~xA, ~xB) = 1 1 n ∈ A if A 6= B. Now surround each point of S by an open sphere of radius 1/2. We obtain an uncountable of pairwise disjoint spheres. So if some set K is dense in `∞, we should have at least one point of K in every sphere. So K is not countable, and `∞ is not separable.

7 Theorem. Let (M, d) be a separable metric space and Y ⊂ M. Then (Y, d) is also separable.

Proof. Let C = {x1, x2, x3, ...} be a dense countable subset of M. One might be tempted to use Y ∩ C as a dense and countable subset of Y , but this does not always work: if M 6= C, take Y = M \ C. Now let

 1  A = (n, m) ∈ × | ∃y ∈ Y 3 d(y, x ) < N N n m

1 For each (n, m) ∈ A choose y ∈ Y with d(y , x ) < . Then Y = y ∈ A is a countable n,m n,m n m n,m ε 1 subset of Y . C is also dense in Y . To see this, let y ∈ Y and ε > 0 with ≥ . C is Y 2 m 1 dense in M so there existsn ∈ N such that d(y, x ) < . So (n, m) ∈ A. Therefore we get that n m 2 d(y, y ) ≤ d(y, x ) + d(x , y ) < ≤ ε. So y ∈ Y . n,m n n n,m m

Exercises

1. Let 0 ≤ ρ < r and x, y two points of a metric space M.

c c (a) Prove that Bρ(x) ⊂ Br(x) ⊂ Br(x) c (b) Prove that if d(x, y) < r − ρ then Bρ(x) ⊂ Br(x)

2. Find the relations (⊆, ( or =) between the following sets:

(a) B˚r(x) and Br(x) ˚c (b) Br(x) and Br(x) c (c) Br(x) and Br(x)

c c ˚ c 3. Let M = {−2}∪[−1, 2] with the metric d(x, y) = |x−y|. Find B1(1), B1(1), B1(1), B1(1), B1(−1) and B1(−1). Comment on your results.

4. Find ∂E for the following subsets of R

(a) E = (1, 2) ∪ (2, 3]

(b) E = R (c) E = φ

5. Let (M, d) be a metric space and A ⊂ M. Show that A = {x ∈ M | d(x, A) = 0}.

6*. Prove that the space C[a, b] is separable.

8 1.3 Convergence and Continuity

Definition. A sequence (xn) of elements in a metric space (M, d) is said to be convergent if there exists some x in M such that d(xn, x) → 0 as n → ∞. In this case, x is the limit of the sequence and we write lim xn = x. A sequence that is not convergent is divergent. n→∞

In general, x is the limit if for every neighborhood Nx of x, there exists N ∈ N such that for all n ≥ N, an ∈ Nx. However, our definition uses the distance function and is most commonly used in metric spaces. Instead of verifying an abstract concept, we simply need to show that a sequence tends to 0. In topological spaces, it is possible for sequences to converge to more than one limit. For example, let X be the real line without the origin, and with two points a 6= b, that are not in R. The topology is generated by the basic open intervals in R \{0} along with sets of the form (u, 0) ∪ {a} ∪ (0, v) and (u, 0) ∪ {b} ∪ (0, v), where u < 0 < v. X is not Hausdorff because a and b cannot be separated by disjoint open sets. Every sequence that converges to a also converges to b. However, this is not the case for metric spaces. The following theorem elaborates this.

Theorem. Convergent sequences in metric spaces have unique limits.

0 0 0 Proof. Suppose lim xn = x and lim xn = x where x 6= x . Let ε = d(x, x )/2. So there exists n→∞ n→∞ 0 0 n ∈ N such that d(xn, x) < ε for all n ≥ n1 and there exists n ∈ N 3 d(xn, x ) < ε ∀n ≥ n1. 0 0 0 0 Let N = max{n, n }. So d(x, x ) ≤ d(x, xn) + d(xn, x ) < ε + ε = d(x, x ), which makes no sense. Therefore x = x0.

Definition. Let (M, d) and (M 0, d0) be two metric spaces where x ∈ M. Then we say that a 0 0 function f : M → M is continuous at x when for all Nf(x) ⊂ M neighborhood of f(x), there exists Nx ⊂ M neighborhood of x such that f(Nx) ⊂ Nf(x). Alternatively, this can be verified using the following theorem:

Theorem. The following are equivalent:

(a) f is continuous at x

(b) For all sequences (xn) → x in M, we have f(xn) → f(x)

(c) For all ε > 0, there is a δ > 0 such that if x, y ∈ E and d(x, y) ≤ δ, then d0(f(x), f(y)) ≤ ε.

Proof.

(a) ⇒ (b): If Nf(x) is an arbitrary neighborhood of f(x) then ∃Nx neighborhood of x

such that f(Nx) ⊂ Nf(x). If xn → x then there is a certain tail N that guarantees

xn ∈ Nx ∀n ≥ N. Consequently, from that same tail, f(xn) ⊂ Nf(x), which means that f(xn) → f(x).

9 (b) ⇒ (a): Suppose that (a) does not hold. Then there exists Nf(x) such that ∀Nx and in

particular for B 1 (x), f(Nx) 6⊂ Nf(x). This means that there exists xn ∈ B 1 (x) such that n n f(xn) 6∈ Nf(x). We now have that xn → x and f(xn) 9 f(x). So ¬(a) ⇒ ¬(b).

(c) ⇒ (a): ∀Nf(x) there exists ε > 0 such that Bε(f(x)) ∈ Nf(x). Choose V = Bδ(x).

(a) ⇒ (c): Let ε > 0. Bε(f(x)) is a neighborhood of f(x). Since (a) is verified, there

exits Nx such that f(Nx) ⊂ Bε(f(x)). But ∃δ > 0 3 Bδ(x) ⊂ Nx. Therefore f(Bδ(x)) ⊂

Bε(f(x)).

Definition. Let (M, d) and (M 0, d0) be two metric spaces where x ∈ M. A function f : M → M 0 is said to be uniformly continuous if for all ε > 0 there exists δ > 0 such that if d(x, y) < δ then d0(f(x), f(y)) < ε. In other words, δ > 0 can be chosen independently from x ∈ M. If a function is uniformly continuous, then it is continuous. The converse is not true.

2 Example. (a) The function f(x) = x from R to R is continuous but not uniformly continuous when R is equipped with the metric d(x, y) = |x − y|.

(b) Let (M, d) be a metric space. Then ∀a ∈ M the function da : M → R defined by d(x) = d(a, x) is uniformly continuous. Proof. By the triangle inequality, we obtain the following: d(a, p) ≤ d(a, q) + d(p, q) ⇒ d(a, p) − d(a, q) ≤ d(p, q), and d(a, q) ≤ d(a, p) + d(p, q) ⇒ d(a, q) − d(a, p) ≤ d(p, q). Together, these inequalities imply |d(a, p) − d(a, q)| = |d(p) − d(q)| ≤ d(p, q). Now, let ε > 0 and δ = ε. Then for any p, q ∈ S, d(p, q) < δ implies |d(p) − d(q)| ≤ d(p, q) < δ = ε, so g is uniformly continuous.

Theorem. Let (M,D) and (M 0, d0) be metric spaces. Then f is continuous if and only if f −1(U) is open in M for each open subset U of M 0.

Definition. Two metrics d1 and d2 on a space M are equivalent if the identity map f(x) = x on M is continuous from (M, d1) to (M, d2) and from (M, d2) to (M, d1). n Example. (a) The Euclidean metric on R and the discrete metric are not equivalent. n Q (b) Let {(Mi, di)}i=1..n be a finite collection of metric spaces, M = Mi and suppose that i=1 n P ~x = (x1, x2, .., xn), ~y = (y1, y2, .., yn) ∈ M. We define the metrics D(x, y) = di(xi, yi) i=1 and D∞(x, y) = max di(xi, yi). Then D and D∞ are equivalent since i=1..n

D∞(x, y) ≤ D1(x, y) ≤ nD∞(x, y) d(x, y) (c) Let (X, d) be a metric space, and let d˜(x, y) = . Then d and d˜ are equivalent. In 1 + d(x, y) t fact, by letting f(t) = then d˜= f ◦ d, and d = f −1 ◦ d˜. 1 + t

10 Exercises  1  1. Consider the real sequence (x ) = . Give an example of a metric d on so that n n R the sequence converges to 2 in (R, d). Then give an example of a metric that makes this sequence diverge.

n 2. Let (M, d) be a metric space, and let (xn) be a sequence in M . Prove that (xn) → x in

(M, d) if and only if every subsequence of (xn) converges to x in (X, d)

3. Suppose that xn → x and yn → y via a metric d. Prove that d(xn, yn) → d(x, y).

4. Let f :(M, d) → (M 0, d0) be continuous at p ∈ M, and g :(M 0, d0) → (M 00, d00) be continuous at f(p). Prove that g ◦ f is continuous at p. (The proof is almost identical when continuity in f and g is uniform).

5. Let (M, d) be a metric space, and let f, g : M → R be uniformly continuous. Show that 2 the direct sum f ⊕g : M → R defined by f ⊕g (x) = (f(x), g(x)) is uniformly continuous.

6*. (a) Prove that the square function is not uniformly continuous in R equipped with the absolute value metric d(x, y) = |x − y|. 0 (b) Show that d (x, y) = | arctan(x) − arctan(y)| is metric on R. 0 (c) Is the square function uniformly continuous under d in R?

1.4 Completeness and Completion

The notion of completeness is of utmost importance when constructing solutions to problems in [functional] analysis. More often, approximate solutions are found, and then the sequence of these approximations is proven to converge towards the exact solution. It is sometimes tedious to find the exact limit. Therefore, we need criteria to verify that a sequence converges without finding the limit.

Definition. A sequence (xn) in a metric space (M, d) is called Cauchy when for all ε > 0 there is an N ∈ N such that for all n, m ≥ N, d(xn, xm) ≤ ε. In other terms, d(xn, xm) → 0.

Every convergent sequence in a metric space is Cauchy. The converse is not true. We will discuss this idea in detail once we get to Banach spaces.

Theorem. Every convergent sequence in a metric space is Cauchy.

Proof. Suppose xn → x in a metric space (M, d). Then ∀ε > 0, ∃N ∈ N 3 d(xn, x) < ε/2 ∀n > N. So by the triangle inequality, d(xm, xn) ≤ d(xm, x) + d(xn, x) < ε/2 + ε/2 = ε.

Theorem. Every in a metric space is bounded.

11 Proof. Let (xn) be a Cauchy sequence of (M, d). We apply the definition of Cauchy sequence with ε = 1. There exists N such that d(xm, xn) < 1 for all m, n ≥ N so (xn) ∈ B1(xN ) ∀n ≥ N.

Define r = 1 + max{1, d(x1, xN ), d(x2, xN ), ..., d(xN1 , xN )}. Then xn ∈ Br(xN ) for all n, and thus (xn) is bounded.

Theorem (Bolzano-Weierstrass). A bounded sequence of real numbers has a convergent sub- sequence.

Proof. Let (xn) be a sequence of real numbers which is bounded, i.e. there exists α, β, such that

α ≤ xn ≤ β. Then at least one of the sets  α + β   α + β   α + β  x : α < x < , x : < x < β , x : x = n n 2 n 2 n n n 2 is infinite. If the third set is, we are done. Otherwise, consider the first element from the infinite set, say xk1 . Repeat this process for all (xn)n>k1 . We then obtain the sequence (xkn ). This sequence is Cauchy, and therefore converges.

Definition. A metric space (M, d) is complete if all of its Cauchy sequences converge.

Example. The real line R equipped with its standard metric is complete.

Proof. Let (x1, x2, ...) be a Cauchy sequence in R. Cauchy sequences are bounded, and the

Bolzano-Weierstrass theorem tells us that there is a subsequence (xnk ) converging to a limit, say `. We shall prove that the whole sequence tends to `. Let ε > 0, and let N and K be large enough that |xm − xn| < ε/2 for n, m > N, and |xnk − `| < ε/2 for k > K. When n > max{N, nK }, |xn − `| ≤ |xn − xnk | + |xnk − `| < ε/2 + ε/2 = ε. Since nk > N, we get that (xn) is convergent with xn → `.

n Example. R equipped with the Euclidean metric is complete.

n Proof. Let (xn) be a Cauchy sequence in R . So each term of the sequence will have the form

(1) (2) (k) xm = (xm , xm , ..., xm )

Let ε > 0 and choose N such that d(xn, xm) < ε for all n, m > N. So for each 1 ≤ i ≤ k, (i) (i) (i) |xn − xm | ≤ d(xn, xm) < ε. Therefore the sequence (xn ) is also Cauchy, so converges to some (i) (1) (2) (n) limit x ∈ R. Thus (xn) converges to x = (x , x , ..., x ).

Example. Q is not complete.

Proof. We just need to find a sequence in Q that converges to a value that is not in Q. The 2 xn + 2 sequence defined by x0 = 1, xn+1 = (i.e. the rationals 1, 3/2, 17/12...) lies in Q. However √ 2xn it converges to the 2. See the Babylonian method for computing squared roots.

12 Example. The continuous function space C[a, b] is not complete when equipped with the metric Z b d1 = |f(x) − g(x)|dx. a Proof. Consider the sequence (fn) ⊂ C[0, 1] defined by:

 −1 0 ≤ x < 1/2 − 1/n  fn(x) = n(x − 1/2) 1/2 − 1/n ≤ x ≤ 1/2 + 1/n  1 1/2 + 1/n < x ≤ 1

Then (fn) is Cauchy but it is not convergent in C[0, 1]. In fact, it converges to the discontinuous function  −1 0 ≤ x < 1/2  f(x) = 0 x = 1/2  1 1/2 < x ≤ 1

Proofs for completeness can get more tedious than for the examples we provided. We will list the most relevant results to our field, and leave the proofs to the curiosity of the reader from n ∞ our references. These include the complex plane, C equipped with the modulus metric, ` equipped with the metric defined in section 1.1, c (the space of convergent sequences in `∞) equipped with the metric induced from `∞, `p equipped with the metric defined in section 1.1, and C[a, b] equipped with d∞ defined in section 1.1.

This next fundamental result in functional analysis guarantees the existence and uniqueness of solutions to nonlinear problems, and requires the notion of completeness. We will start with a necessary definition: Definition. Let (M, d) be a . f : M → M is called a contraction when there exists a k ∈ (0, 1) such that ∀x, y ∈ M, d(f(x), f(y)) ≤ k d(x, y). Theorem. If f is a contraction in a metric space (M, d), then f has a unique fixed point. This means that there exists a unique a ∈ M such that a = f(a). Moreover, every recursive sequence n (an) where an+1 = f(an) converges to a, and d(an, a) ≤ k d(a0, a). Proof. This theorem requires three “subproofs”: Existence: Consider a sequence (an) verifying for all n ∈ N, an+1 = f(an) . Therefore

∀n ≥ 1, d(an+1, an) = d(f(an), f(an−1)) ≤ k d(an, an−1)

n By induction, we obtain for all n ≥ 1, d(an+1, an) ≤ k d(a1, a0). Hence ∀n, p ≥ 1

d(an+p, an) ≤ d(an+p, an+p−1) + d(an+p−1, an+p−2) + ... + d(an+1, an) n+p n+p−1 n ≤ k d(a1, a0) + k d(a1, a0) + ... + k d(a1, a0) kn ≤ d(a , a ). 1 − k 1 0

13 Therefore, the sequence is Cauchy and given that (M, d) is complete, it converges to some a ∈ M.

A contraction is clearly continuous, hence f(an) → f(a). Therefore a = f(a).

Uniqueness: Suppose a0 is another fixed point for f. So d(a, a0) = d(f(a), f(a0)) ≤ k d(a, a0) and thus (1 − k)d(a, a0) ≤ 0. But 1 − k > 0 so d(a, a0) ≤ 0 ⇒ d(a, a0) = 0 ⇒ a = a0.

Order of Convergence: We have that d(an+1, a) = d(f(an), f(a)) ≤ k d(an, a). So by n+1 induction, d(an+1, a) ≤ k d(a0, a).

We proved that the set of rationals is not complete. However, we can intuitively “extend” this set to the real numbers which form a complete space. This should also be the case for any incomplete metric space. For this purpose, we next define the completion of a metric space. We start with the definition of an isometry.

Definition. Let (M, d) and (M 0, d0) bet two metric spaces. A mapping T from M into M 0 is called an isometry if it preserves distance: d0(T (x),T (y)) = d(x, y) for all x, y ∈ M.

Any metric space (M, d) can be completed, which means that there exists a complete metric 0 0 0 0 0 space (M , d ) such that M ⊂ M that is dense in M and d |M×M = d. This space is unique to within isometries. We construct the completion of a metric space using the following method: let M 0 be the set of equivalence classes Cauchy sequences to those in M. Then an element 0 x ∈ M can be seen as an element in M via the equivalence class [(xn)] of the constant sequence xn = x in M. Define the distance

0 0 d¯((xn), (x n)) = lim d(xn, x ) n→∞ n

0 This limit exists because the sequence (d(xn, xn)) is Cauchy in R. Because this distance stays 0 the same, when we replace (xn) or (xn) with a corresponding equivalent Cauchy sequence, the distance d¯ induces a distance d on M 0.

Theorem. The metric space (M 0, d0) defined above is the completion of (M, d)

0 0 Proof. If [(xn)] and [(xn)] are equivalent classes of the constant sequences (x) and (x ). Then, 0 with d |M×M = d,

0 0 0 0 0 d ([(xn)], [(x )]) = d¯((xn), (x )) = lim d(x, x ) = d(x, x ) n n n→∞

0 0 M is dense in M because each equivalence class [xn] ∈ M is Cauchy. There exists constant (k) (k) 0 (k) sequences (yn = y of elements in E such that d ([(xn)], [(yn )]) → 0 when k → ∞. It’s (k) 0 0 sufficient then to take y = xk ∀k ∈ N. The metric space (M , d ) is thus complete, because (k) 0 if the elements [(xn )] ∈ M are Cauchy, then this sequence converge to the equivalence class 0 [(yn)] ∈ M of the sequence (yn) such that yn = xn ∀n ∈ N.

14 Exercises

1. Prove that a convergent sequence in a metric space is Cauchy.

2. Using the completeness of R, prove the completeness of C.

3. Let a, b ∈ R with a < b. Is (a, b) complete? What about [a, b]?

4. What is the completion of (Q, d) where d(x, y) = |x − y|?

5. If M and M 0 are isometric and M is complete, show that M 0 is complete.

6*. (a) Let M = [0, ∞) and d(x, y) = | arctan(x) − arctan(y)|. Show that (M, d) is a metric space (you can use section 1.3’s exercises), then show that the sequence (n2) is Cauchy in (M, d). Is it convergent in M?

(b) Show that the space (R, d) is incomplete.

7*. Let M = {f : R → R | f ≥ 0, f continuous and bounded} and

d(f, g) = sup |f(x) − g(x)| x∈R Let g ∈ E and k be a nonnegative continuous function in 2 such that there exists an R Z integrable function ϕ : R → R with ∀x, y ∈ R, k(x, y) ≤ ϕ(y) and ϕ(t) dt < 1. Let Z R I(x) = k(x, y)f(y). We seek a function f ∈ M 3 ∀x ∈ R, f(x) = I(x) + g(x). R (a) Show that (M, d) is complete.

(b) Show that for all u ∈ E and x ∈ R,I(x) ∈ M. (c) Using a fixed point, show that the solution function f is unique.

1.5 Compactness

Another key concept from analysis is compactness. This also allows us to obtain convergent sequences. The standard example of a compact set is closed and bounded intervals in R. We start by reviewing some basics definitions from topology.

Definition. An (open) cover of a topological space T is a collection of open sets of the form [ 0 {Ui | i ∈ I} with Ui = T .A subcover [of a cover] is a subcollection {Ui | i ∈ I } for some i∈I 0 [ 0 I ⊆ I with Ui = T . If I is finite, then the subcover is finite. i∈I0 Definition. A topological space is compact if each of its open covers has a finite subcover.

15 The same definition applies to metric spaces. For the purpose of our course, we will therefore continue while limiting ourselves to metric spaces.

Definition. A metric space is sequentially compact if every infinite sequence has a convergent subsequence. A metric space (M, d) is precompact1 if for every ε > 0, there exists a finite collection of open balls in M of radius ε whose union contains M.

A compact metric space is separable. A discrete metric space is precompact if and only if it is n finite. A compact metric space is precompact. The compact sets of R are exactly the closed T and bounded balls. The converse is not true. The set Q [0, 1] is closed but not compact since it is not complete.

Theorem. Let (M, d) be a metric space. Then the following are equivalent.

(a)( M, d) is compact

(b)( M, d) is sequentially compact

(c)( M, d) is complete and precompact.

Proof. For notational purposes, we will represent (M, d) by M. \ (a) ⇒ (b): Let Fn be the closure of the set {xn, xn+1,... }, F = Fn and suppose that n∈N F 6= φ. Then the collection of open sets {M − Fn, n ∈ N} is an open cover of M which

is compact, so we can find a finite subcover of M: {M − Fik | k = 1 . . . n}. Therefore n n [ \ M = (M − Fik ) = M − Fik which implies F = φ (contradiction). So F 6= φ. Now k=1 k=1 let x ∈ F . We define the recursive sequence (nk) with n1 = 1 and nk = min{nk | nk >

nk−1 and xnk ∈ B1/k(x)}. This always exists because x ∈ F . Therefore (xnk ) → x.

(b) ⇒ (c): M is sequentially compact so every Cauchy sequence (xn) ∈ M has a sub- sequence that converges to some x ∈ M. We then use the triangle inequality with the

Cauchy condition to show that (xn) → x. So M is complete. Now suppose that M is not precompact, so there exists ε > 0 such that M has no finite subcover of open balls of k−1 [ radius ε. We can therefore define a sequence (xn) with x1 ∈ M and xk ∈/ Bε(xi). xk i=1 k−1 [ will always exist because Bε(xi) ∈/ M (we assumed M is not precompact). Therefore i=1 d(xn, xm) > ε, so (xn) cannot contain any convergent subsequence. This contradicts with the fact that M is sequentially compact.

1Also called totally bounded

16 (c) ⇒ (a): The proof of this implication is technical, and will be left as a project or to the student’s curiosity.

Theorem. A continuous function from a compact metric space M to a (non necessarily compact) metric space M 0 is uniformly continuous.

Proof. f is continuous so for all ε > 0 and x ∈ M there exists δ such that when d(x, y) < δ, d(f(x), f(y)) < ε. The open balls Bδi/2(xi) with xi ∈ M is an open cover of the compact space M so we can find a finite subcover {Bδi/2(xi)}, i = 1..n. Let δ = min{δi}. For each x ∈ M, x ∈ Bδi/2(xi) such that Bδ(x) ⊂ Bδi (xi) for i = 1..k. Hence, for each y ∈ Bδ(x), d(f(y), f(xi)) < ε/2. Since we have d(f(x), f(xi)) < ε/2, we get d(f(x), f(y)) < ε.

Exercises

1. Prove that a compact metric is separable.

2. Let (M, d) be a metric space and M 0 ⊂ M. If M 0 is compact, prove that for each x ∈ M there exists a y ∈ M 0 such that d(x, M 0) = d(x, M).

( n ) X 3. Show that ∆n = (x1, ..., xn) | 0 ≤ xi for 1 ≤ i ≤ n and xi = 1 is compact. i=1 4. Let T be a continuous map from one metric space M to another one M 0. Show that the image of a compact subset of M under T is compact.

5. Consider the same map T as in the previous exercise. Let X ⊂ M be compact. Prove that 1 T |X : X → R (the restriction of T to X) attains a maximum and minimum value .

1This is the extreme value theorem

17 Chapter 2

Banach Spaces

From now on, the field K will denote C or R. In the case of a , |c| will denote the absolute value of c. In the case of a c = a + bi, |c| will denote the modulus of √ c, that is, |c| = a2 + b2.

2.1 Normed Vector Spaces

2.1.1 Vector Spaces

We will start with some basic definitions of vector spaces from undergraduate linear algebra.

A V over a field K is a set equipped with two maps: the addition map denoted +: V × V → V :(~v, ~w) 7→ ~v + ~w and the · : K × V → V :(k,~v) 7→ k · ~v such that for all c, c1, c2 ∈ K and ~v, ~w ∈ V .

(a)( V, +) is an abelian group (b)1 · ~v = ~v (c) c · (~v + ~w) = c · ~v + c · ~w

(d)( c1 + c2) · ~v = c1 · ~v + c2 · ~v

(e) c1 · (c2 · ~v) = (c1 · c2) · ~v

n For notational purposes, we will denote c · ~v by c~v. Commonly used vector spaces include K equipped with its regular componentwise vector addition and scalar multiplication. If V is a X vector space on K and X is a set, then the set V of functions from X to V equipped with addition (f +g)(x) = f(x)+g(x) and multiplication (cf)(x) = cf(x) is a vector space.

The sequence space `∞ is a vector space under regular vector addition and scalar multiplication of vectors. The function space C[a, b] is a vector space under regular function addition and scalar multiplication.

18 n X Let S ⊆ V .A linear combination of elements of S is a vector ~v = ci ~si where ci ∈ K and i=1 ~si ∈ S. If every vector of V is a linear combination of elements in S, then S is a generating set or spanning set of V . The span of S is the set of all finite linear combinations of elements of S. n X Vectors in S in are linearly independent if for each ci ∈ K and si ∈ S, ci ~si = 0 ⇒ ci = 0 i=1 for all i = 1 ··· k.A basis of V is a is a linearly independent spanning set. The dimension of V , denoted dim V , is the number of elements in a basis of V .

n n Example. The dimension of K as a vector space on K is n. The dimension of C , as a X vector space on R, is 2n. If X is an infinite set, then for all non-empty sets E, E is infinite dimensional.

A subspace W of V is a subset of V that is a vector space over the same field as V . Let S ⊂ V . The vector space generated (or spanned) by S is the smallest subspace of V containing S; it is also the set of linear combinations of elements of S. If W is a subspace of V , then dim W ≤ dim V .

A map M : V → W between two vector spaces over the same field K is a if M( ~v1 + ~v2) = M( ~v1) + M( ~v2) and M(c~v) = cM(~v) for all c ∈ K and ~v, ~v1, ~v2 ∈ V . The kernel of a linear map M is ker M = {~v ∈ V | M(~v) = ~0}. It is a subspace of V . A linear map is an if it is bijective. Two vector spaces are isomorphic if there exists an isomorphism between them. Vector spaces are isomorphic if and only if they have the same dimension.

n Example. Any finite dimensional vector space on K is isomorphic to K . Any linear map from n m K to K can be represented by an m × n matrix with coefficients in K.

Let W1 and W2 be two subsets of V . Their sum W1 + W2 = { ~w1 + ~w2 | ~w1 ∈ W1 and ~w2 ∈ W2}.

If W1 and W2 are subspaces of W , their sum is a subspace generated by W1 ∪ W2. It is a L direct sum denoted W1 W2 if W1 ∩ W2 = {~0}. Finally, we have the following equalities: L dim(W1 + W2) = dim W1 + dim W2 − dim(W1 ∩ W2) and dim(W1 W2) = dim W1 + dim W2.

2.1.2 Normed Spaces

Definition. A normed space N = (N, k·k) is a vector space V over K equipped with a , + i.e. an application k · k : V → R such that for all v, w ∈ V and c ∈ K: (a) kvk = 0 ⇐⇒ v = ~0, (b) kcvk = |c|kvk , (c) kv + wk ≤ kvk + kwk (triangle inequality).

It is clear that a (V, k.k) is a metric space with d(x, y) = kx − yk. This will allow us to use a lot of results from our previous chapter.

19 n Example. (a) We define the p-norm on K for all vectors ~xi = (x1, ..., xn) as follows:

n !1/p X p k~xkp = |xi| if 1 ≤ p < ∞ and k~xk∞ = max |xi| i=1..n i=1

The following figure pictures the set of vectors of norm 1 when using the norms where 2 p = 1, 2, ∞ in R :

1-norm 2-norm ∞-norm

∞ (b) A norm on ` is k(xn)k = sup |xn|. Norms on C[a, b] include kfk∞ = sup |f(x)| and n∈N x∈[a,b] Z b kfk1 = |f(x)|dx. a (c) If X is a set and N is a normed space, we denote by B(X,N) the set of bounded maps from X X to N. Then B(X,N) is a subspace of N . We define the norm k.k∞ on B(X,N) using

the norm on N by kfk∞ = sup kf(x)kN where f ∈ B(X,N). x∈X

Definition. A V is a vector space on K equipped with a topology such that the addition and scalar multiplication are continuous functions.

Theorem. Let W be a subspace of a normed vector space V . Then the closure W of V is also a subspace of V .

0 0 0 Proof. Let w, w ∈ W , c ∈ K and (wn), (wn) sequences in W converging to w and w respectively. We have cwn ∈ W and lim cwn ∈ W . By the continuity of scalar multiplication, this limit can n→∞ 0 0 be written as c lim wn = cw, so cw ∈ W . Similarly, wn + w ∈ W and lim wn + w ∈ W . By n→∞ n n→∞ n the continuity of addition, this limit will equal w + w0 so w + w0 ∈ W .

Definition. Two norms k.k1 and k.k2 on a vector space V are equivalent if there exists con- stants α > 0 and β ≥ 0 such that αkvk1 ≤ kvk2 ≤ βkvk1 for all v1, v2 ∈ V .

We define balls in normed spaces in a similar fashion to those in metric spaces (simply replace the distance function by the norm). Then the inequalities above are equivalent to a nesting of balls using the corresponding metric. If two norms on a vector space are equivalent, then every Cauchy sequence in one is Cauchy in the other.

20 Theorem. On a finite dimensional vector space, all norms are equivalent.

n Proof. We will prove this for K with the standard basis e1, e2, ..., en. The proof can be gen- eralized for any finite dimensional vector space, but is more tedious and requires the Cauchy n X inequality. For any ~x = xiei, we simply need to show that any norm for ~x is equivalent to i=1 kxk∞. To see this, let k.k be an arbitrary norm. On one hand

n n X X k~xk = xiei ≤ kxikkeik ≤ βkxk∞ i=1 i=1

n X + where β = keik > 0. Note that the mapping N : V → R : x 7→ kxk is continuous. On i=1 n the other hand, the set S = {x ∈ K | kxk∞ = 1} is compact. Since N is continuous on S and never vanishes, there exists and α > 0 such that N(x) ≥ α ∀x ∈ S, i.e. if kxk∞ = 1 then kxk ≥ α = αkxk∞.

In this previous proof, we mentioned the notion of compactness. We know that in finite dimen- sions, compact sets are closed and bounded. This is not the case for infinite dimensions. The following are known results of the functional analyst Frigyes Riesz.

Lemma. Let W ( V be a closed subspace of a vector space V . Then for all ε > 0 there exists v ∈ V such that kvk = 1 and kv − xk ≥ 1 − ε for each x ∈ W .

Proof. Let w ∈ V − W . W is closed so d = d(w, W ) > 0. Choose x◦ ∈ W such that d d ≤ kw − x k ≤ ◦ 1 − ε

w − x◦ Then choose x◦ such that v = works. In fact, if x ∈ W , since x◦ + kw − x◦kx ∈ W , kw − x◦k

w − x◦ 1 1 − ε kv − xk = − x = kw − (x◦ + kw − x◦kxk ≥ d kw − x◦k kw − x◦k d

Theorem. A normed space N is finite dimensional if and only if B1(0) is compact.

Proof. Only one direction’s proof needs to be elaborated. We will prove the contrapositive statement. If N is infinite dimensional, there exists an infinite collection of finite dimensional subspaces Vn, n ∈ N such that Vn ( Vn+1 for all n. Apply the previous Lemma to Vn with ε = 1/2 to see that vn+1 ∈ Vn+1 ∩ B1(0). This implies that kvn+k − vnk > ε, so we can not find a subsequence of (vn) converging in B1(0). Hence, B1(0) is not compact.

21 2.1.3 Bounded Linear Operators

Definition. An T : V → W is a map between two vector spaces. It is linear if for 1 all x, y in its domain, T (x + y) = T x + T y and T (ax) = aT x where a ∈ K. It is bounded if there exists α > 0 such that kT xkW ≤ αkxkV for all x ∈ V .

Theorem. Let T be a linear operator from V to W . Then the following are equivalent:

(a) T is bounded

(b) T is uniformly continuous

(c) T is continuous at the origin

Proof. (a) ⇒ (b): Use the definition of boundedness to see that for all ε > 0, if v1, v2 ∈ V and

kv1 − v2kV < ε/α then kT v1 − T v2kW < ε.

(b) ⇒ (c): T is uniformly continuous, hence continuous. But T is linear, so the image of the origin under T exists. Thus T is continuous at the origin.

(c) ⇒ (a): If T is continuous at the origin, there exists α > 0 such that T (B 1 (0))v ⊂ (B1(0))W . α Thus if kvkV = k/α with 0 < k < 1 then kT vkW ≤ k = αkvkV

kT vk Definition. The norm of a bounded linear operator T : V → W is kT k = sup . v∈V,v6=0 kvk Theorem. Let U, V, W be normed vector spaces and A : U → V , B : V → W and C : V → W bounded linear operators. Then

(a) kB + Ck ≤ kBk + kCk

(b) kB ◦ Ak ≤ kBkkAk

The proof of this previous theorem will be left as an exercise.

We conclude this section with a brief discussion. The previous content, along with the upcoming exercises, clarify two keys concepts. First, we can consider the set of continuous functions as a vector space where the vectors are functions. This is central in functional analysis. We can find solutions to problems where the unknown variable is a function. The difficulty is that functional spaces are usually infinite dimensional. In addition, we already proved that in finite dimensional vector spaces all norms are equivalent, but this is not the case for infinite dimensions. The choice of the norm for a function space is essential.

1The notation T x is standard for T (x) in functional analysis

22 Exercises

ax bx 1. Let f(x) = e and g(x) = e where a, b ∈ R and x ∈ [0, 1]. Show that f and g are linearly independent in C[0, 1].

2. Show that a normed vector space is a metric space with d(x, y) = kx − yk. Is a metric space necessarily a normed space? Why or why not?

3. Let V be a normed vector space over K. A subset A ⊆ V is said to be convex if for all x, y ∈ A and t ∈ [0, 1] : tx + (1 − t) y ∈ A.

(a) Show that an open ball in the metric induced by the norm of V is a convex set. (b) Show that the closure and the interior of A are also convex.

2 4. Is kxk = |x| a norm on R?

5. Show that in a normed vector space, finite dimensional subspaces are complete.

6. Let N be a normed space where absolutely convergent sequences are convergent. Prove that N is complete.

7. If (N1, k.k1) and (N2, k.k2) be two normed spaces. Show that the vector space N = N1 ×N2

becomes a normed space if we define k.k = max{k.k1, k.k2}.

8. Show that the satisfies all of the properties of a norm.

9. Prove the last theorem of this section. Z b 10. Let u ∈ C[a, b] and define kuk1 = |u(t)|dt , kuk∞ = sup |u(t)|. a t∈[a,b]

(a) Show that (E, k.k1) and (E, k.k∞) are normed vector spaces. 1 ∞ ∞ 1 (b) Let B and B denote the balls for the respective norms. Show that B1 (0) ⊂ B1 (0). 1 ∞ (c) Is there an R > 0 such that BR(0) ⊂ B1 (0)? 1 ∞ (d*) Are the closed balls B1 (0) and B1 (0) closed in (E, k.k1)? Hint: Consider the −n(x−a) sequence fn(x) = ne .

11*. Verify that a norm on the space of matrices Mn(R) is kAk = n max |ai,j| if A = (ai,j). i,j Z 1 12*. Let u ∈ C[0, 1] with kuk = |u|dt and Φ : C[0, 1] → C[0, 1] with Φ(u(t)) = t u(t). 0

(a) Show that Φ is a linear operator and that kΦkC[0,1] ≤ 1. n (b) Use the sequence un(t) = (n + 1)t to show that kΦkC[0,1] = 1.

23 2.2 Banach Spaces

Definition. A is a complete normed vector space. Recall that a space is complete if every Cauchy sequence in it converges.

n Example. (a) K equipped with the p-norm is a Banach space.

(n) n Proof. Let (x ) be a Cauchy sequence in K . Then there is some N > 0 such that kx(k) − x(m)k < ε for all m, k > N. Without loss of generality assume ε < 1 so ε2 < ε. n X (k) (m) 2 This means that |xi − xi | < ε for all m, k > N. Consequently, for each i = 1 . . . n, i=1 (k) (m) (m) |xi − xi | < ε, which means that (xi ) is Cauchy in K, therefore it converges to some n X (m) 2 (m) limit xi. Let ~x = (x1, x2, . . . , xn). When k → ∞, |xi − xi | < ε , that is kx − x k ≤ ε i=1 and so lim x(m) = ~x in the normed space. m→∞

p (b) If B is a Banach space and 1 ≤ p ≤ ∞ then ` (B) equipped with the norm k.kp is Banach.

Proof. We will sketch a proof of this for `2. Details are inspired from the previous proof.

(n) (n) 2 STEP 1. Show that components (xi ) of any Cauchy sequence (x ) in ` converge to a sequence ~x = (xi) that is Cauchy. 2 STEP 2. Show that ~x = (xi) ∈ `

STEP 3. Conclude that for all ε > 0, k~x − x(n)k ≤ ε eventually.

(c) Let X be a set and B a Banach space. Then the vector space B(X,B) equipped with the

norm k.k∞ is a Banach space.

(d) C[a, b] equipped with its infinity norm is Banach.

Proof. Let (fn) be a Cauchy sequence in C[a, b]. Since [a, b] ⊂ R which is complete, the limit of (fn) exists, call it f. Note that |f(x) − fn(x)| ≤ ε for any . We will use the result from real analysis and state that in this case, f is continuous. Thus f ∈ C[a, b]. This is

equivalent to kf − fnk ≤ ε for all ε > 0, concluding our proof.

 Z  n p (e) Let 1 ≤ p < ∞. The vector space S = f : R → K |f(x)| dx < ∞ can be equipped Rn with the norm 1 Z  p p kfkp = |f(x)| dx Rn p n The completion of this normed space is a Banach space denoted by L (R , K).

24 Theorem. Let V be a normed vector space and B a Banach space. Then the set of all linear operators (denoted L(V ; B)) is a Banach space.

Proof. Let (un) be Cauchy in L(V ; B). For each x ∈ V ,

kun(x) − um(x)kB ≤ kun − umkL(V ;B)kxk.

Thus (un(x)) is Cauchy in B and converges to some limit u(x). For each x, y ∈ V and c ∈ K, un(cx + y) = cun(x) + un(y). Taking limits on both side, u(cx + y) = cu(x) + u(y) and so u is a linear map. Let ε > 0. For each x ∈ V there is some N > 0 such that for each n, m ≥ M,

kun − umkL(V ;B) ≤ ε ⇒ kun(x) − um(x)kL(V ;B) ≤ εkxk.

If we fix n and let m → ∞ , then we get that u is continuous with kukL(V ;B) ≤ ε + kunkL(V ;B) with ku − unkL(V ;B) ≤ ε. So un → u in L(V ; B).

∞ ∞ ∞ X X X Definition. fn(x) is normally convergent if kfn(x)k = sup |fn(x)| converges. n=0 n=0 n=0 x∈I ∞ X Theorem. xn converges normally if and only if there exists a nonnegative n=0 ∞ X an such that for all n ∈ N and x ∈ I, |fn(x)| ≤ an. n=0

Note that normal convergence is equivalent to when we equip R with its absolute value norm. In general, it is not true that if a normally convergent series is convergent, N X then the sequence of partial sums fn(x) converges when N → ∞. For this to be true, the n=0 vector space where fn lies needs to be complete.

Theorem. In a Banach space, normally convergent series converge.

The proof is very straightforward. One just needs to notice that partial sums of normally convergent series are Cauchy sequences.

2.2.1 Introduction to Differential Equations in Banach Spaces

Results from basic calculus can be used in Banach spaces. Proofs tend to be more tedious, but we will list some necessary results. First of all, we will limit integrals in this course to Riemann integrals. Lebesgue integrals tend to be more tedious especially in infinite dimensional vector spaces (in finite dimensions, we can just integrate each coordinate). We are considering continuous functions on bounded intervals.

25 Z b Let I = f(t) dt where f ∈ C[a, b] of a Banach space B . Then I is a linear continuous map a equipped with the norm k.k = sup kf(t)kB. t∈[a,b]

Let f ∈ C[a, b] in a Banach space B. Then the antiderivatives of f are functions F ∈ C[a, b] Z t such that F (t) = x + f(s) ds where x ∈ B is arbitrary. In particular, F is constant on [a, b] a if and only if f = 0 on [a, b].

This will allow us to solve differential equations in infinite dimensional spaces. We also have all versions of the Cauchy-Lipschitz theorem and we will cite one of the simplest versions. First, we start with a definition.

Definition. Let B be a Banach space in K and J an interval in R. F : J × B → B is globally Lipschitz with respect to its second variable if the following holds:

∃L > 0, ∀t ∈ J, ∀x, y ∈ B, kF (t, x) − F (t, y)kB ≤ Lkx − ykB

Theorem. Let F : J × B → B be a continuous and globally Lipschitz function. Then for each t◦ ∈ J and x◦ ∈ B there exists a unique solution ϕ ∈ C(J) in B to the differential equation 0 y = F (t, y) with ϕ(t◦) = x◦.

Proof. The proof is an application of the fixed point theorem covered in Chapter 1. We start by noting that this problem is equivalent to finding ϕ ∈ C(J) in B such that for all t ∈ J, Z t ϕ(t) = x◦ + F (s, ϕ(s)) ds, which is equivalent to proving the uniqueness of the fixed point for t◦ Z t the application Φ : C(J) → C(J) in B defined by Φ(u)(t) = x◦ + F (s, u(s)) ds with u ∈ C(J) t◦ in B. For this purpose, we will show that for a well chosen norm, Φ is a contraction. Let r > 0 −r |t−t◦| and define kukr = sup e ku(t)kB. Let Sr(J) be the normed space where functions have t∈J a finite r norm. Then Sr(J) is a Banach space. We can then apply the fixed point theorem (a

Banach space is a complete metric space). Then for each t ≥ t◦

Z t Φ(u)(t) − Φ(v)(t) = F (s, u(s)) − F (s, v(s)) ds t◦

Z t so kΦ(u)(t) − Φ(v)(t)kB ≤ Lku(s) − v(s)kB ds and t◦ Z t r(s−t◦) L −r |t−t◦| kΦ(u)(t) − Φ(v)(t)kB ≤ Lku − vkre ds = ku − vkre t◦ r L We get a similar bound when t ≤ t , so kΦ(u) − Φ(v)k ≤ ku − vk . If we let r = 2L, Φ is a ◦ r r r contraction and has a unique fixed point. Note that any solution in C(J) belongs to Er(J).

26 Exercises

1. Prove that any finite dimensional normed space is Banach.

2. Check if the following spaces (V, k.k) are Banach:  Z 1  (a) C[0, 1], |u(x)|2 dx 0   (b) {(xn) ∈ K | xn → 0}, sup |xn| n≥0  1/2 X 2 X 2 (c) {(xn) ∈ K | |xn| < ∞}, |xn|

3. Prove that if a series is normally convergent, then it is uniformly convergent.

4. Check if the series are normally convergent. X sin(nx) (a) n2 X 2 (b) xe−nx

5. Let E and F be Banach spaces with norms k · kE and k · kF respectively. Let U ⊆ E be an open subset. A map f : U → F is said to be Fr´echet differentiable at x ∈ U if there is a

continuous linear operator Ax : E → F such that:

kf(x + h) − f(x) − A hk lim x F = 0 h→0 khkE Show that if the Fr´echet derivative exists, it is unique.

n 6*. Let Ω ⊂ R and I an bounded interval in R. Let E denote the Banach space of real bounded continuous functions on Ω equipped with the sup norm. Let F : I → E and f : I × Ω → R with f(t, x) = F (t)(x).

(a) Show that if F ∈ C(I) then f is continuous, and that the converse is true when f is uniformly continuous. Z Let k ∈ C(I × Ω × Ω) such that for some L > 0, ∀t ∈ I, x ∈ Ω |k(t, x, y)| dy ≤ L, Ω and u ∈ E (b) Show that there exists a unique function f ∈ C(I × Ω) that has a partial derivative with respect to t that verifies: ∂f Z ∀x ∈ Ω, f(0, x) = u(x), and ∀t ∈ T (t, x) = k(t, x, y)f(t, y) dy ∂t Ω

27 2.3 Linear Functionals and Duality

Definition. A functional is an operator whose range lies in K. Consequently, a linear func- tional is a linear operator from a vector space V to K.

Recall the definition of bounded operators from the previous section. A linear functional f with domain D is bounded if there exists a real number c such that for all x ∈ D, |f(x)| ≤ ckxk. Furthermore, the norm of f becomes kfk = sup |f(x)|. Thus ∀x ∈ D |f(x) ≤ kfk kxk. x∈D kxk=1 Examples. The dot product of two vectors defines a linear functional if we fix one factor, say ~v: n n X dot~v : R → K ~x 7→ ~x · ~v = xi · vi i=1

It is also bounded since |dot~v| = |~x·~v| ≤ k~xkk~vk. Since kdot~vk = sup |~x·~v|, we get kdot~vk ≤ k~vk. x∈D kxk=1 When we have ~x = ~v, we get kdot~vk ≥ k~vk. Thus kdot~vk = k~vk.

Another example of a linear functional is the definite integral. However, we are not considering the integral over a single function (although this would still be a functional). In this case, we will consider that integral for all functions in a certain function space. We will consider C[a, b]. Z b Let I be the integral. Then I(x) = x(t) dt where x ∈ C[a, b]. I is obviously linear. We will a prove that it is bounded and find its norm using the standard max norm on C[a, b]. Then

Z b

|f(x)| = x(t) dt ≤ (b − a) max |x(t)| = (b − a)kxk a t∈J Now take the sup over all x of norm 1 to get kfk ≤ b−a and a particular x = 1 to get kfk ≥ b−a. Thus kfk = b − a.

One final example is a functional on the Hilbert sequence space `2 (see section 1.1) by choosing ∞ 2 X 2 a fixed α = (αi) ∈ ` and defining the functional f(x) = xiαi for x = (xi) ∈ ` . This is an i=1 absolutely convergent sequence and f is bounded. Using the Cauchy-Schwarz inequality,

v v ∞ ∞ u ∞ u ∞ X X uX 2uX 2 |f(x)| = xiαi ≤ |xiαi| ≤ t |xi| t |αi| = kxk kαk i=1 i=1 i=1 i=1

Definition. The algebraic dual V ∗ of a vector space V is the set of all linear functionals on V . The topological dual V 0 of V is the set of all continuous linear functional on V .

28 When V is finite dimensional, V ∗ and V 0 coincide since every linear functional is bounded in this case. For this purpose, we will just refer to either V ∗ or V 0 by the dual.

|f(v)| The dual norm (recall the definition of an operator norm) is kfkV 0 = sup . v∈V kvkV

The duality between the normed space V and V 0 can be represented by the bilinear canonical 0 0 form h·, ·i : V × V → K defined by hf, vi = f(v). It is a linear application at f ∈ V and v ∈ V . The definition of the dual norm gives us |hf, vi| = |f(v)| ≤ kfk|V 0 kvkV .

The duality relation between V and V 0 might seem interestingly enough symmetric, but the topological dual has better properties than the original space V in general.

Theorem. Let V be a normed vector space. Then the topological dual V 0 is Banach.

0 Proof. Let (fn) be a Cauchy sequence in V . For each v ∈ V , we have

|fn(v) − fm(v)| = |hfn − fm, vi| ≤ kfn − fmkV 0 kvkV so (fn(v)) is Cauchy in R. For our purpose, we will define the functional

f : V → withf(v) = lim |fn(v)| ≤ lim kfnkV 0 kvkV R n→∞ n→∞

Since (fn) is Cauchy, then it is bounded and lim kfnkV 0 < ∞. n→∞

0 We now need to verify that (fn) converges to f in V . For each ε > 0 there exists N ∈ N such that if m, n > N then kfn − fmk ≤ ε. Thus for each v ∈ V , |fn(v) − fm(v)| ≤ εkvkV . Fix n and let m → ∞ to get |fn(v) − f(v)| ≤ εkvkV and thus kf − fnkV 0 → 0 when n → ∞

A fundamental in functional analysis is to investigate dual spaces of functions. Therefore we will give a brief overview of the basic dual spaces. For this purpose, we will need to recall . In fact, isomorphisms T are merely bijective linear operators that are norm preserving, that is kT (x)k = kxk. Therefore, if there exists a linear isomorphism between two vector spaces, then one is the dual of another. The proofs are tricky, so we will leave them to the curiosity of the reader.

n n (a) K is the dual of K .

(b) The of `1 is `∞.

(c) The dual space of `p is `q where q is the conjugate of p.

Definition. The bidual or second dual of a Banach space V is the dual of V 0. It is denoted V 00. V is reflexive if V 00 and V are isomorphic.

29 Exercises

1. Let x ∈ C[a, b] and t fixed in [a, b]. Prove that the functional f(x) = x(t) is linear and bounded with norm kfk = 1.

2. Show that the following functionals on C[a, b] are linear and bounded: Z b (a) f(x) = x(t) y◦(t) dt with y◦ ∈ C[a, b] a (b) g(x) = αx(a) + βx(b) with α, β ∈ K 3. The space C0[a, b] is the normed space of all continuously differentiable functions on [a, b] with the norm kxk = max |x(t)| + max |x0(t)|. t∈[a,b] t∈[a,b] (a) Verify that k.k is a norm. a + b (b) Let f(x) = x0 . Prove that f is a bounded linear functional on C0[a, b]. 2

4. Let P (x) ∈ R[X] be the set of polynomials with real coefficients. Define kP k = sup |P (x)| x∈[0,1]

(a) Prove that R[X] is a vector space. (b) Prove that k · k is a norm on R[X]. (c) Let c > 0 and L : R[X] → R be the functional defined by L(P ) = P (c). Prove that L is linear and continuous if and only if c ∈ [0, 1]. Calculate its norm. 0 (d) Is the map P → P continuous in R[X]?

5. Show that any subspace of a reflexive space is reflexive.

6. Let T : V → W be a continuous linear map between two topological vector spaces. We define the continuous transpose T 0 : W 0 → V 0 by T 0(ϕ) = ϕ ◦ T, ϕ ∈ W 0 Show that the resulting functional T 0(ϕ) is in V 0.

7*. Equip C[0, π] with the infinity norm and let ϕ ∈ C[0, π] be fixed. Consider the map from Z π C[0, π] to R defined by T (f) = f(x) ϕ(x) dx 0 (a) Show that T is a continuous linear functional. (b) Find kT k when ϕ > 0 (c) Find kT k when ϕ(x) = cos(x)

8*. Equip N = C[0, 1] and N 0 = C[0, 1] with the respective norms kfk = sup |f(x)| and x∈[0,1] Z 1 Z 1 kgk = |g(x)| dx where f ∈ N and g ∈ N 0. Prove that ϕ(f, g) = f(x) g(x) dx is 0 0 bilinear and calculate its norm.

30 Chapter 3

Hilbert Spaces

3.1 Inner Product Spaces and Orthogonality

We start this section with some basic definitions that characterize maps or forms with range in K, i.e. functionals.

Definitions.

(a) Let V and W be two vector spaces. A form f : V × W → K is bilinear if f(v, w) is linear in both arguments, that is:

• f(u + v, w) = f(u, w) + f(v, w) • f(u, w + v) = f(u, v) + f(u, w) • f(αv, w) = αf(v, w) = f(v, αw)

(b) A bilinear form f : V × V → K is symmetric if f(v1, v2) = f(v2, v1) for all v1, v2 ∈ V .

(c) A functional f : V → K is semi linear if f(c1v1 +c2v2) = c1f(v1)+c2f(v2) for all v1, v2 ∈ V and c1, c2 ∈ K.

(d) f : V × W → K is sesquilinear if it is semi linear in its first argument and linear in its second argument.

(e) f : V × V → K is hermitian if it is linear in its second argument and f(v1, v2) = f(v2, v1) for all v1, v2 ∈ V . This implies that f is sesquilinear.

(f) A bilinear hermitian map is positive definite if f(v, v) > 0 for all v 6= 0 in V

Some things need to be clarified. First, if f is hermitian, then f(v, v) ∈ R for all v ∈ V . Second, if K = R, semi linear coincides with linear and sesquilinear is just bilinear.

31 Definition. Let V be a vector space. A functional V × V → K is an inner product if it is hermitian and positive definite. We will generally denote the inner product of two vectors v1, v2 by hv1, v2i. A vector space equipped with an inner product is a pre-Hilbert or an .

Theorem. Let V be an inner product space. Then for all v, w ∈ V , |hv, wi|2 ≤ hv, vihw, wi. Equality holds if and only if v and w are linearly dependent. This is the Cauchy-Schwarz inequality for norms.

Proof. Let v, w ∈ V and c ∈ K and <(z) = <(x + iy) = x for all complex quantities z. Then

hv + cw, v + cwi = hv, vi + chv, wi + chw, vi + |c|2hw, wi = hv, wi + 2<(c, hv, wi) + |c|2hw, wi

Without loss of generality, choose c ∈ K such that chv, wi ∈ R and let t = |c|. An inner product is positive definite, so we get hv, vi + 2t|hv, wi| + t2hw, wi ≥ 0 which means that the discriminant of this second degree equation is non-positive, i.e 4|hv, wi|2 − 4hv, vihw, wi ≤ 0 which gives us the desired result. Equality holds if and only if the previous second degree equation has a root, which means that there is some c ∈ K such that hv + cw, v + cwi = 0, so v + cw = 0.

Let V be a vector space equipped with an inner product h·, ·i. Then we can define a norm on V that is induced by the inner product by kvk = phv, vi for all v ∈ V . See this section’s exercises for more details on the reason why this is valid. The Cauchy-Schwarz inequality can be therefore written as |hv, wi| ≤ kvk kwk.

Definition. Let x and y be elements of an inner product space V . We say that they are orthogonal if hx, yi = 0 and we write x ⊥ y. A sequence (xi)i∈I in V forms an orthogonal system if hxn, xmi = 0 for all n 6= m ∈ I. The system is orthonormal if kxik = 1 for all  1 if i = j i ∈ I. We define the Kronecker delta of ui and uj by δij = if u ⊥ j. 0 if i 6= j

Lemma. Let (xi) be an orthogonal system in an inner product space. Then

2 2 2 2 kx1 + x2 + ··· + xnk = kx1k + kx2k + ··· + kxnk

Proof. Suppose x ⊥ y. Then hx, yi = hy, xi = 0 and

kx + yk2 = kxk2 + hx, yi + hy, xi + kyk2 = kxk2 + kyk2.

The rest follows by induction.

Theorem. Let (xi) be an orthonormal system in an inner product space V . ∞ X 2 2 Then for each y ∈ V , hxn, yi ≤ kyk . This is known as the Bessel-Parseval inequality. n=1

32 N X Proof. Let yN = hxi, yixi. Since (xi) is an orthonormal (thus orthogonal) system, we have i=1 from the previous lemma and the fact that

N N 2 X 2 X 2 kyN k = khxi, yixik = hxi, yi i=1 i=1

N N N 2 X 2 X X 2 the following: kxN k = |hxi, xi| . But hxN , yi = hhxi, yixi, yi = hxi, yihxi, yi = kxN k i=1 i=1 i=1 so hyN , y − yN i = 0. Also, by the previous lemma,

N 2 2 2 2 2 X 2 kyk = ky − yN + yN k = ky − yN k + kyN k = ky − yN k + |hyi, yi| i=1 giving us the desired result.

Note that in the previous proof, the sequence (yN )N≥1 is Cauchy. In fact, if M > N,

M 2 X 2 kyN − yM k = |hxi, xi| i=N+1 converges to 0 as N,M → ∞. However the sequence doesn’t necessarily converge. The space needs to be complete in order for this to happen.

We now consider relations of orthogonality between sets. Recall first that if B is a subset of K then for some N ∈ N ( N ) X span(B) = cixi | ci ∈ K, xi ∈ B, i = 1 ...N i=1 Definition. Let B ⊂ V , inner product space. The orthogonal of B is the set

B⊥ = {x ∈ V | ∀b ∈ B hb, xi = 0}

Theorem. B⊥ is a closed subspace of V , and B ∩ B⊥ ⊂ {0}.

Proof. Let b ∈ V . Then b⊥ = {b}⊥ is, by definition, the null space (kernel, denoted by ker) of the functional x 7→ hb, xi (see this section’s exercises). One can easily show that the kernel is a closed vector space, therefore \ B⊥ = b⊥ b∈B which is the intersection of closed vector spaces and thus, a closed vector space.

If x ∈ B ∩ B⊥ then x is orthogonal to itself and thus kxk2 = hx, xi = 0.

33 Definition. Let T ⊂ V , inner product space. Then T is total if span(T ) is dense in V .

If T ⊂ V and x ∈ T ⊥, then for all t ∈ T

kx − tk2 = kxk2 + ktk2, kxk ≤ kx − tk ≤ inf{kx − tk; t ∈ span(T )} = dist (x, T )

Thus if x 6= 0, its distance to span(T ) is strictly positive, so span(T ) cannot be dense. Conse- quently, the following holds:

Theorem. If T ⊂ V where V is an inner product space and T is total, then T ⊥ = {0}

The converse is not true in general. As a counterexample, consider C[0, 1] equipped with the Z 1  Z 0 Z 1  inner product hf, gi = f(x) g(x) dx and T = f ∈ E f(x) dx = f(x) dx . We can −1 −1 0 verify that T ⊥ = {0} but T is not dense.

Exercises

1. Let V be a pre- and fix a ∈ V . Show that x → ha, xi is a linear functional and determine its norm. m Y 2. Let V1, ··· ,Vm be inner product spaces. Find an inner product on Vi and prove that i=1 your suggestion is valid.

3. Verify the following identities in an inner product space:

Parallelogram law: 2kxk2 + 2kyk2 = kx + yk2 + kx − yk2 Polarization identity: 4hx, yi = kx + yk2 − kx − yk2 + ikx + iyk2 − ikx − iyk2

4. Verify that the following spaces are pre-Hilbert: Z b (a) C[a, b] with hf, gi = f(t) g(t) dt ∀f, g ∈ C[a, b] a ∞ 2 X 2 (b) ` with ha, bi = an bn ∀a, b ∈ ` n=0 5. Let V be a finite-dimensional inner product space over K. Given any vectors u, v ∈ V , prove that hu, vi = 0 if and only if kuk ≤ ku + αvk for every α ∈ K. ⊥ 6*. Let B be an inner product space. Show that B⊥ = span(B)⊥ = span(B) .

34 3.2 Hilbert Spaces

Definition. A Hilbert space H is a complete inner product space.

Completeness here relates to the distance function d(x, y) = kx−yk , where the norm is induced by the inner product, kxk = phx, xi

Examples. n n X (a) R is a Hilbert space with the inner product hx, yi = xi yi (the regular dot product). One can easily verify that this inner product induced the 2-norm which generates the Euclidean metric. We had already proved in Chapter 1 that this metric space is complete. In addition, orthogonality agrees with regular perpendicularity.

n (b) When we move to C we stick to the same inner product as above but have to take the com- plex conjugate of the components of one of the vectors since an inner product is Hermitian. 2n This, as a normed space, is the same as R (thus complete). ∞ 2 X 2 (c) The sequence space ` is Hilbert with the inner product hx, yi = xi yi where x, y ∈ ` . This sequence converges and can be proven using the Cauchy Schwarz inequality. We had ∞ !1/2 X 2 already proved that this space is complete. The norm is thus kxk = |xi| . i=1 (d) The space `p with p 6= 2 is a Banach space that is not Hilbert since it is not an inner product space (the `p norm cannot be induced from an inner product). This is because the regarded norm does not satisfy the parallelogram identity. Choose x = (1, 1, 0, 0,... ) and y = (1, −1, 0, 0,... ) to see this.

The following results are valid for inner product spaces and not just Hilbert spaces. We choose to mention them in this section due to their use in the exercises.

Lemma. In any inner product space, we have the Schwarz inequality |hx, yi| ≤ kxk kyk and the Triangle inequality kx + yk ≤ kxk + kyk.

Proof. We choose to prove the Schwarz inequality. If x or y is the zero vector then the equality holds. If not, let hy, xi hx, yi α = = hx, yi hx, yi and evaluate kx − αyk to get |hx, yi|2 0 ≤ kx − αyk = kxk2 − kyk2 which gives us the desired result. Note that equality holds if kx − αyk = 0 which shows linear dependence.

35 A major result that requires both of these inequalities is that of the continuity of the inner product. The proof is left as an exercise, but the statement is used to prove that any inner product space can be completed, with the completion being a Hilbert space. We first need to state a definition.

Definition. An isomorphism between two inner product spaces is a bijective linear operator that preserves the inner product.

Theorem. Let X be an inner product space. Then there exists a Hilbert space H for X with an isomorphism A : X → W ⊂ H where W is dense in H.

Example. Recall from the previous chapter that the vector space of all continuous functions C[a, b] is a normed space with the norm

Z b 1/2 kxk = |x(t)|2 dt a This is not a Banach space. It can however be completed and its completion is denoted L2[a, b] which is Banach. In general, for any p ≥ 1 the Banach space Lp[a, b] is the completion of C[a, b] equipped with the p-norm. This norm can be obtained from the inner product Z b hx, yi = x(t)y(t) dt a This last theorem tells us that the inner product can be extended from C[a, b] to L2[a, b], and that L2[a, b] is a Hilbert space. Note that in relation to Lebesgue’s measure theory, this space can be constructed by using Lebesgue measurable functions such that the Lebesgue integral of |x|p exists and is finite on [a, b]. Lp[a, b] would be the equivalence class of those functions where x is equivalent to y if the Lebesgue integral of |x − y|p over [a, b] is zero.

Definition. A subspace Y of an inner product space X is a vector subspace of X taken with the inner product on X restricted to Y × Y . If X is Hilbert, Y doesn’t need to be.

Theorem. Let S be a closed subspace of an Hilbert space H. Then H = S ⊕ S⊥ and for each ⊥ v ∈ H the projection Psv of v on S along S is the best approximation of v in S and

kv − Psvk = d(v, S) = inf kv − wk w∈S

Proof. For each v ∈ H there exists v1 ∈ S such that kv − v1k = inf kv − wk = d since S is w∈S closed. In fact, by our definition of d, there exists a sequence wi ∈ S such that kv − wik → d as i → ∞. Let ε > 0 and let N ∈ N such that kv − wik ≤ d + ε if i > I. For all i, j ≥ N, the gives us

2 2 2 2 2 k(v − wi) − (v − wj)k + k(v − wi) + (v − wj)k = 2kv − wik + 2kv − wjk ≤ 4(d + ε)

36 Thus 1 kw − w k2 ≤ 4(d + ε)2 − 4kv − (w + w )k2 ≤ 4(d + ε)2 − 4d2 = 4ε(2d + ε) j i 2 i j The sequence wi is thus Cauchy in H. Let v1 ∈ S be its limit so that kv−v1k = lim kv−wik = d. i→∞

⊥ + 2 We now need to show that v − v1 ∈ S . For each u ∈ S and c ∈ R , we have d ≤ kv − v1 ± c cuk2 = kv − v k2 ± 2c

⊥ ⊥ We can now decompose v = v1 +(v −v1) with v1 ∈ S and v −v1 ∈ S . In addition, if w ∈ S ∩S 2 ⊥ then hw, wi = kwk = 0 so S and S are algebraic supplements in H. Since v1 = Psv, we get by 2 2 2 2 2 the Pythagorean theorem kvk1 + kv − v1k = kPsvk + k(I − Ps)vk = kvk . Thus kPsvk ≤ kvk 2 and k(I − Ps)vk ≤ kvk, and so the projectors Ps and (I − Ps) are bounded and so the direct sum H = S ⊕ S⊥ is topological.

Exercises 1. Show that the space C[a, b] equipped with its maximum norm is not a Hilbert space.

2. Prove the triangle equality in inner product spaces. When does equality hold?

3. (a) Prove that if xn → x and yn → y then hxn, yni → hx, yi (this proves the continuity of the inner product). (b) Let f be a sesquilinear form from E × E to K. Show that it is continuous if and only if it is bounded (there exists a C > 0 such that for all x, y ∈ E |f(x, y) ≤ Ckxkkyk )

4. Let Y be a subspace of a Hilbert space H. Prove the following theorems

(a) Y is complete ⇐⇒ Y is closed in H. (b) Y is finite dimensional ⇒ Y is complete. (c) H is separable ⇒ Y is separable. Does this hold if H is pre-Hilbert?

5. Let V = C[a, b] and V1 = (V, k · k∞), V2 = (V, k · k2). Show that the identity map from X1

onto X2 is continuous.

6. Let V be a subspace of a Hilbert space H. Show that V ⊂ (V ⊥)⊥. Let P be the orthogonal projection onto V . Show that for every v ∈ (V ⊥)⊥ we have h(I − P)v, vi = 0 then deduce that V = (V ⊥)⊥.

7. Prove the following properties (V1 and V2 are subspaces of an Hilbert space V )

⊥ ⊥ (a) V1 ⊂ V2 ⇒ V2 ⊂ V1 ⊥ ⊥ ⊥ (b)( V1 + V2) = V1 ∩ V2 ⊥ ⊥ ⊥ (c)( V1 ∩ V2) = V1 + V2

37 3.3 Orthonormal Sequences and Bases

We start by referring students to section 3.1 for definitions about orthogonality and orthonor- mality. We will list some examples of orthonormal sequences of usual spaces.

n Examples. (a) In R , the n unit vectors {ei} for i = 1 . . . n where ei denotes the vector with a 1 in the ith coordinate and 0’s elsewhere.

2 (b) In ` , an orthonormal sequence is (en) where en = (δnj) has the nth element 1 and all other zero.

(c) Let X be the inner product space of all real valued continuous functions on [0, 2π] with Z 2π the inner product hx, yi = x(t)y(t) dt. An orthogonal sequence in X is (un) where 0 un(t) = cos nt or un(t) = sin nt. The corresponding orthonormal sequences are necessary for Fourier analysis.

Recall from linear algebra the Gram-Schmidt algorithm which is used to make a set of vectors n in R orthonormal. The resulting set spans the same subspace as the original one. We define the projection operator by hv, ui proj (v) = u u hu, ui

The Gram-Schmidt process then works as follows to get a sequence (ui) of orthogonal vectors, and then the normalized vectors (ei) form an orthonormal set. u1 u1 = v1, e1 = ku1k u2 u2 = v2 − proju1 (v2), e2 = ku2k u3 u3 = v3 − proju1 (v3) − proju2 (v3), e3 = ku3k u4 u4 = v4 − proju1 (v4) − proju2 (v4) − proju3 (v4), e4 = ku4k . . . . k−1 X uk u = v − proj (v ), e = . k k uj k k ku k j=1 k This algorithm can be adopted for any inner product space V to obtain an orthonormal sequence

(ei) from another sequence (vi) in V if for any i, vi+1 is linearly independent from v1 . . . vi.

Definition. Let E = (ei) be an orthonormal system in V and v ∈ V . Then hei, vi ∈ K is called the Fourier coefficient of v in E.

Theorem. Let E = (ei) be an orthonormal system in an inner product space V and let v ∈ V . n X Then for each n ∈ N, the best approximation of v of the form ciei is given by ci = hei, vi. i=1

38

n X Proof. For each i ∈ N,the error v − aiei is minimal when ai = ci, since i=1 2 n n n n n X 2 X X 2 2 X 2 X 2 v − aiei = kvk − 2< aici + |ai| = kvk − |ai − ci| − |ci| i=1 i=1 i=1 i=1 i=1

n X The sum hei, viei gives the best approximation of v as n gets larger, so it is natural to study i=1 ∞ X the convergence of expressions of the form aiei with ai ∈ K. i=1

Theorem. Let E = (ei) be an orthonormal system in a Hilbert space H and (ai) ∈ K. Then ∞ ∞ X X 2 the sequence aiei converges in H if and only if the sequence |ai| converges in K. This i=1 i=1 is known as the Riesz-Fisher theorem. ∞ n+k X X Proof. Since H is complete, the sequence aiei converges in H if and only if aiei converges i=1 i=n ∞ X 2 to 0 ∈ H as n → ∞ for each k ≥ 0. In addition, since K is complete, the sequence |ai| i=1 n+k X 2 converges in K if and only if |ai| converges to 0 ∈ K as n → ∞ for each k ≥ 0. Both i=n 2 n+k n+k X X 2 convergences are equivalent because aiei = |ai| since ei is orthonormal. i=n i=n This theorem, along with Bessel-Parseval’s inequality from the previous section, guarantee that the considered sequence converges in a Hilbert space. However, to guarantee that this limit converges to v, we need to introduce an additional notion for orthonormal sequences.

Definition. Let V be an inner product space. An orthonormal or orthogonal system (vi) in V is complete if the only v ∈ V satisfying hv, vii = 0 for each i is the zero vector. If V is Hilbert and (vi) is orthonormal, then (vi) is a Hilbert basis for V .

Theorem. Every separable Hilbert space has a Hilbert basis.

0 0 Proof. Let V = {v1, v2 ... } be a countable dense subset of H and {v1, v2 ... } the subset of V obtained by removing the vectors vi ∈ V that are linearly dependent of {v1, v2 . . . vi−1}. 0 0 0 Applying the Gram-Schmidt procedure to {v1, v2 . . . vn} we get an orthonormal sequence (wi) 0 0 0 in H. Let En be the subspace of H spanned by {v1, v2 . . . vn} and thus by {w1, w2 . . . wn}. Then [ En ⊂ En+1 for each n ∈ N and S = En is dense in H. Then by exercise 6 of section 1.1, n∈N ⊥⊥ ⊥ S = H and thus S = 0. Whereupon, the sequence (wi) is complete.

39 Theorem. Let H be a Hilbert space and E = (ei) an orthogonal sequence in H. Then for all v, w ∈ H, the following are equivalent: (a) E is complete ∞ X (b) v = hei, viei for all v ∈ H i=1 ∞ X (c) hv, wi = hv, eiihei, wi for all v, w ∈ H i=1 ∞ X 2 2 Proof. (a) ⇒ (b): By Bessel’s inequality, |hei, vi| ≤ kvk . Consequently, by the Riesz- i=1 ∞ X Fisher theorem, the series hei, viei converges in H. Since the Fourier coefficients of the i=1 ∞ ∞ X X term v − hei, viei with respect to E are all zero and E is complete, we get v = hei, viei i=1 i=1 n n X X (b) ⇒ (c): Since v = lim hei, viei and w = lim hei, wiei, and using the continuity of n→∞ n→∞ i=1 j=1 the inner product:

* n n + n ∞ X X X X hv, wi = lim hei, viei, hei, wiei = lim hei, vihej, wiδij = hv, eiihej, wi n→∞ n→∞ i=1 j=1 i,j=1 i,j=1

∞ 2 X 2 (c)⇒ (a): Choose w = v and apply (c) to get kvk = |hei, vi| . Thus if hei, vi = 0 for each i=1 2 i ∈ N, kvk = 0 and v = 0, which means that E is complete.

2 Corollary. Every separable Hilbert space over K is isometric to ` (K).

Proof. Let H be a separable Hilbert space over K and let (vi) be a Hilbert basis of H. Then ∞ X using the previous theorem, every element v ∈ H can be written as v = hei, viei and we i=1 ∞ 2 X 2 2 hence get kvk = |hei, vi| < ∞. Consequently, we can define the linear map Φ : H → ` (K) i=1 by

Φ(v) = (hei, vi)i∈N 2 for each v ∈ H. By Riesz-Fisher’s theorem, each element a = (ai) ∈ ` (K) will allow us to define ∞ X an element v ∈ H by v = aiei. Thus Φ is an invertible. Finally, use part (c) of the previous i=1 theorem to verify that Φ is an isometry.

40 Exercises

1. Prove that an orthonormal set is linearly independent.

2. Let X be the inner product space of all real valued continuous functions on [0, 2π].

(a) Prove that an orthogonal sequence in X is (un) where un(t) = cos nt or un(t) = sin nt. (b) Normalize the sequences in (a).

(c) Prove that um ⊥ un for all m and n.

3. Let (ek) be an orthonormal sequence in a Hilbert space H. Prove that for any x, y ∈ H,

∞ X |hx, ekihy, eki| ≤ kxkkyk k=1

4. (a) If (xi) is a sequence in an inner product space X such that the series kx1k+kx2k+...

converges, show that (sn) is a Cauchy sequence, where sn = x1 + ··· + xn. P P (b) Show that if X is Hilbert and kxjk converges, then xj converges.

5*. Let (ek) be orthonormal in an Hilbert space H, and let M = span(ek). Show that for any ∞ X x ∈ H, x ∈ M if and only if x = hx, ekiek. i=1

3.4 Representation Theorems

In this section H refers to an Hilbert space and H0 its dual, the set of linear functionals on H. We start with the representation theorem of linear functional by vectors:

The goal of this section is to introduce students to major problems in functional analysis and how to find solutions to those problems. Sometimes the mere existence of a solution turns out to be enough. A family of problems is known as the nonsymmetric variational problems, and is formulated as follows: Let H be a Hilbert space and V a closed subspace of H. Define a(·, ·) to be a continuous bilinear form on V, not necessarily symmetric. Then given F ∈ V 0 the problem asks one to find u ∈ V such that a(u, v) = F (v) for all v ∈ V . We will prove this via the Lax-Milgram theorem, but we first need to provide a few tools.

Theorem (Riesz’ representation theorem). Let ` ∈ H0 be a linear functional. Then there exists a unique vector a ∈ H such that for all v ∈ H, `(v) = ha, vi. In addition, kakH = k`kH0 .

Proof. Let’s start by verifying that a ∈ H is necessarily unique. In fact, if a and a0 ∈ H are such that ha, vi = ha0, vi for all v ∈ H then ha − a0, vi = 0 and thus a − a0 ∈ H⊥ = {0}. Now let’s prove the existence of such a vector. If l = 0 then we can choose a = 0. If ` 6= 0 then by

41 the previous chapter, S = `−1(0) = {~v ∈ H | `(~v) = 0} is a subspace of codimension 1 in H. Since H = S ⊕ S⊥, S⊥ has dimension 1. Let b be a nonzero vector of S⊥, so `(b) 6= 0. Let

|`(b)| a = b ∈ S⊥ kbk2

⊥ Then for each v ∈ S, ha, vi = 0 = `(v). If v ∈ S then v = λb for some λ ∈ K, and thus `(b) ha, vi = λkbk2 = `(λb) = `(v) kbk2

Finally, we use the Cauchy Schwarz inequality to prove the final proposition in the theorem, since equality is guaranteed when a and v are linearly independent:

|`(v)| ha, vi k`kH0 = sup = sup = kakH v∈H kvkH v∈H kvkH

Recall that L(H) denotes the set of bounded operators on a Hilbert space H. Then a consequence of Riesz’ representation theorem is the following:

Theorem. Let A ∈ L(H). Then there exists a unique operator A∗ such that for all (u, v) in H × H, hAu, vi = hu, A∗vi.

In this case, A is said to be a self adjoint operator and this will be covered in detail in the next section.

Definition. A self-adjoint operator A : H → H, where H is an Hilbert space, is called coercive if there exists a constant c > 0 such that hAx, xi ≥ ckxk2 for all x in H. Similarly, A bilinear 2 form a : H ×H → R is called coercive if there exists a constant c > 0 such that a(x, x) ≥ ckxk for all x in H.

It follows from the Riesz representation theorem that any symmetric continuous and coercive bilinear form a has the representation a(x, y) = hAx, yi for some self-adjoint operator A : H → H, which then turns out to be a coercive operator. Also, given a coercive self-adjoint operator A, the bilinear form a defined as above is coercive.

Lemma (Contraction Mapping Principle). Let V be a Banach space. Fix M ∈ [0, 1] and let

T : V → V such that for all v1, v2 ∈ V

kT (v1) − T (v2)k ≤ Mkv1 − v2k

Then there exists a unique u ∈ V such that u = T (u). In other words, the mapping T has a unique fixed point u.

42 Theorem (Lax Milgram). Let V be a Hilbert space, b a continuous bilinear coercive form and F continuous linear functional ∈ V 0. Then there exists a unique vector u ∈ V such that for all u ∈ V b(u, v) = F (v).

Proof. For any u ∈ V define the functional (A(u))(v) = b(u, v) for all v ∈ V . A(u) is linear and continuous. Then (A(u))(v) k(A(u))(v)kV 0 = sup ≤ Ckuk < ∞ v6=0 kvk where C is the constant of continuity. Thus A(u) ∈ V 0. In addition, the mapping u → A(u) is a linear map from V to V 0. By Riesz’ representation theorem, for all ϕ ∈ V 0 and v ∈ V there exists a unique τ ∈ V such that ϕ(τ) = (τ, v).

We must find a unique vector u such that (A(u))(v) = F (v), or A(u) = f in V 0, which means that τAu = τF in V since τ : V 0 → V is injective. We now use the Contraction Mapping Principle (CMP) to find ρ 6= 0 such that the mapping T : V → V is a contraction, with T mapping v ∈ V to v − ρ(τAv − τF ). But if T is a contraction, then by the CMP there exists a unique u ∈ V with T (u) = u − ρ(τAu − τF ) = u

Thus ρ(τAu − τF ) = 0 or τAu = τF . We now just need to show that such a ρ exists.

For all v1, v2 ∈ V let v = v1 − v2, then

2 2 kT (v1) − T (v2)k = kv1 − v2 − ρ(τAv1 − τAv2)k = kv − ρ(τAv)k2 = kvk2 − 2ρhτ(Av), vi + ρ2kτAvk2 = kvk2 − 2ρAv(v) + ρ2Av(τAv) = kvk2 − 2ρa(v, v) + ρ2a(v, τAv) ≤ kvk2 − 2ραkvk2 + ρ2Ckvk · kτAvk since A is coercive and continuous ≤ (1 − 2ρα + ρ2C2)kvk2 since A is bounded and kτAvk = kAvk ≤ Ckvk 2 2 2 ≤ (1 − 2ρα + ρ C )kv1 − v2k 2 2 ≤ M kv1 − v2k

 2α  So we need 1 − 2ρα + ρ2C2 < 1 for some ρ. If we choose ρ ∈ 0, then M < 1 , concluding C2 the proof.

Corollary. The nonsymmetric variational problem has a unique solution.

43 Exercises

n 1. Show that any linear functional f on R can be represented by a dot product, that is, for n n X n all x ∈ R , f(x) = xiyi for some fixed y ∈ R . i=1 2. (a) Use Riesz’ representation theorem to prove that if f is a continuous sesquilinear form from H × H to K then there exists a A ∈ L(H) such that for all (u, v) ∈ H × H, f(u, v) = hAu, vi. (b) Recall from section 3.1’s exercises that f is continuous if and only if there exists a C > 0 such that for all x, y ∈ E |f(x, y)| ≤ Ckxkkyk. Show that in this case,

kAkL(H) ≤ C.

3. (a) Let z be a fixed element of an inner product space X. Show that f(x) = hx, zi defines a bounded linear functional f on X of norm kzk. (b) Let X0 be the dual of X. If the mapping X → X0 given by z → f is surjective, show that X must be Hilbert.

4. Prove the Contraction Mapping Principle lemma from this section.

5*. Suppose that conditions for nonsymmetric variational problem hold and that u solves

a(u, v) = F (v) for all v ∈ V . For the finite element variational problem a(uh, v) = F (v)

for all v ∈ Vh, prove that we have C ku − uhkV ≤ min ku − vkV α v∈Vh

where C is the continuity constant and α is the coercivity constant of a(·, ·).

3.5 Special Operators

Definition. Let H1 and H2 be Hilbert spaces and A ∈ L(H1,H2). The adjoint operator is ∗ ∗ the A : H2 → H1 characterized by hA v2, v1iH1 = hv2, Av1iH2 for all v1 ∈ H1 and v2 ∈ H2.

Theorem. Let A ∈ L(H1,H2) and B ∈ L(H2,H3). Then:

(a) A∗ is bounded and kA∗k = kAk

(b)( A∗)∗ = A

(c) ker A = (im A∗)⊥ and im A = (ker A∗)⊥

(d)( B ◦ A)∗ = A∗ ◦ B∗

44 (e) if A is invertible, then (A−1)∗ = (A∗)−1

Proof. (a) Recall that for v1 ∈ H1 and v2 ∈ H2, hv , Av i kAk = sup 2 1 v1,v2 kv1kkv2k

By the Cauchy-Schwarz inequality, this supremum does not exceed kAk . Note that there kaunk also exists some sequence un ∈ H1 such that → kAk. Let v1 = un and v2 = Aun to kunk see that the supremum coincides with kAk. Thus

hv , Av i hA∗v , v i kAk = sup 2 1 = sup 2 1 = kA∗k v1,v2 kv1kkv2k v1,v2 kv1kkv2k

(b) For each v1 ∈ H1 and v2 ∈ H2, we get

∗ ∗ ∗ hv2, Av1iH2 = hA v2, v1iH1 = hv2, (A ) v1iH2

∗ (c) v1 ∈ ker A if and only if hAv1, v2iH2 = hv1,A v2iH1 = 0 for all v2 ∈ H2, which means that ∗ ⊥ ∗ v1 ∈ (im A ) . We get the second relation by using A instead of A and part (b).

(d) For each v1 ∈ H1 and v3 ∈ H3, we have on one hand

∗ ∗ ∗ hv3, (B ◦ A)v1iH3 = hB v3, Av1iH2 = h(A ◦ B )v3, v1iH1

and on the other hand

∗ hv3, (B ◦ A)v1iH3 = h(B ◦ A) v3, v1iH1

Thus A∗ ◦ B∗ = (B ◦ A)∗.

(e) By replacing B with A−1 in part (d) we get I∗ = I = A∗ ◦ (A−1)∗. By exchanging the roles of A and B we get I = (A−1)∗ ◦ A∗. Consequently, A∗’s inverse is (A−1)∗.

Definition. Let H be a Hilbert space. The operator A ∈ L(H) is self-adjoint if A∗ = A.

Theorem. Let H be a Hilbert space over K and A ∈ L(H). Then

(a) If A is self-adjoint then hAv, vi ∈ R for all v ∈ H

(b) If K = C and hAv, vi ∈ R for each v ∈ H then A is self-adjoint.

Proof. (a) If A is self-adjoint then for each v ∈ H

hAv, vi = hv, A∗vi = hv, Avi = hAv, vi

45 (b) If hAv, vi ∈ R for each v ∈ H then

hAv, vi = hv, Avi = hv, Avi = hA∗v, vi

∗ so h(A − A )v, vi = 0. Let v = v1 + cv2 with v1, v2 ∈ H and c ∈ K then

∗ ∗ ∗ 2 ∗ h(A − A )v1, v1i + ch(A − A )v1, v2i + ch(A − A )v2, v1i + |c| h(A − A )v2, v2i = 0

The first and the last terms of this sum are zero. If c = 1, we get

∗ ∗ h(A − A )v1, v2i + h(A − A )v2, v1i = 0

and if c = i we get ∗ ∗ h(A − A )v1, v2i − h(A − A )v2, v1i = 0

∗ ∗ so h(A − A )v1, v2i = 0 and A = A

Definition. Let H be an Hilbert space and U ∈ L(H). U is called a unitary operator if U is invertible and U −1 = U ∗

Let U, V ∈ L(H) be unitary operators. Then kUk = 1, U −1,U ∗, and UV are unitary. The proofs of these statements are very simple and are left to the student’s curiosities.

Recall that an operator is isometric if it preserves norms. A unitary operator is an isometry, but a linear isometry is not necessarily unitary. An example of a unitary operator is the right shift 2 ∗ operator R ∈ L(` (K)). Then R is isometric and its adjoint R is the left shift operator L. Thus R∗R = LR = I. However, R is not unitary since R is not surjective, and thus RR∗ = RL 6= I.

Definition. Let H be an Hilbert space and N ∈ L(H). N is called a if N ∗N = NN ∗.

Clearly, a self-adjoint or unitary operator is normal. However, a normal operator is not neces- sarily self-adjoint or unitary. Take for instance the operator T = 2iI. T ∗ = −2iI = −T so T is neither self-adjoint nor unitary. However, T is normal since T ∗T = TT ∗ = −T 2.

Definition. An operator P ∈ L(H) in a Hilbert space H is called an orthogonal projector if ker P ⊥ im P .

Theorem. A linear projector P ∈ L(H) is orthogonal if and only if P is self-adjoint.

Proof. Let P be orthogonal. Then for each u, v ∈ H,

hu, P vi = hP u + (I − P )u, P vi = hP u, P vi

46 We can similarly get that hP u, vi = hP u, P vi. So P is self-adjoint. Now Let P be self-adjoint. Then for each u, v ∈ H, hP u, (I − P )vi = hu, P (I − P )vi = 0 Thus ker P ⊥ im P and P is orthogonal.

Exercises

1. Let H1 and H2 be Hilbert spaces and S : H1 → H2 and T : H1 → H2 bounded linear operators. Prove the following propositions:

∗ ∗ ∗ ∗ ∗ (a)( S + T ) = S + T and (cT ) = cT for all c ∈ K. (b) kSS∗k = kS∗Sk = kSk2. (c) If SS∗ = 0 or S∗S = 0 then S = 0.

2. Let P ∈ L(H). Prove that the following are equivalent:

(a) P is a projection; (b)1 − P is a projection; (c) P 2 = P and ker P ⊥ im P ; (d) P 2 = P and P is self adjoint and normal;

3. Let P ∈ L(H) be an orthogonal projector. Prove the following propositions:

(a) P is non-negative (b) If P 6= 0 thenkP k = 1

4. Let V be a closed subspace of a separable Hilbert space H. Let P ∈ L(H) be an orthogonal

projector on V and (ei)i∈V be a complete orthonormal system for V . X (a) Show that P v = hei, viei for all v ∈ H i∈I n (b) If V ∈ H = C is the subspace spanned by f1 = (1, 1,..., 1) and f2 = (1, 0,..., 0), find the matrix associated to the linear projector P . Z n n 2 n 5*. Let K ∈ C(R × R , K) with |K(x, y)| dx dy < ∞. Let f ∈ C(R , K) and define Rn×Rn a linear operator Z n 2 n 2 n A : C(R , K) ⊂ L (R , K) → L (R , K) by (Af)(x) = Kf(x, y)f(y)dy Rn 2 n 2 n (a) Show that A can be extended into a bounded linear map A : L (R , K) → L (R , K) and calculate its norm. ∗ (b) Find the adjoint A of A. (c) Find an example of K such that A is normal but neither self-adjoint nor unitary.

47 3.6 Weak-Weak∗ Topologies and Convergence

So far, normed vectored spaces’ topologies were generated by open balls. We will call those strong topologies. We will define the with less open sets, but we will see that it will have more compact sets than the strong topology.

Definition. The weak topology on a Banach Space X is the coarsest topology such that all 0 functionals f : X → K with f ∈ X (dual of X) are continuous.

This definition will allow us to construct the weak topology. It needs to contain at least all sets −1 0 of the form f (U) where U is open in K and f ∈ X . Then consider all arbitrary unions and finite intersections of those, and we will definitely get the finest topology satisfying the definition. In finite dimensions, the weak topology coincides with the strong topology (see this section’s exercises). In infinite dimensions, the weak topology is coarser than the strong topology.

Recall that a neighborhood of an element x is a set containing an open set that contains x. A neighborhood basis is a collection of neighborhoods such that every open set containing x contains at least one of those neighborhoods.

Definition. A basis of weak neighborhoods of x◦ in a Banach space X is generated by weak 0 open sets of the form {x ∈ X | fi(x − x◦) < ε ∀i ∈ I} where ε > 0,I is finite and fi ∈ X .

We can now distinguish weak and strong convergences. The regular notion of convergence will now be considered strong and denoted →.

0 Definition. A sequence xn in a Banach space converges weakly to x when for all f in X , f(xn) → f(x). This is denoted xn * x .

Evidently, strong converge implies weak convergence, since kf(xn) − f(x)k ≤ kfk kxn − xk.

Theorem. If xn * f and fn * f then fn(xn) → f(x).

Proof. The proof is very straightforward. We can simply write

|fn(xn) − f(x)| ≤ |fn(xn) − f(xn)| + |f(xn) − f(x)| ≤ kfn − fk kxnk + |f(xn) − f(x)| where each of the latter terms converges to zero.

Theorem. If xn * x in a Banach space X, then (kxnk) is bounded.

Proof. xn * x so f(xn) is convergent and thus bounded by some cf that depends on f and not on n. Using the bilinear canonical mapping from the previous section, call it C : X → X00 where X00 0 00 is the dual of X , we can define gn ∈ X by gn(x) = f(xn). Then for all n |gn(f)| = |f(xn)| ≤ cf .

Since the dual space is complete, (kgnk) is bounded. But kgnk = kxnk , concluding our proof.

48 We can now extend the definition of weak topology from X to its dual X0. This will give us an even finer topology .

Definition. The weak∗ topology on a dual space X0 is the finest topology such that the 0 applications f from X to K are continuous.

Definition. The weak∗ topology is separable.

0 Proof. This results from the definitions. If f1 and f2 are two distinct elements in X , there exists 0 some x ∈ X such that f1(x) < f2(x). Let α ∈ (f1(x), f2(x)) and define O1 = {f ∈ X | f(x) < α} 0 ∗ and O2 = {f ∈ X | f(x) < α} as weak open sets separating f1 and f2.

0 ∗ ∗ For all f◦ ∈ X , a basis of weak neighborhoods of f◦ is formed by the weak open sets containing 0 f ∈ X with |f(xi) − f◦(xi)| < ε ∀i ∈ I}.

∗ * Definition. The weak convergence is defined by fn −* f ⇐⇒ ∀x ∈ X fn(x) → f(x)

Strong convergence implies weak convergence which implies weak∗ convergence. If the considered space is reflexive and Banach, then weak and weak-* convergence are equivalent. The proof of this will be left as an exercise to the students, but we will use the result to give some examples on such convergences.

Example. Let X be Hilbert. By the Riesz representation theorem, every bounded linear func- tional on X is of the form hx, ai where a ∈ X. So xn * x in X means that hxn, ai → hx, ai.

Thus, if (en) is a complete orthonormal system, then hen, ai → 0 for all a, because the series X |hen, ai| is summable by Bessel-Parseval’s inequality. So en * 0. However, en 9 0.

∗ Lemma. Let X be a normed space, Q a dense subset of X and (φn) in X (weak-* topology on X). Suppose that kφnk ≤ M and for all q ∈ Q, φn(q) converges. Then φn(x) is convergent for all x ∈ X and the limit φ(x) defined a bounded linear functional on X.

The proof of this lemma is left as an exercise. The idea here is that φn weak-* converges to φ. This also holds for a sequence of operators for a Banach space. This lemma will be used to prove the following theorem:

∗ Theorem. Let X be a separable normed space and (φn) a sequence in X that is bounded in norm. Then there is a subsequence of (φn) that is weak-* convergent.

Proof. Let Q = {q1, q1, ···} be a countable dense subset of the separable normed space X. Then for each qi the sequence φ(qi) is bounded. So there is a subsequence indexed by Nq ⊂ N such that this numbers converge, say lim φn(q) → φ(q).Now suppose that Nq1 ⊃ Nq2 ⊃ · · · . We n∈Nq th can thus construct a new subsequence φnj by taking nj to be the j element of Nqj . Then

φnj (q) → φ(q) for all q ∈ Q. We can now apply the previous lemma to conclude the proof.

49 Exercises

1. Show that in finite dimensional Banach spaces, the weak topology coincides with the strong topology.

2. Let (xn) weakly converge in a Banach space X. Prove that its limit is unique.

* 3. If xn → x and fn −* f, prove that fn(xn) → f(x).

4. Let X be a reflexive Banach space. Prove that a sequence (xn) converges weakly in X if and only if it converges weak-*.

5. Prove the last Lemma of this section.

6. Prove that sequence of functions fn such that kfnk ≤ M is weakly convergent if and only Z b if for every a < b the sequence of integrals fn is convergent. a

7. Let xn ∈ C[a, b] and xn * x ∈ C[a, b]. Show that (xn) converges pointwise on [a, b], that

is, (xn(t)) converges for every t ∈ [a, b].

8. Show that if xn * x◦ then lim kxnk ≥ kx◦k. n→∞

9*.A weak Cauchy sequence is a real or complex normed space X is a sequence (xn) ∈ X 0 such that for every f ∈ X , f(xn) is Cauchy in K. Show that a weak Cauchy sequence is bounded.

10*. A normed space X is said to be weakly complete if each weak Cauchy sequence in X converges weakly in X. Show that X is weakly complete if X is reflexive.

50 Bibliography

[1] Erwin Kreyszig, Introductory Functional Analysis with Applications, Wiley Classics, 1989.

[2] Richard Bass, Real Analysis for Graduate Students, Self published, 2011.

[3] Haim Brezis, Analyse fonctionnelle : Th´eorieet applications, Dunaud, 2005.

[4] Frigyes Riesz and Bela Sz.-Nagy, Functional Analysis, Dover, 1990.

[5] James Munkres, Topology (2nd Edition), Prentice Hall, 2000.

51