Analysis III

Home , Lipschitz continuity, Regulated function, Regulated integral, Step function

Analysis III

Lecturer: Prof. Oleg Zaboronski, Typesetting: David Williams

Term 1, 2015 Contents

I Integration 2

1 The Integral for Step Functions 2 1.1 Deﬁnition of the Integral for Step Functions ...... 2 1.2 Properties of the Integral for Step Functions ...... 4

2 The Integral for Regulated Functions 6 2.1 Deﬁnition of the Integral for Regulated Functions ...... 6 2.2 Properties of the Integral for Regulated Functions ...... 8

3 The Indeﬁnite Integral & Fundamental Theorem of Calculus 10 3.1 The Indeﬁnite Integral ...... 10 3.2 The Fundamental Theorem of Calculus ...... 11

4 Practical Methods of Integration 12 4.1 Integration by Parts ...... 12 4.2 Integration by Substitution ...... 13 4.3 Constructing Convergent Sequences of Step Functions ...... 14

5 The Characterisation of Regulated Functions 15 5.1 Characterisation & Proof ...... 15

6 Improper & Riemann Integrals 17 6.1 Improper Integrals ...... 17 6.2 The Riemann Integral ...... 19

7 Uniform & Pointwise Convergence 21 7.1 Deﬁnitions, Properties and Examples ...... 21 7.2 Uniform Convergence & Integration ...... 25 7.3 Uniform Convergence & Diﬀerentiation ...... 27

8 Functional Series 28 8.1 The Weierstrass M-Test & Other Useful Results ...... 28 8.2 Taylor & Fourier Series ...... 29

II Norms 33

9 Normed Vector Spaces 33 9.1 Deﬁnition & Basic Properties of a Norm ...... 33 9.2 Equivalence & Banach Spaces ...... 35

10 Continuous Maps 40 10.1 Deﬁnition & Basic Properties of Linear Maps ...... 40 10.2 Spaces of Bounded Linear Maps ...... 42

11 Open and Closed Sets of Normed Spaces 43 11.1 Deﬁnition & Basic Properties of Open Sets ...... 43 11.2 Continuity, Closed Sets & Convergence ...... 44 11.3 Lipschitz Continuity, Contraction Mappings & Other Remarks ...... 45

12 The Contraction Mapping Theorem & Applications 46 12.1 Statement and Proof ...... 46 12.2 The Existence and Uniqueness of Solutions to ODEs ...... 47 12.3 The Jacobi Algorithm ...... 50 12.4 The Newton-Raphson Algorithm ...... 51

1 Part I Integration

1 The Integral for Step Functions

1.1 Deﬁnition of the Integral for Step Functions

Motivation: “Integrals = Areas Under Curves”

Consider constant functions of the form fc : x 7→ f0 ∈ R for x ∈ [a, b]. Let D be the area under the line at f0 from a to b. Then it makes sense to deﬁne

Z b . fc(x) dx .= sign(f0) · Area(D) a = sign(f0) · (|f0| · (b − a))

= f0 · (b − a)

Now this deﬁnition will be extended to piecewise constant or step functions.

Deﬁnition 1: ϕ : [a, b] → R is called a step function if there exists a ﬁnite set P ⊂ [a, b] with a, b ∈ P such that ϕ is constant on the open sub-intervals of [a, b] \ P . P is called a partition of [a, b]. More concretely, P = {a = p0, p1, p2, . . . , pk−1, pk = b}, with p0 < p1 < p2 < . . . < pk−1 < pk, and

ϕ = ci (an arbitrary constant), for i = 1, 2, . . . , k − 1, k. (pi−1,pi)

Notes: If φ is a step function then there are many partitions of [a, b] such that φ is constant on the open sub-intervals defined by the partition, call one P . There exists at least one other point greater than a in P (since both a and b belong to P by definition), call the first p. Then φ is also constant on the open intervals of [a, b] \ (P ∪ {n}), where a < n < p. Let Q be a partition of [a, b], with Q ⊃ P . Then Q is called a refinement of P (so (P ∪ {n}) is a refinement of P ). The minimal common refinement of P and Q is denoted by P ∪ Q.

If φ is a step function on [a, b], then any partition P = {a = p0, p1, p2, . . . , pk−1, pk = b}, with p0 < p1 <

p2 < . . . < pk−1 < pk, such that φ = ci (an arbitrary constant), for i = 1, 2, . . . , k − 1, k, is called (pi−1,pi) compatible with, or adapted to, φ.

Proposition 1: Let S[a, b] be the space of step functions on [a, b]. If f, g ∈ S[a, b], then ∀µ, λ ∈ R, (λf + µg) ∈ S[a, b] (so S[a, b] is closed under taking ﬁnite linear combinations of its elements).

Proof: f ∈ S[a, b], so ∃P = {a = p0, p1, p2, . . . , pk−1, pk = b}, with p0 < p1 < p2 < . . . < pk−1 < pk, which is compatible with f, and g ∈ S[a, b] so ∃Q = {a = q0, q1, q2, . . . , ql−1, ql = b}, with q0 < q1 < q2 < . . . < ql−1 < ql, which is compatible with g. Consider P ∪ Q = {a = r0, r1, r2, . . . , rm−1, rm = b}, with r0 < r1 < r2 < . . . < rm−1 < rm. Then this partition is compatible with (λf + µg). It remains to

show that (λf + µg) = ci (an arbitrary constant), for i = 1, 2, . . . , m − 1, m. (ri−1,ri)

P ⊂ (P ∪ Q), so (ri−1, ri) ⊂ (pa−1, pa) for some a, and Q ⊂ (P ∪ Q), so (ri−1, ri) ⊂ (qb−1, qb) for

some b. Then, since f is constant, f is constant, and since g is constant, (p ,p ) (r ,r ) (q ,q ) a−1 a i−1 i b−1 b g is constant, so (λf + µg) is constant. Since the choice of i was arbitrary, this proves (ri−1,ri) (ri−1,ri) the proposition.

2 Notes: This demonstrates that S[a, b] is a vector space over R. Define C[a, b] as the space of continuous functions on [a, b], and define B[a, b] as the space of bounded functions on [a, b]. These are also vector spaces. It was demonstrated in Analysis II that C[a, b] ⊂ B[a, b], and by definition S[a, b] ⊂ B[a, b].

k Definition 2: For φ ∈ S[a, b], let P = {pi}i=0 be a partition compatible with φ. Define (temporarily) b . R φi = φ . Then a : S[a, b] → R is defined as (pi−1,pi)

Z b k . X φ .= φi(pi − pi−1) a i=1

R b Remark: If φ ≡ φ0 ∈ R, then P = {a, b}, so a φ = φ0(b − a), which is precisely the area described in the motivation.

R b Lemma 2: The value of a φ does not depend on the choice of partition compatible with [a, b].

Proof: For any φ ∈ S[a, b], let P and Q be partitions compatible with [a, b]. Suppose there are k points R b in Q that are not in P . Denote them n1, n2, . . . , nk. Temporarily let φ be the integral of φ from a a P to b deﬁned by the partition P which is compatible with [a, b].

Now, consider P ∪ {n1}. Since Q is compatible with [a, b], we know that n1 ∈ [a, b], so ∃i ∈ N such that n1 ∈ (pi−1, pi) for some pi−1, pi ∈ P . φ is constant on (pi−1, pi), and so is also constant on (pi−1, n1) and (n1, pi), so P ∪ {n1} is compatible with φ. By deﬁnition

Z b Z b Z b Z b

φ − φ = φi(n1 − pi−1) + φi(pi − n1) − φi(pi − pi−1) = 0, so φ = φ a P ∪{n1} a P a P ∪{n1} a P The proof is completed by induction, since there are a ﬁnite number of points. This is left as an exercise.

3 1.2 Properties of the Integral for Step Functions

1. Proposition 3 - Additivity: For any φ ∈ S[a, b], ∀v ∈ (a, b),

Z b Z v Z b φ = φ + φ a a v

2. Proposition 4 - Linearity: For any φ, ψ ∈ S[a, b], ∀λ, µ ∈ R, Z b Z b Z b λφ + µψ = λ φ + µ ψ a a a

3. Proposition 5 - The Fundamental Theorem of Calculus: Let φ ∈ S[a, b], and let P be a R x partition of [a, b] compatible with φ. Consider I : [a, b] → R, I : x 7→ a φ. Then (I) I is continuous on [a, b].

Sk Sk 0 (II) I is diﬀerentiable on i=1(pi−1, pi), and, ∀x ∈ i=1(pi−1, pi), I (x) = φ(x).

Proof: k 1. Let P = {pi}i=0 be a partition compatible with φ ∈ S[a, b]. Then there are two cases: (i) If v ∈ P , then by deﬁnition

k v k Z b X X X Z v Z b φ = φi(pi − pi−1) = φi(pi − pi−1) + φi(pi − pi−1) = φ + φ a i=1 i=1 i=v+1 a v

(ii) If v∈ / P , then ∃i ∈ N, 1 ≤ i ≤ k, such that v ∈ (pi−1, pi). Then {a = p0, . . . , pi−1, v},

{v, pi, . . . , pk = b} are partitions compatible with φ, so φ ∈ S[a, v], φ ∈ S[v, b], so (a,v) (v,b) their integrals are well-deﬁned on those intervals. Then

k Z b X φ = φj(pj − pj−1) a j=1

= φ1(p1 − p0) + φ2(p2 − p1) + ... + φi(pi − pi−1) + ... + φk(pk − pk−1)

= φ1(p1 − p0) + ... + φi(pi + v − v − pi−1) + ... + φk(pk − pk−1)

= φ1(p1 − p0) + ... + φi(v − pi−1) + φi(pi − v) + ... + φk(pk − pk−1)

i−1 # " k  X X =  φj(pj − pj−1) + φi(v − pi−1) + φi(pi − v) + φj(pj − pj−1) j=1 j=i+1 Z v Z b = φ + φ a v

2. Let P be a partition compatible with φ ∈ S[a, b], and let Q be a partition compatible with ψ ∈ S[a, b]. Then, as was proved in the previous section, P ∪ Q is compatible with both φ and ψ. Let P ∪ Q = {a = x0, x1, . . . , xk−1, xk = b}, with a = x0 < x1 < . . . < xk−1 < xk = b. Then

φ = φi and ψ = ψi, so (λφ + µψ) = (λφ + µψ)i, so P ∪ Q is compatible (xi−1,xi) (xi−1,xi) (xi−1,xi) with λφ + µψ. Then

k k Z b X X λφ + µψ = (λφ + µψ)j(xj − xj−1) = (λφj + µψj)(xj − xj−1) a j=1 j=1 k k X X Z b Z b = λ φj(xj − xj−1) + µ ψj(xj − xj−1) = λ φ + µ ψ j=1 j=1 a a

4 k Sk 3. Let P = {pi}i=0 be a partition compatible with φ ∈ S[a, b]. Choose x ∈ j=1(pj−1, pi), then ∃i ∈ N, 1 ≤ i ≤ k such that x ∈ (pi−1, pi). Then

Z x Z pi−1 Z x I(x) = φ = φ + φ = I(pi−1) + φi(x − pi−1) = φix + constant a a pi−1

Sk 0 so I is continuous on j=1(pj−1, pi) and I (x) = φi = φ(x). Furthermore, limx↓pi−1 I(x) = I(pi−1), R pi−1 so I is right continuous on [a, b). Similarly, limx↑pi I(x) = I(pi−1) + φi(pi − pi−1) = a φ + R pi−1 R pi R pi φi(pi − pi−1) = φ + φ = φ = I(pi), so I is left continuous on (a, b], so I is continuous a pi−1 a on [a, b].

Deﬁnition 3: Let f ∈ B[a, b]. Then the supremum norm (or sup norm) of f, denoted by kfk∞, with k·k∞ : B[a, b] → R is deﬁned as . kfk∞ = sup f(x) x∈[a,b]

Proposition 6: Let φ ∈ S[a, b] be bounded between m, M ∈ R, so m ≤ φ(x) ≤ M ∀x ∈ [a, b]. Then

Z b Z b

m(b − a) ≤ φ ≤ M(b − a), in particular φ ≤kφk∞ (b − a). a a

Proof: For φ ∈ S[a, b], choose a compatible partition P . Now

k Z b X φ = φi(pi − pi−1) a i=1 then k k k X X X m(pi − pi−1) ≤ φi(pi − pi−1) ≤ M(pi − pi−1) i=1 i=1 i=1 so, due to telescoping sums on the upper and lower bounds,

k X Z b m(b − a) ≤ φi(pi − pi−1) ≤ M(b − a), so m(b − a) ≤ φ ≤ M(b − a). i=1 a

If kφk is given, then, by deﬁnition, −kφk ≤ φ(x) ≤ kφk , so using the same argument as above, ∞ ∞ ∞ −kφk (b − a) ≤ R b φ ≤kφk (b − a), so R b φ ≤kφk (b − a). ∞ a ∞ a ∞

b b b Note: So far R φ has been deﬁned for b > a. Deﬁne, for a > b, R φ .= − R φ. Then the bound proved a a a in the previous proposition becomes R b φ ≤ kφk |b − a|. It is a simple exercise to demonstrate that a ∞ such an extension preserves the three properties of the integral for step functions proved previously.

5 2 The Integral for Regulated Functions

2.1 Deﬁnition of the Integral for Regulated Functions

Deﬁnition 4: A function f on [a, b] is called a regulated function if, ∀ε > 0, ∃φ ∈ S[a, b] such that kφ − fk∞ < ε, or, alternatively, if, ∀ε > 0, f(x) − φ(x) < ε ∀x ∈ [a, b]. We denote the set of regulated functions on [a, b] by R[a, b].

Deﬁnition 4.1: A sequence (φn)n≥1 ⊂ S[a, b] is said to converge uniformly to a function f : [a, b] → R if limn→∞kf − φnk∞ = 0.

Lemma : f : [a, b] → R is regulated iﬀ ∃(φn)n≥1 ⊂ S[a, b] such that, as n → ∞,(φn) converges uniformly to f.

Proof: The proof is routine and is left as an exercise.

Deﬁnition 4.2: A function f : A ⊂ R → R is called uniformly continuous on A if, ∀ε > 0, ∃δ such

that ∀x, y ∈ A, |x − y| < δ and f(x) − f(y) < ε.

Note: In general, uniform continuity implies continuity, but not the other way around. For example, 1 f : (0, 1] → R, f : x 7→ x , is continuous, but not uniformly continuous. However, on closed intervals, the two are equivalent.

Lemma: For a function f : A → R, if A = [a, b], continuity implies uniform continuity.

Proof: Suppose f is continuous on [a, b] but not uniformly continuous. Then ∃ε > 0 such that ∀δ > 0, 1 ∃x, y ∈ [a, b] such that |x − y| < δ but f(x) − f(y) ≥ ε. Taking δn = n → 0 as n → ∞ yields two sequences (xn)n≥1, (yn)n≥1 ⊂ [a, b] with |xn − yn| < δn. Since (xn)n≥1 is bounded, it converges by the

Bolzano-Weierstrass Theorem, so ∃(xnk )k≥1 ⊂ (xn)n≥1 which converges in [a, b], since [a, b] is closed. So 1 limk→∞ xnk = u ∈ [a, b]. Choose (ynk )k≥1, which converges by a similar argument, and choosing δ = n k demonstrates that limk→∞ xn − yn = 0, so limk→∞ yn = u. But f is continuous, so limk→∞ f(xn ) = k k k k f(u) = limk→∞ f(ynk ), which contradicts the condition that |x − y| < δ but f(x) − f(y) ≥ ε, so f is uniformly continuous on [a, b].

Proposition 7: If a function f : [a, b] → R is continuous on [a, b], then it is also regulated on [a, b].

k Proof: For any ε > 0, let P = {pi} be a partition of [a, b] such that |pi − pi−1| < δ, where δ i=0 is such that |x − y| < δ =⇒ f(x) − f(y) < ε. Such a δ exists because f is continuous and so 

φ = f(pi−1) for i = 1, . . . , k uniformly continuous on [a, b]. Let φ ∈ S[a, b] be such that [pi−1,pi) . Now, φ(b) = f(b)

∀x ∈ [a, b), ∃j such that x ∈ [pj−1, pj), then f(x) − φ(x) = f(x) − f(pj−1) < ε since x − pj−1 < δ.

Also, f(b) − φ(b) = 0, so, ∀x ∈ [a, b], f(x) − φ(x) < ε, so f is regulated by deﬁnition.

Exercise: Prove that any monotone function is regulated.

6 Proposition 8: For any f, g ∈ R[a, b] and λ, µ ∈ R, λf + µg ∈ R[a, b].

Proof: First prove that ∀f, g ∈ B[a, b], ∀λ ∈ R, kf + gk∞ ≤kfk∞ +kgk∞ and kλfk∞ =|λ|kfk∞. The remainder of the proof is left as an exercise.

R b Deﬁnition 5: Let f ∈ R[a, b]. Then a : R[a, b] → R is deﬁned as Z b Z b . f .= lim φn a n→∞ a where (φn)n≥1 ⊂ S[a, b] converges to f uniformly as n → ∞.

R b Proposition 9: limn→∞ a φn exists and is independent of the choice of (φn)n≥1 ⊂ S[a, b], which converges to f ∈ R[a, b] uniformly as n → ∞.

Proof: Let f ∈ R[a, b] and (φn)n≥1, (ψn)n≥1 ⊂ S[a, b] converge to f uniformly as n → ∞.(φn) con- ε verges to f uniformly, so for any ε > 0, eventually there will be an n such that kφn − fk∞ < 2(b−a) , so eventually there will be n and m, when suﬃciently large, such thatkφn − φmk∞ = (φn − f) + (f − φm) ∞ ≤kφ − fk +kf − φ k ≤ ε . Then R b φ − R b φ = R b(φ − φ ) ≤kφ − φ k (b − a) < ε, so n ∞ m ∞ b−a a n a m a n m n m ∞ R b ( a φn)n≥1 is Cauchy, so the limit exists.

Let ωn = (φn, ψn)n−1 = (φ1, ψ1, φ2, ψ2,...) ⊂ S[a, b]. Then ωn converges to f uniformly as n → ∞, R b R b the proof of this fact is routine and is left as an exercise. limn→∞ a ωn exists, so the subsequences a φn R b R b R b R b and a ψn must converge to the same limit a f, so a φn = a ψn.

7 2.2 Properties of the Integral for Regulated Functions

Proposition 10: For any f ∈ R[a, b], u ∈ [a, b], Z b Z u Z b f = f + f a a u

Proof: f is regulated, so ∃(φn)n≥1 ⊂ S[a, b] which converges uniformly to f. Then (φn) converges [a,u]

uniformly to f , since restrictions are regulated. The same holds for (φn) . So [a,u] [u,b]

Z u Z b Z u Z b ! Z b Z b f + f = lim φn + φn = lim φn = f a u n→∞ a u n→∞ a a

by the additivity of integrals of step functions.

R b Proposition 11: The map I : R[a, b] → R, I : f 7→ a f is linear.

Proof: The proof is similar to a combination of the last proof and the proof for the linearity of the integral for step functions, and so is left as an exercise.

Note: Not all functions are regulated.

Example: Consider f : [0, 1] → R, with ( 1 x ∈ f : x 7→ Q 0 x∈ / Q 1 f is also denoted by Q - the indicator of Q. Let φ be any step function on [a, b], with a compatible

partition P , and let φ = φ . Then kf − φk ≥ f − φ = max(|φ | ,|φ − 1|) ≥ 1 , so 1 ∞ 1 1 1 2 (po,p1) (p0,p1) ∞ there cannot exist a sequence of step functions converging uniformly to f.

Proposition 12: Suppose that f ∈ R[a, b], and that, ∀x ∈ [a, b], m ≤ f(x) ≤ M for some m, M ∈ R. Then Z b m(b − a) ≤ f ≤ M(b − a) a

Proof: f ∈ R[a, b], so, ∀ε > 0, ∃φε ∈ S[a, b] such thatkf − φεk∞ < ε, so ∀x ∈ [a, b], f(x)−ε ≤ φε(x) ≤ f(x) + ε, so m − ε ≤ φε(x) ≤ M + ε, now using the result proved for step functions Z b (m − ε)(b − a) ≤ φε ≤ (M − ε)(b − a) a 1 1 Choose ε = n for n ∈ N, then (φn)n≥1 converges uniformly to f. Take the limit of the above with ε = n , then Z b m(b − a) ≤ f ≤ M(b − a) a

Exercise: Prove that, for any f ∈ R[a, b],

Z b

f ≤kfk∞ (b − a) a

Example: f(x) = x is uniformly continuous on R, since f(x) − f(y) =|x − y|, so when|x − y| < δ = ε,

f(x) − f(y) < ε.

α Proposition 13: Let fα : [1, ∞) → R, fα : x 7→ x for some α ≥ 0. Then fα is uniformly continuous for any α ≥ 0 but not uniformly continuous for α > 1.

α α α δ α α δ α−1 Proof: For α > 1, fα(x+δ)−fα(x) = (x+δ) −x = x ((1+ x ) −1) ≥ x (1+α x −1) = αδx → ∞ as x → ∞, which contradicts the condition for uniform continuity. α δ α α αδ α For α < 1, α 6= 0, 0 ≤ fα(x + δ) − fα(x) = x ((1 + x ) − 1) ≤ x (1 + x − 1), (α − 1) < 0, so x → 0 ε α as x → ∞. For any ε > 0, choose δ = 2αδ , then x is uniform continuous. The case when α = 0 is trivial and is left as an exercise.

Proposition 14: If f : [a, b] → R is weakly increasing, then f ∈ R[a, b].

f(b)−f(a) Proof: For f(b) > f(a), let d = n . Let P be a partition with n + 1 elements such that p0 = a, pj = supx∈[a,b]{x : f(x) ≤ f(a) + dj}, and pn+1 = b. By construction, since pj > pj−1, then

∀x ∈ (pj−1, pj), f(a)+d(j −1) ≤ f(x) ≤ f(a)+dj. Let ϕn ∈ S[a, b] such that ϕn = f(a)+dj and (pj−1,pj ) ϕn(pj) = f(pj). Now, let x ∈ (pj−1, pj). Then, using the previous inequality, −d ≤ f(x) − ϕn(x) ≤ 0, so f(b)−f(a) kf − ϕnk∞ ≤ d = n → 0 as n → ∞, so f is regulated.

Exercise: Prove that if f : [a, b] → R is weakly decreasing, then f is regulated. This proves that all monotonic functions on bounded intervals are regulated.

0 1 1 1 2 1 3 1 Example: Let Q ∩ [0, 1] = { 1 , 1 , 2 , 3 , 3 , 4 , 4 , 5 ,,...} = {q1, q2, q3,...}, and let ( 1 x > qn 1(x > qn) = 0 x ≤ qn

Let f : [0, 1] → R be given by ∞ X −n f : x 7→ 2 1(x > qn) n=1 which exists ∀ x ∈ [0, 1]. Then f is non-decreasing, and is regulated, but jumps an inﬁnitely many times, so is impossible to draw.

9 3 The Indeﬁnite Integral & Fundamental Theorem of Calculus

3.1 The Indeﬁnite Integral

Definition 6: Let f ∈ R[a, b]. Define F : [a, b] → R by Z x F : x 7→ f a F is called the indefinite integral of f.

Lemma: R[a, b] ⊂ B[a, b]

Proof: f ∈ R[a, b], so ∃φ ∈ S[a, b] such that kφ − fk∞ ≤ 1, so ∀ x ∈ [a, b], φ(x) − 1 ≤ f(x) ≤ φ(x) + 1. Since step functions are bounded, φ ∈ B[a, b], so f ∈ B[a, b].

Proposition 15: F is continuous on [a, b].

Proof: Let x, y ∈ [a, b]. Then

Z x Z y Z x

F (x) − F (y) = f − f = f ≤kfk∞|x − y| , a a y so by the sequential deﬁnition of continuity, f is continuous.

Note: Let f : [a, b] → R. Then, if ∃ L > 0 such that, ∀ x, y ∈ [a, b], f(x) − f(y) ≤ L|x − y|. Then f is called Lipschitz continuous or Lipschitz. Lipschitz continuity implies continuity, but the reverse is not generally true.

√ Example: f(x) = e−x is Lipschitz on [0, 1], but g(x) = x is not Lipschitz on [0, 1].

10 3.2 The Fundamental Theorem of Calculus

R x Theorem 16 - FTC1: Let f ∈ R[a, b] such that f is continuous at c ∈ (a, b). Then F (x) = a f is diﬀerentiable at c and F 0(c) = f(c).

Proof: f is continuous at c, so, ∀ε > 0, ∃δ > 0 such that, ∀x ∈ (c − δ, c + δ), f(x) − f(c) < ε, so f(c) − ε < f(x) < f(c) + ε. Choose 0 < h < δ. Then, by integrating the previous statement

Z c+h Z c+h Z c h(f(c) − ε) < f < h(f(c) + ε), so h(f(c) − ε) < f − f = F (c + h) − F (c) < h(f(c) + ε) c a a So, for any h ∈ (0, δ), h(f(c) − ε) < F (c + h) − F (c) < h(f(c) + ε), so

F (c + h) − F (c) −ε < − f(c) < ε h So F is diﬀerentiable at c, and F 0(c) = f(c). The proof for the case where −δ < h < 0 is similar and so is left as an exercise.

Corollary 17: If f ∈ C[a, b], then f is diﬀerentiable on (a, b) and F 0 = f.

Proof: f is continuous, so it is regulated. Applying FTC1 completes the proof.

Theorem 18 - FTC2: Let f : [a, b] → R be continuous, and let g : [a, b] → R be diﬀerentiable, with g0(x) = f(x) ∀x ∈ [a, b]. Then Z b f = g(b) − g(a) a This is the Newton-Leibniz Formula, and g is called the antiderivative of f. This is sometimes written

Z b g0 = g(b) − g(a) a

R x Proof: Let δ : [a, b] → R, with δ(x) = g(x)− a f. Then δ is continuous on [a, b]. it is also diﬀerentiable on (a, b) by FTC1. By the MVT, δ(b)−δ(a) = δ(ξ)(b−a) for some ξ ∈ (a, b). But δ0(ξ) = g0(ξ)−f(ξ) by 0 R a R b R b FTC1, so δ (ξ) = f(ξ) − f(ξ) = 0, so δ(a) = δ(b), so g(a) − a f = g(b) − a f, so a f = g(b) − g(a).

11 4 Practical Methods of Integration

4.1 Integration by Parts

Proposition 20: If f, g ∈ R[a, b], then f · g ∈ R[a, b].

Proof: kf + gk ≤kfk +kgk . Then ∃(ϕn), (ψn) ∈ S[a, b] such that (ϕn) → f, (ψn) → g uniformly. ∞ ∞ ∞ 0 ≤kf · g − ϕnψnk∞ =kf · g − ϕng + ϕng − ϕnψnk∞ ≤ (f − ϕn)g ∞ + ϕn(g − ψn) ∞.

Since supx∈[a,b] a(x) · b(x) ≤ supx∈[a,b] a(x) ·supx∈[a,b] b(x) , this is less than or equal tokgk∞kf − ϕnk∞+ kϕnk∞kg − ψnk∞. Now, kgk∞ is regulated, and so is bounded. kf − ϕnk∞ ,kg − ψnk∞ → 0 as n → ∞, and kϕnk∞ = kϕn − f + fk∞ ≤ kϕn − fk∞ +kfk∞. Also, kϕn − fk∞ → 0 as n → ∞, and kfk∞ is regulated, and so is bounded. Taken together, this means that kf · g − ϕnψnk∞ → 0 as n → ∞, so (ϕnψn) converges uniformly to f · g, so f · g is regulated.

Corollary 19 - Integration by Parts: If F and G are diﬀerentiable such that F 0 and G0 are continuous, hence regulated, then

Z b Z b F 0G = F (b)G(b) − F (a)G(a) − FG0 a a

Proof: (FG)0 = FG0 + F 0G. Then

Z b Z b Z b Z b (FG)0 = FG0 + F 0G = FG0 + F 0G a a a a By FTC2 Z b Z b Z b F (b)G(b) − F (a)G(a) = FG0 + F 0G = FG0 + F 0G a a a So Z b Z b 0 0 F G = F (b)G(b) − F (a)G(a) − FG a a

b Note: F (b)G(b) − F (a)G(a) can be denoted by FG|a.

Example: The Gamma Function Γ : (0, ∞) → R is given by Z ∞ Z L Γ : x 7→ tx−1e−t dt .= lim tx−1e−t dt 0 ε↓0,L→∞ ε Then

Z L 1 0 Γ(x) = lim tx e−t dt ε↓0,L→∞ ε x Z L 1 x −t L x −t = lim (t e )|ε − t 1(−e ) dt x ε↓0,L→∞ ε 1 Z ∞ = txe−t dt x 0 1 = Γ(x + 1) x So Γ(x + 1) = xΓ(x), and Γ(1) = 1, so Γ(n + 1) = n!

12 4.2 Integration by Substitution

Corollary 21 - Integration by Substitution: Let f ∈ C[a, b], and let g : [c, d] → [a, b] be diﬀeren- tiable with Im g ⊂ [a, b]. Then

Z d Z g(d) f(g(x)) · g0(x) dx = f(t) dt c g(c)

R x 0 Proof: Let F (x) = a f. Since f is continuous, F is diﬀerentiable and F (x) = f(x) by FTC1, then by FTC2 Z g(d) f(t) dt = F (g(d)) − F (g(c)) g(c) d 0 0 0 Also, dx F (g(x)) = F (g(x)) · g (x) = f(g(x)) · g (x) which is the initial integrand. Now, by FTC2, d F (g(x))|c = F (g(d)) − F (g(c)), so Z d Z g(d) 0 f(g(x)) · g (x) dx = f(t) dt c g(c)

Note: g does not need to be injective or surjective.

Example: Let g(x) = x2. Then Z a 2 2 0 2 a f(x )(x ) dx = F (x )|−a −a

Example: Let

∞ L1 Z 2 Z 2 G(a, j) .= e−ax +2jx dx = lim e−ax +2jx dx L1→∞,L2→∞ −∞ −L2 for some a > 0, j ∈ R. Then ∞ Z 2 G(a, j) = e−ax +2jx dx −∞ Z ∞ −a(x2+ 2j x) = e a dx −∞ Z ∞ 2 −a((x− j )2− j ) = e a a2 dx −∞ 2 Z ∞ j −a(x− j )2 = e a e a dx −∞

j Now, substituting g(x) = x − a ,

2 Z ∞ 2 Z ∞ 2 Z ∞ √ j −a(x− j )2 j −at2 j −( at)2 e a e a dx = e a e dt = e a e dt −∞ −∞ −∞ √ Now, substituting h(t) = at,

2 Z ∞ √ 2 Z ∞ 2 r j −( at)2 j 1 −y2 j π e a e dt = e a √ e dy = e a −∞ a −∞ a So r j2 π G(a, j) = e a a

13 4.3 Constructing Convergent Sequences of Step Functions

Theorem 22: Let f : [a, b] → R be regulated. Then

n X b − a Z b b − a lim f(aj) = f(x) dx, where aj = a + j n→∞ h h j=1 a

Proof: First, consider φ ∈ S[a, b]. Let P be a partition compatible with φ, with |P | = k. Then

Z b n X b − a b − a 0 ≤ φ − φ(aj) ≤ k 2kφk → 0 as n → ∞ h n ∞ a j=1 since errors only occur and contribute to the diﬀerence at jumps on the step function, of which there are k, and the maximum error this can be is twice the maximum value of the step function across the whole section. Then n Z b X b − a φ = lim φ(aj) n→∞ h a j=1 Now, for f ∈ R[a, b], let n (n) X b − a b − a R = f(a ), where a = a + j f h j j h j=1 ε Since f is regulated, then ,∀ε > 0, ∃φ ∈ S[a, b] such that kf − φk∞ < 3 . Then

Z b Z b Z b Z b (n) (n) (n) (n) Rf − f = Rf − Rφ + Rφ − φ + φ − f a a a a

Z b Z b (n) (n) (n) ≤ Rf − Rφ + Rφ − φ + φ − f a a Now, ε R(n) − R(n) ≤kφ − fk (b − a) < (b − a) f φ ∞ 3 and

Z b ε (n) Rφ − φ < (b − a) eventually in n a 3 and

Z b n n X b − a X b − a ε φ − f = (f(aj) − φ(aj)) ≤ f(aj) − φ(aj) < (b − a) n n 3 a j=1 j=1 ε since f(aj) − φ(aj) < 3 , so

Z b (n) Rf − f < ε(b − a) eventually in n a so Z b (n) lim Rf = f n→∞ a

14 5 The Characterisation of Regulated Functions

5.1 Characterisation & Proof

Proposition 23: A function f ∈ R[a, b] iﬀ, ∀x ∈ (a, b), f(x±) both exist, and both f(a+) and f(b−) exist, where f(x+) = limy↓x f(y) (the right limit), and f(x−) = limy↑x f(y) (the left limit). Alternatively, this can be stated as “all one-sided limits exist”.

Note: This characterisation only requires the existence of these limits, not equality, f(x+) 6= f(x−) is acceptable so long as they both exist.

Proof =⇒ : f is regulated, so, ∀ε > 0, ∃φ ∈ S[a, b] such that kf − φk∞ < ε. Let x ∈ (a, b). There are now two cases, φ is continuous at x, or φ is discontinuous at x. In either case, ∃δ > 0

such that φ = φ0, a constant. Let (xn)n≥1 ⊂ R be any sequence such that, ∀n ∈ N, xn > x, (x,x+δ)

and limn→∞ xn = x. Eventually xn ∈ (x, x + δ). Then, eventually in n and m, f(xn) − f(xm) =

f(xn) − φ0 + φ0 − f(xm) ≤ f(xn) − φ0 + f(xm) − φ0 < 2ε, since f(xn) − φ0 , f(xm) − φ0 < ε, so f(xn)n≥1 is Cauchy, and so converges. If yn is any other sequence with yn ↓ x as n → ∞, then (f(yn))n≥1 converges to the same limit. The proof of this fact is routine after considering (xn, yn)n≥1, and so is left as an exercise. It has been shown that for any (xn)n≥1 ⊂ R such that xn ↓ x as n → ∞, limn→∞ f(xn) = f(x+) exists and does not depend on xn. A similar argument can be used to prove this result for the left limit, and not much modiﬁcation is required for x = a, b, so these proofs are left as exercises.

Proof ⇐= : All one-sided limits exist. Now, ﬁx any ε > 0. Let

Aε = {γ ∈ [a, b] : ∃ψ ∈ S[a, b] such that (f − ψ) < ε} [a,γ] ∞ The proof shall follow from proving three statements:

(i) Aε 6= ∅: f(a+) exists, so, ∀ε > 0, ∃δ > 0 such that, ∀x ∈ (a, a + δ), f(x) − f(a+) < ε. Take γ = δ . Then deﬁne 2 ( f(a) x = a φ(x) = δ f(a+) x ∈ (a, a + 2 ) δ It is clear that φ(x) ∈ S[a, a + 2 ]. Since f(x) − f(a+) < ε and f(a) − φ(a) = 0,

δ (f − φ) < ε, so γ = 2 ∈ Aε, so Aε 6= ∅. [a,a+ δ ] 2 ∞ . (ii) sup Aε = b: Assume that c .= sup Aε < b. Then, ∀δ > 0, (c−δ) ∈ Aε, so ∃φ ∈ S[a, c−δ] such that

(f − φ) < ε. But f has limits f(c±) at c, so ∀ε > 0, ∃δ1, δ2 such that, ∀x ∈ (c, c + δ2), [a,c−δ] ∞ f(x) − f(c+) < ε and, ∀x ∈ (c − δ1, c), f(x) − f(c−) < ε. Choose δ < δ1. Now, let ψ ∈ S[a, b] be given by  f(c−) x < c  ψ(x) = f(c) x = c  f(c+) x > c

and let φ˜ ∈ S[a, b] be given by ( φ(x) x ∈ [a, c − δ] φ˜(x) = δ2 ψ(x) x ∈ (c − δ, c + 2 ]

δ2 δ2 Then (c + 2 ) ∈ Aε, with 2 > 0, which contradicts the fact that c = sup Aε, so b = c.

(iii) b ∈ Aε: It has been shown that, ∀δ > 0, ∃φ ∈ S[a, b − δ] such that (f − φ) < ε. But [a,b−δ] ∞ f(b−) exists, so ∃δ1 > 0 such that, ∀x ∈ (b − δ1, b), f(x) − f(b−) < ε. Choose δ < δ1. Now, let ψ ∈ S[a, b] be given by ( f(b) x = b ψ(x) = f(b−) x < b

and let φ˜ ∈ S[a, b] be given by ( φ(x) x ∈ [a, b − δ] φ˜(x) = ψ(x) x ∈ (b − δ, b]

˜ Now, by deﬁnition, f − φ < ε, so b ∈ Aε. ∞

So γ = b, so ∃ϕ ∈ S[a, b] such that kf − ϕk∞ < ε, so f ∈ R[a, b].

Exercise: Prove that if f ∈ R[a, b], then the set of discontinuities of f is countable. Hint: Check that, ∀ε > 0, the set of jumps of size ε or greater is countable.

16 6 Improper & Riemann Integrals

6.1 Improper Integrals

Motivation: It is possible to generalise the regulated integral to unbounded functions and/or unbounded domains.

Example: Z 1 − 1 x 2 dx = 2 0 − 1 − 1 because, although x 2 is not regulated on (0, 1], ∀ε > 0, x 2 ∈ R[ε, 1]. Then

1 Z 1 1 1 − 2 2 1 2 x dx = 2x |ε = 2 − 2ε → 2 as ε → 0 ε

Example: Z ∞ 1 2 dx = 1 1 x 1 because, ∀L > 1, x2 ∈ R[1,L]. Then

Z L L 1 1 1 2 dx = − = 1 − → 1 as L → ∞ 1 x x 1 L

Example: Z 0 Z 1 1 1 1 1 dx = lim dx = lim log x|ε = lim log 1 x ε↓0 ε x ε↓0 ε↓0 ε which does not exist, therefore the integral is not deﬁned.

Deﬁnition 7.1: If, for any ε > 0, f : (a, b] → R is regulated on [a + ε, b] and Z b l .= lim f ε↓0 a+ε

R b R b exists, then a f converges and is equal to l. If the limit does not exist, then a f diverges.

Deﬁnition 7.2: If, ∀L > 0, f ∈ R[a, L] and

Z L l .= lim f L→∞ a R ∞ R ∞ exists, then a f converges and is equal to l. If the limit does not exist, then a f diverges.

17 R ∞ R c R ∞ Note: Let f ∈ R[−L1,L2] for some L1,L2 > 0. Then −∞ f converges if, ∀c ∈ R, −∞ f and c f R c R ∞ R ∞ R c R ∞ converge. In this case, −∞ f + c f = −∞ f. As an exercise, show that if −∞ f and c f exist, then R c R ∞ −∞ f + c f does not depend on the choice of c.

Example: Let f(x) = x ∈ R[−L, L] ∀L > 0. Then Z L Z L x dx = 0, so lim x dx = 0 −L L→∞ −L However, take any c ∈ R. Then Z L Z c lim x dx = ∞, and lim x dx = −∞ L→∞ c L→∞ −L so do not exist. Then Z ∞ x dx −∞ does not exist, since the limits must exist independently.

R b Notes: Suppose f : (a, b) → R, and, for any ε1, ε2 > 0, f ∈ R[a + ε1, b − ε2]. Then a f exists if, for R c R b R b R b R c R b any c ∈ (a, b), a f and c f both exist (converge). Then a f converges, and a f = a f + c f. R c R b R c Suppose f is deﬁned on [a, b) ∪ (b, c]. Then a f exists (converges) if a f and b f both exist (indepen- R c R b R c dently), with a f = a f + b f.

Example: f(x) = √1 for x ∈ [−1, 0) ∪ (0, 1]. |x|

Note: Let f : (a, b) → R such that, for any ε1, ε2 > 0, f ∈ [a + ε1, b − ε2]. Then ∃c ∈ (a, b) such that Z c Z b−ε2 l1(c) = lim f and l2(c) = lim f ε1↓0 ε2↓0 a+ε1 c 0 0 0 exist iﬀ, ∀c ∈ (a, b), l1(c ) and l2(c ) exist. The ﬁrst formulation is often the more convenient in practice R b R b however. If this formulation is true of f, then a f converges, and a f = l1(c) + l2(c). As an exercise, 0 0 check that l1(c) + l2(c) − l1(c ) − l2(c ) = 0 to demonstrate that this integral is independent of the choice of c.

Deﬁnition 7.3: Let f ∈ R[−L1,L2] for some L1,L2 > 0. Then

Z ∞ Z L1 Z c f .= lim f + lim f L1→∞ L2→∞ −∞ c −L2

Deﬁnition 7.4: If f : [a, b) ∪ (b, c] → R, then Z c Z b−ε1 Z c f .= lim f + lim f ε1↓0 ε2↓0 a a b+ε2

Note: All integrals in the limit must make sense as regulated or improper integrals.

Note: It is possible to deﬁne the integral of functions with domains such as [ f : (j, j + 1) → R j∈Z Situations like this occur when studying gamma functions.

18 6.2 The Riemann Integral

Deﬁnition: Let f : [a, b] → R. Then Z b . Uf .= inf φ : φ ≥ f (“upper sum”) φ∈S[a,b] a Z b . Lf .= sup φ : φ ≤ f (“lower sum”) φ∈S[a,b] a

Deﬁnition: f : [a, b] → R is Riemann integrable if Uf = Lf . Then Z b . R f .= Uf = Lf a

Proposition: If f ∈ R[a, b], then it is Riemann integrable, and

Z b Z b R f = f a a

Proof: f ∈ R[a, b], so, ∀ε > 0, ∃φ ∈ S[a, b] such that, ∀x ∈ [a, b], φ(x) − ε ≤ f(x) ≤ φ(x) + ε. Since φ(x) − ε, φ(x) + ε ∈ S[a, b],

Z b Z b φ − ε(b − a) ≤ Lf ≤ Uf ≤ φ + ε(b − a) a a Taking ε ↓ 0, Z b Z b Lf = Uf = f = R f a a

Note: There are functions which are Riemann integrable but not regulated.

Example: Let f : [0, 1] → R be given by ( 1 x = 2−n for n = 0, 1, 2,... f : x 7→ 0 otherwise

Then f is not regulated since, ∀p > 0, ∃x ∈ (0, p) such that f(x) = 1 and ∃y ∈ (0, p) such that f(y) = 0, so a step function can get no closer than a half to both “sections” of the function on any interval, so R b a f is not deﬁned. However, let ( 1 x ≤ 2−N φN = f(x) x > 2−N

Now, φN ≥ f ≥ 0, so Z 1 Z 1 0 ≤ Lf ≤ Uf ≤ φN = lim φN = 0 0 N→∞ 2−N

Then Lf = Uf = 0, so Z 1 R f = 0 0 so the Riemann integral of f is deﬁned.

19 Note: There are functions which are not Riemann integrable.

Example: Let ( 1 1 x ∈ Q Q(x) = 0 x ∈ R \ Q

Then U1 = 1 6= 0 = L1 , so 1 is not Riemann integrable. However, it is Lebesgue integrable, with Q Q Q Z b 1 L Q = 0 a

20 7 Uniform & Pointwise Convergence

7.1 Deﬁnitions, Properties and Examples

Example: Let (fn)n≥1 be a sequence of functions on [-1,1], with

1 − n fn : x 7→ x 2 −1 for n = 1, 2,...

Fix x and consider  1 x > 0  lim fn(x) = 0 x = 0 , so lim fn(x) = sign x. n→∞  n→∞ −1 x < 0

So the limit of (fn(x))n≥1 as n → ∞ exists ∀x ∈ [−1, 1], and fn is continuous ∀n ∈ N, but the limit is not continuous at 0.

lim lim fn(x) = 0 6= ±1 = lim sign x = lim lim fn(x) n→∞ x→0± x→0± x→0± n→∞

This function has non-commutative limits and a sequence of continuous functions converges to a discontinuous function, both of which are undesirable properties of pointwise convergence.

Deﬁnition 8: Let (fn)n≥1 be a sequence of functions, and let A ⊂ R. Then fn → f pointwise as n → ∞ if, ∀x ∈ A, limn→∞ fn(x) = f(x).

Note: Pointwise convergence is very “loose”, it does not preserve properties of the fns such as continuity − 1 (as in the case of x 2n−1 → sign x). Pointwise convergence can also be very non-uniform (the same example converges pointwise very slowly near 0).

Note: Let f and (fn)n≥1 be functions on A ⊂ R. If (fn) → f pointwise as n → ∞, then f can fail to be continuous, even if all fns are. Alternatively, ∃x0 ∈ A such that

lim lim fn(x) 6= lim lim fn(x) n→∞ x→x0 x→x0 n→∞

Example (The Witch’s Hat): Let fn : [0, 1] → R, with fn(0) = 0 and  2n2x x ∈ [0, 1 ]  2n 2 1 1 1 fn(x) = n − 2n (x − 2n ) x ∈ ( 2n , n ]  1 0 x > n

Then fn ∈ C[0, 1]. Fix x > 0, then limn→∞ fn(x) = 0. Now, as n → ∞, fn → f ≡ 0 on [0, 1], which is R 1 R 1 1 1 continuous. However, 0 f = 0, but 0 fn = n · 2n = 2 , so

1 Z 1 Z 1 = lim fn 6= lim fn = 0 2 n→∞ 0 0 n→∞ It is clear from examples such as this that pointwise convergence must be strengthened.

Deﬁnition 9: Let f, fn : A ⊂ R → R for n = 1, 2,..., then (fn)n≥1 → f converges uniformly as n → ∞

if, ∀ε > 0, ∃Nε such that, ∀n > Nε, fn(x) − f(x) < ε ∀x ∈ A, or, equivalently, if limn→∞kf − fnk∞ = 0.

21 Note: For the remainder of this section, assume, unless speciﬁed otherwise, that f, fn : A ⊂ R → R for n = 1, 2,....

Remark: Uniform convergence =⇒ pointwise convergence, but pointwise convergence =6 ⇒ uniform convergence.

Deﬁnition 9.1: A sequence (fn)n≥1 (of functions on A ⊂ R) is called uniformly Cauchy if, ∀ε > 0,

∃Nε such that, ∀n, m > Nε, fn(x) − fm(x) < ε ∀x ∈ A.

Note: For a ﬁxed x ∈ A, if (fn(x))n≥1 is Cauchy, then it is pointwise convergent (to a common limit f).

Lemma: A uniformly Cauchy sequence converges uniformly.

Proof: Let (fn)n≥1, (fm)m≥1 be uniformly Cauchy sequences converging to f. Then, ∀ε > 0, ∃Nε ε ε such that, ∀n, m > Nε, fm(x) − 2 ≤ fn(x) ≤ fm(x) + 2 ∀x ∈ A. Take m → ∞, then fm → f, so ε ε f(x) − 2 ≤ fn(x) ≤ f(x) + 2 ∀x ∈ A, so fn(x) − f(x) < ε ∀x ∈ A, so (fn)n≥1 converges uniformly.

Theorem 24: Let (fn)n≥1 ⊂ R[a, b]. Assume that fn → f uniformly. Then (i) f ∈ R[a, b] R b R b R b (ii) limn→∞ a fn = a limn→∞ fn = a f

Proof: ε (i) fn ∈ R[a, b], so, ∀ε > 0, ∃φn ∈ S[a, b] such thatkfn − φnk∞ < 2 . Now,kf − φnk∞ =kf − fn + fn − φnk∞ ≤ ε kf − fnk∞ +kfn − φnk∞. Assume that n is large enough such that kf − fnk∞ < 2 , which is guar- anteed since fn → f, so kf − φnk∞ < ε, so f ∈ R[a, b]. R b (ii) It was shown in (i) that f is regulated, so a f is well deﬁned and

Z b Z b

0 ≤ f − fn ≤kf − fnk∞ (b − a) a a

kf − fnk∞ → 0 as n → ∞, so Z b Z b Z b lim fn = lim fn = f n→∞ a a n→∞ a

Theorem 25: Suppose that (fn)n≥1 ⊂ C[a, b], and fn → f uniformly as n → ∞. Then f ∈ C[a, b]. This theorem also holds for some arbitrary A ⊂ R in place of [a, b], and shall be proven for this more general case.

Proof: fn → f uniformly, so ∀ε > 0, ∃Nε such that, ∀n > Nε, fn(x) − f(x) < ε ∀x ∈ A, so fn is 0 0 0 ε continuous ∀n ∈ N. Then ∃δε such that, ∀x ∈ (x0 − δε, x0 + δε) ∩ A, fn(x) − fn(x0) < 3 0 0 0 0 0 ∀x ∈ (x0 − δε, x0 + δε) ∩ A. Set δε = δε. Then, ∀x ∈ (x0 − δε, x0 + δε) ∩ A, f(x) − f(x0) =

f(x) − fn(x) + fn(x) − fn(x0) + fn(x0) − f(x0) ≤ fn(x) − fn(x0) + fn(x) − fn(x0) + fn(x0) − f(x0) . ε fn → f uniformly as n → ∞, so the ﬁrst and last of these terms are less than 3 , and the middle term ε 3ε was shown to be less than 3 earlier, so f(x) − f(x0) < 3 = ε, so f is continuous at x0.

22 Example (Cantor’s Function or The Devil’s Staircase): Let (fn)n≥1 ⊂ C[0, 1] be such that f ≡ 1 and 0  1 1  2 fn(3x) x ∈ [0, 3 )  1 1 2 fn+1(x) = 2 x ∈ [ 3 , 3 ]  1 1 2  2 + 2 fn(3x − 2) x ∈ ( 3 , 1]

Proposition: limn→∞ fn exists and is continuous. It is called Cantor’s function.

1 2 Proof: Clearly f0 ∈ C[0, 1]. Now, suppose that fn ∈ C[0, 1]. Then fn+1 is clearly continuous on ( 3 , 3 ). 1 1 1 Now, fn is continuous on [0, 3 ) by assumption, so 2 fn(3·) is continuous, so fn+1 is continuous on [0, 3 ). 2 1 1 Also, fn is continuous on ( 3 , 1] by assumption, so 2 + 2 fn(3· − 2) is continuous, so fn+1 is continuous 2 1 2 on ( 3 , 1]. Now the only points remaining to investigate the continuity of are 3 and 3 . 1 1 fn+1(1) = 2 + 2 fn(1), which is 1 if fn(1) = 1. f0(1) = 1, so fn(1) = 1 ∀n ∈ N by induction. 1 1 1 1 1 1 Concerning the continuity of 3 , fn+1( 3 ) = 2 , and fn+1( 3 +) = 2 , so fn+1 is right-continuous at 3 , and 1 1 1 1 1 1 fn+1( 3 −) = 2 fn(3( 3 −)) = 2 fn(1−) = 2 fn(1) = 2 since fn is continuous at 1, so fn+1 is left-continuous 1 1 2 at 3 , so fn+1 is continuous at 3 . The proof for 3 is similar and is left as an exercise. Now, !

kfn+1 − fnk∞ = max (fn+1 − fn) , (fn+1 − fn) , (fn+1 − fn) x∈[0, 1 ) x∈[ 1 , 2 ] x∈( 2 ,1] 3 ∞ 3 3 ∞ 3 ∞ !

= max (fn+1 − fn) , 0, (fn+1 − fn) x∈[0, 1 ) x∈( 2 ,1] 3 ∞ 3 ∞   1 1 1 1 = max  fn(3·) − fn−1(3·) , fn(3· − 2) − fn−1(3· − 2)   2 2 2 2  x∈[0, 1 ) x∈( 2 ,1] 3 ∞ 3 ∞ 1 ≤ kf − f k 2 n n−1 ∞

So, for some c, d ∈ R, 1 1 c d kf − f k ≤ kf − f k ≤ kf − f k ≤ ... ≤ kf − f k ≤ n−1 n ∞ 2 n n−1 ∞ 22 n−1 n−2 ∞ 2n+1 1 0 ∞ 2n+1

The constants are introduced to correct for f0. Now, ﬁx n > m, then, for some D ∈ R,

0 ≤kfn − fmk∞ =kfn − fn+1 + fn+1 − fn−2 + ... + fm−1 − fmk∞

≤kfn − fn−1k∞ +kfn−1 − fn−2k∞ + ... +kfm+1 − fmk∞ n−m−1 1 1 1 D X 1 ≤ D + + ... + ≤ 2n 2n−1 2m+1 2m+1 2k k=0 ∞ D X 1 D ≤ = → 0 as m → ∞ 2m+1 2k 2m k=0

Since kfn − fmk∞ can be made arbitrarily small by taking n > m large enough, (fn) is uniformly Cauchy. Therefore, f = limn→∞ fn exists and is continuous, since a uniformly Cauchy sequence converges uniformly and fn is continuous ∀n ∈ N. The limit is called Cantor’s function or the Devil’s Staircase.

23 Now, f ∈ C[0, 1], so f ∈ R[0, 1]. Taking the limit of fn(x), it is clear that  1 1  2 f(3x) x ∈ [0, 3 )  1 1 2 f(x) = 2 x ∈ [ 3 , 3 ]  1 1 2  2 + 2 f(3x − 2) x ∈ ( 3 , 1]

So 1 1 2 1 1 2 1 Z Z 3 Z 3 Z Z 3 1 Z 3 1 Z 1 1 f = f + f + f = f(3x) dx + dx + + f(3x − 2) dx 1 2 2 1 2 2 2 2 0 0 3 3 0 3 3 Let y = 3x, z = 3x − 2. Then

Z 1 1 1 Z 1 1 1 Z 1 1 1 Z 1 2 Z 1 1 Z 1 1 f = + f(y) dy + + f(z) dz = + f, so f = , so f = 0 6 6 0 6 6 0 3 3 0 3 0 3 0 2

1 2 1 2 1 Remark: Assume that f is constant on E ⊂ [0, 1]. Then f is constant on ( 2 , 3 ) ∪ 3 E ∪ ( 3 + 3 E), so 1 1 1 1 1 |E| = 3 + 3 |E| + 3 |E|, so 3 |E| = 3 , so |E| = 1, so f is locally constant on some E ⊂ [0, 1] of length 1, but f : [0, 1] → [0, 1] continuously.

Note: It is possible to map [0, 1] ⊂ R onto (surjectively) [0, 1] × [0, 1] ⊂ R2 continuously. If γ : [0, 1] → [0, 1] × [0, 1] is such a map, then, ∀x ∈ [0, 1] × [0, 1], ∃t ∈ [0, 1] such that γ(t) = x.A surjective map is relatively simple to construct, such as τ : [0, 1] → [0, 1] × [0, 1], with τ : 0.x1x2x3 ... 7→ (0.x1x3x5 ..., 0.x2x4x6 ...), but τ is not continuous. A continuous map satisfying these criteria is called a space ﬁlling curve, and examples were constructed by Hilbert and Peano in the 1890s.

24 7.2 Uniform Convergence & Integration

Note: Throughout this section, let D = [a, b] × [c, d] ⊂ R2, and (x, t) ∈ D.

Deﬁnition 10: A function f : D → R is continuous at (x0, t0) ∈ D if, ∀ε > 0, ∃δε(x0, t0) > 0 such that,

∀(x0, t0) ∈ D with |x − x0| < δε(x0, t0) and |t − t0| < δε(x0, t0), f(x, t) − f(x0, t0) < ε. An equivalent deﬁnition, which shall be introduced later, uses Euclidean distance.

Deﬁnition 10.1: A function f : D → R is uniformly continuous on D if, ∀ε > 0, ∃δε such that,

∀(x, t), (x0, t0) ∈ D with |x − x0| < δε and |t − t0| < δε, f(x, t) − f(x0, t0) < ε.

Lemma 26.2: If f is continuous on D, then it is also uniformly continuous if D is closed.

Proof: The proof is left as an exercise. Hint: Fix t ∈ [c, d], and consider f(·, t) : [a, b] → R with f : x 7→ f(x, t).

Exercise: Prove that, if f is continuous on D, then f(·, t) is continuous on [a, b] (so f(·, t) ∈ C[a, b]).

Lemma 26.1: If a map I is such that

Z b I : t ∈ [c, d] 7→ f(x, t) dx a then, if f ∈ C(D), I ∈ C[c, d](I is often called the integral depending on a parameter).

Proof: Let (tn)n≥1 ⊂ [c, d] with limn→∞ tn = t0, and let fn : [a, b] → R with fn : x 7→ f(x, tn). Now, f is continuous on D, so f is uniformly continuous on D since D is closed, so, ∀ε > 0, ∃δε>0 such that,

if |x − x0| < δε and |t − t0| < δε, then f(x, t) − f(x0, t0) < ε. Since (tn), (tn+m) → t0 as n → ∞, then,

eventually in n, |tn − tn+m| < δε, so fn(x) − fn+m(x) = f(x, tn) − f(x, tn+m) < ε eventually in n, so (fn) is uniformly Cauchy, so (fn) converges uniformly to f. Now,

Z b Z b Z b Z b lim I(tn) = lim f(x, tn) dx = lim fn(x) dx = lim fn(x) dx = f(x, t0) = I(t0) n→∞ n→∞ a n→∞ a a n→∞ a

So I is continuous at t0, and since t0 ∈ [c, d] was arbitrary, I is continuous on [c, d].

∂f Proposition 27: If f, ∂t are continuous on [a, b] × [c, d], then, ∀t ∈ (c, d),

∂ Z b Z b ∂f f(x, t) dx = (x, t) dx ∂t a a ∂t Or, alternatively, ∀t ∈ (c, d),

Z b Z b ∂f F (t) = f(x, t) dx and G(t) = (x, t) dx a a ∂t both exist on (c, d), F is diﬀerentiable on (c, d), and F 0 = G.

25 ∂f Proof: Fix t ∈ (c, d). Then f(·, t), ∂t (·, t) ∈ C[a, b], so F and G exist. Now, ∃[c1, d1] ⊂ [c, d] such that ∂f t ∈ (c1, d1). But ∂t is continuous on [a, b] × [c, d], and so is uniformly continuous on [a, b] × [c, d]. Then

F (t + h) − F (t) Z b f(x, t + h) − f(x, t) ∂f

− G(t) = − (x, t) dx h a h ∂t

Z b ∂f ∂f MVT = (x, τ) − (x, t) dx a ∂τ ∂t

∂f for some τ ∈ (t, t + h). Now, ∂τ is uniformly continuous on [a, b] × [c, d], so ∀ε > 0, ∃δε > 0 such that ∀h with |h| < δε,

∂f ∂f Z b ∂f ∂f Z b

(x, τ) − (x, t) < ε, so (x, τ) − (x, t) dx < ε = ε(b − a) ∂τ ∂t a ∂τ ∂t a so

F (t + h) − F (t) lim = G(t) h→0 h 0 so F (t) = G(t).

Theorem 28 - Fubini’s Theorem: If f : D → R is continuous, then

Z b Z d ! Z d Z b ! f(x, y) dy dx = f(x, y) dx dy a c c a

Proof: Let Z t Z d ! Z d Z t ! F (t) = f(x, y) dy dx − f(x, y) dx dy a c c a for t ∈ [a, b]. Now, F (a) = 0, and

Z d Z d F 0(t) FTC= f(t, y) dy − f(t, y) dy = 0 c c F is continuous on [a, b] and diﬀerentiable on (a, b), and F 0(t) = 0, so F (b) − F (a) = 0, so

Z b Z d ! Z d Z b ! f(x, y) dy dx = f(x, y) dx dy a c c a

Note: f must be continuous on the whole of D, or counterexamples such as the following can arise:

Let D = [0, 2] × [0, 1], and let f(x, y) : D → R with

 2 2  xy(x −y ) (x, y) 6= (0, 0) f : (x, y) 7→ (x2+y2)3 0 (x, y) = (0, 0)

Then ! ! Z 2 Z 1 1 1 Z 1 Z 2 f(x, y) dy dx = 6= = f(x, y) dx dy 0 0 5 20 0 0

26 7.3 Uniform Convergence & Diﬀerentiation

Note: Suppose that (fn)n≥1 is a sequence of functions on [a, b], and that (fn) → f uniformly as n → ∞. Then, even if every fn is diﬀerentiable on [a, b], f is not necessarily diﬀerentiable on [a, b]. And even if 0 0 f is smooth, (fn) does not necessarily converge to f .

1 0 Example: Suppose that fn(x) = n cos nx on R. This function is smooth, and fn(x) = sin nx. Let 1 1 f : x 7→ 0. Then kfn − fk∞ = kfnk∞ = n kcos n·k∞ = n → 0 as n → ∞, so (fn) → f uniformly as 0 0 0 n → ∞. All fns are smooth, and f is smooth, since f ≡ 0. But (fn) 6→ 0 as n → ∞, in fact, (fn) does not converge at all, except for at certain points such as x = kπ for k ∈ N.

q 2 1 Example: Let fn : R → R, with fn : x 7→ x + n . Then (fn(x)) → |x| pointwise as n → ∞. Let 2 2 1 1 |fn(x) −f(x) | 1 1 1 f(x) =|x|. Then f (x)2 − f(x)2 = x2 + − x2 = . f (x) − f(x) = ≤ √ = √ → n n n n f (x)+f(x) n 1 n | n | n 0 as n → ∞, so (fn) → f uniformly as n → ∞. Now, all fns are smooth, and (fn) → f uniformly as n → ∞, but f is not diﬀerentiable at 0.

Note: If a function f : [a, b] → R is continuously diﬀerentiable, it is called a C1 function, or, alternatively, f ∈ C1[a, b] is written, where C1[a, b] is the space of continuously diﬀerentiable functions on [a, b].

1 Theorem 29: Let (fn) be a sequence of C functions on [a, b], with (fn) → f pointwise as n → ∞ on 0 1 0 0 [a, b], and suppose that (fn) converges uniformly. Then f ∈ C [a, b] and limn→∞(fn) = f .

0 0 0 Proof: Let g = limn→∞(fn). Now, ∀n ∈ N, fn ∈ C[a, b], so g ∈ C[a, b] since (fn) → g uniformly, so Z x Z x Z x 0 0 FTC g = lim (fn) = lim fn = lim [fn(x) − fn(a)] = f(x) − f(a) a a n→∞ n→∞ a n→∞ so, since Z x f(x) = f(a) + g a R x 0 a g is continuous, so by the FTC, f = g = limn→∞(fn), so 0 0 FTC 0 lim (fn) = f = lim (fn) n→∞ n→∞

27 8 Functional Series

8.1 The Weierstrass M-Test & Other Useful Results

P∞ Note: Let (fn)n≥1 be a sequence of functions on A ⊂ R. Then thei functional series k=1 fk converges Pn pointwise (or uniformly) on A if the sequence of partial sums Sn = ( k=1 fk)n≥1 converges pointwise (or uniformly).

0 Pn Theorem 24 : Let (fn)n≥1 ⊂ R[a, b]. Suppose that Sn = k=1 fk converges uniformly. Then P∞ k=1 fk ∈ R[a, b] and ∞ ∞ Z b X X Z b fk = fk a k=1 k=1 a

Pn R b P∞ P∞ R b Proof: If (fk) ∈ R[a, b], then Sn = k=1 fk ∈ R[a, b] ∀n ∈ N, so a k=1 fk = k=1 a fk.

0 Pn Theorem 25 : Suppose that (fn)n≥1 ⊂ C[a, b] such that Sn = k=1 fk converges uniformly. Then P∞ k=1 fk ∈ C[a, b], and, ∀x ∈ [a, b],

∞ ∞ X X lim fk(x) = lim fk(x) x→x0 x→x0 k=1 k=1

P∞ P∞ Proof: Sn ∈ C[a, b] ∀n ∈ N, so limx→x0 k=1 fk(x) = k=1 limx→x0 fk(x).

0 1 0 Pn 0 Theorem 29 : If (fn)n≥1 ⊂ C [a, b] and Sn = k=1 fk converges uniformly on [a, b], then

∞ ∞ d X X d f (x) = f (x) dx k dx k k=1 k=1

Proof: The proof is left as an exercise.

Theorem 30 - Weierstrass M-Test: Let (fn)n≥1 be a sequence of functions on A ⊂ R. Then, if P∞ P∞ ∃(Mk)k≥1 ⊂ R such that, ∀x ∈ A, fn(x) < Mn and k=1 Mk converges, then n=1 fn converges uniformly.

P∞ Pn Proof: Mk converges, so ( Mk)n≥1 is Cauchy, so, ∀εn > 0 ∃Nε such that ∀n, m > Nε , k=1 k=1 n n Pn Pm Pn Pn with n > m, k=1 Mk − k=1 Mk < εn. Then k=m+1 Mk < εn, so, since MK ≥ 0, k=m+1 Mk < εn. Therefore

n m n n n X X X X X fk(x) − fk(x) = fk(x) ≤ fk(x) ≤ Mk < εn

k=1 k=1 k=m+1 k=m+1 k=m+1 P∞ This holds ∀x ∈ A, so the sequence of partial sums of fk is uniformly Cauchy, so k=1 fk converges uniformly to its pointwise limit.

P∞ P∞ Corollary 30.1: If (fn)n≥1 is such that k=1kfkk∞ converges, then k=1 fk converges uniformly.

28 8.2 Taylor & Fourier Series

Motivation: Taylor polynomials are of the form

n X 1 S = f (k)(a)(x − a)k n k! k=0 and were studied in Analysis II. Fourier series are of the form n 1 X a + (a cos kx + b sin kx), x ∈ [0, 2π] 2 0 k k k=1 Applications of functional series include solving PDEs, ODEs and time series.

Deﬁnition: For any f ∈ R[−π, π], the series

∞ a0 X + (a cos kx + b sin kx) 2 k k k=1 with coeﬃcients given by 1 Z π ak = f(x) cos kx dx, k ≥ 0 π −π 1 Z π bk = f(x) sin kx dx, k > 0 π −π is called the Fourier series generated by f.

Note: It is not immediately obvious for which classes of functions this series is convergent, or if it converges to f.

P∞ P∞ Theorem 31: Let (ak)k≥0, (bk)k≥1 ⊂ R, with k=0|ak| and k=1|bk| convergent. Then

a0 P∞ • 2 + k=1(ak cos kx + bk sin kx) converges uniformly on R. • The pointwise limit f : R → R is continuous. • f is 2π-periodic (f(x + 2π) = f(x)). Moreover, 1 Z π ak = f(x) cos kx dx, k ≥ 0 π −π 1 Z π bk = f(x) sin kx dx, k > 0 π −π so the terms of the series can be recovered.

. Proof (Convergence): ∀x ∈ R, |ak cos kx + bk sin kx| ≤ |ak cos kx| +|bk sin kx| ≤ |ak| +|bk| .= Mk. P∞ P∞ P∞ P∞ P∞ Then k=1 Mk = k=1|ak| +|bk| = k=1|ak| + k=1|bk|, so k=1 Mk converges, so the Fourier series converges by the M-test.

∞ . a0 P Proof (Continuity): Let f(x) = 2 + k=1(ak cos kx + bk sin kx). Then f ∈ C(R).

n . a0 P Proof (Periodicity): Let SN (y) = 2 + k=1(ak cos kx + bk sin kx). Then f(x + 2π) − f(x) = limn→∞ Sn(x+2π)−limn→∞ Sn(x). Both of these limits exist, so f(x+2π)−f(x) = limn→∞ Sn(x + 2π)− Sn(x) = 0, since cos (k(x + 2π)) = cos kx and sin (k(x + 2π)) = sin kx.

29 Lemma: ∀m ∈ Z Z π eimxdx = 2π1(m = 0) −π where 1(m = 0) is the indicator function for m = 0.

Proof: For m ∈ Z \{0}: Z π Z π Z π Z π 2 sin πm eimx dx = cos mx + i sin mx dx = cos mx dx + i sin mx dx = = 0 −π −π −π −π m For m = 0: Z π Z π Z π eimx dx = e0 dx = 1 dx = 2π −π −π −π So Z π imx e dx = 2π1(m = 0) −π

Proof (Recovery): Z π Z π ∞ a0 X f(x) dx = + (a cos kx + b sin kx) dx 2 k k −π −π k=1 Z π ∞ " Z π Z π # a0 X = 1 dx + a cos kx dx + b sin kx dx = πa 2 k k 0 −π k=1 −π −π so 1 Z π 1 Z π a0 = f(x) dx = f(x) cos 0x dx π −π π −π Now,

∞ " # ∞ " # Z π 1 Z π X Z π X Z π f(x) cos mx dx = a cos mx dx+ a cos kx cos mx dx + b sin kx cos mx dx 2 0 k k −π −π k=1 −π k=1 −π and Z π Z π 1 1 sin kx cos mx dx = (eikx − e−ikx) (eimx − e−imx) dx −π −π 2i 2 1 Z π = ei(k−m)x − ei(m−k)x + ei(k+m)x − ei(m+k)x dx 4i −π 1 = (2π1(k − m = 0) − 2π1(m − k = 0)) = 0 4i and Z π Z π 1 1 cos kx cos mx dx = (eikx + e−ikx) (eimx − e−imx) dx −π −π 2 2 1 Z π = ei(k−m)x + ei(m−k)x + ei(k+m)x + ei(m+k)x dx 4 −π 1 = 2π((1(k − m = 0) + 1(m − k = 0)) = π1(k = m) 4 then ∞ Z π X f(x) cos mx dx = (akπ1(k = m)) = πam −π k=1 so 1 Z π am = f(x) cos mx dx π −π

The proof for bk is similar and so is left as an exercise.

30 Note: This proof relies on an application of Theorem 24, where the sum is multiplied by a bounded function. The proof that the result holds is left as an exercise.

Corollary 31.1: If f : [−π, π] → R is a C2 function, ( or f 0 ∈ C[−π, π]), and f(π) = f(−π). Then the Fourier series generated by f converges to f uniformly.

Note: It must be established that (i) The Fourier series converges uniformly. (ii) The limit is f. However, (ii) is a fairly clunky argument using existing tools, and so shall be omitted and left as an exercise, whilst (i) shall be proved.

Proof:

Z π Z π Z π ! 1 1 0 1 π 0 bk = f(x) sin kx dx = f(x) cos kx dx = f(x) cos kx −π − f (x) cos kx dx π −π πk −π πk −π Z π Z π Z π ! 1 0 1 0 0 1 0 π 00 = f (x) cos kx dx = 2 f (x) sin kx dx = 2 f (x) sin kx −π − f (x) sin kx dx πk −π πk −π πk −π Z π 1 00 = − 2 f (x) sin kx dx πk −π

So Z π 1 1 00 2 00 cb |b | = f(x) sin kx dx ≤ f · sin k · 2π = f .= k 2 2 · ∞ 2 ∞ 2 πk −π πk k k

ca and cb is independent of k. A similar argument establishes that |ak| ≤ k2 for k > 0, where ca is also P∞ 1 ca+cb independent of k. k=1 k2 converges, so the Fourier series converges by the M-test with Mk = k2 for k > 0.

Example (Temperature Equilibration on a Non-Uniformly Heated Ring): Consider the unit circle S1. Let any point on it be described by the angle ϕ ∈ [−π, π] it makes with the x-axis, and let the initial temperature on the ring be given by f(ϕ) ∈ C2[−π, π] such that f(−π) = f(π). Then the temperature T at time t at the point at ϕ, for t > 0, ϕ ∈ [−π, π], is given by the Fourier heat equation

∂T ∂2T (t, ϕ) − (t, ϕ) = 0 ∂t ∂ϕ2 Since the point at π is the same as the point at −π, there are some boundary conditions to consider also:

T (t, −π) = T (t, π) ∂T ∂T (t, −π) = (t, π) ∂ϕ ∂ϕ The initial condition is given by T (0, ϕ) = f(ϕ) Throughout this example, ∂T ∂T ∂2T T˙ .= ,T 0 .= ,T 00 .= ∂t ∂ϕ ∂ϕ2

31 Now, (cos kϕ)0 = −k sin kϕ, and (cos kϕ)00 = −k2 cos kϕ, so A(t) cos kϕ seems a suitable ansatz to make for a solution. Substituting this into the heat equation yields A˙ cos kϕ + k2A cos kϕ = 0, so A˙(t) + k2A(t) = 0, which is an ODE. The boundary conditions are in fact satisfied by either this solution, or one using the ansatz B(t) sin kϕ, but neither of these satisfy the initial conditions. Since a finite linear combination of any of these functions will have the same shortcoming, it seems sensible to take the next ansatz as being an infinite linear combination of them, such as

∞ a0(t) X T (t, ϕ) = + (a (t) cos kϕ + b (t) sin kϕ) 2 k k k=1

Assume that T , T˙ , T 0 and T 00 all converge uniformly for t > 0, ϕ ∈ [−π, π]. Then substituting T into the heat equation and diﬀerentiating pointwise yields

∞ ∞ a˙0 X X + (a ˙ cos kϕ + ·b sin kϕ) + (a k2 cos kϕ + b k2 sin kϕ) = 0 2 k k k k k=1 k=1 So ∞ a˙0 X h i + (a ˙ + a k2) cos kϕ + (b˙ + b k2) sin kϕ = 0 2 k k k k k=1 which is the Fourier series generated by 0, so

a˙0 = 0 2 a˙k + k ak = 0 ˙ 2 bk + k bk = 0

for k > 0. This yields inﬁnitely many ODEs, which require inﬁnitely many initial conditions. Expanding f(ϕ) ∈ C2[−π, π] by a Fourier series yields

(f) ∞ a X (f) (f) f(ϕ) = 0 + a cos kϕ + b sin kϕ 2 k k k=1

(f) where nk denotes the coeﬃcient nk dependent on the function f. This gives the required amount of initial conditions. Then

(f) a0(t) = a0 (f) −k2t ak(t) = ak e (f) −k2t bk(t) = bk e So (f) ∞ a X 2 (f) (f) T (t, ϕ) = 0 + e−k t a cos kϕ + b sin kϕ 2 k k k=1

Whilst this is a solution, its uniqueness must also be established, however that shall not be done here.

Exercise: Check that T converges uniformly for t ≥ 0, and that T˙ , T 0, T 00 do so also for t > 0.

Note: Over time, the temperature becomes distributed evenly across the whole ring, since

a(f) 1 Z π lim T (t, ϕ) = 0 = f(ϕ) dϕ = Average Temperature. t→∞ 2 2π −π The approach to this equilibrium is exponential.

32 Part II Norms

9 Normed Vector Spaces

9.1 Deﬁnition & Basic Properties of a Norm

Deﬁnition 11: Let V be a vector space over R. Then a function k·k : V → R, with k·k : v ∈ V 7→kvk which satisﬁes the following: (i) ∀v ∈ V , kvk ≥ 0 (positivity). Moreover, kvk = 0 ⇐⇒ v = 0 ∈ V (the ability to separate points).

(ii) ∀λ ∈ R, ∀v ∈ V , kλvk =|λ| ·kvk (absolute homogeneity). (iii) ∀u, v ∈ V , ku + vk ≤kuk +kvk (triangle inequality). is called a norm on V . A pair (V,k·k) is called a normed vector space, or a normed space.

Remarks: • (ii) and (iii) use the linear structure of V .

• Positivity follows from (ii) and (iii), since, ∀v ∈ V , 0 = 0 ·kvk abs.= k0 · vk −kv − vk ≤kvk +k−vk = hom. kvk +|−1| ·kvk =kvk +kvk = 2kvk, so 0 ≤kvk.

Examples:

1. |·|, the absolute value, is a norm on R. The triangle inequality is satisﬁed since |x + y| ≤ |x| +|y|. The proofs of the other properties are left as an exercise.

2. k·k∞, the sup norm, is a norm on B[a, b].

(i) : ∀f ∈ B[a, b], if kfk∞ = 0, then supx∈[a,b] f(x) = 0, so, ∀x ∈ [a, b], 0 ≤ f(x) ≤ 0, so f ≡ 0 on [a, b], so k·k∞ can separate points (kf − gk∞ = 0 ⇐⇒ f = g, kf − gk∞ > 0 ⇐⇒ f 6= g).

(ii) : ∀λ ∈ R, ∀f ∈ B[a, b],kλfk∞ = supx∈[a,b] λf(x) = supx∈[a,b]|λ| f(x) =|λ| supx∈[a,b] f(x) = λkfk∞, so k·k∞ has absolute homogeneity.

(iii) ∀f, g ∈ B[a, b],kf + gk∞ = supx∈[a,b] f(x) + g(x) = supx∈[a,b]( f(x) + g(x) ) ≤ supx∈[a,b] f(x) +

supx∈[a,b] g(x) = kfk∞ +kgk∞, so kf + gk∞ ≤ kfk∞ +kgk∞, so k·k∞ satisﬁes the triangle equality.

Note: Norms generalise the notion of magnitude of vectors in Rn.

n n Proposition 34: Let x = (x1, . . . , xn) ∈ R (a vector). Then the following functions on R are norms: 1. n X kxk1 = |xi| (the taxicab or Manhattan norm) i=1 2. v u n uX 2 kxk2 = t xi (the Euclidean norm or Euclidean distance) i=1

3. kxk = max |xi| ∞ 1≤i≤n

33 Proof: Triangle inequality: 1. n n n n X X X X ku + vk1 = |ui + vi| ≤ (|ui| +|vi|) = |ui| + |vi| =kuk1 +kvk1 i=1 i=1 i=1 i=1

2. For any u, v ∈ Rn, n X u · v = ui · vi =kukn ·kvk2 · cos θ ≤kuk2 ·kvk2 i=1 for some θ ∈ [0, 2π]. This is known as the Cauchy-Schwarz inequality for Rn. Then

2 ku + vk2 = (u + v) · (u + v) n X = (ui + vi) · (ui + vi) i=1 = u · (u + v) + v · (u + v) Cauchy ≤ kuk2 ·ku + vk2 +kvk2 ·ku + vk2 Schwarz So 2 ku + vk2 ≤kuk2 ·ku + vk2 +kvk2 ·ku + vk2 The case where u + v = 0 is trivial to check, otherwise

ku + vk2 ≤kuk2 +kvk2

3. ku + vk = max |ui + vi| ≤ max (|ui| +|vi|) ≤ max |ui| + max |vi| =kuk +kvk ∞ 1≤i≤n 1≤i≤n 1≤i≤n 1≤i≤n ∞ ∞

The veriﬁcation of the other properties is left as an exercise.

Remark: All of these norms are instances of the following family of norms:

1  n  p X p kxkp =  |xi|  , p = 1, 2,... i=1

In particular, kxk = lim kxk ∞ p→∞ p That this family of functions is a family of norms follows from the Minkowski inequality:

1 1 1  n  p  n  p  n  p X p X p X p n  |xi + yi|  ≤  |xi|  +  |yi|  ∀x, y ∈ R i=1 i=1 i=1

n Note : R can be thought of as the space of functions on Nn = {1, 2, . . . , n} by considering x : Nn → R, with x : j 7→ xj.

34 9.2 Equivalence & Banach Spaces

Deﬁnition 12: Norms k·ka and k·kb on V are called equivalent (or Lipschitz equivalent) if ∃k1, k2 ≥ 0 such that, ∀v ∈ V , k1 ·kvkb ≤kvka ≤ k2 ·kvkb. This equivalence is denoted by k·ka ∼k·kb.

Remark: The equivalence of norms is an equivalence relation on the set of all norms on V . That is,

1. k·ka ∼k·ka (choose k1 = k2 = 1).

2. k·ka ∼k·kb ⇐⇒ k·kb ∼k·ka.

3. If k·ka ∼k·kb and k·kb ∼k·kc, then k·ka ∼k·kc. The proof is left as an exercise.

n Lemma 35: k·k1, k·k2 and k·k∞ are all equivalent on R .

Proof: Because equivalence is an equivalence relation, it is enough to prove that k·k1 ∼ k·k∞ and n k·k2 ∼k·k∞. Now, ∀x ∈ R , n n X X kxk = max |xi| ≤kxk = |xi| ≤ max xj = nkxk ∞ 1≤i≤n 1 1≤j≤n ∞ i=1 i=1

So k·k1 ∼k·k∞ with k1 = 1, k2 = n. There is a similar argument for the second part of the proof, although a much more general statement will be proved shortly, and so this shall be left as an exercise.

Deﬁnition 13: Let (V,k·k) be a normed space. Then

(i) A sequence (σn)n≥1 ∈ V converges to σ ∈ V if limn→∞ σn − σ = 0.

(ii) A sequence (σn)n≥1 ∈ V is Cauchy if, ∀ε > 0, ∃Nε such that, ∀n, m > Nε, σn − σm < ε. (iii) If every Cauchy sequence in V converges, then (V,k·k) is called Banach (or complete). A complete normed space is also called a Banach space.

Remarks:

• Any Cauchy sequence on (R,|·|) converges, so (R,|·|) is Banach.

• Consider (S[a, b],k·k∞). Let (sn)n≥1 ⊂ S[a, b] be a uniform Cauchy sequence, so (sn) is Cauchy with respect to k·k∞. Then sn → r ∈ R[a, b] as n → ∞, so, in general, (sn) doesn’t converge to a “point” in S[a, b] because not all regulated functions are step functions, so (S[a, b],k·k∞) is not Banach.

Note: All ﬁnite-dimensional normed spaces are complete. A proof for the case when the space is over R shall be given later.

Remarks:

1. If limn→∞ σn exists, then it is unique.

Suppose that σn → a ∈ V and σn → b ∈ V as n → ∞. Then 0 ≤ ka − bk = a − σn + σn − b ≤

a − σn + σn − b → 0 as n → ∞, so ka − bk = 0, so a = b by separation of points.

2. (R[a, b],k·k∞) is complete, because a uniformly Cauchy sequence of regulated functions converges to a regulated function. Also, (C[a, b],k·k∞) and (B[a, b],k·k∞) are complete by results demonstrated earlier.

35 3. Suppose that k·ka ∼k·kb. Then, if limn→∞ σn = σ with respect to k·ka, then limn→∞ σn = σ with respect to k·kb. The proof is left as an (important) exercise.

4. If k·ka ∼ k·kb, then (V,k·ka) is Banach iff (V,k·kb) is Banach. This is essentially a consequence of the previous remark. P∞ 5. (V,k·k) is Banach iff any series n=1 σn which converges absolutely converges to an element in P∞ P∞ V . n=1 σn converges absolutely if n=1 σn converges. The proof of this is left as an exercise, although the ⇐= direction is difficult.

Theorem 36: All norms on Rn are (Lipschitz) equivalent.

n n n Pn Proof: Let (e1, . . . , en) ⊂ R be the standard basis of R . Then, ∀x ∈ R , x = k=1 xkek. Now,

n n n n n X X X X X kxk = xkek ≤ xkek = |xk| ek ≤ max xj ek =kxk ek 1≤j≤n ∞ k=1 k=1 k=1 k=1 k=1 Pn n Let k2 = k=1 ek . Then ∃k2 such that, ∀x ∈ R , kxk ≤ k2kxk∞. Now, let ( ) kxk kxk . . . J = inf , A = : x 6= 0 , B = kxk :kxk∞ = 1 x6=0 kxk∞ kxk∞ Clearly B ⊂ A by deﬁnition. If α ∈ A, then ∃u, v ∈ Rn such that

kuk 1

α = = · u =kvk kuk∞ kuk∞ Then

u 1 kvk = = ·kuk = 1 ∞ kuk kuk ∞ ∞ ∞ ∞ so α ∈ B, so A ⊂ B, so B = A. Then J = inf A = inf B, and clearly J ≥ 0. n 1 Suppose that J = 0. Then ∃x1, x2, x3,... ∈ R with xk = k (so limk→∞ xk = 0) and xk = 1 ∀k. ∞ Now, it must be established that there is a convergent subsequence xkp ⊂ (xk)k≥1 such that p≥1 (1) (2) (n) n (1) (2) (n) xkp → x = (x , x , . . . , x ) ∈ R as p → ∞ with respect to k·k∞. Let xk = xk , xk , . . . , xk . (1) (1) (1) Since xk = 1, xk ≤ 1, so xk is bounded, so by the Bolzano-Weierstrass theorem ∃ xk ⊂ ∞ k≥1 p p≥1 (1) (1) xk which converges to x ∈ R. Applying this same argument to each component of xk k≥1 (j) (j) yields xkp such that xk converges to x ∈ R for any 1 ≤ j ≤ n, so xkp → x p≥1 p p≥1 p≥1 (j) (j) “component-wise” as p → ∞. Since xk → x , then, ∀ε > 0, ∃Nε(j) such that, ∀n > Nε(j), p p≥1 (j) (j) (j) (j) x − x < ε. Let Nε = max1≤j≤n Nε(j). Then, ∀k > Nε, xk − x = max1≤j≤n x − x < ε, kp p kp ∞ so limp→∞ xkp − x = 0, so xkp → x as p → ∞ with respect to k·k∞. ∞ p≥1

Now, 1 = x = x − x + x ≤ x − x + kxk , so kxk ≥ 1 − x − x > 0 since kp kp kp ∞ ∞ kp ∞ ∞ ∞ ∞ xkp − x ∞ → 0 as p → ∞, so x 6= 0 by the separation of points property of k·k∞.

Also, 0 ≤kxk = x − x + x ≤ x + x − x ≤ x +k x − x .(x ) ⊂ (x ), so x → kp kp kp kp kp 2 kp kp k kp ∞

0 as p → ∞, and it was established that x − xkp → 0 as p → ∞, so xkp + k2 x − xkp → 0 as ∞ ∞ p → ∞, so kxk = 0, so x = 0 by the separation of points property of k·k.

However, this is a contradiction, so J > 0, so let k1 = J. Then, from the deﬁnition of J, k1kxk∞ ≤kxk n n ∀x ∈ R \{0}, and if x = 0, then kxk∞ = kxk = 0, so k1kxk∞ = kxk, so k1kxk∞ ≤ kxk ∀x ∈ R , so n n n k1kxk∞ ≤kxk ≤ k2kxk∞ ∀x ∈ R , so k·k ∼k·k∞ on R , so all norms are equivalent on R .

36 Proposition 37: The space (Rn,k·k) is Banach for any norm k·k on Rn.

n Proof: It is enough to show that (R ,k·k∞) is complete, since if k·ka ∼ k·kb, then (V,k·ka) is Banach n ⇐⇒ (V,k·kb) is Banach, and k·k∞ ∼k·k for all norms k·k on R as shown previously.

n (j) (j) Consider any Cauchy subsequence (xk)k≥1 ⊂ R . Then, for any 1 ≤ j ≤ n, 0 ≤ xk+m − xk ≤ (j) (j) n (j) max1≤j≤n xk+m − xk = xk+m − xk → 0 as k → ∞ since (xk)k≥1 is Cauchy in R , so xk ∞ k≥1 (j) is Cauchy in R, so ∃x(j) ∈ R such that x → x(j) as k → ∞. Let x = (x(1), x(2), . . . , x(n)) ∈ Rn. It k was shown in the proof of the previous theorem that limk→∞ xk − x ∞ = 0, so (xk)k≥1 converges in n n n R with respect to k·k∞, so every Cauchy sequence in R converges with respect to k·k∞, so (R ,k·k∞) n n is complete, so (R ,k·k) is Banach for any norm k·k on R .

Corollary: (V,k·k) is Banach for any norm k·k on V , provided dim V is ﬁnite. This is because any n-dimensional vector space V over R is isomorphic to Rn, simply choose a basis in V .

Lemma (Cauchy-Schwarz Inequality for Integrals): ∀f, g ∈ C[a, b], s Z b Z b 2 fg ≤kfk2kgk2 , where kfk2 = f a a

Proof: ∀λ ∈ R, Z b Z b Z b Z b λ2 g2 + 2λ fg + f 2 = (f + λg)2 ≥ 0 (1) a a a a This is a polynomial in λ of degree 2 with at least one real root, so the discriminant

2 Z b ! Z b 2 2 D = 4 fg − 4kgk2kfk2 ≤ 0, so fg ≤kfk2kgk2 a a

Proposition 38: 1. The following functions are norms on C[a, b]: (a)

kfk∞ = sup f(x) (the sup norm) x∈[a,b]

(b) Z b kfk1 = |f| (a generalisation of the taxicab norm) a

2. (C[a, b],k·k∞) is Banach, but (C[a, b],k·k1,2) is not Banach.

3. ∀f ∈ C[0, 1], kfk1 ≤kfk2 ≤kfk∞. 4. No pair of norms (a), (b) or (c) is equivalent.

37 Proof:

1. (a) It was demonstrated that k·k∞ is a norm on B[a, b], and C[a, b] ⊂ B[a, b], so k·k∞ is a norm on C[a, b]. (b) Separation of points: Assume that ∃f such that kfk = 0. Then R b|f| = 0. Assume that 1 a f 6≡ 0. Then ∃x0 ∈ [a, b] such that f(x0) > 0. But f ∈ C[a, b], so ∃[t1, t2] ⊂ [a, b] such that

R b R t2 f > 0. Now, 0 =kfk = |f| ≥ |f| > 0, so 0 > 0, which is clearly a contradiction, 1 a t1 [t1,t2] so f ≡ 0. R b R b R b Absolute homogeneity: ∀λ ∈ R, kλ · fk1 = a |λ · f| = a (|λ| ·|f|) =|λ| · a |f| =|λ| ·kfk1 R b R b R b R b Triangle inequality: ∀f, g ∈ C[a, b], kf + gk1 = a |f + g| ≤ a (|f| +|g|) = a |f| + a |g| = kfk1 +kgk1 (c) The proofs of the ﬁrst two properties are very similar to those in (b), and so they are left as exercises. Triangle inequality: Using the Cauchy-Schwarz inequality Z b Z b Z b 2 2 kf + gk2 = (f + g) = f(f + g) + g(f + g) ≤kfk2kf + gk2 +kgk2kf + gk2 a a a Then, assuming that (f + g) 6≡ 0,

kf + gk2 ≤kfk2 +kgk2 If f + g ≡ 0 then the proof is trivial.

2. Consider the sequence of functions (fk)k≥1 ⊂ C[a, b] given by  a+b 1 −1 a ≤ x < 2 − k  a+b a+b 1 a+b 1 fk(x) = k(x − 2 ) 2 − k ≤ x ≤ 2 + k  a+b 1 1 2 + k < x ≤ b Then, for any k, m ∈ N, a+b 1 Z b Z 2 + k kfk − fk+mk1 = |fk − fk+m| = |fk − fk+m| a+b 1 a 2 − k

since the functions are equal outside of these bounds. Now, fk is bounded by [−1, 1], so|fk − fk+m| ≤ 2, so a+b + 1 Z 2 k 2 4 kfk − fk+mk1 ≤ 2 = 2 · = → 0 as k → ∞ a+b 1 k k 2 − k

so (fk)k≥1 is Cauchy with respect to k·k1. Suppose that it has a limit f ∈ C[a, b]. Then, ∀ε > 0, a+b a+b 1 R b f 2 − ε = −1 and f 2 + ε = 1, since otherwise, for any k > ε , a |fk − f| > 0 which contradicts f being the limit of (fk)k≥1. But then f is not continuous, since −1 = lim f(x) 6= lim f(x) = 1 a+b a+b x↑ 2 x↓ 2

This is a contradiction, so (fk)k≥1 does not have a limit in C[a, b] with respect to k·k1, but it is Cauchy with respect to k·k1, so (C[a, b,k·k1) is not Banach.

3. For any f ∈ C[0, 1], and some x0 ∈ [0, 1] Z 1 Z 1 kfk1 = |f| ≤ kfk∞ =kfk∞ 0 0 Z 1 kfk1 = |f| · 1 ≤kfk2 ·k1k2 =kfk2 0 1 1 ! 2 1 ! 2 1 2 1 Z 2 2 2 2 2 kfk2 = f ≤ f = sup f(x) = f(x0) = f(x0) =kfk∞ 0 ∞ x∈[0,1]

38 4. This is mostly left as an exercise, however a proof that k·k1 6∼ k·k2 will be given. For simplicity, this shall be proven with respect to C[0, 1], the generalisation is left as an easy exercise. Let, for any ε > 0,  √1  ε 0 ≤ x < ε fε(x) = ∈ C[0, 1] √1  x ε ≤ x ≤ 1 Then Z 1 Z ε 1 Z 1 1 √ √ √ √ √ ε kfεk1 = |fε| = dx + dx = ε + 2 x 1 = 2 − ε ≤ 2 0 0 ε ε x Z 1 Z ε Z 1 2 2 1 1 1 1 kfεk2 = fε = dx + dx = [1 + log x]ε = 1 + log → ∞ as ε ↓ 0 0 0 ε ε x ε

So @k > 0 such that kfk2 ≤ kkfk1 ∀f ∈ C[0, 1] .

Exercise: Using a proof similar to the one in 3, show that (C[a, b],k·k2) is not Banach.

39 10 Continuous Maps

10.1 Deﬁnition & Basic Properties of Linear Maps

Deﬁnition 14: Let f : (V,k·kV ) → (W,k·kW ) be a map from V to W . Then f is called (k·kV ,k·kW ) continuous (or simply continuous when the spaces in question are clear) at u ∈ V if, ∀ε > 0, ∃δε > 0

such that, ∀v ∈ V , ku − vkV < δε =⇒ f(u) − f(v) W < ε.

Example: Let I : (C[a, b],k·k∞) → (R,|·|) be given by Z b I : f ∈ C[a, b] 7→ f ∈ R a It was demonstrated in the ﬁrst section that this map is linear.

Theorem 39: Let T : (V,k·kV ) → (W,k·kW ) be a linear map. Then the following conditions are equivalent:

(i) T is continuous at 0V ∈ V . (ii) T is continuous on V (T is continuous at every point of V ). n o : (iii) The set T (x) W x ∈ V such that kxkV ≤ 1 ⊂ R is bounded (then T is called bounded).

Proof (i) ⇐⇒ (ii): The ⇐= direction is clear, if T is continuous on V , it is continuous at 0V ∈ V .

For the =⇒ direction, suppose that T is continuous at 0V . T (0V ) = 0W , so, ∀ε > 0, ∃δε > 0 such that kxkV < δε =⇒ T (x) < ε. Choose some u ∈ V . Then, for any v ∈ V , let x = u − v. Now, ∀ε > 0, W ∃δε > 0 such that ku − vkV < δε =⇒ T (u) − T (v) W = T (u − v) W < ε, so T is continuous at u. Since the choice of u was arbitrary, T is continuous on V .

Proof (i) ⇐⇒ (iii): For the =⇒ direction, by continuity, ∀ε > 0, ∃δε > 0 such that, ∀x ∈ V , ˜ δε x x ε kxk ≤ δε = =⇒ T (x) < ε. Then ≤ 1 =⇒ T ≤ , so ∀x ∈ V , kxk ≤ 1 =⇒ V 2 W δε δε δε V V W ε n o ε T (x) < , so T (x) : x ∈ V such that kxk ≤ 1 is bounded by , so it is bounded. W δ˜ε W V δ˜ε

For the ⇐= direction, T is bounded, so ∃B > 0 such that ∀u, kukV ≤ 1 =⇒ T (u) < B. Then, W ε εu ε εu for any ε > 0, multiplying this inequality by B yields B ≤ B =⇒ T B < ε, so ∀v ∈ V , V W ε ε kvkV < δε = B =⇒ T (v) W < ε, so T is continuous at 0V with δε = B .

Example: For any f ∈ C[a, b], kfk∞ ≤ 1, Z b ≤kfk∞ ·|b − a| ≤|b − a| a

so I : C[a, b] → R, Z b I : f ∈ C[a, b] 7→ f ∈ R a

is bounded, so I is continuous on (C[a, b],k·k∞).

40 Proposition 40: Let V and W be vector spaces, with norms k k ∼ k k and k k ∼ k k · V1 · V2 · W1 · W2 respectively, and let T : V → W be a linear map. Then (i) T is (k k ,k k ) continuous iﬀ T is (k k ,k k ) continuous. · V1 · W1 · V2 · W2 n (ii) Any linear map (R ,k·k1) → (W,k·kW ) is continuous. n (iii) Any linear map (R ,k·k) → (W,k·kW ) is continuous, where k·k is an arbitrary norm.

(iv) Let F : (C[0, 1],k·k1,∞) → (R,|·|) be an evaluation map with F : f 7→ f(0) ∈ R. Then F is (k·k∞ ,|·|) continuous but not (k·k1 ,|·|) continuous.

Proof: (i) T is (k k ,k k ) continuous, so T is bounded with respect to (k k ,k k ), so ∃B ≥ 0 such that · V1 · W1 · V1 · W1 ∀u ∈ V , kuk ≤ 1 =⇒ T (u) ≤ B. Since k k ∼ k k , ∃K > 0 such that, ∀u ∈ V , kuk ≤ V1 W1 · V1 · V2 V1 Kkuk . Consider all u ∈ V such that Kkuk ≤ 1. Then kuk ≤ 1, so T (u) ≤ B. Now, V2 V2 V1 W1 k k ∼ k k , so ∃L > 0 such that, ∀v ∈ W , Lkvk ≤ kvk , so L T (u) ≤ T (u) ≤ B. · W1 · W2 W2 W1 W2 W1 Then, Kkuk ≤ 1 =⇒ L T (u) ≤ B. Let w = Ku. Then, ∀w ∈ V , kwk ≤ 1 =⇒ V2 W2 V2 T (w) ≤ KB = B0, so T is bounded with respect to (k k ,k k ), so T is (k k ,k k ) W2 L · V2 · W2 · V2 · W2 continuous. n n (ii) For any x ∈ R , ∃x1, . . . , xn ∈ R such that, for some basis e1, . . . , en ∈ R ,

n n X X x = xiei, so kxk1 = |xi| i=1 i=1

n Now, consider any x ∈ R with kxk1 ≤ 1. Then   n n X X T (x) = T  xiei = xiT (ei) W i=1 i=1 W W n n X X . ≤ |xi| T (ei) ≤ max T (ei) |xi| ≤ max T (ei) .= B W 1≤i≤n 1≤i≤n i=1 i=1

n So ∃B > 0 such that, ∀x ∈ R with kxk1 ≤ 1, T (x) W ≤ B, so T is bounded, so T is continuous. (iii) This is a direct consequence of (i) and (ii), since all norms on Rn are equivalent.

(iv) For F : (C[0, 1],k·k ) → (R,|·|), take any f ∈ C[0, 1] such that kfk ≤ 1. Then f(0) ≤ 1, so ∞ ∞ F (f) ≤ 1, so F is bounded by 1, so F is continuous.

For F : (C[0, 1],k·k1) → (R,|·|), let (fn)n≥1 ⊂ C[a, b] be the sequence given by fn : [0, 1] → R, fn : 1 x 7→ ( n − n)x + n. Then

Z 1 Z 1 1 1 kfnk1 = fn = − n x + n dx = ≤ 1 ∀n ∈ N 0 0 n n n o However, F (fn) = fn(0) = n → ∞ as n → ∞, so F (f) :kfk1 ≤ 1 is not bounded, so F is not continuous.

41 10.2 Spaces of Bounded Linear Maps

Theorem 41: Suppose (V,k·kV ) and (W,k·kW ) are normed spaces, let L(V,W ) = {T : V → W : T is linear and continuous/bounded}

and let : k·k T ∈ L(V,W ) 7→ sup T (x) W x∈V :kxkV ≤1 Then (L(V,W ),k·k) is a normed vector space.

Proof: For any λ, µ ∈ R, T,G ∈ L(V,W ), x ∈ V ,(λT + µG)(x) = λT (x) + µG(x) - a map from V to W . Take any α, β ∈ R, u, v ∈ V . Then (λT + µG)(αu + βv) = λT (αu + βv) + µG(αu + βv) = λ(αT (u) + βT (v)) + µ(αG(u) + βG(v)) = α(λT + µG)(u) + β(λT + µG)(v), so (λT + µG) is linear. Now, kλT + µGk = sup λT (x) + µG(x) ≤ |λ| sup T (x) +|µ| sup G(x) = kxkV ≤1 W kxkV ≤1 W kxkV ≤1 W |λ|kT k +|µ|kGk. Then, since T and G are both bounded, λT + µG is bounded, so λT + µG is continuous, so L(V,W ) is closed under linear combinations of maps. Veriﬁcation of the remaining axioms of a vector space is left as an exercise. It remains to show that k·k is a norm. Taking λ = µ = 1 in the previous section of the proof yields kT + Gk ≤kT k+kGk which proves the triangle inequality. The proof of absolute homogeneity is left as an

easy exercise. To prove separation of points, suppose that kT k = 0L(V,W ). Then supkuk ≤1 T (u) = V W 0L(V,W ), so T (u) = 0L(V,W ) ∀u ∈ V such that kukV ≤ 1. Since k·kW is a norm, T (x) = 0L(V,W ) ∀u ∈ W v V such that kuk ≤ 1. Now, take any v ∈ V such that kvk > 1. Then T (v) =kvk · T V V W V kvkV W = 0L(V,W ), so T (x) = 0L(V,W ) ∀x ∈ V , so T = 0L(V,W ), which completes the proof.

Remarks: 1. The norm k·k is called an operator norm. 2. Consider L(Rm, Rn) - this is the space Rm×n of all m × n matrices. n 3. Suppose then that m =√n and that both of these R s are equipped with the k·k2 norm. Then for n×n 0 0 any T ∈ R , kT k = λmax, where λmax is the maximum eigenvalue of TT , with T being the transpose of T .

4. If (W,k·kW ) is Banach, then (L(V,W ),k·k) is Banach also.

√ Exercise: Prove 4 and that kT k = λmax as given in 3 is a norm.

42 11 Open and Closed Sets of Normed Spaces

11.1 Deﬁnition & Basic Properties of Open Sets

Deﬁnition 15: Let (V,k·k) be a normed space, and let u ∈ V , δ > 0. Then the set B(u, δ) = {v ∈ V :kv − uk < δ} ⊂ (V,k·k) is called an open ball of radius δ centred at u. B(u, δ,k·k) denotes an open ball deﬁned with respect to k·k, B(u, δ) is shorthand when the norm in question is obvious.

Examples:

1. For (R,|·|), B(x, δ) = (x − δ, x + δ). 2 2 2. For (R ,k·k1), B1(0, 1) = {(x1, x2 ∈ R : x1 + x2 < 1)}. 2 2 p 2 2 For (R ,k·k2), B2(0, 1) = {(x1, x2 ∈ R : x1 + x2 < 1)}. 2 2 For (R ,k·k∞), B∞(0, 1) = {(x1, x2 ∈ R : max(x1, x2) < 1)}.

3. For (C[a, b],k·k∞), B(g, ε) = {f ∈ C[a, b] : supx∈[a,b] f(x) − g(x) < ε}

Deﬁnition 15.1: Let (V,k·k) be a normed space. A subset U ⊂ V is called open if, ∀x ∈ V , ∃δx such that B(x, δx) ⊂ U.

Exercise: Prove that the empty set is open.

Lemma 42: Let (V,k·k) be a normed space. Then (i) Open balls in V are open sets.

(ii) ∀u ∈ V, δ > 0, B(u, δ) = u + δB(0, 1) .= {v ∈ V : v = u + δw, w ∈ B(0, 1)}.

Proof: (i) v ∈ B(u, δ) ⇐⇒ kv − uk < δ. Let δ0 = δ − kv − uk > 0. Consider B(v, δ0). Then, for any w ∈ B(v, δ0), kw − vk < δ0. Then ku − wk =ku − v + v − wk ≤ku − vk +kv − wk

(ii) Take v ∈ B(u, δ), then ku − vk < δ, so u−v < 1. Let w = u−v . Then w ∈ B(0, 1), so v = u + δw δ δ with w ∈ B(0, 1), so v ∈ u + δB(0, 1), so B(u, δ) ⊆ u + δB(0, 1). Now, take any v ∈ u + δB(0, 1). Then ∃w ∈ B(0, 1) such that v = u + δw, so ku − vk = kδwk = δkwk < δ, so v ∈ B(u, δ). Then u + δB(0, 1) ⊆ B(u, δ), so B(u, δ) = u + δB(0, 1).

Lemma 43: Let k·kA ∼k·kB be equivalent norms on V . Then U ⊂ V is open with respect to k·kA iﬀ U ⊂ V is open with respect to k·kB.

Proof: Assume without loss of generality that V is open with respect to k·kA. Then, ∀u ∈ U, ∃δu > 0 0 0 0 such that B(u, δu,k·kA) ⊂ U. Now, consider some δu > 0. Then v ∈ B(u, δu,k·kB) ⇐⇒ kv − ukB < δu. 0 0 δu 0 Sincek·kA ∼k·kB, ∃K > 0 such thatkv − ukA < Kkv − ukB < Kδu. Let δu = K . Then B(u, δu,k·kB) ⊂ B(u, δu,k·kA), so V is open with respect to k·kB.

n Example: If v ∈ R is open with respect to k·k1, then V is open with respect to an arbitrary norm k·k on Rn since all norms on Rn are equivalent.

43 11.2 Continuity, Closed Sets & Convergence

Deﬁnition 15.2: A function f : (V,k·kV ) → (W,k·kW ) is continuous at x ∈ V if, ∀ε > 0, ∃δ > 0 such that f(B(x, δ,k·kV )) ⊂ B(f(x), ,k·kW ).

Exercise: Prove that this deﬁnition is equivalent to the previous deﬁnition of continuity.

Deﬁnition 15.3: A set U ⊂ (V,k·k) is bounded if it is contained in a ball of ﬁnite radius. Equivalently, U ⊂ V is bounded if ∃x ∈ V, δ > 0 such that U ⊂ B(x, δ).

Proposition 44: Let f : (V,k·kV ) → (W,k·kW ). Then f is continuous on V iﬀ the preimage of any open subset U ⊂ W is open. Equivalently, f : V → W is continuous iﬀ, ∀U ⊂ W such that U is open, f −1(U) .= {x ∈ V : f(x) ∈ U} is open.

Proof =⇒ : Let U ⊂ W be an open subset. Then, either f −1(U) = ∅ which is open, or f −1(U) 6= ∅. If f −1(U) 6= ∅, then let u ∈ f −1(U) ⊂ V be any point in the preimage. Let v = f(u) ∈ U. Then ∃ε > 0 such that B(v, ε,k·kW ) ⊂ U. But f is continuous on V , so f is continuous at u, so ∃δε > 0 such that −1 −1 f(B(u, δε,k·kV )) ⊂ B(v, ε,k·kW ) ⊂ U, so B(u, δε,k·kV ) ⊂ f (U), so f (U) is open.

−1 Proof ⇐= : Take any u ∈ V , and let v = f(u). For any ε > 0, consider B(v, ε,k·kW ). f (B(v, ε,k·kW )) −1 −1 is open, and u ∈ f (B(v, ε,k·kW )). Since f (B(v, ε,k·kW )) is open, ∃δε such that B(u, δε,k·kV ) ⊂ −1 f (B(v, ε,k·kW )), so f(B(u, δε,k·kV )) ⊂ (B(v, ε,k·kW ), so f is continuous on V .

Deﬁnition 16: A set U ⊂ (V,k·k) is called closed if V \ U is open.

Examples:

(i) [a, b] ⊂ R is closed since R \ [a, b] = (−∞, a) ∪ (b, ∞) is open. (ii) [a, b) ⊂ R is neither open nor closed (it is not open since B(a, ε) 6⊂ [a, b) for any ε > 0, and it is not closed since R \ [a, b) = (−∞, a) ∪ [b, ∞) is not open).

Theorem 45: Let (V,k·k) be a normed space. Then U ⊂ V is closed iﬀ, for any sequence (xn) ⊂ U which converges to x ∈ V , x ∈ U.

Proof =⇒ : Let U ⊂ V be a closed subset. Assume that U 6= ∅, the case when U = ∅ is trivial.

Then let (xn) ⊂ U be a convergent sequence, with x = limn→∞ xn, so limn→∞ x − xn = 0. Suppose that x ∈/ U, then x ∈ V \ U. Since U is closed, V \ U is open, so ∃δ > 0 such that B(x, δ) ⊂ V \ U.

However, since xn → x as n → ∞, ∃Nδ such that, ∀n > Nδ, x − xn < δ, so xn ∈ B(x, δ) ⊂ V \ U, which is a contradiction since xn ∈ U, so x ∈ U.

Proof ⇐= : Now, any sequence in U which converges does so to a point in U. Suppose that U is not closed. Then V \U is not open, so ∃x ∈ V \U such that, ∀ε > 0, B(x, ε) 6⊂ V \U, so B(x, ε)∩U 6= ∅. Let

(εn) be any null sequence. Then, for any n ∈ N, B(x, εn) ∩ U 6= ∅, so ∃xn ∈ U such that x − xn < εn,

so limn→∞ x − xn = 0. Then (xn) ⊂ U is a sequence which converges to a point x ∈/ U, which is a contradiction, so U is closed.

Notes: Consider the normed space (V,k·k). For any x ∈ V and any δ > 0, B(x, δ) ⊂ V , so V is open. However, any sequence in V which converges does so in V , so V is closed also. ∅ is also both closed and open, since ∅ = V \ V . These sets are called clopen.

44 11.3 Lipschitz Continuity, Contraction Mappings & Other Remarks

Remarks: Let (V,k·k) be a normed space. Then 1. The map k·k : (V,k·k) → (R,|·|) is continuous. To see this, ﬁx any u ∈ V . Then, for any v ∈ V ,

kuk −kvk ≤ ku − vk by the triangle inequality. Now, if ku − vk < ε, then kuk −kvk < ε, so B(u, ε,k·k) ⊂ B kuk , ε,|·| , so setting δε = ε demonstrates that k·k : V → R is continuous.

2. Suppose that, for a map f : (V,k·k) → (V,k·k), ∃K ∈ (0, 1) such that, ∀u, v ∈ V , f(u) − f(v) ≤ ε Kku − vk. Then f is continuous on V . To see this, note that f(B(v, 2K )) ⊂ B(f(v), ε), the details of the proof are left as an exercise. Such a map is called a contraction mapping. In fact, if this condition holds for any k > 0, continuity is preserved, and the map is called Lipschitz continuous. 3. A closed ball of radius δ centred at u is deﬁned as B(u, δ) .= {v ∈ V :ku − vk ≤ δ} ⊂ (V,k·k). This set is closed. To see this, let (un)n≥1 ⊂ B(u, δ) such that un → w ∈ V as n → ∞. Then, by the

continuity of k·k at u ∈ V , kw − uk = limn→∞ un − u ≤ δ, so w ∈ B(u, δ), so B(u, δ) is closed. 4. Let U ⊂ (V,k·k). Then w ∈ V is a boundary point of U if, ∀ε > 0, B(w, ε) ∩ U 6= ∅ and B(w, ε) ∩ V \ U 6= ∅. w ∈ V is a limit point of U if ∃(xn)n≥1 ⊂ U which converges to w.

45 12 The Contraction Mapping Theorem & Applications

12.1 Statement and Proof

Theorem 46 - The Contraction Mapping/Banach Fixed Point Theorem: Let (V,k·k) be a Banach space, U ⊂ V a closed subset of this space, and f : U → U a contraction mapping. Then ∃!w ∈ U such that f(w) = w. Moreover, for any u ∈ U, the sequence (un)n≥0 ⊂ U converges to w as n → ∞, where u0 = u and un+1 = f(un).

Proof: f is a contraction mapping, so ∃K ∈ (0, 1) such that, ∀u, v ∈ U, f(u) − f(v) ≤ Kku − vk.

2 n Then un+1 − un = f(un) − f(un−1) ≤ K un − un−1 ≤ K un−1 − un−2 ≤ ... ≤ K u1 − u0 . Now,

un+m − un = un+m − un+m−1 + un+m−1 − un+m−2 + un+m−2 − ... + un+1 − un

≤ un+m − un+m−1 + ... + un+1 − un n+m−1 n+m−2 n ≤ (K + K + ... + K ) u1 − u0 ∞ X n+j ≤ K u1 − u0 j=0 n K = u1 − u0 → 0 as n → ∞, 1 − K

so (un)n≥0 is Cauchy. Since (V,k·k) is Banach and U is closed, (un) → w ∈ U as n → ∞. Now, consider un+1 = f(un). Then limn→∞ un+1 = limn→∞ f(un). limn→∞ un+1 = w, and limn→∞ f(un) = f(limn→∞ un) = f(w) since f is a contraction mapping and therefore Lipschitz continuous. Then w = f(w), so w is a ﬁxed point of f. It remains to establish the uniqueness of w. Suppose that ∃v ∈ U such that v = f(v). Then kw − vk =

f(w) − f(v) ≤ Kkw − vk, so (1 − K)kw − vk ≤ 0. Now, (1 − K) > 0 and kw − vk ≥ 0, so kw − vk = 0, so w = v since k·k separates points.

Example: Let α ∈ (0, 1). Then, using the Contraction Mapping Theorem, it can be shown that the equation x = e−αx, x ∈ [0, 1], has a unique solution x∗ ∈ (0, 1).

Now, (R,|·|) is Banach, and U = [0, 1] is closed. It remains to show that f : [0, 1] → [0, 1], f : x ∈ [0, 1] 7→

e−αx, is a contraction mapping. ∀x, y ∈ [0, 1], f(x) − f(y) = f 0(ξ)(x − y) = −αe−αξ(x − y) ≤ α|x − y| for some ξ ∈ (x, y) by the Mean Value Theorem, which can be applied since f is continuously diﬀerentiable. Then, since α ∈ (0, 1), f is a contraction mapping, so the Contraction Mapping Theorem can be applied, so ∃!x∗ ∈ (0, 1) such that x∗ = f(x∗). Then x∗ solves the equation x = e−αx. However, the Contraction Mapping Theorem not only guarantees the existence of x∗, it provides a 1 1 method for constructing it, or at least an approximation to it. Let α = 2 and take x0 = 2 . Then 1 x = 0 2 − 1 x1 = e 4 ≈ 0.78

x2 ≈ 0.68

x3 ≈ 0.71

x4 ≈ 0.70

x5 ≈ 0.70

∗ 1 1 It happens that x∞ = x = W ( 2 ), where W is Lambert’s W -function, with W ( 2 ) ≈ 0.70.

46 12.2 The Existence and Uniqueness of Solutions to ODEs

Deﬁnition: An ordinary diﬀerential equation (ODE) is of the form dy (t) = y˙(t) = F (t, y(t)), t ∈ A ⊂ dt R

y(t0) = y0, t0 ∈ A with y(t) the unknown to be solved for subject to the conditions stated. The second equality is known as the intial condition(s).

Remark: Applying R t to the first equality in the ODE yields t0 Z t Z t y˙ = y(t) − y(t0) = F (τ, y(τ)) dτ t0 t0 so Z t y(t) = y(t0) + F (τ, y(τ)) dτ t0 2 Suppose that F : R → R is continuous, and that y ∈ C[t0 − δ, t0 + δ] for some δ > 0. Then y(t0) + R t F (τ, y(τ)) dτ is well-defined and differentiable, and so y(t) solves the ODE. t0

Deﬁnition: The operator Z t . T (f) .= y0 + F (τ, f(τ)) dτ t0

acting from C[t0 − δ, t0 + δ] to C[t0 − δ, t0 + δ] is called the Pickard operator. An operator is simply a mapping from one vector space to another, and in many contexts they are linear, however the Pickard operator is not.

Remark: A ﬁxed point of T occurs when f = T (f), so Z t f(t) = y0 + F (τ, f(τ)) dτ t0 so f solves the ODE.

Theorem 47 - The Picard-Lindel¨ofTheorem: Suppose that F ∈ C[t0 − a, t0 + a] × [y0 − ε, y0 + ε] for some a, ε > 0, and that F is Lipschitz continuous on D = [t0 − a, t0 + a] × [y0 − ε, y0 + ε] with respect to y, that is, ∃L > 0 such that, ∀(t, y), (t, y0) ∈ D, F (t, y) − F (t, y0) ≤ L y − y0 . Then ∃δ ∈ (0, a] such that the ODE as given in the deﬁnition has a unique solution in C[t0 − δ, t0 + δ].

Remark: Suppose that ∂F ∈ C(D) ∂y Then, ∀(t, y)(t, y0) ∈ D and for some ξ(t, y, y0) ∈ D,

0 MVT ∂F 0 0 ∂F 0 F (t, y) − F (t, y ) = (ξ(t, y, y )) y − y ≤ y − y ∂y ∂y ∞ so F is Lipschitz continuous with respect to y, with

∂F L = ∂y ∞

47 Proof: The aim of the proof is to ﬁnd some δ > 0 such that T acts on C[t0 − δ, t0 + δ] and is a contraction mapping for some closed U ⊂ C[t0 − δ, t0 + δ].

For any δ > 0, T : C[t0 − δ, t0 + δ] → C[t0 − δ, t0 + δ]. Let y0 represent the constant function of value y0, and consider B(y0, ε) = {f ∈ C[t0 − δ, t0 + δ] :kf − y0k∞ ≤ ε} ⊂ C[t0 − δ, t0 + δ]. For any f ∈ B(y0, ε),

Z t T (f) − y = y + F (τ, f(τ)) dτ − y ≤ sup |F | ·|t − t | ≤ sup |F | · δ 0 ∞ 0 0 0 t (τ,y)∈D (τ,y)∈D 0 ∞ So choose δ > 0 suﬃciently small such that

sup |F | · δ ≤ ε (τ,y)∈D

This can be done since sup |F | < ∞ (τ,y)∈D because F is continuous on D and D is bounded. Then T : B(y0, ε) → B(y0, ε).

Now, ∀f, g ∈ B(y0, ε),

Z t T (f) − T (g) = [F (τ, f(τ)) − F (τ, g(τ))] dτ ∞ t 0 ∞

Z t

≤ L f(τ) − g(τ) dτ t 0 ∞

≤ Lkf − gk∞kt − t0k∞

≤ Lδkf − gk∞

So choose δ > 0 suﬃciently small such that Lδ < 1 also. Then

1. (C[t0 − δ, t0 + δ],k·k∞) is Banach.

2. B(y0, ε) ⊂ C[t0 − δ, t0 + δ] is closed.

3. T : B(y0, ε) → B(y0, ε) is a contraction mapping.

Then, by the Contraction Mapping Theorem, T has a unique ﬁxed point y ∈ C[t0 − δ, t0 + δ] such that

Z t y(t) = y0 + F (τ, y(τ)) dτ t0 which solves the ODE as given in the deﬁnition.

Note: Because of the Contraction Mapping Theorem, it is possible to construct this solution iteratively. This process is known as Picard’s iteration. The iteration is given by y(n+1)(t) = T (y(n)(t)).

Example: Consider the ODE

y˙(t) = py(t) y(0) = 0

2 Then y ≡ 0 is a solution. However, y (t) = t is also a solution. This can occur because F is not 1 √ 2 4 Lipschitz in this case, y is much greater than y close to 0, so there is no L which bounds the diﬀerence between these functions.

48 Example: Consider the ODE

y˙(t) = 2ty(t) y(0) = 1 with D = [−1, 1] × [0, 2]. Then

F (t, y) = 2ty

(t0, y0) = (0, 1)

This satisﬁes the conditions of the Picard-Lindel¨ofTheorem, so a unique solution exists, and can be constructed iteratively using Picard’s iteration, in this case

Z t (n+1) (n) (n) y = T (y ) = y0 + 2τy (τ) dτ 0

(0) Now, taking y (t) = y0 = 1 the initial approximation to the solution,

y(0) = 1 Z t Z t y(1) = 1 + 2τy(0)(τ) dτ = 1 + 2τ dτ = 1 + t2 0 0 Z t Z t Z t t4 y(2) = 1 + 2τy(1)(τ) dτ = 1 + 2τ(1 + τ 2) dτ = 1 + 2τ + 2τ 3 dτ = 1 + t2 + 0 0 0 2 . .

Using an inductive argument, it can be shown that

 n  t4 t2n X t2k y(n)(t) = 1 + t2 + + ... + = 2 n!  k!  k=0

since y(0) is of this form, and, assuming the statement holds for y(n),

Z t y(n+1)(t) = 1 + 2τy(n)(τ) dτ 0 ! Z t τ 4 τ 2n = 1 + 2τ 1 + τ 2 + + ... + dτ 0 2 n! Z t 2τ 2n = 1 + 2τ + 2τ 3 + τ 5 + ... + dτ 0 n! t4 t2n t2(n+1) = 1 + t2 + + ... + + 2 n! (n + 1)!

which is of the correct form. Then, with respect to k·k∞,

∞ 2k X t 2 y(t) = lim y(n)(t) = = et n→∞ k! k=0 which satisﬁes the ODE.

1 Exercise: Check the existence and uniqueness of a solution to this ODE on C[−δ, δ], where δ ∈ (0, 4 ).

49 12.3 The Jacobi Algorithm

Motivation: Let A ∈ Rn×n be an n × n real-valued matrix, and let b ∈ Rn be a real-valued n- dimensional vector. Then, if det A 6= 0, the equation Ax = b has the unique solution x = A−1b for some x ∈ Rn. However, inverting A is computationally expensive, typically O(n3). Whilst not a problem for small n, if, for example, n = 108, this quickly becomes impractical to compute.

However, if A = D + R for some D,R ∈ Rn×n, where D is diagonal and non-degenerate (det D 6= 0) and R is “small”, so     d11 0 ...... 0 0 r12 ...... r1n  0 d 0 ... 0  r 0 r . . . r   22   21 23 2n   . . . .   . . . .   ......   ......  A =  . 0 .  +  . r32 .   . . . .   . . . .   ......   ......   . . 0   . . rn−1n 0 0 ... 0 dnn rn1 rn2 . . . rnn−1 0 then Dx + Rx = b, so Dx = b − Rx. If R is “small”, then Dx ≈ b. Let x(n) ⊂ Rn be a sequence of approximations to x. Then it seems reasonable to set x(0) = D−1b, and let

x(1)=D−1(b − Rx(0)) x(2)=D−1(b − Rx(1)) . . x(n+1)=D−1(b − Rx(n))

This is called the Jacobi iteration. Using the Contraction Mapping Theorem, under certain conditions it can be shown that this sequence converges to x. Since diagonal inversion is only of order O(n), this is very fast to compute, so these approximations are extremely useful in situations where inverting A is not practical. Additionally, the sequence converges to x exponentially.

Proposition: Suppose that

D−1Rw −1 . 2 D R = sup = λ < 1 w6=0 kwk2 Then the Jacobi iteration as deﬁned above converges to x, the solution to Ax = b.

n n −1 n Proof: Let T : R → R , T : x 7→ D (b − Rx). T acts on (R ,k·k2), which is a Banach space, and Rn ⊆ Rn is closed. Now, ∀u, v ∈ Rn,

−1 −1 −1 −1 −1 kT u − T vk2 = (D b − D Ru) − (D b − D Rv) = D R(u − v) 2 2

−1 ≤ D R ku − vk2 = λku − vk2

Then, since λ < 1, T is a contraction mapping on Rn, so, by the Contraction Mapping Theorem, T has a unique ﬁxed point h ∈ Rn. h = T (h) = D−1(b − Rh), so Dh = b − Rh, so (D + R)(h) = b, so Ah = b, so the Jacobi iteration constructs a solution to Ax = b. Moreover

(n) (n−1) (n−1) n (0) −n log 1 −1 λ h − x = T (h) − T x ≤ λ h − x ≤ ... ≤ λ h − x = e h − D b 2 2 2 2 2 so the sequence converges exponentially, and the number of iterations needed does not depend on the n dimension of R .

50 12.4 The Newton-Raphson Algorithm

Motivation: For some continuously diﬀerentiable f : R → R, ﬁnding an x ∈ R satisfying the equation f(x) = 0 is non-trivial, and a closed-form expression for x cannot always be found, quintic polynomials are a simple example of this. It would be useful then to construct a series approximating x, and converging to this solution. Initially, choosing some x(0) “close” to the root, then taking the Taylor expansion about this point yields g(x) ≈ g(x(0)) + g0(x(0))(x − x(0)). Now, for some x(1) ∈ R, 0 ≈ g(x(1)) = g(x(0)) + g0(x(0))(x(1) − x(0)), so it seems reasonable to set g(x(0)) x(1) − x(0) = − g0(x(0)) then g(x(0)) x(1) = x(0) − g0(x(0)) Now, let g(x(n)) x(n+1) = N(x(n)) .= x(n) − g0(x(n)) This is called the Newton-Raphson method. N : (R,|·|) → (R,|·|), and is called the Newton-Raphson (n) operator.(x )n≥0 ⊂ R is a sequence of approximations to x, and, using the Contraction Mapping Theorem, under certain conditions it can be shown that this sequence converges to x.

Note: This sequence does not always converge, especially if the initial guess is poor. For example, if f is concave on the right and convex on the left, then the sequence can diverge if the initial guess is particularly bad.

Proposition: For any continuously diﬀerentiable f : R → R, if 00 f · f 0 2 = λ < 1 (f ) ∞ then the Newton-Raphson method as deﬁned above converges to x, the solution to f(x) = 0.

Proof: ∀x, y ∈ R and for some ξ ∈ (x, y),

f 0(ξ) f(ξ) · f 00(ξ) f(ξ) · f 00(ξ) N(x) − N(y) MVT= N 0(ξ)|x − y| = 1 − + |x − y| = |x − y| 0 0 2 0 2 f (ξ) f (ξ) f (ξ) 00 f · f ≤ 0 2 |x − y| = λ|x − y| (f ) ∞

Then, since λ < 1, N is a contraction mapping on R, so, by the Contraction Mapping Theorem, N has a unique ﬁxed point h ∈ R. f(h) h = N(h) = h − f 0(h) so f(h) h − h = − f 0(h) so f(h) = 0.