Banach Contraction Mapping Principle

Oksana Bihun

March 2, 2010

Department of Mathematics and Computer Science Concordia College, Moorhead, MN

Metric Spaces

Banach Contraction Mapping Principle

Applications to Ordinary Differential Equations

Applications to Numerical Solutions of Linear Systems Stefan Banach

Born: March 30, 1892, in Ostrowsko, nearby Krak´ow (now Poland). Never met his mother. Father Stefan Greczek arranged for his son to be brought up by Franciszka Plowa. Received early education from a French intellectual Juliusz Mien. Graduated from Henryk Sienkiewicz Gymnasium No. 4 in 1910 without distinction. Felt that nothing new can be discovered in mathematics and chose to study engineering at the Polytechnic (1910-1916). Lviv Lvov Lw´owLemberg Leopolis

Lviv was founded in 1256 by King Danylo Halytskiy of the Ruthenian principality of Halych-Volhynia, and named in honor of his son, Lev (Lion). Throughout history, Lviv belonged to Halych-Volynia, Lithuania, Poland, Austrian-Hungarian Empire, and . Now it belongs to . As of 1910 Lviv and Krak´owbelonged to Austro-Hungarian Empire. Banach and Steinhaus In his memoirs, wrote: “During one such walk I overheard the words“Lebesgue measure”. I approached the park bench and introduced myself to the two young apprentices of mathematics... From then on we would meet on a regular basis, and ... we decided to establish a mathematical society.” Banach solved a problem Steinhaus posed, since then they wrote many papers together. In 1920 Banach became an assistant to Lomnicki at . Banach’s thesis “On Operations on Abstract Sets and their Application to Integral Equations” is sometimes said to mark the birth of . In 1922 Banach was awarded habilitation by Lviv . The Scottish Caf´ein Lviv

The caf´ewas a meeting place for many mathematicians including Banach, Steinhaus, Ulam, Mazur, Kac, Schauder, Kaczmarz, and others. Problems were written in a book kept by the landlord and often prizes were offered for their solution. A collection of these problems appeared later as the . R. D. Mauldin, The Scottish Book, Mathematics from the Scottish Caf´e(1981). In 1936 Mazur posed an approximation problem and offered live goose to the one who solves it. The problem was solved only in 1972 by a Swedish mathematician Per Enflo. He was awarded a live goose in Wroclaw. Metric Spaces

A metric space (X , d) is a set X together with a function d : X × X → R that satisfies the properties 1. d(x, y) ≥ 0 for all x, y ∈ X ; 2. d(x, y) = 0 iff x = y; 3. d(x, y) ≤ d(x, z) + d(z, y) for all x, y, z ∈ X (triangle inequality); 4. d(x, y) = d(y, x) for all x, y ∈ X . The function d is called a metric on X . The metric on X gives a way to measure distance on X . Examples of Metric Spaces

2 p 2 2 1.( R , d) with d(X , Y ) = (y1 − x1) + (y2 − x2) for T T X = (x1, x2) and Y = (y1, y2) .

2 2 3 2.( S , d), where S is the 2-sphere in R and for every pair of 2 points x, y on S , d(x, y) is defined as the length of the shortest arc of the great circle that passes through x and y.

3.( C[0, 1], d), where C[0, 1] is the set of all continuous functions on [0, 1] and for every pair of continuous functions f , g : [0, 1] → R the distance between them is defined to be d(f , g) = max |f (x) − g(x)|. x∈[0,1] Convergent sequences and Cauchy sequences

Let (X , d) be a metric space. An ε-neighborhood of x ∈ X is the set Bε(x) = {y ∈ X : d(x, y) < ε}. ∞ Let {x1, x2,...} = {xn}n=1 be a sequence on X . ∞ The sequence {xn}n=1 converges to x ∈ X , limn→∞ xn = x, if every ε-neighborhood of x contains all but finitely many terms of ∞ {xn}n=1. ∞ The sequence {xn}n=1 is a Cauchy sequence if for every ε > 0 there exists an index N such that the distance d(xn, xm) < ε as long as n, m > N. The metric space X is complete if every Cauchy sequence in X converges to an element of X . Banach Contraction Mapping Principle

Let (X , d) be a complete metric space. A map T : X → X is a contraction if there exists a nonnegative number ρ ≤ 1 such that

d(T (x), T (y)) ≤ ρd(x, y).

T is a strict contraction if ρ < 1. A point x ∈ X is called a fixed point of T if T (x) = x. Theorem Every strict contraction T on a complete metric space (X , d) has a unique fixed point x. Moreover, for every x0 ∈ X , x is the limit of the sequence {x0, T (x0), T (T (x0)),...}, which can be defined recursively:

xn = T (xn−1) for all n = 1, 2, 3,.... Application to Ordinary Differential Equations

Consider the initial value problem

 x˙ = f (x), (1) x(0) = x0,

where f : R → R is a Lipschitz continuous function, i.e. there exists L > 0 such that |f (x) − f (y)| ≤ L|x − y| for all x, y ∈ R. We would like to prove the existence of a continuously differentiable function x : [0, δ] → R, where δ > 0, that solves the initial value problem (1). The latter problem is equivalent to the problem of finding a continuous function x : [0, δ] → R that solves the integral equation Z t x(t) = x0 + f (x(s)) ds. 0 Existence of Solutions of ODEs

Z t x(t) = x0 + f (x(s)) ds. 0 Let X = C[0, δ] with the same distance d as before. R t Define T : X → X by T (x)(t) = x0 + 0 f (x(s)) ds for all t ∈ [0, δ]. Claim. T is a strict contraction provided δ > 0 is small enough.

Z t |T (x)(t) − T (y)(t)| ≤ |f (x(s)) − f (y(s))| ds 0 Z t ≤ L |x(s) − y(s)| ds 0 ≤ Lδ max |x(s) − y(s)|. s∈[0,1]

Therefore, d(T (x), T (y)) ≤ Lδd(x, y) for all x, y ∈ X ; T is a 1 strict contraction provided δ < L . Existence of Solutions of ODEs

Theorem If f : R → R is Lipschitz continuous, then there exists δ > 0 and a unique continuously differentiable function x : [0, δ] → R that solves the initial value problem

 x˙ = f (x), x(0) = x0,

where x0 ∈ R. Applications to Numerical Solutions of Linear Systems

Consider the problem of solving a linear system

Ax = b,

where A is an n × n nonsingular matrix and b is an n-vector. Assume that the diagonal elements of A are equal to 1. Gauss elimination method produces large errors when used on a computer. Rewrite (I − A)x + b = x.

n Consider the (complete) metric space (R , d), where T d(X , Y ) = max |xi − yi | for X = (x1, x2,..., xn) and 1≤i≤n T Y = (y1, y2,..., yn) . Applications to Numerical Solutions of Linear Systems

n Define T (x) = (I − A)x + b for every x ∈ R . Because T (x) − T (y) = (I − A)(x − y).

n X max |T (x)i − T (y)i | ≤ max |(I − A)ij ||xj − yj | 1≤i≤n 1≤i≤n j=1 n X = max |Aij ||xj − yj |. 1≤i≤n j6=i,j=1

n X Thus, d(T (x), T (y)) ≤ max |Aij | d(x, y). 1≤i≤n j6=i,j=1 Applications to Numerical Solutions of Linear Systems

Theorem Let A be a nonsingular n × n matrix and let b be an n-vector. If a nonsingular matrix A has diagonal dominance, that is, n X n |Aij | ≤ |Aii | for all 1 ≤ i ≤ n, then for every x0 ∈ R the j6=i,j=1 approximation scheme

xn = (A − I )xn−1 + b,

where n = 1, 2,..., converges to the solution of the linear system Ax = b. References

I Michael Reed and Barry Simon, Methods of Modern Mathematical Physics, I: Functional Analysis, Academic Press, 1972.

I David Luenberger, Optimization by Vector Space Methods, John Wiley and Sons, Inc., 1969.

I Banach Biography from http://www.gap- system.org/ history/Biographies/Banach.html.

I Images: http://www.mathematik.de/ger/information/landkarte/ gebiete/linearealgebra/bilder/banach.jpg (photo of Stefan Banach); http://www.gap-system.org/˜ history/Miscellaneous/Scottish Cafe.html (Scottish Cafe in Lviv).

I The Scottish Book, http://banach.univ.gda.pl/e-scottish-book.html.