Mathematics for Control Theory

Elements of Analysis p.1 Metric Spaces Contraction Mapping Principle

Hanz Richter Mechanical Engineering Department Cleveland State University Reading materials

We will use: ■ Anthony N. Michel and Charles J. Herget (2007) [1981], Algebra and Analysis for Engineers and Scientists, Birkhäuser, e-ISBN-13: 978-0-8176-4707-0 (chapter 5, skip sections indicated by the instructor) ■ Kenneth R. Davidson and Allan P. Donsig [2002], Real Analysis with Real Applications, Prentice-Hall, ISBN 0-13-041647-9 (use chapter 4 as a reference).

2 / 30 Introduction

Metric spaces are the most general setting in which we can study important aspects of sequences and functions such as convergence and continuity.

Essential concepts pertaining to the base sets of metric spaces, such as openness and compactness and their relation to functions defined on them will be covered in this section.

Metric spaces are sets where a distance function (the metric) is defined for any pair of its elements. Only a few requirements are made for the metric; otherwise, the definition is very general.

3 / 30 Definition of metric space

Let X be any nonempty set and let ρ be a real-valued function defined on X X satisfying: × 1. ρ(x, y) 0 x, y X and ρ(x, y)=0 x = y. ≥ ∀ ∈ ⇐⇒ 2. ρ(x, y) = ρ(y,x) x, y X ∀ ∈ 3. ρ(x, y) ρ(x,z) + ρ(z, y) x, y X ≤ ∀ ∈ Metric spaces defined as above will be denoted X; ρ . A given X admits { } infinitely many metrics.

Examples: ρ(x, y) = x y is a metric when X = R. Also, | − | ρ(z1,z2) = (z1 z2)(z1 z2) is a metric when X = C. − − q

4 / 30 Discrete metric, diameter

The following function is a metric (show) which works for any X:

0, x = y ρ(x, y) = 1, x = y  6 We say that a metric space X; ρ is bounded if M > 0 s.t. ρ(x, y) M { } ∃ ≤ x, y X (the metric is bounded on X). ∀ ∈ The diameter of a subset Y X in a metric space X; ρ is defined by ∈ { } δ(Y ) = sup ρ(x, y): x, y Y { ∈ } We solve Ex.5.1.20 in Michel and Herget: an inclusion relationship among subsets of a metric space implies an order relationship in the corresponding diameters. Is the opposite true?

5 / 30 Important metric spaces

We can make a metric space with complex n-tuples (Cn, with Rn a particular case) and a very general metric. Let x =(ξ1,ξ2,...ξn) and y =(η1, η2,...ηn). Define the p-metric as

1 n p p ρp(x, y) = ξi ηi | − | "i=1 # X for 1 p < . ≤ ∞ This metric satisfies Minkowski’s inequalities:

ρp(x, y) ρp(x, 0) + ρp(y, 0) ≤

ρp(x, y) ρp(x, 0) + ρp(y, 0) − ≤ When p =2, we recover the usual Euclidean distance. Matlab: ρp(x, 0) = norm(x,p) .

6 / 30 p-metrics...

When x and y are infinite sequences rather than vectors in Cn, we can use the above metric for n = provided the sequences converge (see Ex. 5.3.5 in ∞ Michel and Herget): ∞ ξ p < | |i ∞ i=1 X ∞ η p < | |i ∞ i=1 X When p =1, we obtain an absolute value measure of distance. At the other end of the range of p, we define for x, y Cn: ∈

ρ∞(x, y) = max ξi ηi {| − |} Show that this is indeed a metric space.

7 / 30 Distance between continuous functions

Another important metric space can be formed with all real-valued continuous functions on an interval [a, b]. This set is denoted as [a, b]. The following C metric works: b p p ρp(x, y) = x(t) y(t) dt | − | Za  for all x, y [a, b]. ∈C Homework: (0.5pt) Solve 5.3.18 in Michel and Herget.

8 / 30 Product Metric Spaces

How would we capture objects such as vectors whose entries themselves are elements of metric spaces? What are possible metrics? n Let X1; ρ1 , X2; ρ2 .... Xn; ρn be n metric spaces. Let X = Xi. { } { } { } i=1 Consider the following metrics: Q n ρ(x, y) = ρi(xi, yi) i=1 X 1 n 2 2 ρ(x, y) = (ρi(xi, yi)) i=1 ! X for any x =(x1,x2...xn) and y =(y1, y2,...yn) in X.

Th. 5.3.21 shows that both metrics can be used to form metric spaces on X.

9 / 30 Open and Closed Sets

This topic is of the highest importance and should remain with you forever. We will introduce the topological definition later in the course. Let X; ρ be any metric space. { } Definition: Let x0 X and let r be a positive real. The open ball S(x0; r) ∈ around x0 is the set:

S(x0; r) = x X : ρ(x,x0) < r { ∈ }

An equivalent terminology is to say that S is a ball of radius r around x0.

Definition: A subset Y of X is open if for every x X there exists some open ∈ ball S(x, r) such that S(x, r) Y . ⊂ X

Y

Z

10 / 30 Closed Sets

Again let Y X. ⊂ Definition: A point x X is called an adherent point of Y if every open ball ∈ with center x contains at least one point of Y . The set of all adherent points of Y is called the closure of Y , denoted Y¯ .

Definition: Y is a closed subset of X is Y¯ = Y .

Are the following implications true?: A is not open A is closed ⇒ A is not closed A is open ′⇒ A is open A is closed. ⇒ Homework: (0.5pt) Solve 5.4.26 in Michel and Herget.

11 / 30 Important results

Let X; ρ be a metric space. { } Theorem (5.4.15 in Michel and Herget): 1. X and ∅ are open

2. Let Yα be an arbitrary family of open subsets of X where α belongs to { } some set A. Then α∈A Sα is open. 3. The intersection ofS a finite number of open sets of X is open. In class, we give an example of 2. and a counterexample for 3. when the intersection involves a non-finite set of subsets of X.

12 / 30 Important results...

Theorem (5.4.18 - ii in Michel-Herget): If Y X is open, there is a family of ⊂ open balls Sα α∈A such that Y = Sα. { } α∈A S Proof (by double inclusion) : Let x Y . Since Y is open, x is the center of ∈ some open ball Sx contained in Y . Then x Sx. All Sx form the required ∈ family. Now assume x Sα. Then x Sα for some α A, and Sα Y ∈ α∈A ∈ S ∈ ∈ because Sα is an open subset of Y (Th. 5.4.18 -i). Therefore x Y . S ∈ Examine Theorem 5.4.27 independently. These results will be recalled in upcoming material or exercises. Definition: Let x0 X and r > 0. The closed sphere centered at x0 with ∈ radius r is defined as:

K(x0; r) = x X : ρ(x0,x) r { ∈ ≤ } It is actually closed (Th. 5.4.27 -ii).

13 / 30 Sequences on Metric Spaces

The following material is relative to a metric space X; ρ . We denote the + { } positive integers as Z .

+ Definition: A sequence xn in a set Y X is a function f : Z Y , so { } ⊂ → that f(n) = xn.

Definition (convergence): Sequence xn converges to x X if ǫ > 0, + { } ∈ ∀ N Z s.t. ρ(xn,x) <ǫ n N. ∃ ∈ ∀ ≥ With convergence, we write: lim xn = x n→∞ Example: an : n =0, 1, 2.. converges to zero for a < 1 (show using above { } | | definition).

14 / 30 Cauchy Sequences

We can study convergence without involving the value of the limit. This is important because there are sequences whose limit is outside X (these appear in incomplete metric spaces).

Definition: A sequence xn in a metric space X; ρ is Cauchy if + { } { } ǫ > 0, N Z s.t. ρ(xn,xm) ǫ whenever n, m N. ∀ ∃ ∈ ≤ ≥ This definition says that in a Cauchy sequence, any pair of elements coming at or after an index N will be separated by a distance no greater than some ǫ.

Theorem (5.5.12): If xn converges then it is Cauchy. { }

Theorem (5.5.14) If a Cauchy sequence xn contains a convergent { } subsequence, then xn converges. { } The proof is slightly more elaborate than previous ones in the course. We develop it in class.

15 / 30 Complete Metric Spaces

Definition: If every Cauchy sequence in a metric space X; ρ converges to { } some element of X, then the metric space is complete.

This is an important definition. We will see in the next example that X = [a, b] (with the ρ2 metric) contains function sequences which converge C to discontinuous functions. Oddly enough, completeness is obtained with the ρ∞ metric.

This is one reason to seek more inclusive spaces that allow discontinuous functions, crucial in control theory. Relaxing continuity to Lebesgue measurability will allow us to form a complete metric space with ρ2 as metric.

We work on various examples at this point.

16 / 30 Compactness

We will focus on metric spaces based on Cn (Rn as a particular case).

Definition: Let X; ρ be a metric space and let Y X. A collection of { } ⊂ subsets Yα , α A of X is called a covering of Y if Y Yα. A { } ∈ ⊂ α∈A subcollection Yβ , β B, B A such that Y Yβ is called a { } ∈ ⊂ ⊂ β∈B S subcovering of Yα . { } S If Yα and Yβ are open, the covering and the subcovering are open. If A is finite, the covering is finite.

Definition: A metric space X; ρ is compact if every open covering of it { } contains a finite open subcovering.

Examples: i. The set 1, 2, 3 : Find an open covering that doesn’t have any { } proper subcovering. ii. Consider the interval [1, 3]. Give one open non-finite covering and one finite open subcovering.

17 / 30 Two important compactness theorems

Theorem (5.6.27): Consider the complex metric space with Euclidean metric n n C ; ρ2 . A set Y C is compact if and only if it is closed and bounded. { } ⊂ Is 1, 2, 3 compact? { } Theorem (5.6.29): Let X; ρ be a compact metric space and let Y X. If { } ⊂ Y is closed, then Y is compact.

18 / 30 Continuity of functions on metric spaces

In Calculus, we are restricted to functions defined on subsets of Rn. Some Calculus results actually generalize to functions defined on metric spaces.

Recall the definition of continuity: A function f : R R is continuous at → x = a if lim f(x) = f(a) x→a

For a mapping f : X Y between metric spaces X; ρx and Y ; ρy , f is → { } { } continuous at x0 X if ǫ > 0, δ > 0 s.t. ρy(f(x),f(x0)) <ǫ whenever ∈ ∀ ∃ ρ(x,x0) <δ. The mapping is continuous on X if it is so at every point of X.

In general, δ depends on ǫ and x0. If δ is independent of x0 (it only depends on the margin ǫ, but not on location x0), the function is uniformly continuous.

19 / 30 Examples

Show that f(x) = ax + b is uniformly continuous in R, but that f(x) = ex is not. In this case, find a δ = δ(ǫ,x0).

20 / 30 Continuity and Convergence

Theorem: Let X; ρx and Y ; ρy be two metric spaces. A function { } { } f : X Y is continuous at x0 X iff for every sequence xn in X which → ∈ { } converges to x0, the sequence f(xn) converges to f(x0) Y . That is, { } ∈

lim f(xn) = f( lim xn) = f(x0) n→∞ n→∞ whenever lim xn = x0 n→∞

Example: The result is a double implication, so the above limit identity must fail for some sequence if f is discontinuous. Find an example function and sequence that illustrates this.

21 / 30 Continuity and Open/Closed Sets

In brief, open sets in the range of a continuous function can only come from open sets in the domain. The same with closed sets.

Theorem (5.7.9) Let X; ρx and Y ; ρy be two metric spaces and let { } { } f : X Y . Then: → 1. f is continuous on X iff the inverse image of each open subset of Y ; ρy { } is open in X; ρx . { } 2. f is continuous on X iff the inverse image of each closed subset of Y ; ρy is closed in X; ρx . { } { } Warning: A continous function can produce a non-open image from an open set. Example in Michel-Herget: Domain: ( 1, 1) (open). Function: 2 − f(x) = x . Image: ( 1, 0] (not open). Try with the inverse image. −

22 / 30 Continuity and Compactness

Theorem (5.7.12) Let X; ρx and Y ; ρy be two metric spaces and let { } { } f : X Y be continuous on X. Then: → 1. If X; ρx is compact then f(X) is a compact subset of Y . { } 2. If U X is compact, then f(U) is a compact subset of Y . ⊂ 3. If X; ρx is compact and U X is closed, then f(U) is a closed subset { } ⊂ of Y .

4. If X, ρx is compact, then f is uniformly continuous on X. { } The last result is important. Continuous functions defined on a compact set must be uniformly continuous.

23 / 30 Continuity and Uniform Convergence

Theorem (5.7.14) Let X; ρx and Y ; ρy be two metric spaces and let fn { } { } { } be a sequence of functions from X into Y such that each fn is continuous on X. If the sequence fn converges to f uniformly on X, then f is continuous { } on X.

24 / 30 Real-valued functions on a metric space

Theorem (5.7.15) Let X; ρx be a metric space and let R; ρ denote the { } { } real line with the usual metric. Let f : X R and let U X. If f is → ⊂ continuous on X and U is compact then: 1. f is uniformly continuous on U 2. f is bounded on U 3. The supremum and infimum of f on U are actually a minimum and a maximum, that is, x0,x1 U s.t. f(x0) =inf [f(x): x U] and ∃ ∈ ∈ f(x1) = sup [f(x): x U]. ∈

25 / 30 Contraction Mapping Theorem in Metric Space

Let X; ρ be a metric space and let f : X X. Function f is a contraction { } → mapping if c R, 0 < c < 1 s.t. ∃ ∈ ρ(f(x),f(y)) cρ(x, y) ≤ for all x, y X. ∈ It can be proven that every contraction mapping is uniformly continuous. The following key result is known as the fixed-point principle:

Theorem (5.8.5) Let X; ρ be a complete metric space and let f be a { } contraction mapping from X into X. Then:

1. ! x0 X s.t. f(x0) = x0 ∃ ∈ 2. For any x1 X, the sequence xn defined by xn+1 = f(xn), n =1, 2.. ∈ converges to x0.

26 / 30 Example

Let’s take X = Rn. We know we can form a complete metric space if we use the p-metrics studied above (see Exercise 5.5.25 in Michel and Herget). Consider the mapping f : x Ax for x X, where A is a diagonal matrix 7→ ∈ and use p =2. We answer the following: 1. What is required of A so that f is a contraction from X into X? (using the definition)

2. What is x0?

27 / 30 Set Topology Definitions

We aim to establish that p-metrics are in a sense equivalent (for instance, open sets should remain open if we change p). This requires some basic definitions from set topology. A topology on a set A is a collection of subsets of A satisfying: T 1. ∅ and A ∈T ∈T 2. Finite intersections of members of remain in T T 3. Arbitrary unions of members of remain in . T T The members of are called open sets, and X; is called a topological T { T} space. Now, following Michel and Herget: Let X; ρ be a metric space. The topology of X determined by ρ is the { } collection of all open subsets of X.

The collection of all open subsets of X is actually a topology in the general sense, and thus it satisfies 1,2 and 3.

28 / 30 Equivalence of Metrics

Let X; ρ1 and X; ρ2 be two metric spaces defined on X. Let 1 and 2 { } { } T T be the topologies determined by ρ1 and ρ2, respectively. Then we say that ρ1 and ρ2 are equivalent if 1 = 2. T T The consequences of metric equivalence are summarized in Theorems 5.9.2 and 5.9.3 in Michel and Herget.

Important: If a sequence converges to a point according to one metric, it must converge to the same point according any equivalent metric.

We now focus on the following results for Rn and the p metrics: −

ρ∞(x, y) ρ2(x, y) √nρ∞(x, y) ≤ ≤

ρ∞(x, y) ρ1(x, y) nρ∞(x, y) ≤ ≤ n Metrics ρ1, ρ2 and ρ∞ are equivalent in R .

29 / 30 Hausdorff Topological Spaces

You may find that topological spaces are assumed to be Hausdorff in certain control theory papers.

A topological space X; is Hausdorff if x, y, x = y, Ux,Uy, { T} ∀ 6 ∃ Ux Uy = ∅ such that Ux and Uy are open and x Ux, y Uy. ∩ ∈ ∈ Consider X = R, the 1-metric and the topology generated by it. Is this a Hausdorff topological space?

All metric spaces are automatically Hausdorff. The proof is simple, we do it in class.

Consider any X and the trivial topology = X, ∅ . Is X, Hausdorff? T { } { T} -Note that no metric is defined, and that the only open sets are X and ∅.

30 / 30