Mathematics for Control Theory
Introduction Logic and Sets Relations and Functions Mathematical Induction
Hanz Richter Mechanical Engineering Department Cleveland State University Course rationale and aims
The course is primarily directed at PhD students specializing in control and related areas (dynamic systems, mechatronics, optimization). The decision to offer the course was based on: 1. Even for students focusing more on control applications (as opposed to development of new algorithms and analysis methods), the controls literature stands out from other engineering fields for its widespread reliance on advanced mathematics (topics not normally covered in the typical graduate curriculum). PhD students in controls have much more pressure to catch up with the language and basic techniques used by leading researchers in the field.
2 / 40 Course rationale and aims...
2 The course prepares you to read: Frequently, the formality of the mathematical language used in the literature obstructs understanding of the key concepts presented in the paper. Familiarity with these concepts, even at a basic level, can make a big difference. 3 The course prepares you to write: Suppose you found an interesting pattern or result in your research, mostly by simulations. Formalizing the result in mathematical terms and offering a proof or a rational conjecture is key to publishing your work. The course will give you the opportunity to practice writing mathematical content for an engineering audience.
3 / 40 Course rationale and aims...
4 The topics themselves have been carefully chosen to bridge key topics in systems and control and to give you a good foundation for future independent study. 5 These slides are only the rythm of the piece. The accompanying book chapters assigned to you for reading are the rest of the music. If you only listen to the drum section, you will miss most of the piece. You must read along and do recommended exercises and homework. Maybe 1 hour a day of exclusive dedication to reading will be sufficient, plus time for homework. You will be asked to explain your solutions or concepts to the class as one way to earn points towards your grade.
4 / 40 Course organization
The course is roughly divided in 3: 1. Foundations: We cover essentials such as set theory, functions, metric spaces, convergence.
2. Analysis: Includes: Normed spaces and inner product spaces, Lp spaces. 3. Geometric methods: Includes: Differential equations on manifolds, Lie groups, applications to nonlinear control and dynamics. Several important papers in controls and/or dynamics will be used as partial projects. A paper will be chosen by students for the final project, which could be a draft paper written by the student, with significant mathematical content (instructor will approve).
Grading will be through progressive point accumulation, up to 80. The final project will be worth 20 points.
5 / 40 Reading materials
We will use multiple book chapters, sometimes with overlapping material. Book chapters and papers will be made available at the beginning of each section or subsection.
For this section (introduction, logic, sets, relations, functions, mathematical induction), we will use: ■ Hofstadter, Douglas R. (1999) [1979], Gödel, Escher, Bach: An Eternal Golden Braid, Basic Books, ISBN 0-465-02656-7 (provided material from chapter 1). ■ Landin, Joseph, (1989) [1969], An Introduction to Algebraic Structures, Dover Publications, ISBN 0-486-65940-2 (provided material from chapter 1, pp. 1-28) ■ Anthony N. Michel and Charles J. Herget (2007) [1981], Algebra and Analysis for Engineers and Scientists, Birkhäuser, e-ISBN-13: 978-0-8176-4707-0 (chapter 1) ■ Kenneth R. Davidson and Allan P. Donsig [2002], Real Analysis with Real Applications, Prentice-Hall, ISBN 0-13-041647-9 (chapter 1, except 1.3 Linear Algebra and 1.4 Calculus).
6 / 40 A Recommended: use LTEX Incorporating mathematical notation into a document shouldn’t be a nightmare. The same goes for scientific documents containing numbered bibliographic references, figures, and tables. A 1. LTEXsources for sample documents will be made available. A 2. LTEX- oriented document sharing online tools are available for free. You may submit assignments simply by sharing them with the instructor using Overleaf. A 3. Online resources for learning LTEXand resolving issues are plentiful. A 4. Most students gain working knowledge of LTEXin a couple of weeks. A 5. LTEXis highly recommended, not mandatory.
7 / 40 Logic: the rules of the game in mathematics
The ideas presented below are inspired by reading the book “Gödel, Escher, Bach: An Eternal Golden Braid”, by computer scientist Douglas Hofstadter. We will refer to this book as GEB. ■ Mathematics advances by generating new “truths” on the basis of previously known ones and following rules of deduction: logic. ■ This process is not unlike a game, where each new position of the pieces (a “truth”) is generated from others by followig the rules. ■ The process starts with some accepted truths (the initial layout of pieces is always the same and universally accepted). These are called axioms or postulates. One axiom of Euclidean geometry (one of Euclid’s five postulates) is “given a straight line and a point not belonging to it, there’s exactly one line passing through the point which does not intersect the first line”. ■ Axioms are not proven. Several axioms may be used to obtain new truths by a process of proof. Proof may be reduced to a series of symbol permutations per the rules of the game (see GEB). In other words, the generation of truth may be mechanized. 8 / 40 Hierarchy used in math writing
These types of “truth” require a proof. Labeling results as proposition, lemma or theorem is a personal decision, but you will observe the following in the literature: ■ Truths are labeled “proposition”, “lemma”, “theorem” according to their generality and importance. ■ A proposition is a minor result, an accessory to build an upcoming more general result. ■ A lemma is more far-reaching and significant than a proposition, but still an accessory to build upcoming theorems. Sometimes a lemma ends up having the same reach as theorems: the Kalman-Yakubovich lemma of passivity theory in controls. The matrix inversion lemma. ■ A theorem is the type of truth at the top of the hierarchy in a theory. The Cayley-Hamilton Theorem has large importance in Linear Algebra. ■ A corollary is an immediate consequence of a theorem. It’s proven simply by invoking the theorem (applying it to a special situation, for example).
9 / 40 Relative truth, incomplete truth, vacuous truth
Observe the following: 1. Axioms can be overriden, to create a new theory. A good example is hyperbolic geometry (due to Lobachevsky). Euclid’s parallel postulate is replaced with: “given a line and a point outside it, there are at least two distinct lines passing through the point which do not intersect the first line”. This results in a new theory, essential to Einstein’s relativity. 2. A consistent system with axioms and rules of deduction meant to derive truths about the natural numbers will never reach them all. Kurt Gödel proved this for arithmetic in 1931: there are truths that can never be reached by using the axioms and the rules, no matter the procedure. These are called undecidable propositions. Gödel’s result is known as Incompleteness Theorem. 3. Reading: Pages 33-41 in GEB, try the “MU puzzle” provided there. The following proposition is vacuously true: If x ∈ R is such that x2 = −1, then x =3.5. This kind of proposition establishes that the members of the empty set satisfy any arbitrary property. Vacuous propositions may arise in the proof process. 10 / 40 Logical Propositions
A proposition in logics is a statement that can be evaluated as true or false:
All cats have at most 4 legs There are exactly 24 primes between 1 and 100
Statements whose truth cannot be evaluated due to subjective factors or ambiguity are not propositions:
Let’s eat snow This statement is false
Conditional truth: Mathematical results (propositions, lemmas, theorems) are often implications, one-sided or two-sided:
Suppose A. Then B. (A ⇒ B)
Suppose A. Then B if and only if C. (A ⇒ (B ⇐⇒ C))
11 / 40 Truth of Implication
The validity of a statement like A ⇒ B is independent of whether A is true or not. What must be done is to establish the truth of the implication. Informally:
If a tire becomes deflated, contact between it and the surface occurs at more than one point.
The implication is true, we don’t have to think about the specific condition of the tire. In a statement like (A ⇒ (B ⇐⇒ C)), we must establish that if A is true, then B and C are equivalent (B will occur when and only when C occurs).
Finally note that one-sided implications are not reversible. If contact with the surface occurs at more than one point, we can’t conclude that the tire is deflated. It could have been placed on a V-shaped surface.
12 / 40 Modus Ponens and Truth Tables
Sometimes we do want to go farther than just the truth of the implication. We can establish the truth of the actual propositions. Modus ponens is one such scheme: A ⇒ B A is true Then B must be true.
A truth table maps out all outcomes of assumed truths in a proposition. The truth table for A ⇒ B is:
A B A ⇒ B T T T T F F F T T F F T
Clearly, the implication is false (invalid) when B fails to occur when A has happened. Think about the interpretation of this table for the tire-surface example.
13 / 40 Logic Operators
The following are used to construct statements: ■ AND, symbol ∧ ■ OR, symbol ∨ ■ NOT, symbol ∼ ■ XOR (exclusive OR), symbol ⊻ As an exercise, review the truth tables of all these.
14 / 40 Logic Quantifiers
Quantifiers are used to make statements that apply to certain elements in sets. We use: ■ (There) exists (a): ∃ (this means that at least one element exists). This is the existential quantifier. ■ There is no : ∄ ■ There exists only one: ∃! ■ For all: ∀. This is the universal quantifier ■ For some (no symbol): It’s used to say there is some element (or elements) for which a property holds. It’s pretty much the same as ∃. Logical equivalence We will say that two propositions are equivalent if they yield the same truth table. For instance p ⇒ q ≡∼ p ∨ q
15 / 40 Examples
1. A tautology is a proposition which always evaluates to true regardless of the truth values of its variables. Determine if the following is a tautology (by inspection, then verify manually or with an online truth table generator: http://mrieppel.net/prog/truthtable.html
(((p ∨ q) ⇒ r) ∧ q) ⇒ r
2. Is the following proposition true? (Q denotes the rationals)
∀x ∈ R, ∃! y ∈ R s.t. x 6=0 ⇒ y/x ∈ Q
3.Prove the following DeMorgan’s laws by showing truth table equality:
∼ (p ∧ q) ≡∼ p∨ ∼ q
∼ (p ∨ q) ≡∼ p∧ ∼ q 4. What is ∼∀?
16 / 40 Sets and Notation
We define a set as an unambiguous collection of objects, called elements of the set. A set needs to satisfy:
Given an arbitrary element, exactly one of two possibilities must hold: 1. The object belongs to the set 2. The object does not belong to the set Can we unambiguously define the set of people who will buy a new car this year?
17 / 40 Explicit and Implicit Set Descriptions
An explicit description lists the elements; an implicit description summarizes the rules for belonging to the set. For example, these descriptions correspond to the same set:
A = {0, 1, 1, 2, 3, 5, 8}
A = {x : x ≤ 5 and x is a term of the Fibonacci series} Give implicit descriptions of the following sets:
1 1 1 A = 1, , , .... 2 4 8 A = {2, 3, 5, 7, 11}
18 / 40 Set Inclusion and Equality
■ Set A is contained in B (notation: A ⊂ B) if the following implication is true: x ∈ A ⇒ x ∈ B
■ Set A is the same as B (notation: A = B) if the following double implication is true:
x ∈ A ⇒ x ∈ B and y ∈ B ⇒ y ∈ A
■ Frequently, we must show that two sets are the same. This can be done by separately proving each inclusion. ■ We can also use known relationships of set algebra. Exercise: Show by double inclusion that the following sets are equal:
A = {n : n = 4k + 1 for some k ∈ Z}
B = {n : n = 4k − 3 for some k ∈ Z} 19 / 40 Empty Set and Complement
■ The empty set, denoted ∅, is defined as the set without elements. ■ For any set A, ∅ ⊂ A. ■ Let X be a nonempty set such that A ⊂ X. We use X in context to define complement. ■ The complement of A is denoted A′:
A′ = {x ∈ X : x∈ / A}
Show by double inclusion: ■ (A′)′ = A ■ A = B if and only if A′ = B′. Practice rigorous and formal writing with these very simple proofs.
20 / 40 Binary Operations with Sets
Let A and B be subsets of X ■ Union: A ∪ B = {x ∈ X : x ∈ A or x ∈ B} ■ Intersection: A ∩ B = {x ∈ X : x ∈ A and x ∈ B} ■ Difference: A − B = {x ∈ X : x ∈ A and x∈ / B} More definitions: ■ A set with a natural number of elements is finite. A set with one element is a singleton. ■ Is A a singleton or an empty set? : A = {∅} ■ Let A be any set. The power set of A is: P(A) = {B : B ⊂ A} If A is finite with n elements, how many elements does P(A) have?
21 / 40 Some Set Identities
Let A and B be subsets of X. Then the following holds:
■ A ∩ B ⊂ A ■ A ∩ B = B ∩ A ■ A ∩ B = A iff A ⊂ B ■ A ∪ B = B ∪ A ■ A ⊂ A ∪ B ■ A ∩ ∅ = ∅ ■ A = A ∪ B iff B ⊂ A ■ A ∪ ∅ = A ■ A ∩ (B ∩ C)=(A ∩ B) ∩ C ■ A ∩ A = ∅ ■ A ∪ (B ∪ C)=(A ∪ B) ∪ C ■ A ∪ A = A ■ A∩(B∪C)=(A∩B)∪(A∩C) ■ A ∪ A′ = X ■ A ∩ A′ = ∅ ■ (A∩B)∪C =(A∪B)∩(B∪C)
22 / 40 Indexed Intersections and Unions
Let A1, A2, ....Ai,... be a collection of sets indexed by i, where i belongs to a subset I of the natural numbers. Then: Ai = {x ∈ X : x ∈ Ai for some i ∈ I} i∈I [ Ai = {x ∈ X : x ∈ Ai ∀ i ∈ I} i∈I \ DeMorgan’s Laws:
′ ′ Ai = Ai "i∈I # i∈I [ ′ \ ′ Ai = Ai "i∈I # i∈I \ [ Homework: Write your own proof to one of the DeMorgan’s identities (0.5 pt) above and prepare yourself to explain it to the class (1 pt)
23 / 40 Ordered Pairs and Cartesian Product
Let A and B be two sets. An ordered pair (a, b) is formed with an element a ∈ A and an element b ∈ B. The order is important, so:
(a, b)=(c,d) iff a = c and b = d
The Cartesian product A × B is defined as the set of all ordered pairs formed with elements from A and B:
A × B = {(a, b): a ∈ A, b ∈ B}
Ordered n-tuples and products can also be considered:
(x1,x2,..xn) and X1 × X2 × ...Xn Note that, in general, A × B 6= B × A.
24 / 40 Functions (Mappings)
Given to nonempty sets X and Y , a function f : X → Y is a subset of X × Y such that (x, y1)=(x, y2) ⇒ y1 = y2. We say that f maps X into Y . X is the domain of f, while the set of y s.t. (x, y) ∈ f is the range, R(f). Functions can be: ■ Surjective: R(f) = Y (also called onto Y )
■ If y1 = y2 ⇒ (x1, y1)=(x2, y2) for all (x1, y1), (x2, y2) ∈ f, the function is injective or one-to-one ■ If f is one-to-one and onto, it is bijective You should make a clear distinction between a function and its value at a given element of the domain. sin is a function, sin(x) for some real number x is a real number.
25 / 40 Inverse Function
Theorem: (see proof in Michel and Herget): Let f : X → Y . Consider the following subset of R(f) × X:
g = {(y,x) ∈ R(f) × X :(x, y) ∈ f}
Then g is a function from R(f) into X if and only if f is injective.
In other words, f must be injective for an inverse (f −1 = g) to exist.
26 / 40 Composition of Functions
Let X, Y and Z be nonempty sets and let f : X → Y and g : Y → Z be functions. The set
{(x,z) ∈ X × Z : z = g(f(x))} is a function from X into Z, denoted g ◦ f. It can be shown that g ◦ f is onto if g and f are themselves onto. Moreover, g ◦ f is 1-1 if g and f themselves are 1-1. In other words, onto and 1-1 are preserved by composition.
Further, if f and g are bijective, then:
(g ◦ f)−1 =(f −1 ◦ g−1)
Homework (0.5 pt): The above formula would fail without the “onto” requirement. Give a counter-example using injective functions which are not onto.
27 / 40 Equivalence Relations
A relation is more general than a function. Given sets X and Y , a relation from X to Y is simply a subset of X × Y . A relation from X to X is called a relation on X
Let ρ be a relation on X. If (x, y) ∈ ρ we write xρy (“x is related to y”). An equivalence relation on X satisfies: 1. xρx ∀x ∈ X (reflexive) 2. xρy ⇒ yρx (symmetric) 3. xρy and yρz ⇒ xρz (transitive) Define ρ on the set of all humans that exist now or ever existed as ”having the same mother”. Is ρ an equivalence relation?
28 / 40 Equivalence Relations
Let A 6= ∅. A partition P of set A is a collection of subsets S ⊂ A satisfying: 1. If S ∈ P , then S 6= ∅ 2. If S ∈ P and T ∈ P , then either S = T or S ∩ T = ∅
3. S∈P S = A In roughS words, a partition is made up of non-overlapping sets which cover A. (Warning: A separate mathematical definition exists for “covering”).
The subsets S are called equivalence classes of the partition. There’s an important connection between partitions and equivalence classes: Elements of A can be related by “belonging to the same S”. This is formalized next.
29 / 40 Equivalence Relations and Partitions
First, we establish that a partition induces an equivalence relation:
Theorem (Landin Th. 8) Let P be a partition of A. Define a relation ρ on A as xρy if ∃S ∈ P such that x ∈ S and y ∈ S. Then ρ is an equivalence relation.
Now we establish the opposite:
Theorem (Landin Th. 9) Let ρ be an equivalence relation on the set A. For each x ∈ A let Sx = {y : y ∈ A and yρx} Let P (ρ) = {S : S = Sx for some x ∈ A} Then P (ρ) is a partition of A.
Following both proofs in Landin to full understanding is highly suggested.
30 / 40 More on Equivalence Relations and Partitions
■ Start with a nonempty set A, define some partition P . An equivalence relation arises, ρ(P ). In turn, ρ induces a partition P ′(ρ(P )). We end up with the initial partition: P ′ = P . The same happens if we start with an equivalence relation, induce a partition and then again an equivalence relation.
■ If ρ1 and ρ2 are equivalence relations on A, ρ1 = ρ2 ⇐⇒ P (ρ1) = P (ρ2).
■ If P1 and P2 are partitions of A, P1 = P2 iff the equivalence relations they induce are equal. Because of these results, there’s a unique P to a given ρ on A, called “the partition determined by ρ. Viceversa, there’s a unique ρ to a given P of A, called “the equivalence relation determined by P .
31 / 40 Three proof techniques
Many proofs are conducted by one or more of the following methods: 1. Proof by exhaustion: All cases are covered, and the proof is direct. 2. Proof by contradiction: In a statement of the type A ⇒ B, we assume A and then we incorrectly assume ∼ B. We then try to uncover a contradiction. This proves that ∼ B is impossible whenever A occurs. Then the implication must be true and the proof is concluded. The same idea can be used for double implications and for statements involving quantifiers. For example, if we say A holds ∀x, the negative assumption is ∃x s.t ∼ A holds. 3. Proof by induction: applies to properties stated for elements of countable sets (more next).
32 / 40 Mathematical Induction Principle
There is a formal basis for this principle (hereditary sets). See Landin if you wish to see details.
A countable set is one where a bijection exists between its elements and the set of natural numbers.
The induction principle can be used to show that certain property holds for all elements of the countable set. Let xn denote an element of the set, where n ∈ N. 1. Prove that the property holds for n =0 (index shifting may be necessary in some cases) 2. Prove the following implication: If the property holds for n, then if holds for n +1. The left-hand side of the implication is called inductive hypothesis.
33 / 40 Example: Simple proof by induction
Proposition: All integers whose last digit is 0 or 5 are divisible by 5.
Proof: Without loss of generality (w.l.o.g.) assume that the integer is non-negative. If the integer is zero, there’s nothing to prove. A positive integer ending in a0 has the form
n i 10 ai i=0 X for some constants a0, a1..an ∈ N which are not all zero. Proceed by induction. For n =0, a0 is either 0 or 5, which is clearly divisible by 5. Now assume that
n i 10 ai i=0 X is divisible by 5.
34 / 40 Example...
We must show that n+1 i 10 ai i=0 X is divisible by 5. But:
n+1 n i i n+1 10 ai = 10 ai + 10 an+1 i=0 i=0 X X The first term is divisible by 5 (inductive hypothesis). Further, the second term n+1 n is divisible by 5 because an+1 ∈ N and 10 an+1/5=2 × 10 an+1 ∈ N. .
Note that because of the bijection of the set 0, 5, 10, 15, 20... with N, we are just proving a property of the natural numbers. This is the case in any proof by induction!
35 / 40 Equivalence between logic and sets
Set operations (union, intersection, complement..etc.) and logic operations (OR, AND, NOT..., etc) are equivalent in the following sense:
Suppose ϕ(x) means “property ϕ applies to x, and similarly for ψ(x). Define:
A = {x | ϕ(x)} and B = {x | ψ(x)}
Then
A ∪ B = {x | x ∈ A ∨ x ∈ B} = {x | ϕ(x) ∨ ψ(x)} = {x | ϕ ∨ ψ (x)}
Likewise: A ∩ B = {x | ϕ ∧ ψ (x)} ′ A = {x |∼ ϕ(x)} A ⊂ B = {x | ϕ ⇒ ψ (x)} The last formula is interpreted as: “elements for which property ψ implies property ϕ”, or “property ϕ holds whenever ψ holds”, which is clearly an inclusion. 36 / 40 Ordered sets, sup and inf
The familiar relations >, = and < defined for the real numbers fit the definition of “relation” given earlier (are they equivalence relations?).
The real numbers are totally ordered, that is, any two reals satisfy exactly one of the above relations. ■ A subset S of R has an upper-bound if ∃b s.t. x ≤ b ∀x ∈ S. ■ A subset S of R has an lower-bound if ∃l s.t. x ≥ l ∀x ∈ S. ■ b is the supremum of S if no number smaller than b is an upper bound for S (b is the lowest upper bound). Notation: b = sup S. ■ l is the infimum of S if no number greater than l is a lower bound for S (b is the greatest lower bound). Notation: l = inf S. ■ The completeness axiom states that every bounded nonempty subset of R must have a supremum.
37 / 40 Examples
Let x(t) be the unique solution to the differential equation
x˙ + x =1 with initial condition x(0) = 0. Let X = {x(t),t ≥ 0}. What is sup X? Is sup X an element of X?
The infinity norm of a transfer matrix such that Y (s) = G(s)U(s) is given by:
||G||∞ = sup σ¯[G(jw)] w∈R where σ denotes singular value.
For the SISO case ||G||∞ is simply the peak frequency response.
38 / 40 sup and inf for real sets
Given a set A ∈ R, sup A = a if and only if: 1. a is an upper bound of A 2. For all ǫ > 0, ∃s ∈ A s.t. s > a − ǫ.
Similarly, inf A = b if and only if: 1. b is a lower bound of A 2. For all ǫ > 0, ∃s ∈ A s.t. s < b + ǫ. When a ∈ A or b ∈ A we speak of maximum and minimum, respectively. Proofs involving sup and inf may require direct use of the definitions or may take advantage of useful identities (next).
39 / 40 Properties of sup and inf for real sets
Let A and B be nonempty sets of real numbers and let c be a real constant. Define: A + B = {a + b : a ∈ A, b ∈ B} cA = {ca : a ∈ A} Then: 1. A ⊂ B =⇒ (sup A ≤ sup B and inf A ≥ inf B) 2. A 6= ∅ =⇒ inf A ≤ sup A 3. If A 6= ∅ and B 6= ∅ then:
inf (A + B) = inf A + inf B
sup (A + B) = sup A + sup B
4. inf (cA) = c inf A if c ≥ 0 and inf (cA) = c sup A if c < 0 5. sup (cA) = c inf A if c < 0 and sup (cA) = c sup A if c ≥ 0
40 / 40