Linear Systems Professor Yi Ma Professor Claire Tomlin GSI: Somil Bansal Scribe: Chih-Yuan Chiu Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, CA, U.S.A. May 22, 2019 2 Contents Preface 5 Notation 7 1 Introduction 9 1.1 Lecture 1 . .9 2 Linear Algebra Review 15 2.1 Lecture 2 . 15 2.2 Lecture 3 . 22 2.3 Lecture 3 Discussion . 28 2.4 Lecture 4 . 30 2.5 Lecture 4 Discussion . 35 2.6 Lecture 5 . 37 2.7 Lecture 6 . 42 3 Dynamical Systems 45 3.1 Lecture 7 . 45 3.2 Lecture 7 Discussion . 51 3.3 Lecture 8 . 55 3.4 Lecture 8 Discussion . 60 3.5 Lecture 9 . 61 3.6 Lecture 9 Discussion . 67 3.7 Lecture 10 . 72 3.8 Lecture 10 Discussion . 86 4 System Stability 91 4.1 Lecture 12 . 91 4.2 Lecture 12 Discussion . 101 4.3 Lecture 13 . 106 4.4 Lecture 13 Discussion . 117 4.5 Lecture 14 . 120 4.6 Lecture 14 Discussion . 126 4.7 Lecture 15 . 127 3 4 CONTENTS 4.8 Lecture 15 Discussion . 148 5 Controllability and Observability 157 5.1 Lecture 16 . 157 5.2 Lecture 17 . 163 5.3 Lectures 16, 17 Discussion . 176 5.4 Lecture 18 . 182 5.5 Lecture 19 . 185 5.6 Lecture 20 . 194 5.7 Lectures 18, 19, 20 Discussion . 211 5.8 Lecture 21 . 216 5.9 Lecture 22 . 222 6 Additional Topics 233 6.1 Lecture 11 . 233 6.2 Hamilton-Jacobi-Bellman Equation . 254 A Appendix to Lecture 12 259 A.1 Cayley-Hamilton Theorem: Alternative Proof 1 . 259 A.2 Cayley-Hamilton Theorem: Alternative Proof 2 . 262 B Appendix to Lecture 15 265 B.1 Rate of Decay . 265 B.2 Basic Lyapunov Theorems: . 267 B.3 Exponential Stability Theorem: . 270 B.4 Lyapunov Equation: Uniqueness of Solution . 273 B.5 Indirect Lyapunov's Method . 280 Preface The Fall 2018 graduate-level EE221A Linear Systems Theory course, offered by Professor Yi Ma in the Department of Electrical Engineering and Computer Sciences (EECS) at the University of California, Berkeley, included content largely drawn from the following sources: • Tomlin, Claire. Lecture Notes on Linear Systems Theory [10]. • Ma, Yi. Lectures Notes on Linear System Theory [7]. • Callier, Frank and Desoer, Charles. Linear System Theory [4]. • Sastry, Shankar. Nonlinear Systems: Stability, Analysis, and Control [9] • Liberzon, Daniel. Calculus of Variations and Optimal Control, A Concise Introduction [6] • Yung, Chee-Fai. Linear Algebra, 3rd Edition. [11] • Yung, Chee-Fai. Lecture Notes on Mathematical Control Theory. [12] The following collection of notes represents my attempt to organize this ensemble of linear systems-related material into a friendly introduction to the subject of linear systems. The chapters contain collections of lectures in Professor Claire Tomlin's Lecture Notes on Linear Systems Theory [10], presented in roughly the same order, with the exception of Lecture 11 (Linear Quadratic Regulator), which has been relocated to the end of the text. Chapter 1 contains content from Lecture 1, which gives an introduction to linear systems. Chapter 2 includes material from Lectures 2-6, and primarily reviews concepts in linear algebra, such as fields, vector spaces, linear independence and dependence, linear maps and matrix representations, norms, orthogonality, inner product spaces, adjoint maps, projection, least squares optimization, and the singular value decomposition. Chapter 3 was organized from Lectures 7-10, and formally introduces dynamical systems and their properties, beginning with the Fundamental Theorem of Differential Equations, particular classes of dynamical systems (linear, non-linear, time-invariant, time-varying), and concluding with properties of the matrix exponential and an inverted pendulum example. Chapter 4, compiled from material in Lectures 12-15, discusses notions of stability, as well as necessary and sufficient conditions for these different definitions of stability. Chapter 5, which spans Lectures 16-22, defines controllability, observability, stabilizability, and detectability, and explores different criteria that different types of systems 5 6 CONTENTS must satisfy in order to be controllable, observable, stabilizable, and/or detectable. Chapter 6 discusses the Linear Quadratic Regulator, the subject of Lecture 11, as well as the Hamilton- Jacobi-Bellman Equation, as discussed in Chapter 2 of [6]. Finally, appendices including the Basic Lypaunov Theorem and other stability theorems for non-linear systems, among other material, have been added, partly for completeness, and partly to interest readers in more advanced topics in control theory. This material originates largely from Chapter 5 of [9]. These notes have several possible shortcomings. To minimize the reader's confusion, I have attempted to unify the notation used in the references cited above, and correct most of the typos in the text. Nevertheless, it is inevitable that minor errors or inconsistencies in notation remain scattered throughout the notes (Readers have discovered such mistakes are welcome to contact me at chihyuan [email protected]). Regarding the material itself, Professors Yi Ma and Claire Tomlin often gave useful remarks in their lectures that were not included in their written notes. Although I have added as many of these comments into the notes as possible, it is certain that I have missed many others. Moreover, some supplementary material from the Fall 2017 EE221 Linear Systems Theory course were omitted, since I found the material to be similar to content already included. (e.g. Somil Bansal's "Special Lecture on the Linear Quadratic Regulator," which discusses dynamic programming solutions to the finite LQR problem, and Dr. Jerry Ding's "Alternative Derivation of Linear Quadratic Regulator," which describes how the Pontryagin Minimum Principle can be applied to solve the linear quadratic optimization problem.) My inexperience with the subject of linear systems may also have contributed towards errors in the notes. Nonetheless, it is my hope that the text remains a useful, introductory reference to readers studying linear systems theory for the first time. It is an honor for me to dedicate these notes to the following individuals. Naturally, without the carefully prepared lectures and handouts given by Professor Yi Ma, and the painstakingly detailed notes and figures organized by Professor Claire Tomlin, this work would not exist. I would also like to acknowledge Professors Shankar Sastry, author of "Nonlinear Systems: Stability, Analysis, and Control" [9], and Professor Daniel Liberzon, author of "Calculus of Variations and Optimal Control, A Concise Introduction" [6], for their time and effort into compiling these works, which serve as the foundation of most of the material in the last chapter and appendix. Somil Bansal, the Graduate Student Instructor for this course in the Fall semester of 2018, deserves gratitude not only for meticulously preparing the discussion notes included in this text, but also for taking the extra effort to arrange office hours, and organize midterm and final discussion sessions. My appreciation extends to Professor Chee-Fai Yung, who first sparked my interest in control theory, and who graciously allowed me to use sections of his lectures notes in this work. I would also like to acknowledge my fellow classmates in this course, many of whom provided helpful suggestions throughout the semester. Lastly, I would like to thank my parents for their unending support and encouragement. Chih-Yuan Chiu University of California, Berkeley Department of Electrical Engineering and Computer Sciences December 2018 Notation The following notation will be employed throughout this text: N : Set of all positive integers Z : Set of all integers' Q : Set of all rational numbers R : Set of all real numbers, i.e. the real line C : Set of all complex numbers C− : Set of all complex numbers with a (strictly) negative real part C− : Set of all complex numbers with a non-positive real part C0 : Set of all purely imaginary numbers C+ : Set of all complex numbers with a (strictly) positive real part C+ : Set of all complex numbers with a non-negative real part 2: Is an element of (Is contained in) 8: For each (for all) 9: There exists 9!: There exists a unique 9?: Does there exist 3: Such that A ) B: A implies B B ( A: B implies A A , B: A and B are equivalent In: Identity matrix of dimension n × n On: Zero matrix of dimension n × n ◦: Composition of Functions 1X : Identity map from X to X S1 ⊂ S2: The set S1 is a subset of the set S2 W ≤ V: The vector space W is a subspace of the vector space V W ⊕ V: The direct sum of W and V. ? W ⊕ V: The orthogonal direct sum of W and V. j · j: Norm of a vector k · k: Norm of a matrix or operator X(s): Unilateral Laplace transform of x(t) (If the time-domain argument is capitalized, e.g. X(t), a hat is used, e.g. X^(s)). ust(t): Unit step function 7 8 CONTENTS LTI: Linear time-invariant LTV: Linear time-variant SISO: Single-input-single-output MIMO: Multiple-input-multiple-output Chapter 1 Introduction 1.1 Lecture 1 Goals of Lecture 1: 1. An introduction to the broad concepts of modeling and analysis of engineering systems| Modeling, Analysis and control, Verification, Simulation, Validation 2. An overview of the course The difference between science and engineering is sometimes expressed in terms of interaction with a physical phenomenon. Physical sciences study the phenomenon, while the engineering disciplines design, manipulate, and control the phenomenon. Simply put, scientists describe while engineers control. The main purpose of control is to choose an input u(t) to a system such that some pre-defined reward or cost is optimized. 9 10 CHAPTER 1. INTRODUCTION 1. Modeling: The same physical system may have different models, the best choice depends on the problem at hand: • MEMS Device: • Hubble Space Telescope: Which model makes the most sense to use to move the telescope from one altitude to another? The utility of a model is in its predictive power: the ability to use it to forecast what the system will do.