Linear Control Theory (Lecture Notes)

Linear Control Theory (Lecture notes)

Version 0.9

Dmitry Gromov

August 25, 2017 Contents

Preface 5

I CONTROL SYSTEMS: ANALYSIS6

1 Introduction7 1.1 General notions...... 7 1.2 Typical problems solved by control theory...... 8 1.3 Linearization...... 8 1.3.1 * Hartman-Grobman theorem...... 10

2 Solutions of an LTV system 11 2.1 Fundamental matrix...... 11 2.2 State transition matrix...... 12 2.3 Time-invariant case...... 13 2.4 Controlled systems: variation of constants formula...... 14

3 Controllability and observability 17 3.1 Controllability of an LTV system...... 17 3.1.1 * Optimality property ofu ¯ ...... 19 3.2 Observability of an LTV system...... 21 3.3 Duality principle...... 22 3.4 Controllability of an LTI system...... 23 3.4.1 Kalman’s controllability criterion...... 23 3.4.2 Decomposition of a non-controllable LTI system...... 24 3.4.3 Hautus’ controllability criterion...... 25 3.5 Observability of an LTI system...... 26 3.5.1 Decomposition of a non-observable LTI system...... 27 3.6 Canonical decomposition of an LTI control system...... 28

4 Stability of LTI systems 29 4.1 Matrix norm and related inequalities...... 29 4.2 Stability of an LTI system...... 31 4.2.1 Basic notions...... 31 4.2.2 * Some more about stability...... 32 4.2.3 Lyapunov’s criterion of asymptotic stability...... 33

2 4.2.4 Algebraic Lyapunov matrix equation...... 35 4.3 Hurwitz stable polynomials...... 36 4.3.1 Stodola’s necessary condition...... 36 4.3.2 Hurwitz stability criterion...... 36 4.4 Frequency domain stability criteria...... 37

5 Linear systems in frequency domain 42 5.1 Laplace transform...... 42 5.2 Transfer matrices...... 43 5.2.1 Properties of a transfer matrix...... 44 5.2.2 * Computing eAt using the Laplace transform...... 45 5.3 Transfer functions...... 45 5.3.1 Physical interpretation of a transfer function...... 46 5.3.2 Bode plot...... 46 5.3.3 * Impulse response function...... 47

II CONTROL SYSTEMS: SYNTHESIS 49

6 Feedback control 50 6.1 Introduction...... 50 6.1.1 Reference tracking control...... 51 6.1.2 Feedback transformation...... 52 6.2 Pole placement procedure...... 54 6.3 Linear-quadratic regulator (LQR)...... 55 6.3.1 Optimal control basics...... 55 6.3.2 Dynamic programming...... 57 6.3.3 Linear-quadratic optimal control problem...... 59

7 State observers 61 7.1 Full state observer...... 61 7.2 Reduced state observer...... 63

APPENDIX 65

A Block matrices 66 A.1 Matrix inversion...... 66 A.2 Determinant of a block matrix...... 67

B Canonical forms of a matrix 68 B.1 Similarity transformation...... 68 B.2 Frobenius companion matrix...... 69 B.2.1 Transformation of A to AF ...... 70 B.2.2 Transformation of A to A¯F ...... 72 B.3 Jordan form...... 72

C * Linear operators 73

3 C.1 General properties...... 73 C.2 Adjoint of L(u)...... 74 C.3 Solving homogeneous ODEs...... 74

D Miscellaneous 76

Bibliography 77

4 Preface

These lecture notes are intended to provide a supplement to the 1-semester course “Linear control systems” taught to the 3rd year bachelor students at the Faculty of Applied Mathematics and Control Processes, Saint Petersburg State University. This course is developed in order to familiarize the students with basic concepts of Linear Control Theory and to provide them with a set of basic tools that can be used in the subsequent courses on robust control, nonlinear control, control of time-delay systems and so on. The main emphasis is put on understanding the internal logic of the theory, many particular results are omitted, some parts of the proofs are left to the students as exercises. On the other hand, there are certain topics, marked with asterisks, that are not taught in the course, but included in the lecture notes because it is believed that these can help interested students to get deeper into the matter. Some of these topics are included in the course “Modern control theory” taught to the 1st year master students of the specialization “Operations research and systems analysis”. The lecture notes do not include homework exercises. These are given by tutors and elaborated during the weekly seminars. All exercises and examples included in the lecture notes are intended to introduce certain concepts that will be used later on in the course. When preparing the lecture notes the author extensively used the material from the classical book by Roger Brockett, [3], as well as from the lecture notes by Vladimir L. Kharitonov, [6].

5 Part I

CONTROL SYSTEMS: ANALYSIS

6 Chapter 1

Introduction

1.1 General notions

We begin by considering the following system of ﬁrst order nonlinear diﬀerential equations: ( x˙(t) = f(x(t), u(t), t), x(t ) = x , 0 0 (1.1) y(t) = g(x(t), u(t), t),

n m k 1 where x(t) ∈ R , u(t) ∈ R , and y(t) ∈ R for all t ∈ I, I ∈ {[t0,T ], [t0, ∞)} ; f(x(t), u(t), t) and g(x(t), u(t), t) are continuously differentiable w.r.t. all their argu- ments with uniformly bounded first derivatives, and u(t) is a measurable function. With these assumptions, system (1.1) has a unique solution for any pair (t0, x0) and any u(t) which can be extended to the whole interval I. In the following we will say that x(t) is the state, u(t) is the input (or the control), and y(t) is the output. Below we consider these notions in more detail. State. The state is defined as a quantity that uniquely determines the future system’s evolution for any (admissible) control u(t). We consider systems with x(t) being an n element of a vector space R , n ∈ {1, 2,...}. NB: Other cases are possible! For instance, the state of a time-delay system is an element of a functional space. Control. The control u(·) is an element of the functional space of admissible controls: u(·) ∈ U, where U can be defined, e.g., as a set of measurable, L2 or L∞, piecewise m continuous or piecewise constant functions from I to U ⊆ R , where U is referred to m as the set of admissible control values. In this course we assume that U = R and U is the set of piecewise continuous functions.

Deﬁnition 1.1.1. Given (t0, x0) and u(t), t ∈ I,x ˜(t) is said to be the solution of (1.1) d ifx ˜(t0) = x0 and if dt x˜(t) = f(˜x(t), u(t), t) almost everywhere. We will often distinguish the following special cases:

1Whether we will consider a closed and finite or a half-open and semi-infinite interval will depend on the studied problem. For instance, the time-dependent controllability problem is considered on a closed interval while the feedback stabilization (typically) requires infinite time.

7 • Uncontrolled dynamics.

If u(t) = 0 for all t ∈ [t0, ∞) the system (1.1) turns into ( x˙(t) = f (x(t), t), x(t ) = x , 0 0 0 (1.2) y(t) = g0(x(t), t),

where f0(x, t) = f(x, 0, t), resp., g0(x, t) = g(x, 0, t). The dynamics of (1.2) depends only on the initial values x(t0) = x0. • Time-invariant dynamics. Let f and g do not depend explicitly on t. Then (1.1) turns into ( x˙(t) = f(x(t), u(t)), x(t ) = x , 0 0 (1.3) y(t) = g(x(t), u(t)).

The system (1.3) is invariant under time shift and hence we can set t0 = 0.

1.2 Typical problems solved by control theory

Below we list some problems which are addressed by control theory.

1. How to steer the system from the point A (i.e., x(t0) = xA) to the point B (x(T ) = xB) ; Open-loop control. 2. Does the above problem always possess a solution? ; Controllability analysis. 3. How to counteract the external disturbances resulting in deviations from the pre- computed trajectory? ; Feedback control. 4. How to get the necessary information about the system’s state? ; Observer design. 5. Is the above problem always solvable? ; Observability analysis. 6. How to drive the system to an equilibrium from any initial position? ; Stabiliza- tion. 7. And so on and so forth ... many problems are beyond the scope of our course.

1.3 Linearization

Typically, there are two ways to study a nonlinear system: a global and a local one. The global analysis is done using the methods from nonlinear control theory while the local analysis can be performed using linear control theory. The reason for this is that locally the behavior of most nonlinear systems can be well captured by a linear model. The procedure of substituting a nonlinear model with a linear one is referred to as the linearization.

8 Linearization in the neighborhood of an equilibrium point. The state x∗ is said to be an equilibrium (or ﬁxed) point of (1.1) if f(x∗, 0, t) = 0, ∀t. One can consider also controlled equilibria, i.e. the pairs (x∗, u∗) s.t. f(x∗, u∗, t) = 0, ∀t. Let x∗ be an equilibrium point of (1.1). Consider the dynamics of (1.1) in a suﬃciently small neighborhood of x∗, denoted by U(x∗). Let ∆x(t) = x(t) − x∗ be the deviation from the equilibrium point x∗. We write the DE for ∆x(t) expanding the r.h.s. into the Taylor series:

d ∗ ∂ ∂ ∆x(t) = f(x , 0, t)+ f(x, u, t) ∆x(t)+ f(x, u, t) u(t)+H.O.T.2 dt ∂x x=x∗,u=0 ∂u x=x∗,u=0

∂ ∂ Introducing notation A(t) = ∂x f(x, u, t) and B(t) = ∂u f(x, u, t) , re- x=x∗,u=0 x=x∗,u=0 calling that f(x∗, 0, t) and, ﬁnally, dropping the high-order terms we get

d ∆x(t) = A(t)∆x(t) + B(t)u(t). (1.4) dt The equation (1.4) is said to be Linear Time-Variant (LTV). If the initial nonlinear equation was time-invariant, we had the Linear Time-Invariant (LTI) equation:

d ∆x(t) = A∆x(t) + Bu(t). (1.5) dt Note that the linearization procedure can be applied to the second equation in (1.1) as well, thus yielding y(t) = C(t)∆x(t)+D(t)u(t) in the LTV case or y(t) = C∆x(t)+Du(t) in the LTI case (there could also be a constant term which can be easily eliminated by passing toy ˜(t) = y(t) − g(x∗, 0, t)).

Linearization in the neighborhood of a system’s trajectory. Consider the time- invariant nonlinear system (1.3). Let (x∗(t), u∗(t)) be the system’s trajectory and the corresponding control. Denote δx(t) = x(t) − x∗(t) and δu(t) = u(t) − u∗(t). The DE for δx(t) is

d δx(t) =x ˙(t) − x˙ ∗(t) = f(x(t), u(t)) − f(x∗(t), u∗(t)) = dt

∂ ∂ f(x, u) δx(t) + f(x, u) δu(t) + H.O.T. (1.6) ∂x x=x∗(t),u=u∗(t) ∂u x=x∗(t),u=u∗(t)

∂ ∂ Denoting A(t) = ∂x f(x, u) and B(t) = ∂u f(x, u) and drop- x=x∗(t),u=u∗(t) x=x∗(t),u=u∗(t) ping the high-order terms we get an LTV system (1.4). Note that even though the initial nonlinear system was time-invariant, its linearization around the system’s trajectory (x∗(t), u∗(t)) is time-variant!

2H.O.T. = high order terms.

9 1.3.1 * Hartman-Grobman theorem

A justification for using linearized models is given by the Hartman-Grobman theorem which is based on the notion of a hyperbolic fixed point. Definition 1.3.1. The equilibrium (fixed) point x∗ is said to be hyperbolic if all eigenvalues of the linearization A(t) have non-zero real parts. Theorem 1.3.1 (Hartman-Grobman). The set of solutions of (1.1) in the neighborhood of a hyperbolic equilibrium point x∗ is homeomorphic to that of the linearized system (1.4) in the neighborhood of the origin. Quoting Wikipedia: The Hartman–Grobman theorem ... asserts that linearization — our first resort in applications — is unreasonably effective in predicting qualitative patterns of behavior.

10 Chapter 2

Solutions of an LTV system

2.1 Fundamental matrix

Consider the set of homogeneous (i.e., uncontrolled) LTV diﬀerential equations:

x˙(t) = A(t)x(t), x(t0) = x0, (2.1)

n where x(t) ∈ R , t ∈ [t0,T ]. A(t) is component-wise continuous and bounded. Proposition 2.1.1. The set of all solutions of (2.1) forms an n-dimensional vector space over R. n Deﬁnition 2.1.1. A fundamental set of solutions of (2.1) is any set {xi(·)}i=1 such that n n for some t ∈ [t0,T ], {xi(t)}i=1 forms a basis of R . An n × n matrix function of t, Ψ(·) is said to be a fundamental matrix for (2.1) if the n columns of Ψ(·) consist of n linearly independent solutions of (2.1), i.e.,

Ψ(˙ t) = A(t)Ψ(t),

where Ψ(t) = ψ1(t) . . . ψn(t) . Exercise 2.1.1. Prove that rank Ψ(t) = n, t ∈ I ⇒ rank Ψ(t) = n, ∀t ∈ I .

Note that there are many possible fundamental matrices. For instance, an n × n matrix Ψ(t) satisfying Ψ(˙ t) = A(t)Ψ(t) with Ψ(t0) = In×n is a fundamental matrix. Example 2.1.2. Consider the system

0 0 x˙(t) = x(t). (2.2) t 0

That is,x ˙ 1(t) = 0,x ˙ 2(t) = tx1(t). The solution is:

1 1 x (t) = x (t ), and x (t) = t2x (t ) − t2x (t ) + x (t ). 1 1 0 2 2 1 0 2 0 1 0 2 0

11 x1(0) 0 0 2 Let t0 = 0 and ψ1(0) = = . Then ψ1(t) = . Now let ψ2(0) = . Then x2(0) 1 1 0 2 0 we have ψ (t) = = . Thus a fundamental matrix for the system is given by: 2 t2 1

0 2 Ψ(t) = . 1 t2

Proposition 2.1.2. Null space of a fundamental matrix is invariant for all t ∈ [t0,T ] and is equal to {0}. Corollary 2.1.3. Given a fundamental matrix Ψ(t), its inverse Ψ−1(t) exists for all t ∈ [t0,T ].

2.2 State transition matrix

Deﬁnition 2.2.1. The state transition matrix Φ(t, t0) associated with the system (2.1) is the matrix-valued function of t and t0 which:

1. Solves the matrix diﬀerential equation Φ(˙ t, t0) = A(t)Φ(t, t0), t ∈ [t0,T ],

2. Satisﬁes Φ(t, t) = In×n for any t ∈ [t0,T ]. Proposition 2.2.1. Let Ψ(t) be any fundamental matrix of (2.1). Then Φ(t, τ) = −1 Ψ(t)Ψ (τ), ∀t, τ ∈ [t0,T ].

−1 Proof. We have Φ(t0, t0) = Ψ(t0)Ψ (t0) = I . Moreover,

−1 −1 Φ(˙ t, t0) = Ψ(˙ t)Ψ (t0) = A(t)Ψ(t)Ψ (t0) = A(t)Φ(t, t0).

Proposition 2.2.2. The solution of (2.1) is given by x(t) = Φ(t, t0)x0.

Proof. The initial state is x(t0) = Φ(t0, t0)x0 = x0. Next, we show that x(t) = Φ(t, t0)x0 satisﬁes the diﬀerential equation:

x˙(t) = Φ(˙ t, t0)x0 = A(t)Φ(t, t0)x0 = A(t)x(t).

Lemma 2.2.3. Properties of the state transition matrix:

1. Φ(t, t1)Φ(t1, t0) = Φ(t, t0) — semi-group property.

−1 −1 −1 −1 2. Φ (t, t0) = Ψ(t)Ψ (t0) = Ψ(t0)Ψ (t) = Φ(t0, t).

3. Φ(˙ t0, t) = −Φ(t0, t)A(t) (hint: diﬀerentiate Φ(t0, t)Φ(t, t0) = I). T 4. If Φ(t, t0) is the state transition matrix of x˙(t) = A(t)x(t), then Φ (t0, t) is the state transition matrix of the system z˙(t) = −AT (t)z(t) — adjoint equation.

R t t tr(A(s))ds 5. det(Φ(t, t0)) = e 0 , where tr(A(t)) denotes the trace of matrix A(t).

12 R t t A(s)ds 6. If A(t) is a scalar, we have Φ(t, t0) = e 0 (NB: does not hold in general !). Example 2.2.1. The state transition matrix corresponding to the fundamental matrix found in Example 2.1.2 has the following form:   1 0 Φ(t, τ) = t2 − τ 2  1 2 Exercise 2.2.2. Check that the obtained state transition matrix deﬁnes solutions to (2.2).

2.3 Time-invariant case

Consider the time-invariant diﬀerential equation:

x˙(t) = Ax(t), x(t0) = x0. (2.3)

In this case, Φ(t, t0) = Φ(t − t0, 0) = Φ(t − t0) and

Φ(˙ t − t0) = AΦ(t − t0), Φ(t0) = I.

We can set t0 = 0 and consider Φ(t).

n×n Matrix Exponential If A ∈ R , the state transition matrix is (note that 0! = 1):

∞ 1 X ti Φ(t) = I + At + A2t2 + ... = Ai = eAt, 2 i! i=0 where the series converges uniformly and absolutely for any ﬁnite t. Henceforth, eAt will be referred to as the matrix exponential. Lemma 2.3.1. Properties of the matrix exponential: 1. AeAt = eAtA, that is A commutes with its matrix exponential. 2. eAt−1 = e−At.

3. If P is a nonsingular [n × n] matrix, then eP −1AP = P −1eAP (similarity transformation = change of the basis).

A a an 4. If A is a diagonal matrix, A = diag(a1, . . . , an), then e = diag(e 1 , . . . , e ). 5. If A and B commute, i.e., AB = BA, we have eA+B = eAeB. Example 2.3.1 (Harmonic motion). Consider the equation

x˙ (t) 0 ω x (t) 1 = 1 = Ax(t). x˙ 2(t) −ω 0 x2(t)

13 The exponential matrix is thus:

1 0 0 1 1 0 ω2t2 0 1 ω3t3 1 0 ω4t4 eAt = + ωt − − + + ... 0 1 −1 0 0 1 2 −1 0 3! 0 1 4!

x3 x5 x7 x2 x4 x6 Taking into account that sin(x) = x− 3! + 5! − 7! +... and cos(x) = 1− 2! + 4! − 6! +... we readily obtain: cos(ωt) sin(ωt) eAt = , − sin(ωt) cos(ωt) which is the rotation matrix that rotates the points of the Cartesian plane clockwise. Exercise 2.3.2. Using the result of the preceding example and the properties of the matrix exponential determine the matrix exponential eAt for the matrix

r φ A = . −φ r

Example 2.3.3 (Matrix exponential of a Jordan block). Let the [m × m] matrix J be of the form s 1 0 ··· 0 0 s 1 ··· 0    .. ..  J =  . .  ,   0 ··· 0 s 1 0 0 0 0 s where s ∈ C. The matrix J can be written as J = sI + U, where U is the upper shift matrix. First, we observe that I and U commute (as the identity matrix commutes with any square matrix). Thus we can write

eJt = esIteUt.

Next, note that U is nilpotent, i.e., U m = 0. (NB: every matrix with zero main diagonal is nilpotent). Finally, we have

m−1 X ti eJt = estI U i. i! i=0

2.4 Controlled systems: variation of constants formula

Consider the LTV system

x˙(t) = A(t)x(t) + B(t)u(t), x(t0) = x0, (2.4) whose homogeneous (uncontrollable) solution is x(t) = Φ(t, t0)x0.

14 Theorem 2.4.1. If Φ(t, t0) is the state transition matrix for x˙(t) = A(t)x(t), then the unique solution of (2.4) is given by

t Z x(t) = Φ(t, t0)x0 + Φ(t, s)B(s)u(s)ds. (2.5)

Proof. Deﬁne the new variable z(t) = Φ(t0, t)x(t). Diﬀerentiating z(t) w.r.t. t we get

z˙(t) = Φ(˙ t0, t)x(t) + Φ(t0, t)x ˙(t) =

− Φ(t0, t)A(t)x(t) + Φ(t0, t)A(t)x(t) + Φ(t0, t)B(t)u(t),

where the ﬁrst two terms cancel. The resulting expression does not contain z(t) in the r.h.s. thus we can integrate it to get the solution:

t Z z(t) = z(t0) + Φ(t0, s)B(s)u(s)ds,

whence follows

t t Z Z −1 x(t) = Φ (t0, t) x0 + Φ(t0, s)B(s)u(s)ds = Φ(t, t0)x0 + Φ(t, s)B(s)u(s)ds.

t0 t0

Corollary 2.4.2. The solution of a linear time-invariant equation is given by

t Z At A(t−s) x(t) = e x0 + e Bu(s)ds. (2.6) 0

Example 2.4.1 (Exponential input). Consider a (complex valued)1 LTI system with a scalar exponential input signal eσt:

σt x˙(t) = Ax(t) + be , x(·) ∈ C, x(0) = x0 (2.7) where σ ∈ C. The solution of (2.7) is found using (2.6):

t Z At A(t−τ) στ x(t) = e x0 + e be dτ, 0 which can be solved using integration by parts to yield

At −1 σt At x(t) = e x0 + (σI − A) (Ie − e )b. (2.8)

1We consider an LTI system in complex domain as we wish to include also harmonic input signals, iωt e.g. u(t) = e . This condition can be dropped if we assume that σ ∈ R.

15 Assume that the parameter σ is equal to an eigenvalue of A. This is referred to as the resonance. At ﬁrst sight it seems that there is a singularity in the solution. To inspect this case more closely we rewrite (2.8) as x(t) = eAt (σI − A)−1 e(σI−A)t − I b.

−1 Zt P∞ (Zt)k and note that Z (e − I) = t k=0 (k+1)! , which converges everywhere. Hence we conclude that the solution x(t) is well deﬁned for all σ ∈ C. −1 Observe that for x0 = (σI − A) b the solution (2.8) takes a particularly simple form:

x(t) = (σI − A)−1 beσt, that is to say, for a properly chosen initial value the linear system transforms an exponential signal into another exponential signal. Remember this fact. We shall elaborate on it in Sec. 5.3.1.

16 Chapter 3

Controllability and observability

3.1 Controllability of an LTV system

When approaching a control system a ﬁrst step consists in determining whether the system can be controlled and to which extent? This type of problem is referred to as the controllability problem. To make this more concrete we consider the following problem statement.

Two-point controllability. Consider the LTV system (2.4). Given initial state x0 at time t0, ﬁnd an admissible control u such that the system reaches the ﬁnal state x1 at time t1. Solving this problem amounts to determining an admissible controlu ¯(t) ∈ U, t ∈ [t0, t1], (typically non-unique) that solves the following equation:

t Z 1 x1 = Φ(t1, t0)x0 + Φ(t1, s)B(s)¯u(s)ds. (3.1)

Obviously, the two-point controllability problem is stated in a very limited way. We need a general formulation as deﬁned below.

Deﬁnition 3.1.1. The system (2.4) deﬁned over [t0, t1] is said to be completely controllable (or just controllable) on [t0, t1] if, given any two states x0 and x1, there exists an admissible control that transfers (x0, t0) to (x1, t1). Otherwise the system is said to be uncontrollable.

Remark. Note that a system can be completely controllable on some interval [t0, t1] and 0 0 uncontrollable on [t0, t1] ⊂ [t0, t1]. However, it turns out that if a system is controllable 00 00 on [t0, t1] it will be controllable for any [t0, t1] ⊃ [t0, t1]. 00 00 Exercise 3.1.1. Prove that controllability on [t0, t1] implies controllability on [t0, t1] ⊃ [t0, t1]. An LTV system is characterized by its structural elements, i.e. by the matrices A(t) and B(t). In this sense we can speak about controllability of the pair (A(t),B(t)). Thus our goal will be to characterize the controllability property of (2.4) in terms of (A(t),B(t)).

17 To do so we ﬁrst transform (3.1) to a generic form. Denotingx ˆ1 = x1 − Φ(t1, t0)x0 we rewrite (3.1) as t Z 1 xˆ1 = Φ(t1, s)B(s)¯u(s)ds, (3.2)

t0 which amounts to determining an admissible inputu ¯(t) that transfers the zero state at t0 tox ˆ1 at t1. This problem is typically referred to as the reachability problem. One can easily see that for a linear system controllability and reachability problems are equivalent. Using (3.2) we can give the following characterization of two-point controllability. Proposition 3.1.1. The pair (x0, x1) is controllable iﬀ x1 −Φ(t1, t0)x0 belongs to the range of the linear map L(u), where

t Z 1 L(u) = Φ(t1, s)B(s)u(s)ds. (3.3)

The above condition is particularly difficult to check as the map L is defined on the infinite-dimensional space of admissible controls U. We would prefer to have some finite- dimensional criterion. Such criterion will be formulated below but first we present the following formal result. Lemma 3.1.2. Let G(t) be an [n × m] matrix whose elements are continuous functions of t, t ∈ [t , t ]. A vector x ∈ n lies in the range space of L(u) = R t1 G(s)u(s)ds if and 0 1 R t0 only if it lies in the range space of the matrix

Z t1 T W (t0, t1) = G(s)G (s)ds. t0

T Proof. (if) If x ∈ R(W (t0, t1)), then there exists η s.t. x = W (t0, t1)η. Takeu ¯ = G η, then L(¯u) = W (t0, t1)η = x and so, x ∈ R(L(¯u)).

⊥ (only if) Let there be x1 ∈/ R(W (t0, t1)). Then there exists x2 ∈ R (W (t0, t1)), i.e., T T x2 W (t0, t1) = 0. Obviously, x1 ∈/ R(W (t0, t1)) implies that x2 x1 6= 0. Suppose, ad absurdum, that there exists a control u s.t. R t1 G(s)u (s)ds = x . Then we have 1 t0 1 1

Z t1 T T x2 G(s)u1(s)ds = x2 x1 6= 0. (3.4) t0

T But, x2 W (t0, t1) = 0 and so,

Z t1 T T T x2 W (t0, t1)x2 = x2 G(s) G (s)x2 ds = 0. t0

Observe that xT W (t , t )x = R t1 kGT (s)x kds = 0 implies G(t) ≡ 0 for all t ∈ [t , t ], 2 0 1 2 t0 2 0 1 whence a contradiction of (3.4) follows.

18 Now we can use the results of Proposition 3.1.1 and Lemma 3.1.2 to formulate the following fundamental theorem on controllability. Theorem 3.1.3. The pair (x0, x1) is controllable if and only if x1 −Φ(t, t0)x0 belongs to the range space of

Z t1 T T W (t0, t1) = Φ(t1, s)B(s)B (s)Φ (t1, s)ds. (3.5) t0

T T Moreover, if η¯ is a solution of W (t0, t1)η = x1−Φ(t, t0)x0 , then u¯(t) = B (t)Φ (t1, t)¯η is one possible control that accomplishes the desired transfer.

The matrix W (t0, t1) is called the controllability Grammian. The result of Theorem 3.1.3 can be readily seen to encompass the complete controllability case. Corollary 3.1.4. The system (2.4) is completely controllable on [t0, t1] if and only if rank W (t0, t1) = n.

Properties of the controllability Grammian: T 1. W (t0, t1) = W (t0, t1);

2. W (t0, t1) is positive semi-deﬁnite for t1 ≥ t0;

3. W (t0, t) satisfies the linear matrix differential equation (note that we fix t0 and vary t1): d W (t , t) = A(t)W (t, t ) + W (t, t )AT (t) + B(t)BT (t),W (t , t ) = 0; dt 0 1 1 0 0

4. W (t0, t1) satisﬁes the functional equation

T W (t0, t1) = Φ(t1, t)W (t0, t)Φ (t1, t) + W (t, t1).

Note that some authors deﬁne the controllability Grammian as

t Z 1 T T W˜ (t0, t1) = Φ(t0, s)B(s)B (s)Φ (t0, s)ds.

t0 These two forms of the Gram matrix are congruent, i.e. they are related by the following T transformation: W˜ (t0, t1) = Φ(t0, t1)W (t0, t1)Φ (t0, t1).

3.1.1 * Optimality property of u¯

As Proposition 3.1.1 states, the pair of states x0 and x1 is controllable on [t0, t1] if x1 − Φ(t1, t0)x0 belongs to the range of L(u), (3.3). In general, there is a set of −1 controls Ux0,x1 = L x1 − Φ(t1, t0)x0 ⊂ U which satisfy this condition. That is to say,

19 there are potentially inﬁnitely many controls that bring the system from x(t0) = x0 to x(t1) = x1. However, it turns out that the controlu ¯ deﬁned in Theorem 3.1.3 enjoys a very particular property.

We assume for simplicity that the system (2.4) is completely controllable on [t0, t1] and hence W (t0, t1) is full rank. Then we can write

T T −1 u¯(t) = B (t)Φ (t1, t)W (t0, t1) x1 − Φ(t, t0)x0 ∈ Ux0,x1 . (3.6) We have the following result.

Theorem 3.1.5. For any v ∈ Ux0,x1 holds

Z t1 Z t1 ku¯(s)kds < kv(s)kds. (3.7) t0 t0 Proof. Since bothu ¯ and v belong to Ux0,x1 , i.e., L(¯u) = L(v) = x1 − Φ(t, t0)x0 , and due to linearity of L we have

t Z 1 L(¯u − v) = Φ(t1, s)B(s) u¯(s) − v(s))ds = 0.

T −1 T Premultiplying the preceding equation by x1 − Φ(t, t0)x0 W (t0, t1) we get

t Z 1 T −1 T x1 − Φ(t, t0)x0 W (t0, t1) Φ(t1, s)B(s) u¯(s) − v(s))ds =

t0 t Z 1 u¯T (s)u¯(s) − v(s))ds = 0,

t0 whence follows R t1 ku¯(s)k2 = R t1 hu¯(s), v(s)ids. To complete the proof we consider t0 t0

t Z 1 0 < hu¯(s) − v(s), u¯(s) − v(s)ids

t0 t t t Z 1 Z 1 Z 1 = hu¯(s), u¯(s)ids − 2 hu¯(s), v(s)ids + hv(s), v(s)ids

t0 t0 t0 t t Z 1 Z 1 = − hu¯(s), u¯(s)ids + hv(s), v(s)ids,

t0 t0 which implies t t Z 1 Z 1 ku¯(s)k2ds < kv(s)k2ds

t0 t0 as required.

20 This result can be interpreted in the sense that the controlu ¯(t) has the least energy among all controls steering the system (2.4) from x(t0) = x0 to x(t1) = x1. This result is a precursor of optimal control theory that we will touch upon later.

3.2 Observability of an LTV system

When using feedback control it is crucial to be able to determine the system’s state based upon the observed system’s output. This is referred to as the observability problem. Observability Consider the LTV system

x˙(t) = A(t)x(t) + B(t)u(t), x(t ) = x , 0 0 (3.8) y(t) = C(t)x(t).

Given an admissible input u(t) and the observed output function y(t), t ∈ [t0, t1], ﬁnd the initial value x0 at t = t0. First, we note that the output of the system (3.8) can be written as

Z t y(t) = C(t)Φ(t, t0)x0 + Φ(t, s)B(s)u(s)ds (3.9) t0 The last term in (3.9) depends only on the control u and can therefore be computed a priori for any u(t). To simplify the notation we set u(t) = 0 and consider the homogeneous system x˙(t) = A(t)x(t), x(t ) = x , 0 0 (3.10) y(t) = C(t)x(t).

0 00 The observability property can be reformulated as follows: given two states, x0 and x0, 0 00 under which conditions there exists t ∈ [t0, t1] such that C(t)Φ(t, t0)x0 6= C(t)Φ(t, t0)x0? This question can be answered by analyzing the null space of C(t)Φ(t, t0)x0. First, consider the following technical lemma: Lemma 3.2.1. Let H(t) be an [m × n] matrix whose elements are continuous functions n m deﬁned on the interval [t0, t1]. The null space of the mapping O : R → C([t0, t1], R ) deﬁned by O(x) = H(t)x coincides with the null space of the matrix

Z t1 T M(t0, t1) = H (s)H(s)ds. t0

Proof. If x ∈ N (M(t0, t1)), then

Z t1 Z t1 T T T 2 x M(t0, t1)x = x H (s)H(s)xds = kH(s)xk ds = 0, t0 t0 whence H(t)x = 0 for all t ∈ [t0, t1].

With this result, we can formulate a theorem which is closely related to Theorem 3.1.3.

21 Theorem 3.2.2. Consider the system (3.8). Let the matrix M(t0, t1) be deﬁned by

Z t1 T T M(t0, t1) = Φ (s, t0)C (s)C(s)Φ(s, t0)ds. t0 Then we have two cases:

1. M(t0, t1) is non-singular. Then any initial state x0 can be determined uniquely from the observed output y(t). 0 00 2. M(t0, t1) has a non-zero null space. Then any two points x0 and x0 are indistin- 0 00 guishable if x0 − x0 ∈ N (M(t0, t1)).

T T Proof. We write the expression for y(t) in (3.8) and multiply from the right by Φ (t, t0)C (t) to get T T T T Φ (t, t0)C (t)y(t) = Φ (t, t0)C (t)C(s)Φ(s, t0)x0.

Integrating this from t0 to t1 yields

Z t1 T T Φ (s, t0)C (s)y(s)ds = M(t0, t1)x0. t0

If M(t0, t1) is non-singular, we ﬁnd x0 as

Z t1 −1 T T x0 = M (t0, t1) Φ (s, t0)C (s)y(s)ds. t0 0 00 To prove the second statement we suppose that x0 − x0 ∈ N (M(t0, t1)). Then

Z t1 Z t1 0 00 2 0 00 2 ky (s) − y (s)k ds = kC(s)Φ(s, t0)x0 − C(s)Φ(s, t0)x0k ds t0 t0 Z t1 0 00 T T T 0 00 = (x0 − x0) Φ (s, t0)C (s)C(s)Φ(s, t0)ds (x0 − x0) t0 0 00 T 0 00 = (x0 − x0) M(t0, t1)(x0 − x0) = 0,

0 00 whence y (t) = y (t) for all t ∈ [t0, t1].

The matrix M(t0, t1) is called the observability Grammian.

3.3 Duality principle

The duality principle formalizes our intuition about the similarity between the controllability and observability conditions for a linear system. Consider two systems: ( ( x˙(t) = A(t)x(t) + B(t)u(t) x˙(t) = −AT (t) + CT (t)u(t) Σ: and Σ:˜ y(t) = C(t)x(t) y(t) = BT (t)x(t).

We have the following result:

22 Theorem 3.3.1 (Duality). System Σ is completely observable (controllable) iﬀ the dual system Σ˜ is completely controllable (observable).

Proof. The proof is left to the reader as an exercise (Hint: use item 4. of Lemma 2.2.3).

3.4 Controllability of an LTI system

From now on we will consider LTI systems, that is linear systems with constant coeﬃ- cients: x˙(t) = Ax(t) + Bu(t), x(t ) = x , 0 0 (3.11) y(t) = Cx(t), where A, B and C are matrices of appropriate dimensions. For the system (3.11) the controllability Grammian turns out to be

t Z T W (t) = eA(t−s)B(s)BT (s)eA (t−s)ds. (3.12) 0 Exercise 3.4.1. Show that the transpose operation and the matrix exponentiation commute.

3.4.1 Kalman’s controllability criterion

Theorem 3.4.1. The system (3.11) is completely controllable iﬀ

rank B AB A2B...An−1B = n. (3.13)

Proof. (⇒): Let the system (3.11) be completely controllable. Assume that

rank B AB A2B...An−1B = m < n.

Then there exists a non-zero vector c such that

cT B AB A2B...An−1B = 0

which is equivalent to cT B = 0, cT AB = 0,. . . , cT An−1B = 0. By the Cayley-Hamilton theorem this implies that cT AkB = 0 for all k ≥ 0 and hence

cT eAtB = 0 ∀t ≥ 0.

This implies that c belongs to the null space of W (t) and so, since c 6= 0, rank W (t) < n. Finally, this means that the system (3.11) is not completely controllable which contradicts the assumption. (⇐): Let rank B AB A2B...An−1B = n. Assume that the system is not controllable, i.e., rank W (t) < n. Then there exists a non-zero vector c such that W (t)c = 0,

23 which implies that cT eAtB = 0. Expanding the matrix exponential we get cT AiB = 0 for all i ≥ 0. However, this implies that

rank B AB A2B...An−1B < n, which contradicts the assumption.

2 n−1 The matrix KC = B AB A B...A B is called the Kalman’s controllability matrix. The following proposition provides some intuition to the controllability criterion formu- n lated above. First, we recall that a linear subspace S ⊂ R is invariant under the action of A (A-invariant), where A is an [n × n]-matrix, if x ∈ S ⇒ Ax ∈ S. We have the following: n Proposition 3.4.2. Let S ⊂ R be an invariant subspace of A and let rank B = m < n. Then det B AB A2B...An−1B 6= 0 if and only if B/∈ S.

Proof. The proof is left to the reader as an exercise.

Consider two examples. 0 1 1 Example 3.4.2. Let A = and B = . The matrix exponential is eAt = 0 −1 0 −t t 1 e R 1 0 −t and the controllability Grammian is W (t) = dτ. Note that 0 1 − e 0 0 0 rank W (t) = 1 for any t > 0. 0 1 1 Example 3.4.3. Let A = and B = . The matrix exponential is eAt = −1 0 0 cos(t) sin(t) and the controllability Grammian is − sin(t) cos(t)

t " # " # Z cos2(t) − cos(t) sin(t) 1 2t + sin (2 t) −2 sin2(t) W (t) = dτ = . − cos(t) sin(t) sin2(t) 4 −2 sin2(t) 2t − sin (2 t) 0

1 2 2 One can easily check that det W (t) = 4 t − sin (t) 6= 0 for any t > 0. The above illustrates the following important fact.

For an LTI system the property of being completely controllable does not depend on the considered time interval [0,T ].

3.4.2 Decomposition of a non-controllable LTI system

This subsection continues developing geometrical intuition about the controllability property of an LTI system which we brieﬂy touched upon in Prop. 3.4.2.

24 Let us again consider an [n × n] matrix A and its invariant subspace S. Then there n exists a complementary subspace S˜ such that S ⊕ S˜ = R . Let the system (3.11) fails to be completely controllable, i.e.,

rank B AB A2B...An−1B = m < n.

This means that the controllability matrix KC has exactly m independent columns. Consider the linear subspace generated by these columns: S = span(KC ). Then we have the following result. Lemma 3.4.3. Subspace S = span(KC ) is A-invariant.

Proof. Consider

2 3 n AS = span(AKC ) = span AB A BA B...A B .

The columns of ﬁrst n − 1 components of the matrix AKC belong to S. To show that the columns of AnB also belong to S we use the Cayley-Hamilton theorem and write AnB as n n−1 A B = −(anI + an−1A + ... + a1A )B, whence follows that the columns of AnB are linear combinations of vectors from S. Thus we have AS ⊆ S.

Let vectors (v1, v2, . . . vm) form a basis of S. Consider a matrix T = v1 v2 . . . vm T˜ , where T˜ is chosen such that T is non-singular. Lemma 3.4.4. The matrix T deﬁnes a transformation such that the matrices A and B take the following form:

A A B AC = T −1AT = 11 12 ,BC = T −1B = 1 . (3.14) 0 A22 0

Proof. First, we show that AT = TAC . Let us rewrite this as follows: A11 A12 A v1 v2 . . . vm | T˜ = v1 v2 . . . vm | T˜ . A21 A22 The vectors vi are A-invariant, thus the product A v1 v2 . . . vm can be written as a linear combination of the same vectors vi, i = 1, . . . , m. This implies that the matrix A21 = 0. For BC , we need to check that B = TBC . This follows from the fact that span (B) ⊂ S.

3.4.3 Hautus’ controllability criterion

An alternative way to check the controllability of an LTI system is given by Hautus’ controllability condition as detailed below.

25 Theorem 3.4.5. The system (3.11) is completely controllable if and only if

rank(sI − A, B) = n for all s ∈ C. (3.15)

Proof. (⇒): Assume that there exists an s0 ∈ C such that rank(s0I − A, B) < n. Then there exists a non-zero vector c for which T T T T c (s0I − A, B) = 0 ⇒ c A = s0c and c B = 0. T k k T From the last equalities we have c A B = s0c B = 0 for all k ≥ 1, which implies that cT B AB . . . AkB = 0. Hence, (3.11) is not completely controllable. (⇐): Assume that the system (3.11) is not completely controllable. This implies that the controllability subspace S = span(C) has dimension less than n, i.e., dim S < n. Let S⊥ be the orthogonal complement of S, i.e., for any v ∈ S and w ∈ S⊥ we have vT w = 0. We can observe that S⊥ is AT -invariant, i.e., AT S⊥ ⊆ S⊥ as follows from vT AT w = (Av)T w=0. The latter equality follows from the fact that S is A-invariant, whence Av ∈ S. It is known that every invariant subspace contains at least one eigenvector s, i.e., there exist λ ∈ C and s ∈ S⊥ such that AT s = λs or, equivalently, sT (A − λI) = 0. However, we also have sT B = 0 for any s ∈ S⊥. Thus (3.15) does not hold. Remark. Note that the Hautus condition can be simpliﬁed even further by requiring that (3.15) holds for all s ∈ Λ(A) as for s∈ / Λ(A) we have rank(sI − A) = n and (3.15) is satisﬁed trivially.

3.5 Observability of an LTI system

The observability conditions can be derived from controllability conditions by using the duality principle. In particular, we have the following counterparts of the Kalman and Hautus controllability criteria discussed in the previous section. Theorem 3.5.1 (Kalman’s observability criterion). The system (3.11) is completely observable iﬀ  C   CA  rank   = n. (3.16)  ···  CAn−1

Proof. We recall that the observability of (3.11) is equivalent to the controllability of ( x˙(t) = −AT (t) + CT (t)u(t) y(t) = BT (t)x(t). Thus we write the observability condition  C  T T T T n−1 T  CA  rank C −A C ··· (−A ) C = rank   = n.  ···  CAn−1

26 Note that the rank of a matrix is invariant under transposition and elementary row operations (row switching, multiplication, addition).

 C   CA  The matrix KO =   will be referred to as the Kalman’s observability matrix.  ···  CAn−1 Theorem 3.5.2 (Hautus’ observability criterion). The system (3.11) is completely observable if and only if

sI − A rank = n for all s ∈ . (3.17) C C

Proof. Using the duality principle we write

sI − A rank sI + AT CT = rank = n for all s ∈ . (3.18) C C

In the first equality we transposed the fist matrix, multiplied the first n rows by −1 and used the fact that s is arbitrary.

3.5.1 Decomposition of a non-observable LTI system

Similarly to the decomposition into controllable and uncontrollable subsystems, a non- observable linear time-invariant system can be decomposed into an observable and unobservable subsystem as stated below.

Consider an uncontrollable LTI system, i.e., the system (3.11) such that dim KO = T T T T m < n. Let vectors w1 , w2 , . . . wm form a basis of span(KO). Consider a matrix T T T ˜T ˜ T = w1 w2 . . . wm T , where T is chosen such that T is non-singular. Lemma 3.5.3. The matrix T deﬁnes a transformation such that the matrices A and B take the following form: O −1 A11 0 O −1 A = T AT = ,C = CT = C1 0 . A21 A22

T T Proof. To prove this Lemma we ﬁrst show that span(KO) is A -invariant. Next, we show that1 AT T T = T T (AO)T which is equivalent to

T T T T T T ˜ T T T ˜ A11 A21 A w1 w2 . . . wm T = w1 w2 . . . wm T T . 0 A22

The rest is left to the reader as an exercise.

1This a little messy notation is the price that we have to pay for staying with columns and column spaces.

27 3.6 Canonical decomposition of an LTI control system

C Consider a general linear control system (3.11) such that rank K = k1 ≤ n and O n rank K = k2 ≤ n. The vector space R can be decomposed in two ways: as a n ⊥ direct sum of the controllable and uncontrollable subspaces, R = C ⊕ C , where C C = span(K ), dim C = k1 or as a direct sum of the observable and unobservable n ⊥ O subspaces, R = O ⊕ O , where O = span(K ), dim O = k2. Note that some of these subspaces can be trivial. Without going into too much detail we formulate the following result. Theorem 3.6.1. There exists a non-singular [n × n]-matrix T such that     A11 0 A13 0 B1 −1 A21 A22 A23 A24 −1 B2 Aˆ = T AT =   , Bˆ = T B =   ,  0 0 A33 0   0  (3.19) 0 0 A43 A44 0 Cˆ = CT = C1 0 C3 0 .

The transformed system is composed of four subsystems:

• (A11,B1,C1) – controllable, observable;

• (A22,B2, 0) – controllable, unobservable;

• (A33, 0,C3) – uncontrollable, observable;

• (A44, 0, 0) – uncontrollable and unobservable.

Note that these subsystems are not fully decoupled as the oﬀ-diagonal matrices A13, A21 etc. can be non-zero. However, when studying structural properties of the system we can neglect these coupling terms as they do not inﬂuence the controllability/observability properties. Note that the second block-column of Aˆ is all zero except A22; the third row is all zero except A33.

Exercise 3.6.1. Explain why the subsystem (A44, 0, 0) is uncontrollable and unobservable despite A24 6= 0 and A42 6= 0.

28 Chapter 4

Stability of LTI systems

4.1 Matrix norm and related inequalities

A vector space X is said to be normed if for any x ∈ X there exists a nonnegative number kxk which is called a norm and which satisﬁes the following conditions: 1. kxk ≥ 0 and kxk = 0 ⇔ x = 0,

2. kaxk = |a|kxk for any x ∈ X and a ∈ R, 3. kx + yk ≤ kxk + kyk (triangle inequality).

n In the following we will consider X = R and will use the Euclidean norm kxk = s n P 2 xi . i=1

n Note that all presented results remain valid if we consider X = C with a change of the Euclidean norm to the Hermitian one.

m×n For any Q ∈ R , the Euclidean norm induces a matrix norm (a.k.a. spectral norm)

kQxk p kQk = max = max kQxk = max xT QT Qx. x6=0 kxk kxk=1 kxk=1

Consider the matrix QT Q. This matrix is nonnegative and symmetric. Thus all its eigenvalues are nonnegative and real. The square roots of the respective eigenvalues are called the singular values of the matrix Q. m×n T √ Lemma 4.1.1. If Q ∈ R , then kQk = kQ k = λn, where λn is the largest T T eigenvalue of Q Q. Also, λn is equal to the largest eigenvalue of QQ .

Proof. First, we consider the following optimization problem:

µ2 = max xT QT Qx. (4.1) xT x=1

29 The corresponding Lagrange function is L(x, λ) = xT QT Qx + λ(1 − xT x) and the necessary optimality conditions are ∂L = 2QT Qx − 2λx = 0, ∂x ∂L = 1 − xT x = 0. ∂λ We see that the extremal value of x, denoted by x∗ is such that x∗ 6= 0. The ﬁrst equation implies that x∗ is an eigenvector of QT Q and λ is the corresponding eigenvalue. Substituting x∗ into (4.1) we get

2 T T ∗ T ∗ ∗ T ∗ µ = max x Q Qx = max x Q Qx = max λ(x ) x = λn, xT x=1 (x∗)T x∗=1 (x∗)T x∗=1 √ whence µ = λn. Next, we observe that if λ is a nonzero eigenvalue of QT Q then it is also an eigenvalue of QQT . If QT Qx = λx then QQT (Qx) = λ(Qx). Thus we conclude that kQk = kQT k. Lemma 4.1.2. If matrix Q is block-diagonal, i.e.,   Q1 0 0  ..  Q =  0 . 0  , 0 0 Qk we have kQk = max kQik. i=1,...,k

Proof. We note that  T  Q1 Q1 0 0 T  ..  Q Q =  0 . 0  T 0 0 Qk Qk √ √ and so, µ = max λ = max max λi = max kQik. T i=1,...,k T i=1,...,k λ∈Λ(Q Q) λi∈Λ(Qi Qi)

Basic properties of the induced matrix norm. If A and B are real matrices then

1. kAk ≥ 0 and kAk = 0 ⇔ A = 0, 4. kAxk ≤ kAk · kxk,

2. kaAk = |a|kAk for any a ∈ R, 5. kABk ≤ kAk · kBk, 3. kA + Bk ≤ kAk + kBk, 6. kAk = kAT k.

To prove the 4-th and the 5-th properties we will use the deﬁnition of the spectral norm: kAxk kAxk kAk = max ⇒ kAk ≥ ⇒ kAk · kxk ≥ kAxk x6=0 kxk kxk kABxk kAyk kBxk kAyk kBxk kABk = max = max · ≤ max ·max ⇒ kABk ≤ kAk·kBk, x6=0 kxk x6=0 kyk kxk y6=0 kyk x6=0 kxk where in the last construction we denoted y = Bx.

30 n×n Matrix exponential and its estimates. Let A ∈ R . Consider the matrix exponential eAt. We have the following results. Lemma 4.1.3. keAtk ≤ ekAk·|t| for all t ≥ 0.

Proof. Using the properties of the induced matrix norm we have:

∞ i ∞ i At X t i X |t| i kAk·|t| ke k ≤ kA k ≤ kAk ≤ e . i! i! i=0 i=0

Theorem 4.1.4. Given an [n × n] real matrix A, let α = max <(λ). Then for any λ∈Λ(A) > 0 there exists γ() ≥ 1 such that for any t ≥ 0 holds

keAtk ≤ γ()e(α+)t.

Proof. The proof is based on the fact that any square matrix A can be transformed to a Jordan (block-diagonal) form by a similarity transformation: A = PJP −1. It follows that eAt = P eJtP −1 and hence, keAtk ≤ kP k · keJtk · kP −1k. The proof is J t concluded by considering the norms ke i k, where Ji are the elementary blocks of the Jordan matrix.

4.2 Stability of an LTI system

4.2.1 Basic notions

In this and the subsequent sections we will consider the stability property of an uncontrolled LTI system. That is to say, we will set u = 0 and will consider the homogeneous system

x˙(t) = Ax(t), x(0) = x0. (4.2)

Deﬁnition 4.2.1. An LTI system (4.2) is said to be uniformly stable if for any x0 there exists a constant γ ≥ 1 such that the respective solution x(t, x0) to the system (4.2) satisﬁes

kx(t, x0)k ≤ γkx0k for any t ≥ 0. Definition 4.2.2. An LTI system (4.2) is said to be exponentially stable if for any x0 there exist constants γ ≥ 1 and σ > 0 such that the respective solution x(t, x0) to the system (4.2) satisfies −σt kx(t, x0)k ≤ γe kx0k for any t ≥ 0. Theorem 4.2.1. System (4.2) is exponentially stable iff all eigenvalues of A have strictly negative real parts.

31 Proof. (⇒): Suppose that there exists an eigenvalue λ ∈ Λ(A) such that <(λ) ≥ 0. Let v be the corresponding eigenvector, then the system has a particular solution x(t) = eλtv. The norm of x(t) does not decrease to 0 with time, thus the system is not asymptotically stable.

(⇐): Suppose that all eigenvalues of (4.2) lie in the open left half plane of the complex plane and α is the real part of the rightmost eigenvalue. Let > 0 be a suﬃciently small positive constant s.t. α + < 0. Then according to Theorem 4.1.4 we have keAtk ≤ γe(α+)t and so,

At (α+)t kx(t, x0)k ≤ ke k · kx0k ≤ γe kx0k.

Corollary 4.2.2. System (4.2) is exponentially stable iﬀ all roots of the characteristic n n−1 polynomial det(sI −A) = s +a1s +...+an−1s+an have strictly negative real parts. Such polynomials are said to be Hurwitz stable.

4.2.2 * Some more about stability

Multiple eigenvalues. To warm up for this little excursion, the reader is asked to prove the following standard result. Exercise 4.2.1. Show that if for the matrix A • λ is a real eigenvalue with eigenvector v, then there is a solution to the system (4.2) of the form x(t) = veλt. • λ = a ± ib is a complex conjugate pair with eigenvectors v = u ± iw (where at at u, w ∈ R) then x1(t) = e (u cos bt − w sin bt) and x2(t) = e (u sin bt + w cos bt) are two linearly-independent solutions. We can thus observe that the real part of λ is the key ingredient in determining stability (as we already mentioned in the preceding section). However, there are some exceptions. When multiple eigenvalues exist and there are not enough linearly-independent eigen- n k <(λ)t vectors to span R , the solutions behave like |x(t)| ∼ t e , where (k − 1) does not exceed the algebraic multiplicity of the root λ. This implies that a solution can have a large overshoot, but it will eventually decay if λ < 0. Note that an overshoot may occur even for simple eigenvalues. −2 α Example 4.2.2. Consider the system (4.2) with the matrix A = . Its eigen- 0 −1 1 α values are −1 and −2, and the respective eigenvalues are v = and v = . 1 0 2 1 The eigenvalues are real negative, hence the system is stable. However, taking x(0) = −t −2t v2 − αv1, the first coordinate x1(t) = α(e − e ) initially grows from zero to a maxi- mum value of α/4 at t = − ln(1/2). For sufficiently large α, the trajectory first moves far away from the fixed point before approaching it as t → ∞. Finally, consider the matrix A such that there is one eigenvalue λ˜, <(λ˜) = 0, of multiplicity greater than 1. Two cases are possible:

32 1. The geometric multiplicity of λ˜ is less than its algebraic multiplicity, i.e., the eigenvalues associated with λ˜ form a linear subspace of dimension less than the algebraic multiplicity of λ˜. Then, there exists a vectorv ˜ (called the generalized eigenvector) such that the respective solution will be of form x(t) =vt ˜ (depending on the diﬀerence between the algebraic and the geometric multiplicities there could be terms involving tk with k > 1). 2. If the geometric and the algebraic multiplicities of λ˜ coincide, that is λ˜ is a semisimple eigenvalue, the matrix A can be transformed to a diagonal one and hence all solutions either will be constants or will oscillate with constant amplitude depending on the imaginary part od λ˜. The former solution is unstable, while the latter one is uniformly stable (but obviously not exponentially stable).

Stability of time-variant systems. In our course we do not consider stability of time-variant systems. In general, LTV systems behave similar to LTI ones, but some more caution is needed when defining the exponential stability property as the following example demonstrates. Example 4.2.3. Consider the systemx ˙ = −xe−t. It satisfies the condition of Definition 4.2.2 with γ = 1. However, the system does not converge to x(t) = 0 as t → ∞ since the right-hand side decreases faster than exponentially and hence x(t) converges to some ∗ finite non-zero value 0 < x < x(t0). To overcome this difficulty we may extend Definition 4.2.2 by requiring that there exist constants γl < γ and σl > σ such that

−σlt −σt γle kx0k ≤ kx(t; t0, x0)k ≤ γe kx0k.

The left inequality eﬀectively prevents x(t; t0, x0) from decreasing faster than exponentially.

4.2.3 Lyapunov’s criterion of asymptotic stability

Definition 4.2.3. A quadratic form v(x) = xT V x, V = V T , is said to be positive n definite (positive semi-definite) if v(x) > 0 (v(x) ≥ 0) for any x ∈ R \{0}. Since V = V T , all eigenvalues of V are real. Furthermore, a quadratic form is positive definite if and only if all its eigenvalues are strictly positive. This follows, for instance, from the following estimate: 2 2 λminkxk ≤ v(x) ≤ λmaxkxk .

We have the following result. Theorem 4.2.3. The system (4.2) is exponentially stable if and only if there exist two positive deﬁnite quadratic forms v(x) = xT V x and w(x) = xT W x such that the following identity holds along the solutions of (4.2): d v(x(t)) = −w(x(t)). (4.3) dt

33 The quadratic form v(x) satisfying (4.3) is called the quadratic Lyapunov function.

Proof. Necessity: Let the system (4.2) be exponentially stable. We choose a positive deﬁnite quadratic form w(x) = xT W x and show that there exists a positive deﬁnite quadratic form v(x) s.t. (4.3) holds.

Integrating (4.3) from t = 0 to t = T we get

Z T v(x(T, x0)) − v(x(0, x0)) = − w(x(t, x0))dt. 0

Since the system is exponentially stable we have lim x(T ) = 0 and so, lim v(x(T )) = T →∞ T →∞ 0. Thus, Z ∞ Z ∞ T AT t At v(x0) = w(x(t, x0))dt = x0 e W e x0 dt. 0 0 2 −2σt 2 The last integral converges as w(x(t, x0)) ≤ λmax(W )γ e kx0k , t ≥ 0. Furthermore, this integral is a quadratic form in x0 with

∞ Z T V = eA tW eAt dt. 0

T 2 Positive deﬁniteness of V follows from the estimate x (t, x0)W x(t, x0) ≥ λmin(W )kx(t, x0)k , which implies that

Z ∞ Z ∞ T T 2 v(x0) = x (t, x0)W x(t, x0)dt ≥ λmin(W ) kx (t, x0)k dt. 0 0

n Suﬃciency: Let σ > 0 be such that for any x ∈ R 2σv(x) ≤ w(x). In particular, we can choose σ = λmin(W ) . Then we can write 2λmax(V )

d v(x(t)) = −w(x(t)) ≤ −2σv(x(t)), t ≥ 0. dt

This inequality implies that −2σt v(x(t)) ≤ e v(x0), and hence,

2 −2σt −2σt 2 λmin(V )kx(t, x0)k ≤ v(x(t, x0)) ≤ e v(x0) ≤ λmax(V )e kx0k .

The above can be rewritten to yield an exponential estimate on the solution of (4.2):

−σt kx(t, x0)k ≤ γe kx0k, q where γ = λmax(V ) and σ is as deﬁned above. λmin(V )

34 4.2.4 Algebraic Lyapunov matrix equation

Let v(x(t)) = xT (t)V x(t) be a quadratic form. We write explicitly the derivative d dt v(x(t)) taken in virtue of (4.2): d xT V x =x ˙ T V x + xT V x˙ = xT AT V x + xT V Ax = xT (AT V + VA)x. dt Checking condition (4.3) of Theorem 4.2.3 thus boils down to solving the following algebraic Lyapunov matrix equation:

AT V + VA = −W. (4.4)

Theorem 4.2.4. The algebraic Lyapunov matrix equation (4.4) has a unique solution iﬀ the spectrum of the matrix A does not contain an eigenvalue s0 such that −s0 also belongs to the spectrum of A.

Proof. Necessity: Let there exists s0 ∈ Λ(A) such that −s0 ∈ Λ(A). Then there exist two non-zero vectors b and c such that Ab = s0b and Ac = −s0c. This implies that the homogeneous equation AT V + VA = 0 has a non-zero solution given by V = bT c. This implies that the linear matrix equation (4.4) either does not have a solution or the solution is not unique1. Suﬃciency: Consider the homogeneous equation

AT V + VA = 0. (4.5)

We wish to show that the existence of a non-trivial solution to this equation implies that there exists a pair of eigenvalues symmetric w.r.t. origin.

−1 The matrix A can be represented as A = QJAQ , where AJ is the Jordan form of A. T The equation (4.5) takes the form JAV˜ + VJ˜ A, where V˜ = Q VQ. Obviously, V and V˜ are either both trivial or both non-trivial. So, we assume that V˜ 6= 0. Furthermore, letv ˜k be the ﬁrst non-zero column of V˜ , i.e.,v ˜i = 0, i = 1, . . . , k − 1 andv ˜k 6= 0.

Multiplying V˜ and JA with ek, the column vector comprised of zeroes with a single “1” in the kth position, we get:

V˜ ek = vk,JAek = s0ek + δek−1,

where s0 is an eigenvalue of A and δ ∈ {0, 1}. Finally, we have

JAV˜ ek + VJ˜ Aek = JAvk˜ + V˜ (s0ek + δek−1)

= JAv˜k +v ˜ks0 = 0, which implies that −s0 is an eigenvalue along with s0.

1Note that the linear matrix equation (4.4) can always be represented in the regular form A˜v˜ =w ˜ (n2×n2) T with A˜ ∈ R by stacking the columns of A + VA and W . Thus all standard results from linear algebra apply. The reader is urged to convince her-/himself by considering an example

35 We will refer to the condition above (given A, @s0 ∈ C s.t. s0 ∈ Λ(A) and −s0 ∈ Λ(A)) as the Lyapunov condition. Corollary 4.2.5. Let the matrix A satisfy the Lyapunov condition. Then for any symmetric W the unique solution of (4.4) is also a symmetric matrix. Corollary 4.2.6. If the matrix W is positive definite and all eigenvalues of A lie in the open left half plane of C, then the corresponding matrix V is also positive definite. Lemma 4.2.7. Let the system (3.11) be observable (controllable). The matrix A has eigenvalues in the open left half plane of C if and only if the Lyapunov matrix equation AT V + VA = −CT C for the observable case, or AT V + VA = −BBT for the controllable case has a unique positive definite solution V .

4.3 Hurwitz stable polynomials

In this section we will study characteristic polynomials

n n−1 p(s) = det(sI − A) = s + a1s + ... + an−1s