Stability of Dynamic Systems Robert Stengel Optimal Control and Estimation, MAE 546 Princeton University, 2018

• Stability about an equilibrium point • Bounds on the system norm • Lyapunov criteria for stability • Eigenvalues • Transfer functions • Continuous- and discrete- time systems

Dynamic System Stability

• Well over 100 deﬁnitions of stability • Common thread: Response, x(t), is bounded as t –> ∞ • Our principal focus: Initial-condition response of NTI and LTI dynamic systems

2 Vector Norms for Real Variables

• Norm = Measure of length or magnitude of a vector, x

• Euclidean or Quadratic Norm

1/2 1/2 L2 norm = x = xT x = x2 + x2 ++ x2 2 ( ) ( 1 2 n ) • Weighted Euclidean Norm

T 1/2 2 2 2 1/2 y 2 = (y y) = (y1 + y2 +!+ ym )

T T 1/2 xT DT Dx  xT Qx = (x D Dx) = Dx 2 Q DT D Defining matrix  =

Uniform Stability § Autonomous dynamic system § Time-invariant § No forcing input x(t) = f[x(t)]

§ Uniform stability about x = 0

x(t0 ) ≤ δ, δ > 0 Let δ = δ (ε) If, for every ε ≥ 0,

x(t) ≤ ε, ε ≥ δ > 0, t ≥ t0 Then the system is uniformly stable § If system response is bounded, then the system possesses uniform stability 4 Local and Global Asymptotic Stability

• Local asymptotic stability – Uniform stability plus

x(t) ⎯t⎯→∞⎯→0

• Global asymptotic stability System is asymptotically stable for any ε

• If a linear system has uniform asymptotic stability, it also is x(t) = F x(t) globally stable

Exponential Asymptotic Stability

§ Uniform stability about x = 0 plus

x(t) ≤ ke−αt x(0) ; k, α ≥ 0

§ –α = Lyapunov exponent § If norm of x(t) is contained within an exponentially decaying envelope with convergence, system is exponentially asymptotically stable (EAS) § Linear, time-invariant system that is asymptotically stable is EAS

6 Exponential Asymptotic Stability

∞ t ⎛ k ⎞ t ∞ k k e−α dt = − e−α = ∫ ⎜ ⎟ 0 0 ⎝ α ⎠ α Therefore, time integrals of the norm of an EAS system are bounded

∞ ∞ T 1/2 ⎛ k ⎞ ∫ x(t) dt = ∫ ⎣⎡x (t)x(t)⎦⎤ dt ≤ ⎜ ⎟ x(0) 0 0 ⎝ α ⎠ and ∞ 2 ∫ x(t) dt is bounded 0

Exponential Asymptotic Stability

Weighted Euclidean norm and its square are bounded if system is EAS

∞ ∞ T T 1/2 ∫ Dx(t) dt = ∫ ⎣⎡x (t)D Dx(t)⎦⎤ dt is bounded 0 0

with 0 < DT D ! Q < ∞ ∞ T ∫ ⎣⎡x (t)Qx(t)⎦⎤dt is bounded 0 Conversely, if the weighted Euclidean norm is bounded, the LTI system is EAS 8 Initial-Condition Response of an EAS Linear System

x(t) = Φ(t,0)x(0) = eF(t)x(0) 2 x(t) = xT (0)ΦT (t,0)Φ(t,0)x(0) is bounded

• To be shown – Continuous-time LTI system is stable if all eigenvalues of F have negative real parts – Discrete-time LTI system is stable if all eigenvalues of Φ lie within the unit circle

Lyapunovs First Theorem

• A nonlinear system is asymptotically stable at the origin if its linear approximation is stable at the origin, i.e., – for all trajectories that start close enough (in the neighborhood) – within a stable manifold (closed boundary)

x(t) = f[x(t)] is stable at xo = 0 if ∂f[x(t)] Δx(t) = Δx(t) is stable ∂x xo =0 At the origin is a fuzzy concept

10 Lyapunov Function*

Deﬁne a scalar Lyapunov function, a positive deﬁnite function of the state in the region of interest

V ⎣⎡x(t)⎦⎤ ≥ 0 Examples mV 2 E E V 2 V = E = + mgh; = = + h 2 mg weight 2g 1 1 V = xT x; V = xT Px 2 2

T 2 V = ⎣⎡x (t )Px(t )⎦⎤

* Who was Lyapunov? see http://en.wikipedia.org/wiki/Aleksandr_Lyapunov

V ⎣⎡x(t)⎦⎤ ≥ 0 Lyapunovs V[0] = 0 for t ≥ 0 Second Theorem Evaluate the time derivative of the Lyapunov function dV ∂V ∂V ∂V ∂V = + x! = + f ⎡x(t)⎤ dt ∂t ∂x ∂t ∂x ⎣ ⎦ ∂V = f ⎣⎡x(t)⎦⎤ for autonomous systems ∂x dV • If < 0 in the neighborhood of the origin, dt the origin is asymptotically stable • Sufﬁcient but not necessary condition for stability • Lyapunov function is not unique 12 Quadratic Lyapunov Function for LTI System Lyapunov function Linear, Time-Invariant System

T V ⎣⎡x(t)⎦⎤ = x (t)Px(t) x(t) = F x(t) Rate of change for quadratic Lyapunov function dV ∂V = x! = xT (t)Px! (t) + x! T (t)Px(t) dt ∂x = xT (t) PF + FT P x(t) " −xT (t)Qx(t) ( )

Lyapunov Equation

The LTI system is stable if the Lyapunov equation is satisﬁed with positive-deﬁnite P and Q

PF + FT P = −Q with P > 0, Q > 0

14 Lyapunov Stability: 1st-Order Example

Δx(t) = aΔx(t) , Δx(0) given 1st-order initial- condition response

F = a, P = p, Q = q Unstable, a > 0

t t Δx(t) = ∫ Δx!(t)dt = ∫ aΔx(t)dt Stable, a < 0 0 0 at = e Δx(0)

PF + FT P = −Q PF + FT P = −Q with p > 0, a < 0 with p > 0, a > 0 2 pa < 0 and q > 0 2 pa < 0 and q < 0 ∴ system is stable ∴ system is unstable 15

Lyapunov Stability and the HJB Equation

T V ⎣⎡x(t)⎦⎤ = x (t)Px(t)

Lyapunov stability Dynamic programming optimality dV ∂V * < 0 = − min H dt ∂t u(t )

Rantzer, Sys.Con. Let., 2001 16 Lyapunov Stability for Nonautonomous (Time-Varying) Systems

x!(t) = f[x(t),t]; f[0,t] = 0 for t ≥ 0 Time-varying Lyapunov function bounded above and below by time-invariant Lyapunov functions

V ⎣⎡x(t),t ⎦⎤ ≥ Va ⎣⎡x(t)⎦⎤ > 0 in neighborhood of x(0) = 0 For asymptotic stability of time-varying system

V ⎣⎡x(t),t ⎦⎤ ≤ Vb ⎣⎡x(t)⎦⎤ > 0 for large values of x(t) Time-derivative of Lyapunov function must be negative-deﬁnite

dV ⎣⎡x(t ),t ⎦⎤ ⎪⎧∂V ⎣⎡x(t ),t ⎦⎤ ⎪⎫ ⎪⎧∂V ⎣⎡x(t ),t ⎦⎤ ⎪⎫ = ⎨ ⎬ + ⎨ f ⎣⎡x(t )⎦⎤⎬ < 0 in neighborhood of x = 0 dt ⎩⎪ ∂t ⎭⎪ ⎩⎪ ∂x ⎭⎪

Laplace Transforms and Linear System Stability

18 Fourier Transform of a Scalar Variable ∞ ω = frequency, rad / s F [x(t)] = x( jω ) = ∫ x(t)e− jωt dt −∞ j ! i ! −1

x(t) x(t) : real variable x( jω ) : complex variable = a(ω ) + jb(ω ) = A(ω )e jϕ (ω )

x( jω) = a(ω) + jb(ω) A : amplitude ϕ : phase angle

Laplace Transforms of Scalar Variables Laplace transform of a scalar variable is a complex number s is the Laplace operator, a complex variable

∞ L[x(t)] = x(s) = ∫ x(t)e−st dt, s = σ + jω 0 Laplace transformation is a linear operation

Multiplication by a constant x(t) : real variable x(s) : complex variable L a x(t) = a x(s) [ ] = a(ω ) + jb(ω ) jϕ (ω ) Sum of Laplace transforms = A(ω )e

L[x1(t) + x2 (t)] = x1(s) + x2 (s) 20 Laplace Transforms of Vectors and Matrices

Laplace transform of a Laplace transform of a vector variable matrix variable

⎡ ⎤ x1(s) ⎡ a (s) a (s) ... ⎤ ⎢ ⎥ ⎢ 11 12 ⎥ L[x(t)] = x(s) = ⎢ x2 (s) ⎥ L[A(t)] = A(s) = ⎢ a21(s) a22 (s) ... ⎥ ⎢ ... ⎥ ⎢ ...... ⎥ ⎣ ⎦ ⎣ ⎦ Laplace transform of a derivative w.r.t. time ⎡dx(t)⎤ L ⎢ ⎥ = sx(s) − x(0) ⎣ dt ⎦ Laplace transform of an integral over time ⎡ ⎤ x(s) L x(τ )dτ = ⎢∫ ⎥ s ⎣ ⎦ 21

Transformation of the LTI System Equations Time-Domain System Equations x!(t) = Fx(t) + Gu(t) Dynamic Equation y(t) = H x(t) + H u(t) x u Output Equation

Laplace Transforms of System Equations

sx(s) − x(0) = Fx(s) + Gu(s) Dynamic Equation

y(s) = Hxx(s) + Huu(s) Output Equation

22 Laplace Transform of State Vector Response to Initial Condition and Control Rearrange Laplace Transform of Dynamic Equation sx(s) − Fx(s) = x(0) + Gu(s) [sI − F]x(s) = x(0) + Gu(s) 1 x(s) = [sI − F]− [x(0) + Gu(s)] The matrix inverse is

−1 Adj(sI − F) [sI − F] = (n x n) sI − F

Adj(sI − F) : Adjoint matrix (n × n) Transpose of matrix of cofactors sI − F = det(sI − F) : Determinant (1×1) 23

Characteristic Polynomial of a Dynamic System Matrix Inverse

−1 Adj(sI − F) [sI − F] = (n x n) sI − F Characteristic matrix of the system ⎛ ⎞ (s − f11 ) − f12 ... − f1n ⎜ ⎟ ⎜ − f21 (s − f22 ) ... − f2n ⎟ (sI − F) = ⎜ ⎟ (n x n) ⎜ ...... ⎟ ⎜ f f ... s f ⎟ ⎝ − n1 − n2 ( − nn ) ⎠ Characteristic polynomial of the system sI − F = det(sI − F) n n−1 24 ≡ Δ(s) = s + an−1s + ...+ a1s + a0 Eigenvalues

Eigenvalues of the LTI System Characteristic equation of the system

n n−1 Δ(s) = s + an−1s + ...+ a1s + a0 = 0

= (s − λ1 )(s − λ2 )(...)(s − λn ) = 0

Eigenvalues, λi , are solutions (roots) of the polynomial, Δ(s) = 0

λi = σ i + jωi

* λ i = σ i − jωi

s Plane 26 Factors of a 2nd-Degree Characteristic Equation

(s − f11 ) − f12 sI − F = ! Δ(s) − f21 (s − f22 )

2 = s − ( f11 + f22 )s + ( f11 f22 + f12 f21 )

= (s − λ1 )(s − λ2 ) = 0 [real or complex roots] δ = cos−1 ζ = s2 + 2ζω s +ω 2 with complex-conjugate roots n n

λ1 = σ1, λ2 = σ 2

λ1 = σ 1 + jω1

λ2 = σ 1 − jω1

s Plane ω n : natural frequency, rad/s ζ : damping ratio, dimensionless

z Transforms and Discrete-Time Systems

28 Application of Dirac Delta Function to Sampling Process § Periodic sequence of numbers

Δxk = Δx(tk ) = Δx(kΔt)

§ Dirac delta function

⎧ ⎪ ∞, (t0 − kΔt) = 0 δ (t0 − kΔt)=⎨ 0, t − kΔt ≠ 0 ⎩⎪ ( 0 )

(t0 − kΔt)+ε δ (t0 − kΔt)dt = 1 ∫(t0 − kΔt)−ε § Periodic sequence of scaled delta functions Δx kΔt δ t − kΔt ( ) ( 0 ) 29

Laplace Transform of a Periodic Scalar Sequence

§ Periodic sequence of numbers Δxk = Δx(tk ) = Δx(kΔt)

§ Periodic sequence of scaled delta Δx kΔt δ t − kΔt functions ( ) ( )

§ Laplace transform of the delta function sequence

∞ L [Δx(kΔt)δ (t − kΔt)] = Δx(z) = ∫ Δx(kΔt)δ (t − kΔt)e−sΔtdt 0 ∞ ∞ = ∫ Δx(kΔt)e−skΔt dt ! ∑Δx(kΔt)z−k 0 k=0

30 z Transform of the Periodic Sequence z transform is the Laplace transform of the delta function sequence

∞ ∞ L [Δx(kΔt)δ (t − kΔt)] = ∑Δx(kΔt)e−skΔt ! ∑Δx(kΔt)z−k k=0 k=0

z Transform (time-shift) Operator z ! esΔt [advance by one sampling interval] z−1 e−sΔt delay by one sampling interval ! [ ]

z Transform of a Discrete-Time Dynamic System

System equation in sampled time domain

Δxk +1 = Φ Δxk + Γ Δuk + ΛΔwk

Laplace transform of sampled-data system equation (z Transform) zΔx(z) − Δx(0) = Φ Δx(z) + ΓΔ u(z) + ΛΔw(z)

32 z Transform of a Discrete-Time Dynamic System Rearrange zΔx(z) − Φ Δx(z) = Δx(0) + ΓΔ u(z) + ΛΔw(z)

Collect terms (zI − Φ)Δx(z) = Δx(0) + ΓΔ u(z) + ΛΔw(z)

Pre-multiply by inverse Δx(z) = (zI − Φ) −1[Δx(0) + ΓΔ u(z) + ΛΔw(z)]

Characteristic Matrix and Determinant of Discrete-Time System

Δx(z) = (zI − Φ) −1[Δx(0) + ΓΔ u(z) + ΛΔw(z)] Inverse matrix

−1 Adj(zI − Φ) (zI − Φ) = (n x n) zI − Φ

Characteristic polynomial of the discrete-time model zI − Φ = det(zI − Φ) ≡ Δ(z) n n−1 = z + an−1z + ...+ a1z + a0

34 Eigenvalues (or Roots) of the LTI Discrete-Time System Characteristic equation of the system

n n−1 Δ(z) = z + an−1z + ...+ a1z + a0

= (z − λ1 )(z − λ2 )(...)(z − λn ) = 0

Eigenvalues, λi , of the state transition matrix, Φ, are solutions (roots) of the polynomial, Δ(z) = 0 Eigenvalues are complex numbers that can be plotted in the z plane

* λi = σ i + jωi λi = σ i − jωi z Plane

Laplace Transforms of Continuous- and Discrete-Time State-Space Models Initial condition and disturbance effect neglected

Δx(s) = (sI − F) −1GΔ u(s) Δy(s) = H(sI − F) −1GΔ u(s) Equivalent discrete-time model

Δx(z) = (zI − Φ) −1ΓΔ u(z) Δy(z) = H(zI − Φ) −1ΓΔ u(z) 36 Scalar Transfer Functions of Continuous- and Discrete-Time Systems

dim(H) = 1× n dim(G) = n × 1

Δy(s) −1 HAdj(sI − F)G = H(sI − F) G = = Y (s) Δu(s) sI − F

Δy(z) −1 HAdj(sI − Φ)Γ = H(zI − Φ) Γ = = Y (z) Δu(z) sI − Φ

Comparison of s-Plane and z- Plane Plots of Poles and Zeros

Δy(s) −1 HAdj(sI − F)G Δy(z) −1 HAdj(sI − Φ)Γ = H(sI − F) G = = Y (s) = H(zI − Φ) Γ = = Y (z) Δu(s) sI − F Δu(z) sI − Φ § s-Plane Plot of Poles and Zeros § z-Plane Plot of Poles and Zeros § Poles in left-half-plane are stable § Poles within unit circle are stable § Zeros in left-half-plane are § Zeros within unit circle are minimum phase minimum phase

Note correspondence of Increasing sampling rate conﬁgurations as sampling rate moves poles and zeros 38 increases toward the (1,0) point Next Time: Time-Invariant Linear- Quadratic Regulators

Supplementary Material

40 Second-Order Oscillator

Differential Equations for 2nd-Order System ⎡ ⎤ ⎡ ⎤ Δx!1(t) ⎡ 0 1 ⎤ Δx1(t) ⎡ 0 ⎤ ⎢ ⎥ = ⎢ ⎥⎢ ⎥ + ⎢ ⎥Δu(t) Dynamic Equation Δx! (t) −ω 2 −2ζω Δx (t) ω 2 ⎣⎢ 2 ⎦⎥ ⎣⎢ n n ⎦⎥⎣⎢ 2 ⎦⎥ ⎣⎢ n ⎦⎥ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ Δy1(t) ⎡ 1 0 ⎤ Δx1(t) ⎡ 0 ⎤ Δx1(t) ⎢ ⎥ = ⎢ ⎥⎢ ⎥ + ⎢ ⎥Δu(t) = ⎢ ⎥ Output Equation ⎢ Δy2 (t) ⎥ ⎣ 0 1 ⎦⎢ Δx2 (t) ⎥ ⎣ 0 ⎦ ⎢ Δx2 (t) ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ Laplace Transforms of 2nd-Order System

⎡ ⎤ ⎡ ⎤ sΔx1(s) − Δx1(0) ⎡ 0 1 ⎤ Δx1(s) ⎡ 0 ⎤ ⎢ ⎥ = ⎢ 2 ⎥⎢ ⎥ + ⎢ 2 ⎥Δu(s) Dynamic Equation sΔx (s) − Δx (0) −ω −2ζω Δx (s) ω ⎣⎢ 2 2 ⎦⎥ ⎣⎢ n n ⎦⎥⎣⎢ 2 ⎦⎥ ⎣⎢ n ⎦⎥ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ Δy1(s) ⎡ 1 0 ⎤ Δx1(s) ⎡ 0 ⎤ Δx1(s) ⎢ ⎥ = ⎢ ⎥⎢ ⎥ + ⎢ ⎥Δu(s) = ⎢ ⎥ Δy (s) 0 1 Δx (s) 0 Δx (s) Output Equation ⎣⎢ 2 ⎦⎥ ⎣ ⎦⎣⎢ 2 ⎦⎥ ⎣ ⎦ ⎣⎢ 2 ⎦⎥

Small Perturbations from Steady, Level Flight Δx˙ (t) = FΔx(t) + GΔu(t) + LΔw(t)

⎡ ⎤ Δx1 ⎡ V ⎤ velocity, m/s ⎢ ⎥ Δ x ⎢ ⎥ ⎢ Δ 2 ⎥ ⎢ Δγ ⎥ flight path angle, rad Δx(t) = ⎢ ⎥ = Δx3 ⎢ Δq ⎥ pitch rate, rad/s ⎢ ⎥ ⎢ ⎥ ⎢ Δx ⎥ Δα angle of attack, rad ⎣ 4 ⎦ ⎣⎢ ⎦⎥

⎡ ⎤ Δu1 ⎡ ΔδE ⎤ elevator angle, rad Δu(t) = ⎢ ⎥ = ⎢ ⎥ Δu ΔδT throttle setting, % ⎣⎢ 2 ⎦⎥ ⎣ ⎦

⎡ Δw ⎤ ⎡ ΔV ⎤ ~horizontal wind, m/s Δw(t) = ⎢ 1 ⎥ = ⎢ w ⎥ Δw Δα ~vertical wind/V , rad ⎣⎢ 2 ⎦⎥ ⎣⎢ w ⎦⎥ nom

42 Eigenvalues of Aircraft Longitudinal Modes of Motion

sI − F = det(sI − F) ≡ Δ(s) = (s − λ1 )(s − λ2 )(s − λ3 )(s − λ4 ) * * = (s − λP )(s − λ P )(s − λSP )(s − λ SP ) = s2 + 2ζ ω s +ω 2 s2 + 2ζ ω s +ω 2 = 0 ( P nP nP )( SP nSP nSP )

Eigenvalues determine the damping and natural frequencies of the linear systems modes of motion

ζ ,ω :phugoid (long-period) mode ( P nP ) ζ ,ω :short-period mode ( SP nSP ) 43

Initial-Condition ⎡ ⎤ Δx1 ⎡ V ⎤ velocity, m/s ⎢ ⎥ Δ x ⎢ ⎥ ⎢ Δ 2 ⎥ ⎢ Δγ ⎥ flight path angle, rad Response of Business Δx(t) = ⎢ ⎥ = Δx3 ⎢ Δq ⎥ pitch rate, rad/s ⎢ ⎥ ⎢ ⎥ ⎢ Δx ⎥ Δα angle of attack, rad Jet at Two Time Scales ⎣ 4 ⎦ ⎣⎢ ⎦⎥ Same 4th-order responses viewed over different periods of time • 0 - 100 sec • 0 - 6 sec • Reveals Long-Period Mode • Reveals Short-Period Mode