An Introduction to Linear Matrix Inequalities
Total Page:16
File Type:pdf, Size:1020Kb
An Introduction to Linear Matrix Inequalities Raktim Bhattacharya Aerospace Engineering, Texas A&M University Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Linear Matrix Inequalities What are they? Inequalities involving matrix variables Matrix variables appear linearly Represent convex sets polynomial inequalities Critical tool in post-modern control theory AERO 632, Instructor: Raktim Bhattacharya 2 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Standard Form F (x) := F0 + x1F1 + ··· + xnFn > 0 where 0 1 x1 Bx C B 2C m x := B . C ;Fi 2 S m × m symmetric matrix @ . A xn Think of F (x): Rn 7! Sm: Example: 1 x 1 0 0 1 > 0 , + x > 0: x 1 0 1 1 0 AERO 632, Instructor: Raktim Bhattacharya 3 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Positive Definiteness Matrix F > 0 represents positive definite matrix F > 0 () xT F x > 0; 8x =6 0 F > 0 () leading principal minors of F are positive Let 2 3 F11 F12 F13 ··· 6 7 6F21 F22 F23 ··· 7 F = 4 5 F31 F32 F33 ··· ············ n Polynomial Constraints as a Linear Matrix Inequality F11 F12 F13 F11 F12 F > 0 () F11 > 0; > 0; F21 F22 F23 > 0; ··· F21 F22 F31 F32 F33 AERO 632, Instructor: Raktim Bhattacharya 4 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Definiteness Positive Semi-Definite F ≥ 0 () iff all principal minors are ≥ 0 not just leading Negative Definite F < 0 () iff every odd leading principal minor is < 0 and even leading principal minor is > 0 they alternate signs, starting with < 0 Negative Semi-Definite F ≤ 0 () iff every odd principal minor is ≤ 0 and even principal minor is ≥ 0 F > 0 () −F < 0 F ≥ 0 () −F ≤ 0 Matrix Analysis, Roger Horn. AERO 632, Instructor: Raktim Bhattacharya 5 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Example 1 y x y > 0; y − x2 > 0; () > 0 x 1 y x LMI written as > 0 is in general form. x 1 We can write it in standard form as 0 0 1 0 0 1 + y + x > 0 0 1 0 0 1 0 General form saves notations, may lead to more efficient computation AERO 632, Instructor: Raktim Bhattacharya 6 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Example 2 2 3 1 0 x1 2 2 () 4 5 x1 + x2 < 1 0 1 x2 > 0 x1 x2 1 Leading Minors are 1 > 0 1 0 > 0 0 1 1 x2 1 x1 0 1 1 − 0 + x1 > 0 x2 1 x1 1 x1 x2 Last inequality simplifies to − 2 2 1 (x1 + x2) > 0 AERO 632, Instructor: Raktim Bhattacharya 7 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Eigenvalue Minimization n Let Ai 2 S ; i = 0; 1; ··· ; n: Let A(x) := A0 + A1x1 + ··· + Anxn: T Find x := [x1 x2 ··· xn] that minimizes J(x) := min λmaxA(x): x How to solve this problem? AERO 632, Instructor: Raktim Bhattacharya 8 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Eigenvalue Minimization (contd.) Recall for M 2 Sn λmaxM ≤ t () M − tI ≤ 0: Linear algebra result: Matrix Analysis – R.Horn, C.R. Johnson Optimization problem is therefore min t x;t such that A(x) − tI ≤ 0: AERO 632, Instructor: Raktim Bhattacharya 9 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Matrix Norm Minimization n Let Ai 2 R ; i = 0; 1; ··· ; n: Let A(x) := A0 + A1x1 + ··· + Anxn: T Find x := [x1 x2 ··· xn] that minimizes J(x) := min kA(x)k2: x How to solve this problem? AERO 632, Instructor: Raktim Bhattacharya 10 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Matrix Norm Minimization contd. Recall T kAk2 := λmaxA A: Implies min t2 t;x A(x)T A(x) − t2I ≤ 0: or Optimization problem is therefore tI A(x) min t2 subject to ≥ 0: t;x A(x)T tI AERO 632, Instructor: Raktim Bhattacharya 11 / 39 Important Inequalities Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Generalized Square Inequalities Lemma For arbitrary scalar x; y, and δ > 0, we have p y 2 1 δx − p = δx2 + y2 − 2xy ≥ 0: δ δ Implies 1 2xy ≤ δx2 + y2: δ AERO 632, Instructor: Raktim Bhattacharya 13 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Generalized Square Inequalities Restriction-Free Inequalities Lemma Let X; Y 2 Rm×n;F 2 Sm; F > 0, and δ > 0 be a scalar, then XT FY + Y T FX ≤ δXT FX + δ−1Y T FY : When X = x and Y = y 2xT F y ≤ δxT F x + δ−1yT F y: Proof: Using completion of squares. p p T p p δX − δ−1Y F δX − δ−1Y ≥ 0: AERO 632, Instructor: Raktim Bhattacharya 14 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Generalized Square Inequalities Inequalities with Restrictions Let F = fF j F 2 Rn×n;F T F ≤ Ig: Lemma Let X 2 Rm×n;Y 2 Rn×m, then for arbitrary δ > 0 XFY + Y T F T XT ≤ δXXT + δ−1Y T Y ; 8F 2 F: Proof: Approach 1: Using completion of squares. Start with p p T p p δXT − δ−1FY δXT − δ−1FY ) ≥ 0: AERO 632, Instructor: Raktim Bhattacharya 15 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Generalized Square Inequalities Inequalities with Restrictions Let F = fF j F 2 Rn×n;F T F ≤ Ig: Lemma Let X 2 Rm×n;Y 2 Rn×m, then for arbitrary δ > 0 XFY + Y T F T XT ≤ δXXT + δ−1Y T Y ; 8F 2 F: Proof: Approach 2: Use following Lemma. (To do) AERO 632, Instructor: Raktim Bhattacharya 16 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Schur Complements Very useful for identifying convex sets Let Q(x) 2 Sm1 ;R(x) 2 Sm2 Q(x);R(x);S(x) are affine functions of x Q(x) S(x) Q(x) > 0 > 0 () ST (x) R(x) R(x) − ST (x)Q(x)−1S(x) > 0 Generalizing, Q(x) ≥ 0 Q(x) S(x) ≥ 0 () ST (x) I − Q(x)Qy(x) = 0 ST (x) R(x) R(x) − ST (x)Q(x)yS(x) ≥ 0 Q(x)y is the pseudo-inverse This generalization is used when Q(x) is positive semidefinite but singular AERO 632, Instructor: Raktim Bhattacharya 17 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Schur Complement Lemma Let A A A := 11 12 : A21 A22 Define − −1 Sch(A11) := A22 A21A11 A12 − −1 Sch(A22) := A11 A12A22 A21 For symmetric A, A > 0 () A11 > 0;Sch(A11) > 0 () A22 > 0;Sch(A22) > 0 AERO 632, Instructor: Raktim Bhattacharya 18 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Example 1 I x x2 + x2 < 1 () 1 − xT x > 0 () > 0 1 2 xT 1 Here R(x) = 1; Q(x) = I > 0: AERO 632, Instructor: Raktim Bhattacharya 19 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Example 2 P −1 x kxk < 1 () 1 − xT P x > 0 () > 0 P xT 1 or p p p I ( P x) 1 − xT P x = 1 − ( P x)T ( P x) > 0 () p > 0 ( P x)T 1 p where P is matrix square root. AERO 632, Instructor: Raktim Bhattacharya 20 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs LMIs are not unique If F is positive definite then congruence transformation of F is also positive definite F > 0 () xT F x; 8x =6 0 () yT M T F My > 0; 8y =6 0 and nonsingular M () M T F M > 0 Implies, rearrangement of matrix elements does not change the feasible set QS 0 I QS 0 I RST > 0 () > 0 () > 0 ST R I 0 ST R I 0 SQ AERO 632, Instructor: Raktim Bhattacharya 21 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Variable Elimination Lemma Lemma: For arbitrary nonzero vectors x; y 2 Rn, there holds max (xT F y)2 = (xT x)(yT y): F 2F:F T F ≤I Proof: From Schwarz inequality, p p jxT F yj ≤ xT x yT F T F y p p ≤ xT x yT y: Therefore for arbitrary x; y we have (xT F y)2 ≤ (xT x)(yT y): Next show equality. AERO 632, Instructor: Raktim Bhattacharya 22 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Variable Elimination Lemma contd. Let xyT F0 = p p : xT x yT y Therefore, yxT xyT yyT F T F = = : 0 0 (xT x)(yT y) yT y We can show that T T σmax(F0 F0) = σmax(F0F0 ) = 1: ) T ≤ 2 F = F0 F0 1; thus F0 : AERO 632, Instructor: Raktim Bhattacharya 23 / 39 Introduction to LMIs Generalized Square Inequalities Schur Complement Lemma Variable Elimination Lemma Trace of LMIs Variable Elimination Lemma contd.