Applications of Semide nite Programming 1 2 Lieven Vandenb erghe Stephen Boyd August 25, 1998 1 Electrical Engineering Department, University of California, Los Angeles [email protected]. 2 Information Systems Lab oratory, Electrical Engineering Department, Stanford University [email protected]. Abstract A wide variety of nonlinear convex optimization problems can be cast as problems involv- ing linear matrix inequalities LMIs, and hence eciently solved using recently develop ed interior-p oint metho ds. In this pap er, we will consider two classes of optimization problems with LMI constraints: The semide nite programming problem, i.e., the problem of minimizing a linear func- tion sub ject to a linear matrix inequality. Semide nite programming is an imp ortant numerical to ol for analysis and synthesis in systems and control theory. It has also b een recognized in combinatorial optimization as a valuable technique for obtaining b ounds on the solution of NP-hard problems. The problem of maximizing the determinant of a p ositive de nite matrix sub ject to linear matrix inequalities. This problem has applications in computational geometry, exp eriment design, information and communication theory, and other elds. We review some of these applications, including some interesting applications that are less well known and arise in statistics, optimal exp eriment design and VLSI. 1 Optimization problems involving LMI constraints We consider convex optimization problems with linear matrix inequality LMI constraints, i.e., constraints of the form F x= F +x F + + x F 0; 1 0 1 1 m m nn T where the matrices F = F 2 R are given, and the inequality F x 0 means F x is i i m p ositive semide nite. The LMI 1 is a convex constraint in the variable x 2 R . Conversely, a wide variety of nonlinear convex constraints can be expressed as LMIs see the recent surveys by Alizadeh [Ali95], Boyd, El Ghaoui, Feron and Balakrihnan [BEFB94], Lewis and Overton [LO96], Nesterov and Nemirovsky [NN94] and Vandenb erghe and Boyd [VB96]. The purp ose of the pap er is to illustrate the use of linear matrix inequalities with a number of applications from di erent areas. The examples fall in two categories. The rst problem is known as the semide nite programming problem or SDP.InanSDPwe minimize m a linear function of a variable x 2 R sub ject to an LMI: T minimize c x 2 sub ject to F x=F +x F + + x F 0: 0 1 1 m m Semide nite programming can b e regarded as an extension of linear programming where the comp onentwise inequalities b etween vectors are replaced by matrix inequalities, or, equiva- lently, the rst orthant is replaced by the cone of p ositive semide nite matrices. Although the SDP 2 lo oks very sp ecialized, it is much more general than a linear program, and it has many applications in engineering and combinatorial optimization [Ali95, BEFB94, LO96, NN94, VB96]. Most interior-p oint metho ds for linear programming have b een generalized to semide nite programs. As in linear programming, these metho ds have p olynomial worst-case complexity, and p erform very well in practice. The second problem that we will encounter is the problem of maximizing the determinant of a matrix sub ject to LMI constraints. We call this the determinant maximization or maxdet-problem: maximize det Gx sub ject to Gx=G +x G + + x G 0 0 1 1 m m Fx=F +x F + + x F 0: 0 1 1 m m l l T The matrices G = G 2 R are given matrices. The problem is equivalent to minimizing i i 1 the convex function log det Gx sub ject to the LMI constraints. The max-det ob jective arises in applications in computational geometry, control, information theory, and statistics. A uni ed form that includes b oth the SDP and the determinant maximization problem is T 1 minimize c x + log det Gx sub ject to Gx 0 3 F x 0: This problem was studied in detail in Vandenb erghe, Boyd and Wu [VBW98]. 1 2 Ellipsoidal approximation Our rst class of examples are ellipsoidal approximation problems. We can distinguish two basic problems. The rst is the problem of nding the minimum-volume ellipsoid around a given set C . The second problem is the problem of nding the maximum-volume ellip- soid contained in a given convex set C . Both can be formulated as convex semi-in nite programming problems. To solve the rst problem, it is convenient to parametrize the ellipsoid as the pre-image of a unit ball under an ane transformation, i.e., E = fv jkAv + bk1g: T It can be assumed without loss of generality that A = A 0, in which case the volume 1 of E is prop ortional to det A . The problem of computing the minimum-volume ellipsoid containing C can b e written as 1 minimize log det A T sub ject to A = A 0 4 kAv + bk1; 8v 2C; where the variables are A and b. For general C , this is a semi-in nite programming problem. Note that b oth the ob jective function and the constraints are convex in A and b. For the second problem, where we maximize the volume of ellipsoids enclosed in a convex set C , it is more convenient to represent the ellipsoid as the image of the unit ball under an ane transformation, i.e.,as E =fBy + d jkyk 1g: T Again it can b e assumed that B = B 0. The volume is prop ortional to det B ,so we can nd the maximum volume ellipsoid inside C by solving the convex optimization problem maximize log det B T sub ject to B = B 0 5 By + d 2 C 8y; kyk 1 in the variables B and d. For general convex C , this is again a convex semi-in nite opti- mization problem. The ellipsoid of least volume containing a set is often called the Lowner ellipsoid after Danzer, Grun baum, and Klee [DGK63, p.139], or the Lowner-John ellipsoid Grotschel, Lov asz and Schrijver [GLS88, p.69]. John in [Joh85] has shown that if we shrink the n minimum volume outer ellipsoid of a convex set C R by a factor n ab out its center, we obtain an ellipsoid contained in C . Thus the Lowner-John ellipsoid serves as an ellipsoidal approximation of a convex set, with b ounds that dep end only on the ambient dimension, and not in any other way on the set C . 2 Minimum volume ellipsoid containing given p oints The b est known example is the problem of determining the minimum volume ellipsoid that n 1 K contains given p oints x , ..., x in R , i.e., 1 K C = fx ;:::;x g; 1 K or, equivalently, the convex hull Cofx ;:::;x g. This problem has applications in cluster analysis Rosen [Ros65], Barnes [Bar82], and robust statistics in ellipsoidal p eeling metho ds for outlier detection; see Rousseeuw and Leroy [RL87, x7]. Applying 4, we can write this problem as 1 minimize log det A i sub ject to kAx + bk1; i=1;:::;K 6 T A = A 0; nn n T i where the variables are A = A 2 R and b 2 R . The norm constraints kAx + bk1, which are convex quadratic inequalities in the variables A and b, can be expressed as LMIs " i I Ax + b 0; i T Ax + b 1 so 6 is a maxdet-problem in the variables A and b. Maximum volume ellipsoid in p olytop e We assume the set C is a p olytop e describ ed by a set of linear inequalities: T C = fx j a x b ; i =1;:::;Lg: i i To apply 5 we rst work out the last constraint: T By + d 2 C for all ky k 1 a By + d b for all ky k 1 i i T T max a By + a d b i i i ky k1 T kBa k + a d b : i i i This is a convex constraints in B and d, and equivalentto the LMI " T b a dI Ba i i i 0: T T a B b a d i i i We can therefore formulate 5 as a maxdet-problem in the variables B and d: 1 minimize log det B sub ject to B 0 " T dI Ba b a i i i 0; i =1;:::;L: T T b a Ba d i i i 3 Minimum volume ellipsoid containing ellipsoids These techniques extend to several interesting cases where C is not nite or p olyhedral, but is de ned as a combination the sum, union, or intersection of ellipsoids. In particular, it is p ossible to compute the optimal inner approximation of the intersection or the sum of ellipsoids, and the optimal outer approximation of the union or sum of ellipsoids, by solving a maxdet problem. We refer to [BEFB94] and Chernousko [Che94 ] for details. As an example, consider the problem of nding the minimum volume ellipsoid E con- 0 taining K given ellipsoids E ;:::;E . For this problem we describ e the ellipsoids as sublevel 1 K sets of convex quadratic functions: T T E = fx j x A x +2b x+c 0g; i =0;:::;K: i i i i T The solution can b e found by solving the following maxdet-problem in the variables A = A , 0 0 b , and K scalar variables : 0 i 1 minimize log det A 0 T sub ject to A = A 0 0 0 0;:::; 0 1 K 3 3 2 2 7 A b 0 A b 0 i i 0 0 7 7 6 6 T T T 0; i =1;:::;K: b c 0 b 1 b 5 5 4 4 i i i 0 0 0 0 0 0 b A 0 0 1 T c is given by c = b A b 1.