Applications of Semide nite Programming

1 2

Lieven Vandenb erghe Stephen Boyd

August 25, 1998

1

Electrical Engineering Department, University of California, Los Angeles

[email protected].

2

Information Systems Lab oratory, Electrical Engineering Department, Stanford University

[email protected].

Abstract

A wide variety of nonlinear convex optimization problems can be cast as problems involv-

ing linear matrix inequalities LMIs, and hence eciently solved using recently develop ed

interior-p oint metho ds. In this pap er, we will consider two classes of optimization problems

with LMI constraints:

 The semide nite programming problem, i.e., the problem of minimizing a linear func-

tion sub ject to a linear matrix inequality. Semide nite programming is an imp ortant

numerical to ol for analysis and synthesis in systems and control theory. It has also

b een recognized in combinatorial optimization as a valuable technique for obtaining

b ounds on the solution of NP-hard problems.

 The problem of maximizing the determinant of a p ositive de nite matrix sub ject to

linear matrix inequalities. This problem has applications in computational geometry,

exp eriment design, information and communication theory, and other elds.

We review some of these applications, including some interesting applications that are less

well known and arise in statistics, optimal exp eriment design and VLSI.

1 Optimization problems involving LMI constraints

We consider convex optimization problems with linear matrix inequality LMI constraints,

i.e., constraints of the form

F x= F +x F +


+ x F  0; 1

0 1 1 m m

nn

T

where the matrices F = F 2 R are given, and the inequality F x  0 means F x is

i

i

m

p ositive semide nite. The LMI 1 is a convex constraint in the variable x 2 R . Conversely,

a wide variety of nonlinear convex constraints can be expressed as LMIs see the recent

surveys by Alizadeh [Ali95], Boyd, El Ghaoui, Feron and Balakrihnan [BEFB94], Lewis and

Overton [LO96], Nesterov and Nemirovsky [NN94] and Vandenb erghe and Boyd [VB96].

The purp ose of the pap er is to illustrate the use of linear matrix inequalities with a

number of applications from di erent areas. The examples fall in two categories. The rst

problem is known as the semide nite programming problem or SDP.InanSDPwe minimize

m

a linear function of a variable x 2 R sub ject to an LMI:

T

minimize c x

2

sub ject to F x=F +x F +


+ x F  0:

0 1 1 m m

Semide nite programming can b e regarded as an extension of linear programming where the

comp onentwise inequalities b etween vectors are replaced by matrix inequalities, or, equiva-

lently, the rst orthant is replaced by the cone of p ositive semide nite matrices. Although

the SDP 2 lo oks very sp ecialized, it is much more general than a linear program, and it

has many applications in engineering and combinatorial optimization [Ali95, BEFB94, LO96,

NN94, VB96]. Most interior-p oint metho ds for linear programming have b een generalized to

semide nite programs. As in linear programming, these metho ds have p olynomial worst-case

complexity, and p erform very well in practice.

The second problem that we will encounter is the problem of maximizing the determinant

of a matrix sub ject to LMI constraints. We call this the determinant maximization or

maxdet-problem:

maximize det Gx

sub ject to Gx=G +x G +


+ x G 0

0 1 1 m m

Fx=F +x F +


+ x F  0:

0 1 1 m m

l l

T

The matrices G = G 2 R are given matrices. The problem is equivalent to minimizing

i

i

1

the convex function log det Gx sub ject to the LMI constraints. The max-det ob jective

arises in applications in computational geometry, control, information theory, and statistics.

A uni ed form that includes b oth the SDP and the determinant maximization problem

is

T 1

minimize c x + log det Gx

sub ject to Gx 0

3

F x  0:

This problem was studied in detail in Vandenb erghe, Boyd and Wu [VBW98]. 1

2 Ellipsoidal approximation

Our rst class of examples are ellipsoidal approximation problems. We can distinguish two

basic problems. The rst is the problem of nding the minimum-volume ellipsoid around

a given set C . The second problem is the problem of nding the maximum-volume ellip-

soid contained in a given convex set C . Both can be formulated as convex semi-in nite

programming problems.

To solve the rst problem, it is convenient to parametrize the ellipsoid as the pre-image

of a unit ball under an ane transformation, i.e.,

E = fv jkAv + bk1g:

T

It can be assumed without loss of generality that A = A 0, in which case the volume

1

of E is prop ortional to det A . The problem of computing the minimum-volume ellipsoid

containing C can b e written as

1

minimize log det A

T

sub ject to A = A 0

4

kAv + bk1; 8v 2C;

where the variables are A and b. For general C , this is a semi-in nite programming problem.

Note that b oth the ob jective function and the constraints are convex in A and b.

For the second problem, where we maximize the volume of ellipsoids enclosed in a convex

set C , it is more convenient to represent the ellipsoid as the image of the unit ball under an

ane transformation, i.e.,as

E =fBy + d jkyk 1g:

T

Again it can b e assumed that B = B 0. The volume is prop ortional to det B ,so we can

nd the maximum volume ellipsoid inside C by solving the convex optimization problem

maximize log det B

T

sub ject to B = B 0

5

By + d 2 C 8y; kyk 1

in the variables B and d. For general convex C , this is again a convex semi-in nite opti-

mization problem.

The ellipsoid of least volume containing a set is often called the Lowner ellipsoid after

Danzer, Grun  baum, and Klee [DGK63, p.139], or the Lowner-John ellipsoid Grotschel,

Lov asz and Schrijver [GLS88, p.69]. John in [Joh85] has shown that if we shrink the

n

minimum volume outer ellipsoid of a convex set C  R by a factor n ab out its center, we

obtain an ellipsoid contained in C . Thus the Lowner-John ellipsoid serves as an ellipsoidal

approximation of a convex set, with b ounds that dep end only on the ambient dimension,

and not in any other way on the set C . 2

Minimum volume ellipsoid containing given p oints

The b est known example is the problem of determining the minimum volume ellipsoid that

n

1 K

contains given p oints x , ..., x in R , i.e.,

1 K

C = fx ;:::;x g;

1 K

or, equivalently, the convex hull Cofx ;:::;x g. This problem has applications in cluster

analysis Rosen [Ros65], Barnes [Bar82], and robust statistics in ellipsoidal p eeling metho ds

for outlier detection; see Rousseeuw and Leroy [RL87, x7].

Applying 4, we can write this problem as

1

minimize log det A

i

sub ject to kAx + bk1; i=1;:::;K

6

T

A = A 0;

nn n

T i

where the variables are A = A 2 R and b 2 R . The norm constraints kAx + bk1,

which are convex quadratic inequalities in the variables A and b, can be expressed as LMIs

 "

i

I Ax + b

 0;

i T

Ax + b 1

so 6 is a maxdet-problem in the variables A and b.

Maximum volume ellipsoid in p olytop e

We assume the set C is a p olytop e describ ed by a set of linear inequalities:

T

C = fx j a x  b ; i =1;:::;Lg:

i

i

To apply 5 we rst work out the last constraint:

T

By + d 2 C for all ky k 1  a By + d  b for all ky k 1

i

i

T T

 max a By + a d  b

i

i i

ky k1

T

 kBa k + a d  b :

i i

i

This is a convex constraints in B and d, and equivalentto the LMI

" 

T

b a dI Ba

i i

i

 0:

T T

a B b a d

i

i i

We can therefore formulate 5 as a maxdet-problem in the variables B and d:

1

minimize log det B

sub ject to B 0

 "

T

dI Ba b a

i i

i

 0; i =1;:::;L:

T T

b a Ba  d

i i

i 3

Minimum volume ellipsoid containing ellipsoids

These techniques extend to several interesting cases where C is not nite or p olyhedral, but

is de ned as a combination the sum, union, or intersection of ellipsoids. In particular, it

is p ossible to compute the optimal inner approximation of the intersection or the sum of

ellipsoids, and the optimal outer approximation of the union or sum of ellipsoids, by solving

a maxdet problem. We refer to [BEFB94] and Chernousko [Che94 ] for details.

As an example, consider the problem of nding the minimum volume ellipsoid E con-

0

taining K given ellipsoids E ;:::;E . For this problem we describ e the ellipsoids as sublevel

1 K

sets of convex quadratic functions:

T T

E = fx j x A x +2b x+c  0g; i =0;:::;K:

i i i

i

T

The solution can b e found by solving the following maxdet-problem in the variables A = A ,

0

0

b , and K scalar variables  :

0 i

1

minimize log det A

0

T

sub ject to A = A 0

0

0

  0;:::;  0

1 K

3 3 2 2

7

A b 0 A b 0

i i 0 0

7 7 6 6

T T T

 0; i =1;:::;K:  b c 0 b 1 b

5 5 4 4

i i

i 0 0

0 0 0 0 b A

0 0

1

T

c is given by c = b A b 1. See [BEFB94, p.43] for details.

0 0 0

0

0

3 Wire and transistor sizing

We consider linear resistor-capacitor RC circuits describ ed by the di erential equation

dv

C = Gv t ut; 8

dt

n n

where v t 2 R is the vector of no de voltages, ut 2 R is the vector of indep endentvoltage

nn nn

sources, C 2 R is the capacitance matrix, and G 2 R is the conductance matrix. We

assume that C and G are symmetric and p ositive de nite i.e., that the capacitive and

resistive sub circuits are recipro cal and strictly passive. We are interested in problems in

m

which C and G dep end on some design parameters x 2 R . Sp eci cally we assume that the

matrices C and G are ane functions of x, i.e.,

C x=C +x C +


+ x C ; Gx=G +x G +


+ x G ; 9

0 1 1 m m 0 1 1 m m

where C and G are symmetric matrices.

i i

Linear RC mo dels of the form 8 are often used as approximate mo dels for transistors

and interconnect wires in VLSI circuits. When the design parameters are the physical widths

of conductors or transistors, the conductance and capacitance matrices are ane in these

parameters, i.e., they have the form 9. An imp ortant example is wire sizing, where x

i 4

denotes the width of a segment of some conductor or interconnect line. A simple lump ed

mo del of the segment consists of a  section: a series conductance, with a capacitance to

ground on each end. Here the conductance is linear in the width x , and the capacitances

i

are linear or ane. We can also mo del each segment by many such  sections, and still

have the general form 8, 9. Another imp ortant example is an MOS transistor circuit

where x denotes the width of a transistor. When the transistor is `on' it is mo deled as a

i

conductance that is prop ortional to x , and a source-to-ground capacitance and drain-to-

i

ground capacitance that are linear or ane in x . See [VBE96] for details.

i

Signal propagation delay

We are interested in how fast a change in the input u propagates to the di erent no des of

the circuit, and in how this propagation delay varies as a function of the variables x.

We assume that for t<0, the circuit is in static steady-state with ut=vt=v . For

t  0, the source switches to the constant value ut= v . As a result we have, for t  0,

+

1

C Gt

v t=v +e v v  10

+ +

which converges, as t ! 1, to v since our assumption C 0, G 0 implies stability.

+

1

C Gt

The di erence b etween the no de voltage and its ultimate valuev ~t=e v v , and

+

we are interested in how large t must be b efore this is small. To simplify notation, we will

relab el v~ as v , and from here on study the rate at which

1

C Gt

v t= e v 0 11

b ecomes small. Note that this v satis es the autonomous equation C dv =dt = Gv .

1

Let  ;:::; denote the eigenvalues of the circuit, i.e., the eigenvalues of C G, or

1 n

equivalently, the ro ots of the characteristic p olynomial detsC + G. They are real and neg-

1=2 1=2

ative since the matrix is similar to the symmetric, negative de nite matrix C GC .

We assume the eigenvalues are sorted in decreasing order, i.e., 0 >  


  . The

1 n

largest eigenvalue,  , is called the dominant eigenvalue or dominant pole of the RC circuit.

1

The critical dominant time constant is de ned as

dom

T = 1= : 12

1

dom

Note that the dominant time constant T is a very complicated function of G and C ,

i.e., the negative inverse of the largest zero of the p olynomial det sC + G. However it can

be expressed in terms of an LMI, since

dom

T = minf T j TG C  0 g:

In particular,

dom

T x  T  T Gx C x  0: 13

max max

dom

is a quasiconvex function of x, i.e., its sublevel sets We can conclude that T

o n

dom

x T x  T

max 5

are convex sets for all values of T .

max

As a consequence, various optimization problems involving dominant time constant, area,

and power can be cast as convex optimization problems. Therefore we can compute exact,

optimal tradeo curves between these quantities. We discuss this in more detail now. For

more extensive discussion and examples, we refer to [VBE96, VBE97 ], where this approach

to delay optimization is explored in more detail, and compared with conventional techniques

based on geometric programming.

Minimum area sub ject to b ound on delay

Supp ose the area of the circuit is a linear or ane function of the variables x . This o ccurs

i

when the variables represent the widths of transistors or conductors with lengths xed as

l , in which case the circuit area has the form

i

a + x l +


+ x l

0 1 1 m m

where a is the area of the xed part of the circuit.

0

dom

We can minimize the area sub ject to a b ound on the dominant time constant T  T ,

max

and sub ject to upp er and lower b ounds on the widths by solving the SDP

m

X

minimize l x

i i

i=1

14

sub ject to T Gx C x  0

max

x  x  x ; i =1;:::;m:

min i max

By solving this SDP for a sequence of values of T , we can compute the exact optimal

max

tradeo b etween area and dominant time constant. The optimal solutions of 14 are on the

tradeo curve, i.e., they are Pareto optimal for area and dominant time constant.

Minimum power dissipation sub ject to b ound on delay

The total energy dissipated in the resistors during a transition from initial voltage v to nal

T

voltage 0 or b etween 0 and v  is the energy stored in the capacitors, i.e.,1=2v C v . There-

fore for a xed clo ck rate and xed probability of transition, the average power dissipated

in prop ortional to

m

X

T T

v C xv = x v C v ;

k i

i=1

which is a linear function of the design parameters x.

Therefore we can minimize power dissipation sub ject to a constraint on the dominant

time constant by solving the SDP

T

v C xv minimize

sub ject to T Gx C x  0

max

x  x  x ; i =1;:::;m:

min i max

We can also add an upp er b ound on area, which is a linear inequality. By solving this

SDP for a sequence of values of T , we can compute the optimal tradeo between power

max 6

dissipation and dominant time constant. By adding a constraint that the area cannot exceed

A , and solving the SDP for a sequence of values of T and A , we can compute the

max max max

exact optimal tradeo surface b etween p ower dissipation, area, and dominant time constant.

Minimum delay sub ject to area and power constraints

We can also directly minimize the delay sub ject to limits on area and p ower dissipation, by

solving the quasiconvex optimization problem

minimize T

sub ject to TGx Cx  0

x  x  x ; i =1;:::;m

min i max

T

f x  g ; i =1;2

i

i

with variables x and T , where the linear inequalities limit area and power dissipation.

4 Exp eriment design

As a third group of examples, we consider some problems in optimal exp eriment design.

we consider the problem of estimating a vector x from a measurement y = Ax + w , where

w N0;I is measurement noise. The error covariance of the minimum-variance estimator

T

y y T T 1

is equal to A A  = A A . We supp ose that the rows of the matrix A = [a ::: a ]

1 q

p

i

can be chosen among M p ossible test vectors v 2 R , i =1;:::;M:

1 M

a 2fv ;:::;v g; i =1;:::;q:

i

T 1

The goal of exp eriment design is to cho ose the vectors a so that the error covariance A A

i

is `small'. We can interpret each comp onentofy as the result of an exp eriment or measure-

ment that can be chosen from a xed menu of p ossible exp eriments; our job is to nd a set

of measurements that together are maximally informative.

P

T

M

T i i

We can write A A = q  v v , where  is the fraction of rows a equal to

i i k

i=1

i

the vector v . We ignore the fact that the numb ers  are integer multiples of 1=q , and

i

instead treat them as continuous variables, which is justi ed in practice when q is large.

Alternatively,we can imagine that we are designing a random exp eriment: each exp eriment

k 

a has the form v with probability  .

i k

T 1

Many di erent criteria for measuring the size of the matrix A A have b een pro-

p osed. For example, in E -optimal design, we minimize the norm of the error covariance,

T 1 T

 A A , which is equivalent to maximizing the smallest eigenvalue of A A. This is

max

readily cast as the SDP

maximize t

M

X

T

i i

sub ject to  v v  tI

i

i=1

M

X

 =1

i

i=1

  0; i =1;:::;M

i 7

in the variables  ;:::; , and t. Another criterion is A-optimality,in which we minimize

1 M

T 1

TrA A . This can be cast as an SDP:

p

X

minimize t

i

i=1

3 2

P

T

M

i i

 v v e

i i

i=1

5 4

 0; i =1;:::;p; sub ject to

T

e t

i

i

  0; i =1;:::;M;

i

M

X

 =1;

i

i=1

p

where e is the ith unit vector in R , and the variables are  , i = 1;:::;M, and t , i =

i i i

1;:::;p.

T 1

In D -optimal design, we minimize the determinant of the error covariance A A , which

leads to the maxdet-problem

 !

1

M

X

T

i i

minimize log det  v v

i

i=1

15

sub ject to   0; i =1;:::;M

i

M

X

 =1:

i

i=1

Fedorov[Fed71], Atkinson and Donev [AD92], and Pukelsheim [Puk93 ] give surveys and

additional references on optimal exp eriment design.

The formulation of optimal design as maxdet-problems or SDPs has the advantage that

we can easily incorp orate additional useful convex constraints see [VBW98 ].

5 Problems involving moments

Bounds on exp ected values via semide nite programming

k

Let t be a random real variable. The exp ected values E t are called the p ower moments

of the distribution of t. The following classical result gives a characterization of a moment

k

sequence: There exists a probability distribution on R such that x = E t , k =0;:::;2n,if

k

and only if x =1 and

0

2 3

x x x ::: x x

0 1 2 n1 n

6 7

x x x ::: x x

6 7

1 2 3 n n+1

6 7

6 7

x x x ::: x x

2 3 4 n+1 n+2

6 7

H x ;:::;x = 6 7  0: 16

. . . . .

0 2n

. . . . .

6 7

. . . . .

6 7

6 7

x x x ::: x x

4 5

n1 n n+1 2n2 2n1

x x x ::: x x

n n+1 n+2 2n1 2n 8

i

It is easy to see that the condition is necessary: let x = E t , i =0;:::;2n be the moments

i

T

n+1

of some distribution, and let y =[y y


y ] 2 R . Then we have

0 1 n

n

X

2

T i+j 1 n

y H x ;:::;x y = y y E t = E y + y t +


+ y t  0:

0 2n i j 0 1 n

i;j =0

Suciency is less obvious. The pro of is classical and based on convexity arguments; see

e.g., Krein and Nudelman [KN77, p.182] or Karlin and Studden [KS66, p.189{199]. There

are similar conditions for distributions on nite or semi-in nite intervals.

Note that condition 16 is an LMI in the variables x , i.e., the condition that x , ...,

k 0

x b e the moments of some distribution on R can b e expressed as an LMI in x. Using this

2n

fact, we can cast some interesting moment problems as SDPs and maxdet-problems.

Supp ose t is a random variable on R. We do not know its distribution, but we do know

some b ounds on the moments, i.e.,

k

  E t  

k

k

which includes, as a sp ecial case, knowing exact values of some of the moments. Let

2n

pt=c +c t+


+ c t b e a given p olynomial in t. The exp ected value of pt is linear

0 1 2n

i

in the moments E t :

2n 2n

X X

i

E pt= c E t = c x :

i i i

i=0 i=0

We can compute upp er and lower b ounds for E pt,

minimize maximize E pt

k

sub ject to   E t   ; k =1;:::;2n;

k

k

over all probability distributions that satisfy the given moment b ounds, by solving the SDP

minimize maximize c x +


+ c x

1 1 2n 2n

 x  sub ject to   ; k =1;:::;2n

k

k

k

H1;x ;:::;x   0

1 2n

with variables x , ..., x . This gives b ounds on E pt, over all probability distributions

1 2n

that satisfy the known moment constraints. The b ounds are sharp in the sense that there

are distributions, whose moments satisfy the given moment b ounds, for which E pt takes

on the upp er and lower b ounds found by these SDPs.

A related problem was considered by Dahlquist, Eisenstat and Golub [DEG72], who

1 2 i

analytically compute b ounds on E t and E t , given the moments E t , i = 1;:::;n.

Here t is a random variable in a nite interval. Using semide nite programming we can

i

solve more general problems where upp er and lower b ounds on E t , i = 1 :::;n or the

exp ected value of some p olynomials are known.

Another application arises in the optimal control of queuing networks See Bertsimas

[Ber95] and Schwerer [Sch96 ]. 9

Upp er b ound on the variance via semide nite programming

As another example, we can maximize the variance of t, over all probability distributions

that satisfy the moment constraints to obtain a sharp upp er b ound on the variance of t:

2

2

maximize E t E t

k

sub ject to   E t   ; k =1;:::;2n;

k

k

which is equivalent to the SDP

maximize y

" 

x y x

2 1

sub ject to  0

x 1

1

 x   ; k =1;:::;2n 

k

k

k

H1;x ;:::;x   0

1 2n

2

with variables y , x , ..., x . The 2  2-LMI is equivalenttoy  x x . More generally,we

1 2n 2

1

2

2

can compute an upp er b ound on the variance of a given p olynomial E pt E pt . Thus

we can compute an upp er b ound on the variance of a p olynomial pt, given some b ounds

on the moments.

A robust estimate of the moments

Another interesting problem is the maxdet-problem

maximize log det H 1;x ;:::;x 

1 2n

sub ject to   x   ;k =1;:::;2n

17

k

k

k

H1;x ;:::;x  0:

1 2n

The solution can serve as a `robust' solution to the feasibility problem of nding a probability

distribution that satis es given b ounds on the moments. While the SDPs provide lower and

upp er b ounds on E pt, the maxdet-problem should provide a reasonable guess of E pt.

Note that the maxdet-problem 17 is equivalent to

T

maximize log det E f tf t

18

sub ject to   E f t  

T

2 n

over all probability distributions on R, where f t= [1tt ::: t ] . We can interpret this

as the problem of designing a random exp eriment to estimate the co ecients of a p olynomial

n

pt=c +c t+


+ c t .

0 1 n

6 Structural optimization

We consider a truss structure with m bars connecting a set of n no des. External forces are

n

applied at each no de, which cause a small displacement in the no de p ositions. f 2 R will

n

denote the vector of comp onents of  external forces, and d 2 R will denote the vector of 10

corresp onding no de displacements. By `corresp onding' we mean if f is, say, the z -co ordinate

i

of the external force applied at no de k , then d is the z -co ordinate of the displacement of

i

no de k . The vector f is called a loading or load.

We assume damping can be ignored and that the structure is linearly elastic, i.e., the

relation b etween no de displacement and external forces can be describ ed by the di erential

equation



M d + Kd = f: 19

T T

The matrix M = M 0 is called the mass matrix of the truss; the matrix K = K 0is

called the sti ness matrix.

We assume that the geometry unloaded bar lengths and no de p ositions of the truss is

xed; we are to design the cross-sectional areas of the bars. These cross-sectional areas will

b e the design variables x , i =1;:::;m. The sti ness matrix K and the mass matrix M are

i

linear functions of x:

K x=x K +


+ x K ; Mx=x M +


+ x M ;

1 1 m m 1 1 m m

T T

where K = K  0, and M = M  0 dep end on the truss geometry. We assume these

i i

i i

matrices are given For simplicitywe also assume that K x 0 for all nonnegative x.

The total weight W of the truss also dep ends on the bar cross-sectional areas:

tot

W x=w x +


+ w x ;

tot 1 1 m m

where w > 0 are known, given constants density of the material times the length of bar i.

i

Roughly sp eaking, the truss b ecomes sti er, but also heavier, when we increase x ; there is

i

a tradeo between sti ness and weight.

Static measures of sti ness



We rst consider the equilibrium state, i.e., static conditions. In equilibrium, wehaved=0

and d is determined by the set of linear equations

Kd = f:

Our goal is to design the sti est truss, sub ject to b ounds on the bar cross-sectional areas

and total truss weight:

l  x  u; i =1;:::;m; W x  W; 20

i tot

where W is a given limit on truss weight.

There are several ways to form a scalar measure of how sti a truss is, for a given load

f . Perhaps the simplest is the norm squared of the resulting displacementvector d:

T T 2

D x; f  = d d = f K x f:

Another measure is the elastic stored energy,

1

T 1

E x; f  = f K x f:

2 11

Maximizing sti ness corresp onds to minimizing D x; f  or E x; f . These measures are

similar they are b oth small when K is `large', but not exactly the same. In particular, we

will see that using the elastic stored energy E leads to SDP problems, while D in general

do es not.

We can consider several di erent scenarios that re ect our knowledge ab out the p ossible

loadings f that can o ccur. The simplest is that f is a single, xed, known loading. The

design that minimizes the stored energy is the solution of

T 1

minimize f K x f

sub ject to W x  W

21

tot

l  x  u:

i

This problem is convex and can b e cast as the following SDP

minimize t

" 

K x f

 0 sub ject to

T

f t

W x  W

tot

l  x  u

i

in the variables x, t. It should be noted, however, that the SDP is not the most ecient

way to solve 21. More ecient metho ds that directly solve problems of the form 21 have

b een prop osed by Nemirovsky and Ben Tal [BTN92, BTN95a].

In more sophisticated formulations, the loading f mightvary over a given set of p ossible

loads, and we are interested in optimizing the sti ness for the worst case scenario see

also [BTN95b ]. Supp ose for example that f is unknown but b ounded, i.e., it can take

arbitrary values in a ball B = ff jkfk g. The maximum sti ness as f varies over B can

be written as

2

1

max E x; f =  K x:

max

f 2B

2

Therefore we can nd the worst-case design by solving

1

minimize  K x

max

sub ject to W x  W

tot

l  x  u;

i

or, equivalently,by maximizing the smallest eigenvalue of K . This can be cast as an SDP

maximize t

sub ject to K x  tI

W x  W

tot

l  x  u:

i

Finally,we can consider situations where the load is random with known statistics. For

example, supp ose f is a random variable with known mean and covariance:

T

b b b

E f = f; Ef ff f  =: 12

We are interested in minimizing the exp ected stored energy, i.e.,

T 1

minimize E f K f

sub ject to W x  W

tot

l  x  u:

i

This problem can be cast as an SDP, by rst writing the ob jective function as

T 1 T 1 1

b b

E f K f = f K x f + TrK x ;

T

and intro ducing two new variables t and X = X :

minimize t + TrX

" 

b

K x f

sub ject to  0

b

f t

" 

K x I

 0

I X

W x  W

tot

l  x  u:

i

Dynamic measure of sti ness

Wenow return to the dynamical mo del 19. The modal frequencies of the structure describ ed

by 19 are de ned as the values !  0 that satisfy

2

detK M! =0;

i.e., the squarero ots of the generalized eigenvalues of the pair M; K . The fundamental

frequency is the smallest mo dal frequency, i.e.,

1=2

! =  M; K :

1

min

It is the lowest frequency at which the structure can oscillate. The fundamental period of

the structure is 2=! .

1

To avoid oscillations we would like to make the fundamental frequency higher than the

frequencies of the excitations, i.e.,wewould like to imp ose a lower b ound !  . Equiva-

1

lently,we imp ose a maximum fundamental p erio d. This constraint can be expressed as an

LMI, since

2

!   M x K x  0:

1

Various interesting design problems with b ounds on the fundamental frequency can therefore

b e expressed as SDPs. As an example, we can minimize the weight sub ject to the lower b ound

on the fundamental frequency constraintby solving the SDP

minimize W x

tot

2

sub ject to M x K x  0

l  x  u :

i i i 13

7 Quasi-Newton up dates

In quasi-Newton metho ds for unconstrained minimization of a convex function f , the Newton

2 1 1 T

step r f x rf x is replaced by H rf x, where H = H 0 is an approximation

of the Hessian matrix, based on prior information and previous gradientevaluations. In each

+ +

iteration, as the algorithm moves from x to the next p oint x , a new approximation H is

+

determined, based on the current H , and on the di erence b etween the gradients at x and

+

x. A go o d up dating rule for H should satisfy several prop erties: H should b e close to H ,it

1

+ +

should b e easy to compute or, more precisely, the search direction H rf x  should b e

easy to compute, and it should incorp orate the new information obtained byevaluating the

+

gradient rf x . This last prop erty is usually enforced by imp osing the secant condition

+ + +

H x x=rfx rfx: 22

+

Byrd and No cedal [BN89] have prop osed to measure the di erence between H and H

by using the Kullback-Leibler divergence or relativeentropy, given by

1

1=2 + 1=2 1=2 + 1=2

TrH H H log det H H H n

2

see also Dennis and Wolkowicz [DW93]. The Kullback-Leibler divergence is nonnegative

+ + +

for all p ositive de nite H and H , and zero only if H = H . The up date H that satis es

the secant condition and minimizes the Kullback-Leibler divergence is the solution of the

following optimization problem:

1=2 + 1=2 1=2 + 1=2

minimize TrH H H log det H H H n

+

sub ject to H 0 23

+ + +

H x x=rfx rfx:

Fletcher [Fle91] has shown that the solution is given by

T T

Hss H gg

+

H = H + ; 24

T T

s Hs s g

T + +

assuming that s g > 0, where s = x x and g = rf x  rfx. Formula 24 is well

known in unconstrained optimization as the BFGS Broyden, Fletcher, Goldfarb, Shanno

quasi-Newton up date.

Fletcher's observation op ens the p ossibility of adding more complicated LMI constraints

to 23, and solving the resulting problem numerically. For example, we can imp ose a certain

+

sparsity pattern on H , or we can relax the secant condition as

+ + +

kH x x rfx +rfxk;

where  is a given tolerance.

+

Up dating H bynumerically solving a convex optimization problem will obviously involve

far more computation than the BFGS up date. Thus, this formulation for quasi-Newton

up dates is only interesting when gradientevaluations are very exp ensive. 14

8 Conclusion

The applications we describ ed are only a few of the many applications of SDP that have

b een recently investigated, esp ecially in the areas of combinatorial optimization and control

theory. This recent research has b een motivated by the developmentofinterior-p oint meth-

ods for SDP. Most interior-p oint metho ds for linear programming have b een generalized to

SDPs. As in linear programming, these metho ds have p olynomial worst-case complexity,

and p erform very well in practice. Several implementations have b ecome available in the

last few years. These include a commercial SDP solver [GNLC95], and several public do-

main general-purp ose SDP software packages: sdppack [AHNO97], csdp [Bor97], sdpmath

[BPS96], lmitool [END95], sdpa [FK95], sdpt3 [TTT96 ], sp [VB94], sdpsol [WB96]. An

imp ortant di erence with LP is that most of these co des do not exploit problem structure

e.g., sparsity, and if they do, only to a limited extent. Larger SDPs can b e solved success-

fully with interior-p oint metho ds but require sp ecialized techniques that take advantage of

problem structure.

Acknowledgments

We thank Henry Wolkowicz for p ointing out refs. [BN89] and [Fle91].

References

[AD92] A. C. Atkinson and A. N. Donev. Optimum experiment designs. Oxford Statis-

tical Science Series. Oxford University Press, 1992.

[AHNO97] F. Alizadeh, J. P. Haeb erly, M. V. Nayakkankuppam, and M. L. Overton. sdp-

pack User's Guide, Version 0.8 Beta. NYU, June 1997.

[Ali95] F. Alizadeh. Interior p oint metho ds in semide nite programming with applica-

tions to combinatorial optimization. SIAM Journal on Optimization, 51:13{51,

February 1995.

[Bar82] E. R. Barnes. An algorithm for separating patterns by ellipsoids. IBM Journal

of Research and Development, 26:759{764, 1982.

[BEFB94] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequalities

in System and Control Theory, volume 15 of Studies in Applied Mathematics.

SIAM, Philadelphia, PA, June 1994.

[Ber95] D. Bertsimas. The achievable region metho d in the optimal control of queuing

systems; formulations, b ounds and p olicies. Queueing Systems, 21:337{389, 1995.

[BN89] R. H. Byrd and J. No cedal. A to ol for the analysis of quasi-Newton metho ds

with application to unconstrained minimization. SIAM J. on Numerical Analysis,

pages 727{739, 1989. 15

[Bor97] B. Borchers. csdp, a C library for semide nite programming. New Mexico Tech,

March 1997.

[BPS96] N. Brixius, F. A. Potra, and R. Sheng. Solving semide nite programs with Math-

ematica. Technical Rep ort 97/1996, Department of Mathematics, University of

Iowa, 1996.

[BTN92] A. Ben-Tal and A. Nemirovskii. Interior p oint p olynomial time metho d for truss

top ology design. Technical Rep ort 3/92, Faculty of Industrial Engineering and

Management, Technion, Israel Institute of Technology, June 1992.

[BTN95a] A. Ben-Tal and A. Nemirovski. Optimal design of engineering structures. Optima,

pages 4{9, Octob er 1995.

[BTN95b] A. Ben-Tal and A. Nemirovski. Robust truss top ology design via semide nite

programming. Technical Rep ort 4/95, Optimization Lab oratory. Faculty of In-

dustrial Engineering. Technion, August 1995.

[Che94] F. L. Chernousko. State Estimation for Dynamic Systems. CRC Press, Bo ca

Raton, Florida, 1994.

[DEG72] G. Dahlquist, S. C. Eisenstat, and G. H. Golub. Bounds for the error of linear

systems of equations using the theory of moments. Journal of Mathematical

Analysis and Applications, 37:151{166, 1972.

[DGK63] L. Danzer, B. Grunbaum, and V. Klee. Helly's theorem and its relatives. In

V. L. Klee, editor, Convexity, volume 7 of Proceedings of Symposia in Pure

Mathematics, pages 101{180. American Mathematical So ciety, 1963.

[DW93] J. E. Dennis and H. Wolkowicz. Sizing and least change secant metho ds. Techni-

cal Rep ort CORR 90-02, Department of Combinatorics and Optimization. Uni-

versity of Waterlo o, 1993.

[END95] L. El Ghaoui, R. Nikoukha, and F. Deleb ecque. lmitool: a front-end for LMI

optimization, user's guide, 1995. Available via anonymous ftp to ftp.ensta.fr

under /pub/elghaoui/lmitool.

[Fed71] V. V. Fedorov. Theory of Optimal Experiments. Academic Press, 1971.

[FK95] K. Fujisawa and M. Ko jima. SDPA semide nite programming algorithm user's

manual. Technical Rep ort B-308, Department of Mathematical and Computing

Sciences. Tokyo Institute of Technology, 1995.

[Fle91] R. Fletcher. A new variational result for quasi-Newton formulae. SIAM J. on

Optimization, pages 18{21, 1991.

[GLS88] M. Grotschel, L. Lov asz, and A. Schrijver. Geometric Algorithms and Combina-

torial Optimization,volume 2 of Algorithms and Combinatorics. Springer-Verlag,

1988. 16

[GNLC95] P. Gahinet, A. Nemirovski, A. J. Laub, and M. Chilali. LMI Control Toolbox.

The MathWorks, Inc., 1995.

[Joh85] F. John. Extremum problems with inequalities as subsidiary conditions. In

J. Moser, editor, Fritz John, Col lected Papers, pages 543{560. Birkhauser,

Boston, Massachussetts, 1985.

[KN77] M. G. Krein and A. A. Nudelman. The Markov Moment Problem and Extremal

Problems, volume 50 of Translations of Mathematical Monographs. American

Mathematical So ciety, Providence, Rho de Island, 1977.

[KS66] S. Karlin and W. J. Studden. Tchebyche Systems: With Applications in Anal-

ysis and Statistics. Wiley-Interscience, 1966.

[LO96] A. S. Lewis and M. L. Overton. Eigenvalue optimization. Acta Numerica, pages

149{190, 1996.

[NN94] Yu. Nesterov and A. Nemirovsky. Interior-point polynomial methods in convex

programming,volume 13 of Studies in Applied Mathematics. SIAM, Philadelphia,

PA, 1994.

[Puk93] F. Pukelsheim. Optimal Design of Experiments. Wiley, 1993.

[RL87] P. J. Rousseeuw and A. M. Leroy. Robust Regression and Outlier Detection.

Wiley, 1987.

[Ros65] J. B. Rosen. Pattern separation by convex programming. Journal of Mathemat-

ical Analysis and Applications, 10:123{134, 1965.

[Sch96] E. Schwerer. A Linear Programming Approach to the Steady-State Analysis of

Markov Processes. PhD thesis, Graduate Scho ol of Business, Stanford University,

1996. Draft.

[TTT96] K. C. Toh, M. J. To dd, and R. H. Tutunc  u.  SDPT3: a Mat-

lab software package for semide nite programming, 1996. Available at

http://www.math.nus.sg/~ma ttoh kc/ inde x.ht ml.

[VB94] L. Vandenb erghe and S. Boyd. sp: Software for Semide nite Programming.

User's Guide, Beta Version. Stanford University, Octob er 1994. Available at

http://www-isl.stanford.ed u/pe opl e/bo yd.

[VB96] L. Vandenb erghe and S. Boyd. Semide nite programming. SIAM Review,

381:49{95, March 1996.

[VBE96] L. Vandenb erghe, S. Boyd, and A. El Gamal. Optimizing dominant time con-

stantinRC circuits. Technical rep ort, Information Systems Lab oratory, Stanford

University, November 1996. 17

[VBE97] L. Vandenb erghe, S. Boyd, and A. El Gamal. Optimal wire and transistor siz-

ing for circuits with non-tree top ology. In Proceedings of the 1997 IEEE/ACM

International Conference on Computer Aided Design, pages 252{259, 1997.

[VBW98] L. Vandenb erghe, S. Boyd, and S.-P.Wu. Determinant maximization with linear

matrix inequality constraints. SIAM J. on Matrix Analysis and Applications,

April 1998. To app ear.

[WB96] S.-P.Wu and S. Boyd. sdpsol: A Parser/Solver for Semide nite Programming

and Determinant Maximization Problems with Matrix Structure. User's Guide,

Version Beta. Stanford University, June 1996. 18