Sums of Squares, Moment Matrices and Optimization Over Polynomials
Total Page:16
File Type:pdf, Size:1020Kb
SUMS OF SQUARES, MOMENT MATRICES AND OPTIMIZATION OVER POLYNOMIALS MONIQUE LAURENT∗ May 8, 2008 Abstract. We consider the problem of minimizing a polynomial over a semialge- braic set defined by polynomial equations and inequalities, which is NP-hard in general. Hierarchies of semidefinite relaxations have been proposed in the literature, involving positive semidefinite moment matrices and the dual theory of sums of squares of poly- nomials. We present these hierarchies of approximations and their main properties: asymptotic/finite convergence, optimality certificate, and extraction of global optimum solutions. We review the mathematical tools underlying these properties, in particular, some sums of squares representation results for positive polynomials, some results about moment matrices (in particular, of Curto and Fialkow), and the algebraic eigenvalue method for solving zero-dimensional systems of polynomial equations. We try whenever possible to provide detailed proofs and background. Key words. positive polynomial, sum of squares of polynomials, moment problem, polynomial optimization, semidefinite programming. AMS(MOS) subject classifications. 13P10, 13J25, 13J30, 14P10, 15A99, 44A60, 90C22, 90C30. Contents 1 Introduction...............................158 1.1 The polynomial optimization problem . 159 1.2 Thescopeofthispaper . .161 1.3 Preliminaries on polynomials and semidefinite programs . 162 1.4 Contentsofthepaper . .166 2 Algebraic preliminaries . 166 2.1 Polynomial ideals and varieties . 166 2.2 The quotient algebra R[x]/I .................169 2.3 Gr¨obner bases and standard monomial bases . 171 2.4 Solving systems of polynomial equations . 173 3 Positive polynomials and sums of squares . 178 3.1 Somebasicfacts ........................178 3.2 Sums of squares and positive polynomials: Hilbert’s result . 178 3.3 Recognizing sums of squares of polynomials . 180 3.4 SOS relaxations for polynomial optimization . 182 3.5 Convex quadratic polynomial optimization . 183 3.6 Some representation results for positive polynomials . 184 3.7 Proof of Putinar’s theorem . 189 3.8 The cone Σn,d of sums of squares is closed . 191 ∗CWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands. Supported by the Netherlands Organisation for Scientific Research grant NWO 639.032.203 and by ADONET, Marie Curie Research Training Network MRTN-CT-2003-504438. 157 158 MONIQUE LAURENT 4 Moment sequences and moment matrices . 193 4.1 Somebasicfacts ........................193 4.2 Moment relaxations for polynomial optimization . 197 4.3 Themomentproblem . .. .. .. .. .. .198 4.4 The K-momentproblem. .201 5 More about moment matrices . 203 5.1 Finite rank moment matrices . 203 5.2 Finite atomic measures for truncated moment sequences . 205 5.3 Flat extensions of moment matrices . 208 5.4 Flat extensions and representing measures . 211 6 Back to the polynomial optimization problem . 216 6.1 Hierarchies of relaxations . 216 6.2 Duality .............................217 6.3 Asymptotic convergence . 220 6.4 Approximating the unique global minimizer via the mo- mentrelaxations . .221 6.5 Finite convergence . 222 6.6 Optimality certificate . 224 6.7 Extracting global minimizers . 228 6.8 Software and examples . 229 7 Application to optimization - Some further selected topics . 232 7.1 Approximating positive polynomials by sums of squares . 232 7.2 Unconstrained polynomial optimization . 237 7.3 Sums of squares over the gradient ideal . 242 8 Exploiting algebraic structure to reduce the problem size . 245 8.1 Exploiting sparsity . 245 8.2 Exploitingequations . .252 8.3 Exploitingsymmetry. .258 1. Introduction. This survey focuses on the following polynomial op- timization problem: Given polynomials p,g ,...,g R[x], find 1 m ∈ min p := inf p(x) subject to g1(x) 0,...,gm(x) 0, (1.1) x∈Rn ≥ ≥ the infimum of p over the basic closed semialgebraic set K := x Rn g (x) 0,...,g (x) 0 . (1.2) { ∈ | 1 ≥ m ≥ } Here R[x]= R[x1,..., xn] denotes the ring of multivariate polynomials in the n-tuple of variables x = (x1,..., xn). This is a hard, in general non- convex, optimization problem. The objective of this paper is to survey relaxations methods for this problem, that are based on relaxing positiv- ity over K by sums of squares decompositions, and the dual theory of moments. The polynomial optimization problem arises in numerous appli- cations. In the rest of the Introduction, we present several instances of this problem, discuss the scope of the paper, and give some preliminaries about polynomials and semidefinite programming. SUMS OF SQUARES, MOMENTS AND POLYNOMIAL OPTIMIZATION 159 1.1. The polynomial optimization problem. We introduce sev- eral instances of problem (1.1). The unconstrained polynomial minimization problem. This is the problem pmin = inf p(x), (1.3) x∈Rn of minimizing a polynomial p over the full space K = Rn. We now men- tion several problems which can be cast as instances of the unconstrained polynomial minimization problem. Testing matrix copositivity. An n n symmetric matrix M is said T R×n to be copositive if x Mx 0 for all x +; equivalently, M is copositive min ≥ ∈ n 2 2 if and only if p = 0 in (1.3) for the polynomial p := i,j=1 xi xj Mij . Testing whether a matrix is not copositive is an NP-complete problem [94]. P The partition problem. The partition problem asks whether a given sequence a1,...,an of positive integer numbers can be partitioned, i.e., whether xT a = 0 for some x 1 n. Equivalently, the sequence can be min ∈ {± } n 2 partitioned if p = 0 in (1.3) for the polynomial p := ( i=1 aixi) + n (x2 1)2. The partition problem is an NP-complete problem [40]. i=1 i − P RE P The distance realization problem. Let d = (dij )ij∈E be a given set of scalars (distances) where E is a given set of pairs∈ ij with 1 i < j n. Given an integer k 1 one says that d is realizable ≤ k ≤ ≥ k in R if there exist vectors v1,...,vn R such that dij = vi vj for all ij E. Equivalently, d is realizable∈ in Rk if pmin =k 0 for− thek polynomial∈ p := (d2 k (x x )2)2 in the variables x ij∈E ij − h=1 ih − jh ih (i = 1,...,n,h = 1,...,k). Checking whether d is realizable in Rk is an P P NP-complete problem, already for dimension k = 1 (Saxe [123]). Note that the polynomials involved in the above three instances have degree 4. Hence the unconstrained polynomial minimization problem is a hard problem, already for degree 4 polynomials, while it is polynomial time solvable for degree 2 polynomials (cf. Section 3.2). The problem (1.1) also contains (0/1) linear programming. (0/1) Linear programming. Given a matrix A Rm×n and vectors b Rm, c Rn, the linear programming problem can∈ be formulated as ∈ ∈ min cT x s.t. Ax b, ≤ thus it is of the form (1.1) where the objective function and the constraints are all linear (degree at most 1) polynomials. As is well known it can be solved in polynomial time (cf. e.g. [128]). If we add the quadratic 2 constraints xi = xi (i = 1,...,n) we obtain the 0/1 linear programming problem: min cT x s.t. Ax b, x2 = x i =1,...,n, ≤ i i ∀ well known to be NP-hard. 160 MONIQUE LAURENT The stable set problem. Given a graph G = (V, E), a set S V is said to be stable if ij E for all i, j S. The stable set problem⊆ asks for the maximum cardinality6∈ α(G) of a∈ stable set in G. Thus it can be formulated as 2 α(G) = max xi s.t. xi + xj 1 (ij E), xi = xi (i V ) (1.4) x∈RV ≤ ∈ ∈ iX∈V 2 = max xi s.t. xixj = 0 (ij E), xi xi = 0 (i V ). (1.5) x∈RV ∈ − ∈ Xi∈V Alternatively, using the theorem of Motzkin-Straus [93], the stability num- ber α(G) can be formulated via the program 1 = min xT (I + A )x s.t. x =1, x 0 (i V ). (1.6) α(G) G i i ≥ ∈ iX∈V Using the characterization mentioned above for copositive matrices, one can derive the following further formulation for α(G) α(G) = inf t s.t. t(I + A ) J is copositive, (1.7) G − which was introduced in [32] and further studied e.g. in [46] and references therein. Here, J is the all ones matrix, and AG is the adjacency matrix of G, defined as the V V 0/1 symmetric matrix whose (i, j)th entry is 1 precisely when i = j ×V and ij E. As computing α(G) is an NP-hard problem (see, e.g.,6 [40]),∈ we see that∈ problem (1.1) is NP-hard already in the following two instances: the objective function is linear and the constraints are quadratic polynomials (cf. (1.5)), or the objective function is quadratic and the constraints are linear polynomials (cf. (1.6)). We will use the stable set problem and the following max-cut problem in Section 8.2 to illustrate the relaxation methods for polynomial problems in the 0/1 (or 1) case. ± The max-cut problem. Let G = (V, E) be a graph and wij R (ij E) be weights assigned to its edges. A cut in G is the set of edges∈ ij ∈ E i S, j V S for some S V and its weight is the sum of{ the∈ weights| ∈ of its∈ edges.\ } The max-cut⊆ problem, which asks for a cut of maximum weight, is NP-hard [40]. Note that a cut can be encoded V by x 1 by assigning xi = 1 to nodes i S and xi = 1 to nodes ∈i {±V } S and the weight of the cut is encoded∈ by the function− ∈ \ ij∈E (wij /2)(1 xixj ).