Sums of Squares, Moment Matrices and Optimization Over Polynomials
Total Page:16
File Type:pdf, Size:1020Kb
SUMS OF SQUARES, MOMENT MATRICES AND OPTIMIZATION OVER POLYNOMIALS MONIQUE LAURENT∗ Updated version: February 6, 2010 Abstract. We consider the problem of minimizing a polynomial over a semialge- braic set defined by polynomial equations and inequalities, which is NP-hard in general. Hierarchies of semidefinite relaxations have been proposed in the literature, involving positive semidefinite moment matrices and the dual theory of sums of squares of poly- nomials. We present these hierarchies of approximations and their main properties: asymptotic/finite convergence, optimality certificate, and extraction of global optimum solutions. We review the mathematical tools underlying these properties, in particular, some sums of squares representation results for positive polynomials, some results about moment matrices (in particular, of Curto and Fialkow), and the algebraic eigenvalue method for solving zero-dimensional systems of polynomial equations. We try whenever possible to provide detailed proofs and background. Key words. positive polynomial, sum of squares of polynomials, moment problem, polynomial optimization, semidefinite programming AMS(MOS) subject classifications. 13P10, 13J25, 13J30, 14P10, 15A99, 44A60, 90C22, 90C30 Contents 1 Introduction............................... 3 1.1 The polynomial optimization problem . 4 1.2 Thescopeofthispaper .................... 6 1.3 Preliminaries on polynomials and semidefinite programs . 7 1.3.1 Polynomials ..................... 7 1.3.2 Positive semidefinite matrices . 8 1.3.3 Flat extensions of matrices . 9 1.3.4 Semidefinite programs . 9 1.4 Contentsofthepaper ..................... 11 2 Algebraic preliminaries . 11 2.1 Polynomial ideals and varieties . 11 2.2 The quotient algebra R[x]/I ................. 14 2.3 Gr¨obner bases and standard monomial bases . 16 2.4 Solving systems of polynomial equations . 18 2.4.1 Motivation: The univariate case. 18 2.4.2 The multivariate case. 19 2.4.3 Computing VC(I) with a non-derogatory multi- plicationmatrix. 21 2.4.4 Root counting with Hermite’s quadratic form . 23 2.5 Border bases and commuting multiplication matrices . 26 3 Positive polynomials and sums of squares . 29 ∗Centrum Wiskunde & Informatica (CWI), Science Park 123, 1098 XG Amsterdam, Netherlands. Email: [email protected]. 1 2 Monique Laurent 3.1 Somebasicfacts ........................ 29 3.2 Sums of squares and positive polynomials: Hilbert’s result . 29 3.3 Recognizing sums of squares of polynomials . 31 3.4 SOS relaxations for polynomial optimization . 33 3.5 Convex quadratic optimization . 34 3.6 Some representation results for positive polynomials . 35 3.6.1 Positivity certificates via the Positivstellensatz . 35 3.6.2 Putinar’s Positivstellensatz . 37 3.6.3 Representation results in the univariate case . 39 3.6.4 Other representation results . 41 3.6.5 Sums of squares and convexity . 42 3.7 Proof of Putinar’s theorem . 47 3.8 The cone of sums of squares is closed . 49 4 Moment sequences and moment matrices . 52 4.1 Somebasicfacts ........................ 52 4.1.1 Measures ....................... 52 4.1.2 Moment sequences . 52 4.1.3 Moment matrices . 53 4.1.4 Moment matrices and (bi)linear forms on R[x] . 53 4.1.5 Necessary conditions for moment sequences . 54 4.2 Moment relaxations for polynomial optimization . 55 4.3 Convex quadratic optimization (revisited) . 57 4.4 Themomentproblem ..................... 59 4.4.1 Duality between sums of squares and moment se- quences........................ 59 4.4.2 Bounded moment sequences . 61 4.5 The K-momentproblem. 62 4.6 Proof of Haviland’s theorem . 64 4.7 Proof of Schm¨udgen’s theorem . 66 5 Moreaboutmomentmatrices . 68 5.1 Finite rank moment matrices . 68 5.2 Finite atomic measures for truncated moment sequences . 70 5.3 Flat extensions of moment matrices . 74 5.3.1 First proof of the flat extension theorem . 75 5.3.2 A generalized flat extension theorem . 77 5.4 Flat extensions and representing measures . 81 5.5 The truncated moment problem in the univariate case . 85 6 Back to the polynomial optimization problem . 89 6.1 Hierarchies of relaxations . 89 6.2 Duality ............................. 90 6.3 Asymptotic convergence . 93 6.4 Approximating the unique global minimizer via the mo- mentrelaxations . .. .. .. .. .. .. 94 6.5 Finite convergence . 95 6.6 Optimalitycertificate . 97 6.7 Extracting global minimizers . 101 6.8 Software and examples . 102 7 Application to optimization - Some further selected topics . 106 7.1 Approximating positive polynomials by sums of squares . 106 Sums of Squares, Moment Matrices and Polynomial Optimization 3 7.1.1 Bounds on entries of positive semidefinite mo- mentmatrices . .107 7.1.2 Proof of Theorem 7.2 . 109 7.1.3 Proof of Theorem 7.3 . 110 7.2 Unconstrained polynomial optimization . 112 7.2.1 Case 1: p attains its minimum and a ball is known containing a minimizer . 114 7.2.2 Case 2: p attains its minimum, but no informa- tion about minimizers is known . 114 7.2.3 Case 3: p does not attain its minimum . 115 7.3 Positive polynomials over the gradient ideal . 118 8 Exploiting algebraic structure to reduce the problem size . 121 8.1 Exploiting sparsity . 122 8.1.1 Using the Newton polynomial . 122 8.1.2 Structured sparsity on the constraint and objec- tivepolynomials . .122 8.1.3 Proof of Theorem 8.9 . 125 8.1.4 Extracting global minimizers . 127 8.2 Exploitingequations . .128 8.2.1 The zero-dimensional case . 129 8.2.2 The 0/1 case . 131 8.2.3 Exploiting sparsity in the 0/1 case . 133 8.3 Exploitingsymmetry. .134 9 Bibliography ..............................139 Note. This is an updated version of the article Sums of Squares, Moment Matrices and Polynomial Optimization, published in Emerging Applications of Algebraic Geometry, Vol. 149 of IMA Volumes in Mathe- matics and its Applications, M. Putinar and S. Sullivant (eds.), Springer, pages 157-270, 2009. 1. Introduction. This survey focuses on the following polynomial op- timization problem: Given polynomials p,g ,...,g R[x], find 1 m ∈ min p := inf p(x) subject to g1(x) 0,...,gm(x) 0, (1.1) x∈Rn ≥ ≥ the infimum of p over the basic closed semialgebraic set K := x Rn g (x) 0,...,g (x) 0 . (1.2) { ∈ | 1 ≥ m ≥ } Here R[x]= R[x1,..., xn] denotes the ring of multivariate polynomials in the n-tuple of variables x = (x1,..., xn). This is a hard, in general non- convex, optimization problem. The objective of this paper is to survey relaxation methods for this problem, that are based on relaxing positiv- ity over K by sums of squares decompositions, and the dual theory of moments. The polynomial optimization problem arises in numerous appli- cations. In the rest of the Introduction, we present several instances of this problem, discuss the scope of the paper, and give some preliminaries about polynomials and semidefinite programming. 4 Monique Laurent 1.1. The polynomial optimization problem. We introduce sev- eral instances of problem (1.1). The unconstrained polynomial minimization problem. This is the problem pmin = inf p(x), (1.3) x∈Rn of minimizing a polynomial p over the full space K = Rn. We now men- tion several problems which can be cast as instances of the unconstrained polynomial minimization problem. Testing matrix copositivity. An n n symmetric matrix M is said T R×n to be copositive if x Mx 0 for all x +; equivalently, M is copositive min ≥ ∈ n 2 2 if and only if p = 0 in (1.3) for the polynomial p := i,j=1 xi xj Mij . Testing whether a matrix is not copositive is an NP-complete problem [111]. P The partition problem. The partition problem asks whether a given sequence a1,...,an of positive integer numbers can be partitioned, i.e., whether xT a = 0 for some x 1 n. Equivalently, the sequence can be min ∈ {± } n 2 partitioned if p = 0 in (1.3) for the polynomial p := ( i=1 aixi) + n (x2 1)2. The partition problem is an NP-complete problem [45]. i=1 i − P RE P The distance realization problem. Let d = (dij )ij∈E be a given set of scalars (distances) where E is a given set of pairs∈ ij with 1 i < j n. Given an integer k 1 one says that d is realizable ≤ k ≤ ≥ k in R if there exist vectors v1,...,vn R such that dij = vi vj for all ij E. Equivalently, d is realizable∈ in Rk if pmin =k 0 for− thek polynomial∈ p := (d2 k (x x )2)2 in the variables x ij∈E ij − h=1 ih − jh ih (i = 1,...,n,h = 1,...,k). Checking whether d is realizable in Rk is an P P NP-complete problem, already for dimension k = 1 (Saxe [142]). Note that the polynomials involved in the above three instances have degree 4. Hence the unconstrained polynomial minimization problem is a hard problem, already for degree 4 polynomials, while it is polynomial time solvable for degree 2 polynomials (cf. Section 3.2). The problem (1.1) also contains (0/1) linear programming. (0/1) Linear programming. Given a matrix A Rm×n and vectors b Rm, c Rn, the linear programming problem can∈ be formulated as ∈ ∈ min cT x s.t. Ax b, ≤ thus it is of the form (1.1) where the objective function and the constraints are all linear (degree at most 1) polynomials. As is well known it can be solved in polynomial time (cf. e.g. [146]). If we add the quadratic 2 constraints xi = xi (i = 1,...,n) we obtain the 0/1 linear programming problem: min cT x s.t. Ax b, x2 = x i =1,...,n, ≤ i i ∀ well known to be NP-hard. Sums of Squares, Moment Matrices and Polynomial Optimization 5 The stable set problem. Given a graph G = (V, E), a set S V is said to be stable if ij E for all i, j S.