Semidefinite Programming for Discrete Optimization and Matrix Completion
Total Page:16
File Type:pdf, Size:1020Kb
Discrete Applied Mathematics 123 (2002) 513–577 Semideÿnite programming for discrete optimization and matrixcompletion problems Henry Wolkowicz∗;1 , Miguel F. Anjos2 Department of Combinatorics & Optimization, Faculty of Mathematics, University of Waterloo, Waterloo, Ontario, Canada N2L3G1 Abstract Semideÿnite programming (SDP) is currently one of the most active areas of research in opti- mization. SDP has attracted researchers from a wide variety of areas because of its theoretical and numerical elegance as well as its wide applicability. In this paper we present a survey of two ma- jor areas of application for SDP, namely discrete optimization and matrixcompletion problems. In the ÿrst part of this paper we present a recipe for ÿnding SDP relaxations based on adding redundant constraints and using Lagrangian relaxation. We illustrate this with several examples. We ÿrst show that many relaxations for the max-cut problem (MC) are equivalent to both the Lagrangian and the well-known SDP relaxation. We then apply the recipe to obtain new strengthened SDP relaxations for MC as well as known SDP relaxations for several other hard discrete optimization problems. In the second part of this paper we discuss two completion problems, the positive semideÿnite matrixcompletion problem and the Euclidean distance matrixcompletion problem. We present some theoretical results on the existence of such completions and then proceed to the application of SDP to ÿnd approximate completions. We conclude this paper with a new application of SDP to ÿnd approximate matrix completions for large and sparse instances of Euclidean distance matrices. ? 2002 Elsevier Science B.V. All rights reserved. Keywords: Semideÿnite programming; Discrete optimization; Lagrangian relaxation; Max-cut problem; Euclidean distance matrix; Matrix completion problem Survey article for the Proceedings of Discrete Optimization ’99 where some of these results were pre- sented as a plenary address. This paper is available by anonymous ftp at orion.math.uwaterloo.ca in directory pub=henry=reports or with URL. ∗ Corresponding author. Tel.: +1-519-888-4567x5589. E-mail addresses: [email protected] (H. Wolkowicz), [email protected] (M. F. Anjos). URL: http://orion.math.uwaterloo.ca/∼hwolkowi/henry/reports/ABSTRACTS.html 1 Research supported by The Natural Sciences and Engineering Research Council of Canada. 2 Research supported by an FCAR Doctoral Research Scholarship. 0166-218X/02/$ - see front matter ? 2002 Elsevier Science B.V. All rights reserved. PII: S0166-218X(01)00352-3 514 H. Wolkowicz, M. F. Anjos / Discrete Applied Mathematics 123 (2002) 513–577 1. Introduction There have been many survey articles written in the last few years on semideÿnite programming (SDP) and its applicability to discrete optimization and matrixcompletion problems [2,5,36–38,61,76,77,105] This highlights the fact that SDP is currently one of the most active areas of research in optimization. In this paper we survey in depth two application areas where SDP research has recently made signiÿcant contributions. Several new results are also included. The ÿrst part of the paper is based on the premise that Lagrangian relaxation is “best”. By this we mean that good tractable bounds can always be obtained using Lagrangian relaxation. Since the SDP relaxation is equivalent to the Lagrangian re- laxation, we explore approaches to obtain tight SDP relaxations for discrete optimiza- tion by applying a recipe for ÿnding SDP relaxations using Lagrangian duality. We begin by considering the max-cut problem (MC) in Section 2.2. We ÿrst present sev- eral diKerent relaxations of MC that are equivalent to the SDP relaxation, including the Lagrangian relaxation, the relaxation over a sphere, the relaxation over a box, and the eigenvalue relaxation. This illustrates our theme on the strength of the La- grangian relaxation. The question of which relaxation is most appropriate in practice for a given instance of MC remains open. Section 2.5 contains an overview of the main algorithms that have been proposed to compute the SDP bound, and Section 2.6 presents an overview of the known qualitative results about the quality of the SDP bound. We then proceed in Section 2.7 to derive new strengthened SDP re- laxations for MC. To obtain these relaxations we apply the recipe for ÿnding SDP relaxations presented in [100]. This recipe can be summarized as: add as many re- dundant quadratic constraints as possible; take the Lagrangian dual of the Lagrangian dual; remove redundant constraints and project the feasible set of the resulting SDP to guarantee strict feasibility. We also show several interesting properties for the tighter of the new SDP relaxations, denoted SDP3. In particular, we prove that SDP3 is a strict improvement on the addition of all the triangle inequalities to the well-known SDP relaxation, denoted SDP1 [7–9]. This shows that SDP3 often improves on SDP1 whenever the latter is not optimal. In Section 3 we discuss the application of La- grangian relaxation to general quadratically constrained quadratic problems and in Sec- tion 4 we present applications of the recipe to other discrete optimization problems, including the graph partitioning, quadratic assignment, max-clique and max-stable-set problems. The second part of the paper presents several SDP algorithms for the positive semideÿnite and the Euclidean distance matrixcompletion problems. The algorithms are shown to be eLcient for large sparse problems. Section 5.1 presents some theo- retical existence results for completions based on chordality. This follows the work in [41]. An approach to solving large sparse completion problems based on approximate completions [56] is outlined in Section 5.2. In Section 5.3 a similar approach for Eu- clidean distance matrixcompletions [1] is presented. However, the latter does not take advantage of sparsity and has diLculty solving large sparse problems. We conclude in Section 5.4 with a new characterization of Euclidean distance matrices from which we derive an algorithm that successfully exploits sparsity [3]. H. Wolkowicz, M. F. Anjos / Discrete Applied Mathematics 123 (2002) 513–577 515 1.1. Notation and preliminaries We let Sn denote the space of n × n symmetric matrices. This space has dimension t(n):=n(n +1)=2 and is endowed with the trace inner product A; B = trace AB.Welet A◦B denote the Hadamard (elementwise) matrixproduct and A ¡ 0 denote the LNowner partial order on Sn, i.e. for A ∈ Sn;A¡ 0 if and only if A is positive semideÿnite. We denote by P the cone of positive semideÿnite matrices. We also work with matrices in the space St(n)+1. For given Y ∈ St(n)+1, we index the rows and columns of Y by 0; 1;:::;t(n). We will be particularly interested in the vector x obtained from the ÿrst (0th) row (or column) of Y with the ÿrst element dropped. Thus, in our notation, x = Y0;1:t(n). We let e denote the vector of ones and E = eeT the matrixof ones; their dimensions will be clear from the context. We also let ei denote the ith unit vector and deÿne the elementary matrices E := √1 (e eT + e eT), if i = j; E := 1 (e eT + e eT). For any vector √ij 2 i j j i ii 2 i j j i n T v ∈ R ,welet||v||:= v v denote the ‘2 norm of v. We use operator notation and operator adjoints. The adjoint of the linear operator A is denoted A∗ and satisÿes (by deÿnition) Ax; y = x; A∗y; ∀x; y: Given a matrix S ∈ Sn, we now deÿne several useful operators. The operator diag(S) returns a vector with the entries on the diagonal of S. Given v ∈ Rn, the operator Diag(v) returns an n×n diagonal matrixwith the vector v on the diagonal. It is straight- forward to check that Diag is the adjoint operator of diag. We use both Diag(v) and Diag v provided the meaning is clear, and the same convention applies to diag and all the other operators. The symmetric vectorizing operator svec satisÿes s=svec(S) ∈ Rt(n) where s is formed column-wise from S and the strictly lower triangular part of S is ignored. Its inverse is the operator sMat, so S =sMat(s) if and only if s=svec(S). Note that the adjoint of svec is not sMat but svec∗ = hMat with hMat(s) being the operator that forms a symmetric matrixfrom s like sMat but also multiplies the oK-diagonal 1 terms by 2 in the process. Similarly, the adjoint of sMat is the operator dsvec which acts like svec except that the oK-diagonal elements are multiplied by 2. For notational convenience, we also deÿne the symmetrizing diagonal vector operator sdiag(x):=diag(sMat(x)) and the vectorizing symmetric vector operator vsMat(x):=vec(sMat(x)); where vec(S) returns the n2-dimensional vector formed columnwise from S like svec but with the complete columns of the matrix S. Note that the adjoint of vsMat is ∗ 1 T vsMat (x) = dsvec[ 2 (Mat(x) + Mat(x) )]: Let us summarize here some frequently used operators in this paper: diag∗ = Diag; svec∗ = hMat; 516 H. Wolkowicz, M. F. Anjos / Discrete Applied Mathematics 123 (2002) 513–577 svec−1 = sMat; dsvec∗ = sMat; ∗ 1 T vsMat = dsvec[ 2 (Mat(·) + Mat(·) )]: We will frequently use the following relationships between matrices and vectors: y ∼ T ∼ n ∼ 0 T t(n)+1 X = vv = sMat(x) ∈ S and Y = (y0 x ) ∈ S ;y0 ∈ R: x 2. The max-cut problem We begin our presentation with the study of one of the simplest NP-hard problems, albeit one for which SDP has been successful. The max-cut problem (MC) is a discrete optimization problem on undirected graphs with weights on the edges.