Applications and Solution Approaches for Mixed-Integer Semidefinite Programming Tristan Gally joint work with Marc E. Pfetsch and Stefan Ulbrich January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 1 I Efficient solvers for specific applications, but few solvers (and theory) for the general case Mixed-Integer Semidefinite Programming I Mixed-integer semidefinite program MISDP sup bT y m X s.t. C − Ai yi 0, i=1 yi 2 Z 8 i 2 I for symmetric matrices Ai , C I Linear constraints, bounds, multiple blocks possible within SDP-constraint January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 2 Mixed-Integer Semidefinite Programming I Mixed-integer semidefinite program MISDP sup bT y m X s.t. C − Ai yi 0, i=1 yi 2 Z 8 i 2 I for symmetric matrices Ai , C I Linear constraints, bounds, multiple blocks possible within SDP-constraint I Efficient solvers for specific applications, but few solvers (and theory) for the general case January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 2 Contents Applications Solution Approaches Outer Approximation SDP-based Branch-and-Bound Warmstarts Dual Fixing Solvers for MISDP Conclusion & Outlook January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 3 Contents Applications Solution Approaches Outer Approximation SDP-based Branch-and-Bound Warmstarts Dual Fixing Solvers for MISDP Conclusion & Outlook January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 4 n Using variables (xi )i2V 2 {−1, 1g with xi = 1 , i 2 S, this is equivalent to Max-Cut MIQP X 1 − xi xj max c ij 2 i<j s.t. xi 2 {−1, 1g 8 i ≤ n Classical Example: Max-Cut 1 2 4 Max-Cut 2 2 Find Cut δ(S), with S ⊆ V and fi, jg 2 δ(S) 3 iff i 2 S, j 2 V n S, that maximizes 1 6 1 1 X c . 2 ij 3 5 fi,jg2δ(S) January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 5 Classical Example: Max-Cut 1 2 4 Max-Cut 2 2 Find Cut δ(S), with S ⊆ V and fi, jg 2 δ(S) 3 iff i 2 S, j 2 V n S, that maximizes 1 6 1 1 X c . 2 ij 3 5 fi,jg2δ(S) n Using variables (xi )i2V 2 {−1, 1g with xi = 1 , i 2 S, this is equivalent to Max-Cut MIQP X 1 − xi xj max c ij 2 i<j s.t. xi 2 {−1, 1g 8 i ≤ n January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 5 T P With X := xx (and notation A • B := Tr(AB) = ij Aij Bij ), this is equivalent to Max-Cut Rk1-MISDP [Poljak, Rendl 1995] 1 max (Diag(C1) − C) • X 4 s.t. diag(X) = 1 Rank(X) = 1 X 0 Xij 2 {−1, 1g Classical Example: Max-Cut n 0 n n 1 X 1 − xi xj 1 X X X cij = @ cij xi xi − cij xi xj A 2 4 i<j i=1 j=1 j=1 1 = x T (Diag(C1) − C)x 4 January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 6 Classical Example: Max-Cut n 0 n n 1 X 1 − xi xj 1 X X X cij = @ cij xi xi − cij xi xj A 2 4 i<j i=1 j=1 j=1 1 = x T (Diag(C1) − C)x 4 T P With X := xx (and notation A • B := Tr(AB) = ij Aij Bij ), this is equivalent to Max-Cut Rk1-MISDP [Poljak, Rendl 1995] 1 max (Diag(C1) − C) • X 4 s.t. diag(X) = 1 Rank(X) = 1 X 0 Xij 2 {−1, 1g January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 6 Theorem [Laurent, Poljak 1995] Every integral solution satisfies Rank(X) = 1. Classical Example: Max-Cut Max-Cut Rk1-MISDP 1 max (Diag(C1) − C) • X 4 s.t. diag(X) = 1 Rank(X) = 1 X 0 Xij 2 {−1, 1g I Relaxation still non-convex because of rank constraint January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 7 Classical Example: Max-Cut Max-Cut MISDP 1 max (Diag(C1) − C) • X 4 s.t. diag(X) = 1 Rank(X) = 1 X 0 Xij 2 {−1, 1g I Relaxation still non-convex because of rank constraint Theorem [Laurent, Poljak 1995] Every integral solution satisfies Rank(X) = 1. January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 7 I Under certain conditions on A, this is equivalent to `1-Minimization min kxk1 s.t. Ax = b n x 2 R Compressed Sensing I Task: find sparsest solution to underdetermined system of linear equations, i.e. a solution of `0-Minimization min kxk0 s.t. Ax = b n x 2 R where kxk0 := jsupp(x)j. January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 8 Compressed Sensing I Task: find sparsest solution to underdetermined system of linear equations, i.e. a solution of `0-Minimization min kxk0 s.t. Ax = b n x 2 R where kxk0 := jsupp(x)j. I Under certain conditions on A, this is equivalent to `1-Minimization min kxk1 s.t. Ax = b n x 2 R January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 8 Theorem [Foucart, Lai 2008] If Ax = b has a solution x with kxk0 ≤ k and the RIP of order 2k holds for 2 p β2k 2 < 4 2 − 3 ≈ 2.6569, α2k then x is the unique solution of both the `0- and the `1-optimization problem. Compressed Sensing One such condition is the (asymmetric) restricted isometry property (RIP): 2 2 2 2 2 αk kxk2 ≤ kAxk2 ≤ βk kxk2 8x : kxk0 ≤ k January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 9 Compressed Sensing One such condition is the (asymmetric) restricted isometry property (RIP): 2 2 2 2 2 αk kxk2 ≤ kAxk2 ≤ βk kxk2 8x : kxk0 ≤ k Theorem [Foucart, Lai 2008] If Ax = b has a solution x with kxk0 ≤ k and the RIP of order 2k holds for 2 p β2k 2 < 4 2 − 3 ≈ 2.6569, α2k then x is the unique solution of both the `0- and the `1-optimization problem. January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 9 Compressed Sensing 2 2 The optimal constant αk (and similarly βk ) for 2 2 2 2 2 αk kxk2 ≤ kAxk2 ≤ βk kxk2 8x : kxk0 ≤ k can be computed via the following non-convex rank-constrained MISDP: RIP-Rk1-MISDP min Tr(AT AX) s.t. Tr(X) = 1 −zj ≤ Xjj ≤ zj 8 j ≤ n n X zj ≤ k j=1 Rank(X) =1 X 0 z 2f0, 1gn January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 10 Compressed Sensing RIP-MISDP min Tr(AT AX) s.t. Tr(X) = 1 −zj ≤ Xjj ≤ zj 8j ≤ n n X zj ≤ k j=1 Rank(X) = 1 X 0 z 2f0, 1gn Theorem [G., Pfetsch 2016] There always exists an optimal solution for (RIP-MISDP) with Rank(X) = 1. January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 11 Rm I Cross-sectional areas x 2 + for bars that minimize the volume while creating a “stable” truss I Stability is measured by the 1 T compliance 2 f u with node displacements u optimal structure Truss Topology Design d I n nodes V = vi 2 R : i = 1, ... , n I nf free nodes Vf ⊂ V I m possible bars E ⊆ ffvi , vj g : i =6 jg , jEj = m df I Force f 2 R for df = d · nf ground structure 3x3 January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 12 Truss Topology Design Rd Rm I n nodes V = vi 2 : i = 1, ... , n I Cross-sectional areas x 2 + for bars that minimize the volume I nf free nodes Vf ⊂ V while creating a “stable” truss I m possible bars Stability is measured by the E ⊆ ffvi , vj g : i =6 jg , jEj = m I compliance 1 f T u with node Force f 2 Rdf for d = d · n 2 I f f displacements u ground structure 3x3 optimal structure January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 12 Truss Topology Design TTD-SDP [Ben-Tal, Nemirovski 1997] I E : set of possible bars X I `e : length of bar e min `exe e2E I x : cross-sectional areas 2C f T I f : external force s.t. max 0 f A(x) I Cmax : upper bound on compliance xe ≥ 0 8e 2 E I Ae: bar stiffness matrices P with stiffness matrix A(x) = Ae`exe. e2E January 11, 2018 | Applications and Solution Approaches for Mixed-Integer Semidefinite Programming | Tristan Gally | 13 TTD-MISDP [Kocvaraˇ 2010, Mars 2013] X X a min `e axe e2E a2A 2C f T s.t. max 0 fA (x) X a xe ≤ 1 8e 2 E a2A a xe 2 f0, 1g8 e 2 E, a 2 A, P P a where A(x) = Ae `e a xe . e2E a2A Truss Topology Design I In practice, we won’t be able to produce/buy bars of any desired size.