What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion { Israel Institute of Technology Abstract Tremendous recent progress in the performance of optimization techniques for Linear, Conic Quadratic and Semidefinite Programming makes it important to know how to recognize opti- mization problems lending themselves to these advanced techniques. To this end, we present in the talk a kind of simple, powerful and \completely algorithmic" calculus of problems reducible to CQP and SDP. We demonstrate also that problems representable via Conic Quadratic Pro- gramming admit polynomial time Linear Programming approximations. What can be expressed via Conic Quadratic and Semidefinite Programming? Arkadi Nemirovski [email protected] Faculty of Industrial Engineering and Management Technion { Israel Institute of Technology What the story is about | Consider the following optimization program: minimize cT x subject to 8 <> Ax = b (a) :> x ≥ 0 8 ! > 1=3 > P8 3 1=7 2=7 3=7 1=5 2=5 1=5 > > jxij ≤ x2 x3 x4 + 2x1 x5 x6 > i=1 > > 1 2 > 5x2 ≥ + > 1=2 2 1=3 3 5=8 <> 0 x1 x2 x2 1x3x4 (b) x2 x1 > B C > B C > B C > B x1 x4 x3 C > B C > B C 5I > B C > B x3 x6 x3 C > @ A :> 8 0 x3 x8 1 > > B x1 x2 − x1 x3 − x2 x4 − x3 C > B C > B C > B x − x x x − x x − x C > B 2 1 2 3 2 4 3 C < B C 0 B C (c) > B x3 − x2 x3 − x2 x3 x4 − x3 C > @ A > > x − x x − x x − x x > 4 3 4 3 4 3 4 > π :> x1 + x2 sin(φ) + x3 sin(2φ) + x4 sin(4φ) ≥ 0 8φ 2 0; 2 ♠ The problem can be converted, in a systematic way, into an equivalent semidefinite program dim z T X d z ! min j P0 + ziPi 0: (SDP) i=1 ♠ Removing constraints (c), the resulting problem can be converted, in a systematic way, into an equivalent conic quadratic program T T d z ! min j kPiz + pik2 ≤ qi z + ri; i = 1; :::; m: (CQP) ♠ The resulting problem (CQP) can be approximated, in a polynomial time fashion, by a linear programming program dT z ! min j P z + p ≥ 0: (LP) 3 minimize cT x subject to 8 <> Ax = b (a) :> x ≥ 0 8 0 1 0 1 > > xi vi vi xi > @ A @ A > 0; 0; i = 1; :::; 8; > > vi v8+i xi v17 > > 16P > > vi ≤ v17; > i=9 > > 0 1 0 1 0 1 > > v v x v v v > @ 18 19 A @ 4 20 A @ 19 21 A > 0; 0; 0; > > v19 x2 v20 1 v21 x3 > 0 1 0 1 0 1 > > x4 v22 v21 v18 x1 v24 > @ A @ A @ A > 0; 0; 0; > > 0 v22 v20 1 0 v18 v22 1 0 v24 v23 1 > > x v v v x v > @ 6 25 A @ 23 26 A @ 5 27 A > 0; 0; 0; > > v25 1 v26 v24 v27 v25 > 0 1 > > v26 v23 > @ A > 0; > > v23 v27 > > > v17 ≤ v18 + 2v23; > > > > 0 1 0 1 0 1 <> x1 v29 v28 v30 v30 1 @ A 0; @ A 0; @ A 0; (b) > > v 1 v v 1 x > 0 29 1 0 30 291 0 2 1 > > x4 v32 x4 v33 x4 v34 > @ A @ A @ A > 0; 0; 0; > > v32 1 v33 v32 v34 v33 > 0 1 0 1 0 1 > > v31 v35 x2 v36 x4 v37 > @ A @ A @ A > 0; 0; 0; > v 1 v x v v > 0 35 1 0 36 3 1 0 37 34 1 > > v v v v v v > @ 35 38 A @ 36 39 A @ 38 40 A > 0; 0; 0; > > v38 v31 v39 v37 v40 v39 > 0 1 > > x3 1 > @ A > 0; > 1 v > 40 > > > v28 + 2v31 ≤ 5x2; > > > 0 1 > > x x > B 2 1 C > B C > B C > B x1 x4 x3 C > B C > B C 5I > B C > B x3 x6 x3 C > @ A :> x3 x8 4 8 0 1 > > x1 x2 − x1 x3 − x2 x4 − x3 > B C > B C > B C > B x2 − x1 x2 x3 − x2 x4 − x3 C > B C > B C 0 > B C > B x3 − x2 x3 − x2 x3 x4 − x3 C > @ A > > x4 − x3 x4 − x3 x4 − x3 x4 > 0 1 > > x1 0 v41 v42 v43 v44 v45 v46 v47 > B C > B C > B C > B 0 v48 −v42 v49 v50 v51 v52 v53 v54 C > B C > B C > B v −v v v v v v v v C > B 41 42 55 56 57 58 59 60 61 C > B C > B C > B v42 v49 v56 v62 v63 v64 v65 v66 v67 C > B C > B C > B C > B v43 v50 v57 v63 v68 v69 v70 v71 v72 C 0; > B C > B C > B v v v v v v v v v C > B 44 51 58 64 69 73 74 75 76 C > B C > B C > B v45 v52 v59 v65 v70 v74 v77 −v76 v78 C > B C > B C > B C > B v46 v53 v60 v66 v71 v75 −v76 v79 0 C <> @ A (c) v47 v54 v61 v67 v72 v76 v78 0 v80 > > > 2v + v − 8x − 2x − 4x − 8x = 0 > 41 48 1 2 3 4 > > > 2v43 + 2v49 + v55 − 32x1 − 14x2 − 28x3 − 56x4 = 0 > > > > v44 + v50 + v56 = 0 > > > 2v + 2v + 2v + v − 80x − 48x − 88x − 112x = 0 > 45 51 57 62 1 2 3 4 > > > v46 + v52 + v58 + v63 = 0 > > > > 2v47 + 2v53 + 2v59 + 2v64 + v68 − 136x1 − 100x2 − 160x3 = 0 > > > v + v + v + v = 0 > 54 60 65 69 > > > 2v61 + 2v66 + 2v70 + v73 − 160x1 − 136x2 − 176x3 + 224x4 = 0 > > > > v67 + v71 + v74 = 0 > > > 2v + 2v + v − 128x − 120x − 112x + 224x = 0 > 72 75 77 1 2 3 4 > > > 2v78 + v79 − 64x1 − 64x2 − 32x3 + 64x4 = 0 > > : v80 − 16x1 − 16x2 = 0 5 What can be expressed via CQP and SDP? | Let us look at three generic families of convex pro- grams: ♠ Linear Programming: cT x ! min j Ax + b ≥ 0 (LP) ♠ Conic Quadratic Programming: T T c x ! min j kAix + bik2 ≤ ci x + di; i = 1; :::; m (CQP) ♠ Semidefinite Programming: n T X c x ! min j xiAi + B 0 (SDP) i=1 | Geometrically, all these problems are of the form cT x ! min j Ax + b 2 K; (CP) where K is a closed pointed convex cone with a nonempty interior belonging to a specific for the generic problem in question family K of convex cones. 6 cT x ! min j Ax + b 2 K [K 2 K] (CP) LP: K is comprised of direct products of rays R+ CQP: K is comprised of direct products of the Lorentz cones n n+1 L = f(x; t) 2 R : kxk2 ≤ tg SDP: K is comprised of direct products of the semidefinite cones n n S+ = fA 2 S : A 0g Note: The above families of cones are closed w.r.t. (1) taking (finite) direct products of cones; (2) passing from a cone K to its dual cone T K∗ = fη j η x ≥ 0 8x 2 Kg: 7 | Assume that we are given a family K of finite- dimensional convex cones (closed, pointed and with a nonempty interior) and know how to solve problems of the form cT y ! min j Ay + b 2 K (∗) with all possible data (c; A; b) and all K 2 K. Question: What is the family of problems we can actually solve? When an optimization problem eT x ! min j x 2 X ⊂ Rn (P) can be equivalently reformulated in the form of (∗)? An answer: This is the case when X is a K-representable set (K-r.s.), i.e., there exists a K-representation (K-r.) of X X = x 2 Rn j 9u 2 Rk : P x + Qu + b 2 K (R) [K 2 K] Indeed, given the data P; Q; b; K of (R), we can rewrite (P) equivalently in the form of (∗): dT x ! min j x 2 X m cT y ≡ dT x ! min j Ay + b ≡ P x + Qu + b 2 K 2 0 13 x 4y = @ A5 u 8 An example: Consider the optimization problem 0 1 B x C B C B C t ! min j xyzt ≥ 1;A B y C + b ≥ 0; x; y; z ≥ 0 @ A z m (∗) 0 1 B x C B C 1 B C ! min j x; y; z ≥ 0;A B y C + g ≥ 0: xyz @ A z It turns out that the feasible set of this problem is semidefinite representable: 0 1 B x C B C B C xyzt ≥ 1 & x; y; z ≥ 0 & A B y C + b ≥ 0 @ A z m 8 2 > > 6 9u1; u2 : > 6 0 1 0 1 > 6 > 6 x u z u > 6 @ 1 A @ 2 A > 6 (a) 0; 0 > 6 > 6 u1 y u2 t > 6 | {z } > 6 > (1) 6 2 2 > 6 say that xy ≥ u1, zt ≥ u2 and x; y; z; t ≥ 0 > 6 0 1 > 6 > 6 u1 1 < 6 @ A 6 (b) 0 > 6 1 u2 > 4 | {z } > > says that u u ≥ 1 and u ; u ≥ 0 > 1 2 1 2 > > > [(1) says exactly that xyzt ≥ 1 & x; y; z ≥ 0] > 0 1 > > x > B C > B C > B C > (2) A B y C + b ≥ 0 > @ A :> z Thus, (*) is equivalent to the semidefinite problem 0 1 0 1 0 1 0 1 B x C x u z u u 1 B C @ 1 A @ 2 A @ 1 A B C t ! min j 0; 0; 0;A B y C+b ≥ 0: u y u t 1 u @ A 1 2 2 z 9 | We see that a natural interpretation of the question Given a possibility to solve problems cT x ! min j Ax + b 2 K 2 K what can we actually solve? is ♠ What are K-representable sets? ♠ How to recognize K-representability? 10 | Claim: Consider a family K of finite-dimensional closed pointed cones with a nonempty interior, and let this family be closed w.r.t.