A computational practicability study of MIQCQP reformulations Sven Wiese1 MOSEK ApS, Fruebjergvej 3, Symbion Science Park, 2100 Copenhagen, Denmark [email protected] Abstract. We report on computational results regarding several refor- mulation methods for MIQCQPs. The focus is set on the computational practicability of a reformulation, and we include challenging instances in our experiments. We are especially, though not exclusively, interested in reformulations based on SDPs and pursue the question of how such reformulations can be useful in an off-the-shelf solver. Keywords: MIQCQP · Eienvalue-method · Diagonal perturbation · SDP relaxation 1 Introduction Most authors use the term Mixed Integer Quadratic Programming (MIQP) for the minimization of a quadratic form over linear constraints imposed on mixed- integer variables. When also the constraints are allowed to contain quadratic terms, we usually talk about Mixed Integer Quadratically-Constrained Quadratic Programming (MIQCQP). The canonical form we use for describing such a prob- lem throughout this paper is min xT Q0x + cT x s.t. xT Qkx + aT x ≤ b ; k = 1; : : : ; m k k (P) l ≤ x ≤ u x 2 Zp × Rn−p: A decisive question about (P) is whether all involved matrices Qk, that are w.l.o.g. assumed to be symmetric, are positive semidefinite (p.s.d.) or not. If so, it is well known that the continuous relaxation of (P) is a convex optimization problem, thus solvable in polynomial time to any desired accuracy. For solv- ing (P), we may employ non-linear Branch-and-Bound or Outer-Approximation methods in this case. Although (P) is not a convex optimization problem it- self due to the presence of integer variables, we then call it a convex MIQCQP, implicitly referring to its continuous relaxation. If the latter is non-convex instead, we call it a non-convex MIQCQP. In that case we usually have to resort to spatial branching in order to solve it, increasing the computational burden. Yet, it is sometimes possible to translate the non-convexity encoded in the Q-matrices to the integrality of variables, and 2 Sven Wiese we end up with a reformulation of the problem that is a convex MIQCQP, or even a Mixed Integer Linear Program (MILP). Such reformulations and their implementation, with a special focus on practicability, are the topic of this paper. Note that such reformulations can be useful also when (P) is already convex, as we will shortly discuss further below. Two recurring themes in the literature for solving (MI)QCQPs are given by the reformulation-linearization (RLT) technique [5], and by the exploitation of Semidefinite Programming (SDP) relaxations. Our study is largely inspired by the line of research in [9, 7, 8, 14], that can be roughly described as a means of finding suitable perturbations of the matrices Q with desirable properties, again based on SDP-techniques. Perturbations that are restricted to the diagonals of the involved matrices are studied in [12, 13]. The recent paper [11] addresses the issue of choosing automatically between some of the reformulations we treat further below by means of machine learning. Many commercial and non-commercial solvers that allow for mixed inte- ger variables, i.e., Mixed Integer Programming (MIP) solvers, can handle some form of MIQCQP nowadays. Global optimization solvers like ANTIGONE [18], BARON [19], Couenne [6] and SCIP [20] accept both convex and non-convex MIQCQPs. Also CPLEX [3], and more recently Gurobi [2] extended their ca- pability of solving convex MIQCQPs to the non-convex case. [10] provides an overview of the various techniques that contribute to the solution of MIQCQPs in such a software package. FICO Xpress [1] can be used to solve convex MIQCQPs, and non-convex ones are accepted and solved in a heuristic fashion. MOSEK [4] accepts convex MIQCQP in the current release 9.2. The results in Section 3 with non-convex MIQCQPs were obtained using MOSEK and showcase how it will be extended in a future release. A recently vivid topic has been the conversion of convex quadratic constraints to conic form. For Q p.s.d., it is well-known that we can always find a factoriza- tion Q = F T F with some matrix F , leading to the identity T 2 x Qx = kF xk2: (1) Replacing all quadratic forms in (P) with the above right-hand side for some ap- propriate factor F k will transform (P) into a Mixed Integer Second-Order Cone Program (MISOCP). Some of the above solvers accept second-order constraints, or maybe even perform the conversion internally. A discussion about the advan- tages or disadvantages of the transformation of quadratic constraints to conic form is, nevertheless, outside the scope of this paper. The remainder is organized in two sections: In Section 2 we discuss in more detail the various reformulation methods that form the basis of the computa- tional results presented in Section 3. The contributions are twofold: { We present computational results comparing these approaches in a unified computational environment, with a special focus on practical implementa- tions. We include instances in our experiments that to the best of our knowl- edge have not been part of previous computational studies on the subject. A computational practicability study of MIQCQP reformulations 3 { Some of the reformulation methods are based on the solution of SDPs, and to the best of our knowledge there is so far no other MIP solver employing such reformulations. We thus try to answer the question of how useful SPD-based reformulations of MIQCQPs can be in such a software package. 2 Reformulation methods In order to outline the various reformulation approaches, we largely base our notation on [8]. The authors therein introduce an infinite family of reformulations of (P), parametrized in the matrices P 0;:::;P m, as follows: min xT (Q0 + P 0)x + cT x − hP 0;Xi T k k T k s.t. x (Q + P )x + ak x − hP ;Xi ≤ bk; k = 1; : : : ; m T ~ X = xx (RP 0;:::;P m ) l ≤ x ≤ u x 2 Zp × Rn−p; n×n T where X 2 R and X = xx encodes the product relations Xij = xixj. The idea here is to perturb each Qk with some matrix P k, and we are of course interested in those perturbations such that Qk + P k are p.s.d. for all k. If so, ~ the new sources of non-convexity in (RP 0;:::;P m ) are the integrality of variables, and the constraint X = xxT . The latter can, under certain circumstances, again be reformulated so as to translate this non-convexity to the integrality of the variables. The easiest case where this can be seen is when i = j and xi is a binary variable. Then clearly 2 xi = xi; (2) or in other words Xii = xi, which is why it is not even necessary to introduce the variable Xii explicitly. A more difficult case is when still at least one of the two variables in the product xixj, say xi, is binary, and the other has finite bounds lj ≤ xj ≤ uj. Then Xij = xixj is equivalent to ljxi ≤ Xij ≤ ujxi; (3) xj − uj(1 − xi) ≤ Xij ≤ xj − lj(1 − xi): (4) If none of the two variables is binary instead, but at least one is integer with finite bounds, say again li ≤ xi ≤ ui, it can be written via its binary expansion blog(ui−li)c X k xi = 2 tik + li (5) k=0 with auxiliary binary variables tik. Therewith we can rewrite the product blog(ui−li)c X k xixj = 2 tikxj + lixj; (6) k=0 4 Sven Wiese and subsequently introduce yet more auxiliary variables zjik = tikxj. Since these products contain a binary variable now, they can be rewritten using the same idea as in (3) - (4), leading to an extended but linear description of Xij = xixj. k Denote by L = f(i; j) j 9k : pi;j 6= 0g the set of all variable pairs that are ~ perturbed with a non-zero coefficient in (RP 0;:::;P m ), and by L1 ⊆ L those pairs that can be linearized by (2) - (6). For ease of exposition, we assume that L = L1 throughout the rest of this section. We denote by Sx;X;t;z the mixed-integer set collecting all the relations (2) - (6) for the pairs in L, and can thus write the next step in our reformulation process as min xT (Q0 + P 0)x + cT x − hP 0;Xi T k k T k s.t. x (Q + P )x + ak x − hP ;Xi ≤ bk; k = 1; : : : ; m (x; X; t; z) 2 Sx;X;t;z (RP 0;:::;P m ) l ≤ x ≤ u x 2 Zp × Rn−p: (RP 0;:::;P m ) is the generic reformulation we work with, and below we list four different choices for the matrices P k. Remark 1. It can happen that L0 := L n L1 6= ;, for example in the presence of products between two continuous variables, or when required bounds are not finite. One might also resort to a policy of not performing the linearizations (3) - (6) when either max(jljj; jujj) or ui − li are larger than specified constants. In Section 3 we allow that L 6= ;, provided that P qk x x ≥ 0 for all k. 0 i;j2L0 ij i j k Also, we restrict to reformmulations such that pij = 0 for all i; j 2 L0 and all k in such a case. In other words, we never perturb the coefficient of a product that is not linearized, and the non-linearized residual of each Qk is required to be p.s.d.