Parametric Convex Quadratic Relaxation of the Quadratic Knapsack Problem

European Journal of Operational Research 281 (2020) 36–49

Contents lists available at ScienceDirect

European Journal of Operational Research

journal homepage: www.elsevier.com/locate/ejor

Discrete Optimization

Parametric convex quadratic relaxation of the quadratic knapsack problem

∗ M. Fampa a, , D. Lubke a, F. Wang b, H. Wolkowicz c a PESC/COPPE, Universidade Federal do Rio de Janeiro, Caixa Postal 68511, Rio de Janeiro, RJ21941-972, Brazil b Department of Mathematics, Royal Institute of Technology, Sweden c Department of Combinatorics and Optimization, University of Waterloo, Waterloo, Canada

a r t i c l e i n f o a b s t r a c t

Article history: We consider a parametric convex quadratic programming (CQP) relaxation for the quadratic knapsack Received 20 January 2019 problem (QKP). This relaxation maintains partial quadratic information from the original QKP by per-

Accepted 16 August 2019 turbing the objective function to obtain a concave quadratic term. The nonconcave part generated by Available online 20 August 2019 the perturbation is then linearized by a standard approach that lifts the problem to matrix space. We

Keywords: present a primal-dual interior point method to optimize the perturbation of the quadratic function, in a Quadratic knapsack problem search for the tightest upper bound for the QKP. We prove that the same perturbation approach, when Quadratic binary programming applied in the context of semideﬁnite programming (SDP) relaxations of the QKP, cannot improve the Convex quadratic programming relaxations upper bound given by the corresponding linear SDP relaxation. The result also applies to more general Parametric optimization integer quadratic problems. Finally, we propose new valid inequalities on the lifted matrix variable, de-

Valid inequalities rived from cover and knapsack inequalities for the QKP, and present separation problems to generate cuts for the current solution of the CQP relaxation. Our best bounds are obtained alternating between optimizing the parametric quadratic relaxation over the perturbation and applying cutting planes generated by the valid inequalities proposed. ©2019 Elsevier B.V. All rights reserved.

1. Introduction is a generalization of the knapsack problem, which has the same feasible set of the QKP, and a linear objective function in x . The We study a convex quadratic programming (CQP) relaxation of linear knapsack problem can be solved in pseudo-polynomial time the quadratic knapsack problem (QKP), using dynamic programming approaches with complexity of O ( nc ). ∗ The QKP appears in a wide variety of fields, such as biol- p := max x T Qx QKP ogy, logistics, capital budgeting, telecommunications and graph ( QKP ) s.t. w T x ≤ c (1) theory, and has received a lot of attentio0n in the last decades. x ∈ { 0 , 1 } n , Several papers have proposed branch-and-bound algorithms for where Q ∈ S n is a symmetric n × n nonnegative integer profit ma- the QKP, and the main difference between them is the method n trix, w ∈ Z ++ is a vector of positive integer weights for the items, used to obtain upper bounds for the subproblems ( Billionnet & ∈ Z ≥ , ∈ = and c ++ is the knapsack capacity with c wi for all i N : Calmels, 1996; Billionnet, Faye, & Soutif, 1999; Caprara, Pisinger, & { 1 , . . . , n } . The binary (vector) variable x indicates which items are Toth, 1999; Chaillou, Hansen, & Mahieu, 1989; Helmberg, Rendl, & chosen for the knapsack, and the inequality in the model, known Weismantel, 1996; 20 0 0 ). The well known trade-off between the as a knapsack inequality, ensures that the selection of items does strength of the bounds and the computational effort required to not exceed the knapsack capacity. We note that any linear costs in obtain them is intensively discussed in Pisinger (2007) , where the objective can be included on the diagonal of Q by exploiting semidefinite programming (SDP) relaxations proposed in Helmberg the {0, 1} constraints and, therefore, are not considered. et al. (1996) and Helmberg, Rendl, and Weismantel (20 0 0) are The QKP was introduced in Gallo, Hammer, and Simeone presented as the strongest relaxations for the QKP. The linear (1980) and was proved to be NP-Hard in the strong sense by re- programming (LP) relaxation proposed in Billionnet and Calmels duction from the clique problem. The quadratic knapsack problem (1996) , on the other side, is presented as the most computation- ally inexpensive.

∗ Both the SDP and the LP relaxations have a common fea- Corresponding author.

E-mail addresses: [email protected] (M. Fampa), [email protected] ture, they are deﬁned in the symmetric matrix lifted space T (D. Lubke), [email protected] (F. Wang), [email protected] (H. Wolkowicz). determined by the equation X = xx , and by the replacement of https://doi.org/10.1016/j.ejor.2019.08.027 0377-2217/© 2019 Elsevier B.V. All rights reserved. M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49 37 the quadratic objective function in QKP with a linear function in Table 1

X , namely, trace( QX ). As the constraint X = xx T is nonconvex, it is Equations number corresponding to acronyms. relaxed by convex constraints in the relaxations. The well known QKP (1) CI (27)

McCormick inequalities ( McCormick, 1976 ), and also the semidef- QKPlifted (2) ECI (28) inite constraint, X − xx T 0 , have been extensively used to relax LPR (3) LCI (29) CQP (5) SCI (32) = T , Qp the nonconvex constraint X xx in relaxations of the QKP. LSDP (21) CILS (34) In this paper, we investigate a CQP relaxation for the QKP, QSDPQ p (15) SCILS (36) where instead of linearizing the objective function, we perturb SKILS (41) the objective function Hessian Q , and maintain the (concave) perturbed version of the quadratic function in the objective, linearizing only the remaining part derived from the perturbation. Our X − xx T 0 . An IPM could also be applied to this parametric prob- relaxation is a parametric convex quadratic problem, deﬁned as lem in order to generate the best possible bound. Nevertheless, − a function of a matrix parameter Qp , such that Q Qp 0. This we prove an interesting result concerning the relaxations, in case matrix parameter is iteratively optimized by a primal-dual interior the constraint X − xx T 0 is imposed: the tightest bound gener- point method (IPM) to generate the best possible bound for the ated by the parametric quadratic SDP relaxation is obtained when QKP. During this iterative procedure, valid cuts are added to the the perturbation Q p is equal to Q , or equivalently, when we lin- formulation to strengthen the relaxation, and the search for the earize the entire objective function, obtaining the standard linear best perturbation is adapted accordingly. Our procedure alternates SDP relaxation. We conclude, therefore, that keeping the (concave) between optimizing the matrix parameter and applying cutting perturbed version of the quadratic function in the objective of the planes generated by valid inequalities. At each iteration of the SDP relaxation does not lead to a tighter bound. A result that could procedure, a new bound for the QKP is computed, considering be derived from our analysis is that the CQP relaxation cannot gen- the updated matrix parameter and the cuts already added to the erate a tighter bound than the standard linear SDP relaxation. This relaxation. result was already proved in Billionnet et al. (2016) , and we show

In Billionnet, Elloumi, and Lambert (2016) (see also Billionnet, that it still holds for unbounded and nonconvex feasible set.

Elloumi, & Lambert, 2012; Billionnet, Elloumi, & Plateau, 2009 for Another contribution of this work is the development of valid previous results), a similar parametric convex quadratic problem inequalities for the CQP relaxation on the lifted matrix variable. was investigated for the more general problem of minimizing a The inequalities are ﬁrst derived from cover inequalities for the quadratic function of bounded integer variables subject to a set of knapsack problem. The idea is then extended to knapsack inequali- quadratic constraints. However, the authors consider a unique per- ties. Taking advantage of the lifting X := xx T , we propose new valid turbation of the Hessian of each quadratic function in the model, inequalities that can also be applied to more general relaxations to reformulate it as a mixed integer quadratic programming (MIQP) of binary quadratic programming problems that use the same lift- problem with a CQP continuous relaxation. They propose to solve ing. We discuss how cuts for the quadratic relaxation can be ob- the reformulated problem by an MIQP solver. To reformulate the tained by the solution of separation problems, and investigate pos- problem, the authors also seek the best possible perturbations of sible dominance relation between the inequalities proposed. the Hessians, which are considered as the ones, such that the solu- Finally, we present an algorithmic framework, where we iter- tion of the continuous relaxation of the MIQP is maximal. In other atively improve the upper bound for the QKP by optimizing the words, they seek perturbations that lead to the best bound at the choice of the perturbation of the objective function and adding root node of a branch-and-bound algorithm. The authors claim that cutting planes to the relaxation. At each iteration, lower bounds for these perturbations can be computed from an optimal dual solu- the problem are also generated from feasible solutions constructed tion of a standard SDP relaxation of the problem. However, the from a rank-one approximation of the solution of the CQP relax- dual SDP problem is not correctly formulated in the paper, and ation. the proof presented is not correct. In this paper, we prove the re- In Section 2 , we introduce our parametric convex quadratic result following the idea presented in Billionnet et al. (2009), which laxation for the QKP. In Section 3 , we explain how we optimize is based on Lemarchal and Oustry (1999, Theorem 4.4). Further- the parametric problem over the perturbation of the objective; more, we show that the result is valid to the more general problem i.e., we present the IPM applied to obtain the perturbation that where the feasible set is any bounded polyhedron with nonempty leads to the best possible bound. In Section 4 , we present our con- interior. We also note that, if no cuts are added to the relaxation clusion about the parametric quadratic SDP relaxation, and relate during the iterations of our interior point method, it becomes an our results to the results presented in Billionnet et al. (2016) . In alternative way of obtaining the optimal perturbation considered Section 5 , we introduce new valid inequalities on the lifted matrix in Billionnet et al. (2016). variable of the convex quadratic model, and we describe how cut-

Another similar approach to handle nonconvex quadratic func- ting planes are obtained by the solution of separation problems. In tions consists in decomposing it as a difference of convex (DC) Section 6 , we present the heuristic used to generate lower bounds quadratic function (Horst & Thoai, 1999). DC decompositions to the QKP. In Section 7 , we discuss our numerical experiments have been extensively used in the literature to generate convex and in Section 8 , we present our ﬁnal remarks. quadratic relaxations of nonconvex quadratic problems. See, for example, Fampa, Lee, and Melo (2017) and references therein. Unlike the approach used in DC decompositions, we do not necessarily Notation ∈ S n , decompose x TQx as a difference of convex functions, or equiva- If A then svec(A) is a vector whose entries come from A lently, as a sum of a convex and a concave function. Instead, fol- by stacking up its ‘lower half’, i.e., lowing the approach introduced in Billionnet et al. (2016) , we de- T n (n +1) / 2 svec (A ) := (a 11 , . . . , a n 1 , a 22 , . . . , a n 2 , . . . , a nn ) ∈ R . compose it as a sum of a concave function and a quadratic term derived from the perturbation applied to Q . This perturbation can The operator sMat is the inverse of svec, i.e., sMat ( svec (A )) = A . − λ be any symmetric matrix Qp , such that Q Qp 0. We also denote by min (A), the smallest eigenvalue of A and by λ In an attempt to obtain stronger bounds, we also investi- i (A) the ith largest eigenvalue of A. gated the parametric convex quadratic SDP problem, where we To facilitate the reading of the paper, Table 1 relates the add to our CQP relaxation, the positive semideﬁnite constraint acronyms used with the associated equations numbers. 38 M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49

Table 2 where Z ∈ S n and ∈ S n denote, respectively, the slack and the dual

List of abbreviations. symmetric matrix variables. We consider the Lagrangian function CQP Convex Quadratic Programming ∗ μ( , , ) = ( ) − μ + (( − + )) . QKP Quadratic Knapsack Problem L Qp Z : pCQP Qp log det Z trace Q Qp Z

SDP Semidefinite Programming Some important points should be emphasized here. We first MIQP Mixed Integer Quadratic Programming ∗ ( ) MILP Mixed Integer Linear Programming note that the objective function for pCQP Qp is linear in Qp, i.e., this function is the maximum of linear functions over feasible points x , X . Therefore, this is a convex function. We also show the standard abbreviations used in the paper in Moreover, as will be detailed next, the search direction of the Table 2 . IPM, computed at each iteration of the algorithm, depends on = ( ) , = ( ) , the optimum solution x x Qp X X Qp of CQPQ p for a fixed 2. A parametric convex quadratic relaxation matrix Q p . At each iteration of the IPM, we have Z 0, and there- − ≺ fore Q Qp 0. Thus, problem CQPQ p maximizes a strictly con- In order to construct a convex relaxation for QKP , we start by cave quadratic function, subject to linear constraints over a com- considering the following standard reformulation of the problem pact set P, and consequently, has a unique optimal solution (see in the lifted space of symmetric matrices, defined by the lifting e.g. Turlach & Wright, 2015 ). From standard sensitivity analysis re- X := xx T . sults, e.g. Fiacco (1983 , Corollary 3.4.2), Hogan (1973) , and Danskin ∗ = ( ) (1966 , Theorem 1), as the optimal solution x = x (Q p ) , X = X(Q p ) is pQKP : max trace QX lifted ∗ ( ) s.t. w T x ≤ c unique, the function pCQP Qp is differentiable, and therefore, the ( QKP ) (2) lifted X = xx T Lagrangian function is also differentiable. Since Q appears only in the objective function in CQP , and x ∈ { 0 , 1 } n. p Qp T T T We consider an initial LP relaxation of QKP , given by x (Q − Q p ) x + trace (Q p X ) = x Qx + trace (Q p (X − xx )) , max trace (QX ) ( LPR ) (3) we have ( , ) ∈ P, s.t. x X ∗ T ∇ p CQP (Q p ) = X − xx . (8) where P ⊂ [0 , 1] n × S n is a bounded polyhedron, such that

The optimality conditions for (7) are obtained by differentiating { (x, X ) : w T x ≤ c, X = xx T , x ∈ { 0 , 1 } n} ⊂ P. the Lagrangian L μ with respect to Q p , , Z , respectively,

∂ 2.1. The perturbation of the quadratic objective Lμ ∗ : ∇ p CQP (Q p ) − = 0 , ∂Q p Next, we propose a convex quadratic relaxation with the same ∂L μ − + = , (9) feasible set as LPR , but maintaining a concave perturbed version of ∂ : Q Qp Z 0 the quadratic objective function of QKP, and linearizing only the ∂ Lμ − remaining nonconcave part derived from the perturbation. More : −μZ 1 + = 0 , ( or ) Z − μI = 0 . ∂ n Z speciﬁcally, we choose Q p ∈ S such that This gives rise to the nonlinear system Q − Q p 0 , (4) ⎛ ⎞ ∗ ∇ p (Q ) − and we get CQP p G μ(Q , , Z) = ⎝ Q − Q + Z ⎠ = 0 , Z, 0 . (10) T T T T T p p x Qx = x (Q − Q ) x + x Q x = x (Q − Q ) x + trace (Q xx ) p p p p Z − μI T = x (Q − Q p ) x + trace (Q p X ) . ∗ , We use a BFGS approximation for the Hessian of pCQP since it Finally, we deﬁne the parametric convex quadratic relaxation of is not guaranteed to be twice differentiable everywhere, and up- QKP : date it at each iteration (see Lewis & Overton, 2013 ). We denote 2 ∗ ∗ the approximation of ∇ p (Q ) by B , and begin with the ap- p (Q ) := max x T (Q − Q ) x + trace (Q X ) BFGS CQP p ( ) CQP p p p + CQPQ = k, k 1 p s.t. (x, X ) ∈ P. proximation B0 I. Recall that if Qp Qp are two successive it- + ∇ ∗ ( k ) , ∇ ∗ ( k 1 ) , (5) erates with gradients pCQP Qp pCQP Qp respectively, with ∈ S n (n +1) / 2 , current Hessian approximation Bk then we set 3. Optimizing the parametric problem over the parameter Q ∗ + ∗ + p = ∇ ( k 1 ) − ∇ ( k ) , = k 1 − k, Yk : pCQP Qp pCQP Qp Sk : Qp Qp ∗ ( ) The upper bound pCQP Qp in the convex quadratic problem and,

CQPQ depends on the feasible perturbation Qp of the Hessian Q. p υ := Y , S , ω := svec (S ) , B svec (S ) . To find the best upper bound, we consider the parametric problem k k k k k Finally, we update the Hessian approximation with ∗ ∗ = ( ) . paramQKP : min pCQP Qp (6) 1 1 Q−Q p 0 T T B + := B + svec (Y ) svec (Y ) − B svec (S ) svec (S ) B . k 1 k υ k k ω k k k k We solve (6) with a primal-dual interior-point method (IPM), and υ > we describe in this section how the search direction of the algo- We note that the curvature condition 0 should be verified rithm is obtained at each iteration. to guarantee the positive definiteness of the updated Hessian. In We start with minimizing a log-barrier function. We use the our implementation, we address this by skipping the BFGS update when υ is negative or too close to zero. barrier function, B μ( Q p , Z ) with barrier parameter, μ > 0, to obtain the barrier problem The equation for the search direction is ∗ min B μ(Q , Z) := p (Q ) − μ log det Z Q p p CQP p ( , , ) = − μ( , , ) , s.t. Q − Q p + Z = 0 (: ) (7) Gμ Qp Z G Qp Z (11) Z 0 , Z M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49 39 where Algorithm 1 Updating the perturbation Q p . ∗ ∇ ∗ ( ) − k k k ( k ) ( k ) ∇ ( k ) μk τ = pCQP Qp Rd Input: k, Qp , Z , , x Qp , X Qp , pCQP Qp , Bk , , α : . τ = . G μ(Q p , , Z) = Q − Q p + Z =: R p . (12) 0 95, μ : 0 9.

Z − μI R c Compute the residuals: ⎛ ⎞ ∗ ∇ ( k ) − k If B is the current estimate of the Hessian, then (11) becomes Rd pCQP Qp = ⎝ − k + k ⎠ . Rp : Q Qp Z ( ( )) − = − , k k k sMat B svec Qp Rd Rc Z − μ I −Q p + Z = −R p , Solve the linear system for Q : Z + Z = −R c . p k k k k Z sMat (B svec (Q )) + (Q ) = −R − Z R + R . We can substitute for the variables and Z in the third equa- k p p c d p tion of the system. The elimination gives us a simpliﬁed system, Set: and therefore, we apply it, using the following two equations for := sMat (B svec (Q )) + R , Z := −R + Q . elimination and backsolving, k p d p p Update Q p , Z and : = sMat (B svec (Q )) + R , Z = −R + Q . (13) p d p p + + + k 1 = k + α , k 1 = k + α , k 1 = k + α , Qp : Qp ˆ p Qp Z : Zp ˆ p Z : ˆ d Accordingly, we have a single equation to solve, and the system where ﬁnally becomes

α = τα × { , { k + α }} , ˆ p : min 1 argmaxαp Zp p Z 0 ( ( )) + ( ) = − − + . Z sMat B svec Qp Qp Rc ZRd Rp k αˆ := τα × min{ 1 , argmaxα { + α 0 }} . d d d We emphasize that to compute the search direction at each it- + + eration of our IPM, we need to update the residuals deﬁned in (12) , ( k 1 ) ( k 1 ) Obtain the optimal solution x Qp , X Qp of relaxation and therefore we need the optimal solution x = x (Q p ) , X = X(Q p ) k +1 CQP , where Q p := Q . of the convex quadratic relaxation CQP for the current perturba- Qp p Q p ∗ Update the gradient of pCQP : tion Qp. Problem CQPQ p is thus solved at each iteration of the IPM ∗ + + + + method, each time for a new perturbation Q p , such that Q − Q p ≺ ∇ ( k 1 ) = ( k 1 ) − ( k 1 ) ( k 1 ) T . pCQP Qp : X Qp x Qp x Qp 0 . ∗ υ > In Algorithm 1 , we present in details an iteration of the IPM. Update the Hessian approximation of pCQP (if 0): The algorithm is part of the complete framework used to generate ∗ k +1 ∗ k k +1 k Y := ∇ p (Q ) − ∇ p (Q ) , S := Q − Q , bounds for QKP , as described in Section 7 . k CQP p CQP p k p p υ = , , ω = ( ) , ( ) , : Yk Sk : svec Sk Bk svec Sk Remark 1. Algorithm is an interior-point method with a quasi- 1 T B + := B + svec (Y ) svec (Y ) Newton step (BFGS). The object function we are minimizing is k 1 k υ k k differentiable with exception possibly at the optimum. A com- 1 T plete convergence analysis of the algorithm is not in the scope of − B svec (S ) svec (S ) B . ω k k k k this paper, however, convergence analysis for some similar problems can be found in the literature. In Armand, Gilbert, and Jégou Update μ: (20 0 0) , it is shown that if the objective function is always differen- k +1 k +1 tiable and strongly convex, then it is globally convergent to the an- + trace (Z ) μk 1 := τμ . alytic center of the primal-dual optimal set when μ tends to zero n and strict complementarity holds. + + + + k 1 k +1 k +1 ( k 1 ) ( k 1 ) ∇ ∗ ( k 1 ) Output: Qp , Z , , x Qp , X Qp , pCQP Qp , μk +1 Bk +1 , . 4. The parametric quadratic SDP relaxation

In an attempt to obtain tighter bounds, a promising approach might seem to be to include the positive semideﬁnite constraint We now consider the parametric SDP problem given by T X − xx 0 in our parametric quadratic relaxation CQP , and ∗ Qp = T ( − ) + ( ) pQSDP : sup x Q Qp x trace Qp X solve a parametric convex quadratic SDP relaxation, also using Qp ( QSDP ) ( , ) ∈ F an IPM. Nevertheless, we show in this section that the convex Qp s.t. x X − T , quadratic SDP relaxation cannot generate a better bound than the X xx 0 linear SDP relaxation, obtained when we set Q p equal to Q . In fact, (15) as shown below, the result applies not only to the QKP, but to more where Q − Q 0 . general problems as well. We emphasize here that the same result p n n does not apply for CQPQ p . We could observe with our computa- Theorem 2. Let F be any subset of R × S . For any choice of matrix tional experiments that the best bounds were obtained by CQP , Q p Q p satisfying Q − Q p 0 , we have − = , when we had Q Qp 0 for all instances considered. ∗ ∗ p QSDP ≥ p LSDP . (16) Consider the linear SDP problem given by Q p

∗ { ∗ − } = ∗ = ( ) Moreover, inf pQSDP : Q Qp 0 pLSDP . pLSDP : sup trace QX Q p ( LSDP ) s.t. (x, X ) ∈ F (14) ( ) X − xx T 0 , Proof. Let x˜, X˜ be a feasible solution for LSDP. We have ∗ ≥ T ( − ) + ( ˜ ) n n n n pQSDP x˜ Q Qp x˜ trace Qp X (17) where x ∈ R , X ∈ S , and F is any subset of R × S . Qp 40 M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49

T T L (x, X, y ) := x Q x + trace (Q X ) = trace ((Q − Q p )( x˜x ˜ − X˜ )) + trace ((Q − Q p ) X˜ ) + trace (Q p X˜ ) n p (18) q

+ T ( − ( ) − γ T ) . yk bk trace k X k x = T k 1 = trace ((Q − Q p )( x˜x ˜ − X˜ )) + trace (Q X˜ ) (19) The Lagrangian dual function is then deﬁned as

≥ trace (Q X˜ ) . (20) g(y ) := max L (x, X, y ) x,X q

The inequality (17) holds because ( x˜, X˜ ) is also a feasible solution = T + ( ) + ( − ( ) − γ T ) max x Qn x trace Qp X yk bk trace k X k x x,X = for QSDPQ p . The inequality in (20) holds because of the negative k 1 ∗ q q q − T − ˜ semideﬁniteness of Q Qp and x˜x˜ X. Because pQSDP is an up- = − + T − γ T + Qp max trace Qp yk k X x Qn x yk k x yk bk x,X per bound on the objective value of LSDP at any feasible solu- ⎧ k =1 k =1 k =1 ∗ ∗ q q q tion, we can conclude that p ≥ p . Clearly, Q = Q satis- ⎨ QSDP LSDP p x T Q x − y γ T x + y b , if Q − y = 0 , Qp = max n k k k k p k k = = = ﬁes Q − Q p = 0 0 and LSDP is the same as QSDP for this choice x ⎩ k 1 k 1 k 1 Qp + ∞ , otherwise. { ∗ − } = ∗ ⎧ of Qp . Therefore, inf pQSDP : Q Qp 0 pLSDP . Q p ⎪ q q ⎪ ⎪− 1 γ T † γ + T , ⎪ yk k Qn yk k y b Notice that in Theorem 2 we do not require that F be convex ⎨ 4 k =1 k =1 = nor bounded. Also, in principle, for some choices of Q p , we could q q ∗ ∗ ⎪ − = γ ∈ ( ) , have p = + ∞ with p = + ∞ or not. ⎪ if Qp yk k 0 and yk k range Qn QSDP LSDP ⎪ Qp ⎩ k =1 k =1 + ∞ , otherwise, Remark 3. As a corollary from Theorem 2 , we have that the upper , bound for QKP given by the solution of the quadratic relaxation † where Qn is the pseudo-inverse of Qn . CQP , cannot be smaller than the upper bound given by the so- Qp As F has nonempty interior, the Slater condition holds for the lution of the SDP relaxation obtained from it, by adding the SDP inner maximization problem in (22) . Therefore, problem (22) has constraint X − xx T 0 and setting Q equal to Q . p the same optimal value as In Billionnet et al. (2016 , Theorem 1), this result was already proven for the case where F is a particular convex set. The authors q q also claim that in this particular case, the best bound obtained by 1 T † T min − y γ Q y γ + y b , , 4 k k n k k the CQP relaxation is exactly the same as the bound given by the Qp Qn y k =1 k =1 linear SDP relaxation, and that the best perturbation can be de- q − = , rived from the optimal dual variables of the SDP problem. Never- s.t. Qp yk k 0 theless the proof presented in Billionnet et al. (2016) is based on k =1 an incorrect formulation of the dual SDP problem. In the follow- q γ ∈ ( ) , ing, we prove that the result holds, following the same idea used yk k range Qn = in Billionnet et al. (2009) , which is based on Lemarchal and Oustry k 1 Q = Q − Q 0 , y ≥ 0 , (1999 , Theorem 4.4). Furthermore, we show that the result is valid n p to the more general problem where the feasible set is any bounded which is equivalent to polyhedron with nonempty interior. The result then applies to the

CQP relaxations that we use in this work. − max t , , t Qp y Theorem 4. Consider F ⊂ R n × S n as a bounded polyhedron with q q T nonempty interior, deﬁned by: 1 γ T ( − ) † γ − − ≥ , s.t. 4 yk k Q Qp yk k y b t 0

F = { ( , ) ( ) + γ T ≤ , = , . . . , } , k =1 k =1 : x X : trace k X k x bk k 1 q Q − Q 0 , p ∈ S n γ ∈ R n, = , . . . , q where k and k for k 1 q. γ ∈ ( − ) , Deﬁne yk k range Q Qp = ∗ T k 1 p polCQP (Q p ) := max{ x (Q − Q p ) x + trace (Q p X ) : (x, X ) ∈ F} , q − = , Qp yk k 0 where Q − Q p 0 , and k =1 ∗ T ≥ . p polSDP := max { trace (QX ) : (x, X ) ∈ F, X − xx 0 } . (21) y 0

Then By Schur complement ( Horn & Zhang, 2005 ), this is equivalent to ∗ ∗ the following SDP problem min p polCQP (Q p ) = p polSDP . Q−Q p 0 − max t Proof. First we consider the parametric problem , , t Qp y ⎡ ⎤ q ∗ ( ) min ppolCQP Qp − − T γ T / − ⎢ t y b yk k 2⎥ Q Qp 0 ⎢ ⎥ T = = min max x (Q − Q p ) x + trace (Q p X ) ⎢ k 1 ⎥ , − , s.t. ⎢ ⎥ 0 Q Qp 0 x X q T ⎣ ⎦ s.t. trace ( X ) + γ x ≤ b , k = 1 , . . . , q. γ / − k k k yk k 2 Qp Q ( ≥ ) : yk 0 k =1 q (22) − = , Qp yk k 0 Considering Q n := Q n (Q p ) := Q − Q p , the Lagrangian function of k =1 the inner maximization problem in (22) is deﬁned as y ≥ 0 . M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49 41

After substitution, this is equivalent to is a valid inequality for QKPlifted . In this case, we say that (26) is

min −t a valid inequality for QKPlifted derived from the valid inequality t,y ⎡ ⎤ (25) for QKP . q

− − T γ T / ⎢ t y b yk k 2⎥ ⎢ ⎥ = 5.1. Preliminaries: knapsack polytope and cover inequalities ⎢ k 1 ⎥ s.t. ⎢ ⎥ q q ⎣ ⎦ (23) γ / − We begin by recall the concepts of knapsack polytopes and yk k 2 yk k Q cover inequalities. k =1 k =1 The knapsack polytope is the convex hull of the feasible points s z T 0 . : 0 of the knapsack problem, z Z

y ≥ 0 . KF := { x ∈ { 0 , 1 } n : w T x ≤ c} .

We can now derive the dual problem of (23), considering the La- Deﬁnition 5 (Zero-one knapsack polytope) . grangian function, deﬁned as

= ( ) = ({ ∈ { , } n T ≤ } ) . L (t, y, s, z, Z) KPol : conv KF conv x 0 1 : w x c ⎛ ⎡ ⎤ ⎞ q Proposition 6. The dimension

− − T γ T ⎜ ⎢ t y b yk k 2⎥ ⎟ ⎜ ⎢ ⎥ ⎟ ( ) = , s z T = dim KPol n = − − ⎜ ⎢ k 1 ⎥ ⎟ t trace⎜ ⎢ ⎥ ⎟ z Z q q ⎝ ⎣ ⎦ ⎠ and KPol is an independence system, i.e., γ − yk k 2 yk k Q n k =1 k =1 x ∈ KPol, y ∈ { 0 , 1 } , y ≤ x ⇒ y ∈ KPol.

q q ≤ , ∀ ∈ R n , Proof. Recall that wi c i. Therefore, all the unit vectors ei = − − (− − T ) − γ T − − t s t y b yk k z trace Z yk k Q as well as the zero vector, are feasible, and the ﬁrst statement fol- = = k 1 k 1 lows. The second statement is clear. q q

= ( − ) + T − γ T − − Cover inequalities were originally presented in Balas (1975) , t s 1 sy b yk k z trace Z yk k Q

k =1 k =1 Wolsey (1975), see also Nemhauser and Wolsey (1988, Section II.2). These inequalities can be used in general optimization problems q

= ( − ) + ( − γ T − ( )) + ( ) . with knapsack inequalities and binary variables and, particularly, t s 1 yk sbk z trace k Z trace QZ k in QKP . k =1 The Lagrangian dual function is defined as Definition 7 (Cover inequality, CI) . The subset C ⊆ N is a cover if it h (s, z, Z) := min L (t, y, s, z, Z) satisfies ≥ , y 0 t q

= ( − ) + ( − γ T − ( )) + ( ) w j > c. min t s 1 yk sbk k z trace k Z trace QZ y ≥0 ,t ∈ k =1 j C trace (QZ) , = = , − γ T − ( ) ≥ , = , . . . , , The (valid) CI is if s 1 sbk k z trace k Z 0 k 1 q −∞ , otherwise. x ≤ |C | − 1 . (27) Therefore, the dual problem of (23) is given by the maximization j j∈C of the Lagrangian dual function over the SDP cone, as in max trace (QZ) A cover C is minimal if no proper subset of C is a cover. z,Z T ∗ s.t. trace ( Z) + γ z ≤ b , k = 1 , . . . , q, Definition 8 (Extended CI, ECI) . Let w := max ∈ w and define k k k (24) j C j 1 z T the extension of C as 0 . z Z ∗ E(C) := C ∪ { j ∈ N\C : w j ≥ w } . We finally note that problems (24) and (21) are the same. Since The ECI is F has nonempty interior, then (21) is strictly feasible. Therefore, strong duality holds for problem (21), and the result of the theo- x j ≤ |C | − 1 . (28) rem follows. j∈ E(C)

α ≥ ∀ ∈ 5. Valid inequalities Deﬁnition 9 (Lifted CI, LCI). Given a cover C, let j 0, j N\C, α > ∈ and j 0, for some j N\C, such that We are now interested in ﬁnding valid inequalities to + α ≤ | | − , strengthen relaxations of QKP in the lifted space determined by x j j x j C 1 (29) the lifting X := xx T . Let us denote by CRel , any convex relaxation of j∈C j∈ N\C = T QKP in the lifted space, where the equation X xx was relaxed is a valid inequality for KF . Inequality (29) is a LCI . in some manner, by convex constraints, i.e., any convex relaxation of QKPlifted Cover inequalities are extensively discussed in Hammer, We note that if the inequality Johnson, and Peled (1975) , Balas and Zemel (1978) , Balas (1975) ,

τ T x ≤ β (25) Wolsey (1975) , Nemhauser and Wolsey (1988) , and Atamtürk

(2005). Details about the computational complexity of LCI is pre- is valid for QKP , where τ ∈ Z n and β ∈ Z + , then, as x is nonnega- + sented in Zemel (1989) and Gu, Nemhauser, and Savelsbergh tive and X := xx T , (1998) . Algorithm 2 ( Wolsey, 1998 , p.17), shows how to lift a LCI . It −β provides a facet-deﬁning inequality for KPol when C is a minimal ( x X ) ≤ 0 (26) τ cover and t¯ = 1 . 42 M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49

Algorithm 2 Procedure to Lift Cover Inequalities. 5.3. New valid inequalities in the lifted space Input: A cover C and a valid inequality As discussed, after finding any valid inequality in the form of t¯ −1 (25) for QKP , we may add the constraint (26) to CRel when aim- α + ≤ | | − , i j xi j xi C 1 ing at better bounds. We observe now, that besides (26) we can j=1 i ∈C also generate other valid inequalities in the lifted space by taking = T for KF , for some t¯ ∈ [1 , r] , where r := | N \ C| . advantage of the lifting X : xx , and also of the fact that xis bi- ∈ \ nary. In the following, we show how the idea can be applied to Sort the elements i N C in ascending wi order, defining cover inequalities. { i 1 , i 2 , . . . , i r } . = Let For: t t¯ to r x j ≤ β, (33) j∈C t−1 ζ = α + where C ⊂ N and β < | C |, be a valid inequality for KPol . t max i j xi j xi j=1 i ∈C Inequality (33) can be either a cover inequality, CI , an extended t−1 (30) cover inequality, ECI , or a particular lifted cover inequality, LCI , s.t. w x + w x ≤ c − w α ∈ ∀ ∈ i j i j i i it where j {0, 1}, j N\C in (29). Furthermore, given a general LCI, j=1 i ∈C α ∈ Z , ∈ where j + for all j N\C, a valid inequality of type (33) can be |C | +t −1 x ∈ { 0 , 1 } . α α constructed by replacing each j with min { j , 1} in the LCI. α = | | − − ζ Set it C 1 t . Definition 11 (Cover inequality in the lifted space, CILS ) . Let C ⊂ N End and β < | C | as in inequality (33) , and also consider here that β > 1. We define β X ≤ . (34) ij 2 5.2. Adding cuts to the relaxation i, j∈C ,i< j as the CILS derived from (33) . Given a solution ( x¯, X¯ ) of CRel , our initial goal is to obtain Theorem 12. If inequality (33) is valid for QKP , then the CILS (34) is a valid inequality for QKPlifted derived from a CI that is violated T T n by ( x¯, X¯ ) . A CI is formulated as α x ≤ e α − 1 , where α ∈ {0, 1} a valid inequality for QKPlifted . and e denotes the vector of ones. We then search for the CI that β Proof. Considering (33) , we conclude that at most products maximizes the sum of the violations among the inequalities in 2 of variables x x , where i , j ∈ C , can be equal to 1. Therefore, as Y¯ cut (α) ≤ 0 , where Y¯ := x¯ X¯ and i j = Xij : xi xj , the result follows. −e T α + 1 α) = . Remark 13. When β = 1 , inequality (33) is well known as a clique cut ( α cut, widely used to model decision problems, and frequently used as a cut in branch-and-cut algorithms. In this case, using similar To obtain such a CI , we solve the following linear knapsack prob- idea to what was used to construct the CILS , we conclude that it lem, is possible to fix ∗ = { T ¯ (α) T α ≥ + , α ∈ { , } n} . v : maxα e Ycut : w c 1 0 1 (31) X ij = 0 , for all i, j ∈ C, i < j. Let α∗ solve (31) . If ∗ > 0 , then at least one valid inequality v Given a solution ( x¯, X¯ ) of CRel , the following MIQP problem is in the following set of n scaled cover inequalities, denoted in the a separation problem, which searches for a CILS violated by X¯ . following by SCI , is violated by ( x¯, X¯ ) . ∗ = ( ¯ ) − β(β − ) , ( ) z : maxα,β,K trace XK 1 MIQP 1 T ∗ −e α + 1 T α ≥ + , ( ) ≤ . s.t. w c 1 x X α∗ 0 (32) β = e T α − 1 , K(i, i ) = 0 , i = 1 , . . . , n, Based on the following theorem, we note that to strengthen cut K(i, j) ≤ α , i, j = 1 , . . . , n, i < j, (32) , we may apply Algorithm 2 to the CI obtained, lifting it to an i K(i, j) ≤ α , i, j = 1 , . . . , n, i < j, LCI , and finally add the valid inequality (26) derived from the LCI j K(i, j) ≥ 0 , i, j = 1 , . . . , n, i < j, to CRel . K(i, j) ≥ αi + α j − 1 , i, j = 1 , . . . , n, i < j, α ∈ { , } n , β ∈ R , ∈ S n . Theorem 10. The valid inequality (26) for QKPlifted , which is derived 0 1 K from a valid LCI, dominates all inequalities derived from a CI that can α∗ β∗ ∗ ∗ > If , , K solves MIQP1 , with z 0, the CILS given by be lifted to the LCI . ∗ ∗ ∗ ∗ trace (K X ) ≤ β (β − 1) is violated by X¯ . The binary vector α de- Proof. Consider the LCI (29) derived from a CI (27) for QKP . The fines the CI from which the cut is derived. The CI is specifically ∗T T ∗ ∗ ∗ corresponding scaled cover inequalities (26) derived from the CI given by α x ≤ e α − 1 and β (β − 1) determines the right- and the LCI are, respectively, hand side of the CILS . The inequality is multiplied by 2 because we consider the variable K as a symmetric matrix, in order to sim- ≤ (| | − ) , ∀ ∈ , Xij C 1 xi i N plify the presentation of the model. j∈C Theorem 14. The valid inequality CILS for QKP , which is derived lifted and from a valid LCI in the form (33), dominates any CILS derived from a

X ij + α j X ij ≤ (|C | − 1) x i , ∀ i ∈ N, CI that can be lifted to the LCI. ∈ ∈ \ j C j N C Proof. As X is nonnegative, it is straightforward to verify that if X α ≥ ∀ ∈ where j 0, j N\C. Clearly, as all Xij are nonnegative, the second satisfies a CILS derived from a LCI, X also satisfies any CILS derived inequality dominates the first, for all i ∈ N . from a CI that can be lifted to the LCI . M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49 43

( ) > β(β − ) β Any feasible solution of MIQP1 such that trace X¯ K 1 case, if at most elements of C can be selected in the so- ¯ β β generates a valid inequality for QKPlifted that is violated by X. = lution, clearly at most 2 2 subsets of Cs can also be Therefore, we do not need to solve MIQP to optimality to gener- 1 selected. ate a cut. Moreover, to generate distinct cuts, we can solve MIQP 1 several times (not necessarily to optimality), each time adding to it, the following “no-good” cut to avoid the previously generated Given a solution ( x¯, X¯ ) of CRel , we now present a mixed linear cuts: integer programming (MILP) separation problem, which searches

¯ ∈ α( )( − α( )) ≥ , for an inequality in SCILS that is most violated by X. Let A ¯ i 1 i 1 (35) n (n +1) n × i ∈ N { 0 , 1 } 2 . In the first n columns of A we have the n × n identity matrix. In the remaining n (n − 1) / 2 columns of the matrix, there where α¯ is the value of the variable α in the solution of MIQP 1 are exactly two elements equal to 1 in each column. All columns when generating the previous cut. = , α∗ β∗ ∗ α∗T ≤ T α∗ − are distinct. For example, for n 4 We note that, if , , K solves MIQP1 , then x e 1 ⎛ ⎞ is a valid CI for QKP , however it may not be a minimal cover. 1 0 0 0 1 1 1 0 0 0 Aiming at generating stronger valid cuts, based in Theorem 14 , we ⎜ 0 1 0 0 1 0 0 1 1 0 ⎟ = . −δ T α, A : ⎝ ⎠ might add to the objective function of MIQP1 , the term e 0 0 1 0 0 1 0 1 0 1 for some weight δ > 0. The objective function would then favor 0 0 0 1 0 0 1 0 1 1 minimal covers, which could be lifted to a facet-defining LCI , that would finally generate the CILS . We should also emphasize that if The columns of A represent all the subsets of items in N with one the CILS derived from a CI is violated by a given X¯ , then clearly, or two elements. Let ¯ ∗ = ( ) − , ( ) the CILS derived from the LCI will also be violated by X. z : maxα, v ,K,y trace X¯ K 2v MILP 2 Now, we also note that, besides defining one cover inequality s.t. w T α ≥ c + 1 , in the lifted space considering all possible pairs of indexes in C , K(i, i ) = 2 y (i ) , i = 1 , . . . , n, we can also define a set of cover inequalities in the lifted space, n ( ) ≤ , considering for each inequality, a partition of the indexes in C into y i 1 subsets of cardinality 1 or 2. In this case, the right-hand side of i =1 n (n +1) / 2 the inequalities is never greater than β/2. The idea is made precise K(i, j) = A (i, t) A ( j, t )) y (t ) , i, j = 1 , . . . , n, i < j, below. t= n +1 ≥ ( T α − ) / − . , Definition 15 (Set of cover inequalities in the lifted space, v e 1 2 0 5 ≤ ( T α − ) / , SCILS ) . Let C ⊂ N and β < | C | as in inequality (33) . Let v e 1 2 y (t) ≤ 1 − A (i, t) + α(i ) , i = 1 , . . . , n, = { ( , ) , . . . , ( , ) } ( + ) 1. Cs : i1 j1 ip jp be a partition of C, if |C| is even. = , . . . , n n 1 , t 1 2 = { ( , ) , . . . , ( , ) } ࢨ ( + ) 2. Cs: i1 j1 ip jp be a partition of C {i0 } for each α ≤ ≤ α + n n 1 ( − α) , Ay 2 1 i ∈ C , if | C | is odd and β is odd. ( + ) 0 n n n 1 α ∈ { 0 , 1 } , y ∈ { 0 , 1 } 2 , 3. C s : = {(i , i ) , (i , j ) , . . . , (i p , j p )} , where {(i , j ) , . . . , (i p , j p )} 0 0 1 1 1 1 n ࢨ ∈ β v ∈ Z , K ∈ S . is a partition of C {i0 } for each i0 C, if |C| is odd and is even. α∗, ∗, ∗, ∗ ∗ > If v K y solves MILP2 , with z 0, then the particular in-

< = , . . . , equality in SCILS given by In all cases, ik jk for all k 1 p. ∗ ∗ The inequalities in the SCILS derived from (33) are given by ( ) ≤ trace K X 2v (37)

β is violated by X¯ . The binary vector α∗ defines the CI from which X ≤ , (36) ij 2 the cut is derived. As the CI is given by α∗x ≤ e T α∗ − 1 , we can (i, j) ∈C s conclude that the cut generated either belongs to case (1) or (3) in for all partitions Cs defined as above. Definition 15. This fact is considered in the formulation of MILP2 . ∗

, The vector y deﬁnes a partition Cs as presented in case (3), if Theorem 16. If inequality (33) is valid for QKP then the inequalities n = y (i ) = 1 , and in case (1), otherwise. We ﬁnally note that the in the SCILS (36) are valid for QKP . i 1 lifted number 2 in the right-hand side of (37) is due to the symmetry of Proof. The proof of the validity of SCILS is based on the lifting the matrix K ∗. = relation Xij xi x j . We note that if the binary variable xi indicates We now may repeat the observations made for MIQP1 . ( ) > whether or not the item i is selected in the solution, the variable Any feasible solution of MILP2 such that trace X¯ K 2v gener-

Xij indicates whether or not the pair of items i and j, are both se- ates a valid inequality for CRel, which is violated by X¯ . Therefore, lected in the solution. we do not need to solve MILP2 to optimality to generate a cut.

Moreover, to generate distinct cuts, we can solve MILP2 several 1. If | C | is even, C is a partition of C in exactly | C |/2 subsets s times (not necessarily to optimality), each time adding to it, the with two elements each, and therefore, if at most β ele- following suitable “no-good” cut to avoid the previously generated

ments of C can be selected in the solution, clearly at most β cuts:

2 subsets of Cs can also be selected. n (n +1) 2 β ࢨ 2. If |C| and are odd, Cs is a partition of C {i0 } in exactly y¯ (i )(1 − y (i )) ≥ 1 , (38) | − | / C 1 2 subsets with two elements each, where i0 can be i =1 any element of C . In this case, if at most β elements of C β− β where y¯ is the value of the variable y in the solution of MILP2 , can be selected in the solution, clearly at most 1 = 2 2 when generating the previous cut. ∗T T ∗ subsets of C s can also be selected. The CI α x ≤ e α − 1 may not be a minimal cover. Aiming at β 3. If |C| is odd and is even, Cs is the union of {(i0 , i0 )} with generating stronger valid cuts, we might add again to the objective ࢨ | − | / −δ T α, δ > a partition of C {i0 } in exactly C 1 2 subsets with two function of MILP2 , the term e for some weight 0. The ob-

elements each, where i0 can be any element of C. In this jective function would then favor minimal covers, which could be 44 M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49

γ lifted to a facet-defining LCI. In this case, however, after computing if k is even, or α the LCI, we have to solve MILP2 again, with fixed at values that (γ − 1)! represent the LCI , and fixed so that the right-hand side of the in- = γ × k , v NCl : k γ − k ( k 1 ) γ −1 2 ( k ) equality is equal to the right-hand side of the LCI. All components 2 2 ! of y that were equal to 1 in the previous solution of MILP2 should γ if k is odd. also be fixed at 1. The new solution of MILP2 would indicate the Then, the number of inequalities in SKILS is other subsets of N to be added to C s . One last detail should be taken into account. If the cover C corresponding to the LCI , is such q NC . that |C| is odd and the right-hand side of the LCI is also odd, then lk the cut generated should belong to case (2) in Definition 15 , and k =1 MILP should be modified accordingly. Specifically, the second and 2 α γ = Remark 20. If ˜ k is even for every k, such that k : |Ck | is odd, third constraints in MILP , should be modified respectively to 2 β thenthe right-hand side of inequality (41) may be replaced with K(i, i ) = 0 , i = 1 , . . . , n, β 2 × , which will strengthen the inequality in case β is odd. n 2 γ y (i ) = 1 . Note that the case where k is even for every k, is a particular i =1 case contemplated by this remark, where the the tightness of the inequality can also be applied. Remark 17. Let γ := | C |. Then, the number of inequalities in the SCILS is Corollary 21. If inequality (39) is valid for QKP , then the inequalities γ ! (41) , in the SKILS , are valid for QKP , whether or not the modifi- , lifted ( γ ) γ 2 ( ) cation suggested in Remark 20 is applied. 2 2 ! if γ is even, or Proof. The result is again verified, by using the same argument used in the proof of Theorem 16 , i.e., considering that X = 1 , iff (γ − 1)! ij = = γ × , xi x j 1. ( γ −1 ) γ −1 2 ( ) 2 2 ! if γ is odd. 5.4. Dominance relation among the new valid inequalities

Finally, we extend the ideas presented above to the more gen- We start this subsection investigating whether SCILS dominates eral case of knapsack inequalities. We note that the following dis- CILS or vice versa. α ∈ Z , ∀ ∈ \ cussion applies to a general LCI, where j + j N C. Let Theorem 22. Let C be the cover in (33) and consider γ := | C | to be

even. α j x j ≤ β. (39) ∈ j N 1. If β = γ − 1 , then the sum of all inequalities in SCILS is equiv- α , β ∈ Z , β ≥ be a valid knapsack inequality for KPol, with j + alent to CILS. Therefore, in this case, the set of inequalities in α , ∀ ∈ j j N. SCILS dominates CILS. 2. If β < γ − 1 , there is no dominance relation between SCILS

Definition 18 (Set of knapsack inequalities in the lifted space, and CILS . α { , . . . , } SKILS). Let j be the coefficient of xj in (39). Let C1 Cq be the α = α , , ∈ α = partition of N, such that u v if u v Ck for some k, and u Proof. Let sum ( SCILS ) denote the inequality obtained by adding αv , otherwise. The knapsack inequality (39) can then be rewritten all inequalities in SCILS , and let rhs (sum ( SCILS )) denote its right- as hand side (rhs). We have that rhs (sum ( SCILS )) is equal to the number of inequalities in SCILS multiplied by the rhs of each in- q α ≤ β. equality, i.e.: ˜ k x j (40) = ∈ k 1 j Ck γ ! β ( ( )) = × . = , . . . , , = { ( , ) , . . . , ( , ) } , rhs sum SCILS ( γ ) γ Now, for k 1 q let Cl : ik jk ik jk where 2 ( ) 2 k 1 1 pk pk 2 2 ! < ( , ) ∈ , i j for all i j Cl and k ( ) The coefficient of each variable Xij in sum SCILS (coefij ) is given • C is a partition of C , if | C | is even. lk k k by the number of inequalities in the set SCILS in which Xij appears, • C is a partition of C \ { i } , where i ∈ C , if | C | is odd. lk k k0 k0 k k i.e.: The inequalities in the SKILS corresponding to (39) are given by (γ − 2)! coe f = ij ( (γ −2) ) (γ −2) ⎛ ⎞ 2 ( ) 2 2 ! q ⎝ α + α ⎠ ≤ β, Dividing rhs (sum ( SCILS )) by coef , we obtain ˜ k Xi i 2 ˜ k Xij (41) ij k0 k0 k =1 (i, j) ∈C lk β rhs (sum ( SCILS )) /coe f ij = (γ − 1) × . (42) for all partitions C , k = 1 , . . . , q, defined as above, and for all i ∈ 2 lk k0 \ \ = ∅ , Ck Cl . (If |Ck | is even, Ck Cl and the term in the variable k k On the other side, the rhs of CILS is: Xi i does not exist.) k0 k0 β β(β − 1) Remark 19. Consider {C , . . . , C q } as in Definition 18 . For k = rhs ( CILS ) = = . (43) 1 , . . . , , γ = 2 2 1 q let k : |Ck | and define γ β β− k ! β γ − , 1 β NC := γ , 1. Replacing with 1 and 2 with 2 (since is odd), lk ( k ) γ 2 2 ( k !) 2 we obtain the result. M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49 45

2. Consider, for example, C = { 1 , 2 , 3 , 4 , 5 , 6 } and β = 3 ( β < of SKILS . In case | C | is odd, however, the inequalities in SCILS are γ − 1 and odd). In this case, the CILS becomes: stronger.

X 12 + X 13 + X 14 + X 15 + X 16 + X 23 + X 24 Proof. If | C | is even, the result is easily veriﬁed. If | C | is odd, the

+ X + X + X + X + X + X + X + X ≤ 3 . inequalities in SCILS become 25 26 34 35 36 45 46 56 ≤ β − , And a particular inequality in SCILS is 2 Xij 1

(i, j) ∈C s X 12 + X 34 + X 56 ≤ 1 . (44) if β is odd, and = , = , . . . , , The solution X1 j 1 for j 2 6 and all other variables 2 X + 2 X ≤ β, equal to zero, satisﬁes all inequalities in SCILS , because only i0 i0 ij ( , ) ∈ one of the positive variables appears in each inequality in i j Cs the set. However, the solution does not satisfy CILS . On the if β is even, and the inequalities in SKILS become

other side, the solution X 12 = X 34 = X 56 = 1 , and all other X + 2 X ≤ β, variables equal to zero, satisﬁes CILS , but does not satisfy i0 i0 ij ( , ) ∈ (44) . i j Cs = { , , , , , } β = β < γ − β ࢨ ∈ Now, consider C 1 2 3 4 5 6 and 4 ( 1 and for all . In all cases, Cs is a partition of C {i0 }, where i0 C. even). In this case, the CILS becomes: Either with β even or odd, it becomes clear that SCILS is stronger than SKILS . X 12 + X 13 + X 14 + X 15 + X 16 + X 23 + X 24

+ X + X + X + X + X + X + X + X ≤ 6 . 25 26 34 35 36 45 46 56 6. Lower bounds from solutions of the relaxations for QKPlifted And a particular inequality in SCILS is In order to evaluate the quality of the upper bounds obtained + + ≤ . X 12 X34 X56 2 (45) with CRel , we compare them with lower bounds for QKP , given = , = , . . . , , = , = by feasible solutions constructed by a heuristic. We assume in this The solution X1 j 1 for j 2 6 X2 j 1 for j

3 , . . . , 6 , and all other variables equal to zero, satisﬁes all section that all variables in CRel are constrained to the interval

inequalities in SCILS , because at most two of the positive [0,1]. ( , ¯ ) variables appear in each inequality in the set. However, the Let x¯ X be a solution of CRel. We initially apply principal

solution does not satisfy CILS . On the other side, the so- component analysis (PCA) (Jolliffe, 2010) to construct an approximation to the solution of QKP and then apply a special round- lution X 12 = X 34 = X 56 = 1 , and all other variables equal to

zero, satisﬁes CILS , but does not satisfy (45) . ing procedure to obtain a feasible solution from it. PCA selects the largest eigenvalue and the corresponding eigenvector of X¯ , denoted by λ¯ and v¯, respectively. Then λ¯ v¯ v¯ T is a rank-one approximation of γ = 1 Theorem 23. Let C be the cover in (33) and consider : |C| to be X¯ . We set x¯ = λ¯ 2 v¯ to be an approximation of the solution x of QKP . odd. Then there is no dominance relation between SCILS and CILS . λ¯ > ¯ > We note that 0 because Xii 0 for at least one index i in the ¯ Proof. Consider, for example, C = { 1 , 2 , 3 , 4 , 5 } and β = 3 ( β odd). optimal solutions of the relaxations, and therefore, X is not nega-

In this case, the CILS becomes: tive semidefinite. Finally, we round x¯ to a binary solution that satisfies the knapsack capacity constraint, using the simple approach + + + + + + + + + ≤ . X 12 X13 X14 X15 X23 X24 X25 X34 X35 X45 3 described in Algorithm 3 . And a particular inequality in SCILS is Algorithm 3 A heuristic for the QKP. X 23 + X 45 ≤ 1 . (46) Input: the solution X¯ from CRel , the weight vector w , the = , = , . . . , , The solution X1 j 1 for j 1 5 and all other variables equal capacity c. to zero, satisfies all inequalities in SCILS , because only one of the Let λ¯ and v¯ be, respectively, the largest eigenvalue and the positive variables appears in each inequality in the set. However, corresponding eigenvector of X¯ .

1 the solution does not satisfy CILS. On the other side, the solution Set x¯ = λ¯ 2 v¯. = = , X23 X45 1 and all other variables equal to zero, satisﬁes CILS, Round x¯ to xˆ ∈ { 0 , 1 } n. but does not satisfy (46) . While w T xˆ > c Now, consider C = { 1 , 2 , 3 , 4 , 5 } and β = 4 ( β even). In this case, = { | > } the CILS becomes: Set i argmin j∈ N x¯ j x¯ j 0 . = = Set x¯i 0, xˆi 0. X 12 + X 13 + X 14 + X 15 + X 23 + X 24 + X 25 + X 34 + X 35 + X 45 ≤ 6 . And a particular inequality in SCILS is End Output: a feasible solution xˆ of QKP . X 11 + X 23 + X 45 ≤ 2 . (47)

= , = , . . . , , = , = , . . . , , The solution X1 j 1 for j 1 5 X2 j 1 for j 2 5 and all other variables equal to zero, satisfies all inequalities in SCILS , because at most two of the positive variables appear in each in- 7. Numerical experiments equality in the set. However, the solution does not satisfy CILS . On = = = , the other side, the solution X11 X23 X45 1 and all other vari- We summarize our algorithmic framework in Algorithm 4 , ables equal to zero, satisfies CILS, but does not satisfy (47). where at each iteration we update the perturbation Q p of the parametric relaxation and, at every m iterations, we add to the relax- Now, we investigate if SCILS is just a particular case of SKILS , ation, the valid inequalities considered in this paper, namely, SCI , when α ∈ {0, 1}, for all j ∈ N in (39) . j defined in (32) , CILS , defined in (34) , and SCILS , defined in (36) . Theorem 24. In case the modification suggested in Remark 20 is ap- The numerical experiments performed had the following main plied, then if | C | is even in (33) , SCILS becomes just a particular case purposes, 46 M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49

• verify the impact of the valid inequalities, SCI , CILS , and SCILS , Algorithm 4 A cutting plane algorithm for the QKP. when iteratively added to cut the current solution of a relax- n Input: Q ∈ S , max.n cuts . ation of QKP , k := 0 , B := I, μ0 := 1 . • 0 verify the effectiveness of the IPM described in Section 3 in de- λ ( ) , Let i Q vi be the ith largest eigenvalue of Q and corre- creasing the upper bound while optimizing the perturbation Q , p sponding eigenvector. • compute the upper and lower bounds obtained with the pro- n T n −6 Q n := = ( −| λ (Q ) | −1) v v (or Q n : = = ( min{ λ , −10 }) i 1 i i i i 1 i posed algorithmic approach described in Algorithm 4, and com- T 0 v v ), Q := Q − Q . pare them, with the optimal solutions of the instances. i i p n = 0 ( 0 ) ( 0 ) Solve CQP Q p , with Qp : Qp , and obtain x Qp , X Qp . We coded Algorithm 4 in MATLAB, version R2016b, and ran the ∇ ∗ ( 0 ) = ( 0 ) − ( 0 ) ( 0 ) T pCQP Qp : X Qp x Qp x Qp . code on a notebook with an Intel Core i5-4200U CPU 2.30 giga- 0 0 Z := Q − Q. hertz, 6 gigabytes RAM, running under Windows 10. We used the p 0 := ∇ p∗ (Q 0 ) + (2 | λ (∇ p∗ (Q 0 ) | + 0 . 1) I. primal-dual IPM implemented in Mosek, version 8, to solve the re- CQP p min CQP p

, While (stopping criterium is violated) laxation CQPQ and, to solve the separation problems MIQP1 and p + k 1 MILP2 , we use Gurobi, version 8. Run Algorithm 1, where Qp is obtained and re- k +1 The input data used in the first iteration of the IPM described laxation CQP Q , with Q p := Q is solved. Let p p in Algorithm 1 ( k = 0 ) are: B = I, μ0 = 1 . We start with a matrix k +1 k +1 0 (x (Q ) , X(Q )) be its optimal solution. 0 0 p p Q , such that Q − Q is negative definite. By solving CQP , with + p p Qp . k +1 = ∗ ( k 1 ) upper bound : pCQP Qp . = 0, ( 0 ) , ( 0 ) , Qp : Qp we obtain x Qp X Qp as its optimal solution, and ∗ Run Algorithm 3, where xˆ is obtained. ∇ ( 0 ) = ( 0 ) − ( 0 ) ( 0 ) T + set pCQP Qp : X Qp x Qp x Qp . Finally, the positive defi- lower.bound k 1 := xˆ T Q xˆ. 0 0 0 = 0 − niteness of Z and are assured by setting: Z : Qp Q and If k mod m == 0 0 = ∇ ∗ ( 0 ) + ( | λ (∇ ∗ ( 0 ) | + . ) : pCQP Qp 2 min pCQP Qp 0 1 I. Our randomly generated test instances were also used by Solve problem (31) and obtain cuts SCI in (32).

Cunha, Simonetti, and Lucena (2016), who provided us with the Add the max{ n, max.n cuts } cuts SCI with the + + instances data and with their optimal solutions. Each weight w , ( ( k 1 ) , ( k 1 )) j largest violations at x Qp X Qp , to ∈ for j N, was randomly selected in the interval [1, 50], and the ca- CQP Qp . , n pacity c, of the knapsack, was randomly selected in [50 j=1 w j ]. n cuts := 0 . The procedure used by Cunha to generate the instances was based While ( n cuts < max.n cuts & MIQP 1 feasible) on previous works ( Billionnet & Calmels, 1996; Caprara et al., 1999; Solve MIQP 1 and add the CI and CILS ob- Chaillou et al., 1989; Gallo et al., 1980; Michelon & Veillieux, 1996 ). tained to CQP Q . The following labels identify the results presented in Tables 3 , p Add the “no-good” cut (35) to MIQP 1 . 5 and 6 . n cuts := n cuts + 1 . • OptGap (%): = ((upper bound − opt)/opt) × 100, where opt is End

the optimal solution value (the relative optimality gap), n cuts := 0 . • Time (seconds) (the computational time to compute the While ( n cuts < max.n cuts & MILP 2 feasible) bound), Solve MILP 2 and add the CI and SCILS ob- • DuGap (%) : = (upper bound − lower bound)/(lower bound) tained to CQP Q p . × 100, where the lower bound is computed as described in Add the “no-good” cut (38) to MILP 2 . Section 6 (the relative duality gap), n cuts := n cuts + 1 . • Iter (the number of iterations), End • Cuts (the number of cuts added to the relaxation),

• End TimeMIP (seconds) (the computational time to obtain cuts CILS = + and SCILS ). k : k 1. End To get some insight into the effectiveness of the cuts proposed, Output: Upper bound upper.bound k , lower bound we initially applied them to 10 small instances with n = 10 . In lower.bound k , and feasible solution xˆ to QKP . Table 3 we present average results for this preliminary experiment, where we iteratively add the cuts to the following linear relaxation

max trace (QX ) n Fig. 1 depicts the optimality gaps from Table 3 . There is a trade- s.t. w x ≤ c, j j off between the quality of the cuts and the computational time ( ˜ ) = LPR j 1 (48) needed to find them. Considering a unique cut of each type, we 0 ≤ X ≤ 1 , ∀ i, j ∈ N ij note that SCILS is the strongest cut (OptGap = 9 . 121% ), but the 0 ≤ x ≤ 1 , ∀ i ∈ N i computational time to obtain it, if compared to CILS and SCI , is X ∈ S n . bigger. Nevertheless, a decrease in the times could be achieved In the first row of Table 3 , the results correspond to the solution with a heuristic solution for the separation problems, and also by of the linear relaxation LPR˜ with no cuts. In SCI1 , we add only the the application of better stopping criteria for the cutting plane al- most violated cut from the n cuts in SCI to LPR˜ at each iteration, gorithm. We point out that using all cuts together we find a bet- and in the SCI we add all n cuts. In CIL S and SCIL S , we solve MIQP ter upper bound than using each type of cut separately (OptGap and MILP problems to find the most violated cut of each type. The = 3 . 315% ). last row of the table (All) corresponds to results obtained when we We now analyze the effectiveness of our IPM in decreasing the add all n cuts in SCI , and one cut of each type, CILS and SCILS . In upper bound while optimizing the perturbation Q p . To improve these initial tests, we run up to 50 iterations, and in most cases, the bounds obtained, besides the constraints in (48) , we also con- = , stop the algorithm when no more cuts are found to be added to sider in the initial relaxation, the valid equations Xii xi the Mc- ≤ the relaxation. Cormick inequalities Xij Xii , and the valid inequalities obtained by M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49 47

Table 4 40 0 SDP bound vs IPM bound at iterations 1 and 20, for two initial matrices Qp . 0 Qp Inst n gap1 (%) gap20 (%) 30 a Qp I1 50 0.30 0.01 I2 50 0.73 0.03 I3 50 0.14 0.00 20 I4 50 1.02 0.21 I5 50 0.59 0.09

OptGap(%) I1 100 1.47 0.14 10 I2 100 0.59 0.04 I3 100 0.51 0.05 I4 100 1.38 0.26 0 I5 100 0.73 0.06

1 b Qp I1 50 0.01 0.00 ˜ SCI I2 50 0.30 0.09 SCI CILS ALL SCILS I3 50 0.08 0.03 I4 50 0.10 0.02 Fig. 1. Average optimality gaps from Table 3 . I5 50 0.03 0.03 I1 100 0.04 0.00 I2 100 0.02 0.01 I3 100 0.03 0.01 Table 3 I4 100 0.12 0.04 Impact of the cuts added to LPR˜ on 10 small instances ( n = 10 ). I5 100 0.02 0.01

Method OptGap Time Iter Cuts Time MIP (%) (seconds) (seconds) Table 5

LPR˜ 38.082 0.35 1.0 Results for Algorithm 4 ( n = 50 ). SCI 1 36.703 32.38 1.1 28.4

SCI 10.036 39.98 3.0 364.1 Inst OptGap Time DuGap Iter TimeMIP CILS 19.719 9.00 2.7 82.2 6.91 (%) (seconds) (%) (seconds) SCILS 9.121 266.81 50.0 794.3 198.12 I1 0.23 1013.50 0.27 100 641.98 ALL 3.315 315.82 28.3 646.6 264.91 I2 0.00 632.50 0.00 64 411.67 I3 0.00 392.55 0.00 44 205.70 I4 0.00 289.97 0.00 31 160.37 I5 0.21 1093.60 0.37 10 698.04 multiplying the capacity constraint by each nonnegative variable xi , ( − ) , and also by 1 xi and then replacing each bilinear term xi xj by Table 6 X . We then start the algorithm solving the following relaxation. ij Results for Algorithm 4 ( n = 100 ).

T ( − 0 ) + ( 0 ) Inst OptGap Time DuGap Iter Time max x Q Qp x trace Qp X MIP n (%) (seconds) (%) (seconds) ≤ , s.t. w j x j c I1 0.00 2035.30 0.00 20 737.86 j=1 I2 0.25 2177.30 0.65 20 919.41

n I3 0.00 2007.10 0.00 20 773.00 w j X ij ≤ cX ii , ∀ i ∈ N I4 0.12 1885.90 0.84 20 828.98 j=1 I5 0.04 2309.50 0.20 20 970.49 n ( QPR ) (49) w j (X jj − X ij ) ≤ c(1 − X ii ) , ∀ i ∈ N j=1 0 Qp (boundIPM1 ) in relaxation (49) is. The values presented in the X ii = X ii , ∀ i ∈ N table are X ij ≤ X ii , ∀ i, j ∈ N ≤ ≤ , ∀ , ∈ 0 Xij 1 i j N gap 1 (%) = (boundIP M 1 − boundSDP ) /boundSDP ∗ 100 0 ≤ x ≤ 1 , ∀ i ∈ N i gap (%) = (boundIP M − boundSDP ) /boundSDP ∗ 100 X ∈ S n . 20 20 For the experiment reported in Table 4 , we consider 10 In order to evaluate the inﬂuence of the initial decomposition instances with n = 50 and 100. We see from the results in of Q on the behavior of the IPM, we considered two initial decom- Table 4 that in 20 iterations, the IPM closed the gap to the SDP positions. In both cases, we compute the eigendecomposition of Q , a , bounds for all instances. When starting from Qp we end up with = n λ T getting Q i =1 i vi vi . an average bound less than 0.1% of the SDP bound, while when b , starting from Qp this percentage decreases to only 0.03%. We also • n T For the ﬁrst decomposition, we set Q n := = (−| λ | − 1) v v , b i 1 i i i start from better bounds when considering Q , and therefore, we 0 0 a p and Q := Q − Q / 2 . We refer to this initial matrix Q as Q . p n p p use this matrix as the initial decomposition for the IPM in the next n − • = ( { λ , − 6} ) T , For the second, we set Qn : i =1 min i 10 vi vi and experiments. The results in Table 4 show that the IPM developed 0 = − / 0 b Qp : Q Qn 2. We refer to this initial matrix Qp as Qp . in this paper is effective to solve the parametric problem (6), con- verging to bounds very close to the solution of the SDP relaxation, In Table 4 , we compare the bounds obtained by our IPM after which are their minimum possible values.

20 iterations (boundIPM20 ), with the bounds given from the linear We ﬁnally present results obtained from the application of 0 = SDP relaxation obtained by taking Qp Q in (49), and adding to it Algorithm 4 , considering the parametric quadratic relaxation, the the semideﬁnite constraint X − xx T 0 ( boundSDP ). As mentioned IPM, and the cuts. In Tables 5 and 6 we show the results for the in Section 4 , these are the best possible bounds that can be ob- same instances with n = 50 and n = 100 considered in the previ- tained by the IPM algorithm. We also show in Table 4 how close ous experiment. The cuts are added at every m iterations of the to boundSDP , the bound computed with the initial decomposition IPM and the numbers of cuts added at each iteration are n SCI , 48 M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49

5 CILS and 5 SCILS . Note that when solving each MIQP or MILP generate the cuts could also be solved heuristically, in order to ac- problem, besides the cut CIL S or SCIL S , we also obtain a cover in- celerate the process. equality CI . We check if this CI was already added to the relax- We note that the search for the best perturbation Q p , by our ation, and if not, we add it as well. We stop Algorithm 4 when a IPM, is updated with the inclusion of cuts to the relaxation. In maximum number of iterations is reached or when DuGap is suf- the set of cuts added, we could also consider cuts defined by ficiently small. the branching procedure in a branch-and-bound algorithm. In this For the results presented in Table 5 ( n = 50 ), we set the max- case, we could have the perturbation Q p optimized during all the imum number of iterations of the IPM equal to 100, and m = 10 . descend on the branch-and-bound tree, considering the cuts the The execution of each separation problem was limited to 3 sec- have been added to the relaxations. onds, and the best solutions obtained in this time limit was used Finally, we show that if the positive semidefinite constraint to generate the cuts. X − xx T 0 was introduced in the relaxation of the QKP, or any For the results presented in Table 6 ( n = 100 ), we set the max- other indefinite quadratic problem (maximizing the objective func- imum number of iterations of the IPM equal to 20, and m = 4 . In tion), then the decomposition of objective function, that leads to a this case, the execution of each separation problem was limited to convex quadratic SDP relaxation, where a perturbed concave part 10 seconds. of the objective is kept, and the remaining part is linearized, is not We note from the results in Tables 5 and 6 , that the alterna- effective. In this case the best bound is always attained when the tion between the iterations of the IPM to improve the perturba- whole objective function is linearized, i.e., when the perturbation tion Q p of the relaxation and the addition of cuts to the relaxation, Q p is equal to Q . This observation also relates to the well known DC changing the search direction of the IPM, is an effective approach (difference of convex) decomposition of indefinite quadratics that to compute bounds for QKP . Considering the stopping criterion im- have been used in the literature to generate bounds for indefinite posed to Algorithm 4 , it was able to converge to the optimum so- quadratic problems. Once more, in case the positive semidefinite lution of three out of five instances with n = 50 and of two out constraint is added to the relaxation, the DC decomposition is not of five instances with n = 100 . The average optimality gap for all effective anymore, and the alternative linear SDP relaxation leads ten instances is less than 0.1%. The heuristic applied also computed to the best possible bound. As corollary from this result, we see good solutions for the problem. The average duality gap for the 10 that the bound given by the convex quadratic relaxation cannot be instances is less than 0.25%. better than the bound given by the corresponding linear SDP re- We note that our algorithm spends a high percentage of its run- laxation. This last result was already proved in the literature, as ning time solving the separation problems, and also solving the mentioned in Section 4 . linear systems to define the direction of improvement in the IPM algorithm. The running time of both procedures can be improved by a more judicious implementation. There are two parameters in Acknowledgments Algorithm 4 that can also be better analyzed and tuned to improve the results, namely, m and the time limit for the execution of the We thank J. O. Cunha for providing us with the test instances separation problems. As mentioned before, these problems could used in our experiments. We greatly thank the anonymous refer- still be solved by heuristics. Finally, we note that the alternation ees and editor for their constructive comments that significantly between the IPM iterations and addition of cuts to the relaxation improved the paper. M. Fampa was partially supported by CNPq could be combined with a branch-and-bound algorithm in an at- (National Council for Scientific and Technological Development) tempt to converge faster to the optimal solution. In this case, the Grant 303898/2016-0, D. C. Lubke did part of this research at the cuts added to the relaxations would include the cuts that define University of Waterloo, and was supported by Research Grants the branching and the update on Q p would depend on the branch from CAPES (Brazilian Federal Agency for Support and Evaluation of the enumeration tree. These are directions for the continuity of of Graduate Education) Grant 88881.131629/2016-01, CNPq Grant the research on this work. 142143/2015-4 and partially supported by PSR –Power Systems Re- search, Rio de Janeiro, Brazil.

8. Conclusion References In this paper we present a cutting plane algorithm (CPA) to iteratively improve the upper bound for the quadratic knapsack prob- Armand, P. , Gilbert, J. C. , & Jégou, S. J. (20 0 0). A feasible BFGS interior point algorithm for solving convex minimization problems. SIAM Journal on Optimization, lem (QKP). The initial relaxation for the problem is given by a para- 11 (1), 199–222 . metric convex quadratic problem, where the Hessian Q of the ob- Atamtürk, A. (2005). Cover and pack inequalities for (mixed) integer programming. Annals of Operations Research, 139 (1), 21–38 . jective function of the QKP is perturbed by a matrix parameter Q p , Balas, E. (1975). Facets of the knapsack polytope. Mathematical Programming, 8 (1), − such that Q Qp 0. Seeking for the best possible bound, the con- 146–164 . T cave term x (Q − Q p ) x, is then kept in the objective function of Balas, E. , & Zemel, E. (1978). Facets of the knapsack polytope from minimal covers. T SIAM Journal on Applied Mathematics, 34 (1), 119–148 . the relaxation and the remaining part, given by x Q p x is linearized

Billionnet, A., & Calmels, F. (1996). Linear programming for the 0–1 quadratic knap- through the standard approach that lifts the problem to space of sack problem. European Journal of Operational Research, 92 (2), 310–325 . T symmetric matrices defined by X := xx . Billionnet, A. , Elloumi, S. , & Lambert, A. (2012). Extending the QCR method to We present a primal-dual interior point method (IPM), which the case of general mixed integer programs. Mathematical Programming, 131 (1), 381–401 . update the perturbation Q at each iteration of the CPA aiming at p Billionnet, A. , Elloumi, S. , & Lambert, A. (2016). Exact quadratic convex reformu- reducing the upper bound given by the relaxation. We also present lations of mixed-integer quadratically constrained problems. Mathematical Pro- new classes of cuts that are added during the execution of the CPA, gramming, 158 (1–2), 235–266 . which are defined on the lifted variable X , and derived from cover Billionnet, A., Elloumi, S., & Plateau, M. (2009). Improving the performance of standard solvers for quadratic 0–1 programs by a tight convex reformulation: The inequalities and the binary constraints. QCR method. Discrete Applied Mathematics, 157 (6), 1185–1197 . We show that both the IPM and the cuts generated are effec- Billionnet, A. , Faye, A. , & Soutif, E. (1999). A new upper bound for the 0–1 quadratic tive in improving the upper bound for the QKP and note that these knapsack problem. European Journal of Operational Research, 112 (3), 664– 672 . procedures could be applied to more general binary indefinite Caprara, A. , Pisinger, D. , & Toth, P. (1999). Exact solution of the quadratic knapsack quadratic problems as well. The separation problems described to problem. INFORMS Journal on Computing, 11 (2), 125–137 . M. Fampa, D. Lubke and F. Wang et al. / European Journal of Operational Research 281 (2020) 36–49 49

Chaillou, P. , Hansen, P. , & Mahieu, Y. (1989). Best network flow bounds for the Hogan, W. W. (1973). The continuity of the perturbation function of a convex pro- quadratic knapsack problem. In B. Simeone (Ed.), Combinatorial optimization: gram. Operations Research, 21 , 351–352 . Lectures given at the 3rd session of the centro internazionale matematico estivo Horn, R. A. , & Zhang, F. (2005). Basic properties of the Schur complement. In (C.I.M.E.) Held at Como, Italy (pp. 225–235). Berlin, Heidelberg: Springer . August F. Zhang (Ed.), The Schur complement and its applications. Numerical methods and 25–September 2, 1986 algorithms: 4 . Boston, MA, USA: Springer . Cunha, J. O. , Simonetti, L. , & Lucena, A. (2016). Lagrangian heuristics for the Horst, R. , & Thoai, N. V. (1999). DC programming: Overview. Journal of Optimization quadratic knapsack problem. Computational Optimization and Applications, 63 (1), Theory and Applications, 103 , 1–43 . 97–120 . Jolliffe, I. (2010). Principal component analysis (2nd). Berlin: Springer . Danskin, J. M. (1966). The theory of max-min, with applications. SIAM Journal on Lemarchal, C. , & Oustry, F. (1999). Semidefinite relaxations and Lagrangian duality Applied Mathematics, 14 (4), 641–664 . with application to combinatorial optimization . INRIA Rhones-Alpes . Fampa, M. , Lee, J. , & Melo, W. (2017). On global optimization with indefinite Lewis, A. S. , & Overton, M. L. (2013). Nonsmooth optimization via quasi-newton quadratics. EURO Journal on Computational Optimization, 5 (3), 309–337 . 09 methods. Mathematical Programming, 141 (1), 135–163 . Fiacco, A. V. (1983). Introduction to sensitivity and stability analysis in nonlinear McCormick, G. P. (1976). Computability of global solutions to factorable nonconvex programming. Mathematics in science and engineering . Academic Press 165 . programs: part i convex underestimating problems. Mathematical Programming, Gallo, G. , Hammer, P. L. , & Simeone, B. (1980). Quadratic knapsack problems. In 10 , 147–175 . M. W. Padberg (Ed.), Combinatorial optimization (pp. 132–149). Berlin, Heidel- Michelon, P. , & Veillieux, L. (1996). Lagrangean methods for the 0–1 quadratic knap- berg: Springer . sack problem. European Journal of Operational Research, 92 (2), 326–341 . Gu, Z. , Nemhauser, G. L. , & Savelsbergh, M. W. P. (1998). Lifted cover inequalities Nemhauser, G. L. , & Wolsey, L. A. (1988). Integer and combinatorial optimization . New for 0–1 integer programs: Computation. INFORMS Journal on Computing, 10 (4), York, NY: John Wiley & Sons . 427–437 . Pisinger, D. (2007). The quadratic knapsack problem: A survey. Discrete Applied Hammer, P. L. , Johnson, E. L. , & Peled, U. N. (1975). Facet of regular 0–1 polytopes. Mathematics, 155 (5), 623–648 . Mathematical Programming, 8 (1), 179–206 . Turlach, B. A. , & Wright, S. J. (2015). Quadratic programming. In Wiley interdisci- Helmberg, C. , Rendl, F. , & Weismantel, R. (1996). Quadratic knapsack relaxations us- plinary reviews: Computational statistics: 7 (pp. 153–159). John Wiley & Sons, Inc . ing cutting planes and semidefinite programming. In Integer programming and Wolsey, L. A. (1975). Faces for a linear inequality in 0–1 variables. Mathematical Pro- combinatorial optimization (Vancouver, BC, 1996) . In Lecture notes in comput. sci., gramming, 8 (1), 165–178 . pages 190–203: 1084 . Berlin: Springer . Wolsey, L. A. (1998). Integer programming . New York, NY, USA: Wiley-Interscience . Helmberg, C. , Rendl, F. , & Weismantel, R. (20 0 0). A semidefinite programming ap- Zemel, E. (1989). Easily computable facets of the knapsack polytope. Mathematics of proach to the quadratic knapsack problem. Journal of Combinatorial Optimization, Operations Research, 14 (4), 760–764 . 4 (2), 197–215 .