Mathematical Programming manuscript No. (will be inserted by the editor)

G´erard Cornu´ejols Valid Inequalities for Mixed Integer Linear Programs

December 2005, revised July 2006

Abstract. This tutorial presents a theory of valid inequalities for mixed integer linear sets. It introduces the necessary tools from polyhedral theory and gives a geometric understanding of several classical families of valid inequalities such as lift-and-project cuts, Gomory mixed integer cuts, mixed integer rounding cuts, split cuts and intersection cuts, and it reveals the relationships between these families. The tutorial also discusses computational aspects of gen- erating the cuts and their strength.

Key words: mixed integer linear program, lift-and-project, split cut, Gomory cut, mixed integer rounding, elementary closure, polyhedra, union of polyhedra

1. Introduction

In this tutorial we consider mixed integer linear programs. These are problems of the form max cx + hy Ax + Gy ≤ b x ≥ 0integral y ≥ 0, where the data are the row vectors c ∈ Qn, h ∈ Qp, the matrices A ∈ Qm×n, G ∈ Qm×p and the column vector b ∈ Qm; and the variables are the column vectors x ∈ Rn and y ∈ Rp.ThesetS of feasible solutions to this mixed integer linear program is called a mixed integer linear set.Thatis,ifP := {(x, y) ∈ n p n p R+ × R+ : Ax + Gy ≤ b},thenS := P ∩ (Z+ × R+). It can be shown that conv(S) is a polyhedron (see for example Nemhauser and Wolsey [46]). In order to solve a mixed integer linear program, it is therefore equivalent to solve the linear program max cx + hy (x, y) ∈ conv(S) since the extreme points of conv(S)areinS.Thisisthepolyhedral approach. The difficulty, of course, is to construct inequalities defining conv(S)giventhose

Tepper School of Business, Carnegie Mellon University, Pittsburgh, PA 15213, USA; and LIF, Facult´e des Sciences de Luminy, Universit´e d’ Aix-Marseille, 13288 Marseille, France. e-mail: [email protected] Supported by NSF grant DMI-0352885 and ONR grant N00014-03-1-0188. 2 G´erard Cornu´ejols

defining P . This tutorial presents approaches for constructing improved approx- imations of conv(S) recursively. An inequality is said to be valid for a set if it is satisfied by every point in this set. A cut with respect to a point (x, y) ∈/ conv(S) is a valid inequality for conv(S)thatisviolatedby(x, y). The polyhedral ap- proach to solving mixed integer linear programs leads naturally to the cutting plane approach:Solvethelinear programming relaxation obtained by ignoring the integrality requirements on x; if this relaxation is unbounded or infeasible, then stop: the mixed integer linear progam is infeasible or unbounded; otherwise let (x, y) be an optimal extreme point solution; if (x, y) ∈ S, then stop: (x, y) is an optimal solution of the mixed integer linear program; otherwise generate a cut with respect to (x, y), add it to the previous linear programming relaxation, solve this improved relaxation, and repeat. In this tutorial, we give a geometric understanding of several classical families of cuts such as split cuts [25], Go- mory mixed integer cuts [34], intersection cuts [4] and, in the mixed 0,1 case, lift-and-project cuts [54], [43], [7]. We study the relationships between these var- ious families and discuss computational aspects of these cuts. Cuts that exploit specific structures of the constraints are also the object of a vast literature but they are not addressed in this tutorial. We start with some useful results in polyhedral theory.

2. Polyhedra

A polyhedron in Rn is a set of the form P := {x ∈ Rn : Ax ≤ b} where A is a real matrix and b a real vector. If A and b have rational entries, P is a rational polyhedron. The polyhedron {x ∈ Rn : Ax ≤ 0} is called a polyhedral cone.Note that a polyhedral cone is always nonempty since it contains the null vector 0. ⊆ Rn { ∈ Rn For S ,theconvex hull of S is the set conv(S):= x : x = k i ≥ ∈ Rk k 1 k ∈ } i=1 λix where k 1,λ +, i=1 λi =1andx ,...,x S . This is the smallest convex set that contains S. It will sometimes be useful to work with conv(S), the closure of conv(S), which is the smallest closed convex set that ⊆ Rn { ∈ Rn contains S.Theconic hull of a nonempty set S is cone(S):= x : k i ≥ ∈ Rk 1 k ∈ } x = i=1 λix where k 1,λ + and x ,...,x S . The convex hull of a finite set of points in Rn is called a polytope. An important theorem, due to Minkowski-Weyl, states that every polyhedron P can be written as the sum of a polytope Q and a polyhedral cone C.Here Q + C := {x ∈ Rn : x = y + z for some y ∈ Q and z ∈ C}.Notethat P = ∅ if and only if Q = ∅.IfP := {x ∈ Rn : Ax ≤ b} is nonempty, then C = {x ∈ Rn : Ax ≤ 0} is unique. The cone {x ∈ Rn : Ax ≤ 0} is called the recession cone of P . Another useful result is Farkas’s lemma: For a nonempty polyhedron P := {x : Ax ≤ b}, the inequality αx ≤ β is valid for P if and only if there exists a vector v ≥ 0 such that vA = α and vb ≤ β.[ThisisalsoaconsequenceofLP duality: β ≥ max{αx : Ax ≤ b} =min{vb : vA = α, v ≥ 0}.] Another form of Farkas’s lemma is: Ax ≤ b has a solution if and only if vb ≥ 0 for all Valid Inequalities for MILPs 3

v ≥ 0 such that vA = 0. [Equivalently, by LP duality, max{0: Ax ≤ b} = min{vb : vA =0,v≥ 0}.] The proof of these results can be found in textbooks such as Schrijver [53] or Ziegler [58].

2.1. Some properties of polyhedra and convex sets

This section contains two lemmas that will be used later. Lemma 1. Let P := {x ∈ Rn : Ax ≤ b} be a polyhedron and let Π := P ∩{x : πx ≤ π0}.IfΠ = ∅ and αx ≤ β is a valid inequality for Π, then there exists a scalar λ ∈ R+ such that αx − λ(πx − π0) ≤ β is valid for P . Proof. By Farkas’s lemma, since Π = ∅,thereexistu ≥ 0, λ ≥ 0 such that α = uA + λπ

and β ≥ ub + λπ0.

Since uAx ≤ ub is valid for P ,soisuAx ≤ β − λπ0.SinceuAx = αx − λπx,the inequality αx − λ(πx − π0) ≤ β is valid for P .

αx − λ(πx − π0) ≤ β

P Π

πx ≤ π0 αx ≤ β

Fig. 1. Illustration of Lemma 1

Remark 1. Lemma 1 has two possible outcomes: When the hyperplane πx = π0 is parallel to αx = β, then by choosing λ such that α = λπ,wegetthetrivial n inequality λπ0 ≤ β, satisfied by all points in R (for instance, the lemma holds n when P = R ); in the more interesting case when πx = π0 is not parallel to αx = β, the outcome of the lemma is a closed half-space containing P (see Figure 1). 4 G´erard Cornu´ejols

αx ≤ β

αx − λ(πx − π0) ≤ β P

πx ≤ π0 Fig. 2. Lemma1doesnotholdwhenΠ = ∅

Remark 2. The assumption Π = ∅ is necessary in Lemma 1, as shown by the 2 following example (see Figure 2): P := {x ∈ R : x1 ≥ 0,x2 ≥ 0} and Π := P ∩{x : x2 ≤−1}.ThusΠ is empty. The inequality x1 ≤ 1 is valid for Π but there is no scalar λ such that x1 − λ(x2 +1)≤ 1 is valid for P .

Let H ⊆ Rn be a hyperplane and S ⊆ Rn. In general, conv(S) ∩ H = conv(S ∩ H) as shown by the following example: S consists of two points not in H but the line segment connecting them intersects H. The following lemma shows that equality holds when S lies entirely in one of the closed half spaces defined by the hyperplane H (see Figure 3).

Lemma 2. Let H := {x ∈ Rn : ax = b} be a hyperplane and S ⊆{x : ax ≤ b}. Then conv(S) ∩ H = conv(S ∩ H).

S

conv(S)

H

Fig. 3. Illustration of Lemma 2

Proof. We prove first conv(S ∩ H) ⊆ conv(S) ∩ H andthenconv(S) ∩ H ⊆ conv(S ∩ H). Valid Inequalities for MILPs 5

Clearly conv(S ∩ H) ⊆ conv(S)andconv(S ∩ H) ⊆ H so the first inclusion is obvious. ∈ ∩ To prove the other inclusion, let x conv(S) H. This means ax = b and k i 1 k ∈ ≥ k x = i=1 λix where x ,...,x S, λ 0and i=1 λi =1.

k k i b = ax = λiax ≤ λib = b (1) i=1 i=1

i where the inequality follows from ax ≤ b and λi ≥ 0. Relation (1) implies that these inequalities are in fact equations, i.e. axi = b for i =1,...,k. Therefore xi ∈ S ∩ H. This implies x ∈ conv(S ∩ H).

2.2. Facets

Let P := {x ∈ Rn : Ax ≤ b} be a polyhedron. A face of P is a set of the form

F := P ∩{x ∈ Rn : αx = β}

where αx ≤ β is a valid inequality for P (the inequality is said to define the face F ). A face is itself a polyhedron since it is the intersection of the polyhedron P with another polyhedron (the hyperplane αx = β). The dimension of a set is the dimension of the smallest affine space that contains it. The faces of dimension 0 are called vertices of P , those of dimension 1 are called edges and those of dimension dim(P ) − 1 are called facets. The proof of the following theorem can be found in Schrijver [53].

Theorem 1. Let P ⊆ Rn be a nonempty polyhedron. (i) For each facet F of P , at least one of the inequalities defining F is nec- essary in any description Ax ≤ b of P . (ii) Inequalities defining faces of dimension less than dim(P ) − 1 are not needed in the description of P and can be removed.

This result states that, if a polyhedron in Rn has m facets, any representa- tion by a system of linear inequalities in Rn contains at least m inequalities. In integer linear programming, we often consider polyhedra that are given implic- itly as conv(S) (see the Introduction). It is not unusual for such polyhedra to have a number of facets that is exponential in the size of the input. Thus their representation by linear inequalities in Rn is large. In some cases, there is a way to get around this difficulty: a polyhedron with a large number of facets can sometimes be obtained as the projection of a polyhedron with a small number of facets. For this reason, projections turn out to be an important topic in this tutorial. 6 G´erard Cornu´ejols

2.3. Projections

Let P ⊆ Rn+p where (x, y) ∈ P will be interpreted as meaning x ∈ Rn and y ∈ Rp.Theprojection of P onto the x-space Rn is { ∈ Rn ∃ ∈ Rp ∈ } projx(P ):= x : y with (x, y) P .

y

P

x

projx(P )

Fig. 4. Projection

{ ∈ Rn ×Rp ≤ } { ∈ Theorem 2. Let P := (x, y) : Ax+Gy b .Thenprojx(P )= x n t t R : v (b − Ax) ≥ 0 for all t ∈ T } where {v }t∈T is the set of extreme rays of Q := {v ∈ Rm : vG =0,v ≥ 0}.

Proof. Let x ∈ Rn. By Farkas’s Lemma, Gy ≤ b − Ax has a solution y if and only if vt(b − Ax) ≥ 0 for all extreme rays vt of vG =0,v ≥ 0.

Remark 3. Variants of Theorem 2 can be proved similarly: If y ≥ 0inP , then the relevant cone Q is {v : vG ≥ 0,v≥ 0}. If y ≥ 0andAx + Gy = b in P , the relevant cone is {v : vG ≥ 0} with v unrestricted in sign.

Enumerating the extreme rays of Q may not be an easy task in applications. Another way of obtaining the projection of P is to eliminate the variables yi one at a time (Fourier-Motzkin elimination procedure): Consider a polyhedron P ⊆ Rn+1 defined by the system of inequalities: n aij xj + giz ≤ bi for i ∈ I. (2) j=1

0 + − Let I = {i ∈ I : gi =0}, I = {i ∈ I : gi > 0}, I = {i ∈ I : gi < 0}. The Fourier-Motzkin procedure eliminates the variable z as follows: It keeps the inequalities of (2) in I0, and it combines each pair of inequalities i ∈ I+ and l ∈ I− to eliminate z.

Theorem 3. The system of |I0| + |I+||I−| inequalities obtained by the Fourier- Rn Motzkin procedure is the projection projx(P ) of P in the x-space . Valid Inequalities for MILPs 7

Proof. (2) implies 1 n z ≤ (b − a x ) ∀i ∈ I+ g i ij j i j=1

1 n z ≥ (b − a x ) ∀l ∈ I−. g l lj j l j=1 Therefore, every (x, z) in (2) satisfies

1 n 1 n (b − a x ) ≤ (b − a x ). (3) g l lj j g i ij j l j=1 i j=1

Conversely, if x satisfies the inequalities (2) for i ∈ I0 and the system (3) for every i ∈ I+,l∈ I−,then

1 n 1 n max (bl − aljxj ) ≤ min (bi − aij xj). l∈I− g i∈I+ g l j=1 i j=1

Let z be any value in this interval. Then (x, z) satisfies (2).

2.4. Union of polyhedra

In this section, we prove a result of Balas [5], [6] about the union of polyhedra. n i Consider k polyhedra Pi := {x ∈ R : Aix ≤ b }. We will show that the smallest ∪k closed convex set that contains i=1Pi is a polyhedron. This set is denoted by ∪k conv( i=1Pi).

The closure is needed as shown by the following example: P1 consists of a single point, and P2 is a line that does not contain the point P1 (see Figure 5). Let P3 denote the line going through P1 that is parallel to P2.Itiseasytoverifythat conv(P1 ∪P2)=conv(P2 ∪P3)andthatconv(P1 ∪P2)=conv(P2 ∪P3)\(P3 \P1) ∗ (indeed, a point x in P3 \ P1 is not in conv(P1 ∪ P2), but there is a sequence k ∗ of points x ∈ conv(P1 ∪ P2)thatconvergestox ). Here conv(P1 ∪ P2)isnota closed set, and therefore it is not a polyhedron.

P1 P3

conv(P1 ∪ P2)

P2

Fig. 5. conv(P1 ∪ P2) =conv( P1 ∪ P2) 8 G´erard Cornu´ejols

∪k The following example shows that conv( i=1Pi) can have an exponential number of facets in the size of the input Pi,i=1,...,k. i n i Let e denote the i-th unit vector in R .LetPi be the single point e for − i ∪2n i =1,...,n,andPn+i the single point e for i =1,...,n.Thenconv( i=1Pi) has 2n facets, namely

2n n n n conv( Pi)={x ∈ R : 'ixi ≤ 1 for all ' ∈{−1, +1} }. i=1 i=1 Thispolyhedroniscalledtheoctahedron.

P2

P3 P1 conv(∪Pi)

P4 Fig. 6. Octahedron

∪k Although conv( i=1Pi) may have exponentially many facets, Balas [5], [6] proved that it is the projection of a higher dimensional polyhedron Y with a polynomial size representation:  i ≤ i  Aix b yi  i x = x Y :=  (4)  yi =1 yi ≥ 0fori =1,...,k.

i n In this formulation, x is a vector in R and yi is a scalar, for i =1,...,k. The vector x ∈ Rn corresponds to the original space onto which Y is projected. Thus, the polyhedron Y is defined in Rkn+n+k. A formulation with a polynomial number of variables and constraints is said to be compact. The gist of Balas’s ∪k result is that conv( i=1Pi) has a compact representation. The proof is given below. n i n Let Pi := {x ∈ R : Aix ≤ b } be a polyhedron and Ci := {x ∈ R : Aix ≤ 0} its recession cone. By the Minkowski-Weyl theorem, there exists a polytope Qi such that Pi = Qi + Ci.

n i Theorem 4. Consider k polyhedra Pi := {x ∈ R : Aix ≤ b } and let Ci := n {x ∈ R : Aix ≤ 0}.LetPi =: Qi + Ci where Qi is a polytope. If ∪Pi = ∅, ⊆ ∪ ∅ assume that Cj cone i:Pi=∅ Ci for all j such that Pj = . Then the following sets are identical. Valid Inequalities for MILPs 9

∪k (i) conv( i=1Pi) ∪k ∪k (ii) conv( i=1Qi)+cone( i=1Ci) (iii) projxY ,whereY is defined in (4). ∪k ∪k Proof. Define Q := conv( i=1Qi)andC := cone( i=1Ci). We will show that ∪k ⊆ ⊆ ⊆ ∪k conv( i=1Pi) projxY Q + C conv( i=1Pi). ∪k ⊆ Since projxY is a closed set, the first inclusion implies conv( i=1Pi) projxY and the theorem follows.

∪k ⊆ (a) conv( i=1Pi) projxY ∪k ∅ ∪k ∅ The result holds when i=1Pi = , so we assume i=1Pi = . Without loss of generality, P1,...,Ph are nonempty and Ph+1,...,Pk are empty, where 1 ≤ h ≤ k. ∈ ∪k Let x conv( i=1Pi). Then x is a convex combination of a finite number of ∪h points in i=1Pi. Since eachPi is convex, we can write x as a convex combination i ∈ h i ≥ h of points z Pi,sayx = i=1 yiz where yi 0fori =1,...,hand i=1 yi = i i i 1. Let x = yiz for i =1,...,h andyi =0,x =0fori = h +1,...,k.Then i ≤ i k i ∈ Aix b yi for i =1,...k and x = i=1 x . This shows that x projxY and therefore (a) holds.

⊆ (b) projxY Q + C The result holds if Y = ∅, so we assume Y = ∅. ∈ 1 k Let x projxY . By the definition of projection, there exist x ,...,x ,y such k i i ≤ i ≥ { } that x = i=1 x where Ax b yi, yi =1,y 0. Let I := i : yi > 0 . i For i ∈ I,letzi := x .Thenzi ∈ P .SinceP = Q + C ,wecanwrite yi i i i i i i i i i z = w + x where w ∈ Qi and x ∈ Ci. i i For i ∈ I,wehaveAix ≤ 0. That is x ∈ Ci.

i i x = yiz + x i∈I i∈I k i i = yiw + λix where λi ≥ 0 i∈I i=1 ∈conv(∪Qi) ∈cone(∪Ci) ∈ Q + C.

⊆ ∪k (c) Q + C conv( i=1Pi) The result holds when Q = ∅, so we assume Q = ∅. Without loss of generality ≤ ≤ Q1,...,Qh are nonempty and Qh+1,...,Qk are empty, where 1 h k. ∈ h i k i i ∈ ≥ Let x Q+ C.Thenx = i=1 yiw + i=1 x where w Qi, yi 0for h i ∈ i =1,...,h, i=1 yi =1andx Ci for i =1,...k. By the assumption in the 10 G´erard Cornu´ejols j h j ij statement of the theorem we have, for j = h +1,...,k, x = i=1 µi x ,where j ≥ ij ∈ µi 0andx Ci for i =1,...,h. h Based on the above vector y ∈ R+, define I := {i : yi > 0} and consider the point h h 1 k x := (y − ')wi + '(wi + (xi + µjxij )) i |I| ' i i∈I i=1 j=h+1 for '>0 small enough so that y − h ' ≥ 0 for all i ∈ I. i |I| x ∈ conv(∪k P )since (y − h ')+ h ' =1andwi + 1 (xi + i=1 i i∈I i |I| i=1  k j ij ∈ j=h+1 µi x ) Pi. Furthermore x → x as ' → 0. ∈ ∪k
Therefore x conv( i=1Pi).

Remark 4. The assumption in Theorem 4 is necessary as shown by the following 2 2 example (see Figure 7): P1 := {x ∈ R :0≤ x ≤ 1} and P2 := {x ∈ R : x1 ≤ 2 0,x1 ≥ 1}.NotethatP2 = ∅ and C2 = {x ∈ R : x1 =0}.Thetheoremdoes { ∈ R2 ≤ ≤ } not hold in this case since projxY = P1 + C2 = x :0 x1 1 ,whichis different from conv(P1 ∪ P2)=P1.

x2


proj Y



x
1

P1




1 x 0



1

C2

Fig. 7. conv(P1 ∪ P2) =proj xY

Remark 5. The assumption in Theorem 4 is automatically satisfied if (i) Ci = {0} whenever Pi = ∅,or (ii) all the polyhedra Pi have the same recession cone. For example (i) holds when all the Pis are nonempty, or when Ci = {0} for all i.

Remark 6. If all the polyhedra Pi have the same recession cone C = Ci for ∪k ∪k i =1,...,k,thenconv( i=1Pi)=conv( i=1Pi). Indeed, in this case, part (c) of Valid Inequalities for MILPs 11

⊆ ∪k the above proof simplifies and shows that Q + C conv( i=1Pi). Furthermore ∪Pi is the x-projection of  i ≤ i  Aix b yi  i x = x YI :=   yi =1 yi ∈{0, 1} for i =1,...,k.

To prove the last claim, observe that if x ∈∪Pi,thenx ∈ Pj for some j i j =1,...,k. Setting x = x, yj =1andx =0,yi =0fori = j,wehavea 1 k 1 k solution (x, x ,...,x ,y) ∈ YI . Conversely, given (x, x ,...,x ,y) ∈ YI ,letj be j j i the index for which yj =1.WehaveAj x ≤ b and Aix ≤ 0fori = j,namely j ∈ i x Pj and x is in the recession cone of Pi for i =j. Since the recession cone k i of Pi is also the recession cone of Pj,thepointx = i=1 x belongs to Pj.

Remark 7. When the Pis have different recession cones, the x-projection of YI is usually different from ∪Pi as shown by the following example (see Figure 5). 2 2 Let P1 ⊆ R consist of the single point (0, 1), and let P2 ⊆ R be the line x2 = 0. Then we claim that the x-projection of YI is P2 ∪ P3 where P3 denotes 1 2 1 2 the line x2 = 1. Indeed, when y1 =1inYI , x = x + x with x ∈ P1 and x in the recession cone of P2,whichisP2 itself in this example. So the x-projection of YI restricted to y1 = 1 is the line P3.Thex-projection of YI restricted to y2 = 1 is the line P2, which proves the claim. In fact, under the assumption of Theorem 4, we have ∪k ∪k projxYI = i=1Qi +cone( i=1Ci).

Application: Theorem 4 has been applied to several basic models that ap- pear repeatedly in mixed integer programming [37], [45], [56], [3], [23]. As an example, we consider a mixed integer linear set studied by Miller and Wolsey [45] and Van Vyve [56] under the name of continuous mixing set. We present it here in a form that arises in the context of robust mixed 0,1 programming, following Atamt¨urk [3]:

n n S := {(x, y, z) ∈{0, 1} × R+ × R+ : yi + z ≥ dixi for i =1,...,n}

n where d ∈ R+ is a given vector. The polyhedron conv(S) has an exponential number of facets but has a compact formulation in a higher dimensional space. To derive this formulation, observe that the variable z takes either the value 0 or di for i =1,...,n in every vertex of conv(S). Set d0 := 0. Let δ denote any of these n + 1 values, and define

n n S(δ):={(x, y, z) ∈{0, 1} × R+ × δ : yi ≥ dixi − δ for i =1,...,n}. Since the constraints of S(δ) are decoupled, one can strengthen each separately to obtain the convex hull of S(δ):

n n + conv(S(δ)) := {(x, y, z) ∈ [0, 1] × R × δ : yi ≥ (di − δ) xi for i =1,...,n}. 12 G´erard Cornu´ejols

Since z ∈{d0,d1,...,dn} for every vertex of conv(S), we have ∪n conv(S)=conv( i=0conv(S(di))) + C where C := {(x, y, z) ∈ Rn × Rn × R : x =0,y≥ 0,z≥ 0} is the recession cone of conv(S). Defining Pi := conv(S(di)) + C (all Pis have the same recession cone C), it is straightforward to obtain the desired compact formulation Y as defined in (4).

3. Lift-and-Project

In this section, we consider mixed 0,1 linear programs. These are mixed integer linear programs where the integer variables are only allowed to take the values 0 or 1. It will be convenient to write mixed 0,1 linear programs in the form min cx Ax ≥ b xj ∈{0, 1} for j =1,...,n xj ≥ 0forj = n +1,...,n+ p, where the matrix A ∈ Qm×(n+p), the row vector c ∈ Qn+p and the column vector b ∈ Qm are data, and x ∈ Rn+p is a column vector of variables. n+p Consider the polyhedron P := {x ∈ R+ : Ax ≥ b} andthemixed0,1 n p linear set S := {x ∈{0, 1} × R+ : Ax ≥ b}.Thesetconv(S) is a polyhedron and, ideally, we would like to have its linear description in the form conv(S)= Dx ≥ d. Constructing the description Dx ≥ d would reduce the solution of a mixed 0,1 linear program to that of a linear program. But this goal is too ambitious in general as the number of inequalities needed in the description Dx ≥ d is typically enormous. This is not surprising considering that mixed 0,1 linear programming is NP-hard! A more reasonable goal is to obtain an intermediate set between P and conv(S) over which one can optimize a linear function in polynomial time, and then to use recursion to get tighter approximations of conv(S). Sherali and Adams [54], Lov´asz and Schrijver [43] and Balas, Ceria and Cornu´ejols [7] propose an approach for doing this which generates intermedi- ate sets Q between P and conv(S) as projections of higher dimensional sets that have a polynomial description. The polyhedron P ⊆ Rn+p is first lifted into a higher dimensional space Rn+p+q where the formulation is strengthened. This strengthened formulation is then projected back onto the original space Rn+p, thus defining Q. In this process the constraints of the higher dimensional formulation are defined explicitly whereas those of Q are only known implic- itly through projection, thus allowing Q to have a nonpolynomial number of constraints. This approach is known under the name of lift-and-project.

3.1. The lift-and-project relaxation

n+p Consider a polyhedron P := {x ∈ R+ : Ax ≥ b} andthemixed0,1set n p S := {x ∈{0, 1} × R+ : Ax ≥ b}. Without loss of generality, we assume Valid Inequalities for MILPs 13

that the constraints Ax ≥ b include xj ≥ 0forj =1,...,n+ p,andxj ≤ 1 for j =1,...,n. Balas, Ceria and Cornu´ejols [7] study the following “lift-and- project” procedure:

Step 0: Select j ∈{1,...,n}. Step 1: Generate the nonlinear system xj (Ax−b) ≥ 0, (1−xj)(Ax−b) ≥ 0. Step 2: Linearize the system by substituting yi for xixj, i = j,andxj for 2 xj . Call this polyhedron Mj. Step 3: Project Mj onto the x-space. Let Pj be the resulting polyhedron.

S ⊆ Pj follows from the fact that, for any x ∈ S,wehave(x, y) ∈ Mj by 2 choosing yi = xixj for i = j since xj = xj (this holds because xj is a 0,1 variable). We also have Pj ⊆ P since Ax ≥ b is obtained by adding the constraints defining Mj. How tight is the relaxation Pj of S compared to the initial relaxation P ? The next theorem shows that it is tightest possible among the relaxations that ignore the integrality of all the variables xi for i = j.

Theorem 5. Pj = conv{(Ax ≥ b, xj =0)∪ (Ax ≥ b, xj =1)}

P

Pj

xj =0 xj =1

Fig. 8. Illustration of Theorem 5

∗ Proof. Call P the set conv{(Ax ≥ b, xj =0)∪ (Ax ≥ b, xj =1)}. ∗ First we show Pj ⊆ P . We assume P = ∅ since otherwise the result is trivial. ∗ If P ∩{xj =0} = ∅,thenP = P ∩{xj =1}.WealreadyknowthatPj ⊆ P . ∗ Thus, to show that Pj ⊆ P , it suffices to show that Pj ⊆{xj =1}, i.e. that xj ≥ 1 is valid for Pj.Let' := min{xj : x ∈ P }.SinceP ∩{xj =0} = ∅,wehave '>0. The inequality xj ≥ ' is valid for P and, by Farkas’s lemma, it is implied by a nonnegative combination of rows of Ax ≥ b. Therefore (1−xj)xj ≥ (1−xj)' 2 is valid for the nonlinear system of Step 1. Step 2 replaces xj by xj.Thisgives that xj ≥ 1 is valid for Pj . ∗ Similarly if P ∩{xj =1} = ∅,wegetPj ⊆ P . Now assume P ∩{xj =0} = ∅ and P ∩{xj =1} = ∅.Takeαx ≥ β valid ∗ for P . Since it is valid for Π := P ∩{x : xj ≤ 0} we can find a λ such that 14 G´erard Cornu´ejols

αx + λxj ≥ β is valid for P (Lemma 1). Similarly, we can find µ such that αx + µ(1 − xj) ≥ β is valid for P . Therefore (1 − xj )(αx + λxj − β) ≥ 0andxj(αx + µ(1 − xj ) − β) ≥ 0are valid for the nonlinear system of Step 1, and their sum is too:

− 2 − ≥ αx +(λ + µ)(xj xj ) β 0.

2 ≥ Step 2 replaces xj by xj.Thisgivesαx β valid for Mj, and thus for Pj.This ∗ completes the proof that Pj ⊆ P .

∗ ∗ Now we show P ⊆ Pj . We assume P = ∅ since otherwise the result is ∩{ } ∩{ } trivial. Let x be a point in P xj =0 or in P xj =1 . Define yi = xixj for ∈ 2 ∈ i = j.Then(x, y) Mj since xj = xj.So,x Pj. By convexity of Pj it follows ∗ that P ⊆ Pj.

∩n The set j=1Pj is called the lift-and-project closure. It is a better approxi- mation of conv(S)thanP :

⊆∩n ⊆ conv(S) j=1Pj P.

How much better is it in practice? Bonami and Minoux [18] performed computa- tional experiments (see also Bonami’s dissertation [15]). On 35 mixed 0,1 linear programs from the MIPLIB library [13], they found that the lift-and-project closure reduces the integrality gap by 37 % on average (the integrality gap is the difference between the objective value optimized over conv(S)andoverP ).

Sherali and Adams [54] define a stronger relaxation by skipping Step 0 and considering the nonlinear constraints xj(Ax − b) ≥ 0and(1− xj)(Ax − b) ≥ 0 for all j =1,...,nin Step 1. Then, in Step 2, variables yij are introduced for all i =1,...,n+p and j =1,...,nwith i>j. Note that the size of the linear system generated in Step 2 is much larger than in the previous lift-and-project procedure n(n−1) ( 2 + np new variables and 2nm constraints, instead of just n + p − 1new variables and 2m constraints before). Let N denote the resulting polyhedron obtained by projection in Step 3. Clearly, the Sherali-Adams relaxation N is at least as strong as the lift-and-project closure defined above, and it can be strictly stronger since the Sherali-Adams procedure takes advantage of the fact ∩n that yij = yji whereas this is not the case for the lift-and-project closure j=1Pj. How much better is the Sherali-Adams relaxation in practice? We turn again to Bonami and Minoux [18]. For the 19 MIPLIB intances for which they could compute the Sherali-Adams bound within an hour, the improvement over the lift-and-project bound was 10 % on average. Specifically, on these 19 instances, the average integrality gap closed went from 29 % for the lift-and-project closure to 39 % for the Sherali-Adams relaxation. An even stronger relaxation can be obtained as follows: Valid Inequalities for MILPs 15

3.2. The Lov´asz-Schrijver relaxation

Step 1: Generate the nonlinear system xj(Ax−b) ≥ 0and(1−xj)(Ax−b) ≥ 0 for all j =1,...,n. Step 2: Linearize the system by substituting yij for xixj, for all i =1,...,n+ 2 p, j =1,...,n such that j

In this procedure, the fact that we get a relaxation N+ instead of the set 1 S itself is because we replace the constraint Y = 1 xT by “Y positive x semidefinite” in Step 2. ⊆ ⊆ ⊆∩n ⊆ We have conv(S) N+ N j=1Pj P . The convex set N+ is not a polyhedron in general. Nevertheless, a major interest in the above Lov´asz-Schrijver procedure [43] is due to the fact that a linear function can be optimized over N+ in polynomial time. Indeed, optimizing a linear function over the constraint set defined in Step 2 is a semidefinite pro- gram: linear objective, linear constraints plus a symmetric positive semidefinite constraint on the matrix Y of variables. Semidefinite programs can be solved in polynomial time by interior point algorithms [48]. How tight is the Lov´asz-Schrijver relaxation N+? From a practical perspec- tive the size of the semidefinite program creates a tremendous challenge: the number of variables has been multiplied by n and the number of constraints as well! Burer and Vandenbussche [19] solve it using an augmented Lagrangian method and they report computational results on three classes of combinatorial problems, namely the maximum stable set problem, the quadratic assignment problem and the following problem of Erd¨os and Turan: Calculate the maximum size of a subet of numbers in {1,...,n} such that no three numbers are in arith- metic progression. In all three cases, the Lov´asz-Schrijver bound given by N+ is substantially tighter than the Sherali-Adams bound given by N. To illustrate this, we give below the results obtained in [19] for the size of a maximum stable set (graphs with more than 100 nodes):

Name nodes edges optimum N+ N brock200-1 200 5066 21 27.966.6 c-fat200-1 200 18366 12 14.966.6 johnson16-2-4 120 1680 8 10.223.3 keller4 171 5100 11 15.457.0 rand-200-05 200 982 64 72.775.1 rand-200-50 200 10071 11 17.166.6 16 G´erard Cornu´ejols

From a theoretical point of view, Goemans and Williamson [33] proved a striking result. They showed that, for the max-cut problem, the semidefinite relaxation is never more than 14% above the optimum value. Is it possible to improve upon the Lov´asz-Schrijver relaxation in polynomial time? The answer is yes. See for example Lasserre [40], Laurent [41], and Bi- enstock and Zuckerberg [12]. These relaxations are even more computationally demanding than N+. We do not discuss them here. Instead we return to relax- ations that can yield cutting planes in a reasonably short amount of computing time.

3.3. Lift-and-project cuts

In this section we return to the lift-and-project relaxation Pj. Optimizing a linear function over Pj amounts to solving a linear program. So does the problem of generating a valid inequality for Pj that cuts off a given point x, or showing that none exists, as we explain below. Step 2 of the lift-and-project procedure in Section 3.1 constructs the following polyhedron Mj. n+p n+p Mj := {x ∈ R+ ,y ∈ R+ : Ay − bxj ≥ 0,Ax+ bxj − Ay ≥ b, yj = xj}. The first two constraints come from linearizing the inequalities of Step 1. Actu- ally, the variable yj is not introduced explicitly in Step 2. So, in fact, Mj is the n+p n+p−1 polyhedron in R+ × R+ obtained by identifying the variables yj = xj in the above set. Let Aj be the m×(n+p−1) matrix obtained from A by removing its j-th column aj.Wehave

n+p n+p−1 j j Mj := {x ∈ R+ ,y ∈ R+ : Ajy+(a −b)xj ≥ 0,Ax+(b−a )xj −Ajy ≥ b}. By renaming the coefficient matrices of x,weget

n+p n+p−1 Mj = {x ∈ R+ ,y ∈ R+ : B˜jx + Aj y ≥ 0, A˜jx − Ajy ≥ b} We want to project out the y variables. This is done using Theorem 2. The appropriate cone is

Q := {(u, v): uAj − vAj =0,u≥ 0,v≥ 0}.

Namely the set Pj can be written as:

n+p Pj = {x ∈ R+ :(uB˜j + vA˜j)x ≥ vb for all (u, v) ∈ Q}. Given a fractional solution x, we can now express that an inequality αx ≥ β is valid for Pj and cuts off x. Indeed α = uB˜j + vA˜j and β = vb for (u, v) ∈ Q ex- presses the validity for Pj and αx<βexpresses that it is a cut. To get a deepest cut, one can solve the following linear program, the so-called cut generating LP:

max vb − (uB˜j + vA˜j)x uAj − vAj =0 u ≥ 0,v≥ 0. Valid Inequalities for MILPs 17

These constraints along with a normalization constraint to truncate the cone will do. For example, we could add the constraint ui + vi =1. Therefore, the cut generating LP has 2m variables and n + p constraints other than the nonnegativity conditions. This is a fairly large linear program to solve just to generate one cut. Various tricks have been used to speed up the solution, such as working in the subspace of variables where xj > 0forj =1,...,n+p and xj < 1 for j =1,...,n, see [7]. Is it possible to generate lift-and-project cuts without explicitly formulating and solving the cut generating LP? Balas and Perregaard [10] give a positive answer. We explain this below.

Instead of expressing Pj as the projection of Mj,itisalsopossibletoex- press Pj using Theorems 5 and 4 and then projecting onto the x-space. This is equivalent of course. By Theorem 5, Pj is the convex hull of the union of two polyhedra: Ax ≥ b Ax ≥ b x ≥ 0 and x ≥ 0 −xj ≥ 0 xj ≥ 1.

In this part, we assume that the inequalities Ax ≥ b contain −xj ≥−1for j =1,...,n, but not x ≥ 0. By Theorem 4,  0  Ax ≥ by0  − 0 ≥  xj 0  1  Ax ≥ by1 1 ≥ Pj =projx xj y1  0 1  x + x = x   y0 + y1 =1  0 1 x, x ,x ,y0,y1 ≥ 0.

Let ej denote the j-th unit vector. Using the projection theorem (Theorem 2), we get that Pj is defined by the inequalities αx ≥ β such that

α −uA +u0ej ≥ 0 α −vA −v0ej ≥ 0 β −ub ≤ 0 β −vb −v0 ≤ 0 u, u0,v,v0 ≥ 0. Adding a normalization constraint, we obtain the cut generating LP:

min αx −β α −uA +u0ej ≥ 0 α −vA −v0ej ≥ 0 β −ub ≤ 0 (5) β −vb −v0 ≤ 0 m m i=1 ui +u0 + i=1 vi +v0 =1 u, u0,v,v0 ≥ 0. 18 G´erard Cornu´ejols

Balas and Perregaard [10] give a precise correspondence between the basic feasible solutions of (5) and the basic solutions (possibly infeasible) of the usual LP relaxation (R) min{cx : Ax ≥ b, x ≥ 0}.

basic solution 2 basic solution 1

cut 1

cut 2

P

01 Fig. 9. Correspondence between basic solutions and lift-and-project cuts

A geometric view of this correspondence may be helpful: The n + p extreme rays emanating from a basic solution of (R) intersect the hyperplanes xj =0and xj =1inn + p points (some of these points may be at infinity). These points uniquely determine a hyperplane αx = β where (α, β) are associated with a basic feasible solution of the cut generating LP (5). For example, in Figure 9, cut 1 corresponds to the basic solution 1 of (R) and cut 2 corresponds to the basic (infeasible) solution 2 of (R). Using the correspondence, Balas and Perregaard show how simplex pivots in the cut generating LP (5) can be mimicked by pivots in (R). The major practical consequence is that the cut generating LP (5) need not be formulated and solved explicitly. A sequence of increasingly deep lift-and-project cuts can be computed by pivoting directly in (R). We elaborate on these pivoting rules in Section 4.3.

3.4. Strengthened lift-and-project cuts

n p Again we consider the mixed 0,1 linear set S := {x ∈{0, 1} ×R+ : Ax ≥ b}.We assume that the constraints Ax ≥ b contain −xj ≥−1forj =1,...,n, but not x ≥ 0. The cut generating LP (5) produces a lift-and-project inequality αx ≥ β that is valid for Pj. The derivation only uses the integrality of variable xj,not of the variables xk for k =1,...,n and k = j. Balas and Jeroslow [9] found a simple way to use the integrality of the other variables to strengthen the lift-and- project cut. This strengthening has the nice property that it is straightforward to implement once the cut generating LP (5) has been solved. Valid Inequalities for MILPs 19

Note that, given u, u0,v,v0, the optimal values of αk and β in (5) are: max(uak,vak)fork = j αk = j j (6) max(ua − u0,va + v0)fork = j, where ak denotes the k-th column of A,and

β =min(ub, vb + u0). To strengthen the inequality αx ≥ β, one can try to decrease the coefficients αk. Balas and Jeroslow [9] found a way to do just that by using the integrality of the variables xk for k =1,...,n. Theorem 6. (Balas and Jeroslow [9]) Let x¯ satisfy Ax ≥ b, x ≥ 0. Given an vak −uak optimal solution u, u0,v,v0 of the cut generating LP (5), define m = , k u0+v0 k k min(ua + u0m ,va − v0m ) for k =1,...,n α = k k k max(uak,vak) for k = n +1,...,n+ p

and β =min(ub, vb + u0). Then the inequality αx ≥ β is valid for conv(S). Proof. For π ∈ Zn, the following disjunction is valid for conv(S): n n either πkxk ≥ 0or − πkxk ≥ 1. k=1 k=1

Let us repeat the derivation of (5) with this disjunction in place of −xj ≥ 0or xj ≥ 1 as before. We consider the union of Ax ≥ b Ax ≥ b x ≥ 0 and x ≥ 0 n ≥ − n ≥ k=1 πkxk 0 k=1 πkxk 1. Using Theorem 4 and the projection theorem (Theorem 2), we get that any inequality αx ≥ β that satisfies − − n ≥ α uA u0( k=1 πkek) 0 − n ≥ α vA +v0( k=1 πkek) 0 β −ub ≤ 0 β −vb −v0 ≤ 0 u, u0,v, v0 ≥ 0

is valid for conv(S). We can choose u, u0,v,v0 to be an optimal solution of the original cut generating LP (5). This implies that, for k =1,...,n,wecanchoose k k αk =max(ua + u0πk,va − v0πk). Smaller coefficients αk produce stronger inequalities since the variables are nonnegative. What is the best choice of πk ∈ Z k k to get a small αk? It is obtained by equating ua + u0πk and va − v0πk,which yields the value mk in the statement of the theorem (both u0 and v0 are strictly positive since otherwise αx ≥ β is valid for P , contradicting that it is a cut for x¯), and then rounding this value mk either up or down since πk must be integer. The best choice is the minimum stated in the theorem.
20 G´erard Cornu´ejols

Bonami and Minoux [18] found that applying the Balas-Jeroslow strength- ening step improves the average gap closed by an additional 8 %, as compared to the lift-and-project closure, on the 35 MIPLIB instances in their experiment. Specifically, the integrality gap closed goes from 37 % to 45 %. The time to perform the strengthening step is negligible.

3.5. Sequential convexification

Theorem 7. (Balas [5]) Pn(Pn−1(...P2(P1) ...)) = conv(S).

t n−t+p Proof. By induction. Let St := {x ∈{0, 1} × R+ : Ax ≥ b}.Wewantto show Pt(Pt−1(...P2(P1) ...)) = conv(St). This is true for t =1byTheorem5 applied to j =1,soconsidert ≥ 2. Suppose that this is true for t − 1. By the induction hypothesis we have

Pt(Pt−1(...P2(P1) ...)) = Pt(conv(St−1)) and by Theorem 5 applied to j = t,thisis

=conv(conv(St−1) ∩{xt =0}) ∪ (conv(St−1) ∩{xt =1}). For any set S that lies entirely on one side of a hyperplane H, Lemma 2 states that conv(S) ∩ H =conv(S ∩ H).

Therefore conv(St−1)∩{xt =0} =conv(St−1 ∩{xt =0})andconv(St−1)∩{xt = 1} =conv(St−1 ∩{xt =1}). Thus

Pt(Pt−1(...P2(P1) ...)) = conv(conv(St−1 ∩{xt =0}))∪(conv(St−1 ∩{xt =1}))

=conv((St−1 ∩{xt =0}) ∪ (St−1 ∩{xt =1})) = conv(St).

3.6. Rank

Let P 1 denote the lift-and-project closure relative to P , i.e. n 1 P := Pj. j=1

For k ≥ 2, let P k denote the lift-and-project closure relative to P k−1.The set P k is a polyhedron. It is called the k-th lift-and-project closure relative to P . A valid inequality for P k but not P k−1 is said to be of rank k. The lift-and- project rank of P is the smallest integer such that P k =conv(S). It follows from Theorem 7 that the lift-and-project rank of P is at most n. Similarly, N+ is the Lov´asz-Schrijver closure relative to P ,andthek-th k Lov´asz-Schrijver closure N+ is defined iteratively. The Lov´asz-Schrijver rank k 1 relative to P is the smallest k such that N+ =conv(S). Since N+ ⊆ P ,the Valid Inequalities for MILPs 21

Lov´asz-Schrijver rank is at most n. Cook and Dash [24] and Goemans and Tun¸cel [32] give examples showing that the Lov´asz-Schrijver rank can be equal to n. { ∈ n n ≥ 1 } ∩Zn { ∈ Let P := x [0, 1] : j=1 xj 2 and S := P .Hereconv(S)= x n n ≥ } 1 1 [0, 1] : j=1 xj 1 . Cook and Dash show that the point ( 2n−k ,..., 2n−k ) k belongs to N+ for k ≤ n. Note that this point does not belong to conv(S)for k k

4. Mixed Integer Inequalities

n p In this section, we return to mixed integer linear sets S := {(x, y) ∈ Z+ × R+ : Ax+Gy ≤ b} where the variables xj can take general nonnegative integer values, instead of just 0 or 1 in the previous section. We present four families of valid inequalities for conv(S): Gomory mixed integer inequalities, mixed integer rounding inequalities, split inequalities, and intersection inequalities. We compare the resulting elementary closures and ad- dress computational issues. We do not address the general theory of superadditive functions [36] in this tutorial. The interested reader is referred to [46] pages 247-253 for an introduc- tion to superadditivity applied to mixed integer linear sets.

4.1. Gomory’s derivation

Let us consider a mixed integer linear set comprising a single equality constraint: n p n p S := {(x, y) ∈ Z+ × R+ : ajxj + gjyj = b}. j=1 j=1

Let b = b + f0 where 0 f0 j=1 where k is some integer. Since k ≤−1ork ≥ 0, any x ∈ S satisfies the disjunction p fj 1 − fj gj xj − xj + yj ≥ 1(7) f0 f0 f0 fj ≤f0 fj >f0 j=1 OR p fj 1 − fj gj − xj + xj − yj ≥ 1. (8) 1 − f0 1 − f0 1 − f0 fj ≤f0 fj >f0 j=1 22 G´erard Cornu´ejols 1 ≥ 2 ≥ 1 2 ≥ This is of the form a z 1ora z 1 which implies j max(aj ,aj )zj 1for any z ≥ 0. For each variable, what is the maximum coefficient in (7) and (8)? The answer is easy since one coefficient is positive and the other is negative. We get fj 1 − fj gj gj xj + xj + yj − yj ≥ 1. (9) f0 1 − f0 f0 1 − f0 fj ≤f0 fj >f0 gj >0 gj <0 We have just proved that inequality (9) is valid for S.ThisistheGomory mixed integer inequality (GMI inequality) [34]. Remark 8. In the pure integer programming case, the GMI inequality reduces to fj 1 − fj xj + xj ≥ 1 f0 1 − f0 fj ≤f0 fj >f0

1−fj fj Since < when f >f0, it follows that the GMI inequality dominates 1−f0 f0 j n fjxj ≥ f0, j=1 which is known as the fractional cut. The fractional cut can also be derived using Chv´atal’s procedure [21].

Consider now mixed integer linear sets with m inequality constraints, instead n p of one equality constraint above, S := {(x, y) ∈ Z+ × R+ : Ax + Gy ≤ b}. Let P := {(x, y) ∈ Rn × Rp : Ax + Gy ≤ b, x ≥ 0,y ≥ 0} denote the underlying polyhedron. If P = ∅ then S = ∅ and we are done. Therefore we may assume that P = ∅.Letαx + γy ≤ β be any valid inequality for P (Recall that, by Farkas’s lemma, the valid inequalities for P are of the form uAx+uGy −vx−wy ≤ ub+t m n p where u ∈ R+ , v ∈ R+, w ∈ R+, t ∈ R+). Add a nonnegative slack variable αx + γy + s = β, derive a GMI inequality (9) from this equation, and eliminate s = b − αx − γy from this inequality. The result is a valid inequality for S,inthe space Rn ×Rp of the variables x, y. Let us call these inequalities GMI inequalities for S.TheGomory mixed integer closure is obtained from P by adding all the GMI inequalities for S. In the previous section, we saw that one can optimize in polynomial time over the lift-and-project closure, the Sherali-Adams closure and the Lov´asz-Schrijver closure. A natural question is whether the same is true of the Gomory mixed integer closure. It turns out that the situation is totally different: It is NP-hard to optimize a linear function over the Gomory mixed integer closure relative to a polyhedron P (Caprara and Letchford [20], Cornu´ejols and Li [28]). Equivalently, given a point (x, y) ∈ P \ S, it is NP-hard to find a GMI cut with respect to (x, y) or show that none exists. A similar NP-hardness result was proved earlier by Eisenbrand [30] for the Chv´atal closure [21]. Note that this is in contrast with the problem of finding a GMI cut that cuts off a basic solution (x, y) ∈ P \ S. Indeed, any row of the simplex tableau where xj is fractional for some j =1,...,n can be used to generate a GMI cut. We illustrate this in the next section. Valid Inequalities for MILPs 23

4.2. Gomory mixed integer cuts from the simplex tableau

We illustrate the application of GMI cuts on a small example. max x +2y −x +y ≤ 2 x +y ≤ 5 2x −y ≤ 4 x ∈ Z+,y∈ R+.

y 3.5 3 GMI cut

011.523x Fig. 10. Gomory mixed integer cut from the simplex tableau

Add nonnegative slacks s1,s2,s3 and solve the LP relaxation max z where z −x −2y =0 −x +y +s1 =2 x +y +s2 =5 2x −y +s3 =4.

The optimal tableau is

z +0.5s1 +1.5s2 =8.5 y +0.5s1 +0.5s2 =3.5 x −0.5s1 +0.5s2 =1.5 0.5s1 −0.5s2 +s3 =4.5 and the corresponding solution is x =1.5, y =3.5. Since x is not integer, we generate a cut from the row in which x is basic, namely

x − 0.5s1 +0.5s2 =1.5

Here f0 =0.5 and applying the GMI formula, we get

s1 + s2 ≥ 1. 24 G´erard Cornu´ejols

Since s1 + s2 =7− 2y, we can express the cut in the (x, y)-space:

y ≤ 3.

Adding this cut and solving the strengthened LP relaxation gives the solution x =2,y =3.Since¯x is integer in this solution, it is the optimal solution of the mixed integer program. Gomory [34] introduced GMI cuts in the early 60s but, for decades, they were not used in commercial integer programming software. Most textbooks from the 80s considered them impractical for various reasons. They were revived in the 90s when Balas, Ceria, Cornu´ejols and Natraj [8] demonstrated that a branch-and-cut algorithm based on GMI cuts was practical and superior to state- of-the-art algorithms of the time. GMI cuts turn out to be surprisingly good in practice (Bixby, Gu, Rothberg, Wunderling [14]). On 41 MIPLIB instances, adding the GMI cuts generated from the optimal simplex tableau reduces the integrality gap by 24 % on average [17]. Greater improvements can be obtained by performing several rounds of cutting. A round consists of solving the current linear programming relaxation, generating a GMI cut for each basic variable xi with a fractional valuex ¯i in the current solution, and updating the linear program by adding these cuts. Since GMI cuts from the simplex tableau are extremely easy to generate and effective in reducing the integrality gap, they are widely used in commercial codes today. Numerical issues need to be addressed, of course: in order to avoid numerical instability (and possibly cutting off the optimal solution), one usually discards cuts that have too large a ratio between the largest and smallest coefficient; in order to avoid fill in of the basis inverse, one also discards cuts that are too dense.

4.3. Improving mixed integer Gomory cuts by lift-and-project

In this section we return to mixed 0-1 programming and the correspondence between basic feasible solutions of the cut generating LP (5) and basic solutions (possibly infeasible) of the usual LP relaxation (R) introduced in Section 3.3. Balas and Perregaard [10] showed how this correspondence can be used to im- prove Gomory mixed integer cuts by mimicking simplex pivots in (5) through pivots in (R). The simplex tableaux of (5) and (R) will be referred to as large and small respectively. Let xj = aj0 − ajhxh (10) h∈J

be a row of the small optimal simplex tableau such that 0 subsets M1 and M2, such that h ∈ M1 if ajh < 0, and h ∈ M2 if ajh > 0(h ∈ J such that ajh =0cangointoeither subset). Then eliminating α, β from (5), the n columns indexed by M1 ∪ M2 Valid Inequalities for MILPs 25 together with the two columns indexed by u0 and v0 define a feasible basis of the resulting system of n + 2 equations. The strengthened lift-and-project cut associated with this basic feasible solution to (5) is equivalent to the GMI cut from (10). To evaluate the GMI cut generated from the small simplex tableau (10) as a lift-and-project cut, we calculate the reduced costs in the large tableau of the nonbasic variables of the above solution to (5). Each row xi of the small tableau corresponds to a pair of columns of the large tableau, associated with variables ui and vi. The reduced costs r(ui), r(vi) of these variables in the large tableau are known simple functions of the entries aih and ajh,forh ∈ J,ofrowsj and i of the small tableau. If they are all nonnegative, the current large tableau is optimal, hence the GMI cut from (10) cannot be improved. Otherwise, the cut can be improved by executing a pivot in a row i of the small tableau, such that r(ui) < 0orr(vi) < 0. To identify the nonbasic variable xk to replace xi in the basis of the small tableau, we calculate for each h ∈ J the objective function value f(aih)of(5) resulting from the corresponding exchange in the large tableau. This value is a known simple function of the ratio ajh/aih and of the coefficients of rows j and i of the small tableau. Any column h for which f(aih) < 0 is a candidate for an improving pivot, and the most negative value indicates the best column k. Executing the pivot in the small tableau that exchanges xi for xk yields a new simplex tableau (whose solution is typically infeasible), whose j-th row (the same j as before!) is of the form xj = aj0 + tai0 − (ajh + taih)xh, (11) h∈J∪i\k with t := ajk/aik. The GMI cut from (11) is then stronger than the one from (10), in the sense that it cuts off the LP optimum of (R) by a larger amount. These steps can then be repeated with (11) replacing (10) for as long as improvements are possible. Practical experience shows that in about three quarters of the cases GMI cuts from the optimal simplex tableau can be improved by the pivoting procedure described above. On the other hand, improvements beyond 10 pivots are not frequent, and beyond 20 pivots they are very rare. This procedure was extensively tested and has been incorporated into the mixed integer module of XPRESS, with computational results reported in [50].

4.4. K-cuts and reduce-and-split cuts

What happens if, instead of generating a GMI inequality from an equation n p j=1 ajxj + j=1 gjyj = b, we first multiply the equation by k before gen- erating the GMI inequality? Let us try it on an example. 26 G´erard Cornu´ejols

max 10x1 +13x2 10x1 +14x2 ≤ 43 x1,x2 ∈ Z+.

Let z =10x1 +13x2 and introduce a nonnegative integer slack x3.The optimal tableau is

z +x2 +x3 =43 x1 +1.4x2 +0.1x3 =4.3

1−0.4 0.1 ≥ ≥ The GMI cut is 1−0.3 x2 + 0.3 x3 1, i.e. 18x2 +7x3 21. If we multiply the equation x1 +1.4x2 +0.1x3 =4.3byk =2,weget 2x1 +2.8x2 +0.2x3 =8.6. The resulting GMI cut is 3x1 +2x2 ≥ 6. If we multiply by k = 3, we get the GMI cut 2x2 +3x3 ≥ 9. If we multiply by k =4,weget2x2 +3x3 ≥ 4. If we multiply by k =5,wegetx3 ≥ 1.

x3

3 3-cut

2 GMI cut 5-cut 1 4-cut 2-cut x 0 12345 2 Fig. 11. K-cuts for the 2-variable example

Clearly, these simple operations on the equation x1 +1.4x2 +0.1x3 =4.3 produce very different GMI cuts. See Figure 11.

More generally, if we have an optimal simplex tableau, what sort of operations should we perform on the rows in order to generate useful GMI cuts? n p Observe that, if we multiply j=1 ajxj + j=1 gjyj = b by a factor k>1, the coefficients of the xj variables in the GMI inequality remain between 0 and 1, but the coefficients of the yj variables are multiplied by the factor k,which makes the inequality weaker. Valid Inequalities for MILPs 27

This suggests the following idea (Andersen, Cornu´ejols and Li [2]): Take linear combinations of the constraints in order to produce equations n p ajxj + gjyj = b j=1 j=1 where the norm of the vector g is small, and generate GMI inequalities from these “reduced” equations. Specifically, consider the rows of the optimal simplex tableau corresponding to the basic variables xi (but not the basic yis): − − ∈ xi = xi aij xj gihyh for i B. j∈J h∈H { } This gives a set of vectors gi i∈B. Iteratively reduce the norm of at least one vector gi in this set by integral combinations of the others (Integrality is useful to guarantee that the basic variables end up with integral coefficients when { } combining the rows). The procedure for reducing the vectors gi i∈B is similar to Lov´asz’s basis reduction algorithm [42]. It terminates with a set of vectors { } gi i∈B with reduced norms. The corresponding combinations of the tableau rows are i − − ∈ π xB = xi aij xj gihyh for i B. j∈J h∈H ∈ Z The GMI inequalities generated from the equations where xi cut off (x, y) since only the nonbasic variables end up with nonzero coefficients in these in- equalities (indeed, the vectors πi are integral). These cuts are called reduce-and- split cuts (the name comes from the equivalence between GMI inequalities and split inequalities, which we prove in Theorem 10). Computational experiments reported in Andersen, Cornu´ejols and Li [2] show that the reduce-and-split cuts are usually quite different from the GMI cuts generated from the rows of the optimal tableau and that, in some cases, they can be significantly stronger. To illustrate this, we present a few instances of the MIPLIB where the improvement was particularly striking. The gap closed is reported after 20 rounds of cutting. The last two columns give the number of nodes in a branch-and-bound algorithm using these cuts in the formulation. Name GMI gap R&S gap GMI nodes R&S nodes flugpl 14% 100% 184 0 gesa2 46% 97% 743 116 gesa2o 92% 98% 9145 75 mod008 47% 88% 1409 82 pp08a 83% 92% 7467 745 rgn 15% 100% 874 0 vpm1 44% 98% 7132 1 vpm2 41% 61% 38946 4254 In other instances the reduce-and-split cuts did not improve on the GMI cuts from the optimal tableau. Therefore it seems that a hybrid approach that uses both types of cuts is a reasonable strategy. 28 G´erard Cornu´ejols

The idea of using basis reduction to generate strong GMI cuts has been suggested independently by K¨oppe and Weismantel [39].

4.5. Mixed integer rounding inequalities

Nemhauser and Wolsey [46], [47] and Wolsey [57] introduced two definitions of mixed integer rounding (MIR) inequalities. We follow Wolsey [57].

Lemma 3. Consider the 2-variable mixed integer set S := {(x, y) ∈ Z × R+ : x − y ≤ b}. Let f0 := b −b. Then the inequality 1 x − y ≤b (12) 1 − f0 is a valid inequality for conv(S). Proof. We show the validity of (12) in two different ways, for x ≤b and x ≥b +1. If x ≤b, then adding this inequality to 1 times −y ≤ 0 yields (12). 1−f0 If x ≥b+1, then adding f0 times −x ≤−b−1to 1 times x−y ≤ b 1−f0 1−f0 yields (12).
Note that the assumption y ≥ 0 is critical in the above derivation, whereas we can have indifferently x ∈ Z or x ∈ Z+.

y

cut

x b b

Fig. 12. Illustration of Lemma 3

Theorem 8. Consider a mixed integer set defined by a single inequality: S := n p {(x, y) ∈ Z+ × R+ : ax + gy ≤ b}. Let f0 := b −b and fj := aj −aj.Then the inequality n + (fj − f0) 1 (aj + )xj + gjyj ≤b (13) 1 − f0 1 − f0 j=1 j:gj <0 Valid Inequalities for MILPs 29

is a valid inequality for conv(S).

Proof. Relax ax + gy ≤ b to ajxj + ajxj + gjyj ≤ b. j:fj ≤f0 j:fj >f0 j:gj <0

The validity of the relaxation follows from x ≥ 0andy ≥ 0. Let w := ajxj + ajxj and j:fj ≤f0 j:fj >f0 z := − gjyj + (1 − fj)xj . j:gj <0 j:fj >f0

We have w − z ≤ b and since w ∈ Z and z ∈ R+, we can apply Lemma 3. Thus 1 w − z ≤b. 1 − f0 Subtituting w and z yields (13).

Wolsey [57] calls the inequality (13) MIR inequality.

Lemma 4. Consider a mixed integer set defined by a single inequality: S := n p {(x, y) ∈ Z+ × R+ : ax + gy ≤ b}. The MIR inequality (13) is identical to the GMI inequality.

Proof. The Gomory mixed integer inequality is obtained by adding a slack vari- able ax + gy + s = b, generating (9) in the (x, y, s)-space: fj 1 − fj gj gj 1 xj + xj + yj − yj + s ≥ 1 f0 1 − f0 f0 1 − f0 f0 fj ≤f0 fj >f0 gj >0 gj <0

and substituting s = b − ax − gy in this inequality to get the GMI inequality in the (x, y)-space. It is straightforward to check that the resulting inequality is identical to (13).

n Next we return to the general mixed integer linear set S := {(x, y) ∈ Z+ × p R+ : Ax + Gy ≤ b}. The proof of the previous lemma can be slightly modified to show the next result.

Lemma 5. Consider a mixed integer set with m constraints S := {(x, y) ∈ n p Z+ × R+ : Ax + Gy ≤ b}.Lets := b − Ax − Gy be a nonnegative vector of slack m variables. For any λ ∈ R ,leta := λA, g := λG, β := λb, fj := aj −aj and f0 := β −β. The GMI inequality generated from λAx + λGy + λs = λb is

n + (fj − f0) 1 1 (aj + )xj + gjyj + λisi ≤β (14) 1 − f0 1 − f0 1 − f0 j=1 j:gj <0 i:λi<0 30 G´erard Cornu´ejols

n p Any valid inequality for P := {(x, y) ∈ R+×R+ : Ax+Gy ≤ b, x ≥ 0,y ≥ 0} can be used to generate a MIR inequality (13) valid for S. Define the MIR n p closure relative to P := {(x, y) ∈ R+ × R+ : Ax + Gy ≤ b} as the set obtained from P by adding all these MIR inequalities. Lemma 4 implies that the MIR closure contains the GMI closure. This inclusion can be strict as observed by Bonami and Cornu´ejols [16], and Dash, G¨unl¨uk and Lodi [29]: For the integer 2 set S := {(x1,x2) ∈ Z+ :2x1 + x2 ≤ 2, −2x1 + x2 ≤ 0}, the inequality x2 ≤ 0is 1 1 a GMI inequality (to see this, use λ1 = 4 and λ2 = − 4 ) but one can show that it is not a MIR inequality. Thus, in general, inequalities (14) can be stronger than (13). This is because the vector λ in Lemma 5 can have both positive and negative components, whereas (13) corresponds to the special case of (14) where only nonnegative multipliers λi are used. With the other definition of MIR inequalities (Nemhauser and Wolsey [47]), one can show that the MIR closure is identical to the GMI closure [47], [26].

Marchand and Wolsey [44] showed that several classical inequalities in integer programming, such as flow cover inequalities, are MIR inequalities. They imple- mented an aggregation heuristic to generate MIR cuts with excellent computa- tional results in the resolution of mixed integer linear programs. For example, on 41 MIPLIB instances, the Marchand-Wolsey aggregation heuristic (as available in the COIN-OR repository) reduces the integrality gap by 23% on average [17].

4.6. Split inequalities

Let P := {(x, y) ∈ Rn × Rp : Ax + Gy ≤ b} where A, G, b have rational entries, n p n and let S := P ∩ (Z × R ). For π ∈ Z and π0 ∈ Z, define

Π1 := P ∩{(x, y): πx ≤ π0}

Π2 := P ∩{(x, y): πx ≥ π0 +1}

Clearly S ⊆ Π1 ∪ Π2. Therefore any inequality cx + hy ≤ c0 that is valid for Π1 ∪ Π2 is also valid for S. An inequality cx + hy ≤ c0 is called a split inequality n [25] if there exists (π, π0) ∈ Z ×Z such that cx+hy ≤ c0 is valid for Π1 ∪Π2 (see Figure 13). The disjunction πx ≤ π0 or πx ≥ π0 + 1 is called a split disjunction. Many of the inequalities that we have studied in this tutorial are split in- ∩n equalities. Lift-and-project inequalities (i.e. the valid inequalities for j=1Pj)are split inequalities as a consequence of Theorem 5 (the split disjunction is xj ≤ 0 ≥ or xj 1). Strengthened lift-and-project cuts are also split inequalities (recall n ≥ − n ≥ the split disjunction k=1 πkxk 0or k=1 πkxk 1usedintheproofof Theorem 6). GMI inequalities are split inequalities (indeed the disjunction (7) or (8) is a split disjunction). k-cuts and reduce-and-split cuts are split inequal- ities since they are special types of GMI cuts. MIR inequalities are also split inequalities (remember the derivation in the proof of Lemma 3).

The intersection of all split inequalities, denoted by P 1, is called the split closure relative to P . Valid Inequalities for MILPs 31

πx ≤ π0 πx ≥ π0 +1

split inequality

Π1 P Π2

Fig. 13. A split inequality

Theorem 9. (Cook, Kannan and Schrijver [25]) If P is a rational polyhedron, the split closure relative to P is a rational polyhedron.

For k ≥ 2, P k denotes the split closure relative to P k−1 anditiscalledthe k-th split closure relative to P . It follows from the above theorem that P k is a polyhedron. Unlike for the pure integer case [21], [52], there is in general no finite r such that P r =conv(S) in the mixed integer case, as shown by the following example [25].

2 Example 1. Let S := {(x, y) ∈ Z+ × R+ : x1 ≥ y, x2 ≥ y, x1 + x2 +2y ≤ 2}. 3 Starting from P := {(x1,x2,y) ∈ R+ : x1 ≥ y, x2 ≥ y, x1 + x2 +2y ≤ 2},we claim that there is no finite r such that P r =conv(S). Toseethis,notethatP is a simplex with vertices O =(0, 0, 0), A =(2, 0, 0), 1 1 1 B =(0, 2, 0) and C =(2 , 2 , 2 ) (see Figure14). S is contained in the plane y =0. 1 1 More generally, consider a simplex P with vertices O, A, B and C =(2 , 2 ,t)with t>0. Let C1 = C,letC2 be the point on the edge from C to A with coordinate x1 =1andC3 the point on the edge from C to B with coordinate x2 =1. Observe that no split disjunction removes all three points C1, C2, C3.LetQi be the intersection of all split inequalities that do not cut off Ci. All split inequalities 1 belong to at least one of these three sets, thus P = Q1 ∩ Q2 ∩ Q3.LetSi be the 1 simplex with vertices O, A, B, Ci. Clearly, Si ⊆ Qi.ThusS1 ∩ S2 ∩ S3 ⊆ P .It 1 1 t 1 1 t 1 is easy to verify that ( 2 , 2 , 3 ) ∈ Si for i =1, 2and3.Thus(2 , 2 , 3 ) ∈ P .By 1 1 t ∈ k induction, ( 2 , 2 , 3k ) P .

Remark 9. For mixed 0,1 programs, Theorem 7 implies that P n =conv(S)(In- deed, the lift-and-project polytope P1 contains the split closure relative to P by nd Theorem 5. Similiarly, P2(P1)) contains the 2 split closure, etc).

Example 2. Cornu´ejols and Li [27] observed that the n-th split closure is needed for 0,1 programs, i.e. there are examples where P k =conv( S) for all k

C

B

OA

Fig. 14. Example showing that the split rank can be unbounded use the following well-known polytope studied by Chv´atal, Cook, and Hartmann [22]: 1 P := {x ∈ [0, 1]n : x + (1 − x ) ≥ , for all J ⊆{1, 2, ···,n}} CCH j j 2 j∈J j∈J

n 1 Let Fj be the set of all vectors x ∈ R such that j components of x are 2 and each of the remaining n − j components are equal to 0 or 1. The polytope PCCH is the convex hull of F1.

n 1 Lemma 6. If a polyhedron P ⊆ R contains Fj, then its split closure P con- tains Fj+1.

n Proof. It suffices to show that, for every (π, π0) ∈ Z × Z, the polyhedron Π = conv((P ∩{x : πx ≤ π0})∪(P ∩{x : πx ≥ π0 +1})) contains Fj+1.Letv ∈ Fj+1 1 and assume w.l.o.g. that the first j + 1 elements of v are equal to 2 .Ifπv ∈ Z, then clearly v ∈ Π.Ifπv ∈ Z, then at least one of the first j + 1 components of π is nonzero. Assume w.l.o.g. that π1 > 0. Let v1,v2 ∈ Fj be equal to v except v1+v2 for the first component which is 0 and 1 respectively. Notice that v = 2 . Clearly, each of the intervals [πv1,πv]and[πv, πv2] contains an integer. Since πx is a continuous function, there are pointsv ˜1 on the line segment conv(v, v1) andv ˜2 on the line segment conv(v, v2)withπv˜1 ∈ Z and πv˜2 ∈ Z. This means thatv ˜1 andv ˜2 are in Π.Sincev ∈ conv(˜v1, v˜2), this implies v ∈ Π.

Starting from P = PCCH and applying the lemma recursively, it follows that the (n − 1)-st split closure relative to PCCH contains Fn, which is nonempty. n Since conv(PCCH ∩{0, 1} )isempty,then-th split closure is needed to obtain n conv(PCCH ∩{0, 1} ). End of Example 2.

Remark 10. In view of Example 1 showing that no bound may exist on the split rank when the integer variables are general, and Remark 9 showing that the rank is always bounded when they are 0,1 valued, one is tempted to convert general integer variables into 0,1 variables. For a bounded integer variable 0 ≤ x ≤ u, there are several natural tranformations: (i) a binary expansion of x (see Owen and Mehrotra [49]); Valid Inequalities for MILPs 33 u ≤ ∈{ } (ii) x = i=1 izi, zi 1, zi 0, 1 (see Sherali and Adams [55] and K¨oppe, Louveaux and Weismantel [38]); u ≤ ∈{ } (iii) x = i=1 zi, zi zi−1, zi 0, 1 (see Roy [51]). Roy [51] also shows that practical benefits can be gained from such a transfor- mation.

4.7. Chv´atal inequalities

n p Let P := {(x, y) ∈ R+ × R+ : Ax + Gy ≤ b} be a rational polyhedron and let n p S := {(x, y) ∈ Z+ × R+ : Ax + Gy ≤ b}. Define a Chv´atal inequality to be an n inequality πx ≤ π0 with (π, π0) ∈ Z × Z,where

P ∩{(x, y): πx ≥ π0 +1} = ∅.

It follows from the definition that Chv´atal inequalities are split inequalities where one of the sets Π1 or Π2 is empty.

Lemma 7. Chv´atal inequalities are GMI inequalities.

Proof. The lemma holds when P = ∅. Assume now that P = ∅.Let

β =maxπx (x, y) ∈ P.

Then πx ≤ β is a valid inequality for P .SinceP ∩{(x, y): πx ≥ π0 +1} = ∅, it follows that β<π0 + 1. Applying the MIR formula (13) to πx ≤ β, it follows that the inequality πx ≤ π0 is a MIR inequality. By Lemma 4, this inequality is a GMI inequality.

Define the Chv´atal closure relative to P as the intersection of P with all the Chv´atal inequalities. How does the Chv´atal closure compare to the lift-and- project closure? The answer is that neither is included in the other in general, 2 as shown by the following 2-variable examples: For P := {x ∈ R+ : x2 ≤ 2 2x1, 2x1 + x2 ≤ 2} and S := P ∩{0, 1} , the inequality x2 ≤ 0 is a lift- and-project inequality but not a Chv´atal inequality. On the other hand, for 2 2 P := {x ∈ R+ : x1 + x2 ≤ 1.5,x1 ≤ 1,x2 ≤ 1} and S := P ∩{0, 1} , the inequality x1 + x2 ≤ 1isaChv´atal inequality but not a lift-and-project inequality. Although it is NP-hard to optimize over the Chv´atal closure, there are empir- ical results on its strength. Bonami, Cornu´ejols,Dash,Fischetti,Lodi[17]found that the Chv´atal closure closes at least 29 % of the integrality gap on average on 41 MIPLIB instances (all the MIPLIB 3 instances that have at least one contin- uous variable and nonzero integrality gap). For the remaining 24 instances (pure integer programs in MIPLIB 3 with nonzero integrality gap), Fischetti and Lodi [31] found that the Chv´atal closure closes at least 63 % of the integrality gap on average. 34 G´erard Cornu´ejols

4.8. Equivalence between split inequalities and GMI inequalities

Nemhauser and Wolsey [46], [47] studied the relation between split closure and GMI closure. The equivalence proof given here is based on [27]. It uses Lemma 1.

n p Theorem 10. Let P := {(x, y) ∈ R+ × R+ : Ax + Gy ≤ b} be a rational polyhedron and let S := P ∩(Zn ×Rp). The split closure relative to P is identical to the Gomory mixed integer closure relative to P .

Proof. We may assume that the constraints x ≥ 0andy ≥ 0 are part of Ax + Gy ≤ b in the description of P . Consider first a GMI inequality. Its derivation in Section 4.1 was obtained by  −   −   arguing that k = b fj ≤f0 aj xj fj >f0 aj xj is an integer, and either k ≤−1ork ≥ 0. This is a disjunction of the form πx ≤ π0 or πx ≥ π0 +1with n (π, π0) ∈ Z × Z. Thus the derivation of the GMI inequality implies that it is a split inequality.

Conversely, let cx+ hy ≤ c0 be a split inequality. Let πx ≤ π0 or πx ≥ π0 +1 denote the split disjunction used in deriving this inequality, where (π, π0) ∈ n Z × Z,andletΠ1, Π2 be the corresponding intersections with P . First assume that Π2 = ∅. Then the inequality cx + hy ≤ c0 is valid for Π1. Since all the inequalities that define Π1 are valid for P except possibly for πx ≤ π0, it suffices to show that πx ≤ π0 is a GMI inequality. This follows from the fact that πx ≤ π0 is a Chv´atal inequality (since Π2 := P ∩{(x, y): πx ≥ π0 +1} = ∅) and from Lemma 7. A similar argument holds when Π1 = ∅, so we now assume that Π1 = ∅ and Π2 = ∅. By Lemma 1, there exist α, β ∈ R+ such that

cx + hy − α(πx − π0) ≤ c0 and (15)

cx + hy + β(πx − (π0 +1))≤ c0 (16)

are both valid for P . We can assume α>0andβ>0 since, otherwise, cx+hy ≤ c0 is valid for P and therefore also for its Gomory mixed integer closure. We now apply the Gomory mixed integer procedure to (15) and (16). Introduce slack variables s1 and s2 in (15) and (16) respectively and subtract (15) from (16).

(α + β)πx + s2 − s1 =(α + β)π0 + β.

Dividing by α + β we get

s2 s1 β πx + − = π0 + . α + β α + β α + β Valid Inequalities for MILPs 35

β From this equation, we can derive a GMI inequality. Note that f0 = α+β and that the continuous variable s2 has a positive coefficient while s1 has a negative coefficient. So the GMI inequality is

1 1 α+β α+β s2 + s1 ≥ 1 β − β α+β 1 α+β which simplifies to 1 1 s1 + s2 ≥ 1. α β

We now replace s1 and s2 as defined by the equations (15) and (16) to get the GMI inequality in the space of the x, y variables. The resulting inequality is

cx + hy ≤ c0.

Therefore cx + hy ≤ c0 is a GMI inequality.

The split closure is identical to the GMI closure. How tight is it in practice? Balas and Saxena [11] addressed this question by formulating the separation problem for the split closure as a parametric mixed integer linear program with a single parameter in the objective function and the right hand side. They found that the split closure closes 82 % of the integrality gap on average for 33 MI- PLIB instances of mixed integer programs, and 71 % on average for 24 MIPLIB instances of pure integer programs. This experiment shows that the split closure is surprisingly strong. It is interesting to compare the above 82 % figure with the 48 % obtained by adding GMI cuts from the optimal tableau + MIR cuts + lift-and-project cuts [17]. This is a clear indication that there are useful split cuts that are currently not exploited in integer programming codes. Finding deep split cuts efficiently remains a challenging practical issue.

4.9. Intersection cuts

Intersection cuts were introduced by Balas [4]. They are obtained from a basis of the linear programming relaxation and a convex set that contains the cor- responding basic solution but no integer feasible solution in its interior. In this section, we focus on convex sets derived from split disjunctions. For convenience, we assume that the constraints are in equality form. Let S := P ∩ (Zn × Rp) with n+p P := {x ∈ R+ : Ax = b} where A ∈ Rm×(n+p) and b ∈ Rm. Wlog assume that A is of full row rank. Let B index m linearly independent columns of A (B is a basis)andletJ index the remaining columns of A (the nonbasic variables). Let P (B) be the relaxation of P obtained by deleting the nonnegativity constraints on the basic variables:

n+p P (B):={x ∈ R : Ax = b and xj ≥ 0forj ∈ J}. (17) 36 G´erard Cornu´ejols

The convex hull of P (B) ∩ (Zn × Rp) was studied by Gomory [35] under the name of corner polyhedron. Clearly, S is contained in the corner polyhedron and, therefore, valid inequalities for the corner polyhedron are also valid for S. We may write P (B)=¯x + C,whereC is the polyhedral cone C := {x ∈ n+p R : Ax =0 andxj ≥ 0for j ∈ J},and¯x is the basic solution that solves the system Ax = b and xj =0forj ∈ J: xi =¯xi − a¯ij xj,i∈ B. (18) j∈J The extreme rays of C are the following vectors rj for j ∈ J:   −a¯kj if k ∈ B, j 1ifk = j, rk :=  (19) 0ifk ∈ J \{j}. { j} { j } Thus P (B)=¯x +cone( r j∈J ), where cone( r j∈J ) denotes the polyhedral { j} cone generated by the vectors r j∈J . We now derive the intersection cut. Consider an arbitrary split disjunction n p πx ≤ π0 or πx ≥ π0 +1,where π ∈ Z ×{0} and π0 ∈ Z. Assumex ¯ violates the split disjunction, and define ' := πx¯ − π0 to be the amount by whichx ¯ violates the first term of the disjunction. Since π0 <πx<π¯ 0 +1,wehave0<'<1. Also, for j ∈ J, define scalars:   −  j πrj if πr < 0, 1− j α := j if πr > 0, (20) j  πr +∞ otherwise.

πx ≤ π πx ≥ π +1 0 x¯ 0

2 x¯ + α2r

1 x¯ + α1r r2

r1

Fig. 15. Intersection cut

j The interpretation of αj is the following. Consider the half-linex ¯ + αr , j where α ∈ R+, starting fromx ¯ in the direction r .Thevalueαj is the smallest Valid Inequalities for MILPs 37

j α ∈ R+ such thatx ¯+αr satisfies the split disjunction. In other words, the point j j x¯ + αjr is the intersection of the half-line starting inx ¯ in direction r with the hyperplane πx = π0 or the hyperplane πx = π0 + 1 (see Figure 15). Note that j αj =+∞ when the direction r is parallel to the hyperplane πx = π0.Given the numbers αj for j ∈ J, the intersection cut associated with B and the split disjunction πx ≤ π0 or πx ≥ π0 +1,isgivenby: x j ≥ 1. (21) ∈ αj j J This inequality is valid for the corner polyhedron conv P (B) ∩ (Zn × Rp) since it is a split inequality. In fact, the intersection cut gives a complete de- scription of the set of points in P (B) that satisfy the split disjunction: xj conv P (B) ∩ ({x : πx ≤ π0}∪{x : πx ≥ π0 +1}) = P (B) ∩{x : ≥ 1}. αj j∈J Andersen, Cornu´ejols and Li [1] showed that intersection cuts are sufficient to describe the split closure P 1 relative to P .LetB denote the set of all bases of A.Wehave: Theorem 11. 1 P = conv P (B) ∩ ({x : πx ≤ π0}∪{x : πx ≥ π0 +1}) . n p B∈B π∈Z ×{0} ,π0∈Z The following lemma shows that the GMI cuts derived from rows of the simplex tableau are intersection cuts: Lemma 8. Let B be a basis of A,andletx¯ be the corresponding basic solution. Also, let xi be a basic integer constrained variable, and suppose x¯i is fractional. The GMI cut obtained from the row of the simplex tableau in which xi is basic is given by the inequality xj ≥ 1,whereα is computed using formula (20) j∈J αj j with π0 := x¯i,andforj =1,...,n:     ∈ ≤  a¯ij if j J and fj f0, a¯ij  if j ∈ J and fj >f0, πj :=  (22)  1 if j = i, 0 otherwise.

Proof. Let us compute αj for the above disjunction using formula (20), where j ∈ J.Wehave:

' = πx¯ − π0 =¯xi −x¯i = f0. Using (19) and (22), we get   −fj if 1 ≤ j ≤ n and fj ≤ f0, j j − j − ≤ ≤ πr = πiri πjrj =  1 fj if 1 j n and fj >f0, (23) −a¯ij if n +1≤ j ≤ n + p.

Now αj follows from formula (20). This yields the GMI cut as claimed.
38 G´erard Cornu´ejols

5. Conclusion

Lasserre GMI ≡ Split ≡ Intersection Lov´asz-Schrijver

Chv´atal Lift−and−project Sherali−Adams + strengthening Reduce−and−split

MIR GMI from basic Lift−and−project heuristics feasible solutions

Fig. 16. Relations between families of cuts

n p This tutorial considered mixed integer linear sets S := P ∩ (Z+ × R+)where n p P := {(x, y) ∈ R+ × R+ : Ax + Gy ≤ b} is a polyhedron. It presented several families of valid inequalities for S and compared the corresponding elementary closures. See Figure 16. The Gomory mixed integer (GMI) closure is identical to the split closure. This set can also be obtained using intersection cuts generated from all the basic solutions (both feasible and infeasible). Computational exper- iments show that the split closure is surprisingly tight (it closes over 75 % of the integrality gap on average on MIPLIB 3 instances). Unfortunately separating or optimizing over the split closure is NP-hard. This justifies our interest in families of split cuts that can be generated efficiently: GMI cuts from the optimal basis, MIR cuts obtained heuristically, reduce-and-split cuts and lift-and-project cuts. They appear in the bottom of the figure. On the right hand side of Figure 16 appear elementary closures that are defined for mixed 0,1 programs only, from weakest at the bottom to tightest at the top. Integer programming solvers went from using no general cutting planes twenty years ago to using many rounds of such cuts nowadays. Much larger instances can be solved, but this aggressive cutting may sometimes result in the optimum solution being cut off due to numerical difficulties with cuts from later rounds. Have we reached the limits of what can be acheived with general cuts? I do not think so. The split closure is surprisingly tight and integer programming solvers should probably do a better job of approximating it before generating cuts of higher rank. We conclude with three possible research directions. The previous paragraph suggests that one should look for new classes of deep split cuts that can be sepa- rated rapidly, or just use the currently known families more efficiently. Variable transformations are intriguing (see Remark 10) and deserve more attention in the context of cutting planes. Obtaining compact higher dimensional formulations by applying Theorem 4 is another promising direction for research. Valid Inequalities for MILPs 39

Aknowledgements: This tutorial benefited from PhD students’ feedback in short courses in Lausanne (2004), Lisbon (2005) and Zinal (2006) and from discussions with colleagues. I am most grateful to Egon Balas for his help with Theorem 4 and Section 4.3. Thanks also to Claude Lemarechal, Laurence Wolsey and the referees for their useful comments on an earlier draft.

References

1. K. Andersen, G. Cornu´ejols and Y. Li, Split closure and intersection cuts, Mathematical Programming A 102 (2005) 457–493. 2. K. Andersen, G. Cornu´ejols and Y. Li, Reduce-and-split cuts: Improving the performance of mixed integer Gomory cuts, Management Science 51 (2005) 1720–1732. 3. A. Atamt¨urk, Strong formulations of robust mixed 0–1 programming, to appear in Math- ematical Programming (2006). 4. E. Balas, Intersection cuts – A new type of cutting planes for integer programming, Op- erations Research 19 (1971) 19–39. 5. E. Balas, Disjunctive programming: properties of the convex hull of feasible points, GSIA Management Science Research Report MSRR 348, Carnegie Mellon University (1974), published as invited paper in Discrete Applied Mathematics 89 (1998) 1–44. 6. E. Balas, Disjunctive programming and a hierarchy of relaxations for discrete optimization problems, SIAM Journal on Algebraic and Discrete Methods 6 (1985) 466–486. 7. E. Balas, S. Ceria and G. Cornu´ejols, A lift-and-project cutting plane algorithm for mixed 0-1 programs, Mathematical Programming 58 (1993) 295–324. 8. E. Balas, S. Ceria, G. Cornu´ejols and R.N. Natraj, Gomory cuts revisited, Operations Research Letters 19 (1996) 1–9. 9. E. Balas and R. Jeroslow, Strengthening cuts for mixed integer programs, European Jour- nal of Operations Research 4 (1980) 224–234. 10. E. Balas and M. Perregaard, A Precise correspondence between lift-and-project cuts, sim- ple disjunctive cuts and mixed integer Gomory cuts for 0-1 programming, Mathematical Programming B 94 (2003) 221–245. 11. E. Balas and A. Saxena, Optimizing over the split closure, Tepper School of Business Management Science Research Report MSRR 674, Carnegie Mellon University (2006). 12. D. Bienstock and M. Zuckerberg, Subset algebra lift operators for 0-1 integer programming, SIAM Journal on Optimization 15 (2004) 63–95. 13. R.E. Bixby, S. Ceria, C.M. McZeal and M.W.P. Savelsbergh, An updated mixed integer programming library: MIPLIB 3.0, Optima 58 (1998) 12–15. 14. R.E. Bixby, Z. Gu, E. Rothberg and R. Wunderling, Mixed integer programming: A progress report, M. Gr¨otschel ed., The sharpest cut: The impact of Manfred Padberg and his work, MPS/SIAM Series in Optimization (2004) 309–326. 15. P. Bonami, Etude et mise en oeuvre d’approches poly´edriques pour la r´esolution de pro- grammes en nombres entiers ou mixtes g´en´eraux, PhD Thesis, Universit´e de Paris 6 (2003). 16. P. Bonami and G. Cornu´ejols, On MIR inequalities (2006). 17. P. Bonami, G. Cornu´ejols, S. Dash, M. Fischetti and A. Lodi, Projected Chv´atal-Gomory cuts for mixed integer linear programs, technical report (2005). 18. P. Bonami and M. Minoux, Using rank-1 lift-and-project closures to generate cuts for 0–1 MIPs, a computational investigation, Discrete Optimization 2 (2005) 288–307. 19. S. Burer and D. Vandenbussche, Solving lift-and-project relaxations of binary integer programs, SIAM Journal on Optimization 16 (2006) 726–750. 20. A. Caprara and A.N. Letchford, On the separation of split cuts and related inequalities, Mathematical Programming B 94 (2003) 279–294. 21. V. Chv´atal, Edmonds polytopes and a hierarchy of combinatorial optimization, Discrete Mathematics 4 (1973) 305–337. 22. V. Chv´atal, W. Cook, and M. Hartmann, On cutting-plane proofs in combinatorial opti- mization, Linear Algebra and its Applications 114/115 (1989) 455–499. 23. M. Conforti and L. Wolsey, Compact formulations as a union of polyhedra, CORE discus- sion paper, Louvain-la-Neuve, Belgium (2005). 24. W. Cook and S. Dash, On the matrix-cut rank of polyhedra, Mathematics of Operations Research 26 (2001) 19–30. 40 G´erard Cornu´ejols

25. W. Cook, R. Kannan and A. Schrijver, Chv´atal closures for mixed integer programming problems, Mathematical Programming 47 (1990) 155–174. 26. G. Cornu´ejols and Y. Li, Elementary closures for integer programs, Operations Research Letters 28 (2001) 1–8. 27. G. Cornu´ejols and Y. Li, On the rank of mixed 0,1 polyhedra, Mathematical Programming A91(2002) 391–397. 28. G. Cornu´ejols and Y. Li, A connection between cutting plane theory and the geometry of numbers, Mathematical Programming A 93 (2002) 123–127. 29. S. Dash, O. G¨unl¨uk and A. Lodi, Separating from the MIR closure of polyhedra, presen- tation at the International Symposium on Mathematical Programming, Rio de Janeiro, Brazil, 2006, paper in preparation. 30. F. Eisenbrand, On the membership problem for the elementary closure of a polyhedron, Combinatorica 19 (1999) 297–300. 31. M. Fischetti and A. Lodi, Optimizing over the first Chv´atal closure, in M. J¨unger and V. Kaibel eds., IPCO 2005, LNCS 3509 (2005) 12–22. 32. M.X. Goemans and L. Tun¸cel, When does the positive semidefiniteness constraint help in lifting procedures, Mathematics of Operations Research 26 (2001) 796–815. 33. M.X. Goemans and D.P. Williamson, Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming, Journal of the ACM 42 (1995) 1115–1145. 34. R.E. Gomory, An algorithm for integer solutions to linear programs, Recent Advances in Mathematical Programming, R.L. Graves and P. Wolfe eds., McGraw-Hill, New York (1963) 269–302. 35. R.G. Gomory, Some polyhedra related to combinatorial problems, Linear Algebra and Applications 2 (1969) 451–558. 36. R.E. Gomory and E.L. Johnson, Some continuous functions related to corner polyhedra I, Mathematical Programming 3 (1972) 23–85. 37. O. G¨unl¨uk and Y. Pochet, Mixing mixed-integer inequalities, Mathematical Programming 90 (2001) 429–458. 38. M. K¨oppe, Q. Louveaux and R. Weismantel, Intermediate integer programming represen- tations using value disjunctions, technical report, Department of Mathematics, University of Magdeburg (2005). 39. M. K¨oppe and R. Weismantel, A mixed-integer Farkas lemma and some consequences, Operations Research Letters 32 (2004) 207–211. 40. J.B. Lasserre, An explicit exact SDP relaxation for nonlinear 0-1 program, Lecture Notes in Computer Science 2081 (2001) 293–303. 41. M. Laurent, A comparison of the Sherali-Adams, Lov´asz-Schrijver and Lasserre relaxation for 0-1 programming, SIAM Journal on Optimization 28 (2003) 345–375. 42. A.K. Lenstra, H.W. Lenstra and L. Lov´asz, Factoring polynomials with rational coeffi- cients, Mathematische Annalen 261 (1982) 515–534. 43. L. Lov´asz and A. Schrijver, Cones of matrices and set-functions and 0-1 optimization, SIAM Journal of Optimization 1 (1991) 166–190. 44. H. Marchand and L.A. Wolsey, Aggregation and mixed integer rounding to solve MIPs, Operations Research 49 (2001) 363–371. 45. A. Miller and L.A. Wolsey, Tight formulations for some simple MIPs and convex objective IPs, Mathematical Programming B 98 (2003) 73–88. 46. G.L. Nemhauser and L.A. Wolsey, Integer and Combinatorial Optimization, Wiley, New York (1988). 47. G.L. Nemhauser and L.A. Wolsey, A recursive procedure to generate all cuts for 0-1 mixed integer programs, Mathematical Programming 46 (1990) 379–390. 48. Y.E. Nesterov and A.S. Nemirovski, Interior Point Polynomial Algorithms in Convex Programming, SIAM Publications, SIAM, Philadelphia (1994). 49. J.H. Owen and S. Mehrotra, On the value of binary expansions for general mixed-integer linear programs, Operations Research 50 (2002) 810–819. 50. M. Perregaard, A practical implementation of lift-and-project cuts: A computational ex- ploration of lift-and-project cuts with XPRESS-MP, 18th ISMP, Copenhagen (2003). 51. J.-S. Roy, “Binarize and project” to generate cuts for general mixed-integer programs, technical report, EDF R&D, Clamart, France (2006). 52. A. Schrijver, On cutting planes, Annals of Discrete Mathematics 9 (1980) 291–296. 53. A. Schrijver, Theory of Linear and Integer Programming, Wiley, New York (1986). Valid Inequalities for MILPs 41

54. H. Sherali and W. Adams, A hierarchy of relaxations between the continuous and con- vex hull representations for zero-one programming problems, SIAM Journal on Discrete Mathematics 3 (1990) 311–430. 55. H. Sherali and W. Adams, A reformulation-linearization technique for solving discrete and continuous nonconvex problems, Chapter 4, Kluwer Academic Publishers, Norwell, MA (1999). 56. M. Van Vyve, The continuous mixing polyhedron, Mathematics of Operations Research 30 (2005) 441–452. 57. L.A. Wolsey, Integer Programming (1999). 58. G.M. Ziegler, Lectures on Polytopes, Springer, New York (1995).