Heuristics in Mixed Integer Programming

HEURISTICS IN MIXED INTEGER depending on the main ingredient (building PROGRAMMING block) they use, namely LP-based heuristics in the section titled ‘‘LP-Based Heuristics’’ MATTEO FISCHETTI and MILP-based approaches in the section DEI, UniversitadiPadova,Padova,´ titled ‘‘MILP-Based Heuristics.’’ Italy With a little abuse of terminology, in what ANDREA LODI follows, we will say that a point x is integer ∈ I DEIS, Universita´ di Bologna, if xj is integer for all j (no matter what Bologna, Italy the value of the other components), whereas the rounding x˜ of a given x will be defined as x˜j := [xj]ifj ∈ I and x˜j := xj otherwise ([·] representing scalar rounding to the nearest INTRODUCTION integer). We end the introduction by noting that, We consider a generic mixed integer linear although Achterberg [1] has shown that the program (MILP) in the form impact of heuristics is not dramatic in terms of ability of an MILP solver to prove optimal- min cTx (1) ity in a (much) quicker way, the psychological impact for the user who sees a high quality Ax ≥ b (2) feasible solution provided very early is huge. xj ∈{0, 1}∀j ∈ B,(3)For this reason, the availability of very effective general-purpose heuristics for MILP is x integer ∀ j ∈ G,(4) j among the most crucial improvements over xj continuous, ∀ j ∈ C,(5)the last 10 years. where A is an m × n input matrix, and b and c are input vectors of dimension m and LP-BASED HEURISTICS n, respectively. Here, the variable index set N :={1, ..., n} is partitioned into (B, G, C), In this section we address some basic heuris- where B is the index set of the 0–1 variables tics that only assume the availability of a (if any), while the sets G and C index the ‘‘black-box’’ LP solver. general integer and the continuous vari- Folklore ables, respectively. Bound on the variables, including the 0–1 bounds on the binary The so-called rounding as well as diving ones, are assumed to be part of system heuristics belong to folklore. (2). Removing the integrality requirement Rounding methods solve the LP relaxation on variables indexed by I := B ∪ G leads to to get an optimal point x∗, whose fractional { T ∈ } ∗ ∈ I the LP relaxation min c x : x P where components xj with j are rounded, for P :={x ∈ Rn : Ax ≥ b}. example, to their nearest integer value. Con- MILP heuristics aim at finding a feasible tinuous variables, if any, can optimally be (and hopefully good) solution of the prob- recomputed once the integer variables have lem above, which is an NP-hard problem by been fixed by solving an LP. itself. We next present the main ideas under- Diving (also known as Relax-and-Fix) lying some of the heuristics proposed in the methods, instead, mimic an enumerative literature. In this article, in particular, we scheme by sequentially fixing some integer- focus on those algorithms developed with the valued variables that assume a fractional aim of being tightly integrated within MILP value in the solution of the current LP- solvers. We group and discuss the algorithms relaxation. The sequence can be viewed

Wiley Encyclopedia of Operations Research and Management Science, edited by James J. Cochran Copyright © 2010 John Wiley & Sons, Inc. 1 2 HEURISTICS IN MIXED INTEGER PROGRAMMING as ‘‘diving’’ on a branch-and-bound tree In the mid-1980s Balas and Martin [4] until a feasible (and hopefully good) MILP introduced a generalization of P&C to MILPs, solution is found. A limited backtracking called pivot and shift, where general integer is often allowed to investigate alternative variables are treated as 0–1 variables ‘‘cen- fixing patterns. Various variable-fixing (i.e., tered’’ on the current LP solution, and some branching) policies have been used by dif- new types of ‘‘nonstandard’’ pivots are intro- ferent authors, all meant to hopefully reach duced. An elaboration with the same name of deep branching nodes while maintaining a this idea was later proposed by Balas et al. good LP relaxation lower bound value. [5]. Different extensions called pivot, cut, and dive and pivot and Gomory cut were proposed by Nediak and Eckstein [6] and by Gosh and Pivoting Methods Hayward [7], respectively. These methods originate from the seminal work of Balas and Martin [2] in the late OCTANE and Line Search Methods 1970s. The basic heuristic, called Pivot and Line search methods are based on the fol- Complement (P&C), applies to pure 0–1 inte- lowing construction. One starts from an LP G = C =∅ ger linear programs (ILPs) with .It relaxation optimal vertex x∗ and draws a is based on the observation that a feasible line toward a second point, for example, y, solution is just a basic solution of the LP to be chosen according to a certain criterion. relaxation where all 0–1 variables are non- Thus, moving in a discretized way from x∗ basic (either at their lower or upper bound), to y traces a discrete sequence of (say) K that is, only the slack (continuous) variables k ∗ ∗ points x := x + αk(y − x )for0= α1 < ···< on Equation (2) are allowed to be basic. Given k αK = 1. Each x can therefore be associated an optimal basis of the LP relaxation, P&C (e.g., by rounding) with an integer point x˜k then performs a systematic series of ‘‘non- which is then checked for feasibility and pos- standard’’ pivots in the attempt of driving as sibly used to update a current-best MILP many 0–1 variables as possible out of the solution. Local shifts of the rounded vari- basis, while preserving primal feasibility and ables are typically allowed in the attempt of worsening the objective function as little as improving the feasibility and/or quality of the possible. rounded points. More specifically, the method prescribes Hillier [8] introduced this scheme by defin- the use of three types of pivots: (i) Type ing y so as to go inside P along a so-called 1 pivots that maintain primal feasibility interior path, the rational being supported and increase the number of nonbasic 0–1 by the geometrical intuition that (mainly variables; (ii) Type 2 pivots that also main- for general integer MILPs) going inside P tain primal feasibility but leave the number increases the chances the rounded point be of nonbasic 0–1 variables unchanged, feasible. A similar scheme was later adopted while reducing the infeasibility degree of by Ibaraki et al. [9] and by Faaland and ∗ the current LP solution x ,computedas Hiller [10], where the sampling line segment { ∗ − ∗} ∗ j∈B min xj ,1 xj ; (iii) Type 3 pivots [x , y] is replaced by a sequence of linear that increase the number of nonbasic 0–1 segments [x∗, y1], [y1, y2], ... approximating variables at the price of producing a primal- a curved search trajectory. Very recently, infeasible point—a situation to be repaired Naoum-Sawaya and Elhedhli [11] investi- at a later step of the procedure. gated the effect of replacing the LP solver Nonbasic 0–1 variables can also be com- by an analytic-center method. plemented with the aim of improving the OCTANE (for Octahedral Neighborhood solution quality or reducing its primal infea- Enumeration) is a line search method pro- sibility, and reduced costs are exploited to posed by Balas et al.[12]for0–1ILPs.It fix variables when mathematically correct. also starts with an LP optimal vertex x∗ The P&C heuristic was later improved by and moves it along a certain search direction Løkketangen et al. [3] by adding tabu search y − x∗. However, the integer (possibly infea- to the basic mechanisms. sible) points discovered along this direction HEURISTICS IN MIXED INTEGER PROGRAMMING 3

∗ are not defined through rounding, but exploit FP scheme above may stall in case (x , x˜) the following construction. Consider the cen- is not reduced at some iteration, hence a ter x0 = (1/2, ...,1/2) of the unit hypercube. number of diversification mechanisms are For any vertex x˜ of the unit hypercube (not introduced. necessarily in P), define the hyperplane H(x˜) As stated, FP is meant to only produce passing through x˜ and orthogonal to x˜ − a feasible MILP solution, as the objective x0.1 Starting from x∗ and going toward y function is only taken into account implicitly ∗ one crosses, in sequence, certain hyperplanes in the definition of the very first x .In H(x˜1), H(x˜2), ...whose generating 0–1 points order to produce good quality solutions, the x˜1, x˜2, ... are checked for feasibility and used original FP approach [13] uses a ‘‘sliding to possibly update the current-best feasible window’’ method that introduces the cut T solution. OCTANE prescribes to visit only a c x ≤ UB into the LP model and updates limited number of hyperplanes, and exploits UB, on the fly, by taking an intermediate an effective method for their enumeration. value between the optimal LP-relaxation and the UB values. (Note that the value UB can also be a ‘‘guess’’ of a feasible Feasibility Pump solution value and in such a case the The Feasibility Pump (FP) is a method orig- cut can be possibly invalid.) A different inally proposed by Fischetti et al. [13] for approach, called objective FP,wasdevel- 0–1 MILPs (G =∅),andthenextendedby oped by Achterberg and Berthold [15], Bertacco et al. [14] to the general case. It who modified the objective function of is based on the observation that a feasible the auxiliary LP to (x∗, x˜) + α cTx,where MILP solution is a point x of P that coincides α is a dynamically updated parameter. with its rounding. Replacing ‘‘coincides’’ with Very recently, Fischetti and Salvagnin [16] ‘‘is as close as possible’’ leads to the follow- proposed a more elaborated FP version, ing iterative scheme. FP works with a pair called FP 2.0, where the original rounding ∗ of points (x∗, x˜)withx∗ ∈ P and x˜ integer, operation used to construct x˜ from x is that are iteratively updated with the aim replaced by a more clever rounding heuristic of reducing as much as possible their L1- based on constraint propagation—a basic ∗ ˜ = | − ˜ | tool in constraint programming (see Con- distance (x , x): j∈I xj xj .Inasense, the method is aimed at ‘‘pumping’’ the inte- straint Programming Links with Math grality of x˜ into x∗, and the LP-feasibility of x∗ Programming). into x˜ —hence its name.2 To be more specific, ∗ ∈ one starts with any x P, for example, an MILP-BASED HEURISTICS optimal LP vertex, and initializes an integer ∗ x˜ as the rounding of x .AteachFP itera- We next address a new generation of MILP tion, called a pumping cycle, x˜ is fixed and heuristics that emerged in the late 1990s. ∗ one finds a point x ∈ P whichisasclose Their hallmark is the use of a ‘‘black-box’’ as possible to x˜ by solving the auxiliary LP external MILP solver to explore a solution ∗ ∗ min{(x, x˜):x ∈ P}.If(x , x˜) = 0, then x is neighborhood defined by invalid linear con- integer and the method is completed. Oth- straints. The use of an exact MILP solver ∗ erwise, x˜ is replaced by the rounding of x inside an MILP heuristic may appear naive ∗ so as to hopefully reduce (x , x˜)evenfur- at first glance, but it turns out to be effective ther, and the process is iterated. The basic in the cases where the added invalid con- straints lead to a structural simplification of the MILP at hand and allow, for example, for a more powerful instance preprocessing 1The half-spaces induced by these hyperplanes that contain x0 define an octahedron, hence the and/or for extensive node pruning. name of the method. Local Branching 2The name was actually inspired by the Electron Pump of the Asimov’s science fiction novel ‘‘The The local branching (LB) scheme of Fischetti Gods Themselves’’. and Lodi [17] appears to be the first method 4 HEURISTICS IN MIXED INTEGER PROGRAMMING embedding an MILP solver within a general account. At specified nodes of the branch- MILP heuristic framework. Suppose a fea- and-bound tree, the current LP relaxation sible reference solution x of an MILP with solution x∗ and the incumbent x are com- B =∅ is given, and one aims at finding an pared and all integer-constrained variables improved solution that is ‘‘not too far’’ from that agree in value are fixed and projected x. To this end, one can define the k-OPT out of the MILP. In this way, such an MILP neighborhood N(x, k)ofx as the set of the is not only simplified but generally speaking MILP solutions satisfying the invalid local (provided the number of components fixed is branching constraint sufficiently large) it is also reduced in size. This is probably the main difference between (x, x):= xj + (1 − xj) ≤ k,(6) LB and RINS, namely the fact that the former ∈B = ∈B = j :xj 0 j :xj 1 performs a soft fixing by means of the cardi- nality constraint (6), while in the latter the for a small parameter k (typically, k = 10 fixing is hard with clear benefits associated or k = 20), and explore it by means of an with dealing with smaller MILPs. external MILP solver, often heuristically, The RINS framework is also particularly that is, within a prefixed number of branch- suitable for an integration within a classical and-bound nodes. The method is in the branch-and-bound (and then within classi- spirit of local search metaheuristics and cal MILP solvers) because it can be virtu- in particular of large neighborhood search ally invoked at any node of the tree where [18], with the novelty that neighborhoods solutions of the LP relaxations are always are obtained through a so-called soft fixing, different. Clearly, invoking the algorithm too that is, through invalid cuts to be added often would result in slowing down the over- to the original MILP model. Diversification all running time for proving optimality, thus cuts can be defined in a similar way, thus RINS is only executed every L (say) nodes in leading to a flexible toolkit for the definition the tree. On the other hand, the size of the of metaheuristics for general MILPs. For resulting reduced MILP cannot be predicted example, Hansen et al. [19] embedded in advance and the MILP is solved only if the LB within variable neighborhood search, reduction is relevant enough. whereas Fischetti et al. [20] specialized LB to The distance induced neighborhood search MILPs with a two-level variable structure. (DINS) approach by Ghosh [23] is a RINS Integration between FP and LB was instead variant which takes an intermediate step proposed by Fischetti and Lodi [21]. between LB and RINS by performing both As its name suggests, LB can also be soft and hard fixing within the same algo- used within an exact enumerative scheme by rithm. The idea behind DINS is that the most branching on the ‘‘local’’ disjunction (x, x) ≤ promising solutions are those ‘‘close’’ to the k (to be explored first) or (x, x) ≥ k + 1with solution of the continuous relaxation, thus respect to the incumbent MILP solution x (no the neighborhood is defined by the distance matter the LP solution at a given node). This inequality branching strategy is intended to favor the | − ∗|≤ | − ∗| detection of good solutions very early in the xj xj xj xj ,(7) enumeration. j∈B∪G j∈B∪G

where again x∗ and x denote the current LP Relaxation Induced Neighborhood Search and solution of the LP relaxation and the incum- Variants bent, respectively. For a variable xj such that | − ∗| The relaxation induced neighborhood search xj xj < 0.5 the only way for the new solu- (RINS) framework of Danna et al. [22] also tion to decrease its distance with respect uses the (heuristic) solution of a simplified to xj is to hard fix it, that is, xj = xj.For MILP through an external MILP solver as the other variables, instead, a rebounding a main ingredient, but extends the idea by phase is performed before the neighborhood taking the solution of the LP relaxations into is explored (as usual by means of a call to an HEURISTICS IN MIXED INTEGER PROGRAMMING 5 external MILP solver), in which the bounds solutions are either obtained within the on the variables are set so as the absolute tree (by other heuristics, including div- value of the distance between the new solu- ing) or computed by the polishing algo- tion and x cannot increase. A bit more of rithm itself. flexibility can be granted by allowing some • Combination. Twoormoresolutions of the variables to increase their distance (the parents) are combined with the but reducing the overall distance (as guaran- aim of creating a new member of the teed by Equation 7). This is again obtained population (the child) with ‘‘improved’’ by a sophisticated mixture of hard and soft characteristics. The RINS scheme is fixing. adopted, that is, all variables whose The relaxation enforced neighborhood value coincides in the parents are fixed search (RENS) scheme by Berthold [24] is and the reduced MILP is heuristically another RINS approach whose main charac- solved by an external MILP solver teristic is to define a reduced MILP (to be within a limited number of branch- explored by a call to an external solver) as and-bound nodes. This scheme, clearly, the set of all integer solutions which can be is much more time-consuming than a obtained by rounding a solution x∗ ∈ P.More ∗ classical combination step in evolution- precisely, for a given x the reduced MILP is ary algorithms, but guarantees that constructed by the child solution is feasible (for the original problem). 1. hard fixing those integer-constrained variables whose current value is • Mutation. A sufficient degree of diversi- = ∗ ∀ ∈ fication is guaranteed by performing a already integer, that is, xj xj , j F where F :={j : x∗ = x∗ , j ∈ B ∪ G}; classical mutation action which consists j j in (i) selecting at random a seed solution 2. rebounding the remaining variables by in the population, (ii) fixing at random using the two rounded values as lower = ∗ some of its variables, and (iii) heuristi- j and upper uj bounds, that is, j xj ∗ cally solve the resulting reduced MILP. and uj = x for all j ∈ (B ∪ G) \ F. j • Selection. The parents’ selection is per- Interestingly, RENS does not require an formed in a simple way by randomly incumbent solution to work with and it can picking a solution in the population and be used offline as a tool for the analysis of the then choosing, again at random, the sec- effectiveness of MILP rounding heuristics. ond one but only among those solutions Note that pivot-and-shift [5] uses a similar with a better objective value. Although mechanism to restrict the range of the integer it is easy to extend this criterion to select variables depending on their LP value and more than two parents, in Rothberg [25] such a rebounding is similar to that of DINS either two or all solutions are used as as well. parents, that is, in the latter case no selection is performed. An Evolutionary Algorithm within MILP The farthest (to date) step in the direction The polishing algorithm is then a fully of using metaheuristic ideas within an MILP general metaheuristic implemented within has been taken by Rothberg [25]. The result- an MILP framework and, in turn, making use ing approach, often called polishing algo- of an external MILP solver as main ‘‘engine.’’ rithm, implements an evolutionary heuristic which is invoked at selected nodes of a REFERENCES branch-and-bound tree and includes all classical ingredients of genetic computation. 1. Achterberg T. Constraint integer program- More precisely, ming. PhD thesis, Berlin: ZIB; 2007. 2. Balas E, Martin CH. Pivot-and-complement: • Population. A fixed-size population of a Heuristic for 0-1 programming. Manage Sci feasible solutions is maintained. Those 1980;26:86–96. 6 HEURISTICS IN MIXED INTEGER PROGRAMMING

3. Løkketangen A, Jornsten K, Storoy S. Tabu 16. Fischetti M, Salvagnin D. Feasibility pump search within a Pivot and complement frame- 2.0. Math Program Comput 2009;1:201–222. work. Int Trans Oper Res 1994;1:305–317. 17. Fischetti M, Lodi A. Local branching. Math 4. Balas E, Martin CH. Pivot and shift: a heuris- Program 2003;98:23–47. tic for mixed integer programming. Technical 18. Shaw P. Using constraint programming and report, GSIA, Carnegie Mellon University; local search methods to solve vehicle routing 1986. problems. In: Maher M, Puget JF, editors. 5. Balas E, Schmieta S, Wallace C. Pivot and Principles and practice of constraint program- shift - a mixed integer programming Heuristic. ming CP98. Volume 1520, Lecture notes in Disc Optim 2004;1:3–12. computer science. Heidelberg: Springer; 1998. 6. Nediak M, Eckstein J. Pivot, cut, and dive: a pp. 417–431. Heuristic for 0-1 mixed integer programming. 19. Hansen P, MladenovicN,Urosevi´ cD.Vari-´ J Heuristics 2007;13:471–503. able neighborhood search and local branching. 7. Ghosh S, Hayward RB. Pivot and Gomory cut: Comput Oper Res 2006;33:3034–3045. a MILP feasibility Heuristic. Technical report, 20. Fischetti M, Polo C, Scantamburlo M. A local University of Alberta; 2009. branching Heuristic for mixed-integer pro- 8. Hillier FS. Efﬁcient Heuristic procedures for grams with 2-level variables. Networks integer linear programming with an interior. 2004;44:61–72. Oper Res 1969;17:600–637. 21. Fischetti M, Lodi A. Repairing MILP infeasi- 9. Ibaraki T, Ohashi T, Mine H. A Heuristic algo- bility through local branching. Comput Oper rithm for mixed-integer programming prob- Res 2008;35:1436–1445. lems. Math Program Study 1974;2:115–136. 22. Danna E, Rothberg E, Le Pape C. Explor- 10. Faaland BH, Hillier FS. Interior path methods ing relaxation induced neighborhoods to for heuristic integer programming procedures. improve MILP solutions. Math Program Oper Res 1979;27:1069–1087. 2005;102:71–90. 11. Naoum-Sawaya J, Elhedhli S. An interior 23. Ghosh S. DINS, a MILP improvement Heuris- point cutting plane Heuristic for mixed integer tic. In: Fischetti M, Williamson DP, editors. programming. Technical report, University of Integer programming and combinatorial opti- Waterloo; 2010. mization, IPCO 2007. Volume 4513, Lecture 12. Balas E, Ceria S, Dawande M, et al. OCTANE: notes in computer science. Springer; 2007. pp. 310–323. a new Heuristic for pure 0-1 programs. Oper Res 2001;49:207–225. 24. Berthold T. RENS - relaxation enforced neighborhood search. Technical Report 07-28. 13. Fischetti M, Glover F, Lodi A. The feasibility Berlin: ZIB; 2007. pump. Math Program 2005;104:91–104. 14. Bertacco L, Fischetti M, Lodi A. A feasibil- 25. Rothberg E. An evolutionary algorithm for pol- ity pump Heuristic for general mixed-integer ishing mixed integer programming solutions. problems. Disc Optim 2007;4:63–76. Informs J Comput 2007;19:534–541. 15. Achterberg T, Berthold T. Improving the feasibility pump. Disc Optim 2007;4:77–86.