Decomposition and Dynamic Cut Generation in Integer Programming Ted K. Ralphs Matthew V. Galati Department of Industrial and Systems Engineering Lehigh University, Bethlehem, PA http://www.lehigh.edu/˜tkr2 Columbia University, Thursday June 26, 2003 Decomposition Methods for IP 1 Outline ² Traditional decomposition methods – Lagrangian Relaxation – Dantzig-Wolfe Decomposition – Cutting Plane Algorithms ² A common framework ² Decomposition methods and dynamic cut generation ² Conclusions 1 Decomposition Methods for IP 2 Preliminaries ² We consider the following pure integer linear program: > > zIP = minfc xg = minfc xg x2F x2P where F = fx 2 Zn : A0x ¸ b0;A00x ¸ b00g Q = fx 2 Rn : A0x ¸ b0;A00x ¸ b00g F 0 = fx 2 Zn : A0x ¸ b0g Q0 = fx 2 Rn : A0x ¸ b0g Q00 = fx 2 Rn : A00x ¸ b00g ² We are interested in P = conv(F) and P0 = conv(F 0). ² Assumptions – All input data are rational. – P is bounded. – Optimization/separation over P is “difficult.” – Optimization/separation over P0 is “easy.” 2 Decomposition Methods for IP 3 Bounding ² Goal: Compute a lower bound on zIP . ² The most straightforward approach is to solve the initial LP relaxation. minfc>xg x2Q ² Decomposition approaches attempt to improve on this bound by utlizing our implicit knowledge of P0. ² Decomposition algorithms – Dantzig-Wolfe decomposition – Lagrangian relaxation – Cutting plane method 3 Decomposition Methods for IP 4 Dantzig-Wolfe Decomposition ² Express the constraints of Q00 explicitly. ² Express the constraints of P0 implicitly through solution of a subproblem. ² The bound is obtained by solving the Dantzig-Wolfe LP: X X X 00 00 0 zDW = minfc( s¸s): A ( s¸s) ¸ b ; ¸s = 1; ¸s ¸ 0 8s 2 F g s2F0 s2F0 s2F0 (1) ² Solution method: Simplex algorithm with dynamic column generation. ² Subproblem: Optimization over P0. ² Suppose ¸ˆ is an optimal solution to (1). > ² Then zIP ¸ zDW = c xˆ ¸ zLP , where X ˆ 0 xˆ = s¸s 2 P (2) s2F0 4 Decomposition Methods for IP 5 Lagrangian Relaxation ² Express the constraints of Q00 explicitly. ² Express the constraints of P0 implicitly through solution of a subproblem. ² The bound is obtained by solving the Lagrangian dual. > > 00 > 00 zLR(u) = minf(c ¡ u A )s + u b g (3) s2F0 zLD = max fzLR(u)g (4) m00 u2R+ ² Solution method: Subgradient optimization. ² Subproblem: Optimization over P0. 5 Decomposition Methods for IP 6 Lagrangian Relaxation (cont.) ² Rewriting (4) as a linear program, we see it is dual to the Dantzig-Wolfe LP. 00 00 0 zLD = max f´ : ´ · (c ¡ uA )s + ub 8s 2 F g (5) m00 ´2R;u2R+ ² So we have zIP ¸ zLD = zDW ¸ zLP . m00 ² The vector uˆ 2 R+ such that zLD = zLR(ˆu) will be called a set of optimal (dual) multipliers. ² Note that these correspond exactly to the dual variables of the Dantzig- Wolfe LP. ² The member of F 0 that yields the optimal bound will be called the optimal primal solution to the Lagrangian dual. 6 Decomposition Methods for IP 7 Cutting Plane Method ² Express the constraints of Q00 explicitly. ² Express the constraints of P0 implicitly through solution of a subproblem. ² The bound is obtained by solving the initial LP relaxation augmented by facets of P0. ² Solution method: Simplex with dynamic cut generation. ² Subproblem: Separation from P0. ² We assumed that separation over P0 was also “easy.” ² This approach yields the bound 00 00 zCP = minfcx : A x ¸ b g (6) x2P0 . ² Note that xˆ from (2) is an optimal solution to (6), so we obtain the same bound again. 7 Decomposition Methods for IP 8 A Common Framework ² The three methods discussed are just three algorithms for computing the same quantity. ² The basic ingredients are the same: – the original polyhedron (P), – an implicit polyhedron (P0), and – an explicit polyhedron (Q00). ² The essential difference between the cutting plane method and the other two is how the implicit polyhedron is represented: – as the interesection of half-spaces (the outer representation), or – as the convex hull of a finite set (the inner representation). 0 ² zDW and zLR are computed using inner representations of P , whereas zCP uses an outer representation. 8 Decomposition Methods for IP 9 Polyhedra, LP Bound, LD/DW/CP Bound 0 00 Q = Q \Q (LP Bound) P0 \Q00 (LD/DW/CP Bound) P = conv(fx 2 Zn : Ax ¸ bg) P0 = conv(fx 2 Zn : A0x ¸ b0g) Q0 = fx 2 Rn : A0x ¸ b0g Q00 = fx 2 Rn : A00x ¸ b00g 9 Decomposition Methods for IP 10 Improving the Bound 00 00 ² So far, we have the bound minx2P0fcx : A x ¸ b g. ² Can we improve on this? ² With the cutting plane method, the bound can be improved using heuristic cut generation. ² We can think of this as a dynamic tightening of the explicit polyhedron. ² Generic cutting plane method 1. Construct the initial LP relaxation. 2. Solve the current LP relaxation to obtain the solution xˆ. 3. Try to separate xˆ from P, generating a set L of valid inequalities violated by xˆ. 4. If L 6= ;, add the set L of valid inequalities to the LP relaxation and go to Step 2. 5. If L = ;, then output cxˆ. ² Step 3 may generate facets of any number of polyhedra containing P, including P itself. ² In principle, there are analogs of this for Dantzig-Wolfe decomposition and Lagrangian relaxation. 10 Decomposition Methods for IP 11 Dynamic Decomposition Methods ² Generic dynamic bounding method 1. Formulate an initial bounding problem using one of the three decomposition approaches by determining appropriate explicit and implicit polyhedra. 2. Solve the current bounding problem to obtain a valid lower bound. 3. Try to generate an inequality valid for P that increases the bound when added to the description of the explicit polyhedron. 4. If successful in Step 3, add the generated valid inequality to the description of the explicit polyhedron and formulate a new dual problem. 5. If unsuccessful in Step 3, then output the current lower bound. ² The difficulty is in performing Step 3. 11 Decomposition Methods for IP 12 Improving Inequalities ² We will call an inequality that improves the bound in Step 3, an improving inequality. ² An inequality is improving if and only if it is violated by all optimal primal solutions to (6). ² With the cutting plane method, we generate inequalities violated by one of the optimal primal solutions to (6) and hope for the best. ² We can do the same for Dantzig-Wolfe decomposition by computing xˆ from (2), resulting in an implementation of branch, cut, and price. ² When employing the Lagrangian dual with subgradient optimization, we don’t have access to primal solution information from (6). ² We can, however, get limited information by examining the optimal primal solution to the Lagrangian dual problem. 12 Decomposition Methods for IP 13 Dynamic Cut Generation in Lagrangian Relaxation ² An optimal primal solution to the Lagrangian dual is a member of F 0. ² If it is a member of F, then it is optimal. ² Otherwise, we attempt to separate it from P0 and add the newly generated inequality to the matrix [A00; b00]. ² When employed in a branch and bound framework, this technique is known as relax and cut. ² One advantage is that it is sometimes much easier to separate a member of F 0 from P than an arbitrary real vector. ² What are the chances of generating an improving inequality? ² What is the relationship of an optimal primal solution to the Lagrangian dual to an optimal solution of (6)? 13 Decomposition Methods for IP 14 Relating the Methods ² The set of alternative optimal primal solutions to the Lagrangian dual is F 0 \ S, where S is the face of P0 defined as S = fx 2 P0 :(c> ¡ uˆ>A00)x = (c> ¡ uˆ>A00)ˆsg (7) and sˆ is any optimal primal solution to the Lagrangian dual. ² The face S contains xˆ, as defined in (2) and is a proper face if and only if xˆ is not an inner point of P0. ² If xˆ is an inner point of P0, then the convexity constraint in the Dantzig- Wolfe LP has a zero dual value and all members of F 0 are optimal for the Lagrangian dual. ² If ¸ˆ is an optimal solution to the Dantzig-Wolfe LP, any s 2 F 0 such ˆ that ¸s > 0 is an optimal primal solution for the Lagrangian dual. 14 Decomposition Methods for IP 15 Some Consequences ² Any improving inequality must be violated by some optimal solution to the Lagrangian dual. ² In fact, any improving inequality must be violated by some optimal ˆ solution s to the Lagrangian dual such that ¸s > 0. ² Hence, if we have access to the optimal decomposition, we will be much more likely to be able to generate an improving inequality. ² This leads us back to the idea of using Dantzig-Wolfe decomposition with dynamic cut generation. ² What if instead of separating the fractional solution xˆ, we separate each 0 ˆ s 2 F such that ¸s > 0? – Any improving inequality must be violated by some s 2 F 0 such that ˆ ¸s > 0. – It might be easier to separate members of F 0 than xˆ. 15 Decomposition Methods for IP 16 Separating the Members of F 0 ² Consider a class of valid inequalities for P with which it is – difficult to separate an arbitrary fractional solution, but – easy to separate members of F 0. ² Does this occur in practice? Yes. ² How can we take advantage of this situation? 16 Decomposition Methods for IP 17 The Vehicle Routing Problem The VRP is a combinatorial problem whose ground set is the edges of a graph G(V; E). Notation: ² V is the set of customers and the depot (0). ² d is a vector of the customer demands.