Duality for Mixed Integer Linear Programming TED RALPHS MENAL GUZELSOY¨ ISE Department COR@L Lab Lehigh University [email protected] University of Newcastle, Newcastle, Australia 13 May 2009 Thanks: Work supported in part by the National Science Foundation lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 1/56 Outline 1 Duality Theory Introduction IP Duality 2 The Subadditive Dual Formulation Properties 3 Constructing Dual Functions The Value Function Gomory’s Procedure Branch-and-Bound Method Branch-and-Cut lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 2/56 What is Duality? It is difficult to give a general definition of mathematical duality, though mathematics is replete with various notions of it. Set Theory and Logic (De Morgan Laws) Geomety (Pascal’s Theorem & Brianchon’s Theorem) Combinatorics (Graph Coloring) The duality we are interested in is a sort of functional duality. We define a generic optimization problem to be a mapping f : X → R, where X is the set of possible inputs and f (x) is the result. Duality may then be defined as a method of transforming a given primal problem to an associated dual problem such that the dual problem yields a bound on the primal problem, and applying a related transformation to the dual produces the primal again. In many case, we would also like to require that the dual bound be “close” to the primal result for a specific input of interest. lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 3/56 Duality in Mathematical Programming In mathematical programming, the input is the problem data (e.g., the constraint matrix, right-hand side, and cost vector for a linear program). We view the primal and the dual as parametric problems, but some data is held constant. Uses of the Dual in Mathematical Programing If the dual is easier to evaluate, we can use it to obtain a bound on the primal optimal value. We can also use the dual to perform sensitivity analysis on the parameterized primal input data. Finally, we can also use the dual to warm start solution procedure based on evaluation of the dual. lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 4/56 Duality in Integer Programming We will initially be interested in the mixed integer linear program (MILP) instance zIP = min cx, (P) x∈S n r n−r m×n m where, c ∈ R , S = {x ∈ Z+ × R+ | Ax = b} with A ∈ Q , b ∈ R . We call this instance the base primal instance. To construct a dual, we need a parameterized version of this instance. For reasons that will become clear, the most relevant parameterization is of the right-hand side. The value function (or primal function) of the base primal instance (P) is z(d) = min cx, x∈S(d) m r n−r where for a given d ∈ R , S(d) = {x ∈ Z+ × R+ | Ax = d}. lehigh-logo We let z(d) = ∞ if d ∈ Ω = {d ∈ Rm | S(d) = ∅}. Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 5/56 Example: Value Function 1 zIP = min 2 x1 + 2x3 + x4 3 s.t x1 − 2 x2 + x3 − x4 = b and x1, x2 ∈ Z+, x3, x4 ∈ R+. z(d) 3 5 2 2 3 2 1 1 2 d -4 − 7 -3 − 5 -2 − 3 -1 − 1 0 1 1 3 2 5 3 7 4 2 2 2 2 2 2 2 2 lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 6/56 Dual Functions A dual function F : Rm → R is one that satisfies F(d) ≤ z(d) for all d ∈ Rm. How to select such a function? We choose may choose one that is easy to construct/evaluate and/or for which F(b) ≈ z(b). This results in the base dual instance m m zD = max {F(b) : F(d) ≤ z(d), d ∈ R , F ∈ Υ } where Υm ⊆ {f | f : Rm→R} We call F∗ strong for this instance if F∗ is a feasible dual function and F∗(b) = z(b). This dual instance always has a solution F∗ that is strong if the value function is bounded and Υm ≡ {f | f : Rm→R}. Why? lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 7/56 The LP Relaxation Dual Function It is easy to obtain a feasible dual function for any MILP. Consider the value function of the LP relaxation of the primal problem: FLP(d) = max {vd : vA ≤ c}. v∈Rm m By linear programming duality theory, we have FLP(d) ≤ z(d) for all d ∈ R . Of course, FLP is not necessarily strong. lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 8/56 Example: LP Dual Function FLP(d) = min vd, 1 s.t 0 ≥ v ≥ − 2 , and v ∈ R, which can be written explicitly as 0, d ≤ 0 F (d) = . LP − 1 d, d > 0 2 z(d) FLP(d) 3 5 2 2 3 2 1 1 2 lehigh-logo d -4 − 7 -3 − 5 -2 − 3 -1 − 1 0 1 1 3 2 5 3 7 4 2 2 2 2 2 2 2 2 Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 9/56 The Subadditive Dual By considering that F(d) ≤ z(d), d ∈ Rm ⇐⇒ F(d) ≤ cx , x ∈ S(d), d ∈ Rm n ⇐⇒ F(Ax) ≤ cx , x ∈ Z+, the generalized dual problem can be rewritten as r n−r m zD = max {F(b) : F(Ax) ≤ cx, x ∈ Z+ × R+ , F ∈ Υ }. Can we further restrict Υm and still guarantee a strong dual solution? The class of linear functions? NO! The class of convex functions? NO! The class of sudadditive functions? YES! lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 10/56 The Subadditive Dual Let a function F be defined over a domain V. Then F is subadditive if F(v1) + F(v2) ≥ F(v1 + v2)∀v1, v2, v1 + v2 ∈ V. Note that the value function z is subadditive over Ω. Why? If Υm ≡ Γm ≡ {F is subadditive | F : Rm→R, F(0) = 0}, we can rewrite the dual problem above as the subadditive dual zD = max F(b) j F(a ) ≤ cj j = 1, ..., r, j F¯(a ) ≤ cj j = r + 1, ..., n, and F ∈ Γm, where the function F¯ is defined by F(δd) F¯(d) = lim sup ∀d ∈ Rm. δ→0+ δ Here, F¯ is the upper d-directional derivative of F at zero. lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 11/56 Example: Upper D-directional Derivative The upper d-directional derivative can be interpreted as the slope of the value function in direction d at 0. For the example, we have z(d) ¯z(d) 3 5 2 2 3 2 1 1 2 d -4 − 7 -3 − 5 -2 − 3 -1 − 1 0 1 1 3 2 5 3 7 4 2 2 2 2 2 2 2 2 lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 12/56 Weak Duality Weak Duality Theorem Let x be a feasible solution to the primal problem and let F be a feasible solution to the subadditive dual. Then, F(b) ≤ cx. Proof. Corollary For the primal problem and its subadditive dual: 1 If the primal problem (resp., the dual) is unbounded then the dual problem (resp., the primal) is infeasible. 2 If the primal problem (resp., the dual) is infeasible, then the dual problem (resp., the primal) is infeasible or unbounded. lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 13/56 Strong Duality Strong Duality Theorem If the primal problem (resp., the dual) has a finite optimum, then so does the subadditive dual problem (resp., the primal) and they are equal. Outline of the Proof. Show that the value function z or an extension to z is a feasible dual function. Note that z satisfies the dual constraints. Ω ≡ Rm: z ∈ Γm. m m m Ω ⊂ R : ∃ ze ∈ Γ with ze(d) = z(d) ∀ d ∈ Ω and ze(d) < ∞ ∀d ∈ R . lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 14/56 Example: Subadditive Dual For our IP instance, the subadditive dual problem is max F(b) 1 F(1) ≤ 2 F(− 3 ) ≤ 0 2 . F¯(1) ≤ 2 F¯(−1) ≤ 1 F ∈ Γ1. and we have the following feasible dual functions: 1 d F1(d) = 2 is an optimal dual function for b ∈ {0, 1, 2, ...}. 2 3 F2(d) = 0 is an optimal function for b ∈ {..., −3, − 2 , 0}. d d d d 3 1 ⌈⌈ ⌉− ⌉ 3 ⌈⌈ ⌉− ⌉ F3(d) = max{ 2 ⌈d − 4 ⌉, 2d − 2 ⌈d − 4 ⌉} is an optimal function for 1 5 9 b ∈ {[0, 4 ] ∪ [1, 4 ] ∪ [2, 4 ] ∪ ...}. 2⌈⌈ 2d ⌉− 2d ⌉ 2⌈⌈ 2d ⌉− 2d ⌉ 4 3 2d 3 3 3 2d 3 3 d F4(d) = max{ 2 ⌈ 3 − 3 ⌉ − d, − 4 ⌈ 3 − 3 ⌉ + 2 } is an optimal 7 3 1 lehigh-logo function for b ∈ {... ∪ [− 2 , −3] ∪ [−2, − 2 ] ∪ [− 2 , 0]} Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 15/56 Example: Feasible Dual Functions z(d) F(d) 3 5 2 2 3 2 1 1 2 d -4 − 7 -3 − 5 -2 − 3 -1 − 1 0 1 1 3 2 5 3 7 4 2 2 2 2 2 2 2 2 Notice how different dual solutions are optimal for some right-hand sides and not for others. Only the value function is optimal for all right-hand sides. lehigh-logo Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 16/56 Farkas’ Lemma (Pure Integer) For the primal problem, exactly one of the following holds: 1 S= 6 ∅ 2 There is an F ∈ Γm with F(aj) ≥ 0, j = 1, ..., n, and F(b) < 0.