Duality for Mixed Integer Linear Programming

Home , Duality (optimization)

TED RALPHS MENAL GUZELSOY¨ ISE Department COR@L Lab Lehigh University [email protected]

University of Newcastle, Newcastle, Australia 13 May 2009

Thanks: Work supported in part by the National Science Foundation

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 1/56 Outline

1 Duality Theory Introduction IP Duality

2 The Subadditive Dual Formulation Properties

3 Constructing Dual Functions The Value Function Gomory’s Procedure Branch-and-Bound Method Branch-and-Cut

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 2/56 What is Duality?

It is difficult to give a general definition of mathematical duality, though mathematics is replete with various notions of it. Set Theory and Logic (De Morgan Laws) Geomety (Pascal’s Theorem & Brianchon’s Theorem) Combinatorics (Graph Coloring) The duality we are interested in is a sort of functional duality. We define a generic optimization problem to be a mapping f : X → R, where X is the set of possible inputs and f (x) is the result. Duality may then be defined as a method of transforming a given primal problem to an associated dual problem such that the dual problem yields a bound on the primal problem, and applying a related transformation to the dual produces the primal again. In many case, we would also like to require that the dual bound be “close” to the primal result for a specific input of interest.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 3/56 Duality in Mathematical Programming

In mathematical programming, the input is the problem data (e.g., the constraint matrix, right-hand side, and cost vector for a linear program). We view the primal and the dual as parametric problems, but some data is held constant.

Uses of the Dual in Mathematical Programing If the dual is easier to evaluate, we can use it to obtain a bound on the primal optimal value. We can also use the dual to perform sensitivity analysis on the parameterized primal input data. Finally, we can also use the dual to warm start solution procedure based on evaluation of the dual.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 4/56 Duality in Integer Programming

We will initially be interested in the mixed integer linear program (MILP) instance

zIP = min cx, (P) x∈S

n r n−r m×n m where, c ∈ R , S = {x ∈ Z+ × R+ | Ax = b} with A ∈ Q , b ∈ R . We call this instance the base primal instance. To construct a dual, we need a parameterized version of this instance. For reasons that will become clear, the most relevant parameterization is of the right-hand side. The value function (or primal function) of the base primal instance (P) is

z(d) = min cx, x∈S(d)

m r n−r where for a given d ∈ R , S(d) = {x ∈ Z+ × R+ | Ax = d}. lehigh-logo We let z(d) = ∞ if d ∈ Ω = {d ∈ Rm | S(d) = ∅}.

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 5/56 Example: Value Function

1 zIP = min 2 x1 + 2x3 + x4 3 s.t x1 − 2 x2 + x3 − x4 = b and x1, x2 ∈ Z+, x3, x4 ∈ R+.

z(d)

5 2

3 2

1 2

d -4 − 7 -3 − 5 -2 − 3 -1 − 1 0 1 1 3 2 5 3 7 4 2 2 2 2 2 2 2 2

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 6/56 Dual Functions

A dual function F : Rm → R is one that satisﬁes F(d) ≤ z(d) for all d ∈ Rm. How to select such a function? We choose may choose one that is easy to construct/evaluate and/or for which F(b) ≈ z(b). This results in the base dual instance

m m zD = max {F(b) : F(d) ≤ z(d), d ∈ R , F ∈ Υ }

where Υm ⊆ {f | f : Rm→R} We call F∗ strong for this instance if F∗ is a feasible dual function and F∗(b) = z(b). This dual instance always has a solution F∗ that is strong if the value function is bounded and Υm ≡ {f | f : Rm→R}. Why?

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 7/56 The LP Relaxation Dual Function

It is easy to obtain a feasible dual function for any MILP. Consider the value function of the LP relaxation of the primal problem:

FLP(d) = max {vd : vA ≤ c}. v∈Rm

m By linear programming duality theory, we have FLP(d) ≤ z(d) for all d ∈ R .

Of course, FLP is not necessarily strong.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 8/56 Example: LP Dual Function

FLP(d) = min vd, 1 s.t 0 ≥ v ≥ − 2 , and v ∈ R, which can be written explicitly as 0, d ≤ 0 F (d) = . LP − 1 d, d > 0 2

z(d) FLP(d) 3

5 2

3 2

1 2 lehigh-logo d -4 − 7 -3 − 5 -2 − 3 -1 − 1 0 1 1 3 2 5 3 7 4 2 2 2 2 2 2 2 2

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 9/56 The Subadditive Dual

By considering that

F(d) ≤ z(d), d ∈ Rm ⇐⇒ F(d) ≤ cx , x ∈ S(d), d ∈ Rm n ⇐⇒ F(Ax) ≤ cx , x ∈ Z+, the generalized dual problem can be rewritten as

r n−r m zD = max {F(b) : F(Ax) ≤ cx, x ∈ Z+ × R+ , F ∈ Υ }.

Can we further restrict Υm and still guarantee a strong dual solution? The class of linear functions? NO! The class of convex functions? NO! The class of sudadditive functions? YES!

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 10/56 The Subadditive Dual

Let a function F be deﬁned over a domain V. Then F is subadditive if F(v1) + F(v2) ≥ F(v1 + v2)∀v1, v2, v1 + v2 ∈ V. Note that the value function z is subadditive over Ω. Why? If Υm ≡ Γm ≡ {F is subadditive | F : Rm→R, F(0) = 0}, we can rewrite the dual problem above as the subadditive dual

zD = max F(b) j F(a ) ≤ cj j = 1, ..., r, j F¯(a ) ≤ cj j = r + 1, ..., n, and F ∈ Γm,

where the function F¯ is deﬁned by

F(δd) F¯(d) = lim sup ∀d ∈ Rm. δ→0+ δ

Here, F¯ is the upper d-directional derivative of F at zero. lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 11/56 Example: Upper D-directional Derivative

The upper d-directional derivative can be interpreted as the slope of the value function in direction d at 0. For the example, we have

z(d) ¯z(d) 3

5 2

3 2

1 2

d -4 − 7 -3 − 5 -2 − 3 -1 − 1 0 1 1 3 2 5 3 7 4 2 2 2 2 2 2 2 2

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 12/56 Weak Duality

Weak Duality Theorem Let x be a feasible solution to the primal problem and let F be a feasible solution to the subadditive dual. Then, F(b) ≤ cx.

Proof. Corollary For the primal problem and its subadditive dual: 1 If the primal problem (resp., the dual) is unbounded then the dual problem (resp., the primal) is infeasible. 2 If the primal problem (resp., the dual) is infeasible, then the dual problem (resp., the primal) is infeasible or unbounded.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 13/56 Strong Duality

Strong Duality Theorem If the primal problem (resp., the dual) has a ﬁnite optimum, then so does the subadditive dual problem (resp., the primal) and they are equal.

Outline of the Proof. Show that the value function z or an extension to z is a feasible dual function. Note that z satisﬁes the dual constraints. Ω ≡ Rm: z ∈ Γm. m m m Ω ⊂ R : ∃ ze ∈ Γ with ze(d) = z(d) ∀ d ∈ Ω and ze(d) < ∞ ∀d ∈ R .

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 14/56 Example: Subadditive Dual

For our IP instance, the subadditive dual problem is

max F(b) 1 F(1) ≤ 2 F(− 3 ) ≤ 0 2 . F¯(1) ≤ 2 F¯(−1) ≤ 1 F ∈ Γ1.

and we have the following feasible dual functions:

1 d F1(d) = 2 is an optimal dual function for b ∈ {0, 1, 2, ...}.

2 3 F2(d) = 0 is an optimal function for b ∈ {..., −3, − 2 , 0}. d d d d 3 1 ⌈⌈ ⌉− ⌉ 3 ⌈⌈ ⌉− ⌉ F3(d) = max{ 2 ⌈d − 4 ⌉, 2d − 2 ⌈d − 4 ⌉} is an optimal function for 1 5 9 b ∈ {[0, 4 ] ∪ [1, 4 ] ∪ [2, 4 ] ∪ ...}. 2⌈⌈ 2d ⌉− 2d ⌉ 2⌈⌈ 2d ⌉− 2d ⌉ 4 3 2d 3 3 3 2d 3 3 d F4(d) = max{ 2 ⌈ 3 − 3 ⌉ − d, − 4 ⌈ 3 − 3 ⌉ + 2 } is an optimal 7 3 1 lehigh-logo function for b ∈ {... ∪ [− 2 , −3] ∪ [−2, − 2 ] ∪ [− 2 , 0]}

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 15/56 Example: Feasible Dual Functions

z(d) F(d) 3

5 2

3 2

1 2

d -4 − 7 -3 − 5 -2 − 3 -1 − 1 0 1 1 3 2 5 3 7 4 2 2 2 2 2 2 2 2

Notice how different dual solutions are optimal for some right-hand sides and not for others. Only the value function is optimal for all right-hand sides. lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 16/56 Farkas’ Lemma (Pure Integer)

For the primal problem, exactly one of the following holds: 1 S= 6 ∅ 2 There is an F ∈ Γm with F(aj) ≥ 0, j = 1, ..., n, and F(b) < 0.

Proof. Let c = 0 and apply strong duality theorem to subadditive dual.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 17/56 Complementary Slackness (Pure Integer)

For a given right-hand side b, let x∗ and F∗ be feasible solutions to the primal and the subadditive dual problems, respectively. Then x∗ and F∗ are optimal solutions if and only if

1 ∗ ∗ j xj (cj − F (a )) = 0, j = 1, ..., n and n 2 ∗ ∗ j ∗ F (b) = j=1 F (a )xj . P Proof. For an optimal pair we have

n ∗ ∗ ∗ ∗ j ∗ ∗ F (b) = F (Ax ) = F (a )xj = cx . j 1 X=

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 18/56 Example: Pure Integer Case

If we require integrality of all variables in our previous example, then the value function becomes

z(d)

5 2

3 2

1 2

d -4 − 7 -3 − 5 -2 − 3 -1 − 1 0 1 1 3 2 5 3 7 4 2 2 2 2 2 2 2 2

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 19/56 Constructing Dual Functions

Explicit construction The Value Function Generating Functions Relaxations Lagrangian Relaxation Quadratic Lagrangian Relaxation Corrected Linear Dual Functions Primal Solution Algorithms Cutting Plane Method Branch-and-Bound Method Branch-and-Cut Method

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 20/56 Properties of the Value Function

It is subadditive over Ω. It is piecewise polyhedral. For an ILP, it can be obtained by a ﬁnite number of limited operations on elements of the RHS:

(i) rational multiplication (ii) nonnegative combination Chvatal´ fcns.   Gomory fcns. (iii) rounding   (iv) taking the minimum    

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 21/56 The Value Function for MILPs

There is a one-to-one correspondence between ILP instances and Gomory functions. The Jeroslow Formula shows that the value function of a MILP can also be computed by taking the minimum of different values of a single Gomory function with a correction term to account for the continuous variables. The value function of the earlier example is

3 ⌊2d⌋ ⌊2d⌋ 3⌈2d⌉ 2 max 3 , 2 + 2 + 2d, z d min ( ) =  nl m l mo  3 max ⌈2d⌉ ⌈2d⌉ d  2 3 , 2 − ,  nl m l mo  

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 22/56 Jeroslow Formula

Let the set E consist of the index sets of dual feasible bases of the linear program 1 1 min{ c x : A x = b, x ≥ 0} M C C M C C E −1 j m where M ∈ Z+ such that for any E ∈ , MAE a ∈ Z for all j ∈ I. Theorem (Jeroslow Formula) There is a g ∈ G m such that

m z(d) = min g(⌊d⌋E) + vE(d − ⌊d⌋E) ∀d ∈ R with S(d) 6= ∅, E∈E

E −1 where for E ∈ , ⌊d⌋E = AE⌊AE d⌋ and vE is the corresponding basic feasible solution.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 23/56 The Single Constraint Case

Let us now consider an MILP with a single constraint for the purposes of illustration:

min cx, (P1) x∈S

n r n−r ′ n c ∈ R , S = {x ∈ Z+ × R+ | a x = b} with a ∈ Q , b ∈ R. The value function of (P) is

z(d) = min cx, x∈S(d)

r n−r ′ where for a given d ∈ R, S(d) = {x ∈ Z+ × R+ | a x = d}. Assumptions: Let I = {1,..., r}, C = {r + 1,..., n}, N = I ∪ C. z(0) = 0 =⇒ z : R → R ∪ {+∞}, + − N = {i ∈ N | ai > 0}= 6 ∅ and N = {i ∈ N | ai < 0}= 6 ∅, r < n, that is, |C| ≥ 1 =⇒ z : R → R.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 24/56 Example

7 min 3x1 + 2 x2 + 3x3 + 6x4 + 7x5 + 5x6 s.t 6x1 + 5x2 − 4x3 + 2x4 − 7x5 + x6 = b and (SP) x1, x2, x3 ∈ Z+, x4, x5, x6 ∈ R+.

d -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 25/56 Example (cont’d)

1 3 C C η = 2 , ζ = − 4 , η = 3 and ζ = −1:

z FU 18 FL

16 ζC 14 ηC

ζ 8 η 6

d -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28

C {η = η }⇐⇒{z(d) = FU(d) = FL(d) ∀d ∈ R+} C {ζ = ζ }⇐⇒{z(d) = FU(d) = FL(d) ∀d ∈ R−} lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 26/56 Observations

+ − + − Consider dU , dU , dL , dL :

2 − − − + + + − + + 2d 3d 4d 2d 2d d d d d L L L L L L U U L d -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28

C C The relation between FU and the linear segments of z: {η , ζ }

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 27/56 Redundant Variables

Let T ⊆ C be such that c t+ ∈ T if and only if ηC < ∞ and ηC = t+ and similarly, at+ − − C C ct t ∈ T if and only if ζ > −∞ and ζ = − . at and deﬁne

ν(d) = min cIxI + cT xT s.t. aIxI + aT xT = d I T xI ∈ Z+, xT ∈ R+ Then ν(d) = z(d) for all d ∈ R. The variables in C\T are redundant. z can be represented with at most 2 continuous variables.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 28/56 Example

min x1 − 3/4x2 + 3/4x3 s.t 5/4x1 − x2 + 1/2x3 = b, x1, x2 ∈ Z+, x3 ∈ R+.

ηc = 3/2, ζC = −∞.

5 2 2 3 2 1 1 2 − 7 -3 − 5 -2 − 3 -1 − 1 1 1 3 2 5 3 7 2 2 2 2 2 2 2 2 − 1 2 -1 − 3 2 -2 − 5 2

i i For each discontinuous point di, we have di − (5/4y1 − y2) = 0 and each linear segment has the slope of ηC = 3/2. lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 29/56 Jeroslow Formula

Let M ∈ Z be such that for any t ∈ T, Maj ∈ Z for all j ∈ I. + at Then there is a Gomory function g such that

ct at Md z(d) = min{g(⌊d⌋t) + (d − ⌊d⌋t)}, ⌊d⌋t = , ∀d ∈ R t∈T a M a t t Such a Gomory function can be obtained from the value function of a related PILP. For t ∈ T, setting

ct ωt(d) = g(⌊d⌋t) + (d − ⌊d⌋t) ∀d ∈ R, at we can write z(d) = min ωt(d) ∀d ∈ R t∈T

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 30/56 Piecewise Linearity and Continuity

For t ∈ T, ωt is piecewise linear with ﬁnitely many linear segments on any closed interval and each of those linear segments has a slope of ηC if t = t+ or ζC if t = t−.

ωt+ is continuous from the right, ωt− is continuous from the left.

ωt+ and ωt− are both lower-semicontinuous.

Theorem z is piecewise-linear with ﬁnitely many linear segments on any closed interval and each of those linear segments has a slope of ηC or ζC. (Meyer 1975) z is lower-semicontinuous. ηC < ∞ if and only if z is continuous from the right. ζC > −∞ if and only if z is continuous from the left. Both ηC and ζC are ﬁnite if and only if z is continuous everywhere.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 31/56 Maximal Subadditive Extension

Let f : [0, h]→R, h > 0 be subadditive and f (0) = 0.

The maximal subadditive extension of f from [0, h] to R+ is

f (d) if d ∈ [0, h] fS(d) = inf f (ρ) if d > h ,  C∈C(d) ρ  X∈C  C(d) is the set of all ﬁnite collections {ρ1, ..., ρR} such that ρi ∈ [0, h], i = 1, ..., R R and Pi=1 ρi = d. Each collection {ρ1, ..., ρR} is called an h-partition of d. We can also extend a subadditive function f : [h, 0]→R, h < 0 to R− similarly.

(Bruckner 1960) fS is subadditive and if g is any other subadditive extension of f from [0, h] to R+, then g ≤ fS (maximality).

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 32/56 Extending the Value Function

Suppose we use z itself as the seed function. Observe that we can change the “inf” to “min”:

Lemma Let the function f : [0, h]→R be deﬁned by f (d) = z(d) ∀d ∈ [0, h]. Then,

z(d) if d ∈ [0, h] fS(d) = min z(ρ) if d > h .  C∈C(d) ρ  X∈C  For any h > 0, z(d) ≤ fS(d) ∀d ∈ R+.

Observe that for d ∈ R+, fS(d)→z(d) while h→∞.

Is there an h < ∞ such that fS(d) = z(d) ∀d ∈ R+?

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 33/56 Extending the Value Function (cont.)

Yes! For large enough h, maximal extension produces the value function itself. Theorem

Let dr = max{ai | i ∈ N} and dl = min{ai | i ∈ N} and let the functions fr and fl be the maximal subadditive extensions of z from the intervals [0, dr] and [dl, 0] to R+ and R−, respectively. Let f (d) d ∈ R F(d) = r + fl(d) d ∈ R − then, z = F.

Outline of the Proof. z ≤ F : By construction. z ≥ F : Using MILP duality, F is dual feasible.

In other words, the value function is completely encoded by the breakpoints in [dl, dr] and 2 slopes. lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 34/56 General Procedure

We will construct the value function in two steps

Construct the value function on [dl, dr]. Extend the value function to the entire real line from [dl, dr]. Assumptions We assume ηC < ∞ and ζC < ∞. We construct the value function over R+ only. These assmuptions are only needed to simplify the presentation.

Constructing the Value Function on [0, dr] If both ηC and ζC are ﬁnite, the value function is continuous and the slopes of the linear segments alternate between ηC and ζC. C C For d1, d2 ∈ [0, dr], if z(d1) and z(d2) are connected by a line with slope η or ζ , then z is linear over [d1, d2] with the respective slope (subadditivity). With these observations, we can formulate a ﬁnite algorithm to evaluate z in [dl, dr].

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 35/56 Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

Figure: Evaluating z in [0, 6]

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 36/56 Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

Figure: Evaluating z in [0, 6]

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 37/56 Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

Figure: Evaluating z in [0, 6]

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 38/56 Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

Figure: Evaluating z in [0, 6]

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 39/56 Example (cont’d)

dr = 6:

ηC —— ζC ——

0 2 4 6

Figure: Evaluating z in [0, 6]

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 40/56 Example (cont’d)

Extending the value function of (SP) from [0, 12] to [0, 18]

z FU 18 FL Υ 16

d -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 41/56 Example (cont’d)

Extending the value function of (SP) from [0, 12] to [0, 18]

z FU 18 FL Υ 16

d -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 42/56 Example (cont’d)

Extending the value function of (SP) from [0, 18] to [0, 24]

z FU 18 FL Υ 16

d -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 43/56 Example (cont’d)

Extending the value function of (SP) from [0, 18] to [0, 24]

z FU 18 FL Υ 16

d -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 44/56 General Case

For E ∈ E , setting

m ωE(d) = g(⌊d⌋E) + vE(d − ⌊d⌋E) ∀d ∈ R with S(d) 6= ∅, we can write m z(d) = min ωE(d) ∀d ∈ R with S(d) 6= ∅. E∈E Many of our previous results can be extended to general case in the obvious way. Similarly, we can use maximal subadditive extensions to construct the value function. However, an obvious combinatorial explosion occurs. Therefore, we consider using single row relaxations to get a subadditive approximation.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 45/56 Gomory’s Procedure

There is a Chvatal´ function that is optimal to the subadditive dual of an ILP with RHS b ∈ ΩIP and zIP(b) > −∞. The procedure: In iteration k, we solve the following LP

k−1 zIP(b) = min cx s.t. Ax = b n i i j=1 f (aj)xj ≥ f (b) i = 1, ..., k − 1 x ≥ 0 P The kth cut, k > 1, is dependent on the RHS and written as:

m k−1 k k−1 k−1 i k−1 k−1 k−1 f (d) = λi di + λm+if (d) where λ = (λ1 , ..., λm+k−1) ≥ 0 & i 1 i 1 ' X= X=

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 46/56 Gomory’s Procedure (cont.)

Assume that b ∈ ΩIP, zIP(b) > −∞ and the algorithm terminates after k + 1 iterations. If uk is the optimal dual solution to the LP in the ﬁnal iteration, then

m k k k k i F (d) = ui di + um+if (d), i 1 i 1 X= X= k is a Chvatal´ function with F (b) = zIP(b) and furthermore, it is optimal to the subadditive ILP dual problem.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 47/56 Gomory’s Procedure (cont.)

Example: Consider an integer program with two constraints. Suppose that the following is the ﬁrst constraint:

2x1 + −2x2 + x3 − x4 = 3

At the ﬁrst iteration, Gomory’s procedure can be used to derive the valid inequality

⌈2/2⌉x1 + ⌈ − 2/2⌉x2 + ⌈1/2⌉x3 + ⌈ − 1/2⌉x4 ≥ ⌈3/2⌉

by scaling the constraint with λ = 1/2. In other words, we obtain x1 − x2 + x3 ≥ 2. Then the corresponding function is:

1 F (d) = ⌈d1/2⌉ + 0d2 = ⌈d1/2⌉

By continuing Gomory’s procedure, one can then build up an optimal dual function in

this way. lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 48/56 Branch-and-Bound Method

Assume that the primal problem is solved to optimality. Let T be the set of leaf nodes of the search tree. Thus, we’ve solved the LP relaxation of the following problem at node t ∈ T zt(b) = min cx , s.t x ∈ St(b)

t t n t t r where St(b) = {Ax = b, x ≥ l , −x ≥ −u , x ∈ Z } and u , l ∈ Z are the branching bounds applied to the integer variables. Let (vt, vt, vt) be the dual feasible solution used to prune node t, if t is feasibly pruned a dual feasible solution (that can be obtained from it parent) to node t, if t is infeasibly pruned Then,

t t t t t FBB(d) = min{v d + v l − v u } t∈T lehigh-logo is an optimal solution to the generalized dual problem.

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 49/56 Branch-and-Cut Method

It is thus easy to get a strong dual function from branch-and-bound. Note, however, that it’s not subadditive in general. To obtain a subaditive function, we can include the variable bounds explicitly as constraints, but then the function may not be strong. For branch-and-cut, we have to take care of the cuts. Case 1: Do we know the subadditive representation of each cut? Case 2: Do we know the RHS dependency of each cut? Case 3: Otherwise, we can use some proximity results or the variable bounds.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 50/56 Case 1

If we know the subadditive representation of each cut: At a node t, we solve the LP relaxation of the following problem

zt(b) = min cx s.t Ax ≥ b x ≥ lt −x ≥ −gt Htx ≥ ht r n−r x ∈ Z+ × R+

where gt, lt ∈ Rr are the branching bounds applied to the integer variables and Htx ≥ ht is the set of added cuts in the form

t k ¯t k t Fk(aj )xj + Fk(aj )xj ≥ Fk(σk(b)) k = 1, ..., ν(t), j∈I X j∈XN\I ν(t): the number of cuts generated so far, k th lehigh-logo aj , j = 1, ..., n: the columns of the problem that the k cut is constructed from, σk(b): is the mapping of b to the RHS of the corresponding problem.

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 51/56 Case 1

Let T be the set of leaf nodes, ut, ut, ut and wt be the dual feasible solution used to prune t ∈ T. Then,

ν(t) t t t t t t t F(d) = min{u d + u l − u g + w kFk(σk(d))} t∈T k 1 X= is an optimal dual function, that is, z(b) = F(b). Again, we obtain a subaddite function if the variables are bounded. However, we may not know the subadditive representation of each cut.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 52/56 Case 2

If we know the RHS dependencies of each cut: We know for Gomory fractional cuts. Knapsack cuts Mixed-integer Gomory cuts ? Then, we do the same analysis as before.

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 53/56 Case 3

In the absence of subadditive representations or RHS dependencies: For each node t ∈ T, let hˆt be such that n lt if ht ≥ 0 hˆt = ht ˆx with ˆx = j kj , k = 1, ..., ν(t) k kj j j gt otherwise j 1 j X= t th t where hkj is the k entry of column hj. Furthermore, deﬁne n lt if wtht ≥ 0 h˜t = ht˜x with ˜x = j j . j j j gt otherwise j 1 j X= Then the function

F(d) = min{utd + utlt − utgt + max{wth˜t, wthˆt}} t∈T

is a feasible dual solution and hence F(b) yields a lower bound. This approach is the easiest way and can be used for bounded MILPs (binary programs), however it is unlikely to get an effective dual feasible solution forlehigh-logo general MILPs.

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 54/56 Conclusions

It is possible to generalize the duality concepts that are familiar to us from linear programming. However, it is extremely difﬁcult to make any of it practical. There are some isolated cases where this theory has been applied in practice, but they are far and few between. We have a number of projects aimed in this direction, so stay tuned...

lehigh-logo

Ralphs and Guzelsoy¨ (Lehigh University) Duality for MILP 18February2009 55/56