DM545 Linear and Integer Programming Lecture 13 Cutting Planes and Branch and Bound Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Cutting Plane Algorithms Outline Branch and Bound 1. Cutting Plane Algorithms 2. Branch and Bound 2 Cutting Plane Algorithms Outline Branch and Bound 1. Cutting Plane Algorithms 2. Branch and Bound 3 Cutting Plane Algorithms Valid Inequalities Branch and Bound T n • IP: z = maxfc x : x 2 X g; X = fx : Ax ≤ b; x 2 Z+g • Proposition: conv(X ) = fx : A~x ≤ b~; x ≥ 0g is a polyhedron • LP: z = maxfcT x : A~x ≤ b~; x ≥ 0g would be the best formulation • Key idea: try to approximate the best formulation. Definition (Valid inequalities) ax ≤ b is a valid inequality for X ⊆ Rn if ax ≤ b 8x 2 X Which are useful inequalities? and how can we find them? How can we use them? 4 Cutting Plane Algorithms Example: Pre-processing Branch and Bound • X = f(x; y): x ≤ 999y; 0 ≤ x ≤ 5; y 2 B1g x ≤ 5y n • X = fx 2 Z+ : 13x1 + 20x2 + 11x3 + 6x4 ≥ 72g 13 20 6 72 6 2x + 2x + x + x ≥ x + x + x + x ≥ = 6 + 1 2 3 4 11 1 11 2 3 11 4 11 11 2x1 + 2x2 + x3 + x4 ≥ 7 • Capacitated facility location: X xij ≤ bj yj 8j 2 N xij ≤ bj yj i2M X xij = ai 8i 2 M xij ≤ ai j2N n xij ≥ 0; yj 2 B xij ≤ minfai ; bj gyj 5 Cutting Plane Algorithms Chvátal-Gomory cuts Branch and Bound n n m×n • X 2 P \ Z+; P = fx 2 R+ : Ax ≤ bg; A 2 R m • u 2 R+ , fa1; a2;::: ang columns of A CG procedure to construct valid inequalities n X 1) uaj xj ≤ ub valid: u ≥ 0 j=1 n X X X 2) buaj cxj ≤ ub valid: x ≥ 0 and buaj cxj ≤ uaj xj j=1 n X n 3) buaj cxj ≤ bubc valid for X since x 2 Z j=1 Theorem by applying this CG procedure a finite number of times every valid inequality forX can be obtained 6 Cutting Plane Algorithms Cutting Plane Algorithms Branch and Bound n • X 2 P \ Z+ • a family of valid inequalities F : aT x ≤ b; (a; b) 2 F for X • we do not find them all a priori, only interested in those close to optimum Cutting Plane Algorithm Init.: t = 0; P0 = P Iter. t: Solve z¯t = maxfcT x : x 2 Pt g let xt be an optimal solution if xt 2 Zn stop, xt is opt to the IP if xt 62 Zn solve separation problem for xt and F if (at ; bt ) is found with at xt > bt that cuts off x t Pt+1 = P \ fx : ai x ≤ bi ; i = 1;:::; tg else stop (Pt is in any case an improved formulation) 7 Cutting Plane Algorithms Gomory’s fractional cutting plane algorithmBranch and Bound Cutting plane algorithm + Chvátal-Gomory cuts • maxfcT x : Ax = b; x ≥ 0; x 2 Zng • Solve LPR to optimality 2 3 x = b¯ − P ¯a x ; u 2 B ¯ −1 ¯ u u uj j 6 I AN = AB AN 0 b 7 j2N 6 7 ¯ P 4 5 z = d + c¯j xj ¯ j2N c¯B c¯N (≤ 0) 1 −d • If basic optimal solution to LPR is not integer then 9 some row u: ¯ 1 bu 62 Z . The Chvatál-Gomory cut applied to this row is: X ¯ xBu + b¯auj cxj ≤ bbuc j2N (Bu is the index in the basis B corresponding to the row u) (cntd) 8 Cutting Plane Algorithms Branch and Bound ¯ P • Eliminating xBu = bu − ¯auj xj in the CG cut we obtain: j2N X ¯ ¯ (¯auj − b¯auj c)xj ≥ bu − bbuc j2N | {z } | {z } 0≤fuj <1 0<fu <1 X fuj xj ≥ fu j2N fu > 0 or else u would not be row of fractional solution. It implies that ∗ ∗ x in which xN = 0 is cut out! • Moreover: when x is integer, since all coefficient in the CG cut are integer the slack variable of the cut is also integer: X s = −fu + fuj xj j2N (theoretically it terminates after a finite number of iterations, but in practice not successful.) 9 Cutting Plane Algorithms Example Branch and Bound max x1 + 4x2 x2 x1 + 6x2 ≤ 18 x1 ≤ 3 x1 = 3 x1; x2 ≥ 0 x1; x2integer x1 + 6x2 = 18 x1 x1 + 4x2 = 2 | | x1 | x2 | x3 | x4 | -z | b | |---+----+----+----+----+----+----| | | 1 | 6 | 1 | 0 | 0 | 18 | | |1|0|0|1|0|3| |---+----+----+----+----+----+----| | |1|4|0|0|1|0| | | x1 | x2 | x3 | x4 | -z | b | |---+----+----+------+------+----+------| | | 0 | 1 | 1/6 | -1/6 | 0 | 15/6 | | |1|0|0 |1 |0| 3| x = 5=2; x = 3 |---+----+----+------+------+----+------| 2 1 | | 0 | 0 | -2/3 | -1/3 | 1 | -13 | Optimum, not integer 10 Cutting Plane Algorithms Branch and Bound • We take the first row: | | 0 | 1 | 1/6 | -1/6 | 0 | 15/6 | • P 1 5 1 CG cut j2N fuj xj ≥ fu 6 x3 + 6 x4 ≥ 2 • Let’s see that it leaves out x ∗: from the CG proof: 1=6 (x1 + 6x2 ≤ 18) 5=6 (x1 ≤ 3) x1 + x2 ≤ 3 + 5=2 = 5:5 since x1; x2 are integer x1 + x2 ≤ 5 • Let’s see how it looks in the space of the original variables: from the first tableau: x3 = 18 − 6x2 − x1 x4 = 3 − x1 1 5 1 (18 − 6x − x ) + (3 − x ) ≥ x + x ≤ 5 6 2 1 6 1 2 1 2 11 Cutting Plane Algorithms Branch and Bound • Graphically: x2 x1 = 3 x1 + 6x2 = 18 x1 + x2 = 5 x1 x1 + 4x2 = 2 • Let’s continue: We need to apply dual-simplex | | x1 | x2 | x3 | x4 | x5 | -z | b | (will always be the case, why?) |---+----+----+------+------+----+----+------| | | 0 | 0 | -1/6 | -5/6 | 1 | 0 | -1/2 | | | 0 | 1 | 1/6 | -1/6 | 0 | 0 | 5/2 | ratio rule: min j cj j | |1|0|0 |1 |0|0|3 | aij |---+----+----+------+------+----+----+------| | | 0 | 0 | -2/3 | -1/3 | 0 | 1 | -13 | 12 Cutting Plane Algorithms Branch and Bound • After the dual simplex iteration: We can choose any of the three | | x1 | x2 | x3 | x4 | x5 | -z | b | rows. |---+----+----+------+----+------+----+-------| | | 0 | 0 | 1/5 | 1 | -6/5 | 0 | 3/5 | | | 0 | 1 | 1/5 | 0 | -1/5 | 0 | 13/5 | Let’s take the third: CG cut: | | 1 | 0 | -1/5 | 0 | 6/5 | 0 | 12/5 | 4 1 2 |---+----+----+------+----+------+----+-------| 5 x3 + 5 x5 ≥ 5 | | 0 | 0 | -3/5 | 0 | -2/5 | 1 | -64/5 | • In the space of the original variables: x2 4(18 − x1 − 6x2) + (5 − x1 − x2) ≥ 2 x1 + 5x2 ≤ 15 x1 • ... 13 Cutting Plane Algorithms Outline Branch and Bound 1. Cutting Plane Algorithms 2. Branch and Bound 14 Cutting Plane Algorithms Branch and Bound Branch and Bound • Consider the problem z = maxfcT x : x 2 Sg • Divide and conquer: let S = S1 [ ::: [ Sk be a decomposition of S into k T smaller sets, and let z = maxfc x : x 2 Sk g for k = 1;:::; K. Then k z = maxk z For instance if S ⊆ f0; 1g3 the enumeration tree is: S x1 = 0 x1 = 1 S0 S1 x2 = 0 S00 S01 S10 S11 x3 = 0 S000 S001 S010 S011 S100 S101 S110 S111 15 Cutting Plane Algorithms Bounding Branch and Bound • Let zk be an upper bound on zk • Let zk be an lower bound on zk • (zk ≤ zk ≤ zk ) k • z = maxk z is an upper bound on z k • z = maxk z is a lower bound on z 16 Cutting Plane Algorithms Branch and Bound 27 13 z = 25 z = 20 pruned by optimality 20 25 20 15 27 13 z = 26 z = 21 pruned by bounding 20 26 18 21 40 −∞ z = 37 z = 13 nothing to prune 24 37 13 −∞ 17 Cutting Plane Algorithms Example Branch and Bound max x1 + 2x2 x2 x1 + 4x2 ≤ 8 4x1 + x2 ≤ 8 x1; x2 ≥ 0; integer x1 + 4x2 = 8 x1 x1 + 2x2 = 1 4x1 + x2 = 8 • Solve LP | | x1 | x2 | x3 | x4 | -z | b | |---+----+----+----+----+----+---| | |1|4|1|0|0|8| | |4|1|0|1|0|8| |---+----+----+----+----+----+---| | |1|2|0|0|1|0| | | x1 | x2 | x3 | x4 | -z | b | |--------------+----+------+----+------+----+----| | I’=I-II’ | 0 | 15/4 | 1 | -1/4 | 0 | 6 | | II’=1/4II | 1 | 1/4 | 0 | 1/4 | 0 | 2 | |--------------+----+------+----+------+----+----| | III’=III-II’ | 0 | 7/4 | 0 | -1/4 | 0 | -2 | 18 Cutting Plane Algorithms Branch and Bound • continuing x2 = 1 + 3=5 = 1:6 | | x1 | x2 | x3 | x4 | -z | b | |----------------+----+----+-------+-------+----+---------| x1 = 8=5 | I’=4/15I | 0 | 1 | 4/15 | -1/15 | 0 | 24/15 | The optimal solution | II’=II-1/4I’ | 1 | 0 | -1/15 | 4/15 | 0 | 24/15 | |----------------+----+----+-------+-------+----+---------| will not be more than | III’=III-7/4I’ | 0 | 0 | -7/15 | -3/5 | 1 | -2-14/5 | 2 + 14=5 = 4:8 • Both variables are fractional, we pick one of the two: x2 4:8 x1 = 1 x1 ≤ 1 x1 ≥ 2 x1 + 4x2 = 8 x1 x1 + 2x2 = 1 4x1 + x2 = 8 19 Cutting Plane Algorithms Branch and Bound • Let’s consider first the left branch: | | x1 | x2 | x3 | x4 | x5 | -z | b | |---+----+----+-------+-------+----+----+-------| | |1|0|0 |0 |1|0|1 | | | 0 | 1 | 4/15 | -1/15 | 0 | 0 | 24/15 | | | 1 | 0 | -1/15 | 4/15 | 0 | 0 | 24/15 | |---+----+----+-------+-------+----+----+-------| | | 0 | 0 | -7/15 | -3/5 | 0 | 1 | -24/5 | always a b term negative after | | x1 | x2 | x3 | x4 | x5 | b | -z | branching: |----------+----+----+-------+-------+----+---+-------| ¯ | I’=I-III | 0 | 0 | 1/15 | -4/15 | 1 | 0 | -9/15 | b1 = bb3c | | 0 | 1 | 4/15 | -1/15 | 0 | 0 | 24/15 | ¯ ¯ b1 = bb3c − b3 < 0 | | 1 | 0 | -1/15 | 4/15 | 0 | 0 | 24/15 | |----------+----+----+-------+-------+----+---+-------| | | 0 | 0 | -7/15 | -3/5 | 0 | 1 | -24/5 | Dual simplex: cj minj j j ai j | | x1 | x2 | x3 | x4 | x5 | b | -z | |-------------+----+----+--------+----+-------+---+--------| | I’=-15/4I | 0 | 0 | -1/4 | 1 | -15/4 | 0 | 9/4 | | II’=II-1/4I | 0 | 1 | 15/60 | 0 | -1/4 | 0 | 7/4 | | III’=III+I | 1 | 0 | 0 | 0 | 1 | 0 | 1 | |-------------+----+----+--------+----+-------+---+--------| | | 0 | 0 | -37/60 | 0 | -9/4 | 1 | -90/20 | 20 Cutting Plane Algorithms Branch and Bound • Let’s branch again x 4:8 2 x1 ≤ 1 x1 ≥ 2 4:5 C x1 + 4x2 = 8 x2 ≤ 1 x2 ≥ 2 x1 x1 + 2x2 = 1 B A 4x1 + x2 = 8 We have three open problems.